Dialog, 19(3), 42-59 Copyright © 2016, ISSN: 1930-9325 This research was made possible by Grant Number 90YR0055 from the Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Office of Planning, Research, and Evaluation, the Administration for Children and Families, or the U.S. Department of Health and Human Services. RESEARCH ARTICLE Lessons from Training Early Head Start Staff to Implement an Evidence-Based Parenting Intervention Myae Han, Rena Hallam, Jason T. Hustedt, Jennifer A. Vu, and Juana Gaviria-Loaiza University of Delaware The purpose of this study was to understand the process of implementing Promoting First Relationships (PFR; Kelly, Zuckerman, Sandoval, & Buehlman, 2008b), an evidence- based parent-child interaction intervention, within an Early Head Start (EHS) home visiting program. This study documented seven home visitors’ early experiences with intervention implementation, specifically preparation and training on the intervention, overall experience to use of the intervention, challenges and issues identified during implementation, as well as recommendations and suggestions for sustainability of PFR within the EHS program. Data were collected via semi-structured individual interviews. We employed a constant-comparative method and a systematic coding process. Findings indicate that overall, home visitors reported an alignment between the intervention content and their own philosophies, a belief that implementing the intervention improved their home visiting practice, and a sense that participating families benefited from the experience. Some negative experiences were identified in relationship to the lengthy training process and changes in home visiting practice. Major challenges included learning about intervention content and managing household issues. Recommendations for sustaining the intervention after initial implementation are also reported. Keywords: Early Head Start; parenting intervention; home visiting; implementation Evidence-based programs are of high interest to researchers and policymakers and are now frequently required when conducting federally-funded research in areas such as human services and education. However, there have been growing concerns about scaling up evidence-based interventions in community settings because the local context can be very different from the context where the intervention was initially developed. Agencies adopting evidence-based interventions often experience unexpected issues during implementation. According to Supplee and Metz (2015), developers of evidence-based programs may face serious challenges meeting the needs of local communities and users. Successful outcomes are achievable only if the 43 HAN ET AL. intervention is implemented with high levels of fidelity and quality (Supplee & Metz, 2015). In order to achieve successful implementation in practice settings, it is critical to understand the context and the conditions of implementation and to systematically examine implementation processes. Such information is vital in applying evidence-based practice in human service prevention programs. Early Head Start and Home Visiting as a Context for Intervention Early Head Start (EHS) is a comprehensive child development initiative with a strong emphasis on parent involvement and research-informed practice, including a program of federally-funded research on the use of evidence-based parenting interventions aimed at reducing families’ negative experiences with stress (Buffering Toxic Stress Consortium Principal Investigators, Meyer, & Fortunato, 2013). Established in 1994 as a downward extension of the federal Head Start initiative for preschool-age children, EHS offers services to children up to age 3 and their parents, including pregnant women. EHS provides an ideal context to study implementation, as the comprehensive nature of the program promotes a range of different types of interventions (e.g., health, parent education, early education experiences) and the federal Head Start program has historically been viewed as a “national laboratory” for services to young at-risk children and their families (Zigler & Seitz, 1982). Further, prior research suggests that EHS offers a strong foundation for program implementation. In a nationally representative randomized control trial program evaluation, Love and colleagues (2002) found that children who participated in Early Head Start had more positive outcomes on measures of cognitive and language development and were also less aggressive, compared to control group children. The national Early Head Start initiative has incorporated home visitation as a model practice, with local Early Head Start grantees having the option to deliver either home-based or center-based service models dependent on the needs of families in their communities. Across the U.S., home visits have become a key component of practice in early care and education and public health services. Home visitation is often considered an essential strategy for intervening with high-risk families of very young children, as it provides a more intimate context for delivering information and assistance to families and provides an opportunity for more dyadic types of intervention. Home visiting efforts across different agencies tend to have similar broad goals of enhancing child and family development. However, programs vary greatly in targeted outcomes (e.g., health, education), method of delivery (e.g., use of a specified curriculum), and target population (Zero to Three, 2009). Prior research suggests that the efficacy of home visiting is quite mixed (e.g. Daro, 2006; Filene, Kaminski, Valle, & Cachat, 2013; Goyal, Teeters, & Ammerman, 2013; Sweet & Appelbaum, 2004). In general, positive outcomes have been found more often and effects when found are modest (Filene et al., 2013; Sweet & Appelbaum, 2004). Researchers in home visiting practice point to the need to study implementation, and specifically fidelity, in order to better understand parent and child outcomes (Jones Harden, Chazen-Cohen, Raikes, & Vogel, 2012). Studying the Implementation of Interventions The implementation or installation of evidence-based interventions in community-based EVIDENCE-BASED PARENTING INTERVENTION 44 programs is of increasing significance, as recent research has demonstrated that even well- developed programs may fail or be less effective due to implementation issues “on the ground” (Jones-Harden, Chazan-Cohen, Raikes, & Vogel, 2012; Supplee & Metz, 2015). Particular attention has been paid to the role of implementation in the professional development of the early care and education workforce as an important bridge to high quality practice (Halle, Metz, & Martinez-Beck, 2013; Odom, 2009). Implementation science provides the field with a framework for conceptualizing how to study and monitor processes regarding the preparation and support for field staff as well as the quality of intervention implementation with children and families. This offers researchers a model for examining the conditions of implementation and modifying those conditions as needed (Blase, Fixsen, Sims, & Ward, 2014). Implementation science posits that a set of organizational drivers scaffolds the infrastructure around a particular intervention and that attention to these drivers increases the likelihood of successful operations as well as enhances the potential for sustainability of the intervention (Fixsen & Blase, 2008). From an implementation perspective, the likelihood of successfully launching and sustaining an evidence-based intervention relies on three types of drivers: competency, organization, and leadership (Fixsen & Blasé, 2008). Competency drivers relate to the selection, professional development, and support of staff who are the actual implementers of a specific intervention. Organizational drivers include the gathering and utilization of data to inform decision-making, the development of responsive administrative procedures to support effective implementation, and the need to collaborate with external parties when necessary. Leadership drivers refer to the importance of effective agency leaders and leadership teams in managing the overall process. One of the key premises of implementation science and the focus of this study is the importance of feedback loops to ensure adequate conditions are in place to promote high quality execution of an intervention as aligned to the above three organizational drivers (Fixsen & Blase, 2008). This aspect is particularly significant when field or agency staff are the primary agents of change. Feedback loops enable researchers and practitioners to collaboratively make implementation decisions based on authentic data from the field. From an implementation perspective, feedback loops between field staff, program administrators, and researchers should be intentional – meaning that both the type of information and the frequency of its collection should be considered in light of agency operations and features of the intervention. The Current Study The current study uses Early Head Start as a context for examining the implementation of the Promoting First Relationships (PFR; Kelly, Zuckerman, Sandoval, & Buehlman, 2008b) intervention. The study examined the perspectives of the home visitors responsible for a new parent-child intervention during the first year of implementation as part of their regular home visits with families. By examining stakeholders’ perspectives in the early phases of implementation, researchers and agency staff can more carefully monitor fidelity of implementation and adjust the infrastructure as needed to bring about needed conditions for effective implementation of the intervention. The Early Head Start (EHS) program involved in this research serves approximately 200 young children and their families in the Mid-Atlantic area. The partnering EHS program is mature and well-established, having started in 1997 during the third wave of EHS expansion. It 45 HAN ET AL. operates both home- and center-based programs in urban, suburban, and rural settings. All home visiting staff are trained to use Parents as Teachers (e.g., Zigler, Pfannenstiel, & Seitz, 2008), which is employed as the program’s primary curriculum. In the current study, which is part of a larger investigation of families’ experiences with the PFR intervention, a subset of the program’s current EHS home visiting staff was trained in PFR and began delivering PFR during their home visits with some families. This was possible due to a convergence of the EHS program’s goal of improving the quality of home visiting and the supervision of home visitors and the researchers’ interests in integrating an evidence-based parent-child interaction model into typical EHS practice. Interviews with the staff members were conducted during initial implementation of the PFR intervention, in order to inform scale-up of the intervention within a large and well- established EHS program. The purpose of this study was to understand the process of implementing the PFR intervention as an element of existing EHS home visiting services. Our research was guided by three questions: 1. How did the home visitors perceive implementation of the PFR intervention as part of an Early Head Start home visiting program? 2. What kinds of challenges and issues were identified during implementation? 3. What kinds of suggestions and recommendations were identified by the home visitors to promote sustainability of the PFR intervention in the Early Head Start program? By examining these three questions, we aim to identify lessons related to the implementation of evidence-based interventions within Early Head Start programs. METHOD Participants The participants were seven Early Head Start home visitors in the partnering EHS agency. These home visitors were randomly selected from amongst the existing 14 home visitors in the EHS program as a first cohort to receive PFR training and began implementing this intervention as part of their home visiting sessions. All home visitors selected for this initial cohort agreed to participate in the study. Demographic information for the home visitors is reported in Table 1. The majority of home visitors serve families in home-based programs (N=6). Only one served families in a center-based program, and this home visitor had the largest caseload. This distribution of home visitors and their caseloads is typical of the Early Head Start program, where the majority of the home visiting staff provide home-based services. Fewer staff members provide home visits to center-based families, and these staff members have larger caseloads because the home visits occur less frequently. Home visitors’ education levels and years of employment varied as shown in Table 1. Home visitors selected one family from their current caseload to participate in this study. EVIDENCE-BASED PARENTING INTERVENTION 46 TABLE 1 Demographic information Demographics Participant information (N=7) Age range 36 - 61 years Ethnicity (N) Hispanic (2), non-Hispanic (5) Highest education level (N) Some college credit (1), Associate’s degree (2), Bachelor’s degree (3), Master’s degree (1) Years of employment with current EHS program (N) 1-5 years (5), 6-10 years (2) Program type (N) Home-based (6), Center-based (1) Average caseload (N) 2 families (1), 6 families (1), 12 families (4), 20 families (1) Intervention: Promoting First Relationships Promoting First Relationships (PFR; Kelly et al., 2008b) was developed to guide caregivers in building nurturing and responsive relationships with their young children from birth to age three, thus encouraging growth in the children’s social-emotional development, language, and cognition. PFR is a manualized, strengths-based intervention delivered by community-based service providers who first participate in an extensive training and fidelity process. Providers may be individuals who work with caregivers and young children, such as early childhood professionals, home visitors, and social workers. PFR has been used in both home and center- based practice to enhance family engagement in early care and education (Hallam, Han, Vu, & Hustedt, 2017) The intervention is based on video reflection between caregivers and trained PFR staff, focusing on positive feedback and based on principles of attachment theory. Home visitors lead 10 weekly sessions, using four key strategies aimed at teaching parents how to provide sensitive and responsive caregiving: (1) joining, or establishing rapport with the caregiver; (2) giving positive, instructive, verbal feedback; (3) using videotapes of caregiver-child interactions to show caregivers their strengths in practicing parenting skills; and (4) asking reflective questions to discover needs of the caregiver and child (Kelly, Zuckerman, & Rosenblatt, 2008a). During five of the 10 hour-long sessions, caregiver-child dyads are videotaped during brief interaction sessions. On the alternate weeks, the service provider and caregiver review the unedited videotapes together. This is used to support a consultation strategy where the service provider reviews PFR principles and guides discussion with the caregiver (Kelly et al., 2008a; Spieker, Oxford, Kelly, Nelson & Fleming, 2012) using actual examples of the caregiver’s strengths. Initial results from prior PFR research (Kelly et al., 2008a) indicated that caregivers participating in the intervention were more sensitive and responsive to their children and that the children in turn were more responsive as well. A community-based randomized control trial (Spieker et al., 2012) examined the effects of PFR with a sample of 210 caregivers and toddlers in state 47 HAN ET AL. dependency, where the toddlers had experienced recent changes in their caregiver placement. For caregivers in the PFR intervention group, the researchers found significantly greater understanding of their toddlers’ social-emotional needs compared to caregivers in the control group (d = 0.36 immediately post-intervention). Caregivers that received PFR also showed higher levels of sensitivity compared to caregivers in the control group (d = 0.41 immediately post-intervention). Child outcomes were similarly positive, as caregiver reports of child competence were significantly more positive in the PFR group than in the control group (d = 0.42 immediately post-intervention. The evidence base related to PFR has continued to grow. Additional studies of PFR provide further evidence of the intervention’s impacts on physiological regulation (cortisol production patterns; Nelson & Spieker, 2013) as well as sleep regulation (d = 0.67 at a 6-month follow-up; Oxford, Fleming, Nelson, Kelly, & Spieker, 2013). Training and Delivery of PFR PFR can be delivered by anyone who is trained and certified by the PFR developer. However, there is a lengthy training and post-training process in order to maintain one’s status as a PFR provider. The PFR training, in which the home visitors in this study participated in order to become certified on the intervention, involved attending a 3-day workshop followed by approximately 20 weeks of mentored online support, involving distance learning with a coach employed by the PFR developers at the University of Washington. During the first 10 weeks of mentored support, the PFR coach worked with a pair of home visitors and provided online coaching on a weekly basis, reviewing training videotapes of parent-child interaction exemplars and discussing the key PFR concepts. During the second 10 weeks, home visitors delivered PFR with an initial EHS family from their caseload and received support from the PFR coach. The process usually involved the PFR coach reviewing a video recorded home visiting session with a participating family, asking the home visitor to reflect on their practice, and providing feedback to the home visitor. The home visitors also received additional support from a research project staff member who was also receiving PFR training. This support included additional individual coaching sessions often repeating and emphasizing PFR concepts. The amount of coaching support was individualized based on each home visitor’s need. All home visitors completed 10 PFR sessions (e.g., the second 10 weeks of training) with one family prior to the current implementation study. The PFR training process also included assessments of fidelity to the intervention. For each home visitor, the process began with videotaping a PFR home visit, including reflection and conversation between the home visitor and caregiver. Then this video was sent to PFR staff at the University of Washington for an external fidelity check and scored using their fidelity checklist. A score of 38 points out of a possible 40 was considered to be a passing score. If home visitors did not initially pass, additional fidelity checks were conducted until a passing score was received. Data Collection The study protocol was approved by the University’s Institutional Research Review Board prior to the study. Consent from home visitors and families was obtained before data collection. We EVIDENCE-BASED PARENTING INTERVENTION 48 employed semi-structured interviews as our means of gathering home visitor perspectives on implementation. Interview questions were developed by a leadership team consisting of four investigators, who are university faculty and experts in the areas of early childhood education and early intervention, and a project coordinator who received PFR training. The interview questions were about overall experiences with the PFR training process and content, challenges and issues of implementation, home visitor experiences with participating families, and recommendations for sustainability. During the interviews, a base set of common questions was employed, and follow-up questions and/or prompts were used when needed for a greater depth of information. The project coordinator conducted an individual interview with each of the home visitors. Each interview session took about 30-40 minutes and was audio-taped. All interviews were then transcribed verbatim. The lengths of transcripts were between 14 -27 pages per interview, totaling 126 pages of 12-point font, single-spaced Word documents. No names were used in the transcripts. Code numbers were used to protect the confidentiality of subject identities in the transcripts. Data Analysis The constant-comparative method (Merriam, 1998) and systematic coding (Miles & Huberman, 1994) were used for the qualitative analysis process. We used qualitative data analysis software, NVivo 10 (QSR International, 2014). Verbatim transcripts were downloaded to NVivo. Transcripts were read in their entirety by two researchers several times prior to the analysis in order to develop familiarity with the data. During this process, a table of summary content was developed for each interview transcript. Two researchers, a principal investigator (PI) of the current project and a doctoral student, analyzed and coded the transcript together. Both have prior experience with qualitative analysis. Peer debriefing and member checks were then conducted twice with the entire research team including the three faculty members who also share the role of co-principal investigators and the project coordinator who developed the interview questions. For code development, an initial coding scheme category was created based on research questions consisting of three major themes addressing different aspects of the implementation process: home visitors’ experience with PFR implementation, challenges/issues with implementation, and recommendations for sustainability. Home visitors’ experiences were categorized as positive, negative, or neutral experiences. This is represented in Figure 1. Two researchers, PI and doctoral student, reviewed each transcript together and identified sub-codes under each category of home visitors’ experiences (positive, negative, neutral), challenges and issues with implementation, and recommendations for sustainability. When there was disagreement in identifying or defining the code, constant comparative methods were used by discussing the code names, scopes, and definitions until consensus was achieved. When consensus was not reached on some codes, these codes were discussed during the peer debriefing and member checking process. Next, a codebook was developed and included the definition of each code and examples of cases. Using the codebook, two researchers reviewed and recoded entire transcript. This process took more than 8 months. Initial results were shared with the entire project team during peer debriefing. 49 HAN ET AL. Figure 1. Initial coding scheme categories. Using NVivo allowed us to calculate the frequency of each code and the number of sources of transcripts. In total, 857 interview segments were identified that related to the implementation process and were coded. Among them were 274 segments related to home visitors’ experiences with PFR implementation, 339 segments related to implementation challenges and issues, and 167 segments related to recommendations for sustainability. Some interview segments were double-coded if they encompassed multiple scheme categories, which is why the frequency of total number of segments is greater than the sum of segments across the three categories. No double codes were assigned within the same category. To increase credibility of the qualitative analysis process (Mertens, 2010), the researchers had prolonged and persistent engagement with data, and multiple researchers engaged in process of coding development and checked each other’s subjectivity. Peer debriefing and member checking were used to monitor the trustworthiness of the results. In addition, NVivo qualitative software was used to keep notes and comments, which allowed us to monitor our own biases during coding process. RESULTS Our overall goal was to examine the implementation process of the evidence-based Promoting First Relationships intervention as it was incorporated into Early Head Start home visiting practice. Our interviews with home visitors who carried out the intervention revealed the following results which are presented in order of our research questions. Implementation Process Home visitors’ experience with PFR implementation Positive Negative Neutral Challenges/ issues with implementation Recommendations for sustainability EVIDENCE-BASED PARENTING INTERVENTION 50 Q1. Home Visitors’ Experience of the PFR Implementation Process NVivo qualitative analysis software (QSR International, 2014) provides a visual distribution of codes in relation to the frequency and amount of language content coded in any particular set of data. Figure 2 shows the frequency of major topics of codes related to the home visitors’ experience with the PFR implementation process. Thus, the visual displays the relative size of each code across all the interview transcripts coded. Table 2 reports the description/definition of these codes. Figure 2. Home visitors’ experiences with PFR implementation. Note. Each parenthesis reports total frequency of each code and the number of unique references/the number of home visitors (maximum 7). 51 HAN ET AL. TABLE 2 Description of codes for home visitors’ experiences Code Description Positive Expressing explicit positive feelings PFR content Positive experience with PFR strategies, content, materials Video Expressing positive experience with the use of video Experience from Families Family reporting positive feelings or experiences during PFR visit Training Positive experience with training process in general Self-confidence/skill Increasing self-confidence or skills with home visiting practice. expressing personal or professional growth Other support Positive experience with support provided by research project staff Family condition Characteristics or situation of family which made PFR visit more successful Family relationship Improvement of relationship between family and home visitors Negative Expressing negative feelings such as frustration Training Negative experience with trainer or training process PFR content Negative experience with questioning techniques, number of handouts Repetition Negative experience with repetitive sessions or reflection Video Resistance to video or uncomfortableness Neutral Neutral feelings about the PFR in general - not positive, not negative Video Neutral experience with videotaping Training Neutral experience with training process Within Figure 2, the size of each square represents how comparatively how much the home visitors talked about the topics of codes within the total Experience segment. We divided these codes by Positive, Negative, and Neutral experiences related to the PFR implementation process. It is important to note how many home visitors mention each code in addition to overall EVIDENCE-BASED PARENTING INTERVENTION 52 frequency of codes. Therefore we report both in our analysis. The first number in the parenthesis indicates the total frequency of codes across all transcripts and the second number indicates the number of unique references, in other words, the number of home visitors whose interview included that code (maximum 7). For example, Positive (214, 7) means that there were 214 segments related to the positive experience with the PFR implementation among all 274 segments related to Experience codes and that these codes were drawn from interviews with all 7 home visitors’ transcripts. Overall, the majority of comments by home visitors were related to positive experiences with implementation. The most prevalent positive experiences were about the content of the PFR intervention, use of video, positive experiences with families, the training process, and increasing self-confidence and skills. Some examples of actual quotes from home visitors include the following.  “I can’t say enough good things about it, I love it (about PFR content),”  “It (addressing social emotional issues) was easier in PFR to do that because it gave us the tools, kind of a step-by step questions to ask and things to say and even the video,”  “the video was so concrete in that you‘re going back and looking at that segment, it was very easy for me to zero in on things,”  “I think that was good for the mom personally, she’s gonna feel so good after we’re done,”  “it (reflection) made mom more comfortable with the ideas, she seemed to be much more okay with reflecting on her child’s inner world, her inner world,”  “PFR definitely honed my skills in being able to notice little things, it was good for me professionally, It is really rewarding, that made me feel so much better and the way we discuss.” Some negative experiences were also expressed by the home visitors in regard to the training process and PFR content. These comments addressed the lengthy training process and experiences with the trainer, repetition of the same information in PFR content, and discomfort with videotaping. Some examples of quotes from home visitors include the following.  “I think at the end they got a little tired because there were a lot of video-taping and a lot of talking,”  “I feel like I wasn’t properly trained in the video” “It (online coaching) is certainly not ideal, it’s not my favorite, just because you’re not getting person-to person interaction.” Q2. Challenges and Issues with Implementation Figure 3 reports the challenges and issues identified during the PFR implementation process and Table 3 presents descriptions of the most frequent codes and examples of actual quotes by the home visitors. A total of 339 segments were identified as codes related to challenges addressing various issues during the implementation process. As part of our PFR implementation plan, we 53 HAN ET AL. placed greater importance on identifying challenges and issues with the initial implementation process. Figure 3. Challenges and issues identified. Note. Each parenthesis reports total frequency of each code and the number of unique references/the number of home visitors (maximum 7). PFR session/content (63, 7) Household issues (61, 7) Doing two programs (39, 7) Scheduling (31, 7) Length of time (22, 7) Technology (35, 7) Research process (19, 5) Decision making (11, 4) Time commitment (18, 4) Trainer (16, 4) Resource org. (15, 5) Fidelity (8, 3) Challenges and issues (339, 7) EVIDENCE-BASED PARENTING INTERVENTION 54 TABLE 3 Descriptions and summarized examples reflecting major implementation challenges addressed by home visitors Major Codes Code Definitions and Examples PFR session/content Definition: Any challenges with PFR session/contents such as strategies, techniques, handouts, reflection process, etc. Examples: Uncomfortable with video reflection or recording by home visitors and family, not intervening in caregiver-dyad interaction when the caregiver was having a hard time, letting the caregiver lead, not giving suggestions during dyadic interaction. Household issues Definition: Situations presented by EHS family that interfere with PFR home visit. Examples: Family sickness, sibling distraction, presence of other family members (large family), family emergency Doing two programs Definition: Challenges related to delivering both EHS program and PFR intervention. Examples: Helping family to understand both curricula [Parents as Teachers (PAT) & PFR], balancing time between PAT and PFR. Technology Definition: Issues with the use of technology Examples: Internet accessibility issues during online training, lack of experience with video recording and uploading. Scheduling Definition: Challenges with scheduling with trainer, family, or other home visitors. Examples: Scheduling difficulty with a PFR trainer due to different time zones (East coast vs. West coast), scheduling difficulty with families due to holidays or missing visits. One strategy used in the PFR intervention involves asking reflective questions to discover needs of the caregiver while the home visitor and caregiver review a video recording of a previous caregiver-child dyad interaction. Home visitors identified this strategy as one of the challenges, because some were used to actively providing suggestions right away instead of serving as a guide who helps the caregiver to identify an area for further development through a reflective questioning process. Uncomfortableness with video recording was also identified as a challenge as part of the PFR sessions. Various household issues were identified and those were often issues the home visitors cannot control such as family sickness. Doing two programs, technology and scheduling issues were also identified by all seven home visitors. Examples from the transcripts are included in 55 HAN ET AL. the Table 3. Our goal was to identify these challenges and issues early on and address them as we move forward with implementation. We will describe how we addressed some of these issues in the discussion. Q3. Suggestions and Recommendations Based on their initial experience in completing the 10-session PFR intervention with one family, we asked home visitors about any suggestions or recommendations to promote the sustainability of PFR in EHS. Figure 4 reports these suggestions and the frequency of codes. Table 4 describes the examples of summary quotes by home visitors related to each code. Figure 4. Suggestions for the sustainability of PFR implementation by home visitors. Note. Each parenthesis reports total frequency of each code and the number of unique references/the number of home visitors (maximum 7). Home visitors addressed ‘ongoing support’ as a critical element for sustainability. They presented various ideas of different forms of ongoing support as indicated in Table 4. They also suggested some ideas for the PFR trainer and PFR coach as well as ways to integrate PFR with EHS programs. These suggestions are also presented in Table 4. EVIDENCE-BASED PARENTING INTERVENTION 56 TABLE 4 Suggestions/recommendations for sustaining the Promoting First Relationships intervention in Early Head Start Suggestions/ Recommendations Examples Ongoing support  Need for ongoing support such as group reflective practice sessions, coaching, or supervision.  Train supervisors, not just home visitors  Need for individual coaching as well as group coaching  At least monthly sessions for any types of support  Having a local trainer/coach Suggestions for PFR  Flexibility with length of training and trainer  Depending less on video. Too much video watching  One-on-one online coaching instead of paired coaching for PFR coach  PFR trainer in close proximity Integrating PFR and EHS  Beneficial to integrate PFR into EHS  Various ideas on how to implement: e.g., doing PFR first and doing PAT after, doing PFR 3 weeks and 1 week exclusively for other EHS services in each month, meshing PFR with PAT, using PFR as supplement Organization skills  Better organization skills are needed when using manualized program Review with families  Need for continued review of PFR with family even after they are done with sessions. Despite a number of challenges and implementation issues home visitors faced, overall these data show that the majority of home visitors identified benefits of integrating PFR into EHS. Furthermore, they suggested numerous ideas for sustainability such as having ongoing support and developing better organization skills to promote the use of PFR within the EHS program. DISCUSSION This study uses Early Head Start home visitor interview data to identify lessons that can be drawn from our experiences in embedding an evidence-based intervention within typical EHS program activities. Our research questions focused on home visitors’ experiences with 57 HAN ET AL. implementation process of PFR into their home visits, challenges and issues identified while beginning to implement the intervention, and their recommendations and suggestions for sustaining the PFR intervention as part of EHS practice. At the time of their interviews, each home visitor had completed the 10-session PFR intervention with their first family. Thus, the interviews provided a timely means of investigating initial experiences in implementing the new PFR content home visitors had just learned. The home visitors provided rich information relevant to the further implementation of PFR within Early Head Start home visits. They were receptive to implementing PFR and reported largely positive experiences. They also identified a number of issues and challenges and offered suggestions for sustainability. An immediate goal of this research was to inform our implementation process as the PFR intervention was being scaled to reach more EHS families, prior to training an additional cohort of EHS home visitors. However, given potential challenges related to a mismatch between settings where interventions are developed and the settings in which they are later delivered (Supplee & Metz, 2015), these findings have more general applicability to researchers (or EHS agencies) seeking to apply evidence-based prevention and intervention strategies as part of EHS practice. We will describe the lessons we learned from this study of initial implementation and will identify some ways in which we were able to employ this information in feedback loops that informed further implementation processes. The lessons below apply to PFR as well as to other interventions that are delivered by home visitors or other agency-based staff. First, gathering intentional implementation data aids the researcher in understanding whether and how the target intervention can be successfully situated within the existing agency. There are several key considerations in establishing a match between an intervention and an EHS program. It is important to have an in-depth understanding of the requirements of the intervention, its timing and duration, as well as staff requirements and program capacities (Vu, Hustedt, Pinder, & Han, 2015). Once that initial match has been made, notions of intervention philosophy, time investment on the part of field staff, and technical issues are important indicators of whether the intervention will be sustainable. For example, if staff members lack the technical expertise to upload or edit videos and this is a necessary component of the intervention, supplemental training will be needed or programs may need to seek an intervention with requirements that better match the skills of current staff. In our case, members of the research team provided EHS staff with additional support needed to master the technology involved in PFR. This allowed home visitors to be more independent with the use of technology after receiving the training. As a general lesson, it is important to gather implementation data on an ongoing basis to inform continued implementation. Second, gathering data specifically from field staff allows for evolution of infrastructure to better support implementation. Timing of data collection is also key in information gathering. In our case we were able to adapt conditions within the EHS agency in response to home visitor feedback. All the home visitors that we interviewed provided suggestions about the need for ongoing additional support for PFR, beyond the support already provided by the intervention developers through the initial training process. In response, we created a position within the EHS agency for a job-embedded coach, trained by the PFR developers but available locally to provide consultation as needed. This job-embedded coach provided within-agency infrastructure for in–the-moment coaching and technical support related to intervention implementation. Also in response to home visitor feedback, we made modifications to the supervision process, and revisions to our plans for group reflective practice sessions, which are now led by the job- embedded coach. EVIDENCE-BASED PARENTING INTERVENTION 58 Third, the creation of systematic feedback loops builds a process within the agency that allows for bi-directional sharing of information as well as the establishment of time and energy devoted to examining implementation data. In our work, we have identified specific time points to collect feedback from field staff and participating families in order to garner timely implementation information. Important to this process is the need to collect information that is essential to the implementation (and not just the content of the intervention) as well as the systematic analysis of information that accompanies effective implementation. Lastly, another lesson involves the need for researchers to maintain an open dialogue with the intervention developers and/or intervention resource as implementation proceeds. This provides a capacity for rapid response to emerging implementation issues and helps build a foundation for sustainability. Intervention fidelity is a key issue that will need to be addressed when training field staff on any intervention. Additionally, in our case, there were scheduling difficulties based on the availability of local EHS staff and the PFR trainer, and we were able to reschedule future trainings so that they were concentrated on days of the week when home visitors were more likely to be working in the office and less likely to be in the field. On a larger scale, when we began developing the position for an embedded PFR coach within the EHS program, we were able to gain input from the PFR developers on how to turn a new staff member into an expert trainer and what types of support for home visiting staff to transition to this new position. Although this level of communication with developers is not possible in all cases, agencies can connect with the entity that is sponsoring or supporting the target intervention (e.g. professional development agency, school district, local non-profit, etc.) In conclusion, when an evidence-based-intervention is implemented in the local setting, it is important to examine the process and conditions of implementation and identify potential challenges at early stages of implementation. Our study provides an example of how data-driven conversation and feedback from field staff could facilitate sustainability of the intervention. Successful implementation depends on data-driven conversation among field staff, researchers, and administrators during the implementation process. Ongoing and frequent feedback loops between researcher and field staff are crucial to successful implementation of the intervention. While our study identified key themes in the context of interviews with home visiting staff during initial implementation of PFR, the home visitors raised issues that are likely to be central to implementation of new interventions in other contexts. In our case, the collection of data during the initial implementation process allowed us to make necessary adjustments that facilitated continued implementation and promoted sustainability of the intervention in the longer term. In the context of an increasing emphasis on adapting evidence-based interventions for use in settings such as Early Head Start, and examining the outcomes associated with these interventions, the scientific study of implementation plays a critical role. REFERENCES Blase, K., Fixsen, D.L., Sims, B. J. & Ward, C.S. (2014). Implementation science: Changing hearts, minds, behavior, and systems to improve educational outcomes. Oakland, CA: The Wing Institute. Buffering Toxic Stress Consortium Principal Investigators, Meyer, A. L., & Fortunato, C. (2013). Parenting interventions in Early Head Start: The Buffering Toxic Stress Consortium. Zero to Three, 34(2), 73-86. Daro, D. (2006). Home visitation: Assessing progress, managing expectations. Chicago, IL: Ounce of Prevention Fund and Chapin Hall Center for Children. Retrieved from http://www.theounce.org/pubs/HomeVisitation.pdf http://www.theounce.org/pubs/HomeVisitation.pdf 59 HAN ET AL. Filene, J. H., Kaminski, J. W., Valle, L. A., & Cachat, P. (2013). Components associated with home visiting program outcomes: A meta-analysis. Pediatrics, 132, S100. doi: 10.1542/peds.2013-1021H Fixsen, D.L., & Blase, K.A. (2008). Drivers framework. Chapel Hill, NC: The National Implementation Research Network, Frank Porter Graham Child Development Institute, University of North Carolina at Chapel Hill. Goyal, N. K., Teeters. A/. & Ammerman, R. T. (2013). Home visiting and outcomes of preterm infants: A systematic review. Pediatrics, 132, 502-516. doi: 10.1542/peds.2013-0077 Hallam, R., Han, M., Vu, J., & Hustedt, J. T. (2017). Meaningful family engagement in early care and education programs: The role of home visits in promoting positive parent-child interaction. Advance in Early Education and Day Care, 20, 51-66. doi:10.1108/S0270-402120160000020007 Halle, T., Metz, A., & Martinez-Beck, I. (2013). Applying implementation science in early childhood programs and systems. Baltimore, MD: Brookes Publishing. Jones Harden, B., Chazan-Cohen, R., Raikes, H., & Vogel, C. (2012). Early Head Start home visitation: The role of implementation in bolstering program benefits. Journal of Community Psychology, 40(4), 438-455. doi: 10.1002/jcop.20525 Kelly, J. F., Zuckerman, T., & Rosenblatt, S. (2008a). Promoting First Relationships: Relationship-focused early intervention approach. Infants & Young Children, 21, 285-295. doi: 10.1097/01.IYC.0000336541.37379.0e Kelly, J. F., Zuckerman, T. G., Sandoval, D., & Buehlman, K. (2008b). Promoting First Relationships: A program for service providers to help parents and other caregivers nurture young children’s social and emotional development (2 nd ed.). Seattle, WA: NCAST Programs, University of Washington. Love, J., Kisker, E., Ross, C. M., Schochet, P. Z., Brooks-Gunn, J., Paulsell, D., et al. (2002). Making a difference in the lives of infants and toddlers and their families: The impacts of Early Head Start. Volumes I-III: Final technical report [and] appendixes [and] local contributions to understanding the programs and their impacts. Washington, DC: U.S. Department of Health and Human Services, Head Start Bureau. Merriam, S.B. (1998). Qualitative research and case study applications in education (2 nd ed.). San Francisco: Jossey-Bass. Mertens, D. (2010). Research and evaluation in education and psychology. Integrating, diversity with quantitative, qualitative, and mixed methods. California: SAGE. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2 nd ed.). Thousand Oaks, CA: Sage. Nelson, E. M., & Spieker, S. (2013). Intervention effects on morning and stimulated cortisol responses among toddlers in foster care. Infant Mental Health Journal, 34(3), 211-221. doi: 10.1002/imhj.21382 Odom, S. (2009). The tie that binds: Evidence-based practice, implementation science, and outcomes for children. Topics in Early Childhood Special Education, 29(1), 53-61. doi:10.1177/0271121408329171 Oxford, M. L., Fleming, C. B., Nelson, E. M., Kelly, J. F., & Spieker, S. J. (2013). Randomized trial of Promoting First Relationships: Effects on maltreated toddlers' separation distress and sleep regulation after reunification. Children and Youth Services Review, 35(12), 1988-1992. doi: 10.1016/j.childyouth.2013.09.021 QSR International Pty. (2014). NVivo qualitative data analysis software Version 10. Spieker, S. J., Oxford, M. L., Kelly, J. F., Nelson, E. M., & Fleming, C. B. (2012). Promoting First Relationships: Randomized trial of a relationship-based intervention for toddlers in child welfare. Child Maltreatment, 17, 271-286. doi:10.1177/1077559512458176 Supplee, L. H., & Metz, A. (2015). Opportunities and challenges in evidence-based social policy. Social Policy Report, 28 (4), 3–16. Sweet, M., & Appelbaum, M. I. (2004). Is home visiting an effective strategy? A meta-analytic review of home visiting programs for families with young children. Child Development, 75, 1435-1456. doi: 10.1111/j.1467.8624.2004.00750.x Vu, J. A., Hustedt, J. T., Pinder, W. M., & Han, M. (2015). Building early relationships: A review of caregiver-child interaction interventions for use in community-based early childhood programs. Early Child Development and Care, 185, 138-154. doi: 10.1080/03004430.2014.908864 Zero to Three (2009). Reaching families where they live. Retrieved from http://www.zerotothree.org/public- policy/policy-toolkit/homevisitssing_mar5.pdf Zigler, E. & Seitz, V. (1982). Social policy and intelligence. In R.J. Sternberg (Ed.). Handbook of human intelligence. New York: Cambridge University Press. Zigler, E., Pfannenstiel, J. C., & Seitz, V. (2008). The Parents as Teachers program and school success: A replication and extension. Journal of Primary Prevention, 29, 103-120. doi: 10.1007/s10935-008-0132-1 http://www.zerotothree.org/public-policy/policy-toolkit/homevisitssing_mar5.pdf http://www.zerotothree.org/public-policy/policy-toolkit/homevisitssing_mar5.pdf