Dialog, 19(3), 99-103 Copyright © 2016, ISSN: 1930-9325 RESEARCH-TO-PRACTICE SUMMARY Studying Implementation Process Matters: Implementing an Evidence- Based-Intervention in Early Head Start Myae Han, Rena Hallam, Jason T. Hustedt, Jennifer A. Vu, and Juana Gaviria-Loaiza University of Delaware The present study summarizes lessons from implementing an evidence-based parent-child interaction intervention, Promoting First Relationships, within an Early Head Start (EHS) home visiting program. The study documented seven home visitors’ early experiences and feedback with intervention implementation examining overall experiences in using the intervention, challenges and issues identified during implementation, and recommendations and suggestions for sustainability of the intervention within the EHS program. Findings indicate that overall, home visitors reported positive experiences with the implementation process. Some negative experiences were identified in relation to the lengthy training process and changes in home visiting practice. Challenges included learning about intervention content and managing household issues. Recommendations for sustaining the intervention included providing ongoing support and having flexibility with the training process. Keywords: Early Head Start, parenting intervention, home visiting, implementation Evidence-based programs are of high interest to researchers and policymakers in the field of human service and education. However, there have been growing concerns about scaling evidence-based interventions in community settings because the context can be very different from the context where the intervention was initially developed. Recent research has demonstrated that even well-developed programs may fail or be less effective due to implementation issues “on the ground” (Jones-Harden, Chazan-Cohen, Raikes, & Vogel, 2012; Supplee & Metz, 2015). Particular attention has been paid to the role of implementation in the professional development of the early care and education workforce as an important bridge to high quality practice (Halle, Metz, & Martinez-Beck, 2013; Odom, 2009). Implementation science provides the field with a framework for conceptualizing how to study and monitor processes regarding the preparation and support for field staff as well as the quality of intervention implementation with children and families. From an implementation perspective, the likelihood of successfully launching and sustaining an evidence-based intervention relies on three types of drivers: competency, organization, and leadership (Fixsen & Blase, 2008). Competency drivers relate to the selection, professional development, and support of staff who are the actual implementers of a specific intervention. Organizational drivers 100 HAN ET AL. include the gathering and utilization of data to inform decision-making, the development of responsive administrative procedures to support effective implementation, and the need to collaborate with external parties when necessary. Leadership drivers refer to the importance of effective agency leaders and leadership teams in managing the overall process. One of the key premises of implementation science and the focus of this study is the importance of feedback loops to ensure adequate conditions are in place to promote high quality execution of an intervention as aligned to the above three organizational drivers (Fixsen & Blase, 2008). Feedback loops are established when structures are put in place to gather and effectively use data to make decisions to refine the implementation of an intervention. This process allows for the agency or entity to quickly solve problems and barriers that arise during early implementation (Metz & Albers, 2014). This aspect is particularly significant when field or agency staff are the primary agents of change. Feedback loops enable researchers and practitioners to collaboratively make implementation decisions based on authentic data from the field. Feedback loops between field staff, program administrators, and researchers should be intentional – meaning that both the type of information and the frequency of its collection should be considered in light of agency operations and features of the intervention. The Current Study To understand the process of implementation, the current study uses Early Head Start (EHS) as a context for carrying out the Promoting First Relationships (PFR; Kelly, Zuckerman, Sandoval, & Buehlman, 2008) intervention and examined the perspectives of seven EHS home visitors during the first year of implementation. PFR was developed to guide caregivers in building nurturing and responsive relationships with their young children from birth to age three, thus encouraging growth in the children’s social-emotional development, language, and cognition. PFR is a manualized, strengths-based intervention delivered by community-based service providers who first participate in an extensive training and fidelity process. The intervention is based on video reflection between caregivers and trained PFR staff, focusing on positive feedback and based on principles of attachment theory. The PFR training process involved attending a 3-day workshop followed by approximately 20 weeks of mentored online support. Our study addresses three questions: 1. How did the home visitors perceive implementation of the PFR intervention as part of an Early Head Start home visiting program? 2. What kinds of challenges and issues were identified during implementation? 3. What kinds of suggestions and recommendations were identified by the home visitors to promote sustainability of the PFR intervention in the Early Head Start program? METHOD We employed semi-structured interviews. The interview questions were about overall experiences with the PFR training process and content, challenges and issues of PFR implementation, home visitor experiences with participating families, and recommendations for sustainability. Each interview session took about 30-40 minutes and was audio-taped. IMPLEMENTATION PROCESS MATTERS 101 For data analysis, the constant-comparative method (Merriam, 1998) and systematic coding (Miles & Huberman, 1994) were used for the qualitative analysis process. We used qualitative data analysis software, NVivo 10 (QSR International, 2014) for the coding process and visualization of the data. Peer debriefing and member checks were then conducted twice with the entire research team. SUMMARY OF FINDINGS Q1. Home Visitors’ Experience of the PFR Implementation Process Overall, the majority of comments by home visitors were related to positive experiences with implementation. The most prevalent positive experiences were about the content of the PFR intervention, use of video, positive experiences with families, the training process, and increasing self-confidence and skills. Some negative experiences were also expressed by the home visitors in regard to the lengthy training process and discomfort with videotaping. Q2. Challenges and Issues with Implementation Asking reflective questions and uncomfortableness with video recording were identified as a challenge for PFR intervention content. Various household issues were also identified and those were often issues the home visitors could not control such as family sickness and sibling presence, etc. Doing two programs (Parents as Teachers as well as PFR), technology, and scheduling issues were also identified as challenges. Q3. Suggestions and Recommendations Home visitors addressed ‘ongoing support’ as a critical element for sustainability. They suggested various different forms of ongoing support such as individual and group coaching, training supervisors, and changes in intensity of coaching. Home visitors also suggested having flexibility with the length of training, depending less on video during the training, and having a local trainer. RECOMMENDATIONS FOR IMPLEMENTATION This study was conducted as part of our PFR implementation process to facilitate feedback loops between home visitors and researchers, and with a goal of informing further implementation. The lessons we learned from this study are described below and are applicable to other interventions delivered by home visitors or agency-based staff. First, gathering intentional data on the target intervention prior to implementation is critical to decide whether the intervention can be successfully situated within the existing agency. The data could include the requirements of the intervention, its timing and duration, staff 102 HAN ET AL. requirements, program capacities, intervention philosophy, time investment on the part of field staff, and technical issues (Vu, Hustedt, Pinder, & Han, 2015). Second, conducting well-timed data collection while early implementation is underway can provide key information to guide further refinement in intervention implementation. We conducted interviews right after the home visitors had delivered the intervention to their first family. In response to feedback from the home visitors, we created a position within the EHS agency for a job-embedded coach who is available locally to provide consultation as needed, as well as in–the-moment coaching and technical support including assistance with technology. Written resources were also developed to support use of technological devices and applications that may have been unfamiliar to some. Third, another lesson involves the need for researchers to maintain an open dialogue with the intervention developers and/or other intervention resources as implementation proceeds. This provides a capacity for rapid response to emerging issues and helps build a foundation for sustainability. For example, in our case, there were scheduling difficulties based on the availability of home visitors and the PFR trainer located in different time zone. By addressing the issue early with the intervention developers, the trainers were able to accommodate scheduling issues more promptly with home visitors. We also developed a plan for a locally available PFR coach for future training in the intervention. Lastly, having a plan for creation of systematic feedback loops is a key for successful implementation. In our work, we have identified specific time points to collect feedback from field staff and participating families in order to garner timely implementation information. Important to this process is the need to collect information that is essential to the implementation (and not just the content of the intervention) as well as the systematic analysis of information that accompanies effective implementation. This allows for bi-directional sharing of information as well as the establishment of time and energy devoted to examining implementation data. In conclusion, when an evidence-based-intervention is implemented in the local setting, it is important to examine the process and conditions of implementation and identify potential challenges at early stages of implementation. Our findings illustrate how data-driven conversation and feedback from field staff allow for adjustments while implementation is underway and thus promote future sustainability of the intervention. REFERENCES Fixsen, D.L., & Blase, K.A. (2008). Drivers framework. Chapel Hill, NC: The National Implementation Research Network, Frank Porter Graham Child Development Institute, University of North Carolina at Chapel Hill. Halle, T., Metz, A., & Martinez-Beck, I. (2013). Applying implementation science in early childhood programs and systems. Baltimore, MD: Brookes Publishing. Jones Harden, B., Chazan-Cohen, R., Raikes, H., & Vogel, C. (2012). Early Head Start home visitation: The role of implementation in bolstering program benefits. Journal of Community Psychology, 40(4), 438-455. doi: 10.1002/jcop.20525 Kelly, J. F., Zuckerman, T. G., Sandoval, D., & Buehlman, K. (2008). Promoting First Relationships: A program for service providers to help parents and other caregivers nurture young children’s social and emotional development (2 nd ed.). Seattle, WA: NCAST Programs, University of Washington. Metz, A., & Albers, B. (2014). What does it take? How federal initiatives can support implementation of evidence- based programs to improve outcomes for adolescents. Journal of Adolescent Health, 54, 592-596. Merriam, S.B. (1998). Qualitative research and case study applications in education (2 nd ed.). San Francisco: Jossey-Bass. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2 nd ed.). Thousand Oaks, CA: Sage. IMPLEMENTATION PROCESS MATTERS 103 Odom, S. (2009). The tie that binds: Evidence-based practice, implementation science, and outcomes for children. Topics in Early Childhood Special Education, 29(1), 53-61. doi:10.1177/0271121408329171 QSR International Pty. (2014). NVivo qualitative data analysis software Version 10. Supplee, L. H., & Metz, A. (2015). Opportunities and challenges in evidence-based social policy. Social Policy Report, 28 (4), 3–16. Vu, J. A., Hustedt, J. T., Pinder, W. M., & Han, M. (2015). Building early relationships: A review of caregiver-child interaction interventions for use in community-based early childhood programs. Early Child Development and Care, 185, 138-154. doi: 10.1080/03004430.2014.908864