International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 DOI: 10.18357/ijcyfs102-3201918853 AN EVIDENCE-BASED PROGRAM MODEL FOR FACILITATING THERAPEUTIC RESPONSES TO PAIN-BASED BEHAVIOR IN RESIDENTIAL CARE Martha J. Holden and Deborah E. Sellers Abstract: Children and young people in residential care have often lived lives saturated with loss, neglect, rejection, and traumatic experiences. Children express the pain of trauma in various ways, namely pain-based behaviors manifesting in ways that often leave their care givers confused, frustrated, frightened, angry or exhausted. For residential caregivers to respond to children and young people in a consistent and therapeutic manner, residential environments must provide an ethos of respect, caring, and trust, creating a safe place for children and staff to live and learn together. This paper describes the Children and Residential Experiences (CARE) model, its implementation, and evidence for its effectiveness. CARE is a trauma-informed, principle-based, multi-component program designed to enhance the social dynamics in group care settings and help agencies create a living environment that provides developmentally enriching experiences for children in their care. By incorporating the CARE principles throughout all levels of the organization and into daily practice, the CARE program model has been shown to improve the capacity of staff to establish positive developmental relationships with the children in their care, offer developmentally enriching experiences and a “sense of normality”, and create cohesion and congruence throughout the organization. Through consistent and predictable compassionate and responsive interactions with adults, as well as opportunities to overcome challenges and to experience successful learning opportunities, children can grow, develop and thrive. Keywords: residential child care, therapeutic residential care, trauma-informed care, evidence based programs, group care Martha Holden MS (corresponding author) is the Project Director of the Residential Child Care Project at the Bronfenbrenner Center for Translational Research at the College of Human Ecology at Cornell University, 35 Thornwood Dr. Ithaca, NY 14850. Email: mjh19@cornell.edu Deborah Sellers PhD is the Director of Research of the Residential Child Care Project at the Bronfenbrenner Center for Translational Research at the College of Human Ecology at Cornell University, 35 Thornwood Dr. Ithaca, NY 14850. Email: des256@cornell.edu mailto:mjh19@cornell.edu mailto:des256@cornell.edu International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 64 Addressing Pain-Based Behavior in Residential Care Imagine a child falling off a bike, feeling pain, and seeing the blood running down her leg. She screams in pain, cries, and throws the bike into the ditch. No reasonable adult would walk over and yell at the child for throwing her bike and tell her that the bike will be taken away from her as a consequence. Seeing the blood and understanding the physical pain the child is experiencing, most caring adults would provide a comforting and supportive response (e.g., close physical proximity and help with getting the bike back home) as well as tangible aid (e.g., clean and bandage the cut). In addition, the adult might soon motivate the child to get back on the bike and avoid making the same mistake that resulted in the fall. Providing comfort and a safe haven when hurt, and helping children overcome their fears, meet challenges, and learn from experience are responses that help children grow and develop. By overcoming adversity and making sense of their pain and emotions, children become stronger. When children are experiencing psychoemotional pain and scream in pain, cry, or throw something, they are sometimes punished by the adults responsible for caring for them. This is the pattern of behaviors and responses for many children in care who have experienced trauma, loss, and separation. Without an apparent trigger or cause, they may be experiencing psychoemotional pain and expressing it with pain-based behaviors (Anglin, 2002). Without the tangible or physical evidence of what is causing the behavior, sometimes adults are not in tune with the pain the child is experiencing. No child should be punished for behavior that is a result of pain — either physical or emotional. That is inflicting pain on top of the pain they already feel, which only increases the damage. The ability to deal with children’s psychoemotional pain without inflicting additional painful experiences on them is one of the biggest challenges for caregivers and therapeutic residential care. The Children Children enter the child welfare system for a variety of reasons, including parents that are unwilling or unable to care for them, abuse or neglect in their homes, and violence and poverty in their communities. Although these children are most often placed in foster or kinship care, children with serious developmental concerns or behaviors that put them “at risk” at home or in the community are often placed in residential care or end up in the juvenile justice system. Many of these children are on developmental trajectories that place them outside of social norms, such as antisocial, homeless, delinquent, or poor mental health trajectories. When compared with the general population, the population of young people living in residential care has a disproportionately high rate of emotional and behavioral problems (Burns et al., 2004) and is at high risk of experiencing poor developmental outcomes throughout the life course. In addition to experiencing parental maltreatment and other forms of trauma, children in residential care often have a history of unsuccessful placement in foster care (Zinn, DeCoursey, George, & Courtney, 2006), and a host of other risk factors that impair their healthy development (Ryan & Testa, 2005). International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 65 To ameliorate the risk of poor developmental outcomes, children typically receive some form of treatment by professional clinicians to address specific behavioral and emotional problems while in care. Equally important is their need for healthy developmental and normalizing experiences throughout the day as well as protection from experiencing additional trauma, negative and defeating life situations, and other toxic experiences. In residential care, children need to experience a therapeutic milieu that supports their rehabilitation (James, 2011) and provides them opportunities to grow, develop and thrive. Providing a therapeutic milieu during the time that children are not in individual clinical care — the “other 23 hours” (Trieschman, Whittaker, & Brendtro, 1969) — requires a program model that helps residential staff understand the impact of trauma on a child’s development, how to respond to the pain-based behavior caused or triggered by trauma, and how to provide trauma-informed care. All activities, routines, expectations, and interactions should be designed taking into account the impact of overwhelming stress and trauma on a child. The Purpose of Care Thus, the purpose of therapeutic residential care is to provide a safe place for children to experience caring relationships, learn new skills, and develop a sense of normality, including the everyday small pleasures of life we all cherish. A therapeutic milieu offers children some breathing room, a respite from the stress and overwhelming challenges that they experience in their homes, schools, and communities. The child is surrounded with adults who act as teachers, empathizers, coaches, defenders, advocates, and mentors to help support and protect them as well as assist them in developing and practicing necessary life skills. By providing opportunities for normal developmental experiences and support to children in their attempts to “catch up” in areas where they are lagging, therapeutic residential care can enhance the child’s opportunities for achieving normal developmental outcomes (Hawkins-Rodgers, 2007; Maier, 1987, 1991). This is best accomplished by providing a well-designed and nurturing developmental setting where children can develop a core set of assets (McNeely & Blanchard, 2009) such as competence, self-efficacy, relationship skills, emotional regulation, and empathy. This requires active support, carefully orchestrated opportunities to practice and learn psychosocial skills, and intentional interactions with caring, informed, authentic, and engaged adults. Providing a safe place to breathe and practice new skills supported by competent and caring adults helps children to thrive and realize more of their potential. The Challenge This is not an easy task. Many children enter therapeutic residential settings with negative ideas of the people and the place based on past experiences. They walk in the door hostile, or perhaps withdrawn, with little optimism or hope of being successful, being cared for, or being capable of thriving in the environment. This is not a surprising position considering the cumulative effect of their life experiences that have left them alone, rejected, sometimes feared and often fearful, and in care but not cared about. The challenge is to provide a robust developmental setting filled with supportive and empathic adults providing positive and appropriate developmental life International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 66 experiences, including time for play and having fun together, that prepare them to meet life’s challenges as well as take opportunities to expand their learning. The Active Ingredient The child–adult relationship is commonly recognized as the key to providing effective interventions. The quality of that relationship has been described as “the active ingredient on which effectiveness of all other program elements depend” (Li & Julian, 2012, p. 163). Studies, both empirical and those representing the voice of the child, continually indicate the importance of positive and supportive worker–child relationships (Harder, Knorth, & Kalverboer, 2013; Izzo et al., 2014; Lester, Goodloe, Johnson, & Deutsch, 2018; Moore, Moretti, & Holland, 1997). This relationship not only provides a safe haven for children when distressed, but provides the secure base children need to seek out new experiences and take on new learning and challenges. Most children, when feeling threatened and experiencing fear and distress, naturally respond by turning to a trusted adult for assistance. Engaging with responsive, caring, and competent adults helps children learn to deal with and manage overwhelming emotions. Children develop and thrive by turning to adults who provide support and comfort when they experience distress; in this way children learn to self-regulate and cope with these emotions. These adults also provide a secure base that allows children to be free to engage in growth opportunities (Feeney & Collins, 2015). Unfortunately, children with histories of interactions with inadequate, uncaring, unresponsive, or abusive adults have abandoned the strategy of depending on adults for assistance and have developed other maladaptive or ultimately self-defeating responses. They have developed patterns of pain-based behaviors and responses such as aggression, rigid and inflexible behaviors, withdrawal, impulsive outbursts, and self-injury. At the core, the child’s environment should be and feel safe — the adult–child relationship is linked to perceptions of safety (Moore, McArthur, Death, Tilbury & Roche, 2017). This is also critical to ensuring that the child feels safe and secure and is willing to seek out adults for assistance when distressed. These same adults also are key to motivating children to learn new skills and practice a new way of being. They provide the encouragement and security children need to move outside their comfort zone and take on new challenges (Feeney & Collins, 2015). Providing a safe and therapeutic environment filled with responsive adults and opportunities to meet and overcome challenges, learn new skills, engage in meaningful and satisfying relationships and experiences, and find some joy in their daily lives is the core of therapeutic residential care. We will now offer an overview of the Children And Residential Experiences (CARE) program model and discuss the CARE implementation process before presenting some of the evidence of effectiveness including impacts on selected pain-based behaviors. International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 67 The CARE Program Model and its Implementation The CARE Model If children are to thrive, they need to have predictable, engaging, responsive, compassionate, secure, purposeful, and stimulating experiences on an ongoing basis. This enormous challenge can be accomplished by creating a culture and living environment that provides developmentally enriching relationships and experiences and a “sense of normality” (Anglin, 2002; Li & Julian, 2012). CARE is designed to accomplish this task. By incorporating six evidence-informed principles (see Table 1) and three key processes (reflective practice, data- informed decision-making, and participatory management) throughout all levels of the organization and into daily practice, an ethos develops that supports and expects developmental relationships in a trauma-sensitive environment (Holden, 2009; Izzo et al., 2016; Nunno, Smith, Martin, & Butcher, 2017). Table 1 CARE Principles that Guide Agency-Wide Programming Programming in residential care settings needs to be… Rationale Relationship based Positive relational experiences with caregivers help children form healthy internal working models of adult–child relationships and build their capacity for healthier relationships. Trauma informed Trauma histories influence children’s programming needs and their abilities to meet expectations and participate in activities. Developmentally focused Children’s life trajectories improve when caregivers provide opportunities for normative developmental experiences and adapt their expectations to meet each child’s unique needs. Family involved Children benefit when caregivers understand and adapt to families’ cultural norms and beliefs, and promote active family involvement. Competence centered Opportunities for children to practice problem solving, coping skills, and other life skills allow children to develop greater self-efficacy, social competence, and capacity to manage life circumstances. Ecologically oriented Children’s opportunities for growth and development increase by adapting key features of the physical and social environment to support their successful engagement. The CARE principles serve as an organizing framework that enables the organization to realign or reallocate resources, to set priorities, and to create a culture that helps children grow and develop through enhanced interactions focused on strengthening attachments and relationships, building competencies, adjusting expectations to account for children’s developmental stage and trauma history, involving families in the child’s care and treatment, and enriching dimensions of the environment to create a more therapeutic milieu (Holden, 2009). High-expectation messages, respectful interactions, reflective dialogue, collaboration, and flexibility are practiced throughout the organization. CARE offers a set of principles and processes to guide practice by training and supporting all staff to use their relationships, creativity, and professional judgment in creating International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 68 enriching experiences for children. This approach cultivates personal investment on the part of staff members and serves to reduce the sense of being constrained or controlled by bureaucratic rules. Staff are able to respond with more sensitivity to emotionally distressed children and pain- based behavior. With the focus on the developmental relationship as their primary therapeutic tool, staff are better equipped to diffuse momentary situations that could otherwise culminate in dangerous or destructive outcomes. Potential crisis events become opportunities to work together with the children to make sense of their feelings and find other ways to cope with their emotions. The relationship becomes a source of support when children are feeling alone, angry, fearful, or overwhelmed. Children learn to go to staff when they are feeling stressed and are flooded by emotions and see staff as a source of support and strength (Feeney & Collins, 2015). At the same time, as staff learn to understand and apply developmentally and trauma-sensitive approaches, they create fewer situations where children feel frustrated or angry. The environment becomes less chaotic, more predictable, and feels more normal to all involved. Organizational Congruence and Collective Efficacy To create a true therapeutic milieu, the children need to have a consistent, congruent experience regardless of the individual staff member or activity. The organization itself must provide the culture, climate, and underlying philosophy that creates the safe haven, the relief from stress and burdens, the motivation to engage and stay engaged even when it is difficult, and the desire to recognize and seize opportunities for growth. Children and staff alike need to feel cared for and about, and appropriately challenged to grow, develop, and thrive. The CARE model honors the true complexity of residential work and strives to help staff develop their abilities and achieve the necessary growth mindset (Dweck, 2008; Kegan & Lahey, 2009) to deal effectively with such complexity. CARE organizations create “holding environments” that allow staff to be reflective and to examine and develop insights about their experiences (Kahn, 2005). By working in alignment with the six principles and building a developmental relationship, staff are able to be genuine within their individual relationships with children and, at the same time, provide a consistent, predictable environment for the children. A key dimension of this alignment is recognizing and responding proactively to pain and pain-based behaviors; their own as well as the childern’s. Implementing the CARE Model Cornell consultants collaborate with agency leaders and an agency CARE implementation team to implement the CARE model. This 3- to 4-year systematic undertaking involves re- orienting agency practices around the six evidence-informed principles of the CARE model (see Table 1). For most agencies, this reorientation of practices requires changes in theoretical perspectives, organizational norms, and role expectations. The role of the CARE consultants is to help build capacity within the organization to implement and sustain the CARE program model in International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 69 a manner that helps the organization achieve its mission. Every organization implements CARE in a slightly different manner based on its mission, clients, and primary tasks. The implementation is a collaborative process. Agency leaders make decisions about how, when, and where to do what, with input from the CARE consultants. CARE consultants do not tell agencies what they should or should not do. Instead, they coach and counsel agency leaders in how to apply the CARE principles to their decisions and actions. The relationship between the Cornell team and the agency is based on the CARE principles and practices (e.g., using high-expectations messages, respectful interactions, open and reflective dialogue, collaborative decision-making, and flexibility based on the agency’s zone of proximal development; Vygotsky, 1978). The development of a CARE implementation team is an essential implementation activity. The role of the implementation team, which includes agency leadership, supervisors, and key training and clinical staff, involves developing an implementation plan, allocating adequate resources, creating a process for collecting data and using the results to help guide the implementation process, adjusting policies and procedures, job descriptions, and integrating the principles within their own work. In addition, the members of the implementation team provide support for frontline supervisors so that they can model and mentor staff as they incorporate the CARE principles into their practice. The team also builds structures and processes that facilitate application of the CARE principles and their eventual integration into the agency culture. The leadership and implementation team attend a 4-day leadership retreat to develop the initial implementation plan and learn more about the CARE model. Selected agency staff (CARE educators) are prepared to deliver the 5-day manualized training to all staff. CARE consultants provide quarterly on-site technical assistance (TA) visits to work with the implementation team and other agency staff. TA activities include observation and feedback, training and coaching for front-line supervisors, developing routines for reflective practice, and addressing organizational barriers to creating a more therapeutic milieu. When the CARE implementation process is 18 to 24 months in, or when all staff have been trained in CARE and other major milestones in the CARE implementation plan have been achieved, a midterm leadership retreat is scheduled. The CARE midterm leadership retreat is devoted to assessing progress with the implementation of CARE through review of the observations of the CARE consultants during the midterm visit, the results from CARE surveys administered in advance of the retreat, the administrative data addressing key goals or desired outcomes of CARE implementation, and the use of the CARE fidelity tools. At the midterm retreat the CARE fidelity tools guide examination of CARE implementation to date and help generate action plans to accomplish components of implementation that need additional attention and effort. In most organizations, full implementation of CARE is achieved towards the end of the 3- to 4-year initial implementation process. As with any intervention, however, maintaining and sustaining fidelity of CARE implementation requires on-going attention. Many agencies choose to continue their relationship with Cornell consultants who help to facilitate the on-going attention to implementation fidelity through annual visits. In addition, a community of practice has International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 70 developed among organizations that implement the CARE program model. Cornell staff facilitate regional and national events as well as on-going certification of, and additional training for, CARE educators, supervisors, and leaders. Approximately 4 to 5 years after initiating CARE implementation, agencies are offered the opportunity to apply for CARE certification. To do so, a CARE fidelity assessment visit is scheduled to determine if the agency has met the certification criteria. Prior to the visit the agency administers the post-implementation surveys, completes a self-assessment, and compiles selected administrative data to assist in assessing the fidelity and impact of CARE implementation. Two Cornell consultants make a 2- to 3-day on-site visit that includes interviews, focus groups, observations, and a CARE leadership meeting day to process all data, observations, and review the CARE fidelity tool. The written results of the assessment visit are reviewed by the Cornell CARE Certification committee to make certification recommendations. If the agency has not met the certification criteria, concrete recommendations for improvements will facilitate the agency’s effort to improve their implementation of CARE and meet the certification criteria. Figure 1. CARE theory of change. Evidence Supporting the Effectiveness of the CARE Model The CARE Theory of Change The CARE Theory of Change (TOC; see Figure 1) articulates the pathways through which CARE is expected to improve the socio-emotional and developmental outcomes for children (Holden et al., 2010; Holden, Anglin, Nunno, & Izzo, 2014). As outlined in the TOC, intensive agency-wide training sessions and a variety of technical assistance activities are expected to help personnel at all levels of the facility learn how to enhance interactions with children by consistently using the CARE principles (see Table 1) to guide those interactions. The TOC further indicates International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 71 that when staff–child interactions are enhanced as outlined, children develop more positive perceptions about themselves and their relationships and interactions with staff. These perceptions are ultimately expected to contribute to improving children’s social and emotional well-being and to reducing critical incidents involving pain-based behaviors (e.g., runaways, assault, and restraints). The addition of ongoing consultation is expected to help direct-care staff internalize the principles and thus incorporate them more readily and to help supervisors motivate and support their staff around the use of CARE principles. The TOC also guides evaluation efforts. A 5-year quasi-experimental evaluation included examination of the impact of CARE on staff knowledge and beliefs, adult–child interactions and relationships, and child behavior (Izzo et al., 2016). In addition, a single agency interrupted time series used 12 years of data to examine the impact of CARE implementation on restraint rates (Nunno, Smith, Martin, & Butcher, 2017). Results from both of these studies are summarized below. Quasi-Experimental CARE Evaluation A rigorous evaluation of the CARE program model was conducted with funding from The Duke Foundation. Agencies in North Carolina were invited to participate in a quasi-experimental, waitlist trial through presentations to a statewide association of group care agencies, letters sent directly to eligible agencies, and word of mouth. To participate agencies had to (a) serve primarily children referred by social services, (b) be licensed by a state agency, (c) be willing to be placed on a 12-month waitlist, and (d) have no previous exposure to the CARE program model. Sixteen agencies agreed to participate, including seven which were part of a larger parent organization with campuses across the state. Each campus was treated as a separate entity. Of the 16 agencies, one became ineligible due to a change in their target population, one discontinued due to change in administrative priorities, and one closed before implementation began. Though random assignment to the first or second cohort would have been ideal, various considerations necessitated a quasi-experimental design rather than a randomized group trial. Ultimately, six agencies participated in the first cohort, which began implementation in 2010, and seven agencies participated in the second cohort which began implementation in 2011. Figure 2 depicts the design of the study. Data collection included anonymous staff and confidential child surveys completed annually for 4 years in Cohort 1 and 5 years in Cohort 2. (As noted above, CARE implementation takes place over a 3-year period.) The first annual survey served as the baseline for Cohort 1 and the pre-baseline for Cohort 2. The staff survey included the Organizational Social Context instrument (Glisson & Hemmelgarn, 1998) as well as questions about the respondent’s current practice, knowledge, beliefs, and demographic characteristics. The child survey included questions about the child’s relationship with and attachment to the staff, as well as gender and race. In addition, the agency provided individual client characteristics (e.g., type of placement and length of time at the agency) and agency-wide counts of the behavioral incidents (aggression toward staff International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 72 or peers, property damage, runaways, and self-harm) that occurred each month for the duration of the study. Figure 2. Quasi-experimental wait-list design of CARE evaluation Staff reactions to the CARE training: Staff reactions to the CARE training were examined in surveys completed at the end of the 5-day CARE training. Of the 501 staff who completed post-training surveys, 85% reported the level of the subject matter to be “just right,” and 95% or more considered the presentations, student workbook, group discussions, and the overall program to be “good” or “excellent”. More than 90% of participants reported that they plan to (80%) or already (10%) use the material in their jobs and that they definitely (45%) or probably (47%) know the material. Figure 3. Percent of knowledge questions answered correctly by cohort and time. Staff knowledge and beliefs: Across the 13 participating agencies and the 5-year duration of the study, 701 staff members contributed a total of 1488 completed surveys. The survey included 13 questions that assessed knowledge about CARE. Figure 3 reports the average percentage correct International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 73 on the knowledge assessment by cohort and time point. Most staff were trained during the first 12 months of implementation so the primary increase in knowledge occurred over that time period. The staff survey also included a series of items designed to assess childcare beliefs, particularly reliance on rules and consequences (e.g., If you change the expectations or relax the rules, it will be harder to maintain compliance later) and extent of flexibility (e.g., Make it clear that the homework must be done before going to school tomorrow. Then give him a choice about doing it now, or waking up early to do it before school). Implementation of the CARE program requires less reliance on rules and consequences and increased flexibility. Changes in beliefs that occurred during the first year of implementation are presented in Figure 4 (Izzo & Smith, 2016). More detailed analyses of changes in staff knowledge and beliefs are currently underway. Flexibility Reliance on Rules and Consequences Figure 4. Changes in staff childcare beliefs. Child perceptions of relationship quality: As noted above, the child survey was administered annually in all 13 agencies by a member of the Cornell research team. Across the 13 participating agencies and the 5-year duration of the study, 933 children completed 1264 surveys. The average length of stay was 6.7 months, so relatively few children completed more than one survey. About 55% of respondents were male and 37% were non-white. The average age was 14.5 years. Just over half of the children were placed in the agencies through the child welfare system. For one-third of the children, residential care was their first placement, 23% had one previous placement, and 23% had two or more prior placements. The placement history of 22% of the children completing surveys was unknown. Table 2. Child Perceptions Survey: Six Scenarios for Measuring Quality of Relationship with Staff Type of scenario Sample item When you needed help: They tried to understand what I wanted. During recreational activities: I felt that they enjoyed spending time with me. When you misbehaved: Listened to me to try and understand why I acted that way. During routine day-to-day life: Gave me privacy when I needed it. When you were feeling upset: Talking with them made me feel better. How staff deal with your family: Respected my family or loved ones. International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 74 In the child perceptions survey, the sum of responses to 33 questions measured the quality of the relationship with staff as perceived by the respondent. The respondent was asked to rate how often the staff interact well with them in six scenarios listed in Table 2. The wording of the five response options varied slightly by scenario but in general represented frequency ranging from never to every day or never to always. As depicted in Figure 5 (Izzo & Smith, 2016), child perceptions of the quality of their relationship with staff improved during the implementation of CARE. Additional analyses are underway to assess whether and how agency and child characteristics influence these perceptions. Figure 5. Youth perceptions of the quality of their relationships with staff. Behavioral incidents: As with most agencies in the United States, licensing and accreditation regulations required every agency in the study to complete a written report for any incident involving substantial risk of danger to a child or staff member. These behavioral incidents indicate behavioral dysregulation and thus, in aggregate, represent a marker for the broader pattern of interaction between children and staff. For example, staff who learn how to set developmentally appropriate expectations will create fewer opportunities for children to feel frustrated or angry when they are unable to meet expectations. Similarly, staff that understand and respond with more sensitivity will be better able to diffuse momentary situations that might otherwise culminate in dangerous or destructive behaviors that would require formal documentation. Consequently, a decrease in behavioral incidents — verbal threats or physical aggression towards staff, verbal threats or physical aggression towards peers, an act or threat of self-harm, property destruction, and attempted or completed runaway — during the CARE implementation period relative to baseline would indicate improved interactions and relationships with staff. As reported by Izzo et al. (2016), agencies provided monthly counts of each type of incident and the average number of children each month so that per capita rates could be computed. 56.2 56.7 54.7 60.9 56.6 53.6 62.3 62.0 12mPre Baseline 12m 24m 36m Yo u th -A d u lt I n te ra ct io n Q u a li ty Stage of CARE Implementation Cohort 1 Cohort 2 International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 75 Interrupted time series regression models that controlled agency characteristics and accounted for the nesting of incidents within agencies were used to compare the trends during baseline to those during CARE implementation while controlling for agency characteristics (e.g., cohort, Organization Social Context profile). For all five types of incidents, there were significant decreases in incident rates of 4% to 8% per month for cohort 1. For cohort 2, the trends were different than cohort 1 for aggression towards peers and self-harm. Figure 6 illustrates the model- adjusted estimates for the frequency of incidents per child over the entire 4-year study period for the three outcomes that were consistent across cohorts. Figure 6. Estimated rates (and 95% confidence intervals) of behavioral incidents. Single Agency Interrupted Time Series Given the well-known challenges of evaluating organizational-level interventions (e.g., Murray, 1998), the interrupted time series (Nunno, Smith, Martin, & Butcher, 2017) provides a compelling alternative, particularly if data are available for substantial periods of time both prior to and after the implementation of the intervention (Cook, Campbell, & Shadish, 2002). The Waterford Country School (WCS) in Connecticut has collected the number of restraints per month for over 20 years. The frequency of restraints is considered a meaningful indicator of interactional quality because the use of restraint typically represents a staff–child interaction that resolved unsuccessfully. At WCS, CARE implementation began in January 2009, so the monthly restraint rate, the number of restraints per month divided by the average number of children per month, was available for 72 months prior to and 72 months after the start of CARE. The restraint rate was available for three different WCS programs: residential, shelter, and day school. Application of interrupted time series regression methods (Shadish, Kyse, & Rindskopf, 2013) yielded the estimated restraint rates depicted in Figure 7. Note that the residential restraint rate declined substantially during the 6 years after the implementation of CARE began. Indeed, the residential restraint rate continued to decline and, as of September 2018, has been zero for at least 12 months (William Martin, personal International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 76 communication). Though substantially lower than the restraint rate in the residential program, the restraint rate for the shelter population also showed a significant decline. For day students, a different trend was detected during the implementation period. The restraint rate increased for about 30 months, then reversed and declined for the remainder of the study period. The results for the day students illustrate the benefits of on-going careful collection and monitoring of a measure that matters. WCS staff recognized the relatively sharp increase in the day student restraint rate, reflected on possible causes, and implemented a plan of action based on the CARE principles. Though it took a number of months, the day student restraint rate eventually declined substantially. Figure 7. Estimated restraint rates for three programs at Waterford Country school. Month is centered at January 2009, the beginning of CARE implementation. Summary and Conclusions In sum, a growing body of evidence supports the effectiveness of the CARE program model, a fact that is acknowledged in the scientific rating of 3 (promising research evidence) for CARE in the California Evidence-Based Clearinghouse (http://www.cebc4cw.org/program/children-and-residential-experiences-care/detailed), where CARE is listed under both the Higher Levels of Placement and the Alternatives to Long-Term Residential Programs topics. In addition, a recent review of organization-wide, trauma-informed care models in out-of-home care settings found the design of the CARE evaluation, as reported in Izzo et al. (2016), to be the only one of seven studies that met inclusion criteria to have moderate rather than high risk of bias (Bailey et al., 2018). The CARE Evaluation also addressed nearly all of the recommendations for the measurement, analysis, design, and reporting of evaluations of trauma-informed organizational interventions in Purtle’s (2018, p. 13) systematic review. http://www.cebc4cw.org/program/children-and-residential-experiences-care/detailed International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 77 As noted earlier in this article, more and more literature recognizes the crucial importance of the relationship between children and staff in the residential care setting. In essence, the CARE program model provides a principle-based compass for improving all relationships within the organization. Leaders and supervisors support their staff by modeling the CARE principles in their relationships with all staff, from direct care to upper management to maintenance and other support staff. The CARE training, including regular refreshers and time for reflection, combined with this support from leaders and supervisors, provides direct care staff with the opportunity to develop the capacity to both respond effectively to pain and pain-based behavior and help children and adolescents to practice and learn new life-skills and coping mechanisms. The milieu becomes therapeutic in that routine daily activities become opportunities for achieving the small successes that reduce pain-based bahaviors and eventually lead to accomplishing developmental milestones and enhancing a sense of normality. International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 78 References Anglin, J. P. (2002). Pain, normality, and the struggle for congruence: Reinterpreting residential care for children and youth. Binghamton, NY: Haworth. Bailey, C., Klas, A., Cox, R., Bergmeier, H., Avery, J., & Skouteris, H. (2018). Systematic review of organization-wide, trauma-informed care models in out-of-home care (OoHC) settings [Online early view]. Health and Social Care in the Community, 1–13. doi:10.1111/hsc.12621 Burns, B. J., Phillips, S. D., Wagner, H. R., Barth, R. P., Kolko, D. J., Campbell, Y., & Landsverk, J. (2004). Mental health need and access to mental health services by youths involved with child welfare: A national survey. Journal of the American Academy of Child & Adolescent Psychiatry, 43(8), 960–970. doi:10.1097/01.chi.0000127590.95585.65 Cook, T. D., Campbell, D. T., & Shadish, W. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin. Dweck, C. S. (2008). Mindset: The new psychology of success. New York, NY: Random House Digital. Feeney, B. C., & Collins, N. L. (2015). A new look at social support: A theoretical perspective on thriving through relationships. Personality and Social Psychology Review, 19(2), 113– 147. doi:10.1177/1088868314544222 Glisson, C., & Hemmelgarn, A. (1998). The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse & Neglect, 22(5), 401–421. doi:10.1016/S0145-2134(98)00005-2 Harder, A. T., Knorth, E. J., & Kalverborer, M. E. (2012). A secure base? The adolescent-staff relationship in secure residential youth care. Child & Family Social Work, 17(3),1–13. doi:10.1111/j.1365-2206.2012.00846.x Hawkins-Rodgers, Y. (2007). Adolescents adjusting to a group home environment: A residential care model of re-organizing attachment behavior and building resiliency. Children and Youth Services Review, 29, 1121–1141. doi:10.1016/j.childyouth.2007.04.007 Holden, M. J. (2009). Children and residential experiences: Creating conditions for change. Arlington, VA: Child Welfare League of America. Holden, M. J., Anglin, J. P., Nunno, M. A., & Izzo, C. V. (2014). Engaging the total therapeutic residential care program in a process of quality improvement: Learning from the CARE model. In J. K. Whittaker, J. Fernandez Del Valle, & L. Holmes (Eds.), Therapeutic residential care with children and youth: Identifying promising pathways to evidence based international practice (pp. 301–315). Philadelphia, PA: Jessica Kingsley. http://dx.doi.org/10.1111/hsc.12621 http://dx.doi.org/10.1097/01.chi.0000127590.95585.65 http://dx.doi.org/10.1177/1088868314544222 http://dx.doi.org/10.1016/S0145-2134(98)00005-2 http://dx.doi.org/10.1111/j.1365-2206.2012.00846.x http://dx.doi.org/10.1016/j.childyouth.2007.04.007 International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 79 Holden, M. J., Izzo, C., Nunno, M., Smith, E. G., Endres, T., Holden, J. C., & Kuhn, F. (2010). Children and residential experiences: A comprehensive strategy for implementing a research-informed program model for residential care. Child Welfare, 89(2), 131–149. Izzo, C. V., Aumand, B. N., Cash, B. M., McCabe, L. A., Holden, M. J., & Bhattacharjee, M. (2014). Exploration of the youth-adult relationship in residential care: Small glimpses from a large sample of youth. International Journal of Child and Family Welfare, 15 (1/2), 10– 23. Izzo, C. V., & Smith, E. G. (2016, March). Creating conditions for healthy development in residential youth care settings: Recent findings from the CARE program. Presented in Talks at Twelve, Bronfenbrenner Center for Translational Research, Cornell University. March 10, 2016. Izzo, C. V., Smith, E. G., Holden, M. J., Norton, C. I., Nunno, M. A., & Sellers D. E. (2016). Intervening at the setting level to prevent behavioral incidents in residential child care: Efficacy of the CARE program model. Prevention Science, 19(5), 554–564. doi:10.1007/s11121-016-0649-0 James, S. (2011). What works in group care? – A structured review of treatment models for group homes and residential care. Children and Youth Services Review, 33(2), 308–321. doi:10.1016/j.childyouth.2010.09.014 Kahn, W. A. (2005). Holding fast: The struggle to create resilient caregiving organizations. London, UK: Brunner-Routledge. Kegan, R., & Lahey, L. L. (2009). Immunity to change: How to overcome it and unlock potential in yourself and your organization. Brighton, MA: Harvard Business. Lester, A. M., Goodloe, C. L., Johnson H. E., & Deutsch N. L. (2018). Understanding mutuality: Unpacking relational processes in youth mentoring relationships [Online]. Journal of Community Psychology, 1–16. https://doi.org/10.1002/jcop.22106 Li, J., & Julian, M. M. (2012). Developmental relationships as the active ingredient: A unifying working hypothesis of “what works” across intervention settings. American Journal of Orthopsychiatry, 82(2), 157–166. doi:10.1111/j.1939-0025.2012.01151.x Maier, H. (1987). Developmental group care of children and youth. New York, NY: Haworth. Maier, H. (1991). Developmental foundations of youth care work. In J. Beker & Z. Eisikovits (Eds.), Knowledge utilization in residential child and youth care practice (pp. 25–48). Washington, D.C.: Child Welfare League of America. http://dx.doi.org/10.1007/s11121-016-0649-0 http://dx.doi.org/10.1016/j.childyouth.2010.09.014 https://doi.org/10.1002/jcop.22106 http://dx.doi.org/10.1111/j.1939-0025.2012.01151.x International Journal of Child, Youth and Family Studies (2019) 10(2-3): 63–80 80 McNeely, C., & Blanchard, J. (2009). The teen years explained: A guide to healthy adolescent development. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health, Center for Adolescent Health. Moore, T, McArthur, M., Death, J., Tilbury, C., & Roche, S. (2017). Young people’s views on safety and preventing abuse and harm in residential care: “It’s got to be better than home”, Children and Youth Services Review, 81, 212–219. doi:10.1016/j.childyouth.2017.08.010 Moore, K., Moretti, M. M., & Holland, R. (1997). A new perspective on youth care programs: Using attachment theory to guide interventions for troubled youth. Residential Treatment for Children & Youth, 15(3), 1–24, doi:10.1300/J007v15n03_01 Murray, D. M. (1998). Design and analysis of group-randomized trials (Vol. 27). Monographs in epidemiology & biostatistics. New York, NY: Oxford University Press. Nunno, M. A., Smith, E. G., Martin, W. R., & Butcher, S. (2017). Benefits of embedding research into practice: An agency-university collaboration. Child Welfare, 94(3), 113–133. Purtle, J. (2018). Systematic review of evaluations of trauma-informed organizational interventions that include staff trainings [Online]. Trauma, Violence, & Abuse, 1524838018791304. doi:10.1177/1524838018791304 Ryan, J. P., & Testa, M. F. (2005). Child maltreatment and juvenile delinquency: Investigating the role of placement and placement instability. Children and Youth Services Review, 27(3), 227−249. doi:10.1016/j.childyouth.2004.05.007 Shadish, W. R., Kyse, E. N., & Rindskopf, D. M. (2013). Analyzing data from single-case designs using multilevel models: New applications and some agenda items for future research. Psychological Methods, 18(3), 385–405. doi:10.1037/a0032964 Trieschman, A., Whittaker, J., & Brendtro, L. (1969). The other 23 hours: Child care work with emotionally disturbed children in a therapeutic milieu. Chicago, IL: Aldine. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Zinn, A., DeCoursey, J., Goerge, R., & Courtney, M. (2006). A study of placement stability in Illinois [Chaplin Hall working paper]. Chicago, IL: Chapin Hall Center for Children. Retrieved from http://www.tndev.net/mbs/docs/reference/Child_Welfare_System_National/ChapinHallDoc ument_1.pdf http://dx.doi.org/10.1016/j.childyouth.2017.08.010 http://dx.doi.org/10.1300/J007v15n03_01 http://dx.doi.org/10.1177/1524838018791304 http://dx.doi.org/10.1016/j.childyouth.2004.05.007 http://dx.doi.org/10.1037/a0032964 http://www.tndev.net/mbs/docs/reference/Child_Welfare_System_National/ChapinHallDocument_1.pdf http://www.tndev.net/mbs/docs/reference/Child_Welfare_System_National/ChapinHallDocument_1.pdf