STOP NOW AND PLAN (SNAP™) is a cognitive-behavioural strategy that helps children and parents regulate angry feelings by getting them to stop, think, and plan positive alternatives before they act impulsively International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 330 ROLLING OUT SNAP® AN EVIDENCE-BASED INTERVENTION: A SUMMARY OF IMPLEMENTATION, EVALUATION, AND RESEARCH Leena K. Augimeri, Margaret Walsh, and Nicola Slater Abstract: This article describes the evolutionary process of developing, disseminating, and implementing an evidence-based intervention model for children in conflict with the law. Stop Now And Plan (SNAP®), a Canadian, evidence-based gender sensitive model for young children in conflict with the law, was initiated in 1985 in response to the de- criminalization of children under 12 in Canada. This community-based model is well validated for its efficacious outcomes on reducing problem behaviours in this high-risk population, helping to shift the trajectory of criminal outcome. The article describes the lessons learned during the evaluation, implementation, and replication of SNAP® and the resulting creation of a stringent implementation approach. Currently under the management of the Centre for Children Committing Offences (CCCO), replication sites known as SNAP® Affiliates, enter into a formalized licensing agreement that includes assessing site readiness and theoretical philosophy, ongoing training and consultation, and an accreditation quality assurance process. This formalized approach has been adopted to ensure sites are able to deliver the highest quality of service and to replicate successful outcomes, changing life course trajectories of these high-risk children and families. Keywords: evidence-based intervention, implementation, replication, quality assurance Address correspondence to: Leena K. Augimeri, Child Development Institute, 46 St. Clair Gardens, Toronto, Ontario, M6E 3V4, Canada. E laugimeri@childdevelop.ca T 416-603-1827 ext. 3112 F 416-654-8996 Leena K. Augimeri, PhD is Director, Centre for Children Committing Offences & Program Development, Child Development Institute, Toronto, Ontario and Adjunct Assistant Professor/Sessional Lecturer, University of Toronto. Margaret Walsh, BA is Manager, Research & Evaluation, Child Development Institute, Toronto, Ontario. Nicola Slater, NNEB is Manager, Centre for Children Committing Offences, Child Development Institute, Toronto, Ontario. mailto:laugimeri@childdevelop.ca� International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 331 This paper describes the genesis and 25-year evolution of Stop Now And Plan (SNAP®), a Canadian evidence-based gender sensitive model for young children in conflict with the law. Focus will be placed upon the lessons learned during the implementation, replication, and evaluation of this multi-faceted intervention program. This journey involved a number of major chronological developments (see Figure 1) beginning with the decriminalization of children under the age of 12 in Canada. Recognizing a need to establish services for these children, a community-based children’s mental health centre specializing in latency aged children with disruptive behaviour problems initiated the development of the program. This was done through a review of the scientific literature, the development of a theoretical framework and treatment model, creation of program manuals, and multiple evaluations and research studies. This was a rarity as such programs typically find their beginnings within university communities versus applied community-based settings (Weisz, Sandler, Durlak, & Anton, 2005). The dissemination of positive treatment effects and research findings generated much attention and interest from various mental health providers and researchers. The model is now regarded as one of the most fully developed and longest sustained evidence-based gender specific interventions for young children in conflict with the law (Garbarino, 2006; Howell, 2001, 2003). Requests for replications have led to the creation of a dissemination and training unit, the Centre for Children Committing Offences (CCCO). The CCCO would be responsible for the development of SNAP® training and consultation modules, as well as licensing, model development, fidelity frameworks, and external evaluations. Interest in SNAP® is growing quickly. In the intervening time, thousands of professionals have been trained in its use. To date, SNAP® licenses have been issued to children’s mental health agencies, educational facilities, and other community and social service organizations across Canada, the United States, Europe, and Australia. International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 332 Brief SNAP® History SNAP® was created as a response to the decriminalization of children under the age of 12 in Canada with the introduction of the Young Offenders Act (YOA) in 1984 (which was subsequently replaced with the Youth Criminal Justice Act in 2002). Under the Act, these young children fall under the jurisdiction of provincial/territorial child welfare or mental health legislation. Police officers have two options when they come into contact with a child under the age of 12 engaging in antisocial behaviour: (a) return the child to their parent/caregiver(s), or (b) notify child welfare services, if the child is deemed to be “in need of protection”. A Canadian research study conducted around the same time (Leschied & Wilson, 1988) indicated that the number of pre-adolescent children involved in antisocial behaviour and/or criminal activity was extensive. Given the limited options for these young children in conflict with the law, the federal government encouraged the provinces and territories to fund and develop innovative initiatives for supporting and treating these at-risk children and their families. This government support resulted in Earlscourt Child and Family Centre (now called Child Development Institute, CDI) to seek funding for a 6-month pilot. Given the Centre’s mandate and expertise in working with latency aged aggressive and antisocial children and their families, the agency was well positioned to develop such a program. In October 1985, the Under 12 Outreach Project (now called SNAP® Under 12 Outreach Project, SNAP® ORP) was launched in Toronto, Canada by a team of scientists and practitioners in partnership with Toronto Police Services. It was the first program of its kind designed specifically to meet the needs of children under the age of 12 engaging in antisocial activities such as physical aggression/assault, break and enter, vandalism, and shoplifting/theft. Given the presenting problems and the complexity of issues and risks that these children and their families experience, there was a need to develop a multi-faceted and multi-systemic approach to intervention (Goldberg & Augimeri, 2007). From its onset, research was an integral part of the model’s development. Researchers worked hand-in-hand with the clinicians to inform the theoretical approach of the model and to provide ongoing evaluation feedback, which further informed program development. This scientist-practitioner approach (Jones & Mehr, 2007; Lane & Corrie, 2006) built a strong foundation for the establishment of SNAP® as an evidence-based program and helped to stimulate ongoing research activities. The Province of Ontario (Ministry of Community and Social Services, now assumed under the Ontario Ministry of Children and Youth Services) subsequently funded the program when the pilot showed promising outcome results (Hrynkiw-Augimeri, 1986). From the onset, the SNAP® model utilized components from a variety of established interventions: skills training (Allan & Narnne, 1984; Goldstein, Sprafkin, Gershaw, & Kline, 1980; McGinnis, Goldstein, Sprafkin, & Gershaw, 1984), cognitive problem solving (Spivack, Platt, & Shure, 1974; Spivack & Shure, 1982), self-control and anger management strategies (Camp, Blom, Hebert, & Van Doorninck, 1977; Carter, Patterson, & Quesebrath, 1979; Cummings, 1987; Cummings, Iannotti, & Zahn-Waxler, 1989; Marion, 1994; Meichenbaum & Goodman, 1971; Schneider & Robin, 1973; Stern & Fodor, 1989), cognitive self-instruction (Corno, 1987; Bornstein & Quevillon, 1976; Kendall, 1977; Snyder & White, 1979), family management skills training (Patterson & Stouthamer-Loeber, 1984), and parent training International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 333 (Forgatch, Bullock, & Patterson, 2004; Forehand & McMahon, 1981). As the model continued to evolve, consultations with research scientists at the Oregon Social Learning Center (OSLC) in Eugene, Oregon helped focus the theoretical orientation and parenting component through its Social Interactional Family Therapy (SIFT) approach in working with families (Forgatch & Patterson, 1987; Patterson, Reid, Jones, & Conger, 1975; Patterson, 1982; Patterson & Forgatch, 1987). Today, SIFT is now referred to as Social Interactional Learning or SIL (Patterson, Forgatch, & DeGarmo, 2010). In the mid-1980s the focused treatment approach for children with conduct problems generally included social skills training (see McGinnis et al., 1984; Pepler & Craig, 1990; Pepler & Rubin, 1991). However, the SNAP® model included a self-control and problem solving focus along with social skills and parent management training. As Strayhorn (2002) indicates, “self- control difficulties are of central importance for many psychiatric disorders…[it] is also a crucial, and often missing, ingredient for success in most treatment programs” (p. 7). In addition, research has found that latency-aged children tend to be good candidates for learning self-control strategies (see Piquero, Jennings, & Farrington, 2010). The SNAP® technique is a cognitive behavioural strategy intended to help children control impulsivity, think about the consequences of their behaviour, and develop a socially appropriate plan. It was first developed in the late 1970s in a former Earlscourt day treatment classroom for children with behavioural problems. In the classroom, the clinician would “snap her fingers” to cue a child to begin the SNAP® process. This technique which formed the basis of the SNAP® model, was formalized with the creation and publication of program manuals (Earlscourt Child and Family Centre, 1990a, 1990b, 2001a, 2001b, 2002; Levene, 1998) and was trademarked in 1998. The technique underlies the entire foundation of this multi-component model. The steps of SNAP® have now been translated onto the image of a stoplight, which creates a visual for children: Red Light (STOP), Yellow Light (NOW AND), Green Light (PLAN). These steps are used to help children regulate angry feelings by helping them to first, calm down through the use of techniques such as taking deep breaths and/or counting to 10 (STOP); second to use coping statements (NOW AND) to think of what to say to remain calm (for example, “this is hard but I can do this”), and third, to generate effective solutions (PLANS) to make their problems smaller instead of bigger, make them feel like a winner and not hurt anyone, anything, or themselves. A key aspect of this strategy is teaching children to identify triggers (what makes them angry or upset), and helping them make the connection between their body awareness (body cues) and what STOPS they can use to effectively help their bodies calm down in order to make a PLAN. The original SNAP® ORP program (1985 to 1995) was a time-limited intervention that consisted of two core structured group components: The Transformers Club (now called SNAP® Boys Club) and the Parent Group (now called the SNAP® Parent Group). The groups lasted for a period of 12 weeks, repeatable once. The focus of these 1.5-hour groups incorporated social skills, self-control and problem solving using group discussion, modelling, behavioural rehearsal/role-playing, home practice exercises, in vivo learning opportunities, and relaxation training. Using the corresponding group manuals (Earlscourt Child and Family Centre, 1990a, 1990b, 2001a, 2001b), the structured groups were facilitated by trained, designated SNAP® staff (e.g., parent group – Family Worker; child group – Child Group Leaders). The Transformers International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 334 Club was comprised of seven children placed according to developmental needs and ages (younger children’s group, 6-7 years; middle children’s group, 8-9 years; and an older children’s group, 10-11 years). Over the course of the 12 weeks, specific topics (e.g., Joining In, Dealing With Anger, Avoiding Trouble, Dealing With Peer Pressure, Apologizing, and Stopping Stealing) were addressed. In addition, the concurrent parent group exposed parents to skills and strategies taught to their children, encouraged prompting of the skill at home, and taught effective parent management strategies. Other components that were offered based on levels of risk and need included: Individual Befriending/Mentoring, Family Counselling, and School Advocacy/Teacher Consultation. The model underwent major modifications in 1996 as a result of the extensive review of a decade’s worth of outcome evaluation and research data (e.g., Augimeri, Farrington, Koegl, & Day, 2007; Augimeri, Jiang, Koegl, & Carey, 2006; Day, 1997; Day & Augimeri, 1996; Day & Hunt, 1996; Hrynkiw-Augimeri, 2005; Hrynkiw-Augimeri, Pepler, & Goldberg, 1993; Koegl, Farrington, Augimeri, & Day, 2008; Webster, Augimeri, & Koegl, 2002). There were three major shifts in the model’s development and implementation: 1. The use of a Structured Professional Judgment (SPJ) approach to clinical risk assessment and management (e.g., Douglas & Kropp, 2002; Hart, 1998, 2001); 2. The move of SNAP® ORP from a time-limited intervention to a continued care model; 3. The introduction of the gender-specific SNAP® Girls Connection (SNAP® GC). Assessment of Children and Youth at Risk In the assessment of children’s potential for continued delinquency, SNAP® clinicians identified the need for the development of a risk and protective factors checklist (Augimeri & Levene, 1994, 1997). After its creation, the checklist proved to be a beneficial tool for reference during case conferences and treatment reviews as it helped to identify child and family risk indicators associated with possible aggression and future violence. In-depth research and consultation on SPJ risk assessment (see Webster, Douglas, Eaves, & Hart, 1997), child delinquency and participation in a Very Young Offender Task Force sponsored by the U.S. Office of Juvenile Justice and Delinquency Prevention (Loeber & Farrington, 2001) led researchers and clinicians at CDI to develop two structured, professional judgment risk schemes: the Early Assessment Risk Lists or EARL-20B V2 for boys (Augimeri, Koegl, Webster, & Levene, 2001), and the EARL-21G V1 for girls (Levene et al., 2001). In their current form, the EARLs allow clinicians to isolate specific areas of concern and need across three domains (Family, Child, and Responsivity) that will be amenable to change (Augimeri, Enebrink, Walsh, & Jiang, 2010; Augimeri, Walsh, Liddon, & Dassinger, 2011). This further enhanced the SNAP® program’s theoretical framework by including this developmental-systemic model of risk and protective processes. Furthermore, it has been found that interventions that address need, responsivity, and risk principles show larger effects in term of recidivism than those programs that do not address these principles (Andrews et al., 1990). International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 335 Subsequent research and outcome evaluation indicated that the program was highly effective for the majority of the SNAP® ORP children (see Augimeri et al., 2006, 2007, 2011). However, there was a sub-group of high-risk boys (approximately 10% to 16%) that did not appear to benefit and/or showed a slower rate of improvement as compared to the low-to- moderate risk SNAP® ORP children (Augimeri et al., 2006). A decision was then made by the multi-disciplinary team (clinicians and researchers) to move to a continued care model of service delivery. Under this model, the child and family would now remain in service as long as an identified level of risk and need is present, as assessed by the EARLs. Now, the SNAP® children and families can access a range of service components that incorporates the developmental need of the children and families as they undergo a transition from pre-adolescence to adolescence (see Table 1). Table 1: SNAP® Under 12 Outreach Project (SNAP® ORP) and SNAP® Girls Connection (SNAP® GC) Treatment Components SNAP® ORP Treatment Components SNAP® GC Treatment Components SNAP® Boys Group (Transformer Club)* The SNAP® Girls Club† SNAP® Parent Group* SNAP® Parent Group† Individual Counselling/Mentoring & Community Connections Girls Growing Up Healthy (GGUH)† School Advocacy/Teacher Consultation School Advocacy/Teacher Consultation Leaders in Training (LIT) Leaders in Training (LIT) Stop Now and Plan Parenting (SNAPP): Individualized Family Counselling Stop Now and Plan Parenting (SNAPP): Individualized Family Counselling Homework Club/Academic Tutoring Homework Club/Academic Tutoring Victim Restitution Victim Restitution TAPP-C (The Arson Prevention Program-Children) TAPP-C (The Arson Prevention Program-Children) Parent Problem Solving Continued Care Croup Parent Problem Solving Continued Care Croup Crisis Intervention Individual Counselling/Mentoring & Community Connections *Core Components of the SNAP® ORP; †Core Components of SNAP® GC At the same time, clinicians had noted that in the SNAP® mixed-sex groups, girls were either not fully engaged in the clinical process (e.g., they were withdrawn, would model boys’ behaviours, and/or would compete for boys’ attention), or were not supportive of one another (Pepler et al., 2010). Given that these girls often showed a high prevalence of risk for abuse, a International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 336 lack of coping skills (Walsh, Pepler, & Levene, 2002), strained mother-daughter relationships, multiple separations from caregivers (Pepler et al., 2010), and troubled family/peer relationships (Ehrensaft, 2005), it was felt that a gender-specific intervention was necessary to address such concerns. The SNAP® GC program’s theoretical foundation is similar to SNAP® ORP’s (e.g., social learning, self-control, and problem solving) but is also guided by feminist perspectives in providing assertiveness and anti-oppression training to these largely mother-led families, as well as a focus on relationship building. SNAP® GC has a third core component, the Girls Growing Up Healthy model, a mother-daughter group that specifically focuses on positive attachment. This component is offered after the SNAP® girl’s core group has been completed and when the girl in question has reached the age of 9. Moving Towards a SNAP® Community Implementation Model Evaluation and research conducted on both SNAP® gender-specific, evidence-based programs demonstrated positive treatment outcomes with moderate to large effect sizes. In general, the program lowers aggressive, bullying, and delinquent behaviours in the short term, with good evidence that these effects can be sustained over the intermediate future (see Augimeri, Enebrink, et al., 2010; Augimeri et al., 2007; Pepler et al., 2010; Jiang, Walsh, & Augimeri, 2011; Walsh, et al., 2002). There is also some indication that the program produces long-term changes such as preventing involvement in criminal activities in adolescence and adulthood, although more work is clearly needed to isolate the SNAP® as the key factor preventing this type of negative outcome (see Augimeri, Pepler, Jiang, Walsh, & Dassinger, 2010). Groundbreaking neurological research provides supporting evidence that SNAP® helps children manifest changes in brain systems responsible for cognitive control and self-regulation (Lewis et al., 2008). Furthermore, following treatment, SNAP® families showed an improved ability to “repair” after engaging in difficult parent-child interactions (Granic, O’Hara, Pepler, & Lewis, 2007). Publication and dissemination of SNAP® research results generated much interest from communities seeking an effective crime prevention program for young children. SNAP® group manuals and information were readily distributed to interested organizations without establishing a formal monitoring process. In 1998, funding was received through the National Crime Prevention Centre (NCPC) to develop a SNAP® Resource Kit containing treatment manuals, videos, risk assessment tools, and education booklets. These kits provided the necessary information and resource materials to fully replicate the model. In 1999, an effort was begun to maintain consistency with a high standard of quality over the SNAP® treatment strategies, a SNAP® license was established. Licenses were issued to organizations, clinicians, and educators interested in using the SNAP® technique for a nominal fee of $1.00. The license simply allowed the developers to monitor which organizations were using SNAP®. In 2000, with funding from the J.W. McConnell Family Foundation, CDI opened the CCCO. In addition, the mandate of this dedicated research, training, and dissemination unit focused on the development of a comprehensive gender-sensitive crime prevention strategy for young children in conflict with the law. This strategy involves (a) police-community referral protocols that establish efficient community-based referral mechanisms, (b) clinical risk management tools that assess the potential risk for future antisocial behaviour in young children, and (c) SNAP® gender-specific programs that serve aggressive and antisocial children and their families (Augimeri et al., 2011; International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 337 Koegl, Augimeri, Ferrante, Walsh, & Slater, 2008). With a Canadian focus and an international presence, the CCCO works collaboratively with researchers, policy-makers, clinicians, and other professionals to advance and implement this three-pronged crime-prevention strategy. In early 2000, the CCCO partnered with a number of national and international organizations (e.g., Government of Yukon, Canada; Banyan Community Services, Ontario, Canada; St. Paul Public School Board, Minnesota, United States; and a number of Scandinavian organizations including the University of Oslo, Norway, and several professional groups in Sweden) to implement partial and full replications of the SNAP® ORP. This resulted in the CCCO creating comprehensive replication budgets, training modules, consultation schedules, and the beginnings of a fidelity and monitoring framework. Preliminary evaluation results from the various external replications supported the previous SNAP® outcomes (see Augimeri et al., 2011). Banyan Community Services, the first organization to implement a full SNAP® ORP program (a site funded by NCPC), conducted a third-party multi-year external evaluation that included a wait-list comparison group. Significant differences were found in terms of rule breaking, aggression, and conduct problems between the SNAP® ORP and the wait-list control group (see Lipman et al., 2008). In an effort to heighten awareness of the need for services for children under 12 in conflict with the law and to determine interest in replicating the SNAP® model, the CCCO focused its energy on “scaling up” during the first six years. Much energy was invested in establishing as many SNAP® Affiliate Sites as possible, as well as developing training modules, distributing SNAP® resource materials, and issuing SNAP® licenses (approximately 80 licenses were issued by 2006). Growing interest was also accelerated when the model received among the highest effectiveness designation from accredited reviewers such as the U.S. Office of Juvenile Justice and Delinquency Prevention – Model Program Guides, and the Canadian National Crime Prevention Centre – Model Programs. Thus, SNAP® emerged as a unique model that is accessible, user friendly, and easily adopted. What was now needed for the program was an understanding of how to support and monitor SNAP® Affiliate Sites to ensure successful program outcomes and sustainable implementations. In a survey conducted by the CCCO (see Augimeri, Koegl, Slater, & Ferrante, 2006, Fall), SNAP® Affiliate Sites reported that the following practice activities would strengthen their SNAP® replication: participation in SNAP® research, access to SNAP® resources and information, ongoing SNAP® training and consultation, and assistance dealing with SNAP® implementation issues. A large percentage of sites indicated that connecting with other SNAP® organizations would also help enhance their abilities to provide sustainable and successful SNAP® programming within their communities. These findings showed strong support for the creation of a SNAP® Community of Practice. A “scaling deeper” process was needed to help focus the CCCO’s idea to create a comprehensive framework that would assess the readiness of communities and sites, as well as put into place the implementation, training, consultation, and support mechanisms. To lay the foundation for developing such a framework, the CCCO undertook the development of a business plan. The focus was also to put into action a marketing strategy that International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 338 included safeguards to preserve control over intellectual properties and ensure product fidelity. The CCCO would now take a measured and strategic approach to dissemination. With this strategy, interested organizations would no longer have only the capacity to replicate, but would also be required to enter into an annual licensing agreement that would include required criteria for implementation sites and a SNAP® accreditation designation rating model. Criteria for implementation includes the following: Sites must participate in SNAP® training (this includes a 5-day core SNAP® training, as well as a 2-day site-specific training to deal with implementation issues); all staff must be trained by the CCCO; staff must participate in ongoing consultation activities between Year One and Year Three (which involves ongoing fidelity and integrity activities); members must agree to an annual licensing fee; and, finally, they must participate in data collection and program evaluation activities. This last criterion is especially vital to the CCCO as it is important to clinical risk management planning and program quality assurance. The accreditation model that is currently being tested involves three possible SNAP® designations. An “A” designation indicates that the SNAP® Affiliate Site is implementing their program with high integrity and fidelity, and is meeting all required criteria. Such a site will be implementing the full SNAP® model and achieving at least 90% program delivery compliance. A “B” designation indicates that the site has not yet achieved 90% compliance and/or has not been able to implement the full SNAP® model at this time; however, the commitment to reach full implementation has been negotiated. Sites that have achieved at least 90% compliance level for implementing their SNAP® group components but are not yet a full replication site fall within this category. The final designation a site can receive is a “C” designation. This means that the site has just completed training and no fidelity and integrity checks have yet been completed and/or the site is transitioning from the status of a partial SNAP® replication site to a full replication site. The developers were aware that this new strategy could result in a decreased number of licensed SNAP® Affiliates due to increased costs and heightened required standards. However, it was not anticipated that this approach would weaken the brand; rather, it was seen as an opportunity to “delve deeper” with a set of partnerships that would provide a strong base to the sustainability of the SNAP® model and the SNAP® Community of Practice. This would include the creation of a dedicated interactive SNAP® website (www.stopnowandplan.com), electronic newsletters, and teleconference/videoconference communications. A New Generation of SNAP® Affiliates The CCCO now requires all professionals and/or organizations using SNAP® to commit to the following implementation standards and principles: 1. Replication sites must enter into a formal SNAP® licensing agreement. 2. The organization’s treatment philosophy should be consistent with CDI’s (e.g., cognitive behavioural family-centred approach). 3. The replication site must have a strong history of collaboration with other social service programs and relevant stakeholders (e.g., child welfare, police, schools). 4. Replication sites should allocate full-time, dedicated, and trained staff to operate programs. http://www.stopnowandplan.com/� International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 339 5. Training and consultation should be built into the implementation plan. 6. Any adaptations to the model must be first approved by the CCCO. 7. Research must be an integral part of the replication. SNAP® Site Selection The business plan helped to further formalize and expand the selection process, and new agencies were required to become a SNAP® Affiliate Site. This site selection program development process includes eight stages where potential sites must: 1. complete a readiness checklist (e.g., SNAP® Request for Qualifications application); 2. participate in a two-hour telephone or on-site needs assessment and information session that determines the availability of appropriate pilot project funding with a plan for sustainability; 3. lead staff to participate in an interview highlighting key program implementation issues (e.g., identifying target population and key stakeholders, developing referral mechanism and evaluation plans); 4. hold a SNAP® Site readiness review meeting that includes a community mobilization meeting with key stakeholder; 5. partake in staff recruitment and training that includes a 5-day full replication training module and an additional 2-day session for lead staff; 6. sign and issue a SNAP® Licensing Agreement; 7. provide ongoing SNAP® Affiliate Site program implementation support; and 8. participate in ongoing fidelity and integrity activities as stated in the SNAP® Program’s Policy and Procedures Manual and by SNAP® developers. SNAP® Fidelity and Integrity Framework Treatment integrity is used to determine the accuracy and consistency with which an intervention is implemented (Wilkinson, 2006). Research has shown that when treatments are implemented with high integrity (e.g., consistency in treatment implementation according to a treatment manual, training from developer/consultant), more successful outcomes are obtained (Sterling-Turner, Watson, & Moore, 2002). When program developers are available to provide direct training and consultation to ensure consistency in program implementation (according to program/treatment protocol), treatment integrity has been shown to be higher (Sterling-Turner et al., 2002). As program developers of SNAP®, our primary concern is that the model be delivered and implemented effectively. Steps have been taken to ensure this (e.g., manualization, integrity checks, licensing standards, and training); however, this does not always guarantee that an affiliate site will be competent in its delivery (Patterson et al., 2010). The need for successful implementation and service delivery stems from the notion that in order to attribute treatment outcome to the intervention, the implementation must be consistently and accurately carried out as determined by previous evaluations of the program, historical experience, and so on. Early in the development of the SNAP® model, fidelity and integrity practices were utilized to ensure quality program implementation and adherence to the manuals (Day & Augimeri, 1996; Pepler et al., 2010). Lipsey (2009) found that “aside from delinquency risk, the International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 340 largest and most consistent relationship with recidivism effects is the quality of program implementation with, of course, higher quality associated with bigger effects on recidivism” (p. 141). There are a number of ways to assess treatment integrity that include both direct and indirect methods (Kazdin & Perepletchikova, 2005), SNAP® being one such model that uses both methods. SNAP® fidelity/integrity practices include file audits (indirect) and ongoing monitoring of the SNAP® parent and children’s groups (direct) using a SNAP® Children’s and/or Parent Group Integrity Checklist. In 2009, the CCCO began to establish a more comprehensive SNAP® Fidelity Rating Index (Walsh & Augimeri, 2009). This emerged from an identified need to monitor new and existing SNAP® Affiliate Sites more stringently as part of the established SNAP® Accreditation Rating Framework and new licensing requirements. In recent years, such practices (fidelity) are becoming the standard for understanding clinical practice by including process of change mechanisms (Patterson et al., 2010; Schoenwald, Heiblum, Saldana, & Henggeler, 2008). The framework is broken down into a number of subsections that afford a rating for each with an overall fidelity rating at the end. Sections include: Pre-implementation Site Selection Procedures/Requirements: (as outlined in the SNAP® Request for Qualification application), this subsection includes a review of the interested organization’s performance standards (e.g., a reputable track record, demonstrated fiscal budget management), a compatible orientation (e.g., client-centred, family focused, cognitive- behavioural), dedicated and adequately trained staff, appropriate physical program space, and a plan for continued program sustainability beyond the pilot period. Ongoing Site Implementation Activities: includes training, consultation (on- and off-site support), program manuals, internal program integrity and fidelity checks, research and evaluation, team meetings, and assistance with the hiring of program staff. Ongoing Process of Service Implementation: includes the tracking of referrals, admission criteria, service utilization, file audits, and ensuring the timely completion of program measures (pre, post, and follow-up). Monitoring of Suggested Clinical Risk Management Strategies: includes reviews of clinical risk management plans, ensuring delivery of the group components according to program manuals, and that SNAP® treatment services/components adhere to SNAP® therapeutic principles and orientation (e.g., cognitive-behavioural). Dosage Monitoring: in relation to risk level utilizing the EARLs. Dosage monitoring consists of the review of a number of treatment components recommended and received, as well as any hours of service. Treatment Quality: relates to the delivery of the service, which includes facilitator preparedness, skills, and understanding of SNAP® principles, client satisfaction, and alliance. Participant Responsiveness: includes child and parent engagement during treatment, goal attainment, and evidence of behavioural change as noted on standardized measures. Process and Outcome Evaluation Framework The scientist-practitioner approach helped to establish SNAP® as an evidence-based practice that enabled the CCCO to share knowledge and evidence with others (nationally and internationally), so that the positive impact on children’s lives could be increased exponentially. It is a formula that works – building knowledge, understanding change, and developing effective interventions. In continuing with the spirit of the scientist-practitioner way of thinking, the International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 341 CCCO introduced an Evidence-Based Implementation Evaluation Checklist/Barometer (See Figure 2) that is used to systematically track the various SNAP® models (SNAP® ORP, SNAP® GC, and SNAP® for Schools) program planning, evaluation, and research activities. The checklist/barometer is a visualization of each SNAP® model’s progress. It outlines three stages along a continuum for establishing efficacious interventions: Program Planning, Process Evaluation, and Outcome Evaluation. Figure 2: Evidence-Based Implementation Evaluation Checklist *Methodology may vary. ^ Fidelity/Integrity Reviews Level I - refers to program level supervision, peer review of cases, pre/de-briefs, external case consultations. Fidelity/Integrity Reviews Level II - refers to a quantifiable activities/measures conducted for fidelity and scored on a regular bases. ^^ CSQ - Client Satisfaction Data is collected at several point across programming (e.g., post group, post treatment components, and discharge). The greater the number of tasks completed in each stage, the longer and darker the barometer becomes. Once the checklist is updated, the barometer indicates (with color-graded bars) the completion rate within the three stages. For example, as seen in Figure 2 our SNAP® ORP and GC models have been involved in a variety of process and outcome evaluation and research activities since their implementation in 1985 and 1996 respectively, and have received a higher bar grading/completion rate across all stages than the new SNAP® for Schools model. This newer model is at the beginning stages of its evaluation and research activities and evolved from the SNAP® ORP and GC. Community consensus about the need to expand the SNAP® program, led to the creation of this school-based model. This program is tailored to meet the needs of elementary school-aged children with serious conduct problems. Preliminary evaluation results are promising with significant positive behavioural changes being identified (Walsh, Hong, & Augimeri, 2009, 2010). Under the Program Planning stage of the barometer, one of the key steps for all new program implementations is the development of program logic models (Millar, Simeone, & International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 342 Carnevale, 2001; United Way of Greater Toronto, 2005). A logic model can help to conceptualize the relationship between the program inputs, activities, target group, outputs, and expected outcomes. As part of a SNAP® replication, sites are required to develop their own program logic model independent of the CCCO’s SNAP® Logic Model. This exercise enables SNAP® Affiliate Sites to begin thinking about their individual program and community needs and linking agency and program resources. Under the Process Evaluation, key activities include tracking: number of referrals, admission criteria, demographic data of clients admitted into service, program and client utilization rates, and cultural competency. Process evaluation analyses on these key indicators will help to provide information that describes the clients and families served, and the level of service delivery they have received. For replication sites, this information is helpful in explaining variations in how different children and families respond to the program. Do those who get more services or certain services do better than those who get fewer or different services? By following such a systematic approach, programs will be better informed about the processing involved in reaching the ultimate goal of being identified and validated as evidence-based programming. The primary function of a process evaluation is designed to assess the overall implementation of services within a program and whether services were delivered to the target clients. Under the Outcome Evaluation stage of the barometer, several steps are outlined to be taken along the pathway to evidence-based programming (e.g., qualitative and quantitative analyses, pre/post data, use of standardized measures, statistically significant results, sustained effects for at least one year, use of a comparison group, random control trials, replication, and third party external evaluation). We included criteria noted for efficacious and well-established treatments (see Brestan & Eyberg, 1998; Chambless & Hollon, 1998; Chambless & Ollendick, 2001; Task Force on Promotion and Dissemination of Psychological Procedures, 1995). This is an extensive step-by-step process and can take many years to accomplish. The SNAP® ORP and GC models have been moving along this stage of the continuum for some 15 to 25 years. Conclusion The trials, tribulations, and triumphs of implementing SNAP® has led to the need to create replication standards and principles, licensing agreements, accreditation, and fidelity frameworks as outlined in this paper to ensure success. Although the new standards and criteria under the annual SNAP® licensing agreement may seem daunting and challenging, these are designed to act as a safety net to protect the SNAP® model and guide SNAP® Affiliate Sites along their implementation process. In the past two years under the new licensing agreement, more than 25 organizations have transitioned into the newly established SNAP® standards and requirements, with a additional number slated for the coming years. These include a variety of different types of organizations (e.g., accredited children’s mental health centres, community- based and government organizations, and schools) across Canada, the United States, Europe, and Australia with promising and replicable results (e.g., Lipman et al., 2008; Lipman, Kenny, Brannan, O’Grady, & Augimeri, 2010; Canfield & Burke, 2010). International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 343 It is apparent that creating an efficacious, well-established, and recognized program takes a team of dedicated scientists and practitioners. It requires commitment, support, and resources from the host organization, particularly when such programs begin their incubation within a community-based setting. This process requires patience and extensive monitoring and documentation that can expand across decades. If successful, replication may occur. The rollout of the model will necessitate that developers initiate a replication, implementation, and dissemination strategy. This requires further support and resources to ensure its viability as evidenced in the evolution of SNAP® and the creation of the CCCO. There are five important aspects of implementation and replication that deserve to be reiterated in regard to ensuring successful SNAP® outcomes: 1. Adherence to the model is critical. An integral tool used to ensure this is the use of program manuals. Manuals can be seen as limiting a clinician’s creativity (Weisz et al., 2005). However, given that SNAP® is a cognitive-behavioural social learning skills-based approach, requiring a certain level of knowledge, understanding, and expertise, the manuals provide clinicians with clear and distinct direction on how to implement the components with the highest integrity and fidelity (Corrigan, Steiner, McCracken, Blaser, & Barr, 2001). 2. Restraint from Making Modifications is essential. We understand the importance of making programs “fit” within an existing practice or make it more “culturally appealing”. For example, during the implementation process at one of our SNAP® Aboriginal sites, it was requested that the SNAP® Learning Log (a form that helps the client document the process of learning what happens when they get angry or upset by identifying the trigger, body cues, thoughts and feelings, how they handle the situation, and how they incorporate SNAP® in the learning) be altered to resemble the shape of a “medicine wheel” to make it more culturally sensitive. Such minor adaptations such as changing the esthetic appeal of a learning tool can be a helpful improvement, one that may offer a different look, but not dramatically change the content of what is to be achieved. Caution and careful consideration must be taken so that key treatment ingredients are not altered in any way. 3. Training and Ongoing Consultation is essential. It is an investment that both the replication and host agencies commit to provide clinical staff with the necessary skills and information to implement the program as intended (Lipsey, 2009). It also provides ongoing communication, feedback support, and a quality assurance mechanism (Schoenwald et al., 2008). 4. Engagement in Ongoing Fidelity/Integrity Audits ensures the program is being delivered with the highest possible efficacy to ensure that the desired outcomes are achieved (Dane & Schneider, 1998; Forgatch, Patterson, & DeGarmo, 2005; Schoenwald et al., 2008). Auditing program integrity and fidelity is the responsibility of both the host organization and the replication agency as part of good clinical supervision and program management. 5. Selecting the Right Staff is paramount to program success. Hiring dedicated, passionate, knowledgeable, and skilled team members is important for such variables as staff turnover and high dropout rates that can affect quality and delivery of services (Lipsey, 2009). International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 344 For example, clinicians have to be able to engage high-risk clients and establish therapeutic relationships. It is obvious that implementing and replicating the model is not an undertaking that we have addressed without an awareness of the complexity for insuring its generalizability. We have attempted to maintain the contents of the program without compromising its initial integrity. We have held true to our belief that achieving successful outcomes and changing life course trajectories of these high-risk children and families has been the principal focus of the SNAP® model, and brings us back to why we began this journey. International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 345 References Allan, A., & Narrne, J. (1984). Class discussions for teachers and counsellors in elementary school. Toronto: Guidance Centre, Faculty of Education, University of Toronto. Andrews, D. A., Zinger, I., Hoge, R. D., Bonta, J., Gendreau, P., & Cullen, F. T. (1990). Does correctional treatment work? A clinically relevant and psychologically informed meta- analysis. Criminology, 28(3), 369-404. Augimeri, L. K., Enebrink, P., Walsh, M., & Jiang, D. (2010). Gender-specific childhood risk assessment tools: Early Assessment Risk Lists for Boys (EARL-20B) and Girls (EARL- 21G). In R. K. Otto & K. S. Douglas (Eds.), Handbook of violence risk assessment (pp. 43-62). Oxford, UK: Routledge, Taylor & Francis. Augimeri, L. K., Farrington, D. P., Koegl, C. J., & Day, D. M. (2007). The Under 12 Outreach Project: Effects of a community based program for children with conduct problems. Journal of Child and Family Studies, 16, 799-807. Published Online, January 10, 2007. DOI: 10.1007/s10826-006-9126-x Augimeri, L. K., Jiang, D., Koegl, C. J., & Carey, J. (2006). Differential effects of the SNAP® Under 12 Outreach Project (SNAP® ORP) associated with client risk & treatment intensity. Program evaluation report submitted to the Centre of Excellence for Child and Youth Mental Health at CHEO. Augimeri, L. K., Koegl, C. J., Ferrante, P., & Slater, N. (2006, Fall). Why and how: Conducting effective clinical risk assessments with children with conduct problems. Canada’s Children, 12(2), 24-27 (English); 28-32 (French). Augimeri, L. K., Koegl, C. J., Webster, C. D., & Levene, K. (2001). Early assessment risk list for boys: EARL-20B, Version 2. Toronto: Earlscourt Child and Family Centre. Augimeri, L. K., & Levene, K. (1994, 1997). Risk factors associated with conduct disorder and non-responders. Toronto: Earlscourt Child and Family Centre. Augimeri, L. K., Pepler, D., Walsh, M. M., Jiang, D., & Dassinger, C. (2010). Aggressive and antisocial young children: Risk prediction, assessment and clinical risk management. Program Evaluation Report submitted to The Provincial Centre of Excellence for Child and Youth Mental Health at CHEO (Grant: # RG-976). Augimeri, L. K., Walsh, M., Liddon, A. D., & Dassinger, C. R. (2011). From risk identification to risk management: A comprehensive strategy for young children engaged in antisocial behavior. In D. W. Springer & A. Roberts, (Eds.), Juvenile justice and delinquency (pp. 117-140). Sudbury, MA: Jones & Bartlett. Bornstein, H., & Quevillon, P. (1976). The effects of a self instructional package on overactive preschool boys. Journal of Applied Behaviour Analysis, 9(2), 179-188. International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 346 Brestan, E. V., & Eyberg, S. M. (1998). Effective psychosocial treatments of conduct disordered children and adolescents: 29 years, 82 studies, and 5,272 kids. Journal of Clinical Child Psychology, 27(2), 180-189. Camp, B. W., Blom, G. E., Hebert, F., & Van Doorninck, W. J. (1977). “Think Aloud”: A program for developing self-control in young aggressive boys. Journal of Abnormal Child Psychology, 5(2), 157-169. Canfield, T. N., & Burke, J. D. (2010). [Evaluation of a two-year demonstration of the Pittsburgh SNAP® Program]. Unpublished raw data. Carter, D. B., Patterson, C. J., & Quasebarth, S. J. (1979). Development of children’s use of plans for self-control. Cognitive Therapy and Research, 3(4), 407-413. Chambless, D. L., & Hollon, S. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7-18. Chambless, D. L., & Ollendick, T. H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review Psychology, 52, 685-716. Corno, L. (1987). Teaching and self-regulated learning. In D. C. Berliner & B. V. Rosenshine (Eds.), Talks to teachers (pp. 249-266). New York: Random House. Corrigan, P. W., Steiner, L., McCracken, S. G., Blaser, B., & Barr, M. (2001). Strategies for disseminating evidence-based practices to staff who treat people with serious mental illness. Psychiatric Services, 52(12), 1598-1606. Cummings, E. (1987). Coping with background anger in early childhood. Child Development, 58(4), 976-984. Cummings, E. M., Iannotti, R. J., & Zahn-Waxler, C. (1989). Aggression between peers in early childhood: Individual continuity and developmental change. Child Development, 60(4), 887-895. Dane. A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18(1), 23-45. Day, D. M. (1997). Long-term follow-up of court contact and criminal behavior among two samples of youth at risk for juvenile delinquency. Report submitted to the Trillium Foundation. Toronto: Earlscourt Child and Family Centre. Day, D. M., & Augimeri, L. K. (1996). Serving children at risk for juvenile delinquency: An evaluation of the Earlscourt under 12 outreach project (ORP). Report submitted to the Department of Justice. Toronto: Earlscourt Child and Family Centre. International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 347 Day, D. M., & Hunt, A. C. (1996). A multivariate assessment of a risk model for juvenile delinquency with an “under 12 offender” sample. Journal of Emotional and Behavioural Disorders, 4(2), 66-72. Douglas, K. S., & Kropp, P. R. (2002). A prevention-based paradigm for violence risk assessment: Clinical and research applications. Criminal Justice and Behavior, 29(5), 617-658. Earlscourt Child and Family Centre. (2002). SNAP™ girls group manual: The girls club. Toronto: Earlscourt Child and Family Centre. Earlscourt Child and Family Centre. (1990a, 2001a). SNAP™ children’s group manual. Toronto: Earlscourt Child and Family Centre. Earlscourt Child and Family Centre. (1990b, 2001b). SNAP™ parent group manual. Toronto: Earlscourt Child and Family Centre. Ehrensaft, M. K. (2005). Interpersonal relationships and sex differences in the development of conduct problems. Clinical Child and Family Psychology Review, 8(1), 39-63. Forehand, R. L., & McMahon, R. J. (1981). Helping the noncompliant child. New York: The Guilford Press. Forgatch, M. S., Bullock, B. M., & Patterson, G. R. (2004). From theory to practice: Increasing effective parenting through role-play. In H. Steiner (Ed.), Handbook of mental health interventions in children and adolescents: An integrated developmental approach (pp. 782-813). San Francisco: Jossey-Bass. Forgatch, M. S., & Patterson, G. R. (1987). Parents and adolescents: Living together, Part II: Family problem solving. Eugene, OR: Castalia. Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training. Behavior Therapy, 36(1), 3-13. Garbarino, J. (2006). See Jane hit: Why girls are growing more violent and what we can do about it (1st ed.). New York: Penguin Press. Goldberg, K., & Augimeri, L. K. (2007). Children under the age of criminal responsibility at risk of delinquency. In The welfare of Canadian children: It’s our business (pp. 126-128). Ottawa, ON: Child Welfare League of Canada. Goldstein, A. P., Sprafkin, R. P., Gershaw, J., & Klein, P. (1980). Skill-streaming the adolescent: A structured learning approach to teaching prosocial skills. Champaign, IL: Research Press. International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 348 Granic, I., O’Hara, A., Pepler, D., & Lewis, M. D. (2007). A dynamic systems analysis of parent-child changes associated with successful “real-world” interventions for aggressive children. Journal of Abnormal Child Psychology: An official publication of the International Society for Research in Child and Adolescent Psychopathology, 35(5), 845-857. Hart, S. D. (1998). The role of psychopathy in assessing risk for violence: Conceptual and methodological issues. Legal and Criminological Psychology, 3, 121-137. Hart, S. D. (2001). Assessing and managing violence risk. In K. S. Douglas, C. D. Webster, S. D. Hart, D. Eaves, & J. R. P. Ogloff (Eds.), HCR-20 violence risk management companion guide (pp. 13-25). Burnaby, BC: Mental Health Law and Policy Institute, Simon Fraser University. Howell, J. C. (2001). Juvenile justice programs and strategies. In R. Loeber & D. P. Farrington (Eds.), Child delinquents: Development, interventions, and service needs (pp. 305-322). Thousand Oaks, CA: Sage Publications. Howell, J. C. (2003). Preventing and reducing juvenile delinquency, a comprehensive framework. Thousand Oaks, CA: Sage Publications. Hrynkiw-Augimeri, L. K. (1986). Earlscourt Child and Family Centre’s outreach pilot project report. Toronto: Earlscourt Child and Family Centre. Hrynkiw-Augimeri, L. K. (2005). Aggressive and antisocial young children: Risk prediction, assessment and management utilizing the Early Assessment Risk List for Boys (EARL- 20B). Unpublished doctoral dissertation, Ontario Institute for Studies in Education, University of Toronto. Hrynkiw-Augimeri, L. K., Pepler, D., & Goldberg, K. (1993). An outreach program for children having police contact. Canada’s Mental Health, 41, 7-12. Jiang, D., Walsh, M., & Augimeri, L. K. (2011). The linkage between bullying behaviour and future offending. Criminal Behaviour and Mental Health, 21, 128-135. Published online, March 8, 2011. doi: 10.1002/cbm.803. Jones, J. L., & Mehr, S. L. (2007). Foundations and assumptions of the scientist-practitioner model. American Behavioral Scientist, 50(6), 766-771. Kazdin, A., & Perepletchikova, F. (2005). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12(4), 365-385. International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 349 Kendall, P. C. (1977). On the efficacious use of verbal self-instructional procedures with children. Cognitive Therapy and Research, 1(4), 331-341. Koegl, C. J., Augimeri, L. K., Ferrante, P., Walsh, M., & Slater, N. (2008). A Canadian programme for child delinquents. In R. Loeber, N. W. Slot, P. van der Laan, & M. Hoeve (Eds.), Tomorrow's criminals: The development of child delinquency and effective interventions (pp. 285-300). Aldershot, UK: Ashgate. Koegl, C. J., Farrington, D. P., Augimeri, L. K., & Day, D. M. (2008). Evaluation of a targeted cognitive-behavioural programme for children with conduct problems – The SNAP® Under 12 Outreach Project: Service intensity, age and gender effects on short- and long- term outcomes. Clinical Child Psychology and Psychiatry, 13(3), 419-434. Lane, D. A., & Corrie, S. (2006). The modern scientist-practitioner: A guide to practice in psychology. New York: Routledge. Leschied, A., & Wilson, S. K. (1988). Criminal liability of children under twelve: A problem for child welfare, juvenile justice or both? Canadian Journal of Criminology, 30(1), 17-29. Levene, K. (1998). SNAPP Stop-now-and-plan-parenting: Parenting children with behaviour problems. Toronto: Earlscourt Child and Family Centre. Levene, K. S., Augimeri, L. K., Pepler, D. J., Walsh, M., Koegl, C. J., & Webster C. D. (2001). Early assessment risk list for girls: EARL-21G, Version 1, Consultation Edition. Toronto: Earlscourt Child and Family Centre. Lewis, M. C., Granic, I., Lamm, C., Zelazo, P. D., Stieben, J., Todd, R. M., et al. (2008). Changes in the neural bases of emotion regulation associated with clinical improvement in children with behavior problems [Special Issue: Integrating biological measures into the design and evaluation of preventive interventions]. Development and Psychopathology, 20(3), 913-939. Lipman, E., Kenny, M., Brannan, E., O’Grady, S., & Augimeri, L. (2010). Qualitative outcomes of participation in a multi-component intervention for boys at-risk of criminal activity. Manuscript submitted for publication (American Journal of Orthopsychiatry). Lipman, E. L., Kenny, M., Sniderman, C., O’Grady, S., Augimeri, L., Khayutin, S., et al. (2008). Evaluation of a community-based program for young boys at risk of antisocial behaviour: Results and issues. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 17(1), 12-19. Lipsey, M. W. (2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims and Offenders, 4(2), 124-147. International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 350 Loeber, R., & Farrington, D. P. (Eds.). (2001). Child delinquents: Development, intervention and service needs. Thousand Oaks, CA: Sage. Marion, M. (1994). Encouraging the development of responsible anger management in young children. Early Child Development and Care, 97, 155-163. McGinnis, E., Goldstein, A. P., Sprafkin, R. P., & Gershaw, N. J. (1984). Skills-streaming the elementary school child: A guide for teaching prosocial skills. Champaign, IL: Research Press. Meichenbaum, D., & Goodman, J. (1971). Training impulsive children to talk to themselves: A means of developing self-control. Journal of Abnormal Psychology, 77(2), 115-126. Millar, A., Simeone, R. S., & Carnevale, J. T. (2001). Logic Models: A systemic tool for performance management. Evaluation and Program Planning, 24(1), 73-81. Patterson, G. R. (1982). A social learning approach to family intervention; III, Coercive family process. Eugene, OR: Castalia. Patterson, G. R., & Forgatch, M. S. (1987). Parents and adolescents living together, part 1: The basics. Eugene, OR: Castalia. Patterson, G. R., Forgatch, M. S., & DeGarmo, D. (2010). Cascading effects following intervention. Development and Psychopathology, 22(4), 949-970. Patterson, G. R., Reid, J. B., Jones, R. R., & Conger, P. E. (1975). A social learning approach to family intervention; I, Families with aggressive children. Eugene, OR: Castalia. Patterson, G. R., & Stouthamer-Loeber, M. (1984). The correlation of family management practices and delinquency. Child Development, 55(4), 1299-1307. Pepler, D. J., & Craig, W. M. (1990). School-based skills training with aggressive children: Necessary, but not sufficient? Exceptionality Education Canada, Special Edition: Teaching Social Skills in School, 3, 177-194. Pepler, D. J., & Rubin, K. H. (1991). (Eds.), The development and treatment of childhood aggression. Hillsdale, NJ: Erlbaum. Pepler, D., Walsh, M., Yuile, A., Levene, K., Vaughan, A., & Webber, J. (2010). Bridging the gender gap: Interventions with aggressive girls and their parents. Prevention Science, 11(3), 229-238. [Electronic version] http://www.springerlink.com/content/ Piquero, A. R., Jennings, W. G., & Farrington, D. P. (2010). Self-control interventions for children under age 10 for improving self-control and delinquency and problem behaviors. Campbell Systematic Reviews 2010:2. doi: 10.4073/csr.2010.2. http://www.springerlink.com/content/� International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 351 Schneider, M., & Robin, A. (1973). The turtle manual: A method for the self-control of impulsive behaviour. (U.S. Office of Education Grant O.E.O., 0 – 7 12872). Point of Woods Laboratory School, State University of New York, Stony Brook. Schoenwald, S. K., Heiblum, N., Saldana, L., & Henggeler, S. W. (2008). The international implementation of Multisystemic Therapy. Evaluation & the Health Professions, 31(2), 211-225. Snyder, J., & White, M. (1979). The use of cognitive self-instruction in the treatment of behaviourally disturbed adolescents. Behavior Therapy, 10(2), 227-235. Spivack, G., Platt, J., & Schure, M. B. (1974). The problem-solving approach to adjustment. San Francisco: Jossey-Bass. Spivack, G., & Shure, M. B. (1982). The congnition of social adjustment, interpersonal cognitive problem-solving thinking. In B. B. Lahey & A. E. Kazdin (Eds.), Advances in clinical child psychology (Vol. 5, pp. 95-136). New York and London: Plenum Press. Strayhorn, J. M. (2002). Self-control: Theory and research. Journal of American Academy of Child and Adolescent Psychiatry, 41(1), 7-16. Sterling-Turner, H. E., Watson, T. S., & Moore, J. W. (2002). The effects of direct training and treatment integrity on treatment outcomes in school consultation. School Psychology Quarterly, 17(1), 47-77. Stern, J. B., & Fodor, I. G. (1989). Anger control in children: A review of social skills and cognitive behavioral approaches to dealing with aggressive children. Child & Family Behavior Therapy, 11(3-4), 1-20. Task Force on Promotion and Dissemination of Psychological Procedures. (1995). Training in and dissemination of empirically validated psychological treatments: Report and recommendations. The Clinical Psychologist, 48, 3-23. United Way of Greater Toronto. (2005). A toolkit for outcome measurement: Building on logic models. Toronto: Author. Walsh, M., & Augimeri, L. K. (2009). SNAP® fidelity rating index. Toronto: Child Development Institute. Walsh, M., Hong, L., & Augimeri, L. K. (2009). The efficacy of school-based mental health services: SNAP® 2008-2009 fiscal year report. Toronto: Child Development Institute. Walsh, M., Hong, L., & Augimeri, L. K. (2010). The efficacy of school-based mental health services: SNAP® 2009-2010 fiscal year report. Toronto: Child Development Institute. International Journal of Child, Youth and Family Studies (2011) 2.1: 330-352 Canadian Research Perspectives for Youth at Risk for Serious and Violent Offending: Implications for Crime Prevention Policies and Practices 352 Walsh, M., Pepler, D., & Levene, K. (2002). A model intervention for girls with disruptive behavior problems: The Earlscourt girls connection. Canadian Journal of Counselling, 36(4), 297-311. Webster, C. D., Augimeri, L. K., & Koegl, C. J. (2002). The under 12 outreach project for antisocial boys: A research-based clinical program. In R. R. Corrado, R. Roesch, S. Hart, & J. K. Gierowski (Eds.), Multi-problem violent youth: A foundation for comparative research on needs, interventions and outcomes (pp. 207-218). Amsterdam: Ios Press. Webster, C. D., Douglas, K. S., Eaves, D., & Hart, S. D. (1997). HCR-20: Assessing risk for violence, Version 2. Burnaby, BC: Mental Health, Law, and Policy Institute, Simon Fraser University. Weisz, J. R., Sandler, I. N., Durlak, J. A., & Anton, B. S. (2005). Promoting and protecting youth mental health through evidence-based prevention and treatment. American Psychologist, 60(6), 628-648. Wilkinson, L. A. (2006). Monitoring treatment integrity. School Psychology International, 27(4), 426-438. Yuile, A. (2007). Developmental pathways of aggressive girls: A gender-sensitive approach to risk assessment, intervention, and follow-up. Unpublished doctoral dissertation, York University, Toronto, Canada.