Francis, J., Vella-Brodrick, D., & Chyuan-Chin, T. (2021). Effectiveness of online, school-based Positive Psychology Interventions to improve mental health and wellbeing: A systematic review. International Journal of Wellbeing, 11(4), 44-67. https://doi.org/10.5502/ijw.v11i4.1465 Jacqueline Francis The University of Melbourne jacqui.francis@unimelb.edu.au Copyright belongs to the author(s) www.internationaljournalofwellbeing.org 44 ARTICLE Effectiveness of online, school-based Positive Psychology Interventions to improve mental health and wellbeing: A systematic review Jacqueline Francis · Dianne Vella-Brodrick · Tan Chyuan-Chin Abstract: Online positive psychology interventions provide a more equitable method for young people to access wellbeing education at school than more traditional face to face programs. This systematic review aimed to examine the effectiveness of universal, online, school-based, positive psychology interventions using recommendations by the Preferred Reporting Items for Systematic reviews and Meta-analyses - protocols (PRISMA-P). Nine articles were identified for the review and were deductively, thematically analyzed using an enhanced RE-AIM framework which adopts a wider systems perspective including evaluation of socio-ecological readiness system wide buy-in and consideration of micro (individual) to macro (governing bodies) levels of influence, on both reach and adoption. Effectiveness assessment identified common factors for success related primarily to implementation (e.g., readiness, reach, outcomes, adoption, implementation, and maintenance). For example, buy- in from stakeholders was found to be highest when PPIs are age appropriate, engaging and helpful. Also brief, more frequent sessions, may be more effective than less frequent longer sessions and multi-level stakeholder buy-in may result in higher completion rates leading to better overall program effectiveness. Keywords: positive psychology; positive education; RE-AIM; online interventions; school students; implementation science 1. Introduction Adolescence is ordinarily a time of dynamic physiological, cognitive, and situational change (Unicef, 2020). Transition from primary school to high school can render young people vulnerable (Evans et al., 2018). COVID-19 has made these changes even more complex (UChicagoMedicine, 2020). Hence, primary schools are well placed to prepare students to thrive through uncertainty and transition. Indeed, the Australian Curriculum requires students to learn about building social and emotional awareness and management capabilities (Australian Curriculum, Assessment and Reporting Authority, 2021) and there is growing interest in positive psychology interventions (PPIs) - activities designed to maintain or improve wellbeing as well as to promote people feeling and functioning at their best (Slemp et al., 2017). How teachers effectively put this into action, however, remains a challenge. Online interventions present a modality that may be both practical and effective for young people to learn about wellbeing (Clarke et al., 2015; Manicavasagar et al., 2014). However, there is limited research examining online positive psychology interventions, particularly within the school context. This systematic review aims to address this gap in knowledge by examining the effectiveness of school-based, about:blank Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 45 online, positive psychology interventions. School-based PPI research provides one pathway to build wellbeing in the school context (Proctor et al., 2011; Waters et al., 2015). Wellbeing is considered a multidimensional construct (Forgeard et al., 2011) including the hedonic or feeling good aspects of subjective wellbeing (Diener et al., 1999), and the eudemonic or doing good aspects of social and psychological wellbeing (Ryff & Keyes, 1995; Keyes & Simoes, 2012; Keyes & Annas, 2009). Interventions for wellbeing typically include singularly focused mindfulness interventions (for review see Waters et al., 2015) or gratitude interventions (for review see Wood et al., 2010), as well as multi- dimensional programs, such as character strength interventions (Madden et al., 2011; Proctor et al., 2011) or multi-component programs for resilience, or social & emotional learning (PENN Resiliency Program: Brunwasser et al., 2009; PATHS curriculum: Greenberg et al., 1995). Multi- component, universal, online programs potentially provide flexibility to meet diverse student, teacher, and contextual needs. Universal interventions, taught to whole classes, avoid the possible stigma associated with targeted intervention (Greenberg & Abenavoli, 2017). Online programs can overcome existing implementation barriers such as geographical access or cost (Bennett & Glasgow, 2009; World Health Organization, 2019) and online PPIs have been successfully disseminated among adult cohorts (Bailey et al., 2018; Kemper et al., 2017; Proyer et al., 2015). However, research into youth online wellbeing promotion and health promotion and prevention is still emerging (for a systematic narrative synthesis into effectiveness of online mental health promotion and prevention interventions for youth aged 12-25 see Clarke et al., 2015). Moreover, very few studies exist evaluating the effectiveness of online, school-based PPIs, particularly in primary school aged cohorts. Evaluations of wellbeing programs such as PPIs often focus on outcome measures and their efficacy, while somewhat neglecting process evaluation, and overall effectiveness (Boiler et al., 2013; Sin & Lyubomirsky, 2009). However, implementation effectiveness is understood as critical in successful program delivery (Durlak et al., 2011). Hence, this review is focused on examining program effectiveness, for online, school-based PPIs. Given the acceptance of mental health outcome measures within PPI study design (Boiler et al., 2013; Sin & Lyubomirsky, 2009), we refer to PPIs in broad terms for this study. PPIs include those primarily concerned with enhancing wellbeing, or social emotional learning, as well as with reducing stress, anxiety, or depression. This review will identify commonalities and also gaps, in terms of readiness, reach, outcomes, adoption, implementation and maintenance. Effective, successful implementation of multi- component, universal approaches of programs occurs where there is readiness (transtheoretical model of change: Prochaska & Velicer, 1997), where the program content is sequenced, active, focused and explicit (SAFE: Durlak et al., 2011), where quality implementation is maintained (Grazczyk et al., 2012; Meyers et al., 2012), there is support from the encompassing system (Wandersman et al., 2008), and barriers to success are overcome. Program effectiveness can be evaluated using guiding frameworks such as RE-AIM (Glasgow et al., 2019). While a number of alternative evaluation guidelines exist such as the American Psychological Association’s JARS (APA Publications and Communications Board, 2008), and the Standards of Evidence (Flay, et al 2005), RE-AIM considers internal validity (sustained intervention effects) and external validity (representativeness) (Hone et al., 2015), and has been widely used within the Australian educational context (Lonsdale et al., 2016; Moore et al., 2013). RE-AIM provides a practical tool to evaluate and report on intervention effectiveness, providing insight into the likely value in upscaling interventions (for tools and resources see http://www.re-aim.org/). Readiness is important for evaluating intervention reach and adoption. The Transtheoretical model of change, proposes temporal stages of health behaviour change, including pre- http://www.internationaljournalofwellbeing.org/ http://www.re-aim.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 46 contemplative, contemplative, preparative, action and maintenance stages (Prochaska & Velicer, 1997). An individual’s readiness for change occurs during preparative and action phases. This model emerged from the comparative analysis of more than 300 theories (Prochaska & DiClemente, 1986), and the empirical analysis of health behaviour change (Prochaska & DiClemente, 1982; McConnaughy et al., 1984). Hence, it is a useful frame by which to consider intervention reach and adoption, potentially at different system levels. Contemplative readiness, in the form of perceiving intervention need, value and fit, provides insight into buy-in and reach. Actioned readiness, in the form of sustained buy-in and applied change, provides insight into intervention adoption. Intervention effectiveness is understood more deeply when implementation is considered as existing within a complex system. Understanding reach at an individual, class, school, and governing body level is valuable within the school context. Similarly understanding adoption at multiple system wide levels is instructive. The socio-ecological framework is informed by Bronfenbrenner’s (1977) multi-level Ecological Systems Theory. This adds further perspective to intervention readiness, reach and adoption, by considering not only individuals but the system in which they exist, at multiple levels including the microsystems, mesosystems, exosystems, macrosystems, and ecosystems (Lomas, 2015). For example, when considering reach, an individual teacher may wish to implement an intervention or program they see value in, however school leadership or departmental hierarchy may not agree, thereby blocking buy-in and limiting reach. Alternatively, leadership may perceive value in and need for a program, buying into the program and expecting teachers to deliver it, however the teacher may not agree, they may not see the point, or not have the resources to run the program, thereby impacting reach, implementation, and adoption. Evaluation of indicators of readiness for change (preparation, action), at multiple system levels, will further inform intervention reach and adoption effectiveness. The Interactive Systems Framework (ISF) (Wandersman et al., 2008) enhances an understanding of implementation via three succinct components, derived specifically to bring research and practice together (Wandersman et al., 2008) including: the synthesis and translation system - transforming theory into a practice friendly format; the delivery system and; the support system. This descriptive framework suits a qualitative evaluation of intervention effectiveness – focused primarily on implementation success for this study (e.g., readiness, reach, outcomes, adoption, implementation and maintenance) and has been embedded within the existing RE- AIM framework for this review. 2. Methods This systematic qualitative review drew on the preferred items for systematic review and meta- analysis protocols (PRISMA-P) (Shamseer et al., 2015) which guided planning, inquiry and reporting. 2.1 Search strategy A systematic search was conducted using the following databases: Ebscohost (ERIC, Medline, Academic Search Complete, Cinahl Complete), Web of Science, Ovid (PsycInfo), and Scopus. Search terms included: PPI, program, intervention, mindfulness, meditation, gratitude, kindness, positive relationships, pro-social, compassion, empathy, emotion regulation, emotional intelligence, strengths, character strength, social emotional learning, cognitive behavior therapy, web, online, computer, child, children, adolescents, youth, teenager, student, school, education, classroom, wellbeing, positive education, positive psychology, mental health, effectiveness, http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 47 evaluation, outcomes. See Appendix A for search strings per database. The screening of reference lists from past reviews and meta-analysis were also used to identify additional suitable studies (Ciocanel et al., 2017; Waters, 2015). The last search run was conducted in May 2020 by the first author. 2.2 Selection of studies Inclusion criteria were based on the structured PICOS approach used to determine study selection criteria, including five key components: population, intervention, comparison group, outcome, and study design (Liberati at al., 2009). In this instance the inclusion criteria for study selection was: (1) the intervention included school-aged children, (2) interventions were undertaken within the school context, during school hours, were web or computer based, and included the general population (not a clinical or specialized sub-group), the intervention must include a common PPI strategy, such as an activity or series of activities delivered for wellbeing or mental health benefit, for example, mindfulness, strengths or empathy interventions, (3) intervention evaluation included at a minimum pre and post measures and included a comparison or control group, (4) intervention measures should include evaluation of program effectiveness, and/or outcome measures including wellbeing and/or prevention of mental health symptomology, (5) the study must include pre and post measures of effectiveness, and (6) be published in peer reviewed journals in the English language. Exclusion criteria included: (1) children from clinical populations, (2) interventions that occurred exclusively out of the school environment, (3) interventions that were not online/web/computer based, (4) studies that did not include pre and post measures, (5) studies published as narrative or theoretical reviews, (6) non-English language articles. 2.3 Data extraction The first author was the main data extractor, who followed a standardized procedure to assess the eligibility of the studies. Duplicates and triplicate studies were removed. The title and abstract were then screened for inclusion and exclusion criteria. The full text of all remaining studies was read by the first author and second author. Discussion and consensus resulted in nine studies meeting the search criteria. 2.4 Quality assessment Methodological quality assessment was undertaken for each of the selected studies by using the revised tool for assessing risk of bias in randomized trials (RoB 2) (Sterne et al., 2019). The assessment involved ‘signaling questions’, a judgement about the risk of bias, and text boxes justifying responses to these, across key domains of interest. Signaling questions could be answered ‘Yes’, ‘Probably yes’, ‘Probably no’, ‘No’, or ‘No information’. Overall risk-of-bias judgements for each domain assessed were rated as ‘Low risk of bias’, ‘Some concerns’, or ‘High risk of bias’. These overall judgements were based on summarized answers to the signaling questions. The domains assessed for bias included: bias in the randomization process, bias from deviation from the intended intervention, bias due to missing outcome data, bias in measurement of the outcome, bias in result reporting selection, and overall bias (Sterne et al., 2019). Quality assessment for each paper was undertaken independently by the first and second authors. Discussion over discrepancies in risk of bias within domain assessment, resulted in selection of the higher risk assessment. There was 100% agreement on overall bias assessments across all nine studies. See Appendix B for Quality Assessment results. http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 48 2.5 Analysis A theoretically based, question driven, deductive thematic analysis (Braun & Clarke, 2006) of the literature was undertaken to create a rich understanding of universal, online, school-based PPIs and their effectiveness. To identify effective elements of the studies reviewed, RE-AIM effectiveness reporting standards were employed (Hone et al., 2015), enhanced by the inclusion of socio-ecological readiness (considering individual, setting and governing body levels of readiness for change) (Prochaska & Velicer, 1997; Bronfenbrenner, 1977) at the reach and adoption RE-AIM stages. Also, the integration of the ISF provided a clearer understanding of implementation effectiveness. Studies were analyzed according to these five enhanced RE-AIM components: Readiness and Reach, Efficacy, Readiness and Adoption, Implementation and Maintenance. Figure 1 (below) illustrates the RE-AIM framework, adapted in the Reach, Adoption and Implementation dimensions. The computer program NVIVO 12 was used to create thematic nodes of Reach, Efficacy, Adoption, Implementation and Maintenance (RE-AIM, Glasgow et al., 2019). Identified papers were systematically coded, resulting in the development of sub-themes. Differing from the existing RE-AIM framework, and in line with the enhanced RE-AIM framework articulated in this paper, both reach and adoption were considered at multiple system levels, including individual, class, school and governing bodies. Table 1 (below) illustrates thematic nodes established during thematic analysis. Table 1. Thematic nodes established during thematic analysis of identified studies Themes Sub-themes Reach Inclusion/exclusion criteria Target population Recruitment Readiness (perceived) Macro-level buy-in Motivation for and against buy-in Participants Representativeness Efficacy Analysis Intention to treat Negative result Low or no significance, low effect size. Low baseline measures Adoption Adherence Motivation for and against Readiness and Support-micro and macro level Implementation Duration Content Delivery Design Flexibility & Fit Replication Maintenance Maintenance Other Barriers Online Value http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 49 Figure 1. Enhanced RE-AIM framework including socio-ecological readiness and ISF 2.5.1 Analysis - Readiness and Reach (micro to macro) Selected studies were examined for readiness and reach. Reach in this review includes elements of RE-AIM associated with establishing buy-in before program commencement (including some http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 50 elements previously allocated to adoption). Visible indicators of readiness were considered in terms of the temporal stages of change including contemplation, preparation, or action toward change (Prochaska & Velicer, 1997). Reach was considered in terms of setting level reach (schools), class level reach (teachers) and individual reach (target student participants). Studies were analyzed for inclusion and exclusion criteria, recruitment methods, buy-in characteristics, buy-in rates, sample size, and motivation for buy-in. Characteristics and motivations of non- participators and drop-out rates were also analyzed (Hone et al., 2015). 2.5.2 Analysis - Adoption Adoption in this review relates to program consumption, and includes factors associated with the use and completion of programs, at individual, class, school and governing body levels. Delivery agent selection and description, sustained buy-in, and barriers to sustained buy-in were considered. Adoption rates were analyzed in terms of completion of pre and post measures, and adherence to the program (dosage). 2.5.3 Analysis - Implementation Studies were analysed for intervention type, intervention content replication capacity, for intended delivery of core program components (Hone et al., 2015) including delivery agent authenticity, core message fidelity and program flexibility for fit. Studies were also analyzed for sufficient support, including support for individuals (training or coaching), and setting level capacity (resources), and for any barriers evidently impeding implementation (Wandersman et al., 2008). 3. Results 3.1 Study selection A total of 203 articles were obtained from the electronic databases, including 22 from Ebscohost, 16 from Web of Science, 22 from PsycInfo, and 143 from Scopus. One hundred and ninety two remained after the removal of duplicates. Title and abstract screening resulted in two remaining articles. Seven additional articles were located in reference lists, resulting in a total of nine articles that met the inclusion criteria. See the PRISMA guided selection process in Figure 2 (below). 3.2 Description of studies Nine studies were identified. These studies included 5157 school children. Of those only one involved primary aged school children. Eight of nine studies drew on secondary school students, including those from single sex, co-educational, private, religious, and government secondary schools. See Table 2 (below) for an overview of the main characteristics of studies included. http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 51 Figure 2. PRISMA analysis illustrating literature search results Ebscohost including ERIC, Medline, Academic Search Complete, Cinahl complete n=22 Web of Sicence n=16 Ovid – PsycInfo n=22 Scopus n=143 Articles obtained from search n=203 Duplicates removed = 11 Remaining articles = 192 Sc re e n in g Id e n ti fi ca ti o n E lig ib ili ty Full text screened for eligibility according to selection criteria N = 2 Title and abstract screened for relevance N = 192 In cl u d e d Articles included for review N = 9 Unable to access full text N=0 Excluded based on title and abstract N=190 -Not child sample (n=35) -Clinical sample (n=16) -Non-school context (n=75) -Not online intervention (n=34) -Purpose not to enhance wellbeing, or decrease depression or anxiety (n=14) -Meta analysis or review (n=14) -Without comparison or control group (n=2) Additional articles located in reference lists N=7 http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 52 Table 2. Main characteristics of studies included First author (year) Location School Type Sample size and age or school year Study design Intervention type Duration Comparison group/s Outcome measures Attwood (2012) UK Co- educational primary school n=13, boys 10-12 years Non- randomised matched control experiment. CBT 6 x 45 minute weekly sessions Placebo. Computer gaming group SCAS; SDQ; perceptions interview Bannink (2012) Nether- lands Co- educational secondary schools n=1702, mean age 15.9 years Cluster randomised controlled trial Mental health, quality of life, health behaviours, personalised information Unclear Care as usual. SDQ; YSR; HQL Burck- hardt (2015) Australia 2 x Anglican girls secondary school, 1 x Jewish co- educational secondary school, 1 x Catholic boys secondary school n=572, student s from grades 7, 9, 10, 11, 12, RCT block random- ization Positive psychology wellbeing intervention ‘Bite Back’ 6 hours delivery over 4-6 weeks Placebo. Altern-ative non- psychology program DASS-12; SLSS; SWEMWBS Calear (2013) Australia 16 governmen t secondary schools, 14 private secondary schools n=1477, mean age 14.3 years, grades 9, 10, 11 Cluster RCT CBT based ‘MoodGym’ One module per week during one class (29 exercises) over five weeks Waitlist Adherence; RCMAS; CES-D Calear (2009) Australia 16 governmen t secondary schools, 14 private secondary schools n=1477, mean age 14.3 years, grades 9, 10, 11 Cluster RCT CBT based ‘MoodGym’ One module per week during one class (29 exercises) over five weeks Waitlist RCMAS; CES-D; Adherence and attendance http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 53 Table 2. Main characteristics of studies included (continued) First author (year) Location School Type Sample size and age or school year Study design Intervention type Duration Comparison group/s Outcome measures Fridrici (2009) Germany Co- educational secondary schools n=904 mean age 14.8 years, grades 8 & 9 Four treatment conditions. Pre and post design. Stress prevention program 8 x 90 minute weekly sessions Three groups: Online- training at home, school- based face- to-face, no inter- vention group Knowledge of stress and coping questions, Thought listing (stress); SSKJ 3-8; training acceptance O’- Kearney (2006) Australia 1 x boys secondary school n=78 15 & 16 years old, from grade 9 Non- randomised control experiment CBT based ‘MoodGym’ 5 x 30-60 minute modules, over 5 weeks Care as usual. CES-D; CASQ-R; RSES; ATTDEP O’- Kearney (2009) Australia 1 x girls secondary school n=157, 15 & 16 years old, from grade 10 RCT block random- isation CBT based ‘MoodGym’ 5 x 30-60 minute modules, over 5 weeks Care as usual CES-D; CASQ-R; DLC; Attitudes to depression Puola- kanaho (2019) Finland Two sub samples from original longitud- inal study of 800 9th grade students from two munici- palities n=249, grade 9 RCT ACT based ‘Youth COMPASS’ One module = 6 exercises (each 5- 10 minutes). 1 x module per week for 5 weeks Three groups: Program with support, Program without support, Care as usual Salzburg Reading Fluency Test, Basic Arithmetic Test, HBSC, Academic Buoyancy Scale, Adherence Note. SCAS: Spence Children’s Anxiety Scale Child & Parent Version (Spence, 1998), SDQ: Strengths and Difficulties Questionnaire Parent Version (Goodman, 1997), YSR: Youth Self Report; HQL: Health related Quality of Life; DASS-12: The Depression, Anxiety, and Stress Scale – Short form; SLSS: Student Life Satisfaction Scale; SWEMWBS: The Short Warwick-Edinburgh Mental Well-Being Scale, RCMAS: Revised Children’s Manifest Anxiety Scale; CES-D: Center for Epidemiological Studies Depression Scale; SSKJ 3-8: Questionnaire for the measurement of stress and coping in children and adolescents, CASQ-R: The Revised Children’s Attributional Style Questionnaire, RSES: Rosenberg Self-Esteem Scale; ATTDEP: Attitudes to depression, DLC: Depression literacy scale; HBSC: Health Behaviour in School-Aged Children, CBT: cognitive behavioral therapy; ACT: acceptance commitment therapy. http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 54 3.2.1 Readiness and Reach – recruitment rationale Recruitment varied widely across studies. One study was based on a convenience sample (Attwood et al., 2012). One German study recruited considerable numbers of participants, although reasons for particular school choice were not articulated (Fridrici & Lohaus, 2008). One Dutch study recruited two youth health care organizations from two Dutch cities, who invited 14 schools to participate (Bannink, et al. 2012). Two studies from one project recruited schools from across Australia, including schools from the Australian Capital Territory (ACT), (Calear et al., 2009; 2013). Two Australian studies invited all students from within defined year levels (year nine and year ten respectively) at two selected single sex schools (O’Kearney et al., 2006; 2009). One Australian study invited schools to participate based on their physical proximity allowing researchers physical access to schools during the study. These schools were also non- governmental to overcome delays associated with state school department of education permission (Burckhardt et al., 2015). One study drew on a subsample of a larger longitudinal Finnish study, involving two municipalities in Central Finland – the subsample inclusion criteria was clearly defined and included ninth grade students with concerning reading and math skill levels (Puolakanaho et al., 2019). 3.2.2 Readiness and Reach – location and school level Most studies occurred within the secondary school context, in Western Countries. Studies included: five Australian secondary schools studies (Burckhardt et al., 2015; Calear et al., 2009; 2013; O’Kearney et al., 2006; 2009), one UK primary school study (Attwood et al., 2012), one Netherlands secondary schools study (Bannink et al., 2012), one Finnish secondary schools study (Puolakanaho et al., 2019), one German secondary schools study (Fridrici & Lohaus, 2009). 3.2.3 Readiness and Reach – participant characteristics and representativeness Studies represented students ranging in age from 10 to 18 years, and from both government and non-government school sectors. Sample sizes also varied widely from 24 to 6,227 students. One study involved over 1000 students from government secondary schools: n=1702, representing 12 Dutch secondary schools and the average Dutch population, with mean age 15.9 years (Bannink et al., 2012). One study involved 33 year eight and nine German secondary school classes, n=904 (Fridrici & Lohaus, 2009). One Australian paper reported on non-parametric analysis of two studies using a small opportunistic sample (13 boys 10-12 years, and 12 girls and boys aged 10- 16 years) (Attwood et al., 2012). Two Australian studies involved students from single-sex secondary schools in the ACT, including 78 year nine boys (O’Kearney, et al., 2001) and 157 year ten girls (O’Kearney, et al., 2009). One further Australian study involved students from two Anglican girls secondary schools, including grade seven and 10 at one school, and grade 10, 11, 12 at another school, one Jewish co-educational secondary school including grade seven students, and one Catholic boys secondary school, including grade nine students, 572 students overall (Burckhardt, et al., 2015). One study involved two sub samples of 125 and 124 students from an original longitudinal study of 800 ninth grade students from two municipalities in Finland (Puolakanaho et al., 2019). Two studies emerged from one Australian project involving 14 schools from private, public, coeducational, single-sex, metropolitan, and rural schools from six states, with 1,477 students, mean age 14.34 years (Calear et al., 2009; 2013). http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 55 3.2.4 Readiness and Reach: Motivation for buy-in and motivation for non-participation Indicators for readiness and reach were evident at multiple system levels. Governing bodies reportedly implement initiatives to promote emotional skills (Attwood et al., 2012), and the Netherlands have established youth health care structures, including preventative health consultations with children (Bannink, et al., 2012). Lack of readiness and reach were evident at individual and setting levels. Individual participant non-involvement resulted from: absence due to illness or school change (Bannink et al., 2012; Fridrici & Lohaus, 2009), unavailability on data collection day (O’Kearney et al., 2007), parental refusal (Bannink et al., 2012; Fridrici & Lohaus, 2009), participant refusal or withdrawal (Bannink et al., 2012; Fridrici & Lohaus, 2009), or based on behavioural concerns (Fridrici & Lohaus, 2009). 3.2.5 Efficacy-measured impact Measured impact varied across studies. Intention to treat (ITT) analysis involves using data from every participant who is randomized (Gupta, 2011). Only three studies reported using ITT analysis (Burckhardt et al., 2015; Calear et al., 2013; O’Kearney et al., 2006). In terms of outcomes measured there were instances of no significant effects, such as for one Australian study reporting no significant change in anxiety, stress, depression and life satisfaction (Burckhardt et al., 2015). Importantly, negative outcomes were rare, although these were reported in one Dutch study where there were improved overall mental health outcomes, but a negative effect on drug use among boys (Bannink et al., 2012). Overall, however there were nuanced observations of positive impact, commonly evidenced in relation to mental health. Statistically significant post intervention reduction in total child anxiety ratings, with reductions on the social anxiety and generalized anxiety subscale, was evident for one treatment group (Attwood et al., 2012). E-health4Uth alone showed significant positive results in health-related quality of life, E-health4Uth and consultation combined demonstrated significant positive results in mental health status (Bannink et al., 2012). The online self-directed Mood-gym treatment condition was used in four studies. High adherence, compared to low adherence, was associated with stronger effects on anxiety, at post- intervention and at 6-month follow-up (Calear et al., 2013). Significantly lower levels of anxiety at post intervention and 6-month follow up, and change in depressive symptoms was only significant for males in the Calear et al., (2009) study. In the O’Kearney et al., (2006) study, compared to a standard personal development condition, no significant between-group differences were observed, although completion of three or more modules was associated with small benefits for depressive symptoms, attributional style, and self-esteem. There was also a reduction in risk of being classified as vulnerable to depression, at immediate follow up only (O’Kearney et al., 2006). Online and self-directed Mood-Gym treatment condition, compared to a control condition, demonstrated a significantly faster decline in depressive symptoms. While no significance was detected immediately post intervention, a moderate and significant effect was present 20 weeks after the intervention. (O’Kearney et al., 2009). The Fridrici and Lohaus (2009) study, showed the greatest ratio of positive cognitions occurred for the school-based face-to-face condition, then online-school condition, then online- home condition, and that all were greater than the control condition. Knowledge gain related to stress and coping was demonstrated post online-program in German Secondary Schools (Fridrici & Lohaus, 2009). Web and mobile delivered Youth COMPASS treatment group, compared to a control group, showed a significant decrease in overall stress and increase in academic buoyancy (Puolakanaho et al., 2019). Participants more depressed or more stressed at baseline, tended to demonstrate the greatest benefit from exposure to the treatment condition (Bannink et al., 2012; http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 56 Calear et al., 2013; O’Kearney et al., 2009; Puolakanaho et al., 2019). 3.2.6 Adoption, delivery agents, sustained buy-in, completion-adherence barriers Completion of pre and post-program measures varied. Participation flow was not recorded with one small sample (Attwood et al., 2012). 52% of pre and post data for the treatment group, and 48% of data for the waitlist group was obtained for the MoodGym studies by Calear and colleagues (2009; 2013), 71% of data was obtained for post measures, and 80% of data for follow- up data for O’Kearney and colleagues (2006). Of 572 randomized students in one study, 59% completed baseline measures, and of those 55% returned post measure questionnaires. This study was teacher supervised and workbook guided and saw two of four schools drop out, one school citing that students found the required activities boring and without clear purpose, while at the other school the teacher in charge of the program left the school (Burckhardt et al., 2015). The Youth COMPASS study obtained 95% of originally randomized students’ pre and post measures (Puolakanaho et al., 2019). Program delivery agents, completion and adherence rates also varied. Completion and adherence rates of one small opportunistic sample, that was researcher led, was unknown as participation flow was not recorded (Attwood et al., 2012). One coach directed program saw 75% of students completing more than half of the program (Puolakanaho et al., 2019). Teacher delivered programs reported 62% or more students, completed more than half of the program (Calear et al., 2009; 2013; Fidrici & Lohaus, 2009). Teacher supervision of student self-directed programs saw 40% of students completing more than half of the program (O’Kearney et al., 2006; 2013). Teacher supervision of a workbook guided program, saw 56% of participants completing 5-6 out of 6 workbooks for the ‘Bite-back’ program (Burckhardt, 2015). 3.2.7 Implementation-intervention type and replication capacity Program content varied. Seven programs were based on Cognitive Behavioural Therapy (CBT) approaches (Attwood et al., 2012; Calaer et al., 2009; Calaer et al., 2013; O’Kearney et al., 2006; O’Kearney et al., 2013). One study focused on promoting adolescent mental health and health behaviours via tailored online messaging (Bannink et al., 2012). One program incorporated an Acceptance Commitment Therapy (ACT) intervention model (Puolakanaho et al., 2019). One program focused on cognitive reconstruction, seeking social support, relaxation and time management (Fidrici, 2009). One program focussed on nine positive psychology domains including gratitude, optimism, flow, meaning, hope, mindfulness, character strengths, healthy lifestyle, and positive relationships (Burckhardt, et al., 2015). 3.2.8 Implementation - delivery agent and dosage Lesson frequency and duration varied. Studies (five) involved weekly sessions over five to six weeks (Calaer et al., 2009; 2013; O’Kearney et al., 2006; 2009; Puloakanaho et al., 2019), and one study involved weekly sessions over four to six weeks (Burckhardt et al., 2015). Lesson length ranged from 20 to 40 minutes (Calaer et al., 2009), 45 minutes (Attwood et al., 2012), 90 minutes (Fidrici & Lauhaus, 2009), 60-90 minutes per week accumulated with ‘regular website use’ (Burckhardt et al., 2015), and 30-60 minutes (O’Kearney, et al., 2006). Agent delivery varied. One program was researcher delivered (Attwood et al, 2012), one program was partly nurse delivered (Bannink et al. 2012), two programs were teacher delivered (Calaer et al, 2009; 2013), four programs were teacher supervised (Burckhardt et al., 2015; Fidrici & Lohaus, 2009; O’Kearney et al, 2006; 2009), and one study was coach directed (Puolakanaho et http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 57 al, 2019). 3.2.9 Quality assessment – Risk of Bias (RoB) Quality assessment of selected studies was undertaken using the RoB2 tool (Sterne et al., 2019), determining risk of bias across five domains. The five domains assessed the risk of bias related to the randomization process, deviation from the intended intervention, missing outcome data, measurement of the outcome, and result reporting selection. Across the nine studies Domain 4 (risk of bias in measurement of the outcome) was assessed with the highest risk of bias. This highlighted for example, areas of weakness associated with appropriate method choice, participant awareness of intervention group, and possible pre-conceptions of intervention benefit, potentially impacting outcomes. Comparatively, Domain 5 (risk of bias in reporting selection) was assessed as low risk for most studies, illustrating an area of strength in relation to data analysis following a pre-determined plan. 4. Discussion This review involved the identification and analysis of nine studies, reporting on the efficacy and process effectiveness of universal, online, school-based PPIs. While PPIs are usually considered to be activities designed to primarily promote wellbeing, this review has broadened the scope to include PPIs designed to improve mental health. Examination of program effectiveness was undertaken using an enhanced RE-AIM framework which incorporated socio-ecological readiness and integrated the transtheoretical model of change (Prochaska & Velicer, 1997) and multi-level ecological systems theory (Bronfenbrenner, 1977), adding aspects of readiness for change at multiple system levels to the evaluation of intervention reach and adoption. Additionally, incorporating the ISF (Wandersman et al., 2008) into RE-AIM, adds depth and detail when evaluating intervention implementation, particularly in relation to aspects of delivery. In terms of reach, the nine studies identified represented school aged students attending Australian, UK, German, Finnish and Canadian schools, and involved diverse study designs. One small exploratory study included a sample of 13 (Attwood et al., 2012), another study allocated 78 study participants in a school-based controlled trial (O’Kearney et al., 2007). All other studies involved samples over 100 (Burckhardt, et al., 2015; Fidrici & Lohaus, 2008; O’Kearney, et al., 2009; Puolakanaho et al., 2019) and three studies, based on three samples, were drawn from samples of over 1000 (Bannink et al., 2012; Calear, et al., 2009; 2013). These studies involved three cluster randomised controlled trials (Burkhardt, 2015; Calear, et al., 2009; 2013), two randomised controlled trials (Bannink et al., 2012; Puolakanaho et al., 2019), and one controlled trial (O’Kearney et al., 2009). Only one study focused on primary aged students (Attwood et al., 2012). This highlights a need for research with primary school aged cohorts and online PPIs. Together, these nine studies represent participants drawn from government, catholic and independent school sectors. Common inclusion criteria for participants included being school aged students, part of regular classroom activity, within the age or year level required for the study focus and having signed parental consent. Why samples were selected for invitation and subsequent recruitment varied and was rarely articulated. Considering readiness, motivators for participation in online PPIs existed at different levels (government, community, school, teacher, student). Motivators were sometimes shared, and sometimes specific to individual context. Often there was stated or assumed over-arching support from governing bodies who are becoming increasingly aware of the economic and societal costs of mental illness and are keen to find ways to improve mental health in society, http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 58 particularly targeting youth (Attwood et al., 2012; Bannink et al., 2012). Individual motivation for participation could be considered in terms of intervention need, value and fit, however most studies did not explicitly measure these aspects (Calear et al., 2013). Noted by researchers was the reluctance of male adolescents to engage with mental health services (O’Kearney et al., 2007), but also their positive attitudes and engagement with computers (Colley & Comber, 2003), supporting the potential of the online delivery mode for some cohorts. Little is known about initial participation refusal, however competing school schedule demands affected data collection timing for one study (O’Kearney et al., 2006). Delivery expertise is an important consideration for implementation fidelity. Delivery occurred in a variety of ways: student driven, teacher supervised, teacher delivered, health specialist delivered or coach facilitated. Puolakanaho et al. (2019), suggested that delivery of online PPIs did not require specialized skills. However, some programs drew on the capacity of experts including health-led programs (Bannink et al., 2012), coach led programs (Puolakanaho et al. 2019), or researcher supervision (Attwood et al., 2012). Teacher delivery capacity was supported via teacher training via the provision of delivery manuals (Calear et al., 2009; 2013). While delivery capacity impacts on readiness and reach in the early phase of intervention buy-in, delivery capacity (human and environmental) remains relevant when considering adoption and sustained buy-in, and implementation aspects of effectiveness. One school noted the lack of flexibility in a fixed online program, where two of four schools dropped out, and students (ranging from grades 7 to 12) reported finding the program ‘boring’ (Burckhardt, et al., 2015). Flexibility for fit, particularly age appropriate, engaging and meaningful content may be key to effective delivery. Adoption is often determined by completion rates of pre and post measures, or exercise completion rates (Calear et al., 2009; 2013). Based on numbers reported, overall completion of pre and post measures varied from 52% of the MoodGYM treatment group (Calear et al., 2009; 2013), to 55% of Bite Back participants who completed pre measures completed post measures (Burckhardt, 2015),to 60% of allocated male participants from MoodGYM treatment and control groups (2006), to 71% of all allocated female participants from MoodGYM treatment and control groups (O’Kearney et al., 2006), 73.5% and 77.6% for the Netherlands E-health and E-health combined with counselling respectively (Bannink et al., 2012), and 95% for the Finnish Youth COMPASS study (Puolakanaho et al., 2019). It is noteworthy that the studies with the greatest pre and post measure completion rates were studies with larger sample sizes and multi-level buy-in and support for the process, either via embedded existing health delivery structures (Bannink et al., 2012), or commitment to broader longitudinal research into wellbeing (Puolakanaho et al., 2019). Closely related to adoption and the acceptability of programs, are implementation considerations such intervention content and delivery. Researcher, coach led, or health led groups demonstrated the highest adoption and completion rates (Bannink et al., 2012; Puolakanaho et al., 2019). Teacher delivered programs (Calear et al., 2009; 2013; Fidrici & Lohaus, 2009) showed higher rates of adoption and completion, compared to student-directed programs supervised by teachers (O’Kearney et al., 2006; 2013). One study compared a face-to-face intervention, with an at school, supervised online intervention, and an at home, unsupervised intervention (Fidrici & Lohaus, 2008). Students preferred people to computers. The face-to-face intervention was the preferred approach, followed by the at school supervised online intervention. The intervention that was entirely student driven was rated as the least acceptable training approach, which resulted in 37% non-participation (compared to 100% participation online at school), as well as the least effective training outcomes. The presence of a human http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 59 delivery agent, and delivery agent expertise, led to better outcomes. Intervention types varied, although seven programs were based on CBT approaches and one used an ACT intervention model – reflecting the two main approaches used in Australian psychological clinical settings. Program duration varied and ranged from 20 to 60 minute lessons, and typically ran once per week over five or six weeks. The programs with the highest adoption levels included Youth COMPASS with six or more 5-10 minute exercises, per week over five weeks (Puolakanaho et al., 2019). It is possible that greater frequency of program dosage, and revisiting core messages, is more valuable than less frequent, longer sessions. These nine studies measured a range of constructs to ascertain program efficacy. In some instances, no significant effects were detected (Burckhardt et al., 2015). In others, small, significant intervention effects were observed including a decrease in overall stress (Puolakanaho et al., 2019). Overall, typically small positive, often non-significant effects were observed universally for participants (O’Kearney et al., 2006; 2013; Bannink et al., 2012). Interestingly, in some instances this positive effect was enhanced at follow-up, illustrating a lag effect, for reduced universal anxiety levels and male depressive symptoms (Calear et al., 2009), as well as reduced female depressive symptoms among those high in depression at baseline (O’Kearney et al., 2009). Importantly, certain sub-groups experienced greater benefit from PPIs. Unsurprisingly, participants who completed more sessions, gained more from the experience (Calear et al., 2013), and participants with lower baseline measures tended to benefit the most from PPIs (Bannink et al., 2012; O’Kearney et al., 2009; Puolakanaho et al., 2019). Fidrici and Lohaus (2009) found a significant intervention effect on adolescents’ knowledge of stress and coping. Perhaps this measurement of knowledge, provides insight into the stages of learning and skills acquisition, indicating participants need for time to digest new knowledge and practice new skills, before changes should be expected in measures of emotion, cognitions and behaviours. Four studies collected measures at multiple times points, including pre, post and follow up data (Calear et al., 2009; 2013; O’Kearney et al., 2006; 2009). This provided useful insight into the trajectory of intervention impact, however no study reported on long term maintained use of the program. This qualitative systematic review of nine studies has provided insight into the effectiveness of online, school-based, universal PPIs. Examining the effectiveness of these studies using the enhanced RE-AIM framework, illustrates both strengths of current studies, as well as gaps and opportunities for future studies. Participant characteristics are typically reported – most participants are secondary school aged students, although emerging research with younger participants appears promising and suggests a future research cohort opportunity in primary school aged participants. Online PPIs appear to overcome barriers related to restricted human and environmental resources, such as personnel and timetabling limitations, although the value of delivery agent capacity and expertise, and of human connection, remains unknown. The rationale for why and how participants are selected and invited is rarely reported, and if done so, could provide further insight into effectiveness. Motivation for systemic buy-in is also rarely reported, although indicators of PPI acceptability in the form of perceptions of need, value and fit provide valuable insight into initial and sustained buy-in. Implementation is also key to understanding intervention effectiveness, however ideal dosage levels for effectiveness are yet to be determined. Measures of impact also vary among studies, although rely on child self-report, and at times also parent or teacher reported evaluations, on feelings and thoughts. Opportunities exist to further explore PPI impact on knowledge and skill development, using a wider range of http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 60 measurement tools, to capture a more comprehensive picture of PPI effectiveness. Quality assessment undertaken for the identified studies, pointed to risk of bias in measurement of the outcomes as a particular area of concern. This suggests future research could be strengthened by choosing appropriate methods, including valid instruments, and suitable data collection procedures. Research design ensuring blinded outcome assessors would also add strength to future research. 4.1 Practical implications This systematic qualitative review has resulted in key learnings detailed in Table 3. Table 3. Key learnings from review Key recommendations for practitioners. Schools, teachers and students are likely to be more motivated to buy-in, where PPIs are age appropriate, engaging and helpful – perhaps where there is perceived need, value and fit. To achieve better intervention fit, include stakeholders in the co-creation of interventions or develop interventions incorporating responsive, real time, adaptability based on user input. People who tend to experience the greatest benefit from PPIs include those with lower baseline measures, and those who complete more intervention sessions. Brief, more frequent sessions, including revisiting core messages, may be more valuable than less frequent, longer sessions. Proactively planning to work around barriers to buy-in, up-take and sustained adoption and implementation, such as access to resources including personnel, time, and internet seem integral to intervention success. An online approach may facilitate program integrity by retaining fidelity to core components. Delivery agent capacity can be supported via training or upskilling. Multi-level buy-in and support, including region wide approaches, result in greater completion rates. System wide buy-in and change support with the intention of sustaining and maintaining change, designed with clear, shared intent and purpose, can result in valuable positive wellbeing and mental health outcomes. How success of PPIs is measured may need to be reconceptualized to include measures of early learning and skill acquisition, to acknowledge the realistic processes and time required to digest new knowledge and practice new skills, before embedded changes in emotion, cognitions and behaviors will be visible. 4.2 Limitations In considering these recommendations it is important to keep in mind limitations of this review. The review included a relatively small number of studies due to the limited research available. The search was limited to peer reviewed articles, and as such may have been biased towards studies reporting significant results. This review was qualitative in nature, and so while providing a deep and detailed inquiry into effectiveness, it did not quantitatively analyse outcome data, which may add further understanding to online PPI outcome efficacy. Overall http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 61 however, this study provides useful initial insights into universal online PPIs in the school context. In the right conditions, universal online school-based PPIs provide a useful approach to improving the wellbeing and mental health of school children. Online PPIs containing the right ingredients for their context, can preserve and build student social, emotional and psychological capacity, equipping young people to survive in the hard times and thrive in the good times. The bottom line is that the context, system and what we elect to measure matter. The challenge now is to create a clear framework for identifying effective conditions, ingredients and contexts that can optimise universal online PPI success. This review has commenced this process and serves to encourage others to extend on this work. Conflict of Interest The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. Data Availability Statement Data queries should be directed to JF: jacqui.francis@unimelb.edu.au. Authors Jacqueline Francis The University of Melbourne jacqui.francis@unimelb.edu.au Dianne Vella-Brodrick The University of Melbourne Tan Chyuan-Chin The University of Melbourne Author Contributions Research design and planning was undertaken by DVB, TCC and JF. Systematic review undertaken by JF with input from DVB. Coding and thematic analysis of articles undertaken by JF. Paper written by JF. Quality assessments of included studies were undertaken by JF and DVB. Paper written by JF. Critical revision and editing by DVB and TCC. Acknowledgments This systematic review was undertaken as part of a PhD thesis (JF), kindly supported through an Australian Government Research Training Program Scholarship. Publishing Timeline Received 25 October 2020 Revised version received 2 September 2021 Accepted 2 September 2021 Published 30 September 2021 References APA Publications and Communications Board. (2008). Reporting standards for research in psychology: Why do we need them? What might they be? The American Psychologist, 63(9), 839–851. https://doi.org/10.1037/0003-066X.63.9.839 http://www.internationaljournalofwellbeing.org/ mailto:jacqui.francis@unimelb.edu.au Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 62 Attwood, M., Meadows, S., Stallard, P., & Richardson, T. (2012). Universal and targeted computerised cognitive behavioural therapy (Think, Feel, Do) for emotional health in schools: Results from two exploratory studies. Child and Adolescent Mental Health, 17(3), 173–178. https://doi.org/10.1111/j.1475- 3588.2011.00627.x Australian Curriculum, Assessment and Reporting Authority. (2021, July). Personal and Social Capability. https://www.australiancurriculum.edu.au/f-10-curriculum/general-capabilities/personal-and-social- capability/ Bailey, N. W., Nguyen, J., Bialylew, E., Corin, S. E., Gilbertson, T., Chambers, R., & Fitzgerald, P. B. (2018). Effect on Well-Being from an Online Mindfulness Intervention: “Mindful in May.” Mindfulness, 9(5), 1637–1647. https://doi.org/10.1007/s12671-018-0910-7 Bannink, R., Joosten - van Zwanenburg, E., van de Looij - Jansen, P., van As, E., & Raat, H. (2012). Evaluation of computer-tailored health education (‘E-health4Uth’) combined with personal counselling (‘E-health4Uth + counselling’) on adolescents’ behaviours and mental health status: design of a three-armed cluster randomised controlled trial. BMC Public Health, 12(1), 1083. https://doi.org/10.1186/1471-2458-12-1083 Bennett, G. G., & Glasgow, R. E. (2009). The Delivery of Public Health Interventions via the Internet: Actualizing Their Potential. Annual Review of Public Health, 30(1), 273–292. https://doi.org/10.1146/annurev.publhealth.031308.100235 Boiler, L., Haverman, M., Westerhof, G. J., Riper, H., Smit, F., & Bohlmeijer, E. (2013). PPIs: a meta- analysis of randomized controlled studies. BMC Public Health, 13(1), 119. https://doi.org/10.1186/1471- 2458-13-119 Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa Bronfenbrenner, U. (1977). Toward an experimental ecology of human development. American Psychologist, 32(7), 513–531. https://doi.org/10.1037/0003-066X.32.7.513 Brunwasser, S. M., Gillham, J. E., & Kim, E. S. (2009). A meta-analytic review of the Penn Resiliency Program's effect on depressive symptoms. Journal of consulting and clinical psychology, 77(6), 1042– 1054. https://doi.org/10.1037/a0017671 Burckhardt, R., Manicavasagar, V., Batterham, P. J., Miller, L. M., Talbot, E., & Lum, A. (2015). A Web- Based Adolescent Positive Psychology Program in Schools: Randomized Controlled Trial. Journal of Medical Internet Research, 17(7), e187. https://doi.org/10.2196/jmir.4329 Calear, A. L., Christensen, H., Mackinnon, A., & Griffiths, K. M. (2013). Adherence to the MoodGYM program: Outcomes and predictors for an adolescent school-based population. Journal of Affective Disorders, 147(1), 338–344. https://doi.org/https://doi.org/10.1016/j.jad.2012.11.036 Calear, A., Christensen, H., Mackinnon, A., Griffiths, K., & O’Kearney, R. (2009). The YouthMood Project: A Cluster Randomized Controlled Trial of an Online Cognitive Behavioral Program with Adolescents. Journal of Consulting and Clinical Psychology, 77(6), 1021–1032. https://doi.org/10.1037/a0017391 Ciocanel, O., Power, K., Eriksen, A., & Gillings, K. (2017). Effectiveness of positive youth development interventions: A meta-analysis of randomized controlled trials. Journal of youth and adolescence, 46(3), 483-504. http://doi.org/10.1007/s10964-016-0555-6 Clarke, A., Kuosmanen, T., & Barry, M. (2015). A Systematic Review of Online Youth Mental Health Promotion and Prevention Interventions. Journal of Youth & Adolescence, 44(1), 90–113. http://10.0.3.239/s10964-014-0165-0 Colley, A., & Comber, C. (2003). Age and gender differences in computer use and attitudes among secondary school students: what has changed? Educational Research, 45(2), 155–165. https://doi.org/10.1080/0013188032000103235 Diener, E., Suh, E. M., Lucas, R. E., & Smith, H. L. (1999). Subjective well-being: Three decades of progress. Psychological bulletin, 125(2), 276. https://doi.org/10.1037/0033-2909.125.2.276 Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The Impact of Enhancing Students’ Social and Emotional Learning: A Meta-Analysis of School-Based Universal Interventions. Child Development, 82(1), 405–432. https://doi.org/10.1111/j.1467-8624.2010.01564.x http://www.internationaljournalofwellbeing.org/ https://www.australiancurriculum.edu.au/f-10-curriculum/general-capabilities/personal-and-social-capability/ https://www.australiancurriculum.edu.au/f-10-curriculum/general-capabilities/personal-and-social-capability/ http://10.0.3.239/s10964-014-0165-0 https://doi.org/10.1080/0013188032000103235 Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 63 Evans, D., Borriello, G. A., & Field, A. P. (2018). A review of the academic and psychological impact of the transition to secondary education. Frontiers in psychology, 9, 1482. https://doi.org/10.3389/fpsyg.2018.01482 Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., … Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151–175. https://doi: 10.1007/s11121-005-5553-y Forgeard, M. J. C., Jayawickreme, E., Kern, M., & Seligman, M. E. P. (2011). Doing the right thing: Measuring wellbeing for public policy. International Journal of Wellbeing, 1(1), 79-106. https://doi:10.5502/ijw.v1I1.15 Fridrici, M., & Lohaus, A. (2009). Stress-prevention in secondary schools: online- versus face-to-face- training. Health Education, 109(4), 299–313. https://doi.org/10.1108/09654280910970884 Glasgow, R. E., Harden, S. M., Gaglio, B., Rabin, B., Smith, M. L., Porter, G. C., … Estabrooks, P. A. (2019). RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review. Frontiers in Public Health,7, 64. https://doi.org/10.3389/fpubh.2019.00064 Greenberg, M. T., & Abenavoli, R. (2017). Universal interventions: Fully exploring their impacts and potential to produce population-level impacts. Journal of Research on Educational Effectiveness, 10(1), 40-67. https://doi.org/10.1080/19345747.2016.1246632 Greenberg, M. T., Kusche, C. A., Cook, E. T., & Quamma, J. P. (1995). Promoting emotional competence in school-aged children The effects of the PATHS curriculum.pdf. Development and Psychopathology, 7, 117–136. Gupta, S. K. (2011). Intention-to-treat concept: A review. Perspectives in Clinical Research, 2(3), 109–112. https://doi.org/10.4103/2229-3485.83221 Hone, L. C., Jarden, A., & Schofield, G. M. (2015). An evaluation of positive psychology intervention effectiveness trials using the re-aim framework: A practice-friendly review. Journal of Positive Psychology, 10(4), 303–322. https://doi.org/10.1080/17439760.2014.965267 Kemper, K. J., Rao, N., Gascon, G., & Mahan, J. D. (2017). Online Training in Mind-Body Therapies: Different Doses, Long-term Outcomes. Journal Of Evidence-Based Complementary & Alternative Medicine, 22(4), 696–702. https://doi.org/10.1177/2156587217701857 Keyes, C., & Annas, J. (2009). Feeling good and functioning well: distinctive concepts in ancient philosophy and contemporary science. The Journal of Positive Psychology, 4(3), 197-201. https://doi:10.1080/17439760902844228 Keyes, C., & Simoes, E. J. (2012). To Flourish or Not: Positive Mental Health and All-Cause Mortality. American Journal of Public Health, 102(11), 2164-2172. https://doi:10.2105/AJPH.2012.300918 Lomas, T. (2015). Positive social psychology: A multilevel inquiry into sociocultural well-being initiatives. Psychology, Public Policy, and Law, 21(3), 338. https://doi.org/10.1037/law0000051 Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., … Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of Clinical Epidemiology, 62(10), e1–e34. https://doi.org/https://doi.org/10.1016/j.jclinepi.2009.06.006 Lonsdale, C., Sanders, T., Cohen, K. E., Parker, P., Noetel, M., Hartwig, T., … Gore, J. M. (2016). Scaling- up an efficacious school-based physical activity intervention: Study protocol for the “Internet-based Professional Learning to help teachers support Activity in Youth” (iPLAY) cluster randomized controlled trial and scale-up implementation evaluat. BMC Public Health, 16(1), 1–16. http://doi.org/10.0.4.162/s12889-016-3243-2 Madden, W., Green, S., & Grant, A. M. (2011). A pilot study evaluating strengths-based coaching for primary school students: Enhancing engagement and hope. International Coaching Psychology Review, 6(1), 71-83. https://doi.org/10.1002/9781119656913.ch16 Manicavasagar, V., Horswood, D., Burckhardt, R., Lum, A., Hadzi-Pavlovic, D., & Parker, G. (2014). Feasibility and effectiveness of a web-based positive psychology program for youth mental health: randomized controlled trial. Journal of medical Internet research, 16(6), e3176. https://doi:10.2196/jmir.3176 http://www.internationaljournalofwellbeing.org/ https://doi.org/10.2196/jmir.3176 Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 64 McConnaughy, E. A., Prochaska, J. O., & Velicer, W. F. (1983). Stages of change in psychotherapy: Measurement and sample profiles. Psychotherapy: Theory, Research & Practice, 20(3), 368-375, https://doi.org/10.1037/h0090198. Meyers, D. C., Durlak, J. A., & Wandersman, A. (2012). The Quality Implementation Framework: A Synthesis of Critical Steps in the Implementation Process. American Journal of Community Psychology, 50(3–4), 462–480. https://doi.org/10.1007/s10464-012-9522-x Moore, L., Silva-Sanigorski, A. de, & Moore, S. N. (2013). A socio-ecological perspective on behavioural interventions to influence food choice in schools: alternative, complementary or synergistic? Public Health Nutrition, 16(6), 1000–1005. https://doi.org/10.1017/S1368980012005605 O’Kearney, R., Gibson, M., Christensen, H., & Griffiths, K. M. (2006). Effects of a Cognitive-Behavioural Internet Program on Depression, Vulnerability to Depression and Stigma in Adolescent Males: A School-Based Controlled Trial. Cognitive Behaviour Therapy, 35(1), 43–54. https://doi.org/10.1080/16506070500303456 O’Kearney, R., Kang, K., Christensen, H., & Griffiths, K. (2009). A controlled trial of a school-based Internet program for reducing depressive symptoms in adolescent girls. Depression And Anxiety, 26(1), 65–72. https://doi.org/10.1002/da.20507 Prochaska, J. O., & DiClemente, C. C. (1982). Transtheoretical therapy: toward a more integrative model of change. Psychotherapy: Theory, Research & Practice, 19(3), 276-288. https://doi.org/10.1037/h0088437. Prochaska, J. O., & DiClemente, C. C. (1986). Toward a Comprehensive Model of Change BT - Treating Addictive Behaviors: Processes of Change. In W. R. Miller & N. Heather (Eds.) (pp. 3–27). Boston, MA: Springer US. https://doi.org/10.1007/978-1-4613-2191-0_1 Prochaska, J. O., & Velicer, W. F. (1997). The Transtheoretical Model of Health Behavior Change. American Journal of Health Promotion, 12(1), 38–48. https://doi.org/10.4278/0890-1171-12.1.38 Proctor, C., Tsukayama, E., Wood, A. M., Maltby, J., Eades, J. F., & Linley, P. A. (2011). Strengths gym: The impact of a character strengths-based intervention on the life satisfaction and well-being of adolescents. Journal of Positive Psychology, 6(5), 377–388. https://doi.org/10.1080/17439760.2011.594079 Proyer, R. T., Gander, F., Wellenzohn, S., & Ruch, W. (2015). Strengths-based PPIs: a randomized placebo-controlled online trial on long-term effects for a signature strengths- vs. a lesser strengths- intervention. Frontiers in Psychology, 6, 456. https://doi.org/10.3389/fpsyg.2015.00456 Puolakanaho, A., Lappalainen, R., Lappalainen, P., Muotka, J. S., Hirvonen, R., Eklund, K. M., … Kiuru, N. (2019). Reducing Stress and Enhancing Academic Buoyancy among Adolescents Using a Brief Web-based Program Based on Acceptance and Commitment Therapy: A Randomized Controlled Trial. Journal of Youth and Adolescence, 48(2), 287–305. https://doi.org/10.1007/s10964-018-0973-8 Ryff, C. D. & Keyes, C.L. (1995). The structure of psychological well-being revisited. Journal Personality and Social Psychology 69 (4): 719-727. https://doi.org/10.1037/0022-3514.69.4.719 Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., … Stewart, L. A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ: British Medical Journal, 349, g7647. https://doi.org/10.1136/bmj.g7647 Slemp, G. R., Chin, T.-C., Kern, M. L., Siokou, C., Loton, D., Oades, L. G., … Waters, L. (2017). Positive Education in Australia: Practice, Measurement, and Future Directions BT - Social and Emotional Learning in Australia and the Asia-Pacific: Perspectives, Programs and Approaches. In E. Frydenberg, A. J. Martin, & R. J. Collie (Eds.) (pp. 101–122). Singapore: Springer Singapore. https://doi.org/10.1007/978-981-10-3394-0_6 Sin, N. L., & Lyubomirsky, S. (2009). Enhancing well-being and alleviating depressive symptoms with PPIs: a practice-friendly meta-analysis. Journal of Clinical Psychology, 65(5), 467–487. http://doi:10.0.3.234/jclp.20593 Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., … Eldridge, S. M. (2019). RoB 2: a revised tool for assessing risk of bias in randomised trials. Bmj, 366. https://doi.org/10.1136/bmj.l4898 http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 65 UChicagoMedicine. (2020, March). Too young to be stressed: Adolescent mental health in the COVID-19 pandemic. https://www.uchicagomedicine.org/forefront/news/too-young-to-be-stressed-out-ingalls- behavioral-health-focuses-on-adolescents-too Unicef. (2020, March). The adolescent brain: Vulnerability and opportunity. https://www.unicef- irc.org/article/1149-the-adolescent-brain-vulnerability-and-opportunity.html Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., … Saul, J. (2008). Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology, 41(3–4), 171–181. https://doi.org/10.1007/s10464-008-9174-z Waters, L., Barsky, A., Ridd, A., & Allen, K. (2015). Contemplative Education: A Systematic, Evidence- Based Review of the effect of Meditation Interventions in Schools. Educational Psychology Review, 27, (1), 103-134. https://doi.org/10.1007/s10648-014-9258-2 Wood, A. M., Froh, J. J., & Geraghty, A. W. A. (2010). Gratitude and well-being: A review and theoretical integration. Clinical Psychology Review, 30(7), 890–905. https://doi.org/10.1016/j.cpr.2010.03.005 World Health Organization (2019). WHO Guideline: Recommendations on Digital Interventions for Health System Strengthening. Geneva: License: CC BY-NC-SA 3.0 IGO. https://apps.who.int/iris/bitstream/handle/10665/311941/9789241550505-eng.pdf?ua=1 http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 66 Appendix A Strings in the search. Ebscohost including ERIC, Medline, Academic Search Complete, Cinahl complete PPI or program or intervention or mindfulness or meditation or gratitude or kindness or positive relationships or pro-social or compassion or empathy or emotion regulation or emotional intelligence or strengths or character strength or social emotional learning or cognitive behavior therapy (title) AND web* or online or computer* (title) AND children or adolescents or youth or child or teenager or student (title) AND school or education or classroom (title) AND wellbeing or well-being or well being or positive education or positive psychology or mental health (All text) AND wellbeing or well-being or well being or positive education or positive psychology or mental health (All text). Limits: English language, peer reviewed, all child. Web of Science PPI or program or intervention or mindfulness or meditation or gratitude or kindness or positive relationships or pro-social or compassion or empathy or emotion regulation or emotional intelligence or strengths or character strength or social emotional learning or cognitive behavior therapy AND web* or online or computer* (All text) AND children or adolescents or youth or child or teenager or student (Title) AND school or education or classroom (All Text) AND wellbeing or well-being or well being or positive education or positive psychology or mental health (All text) AND effectiveness or evaluation or outcomes (All text). Limits: English language, journal article. Ovid including Psych Info. PPI or program or intervention or mindfulness or meditation or gratitude or kindness or positive relationships or pro-social or compassion or empathy or emotion regulation or emotional intelligence or strengths or character strength or social emotional learning or cognitive behavior therapy (title) AND web* or online or computer* (All text) AND children or adolescents or youth or child or teenager or student (Title) AND school or education or classroom (keyword) AND wellbeing or well-being or well being or positive education or positive psychology or mental health (keyword) AND effectiveness or evaluation or outcomes (keyword) AND PPI and online and child and school and WB and effectiveness. Limits: English language and journal articles. Scopus PPI or program or intervention or mindfulness or meditation or gratitude or kindness or positive relationships or pro-social or compassion or empathy or emotion regulation or emotional intelligence or strengths or character strength or social emotional learning or cognitive behavior therapy AND web* or online or computer* (All text) AND children or adolescents or youth or child or teenager or student (Title) AND school or education or classroom (All Text) AND wellbeing or well-being or well being or positive education or positive psychology or mental health" (All text) AND effectiveness or evaluation or outcomes (All Text) AND PPI and online and school and WB and effectiveness plus additional search (pilot and online* and experiment* and quasi-experiment*. Limits: English, Psychology, Social Sciences - included. Excluded -young adult, middle, aged, parenting, child parent relationship, autism, procedures, aged, parents, review, anxiety disorder, parent, clinical, mental disease, mental disorders, aged 80+, burnout, human tissue, university, mother, pathophysiology, theory http://www.internationaljournalofwellbeing.org/ Online, school-based Positive Psychology Interventions Francis, Vella-Brodrick, & Chyuan-Chin www.internationaljournalofwellbeing.org 67 Appendix B Quality assessment of selected studies across domains. Study Domain 1 Domain 2 Domain 3 Domain 4 Domain 5 Overall score Attwood et al., 2012 LR SC HR HR SC HR Bannink et al., 2012 LR SC SC SC LR SC Burckhardt et al., 2015 SC SC SC SC LR SC Calear et al., 2013 LR SC SC HR LR SC Calear et al., 2009 LR SC SC HR LR SC Fridrici & Lohaus, 2009 SC SC SC SC LR SC O’Kearney et al., 2006 SC LR SC SC LR SC O’Kearney et al., 2008 SC SC LR LR LR SC Puolakanaho et al., 2019 LR LR SC SC LR SC Domain 1 risk of bias in the randomization process, Domain 2 risk of bias in deviation from the intended intervention, Domain 3 risk of bias due to missing outcome data, Domain 4 risk of bias in measurement of the outcome, Domain 5 risk of bias in result reporting selection, LR low risk, SC some concerns, HR high risk. http://www.internationaljournalofwellbeing.org/