Collective leadership: A catalyst for school improvement Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 29/4/2022 Facebook: /EPAAA Revisions received: 30/5/2023 Twitter: @epaa_aape Accepted: 29/5/2023 education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 31 Number 80 August 1, 2023 ISSN 1068-2341 Collective Leadership: A Catalyst for School Improvement Jonathan Eckert & Grant Morgan Baylor University United States Citation: Eckert, J., & Morgan, G. (2023). Collective leadership: A catalyst for school improvement. Education Policy Analysis Archives, 31(80). https://doi.org/10.14507/epaa.31.7603 Abstract: Collective leadership of teachers and administrators can be a vehicle for catalyzing school improvement. In chemistry, a catalyst is any substance that inc reases the rate of reaction without itself being consumed. Leadership that accelerates good work without using up the leader is increasingly necessary. We identified schools that demonstrated exemplary Science, Technology, Engineering, and Mathematical (STEM) learning. Using a theoretical model of collective leadership development, we conducted a multiple -case study to identify common themes that provided insight into school improvement. We surveyed STEM leaders (n=113), conducted interviews and focus groups with teachers (n=52) and administrators (n=18), and conducted site visits to five schools. We found seven implications for policymakers: 1) enact policies that support site-based leadership; 2) implement professional learning of teachers and administrators together on work related to shared goals; 3) support peer observation and feedback; 4) provide opportunities for administrators, teachers, and students to design flexible schedules that support cross-curricular STEM connections; 5) develop public/private partnerships that can provide expertise and materials; 6) create opportunities for educators and students to make their work public; 7) engage school -based teams in iterative improvement cycles that rely on collection of observable evidence of improvement using engineering and design principles. http://epaa.asu.edu/ojs/ https://doi.org/10.14507/epaa.31.7603 Education Policy Analysis Archives Vol. 31 No. 80 2 Keywords: school leadership; collective leadership; leadership development; STEM; school improvement Liderazgo colectivo: Un catalizador para la mejora escolar Resumen: El liderazgo colectivo de docentes y administradores puede ser un vehículo para catalizar la mejora escolar. En química, un catalizador es cualquier sustancia que aumenta la velocidad de reacción sin consumirse. Cada vez es más necesario un liderazgo que acelere el buen trabajo sin agotar al líder. Identificamos escuelas que demostraron un aprendizaje ejemplar en Ciencias, Tecnología, Ingeniería y Matemáticas (STEM). Usando un modelo teórico de desarrollo de liderazgo colectivo, llevamos a cabo un estudio de casos múltiples para identificar temas comunes que brindaron información sobre la mejora escolar. Encuestamos a líderes de STEM (n=113), realizamos entrevistas y grupos de enfoque con maestros (n=52) y administradores (n=18), y realizamos visitas a cinco escuelas. Encontramos siete implicaciones para los formuladores de políticas: 1) promulgar políticas que apoyen el liderazgo basado en el sitio; 2) implementar el aprendizaje profesional de maestros y administradores juntos en el trabajo relacionado con objetivos compartidos; 3) apoyar la observación y la retroalimentación de los compañeros; 4) brindar oportunidades para que los administradores, maestros y estudiantes diseñen horarios flexibles que respalden conexiones STEM transversales; 5) desarrollar asociaciones públicas/privadas que pued an proporcionar experiencia y materiales; 6) crear oportunidades para que educadores y estudiantes hagan público su trabajo; 7) involucrar a los equipos escolares en ciclos iterativos de mejora que se basan en la recopilación de evidencia observable de mej ora utilizando principios de ingeniería y diseño. Palabras clave: liderazgo escolar; liderazgo colectivo; desarrollo de liderazgo; STEM; mejoramiento escolar Liderança coletiva: Um catalisador para a melhoria da escola Resumo: A liderança coletiva de professores e administradores pode ser um veículo para catalisar a melhoria da escola. Em química, um catalisador é qualquer substância que aumenta a velocidade da reação sem ser consumida. Uma liderança que acelere o bom trabalho sem esgotar o líder é cada vez mais necessária. Identificamos escolas que demonstraram aprendizado exemplar em Ciências, Tecnologia, Engenharia e Matemática (STEM). Usando um modelo teórico de desenvolvimento de liderança coletiva, realizamos um estudo de caso múltiplo para identificar temas comuns que fornecem informações sobre a melhoria da escola. Pesquisamos líderes STEM (n=113), realizamos entrevistas e grupos focais com professores (n=52) e administradores (n=18) e realizamos visitas locais a cinco escolas. Encontramos sete implicações para os formuladores de políticas: 1) promulgar políticas que apoiem a liderança local; 2) implementar a aprendizagem profissional de professores e gestores juntos no trabalho relacionado a objetivos compartilhados; 3) apoiar a observação e feedback dos pares; 4) fornecer oportunidades para administradores, professores e alunos projetarem horários flexíveis que suportem conexões STEM transcurriculares; 5) desenvolver parcerias público/privadas que possam fornecer expertise e materiais; 6) criar oportunidades para que educadores e alunos tornem seu trabalho público; 7) engajar equipes escolares em ciclos iterativos de melhoria que dependem da coleta de evidências observáveis de melhoria usando princípios de engenharia e design. Palavras-chave: liderança escolar; liderança coletiva; desenvolvimento de liderança; STEM; melhoria escolar Collective leadership 3 Collective Leadership: A Catalyst for School Improvement What if an answer to sustained school improvement resides in the intersection of science, technology, engineering, and mathematics (STEM) practices and leadership work? What if policies could support leadership work and STEM practices in ways that support school improvement? Increasingly policymakers and educational leaders have emphasized STEM as a means of improving educational and economic outcomes (America COMPETES Act, 2007; National Research Council, 2009, 2010, 2012; NGSS Lead States, 2013). Simultaneously, policymakers and educational leaders have encouraged numerous forms of leadership – teacher (Murphy, 2005; Wenner & Campbell, 2017; York-Barr & Duke, 2004), distributed (Spillane et al., 2001; Spillane, 2006), transformational combined with instructional (Day et al., 2016), shared (MacBeath & Dempster, 2009), collaborative (DeWitt, 2017), or collective (Eckert, 2018; Leithwood & Mascall, 2008; Louis et al., 2010). We ground this paper in collective leadership. Collective leadership encompasses the practices through which teachers and administrators influence students, colleagues, policymakers, and others to improve student outcomes (Eckert, 2021). Collective leadership is distinct from other forms of leadership because it is inclusive and draws on the collective expertise of relevant teams of leaders as opposed to delegating tasks or emphasizing positional authority that collaborative, distributed, or transformational leadership emphasize. We describe the practices of collective leadership as catalytic. Drawing from chemistry, a catalyst is any substance that increases the rate of reaction without itself being consumed. Therefore, catalytic leadership does not revolve around an individual leader but accelerates the work of others without leading to burnout. Burnout—an inability to function effectively in a given role—among educational leaders due to cynicism, depersonalization, and emotional exhaustion is prevalent (Doyle Fosco, 2022). Given this reality, collective leadership could build resilience (Day & Gu, 2014) that leads to school improvement. Exemplary STEM learning includes content and context integration in the application of design thinking through the use of science, technology, engineering, and mathematics knowledge (Hansen, 2014; Johnson et al., 2016; Lynch et al., 2015; Peters-Burton, Lynch, et al., 2014; Tofel- Grehl & Callahan, 2014). Across schools where exemplary STEM learning is occurring, there is a sense of independent student learning, inclusion, and an emphasis on research and inquiry (Tofel- Grehl & Callahan, 2014). In another paper, we address the nature of exemplary STEM practices (Eckert, 2021). The STEM emphasis on independence, research, inquiry, and experimentation support and are supported by leadership practices that are predicated on a strong orientation toward improvement, broad ownership of school improvement, and diverse leaders serving in various roles. For any of the innovations in STEM or collective leadership to be scalable, policies must be in place to support improvement—policies that support experimentation, tolerate failure as a possibility predicated on disciplined risk, and support the collective leadership of teachers and administrators (Bryk et al., 2015; Eckert, 2018). To better understand how collective leadership catalyzes school improvement and the policies that support this improvement, we identified elementary, middle, and high schools that were widely recognized for doing outstanding work in STEM. We focused our study in the United States in the state of New York. This paper documents the results of a mixed-methods, multiple-case study of five schools. We are studying how collective leadership involves agency and decision-making of administrators, teachers and students in schools that emphasize STEM. To identify what made the schools distinctive, we used a theoretical model of collective leadership (Eckert, 2018, 2019) to determine the practices teachers and administrators used to catalyze improvement. The research question is: How does collective leadership influence school improvement in exemplary STEM schools? We briefly review the research on collective leadership, STEM, and Education Policy Analysis Archives Vol. 31 No. 80 4 school improvement. We then describe how we collected data. After we share the findings of our cross-case synthesis using collective leadership constructs for analysis, we then describe the ways schools are improving based on educator and student experiences. Finally, we identify implications for policies and practices for sustainable school improvement. Collective Leadership, STEM, and School Improvement Collective leadership emphasizes joint goal setting and strategic implementation of those goals in the service of the mission at the core of schools—teaching and learning. Collective leadership has been found to have a greater influence on student outcomes than individual leadership (Seashore-Louis et al., 2010). Collective leadership is increasingly important because collective teacher efficacy, a subset of this type of leadership, is the most influential factor influencing student learning (Hattie, 2012, 2015). We (Eckert, 2018) based our original theoretical framework for collective leadership development on work redesign literature (e.g., Campion et al., 2005; Hackman & Oldham, 1980), leadership development across organizational type and sector (e.g., Avolio, 2010; Conger, 1992; Day et al., 2004; Van Velsor et al., 2010; Yukl, 2013) and teacher leadership (e.g., Berg, 2018; Lieberman & Miller, 2004; Mangin & Stoelinga, 2008; Murphy, 2005; Wenner & Campbell, 2017; York-Barr & Duke, 2004). This framework describes leadership as a set of functions (Firestone, 1996; Heller & Firestone, 1995; Mayrowetz & Weistein, 1999) or specifically “change leadership functions” (Heller & Firestone, 1995, p. 67). This framing is particularly important in an era when external forces influence the way educators see themselves as professionals (Anderson & Cohen, 2015; Lima, 2021). The theory frames collective leadership as a blending of the work of teachers and administrators do to identify and advance shared goals that will benefit students. The seven conditions are: 1) A shared vision and strategy that is supported by dynamic, strategically “ambidextrous” leadership development (Fullan, 2005; Hargreaves & Fink, 2006; McCauley, 2008; Mumford et al., 2007; Smylie, 2010) that is democratic, participative inclusive leadership (Bryk et al., 2010; Leithwood & Louis, 2012; Light, 1998); 2) Supportive administration that believes in this kind of leadership (Birky et al., 2006; Camburn et al., 2008; Murphy et al., 2009; Smylie & Brownlee- Conyers, 1992; Van Velsor et al., 2010; Weiner & Lamb, 2020) that recalibrates authority relationships (Murphy, 2005) and works toward a democratic school climate (Ascorra et al., 2019); 3) Resources and initial leadership capacity that provide the necessary talent, time, and space to lead together (Avolio & Hannah, 2008; Bond et al., 2008; Conger, 1992; Martineau, 2004); 4) Adequate work design to facilitate collaboration, improvement, and the spread of teaching expertise (Campion et al., 2005; Hackman & Oldham, 1980; Margolis, 2012; Smylie & Denny, 1990); 5) Supportive social norms and working relationships that foster relational trust (Bryk & Schneider, 2002; Hargreaves & O’Connor, 2018; Muijs & Harris, 2007; Tschannen-Moran, 2004; Van Velsor & McCauley, 2004); 6) Shared influence of administrators and teachers on themselves and others (Day et al., 2004; Lai & Cheung, 2015; Muijs & Harris, 2007); and 7) Orientation toward improvement that facilitates growth (Eckert, 2018, 2019). These conditions can either support or constrain professional growth and student learning. Of the seven constructs (see Figure 1), the first four—shared vision and strategy, supportive administration, resources, and work design—are antecedent to the other three—supportive social norms and working relationships, shared influence, and orientation toward improvement—as the green arrow between boxes indicates. However, we also acknowledge the systemic interactive relationships among all seven constructs (Bass, 1990; Firestone, 1996; Smylie, 2010; Yukl, 2013). The gray connecting lines with nodes indicate the influence that all seven constructs have on leadership development experiences. Those development experiences are iterative processes within the larger Collective leadership 5 model as indicated by the circular green arrows. The development experiences influence both the collective leadership capacity of teachers and administrators as well as their leadership practice. The dotted gray lines that intersect the line that flows out of development experiences indicate that the seven constructs can continue to support or constrain leadership efforts. There is a direct arrow from collective leadership capacity to collective leadership practice because as capacity increases, practice improves. The direct arrow from collective leadership practice to student outcomes demonstrates the influence of leadership practice on student outcomes. The gray line and nodes that flow from student outcomes back to collective leadership practice and the seven constructs indicate that those outcomes feedback into leadership practice—a reciprocal relationship which the green arrow indicates—and influence the seven constructs as the process is iterative rather than linear. Figure 1 Theoretical Model for Collective Leadership Development These constructs and the model itself are described in more detail in previous work (Eckert, 2018, 2019, 2021). In those studies, we have found that schools with higher levels of collective leadership tend to have greater degrees of collective teacher efficacy which has been found to have significant influence on student outcomes (Hattie, 2017). Additionally, schools whose ratings on the seven collective leadership constructs are high, seem to facilitate the spread of expertise between teachers and administrators, teachers and teachers, and administrators and teachers. The opportunity to work together and experience professional learning simultaneously appears to increase leadership capacity and prepare teachers for more influential leadership (Eckert, 2018, 2019, 2021). We know that expansion of leadership pipelines (Gates et al., 2019), the spread of teaching expertise (Papay et al., 2020), leadership sustainability (Hargreaves & Fink, 2003), and improved teacher recruitment and retention (Carver et al., 2013; Carver-Thomas & Darling-Hammond, 2019) can also have significant effects on school improvement and could be enhanced by collective leadership (Louis et al., 2010). To study these seven conditions, we have developed a survey for teachers and administrators that effectively measures collective leadership conditions across contexts (Eckert, 2022) and as a school- level factor (Eckert, in press). Education Policy Analysis Archives Vol. 31 No. 80 6 When coupled with collective leadership practices and policies, exemplary STEM practices can catalyze sustainable improvement. Exemplary STEM learning is integrated across disciplines; inclusive; uses design process; requires planning, implementing, testing, and evaluating solutions; includes ethical considerations, effective communication, and teamwork. Collective leadership is an inclusive approach to leadership that can effectively support the development of content expertise, inform the flexible use of learning spaces, and leverage to eh collective expertise of educators in ways that support iterative ideation and improvement. First, developing and maintaining a STEM identity as a school requires a shared vision built on a culture of intellectualism, inclusion, and collaboration (Tofel-Grehl & Callahan, 2014), and this can vary significantly based on instruction, collaboration (Bassok et al., 2016; Diamond & Spillane, 2004; Marx & Harris, 2006), and planning time limitations (Goodpaster et al., 2012; Herro & Quigley, 2017; Lesseig et al., 2016; Margot & Kettler, 2019). These differences extend to pedagogical content knowledge (Shulman, 1986; Zahorik, 1996) and resources (Hsu et al., 2011; Litke & Hill, 2020; Park et al., 2017) as school leaders consider professional learning around inquiry-based teaching as well as materials and physical space considerations. Second, course offerings and student research opportunities, often tied to funding, limit exemplary STEM learning and vary by school level (Peters-Burton, Lynch, et al., 2014; Pfeiffer et al., 2010; Tofel-Grehl & Callahan, 2014). Third, an orientation toward improvement is part of the engineering process predicated on building, testing, and re-designing prototypes. Fourth, the integration of STEM disciplines across all subject areas requires shared influence among teachers with different expertise as well as flexible work design (Johnson et al. 2016; Peters-Burton, Kaminsky, et al. 2014). Collective leadership and STEM principles align well with much of the expanding improvement science work in education. Increasing research-practice partnerships and networked improvement communities (NICs) connect science and engineering principles to leadership efforts in the service of continuous school improvement. Research-practice partnerships are a model for organizing the improvement of practitioners and researchers around common problems of practice (Coburn & Penuel, 2016). Similar to research-practice partnerships, NICs (Bryk et al., 2015) operate on similar principles that support continuous improvement through iterative processes of design, implementation, and evaluation contingent upon school contexts in ways that promote empirical research and insights from practice. Additionally, design thinking increasingly informs schools and improvement communities through its emphasis on empathy and perspective-taking for deeper understanding (Nash, 2019). While continuous improvement in schools is certainly not a new concept (Murphy, 2005; Smylie, 2010), the emphasis on design thinking and tools of improvement science offer connecting points between leadership, STEM, and school improvement in meaningful ways. Researchers are identifying “cultures of improvement” (e.g., Dolle et al., 2013; Harrison et al., 2019) that permeate learning organizations that both tolerate and encourage reflective risk-taking (Eckert, 2016). While this study examines five individual schools, identifying ways to build networks of schools working on common problems of practice could catalyze the spread of collectively-led improvement by exponentially increasing the amount of data collected. Methods We identified an initial list of over 30 schools across New York after consulting with Regents, the State Department of Education, State Teachers of the Year, and reviewing state STEM achievement data (e.g., graduation rates, math and science testing data in Grades 3-8 and Regents exams). To narrow the list of possible schools, we identified schools with above average state achievement data in mathematics and science. Because we could not use student achievement scores alone given the fact that they only address science and mathematics, we conducted videoconferences Collective leadership 7 with school leaders of the recommended schools to determine the quality of their STEM learning outcomes. We also ensured the inclusion of one elementary, middle, and high school, each from different parts of the state. Additionally, we identified one district with an elementary, middle, and high school to see STEM learning develop P-12 (see Table 1). We selected STEM High School1 (SHS), New York Elementary School (NYES), and three schools from the same New York district (see Table 1). The district is a STEM Learning Ecosystem, a national initiative to improve STEM (Wang et al., 2011). The three schools in the district were Central Elementary School (CES), Central Middle School (CMS), and Central High School (CHS). Other than SHS which uses a selective admissions process, the other four schools are traditional district schools that do not have admissions requirements. Additionally, four of the five schools are classified as mid-low to mid-high poverty schools making them similar to approximately 55% of schools in the US (NCES, 2022). Table 1 School Demographics2 Total enrollment Economically disadvantaged White Asian / Pacific Islander Black Hispanic Multiracial Native American NYES 318 7% 83% 4% 1% 10% 3% CHS 1033 41% 88% 2% 5% 2% 2% 1% CES 305 72% 82% 2% 10% 4% 3% CMS 729 46% 87% 2% 4% 3% 3% 1% SHS 1313 39% 48% 47% 1% 4% Over the course of two years, we collected data in all five schools. To address construct validity, we collected multiple primary and supportive sources of evidence for triangulation (Patton, 2002) in a mixed-methods multiple-case study design (Creswell & Clark, 2018; Yin, 2014) based on humble, appreciative inquiry (Schein, 2013; Stavros et al., 2016). The purpose of this type of inquiry is to identify strengths, possibilities, and success while also identifying barriers. Primary evidence included a collective leadership survey (see Appendix A), interviews, focus groups (see Appendix B), and classroom observations. For participating in interviews or focus groups, teachers received $50 gift cards. Supportive evidence included document analyses of school STEM surveys, school climate surveys, student data, program planning documents, committee work products, student achievement data, and school improvement plans. Additionally, administrators and teachers at each school reviewed drafts of their school’s case study and provided feedback. Once we collected the data, we used cross-case synthesis (Yin, 2014) to identify similarities, differences, patterns, and themes across five stages (see Figure 2). First, we analyzed state-level math and science achievement data as well as data from the Collective Leadership Survey (n=113) of teachers and administrators connected to STEM instruction. We identified areas of relative strength in science and math test scores and then identified teachers and instructional coaches at particular grade levels or classes who appeared to be primarily responsible for above average levels of performance to include in interviews, focus groups, and classroom visits. 1 Pseudonyms used for schools and participants 2 All demographic and achievement data retrieved from https://data.nysed.gov/profile.php?instid=800000041734 https://data.nysed.gov/profile.php?instid=800000041734 Education Policy Analysis Archives Vol. 31 No. 80 8 Figure 2 Mixed-Methods Data Collection and Analysis Plan Second, after determining that all five schools exceeded average thresholds for math and science achievement, we aggregated the Collective Leadership Survey data based on the seven conditions. When analyzed by condition, the survey item responses (see Table 2) were very consistent within condition (=.732 to =.926), which provides strong reliability evidence for the instrument’s use. Confirmatory factor analysis (CFA) was used to examine the extent to which the instrument’s factor structure (i.e., conditions) aligned with expectation. The CFA model was estimated using diagonally-weighted least squares, which is most appropriate for Likert-type response data (DiStefano & Morgan, 2014), and the factors were allowed to be correlated. CFA results suggest that there was adequate to good model-data fit (𝜒473 2 = 3541.2, p < .001, CFI = .95, SRMR = 0.10, RMSEALower = 0.10, RMSEAUpper = 0.11), which supports the use of the instrument as designed. Collective leadership 9 Table 2 Cronbach’s Alphas for Survey Items by Condition Condition Number of items Α Shared vision and strategy 1 Supportive administration 4 .926 Resources 4 .912 Work design 5 .808 Supportive social norms and working relationships 4 .819 Shared influence 4 .732 School orientation toward improvement 4 .888 Initial outcomes 4 .753 Third, the survey results informed the semi-structured protocols for responsive interviewing (Rubin & Rubin, 2005) with teachers (n=52) and administrators (n=18). Administrators included superintendents (n=2), central office administrators (n=6), principals (n=5), and assistant principals (n=5). Teacher leaders voluntarily signed up for individual or focus group time slots during planning periods, before, or after school. Interviews and focus groups were 30 to 90 minutes in length. Through site visits to all five schools, we conducted 32 classroom observations that ranged from 7 to 55 minutes and observed three leadership meetings for 15 to 60 minutes each, two staff meetings for 30 minutes each, and five professional learning community sessions of approximately 30 minutes each. During the classroom visits, we were able to speak informally speak to students and captured their quotations in field notes. Fourth, we transcribed, organized, and coded interviews and field notes using Dedoose. We coded the data into categories that comprised the theoretical model of collective leadership development. We used the seven conditions for collective leadership for initial provisional coding (Saldana, 2016). Researchers blind-coded the data concurrently. To ensure inter-rater reliability through testing and regular collaboration to reconcile discrepancies, the researchers’ pooled Cohen’s kappa was .86. This pooled kappa value indicates excellent agreement even by the most conservative estimates (Cicchetti, 1994; Fleiss, 1971; Landis & Koch, 1977; Miles & Huberman, 1994). When there were discrepancies, we used memos to reconcile coding differences across school sites and a second round of coding. Through memo writing, we were able to better understand model conditions as they manifested themselves in different contexts. Fifth, we wrote memos and created tables that compared the five schools’ STEM and leadership practices across all seven collective leadership conditions. We were then able to determine patterns across all five schools by examining convergence, divergence, and contradiction. Using the theoretical model to facilitate analysis, we employed cross-case synthesis (Yin, 2014). By matching observed events to the theoretical model and then comparing them across cases we were able to organize findings and check for disconfirming evidence. The complexity of the theoretical model required a robust analysis of all data as the model itself is comprehensive. Moreover, document analysis including school-level data on climate and culture triangulated similarities and differences across schools. By using the quantitative results of the survey to inform our interviews and focus groups, we were able to identify key leadership themes through effective school practices. We organized the findings first by collective leadership by school and then by condition around themes that emerged across all five schools. Education Policy Analysis Archives Vol. 31 No. 80 10 Findings We organized our findings to elucidate the connection between collective leadership, STEM, and school improvement. In the first section, we describe Our findings are descriptive and therefore, we are not attempting to infer causality. Collective leadership, STEM, and school improvement We will briefly examine how collective leadership practices have influenced improvement at each school based primarily on enhanced student with connections between STEM and collective leadership. We conclude each school section with supportive school and district policies. New York Elementary School Educators attributed changes in work design and supportive working relationships, co- planning, co-teaching, and collectively looking at student work spread to instructional improvement. Administrators and teachers reported that they were more open to feedback from others and are willing to take instructional risks. Teacher expertise was honored and elevated by administrators who see tangible differences in the way students learn. The attention to social-emotional learning, team building, critical thinking, and empathy changed the way students experience school. Both teachers and administrators reported this and highlighted how their colleagues supported their work. These norms and attitudes were evident in staff meetings, PLCs, and in classroom visits as teachers and students worked together toward shared goals. Teachers used STEM lessons for social-emotional learning. Students were frequently engaged as teams in problem-solving requiring them to take on different perspectives. Teachers cited reduced classroom management issues with students demonstrating high levels of engagement. Moreover, teachers emphasized student collaboration and empathy in observed lessons. These benefits accrued to both advanced students and struggling students. One teacher said, We also find that the gifted and talented students are the ones who are the most nervous to fail forward. They don’t want to take that risk. They’re afraid of being wrong. They don’t want anyone to see that they’re making a mistake if it doesn’t work out that way. For students who struggled with reading, these STEM experiences provided outstanding opportunities to stretch academically. One teacher shared, “Our students say, ‘Here is a challenge. Let me acquire the materials and resources I need. Let me work with my team collaboratively, and then let me solve this problem in a way that’s also connected to the real world.’” As evidenced by this representative quotation, students had entered in to shared problem solving. The similarities between classrooms, PLCs, and other team meetings demonstrated that the shared influence between teachers and administrators extended to students influencing each other. Veteran teachers and administrators expressed excitement about the growth they were seeing in their school as a learning community. In interviews, they frequently cited the fact that this type of shared influence “had not always existed” and that their focus on STEM and collaboration had re- oriented them toward ongoing improvement. School and district policies supported peer observation and feedback as well as coaching support from a STEM specialist. Teachers had adequate time to co-plan and observe classroom and classroom lessons and school leaders had the autonomy to allocate classrooms for co-teaching and design spaces. The STEM specialist was a district catalyst for improvement because she was in a position of influence due to district resources and policies. District policies and positions that support collective leadership and the spread of STEM expertise were essential for innovation. Collective leadership 11 Central Elementary School CES identified better outcomes for students due to a shared vision and strategy of tracking their progress as a central focus of collective leadership and their enhanced orientation toward improvement. A kindergarten teacher described what her students were saying, “‘We have to work as a team. It is okay that that thing didn’t work. We are just going to try again.’ Just that attitude shift is what I see being the true outcome of STEM.” She added that she noticed a change in the “perseverance level” of students that reflected improvement in the collective leadership of teachers and administrators. According to her, students had learned that “the only true failure is when you quit.” During an observation, another class was working on storage devices for the international space station to build on the 3-D printer, and a fifth grader said, “I’ve always loved engineering. You get to use your creativity for what does and does not work.” By tapping into students’ passion for STEM and combining that with the strong orientation toward improvement that collective leadership requires, the culture of the school was changing. With an increasing population of students coming from homes with limited financial resources, STEM has been extremely important. One teacher shared, “I think that for kids who may lack resources at home, this opens up a whole world of possibility to them, and that is something that we talk about at the school’s college and career day.” Another teacher added, “I hear more and more parents say to me, especially at the fifth-grade level, that their child is now talking that they want to be engineers, and I didn’t hear that a few years ago.” Teachers and administrators attribute the fact that students are beginning to see different possibilities for themselves to their shared work of making opportunities more accessible to students through their own collaborations and community partners. By connecting to local industry and bringing experts into the classroom, teachers are fundamentally changing the way they are approaching their teaching. Teaching has become much more about what students do and connecting them to resources than what the teachers say or do. Teachers see this reflected in the support they receive from their administrators who have become “the biggest supporters of STEM and our growth.” District policies were particularly important for Central School District (CSD) because they created a pipeline for STEM and collective leadership development. The long-time superintendent created district-community partnerships to enhance STEM, thereby expanding collective leadership to the community. Additionally, CSD had achieved national status as a STEM Ecosystem due to their emphasis on STEM which cemented their identity as a district that invited others in to learn from and with them each summer through shared professional learning. Central Middle School Collective leadership for school improvement manifested itself in the changes in the physical structure of the building and the way students and teachers altered work and work design through shared influence and supportive social norms. To illustrate, during an interview with the principal, two eighth grade students entered his office. They had put together a proposal for an aquarium for the school lobby. They gave a 3-minute pitch that included research on the stress reduction associated with an aquarium, cost, and a care plan for the aquarium. They were confident, polished, and succinct. A year later, there was an aquarium in the lobby. Collective leadership that included the leadership of students brought about this change, and the change was about tangible improvement. This culture of improvement has resulted in improved attendance according to teachers. One teacher shared, “Remember year one of the energy project? We had 100% attendance the first time [for all the students on the team]. We had a student that up to this point was a major attendance issue, but she would come to school during the project. She would come when she knew Education Policy Analysis Archives Vol. 31 No. 80 12 we were working on the project.” Students were coming to school because they were engaged. An eighth science teacher recalled one of the first years of a Mars rover project. The bell rang … They all just kept working… after a minute or so, a student asked, “Do we go home now?” And that engagement of not looking at the clock and not thinking about going home or going to sports but to be thinking about the task I’m trying to solve—that’s a big deal. Learning outcomes extended beyond what students did in STEM classes. In a focus group, a teacher described a time she needed leadership for a club, so she asked for five students to take the lead. “Within two hours, I think I had 13 kids who came. Now they run the class, and every day they have ideas. They are consistently asking questions such as, ‘How about we try this? What could we do?’” Supportive teachers and administrators expanded leadership to include students. Administrators not only ask for input on decisions, they seek it out, and expect teachers and students to have answers that will improve culture and policies. From designing school buildings to determining bell schedules, administrators at CMS depend on teachers and students. Recently, CMS was completely redesigned based on student and teacher needs. Administration also implemented policies that allowed for flexible scheduling without bells during the day. Policies did not just support teachers and students. Teachers and students designed the policies. Central High School Catalytic partnerships with corporations and higher education institutions, community projects, and recognition that CHS and the district have received as a STEM Ecosystem were indicators of their success through shared vision and strategy, shared influence, and a strong orientation toward improvement. Students presented their research to groups during learning tours and at an annual university Environmental Summit. However, the outcomes that teachers and administrators focused on were more student- specific. A technology teacher described the collective leadership at CHS: I was able to visit when they built robots where they had to carry an item. The next year, they had to carry an item and keep it in a straight line. You see that a lot. Kids are being challenged to move further and further ahead. I think there is a lot of teacher leadership in the high school. When teachers feel like they have the freedom to take risks, they tend to do so. You will see teachers and students learning together. The collective leadership of students, teachers, and community partners expanded CHS’s impact through enacted policies. The clean water reclamation project was just one example of this. Students who were passionate about finding ways to turn sewage into clean water used village landscaping waste to heat a greenhouse where they found environmentally friendly solutions alongside research biologists and teachers. Students’ perspectives on the world and their self-efficacy changed. The teacher responsible for the class explained, In developing real-world research projects, students often must deal with failure— things go wrong. Pumps break, things leak, plants die, etc. This might be the second most important things students learn. You can’t quit and have to keep solving problems that arise. According to the teacher, students were “the drivers of their own learning.” This was one illustrative example of the way many community business partnerships built a strong orientation toward improvement that extended beyond the school walls. Policies developed by the superintendent resulted in flourishing community partnerships. Collective leadership 13 STEM High School Ranked as one of the best high schools in the US (U.S. News and World Report, 2020), SHS was building on the strengths of an academically talented student body and an accomplished faculty due to significant resources and capacity. From the perspective of resources and capacity, these strengths were catalysts for remarkable outcomes. Of their students, 98% graduated with an Advanced Regents Diploma, and 99% went on to four-year colleges. A Tech Guild, Backpacks to Briefcases, Underwater Robotics, Student Run Enterprise, and a region-wide Hackathon were just some of the remarkable STEM opportunities available to SHS students due to the collective leadership and expertise of administrators, teachers, and students. These representative programs demonstrate the support that administrators provide for teachers, who in turn, provide a range of opportunities to students based on their interests. Administrators at SHS are willing to share power in ways that support innovative practices by catalyzing good work that is led by teachers and students. Students valued the flexible work design and SHS’s culture of improvement for three reasons. They believed the culture 1) created an environment that gave students choices; 2) provided students with opportunities to create hands-on projects; 3) translated what was taught to their own interests outside of class time. During a class observation, one student described the open environment his engineering teacher created. “Mr. Hernandez gives you the basics and then he makes you get to the endpoint on your own flexible deadline.” Teachers were catalysts for learning outside of the classroom—an expectation at SHS according to students. While observing another class, a student explained, “It’s half about the teacher introducing supplementary materials. The other half is found on my own.” Another student described how this works in her Forensics class. “Mrs. Gordon assigns group projects but has us do our own research in our own time to build our own crime scene. She also gives us multiple opportunities to rework our project and add to our knowledge.” Teachers attribute the success of their students to the trust between teachers and administrators that has developed a strong orientation toward improvement. As the “story-teller-in- chief,” the principal has set the tone of being a supporter of innovative work. By implementing a policy that allows teachers and administrators to film and share students working in classrooms, they are making their improvement public. Other district policies also had significant impact on the way SHS enacted leadership and approached improvement. The ability to partner with community organizations and businesses for internships increased the collective expertise available to students. Additionally, the ability to offer different types of classes and extra-curricular activities based on faculty expertise and not necessarily certification (e.g., Calculus III, Forensics, and Solar Car Design) greatly enhanced the teaching capacity and opportunity for students to learn. Collective Leaders as Catalysts Across all five schools, we found evidence of catalytic leaders. Catalytic leaders accelerated the improvement of others in sustainable ways. This type of leadership manifested itself in exemplary STEM learning that energized educators and students in ways that seemed less likely to lead to burnout. The five schools in the study demonstrated educator retention rates of over 90 percent and high levels of job satisfaction. We have administered the same collective leadership survey to over 10,000 educators—almost none of whom are in STEM schools—and the rates of collective leadership in these five schools is relatively high. We define exemplary STEM learning as a multi-disciplinary integrated approach that addresses global and local challenges through the development of critical thinking, problem-solving, teamwork, communication, and empathy. All five Education Policy Analysis Archives Vol. 31 No. 80 14 schools displayed high levels of collective leadership across all seven conditions and probes on their initial outcomes. The results (see Table 3) indicate that respondents were most positive about the supportive norms and working relationships at their schools (M = 4.56 on a 5-point scale) and reported strong orientations toward improvement (M = 4.06 on a 5-point scale). Teachers and administrators at all schools reported high levels of efficacy (M = 4.07 on a 5-point scale) as measured by probes related to initial outcomes (e.g., “When we try really hard as teachers and administrators, even the most unmotivated students learn.”). Even the lowest average rating (M = 3.30 on a 5-point scale) for enabling work structures was relatively positive. Table 3 Mean survey responses by condition (5-point scale: “Strongly Agree” to “Strongly Disagree”) and school Condition NYES (n=7) CES (n=23) CMS (n=38) CHS (n=26) SHS (n=19) Total (n=113) M SD M SD M SD M SD M SD M SD Shared vision and strategy 4.00 .82 4.13 1.30 3.53 1.01 3.54 1.30 3.89 1.33 3.74 1.16 Supportive administration 4.43* 0.47 4.61* 0.52 3.38 0.89 3.75 0.92 4.51* 0.57 3.97 0.92 Resources 3.74 0.84 4.03 0.72 3.52 0.77 3.72 0.78 3.87 0.73 3.74 0.77 Work design 3.32 0.79 3.59* 0.88 2.90 0.79 3.20 0.82 3.88* 0.85 3.30 0.89 Supportive social norms and working relationships 4.86 0.38 4.62 0.43 4.45 0.50 4.55 0.45 4.61 0.39 4.56 0.46 Shared influence 3.46 0.37 3.36 0.72 2.85 0.93 3.08 0.84 3.17 1.00 3.10 0.87 School orientation toward improvement 4.57* 0.45 4.42* 0.53 3.76 0.57 3.89 0.78 4.24* 0.39 4.06 0.65 Initial outcomes 4.56* 0.38 4.37* 0.49 3.76 0.56 3.97 0.56 4.28* 0.63 4.07 0.61 *significantly greater (<.05) than other schools based on one-way ANOVA. Shared Vision and Catalytic Administrators The survey data demonstrated high levels of sustainable collective leadership. All five schools demonstrated “STEM cultures” through the first two conditions necessary for collective leadership: shared vision and strategy and supportive administration. Particularly at NYES, CES, and SHS where supportive administration was significantly higher than the average response across all schools, administrators were catalysts for collective leadership. Administrators and teachers prioritized an engineering mindset that identified problems, sought alternative solutions, prototyped, collected evidence, and re-designed and in so doing, catalyzed improvement. They adhered to the improvement science principle that what they were doing was “possibly wrong and definitely incomplete” (Bryk et al., 2015, p. 79). The work they were doing was not about the leader, but was about catalyzing the good work of others. In doing this, they led their schools in ways that did not Collective leadership 15 contribute to burnout because they were developing strengths-based leadership in others. All five schools and particularly the three that were in the same district, benefitted from districts where schools had the autonomy to implement their own visions. Central office administrators did not micromanage processes; instead the central office administrators supported policies that allowed teachers and administrators to work together toward shared goals. Additionally, New York state policy allowed for significant school autonomy within district purview. In addition to strong building leadership, district leaders were supportive of building efforts and provided tangible resources. These STEM cultures permeated far more than STEM classes and provided opportunities for more expansive visions of collective leadership and the notion that leadership work was performed by teachers and administrators together. In some cases, administrators catalyzed the STEM culture, and in other cases, teachers led the way. Administrators catalyzed the culture by allowing teachers with significant expertise to lead. For example, a STEM specialist catalyzed NYES’s vision and strategy. Teachers and administrators described her as a “force of nature” and as a “beacon” by the NYES principal. Over the past decade, by co-teaching STEM labs and developing makerspaces, she catalyzed the transformation of elementary STEM learning. Although they sometimes faced pressure from state and federal policymakers, the principal at CES and the district superintendent both have had long careers in which they focused on student outcomes that move beyond test scores and standards. The shared vision and administrative support extended to students. A kindergarten teacher at CES described how the elementary culture built a trajectory for the entire district. “My kindergarten students can tell you what the engineering process is…. It starts at a young age so that we can build on it so that when they get to high school, it’s not this new concept.” The SHS principal was a catalyst for improvement in that he highlighted the good work teachers and students were doing: I replaced every wall in this school with glass metaphorically. I am the Storyteller-in- Chief… It is my job to document everything that is going on here. All teachers and students have signed off to let me take video and pictures. If you want to know what is going on, just follow my Twitter feed. This support of the work occurring in classrooms and beyond catalyzed and identified improvement. Additionally, supportive administrators developed policies that enhanced collaboration, gave permission to publicly share teacher and student work, and created clear avenues for communication between teachers and administrators. Resources and Work Design Through partnerships with local pharmaceutical and chemical companies and the use of local funds from foundations and municipal funds, the schools have developed substantial resources for STEM integration. District policies that support these types of partnerships are necessary. Of the five schools, CMS had the best facilities overall due to a recent renovation, but all schools were relatively positive about the resources they had available based on their survey responses. However, much of the reason for these outstanding facilities that included many collaborative spaces, well- designed labs, and makerspaces, was due to how teams of teachers and administrators designed the facilities together. During and renovation that caused teachers and administrators to disperse across the district, they met regularly to discuss plans for how to allocate resources and what design priorities should be. The design of teachers’ work was also flexible with no bells and the ability to combine classes from 24-96 students with flexibly designed team wings. Education Policy Analysis Archives Vol. 31 No. 80 16 SHS, which reported significantly higher levels of effectively designed work on the survey, provided clear examples of flexibility in the design of teachers and students work as well as high levels of human capital. Most SHS students participated in internships across New York City with the support of an internship coordinator. SHS is also home to a state-of-the-art broadcast studio, a forensics teacher who served as a forensic DNA analyst during the 9/11 terrorist attack, and other teachers who are at the top of their respective fields. In a school that has over 30 students taking Calculus III, this type of high-level expertise and exceptional teaching capacity were essential. The elementary schools were creatively addressing resources and work design where there were more constraints and less obvious community partnerships. Some of the elementary teachers cited a lack of content knowledge in STEM fields as an impediment to improvement. Others struggled to find common planning time with some at NYES resorting to planning instruction on Sunday afternoons and evenings when other teachers were available. While the teachers who cited these challenges through the survey and through the interviews did not connect these conditions to burnout, they did cite them as possible contributors to future burnout as they said that evening and weekend work might not be sustainable over time. Through external grants, business partnerships with local businesses and municipal agencies, teachers and administrators secured additional resources. For example, the district STEM specialist who supported NYES was a national expert who tested new makerspace products for companies and provided feedback in exchange for materials. She co-facilitated makerspaces, learning environments where students create with high and low-tech tools, with grade-level teachers with materials supplied by grants, the parent-teacher association, and materials she was testing for companies. All the NYES teachers interviewed highlighted the importance of designing and implementing these experiences collaboratively and having a designated room for the makerspace. The previous year, no room was available for the makerspace, which made lessons significantly more challenging to implement. At CES, extensive human capital supported exemplary STEM instruction. Three district technology specialists supported teacher-developed projects in addition to support from the library media specialist and an “Inquiry Support Teacher for Math and Science.” The inquiry support teacher ran an innovation lab that is open to all students and saw classes throughout the day. Additionally, she left the lab open during student lunch periods for them to sign up for additional time. School and district policies supported her flexibility to allow students to make better use of the available resources. Relationships, Influence, and Orientation toward Improvement Supportive relationships, shared influence, and leaders’ orientation toward improvement are interrelated in all five schools. Of the seven collective leadership conditions, “supportive norms and working conditions” was the highest rated across the five schools. Where working relationships are strong, teachers and administrators can spread their expertise in influential ways and develop a strong orientation toward improvement that moves beyond tolerance of risk to an expectation of reflective risk-taking. New York Elementary School had a significantly higher rating for “school orientation toward improvement” on the survey when compared to the average score from the five schools. As teams of teachers at NYES developed STEM units and makerspace experiences, they developed and improved working relationships. They collaboratively wrote curriculum and provided professional learning to other teachers. A third-grade teacher described the iterative nature of their work that captures relationships, shared influence, and a strong orientation toward improvement. “Teachers are learning from teachers. It has a little bit more of a personal connection because no one’s really nervous that they’re going to make a mistake or that it’s something that we know, it’s constantly evolving.” Administrators who develop policies that develop collaborative practices and peer Collective leadership 17 observation are key to building cultures where these practices lead to improvement. At all five schools, administrators were implementing policies and developing cultures that catalyzed teacher and student growth. The shared influence and relational trust between teachers were evident in the way CMS teachers discussed peer observation. Teachers were very positive about these opportunities to observe others. They “love peer observations,” and they love “stealing people’s ideas and reworking” them so that they “do not have to reinvent the wheel.” Students reflected their schools’ orientations toward improvement. At one of the site visit observations, one CHS student summed up what she liked about her course at a clean water reclamation project. “You are not working alone. Your team can present ideas to the teachers. When things fail, we go from there…. Every week you have a new problem to solve.” In all five schools, collective leadership was evident in the work of teachers and administrators who had a clear vision, supported each other, and had structured their work to best facilitate student learning. The engineering approach to problem solving permeated student, teacher, and administrative leadership in these schools. STEM culture included a tolerance of risk that was necessary for the engineering process that allowed leaders to “fail fast and fail forward” while prototyping potential improvements. School and district policies that allow disciplined risk-taking were necessary for innovation to occur. Discussion We acknowledge several limitations to this mixed-methods multi-case synthesis. First, as is the case with any case study research, we limited the study to a few sites. Moreover, we selected these schools because they were already demonstrating evidence of exemplary STEM learning which means they are not representative of what is occurring in all schools. Second, the schools selected are not particularly racially/ethnically diverse (see Table 1). We based our selection criteria to schools with strong STEM reputations and evidence as that was the focus of the study. While this creates challenges of determining causality for conditions that promote improvement, the descriptive nature of the collective leadership and STEM practices can provide insights into how other school leaders might catalyze growth in their own schools. Third, this study is not longitudinal, and we can only draw correlational links between exemplary STEM practices and distal student outcomes. Additional research on the long-term benefits of exemplary STEM teaching and leadership with educational, economic, and social outcomes would be valuable. Combining exemplary STEM practices with supportive leadership conditions is an optimal way to catalyze school improvement in policy and practice. As administrators, teachers, and students apply STEM principles in schools that cultivate collective leadership, they can catalyze school improvement across school contexts including those with traditionally underserved students who are underrepresented in STEM disciplines. Each of the five points that follow are evidence-based practices found in contexts where students are more likely to be successful. First, this study’s findings are consistent with the literature on exemplary STEM schools. Each of the five schools demonstrate shared visions built on inclusion and collaboration (Tofel- Grehl & Callahan, 2014). All five schools base their school improvement plans on resource and physical space considerations that advance STEM practices (Hsu et al., 2011; Litke & Hill, 2020; Park et al., 2017) and this extends to the way they prioritize collaborative time (Goodpaster et al., 2012; Herro & Quigley, 2017; Lesseig et al., 2016; Margot & Kettler, 2019). Educators also demonstrate high levels of pedagogical content knowledge (Shulman, 1986; Zahorik, 1996), which is associated with stronger learning outcomes. Education Policy Analysis Archives Vol. 31 No. 80 18 Second, collective leadership combined with STEM practices identifies challenges as design problems. Administrators can provide tangible support and facilitate a cohesive vision for collective leadership (Birky et al., 2006; Camburn et al., 2008; Murphy et al., 2009; Smylie & Brownlee- Conyers, 1992; Van Velsor et al., 2010; Weiner & Lamb, 2020). Part of that vision is a strong orientation toward improvement grounded in high levels of relational trust (Bryk & Schneider, 2002; Hargreaves & O’Connor, 2018; Muijs & Harris, 2007; Tschannen-Moran, 2004; Van Velsor & McCauley, 2004) which support stronger working relationships (Hargreaves & O’Connor, 2018; Muijs & Harris, 2007; Van Velsor & McCauley, 2004). All five schools display high levels of relational trust between teachers and administrators as well as strong orientations toward improvement through reflective risk-taking (Eckert, 2018, 2019), the collection of evidence of improvement, and nimble leadership. Federal, state, and district policies that support site-based leadership and autonomy are critical to allow opportunities for school-level innovation. Collective leadership acts as a through-line for school improvement efforts. As administrators and teachers support students in problem-solving related to real-world problems, they also apply those same approaches to problems of practice. When collective leadership refers to work that is done toward shared goals (Eckert, 2021), then those STEM practices spread across school decision-making structures while supporting innovative thinking. Elementary through high school students have ownership of their learning and can provide input into how they best learn. From NYES students who consider themselves engineers designing solutions to CMS’s complete redesign of their school building to high school students reclaiming sewage water in ways that benefit a city, collective leadership develops as a set of functions (Firestone, 1996; Heller & Firestone, 1995; Mayrowetz & Weistein, 1999) where the work, rather than the people, drive improvement (Heller & Firestone, 1995; Louis & Miles, 1990). This approach to leadership reduces conditions that lead to burnout (Doyle Fosco, 2022) because leadership is not situated in a single person or position; instead, a range of leaders with differing expertise collectively lead the school. Third, collective leadership develops through co-designing, co-teaching, and collective efficacy. Collective leadership facilitates the development of diverse leaders in a community. Through co-teaching, teachers influence others by working alongside one another enhances collective teacher efficacy (Hattie, 2015) in the service of student learning. Administrators and teachers are co-leaders as they determine the most effective ways to teach students. Students “figure it out” in classes where teachers do not always have pre-packaged answers. The conditions that support the collective leadership of teachers and administrators also support the leadership of students in their own learning. Community leaders, administrators, teachers, and students can identify issues to address and then bring together the collective expertise of classes and schools to bring about improvement. Collective leadership contextualizes improvement. The engineering mindset when approached from a collective leadership perspective, requires leaders to work together to identify issues, prototype, collect data, and redesign in an iterative process. For example, at the elementary level, the pressures of standardized accountability and the elementary emphasis on mathematics and reading result in a lack of time for STEM instruction and collaboration (Bassok et al., 2016; Diamond & Spillane, 2004; Marx & Harris, 2006). Elementary teachers and administrators at both schools are collaborating to reduce those pressures and find creative ways to maximize planning and instructional time even with well-documented planning limitations (Goodpaster et al., 2012; Herro & Quigley, 2017; Lesseig et al., 2016; Margot & Kettler, 2019), which exist at all levels (Campion et al., 2005; Hackman & Oldham, 1980; Margolis, 2012; Smylie & Denny, 1990) which is likely to reduce burnout and build resilience (Day & Gu, 2014). State and local polices that support professional learning of teachers and administrators together as they work on shared goals are particularly valuable. Additionally, policies that support peer observation and feedback as well as professional Collective leadership 19 learning communities are likely to benefit collective leadership efforts. Moving toward observation as opposed to evaluation, moves schools closer to improving rather than proving. Fourth, school improvement through collective leadership is a process because collective leadership is a vehicle for continuous improvement. Even with their successes, none of the school leaders at the five schools believed that they had achieved exemplary STEM status or met all their school improvement goals. They saw school improvement as a process that required constant data collection, feedback, and revision. Both high schools were seeking additional external partnerships to increase real-world opportunities for students. CMS was spreading the integrated practices of effective teams to other teams with limited resources (Hsu et al., 2011; Litke & Hill, 2020; Park et al., 2017). The improvement efforts of these five schools could further benefit the schools themselves as well as the broader educational system in which they reside if they were part of networked improvement communities (Bryk et al., 2015). These communities operate in similar principles that are evident in the schools’ collective leadership approaches, but they spread expertise and expand improvement beyond individual schools to networks. Networking schools that use these practices could expand “cultures of improvement” (e.g., Dolle et al., 2013; Harrison et al., 2019). Federal and state policies that support processes of improvement rather than focusing exclusively on outcomes are critical for supporting schools. In schools that focus on improvement, iterative collection of observable evidence of growth is critical. These data can be used as evidence of growth. Fifth, policies should support collective leadership of teachers and administrators who use improvement science tools to collect evidence of growth (Bryk et al., 2010, 2015). Schools that emphasize STEM appear to be more likely to collect evidence of progress based on iterations of improvement efforts. In the five schools in this study, these data collections occurred collectively. In a rapidly shifting policy landscape, evidence of improvement toward shared goals that support varied student outcomes is valuable. Whether the goals relate to academic, social-emotional, or other non- cognitive student outcomes, policies that support collective improvement processes allow flexibility within parameters that require evidence collection. Conclusion Leaders and policies can function as catalysts for improvement when they focus on accelerating good work. These cases demonstrate how collective leadership can lead to school improvement. Because of their orientations toward improvement through an engineering mindset that promotes growth, schools such as these could lead the way on school improvement practices through exemplary STEM learning and collective leadership. Schools with a clear vision of STEM learning outcomes and supportive administration can be ideal testing grounds for school improvement. These schools have work designs that are flexible and the opportunity for collaboration and joint risk-taking of collaborative faculty. Adopting and supporting the following seven policy orientations could enhance the catalytic leadership practices highlighted in this study: One, enact policies that support site-based leadership with autonomy over budget and continuous improvement processes that could allow for schedule, facility, and human capital flexibility. Collective leadership at the school level, with networked improvement at a system level, facilitates disciplined inquiry and wide experimentation across sites. Two, implement professional learning of teachers and administrators together on work related to shared goals. Education Policy Analysis Archives Vol. 31 No. 80 20 Three, support peer observation and feedback that fosters collaboration and shared growth. Because of the integrated nature of STEM, providing opportunities for teachers to give and receive feedback around shared practice is likely to improve schools. Four, provide opportunities for administrators, teachers, and students to design flexible schedules that allow for deeper cross-curricular STEM practices and inquiry. Five, develop public/private partnerships that can provide expertise, materials, and funding that can enhance STEM opportunities. This is particularly relevant at the high school level where students are more likely to be able to do field work such as the clean water reclamation project or internships. Six, as much as a given context will allow, make educators and students work public. By allowing student work to be shared publicly, the more resources could be available. Seven, engage school-based teams in iterative improvement cycles that rely on collection of observable evidence of improvement using engineering and design principles. Future research could include data collection on these policy directions that uses the information collected through improvement efforts. More longitudinal data of improvement instead of relying on status-proficiency would also be beneficial. If school leaders and policymakers were to embrace the combination of collective leadership and STEM practices as a through-line for policies that school improvement, then expertise could spread through collective leadership and contextualized solutions could be the result. References America COMPETES Act, Pub. L. No. 110-69, 121 Stat. 572–718. (2007). https://www.govinfo.gov/app/details/PLAW-110publ69 Anderson, G., & Cohen, M. (2015). Redesigning the identities of teachers and leaders: A framework for studying new professionalism and educator resistance. Education Policy Analysis Archives, 23(85). https://doi.org/10.14507/epaa.v23.2086 Ascorra, P., Carrasco Aguilar, C., Lopez, V., & Morales, M. (2019). Policies for school climate in an era of accountability. Education Policy Analysis Archives, 27(31). https://doi.org/10.14507/epaa.27.3526 Avolio, B. J. (2010). Full range leadership development (2nd ed.). Sage. Avolio, B. J., & Hannah, S. T. (2008). Developmental readiness: Accelerating leader development. Psychology Journal: Practice and Research, 60(4), 331–347. Bass, B. M. (1990). Handbook of leadership: Theory, research, and managerial applications (3rd ed.). Free Press. Bassok, D., Latham, S., & Rorem, A. (2016). Is kindergarten the new first grade? AERA Open, 2(1). https://doi.org/10.1177/2332858415616358 Berg, J. H. (2018). Leading in sync: Teacher leaders and principals working together for student learning. ASCD. Birky, V. D., Shelton, M., & Headley, S. (2006). An administrator’s challenge: Encouraging teachers to be leaders. NASSP Bulletin, 90(2), 87–101. Bond, F. W., Flaxman, P. E., & Bunce, D. (2008). The influence of psychological flexibility on work redesign: Mediated moderation of a work reorganization intervention. Journal of Applied Psychology, 93(3), 645–654. Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Harvard Education Press. Bryk, A. S., Sebring, P. B., Allensworth, E., Luppescu, S., & Easton, J. Q. (2010). Organizing schools for improvement: Lessons from Chicago. University of Chicago Press. Collective leadership 21 Bryk, A., & Schneider, B. (2002). Trust in schools: A core resource for improvement. Russell Sage Foundation. Camburn, E. M., Kimball, S. M., & Lowenhaupt, R. (2008). Going to scale with teacher leadership: Lessons learned from a districtwide literacy coach initiative. In M. M. Mangin & S. R. Stoelinga (Eds.), Effective teacher leadership: Using research to inform and reform (pp. 120–143). Teachers College Press. Campion, M. A., Mumford, T. V., Morgeson, F. P., & Nahrgang, J. D. (2005). Work redesign: Eight obstacles and opportunities. Human Resource Management, 44(4), 367–390. Carver, C., Margolis, J., & Williams, T. (2013). Teacher leadership: Practices, politics, and persistence. The New Educator, 9(3), 167–196. Carver-Thomas, D., & Darling-Hammond, L. (2019). The trouble with teacher turnover: How teacher attrition affects students and schools. Education Policy Analysis Archives, 27(36). http://dx.doi.org/10.14507/epaa.27.3699 Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normal and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. Coburn, C. E., & Penuel, W. R. (2016). Research–practice partnerships in education: Outcomes, dynamics, and open questions. Educational Researcher, 45(1), 48–54. https://doi.org/10.3102/0013189X16631750 Conger, J. (1992). Learning to lead: The art of transforming managers into leaders. Jossey-Bass. Creswell, J. W., & Clark, V. L. P. (2018). Designing and conducting mixed methods research. Sage. Day, C., & Gu, Q. (2014). Resilient teachers, resilient schools: Building and sustaining quality in testing times. Routledge. Day, C., Gu, Q., & Sammons, P. (2016). The impact of leadership on student outcomes. Educational Administration Quarterly, 52(2), 221–258. https://doi.org/10.1177/0013161X15616863 Day, D. V., Zaccaro, S. J., & Halpin, S. M. (2004). Leadership development for transforming organizations: Growing leaders for tomorrow. Psychology Press. DeWitt, P. M. (2017). Collaborative leadership: Six influences that matter most. Corwin Press. Diamond, J. B., & Spillane, J. P. (2004). High-stakes accountability in urban elementary schools: Challenging or reproducing inequality? Teachers College Record, 106(6), 1145–1176. Dolle, J. R., Gomez, L. M., Russell, J. L., & Bryk, A. S. (2013). More than a network: Building professional communities for educational improvement. In Yearbook of the National Society for the Study of Education (Vol. 112, pp. 443–463). Teachers College. Doyle Fosco, S. L. (2022). Educational leader wellbeing: A systematic review. Educational Research Review, 37, 100487. https://doi.org/10.1016/j.edurev.2022.100487 Eckert, J. (2016). The novice advantage: Fearless practice for every teacher. Corwin Press. Eckert, J. (2018). Leading together: Teachers and administrators improving student outcomes. Corwin Press. Eckert, J. (2019). Collective leadership development: Emerging themes from urban, suburban, and rural high schools. Educational Administration Quarterly, 55(3), 477–509. https://doi.org/10.1177/0013161X18799435 Eckert, J., & Butler, J. (2021). Teaching and leading for exemplary STEM learning. The Elementary School Journal, 121(4), 674–699. https://doi.org/10.1086/713976 Eckert, J., & Daughtrey, A. (2019). Teacher leadership development: Tracking one district’s progress over three years. Education Policy Analysis Archives, 27(42), 1–22. http://dx.doi.org/10.14507/epaa.27.4130 Eckert, J. & Morgan, G. B. (in press). Collective school leadership through turbulence. Leadership and Policy in Schools. Education Policy Analysis Archives Vol. 31 No. 80 22 Eckert, J., Morgan, G. B., & Padgett, R. N. (2022). Collective leadership: Developing a tool to assess educator readiness and efficacy. Journal of Psychoeducational Assessment, 07342829211072284. https://doi.org/10.1177/07342829211072284 Firestone, W. A. (1996). Leadership: Roles or functions? In K. Leithwood, J. Chapman, D. Corson, P. Hallinger, & A. Hart (Eds.), International handbook of educational leadership and administration, Part I (pp. 395–418). Kluwer. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378–382. Fullan, M. (2005). Leadership and sustainability: System thinkers in action. Corwin Press. Gates, S. M., Baird, M. D., Master, B. K., & Chavez-Herrerias, E. R. (2019). Principal pipelines: A feasible, affordable, and effective way for districts to improve schools. RAND Corporation. https://doi.org/10.7249/RR2666 Goodpaster, K. P. S., Adedokun, O. A., & Weaver, G. C. (2012). Teachers’ perceptions of rural STEM teaching: Implications for rural teacher retention. Rural Educator, 33. Hackman, J. R., & Oldham, G. R. (1980). Work redesign. Addison-Wesley. Hansen, M. (2014). Characteristics of schools successful in STEM: Evidence from two states’ longitudinal data. Journal of Educational Research, 107(5), 374–391. https://doi.org/10.1080/00220671.2013.823364 Hargreaves, A., & Fink, D. (2003). Sustaining leadership. Phi Delta Kappan, 84(9), 693–700. Hargreaves, A., & Fink, D. (2006). Sustainable leadership. Jossey-Bass. Hargreaves, A., & O’Connor, M. T. (2018). Collaborative professionalism: When teaching together means learning for all. Corwin Press. Harrison, C., Wachen, J., Brown, S., & Cohen-Vogel, L. (2019). A view from within: Lessons learned from partnering for continuous improvement. Teachers College Record, 121(9). Hattie, J. (2012). Visible learning for teachers: Maximizing impact on learning. Routledge. Hattie, J. (2015). What works best in education: The politics of collaborative expertise (Open Ideas). Pearson. https://www.pearson.com/content/dam/corporate/global/pearson-dot- com/files/hattie/150526_ExpertiseWEB_V1.pdf Hattie, J. (2017). Collective teacher efficacy. https://visible-learning.org/2018/03/collective-teacher- efficacy-hattie/ Heller, M. F., & Firestone, W. A. (1995). Who’s in charge here? Sources of leadership for change in eight schools. The Elementary School Journal, 96(1), 65–86. Herro, D., & Quigley, C. (2017). Exploring teachers’ perceptions of STEAM teaching through professional development: Implications for teacher educators. Professional Development in Education, 43(3), 416–438. https://doi.org/10.1080/19415257.2016.1205507 Hsu, M. C., Purzer, S., & Cardella, M. E. (2011). Elementary teachers’ views about teaching design, engineering, and technology. Journal of Pre-College Engineering Education Research, 1(2). https://doi.org/10.5703/1288284314639 Humphrey, S. E., Nahrgang, J. D., & Morgeson, F. P. (2007). Integrating motivational, social, and contextual work design features: A meta-analytic summary and theoretical extension of the work design literature. Journal of Applied Psychology, 92(5), 1332–1356. Johnson, C. C., Peters-Burton, E., & Moore, T. J. (2016). The need for a STEM road map. Routledge. Lai, E., & Cheung, D. (2015). Enacting teacher leadership: The role of teachers in bringing about change. Educational Management Administration & Leadership, 43(5), 673–692. https://doi.org/10.1177/1741143214535742 Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. Leithwood, K., & Louis, K. S. (2012). Linking leadership to student learning. Jossey-Bass. Collective leadership 23 Leithwood, K., & Mascall, B. (2008). Collective leadership effects on student achievement. Educational Administration Quarterly, 44(4), 529–561. Lesseig, K., Slavit, D., Nelson, T. H., & Seidel, R. A. (2016). Supporting middle school teachers’ implementation of STEM design challenges. School Science and Mathematics, 116(4), 177–188. https://doi.org/10.1111/ssm.12172 Lieberman, A., & Miller, L. (2004). Teacher leadership. Jossey-Bass. Light, P. C. (1998). Sustaining innovation: Creating nonprofit and government organizations that innovate naturally. Jossey-Bass. Lima, L. C. (2021). Democracy and education: Dewey in times of crisis in democratic education. Education Policy Analysis Archives, 29(154). https://doi.org/10.14507/epaa.29.5881 Litke, E., & Hill, H. C. (2020). Exploring central tendencies: The teaching of data and statistics in elementary mathematics classrooms. Elementary School Journal, 120(3), 480–506. https://doi.org/10.1086/707007 Louis, K. S., Leithwood, K., Wahlstrom, K. L., & Anderson, S. E. (2010). Investigating the links to improved student learning: Final report of research findings. The Wallace Foundation. http://www.wallacefoundation.org/knowledge-center/Documents/Investigating-the-Links- to-Improved-Student-Learning.pdf Louis, K. S., & Miles, M. B. (1990). Improving the urban high school: What works and why. Teachers College Press. Lynch, S. J., Peters-Burton, E., & Ford, M. (2015). Building STEM opportunities for all. Educational Leadership, 72(4), 54–60. MacBeath, J., & Dempster, N. (Eds.). (2009). Connecting leadership and learning: Principles for practice. Routledge. Mangin, M. M., & Stoelinga, S. R. (2008). Effective teacher leadership: Using research to inform and reform. Teachers College Press. Margolis, J. (2012). Hybrid teacher leaders and the new professional development ecology. Professional Development in Education, 38(2), 291–315. Margot, K. C., & Kettler, T. (2019). Teachers’ perception of STEM integration and education: A systematic literature review. International Journal of STEM Education, 6(2). https://doi.org/10.1186/s40594-018-0151-2 Martineau, J. W. (2004). Evaluating the impact of leader development. In C. D. McCauley & E. Van Velsor (Eds.), The Center for Creative Leadership handbook of leadership development (2nd ed., pp. 234–267). Jossey-Bass. Marx, R. W., & Harris, C. J. (2006). No Child Left Behind and science education: Opportunities, challenges, and risks. Elementary School Journal, 106(5), 467–478. Mayrowetz, D., & Weistein, C. S. (1999). Sources of leadership for inclusive education: Creating schools for all children. Educational Administration Quarterly, 35(3), 423–449. McCauley, C. D. (2008). Leader development: A review of research. Center for Creative Leadership. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. Sage. Muijs, D., & Harris, A. (2007). Teacher leadership in (in)action: Three case studies of contrasting schools. Educational Management, Administration, and Leadership, 35(1), 111–134. Mumford, M. D., Hunter, S. T., Eubanks, D. L., Bedell, K. E., & Murphy, S. T. (2007). Developing leaders for creative efforts: A domain-based approach to leadership development. Human Resource Management Review, 17(4), 402–417. Murphy, J. (2005). Connecting teacher leadership and school improvement. Corwin Press. Murphy, J., Smylie, M. A., Mayrowetz, D., & Louis, K. S. (2009). The role of the principal in fostering the development of distributed leadership. School Leadership and Management, 29, 181–214. Education Policy Analysis Archives Vol. 31 No. 80 24 Nash, J. B. (2019). Design thinking in schools: A leader’s guide to collaborating for improvement. Harvard Education Press. National Research Council. (2009). Engineering in K-12 education: Understanding the status and improving the prospects. The National Academies Press. National Research Council. (2010). Standards for K-12 engineering education? National Academies Press. National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. National Academies Press. NCES. (2022). Public school students eligible for free or reduced-price lunch. https://nces.ed.gov/fastfacts/display.asp?id=898 NGSS Lead States. (2013). Next Generation Science Standards: For states by states. National Academies Press. Papay, J. P., Taylor, E. S., Tyler, J. H., & Laski, M. E. (2020). Learning job skills from colleagues at work: Evidence from a field experiment using teacher performance data. American Economic Journal: Economic Policy, 12(1), 359–388. https://doi.org/10.1257/pol.20170709 Park, M., Dimitrov, D. M., Patterson, L. G., & Park, D. (2017). Early childhood teachers’ beliefs about readiness for teaching science, technology, engineering, and mathematics. Journal of Early Childhood Research, 15(3), 275–291. https://doi.org/10.1177/1476718X15614040 Patton, M. (2002). Qualitative research and evaluation methods. Sage. Peters-Burton, E., Kaminsky, S. E., Lynch, S., Behrend, T., Han, E., Ross, K., & House, A. (2014). Wayne School of Engineering: Case study of a rural inclusive STEM‐focused high school. School Science and Mathematics, 114(6), 280–290. https://doi.org/10.1111/ssm.12080 Peters-Burton, E., Lynch, S., Behrend, T., & Means, B. (2014). Inclusive STEM high school design: 10 critical components. Theory into Practice, 53(1), 64–71. Pfeiffer, A. I., Overstreet, J. M., & Park, A. (2010). The state of science and mathematics education in state- supported residential academies: A nationwide survey. Roeper Review, 32(1), 25–31. Rubin, H. J., & Rubin, I. S. (2005). Qualitative interviewing: The art of hearing data. Sage. Saldana, J. (2016). The coding manual for qualitative researchers. Sage. Schein, E. H. (2013). Humble inquiry: The gentle art of asking instead of telling. Berrett-Koehler Publishers. Seashore-Louis, K., Leithwood, K., Wahlstrom, K., & Anderson, S. (2010). Learning for leadership: Investigating the links to improved student learning. Educational Research Service. Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Research, 15(2), Article 2. Smylie, M. A. (2010). Continuous school improvement. Corwin Press. Smylie, M. A., & Brownlee-Conyers, J. (1992). Teacher leaders and their principals: Exploring the development of new working relationships. Educational Administration Quarterly, 28, 150–184. Smylie, M. A., & Denny, J. W. (1990). Teacher leadership: Tensions and ambiguities in organizational perspective. Educational Administration Quarterly, 26(3), 235–259. Smylie, M. A., & Eckert, J. (2018). Beyond superheroes and advocacy: The pathway of teacher leadership development. Educational Management Administration & Leadership, 46(4), 556–577. https://doi.org/10.1177/1741143217684893 Spillane, J. P. (2006). Distributed leadership. Jossey-Bass. Spillane, J. P., Halverson, R., & Diamond, J. B. (2001). Investigating school leadership practice: A distributed perspective. Educational Researcher, 30(3), Article 3. Stavros, J. M., Godwin, L. N., & Cooperrider, D. L. (2016). Appreciative inquiry. In Practicing organizational development: Leading transformation and change (Fourth). John Wiley and Sons, Inc. https://doi.org/10.1002/9781119176626.ch6 Collective leadership 25 STEM Ecosystems. (2019). East Syracuse Minoa Central School District STEM learning ecosystem. East Syracuse Minoa Central School District STEM Learning Ecosystem Tofel-Grehl, C., & Callahan, C. M. (2014). STEM high school communities: Common and differing features. Journal of Advanced Academics, 25(3), 237–271. https://doi.org/10.1177/1932202X14539156 Tschannen-Moran, M. T. (2004). Trust matters: Leadership for successful schools. Jossey-Bass. Tushman, M. L., & O’Reilly, C. A. (1996). Ambidextrous organizations: Managing evolutionary and revolutionary change. California Management Review, 38(4), 8–30. U.S. News and World Report. (2020). Best high schools. https://www.usnews.com/education/best- high-schools/search?name=Staten%20Island%20Technical%20High%20School Van Velsor, E., & McCauley, C. D. (2004). Our view of leadership development. Jossey-Bass. Van Velsor, E., McCauley, C. D., & Ruderman, M. N. (Eds.). (2010). The center for creative leadership handbook of leadership development (3rd ed.). Jossey-Bass. Weiner, J. M., & Lamb, A. J. (2020). Exploring the possibilities and limits to transfer and learning: Examining a teacher leadership initiative using the theory of action framework. Journal of Educational Change, 21(2), 267–297. https://doi.org/10.1007/s10833-020-09378-z Wenner, J. A., & Campbell, T. (2017). The theoretical and empirical basis of teacher leadership: A review of the literature. Review of Educational Research, 87(1), 134–171. https://doi.org/10.3102/0034654316653478 Yin, R. (2014). Case study research: Design and method (5th ed.). Sage. York-Barr, J., & Duke, D. (2004). What do we know about teacher leadership? Findings from two decades of scholarship. Review of Educational Research, 74(3), 255–316. Yukl, G. (2013). Leadership in organizations (8th ed.). Pearson. Zahorik, J. A. (1996). Elementary and secondary teachers’ reports on how they make learning interesting. Elementary School Journal, 96(5), 551–564. https://doi.org/10.1086/461844 Education Policy Analysis Archives Vol. 31 No. 80 26 Appendix A Sample Collective Leadership Survey Items by Condition All responses are based on a 5-point scale (Strongly disagree, disagree, neither agree nor disagree, agree, strongly agree) Shared vision and strategy: Collective leadership is the work of teachers and administrators toward shared goals. To what extent do you agree or disagree with the following statement? "There is a shared vision, between teachers and administrators, of the goals for my school." Supportive administration: The principal is comfortable expanding the power of teachers. Resources: There are adequate financial resources to support our school's goals. There are adequate human resources to support our school's goals. Work design: There is adequate time available for leadership work. There are regular opportunities for teachers to observe each other’s teaching. Supportive norms and working relationships: I have good working relationships with teachers in my school. I have good working relationships with administrators in my school. Shared influence: I have a great deal of influence on teachers in my school. Administrators in my school. Orientation toward improvement: At my school, risk-taking informed by reflection is encouraged. Collective leadership 27 Appendix B Focus Group Protocol “Your participation is completely voluntary. If you are willing to participate and include you email address on the sign in sheet, we will send you a $50 Amazon gift card. Our goal is to better understand what has made STEM education successful at your school. With your permission, I will be recording this to ensure the accuracy of your comments. Our findings from your school and four others will become the basis of a report from the National Network of State Teachers of the Year. Thanks for taking the time to talk to us.” Conditions: 1) How have you designed teachers’ and administrators’ work to advance shared STEM goals? 2) How are you developing capacity of teachers and administrators to lead STEM learning? 3) What autonomy do teachers have to lead STEM learning? Barriers: 4) What barriers have teachers and/or administrators had to overcome to develop exemplary STEM learning? a. Time? b. Work structures (e.g., PLCs, co-teaching, peer observation, hybrid roles)? c. Curriculum? d. District disconnects? e. Attitudes/politics? Collective Leadership – defined as the work of teachers and administrators toward shared goals (probe on attitudes toward collective leadership where possible): 5) On a scale of 0-10 – how much of the exemplary STEM work has been led by teachers (10 being total control, 0 being no control)? 6) On a scale of 0-10 – how much of the exemplary STEM work has been led by administrators (10 being total control, 0 being no control)? 7) What are effective ways that teachers and administrators have led together? How has that improved student outcomes? 8) What else should we know about your school as it relates to STEM? Education Policy Analysis Archives Vol. 31 No. 80 28 About the Authors Jonathan Eckert Baylor University jon_eckert@baylor.edu https://orcid.org/0000-0002-7171-847X Jonathan Eckert is the Lynda and Robert Copple Professor of Educational Leadership at Baylor University. He has written multiple books and articles on collective leadership, teaching efficacy, STEM, and strategic compensation. Grant Morgan Baylor University grant_morgan@baylor.edu https://orcid.org/0000-0002-3077-4643 Grant Morgan is the Associate Dean for Research and Professor of Quantitative Methods in the School of Education at Baylor University. His research interests include latent variable models, psychometrics, classification, and nonparametric statistics. education policy analysis archives Volume 31 Number 80 August 1, 2023 ISSN 1068-2341 Readers are free to copy, display, distribute, and adapt this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, the changes are identified, and the same license applies to the derivative work. More details of this Creative Commons license are available at https://creativecommons.org/licenses/by-sa/4.0/. EPAA is published by the Mary Lou Fulton Teachers College at Arizona State University. Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), QUALIS A1 (Brazil), SCImago Journal Rank, SCOPUS, SOCOLAR (China). About the Editorial Team: https://epaa.asu.edu/ojs/index.php/epaa/about/editorialTeam Please send errata notes to Audrey Amrein-Beardsley at audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. mailto:jon_eckert@baylor.edu https://orcid.org/0000-0002-7171-847X mailto:grant_morgan@baylor.edu https://orcid.org/0000-0002-3077-4643 https://creativecommons.org/licenses/by-sa/4.0/ http://www.doaj.org/ http://www.doaj.org/ https://epaa.asu.edu/ojs/index.php/epaa/about/editorialTeam mailto:audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE