International Electronic Journal of Elementary Education June 2023, Volume 15, Issue 5, 371-382 371 © 2023 Published by KURA Education & Publish- ing. This is an open access article under the CC BY- NC- ND license. (https://creativecommons. org/licenses/by/4.0/) Copyright © www.iejee.com ISSN: 1307-9298 Teacher Professional Development for STEM Integration in Elementary/ Primary Schools: A Systematic Review Tugba Boz* Abstract Introduction There has been a recent increase in studies devoted to integrated science, technology, and engineering mathematics (STEM) education. However, as the field is emerging, there has not been a large amount of scholarship on how to best provide teacher professional development (PD) on integrated STEM education. On the other hand, well-grounded research is available in the field of effective teacher professional development. Therefore, in this study, a systematic review of empirical studies on training elementary/primary school teachers in integrated STEM education was undertaken. Using an adapted version of Lawless and Pellegrino's analytical framework (2007), the emerging commonalities among eleven studies in conjunction with the literature on effective teacher professional development were analyzed and discussed. This study aims at connecting current STEM-integrated PD activities with the literature on effective PD and laying the groundwork for future professional development programs and evaluations in integrating STEM subject areas in elementary or primary schools. In recent years, there has been an increase in studies dedicated to exploring the conceptual, theoretical, and practical implications of integrated STEM education which refers to “the approach to teaching the STEM content of two or more STEM domains, bound by STEM practices within an authentic context for the purpose of connecting these subjects to enhance student learning” (Kelly & Knowles, 2016, p.3). The literature (e.g., English, 2016; Nadelson & Seifert, 2017; Pearson, 2017) argues that integrated STEM education increases students' interest in and engagement with STEM fields and better prepares students for the future STEM workforce in which disciplines are interrelated and integrated. This attention to integrating STEM subject areas has brought about the development and implementation of integrated STEM programs in real classrooms. Engineering design projects or the development of robotics projects in science and mathematics classrooms are a few examples. As there is more emphasis on initiating integrated STEM Keywords: Teacher Professional Development, Integrated STEM, Review of Literature, Elementary Teachers Received : 27 October 2022 Revised : 5 April 2023 Accepted : 17 June 2023 DOI : 10.26822/iejee.2023.306 * Correspondance Details: Tugba Boz, Department of Educational Theory and Practice, University of Georgia, Athens GA, United States. E-mail: tugba.boz@uga.edu ORCID: https://orcid.org/0000-0002-8020-127X 372 June 2023, Volume 15, Issue 5, 371-382 education at early ages (e.g., Cunningham, 2018), programs have been designed and implemented to train elementary school teachers to implement integrated STEM education in their classrooms. However, since the field is emerging, there has been a limited amount of research on teacher professional development in integrated STEM education. On the other hand, the literature provides well-grounded and comprehensive answers on the essential characteristics of effective teacher professional development (PD) and their effects on teacher beliefs and classroom practices. Therefore, this study begins with an in-depth literature review on the key characteristics of effective teacher professional development (PD). The review draws on highly influential and frequently cited studies, authored by the leading figures in the field systematic review is then presented to identify the current empirical studies on professional development activities in integrated STEM education. The discussion focuses on the connection between the current STEM-integrated professional development activities and the literature on effective PD characteristics. This inquiry is aimed to lay the groundwork for future professional development programs and evaluations in STEM integration since, as Desimone and Garet (2015) argued, specific inquiries in teacher PD research are necessary to "translate the broad features into specific, effective activities in varying contexts" (p. 260). Furthermore, this study aims to provide a framework that outlines the effective components of professional development in the integration of STEM subjects and presents guidelines for program/project developers and policymakers in their consideration of teacher professional development in STEM integration. Overview of Teacher Professional Development Definition and History Professional development (PD) is an activity that aims to improve paid staff members' performance (Little, 1987). It is broadly defined as teachers' professional growth resulting from their formal or informal experiences, such as attending workshops, professional meetings, mentoring, reading teacher magazines, and watching documentaries (Villegas- Reimers, 2003; OECD, 2019). Some scholars prefer using the term professional learning instead of professional development to emphasize "the nature and ownership of teacher learning" and position teachers as "the key decision makers about what matters for their growth" (Smith, 2017, p.2). According to Borko (2004), teacher learning can occur in various contexts, such as brief hallway conversations or counseling a troubled child after school. his broad definition of PD recognizes that professional development can occur in multiple settings and ways." It is worth noting that teacher professional development or learning has not always been defined and considered in such a broad way or as an ongoing process. In the first half of the 20th century, research on teacher professional development began with large survey studies (Cochran-Smith, 2009). Starting in the 1970s, a new approach started to govern teacher education. According to Cochran-Smith and Demers (2008), this new model was "more constructivist than transmission-oriented—the recognition that both prospective and experienced teachers (like all learners) brought prior knowledge and experience to all new learning situations, which are social and contextually specific" (p.1011). Therefore, in this new model, teacher education was not a one-time event but an ongoing and complex process which "was marked by an equally important set of factors" embedded in teachers' everyday practices and interactions (Avalos, 2011, p.17). Today, it is widely accepted that teacher professional development plays a significant role in enhancing the quality of school, influencing teachers' beliefs and practices (Desimone, 2009, 2011; Garet et al., 2001; Villegas-Reimers, 2003), and improving student achievement (Desimone et al., 2005). Effectively designed and implemented PD programs can help teachers in their day-to-day work (Fullan & Miles, 1992). Therefore, PD is not only a government or state- mandated mass learning practice but is also highly desired by teachers. Characteristics of Effective Professional Development The literature proposes well-grounded recommendations for designing and delivering effective teacher PD. One such recommendation is to focus on active learning in PD, as research indicates thar active learning is more likely to result in intended teacher knowledge and skills (Bates & Morgan, 2018; Garet et al., 2001; Penuel et al., 2007). In other words, PD should offer teachers many opportunities to observe the modeling of intended changes in instructional practices, incorporate lesson planning and enactments, and practice and reflect on them. (Birman et al., 2000; Darling-Hammond et al., 2017; Darling-Hammond & Richardson, 2009; Darling- Hammond & McLaughlin, 1995; Desimone & Pak, 2017; Desimone, 2009; Desimone et al., 2013; Knapp, 2003; Popova et al., 2018). Additionally, PD should encourage teachers to discuss their work and their students' work, in order to foster collaboration and shared learning (Desimone et al., 2002). Second, PD should focus on student learning-oriented teaching practices and activities (Guskey, 2000; Darling-Hammond & Richardson, 2009; Desimone et al., 2013; Knapp, 2003): As widely discussed in the literature, the main objective behind PD is to improve student learning. Guskey (2000) emphasizes that all PD 373 Teacher Professional Development for STEM Integration in Elementary/Primary Schools: A Systematic Review / Boz efforts should primarily focus on learning and learners, and that even in planning PD, educators should first decide on the specific student learning outcomes they expect to achieve (Guskey, 2014). Knapp (2003) claimed that professional learning experiences should concentrate on classroom teaching and the evidence of student learning (e.g., Gupta & Lee, 2020). Additionally, analyzing student work can be incorporated into PD and used to develop a shared understanding among teachers of what constitutes good work, common misconceptions students may have, and which instructional strategies are effective (Darling-Hammond & Richardson, 2009). Third, PD should have a specific subject focus (Desimone et al., 2013; Garet et al., 2001; Villagas- Reimeners, 2013): Popova et al. (2018) have indicated that having a specific subject focus is positively associated with student learning. Villagas-Reimeners (2013) has indicated that a specific subject matter is important to choose for PD because the content, design, and implementation of PD vary according to the subject matter. For example, Villagas-Reimeners found that concentrated time for PD is more effective for mathematics, while distributed time is more effective for science. Fourth, PD should be a collaborative and collegial act (Darling-Hammond & Richardson, 2009; Desimone et al., 2013; Garet et al., 2001; Knapp, 2003): Collaborative and collegial learning environments create communities of practice and help improve/increase the change process beyond individual classrooms. However, for such collaborative interactions to emerge, professional development leaders should ensure a safe place to nurture trust and critical dialogue (Borko, 2004). Fifth, teacher learning in PD should be a coherent part of and seamlessly linked to the curriculum, assessment, and standards in place (Darling-Hammond & Richardson, 2009; Desimone, 2009): Desimone and Garet (2015) emphasized the importance of linking professional development (PD) explicitly to curriculum and classroom realities, as this connection can improve the success of PD initiatives. Additionally, PD programs should be designed to ensure that there are no disparities between what teachers learn and what they can implement in their classrooms (Desimone, 2009). To achieve this goal, PD should be connected to other changes that are already in place within schools (e.g., Garet et al., 2001). By making these connections, teachers can more easily integrate new knowledge and skills into their classroom practices. Finally, PD should be sustained over time through on-demand in-site expert support and follow-ups (Darling-Hammond et al., 2017): While there is no consensus on the duration of PD required to lead to positive outcomes, some scholars suggest that 80 hours (Darling-Hammond & Richardson, 2009), 20 hours (Desimone, 2009), or even 8 hours (Parsad, 2001) can yield improvements in teaching in real-world classrooms. It is worth noting that the effectiveness of PD is also influenced by the content, format, and quality of the activities offered to teachers (OECD, 2019). Therefore, PD programs should offer teachers a variety of activities that allow for active learning, such as observation of effective teaching practices, lesson planning, and opportunities for reflection and collaboration with peers. As Darling-Hammond (2009) claimed, the academic community is on its way toward reaching a consensus on the content, design, and implementation of effective professional development. However, contextual and background factors should also be considered when designing and delivering PD programs (e.g., Altun et al., 2021). Research suggests that teacher motivation plays an important role in the success of PD initiatives (Guskey, 2002; Shin & Jun, 2019). The content of PD also influences the impact of PDs on teacher change. According to Desimone and Garet (2015), it is easier for teachers to change procedural classroom behavior than to acquire inquiry-oriented instructional techniques. Boyd et al. (2009) claimed that professional development that focuses on practical aspects of teaching is more likely to produce positive effects on students than professional development that primarily emphasizes teacher behaviors. It seems that the existing body of literature on teacher professional development is still growing. Still, available literature pinpoints the ways in which PD evaluations can be undertaken and PD effectiveness can be determined. Evaluating the Effectiveness of Professional Development As argued by Knapp (2003), "the most immediate target of professional development is pro-learning" (p.112). Therefore, PD evaluations aim to explore the "changes in the thinking, knowledge, skills, and application that form practicing teachers' or administrators' repertoire" P.112). Historically, many early PD evaluations consisted of self-reported changes in the participating teachers' behavior and knowledge at the end of a workshop or series (Desimone, 2011). Some scholars (e.g., Guskey & Sparks, 1991; Smylie, 1989) subsequently made this criticism and called for more in-depth studies on the subject. Today, scholars often suggest that rigorous evaluations should be done during the process of planning and during the process of delivery through formative and summative assessments. For example, Guskey (2014) listed the following questions to be considered in planning a PD: "Is this experience or activity leading to the intended results? Is it better than what was done in the past? Is it better than another, competing activity? Is it worth the costs?" (p.1219). He 374 June 2023, Volume 15, Issue 5, 371-382 also added that "Answers to these [the following] questions require more than a statement of findings. They demand an appraisal of quality and judgments of value, based on the best evidence available" (p.1219). Additionally, more rigorous evidence is expected from researchers or PD practitioners about evaluation. It is recommended that these evaluations should be rigorously handled, including the use and administration of many tools (Desimone, 2009, 2011; Guskey, 2014). Depending on the goal of PD, questionnaires, structured interviews, lesson plans, “oral or written personal reflections, or examinations of participants’ journals and portfolios…direct observations, either by trained observers or using digital recorders” are recommended in the literature to be employed in evaluations (Guskey, 2014, p.1227). Furthermore, more agents are expected to be involved, such as teachers, administrators, parents, and/or students (Guskey, 2014). Furthermore, student learning is recommended as a priority in PD evaluations. However, student learning outcomes are not only confined to increases in test scores or cognitive outcomes, but affective and behavioral outcomes should also be considered (Guskey, 2002), such as "their perceptions of teachers, fellow students, and themselves; their sense of self-efficacy and their confidence in new learning situations can be especially informative" (Guskey, 2014, p.1228). Purpose of the Study After elaborating on the elements of effective professional development and evaluation, the author now turns to what is considered effective in PD planning, implementation, and evaluation in STEM integration. For this particular study, a broad and widely-agreed upon the conceptualization of integrated STEM education was adopted: STEM integration is the intentional and explicit connection of STEM disciplines—science, mathematics, engineering, and technology (English, 2016; Kelly & Knowles, 2016; National Academy of Sciences, 2014; Pearson, 2017). Under this concept of integrated STEM education, the following research questions were posed: 1. What is the current state of teacher PD in integrated STEM education in elementary schools? 2. What connections are present between the current state of PD in integrated STEM education and the literature on effective teacher PD in terms of types, design and evaluation of PD effectiveness? 3. What implications/recommendations can be offered to improve the current state of PD in integrated STEM education for elementary teachers based on the recommendations offered by the literature on effective teacher PD? Methods Screening and Eligibility Process of the Review The main databases in educational research (ERIC, Web of Science, PsychInfo, Academic Search Complete) were used to find the empirical articles published in peer-reviewed journals published between 1999 to 2022. Empirical was defined as having qualitative and quantitative data collection and analysis. Generalists: Elementary classroom teachers (K-5) and primary school teachers (K-6) were targeted. Using the Boolean operators, the following keywords and a combination of keywords were used: (STEM integration* OR interdisciplinary STEM) AND (professional development OR continuing training) AND (elementary teachers* OR primary teachers*). Based on this initial query, 94 articles were retrieved after removing duplicates. For this study, abstracts were examined based on the following criteria: C1. Integration was aimed extensively and intentionally in PD, and PD was the reviewed study's focus. C2. More than one subject area was explicitly targeted and was used as integral to the activities rather than peripheral. For example, technology or engineering alone was not considered integrated. C3. Only in-service elementary or primary school teachers were targeted. C4. The PD aimed to change teacher knowledge, skills, or behaviors. At the end of this review process, 11 studies were included in this current review (see Table 1 below). Figure 1. Identification of the Reviewed Studies (Page et al., 2021) 375 Teacher Professional Development for STEM Integration in Elementary/Primary Schools: A Systematic Review / Boz Analytical Framework and Analysis Lawless and Pellegrino (2007) suggested a schema to review technology TPD programs. In this schema, they recommended evaluating PD programs or research studies based on: Three critical dimensions … [which are] [a] type of professional development, which includes issues of delivery, duration, and content… [b] the unit (or units) of analysis that serves as the focus of any research/ evaluation of the outcomes and efficacy of that program … [c] the nature of the research/evaluation study design and method, above and beyond the issues of a unit of analysis and measures. (p. 582) In this study, the schema proposed by Lawless and Pellegrino was partially adapted. The investigation focused on (a) the types of professional development, including duration, content, and delivery, (b) the unit of analysis or the outcomes evaluated in the professional development, such as student, program, or teacher outcomes, and (c) the designs and methods employed to research the effectiveness of professional development in integrated STEM education. In order to ensure the study's rigor and repeatability (Xiao & Watson, 2019), the author utilized the PRISMA framework for finding and selecting the reviewed studies and strictly followed the schema by Lawless and Pellegrino. Thematic analysis (Joffe, 2011) was employed to identify the main themes of the schema in the 11 reviewed studies, and the similar and different themes for each component of the schema were then reported. The following section discusses the similarities and the differences among the 11 studies in terms of their type, unit of analysis, and design of professional development (PD) programs. Then, the connections between these STEM-integrated PD programs and the literature on effective PD characteristics are explored. Finally, the implications are presented. Table 1 Lists of Empirical Studies on STEM integrated Professional Development in K-5. Study Duration & Delivery Participants Design & Methods Baxter et al., 2014 A two-day workshop in the summer, a six-day workshop during the school year, and two classroom observation sessions for ten contact days 44 K-5 classroom teach- ers Field notes of workshops, an online survey of teacher confi- dence in integrated mathematics and science, self-reported responses to changes in practice Boice et al., 2021 Five-week summer PD, ongoing support during the school year, 17 STEM and art teachers from nine elementary schools Surveys focus groups, written reflec- tions Cook et al., 2020 Two-year PD program that included approximately 130 hours of PD during the school year. 25 classroom teachers Lesson plans, reflections Guzey et al., 2016 A three-week summer institute and year-long support from a coach through monthly meetings 48 K-8 teachers from three large school districts The researchers developed a STEM integration assessment tool Havice et al., 2018 Two-day PD 33 K-5 teachers Pre-Post Survey McFadden and Roehrig, 2017 Three-week intensive summer PD, developing STEM-integrat- ed curricular units in teams, pilot teaching over the univer- sity-based summer camp and in their classrooms over the following academic year Four elementary teach- ers in one group and three elementary teach- ers in the other group Participant reflections, field notes, re- cordings of PD days, interviews, team conversations, curriculum develop- ment artifacts Parker et al., 2015 Two-week summer workshop, application during 4-week summer school in 22 schools Six K-5 teachers from each of the 22 partici- pant schools Focus group interviews and teacher surveys over the academic year Rich et al., 2017a Year-long series of weekly PD followed by classroom practices (Year 1 of a longitudinal study) 27 teachers of K-6 Teacher self-efficacy and beliefs survey for integrating computing and engineering, Interviews and documented obser- vations Rich et al., 2018a Weekly PD meetings for 45 minutes with a PD researcher on Friday afternoons over an academic year (Year 2 of a longitudinal study) 17 elementary teachers Interviews at the end of the lesson study/application, PD practitioners’ fieldnotes Sias et al., 2017 One-week-long PD 39 teachers of the third to fifth grades all 39 lesson plans were coded/ana- lyzed by the researchers developing rubric. Wentworth and Monroe, 2011 Series of PD over two years (Details are not specified) 38 in-service elementary teachers Ten lesson plans (along with enact- ments) produced by 38 teachers in groups of three to four; coded by two mathematics in- structors based on the four criteria outlined at the beginning of the given PD Notes. a = As these two papers report on the same PD, they will be considered together in reporting their findings. 376 June 2023, Volume 15, Issue 5, 371-382 Findings and Discussion In this section, the similarities and the differences among the 11 studies regarding the type, unit of analysis, and the design features of their teacher professional development (PD) programs. For example, under the “type” of PD theme, the reviewed studies were categorized and presented based on the delivery medium employed, the duration of the PD offered, and the content (curriculum). The “unit of analysis” theme revealed the tools employed in the reviewed studies to evaluate the effectiveness of PD programs. The connections between these STEM- integrated PD programs and the literature on effective PD traits were presented for each theme (type, design, and unit of analysis). Type of PD: Delivery, duration, and content Reviewed Studies Regarding the delivery and duration, the majority of the reviewed PD studies were delivered during the summer months and consisted of workshops lasting from two days to five weeks. In addition, the majority of PD providers included year-long support for teachers to ensure ongoing engagement with the PD content and implementation in their classrooms. For example, Baxter et al. (2014) provided a two-day summer PD and workshops for six days during the school year. In addition, their participants engaged in two classroom observations for ten contact days over the year. Guzey et al. (2016) offered a three-week- long summer PD activity and continued to support teachers through monthly meetings throughout the year. Rich et al. (2017; 2018) did not hold a summer academy but provided teachers with weekly 45-minute workshops throughout the school year. Two reviewed programs (McFadden & Roehrig, 2017; Parker et al., 2015) employed one- to five-week-long workshops followed by classroom implementation during the academic year. Only one of the reviewed PD programs (Havice et al., 2018) offered a two-day workshop without any follow-up activity, and they discussed that their two-day-long PD was successful as there were successful changes in their participants’ understanding of integrated STEM education. Regarding the content and curriculum, all the reviewed PD programs aimed to address the existing grade- level curriculum and standards. This aspect of STEM- integrated PD literature agrees with the literature (e.g., Desimone, 2009), as PDs should be naturally linked to the curriculum and standards in place. In addition, as offered by the literature on effective PD, some PD programs included community-building activities such as field trips and family STEM activities. Some studies emphasized particular STEM domains to be integrated. For example, McFadden & Roehrig (2017) focused on the integration of elementary engineering into content areas and standards in place (math, science, and technology). Two of the PD programs were geared toward integrating two domains, mathematics and science (Baxter et al., 2014), or technology and mathematics (Wentworth & Monroe, 2011). In the PD workshops by Rich et al. (2017), teachers were introduced to the integration of engineering and computing units, and in their 2018's study, their participant teachers worked on incorporating engineering or computing into their teaching through lesson study. Parker et al. (2015) focused on fully integrating science with reading, math, and engineering. All studies encouraged integration to align with the existing grade-level curriculum and standards. The majority of the reviewed PD programs consisted of introducing example units and lessons to their participant teachers, such as Elementary is Engineering units, researchers-developed integrated STEM lesson plans (McFadden & Roehrig, 2017; Parker et al., 2015; Rich et al., 2017; Wentworth & Monroe, 2011). In Boice et al.'s study (2021), teachers felt more comfortable surrounded by support staff. When they worked with innovators outside the field of education and collaborated to develop integrated STEM lessons, they were inspired to prepare creative integrated STEM lesson plans. Three of the reviewed PD programs (Boice et al., 2021; Sias et al., 2017) ran special events such as field trips to local companies or engineering design facilities, engineering panels, STEM-related conferences, and family science activities. Overall, the reviewed programs provided some details about the content of the PD activities. However, the majority of the reviewed PD programs did not provide sufficient information about the aspects that could make critical differences in the implementation and consequences of PD programs, such as PD instructors' background, available resources (material and staff capacity), presence/types of instruction and activities during PD. Connections with the literature on effective PD The reviewed studies align with the literature on effective PD that recommends PD to be run over the course of time with an on-site support (e.g., Darling- Hammond & Richardson, 2009; Penuel et al., 2007). However, providing encompassing PD programs is related to the availability of human, physical and financial resources, and as observed in Havece et al.’s study (2018), short workshops might still enhance teachers’ understanding of STEM integration and support teachers to become comfortable with integrated STEM curriculum. All the reviewed PD programs aimed to address the existing grade-level curriculum and standards. This 377 Teacher Professional Development for STEM Integration in Elementary/Primary Schools: A Systematic Review / Boz aspect of STEM-integrated PD literature agrees with the literature (e.g., Desimone, 2009), as PDs should be naturally linked to the curriculum and standards in place. In addition, as offered by the literature on effective PD, some PD programs included community- building activities such as field trips and family STEM activities. The reviewed studies did not emphasize some design features of effective PD. For example, the following concerns were not located in the reviewed studies: how did the content present in PD? What types of inquiry prompts and tasks were presented? What was the level of modeling provided? What kind of opportunities were teachers granted for reflection on their work or their students' work? In particular, the reviewed studies do not explicitly discuss whether and how they employed active learning in their PD setting, an element frequently acknowledged as crucial for effective PD (Bergh et al., 2014). Unit of analysis Reviewed Studies Researchers utilized a variety of tools to evaluate the effectiveness of their PD programs, including analysis of lesson plans created by participant teachers, pre- post surveys, field notes, interviews, and teacher debriefs. However, only one study focused on evaluating program outcomes (Parker et al., 2015), while the others evaluated teacher outcomes, such as changes in beliefs, understandings, self-reported knowledge, perceptions, confidence, self-efficacy, and self-reported levels of practice/implementation. None of the reviewed studies collected student data to assess changes in knowledge, skills, or attitudes resulting from their teachers' participation in integrated STEM programs. Only Baxter et al. (2014) included student work in their PD to allow participant teachers to reflect on students' understanding of integrated science and mathematics concepts, but it was unclear whether participant teachers evaluated their students' work. Additionally, none of the studies included external evaluators to assess the effectiveness of their PD practices. Connections with the literature on effective PD The PD literature encourages the use of multiple tools and methods to evaluate changes in teachers' knowledge, skills, attitudes, classroom teaching, and student learning. The majority of the reviewed PD studies employed various data collection methods, such as field notes and teacher debriefs. However, as recommended in the literature on effective PD, rigorous methods are needed to examine the direct influence of PD on real-life classroom instruction and learning (e.g., Guskey, 2014). None of the reviewed studies examined student learning, and it is unclear whether the integrated STEM activities continued to be taught by teachers after support from the research team was discontinued. Furthermore, the long-term effects of PD on teacher practice were not clearly addressed in the studies. Design Reviewed Studies The majority of the studies discussed the importance of teacher collaboration in STEM-integrated PD. It was argued that having a supportive communal learning environment in the PD setting helped teachers develop confidence and comfort in implementing the targeted curriculum. Boice et al. (2021) argued that collaborative lesson planning was critical to ensure equal STEM integration and create a sense of community among teachers. Parker et al. (2015) emphasized that teachers benefitted from talking about their successes or challenges with the implementation of STEM integration with their colleagues who taught the same grade. The assessment was considered to be a challenging aspect of integrated STEM education. Guzey et al. (2016) recommended to support teachers in assessing student learning in integrated STEM lessons. Cook et al. (2021) claimed that in implementing integrated STEM lessons, their participant teachers used assessments that depended on one STEM subject. They recommended performance-based assessment types (in PD and classroom) that “include student choice on the product they produce or the process through which they showcase their content understanding” (p. 206). Studies focused on the elements of PD that help participants establish links between PD and their teaching and student learning. For example, noticing positive student reactions/responses to integrated STEM materials and lessons increased teacher willingness to dive deeper into STEM integration (Boice et al., 2021; Cook et al., 2020; Rich et al., 2017, 2018). Boice et al. (2021) proposed to present teachers with many hands-on activities to first-hand experience integrated STEM learning and even use these activities in their teaching of integrated STEM lessons. Upon their assessment of the lesson plans, Wentworth and Monroe also (2011) recommended that teachers can develop more complex integrated lesson plans if PD addresses how and where teachers can find and examine integrated STEM lesson examples. The reviewed studies included PD features that supported their participants’ learning and teaching of integrated STEM education. Modeling the STEM- integrated curriculum was discussed in the reviewed 378 June 2023, Volume 15, Issue 5, 371-382 articles as necessary for teachers to become familiar with the new content and pedagogy of integrated STEM education (Parker et al., 2015; Sias et al., 2016; Wentworth & Monroe, 2011). The authors recommended that the PD practitioners need to model the skills and practices that teachers are expected to demonstrate in PD, and teachers should be clear about what skills and knowledge teachers will gain at the end of PD. Sias et al. (2017) even claimed, “without experience or models of the implementation of the educational innovations, teachers are less likely to conceive of how they might integrate the innovations in their lessons” (p. 235). Studies discussed the importance of examining and presenting reflexive connections across STEM subject areas in PD settings (e.g., Baxter et al., 2014; Cook et al., 2020). They also suggested that PD that focuses on the natural relationship between mathematics and science will enhance teacher confidence and practice. Guzey et al. (2016) indicated that science teachers need more subject-matter knowledge to teach mathematics effectively, and teachers might mainly focus on engineering and science integration. Therefore, it is necessary for PD providers to provide opportunities for teachers to have the chance to increase their understanding of all the subject areas that will be integrated. Wentworth and Monroe (2011) similarly emphasized that PD that integrates mathematics and technology should focus on the use of technology in an integral way that “allows candidates to think more deeply about the mathematics in ways they could not without the technology” (p. 271). Some contextual factors were proposed to consider in organizing and implementing PDs. The provision of necessary physical resources (i.e., supplies, materials for engineering lessons, printouts, and teacher and student workbooks) enabled teachers to incorporate computing and engineering in their classes (e.g., Rich et al., 2017, 2018). Boice et al. (2021) indicated that "ongoing financial and material support allowed teachers to engage students in new and otherwise impossible ways" (p. 17). Being provided with support staff (researchers, PD practitioners, or coaches) or having administration/ peer support at school also assisted teachers in teaching integrated STEM (Boice et al., 2021; Rich et al., 2018). Time constraints in developing integrated lessons, unanticipated changes during trainings (participant turnovers, changing expectations), and lack of shared vision among the actors of PD implementation inhibited teachers’ efforts to integrate STEM in implementation (Guzey et al., 2016; Rich et al., 2018). Parker et al. (2015) discussed that their participant teachers expected coaches to have a strong content area and district-level contextual knowledge. In implementing an integrated STEM lesson, teachers surely needed to believe that integration of STEM lessons would help them reach their instructional goals and state standards and that the provided/ presented materials would align well with their classroom realities. Two studies concluded that teachers were not ready to think in nontraditional ways as integrated curriculum and teaching just started to be included in elementary schools (McFadden & Roehrig, 2017; Sias et al., 2017). In addition, working in large schools was correlated with increased engineering integration activity per teacher. Only one study (Baxter et al., 2014) included information about the nature of integration for a successful PD: opportunities of infusion and transfer in connecting mathematics and science. Connections with the literature on effective PD Some implications and recommendations offered in the reviewed studies agreed with the literature on effective PD. For example, modeling the STEM- integrated curriculum was discussed as crucial in the reviewed studies. Collaborative practices and peer/ administration support were also discussed as critical enablers of STEM-integrated curriculum and lesson implementation in most of the reviewed studies. The PD literature also discusses the importance of collaborative and collegial learning environments as conducive to creating communities of practice and increasing the change process beyond individual classrooms (e.g., Bates & Morgan, 2018). The literature on effective PD recommends that PD should have a specific subject focus (e.g., Desimone et al., 2013). Some studies focused on the integration of certain STEM subject areas, such as mathematics and science, and those studies suggested that it would be important to focus on the natural connections across STEM subject areas so that elementary/primary teachers increase their cross-curricular subject matter knowledge. Some studies focused on STEM without any concerns to increase teachers' knowledge of particular subjects (e.g., Boice et al., 2020). From the reviewed studies, it is not possible to conclude whether focusing on individual subject areas or STEM overall leads to different results in teacher knowledge and skills in integrated STEM education. Cochran-Smith and Demers (2018) also underline the importance of "the recognition that both prospective and experienced teachers (like all learners) brought prior knowledge and experience to all new learning situations, which are social and contextually specific" 379 Teacher Professional Development for STEM Integration in Elementary/Primary Schools: A Systematic Review / Boz (p.1011). Therefore, context plays an important role in PD practitioners/researchers' successes and challenges in the field. However, only a few studies reviewed (i.e., Parker et al., 2015) included detailed information regarding teacher, school, and district backgrounds. In alignment with the literature on effective PD, the reviewed studies showed that examining intended change in student work in PD and positive student reactions/responses to integrated STEM materials and lessons increased teacher willingness and motivation to engage with STEM integration in classrooms (e.g., Rich et al., 2017, 2018). Some practices that were raised in the reviewed studies were interesting. The assessment was one area that was discussed as important to focus on in integrated STEM PD. The reviewed studies recommended that teachers are challenged with developing assessment practices that will equally assess student learning in more than one subject area in integrated STEM lessons, and they should be supported in terms of how to assess student learning in integrated STEM classes. Human and material resources made the difference in the way teachers employed integrated STEM education in their practice. The provision of materials, supplies and support from the school community (i.e., administrators) enabled teachers to incorporate intended PD knowledge and skills in their practice. In particular, it requires more time for teachers to plan and develop integrated STEM lessons; therefore, time constraints can significantly influence teachers' efforts to teach integrated STEM lessons. Implications and Conclusion This systematic review identified several key implications for improving professional development (PD) practices and advancing research on integrated STEM education. First, it is important to recognize that some teachers may lack confidence or expertise in certain subject areas, which can hinder their ability to create balanced and effective integrated STEM lessons. To address this issue, PD providers can offer targeted workshops to enhance subject matter knowledge and encourage collaborative planning with content experts. Additionally, PD developers should model active learning strategies and provide teachers with a variety of examples of effective integrated STEM curriculum for primary/elementary classrooms, including lessons, videos, and student artifacts. It is also crucial to ensure ongoing support and highlight successful practices for teachers implementing integrated STEM activities. Furthermore, PD programs should align with school or district-level knowledge and provide opportunities for teachers to receive continuous support from administrators and parents. Field trips to local businesses and community-wide seminars and workshops can also be organized to establish a shared vision and sense of community around integrated STEM education. PD providers should also emphasize assessment practices that equally assess student learning and understanding across each subject area integrated into STEM lessons. Finally, it is important to support teachers in planning and implementing integrated STEM lessons. One way to achieve this is by involving support staff from integrated disciplines such as engineering, who can assist teachers in developing creative ideas for lesson planning and delivery. To further advance research on effective STEM-integrated PD, future studies should explicitly describe the PD process, including details about teacher, instructor, school, and community backgrounds, available resources, and types of instruction and tasks. Despite the insights gained from this systematic review, there are several limitations to consider. The review was based on a relatively small sample of 11 studies; therefore, more research is needed to establish consensus on effective components of STEM- integrated PD. Nevertheless, this review provides a solid foundation for future PD development and evaluation in STEM integration and can inform the development of more comprehensive research designs to explore the important characteristics of effective teacher professional development in integrated STEM education. References References marked with an asterisk indicate studies included in the systematic review Altun, S., Yabas, D., & Bal-Nayman, H. (2021). Teachers’ experiences on instructional design based professional development: A narrative inquiry. International Electronic Journal of Elementary Education, 14(1), 35–50. Avalos, B. (2011). Teacher professional development in teaching and teacher education over ten years. Teaching and Teacher Education, 27(1), 10–20. Bates. C. C., & Morgan, D. N. (2018). Seven elements of effective professional development. The Reading Teacher, 71(5), 623–626. https:// doi:10.1002/trtr.1674 *Baxter, J. A., Ruzicka, A., Beghetto, R. A., & Livelybrooks, D. (2014). Professional development strategically connecting mathematics and science: The impact on teachers' confidence and practice. School Science and Mathematics, 114(3), 102– 113. https://doi:10.1111/ssm.12060 380 June 2023, Volume 15, Issue 5, 371-382 Van den Bergh, L., Ros, A., & Beijaard, D. (2014). Improving teacher feedback during active learning: Effects of a professional development program. American Educational Research Journal, 51(4), 772–809. https://doi. org/10.3102/0002831214531322 Birman, B., Desimone, L., Porter, A., & Garet, M. (2000). Designing professional development that works. Educational Leadership,57(8), 28–33. Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33(8), 3–15. https:// doi:10.3102/0013189X033008003 *Boice, K. L., Jackson, J. J., Alemdar, M., Rao, A. E., Grossman, S., & Usselam, M. (2021). Supporting teachers on their STEAM journey: A collaborative STEAM teacher training program. Education Sciences, 11, 105. https://doi.org/10.3390/ educsci11030105 Cochran-Smith, M., & Demers, K.E. (2008). How do we know what we know? Research and teacher education. In M. Cochran-Smith, S. Feiman- Nemser, & D. J. McIntyre (Eds.), Handbook of research on teacher education: Enduring questions in changing contexts (3rd ed., pp. 1009–1016). Routledge. Cochran-Smith, M., & Zeichner, K. (2005). Studying teacher education: The report of the aera panel on research and teacher education. Lawrence Erlbaum Associates, Inc. *Cook, K., Bush, S., Cox Jr, R., & Edelen, D. (2020). Development of elementary teachers' science, technology, engineering, arts, and mathematics planning practices. School Science and Mathematics, 120(4), 197–208. https://doi:10.1111/ ssm.12400 Darling-Hammond, L. (2000). How teacher education matters. Journal of Teacher Education, 51(3), 166–173. https://doi:10.1177/002248710005100300 2 Darling-Hammond, L. (2006). Constructing 21st- century teacher education. Journal of Teacher Education, 57(3), 300–314. https:// doi:10.1177/0022487105285962 Darling-Hammond, L., & Richardson, N. (2009). Research review/teacher learning: What matters? Educational Leadership, 66(5), 46–53. Darling-Hammond, L., Hyler, M. E., & Gardner, M. (2017). Effective teacher professional development. Learning Policy Institute. Desimone, L. (2009). Improving impact studies of teachers' professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181–199. https:// doi:10.3102/0013189X08331140 Desimone, L. M., & Garet, M. S. (2015). Best practices in teachers’ professional development in the United States. Psychology, Society, & Education, 7(3), 252–263. https://doi:10.25115/psye.v7i3.515 Desimone, L.M., & Pak, K. (2017). Instructional coaching as high-quality professional development. Theory into Practice, 56(1), 3–12. Desimone, L., Porter, A.C., Garet, M.S., & Yoon, K. S., & Birman, B. F. (2002). Effects of professional development on teachers' instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis, 24(2), 81–112. https://doi:10.3102/01623737024002081 Desimone, L., Smith, T.M., & Phillips, K. (2013). Linking student achievement growth to professional development participation and changes in instruction: A longitudinal study elementary students and teachers in Title I schools. Teachers College Record, 115(5), 1–46. Desimone, L. M., Smith, T. M., Hayes, S., & Frisvold, D. (2005). Beyond accountability and average math scores: Relating multiple state education policy attributes to changes in student achievement in procedural knowledge, conceptual understanding and problem solving in mathematics. Educational Measurement: Issues and Practice, 24(A), 5–18. English, L. (2016). STEM education K-12: Perspectives on integration. International Journal of STEM Education, 3(1), 1–8. https://doi:10.1186/s40594- 016-0036-1 Fullan, M., & Milles, M. (1992). Getting reform right: What works and what doesn’t. Phi Delta Kappan, 73, 744–752. Garet, M., Porter, A., Desimone, L., Birman, B., & Yoon, K. (2001). What makes professional development effective? American Educational Research Journal, 38(4), 915–945. https:// doi:10.3102/00028312038004915 Gupta, A., & Guang-Lea, Lee. (2020). The effects of a site-based teacher professional development program on student learning. International Electronic Journal of Elementary Education, 12(5), 417–428. https://doi.org/10.26822/ iejee.2020562132 381 Teacher Professional Development for STEM Integration in Elementary/Primary Schools: A Systematic Review / Boz Guskey, T. R. (2000). Evaluating professional development. Thousand Oaks, CA: Corwin. Guskey, T. R. (2002). Professional development and teacher change. Teachers and Teaching, 8(3), 381–391. https://doi:10.1080/135406002100000512 Guskey, T. R. (2014). Evaluating professional learning. In Billett, S., Harteis, C., & Gruber, H. (Eds.). International handbook of research in professional and practice-based learning. (pp. 1215–1235). Springer. *Guzey, S. S., Moore, T. J., & Harwell, M. (2016). Building up STEM: An analysis of teacher-developed engineering design-based STEM integration curricular materials. Journal of Pre-College Engineering Education Research (J-PEER), 6(1), 11–29. *Havice, W., Havice, P., Waugaman, C., & Walker, K. (2018). Evaluating the effectiveness of integrative STEM education: Teacher and administrator professional development. Journal of Technology Education, 29(2), 73–90. Joffe, H. (2011). Thematic analysis. In D. Harper & A. R. Thompson (Eds.), Qualitative methods in mental health and psychotherapy: A guide for students and practitioners (pp. 209-223). Wiley. Knapp, M.S. (2003). Professional development as a policy pathway. Review of Research in Education, 27, 109–157. Kelley, T., & Knowles, J. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education, 3(1), 1–11. https:// doi:10.1186/s40594-016-0046-z Lawless, K.A., & Pellegrino, J.W. (2007). Professional development in integrating technology into teaching and learning: Knowns, unknowns, and ways to pursue better questions and answers. Review of Educational Research, 77(4), 575–614. https://doi:10.3102/0034654307309921 Little, J. W. (1987). Teachers as colleagues. In V. Richardson-Koehler (Ed.), Educators' handbook: A research perspective (pp. 491–518). Longman. *McFadden, J., & Roehrig, G. (2017). Exploring teacher design team endeavors while creating an elementary-focused STEM-integrated curriculum. International Journal of STEM Education, 4(1), 1–22. https://doi:10.1186/s40594- 017-0084-1 Nadelson, L. S., & Seifert, A. L. (2017). Integrated STEM defined: Contexts, challenges, and the future. The Journal of Educational Research, 110(3), 221–223. https://doi:10.1080/00220671.2017.12897 75 OECD (2019). TALIS 2018 Results (Volume I): Teachers and school leaders as lifelong learners. OECD Publishing. https://doi:10.1787/1d0bc92a-en Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T.C., & Mulrow, C. D., et al. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372(71). https://doi:10.1136/bmj.n71 *Parker, C., Abel, Y., & Denisova, E. (2015). Urban elementary STEM initiative. School Science and Mathematics, 115(6), 292–301. https://doi:10.1111/ssm.12133 Parsad, B., Lewis, L., Farris, E., & Greene, B. (2001). Teacher preparation and professional Development (NCES 2001-088). U.S. Department of Education, National Center for Education Statistics. Pearson, G. (2017). National academies piece on integrated STEM. The Journal of Educational Research, 110(3), 224–226. https://doi:10.1080/00 220671.2017.1289781 Penuel, W. R., Fishman, B. J., & Yamaguchi, R. (2007). What makes professional development effective? Strategies that foster curriculum implementation. American Educational Research Journal, 44(4), 921–958. https:// doi:10.3102/0002831207308221 Popova, A., Evans, D., Breeding, M. E., & Arancibia, V. (2018). Teacher professional development around the World: The gap between evidence and practice (WPS 8572). World Bank Group. http://documents.worldbank.org /curated/ en/349051535637296801/Teacher-Professional- Development-around-the-World-The-Gap- between-Evidence-and-Practice *Rich, P. J., Jones, B., Belikov, O., Yoshikawa, E., & Perkins, M. (2017). Computing and engineering in elementary school: The effect of year-long training on elementary teacher self-efficacy and beliefs about teaching computing and engineering. International Journal of Computer Science Education in Schools, 1(1), 1–20. https:// files.eric.ed.gov/fulltext/ED572692.pdf 382 June 2023, Volume 15, Issue 5, 371-382 *Rich, P.J., Belikov, O., Yoshikawa, E., & Perkins, M. (2018). Enablers and inhibitors to integrating computing and engineering lessons in elementary education. Journal ofTechnology and Teacher Education, 26(3), 437–469. https:// www.learntechlib.org/primary/p/181979/ *Sias, C. M., Nadelson, L. S., Juth, S. M., & Seifert, A. L. (2017). The best laid plans: Educational innovation in elementary teacher generated integrated STEM lesson plans. The Journal of Educational Research, 110(3), 227–238. https://do i:10.1080/00220671.2016.1253539 Smith, K. (2017). Teachers as self-directed learners: Active positioning through professional learning. Springer. https://doi:10.1007/978-981-10-3587-6 Villegas-Reimers, E. (2003). Teacher professional development: An international review of the literature. UNESCO International Institute for Educational Planning. Xiao, Y., & Watson, M. (2019). Guidance on conducting a systematic literature review. Journal of Planning Education and Research, 39(1), 93–112. https://doi.org/10.1177/0739456X17723971 Shin, Y.-S., & Jun, J.S. (2019). The hierarchical effects of individual and organizational variables on elementary school teachers’ lifelong learning competence. International Electronic Journal of Elementary Education, 12(2), 205–212. https:// doi.org/10.26822/iejee.2019257668 *Wentworth, N., & Monroe, E. E. (2011). Inquiry-based lessons that integrate technology: Their development and evaluation in elementary mathematics teacher education. Computers in the Schools, 28(4), 263–277. https://doi:10.1080/07 380569.2011.620938