An Integrative Approach to Professional Development to Support College- and Career- Readiness Standards Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 8/5/2019 Facebook: /EPAAA Revisions received: 2/17/2020 Twitter: @epaa_aape Accepted: 2/28/2020 education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 28 Number 111 August 3, 2020 ISSN 1068-2341 An Integrative Approach to Professional Development to Support College- and Career- Readiness Standards1 Katie Pak University of Pennsylvania Laura M. Desimone University of Delaware & Arianna Parsons University of Pennsylvania United States Citation: Pak, K., Desimone, L. M., & Parsons, A. (2020). An integrative approach to professional development to support college- and career- readiness standards. Education Policy Analysis Archives, 28(111). https://doi.org/10.14507/epaa.28.4970 Abstract: Though scholars agree that professional development (PD) is a key mechanism for implementing education policies that call for teacher change, and that PD generally needs to be content-focused, active, collaborative, coherent, and sustained, the application of this framework has yielded mixed results. In this qualitative study, we employed 1 This research was supported in part by Grant R305C150007 from the Institute of Education Sciences in the U.S. Department of Education to the University of Pennsylvania. The content is solely the responsibility of the author and does not necessarily represent the official views of the Institute of Education Sciences or the U.S. Department of Education. http://epaa.asu.edu/ojs/ https://doi.org/10.14507/epaa.28.4970 Education Policy Analysis Archives Vol. 28 No. 111 2 structured interviewing methods to explore how district leaders across five states are implementing college- and career- readiness (CCR) standards across the United States by creatively adapting and integrating the features of this PD framework in order to meet the demands of this mandated educational policy. We illustrate a revised model for how 70 district officials are conceptualizing these features of PD to support CCR standards-based learning. Keywords: College- and career- readiness standards; standards-based reform; professional development; district leadership Un enfoque integrador para el desarrollo profesional para apoyar los estándares de college and career readiness Resumen: Aunque los académicos coinciden en que el desarrollo profesional (PD) es un mecanismo clave para implementar políticas educativas que exigen un cambio de docentes, y que la PD generalmente debe estar centrada en el contenido, activa, colaborativa, coherente y sostenida, la aplicación de este marco ha dado resultados mixtos. En este estudio cualitativo, empleamos métodos de entrevista estructurados para explorar cómo los líderes de distrito en cinco estados están implementando estándares de college and career readiness (CCR) en los Estados Unidos mediante la adaptación e integración creativas de las características de este marco de DP para cumplir con el demandas de esta política educativa obligatoria. Ilustramos un modelo revisado de cómo 70 funcionarios del distrito están conceptualizando estas características de PD para apoyar el aprendizaje basado en estándares CCR. Palabras-clave: estándares de college and career readiness; reforma basada en estándares; desarrollo profesional; liderazgo del distrito Uma abordagem integrativa ao desenvolvimento profissional para apoiar os padrões de college and career readiness Resumo: Embora os estudiosos concordem que o desenvolvimento profissional (P D) é um mecanismo essencial para a implementação de políticas educacionais que exigem mudança de professor, e que a PD geralmente precisa ser focada em conteúdo, ativa, colaborativa, coerente e sustentada, a aplicação dessa estrutura Você produziu resultados mistos. Neste estudo qualitativo, empregamos métodos estruturado s de entrevista para explorar como os líderes distritais de cinco estados estão implementando os padrões de college and career readiness (CCR) nos Estados Unidos, adaptando e integrando criativamente os recursos dessa estrutura de PD para atender às exigên cias dessa política educacional obrigatória. Ilustramos um modelo revisado de como 70 funcionários distritais estão conceituando esses recursos do PD para apoiar o aprendizado baseado nos padrões da CCR. Palavras-chave: padrões de college and career readiness; reforma baseada em padrões; desenvolvimento profissional; liderança distrital An Integrative Approach to Professional Development 3 Introduction Policymakers in the United States have long held teachers accountable to the goals of the standards-based reform movement, which argues that rigorous academic standards, high quality curriculum and instruction aligned to these standards, and accountability systems that track district and school performance all interdependently boost student achievement (Cohen & Mehta, 2017). The most recent iteration of the standards-based reform movement has produced college- and career- readiness (CCR) standards in English language arts (ELA) and mathematics. CCR standards, currently present in every state plus the District of Columbia, provide guidelines for challenging K–12 learning that is intended to prepare students for colleges and 21st -century careers. Yet CCR standards alone cannot accelerate student outcomes. Teachers must receive professional development (PD) that supports their adaptations to the rigorous teaching and learning expectations embedded in the standards: a wide range of disciplinary content expertise , deep student understanding of key concepts, and critical student engagement in the core subject areas (Floden et al., 2017). This PD is especially critical given that many of these teachers’ pre - service programs, K-12 schools, and/or college programs were not designed with these current CCR standards in mind, suggesting that teachers are now expected to implement rigorous pedagogy that they themselves might not have experienced. The importance of PD is further bolstered in the Every Student Succeeds Act (ESSA), passed by the United States Congress in 2015. ESSA considers PD to be high quality when it is sustained, intensive, collaborative, inclusive of other teachers, job -embedded, data-driven, and grounded in evidence-based practices. ESSA also raises the bar for the general education of students with disabilities (SWDs) and English learners (ELs), as the legislation limits the percentage of SWDs eligible for the alternate assessment to 1% of the total student population, and it moves accountability provisions for ELs from Title III to Title I. These shifts indicate that all teachers are held increasingly accountable to the performance of SWDs and ELs on state assessments aligned to the CCR standards, suggesting a greater need for PD that addresses these demands. ESSA additionally emphasizes the need to professionally develop school principals as instructional leaders, a focus that had not previously been articulated in federal educational legislation (Young et al., 2017). PD is often relied on as a critical lever for implementing educational policies that call for changes to professional practice, as well as for teachers’ commitment to these changes (Carney et al., 2016; Lobman & Ryan, 2008; Smith & Rowley, 2005; Youngs, 2001). Many educational scholars have coalesced around five features of PD —content focus, active learning, collective participation, duration, and coherence (Desimone, 2009)—as the conditions that tend to foster the growth of teacher knowledge, skills, and beliefs about their practice. But the actual effects of these features on student learning have yielded mixed results (see Garet et al., 2011; Fischer et al., 2018; Yoon et al., 2007). In response, researchers have refined the features of PD in different ways—by adding in a focus on curriculum (e.g., Penuel et al. 2011); by including modeling, coaching, and feedback (e.g., Darling-Hammond et al., 2017); or by exploring the contexts in which certain features are more or less effective (e.g., Kennedy, 2016). This work has contributed to a more robust conception of teacher PD, though enactment of these f eatures still has not consistently produced positive teacher and student outcomes. These findings speak to the overall challenge of translating theory into practice when implementing educational policies. It is therefore incumbent upon researchers to cont inuously adapt theoretical frameworks to reflect emerging insights about the complexities of Education Policy Analysis Archives Vol. 28 No. 111 4 implementing policies through PD mechanisms. We argue that the process for this theoretical adaptation involves the following phases: (1) engaging with educational leaders to understand how they are identifying and implementing effective features of PD in service of a specific policy reform; (2) examining if and how organizational knowledge, skills, and beliefs (i.e., organizational capacity) improved in the intended direction; and (3) exploring the impact on teacher change and student learning. Our study addresses the first phase, where we analyze how educational leaders perceive their implementation of teacher PD in order to enact standards - based policies. More specifically, we examine the perceptions of school district leaders (e.g., superintendents, assistant superintendents, directors of curriculum, directors of PD, directors of student support), as they shoulder the daunting responsibility of orchestrating syst em-wide strategies for implementing state policy mandates (Durand et al., 2016; Johnson et al., 2015). The new rigors of CCR standards-based policy expectations and pressures push district le aders to think creatively about the features of their PD and methods for shifting teacher practice. ESSA’s focus on leadership development and the increasing levels of accountability in educating SWDs and ELs in CCR standards-aligned classrooms also require district leaders to implement PD as a mechanism for building organizational capacity throughout a district system rather than just focusing on the instructional practices of general education teachers. Despite these policy demands on district leaders, m ost PD research examines interventions as initiatives or programs developed outside the central office (Crowley, 2017). Recent research documents the tendency for district PD providers to focus more on the logistics of PD than its substance (Morel & Coburn , 2018), concluding that district leaders are less likely to influence what information teachers receive about the standards. Yet these logistical decisions directly impact whether teachers have access to content -focused, collaborative, active, sustained, and coherent PD, and there is little documentation of how and why district leaders make these logistical decisions in light of the field’s mixed evidence on the effectiveness of certain PD features. Additionally, scholarship on PD for general education teachers, special education teachers, teachers of English learners, and principals tends to be conducted in silos, leading to a lack of research on how district leaders provide PD across this interdependent system of education providers. There is therefore a need for literature that showcases (a) how district leaders are strategically leveraging their PD efforts for general education teachers, teachers of SWDs and ELs, and principals to support classroom -level implementation of the CCR standards; (b) how these district decisions reveal new and potentially influential interactions among the features of high -quality PD in the context of the CCR standards; and (c) how they respond to the contextual challenges that often permeate PD environments. We use qualitative data from 2016 to 2017 to examine how district leaders report their PD systems and strategies for different teacher types and school leaders in order to meet the needs of the CCR standards policy, and to what extent these district decisions build on our understanding of the empirically supported features of effective PD identified in the past several decades. Our findings help inform the conceptual model we present in this paper, which illustrates how district leaders in our study are perceiving their st rategic integration of different PD features to cultivate a system that supports teacher learning in the context of the CCR standards. We hope that the PD model we present in this study aids practitioners and researchers in better understanding the ways th at some educational leaders in the United States approach PD in support of a system-wide instructional shift anchored in federal and state CCR policies. Our intention is not to purport a singular conceptual model generalizable across district An Integrative Approach to Professional Development 5 contexts. Instead, we demonstrate one model that emerged from our interviews of district administrators in 24 districts across five states, which we believe can be adapted to other local contexts. Conceptual Framework There is considerable evidence that for PD to be effective in improving teacher practice and student learning, at least five features need to be in place: (a) content focus: activities that are focused on subject matter content and how students learn that content; (b) active learning: opportunities for teachers to observe, receive feedback, analyze student work, or make presentations; (c) coherence: content, goals, and activities that are consistent with the school curriculum and goals, teacher knowledge and beliefs, the needs of students, and school, di strict, and state reforms and policies; (d) sustained duration: PD activities that are ongoing throughout the school year and include 20 hours or more of contact time; and (e) collective participation: groups of teachers from the same grade, subject, or school participate in PD activities together to build a learning community. Evidence supporting these aspects of professional development comes from cross - sectional studies (Garet et al., 2001), longitudinal studies (Desimone et al., 2002; Desimone, et al., 2013), and literature reviews of qualitative and quasi-experimental studies (Fischer et al., 2018; Kennedy, 2016; Kraft et al., 2016). Studies using this framework have shown some positive effects on teachers and students (e.g., Gersten et al., 2010; Penuel et al., 2011), but also documented limited or no effects (e.g., Garet et al., 2008; Garet et al., 2011), or effects that depend on how features interacted with other elements of the PD program (Kennedy, 2016). The Complexity of Translating a PD Conceptual Framework to Practice Recent work has demonstrated how the dynamics of everyday realit ies in schools necessitate more nuanced understanding of the characteristics of PD and how they can be leveraged to support teachers. Thus, refinements of this PD framework are offered below, where we illustrate how scholars observe these features interact ing in real-world settings. Collective participation is effective when teachers engage in productive discourse and activities that allow for teacher reflection and resource sharing, which can result in higher levels of teacher efficacy and adaptive expertise (Carney et al., 2016; DeLuca et al., 2017; Durksen et al., 2017; Ronfeldt et al., 2015; Smith & Rowley, 2005; Von Esch & Kavanagh, 2018). The emphasis on productive discourse is important given that not all collaboration yields substantive conversations about teacher practice (Horn et al., 2017) or explicit connections to a school’s instructional agenda (Stosich et al., 2018). Further, scholars have found that collaborative inquiries are most effective when they are grounded in opportunities to interact with the actual curriculum or assessments being used in classroom settings (Akiba & Wilkinson, 2016; Green et al. 2013; Hodges & Jong, 2014; Stosich, 2016), or when they include teachers from the same schools building their capacities to implement instructional changes together (Youngs, 2001). Collective participation in the form of collaboration between general education teachers and “intervention” teachers (e.g., special teachers, EL teachers) and school leaders is also proving to be a fruitful endeavor, though this literature is more limited in scope. Griffin and colleagues (2018) suggested that collaborative inquiry between general a nd special education math teachers in online forums increased their beliefs about student learning and enhanced their understanding of mathematics content. Babinski and colleagues (2018) observed how the school teams of classroom and English as a Second Language teachers worked together to align content Education Policy Analysis Archives Vol. 28 No. 111 6 and instructional strategies for their ELs, leading to more impactful pedagogy for ELs compared to teachers in the control group. Penner-Williams et al. (2017) also pointed to PLCs as a key PD mechanism for learning strategies specific to ELs. Stosich et al. (2018) raised the potential benefits of principals and teachers engaging in joint -inquiry around the implementation of CCR math standards, and the challenges of doing so in environments that expect princi pals to demonstrate expertise rather than a learning stance. The paucity of the literature focused on collaborative PD for intervention teachers may very well be due to the lack of PD opportunities that show these teachers how to scaffold their instruction in alignment with the content of the standards (Murphy & Haller, 2015; Von Esch & Kavanagh, 2018). There is also limited literature on the inclusion of principals in PD for teachers, despite the value of including principals to help clarify the goals of PD and to encourage teachers to implement the learnings from the PD (Binkorst et al., 2018; Buttram & Farley-Ripple, 2016; Floden et al., 2017). To address these gaps, we examine how districts in our study approach the inclusion of intervention teachers and principals in collective PD experiences. Another common feature of PD, content focus, is often discussed in tandem with coherence, active learning, and collective participation. Content-focused PD alone does not necessarily affect student learning; instead, it proves to be effective when it coherently adapts to local needs and connects to teachers’ daily activities (Allen & Penuel, 2015; Kennedy, 2016; Koellner, & Jacobs, 2015), when it invites teachers to actively unpack assessment systems aligned to the standards (Popp & Goldman, 2016) or make decisions around curriculum design (Penuel et al., 2011), and when it includes opportunities for collaborative knowledge sharing (Griffin et al., 2018). The observation that math teachers who exhibit less anxiety around the content tend to influence higher gains in student achievement (see Kukata et al., 2017) suggests the importance of content-focused PD in deepening teachers’ confidence in their subject areas, as long as the PD contains elements of these other features. These studies do not distinguish, however, whether specific groups of teachers (e.g., general education teachers, intervention teachers) receive a higher volume of content PD compared to other groups in the same district, and if so, why these imbalances exist. In our study, we investigate why and how these content PD decisions are made in the context of diverse school-based stakeholders’ needs. Duration, another prominent feature of PD given the widely accepted notion that one- off PD workshops are minimally effective, is insufficient on its own. However, when active learning is coherently integrated into this extended learning time, the PD is found to be more effective. Ongoing opportunities for teachers to shape their own professional learning goals, learn and practice classroom strategies, engage with instructional tools, interact with coaches who model and then provide feedback on pedagogical moves (Binkhorst et al., 2018; Kennedy, 2016; Kraft et al., 2016; Sailors & Price, 2015; Supovitz & Mayer, 2000) all help to maximize the utility of PD contact hours. Furthermore, in a study of PD spillover effects, Sun and colleagues (2013) found that teachers were more likely to effectively assist their colleagues when they participated in PD of longer duration with active learning strategies related to writing instruction. Duration is a challenge when teachers have limited time built into their schedules for PD, which in turn gives teachers little time to process the coherence of newly introduced standards, aligned curriculum, or other instructional initiatives (Allen & Penuel, 2015). Coherence is also a challenge when district leaders do not play a large role in designing and/ or funding PD to ensure alignment with the overall instructional direction of the school system and with the individual goals of school-based staff (Durand et al., 2016; Johnson et al., 2015). An Integrative Approach to Professional Development 7 District leaders have the ability to shape the conditions for PD and determine the types of professional learning that teachers and principals experience (Akiba & Wilkinson, 2016; Desimone & Garet, 2015; Morel & Coburn, 2018), yet their decision -making is rarely included in PD literature (Whitworth & Chiu, 2015). We therefore pay special attention to providing insights about district leaders’ strategies for addressing duration and coherence of their teachers’ PD experiences. Our brief review of the literature highlights the complexity of translating a conceptual framework into effective PD in practice, given the myriad groups of teachers that influence the uptake of standards-based reform, and the ways these features often overlap with each other. By grounding our study in this conceptual framework, we are able to draw insights about how schools are enacting, combining, and organizing features of PD in ways intended to facilitate professional learning efforts, which in turn leads us to offer a more complex, interactive conceptual model to illustrate how features of PD might interact on the ground to leverage teacher learning in today’s standards-based environment. Methods Our analysis draws on data from case studies of districts in five states —California (CA), Texas (TX), Kentucky (KY), Ohio (OH), and Massachusett s (MA). We collected the data through structured interviews with district officials in each of these five states. In each state, we requested interviews with four to six district officials with knowledge of the district’s approach to standards-based reform through the lenses of curriculum, PD, assessment and accountability, and specialized instruction for SWDs and ELs. We initiated the request with either the Superintendent or the Superintendent’s designee (such as the Director of Teaching and Learning), who then connected us with other district officials to interview. We ultimately recruited 70 district officials in 24 districts across the five states. The interviews were conducted in the spring, summer, and fall of 2016 and 2017. The case study districts were part of a larger study of the implementation of CCR standards; they were selected from a state-representative sample of districts that participated in the survey portion of the larger study. For the purpose of this study of the implementation of PD in service of standards-based reform, we chose districts for variation on urbanicity (we include a rural, suburban, and urban district in each state), choosing districts with a sufficient number of SWDs and ELs, given the focus of the work on understanding standards implementation and supports for all teachers and students, including special populations. The structured interview protocol we developed covered questions related to the adoption and implementation of CCR standards, including questions about PD, curriculum, assessments, district context, and overall successes and challenges of standards -based reform. For each question, we included prompts related to PD supports for SWD and EL teachers, as well as for school leaders. After completion, the intervie ws were transcribed and then coded deductively using the five features of effective PD. In addition, other relevant codes related to key reform areas (e.g., SWDs, ELs, curriculum, leadership, assessment) were included both as they related to the study’s focus on standards implementation and as they emerged in the data. In order to develop inter-rater reliability, we engaged in paired coding and a process of dialogic engagement, which Ravitch and Carl (2016) defined as an ongoing and collaborative process of dialogue among a research team. After each cycle of coding, the research team discussed emergent themes pertaining to how the five features were manifesting in current models of district PD, and whether these themes were linked to other potentially influe ntial characteristics Education Policy Analysis Archives Vol. 28 No. 111 8 such as district urbanicity. This iterative coding and dialogic engagement process resulted in multiple re-readings of the data and, finally, a convergence on three major findings, which we present below. Findings Even though the movement to adopt CCR standards across the United States can be traced back to 2009, when the Common Core State Standards were first released to the public, district leaders in 2016 and 2017 were still reporting how they needed to help their principals and teachers understand fundamental content and instructional demands of their state’s ELA and math standards. Table 1 shows more specifically when each of the states in this study adopted their CCR standards in ELA and math in order to situate district leade rs’ contexts for interacting with state standards policies. Table 1 Standards Adoption and Revision Timeline Name of Standards; Adoption Year Revised California Common Core State Standards: 2010 ELA and math standards were revised in 2013 Kentucky Kentucky Academic Standards; 2010 The standards were under review in 2017 during the time of these interviews Massachusetts Massachusetts Curriculum Frameworks; 2010 ELA and math standards were revised in 2017 Ohio Ohio’s Learning Standards; 2010 ELA and math standards were revised in 2017 Texas Texas Essential Knowledge and Skills; 2008 Math standards were revised in 2012; ELA and math were revised again in 2017 As stated earlier, the role of district leaders is to help translate policy to practice by facilitating the conditions for teachers to learn how best to implement mandated state standards in ELA and math. District leaders agreed that the state’s new CCR standards called for significant shifts in teacher practice, as the ELA standards “ask kids to read at high levels, higher than what we have previously asked kids to read at before” while the math standards ask them to “think like mathematicians” and engage in more authentic and conceptual problem -solving tasks than ever before (Interview 24, KY). Thus, in order to effectively implement their state’s policy requiring the integration of these new ELA and math expectations into classroom instruction, district leaders are continuously relying on PD to he lp teachers understand and apply the instructional shifts. An Integrative Approach to Professional Development 9 Across all of the states in our study, district leaders describe how they incorporate the features of high-quality PD in integrative ways to create stronger learning experiences for teachers implementing these standards. Our interviews unveiled three major themes related to the ways that some district leaders are starting to reconceptualize PD to serve this purpose. First, district leaders are broadening the traditional notion of collective partic ipation by including intervention teachers in the same PD experience as general teachers, and by providing PD opportunities on the same topics for school leaders, which serves to improve coherence and build content knowledge across a policy system. This suggests a more comprehensive landscape of professional learning than has been reflected in previous literature. Second, we are seeing active learning being used as a mechanism for creating collaborative PD opportunities grounded in the state’s content standards. Lastly, our data showed an intentional interaction between duration and coherence—what we call sustained coherence—which indicates that district administrators are attempting to make teacher PD learning consistently oriented towards the policy by relying on instructional coaches and district-facilitated professional learning communities (PLCs). A New Kind of Collective Participation In an effort to expand the traditional notion of collective PD, districts are designing opportunities not only for general education teachers to learn together but also for principals, special education teachers, and teachers of English language learners to partake in the same experiences. District administrators describe the inclusion of school leaders and interventio n teachers in both formal PD workshop settings and in school-based PLCs. This finding demonstrates a promising, inclusionary trend by indicating that standards -based policy implementation is not the responsibility of only one stakeholder group nor the priv ilege of only teachers of certain children. By creating opportunities for this inclusionary learning, administrators are reporting how they are encouraging all teachers to take more ownership over the performance of SWDs and ELs so that they too can access high quality instruction that comes with implementing CCR standards. In at least 17 districts, administrators promote this message by requiring everyone to receive joint PD on instructional methods specific to these special populations, such as Sheltered Instruction Observation Protocol (SIOP) or co-teaching to support SWDs. By including general teachers in district-wide PD sessions for SWDs and ELs, district leaders described “trying to get all of our teachers better informed” (Interview 13, TX) and encou raging the integration of these strategies into the general education classroom. One example of this collaboration is found in a district in California, where a cohort of special education and general education teachers gather in PD to “study and create Co mmon Core aligned lessons together” (Interview 9, CA) to collectively deepen their understanding of differentiation in a CCR standards-based classroom. Similarly, of the 14 districts that describe PLCs, eight spoke about the intentional inclusion of intervention teachers in conversations about student mastery of the standards. These collaborative interactions among teachers with multiple areas of expertise allow for more robust conversations around scaffolding instructional support. It also shifts the disco urse towards the CCR standards as applicable to all students. As one Kentucky district official said: I think in the past there was an exclusionary mindset of separatism from Special Ed. and General Ed. and I think with the professional learning communiti es that they’re coming together, looking at student work, student data, and using one Education Policy Analysis Archives Vol. 28 No. 111 10 another as springboards, and sharing conversations around how to support gaps in knowledge. (Interview 28, KY) This administrator speaks to the value of bringing together general education and special education voices in a single PLC in order to promote inclusion rather than exclusion. An ESL coordinator in another district also testified to the need to subvert exclusi onary practices towards teachers of ELs, as they are often seen as the “stepchild” that does not usually get a seat at the PLC. Thus, he and his “content counterparts” work together with all teachers so the teachers “are hearing one voice and not two” (Interview 23, TX). Sentiments such as these point to the way in which these collaborative opportunities can engender coherence, another feature of effective PD. District administrators are increasingly speaking of PD as a vehicle through which to achieve greater “instructional coherence and organizational coherence” (Interview 28, KY) among teachers of general education students, SWDs, and ELs so that the policies of standards-based reform can seem to connect with each other, instead of operating in silos. In previous waves of standards-based reform, state standards were seen as policies for general education populations, separate from policies for SWDs that mandate individualized instruction, and separate from policies that require English language proficiency testing. The collective practices referenced in this study instead encourage more information sharing, stronger networks, and calibration of instructional expectations, helping to combat potential feelings of isolation for SWD or EL teachers who may be al one at their school sites. Including school leaders in PD further expands this sense of coherence and also lends legitimacy to the message of shared ownership. PD is used to train school leaders in instructional strategies that their teachers are being asked to implement and to gain these leaders’ commitment to integrating PD for general and special teachers. In five districts, school leaders are either required or encouraged to partake in new PD that their teachers receive. The value in this practice, according to one administrator, is that “we want the administrator to support the idea. Cause once you send a teacher [to PD] who’s excited and ignited and you send them back and the principals says we’re not doing that… it doesn’t work. So I need [principals] to get excited too” by also attending the PD (Interview 12, CA). In two of these districts, building leaders are receiving PD on EL strategies so they can serve as EL champions within their own buildings, which entails advocating for their teachers to att end the district’s EL PD or providing feedback to teachers to improve instructional practices for ELs. These practices speak not just to the importance of inclusive PD in creating coherent mindsets, but also to how new notions of collective participation in PD can have impacts on students’ educational experiences. Collective participation can also be a way to address content gaps that exist due to the traditional practice of excluding intervention teachers and principals from content PD. Administrators in three districts specifically direct their content-focused PD towards special education teachers, while four other districts make sure to target principals and assistant principals in their content-focused PD. Because special education teachers “get a little information about being an elementary school teacher and a little about middle school and a little about high school and a little bit about disabilities and a little bit about behavior” without getting “intensive training around how to be a reading interventionist” (Interview 19, TX), some districts are switching gears to invite more special teachers to content PD and focus less on compliance-based PD. One example is from an urban district in California. This district reported that “our Special Ed. teachers are not that fluent in math, you know, the mathematics An Integrative Approach to Professional Development 11 itself, and so, one of our goals is to close that gap, and we have done that by partnering with our General Ed. experts” (Interview 9, CA). A similar move is also being undertaken to include school l eaders in content-focused PD. Four districts reported offering content-focused leadership PD to groups of principals and highlighted the importance in doing so because principals needed to offer instructional feedback that is aligned to the standards. A Texas administrator summarizes this sentiment: The instructional leader who conducts observations and provides feedback needs to be knowledgeable on what is being delivered in the classroom. Because if you’re not involved and don’t understand the standards all you’re doing is reading pedagogy. Reading pedagogy is great, but that’s not going to get you there if the pedagogy is not addressing the standard or what the standard is calling for. (Interview 11, TX) Here, the district official describes the need for principals to develop content expertise aligned to the teaching and learning expectations of the standards in order to provide meaningful feedback. These efforts to widen the reach of content -focused PD to also envelop principals, assistant principals, and special teachers suggest promising signs of building the whole system’s capacity to implement the CCR standards. Collaboration through Active Learning A second major theme relates to pairing collective participation with active learning to target teachers’ content knowledge development. Districts tended to describe three different forms of active learning with regards to building standards-based content knowledge: teachers developing curricular products aligned to the standards as a form of learning ( n = 5), coaches and other district personnel modeling standards-based instruction while teachers act as students (n = 6), and teachers, as well as principals, observing each other’s lessons and debriefing the observations (n = 7). While there are some instances in which district officials described lecture- style PD, the topics of these sessions were less related to the district’s focus on the CCR standards and were more one-off opportunities to hear from researchers on growth mindset and other educational “fads.” While the idea of incorporating active learning into PD experiences is not necessarily new, we are now seeing it being paired with collective participation for both teachers and leaders. With districts merging active learning and collective participation, these features are working as connected mechanisms for elevating PD experiences to better meet the goals of state standards policy. For example, in an urban district in Texas, principals visit each other’s campuses on a monthly basis in a system called instructional rounds. They reflect on instructional strengths and areas of growth, and then they are able to compare “who made more progress” using the “quantitative and qualitative data from the rounds process” (Interview 9, TX). This competitive spirit, coupled with the learning that stems from peer observation experiences, motivates teachers to continuously improve their implementation of the standards, according to officials in this district. Another urban district in Kentucky brings together elementary school teachers, in what they call a “large group PLC,” to collaboratively design lessons. The group observes one of their peers delivering the lesson they designed together, and then they participate in a debrief where they discuss what they noticed, “so it ’s a collective experience in planning, delivery, and then, observation, and then, all that conversation that happens along with it has been very powerful” (Interview 27, KY). This speaks to the power of Education Policy Analysis Archives Vol. 28 No. 111 12 convening teachers across a district who are able to combine their respective “wisdom of practice” (Shulman & Wilson, 2004) to design, deliver, and debrief the same, standards -aligned lesson. A rural district administrator in Massachusetts provided one example of teachers actively role-playing students in a collaborative setting. In a PD session at the beginning of the year, she asked teachers to predict which PD goals were immediate, mid-term, and long-term. After putting each goal on an individual sticky note, the facilitator asked each teacher to create three columns and sort the sticky notes into the immediate, mid-term, and long-term categories by passing the sticky notes around the table. They then had a discussion about the teachers’ choices and whether they align with the district’s intentions. She later explained to them that this sticky note strategy was one that she used with her students: “when they’re reading something, you take the vocab words out before hand and you have them sort them and predict whether the word connects with the setting, the characters, or the conflict of the story, and then afterwards… they see did they have them in the correct categories and if not where should it have gone” (Interview 9, MA). In this example, the PD facilitator is not only asking teachers to reflect on the instructional shifts that they think should be long-term or short-term goals, she also is explicitly demonstrating for them active learning processes that they can apply to their classrooms. As this interview participant later said, her desire is to creat e “intriguing learning opportunities for the adults” so that they are fully engrossed in the PD and then internalize what they learn. District officials note a variety of benefits to these active learning approaches. As one administrator in a suburban Ohio district shared, teachers learn the content standards best when they “have a product that they’re working on, otherwise [the learning] doesn’t happen” (Interview 26, OH). The process of developing useful, standards-based resources together is more engaging for teachers than just handing them the list of standards and asking them to analyze them, according to this administrator. For another administrator in a rural California district, her integration of practice in PD sessions gives teachers opportunities to “fail in a non- threatening environment” so that teachers can “learn from it and do something different next time” (Interview 11, CA). This represents another district’s commitment to active learning through role-playing, a highly impactful mechanism for adult learning (Kilgour et al., 2015). Sustained Coherence The third theme involves the reliance on duration to provide coherence. Both features are contingent on governance structures and union contracts, such as wheth er central offices have the authority to streamline PD to be consistent with system -wide reforms, or whether schools have the authority to determine PD that is most aligned to their unique needs. In addition, the frequency and length of PD often depends on union contract rules in states that have unions. Therefore, it holds that district administrators tend to speak about coherent PD mechanisms while also referring to the duration of these efforts, leading to the concept of sustained coherence. In fact, we find that districts that describe extended PD opportunities for their school staff also frequently comment on how these PD sessions connect to their curriculum and instructional goals, all of which help translate the policies of standards -based reform into practice (Chingos & Whitehurst, 2012) . The most common mechanisms districts are using are providing ongoing district coaching opportunities to schools ( n = 12) and facilitating school-based PLCs (n = 7), both of which provide regular checkpoints to ensur e that teachers are meeting the district’s instructional expectations. An Integrative Approach to Professional Development 13 District leaders describe implementing a range of creative solutions for extending the time teachers spend in PD in order to foster coherent implementation of the CCR standards. In one suburban Texas district, teachers convene before the start of each quarter, so that district administrators can provide PD on the upcoming curriculum units that teachers are expected to follow. This decision is an example of the marriage of coherence and d uration: by previewing the units that teachers will be teaching on a quarterly basis, the district is prioritizing PD that develops teachers’ ongoing, consistent understanding of the curriculum. Nine other districts require monthly check-ins with principals, teachers, or both, and this time is used to follow up on the district’s instructional goals. In an urban district in California, central office administrators provide guidance to schools that outline what building leaders should cover during mandated PD days, which are once or twice a month throughout the school year. One urban district administrator in Ohio explains shifting from the disconnected, half -day PD days to a series of two-day PD sessions with greater district oversight. PD used to be “here’s your book, you know, good luck, go home and figure it out” to an ongoing set of PD trainings that teachers are required to attend and implement in their classrooms (Interview 15, OH). These strategies not only increase the time teachers spend in PD, but a lso work to ensure that time is meaningful and beneficial. Instructional coaches represent one common mechanism for ongoing professional learning in schools. Twelve districts mention some form of instructional coaching, six of which describe their model as a sustainable means of establishing district wide instructional coherence. These coaches serve two purposes. First, they meet regularly with the central office to understand the district’s instructional direction, and then support teachers in aligning wit h this direction. Second, while they are dispatched by the district to promote the central office’s instructional goals inside the schools, they are also expected to personalize their supports to meet individual teacher needs. An example of these dual purp oses is found in a Kentucky district, where coaches gather every other week to read books about instructional shifts embedded in the Common Core State Standards and discuss how they might push this thinking into their respective school sites. Additionally, they differentially work with teachers (e.g., modeling a lesson, co-teaching) based on individual needs so that it’s “never a ‘gotcha’ visit” (Interview 29, KY). These activities show how coaches help “support everyone consistently and uniformly… because we don’t want some people to believe certain things only work for some kids” (Interview 17, OH), while also being flexible enough to provide ad -hoc PD when necessary. The coaches therefore serve as human resources who offer support that is both sustained and coherent, either because they align with district vision or because they meet individual needs. One district in rural Ohio is taking the idea of sustained coherence even further and has been working with the same external PD coaches for over three year s. In this district, the same two university professors have been working with teachers continually on ELA, math, special education, and Universal Design for Learning instruction for the past three years. They work with teachers every other month and “they’ll stay the whole day” and teachers will invite them into their classrooms, and then district staff “meet with them later and then… they’ll make suggestions” (Interview 14, OH). The prolonged nature of this relationship is conducive to the professors building trust with teachers and even being asked in their classrooms to observe their instruction and provide feedback. Furthermore, the consistent nature of their visits is a shift in PD strategy for the district, as this interviewee made clear: Education Policy Analysis Archives Vol. 28 No. 111 14 [Our PD] used to be like you know someone would come in and talk to us and blah blah blah, and then you’d leave and you’d forget it. And now with these guys, I mean continual, you don’t forget about it because they don’t let you forget about it. (Interview 14, OH) Thus, the second benefit of having regular check-ins with the PD providers is the built-in accountability to improving one’s instructional practice. These sustained interactions undoubtedly contribute to a sense of stability within the seemingly unstable p olicy environment that is frequently associated with national standards-based reform movements. Finally, district participation in school-based PLCs represents another example of how coherence and duration can also occur through collective participation a nd active learning, whose pairing we highlighted earlier. While PLCs are ordinarily positioned as school -based initiatives (DuFour et al., 2006), district leaders are increasingly involving themselves in training and supporting PLCs to follow district wide protocols for engaging with the CCR standards. At least seven districts in our study describe having central office staff, including instructional coaches, walk teacher teams through data-based inquiries of their students’ strengths and needs, with one district sharing that they actually created a new job description for an administrator to support PLCs. District participation in school PLCs is particularly prevalent in Ohio, as most of the Ohio districts in our sample engage in stat e-mandated collaborative team meetings where they analyze student data in 90-day continuous improvement cycles. An urban Ohio administrator shared how this PLC structure helps teachers see how “everything comes together,” and that the district’s standards and curriculum “have a connection between them” (Interview 15, OH), a clear endorsement of the utility of PLCs as channels for building coherence. Limitations Our findings should be interpreted with several caveats in mind. One, we do not probe into PD activities that are interdisciplinary, such as cultural competency, social justice, ethical practice, and other important influences on the teaching profession (Sinnema et al., 2017). While this PD is necessary, it is outside the scope of our study. Two, the data for this study are based on the perspectives of district administrators, who may portray more positive than negative examples of PD. This analysis does not include principal and teacher voice to triangulate the themes we identify, nor do we analyze how these PD experiences influence teacher change or student learning. Since this is a study on district leaders, as policy implementers, designing and justifying strategic decisions around PD to support specific policy goals, we rely hea vily on the district perspectives collected through this research. What our study does offer is the identification of PD trends based on district leaders’ descriptions of their work; given the significant influence that district leaders have on the professional learning environments experienced by their principals and teachers, our data from 24 districts in five states offer a unique opportunity to gain insights into how districts across the country are attempting to design PD initiatives to respond to th e policy demands of the CCR standards. An Integrative Approach to Professional Development 15 Discussion In this paper, we refine our understanding of the features of effective PD according to the ways in which district leaders report the design of their PD models. Our first contribution to the literature is our analysis of how district leaders believe they a re integrating PD features to enhance the goals of standards policies in schools. As we suggest throughout the analysis, these features do not manifest in separate and neat categories: instead, they overlap with one another to form an interdependent approach to PD. Second, we highlight how and why district leaders strategically integrate the PD features for their general education teachers, intervention teachers, and principals, whereas previous scholarship on PD has traditionally focused on one or two of these role groups at a time. This inclusionary approach generates specific themes around the affordances or challenges of PD methods that would not have otherwise surfaced if only one or two of the role groups had been the center of analysis. Ultimately, th is study sheds light on district leaders’ understandings of PD and how they implement PD in service of broader policy goals. The focus on district perspectives addresses the limited exploration of the role of district leaders in creating a system of support surrounding PD (Whitworth & Chiu, 2015) despite the large influence they have over the successful or unsuccessful leveraging of PD in support of instructional reform (Akiba & Wilkinson, 2016; Durand et al., 2016; Morel & Coburn, 2018). Figure 1 illustrates the relationships between the three main findings and how they operate together to support the broader goals around instructional ownership over the learning of all students and eventually, organizational capacity to undertake CCR standards -based reform. In this model, the PD features are not merely “characteristics” of PD. Instead, collective participation is a structural precondition, active learning is a mechanism that generates sustained coherence and content/content standards understanding, and the se interactions ideally work to produce the long-term outcomes of ownership and capacity. Figure 1 Conceptual Model for PD Foregrounding most of the approaches that our respondents describe is the collective participation of groups of teachers in joint PD ventures, paralleling a similar focus on teacher collaboration in the literature (e.g., DeLuca et al., 2017; Horn et al., 2 017; Ronfeldt et al., 2015) Education Policy Analysis Archives Vol. 28 No. 111 16 and in national standards for professional learning. A promising trend found in 17 of the 24 districts in this study is the inclusion of SWD teachers, EL teachers, and principals in the PD opportunities typically reserved for general education teachers, and vice versa, in an attempt to hold all teachers responsible for the learning of all students. Such practices bode well, especially given the important role principals play in mobilizing resources and staff around PD priorities (Buttram & Farley-Ripple, 2016) and the value of sharing strategies and expertise among general teachers and special teachers in PLC contexts (Griffin et al., 2018; Li & Peters, 2016). This level of collective participation then yields two outcomes: cohere nce and content focus. The common exposure to the same instructional vision that principals, general education teachers, and teacher specialists receive through this PD engenders instructional coherence, while the extra effort to include principals and non-general education teachers in PD that builds their content expertise means that more stakeholders are equipped to champion the implementation of content standards for all student groups. However, collective participation for the sake of collaboration does not necessarily lead to meaningful professional learning (Kennedy, 2016; Stosich, 2016). Another promising trend is, therefore, the use of active learning mechanisms to make this collective participation more valuable for teachers and principals. District leaders tend to describe three types of active learning in these collaborative settings: teacher development of curricular and instructional tools, teachers acting as students while central office staff model CCR standards-based pedagogy, and teachers and principals engaging in peer observations and debriefs. The benefit of these approaches, according to district leaders, is that when teachers are more engaged in interactive learning environments, they encounter more “ah ha” moments when it comes to understanding their content, and the content of the standards. Furthermore, when teachers are embedded in the sense-making processes themselves, they are more likely to form coherent connections between the standards, their curricular materials, and the policy goals of their school and district leaders (Allen & Penuel, 2015). This discussion of coherence is linked to the idea of duration, as coherence forms and builds when there is extended engagement in learning overtime (Chrispeels et al., Daly 2008; Johnson et al., 2014). We see the two features as symbiotic because district leaders’ abilities to promote coherence and duration both tend to be contingent on the political culture of the district. In other words, governance structures (e.g., site-based decision making) and teacher contracts frequently dictate the extent to which district leaders can choose how, and how often, teachers spend their PD hours. “Sustained coherence” may therefore be more achievable in districts that have centralized processes for requiring teachers to engage in prolonged, common instructional growth initiatives. We do not have data that indicates which of our districts are more centralized, how many hours of PD teachers are required to have, and how they spend these hours, though we do have district narratives that showcase the intentionality with which they attempt to establish sustained coherence. Two commonly referenced mechanisms for forging this sustained coherence are district-trained instructional coaches, who promote both central office goals and individual teacher goals in schools throughout the academic year, and district facilitation of ongoing PLCs. Though district leaders hope that the impact of these PD initiatives will be shared ownership over CCR-aligned instruction for all student subgroups and organizational capacity (i.e., knowledge, skills, and beliefs) to undertake these policy reforms, undergirding these efforts are the enduring challenges of (a) human capital turnover and capacity constraints, (b ) scale in terms of both population density in urban areas and geographic spread in rural areas, and (c) developing shared understanding of the meaning of college - and career- readiness, especially for those adults who have not directly experienced 21st -century CCR expectations. The first An Integrative Approach to Professional Development 17 challenge is partially due to administrative and teacher turnover in the district, especially in rural districts where teachers “use the small town as a lily pad to jump to something bigger... and so we’re constantly in the mindset of starting over” (Interview 25, TX). For administrators, the turnover leads to a rotating door of PD plans, while for teachers, the cost of orienting, inducting, and/or mentoring new teachers is a financial burden that then takes away from other PD opportunities. The second challenge of scale leads to a “fragmented delivery system” (Interview 9, CA) because PD does not end up reaching everyone in equal doses. Finally, the movement to implement CCR standards for students necessitates that all adul ts in the system must also have an understanding of what rigorous college instruction looks like, and what competitive industries are looking for in their future employees. This leads to the third challenge of changing organizational culture to embrace CCR expectations when not everyone in the system experienced those expectations themselves. District leaders attempt to mitigate these challenges by fostering teacher networks as a form of stable PD support in the midst of central office turbulence, exploring digital learning opportunities through online platforms that more people on a larger scale can access, and consistently modeling CCR shifts in PD environments. In one district, to address loss of expertise when administrative and teacher turnover occurs, officials created a network of EL teachers across the district so they can support and train each other. The presence of teacher networks indicates promising new avenues of research on the diffusion of instructional expertise from veteran teachers who do receive focused, content-specific PD to teachers new to the district who missed those opportunities (Sun et al., 2013). To address the second challenge, three districts utilize online learning platforms to provide everyone the same access to PD. Virtual learning may indeed be a viable option for professional learning and resource sharing in the absence of opportunities to meet face-to-face (McConnell et al., 2013; Tseng & Kuo, 2014). To address the third challenge, at least three districts engage in internal PD so they themselves can constantly embody, model, and message the expectations of the CCR standards to their schools. These consistent demonstrations of the district’s instructional values, beliefs, and expectations through their modeling of the standards may also yield the shifts in organizational culture that this standards-based movement calls for (Schein, 2010). The conceptual model for PD presented here demonstrates trends in how district leaders describe their strategic usage of the PD features to facilitate instructional improvement in service of policy reform. We are not claiming that this model should be implemented in all district-wide endeavors that depend on PD as a driving force; rather, we hope that this model is instructive for both researchers and practitioners. Practitioners can adapt the model’s theory of change based on their local contexts. It is typical for a district reform’s theory of change to be underspecified, or to be more based on wishful thinking than intentional planning (Bryk et al., 2015). It would benefit districts to use a conceptual model, such as the one that emerged from this study, as a starting point for their development of a PD strategy that is feasible given the organizational structure of the district system, and given the policy pressures that they face. The model is also instructive to researchers who seek conceptual frameworks for studying the effectiveness of teacher-learning initiatives. The presence of the five PD features is necessary but not sufficient for ensuring the improvement of teacher practice and student learning; instead, researchers should look for how practitioners make sense of this constellation of features given their local contexts and use practitioner-developed models as the basis of evaluation studies. Though practitioners are learning every day about the PD approaches that work or do not work for them, as a field, we “fail to organize, refine, and build on these lessons” in a timely and accessible manner (Bryk et al., 2015, p. 11). Because our conceptual Education Policy Analysis Archives Vol. 28 No. 111 18 model is both grounded in the literature and derived from district administrators’ direct accounts of their PD practices, we present an opportunity for scholars to learn from the front - line professionals and apply these learnings on a more systematic scale. Overall, the goal of this study was to examine how theoretical frameworks, when applied to the messiness of day-to-day practice, warrant continual adaptation and reorganization in order to align with evolving expectations of teachers (e.g., instruction should be rigorously aligned to CCR standards) and structures (e.g., collaborative PLCs with SWD and EL teachers) within the field of education. From this approach, we contribute to the literature b y identifying three main themes that build our understanding of how teacher PD strategically integrates and leverages specific features to support instructional change. Conclusion As the past several decades have taught us, system-wide change will not occur without district and school leaders infusing a PD mission into their everyday roles and responsibilities (Elmore & Burney, 1997; Fink & Resnick, 2001; Spillane & Thompson, 1997; Togneri & Anderson, 2003). The integration of intensive capacity -building efforts into local infrastructures for change is necessary given the ambitious targets set by CCR policy (Cohen & Mehta, 2017). For complex organizations to adapt to changing educational contexts, system leaders need to move beyond tinkering with technical changes to teaching (Richmond et al., 2016) and instead foster a culture of deep learning that affects all members of the organization (Edmondson, 2008). Our work here provides insights from a set of districts across the country on how they are applying features of teacher PD to contribute to this culture of learning. Given that this study is not an evaluation of PD effectiveness, we encourage other researchers to explore whether the trends we discuss here are not just changes to the sy stem but improvements that positively influence teacher and student outcomes (see Bryk et al., 2015 , for this dichotomy). Does the inclusionary approach to collective participation lead to a sense of shared ownership over the education of SWDs and ELLs, an d is this shared ownership associated with student learning gains for these populations? Do active learning approaches improve teachers’ content knowledge, pedagogical content knowledge, and alignment of their instruction with the content standards? Do districts with more centralized governance structures and teacher contracts that allot enough time and district control over PD exhibit more successful implementation of sustained coherence? What are the affordances and challenges of other theories of change that districts develop using different configurations of the features of PD? We hope that our work contributes to future studies that answer these important questions. References Akiba, M., & Wilkinson, B. (2016). Adopting an international innovation for teacher professional development: State and district approaches to lesson study in Florida. Journal of Teacher Education, 67(1), 74–93. http://doi.org/10.1177/0022487115593603 Allen, C. D., & Penuel, W. R. (2015). Studying teachers’ sensemaking to investigate teachers’ responses to professional development focused on new standards. Journal of Teacher Education, 66(2), 136–149. http://doi.org/10.1177/0022487114560646 Babinski, L. M., Amendum, S. J., Knotek, S. E., Sánchez, M., & Malone, P. (2018). Improving young English learners’ language and literacy skills through teacher professional An Integrative Approach to Professional Development 19 development: A randomized controlled trial. American Educational Research Journal, 55(1), 117–143. http://doi.org/10.3102/0002831217732335 Binkorst, F., Poortman, C. L., McKenney, S. E., van Joolingen, W. R. (2018). Revealing the balancing act of vertical and shared leadership in Teacher Design Teams. Teaching and Teacher Education, 72, 1–12. http://doi.org/10.1016/j.tate.2018.02.006 Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Harvard Education Press. Buttram, J. L., & Farley-Ripple, E. N. (2016). The role of principals in professional learning communities. Leadership and Policy in Schools, 15(2), 192. http://doi.org/10.1080/15700763.2015.1039136 Carney, M. B., Brendefur, J. L., Thiede, K., Hughes, G., & Sutton , J. (2016). Statewide mathematics professional development: Teacher knowledge, self -efficacy, and beliefs. Educational Policy, 30(4), 539–572. https://doi.org/10.1177/0895904814550075 Chingos, M. M., & Whitehurst, G. J. (2012). Choosing blindly: Instructional materials, teacher effectiveness, and the Common Core. Brookings Institution. Chrispeels, J. H., Burke, P. H., Johnson, P., & Daly, A. J. (2008). Aligning mental models of district and school leadership teams for reform coherence. Education and Urban Society, 40(6), 730–750. http://doi.org/10.1177/0013124508319582 Cohen, D. K., & Mehta, J. D. (2017). Why reform sometimes succeeds: Understanding the conditions that produce reforms that last. American Educational Research Journal, 54(4), 644–690. http://doi.org/10.3102/0002831217700078 Crowley, C. B. (2017). Professional development as product implementation training. Teaching and Teacher Education, 67, 477-486. Darling-Hammond, L., Hyler, M. E., & Gardner, M. (2017). Effective teacher professional development. Learning Policy Institute. DeLuca, C., Bolden, B., & Chan, J. (2017). Systemic professional learning through collaborative inquiry: Examining teachers' perspectives. Teaching and Teacher Education, 67, 67–78. http://doi.org/10.1016/j.tate.2017.05.014 Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181-199. http://doi.org/10.3102/0013189X08331140 Desimone, L. M., & Garet, M. S. (2015). Best practices in teacher's professional development in the United States. Psychology, Society & Education, 7(3), 252-263. http://doi.org/10.25115/psye.v7i3.515 Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Effects of professional development on teachers’ instruction: Results from a three -year longitudinal study. Educational Evaluation and Policy Analysis, 24(2), 81-112. http://doi.org/10.3102/01623737024002081 Desimone, L., Smith, T. M., & Phillips, K. J. R. (2013). Linking student achievement growth to professional development participation and changes in instruction: A longitudinal study of elementary students and teachers in Title I schools. Teachers College Record 115(5). DuFour, R., DuFour, R., Eaker, R., Many, T. (2006). Learning by doing: A handbook for professional learning communities at work. Solution Tree. Durand, F. T., Lawson, H. A., Wilcox, K. C., & Schiller, K. S. (2016). The role of district office leaders in the adoption and implementation of the Common Core State Standards in elementary schools. Educational Administration Quarterly, 52(1), 45–74. http://doi.org/ 10.1177/0013161X15615391 Education Policy Analysis Archives Vol. 28 No. 111 20 Durksen, T. L., Klassen, R. M., & Daniels, L. M. (2017). Motivation and collaboration: The keys to a developmental framework for teachers’ professional learning. Teaching and Teacher Education, 67, 53–66. http://doi.org/10.1016/j.tate.2017.05.011 Edmondson, A. C. (2008). The competitive imperative of learning. Harvard Business Review, 86(7– 8), 60–67. Elmore, R., & Burney, D. (1997). Investing in teacher learning: Staff development and instructional improvement in Community School District Two, New York City . National Commission on Teaching and America’s Future. Fink, E., & Resnick, L. B. (2001). Developing principals as instructional leaders. Phi Delta Kappan, 82(8), 598–610. Fischer, C., Fishman, B., Dede, C., Eisenkraft, A., Frumin, K., Foster, B., ... & McCoy, A. (2018). Investigating relationships between school context, teacher professional development, teaching practices, and student achievement in respo nse to a nationwide science reform. Teaching and Teacher Education, 72, 107–121. http://doi.org/10.1016/j.tate.2018.02.011 Floden, R. E., Richmond, G., & Andrews, D. C. (2017). Responding to the challenge of new standards. Journal of Teacher Education, 68(3), 236–238. http://doi.org/10.1177/0022487117702380 Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915-945. http://doi.org/10.3102/00028312038004915 Garet, M. S., Wayne, A. J., Stancavage, F., Taylor, J., Eaton, M., Walters, K., Song, M., Brown, S., Hurlburt, S., Zhu, P., Sepanik, S., & Doolittle, F. (2011). Middle school mathematics professional development impact study: Findings after the second year of implementation (NCEE 2011-4025). National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Gersten, R., Dimino, J., Jayanthi, M., Kim, J. S., & Santoro, L. E. (2010). Teacher study group: Impact of the professional development model on reading instruction and student outcomes in first-grade classrooms. American Educational Research Journal, 47, 694–739. http://doi.org/10.3102/0002831209361208 Green, J. D., Gonzalez, E. M., López-Velásquez, A. M., & Howard, E. R. (2013). Hands-on professional development: Middle school teachers' experiences with a curriculum intervention research project. Middle School Journal (J3), 45(2), 27. Griffin, C. C., Dana, N. F., Pape, S. J., Algina, J., Bae, J., Prosser, S. K., & League, M. B. (2018). Prime Online: Exploring teacher professional development for creating inclusive elementary mathematics classrooms. Teacher Education and Special Education, 41(2), 121– 139. http://doi.org/10.1177/0888406417740702 Hodges, T. E., & Jong, C. (2014). School-based communities of practice as mechanisms for standards-based mathematics curriculum implementation. Journal of Education, 194(2), 25–34. http://doi.org/10.1177/002205741419400204 Horn, I. S., Garner, B., Kane, B. D., & Brasel, J. (2017). A ta xonomy of instructional learning opportunities in teachers’ workgroup conversations. Journal of Teacher Education, 68(1), 41–54. http://doi.org/10.1177/0022487116676315 Johnson, S. M., Marietta, G., Higgins, M. C., Mapp, K. L., & Grossman, A. (2015). Achieving coherence in district improvement: Managing the relationship between the central office and schools. Harvard Education Press. An Integrative Approach to Professional Development 21 Kennedy, M. M. (2016). How does professional development improve teaching? Review of Educational Research, 86(4), 945–980. http://doi.org/10.3102/0034654315626800 Kilgour, P., Reynaud, D., Northcote, M. T., & Shields, M. (2015). Role-playing as a tool to facilitate learning, self-reflection and social awareness in teacher education. International Journal of Innovative Interdisciplinary Research, 2(4), 8–20. Retrieved from http://www.auamii.com/jiir/Vol-02/issue-04/2Kilgour.pdf Koellner, K., & Jacobs, J. (2015). Distinguishing models of professional development: The case of an adaptive model’s impact on teachers’ knowledge, instruction, and student achievement. Journal of Teacher Education, 66(1), 51–67. http://doi.org/10.1177/0022487114549599 Kraft, M. A., Blazar, D., & Hogan, D. (2016). The effect of teaching coaching on instruction and achievement: A meta-analysis of the causal evidence. Brown University Working Paper. Kutaka, T. S., Smith, W. M., Albano, A. D., Edwards, C. P., Ren, L., Beattie, H. L., ... & Stroup, W. W. (2017). Connecting teacher professional development and student mathematics achievement: a 4-year study of an elementary mathematics specialist program. Journal of Teacher Education, 68(2), 140–154. http://doi.org/10.1177/0022487116687551 Li, N., & Peters, A. W. (2016). Preparing K-12 teachers for ELLs: Improving teachers L2 knowledge and strategies through innovative professional development. Urban Education. http://doi.org/10.1177/0042085916656902 Lobman, C., & Ryan, S. (2008). Creating an effective system of teacher preparation and professional development: Conversations with stakeholders. Educational Policy, 22(4), 515–540. https://doi.org/10.1177/0895904807307064 Murphy, A. F., & Haller, E. (2015). Teachers' perceptions of the implementation of the literacy Common Core State Standards for English language learners and students with disabilities. Journal of Research in Childhood Education, 29(4), 510–527. http://doi.org/10.1080/02568543.2015.1073200 Penner-Williams, J. Diaz, E., & Gonzales Worthen, D. (2017). PLCs: Key PD component in learning transfer for teachers of English learners. Teaching and Teacher Education, 65, 215–229. http://doi.org/10.1016/j.tate.2017.03.014 Penuel, W. R., Gallagher, L. P., & Moorthy, S. (2011). Preparing teachers to design sequences of instruction in earth systems science: A comparison of three professional development programs. American Educational Research Journal, 48(4), 996–1025. http://doi.org/10.3102/0002831211410864 Popp, J. S., & Goldman, S. R. (2016). Knowledge building in teacher professional le arning communities: Focus of meeting matters. Teaching and Teacher Education, 59, 347–359. http://doi.org/10.1016/j.tate.2016.06.007 Ravitch, S. M., & Carl, N. M. (2016). Qualitative research: Bridging the conceptual, theoretical, and methodological. Sage Publications. Richmond, G., Bartell, T., & Dunn, A. H. (2016). Beyond “tinkering”: Enacting the imperative for change in teacher education in a climate of standards and accountability. Journal of Teacher Education, 67(2), 102–104. http://doi.org/10.1177/0022487116628837 Ronfeldt, M., Farmer, S. O., McQueen, K., & Grissom, J. A. (2015). Teacher collaboration in instructional teams and student achievement. American Educational Research Journal, 52(3), 475–514. http://doi.org/10.3102/0002831215585562 Sailors, M., & Price, L. (2015). Support for the improvement of practices through intensive coaching (SIPIC): A model of coaching for improving reading instruction and reading Education Policy Analysis Archives Vol. 28 No. 111 22 achievement. Teaching and Teacher Education, 45, 115–127. http://doi.org/10.1016/j.tate.2014.09.008 Schein, E. H. (2010). Organizational culture and leadership (Vol. 2). John Wiley & Sons. Shulman, L. S., & Wilson, S. M. (2004). The wisdom of practice: Essays on teaching, learning, and learning to teach (Vol. 8). Jossey-Bass. Sinnema, C., Meyer, F., & Aitken, G. (2017). Capturing the complex, situated, and active nature of teaching through inquiry-oriented standards for teaching. Journal of Teacher Education, 68(1), 9–27. http://doi.org/10.1177/0022487116668017 Smith, T. M., & Rowley, K. J. (2005). Enhancing commitment or tightening control: The function of teacher professional development in an era of accountability. Educational Policy, 19(1), 126–154. https://doi.org/10.1177/0895904804270773 Spillane, J. P., & Thompson, C. L. (1997). Reconstructing conceptions of local capacity: The local education agency’s capacity for ambitious instructional reform. Educational Evaluation and Policy Analysis, 19(2), 185–203. http://doi.org/10.2307/1164209 Stosich, E. L. (2016). Joint inquiry: Teachers’ collective learning about the Common Core in high-poverty urban schools. American Educational Research Journal, 53(6), 1698–1731. http://doi.org/10.3102/0002831216675403 Stosich, E. L., Bocala, C., & Forman, M. (2018). Building coherence for instructional improvement through professional development: A design -based implementation research study. Educational Management Administration & Leadership, 46(5), 864-880. Sun, M., Penuel, W. R., Frank, K. A., Gallagher, H. A., & Youngs, P. (2013). Shaping professional development to promote the diffusion of instructional expertise among teachers. Educational Evaluation and Policy Analysis, 35(3), 344–369. http://doi.org/10.3102/0162373713482763 Supovitz, J. A., Mayer, D. P., & Kahle, J. B. (2000). Promoting inquiry -based instructional practice: The longitudinal impact of professional development in the context of systemic reform. Educational Policy, 14(3), 331–356. https://doi.org/10.1177/0895904800014003001 Togneri, W., & Anderson, S. E. (2003). Beyond islands of excellence: What districts can do to improve instruction and achievement in all schools: A leadership brief. Learning First Alliance. Tseng, F. C., & Kuo, F. Y. (2014). A study of social participation and knowledge sharing in the teachers' online professional community of practice. Computers & Education, 72, 37–47. http://doi.org/10.1016/j.compedu.2013.10.005 Von Esch, K. S., & Kavanagh, S. S. (2018). Preparing mainstream classroom teachers of English learner students: Grounding practice-based designs for teacher learning in theories of adaptive expertise development. Journal of Teacher Education, 69(3), 239–251. http://doi.org/10.1177/0022487117717467 Whitworth, B. A., & Chiu, J. L. (2015). Professional development and teacher change: The missing leadership link. Journal of Science Teacher Education, 26(2), 121–137. http://doi.org/10.1007/s10972-014-9411-2 Yoon, K. S., Duncan, T., Lee, S. W., Scarloss, B., & Shapley, K. L. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007-No. 033). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. An Integrative Approach to Professional Development 23 Young, M. D., Winn, K. M., & Reedy, M. A. (2017). The Every Student Succeeds Act: Strengthening the focus on educational leadership. Educational Administration Quarterly, 53(5), 705–726. http://doi.org/10.1177/0013161X17735871 Youngs, P. (2001). District and state policy influences on professional development and school capacity. Educational Policy, 15(2), 278–301. https://doi.org/10.1177/0895904801015002003 About the Authors Katie Pak University of Pennsylvania kpak@gse.upenn.edu Katie Pak, Ed.D., is a postdoctoral researcher at the University of Pennsylvania Graduate School of Education. Her research includes policy implementation, educational leadership, school improvement, and leadership for critical social change. Laura M. Desimone University of Delaware lauramd@udel.edu Laura M. Desimone, Ph.D., is director of research in the College of Education and Human Development at the University of Delaware and a professor in the School of Education in Educational Statistics and Research Methods and in the Joseph R. Biden, Jr. School of Public Policy & Administration. She studies how state-, district-, and school-level policy can better promote changes in teaching that lead to improved student achievement and to closing the achievement gap between advantaged and disadvantaged students. Arianna Parsons University of Pennsylvania ariannap@gse.upenn.edu Arianna Parsons studies curriculum development and professional learning, in both domestic and international settings. As a former special education teacher, she is especially interested in how these topics play out in inclusive instructional settings for students with disabilities. mailto:kpak@gse.upenn.edu mailto:lauramd@udel.edu mailto:ariannap@gse.upenn.edu Education Policy Analysis Archives Vol. 28 No. 111 24 education policy analysis archives Volume 28 Number 111 August 3, 2020 ISSN 1068-2341 Readers are free to copy, display, distribute, and adapt this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, the changes are identified, and the same license applies to the derivative work. More details of this Creative Commons license are available at https://creativecommons.org/licenses/by-sa/4.0/. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), QUALIS A1 (Brazil), SCImago Journal Rank, SCOPUS, SOCOLAR (China). Please send errata notes to Audrey Amrein-Beardsley at audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. https://creativecommons.org/licenses/by-sa/4.0/ http://www.doaj.org/ mailto:audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE An Integrative Approach to Professional Development 25 education policy analysis archives editorial board Lead Editor: Audrey Amrein-Beardsley (Arizona State University) Editor Consultor: Gustavo E. Fischman (Arizona State University) Associate Editors: Melanie Bertrand, David Carlson, Lauren Harris, Danah Henriksen, Eugene Judson, Mirka Koro-Ljungberg, Daniel Liou, Scott Marley, Molly Ott, Iveta Silova (Arizona State University) Madelaine Adelman Arizona State University Amy Garrett Dikkers University of North Carolina, Wilmington Gloria M. Rodriguez University of California, Davis Cristina Alfaro San Diego State University Gene V Glass Arizona State University R. Anthony Rolle University of Houston Gary Anderson New York University Ronald Glass University of California, Santa Cruz A. G. Rud Washington State University Michael W. Apple University of Wisconsin, Madison Jacob P. K. Gross University of Louisville Patricia Sánchez University of University of Texas, San Antonio Jeff Bale University of Toronto, Canada Eric M. Haas WestEd Janelle Scott University of California, Berkeley Aaron Benavot SUNY Albany Julian Vasquez Heilig California State University, Sacramento Jack Schneider University of Massachusetts Lowell David C. Berliner Arizona State University Henry Braun Boston College Kimberly Kappler Hewitt University of North Carolina Greensboro Noah Sobe Loyola University Casey Cobb University of Connecticut Aimee Howley Ohio University Nelly P. Stromquist University of Maryland Arnold Danzig San Jose State University Steve Klees University of Maryland Jaekyung Lee SUNY Buffalo Benjamin Superfine University of Illinois, Chicago Linda Darling-Hammond Stanford University Jessica Nina Lester Indiana University Adai Tefera Virginia Commonwealth University Elizabeth H. DeBray University of Georgia Amanda E. Lewis University of Illinois, Chicago A. Chris Torres Michigan State University David E. DeMatthews University of Texas at Austin Chad R. Lochmiller Indiana University Tina Trujillo University of California, Berkeley Chad d'Entremont Rennie Center for Education Research & Policy Christopher Lubienski Indiana University Federico R. Waitoller University of Illinois, Chicago John Diamond University of Wisconsin, Madison Sarah Lubienski Indiana University Larisa Warhol University of Connecticut Matthew Di Carlo Albert Shanker Institute William J. Mathis University of Colorado, Boulder John Weathers University of Colorado, Colorado Springs Sherman Dorn Arizona State University Michele S. Moses University of Colorado, Boulder Kevin Welner University of Colorado, Boulder Michael J. Dumas University of California, Berkeley Julianne Moss Deakin University, Australia Terrence G. Wiley Center for Applied Linguistics Kathy Escamilla University of Colorado, Boulder Sharon Nichols University of Texas, San Antonio John Willinsky Stanford University Yariv Feniger Ben-Gurion University of the Negev Eric Parsons University of Missouri-Columbia Jennifer R. Wolgemuth University of South Florida Melissa Lynn Freeman Adams State College Amanda U. Potterton University of Kentucky Kyo Yamashiro Claremont Graduate University Rachael Gabriel University of Connecticut Susan L. Robertson Bristol University Miri Yemini Tel Aviv University, Israel Education Policy Analysis Archives Vol. 28 No. 111 26 archivos analíticos de políticas educativas consejo editorial Editores Asociados: Felicitas Acosta (Universidad Nacional de General Sarmiento), Armando Alcántara Santuario (Universidad Nacional Autónoma de México), Ignacio Barrenechea, Jason Beech ( Universidad de San Andrés), Angelica Buendia, (Metropolitan Autonomous University), Alejandra Falabella (Universidad Alberto Hurtado, Chile), Carmuca Gómez-Bueno (Universidad de Granada), Veronica Gottau (Universidad Torcuato Di Tella), Carolina Guzmán-Valenzuela (Universidade de Chile), Antonia Lozano-Díaz (University of Almería), Antonio Luzon, (Universidad de Granada), María Teresa Martín Palomo (University of Almería), María Fernández Mellizo- Soto (Universidad Complutense de Madrid), Tiburcio Moreno (Autonomous Metropolitan University-Cuajimalpa Unit), José Luis Ramírez, (Universidad de Sonora), Axel Rivas (Universidad de San Andrés), César Lorenzo Rodríguez Uribe (Universidad Marista de Guadalajara), Maria Veronica Santelices (Pontificia Universidad Católica de Chile) Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Ana María García de Fanelli Centro de Estudios de Estado y Sociedad (CEDES) CONICET, Argentina Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Miguel Ángel Arias Ortega Universidad Autónoma de la Ciudad de México Juan Carlos González Faraco Universidad de Huelva, España José Gregorio Rodríguez Universidad Nacional de Colombia, Colombia Xavier Besalú Costa Universitat de Girona, España María Clemente Linuesa Universidad de Salamanca, España Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Xavier Bonal Sarro Universidad Autónoma de Barcelona, España Jaume Martínez Bonafé Universitat de València, España José Luis San Fabián Maroto Universidad de Oviedo, España Antonio Bolívar Boitia Universidad de Granada, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Jurjo Torres Santomé, Universidad de la Coruña, España José Joaquín Brunner Universidad Diego Portales, Chile María Guadalupe Olivier Tellez, Universidad Pedagógica Nacional, México Yengny Marisol Silva Laya Universidad Iberoamericana, México Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Miguel Pereyra Universidad de Granada, España Ernesto Treviño Ronzón Universidad Veracruzana, México Gabriela de la Cruz Flores Universidad Nacional Autónoma de México Mónica Pini Universidad Nacional de San Martín, Argentina Ernesto Treviño Villarreal Universidad Diego Portales Santiago, Chile Marco Antonio Delgado Fuentes Universidad Iberoamericana, México Omar Orlando Pulido Chaves Instituto para la Investigación Educativa y el Desarrollo Pedagógico (IDEP) Antoni Verger Planells Universidad Autónoma de Barcelona, España Inés Dussel, DIE-CINVESTAV, México José Ignacio Rivas Flores Universidad de Málaga, España Catalina Wainerman Universidad de San Andrés, Argentina Pedro Flores Crespo Universidad Iberoamericana, México Juan Carlos Yáñez Velazco Universidad de Colima, México javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/819') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/820') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/4276') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1609') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/825') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/797') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/823') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/798') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/555') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/814') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/2703') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/801') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/826') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/802') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/3264') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/804') An Integrative Approach to Professional Development 27 arquivos analíticos de políticas educativas conselho editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editoras Associadas: Andréa Barbosa Gouveia (Universidade Federal do Paraná), Kaizo Iwakami Beltrao, (Brazilian School of Public and Private Management - EBAPE/FGVl), Sheizi Calheira de Freitas (Federal University of Bahia), Maria Margarida Machado, (Federal University of Goiás / Universidade Federal de Goiás), Gilberto José Miranda, (Universidade Federal de Uberlândia, Brazil), Marcia Pletsch (Universidade Federal Rural do Rio de Janeiro), Maria Lúcia Rodrigues Muller (Universidade Federal de Mato Grosso e Science), Sandra Regina Sales (Universidade Federal Rural do Rio de Janeiro) Almerindo Afonso Universidade do Minho Portugal Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil José Augusto Pacheco Universidade do Minho, Portugal Rosanna Maria Barros Sá Universidade do Algarve Portugal Regina Célia Linhares Hostins Universidade do Vale do Itajaí, Brasil Jane Paiva Universidade do Estado do Rio de Janeiro, Brasil Maria Helena Bonilla Universidade Federal da Bahia Brasil Alfredo Macedo Gomes Universidade Federal de Pernambuco Brasil Paulo Alberto Santos Vieira Universidade do Estado de Mato Grosso, Brasil Rosa Maria Bueno Fischer Universidade Federal do Rio Grande do Sul, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Fabiany de Cássia Tavares Silva Universidade Federal do Mato Grosso do Sul, Brasil Alice Casimiro Lopes Universidade do Estado do Rio de Janeiro, Brasil Jader Janer Moreira Lopes Universidade Federal Fluminense e Universidade Federal de Juiz de Fora, Brasil António Teodoro Universidade Lusófona Portugal Suzana Feldens Schwertner Centro Universitário Univates Brasil Debora Nunes Universidade Federal do Rio Grande do Norte, Brasil Lílian do Valle Universidade do Estado do Rio de Janeiro, Brasil Geovana Mendonça Lunardi Mendes Universidade do Estado de Santa Catarina Alda Junqueira Marin Pontifícia Universidade Católica de São Paulo, Brasil Alfredo Veiga-Neto Universidade Federal do Rio Grande do Sul, Brasil Flávia Miller Naethe Motta Universidade Federal Rural do Rio de Janeiro, Brasil Dalila Andrade Oliveira Universidade Federal de Minas Gerais, Brasil