Microsoft Word - v16n18.doc Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives (Archivos Analíticos de Políticas Educativas), it is distributed for non-commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-nd/2.5/. All other uses must be approved by the author(s) or EPAA/AAPE. EPAA/AAPE is published jointly by the Colleges of Education at Arizona State University and the University of South Florida. Articles are indexed by the Directory of Open Access Journals, H.W. Wilson & Co., and SCOPUS. EDUCATION POLICY ANALYSIS ARCHIVES English Editor: Sherman Dorn Spanish Editor: Gustavo Fischman College of Education Mary Lou Fulton College of Education University of South Florida Arizona State University Volume 16 Number 18 September 30, 2008 ISSN 1068–2341 The Impact of State Intervention on “Underperforming” Schools in Massachusetts: Implications for Policy and Practice Patrick J. McQuillan Boston College Yves Salomon-Fernandez University of Massachusetts, Boston Citation: McQuillan, P. J., & Salomon-Fernandez, Y. (2008). The impact of state intervention on “underperforming” schools in Massachusetts: Implications for policy and practice. Education Policy Analysis Archives, 16(18). Retrieved [date] from http://epaa.asu.edu/epaa/v16n18/. Abstract Since passage of the federal No Child Left Behind Act (NCLB) in 2002, state departments of education across the U.S. have been busy creating or modifying school accountability systems to meet NCLB guidelines. Ultimately, NCLB seeks to have all public school students proficient in English/Language Arts and mathematics by 2014. To identify schools in danger of not meeting this goal, states must establish student performance benchmarks and identify schools not making adequate yearly progress (AYP). Those consistently failing to make AYP can be ordered into “radical restructuring,” which may include having the state intervene in running the school (U. S. Department of Education, 2002). Given these NCLB provisions and the growing number of schools not meeting AYP, the number of state interventions in low-performing schools will certainly increase. Accordingly, this article explores two questions about state-led interventions. First, how have teachers and administrators in underperforming schools in Massachusetts perceived state intervention? In addition, based on their perceptions, what might be done to make the process more effective? At three schools that experienced interventions from the Massachusetts Department of Education, a qualitative study explored the Education Policy Analysis Archives Vol. 16 No. 18 2 process of state intervention. A survey to principals in 22 of the 23 schools deemed underperforming by the state between 2000 and 2004 supplemented the in-depth qualitative work. Drawing on these mixed methods data sources, this article offers a series of proposals aimed at informing future state interventions in Massachusetts and elsewhere. Key words: accountability; educational improvement; educational legislation; No Child Left Behind; politics of education; Massachusetts. El impacto de la intervención del Estado en escuelas de bajo desempeño en Massachusetts: Implicaciones políticas y practicas. Resumen Desde la aprobación en 2002 de la ley federal No Child Left Behind (NCLB/ Ningún Niño Se Quede Atrás), los departamentos estatales de educación. se han ocupado de crear o modificar sus sistemas de rendición de cuentas (accountability) para atender a las demandas de la ley NCLB. En última instancia, NCLB pide que todos los estudiantes de las escuelas públicas demuestren ser competentes en Inglés/Artes del Lenguaje y Matemáticas para el año 2014. Para identificar las escuelas que podrían no cumplir con este objetivo, los estados deben establecer indicadores de rendimiento de los estudiantes e identificar parámetros de referencia de “Progreso Anual Adecuado” para cada escuela. Escuelas que no demuestran tener un Progreso Anual Adecuado pueden ser sometidas a procesos de “Reestructuración radical”, que incluyen que el Departamento de Educación del Estado intervenga en la dirección de la escuela (Departamento de Educación, EE.UU., 2002). Teniendo en cuenta estas disposiciones y el creciente número de instituciones que no cumplen con los objetivos de Progreso Anual Adecuado, el número de intervenciones en escuelas con bajo desempeño va aumentar. En consecuencia, este artículo explora dos preguntas sobre esas intervenciones. En primer lugar, ¿cómo han percibido los maestros y administradores de escuelas de bajo rendimiento en Massachusetts la intervención del estatal? Además, sobre la base de esas percepciones, que se podría hacer para que el proceso de intervención sea más eficaz? Para contestar estas preguntas se realizo un estudio cualitativo en tres escuelas que experimentaron intervenciones del Departamento de Educación de Massachusetts. Para complementar los datos cualitativos se utilizó una encuesta con directores de 22 de las 23 escuelas consideradas de bajo rendimiento en Massachusetts (entre el 2000 y el 2004). En base a la combinación de datos y métodos de análisis, este artículo ofrece una serie de propuestas para futuras intervenciones escolares estatales en Massachusetts y en otros lugares. Palabras clave: rendición de cuentas; mejoramiento educativo; legislación educativa; Ningún Niño Se Quede Atrás (NCLB); políticas educativas; Massachusetts. Introduction Since passage of the federal No Child Left Behind Act (NCLB) in 2002, state departments of education (DOE) across the country have been busy creating or modifying public school Impact of State Intervention on “Underperforming” Schools 3 accountability systems to meet NCLB guidelines, with the ultimate goals of having all public school students proficient in English/language arts and mathematics by 2014. Central to NCLB mandates is that states establish student performance benchmarks and identify schools not making adequate yearly progress (AYP), with proficiency judged through state-specific assessments. Schools that fail to make AYP for two consecutive years are designated as in need of improvement. Those failing to do so for four consecutive years may be referred for various corrective actions. After five years of not making AYP, schools may be ordered into radical restructuring—they may be converted into a charter school, a private company may take over the school, or the state may assume responsibility for running the school (U.S. Department of Education, 2002). Given these NCLB mandates in combination with trends in AYP data, it is virtually certain that state interventions in low-performing schools will increase nationwide (Brady, 2003; Elmore, 2003; Seder, 2000; Tucker & Toch, 2004; Ziebarth, 2004). In Massachusetts, state involvement in public school accountability predates NCLB. Although the State implemented its current accountability system in 1999, the first state intervention occurred in 1988 when, under state direction, Boston University assumed control of Chelsea public schools (Arasim, 1999).1 In 2000, the Massachusetts Department of Education (MADOE) initiated the panel review process, the first step in identifying schools as underperforming, and started reviewing and intervening in low-performing schools. From 2000 to 2004, MADOE organized 77 panels of school teachers, administrators, and state education officials to intervene in schools identified as potentially underperforming2 due to low scores on the state-wide assessment, the Massachusetts Comprehensive Assessment System (MCAS). Also, in 2004 the state declared the two districts of Winchendon and Holyoke underperforming, thus making all schools underperforming in those districts. In this instance, unlike Chelsea, the state initiated intervention and subsequent reforms.3 The 77 schools in which MADOE intervened between 2000 and 2004 share many characteristics. All are urban. All serve disproportionate numbers of low-income, minority, English language learners (ELL), and immigrant students. Of the 77 schools that underwent panel review between 2000 and 2004, 23 were officially declared underperforming. MCAS results of schools reviewed but not declared underperforming, in most instances, were very low but did not differ significantly from those that were declared underperforming. Being one of the first states to have its NCLB accountability plan approved by federal officials (Education Commission of the States, n.d.), Massachusetts represents a pioneer in public school accountability. In light of the coming wave of public school interventions aimed at enhancing student achievement, this study seeks to contribute to the emerging body of knowledge on state-led intervention in low-performing schools by exploring two questions: How do teachers and 1 We excluded these schools from our analyses because Chelsea is run entirely by Boston University, so the intervention is not being implemented by MADOE. 2 Because a school can only be designated underperforming after a panel review, in Massachusetts the term is only used to refer to such schools. In this article, we adhere to the convention. Nonetheless, ‘state intervention’ includes not only work done in underperforming schools but the initial panel review process as well. 3 The Massachusetts Accountability System and NCLB differ slightly. In Massachusetts, schools are deemed underperforming following aggregate-level analyses of several performance indicators and a site visit by a panel of educators and MADOE officials. Under NCLB, a school’s needed level of intervention is determined by failure to make AYP in multiple consecutive years in the aggregate and by sub-groups. NCLB includes no site visits. Massachusetts is currently redesigning its accountability system to align with NCLB requirements. Education Policy Analysis Archives Vol. 16 No. 18 4 administrators in Massachusetts public schools perceive state intervention at their schools? And based on their perceptions, what might be done to make the intervention process more effective? To investigate these questions, we conducted qualitative studies at three secondary schools, one high school and two middle schools, which underwent panel review and were reviewed by MADOE as potentially underperforming,. Ultimately, two were declared underperforming; one was not. To complement these qualitative studies we surveyed principals in 22 of the 23 schools declared underperforming between 2000 and 2004 by the state. Ideally, insights derived from this research will inform future interventions by state officials working with consistently low-performing schools. The State of State Interventions For a host of reasons, at this juncture in the ever-evolving arena of educational reform policy it is critical to understand how teachers and administrators in Massachusetts schools perceived state intervention. Since NCLB provisions mandate that state departments of education intervene in schools that consistently fail to achieve AYP, many states face a new challenge: working to enhance academic achievement in chronically low-performing schools. As Reville, Coggins, Schaefer, and Candon (2004) observed in their extensive study of MADOE, The [Massachusetts] Department of Education has traditionally been perceived as an agency whose primary purposes were distributing funds and ensuring compliance with legal regulations. The charge for the state to be a partner to schools and districts and a support for instructional improvement is an expansion of the state role, which will take considerable planning and effort to enact. (p. 11) From a national perspective, Marc Tucker and Thomas Toch (2004) concluded much the same, “State departments of education have never been equipped to do the kind of work that NCLB now demands” (p. 3). Moreover, most interventions have focused on financial and management issues, not academic problems, presenting a very different and more challenging endeavor (Seder, 2000; Wong & Shen, 2003). Understanding the experience of MADOE interventions may also prove valuable since state interventions in low-performing schools will likely increase simply because “NCLB guidelines require quicker action than many state policies had previously called for” (Elmore, 2003, p. 2; also see Seder, 2000). As of fall 2006 in Massachusetts alone, using NCLB criteria, 37% of all schools (more than 600) were identified as in need of improvement,. Another 47 were identified for corrective action and 57 were identified for restructuring.4 In California, so-called turn-around teams from the state intervened in 24 schools in 2003. Driven by “NCLB’s progressively-tougher standards,” in 2006 as many as 3,600 schools were slated for intervention (Tucker & Toch, 2004, p. 3). Nationally, during the 2005–06 school year states ordered approximately 1,750 schools into some form of radical restructuring, an increase of 44% from the previous year (Feller, 2006). To heighten the challenge, there is limited understanding of what it takes to turn around chronically low-performing schools. Reville, Coggins, and Candon (2005) put the matter in stark terms, saying MADOE must develop an intervention system, “which no state in the country has 4 Corrective action status is assigned to schools that fail to make AYP for four consecutive years. Such schools may choose to implement a new curriculum, replace school staff, provide teacher professional development, or extend the school day or year, among other options. After one year, a school that still fails to make AYP is assigned restructuring status and may close and reopen as a charter school, enter into a contract with a private management company, hire new staff, be taken over by the state, or engage in other (unspecified) restructuring of school governance (U.S. Department of Education, 2003). Impact of State Intervention on “Underperforming” Schools 5 done completely and for which no proven models exist” (p. 15). Research conducted more broadly supports this claim. In a study of state departments of education, Tucker and Toch (2004) found, “[S]tate agencies that NCLB relies on to carry out its sweeping mandates simply don’t have the capacity to do so” (p. 2). Richard Elmore (2003) maintained much the same: “[N]othing in the recent history of state accountability efforts has equipped states or localities to handle the number of schools that will likely be classified as low-performing under NCLB” (p. 5). Given this formidable undertaking, it is no surprise that states have encountered considerable frustration in what efforts they have made to assist low-performing schools. In their study of MADOE, Reville et al. (2005) highlighted this concern: “[N]o concrete strategy for technical assistance at the district level exists.… Currently, the state provides minimal technical assistance [to low-performing schools] where it provides any” (pp. 15–16; emphasis in original). Likewise, Carol Keirstead & Cynthia Harvell (2005) found that in four low-performing Massachusetts schools state intervention strategies “were insufficient for helping schools improve instructional practice” (p. 1). Studies beyond Massachusetts support this view. After investigating fiscal and academic accountability measures promoted through varied intervention strategies, Seder (2000) wrote, “The track record… is mixed.… State intervention strategies return fiscal soundness to districts typically in three to five years, but student achievement often lags behind” (p. i). He further cautioned: “None of the various state-intervention strategies should be seen as panaceas. Even in those districts deemed successful, student performance still lags behind national and state averages and falls below state performance standards” (p. 28). In related research, Brady (2003) examined three prominent intervention strategies. Based on what he found, Brady concluded that in most cases, state intervention did not significantly improve student performance; no particular intervention appeared more successful than any other; interventions were uneven in implementation and hard to sustain; and as school contexts varied markedly, it was nearly impossible to determine which interventions were most effective. Consistent with Seder (2000) and Brady (2003), in a review of research on state takeovers of schools and districts Ziebarth (2004) wrote, “[S]tate takeovers, for the most part, have yet to produce dramatic and consistent increases in student performance” (p. 2). To some degree, these frustrating developments may reflect the fact that little research exists on state intervention in low-performing schools (McRobbie, 1998; Spreng, 2005; Wong & Shen, 2003). While many states have plans to intervene in schools, “little evaluation of the effectiveness of these actions on improving student and school performance has occurred” (Rudo, 2001, p. 1). Ziebarth’s (2004) review of research stated the matter plainly: “There is a limited amount of research… on the effects of state takeovers” (p. 2). Despite limited research, there may be consensus on one issue: successful intervention will likely entail a long-term commitment by all parties (Phenix, Siegel, Zaltsman, & Fruchter, 2005; Reville, et al., 2005; Vu, 2006). Drawing on the British experience, Turner (1998), the director of a school identified as failing, saw a long-term strategy as essential: “[A]n ‘education agency’ taking over a school will not of itself improve the school. Genuine improvement will occur only with the commitment of the staff, so commitment must be encouraged and nurtured” (p. 97). Further, as Odden and Busch (1998) maintained, since “substantive school restructuring requires that teachers develop an array of new professional expertise” successful interventions can be achieved “only through ongoing, long-term professional development” (p. 35). Yet even though since 1988 “more than half of all states have passed laws that allow state authorities to take control of local school districts” (Hammer, 2005, p. 1), few state departments of education have undertaken interventions longer than ten years.5 In a related vein, Brady (2003) questioned the NCLB timeframe: “The law 5 California, for instance, had many NCLB-like accountability measures in place by the mid-1990s, including state interventions. Since 1997, North Carolina’s Department of Public Instruction has sent Education Policy Analysis Archives Vol. 16 No. 18 6 may expect too much too fast. If successful interventions take two to three years to begin to manifest results in terms of AYP, then the measures of success may prove slower than many of the law’s timelines tolerate” (p. 32; emphasis in original). Tucker and Toch (2004) framed the challenge in daunting terms: “[W]e need a long-term solution, which can only lie in building the capacity of the states, districts, and schools to reach the kinds of goals contemplated by the framers of NCLB. This is not a simple matter, but a vast, man-to-the-Moon kind of challenge” (p. 5). Finally, a study of underperforming schools in Massachusetts could prove valuable because most low-performing schools are disproportionately low-income, non-native-English-speaking, highly mobile, and in need of special services (Haycock, 2001; Perryman, 2006; Ziebarth, 2004). Their teachers are less likely to be certified in subjects they teach and more likely to leave the profession or schools where they work (Darling-Hammond, 2004; Peske & Haycock, 2006; Phenix, et al., 2005; Stiefel, Rubenstein, and Schwartz, 2004). If states developed effective strategies to support these schools, as our research suggests, many would welcome intervention. In summary, while increasingly popular, state intervention entails a new role for most departments of education. The mixed impact these efforts have had on low-performing schools suggests that no one fully understands which strategies are most effective, in part, because few have been systematically studied. If consensus exists, it is that states must make a long-term commitment to support low-performing schools, whose demographic profiles once again point to the inability of the U.S. educational system to educate its most challenging students. Methods This study draws on both qualitative and quantitative data. Given the novelty of state intervention, a mixed-methods approach seemed logical. That is, a general sense of the intervention process came through qualitative field investigations in three schools. In turn, insights derived from these studies and analyses of state reports and the existing literature provided a foundation for a broader survey. The study began in spring 2005 with qualitative investigations of three public schools, two middle schools (August and University) and a high school (Morwood).6 All experienced state intervention in the form of Massachusetts Department of Education (MADOE) panel reviews because of consistently low MCAS scores.7 Morwood and University were subsequently declared underperforming. For August, state intervention ended at this point. The qualitative study draws on three data sources: 35 hours of interviews with 16 teachers and administrators at the three schools reviewed as potentially underperforming,8 six site observations (two at each school) totaling 3 hours, and analyses of varied documents, including commissioned reports on state support for low- performing schools, school improvement plans, MADOE panel review reports, MADOE fact- finding reviews, and relevant newspaper articles. Teacher interviews took place in the respective classrooms during normal school hours, except for one teacher who was interviewed in a guidance counselor’s office. Administrator interviews were held in their offices during school hours. In all “assistance teams” into 56 of the state’s lowest performing schools. Kentucky, too, has provided low- performing schools with additional staff (Tucker & Toch, 2004). As already noted, MADOE has intervened in low-performing schools since 1988 (Reville, et al., 2005). These examples, however, are not the norm. 6 All school names are pseudonyms. 7 When considering faculty and administrator reactions to “state intervention,” the reader should keep in mind that no intervention is exactly the same since review panels varied site-to-site as well as the nature of the intervention, among other factors. 8See Appendix A for a copy of our interview protocols from school site visits. Impact of State Intervention on “Underperforming” Schools 7 instances, only the researchers and research participant were present. Each school observation lasted about half an hour and focused on the physical appearance of the school, messages posted in hallways and classrooms, teacher-student interactions in hallways and cafeterias, and student behavior in public spaces. To generate our key codes and categories and later to identify patterns and themes derived from our qualitative data, we employed a constant comparative analytic method, investigating and corroborating our findings in a recursive and iterative fashion (Sipe & Ghiso, 2004; Spradley, 1980). Coding and analysis were driven by our research questions, as the conceptual codes and categories we focused on offered insight into these questions (Coffey & Atkinson, 1996). Ultimately, we organized our data into three domains (Spradley, 1980): initial reactions by faculty and administrators to learning about state intervention at their school; teacher and administrator perceptions of state intervention over time; and the perceived impact of state intervention on students. To assess the validity of our findings—what could be considered a form of theoretical sampling (Charmaz, 2000 )—we shared an outline and draft of our manuscript with MADOE personnel. The Principal Survey9 Building on understandings derived from our qualitative data, in fall 2006 we designed a survey and sent it to principals in the 23 schools declared underperforming by MADOE between 2000 and 2004.10 Of those principals, 22 replied (a 95.6% response rate).11 The survey included 48 questions distributed across six sub-sections. The overarching themes included elements key to intervention effectiveness, MCAS as a valid measure of achievement, financial and technical support associated with the intervention, the degree of school improvement in the intervention, the effectiveness of state intervention, and the principals’ experience with the intervention. At the time of the survey, more than 90% of principals indicated that they had been at their respective schools for five or fewer years. Nearly 75% of respondents indicated that they were not their school’s principal prior to the intervention. (Table 1 presents the distribution of the principals’ years of experience at their schools and in their districts.) Of the schools where we surveyed principals, 50% were elementary schools, 35% were middle schools, 10% were high schools, and 5% were K-8 schools. On average, the elementary schools served fewer students, with the majority 9 Two questions from the original survey were excluded from analysis. We removed question 11, which asked principals about state funding because some indicated they did not know the exact figures, while others included staff salaries, for an instructional coach, for instance, which increased their total considerably. Since we could not determine who received a full- or part-time coach as a result of intervention and whether that was included in the figures provided us, this question introduced bias that could not be remedied. We discarded item 12 because it asked about the clarity of evaluation criteria. Since 75 percent of the responding principals were not at their schools when the schools were reviewed their responses would have low validity, and since those who were constitute a small minority, these responses were not useful. 10 Appendix B includes a copy of the survey. 11 The school that did not return a survey closed at the end of the 2005–06 academic year. From a sampling bias perspective, the missing survey should minimally affect our results because four of five schools from this city returned a survey. However, because the missing school will close because of its status as an underperforming school and its inability to raise student performance—a decision challenged by the principal, some teachers, and parents—survey results of this school might differ in intensity and direction from the overall sample and other schools in the city. The absence of this school’s survey results introduces some bias to our findings. Education Policy Analysis Archives Vol. 16 No. 18 8 enrolling under 500 students. The two high schools had the highest enrollments. (Table 2 presents student enrollment data from the schools.) Table 1 Principals’ Experience in School and District Years Served Any Capacity in School (%) Principal of School (%) Any Capacity in District (%) Less than 1 14 18 14 1 through 5 73 73 35 6 through 10 9 9 14 More than 10 5 0 37 Note: Percentages will not add to 100% when there is rounding error. Table 2 Student Enrollment Data Grade level N Minimum Maximum Mean Standard Deviation Elementary 11 162 713 359 159 Middle 8 350 710 544 149 High School 2 1200 1300 1250 70 K-8 1 600 600 600 N/A The survey provided a means to test our qualitative findings by assessing the extent to which interview data and observations collected in spring 2005 from the three schools that experienced state interventions in 2001 might generalize to the schools reviewed and declared underperforming by the state between 2000 and 2004. Given the high response rate, we believe we gained a more nuanced understanding of principals’ views on state intervention from our qualitative study and feel more confident in our overall findings. Moreover, as these principals experienced intervention for at least one-and-a-half years, and some as long as five-and-a-half years, we feel they could make informed evaluations of this experience. Although 95.6% of principals responded to the survey, the sample is small. Thus, we were limited in the inferential statistical techniques we could apply. Instead, we conducted frequency analyses, examining the distribution of responses in the different categories. Given the high response rate, we feel confident in the reliability of our findings, though as the number of schools found underperforming increases, principal survey responses may differ. In terms of our research design, we treated high schools, middle schools, and elementary schools in a similar fashion. In analyzing interview, archival, survey, and observational data we did not assess how differing school contexts influenced state intervention. While this accords with most research to-date, we realize differences exist among the three levels of schooling. Nonetheless, we found that state intervention affected all schools in some similar ways. That is what we document and analyze. Impact of State Intervention on “Underperforming” Schools 9 Limitations Three limitations of this study require mention. First, our qualitative cases did not include an elementary school. Given that many underperforming schools in Massachusetts are elementary schools, future research should examine how elementary school teachers and administrators perceive state intervention, especially since research suggests that interventions at this level tend to be more successful (e.g., Learmoth & Lowers, 1998). Second, we did not include students in this study. They should be part of any study that seeks to understand the impact of state intervention. And third, some school personnel, especially teachers, may not fully understand what an intervention entails, as some may only experience a small aspect of the larger experience. Nonetheless, we tended to characterize state intervention in ways that suggested it was understood in a uniform fashion by all of our research participants. In future research, teachers and administrators should be offered the opportunity to clarify how they are defining the intervention process. The Three Schools University Middle School Due to overcrowding in the city’s other two middle schools, its district created University Middle School and first enrolled students in the school in fall 2000. In 2001, the school enrolled nearly 700 7th and 8th grade students—50% White students, nearly 30% Hispanic, slightly more than 10% Asian, and less than 10% African American. Compared with the city’s other middle schools, University served a challenging student population. Over half were eligible for free or reduced price lunch—more than twice the state average. Over one-third spoke a first language other than English. More than half of all students were reading below grade level. The school enrolled more than two-thirds of the district's 7th and 8th grade special education population and approximately 60% of its English language learners (ELLs) (MADOE, 2002a). As a new school, University faced additional challenges. One stemmed from student demographics and a selection bias in who attended. As one administrator recalled, the school “did not receive a fair placement of students… [One middle school] literally hand-picked who they wanted. [University] received kids who weren’t scoring quite as well.” The MADOE (2002a) panel review report confirmed this perception, noting that “20 percent of 7th graders [had] failed multiple subjects in 6th grade.” University also had relatively limited resources, as the administrator explained: “This building [had been] a high school and… anything that was worth anything [faculty] took with them when they went to the new building.” University was deemed underperforming by the state in spring 2002, the only school so designated in its district.12 While the school community felt stigmatized by the decision, it also served as impetus for reform. The district hired a new principal, and the school community collectively articulated a vision to which teachers, administrators, and students seemed committed. Throughout interviews, teachers and administrators expressed a desire to see the school through its improvement efforts and free of the underperforming designation. Speaking to the level of faculty commitment, the principal remarked, “It’s been a great experience here. The people really want to 12 Although University only enrolled students for two years at this point, the state made this determination based on how University students had done when they were enrolled in the city’s other middle schools. Education Policy Analysis Archives Vol. 16 No. 18 10 do well.” Moreover, after being labeled underperforming, there was considerable faculty turnover, which has led some, including the principal, to maintain that University is “now a very different school.” In the following year, University made AYP for English/Language Arts (ELA) in the aggregate and for all subgroups. In math, the school made AYP in the aggregate. The following two years the school had a mixed performance, attaining AYP in the aggregate and for all subgroups in ELA but failing to do so in math. In 2006 and 2007, University’s performance declined notably. Both years, the school failed to make AYP in ELA and math in the aggregate and for the school’s subgroups. Having failed to make AYP for two consecutive years in ELA, as of January 2007, it was in the first year of NCLB’s improvement status. 13 As a result, the school had to revise this aspect of its improvement plan, and the district had to offer parents the opportunity to transfer their children to other district schools and provide the school with technical assistance. Having failed to make AYP for four years in math, its 2007 NCLB status for math was in the second year of restructuring (MADOE, 2007). The consequences for this development were much the same as for improvement year 1 except that the district had to implement fundamental reforms that include changing the school’s governance or staffing. August Middle School August Middle School is one of 20 middle schools located in a major city. In 2001, the school enrolled about 700 students, more than 80% African-American. The remaining 20% included a mix of Latino, Asian, and White students. Of this population, 75% were eligible for free or reduced-price lunch and nearly 30% spoke a first language other than English (MADOE, 2001a). August underwent a panel review in spring 2001 because the school failed to make AYP in both math and ELA in the aggregate the previous year (prior to 2003, MADOE did not assess whether schools made AYP for the various subgroups). The school, however, was never declared underperforming. For August, the panel review went rather well. As the report noted: [T]he school has clearly identified its problems and concerns, especially in math and literacy.… Goals in the School Improvement Plan are directly tied to areas of identified weakness.… The specific strategies planned in literacy and math seem likely to yield a good result as they are aligned with a strong data analysis effort on the part of the school and are correlated with State Standards. (MADOE, 2001a) Moreover, the report indicated, “Interviews and observations show that faculty and staff are familiar with the school plan and are working diligently to implement it.… [Further,] there is a quarterly review of progress on meeting the goals” (MADOE, 2001a). To conclude, the panel wrote: “August Middle School does appear to have a sound plan for improving student performance and the conditions appear to be in place for successful implementation of the school’s improvement plan” (MADOE, 2001a). Fully in line with this optimistic assessment, in 2001 and 2002 August made AYP in math and ELA. However, from 2004 to 2007 the school proved unable to sustain these improvements in test scores and failed to make AYP in the aggregate and for certain subgroups in both math and ELA. As of December 2007, the school’s AYP accountability status for both math and ELA was in 13 As of June 2007, MADOE no longer uses the label, “underperforming.” Instead, any school with an unacceptable NCLB status is rated in terms of five dimensions ranked in terms of degree of severity: identified for improvement—year 1, identified for improvement—year 2, corrective action, restructuring—year 1, and restructuring— year 2. Impact of State Intervention on “Underperforming” Schools 11 the first year of NCLB’s restructuring. As a result, the district, working with teachers and parents, must analyze the school’s needs and develop a plan for fundamental reforms aimed at improving student performance (MADOE, 2007). Morwood High Located in a the same city as August Middle School, in 2001 Morwood High enrolled approximately 1300 students, 50% African American, 40% Hispanic, 8% White, and 2% Asian. Of all students, 70% qualified for free/reduced price lunch, nearly three times the state average. The school also enrolled many ELL students, with over 50% of all students speaking a first language other than English. In general, the school has a poor reputation in the local community. A recent article in the local paper spoke to this issue. In a class with 17 students, for instance, nine were absent. On back-to-school night, one teacher had no parents visit his classroom. And by the end of the first quarter, more than 25% of all students had failed five of their six classes. In 2000, Morwood High was declared underperforming. Identifying key factors behind this decision, the panel review report pointed to “the absence of systematic data collection and use to monitor curriculum implementation, instructional practices, and student achievement.” It continued, asserting that “low expectations are evident in many classes” and that “there was no improvement plan in place that used multiple sources of data to address student and staff needs with clear goals, benchmarks, and assessments of progress.” Despite these concerns, in 2001 and 2002 Morwood met AYP standards in the aggregate in both math and ELA. As with the other two study schools, however, Morwood proved unable to sustain these gains, and from 2003 to 2007 the school never made AYP for subgroups in ELA or math, although in alternating years they did make AYP in the aggregate for ELA and math. As of December 2007, the school’s AYP accountability status was the second year of restructuring (MADOE, 2007), the same as University Middle School. State Intervention in Massachusetts State intervention in Massachusetts is a three-stage process. It begins with a review by MADOE of schools performing lowest on the English/Language Arts and mathematics MCAS exams, using trend analyses to identify those showing little or no improvement in the last three or more years.14 Depending on resources, the State then identifies schools for an initial panel review, to evaluate whether a school should be declared underperforming. Before the review, school and district administrators develop a plan to improve student performance. MADOE staff, accompanied by three-to-five practitioners, then review the plan and various indicators of student performance, including school demographic data, MCAS scores, dropout and attendance statistics, information on teacher qualifications, AYP reports, school improvement plans, and a teacher survey, among other data sources. The panel spends one day with school and district administrators and one day on-site with the goal of addressing two questions. First, is the school implementing a sound plan for improvement and what gains have been achieved to date as a result of this implementation? Second, do the conditions appear to be in place for successful implementation of the school’s improvement 14 Of late, there have been some minimal changes made to the underperforming school selection process. With the recent selection of a new Commissioner of Education, more substantial changes are anticipated. Education Policy Analysis Archives Vol. 16 No. 18 12 plan? After reviewing the panel’s report, the Commissioner of Education decides whether to designate a school underperforming (MADOE, 2001b). For schools designated underperforming, the process moves to a third stage, where 3–5 practitioners and one or two MADOE employees conduct a fact-finding review of the school. This review includes two days of research during which panel members observe classes and interview students, teachers, and school and district administrators. The panel then produces a report that highlights the school’s strengths and weaknesses and serves as a guide for revising the school improvement plan, which is authored jointly by district personnel and panel members. Schools that fail to improve student achievement over the next two years are declared chronically underperforming. MADOE then specifies actions the “district shall take to improve the academic performance of students,” which can include granting principals increased flexibility in hiring and dismissing teachers (MADOE, n.d.). Initial Reactions to State Intervention Where one starts with school reform often has a big impact on where he or she ends up (Fullan & Stiegelbauer, 1991; McQuillan & Welner, 2007). To assess how best to promote change through state intervention, it helps to know how teachers and administrators initially reacted to public declaration of their potentially underperforming status since this creates a first impression that shapes how school personnel perceive much of what follows. Welcoming State Intervention Upon learning that MADOE would intervene at their schools in the form of a panel review, some teachers and administrators welcomed the news, viewing state intervention as an appropriate step toward school improvement. In this regard, a University teacher explained: I felt optimistic about [the intervention] because it’s good for someone to step in and look at what’s wrong [because] then they can change things because we should all be… trying to improve things, trying to do better work for the students… .I think accountability is great. Welcoming state intervention, a second University teacher believed it would benefit the school’s underserved English language learners: I was upset because we were labeled. But in a way, I was also very happy because a lot of my ESL students sometimes are not counted as children that could actually be part of the whole picture… [Now,] they have… to be tested. And I was upset because I knew they had to take a new test that I knew they would fail. But I knew this failure was going to not only benefit them, but it was going to benefit the whole school because sometimes I find when something is wrong you get upset because it is wrong but there’s a new door that opens up which helps the whole situation. Often, school personnel who welcomed intervention qualified their remarks, expressing various concerns about the process and its ultimate effect on their schools. As a Morwood teacher who had been at the school over 20 years noted, “I couldn’t imagine what took them so long… [But] I don’t know if it’s going to bring any significant positive change, like so many other things that happen and no significant change comes out of them.” An August teacher expressed support with reservation as well: Impact of State Intervention on “Underperforming” Schools 13 If someone says, “I’m going to help you do better, get more equipment, get better materials,” it’s always welcomed. But how is it going to be done? Will it be saying, “Well, you’re not doing anything good here, I don’t see any positives….” I don’t think anybody is going to refuse [help]. But if the approach is punitive I don’t know if it’s going to be productive. A second August teacher offered a similar reaction: I think there would be [benefits to intervention], yes. The purpose of the state coming would be to help, right? And if they come in like that and that’s what they’re trying to do—“I’m not criticizing, I’m coming in to help. This is where you are, and you need help in these areas, and we have the expertise and the material”—then supposedly everybody should win, supposedly. In her afterthought, “supposedly,” this teacher spoke to a tension expressed by many— acknowledging a need for assistance but, based in part on past experience, questioning whether MADOE intervention would prove effective or respectful. Given the novelty of state intervention, this skepticism seems no surprise. Feeling Unjustly Treated When teachers and administrators first heard MADOE targeted their school for intervention many expressed a sense of inequity, arguing that the state ignored myriad factors that affected student learning but which school personnel considered outside their control. And since these factors were outside their control, was it fair to hold only them accountable? Describing the chaotic nature of her students’ lives, a Morwood teacher explained: Teachers should be held accountable for the performance of our kids, but we’re not solely responsible for the underperformance of students in this school. There are a multitude of problems… they have to overcome before they get to my classroom. It’s not an excuse for what they’re not doing, it’s just reality… Many of our kids live on their own… Many of them live in shelters. I have immigrant kids living in apartments with no parents, working full-time, often taking care of younger siblings. These are just some of the problems. An August teacher offered a comparable reaction, acknowledging her commitment to student achievement though well aware of the challenges her students faced: [W]hat I’m doing just doesn’t overcome… some hurdles [students] have to overcome. And it frustrates me and angers me… How come if I’m putting my heart and soul into this we still aren’t getting anywhere? How do we control the things we can’t control?… If kids haven’t gone to bed at night, if they have no breakfast in the morning? Or [what] if no one will come up [to visit the school]? … What about the kids [and the fact] that it’s January… and this is the third place they’ve been [in school this year]?… There’s so much we can’t control and it is frustrating to say you are underperforming. The number of special needs students a school served represented another structural factor beyond control of school personnel that shaped overall student achievement and potentially MCAS scores. Alluding to options accorded charter and pilot schools, a Morwood administrator observed: Education Policy Analysis Archives Vol. 16 No. 18 14 [Charter and pilot schools]15 can refuse to serve these students. We have no choice but to take them. We have a wide range of different functioning levels of [special education]. Some kids are just physically challenged…. Others need to have their diapers changed several times a day…. We have autistic kids [and students with vision problems for whom] everything has to be translated into Braille. This posed an instructional challenge for teachers, but also triggered feelings of unjust treatment. NCLB limits the proportion of special needs students that can be excluded from the regular assessment and instead be tested via alternate assessment to 1% per state. Teachers felt that the policy does not discriminate between schools that have more severe special needs students who still attend within the regular district school from those who attend specially- designated schools that service students with more serious needs. At our study schools, teachers were held accountable for all students’ performance regardless of any cognitive limitations. An additional complication mentioned by several teachers was the growth in English language learning (ELL) populations, combined with a new state law eliminating bilingual education. Known as the Unz Initiative (named after the Silicon Valley entrepreneur Ron Unz, who sponsored several ballot measures in different states to end bilingual education), the law requires that students with limited English skills are now integrated into mainstreamed classes where instruction is only in English. Consequently, the Unz initiative seems to have exacerbated the challenge of teaching ELL students on multiple levels. First, some teachers now must work with students representing a wide range of English skills: some who know no English, some who have previous exposure to the language, and fluent English speakers. In addition, the state requires that all students be assessed in mathematics regardless of English proficiency levels or the number of years the student has been in the country. Exclusions from testing in ELA are allowed only for students who have been in the country less than a year. This posed a significant challenge for teachers. As an August teacher observed: [S]tudents now who are not proficient in English are taking [the MCAS] and the test is part of the overall score of the school. So if you have a school with a large population of English language learners, then you have a problem…. Before they only had to take the test after three years, but now it’s changed.16 A Morwood teacher spoke to the same tension: We have kids coming from foreign countries without knowing a word of English. They’re thrown into a course and don’t know what is going on because they can’t understand. As a teacher, you have to do your best to accommodate them, and try to keep up with the rest of the class, 25 to 30 kids…. [But] how can I get them ready for MCAS… in one… year?17 Beyond ELL and special needs students, teachers found that some students had such academic deficiencies that there was little chance they would perform at grade level. An August Middle School teacher explained: 15 Pilot schools are essentially in-district charters that are granted autonomy in certain areas, such as curriculum and hiring, but remain in the district. 16 Massachusetts public school students must take the MCAS exam after they have been enrolled in schools for one year. 17 Three years after Massachusetts ended its bilingual programs by voter mandate, after one year of schooling, eighty-three percent of non-native English speakers in grades 3 through 12 could not read, write, or understand English well enough to function in a regular classroom. Of those who were in school at least three years, more than half were not fluent (Schetti & Jan, 2006). Impact of State Intervention on “Underperforming” Schools 15 If we have a child coming to us at the 2nd or 3rd grade level, which does happen, and we bring this child by 8th grade to the 6th grade [achievement] level, we are underperforming because the child is not leaving us with an 8th grade [achievement] level. But are we able to do that in three years?… [W]hat I know is that from where we were, we’ve made progress. The source of this frustration is mainly that the NCLB-approved metric for measuring progress does not do so relative to a student’s starting point. As such, educators considered it unfair to expect a school to meet some absolute expectation insensitive to the school population’s starting point. A more appropriate teacher accountability measure might be one that determines the value teachers add during the school year. Thus, a teacher who brings a student from the second to sixth grade level of proficiency would be recognized as having made progress even if the student is in the eighth grade. Teachers and administrators also questioned whether their schools had the resources to realize goals set by the state. In the words of a University administrator: “[It is damning] to label a school underperforming and you don’t give them enough money for them to do what they need to do. We don’t have technology here, and other schools have that. That really puts teachers at a disadvantage.” Explaining why textbooks were piled up in his classroom, an August teacher addressed the issue of limited resources: [The textbooks] are in the class because [students] are not allowed to take them home…. This year I was surprised to hear I was not going to be able to issue textbooks. So I said, “How are kids supposed to read at home?” They said, “Well they can’t, so you have to make copies for them.” I’m not even thinking about that. Some school personnel questioned the use of a single exam, MCAS, as key to school evaluations. An August teacher elaborated: It’s not that [MCAS] is not important but… I don’t think one test will show you the entire scope of what a child can do. First of all, it is a test that happens at the end of the year. It is high-pressure because you have days and days of testing, which tests the kids’ stamina, as far as I’m concerned. And this is not the only test. We give writing prompts [i.e., assessments similar to many MCAS items]. We give the SRI [Scholastic Reading Inventory], and we have to give finals…. It’s a lot happening at the same time which means [students] come to a point where they’re just tired and they just write without thinking. A University administrator saw matters in a similar light: “As a teacher, I never used one exam as the grade for students…. There’s too much emphasis placed upon MCAS. There’s a tremendous amount of growth happening in kids that you don’t capture in [standardized] testing.” Aspects of these varied tensions linked to equity, morale, and high-stakes assessment arose in the principal surveys as well. For one, when asked whether “student performance on MCAS is an accurate measure of student achievement,” 45% either strongly agreed (14%) or agreed (32%), while 32% disagreed (27%) or strongly disagreed (5%). So while more principals considered MCAS a valid measure of achievement than did not, it was not a majority, and nearly 20% were uncertain. (It should be noted here that about half of the principals are from elementary schools, which is consistent with the levels of the schools reviewed by the state during the time period examined in this study). Here, too, a sizable percentage of school practitioners questioned the validity of MCAS to assess student achievement. When asked whether “student performance on MCAS is an accurate measure of teacher effectiveness,” principals were equally divided, with slightly more than 40% agreeing (36%) or strongly agreeing (5%), while the same percentage disagreed (32%) or strongly Education Policy Analysis Archives Vol. 16 No. 18 16 disagreed (9%). To an extent, those disagreeing with the statement may believe, as did many teachers, that aspects of student MCAS performance are outside teacher control or that the exam simply is not a valid measure of teacher effectiveness. Demoralized As one might expect, many school personnel felt demoralized with being reviewed as potentially underperforming. Asked whether there is a stigma associated with this designation, a University administrator stated, “Absolutely.” A Morwood teacher elaborated: You can’t help but take [the label] personally when the whole country is blaming teachers for student underperformance, including the president. It’s a given. If the kids are not learning, [teachers] are not performing as they should. It’s the teachers. How is staff morale? From what I can see, it’s horrible. An August Middle School teacher described the impact of being designated underperforming as disheartening: Do I take that label to heart? That’s me, underperforming? Or do you think others perceive it as that? Unfortunately, it’s both… [I]f others perceive us that way, then I take it to heart, I take it personally. They’re not going to say, “Well the school is underperforming but Rachel is doing a great job.” They’re not going to put it in the paper that way. It’s going to be, “The school is underperforming,” and they’re not going to mention anyone’s name that’s doing a good job. They can’t. And I would then perceive… “Something is wrong. It’s my fault.” These remarks from the interviews suggest that teachers internalize the underperforming label. They view it as a reflection of the caliber of their work and their contribution to students’ success or failure. In addition, it affects their morale. They perceive themselves as being looked down upon not only by the “president,” as one teacher put it, but more importantly by their local community. The principals’ survey shows how critical they perceive morale to be in ridding their school of the underperforming label. In the survey, principals overwhelmingly endorsed the importance of “good teacher morale” to making an intervention successful, as 91% rated this factor as “essential” (36%) or “very important” (55%) to “the effectiveness of DOE intervention.” Yet when asked whether the intervention “has positively affected teacher morale,” only 26% agreed (22%) or strongly agreed (5%). Fifty percent disagreed (27%) or strongly disagreed (23%), with 23% uncertain. So while principals felt that “good teacher morale” was critical to an intervention, few believed the process promoted positive morale. Upon learning their school would undergo state intervention teachers and administrators all responded at some emotional level (Evans, 1996; Hargreaves, 2004a, 2004b; Keirstead & Harvell, 2005). Some welcomed intervention, while others maintained that their school was not underperforming, at least relative to other schools. Many felt unjustly treated, and some were demoralized. Touching on varied tensions linked to state intervention, an August teacher with 33 years’ experience offered her thoughts on why this experience represents such a challenge for all involved: I guess I’m frustrated… [W]e’re starting at a negative before we even begin. Those schools in Wellesley [an affluent district], the state’s not going to be coming in to them… I want someone to oversee what we’re doing and make sure Impact of State Intervention on “Underperforming” Schools 17 we’re doing a good job. But if it’s in comparison to Wellesley, I’m not too sure I’m happy about that unless you want to give us equal funding and equal resources… I can see both worlds. I know somebody has to oversee us… But I also see stress in the fact that every school is held to the same accountability [standards]. Every school is not teaching the same population of kids, does not have the same resources, does not have the same parental support. It’s not an equal system. But… every school is held to that same standard. In many ways, these remarks speak to the complexity of state interventions in chronically low- achieving schools. Even though this teacher recognized a need for oversight and accountability, she did not welcome intervention; rather, she reacted with skepticism, questioning whether her school was being judged fairly. Teachers and Administrator Perceptions of State Intervention Assessing teacher and administrator perceptions of state intervention reveals a continuum. For some, state intervention has led to positive outcomes. For others, intervention has had a negligible, if any, impact on their schools. Still others view intervention as largely negative. Intertwined with these reactions are issues of respect and trust as well as concern for finding time to do the work required of underperforming schools. Benefits of State Intervention In his role as a British civil servant charged with enacting government policy on school effectiveness, Michael Stark (1998) argued that “the public identification of unacceptable standards [in a public school] tends to speed rather than delay recovery, and indeed is often a precondition for [recovery]” (p. 35). He continued, “[M]ost schools have not been destroyed by special measures [e.g., state interventions]; in most cases they have been revived” (p. 43). Consistent with this view, several teachers and administrators in our study saw benefits to state intervention. A University administrator, for example, felt the intervention energized his faculty, leading them to reflect more intensively on their practice, and to collaborate in addressing their perceived needs: I can’t tell you how much support, cooperation I’ve gotten from teachers… Most people here spend a great deal of time after school, they go the extra mile… [The teachers] have changed… [B]eing under the gun of being underperforming makes my job more difficult, but it’s an advantage too. [Teachers] don’t want this label, so they’ll do whatever it takes to get rid of it. They really are concerned about that, so it’s a positive thing here… They’re not seeking the marginal anymore… This has forced [teachers] to examine what they do… I thank [MADOE] for bringing my staff together and making them even more unified… This has been my most rewarding experience in education. In comparable fashion, a University teacher commented on how faculty mobilized to develop a school improvement plan during a voluntary, two-day summer workshop: I think about 90% of the staff came in… The school administration got a lot of feedback and support from the staff… I thought it was really positive because… [teachers] were really committed. I think there are very few teachers who felt this wasn’t a positive thing because they’d have to do more work… [T]hey want to do things to improve overall [student] scores. Education Policy Analysis Archives Vol. 16 No. 18 18 A second University teacher felt the intervention pushed faculty to think more deeply about teaching: It has had a positive impact in terms of the lower end of the teaching staff who didn’t necessarily give a lot of thought and consideration to what they were doing. They’ve had to raise the bar. [In fact,] we’ve all stepped up to the [challenge]… [W] e’ve been forced to look at ourselves and reflect more. And that’s been the positive part. Likewise, an August teacher remarked on the better documentation and reflection prompted by the intervention: [W]e had to document a lot… That is very helpful and encouraging because you say, “Wow, we do a lot of things. We put a lot of things in place…” [T]he process made us better teachers because we started to be aware of what made better teachers. Although no questions prompted participants to comment on unskilled or unprofessional colleagues, related topics arose, especially at Morwood. An administrator, for instance, bemoaned the impact some teachers have on faculty attitudes: “Teachers can be barriers [to improvement]. The ones that are ineffective and poison the staff, those are the worst. It’s not enough that they’re not helping, but they feel a need to take others down with them.” Another Morwood administrator admitted that some faculty “have no more business teaching in an urban school because they don’t like these children. They have low expectations of them. For them this is just about a paycheck.” She continued, “There are teachers who are opposed to professional development. They refuse to do it… [They] had to be threatened with losing their jobs before they even consider… doing the right thing by their kids.” Administrators and teachers at other schools shared similar views. A University administrator touted the benefits of teacher turnover: “Most of the people who wanted to retire [have] retired. We’ve brought a number of great people to replace them. It’s a different school than it was two-and-a-half, three years ago when [the State] did the review.” A University teacher acknowledged the importance of accountability: I think teachers that can’t be held accountable shouldn’t be here because some people are just waiting to retire, or some people who are not ready to retire are here because of vacation days. I know that because I’ve spoken to people like that. In line with such remarks, every principal who responded to our survey agreed that the ability to remove ineffective teachers was either essential (68%) or very important (27%) to the success of state intervention (5% missing). Yet when teachers—even incompetent ones—lose their jobs, school climate can suffer. A Morwood teacher spoke to this issue, “From what I can see, [morale] is horrible… because a lot of people are concerned about continuing to be here… They feel they are under attack, not just from the state, but from the city, the administration.” So even those who saw state intervention in a positive light acknowledged that school climate and faculty morale could suffer. Limited Impact of State Intervention Although faculty and administrators attributed multiple benefits to state intervention, many also said MADOE efforts had a limited effect on their school, often because the state provided neither adequate funding nor technical assistance. In terms of funding, a Morwood guidance counselor dismissed state intervention: “They gave us $25,000. What can we do systemically with Impact of State Intervention on “Underperforming” Schools 19 that? We can’t even hire a teacher.”18 A University teacher also considered funding inadequate: “If the state’s going to hold us accountable, give us some money. I know there isn’t a lot to give, but cutting funding and then holding us to higher standards, it’s not possible.” A University administrator expressed frustration with MADOE technical support: “I was promised more to help me prepare for the last visit of the sister organization. I don’t think I really got anything.” Teachers and administrators also said the intervention had limited impact because MADOE never fully understood their schools, in part because they spent little time there. As a University administrator explained, “Their job is very difficult… I’ve done [school accreditation] reviews and we’re there three days and we all think we need at least another day to really evaluate the school. I don’t know how they can come in and evaluate a school in one day.”19 Addressing the question, “Was the state justified in labeling your school underperforming?” a Morwood teacher said much the same: “Absolutely!… Whether or not they have actually discerned the true causes for underperforming I’m doubtful… They didn’t spend enough time in classrooms, observing on all the floors, or really talking to people.” A second Morwood teacher concurred: “I felt like they came in, did their thing, and were out… [L]ooking at how little I saw them, I have very little faith in how they’re going to help us.” These qualitative findings align with survey data. In assessing whether “the DOE understands the challenges my school faces with serving our student population,” more than 80% of principals were either uncertain (27%), disagreed (36%), or strongly disagreed (18%). Responding to open-ended survey questions, principals offered comparable criticisms: Monitoring has been less than effective. Last year our monitor came in two times. The feedback was received over three months later and was VERY generic. The reports said the same thing for all schools in our district. DOE process was not helpful. Did not provide technical assistance to improve curriculum, assessment, and instruction. Did not have consistent DOE monitors. There is minimal analysis of actual strategies to change schools (beyond writing plans)… Turning around failing schools is complex work. Most of the official discussion is quite superficial usually a) ideological, or b) bureaucratic, or c) platitudinous. DOE monitor… changed three times in two years. Monitor stayed at school for approximately two hours each visit (three visits per year) and then wrote a report based on that short visit. Monitor did not visit classrooms or talk to teachers… Provided some PIM [performance improvement mapping] training but never gave guidance after initial training. Further, responding to the statement, “DOE intervention at my school has been effective,” 36% agreed and 9% strongly agreed. Despite such criticism, when assessing the impact of state intervention on instruction, curriculum, faculty ability to analyze MCAS data, using MCAS data to enhance student achievement, writing the school improvement plan, achieving performance goals, instructional planning, identifying student learning gaps, parental/caregiver involvement, and aligning professional 18 The guidance counselor cites $25,000 as the total funds the school received. In our research, we were told the school received $50,000. 19 Officially, state intervention includes a two-day school visit. However, the fact finding team spends one day with school and district administrators outside the school. Education Policy Analysis Archives Vol. 16 No. 18 20 development to student learning gaps—all potential aspects of the intervention process— principal perceptions were consistently positive. On all dimensions, at least 76% responded that their schools had made “moderate” or “a great deal” of progress (see Appendix B). To a degree, these conflicting results may happen because school personnel do not fully understand what intervention entails. For many, MADOE presence at their school was seen as the intervention, even though the process comprised multiple dimensions. This could help explain comments such as the University administrator who said, “I feel like the threat of them coming and potentially closing the school was more effective than their actual presence.” Perhaps compiling portfolios and creating an improvement plan—although part of the intervention process and a positive development—was never understood as such. Negative Impacts of Intervention on Schools Though some school personnel saw benefits derived from state intervention, many felt the impact had a negative effect on their schools. This is not uncommon. For instance, in examining school accountability measures linked to standards-based reform, O’Day (2002) noted, Schools respond unevenly to outcomes-based accountability… [S]chools that are better positioned in terms of their socioeconomic composition (i.e., higher SES students) and their prior academic performance respond more readily and coherently to the demands of external performance-based accountability systems than those schools less well situated… [R]esearch suggests that lower performing schools actually lose ground relative to the well-positioned schools once an external accountability system is instituted. (p. 308) After studying four low-performing Massachusetts schools, Keirstead and Harvell (2005) came to a comparable conclusion: “State and district actions that are not well integrated or coordinated… create additional burdens for schools ill-equipped to manage multiple initiatives” (p. 1). Consistent with these findings, many teachers and administrators in this study maintained that state intervention negatively affected their schools. Some felt their schools were disadvantaged by the stigma attached to the underperforming label, which encouraged conscientious parents to enroll their children elsewhere. 20 For a Morwood administrator the logic was undeniable: “Would you want to send your child to a school you know is underperforming according to the state? Of course not.” An August teacher raised the same concern: Parents that do their research certainly are not going to choose [a school] that is underperforming… [O]ften times the ones that do the research are parents who are going to be actively involved in their child’s education. So I think we miss out on those parents. We don’t get help from them, which is vital. Further, many school personnel considered the work associated with state intervention—most derived from MADOE mandates—to be an additional drawback. Voicing a common sentiment, a University teacher stated, “[DOE reports] are overwhelming.” A Morwood administrator elaborated: We have not been able to operate as a real school because all our attention has been focused on doing what the DOE wants… It’s not like we’re doing things because it’s best for our kids… [I]t’s always around the parameters set by the 20 The MADOE has proposed “softening” its tone and changing the terminology to “priority” or “new beginning” schools (Silva, 2006). Impact of State Intervention on “Underperforming” Schools 21 DOE. You do something and you have to think, “Is this going to fit within the guidelines of DOE? Will it meet DOE expectations?” A University administrator expressed concern as well: “I’m constantly preparing information for the state and… it really takes me away from instruction and working in classrooms… It takes a lot of energy…. I should be investing in the school… It’s a good thing to have oversight, but there are limits… It’s the most frustrating part of my job.” An August teacher linked the additional work to low teacher morale: Teacher morale was very, very low because… you had all sorts of additional work put upon you. There was the stress of knowing these people were coming… [T]here were days you felt there is so much time involved in what we’re doing and… I truly want to be working with the kids… I don’t want to be up preparing for the meeting the next… rather than preparing the lesson to do the next day. Such observations suggest that teachers perceive the demands of state intervention as additional factors outside their control that impact student achievement. As many maintained, aspects of state intervention seemed unfair. Given this perception, respect seems essential. State personnel do not want to demoralize hard-working teachers. The MADOE (n.d.) school review process directs panel members to “be polite and sensitive to the school’s needs,” “to try to relieve anxiety through mutual respect and valuing the opinions of others,” and “to be supportive and enabling,” among other directives (p. 11). However, as the next section describes, some school personnel experienced state intervention as disrespectful. Unrespected and Distrusted21 Clearly, state intervention is emotionally difficult—for school personnel and MADOE. Consequently, issues of respect can complicate the process. For a Morwood teacher, a class visit proved disrespectful: “[S]omeone observed my class and I never even got a post-it note, like, ‘This is what I saw, thanks,’ or even introducing themselves. I tried to introduce myself once and I remember the woman saying, ‘You don’t need to know my name.’ I just thought it would be a courtesy.” A University teacher questioned the attitude MADOE personnel brought to the intervention: The state has given us these requirements that, while I think we’ve made a positive change in the school, I don’t think it has to be brought with such force. I think more of a constructive criticism would have been nice rather than finger- pointing and blaming, accusatory almost, like, “Why aren’t your kids doing better? What is your teaching staff doing wrong?” While this teacher considered state intervention too accusatory, a Morwood teacher felt the intervention was too directive, and that the MADOE determined too much of what happened: [The reform plan has] too much emphasis on the fact that to be successful you have to use these methods and only these methods… That never works in education. You never say, “One-size fits all.” It would be better to have more 21 Although a number of teachers felt the MADOE had not been respectful, an August teacher felt differently: “I had the principal of a school. He was very complimentary after. He talked to me and I think because he was the principal he understood what the school went through and he empathized with us and said that he was not out to get us. He was there to gather information and it was, I felt, whatever he came up with was going to be an honest evaluation. They weren’t out to find the bad, but… to see that we really at that point were making an effort to change things around and I thought it was going to be a fair process.” Education Policy Analysis Archives Vol. 16 No. 18 22 faith in the teachers knowing what the students need a lot of the times. Yes, observe and make suggestions, but there’s way too narrow a focus on how education has to be. For a University administrator respect was a comparative phenomenon: “I was at a meeting last year and I saw the reverence the DOE paid to some of the affluent districts, like Wellesley [a wealthy district]. But us, we do get treated differently. I don’t think [the MADOE] is even aware of the deference they give them, and the way they view and treat us.” Related concerns surfaced in the principal survey. Although 46% of respondents agreed that “the DOE values teacher input and opinions in the intervention process” (assuming this constitutes a form of respect), 41% disagreed (32%) or strongly disagreed (9%). In an open response, one principal addressed the issue directly: “The state needs to value the teachers that work under challenging circumstances. Let urban teachers know they are respected.” Though key to effective reform, according to our research various factors contributed to a lack of trust by both sides during state interventions (Bryk & Schneider, 2002; Evans, 1996).22 For schools, MADOE-required reports and documentation suggested a mistrust in their ability to meet expectations without oversight. For a Morwood teacher, the intervention process seemed less collaborative than anticipated, and implicitly less trusting: “What many people and I envisioned could have happened was that there’d be time to come together and work forward, not a top-down, ‘We’re going to do these things,’ because there’s no buy-in.” In other cases trust suffered because of past experience with reform. As another Morwood teacher noted: [W]e’ve restructured here before and now we’re restructuring again… We spent the same amount of time and energy and money breaking up into small learning communities and within a year that started to deteriorate… [T[hen we dismantled the bilingual program… It’s happened over and over and that’s why people don’t look to [state intervention] as the “joy of intervention.” With the Department of Education, I’m wondering, “Is this the same thing? Are we going to experience this revving up of interest and then they lose interest.” In one instance, mistrust was exacerbated by perceptions that MADOE colluded with the district. “I feel like the state… is totally in cahoots with the school district management,” said one practitioner. Concerns with trust also emerged in survey data. On the survey 96% of principals indicated that teacher buy-in was either essential (86%) or very important (9%) “to the effectiveness of MADOE intervention.” Yet when asked whether “the DOE values teacher input and opinions,” less than 50% agreed, although it seems MADOE could display trust and might enhance teacher buy-in by soliciting their input. Further, more than 80% of principals maintained that trust between school personnel and the DOE was either essential (27%) or very important (55%) to an effective intervention, and when principals rated how important it is for school personnel to trust that state “intervention [would] benefit their school”, all agreed that trust was either essential (64%) or very important (36%). However, when asked whether the MADOE “understands the challenges my school faces with serving our student population”—which is disproportionately non-native-English- speaking and with high proportions qualifying for free/reduced price lunch—80% were either uncertain (27%), disagreed (36%) or strongly disagreed (9%). For principals to trust that intervention 22 Trust among teachers within a school can also be a concern, although our research did not examine this in-depth. As Bryk and Schneider (2002) maintained, “[T]eachers must work together to advance educational opportunities for children; but if teachers don’t trust their colleagues (which often is the case in schools most in need of reform), the required collaborative efforts are unlikely to be initiated and sustained” (p. 130). Impact of State Intervention on “Underperforming” Schools 23 will benefit their schools it seems MADOE needs to appreciate the challenges schools face. Yet principals clearly questioned whether MADOE understood those challenges, thereby suggesting that trust might be an issue as well. Time for Schools Reflecting on the emotional impact school reform has had on teacher lives, Robert Evans (1996) noted, “Of all the complaints of teachers about the difficulties of change programs in schools, none is more frequent than ‘not enough time’” (p. 139). This observation seems to hold true for state interventions. In MADOE panel reports on Morwood (2002b) and University (2002a), both teachers and administrators said they needed more time to do the work expected. In our qualitative studies, school personnel identified this need as well. As a University teacher explained, “Time is the biggest issue. Time with the students and time with colleagues to share, and time to do workshops because we have been training and we have to train others.” Drawing on similar thinking, a Morwood teacher said faculty never had time to collectively identify a direction for school reform: We never get any freed up time together, communication time together… [T]here was not a whole school movement toward getting out of this underperforming label… We need to do something together… [U]nless you get time to foster and talk about [reform] we can’t move that far with students… it’s very frustrating. Reflecting on the impact of state intervention teachers and administrators expressed a range of reactions. Some felt energized by the opportunity for reform. Others questioned the value of the intervention, even saying it had a negative impact. Respect and trust were matters of concern as was finding time to do the work to transform these schools. If nothing else, teacher and administrator views of state intervention point to the enormity of this challenge. Perceived Impact on Students When school personnel identified positive aspects of state intervention the benefits to students were often implicit in remarks such as, “We started to become aware of what makes better teachers,” and “teachers really had to get on the ball with their lesson plans,” which suggest that related benefits accrued to students in terms of enhanced achievement. The following section considers what teachers and principals said explicitly about how being in an underperforming school affected students. MCAS Preparation According to most research participants, because of their schools’ underperforming status, MCAS became a driving force behind the school curriculum, which narrowed accordingly, as teachers and administrators focused on state standards and preparing students for the exam (Shepard, 2001; Spreng, 2005). A Morwood administrator explained, “In every school there is emphasis on the MCAS. That’s all the teachers can emphasize. Because we are an underperforming school… teachers have to teach to the MCAS.” A Morwood teacher affirmed this view: “[T]he first day of school, the principal and chief academic officer said, ‘MCAS is our curriculum. ’” An August teacher stated, “There is absolutely an emphasis on MCAS… starting from September.” However, Education Policy Analysis Archives Vol. 16 No. 18 24 the narrowing effects were more pronounced for teachers in the core MCAS content areas, some of whom adopted past MCAS exams as classroom curricula. In addition, two schools rearranged their schedules to maximize time devoted to MCAS content areas, math and English/language arts. This emphasis held true even at August Middle School which was not declared underperforming. Faculty also maintained that this curriculum shift influenced their teaching. As a Morwood teacher noted, “[Our students are] not being exposed to a wider knowledge base because we’re centered on just passing the test.” To a lesser degree, an August teacher also felt constrained: It has limited, not prohibited, [what I can do]… [Y]ou cannot leave this and go into something more challenging. [Some topics have] to be done… I used to do more challenging work because I could pick a concept and bring it to as high as the kids could reach. But now I cannot do that because it’ll take me time to build this up and I have other content that has to be covered and I’m thinking, “I cannot let them go to the test and not go through this.” For a University teacher, teaching to the test had drawbacks for students and teachers alike: Teaching to a test isn’t fun for the kids and isn’t fun for the teacher… [Y]ou’re so concerned with the test because you want to get the scores up and get [the state] off your back you lose track of what school is really about which is also teaching kids how to live in the real world with real skills, and you really can’t do that when you’re teaching to a test. In a related sense, a University teacher spoke to the dilemma she faced: helping students meet state standards while trying to keep them from dropping out of school: [Y]ou have to keep teaching engaging, interactive, and fun, otherwise you lose sight of what’s important… These kids are most important to me. And [it’s critical that] these kids enjoy school and don’t lose their love for school right now because this is a crucial time. This is when they decide [about staying in school]. Some say, “I’m dropping out when I’m 16,” and if they decide that now, they’ll do it. In contrast, some school personnel saw substantial benefit to focusing on MCAS. As a University teacher noted: “I’d say students [have benefited] because we’ve improved our education, our instruction. We’ve focused more on things that they need more than what we think they need. We’ve looked at the data to see what they need to learn.” Moreover, an August administrator felt that preparing students for MCAS did not mean the curriculum had to narrow: Every opportunity we get, we talk to the kids about MCAS…. Do you think that in some ways it’s limited teachers’ ability to be more creative? No, because MCAS is not like the Stanford [9] where you can teach to the test… [I]t’s difficult to teach to the MCAS because it’s not a set of questions. MCAS is different and it asks kids to think critically. It asks them to compare and contrast, to analyze, to associate… It asks them questions about themselves, to connect themselves with past events… I think it’s a good data piece that you can say, “Hey this person’s able to think deeper.” Consistently, teachers and administrators who experienced state intervention acknowledged the related impact of MCAS preparation on curriculum and instruction. For some, it undermined student achievement and teacher autonomy. For others, a curriculum driven by MCAS represented an appropriate and valuable direction. Impact of State Intervention on “Underperforming” Schools 25 Distance from Responsibility for Student Failure In addition to emphasizing the MCAS as driving instruction, teacher responses indicate that the interventions may have created a wedge between teachers and student outcomes. Teachers expressed this insulation in several ways, from student misbehavior to the values that students and their families had. In discussing state intervention, many faculty maintained that student misbehavior was critical to their school’s underperforming status (Datnow, Borman, Stringfield, Overman, & Cartellano, 2003). Accordingly, a Morwood teacher challenged a perceived overemphasis on teachers: “I don’t think this is just a teacher problem… There’s no significant push to hold students accountable. No one asks what students need to do.” A second Morwood teacher had a similar reaction: “[F]or too many years… too much of the problems are blamed on teachers… [W]e have dozens and dozens of kids who never go to class… I write these students up and I get in touch with parents and nothing ever changes.” A third Morwood teacher felt students had “taken over” the school: [Students] are out of control. We say, “They’ve taken over”… I have a senior class and everyday I have four to five kids who show up on time out of 22… And we don’t seem to have an effective way of dealing with absenteeism and tardiness. It’s a total lack of order in the hallways… [L]oud talking, interrupting your class, opening your door and yelling, “Is this person here?” It’s chronic. It’s not a good atmosphere to learn… Would you say chaos is too strong a word to use to characterize this school? No… we may have two, three fire alarms on the same day… Wouldn’t you call that chaos? Teachers also questioned whether students valued education. An August teacher, for instance, alluded to students’ unwillingness to seek extra help: We have a program where kids can come after-school for extra help and they don’t stay and they’re not accountable for not staying. What do I do? … I don’t have any power to make them stay. How can you raise scores if the gaps they have can’t all be addressed during class time, yet they don’t show up for extra help? Implying a comparable perspective, a second August teacher reflected on student perceptions of the school’s underperforming label: I don’t remember the kids having it feel like a stigma. And unfortunately sometimes I think it’s because they don’t value education as much as we would hope… [The attitude] is, “Let’s just get through it… All schools are alike, they’re like jails and they lock us down until 2:30 and then we’re free.” That’s their perception. A Morwood teacher said students attend her school precisely for its poor reputation: [I]f you were to ask them, “What’s the reputation of this school in relation to other schools?”, and I have asked them, they’ll come right out and say, “This is the easy school. This is the SPED school. This is the school where you can do anything and nothing will happen to you.” And there are kids who say, “I picked this school because of those reasons, because I really didn’t want to have to do any work.” Teachers also described problematic relationships with parents. A University teacher, for example, noted an ongoing lack of parent involvement: Education Policy Analysis Archives Vol. 16 No. 18 26 Even after we were declared underperforming, it seems like it’s the same number of parents [who get involved]… We always invite all parents to discuss the plans, but most don’t come… [W]e call for a parent meeting when the kids aren’t coming [to school] or are cutting classes or just not doing any work… and generally they just won’t show up. An August teacher considered parents comparably uninvolved: “[Parent involvement] is getting better, but in general if I send warning notices for the child saying they’re failing, not passing in homework, no one gets back in touch with me. That tells me they’re not involved.” Commenting on expectations of student academic performance, a Morwood teacher questioned the attitude both students and parents brought to schooling: Expectations are very low from students themselves and their parents. You have parents who challenge you, who would take kids out of your class because the kid says it’s too hard. That happens all the time… [T]here are those kids who are satisfied with a C and their parents think that’s not too bad. I think expectations across the board are lacking.23 Moreover, when asked whether in her opinion Morwood High was underperforming, a teacher’s response seems revealing: “I don’t think teachers in this building are underperforming at all. I think the teachers are fantastic, very hard-working.” An August teacher said her colleagues were professional: “I think the majority of [teachers]… are working very hard. Most of them are doing what is expected.” In these remarks, teachers offered various explanations for student failure and their school’s underperforming status, all of which distanced them somewhat from responsibility for these outcomes. Whether it was disregard and disrespect for learning that undermined classroom instruction, or students and parents whose expectations across the board are lacking, teachers identified factors that, from their perspectives, were beyond their control yet undermined student achievement (Datnow et al., 2003). 24 Reinforcing Negative Stereotypes The final way in which teachers and administrators perceived their school’s underperforming status as affecting students is disheartening, though perhaps most pronounced at Morwood. In some educators’ views, this official designation affirmed a negative view of low-income, urban students of color. A Morwood administrator explained: Do you think students are aware of the fact that the school is underperforming? Yeah, they know. The MCAS scores are published in the paper before parents get it. The whole community gets to hear the school is this, the students are this and that. 23 Although in interviews teachers and administrators emphasized the role of parents in student achievement, as compared with teachers and administrators, principals rated parental support the least important of all elements in the success of state intervention. With the emphasis placed on that element in interviews, a mean of 2.5 on that survey question seems low. However, the survey was completed by principals only whereas most interviewees were teachers. Further, relative to other categories, such as teachers and principals, parents are likely not as important. Placed in this perspective, these results seem reasonable. 24 Keirstead & Harvell (2005) identified similar developments in their study of four low-performing Massachusetts schools. However, at some schools state intervention helped them move from “discussions of ‘them’ to discussions of ‘we’” (p. 7), which “provided a mantra for teams and schools to break their habits of blaming things beyond their control and taking responsibility for the things they could control” (pp. 7–8). Impact of State Intervention on “Underperforming” Schools 27 What’s their reaction?… They have gotten apathetic. They think, “We’re dummies, so we’ll act like dummies.” A Morwood teacher shared a similar view: Do you think students are aware that the school is underperforming? They’re aware of it; they’re more aware of it than anybody else. Do you think they have an opinion on it? Yeah, they have an opinion of it. They care about it. They know it’s not fair. They’re angry about it. They also seem resigned because they’re so used to it. They feel powerless… [M]y kids talk about it. They visit other schools and they come back and talk about how the classes are, what people have in other schools. They always make comparisons: “They have this, we don’t.” We have two computers and a printer in my classroom but none of them work. How is student morale? Student morale is just like the teachers’, it is low. A University administrator expressed a similar opinion: Some of the kids aren’t great students and when they’re labeled underperforming, that’s a hard thing for a kid. Do you think the students are or were aware that the school’s underperforming? Yes, they are. They’re aware.25 Although students are the intended beneficiaries of state intervention, the benefits for them seem mixed. In some cases, intervention has led teachers to be more reflective about their work, to pay greater attention to lesson plans, and to mobilize around school improvement plans. The consequences of narrowing the curriculum is less obvious. Some teachers and administrators saw this as a positive development. Others felt constrained and consequently devoted less attention to engaging, interactive, and fun activities. Also, as teacher comments implied, students at underperforming schools were seen as a key piece of the problem by those who described them as “out of control,” “not want[ing] to have to do any work,” and “not valu[ing] education.” To further complicate the impact of state intervention on students, their schools are publicly identified as underperforming, a label that can be demoralizing for low-income students of color who already occupy a tenuous niche in American society and often have conflicted relationships with schools. Finally, students seem largely excluded from the intervention process. While MADOE reports focus on student performance in various academic domains, one hears virtually nothing from students about their performance—why they succeed or why they fail. Although the panel review process and fact-finding review allot time for two or three student focus groups, in the five reports on Morwood, University, and August posted on the MADOE website, there was direct input from students once, as the August report noted: “Students felt that the school was an orderly place and not violent and that the principal ‘knows how to handle situations’” (MADOE, 2001). The brevity and superficiality of this one sentence seems a symbolic statement on the student role in the intervention process. 25 In a few cases teachers said students were not aware of the “underperforming” label, typically explaining that this occurred because students cared little about their education. Therefore, while this designation may not impact their self-image, it suggests they hold a problematic view of education. Education Policy Analysis Archives Vol. 16 No. 18 28 Policy Implications Building on what we learned about state intervention from teachers and administrators in our study schools, we offer a series of policy proposals aimed at helping state officials nationwide think about how to conceptualize and implement reform. Acknowledge the Challenge Given the novelty of the intervention process, the likelihood that mandated state interventions will increase, the limited staffing of many states departments of education, and the difficulty of turning around consistently low-performing schools (among other factors), those who control state budgets must recognize the enormity of this challenge, which means state education budgets must receive a significant infusion of financial resources. Without additional funding, state DOEs will not be able to provide the level of assistance and oversight necessary to help turn around low-performing schools, and state interventions will do little more than frustrate all involved in the process. The following policy implications are dependent on the DOE securing such financial support. Make the Process More Inclusive In interviews, most teachers and administrators acknowledged that their schools faced serious challenges and generally welcomed assistance. Many endorsed the overall approach to accountability promoted by MCAS, though not necessarily as the sole criterion of a school’s underperforming status. Overall, school personnel were clearly committed to improving their schools. However, they did question whether the State had the resources, technical expertise, or long-term commitment to address their needs. In our study sites, this initial skepticism tended to worsen when MADOE representatives had limited time to work with schools, as school personnel came to question whether the State fully understood their needs and, therefore, whether intervention strategies would benefit their schools. This troubling dynamic should be kept at the forefront of all efforts to aid consistently low- performing schools for one overarching reason: student achievement will not improve unless a critical mass of school personnel actively embrace whatever reforms their schools undertake. As Fullan (2001) wrote, “Educational change depends on what teachers [and we would add, “administrators”] do and think—it’s as simple and complex as that” (p. 115). Understanding their point of view is therefore critical. To begin, while state interventions can be energizing, they are emotionally difficult (Evans, 1996; Hargreaves, 2004b). They start with the assumption that schools are failing, which as British educator Linda Turner (1998) observed, “appears to tell all of its staff that they also have failed” (p. 104). Further, those working in schools often feel powerless to change their predicament, believing “many factors that impact their work are outside their control” (Learmoth & Lowers, 1998, p. 133), although they are still held accountable for student achievement. In turn, issues of trust between school and state personnel often complicate this already difficult situation (Hoy & Tschannen-Moran, 2003). One means to address these tensions and build on the initial foundation of support for intervention is to bring teachers and administrators into the intervention process (Erlichson, 2005; Turner, 1998). Creating opportunities for school personnel to plan, deliberate, discuss, and disagree would allow them to shape reform, more fully understand what reform entails, and create a common Impact of State Intervention on “Underperforming” Schools 29 vision—all factors linked to successful intervention (Elmore, 2003; Erlichson, 2005; Rudo, 2001). An inclusive strategy could also promote mutual respect and trust between school and state personnel while using practitioner knowledge to shape a common school culture. In a very pragmatic way, if the opinions of teachers and principals were valued, DOEs might better understand the challenges faced by low-performing schools and thereby address them more effectively. Further, as such an approach will likely require DOE personnel to spend more time in school settings, there would be more opportunities to promote relational trust and respect, two key elements to successful reform (Bryk & Schneider, 2002). In addition, the student voice was nearly absent in the school intervention process. As these reforms ultimately aim to enhance student achievement, it seems the recipients of reform should have a meaningful role in the process, if only to let adults know how they feel about these endeavors (Corbett & Wilson, 1995; Fullan & Stiegelbauer, 1991; Learmoth & Lowers, 1998). Otherwise, any effort at change is “based on an incomplete picture of life in classrooms and schools” (Cook-Sather, 2002, p. 3). As states continue conducting interventions, which we consider appropriate and potentially quite valuable, it seems the process could promote mutual respect, trust, and understanding by more fully integrating the student, teacher, and administrator points of view. If they are to enact reform, it seems reasonable to allow them opportunities to shape reform. In so doing, state departments of education must always make student achievement their highest priority. Indeed, state personnel should make this point explicit at the outset of any intervention and maintain an unrelenting commitment to this overarching ideal throughout, even while they seek to honor the opinions of teachers and administrators. Level the Playing Field To further enhance their relations with school personnel, state officials could reassess their approach to accountability. As noted many times, teachers in underperforming schools in Massachusetts said they faced an unfair disadvantage, largely because of the MCAS exam. Holding all students to the same standards does not feel equitable to teachers who work with sizable numbers of highly mobile, low-income, ELL, and special needs students. It seems no coincidence that every school reviewed as potentially underperforming enrolled disproportionate numbers of these students. Why should these schools and teachers be more likely to face public stigma? All involved would benefit from a more sensitive testing instrument that followed individual student progress, or at least followed cohorts over time, rather than treating each student cohort as equal. They are not, and to treat them as such and then publicly identify them as underperforming can make teachers and students in these schools feel that they are unwilling participants in a rigged game. States might also promote more productive relations with school personnel by explicitly identifying the standards by which schools are judged. Since those schools declared underperforming in our study and those that experienced panel review but avoided the official label often differed little in terms of MCAS performance, vague evaluation criteria could distract school personnel with issues of equity while undermining trust and contributing little to school improvement. In this regard, consider how a Morwood administrator reacted to being declared underperforming:26 26 Of the interviews, this was the only principal who was at his school when it was declared ‘underperforming.’ The others assumed their position after the school was labeled. Education Policy Analysis Archives Vol. 16 No. 18 30 I felt rage. Built up rage… I knew scores were low across the city. Before I actually set foot here, I was given a clear message on what needed to happen once I was selected as principal. [But] I had not had time to implement any of my agenda. When I heard about this, I called the superintendent and asked him, “Why? What was going on? Why us? We’re in the middle of the pack, and others fared much worse in terms of MCAS?”… [T]he school had begun to make improvement and we were starting to address structural and instructional issues to improve student performance. All that didn’t matter. It was basically, “You’re selected, and this is it.” Such anger, resentment, and frustration—fueled in part by vague evaluation criteria—creates a difficult environment within which to begin the intervention process. Directions for Future Research Future research on state intervention in low-performing schools should make student achievement a priority. As state interventions will only increase, there is a need for longitudinal quantitative studies with a larger study population than ours to determine any correlation between being declared underperforming with increases in student performance. This would entail assessing the trajectory of schools designated underperforming with those that were not, to determine what effect, if any, state intervention had and how much of any improvement can be attributed to intervention. Such research should be complemented by qualitative investigations that explore the mechanisms of change, the “how” and “why” behind the outcomes at underperforming schools. Many participants in our study reported changes in school schedules and curricula to maximize time spent preparing for state exams. Others feared that overemphasis of math and English/Language Arts may contribute to students dropping out. The proposed studies could begin to examine these as well as other important issues. Impact of State Intervention on “Underperforming” Schools 31 References Arasim, L. (1999). States’ interventions in school districts. Washington, DC: Senate Fiscal Agency. Retrieved March 12, 2005, from http://www.senate.michigan.gov/sfa/Publications/Notes/1999Notes/NotesJanFeb99A rasim.PDF. Brady, R. (2003). Can failing schools be fixed? Washington, DC: Thomas B. Fordham Foundation. Retrieved February 18, 2005, from http://www.edexcellence.net/doc/failing_schools.pdf. Bryk, A. S., & Schneider, B. (2002). Trust in schools: A core resource for improvement. New York: Russell Sage Foundation. Charmaz, K. (2000). Grounded theory: Objectivist and constructivist methods. In N. K. Denizin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed.; pp. 509–535). Thousand Oaks, CA: Sage Publications. Coffey, A., & Atkinson, P. (1996). Concepts and coding. In A. Coffey (Ed.), Making sense of qualitative data: Complementary research strategies (pp. 26–53). Thousand Oaks, CA: Sage Publications. Cook-Sather, A. (2002). Authorizing students’ perspectives: Toward trust, dialogue, and change in education. Educational Researcher, 31(4), 3–14. Corbett, D., & August, B. (1995). Make a difference with, not for, students: A plea to researchers and reformers. Educational Researcher, 24(5), 12–17. Darling-Hammond, L. (2004). What happens to a dream deferred? The continuing quest for equal educational opportunity. In J. A. Banks & C. A. M. Banks (Eds.), Handbook of research on multicultural education (pp. 607–630). San Francisco: Jossey-Bass. Datnow, A., Borman, G. D., Stringfield, S., Overman, L. T., & Cartellano, M. (2003). Comprehensive School Reform in culturally and linguistically diverse contexts: Implementation and outcomes from a four-year study. Educational Evaluation and Policy Analysis, 25(2), 143–170. Education Commission of the States. (n.d.). No Child Left Behind: State accountability and consolidated plans. Denver, CO: Education Commission of the States. Retrieved June 10, 2003 from http://www.ecs.org/clearinghouse/42/65/4265.htm. Elmore, R. (2003). Knowing the right thing to do: School improvement and performance-based accountability. Washington, DC: National Governor’s Association Center for Best Practices. Retrieved July 21, 2006, from http://www.nga.org/portal/site/nga/menuitem.50aeae5ff70b817ae8ebb856a11010a0/. Education Policy Analysis Archives Vol. 16 No. 18 32 Erlichson, B. A. (2005). Comprehensive school reform in New Jersey: Waxing and waning support for model implementation. Journal of Education for Students Placed at Risk, 10(1), 11–32. Evans, R. (1996). The human side of school change: Reform, resistance, and the real-life problems of innovation. San Francisco: Jossey-Bass. Feller, B. (2006, May 10). Overhaul looms for rising number of schools. Boston Globe, , p. A3. Fullan, M. (2001). The new meaning of educational change (2nd ed). New York: Teachers College Press. Fullan, M., & Stiegelbauer, S. (1991). The new meaning of educational change. New York: Teachers College Press. Good, T. L., & McCaslin, M. M. (2005). CSR principal perceptions of support from the state department of education. Journal of Education for Students Placed at Risk, 10(1), 35–51. Hammer, P. C. (2005). Corrective action: A look at state takeovers of urban and rural districts. Charlestown, WV.: AEL Policy Briefs. Hargreaves, A. (2004a). Inclusive and exclusive educational change: Emotional responses of teachers and implications for leadership. School Leadership and Management, 24(2), 287– 309. Hargreaves, A. (2004b). Distinction and disgust: The emotional politics of school failure. International Journal of Leadership in Education, 7(1), 27–41. Haycock, K. (2001). Closing the achievement gap. Educational Leadership, 58(6), 2–8. Hoy, W. K., & Tschannen-Moran, M. (2003). The conceptualization and measurement of faculty trust in schools. In W. K. Hoy & C. Miskel (Eds.), Studies in leading and organizing schools (pp. 181–207). Charlotte, NC: Information Age Publishing. Keirstead, C., & Harvell, C. (2005). State support for low-performing schools: Lessons from four Massachusetts schools. Providence, RI: The Education Alliance. Lather, P. (1986). Research as praxis. Harvard Educational Review, 56(3), 257–277. Learmoth, J., & Lowers, K. (1998). A trouble-shooter calls: The role of the independent consultant. In L. Stoll & K. Myers. (Eds.), No quick fixes: Perspectives on schools in difficulty (pp. 133–143). London: Falmer Press. Legters, N., Balfanz, R., Jordan, W. J., & McPartland, J. M. (2002). Comprehensive reform for urban high schools: A Talent Development approach. New York: Teachers College Press. Massachusetts Department of Education. (n.d.). The Mass. DOE citations should be listed from the oldest to most recent publication. School review process. Retrieved March 21, 2006, from http://www.doe.mass.edu/sda/review/?section=all. Impact of State Intervention on “Underperforming” Schools 33 Massachusetts Department of Education. (2001a). School panel review report: August Middle School. Retrieved March 21, 2006, from http://www.doe.mass.edu/sda/review/cohorts/2002/. Massachusetts Department of Education. (2001b). School performance rating process: Implementation guidelines and methods. Retrieved March 3, 2005, from http://www.DOE.mass.edu/sda/ayp/cyclei/guide.pdf. Massachusetts Department of Education. (2002a). School panel review report: University Middle School. Retrieved March 21, 2006, from http://www.doe.mass.edu/sda/review/cohorts/2002/. Massachusetts Department of Education. (2002b). School panel review report: Morwood High School. Retrieved March 21, 2006, from http://www.doe.mass.edu/sda/review/cohorts/2002/. Massachusetts Department of Education. (2006a). 2006 glossary of accountability & AYP reporting terms. Retrieved August 5, 2007, from http://www.doe.mass.edu/sda/ayp/cycleIV/glossary.html. Massachusetts Department of Education. (2006b). Adequate yearly progress data. Retrieved January 13, 2007, from http://profiles.doe.mass.edu/ayp2006. Massachusetts Department of Education. (2007). Adequate yearly progress data. Retrieved December 28, 2007 from http://profiles.doe.mass.edu/ayp/ayp_report/school. McQuillan, P. J., & Welner, K. (2007). Understanding small school reform through the lens of complexity theory. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL. McRobbie, J. (1998). Can state intervention spur academic turnaround? San Francisco, CA: WestEd Policy Center. Retrieved June 14, 2006, from http://www.wested.org/pub/docs/policy/intervention/htm. O’Day, J. (2002). Complexity, accountability, and school improvement. Harvard Educational Review, 72(3), 293–329. Odden, A., & Busch, C. (1998). Financing schools for high performance: Strategies for improving the use of educational resources. San Francisco: Jossey-Bass. Perryman, J. (2006). Panoptic performativity and school inspection regimes: Disciplinary mechanisms and life under special measures. Journal of Education Policy, 21(2), 147–161. Peske, H. G., & Haycock, K. (2006). Teaching inequality: How poor and minority students are shortchanged on teacher quality. Washington, DC: The Education Trust. Retrieved July 12, 2006, from http://www2.edtrust.org/EdTrust/Press+Room/teacherquality2006.htm. Education Policy Analysis Archives Vol. 16 No. 18 34 Phenix, D., Siegel, D., Zaltsman, A., & Fruchter, N. (2005). A forced march for failing schools: Lessons from the New York City Chancellor’s District. Education Policy Analysis Archives, 13(40), 1–27. Reville, S. P. Coggins, C. T., Schaefer, B., & Candon, J. (2004). Examining state intervention Capacity: How can the state better support low performing schools and districts? Cambridge, MA: Rennie Center for Education Research & Policy at MassINC. Retrieved February 14, 2006, from http://www.renniecenter.org/research_docs/0411_StateIntervention.html. Reville, S.P. Coggins, C. T., & Candon, J. (2005). Reaching capacity: A blueprint for the state role in improving low performing schools and districts. Cambridge, MA: Rennie Center for Education Research & Policy. Retrieved February 26, 2006, from http://www.renniecenter.org/research_docs/0504_ReachingCapacity.html. Reville, S. P. (2006, January 3) Bring teachers to the table. The Boston Globe, p. A13. Rudo, Z. H. (2001). Corrective action in low-performing schools and school districts. Austin, TX: Southwest Educational Development Laboratory. Retrieved from http://www.sedl.org/pubs/catalog/items/pol91.html. Sacchetti. M., & Jan, T. (2006, May21). Bilingual law fails first test. Boston Globe, p. A1 and A8. Seder, R. (2000). Balancing accountability and local control: State intervention for financial and academic stability. Los Angeles, CA: Reason Public Policy Institute. Retrieved September 15, 2008, from https://www.policyarchive.org/bitstream/handle/10207/5903/ps268.pdf. Shepard, L. A. (2001). The role of classroom assessment in teaching and learning. In V. Richardson (Ed.), Handbook of research on teaching, (pp. 1066–1101). Washington, DC: American Educational Research Association. Silva, C. (2006, June 28). State may soften its tone on MCAS. Boston Globe, p. B4. Sipe, L., & Ghiso, M. (2004). Developing conceptual categories in classroom descriptive research: Some problems and possibilities. Anthropology and Education Quarterly, 35(4), 472–485. Spradley, J. P. (1980) The ethnographic interview. New York: Holt, Rinehart, & Winston. Spreng, C. (2005). Policy options for interventions in failing schools. Santa Monica, CA: Pardee Rand Graduate School. Retrieved March 17, 2007, from http://www.rand.org/pubs/rgs_dissertations/RGSD187/. Stark, M. (1998). No slow fixes either: How failing schools in England are being restored to health. In L. Stoll & K. Myers (Eds.), No quick fixes: Perspectives on schools in difficulty (pp. 34–43). London: Falmer Press. Impact of State Intervention on “Underperforming” Schools 35 Stiefel, L., Rubenstein, R., & Schwartz, A. (2004). From districts to schools: Distribution of resources across schools in big city districts. Paper presented at the annual meeting of the American Educational Finance Association, New York, NY. Storey, A. (2006). The search for teacher standards: A nationwide experiment in the Netherlands. Journal of Education Policy, 21(2), 215–234. Tucker, M. S., & Toch, T. (2004). Hire ed: The secret to making Bush’s reform law work? More bureaucrats. Washington Monthly. Retrieved June 15, 2006, from http://www.washingtonmonthly.com/features/2004/0403.toch.html. Turner, L. (1998). Turning around a struggling school: A case study. In L. Stol & K. Myers (Eds.), No quick fixes: Perspectives on schools in difficulty (pp. 96–106). London: Falmer Press. U.S. Department of Education. (2002). No Child Left Behind Act: A desktop reference. Retrieved March 29, 2005, from http://ww.ode.state.or.us/nclb/pdfs/nclb-desk-reference.pdf. U.S. Department of Education. (2003). No Child Left Behind: Accountability and adequate yearly progress (AYP). Paper presented at the National Title I Directors’ Conference, Washington, DC. Retrieved March 3, 2005, from http://www.ed.gov/admins/lead/account/ayp203/edlite-index.html. Vu, P. (2006, March). States work to narrow teacher equity gap. Stateline.org [online]. Washington, DC: Pew Research Center. Retrieved June 24, 2006, from http://www.stateline.org/live/ViewPage.action?siteNodeld=!&contentld=12138 . Wong, K. K., & Shen, F. X. (2003). Measuring the effectiveness of city and state takeover as a school reform strategy. Peabody Journal of Education, 78(4), 89–119. Ziebarth, T.(2004). State takeovers and reconstitutions policy brief. Denver, CO: Education Commission of the States. Retrieved August 24, 2006, from http://www.ecs.org/ecsmain.asp?page=/html/publications/home_publications.asp. Education Policy Analysis Archives Vol. 16 No. 18 36 Appendix A Teacher and Administrator Interview Questions Although all interviewees were asked the questions below, as other topics and issues emerged they were explored to the extent possible. Most interviews lasted between 45 and 70 minutes. 1) Can you recall how you felt, what your thoughts were, when you learned that your school would be reviewed for being potentially underperforming? 2) Can you share with me what that process was like and has been for you as a teacher and for the school? 3) Is there anyone or group, in particular, that you think this process has or did affect the most? 4) Do you think the school’s designation as underperforming is a reflection of yourself and your work; or a reflection of your peers? 5) Having gone through this process, what are your thoughts on state intervention in schools? 6) Do you think others view you or your school differently? 7) Having gone through this process, what is your view on state intervention in schools? 8) Whom do you think should be held accountable for student performance? 9) Is there an emphasis on the MCAS at your school? 10) What is your opinion of that emphasis or lack thereof? 11) What is your opinion of the MCAS as the primary measure of your performance as a teacher? E ducation P olicy A nalysis A rchives V ol. 16 N o. 18 38 Item E ssential V ery Im portant U ncertain N ot V ery Im portant N ot im portant at all T he D O E values principal input and opinions in the intervention process. 2 11 4 4 1 T he intervention process has positively affected teacher m orale. 1 5 5 6 5 T he D O E has been a visible presence at m y school during this intervention. 1 9 1 8 3 A s part of the intervention, it’s im portant for the D O E to have a regular presence at m y school. 1 13 5 3 0 T he D O E understands the challenges m y school faces w ith serving our student population 1 3 6 8 4 In general, teachers at m y school agreed that the school w as under-perform ing at the tim e of the initial D O E review . 2 7 3 7 2 T he D O E intervention at m y school has been effective, so far. 2 8 6 5 1 Im pact of State Intervention on “U nderperform ing” Schools 37 A p p en d ix B P rin cip a l S u rvey R a w C o u n ts Survey instructions: P lease rate how im portant each of the follow ing elem ents are to the effectiveness of the D O E intervention at your school using the scale [indicated]. Item E ssential V ery Im portant U ncertain N ot V ery Im portant N ot im portant at all T eacher buy-in 19 2 1 0 0 Principal buy-in 20 2 0 0 0 Parental support for the intervention 1 10 10 1 0 T rust betw een school personnel and the D O E 6 12 3 0 0 G ood teacher m orale 8 12 1 0 0 L ow teacher turnover 9 5 4 4 0 T he ability to rem ove ineffective teachers 15 6 0 0 0 A dequate financial support from D O E 13 8 1 0 0 School personnel trusting that the intervention w ill benefit the school 14 8 0 0 0 R eceiving assistance from the D O E for your school’s specific needs upon request 11 8 3 0 0 Student perform ance on M C A S is an accurate m easure of student achievem ent in m y school. 3 7 4 6 1 Student perform ance on M C A S is an accurate m easure of m y perform ance as principal. 1 7 1 8 5 Student perform ance on M C A S is an accurate m easure of teacher effectiveness at m y school. 1 8 2 7 2 T he financial support the D O E has provided m y school since it w as declared underperform ing is adequate. 1 5 6 4 6 T he technical assistance that the D O E has provided m y school since it w as declared underperform ing is adequate. 1 5 3 7 6 T he D O E values teacher input and opinions in the intervention process. 0 10 3 7 2 Im pact of State Intervention on “U nderperform ing” Schools 39 Item A great deal M oderate M inim al N one D id not need im provem ent M issing Instruction 8 3 0 0 0 8 C urriculum 9 4 0 0 0 9 T eachers’ expectation for student achievem ent 11 4 1 0 0 11 Faculty ability to analyze M C A S data 8 4 0 0 1 8 U sing M C A S data to enhance student achievem ent 11 3 0 0 0 11 W riting the school im provem ent plan 10 1 0 0 0 10 A chieving perform ance goals 12 3 0 0 0 12 Instructional planning 12 3 0 0 0 12 U nderstanding the root causes of the school’s under-perform ance 6 6 1 0 0 6 Identifying student learning gaps 12 4 0 0 0 12 parental/caregiver involvem ent 12 2 0 0 0 12 A ligning professional developm ent to student learning gaps 12 2 0 0 0 12 O btaining adequate district financial support 9 5 7 0 0 9 A ligning the school curriculum to the State curriculum fram ew orks 13 6 0 0 0 13 Education Policy Analysis Archives Vol. 16 No. 18 40 About the Authors Patrick J. McQuillan Boston College Yves Salomon-Fernandez University of Massachusetts, Boston Email: mcquilpa@bc.edu Patrick J. McQuillan is an Associate Professor in the Lynch School of Education at Boston College. His research interests include educational change, urban school reform, and complexity theory. His most recent publication, “Understanding Small-School Reform Through the Lens of Complexity Theory: It’s ‘Good to Think With’” (Teachers College Record), is in press. Yves Salomon-Fernandez is Director of Planning and Assessment for the Graduate College of Education at the University of Massachusetts in Boston. She is a Ph.D. candidate in Educational Research, Measurement, and Evaluation at Boston College and a former data analyst for the Massachusetts Department of Education. Impact of State Intervention on “Underperforming” Schools 41 EDUCATION POLICY ANALYSIS ARCHIVES http://epaa.asu.edu Editor: Sherman Dorn, University of South Florida Production Assistant: Chris Murrell, Arizona State University Editorial Assistant: Judy Castillo, University of South Florida General questions about appropriateness of topics or particular articles may be addressed to the Editor, Sherman Dorn, epaa-editor@shermandorn.com. Editorial Board Noga Admon Jessica Allen Cheryl Aman Michael W. Apple David C. Berliner Damian Betebenner Robert Bickel Robert Bifulco Anne Black Henry Braun Nick Burbules Marisa Cannata Casey Cobb Arnold Danzig Linda Darling-Hammond Chad d'Entremont John Diamond Amy Garrett Dikkers Tara Donohue Gunapala Edirisooriya Camille Farrington Gustavo Fischman Chris Frey Richard Garlikov Misty Ginicola Gene V Glass Harvey Goldstein Jake Gross Hee Kyung Hong Aimee Howley Craig B. Howley William Hunter Jaekyung Lee Benjamin Levin Jennifer Lloyd Sarah Lubienski Les McLean Roslyn Arlin Mickelson Heinrich Mintrop Shereeza Mohammed Michele Moses Sharon L. Nichols Sean Reardon A.G. Rud Ben Superfine Cally Waite John Weathers Kevin Welner Ed Wiley Terrence G. Wiley Kyo Yamashiro Stuart Yeh Education Policy Analysis Archives Vol. 16 No. 18 42 EDUCATION POLICY ANALYSIS ARCHIVES http://epaa.asu.edu New Scholar Board English Language Articles 2007–2009 Wendy Chi Corinna Crane Jenny DeMonte Craig Esposito Timothy Ford Samara Foster Melissa L. Freeman Kimberly Howard Nils Kauffman Felicia Sanders Kenzo Sung Tina Trujillo Larisa Warhol Impact of State Intervention on “Underperforming” Schools 43 Archivos Analíticos de Políticas Educativas http://epaa.asu.edu Editor Gustavo E. Fischman Arizona State University Asistentes editoriales: Rafael O. Serrano (ASU) & Lucia Terra (UBC) Hugo Aboites UAM-Xochimilco, México Armando Alcántara Santuario CESU, México Claudio Almonacid Avila UMCE, Chile Dalila Andrade de Oliveira UFMG, Brasil Alejandra Birgin FLACSO-UBA, Argentina Sigfredo Chiroque IPP, Perú Mariano Fernández Enguita Universidad de Salamanca. España Gaudêncio Frigotto UERJ, Brasil Roberto Leher UFRJ, Brasil Nilma Lino Gomes UFMG, Brasil Pia Lindquist Wong CSUS, USA María Loreto Egaña PIIE, Chile Alma Maldonado University of Arizona, USA José Felipe Martínez Fernández UCLA, USA Imanol Ordorika IIE-UNAM, México Vanilda Paiva UERJ, Brasil Miguel A. Pereyra Universidad de Granada, España Mónica Pini UNSAM, Argentina Romualdo Portella de Oliveira Universidade de São Paulo, Brasil Paula Razquin UNESCO, Francia José Ignacio Rivas Flores Universidad de Málaga, España Diana Rhoten SSRC, USA José Gimeno Sacristán Universidad de Valencia, España Daniel Schugurensky UT-OISE Canadá Susan Street CIESAS Occidente,México Nelly P. Stromquist USC, USA Daniel Suárez LPP-UBA, Argentina Antonio Teodoro Universidade Lusófona, Lisboa Jurjo Torres Santomé Universidad de la Coruña, España Lílian do Valle UERJ, Brasil