Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 19/4/2017 Facebook: /EPAAA Revisions received: 8/10/2017 Twitter: @epaa_aape Accepted: 9/10/2017 POLICY COLLECTION National Center for Education Policy – EPAA/AAPE education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 25 Number 109 October 23, 2017 ISSN 1068-2341 Neglecting Democracy in Education Policy: A-F School Report Card Accountability Systems Kevin Murray & Kenneth R. Howe University of Colorado Boulder United States Citation: Murray, K., & Howe, K. R. (2017). Neglecting democracy in education policy: A-F school report card accountability systems. Education Policy Analysis Archives, 25(109). http://dx.doi.org/10.14507/epaa.25.3017 This article is the result of an ongoing collaboration between the National Education Policy Center and EPAA/AAPE. Abstract: Sixteen states have adopted school report card accountability systems that assign A-F letter grades to schools. Other states are now engaged in deliberation about whether they, too, should adopt such systems. This paper examines A-F accountability systems with respect to three kinds of validity. First, it examines whether or not these accountability systems are valid as a measure, that is, do these systems validly measure school quality? Second, it examines whether or not they are valid as a policy instrument. or, how far do A-F accountability systems fulfill the stated aims of their proponents—empowering parents, providing “simple” and “common sense” measures of educational quality, and so on? Finally, it examines whether or not A-F systems are valid as a democratic framework:, how well do these systems align with the broader goals of educating students http://epaa.asu.edu/ojs/ http://dx.doi.org/10.14507/epaa.25.3017 Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 2 for democratic citizenship and of incorporating parents and community members in democratic deliberation about policies for their public schools? The paper concludes that A-F accountability systems are invalid along each of these lines, and provides recommendations for democratically developing and implementing criteria for school assessment. Keywords: Accountability; Democracy; Education; Educational Policy; Politics of Education El descuido de la democracia en política de educación: Sistemas de responsabilidad con calificaciones de letra A-F a las escuelas Resumen: Dieciséis estados han adoptado sistemas de rendición de cuentas de tarjetas de calificaciones escolares que asignan calificaciones de letra A -F a las escuelas. Otros estados ahora están involucrados en la deliberación sobre si ellos también deberían adoptar tales sistemas. Este documento examina los sistemas de rendición de cuentas de A -F con respecto a tres tipos de validez. Primero, examina si estos sistemas de rendición de c uentas son válidos como medida, es decir, ¿estos sistemas valen válidamente la calidad de la escuela? En segundo lugar, examina si son o no válidos como un instrumento de política. o, ¿hasta qué punto los sistemas de rendición de cuentas de A -F cumplen los objetivos declarados de sus defensores -apoderando a los padres, proporcionando medidas "simples" y de "sentido común" de calidad educativa, etc.? Finalmente, examina si los sistemas A -F son válidos como marco democrático: ¿qué tan bien se alinean estos s istemas con los objetivos más amplios de educar a los estudiantes para una ciudadanía democrática y de incorporar a padres y miembros de la comunidad en una deliberación democrática sobre las políticas para sus escuelas públicas? El documento concluye que los sistemas de rendición de cuentas A-F no son válidos a lo largo de cada una de estas líneas y ofrece recomendaciones para desarrollar y aplicar criterios democráticamente para la evaluación escolar. Palabras-clave: Responsabilidad; Democracia; Educación; Política educativa; Politica de la educacion Desprezando a democracia na política educacional: Sistemas de responsabilização do cartão de relatório da escola A-F Resumo: Dezesseis estados adotaram sistemas de responsabilidade de cartões de nível escolar que atribuem notas de grau A-F às escolas. Outros estados estão agora envolvidos na deliberação sobre se eles também devem adotar tais sistemas. Este artigo examina os sistemas de responsabilização da A-F em relação a três tipos de validade. Em primeiro lugar, examina se esses sistemas de prestação de contas são válidos como medida, ou seja, esses sistemas são válidos para a qualidade da escola? Em segundo lugar, ele examina se eles são ou não válidos como uma ferramenta de política. ou, em que medida os sistemas de responsabilização da A-F atendem aos objetivos declarados de seus defensores - capacitando os pais fornecendo medidas "simples" e de "senso comum" de qualidade educacional, etc.? Finalmente, examina se os sistemas AF são válidos como um quadro democrático: quão bem esses sistemas se alinham com os objetivos mais amplos de educar estudantes para a cidadania democrática e incorporar os pais e membros da comunidade em uma deliberação democrática sobre as políticas para as escolas públicas? O documento conclui que os sistemas de responsabilização da A-F não são válidos ao longo de cada uma dessas linhas e fornece recomendações para desenvolver e aplicar critérios de forma democrática para avaliação escolar. Palavras-chave: Responsabilidade; Democracia; Educação; Política educacional; Política educacional Neglecting Democracy in Education Policy 3 Neglecting Democracy in Education Policy: A-F School Report Card Accountability Systems In this paper, we examine school report card accountability systems that assign A-F letter grades to schools, which many states have adopted.1 We will argue that A-F accountability schemes are an instance where democracy and democratic values have been neglected in education research and policy. The premise of A-F report card systems is simple: they make information about school quality available to students, parents, schools, and communities in simple and direct terms, pressuring schools to improve their academic performance. We begin by describing A-F report card accountability systems and delineating the rationales advanced for them. We proceed to examine A- F school report cards with respect to three kinds of validity. The first is whether they are valid as a measure: do these systems validly measure school quality? The second is whether they are valid as a policy instrument: how far do report cards fulfill the stated aims – namely, providing clear and commonsense measures of school quality that empower parents and drive school improvement – of their proponents? The third is whether they are valid as a democratic assessment framework: how well do A-F report cards align with the broader goals of educating students for democratic citizenship and incorporating parent and community members in democratic deliberation about policies for their public schools? To forecast the argument, we will contend that there is good reason to think that many such A-F report card systems are technically flawed: they fail to validly measure and represent school quality, and they typically fail to drive the school improvement they promise. There is also good reason to think that they are democratically flawed: they typically fail to measure, and reward or punish, how well schools promote good democratic citizenship. They are, in effect, blind to the democratic educational outcomes required of schooling in democratic society. And they most often smuggle in, and present as given, conceptions of schooling and school quality that should be subject to deliberation among citizens. In particular, we find that they often covertly promote a neoliberal view of schooling, which holds, roughly, that schooling should be economically oriented and prepare students to interface properly with markets. (We say more about neoliberalism later.) In the case of A-F accountability systems, education researchers and policymakers appear to labor under the illusion that “pure” conceptions of schooling and school quality, insulated from the moral and political values of researchers, policymakers, and citizens, can be discovered and used to drive educational improvement. But they are mistaken, to the detriment of democratic education and education policy: beyond the general requirement of education in democratic society to sustain democracy itself, there can be no fixed or pure view of schooling and school quality, but rather only a succession of educational aims that emerge from deliberation among citizens, shot through with the values and aspirations of those citizens. It would be better, we contend, to recognize that democracy is foundational – not corollary or supererogatory – to education research and policy and, for that reason, to abandon, or at the very least substantially remediate, A-F school grade systems.2 1 For simplicity, we often use “A-F letter grades” or “school report cards” to refer to A-F school report card accountability systems. 2 A number of states have adopted school rating and accountability systems that function similarly to A-F grades. Instead of letter grades, these systems employ five scale rating schemes that range, for example, from “exemplary school” to “focus school.” While in this paper we confine ourselves to evaluating A-F grading schemes, in particular, the argument developed here may apply to sufficiently similar rating systems. Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 4 To be clear, we evaluate A-F accountability systems from the normative framework of democracy and democratic values.3 We take public education to be the bedrock of democracy and the wellspring of robust democratic citizenship. On our view, there can be no evaluation of education policy, and no inquiry more generally, entirely shielded from values (Longino, 1990; Putnam, 2002). Because values cannot be drained away from policy evaluation, we stand upon defensible values, namely, democratic values. School Report Cards Sixteen states have adopted accountability systems that assign A-F grades to schools.4 Following the passage of the Every Student Succeeds Act (ESSA), other states are now engaged in deliberation, often contentious, about whether they too should adopt such systems, and how they should be conceived and implemented. Measures used to determine A-F grades for schools vary by state but often include graduation rates, ACT/SAT participation and scores, standardized student achievement test scores, growth in academic test scores, and attendance rates. A-F grades have associated rewards and punishments, which vary by state. In Florida, for example, the Opportunity Scholarship Program allows students who have attended schools earning either one “F” or three consecutive years of “D” grades to exit and enroll in higher-performing public schools within their district or any other district in the state, provided space is available.5 The A-F accountability system in Indiana requires the State Board of Education to intervene with a menu of options in schools that have received an “F” grades. Options include merging the school with a nearby higher-performing school, assigning a “special management team” to operate all or some part of the school, closing the school, and revising the school’s improvement plan, among others.6 Such state sanctions are examples of direct or bureaucratic accountability – systems where state officials determine rewards and punishments. Typically, however, A-F school grading systems also incorporate market accountability – systems that allow parents and students to make choices about leaving one particular school for another, taking funding with them. Vehicles for market accountability are often choice and voucher programs. For example, the Indiana Choice Scholarship Program provides eligible students with state funding for partial or full tuition costs at participating choice schools, including religiously affiliated schools.7 Such programs make schools indirectly accountable; when information about their performance is disseminated in A-F grades, families decide whether students will remain in a school or not. Proponents of choice systems maintain that allowing parents to remove their children from schools receiving low grades will ultimately ensure that only high-performing schools survive. A-F school grading systems have considerable intuitive appeal to policymakers and parents 3 For an account of the relationship between democracy and education, see: Gutmann (1999). For an account of deliberative democracy more generally, see: Gutmann & Thompson (2004). 4 For detailed information about state accountability systems, see Table 1 below. 5 For additional information about the Florida Opportunity Scholarship Program (OSP), see Florida Department of Education (n.d.). The OSP as initially implemented allowed students to enroll in participating private schools. The Florida Supreme Court declared the OSP private school option unconstitutional in 2006. See: Dillon, S. (2006). 6 For detailed information about Indiana School accountability, see: Ind. Code IC § 20-31-9 (Office of Code Revision, Indiana Legislative Services Agency [2010]). Retrieved from https://iga.in.gov/legislative/laws/2016/ic/titles/020/. IC § 20-31-9-4 details state intervention into schools that have received “F” grades. 7 For a list of Indiana participating choice schools, see: Indiana Department of Education (2016). Neglecting Democracy in Education Policy 5 as a good way to convey the quality of schools, to foster parental participation, and to spur school improvement. There is reason to become skeptical of the validity of A-F school grading systems, however, when one considers their rationales and features more carefully, as we do in this paper. Below we look closely at rationales states have offered for implementing state A-F report cards systems. Rationales for School Report Cards Implemented over the last 17 years or so, the A-F grading systems are a somewhat recent variation within the accountability movement in public education (Meens & Howe, 2015). Florida was the first to adopt an A-F system. Jeb Bush, then governor of Florida, worked with the state legislature to craft and implement his “A+ Education Plan” in 1999, which put school A-F grades at the center. Students who attended schools that received an “F” two out of four years were eligible either to attend a higher-performing public school or to receive a voucher that could be used to attend a participating private school (Figlio & Lucas, 2004). While Florida policymakers have substantially revised the original A+ Plan, A-F grades remain central to Florida’s accountability system. Fifteen states have now followed Florida in constructing accountability measures around A- F school grades. Rationales given for A-F systems are strikingly similar across states, as if they reverberate in an echo chamber. Florida is frequently cited as an obvious success of A-F systems, and other states frequently cite similar—or indeed, identical—rationales when they choose the A-F path. For example, Jeb Bush’s Foundation for Florida’s Future (n.d.) argues: “Assigning a letter grade (A-F) is a way to report a school’s effectiveness in a manner everyone can understand. Used along with rewards for improving schools and support for schools that need to improve, grading schools encourages them to make student achievement their primary focus.” Similarly, the Arizona Department of Education (2013) states “the A -F Letter Grade System was created to provide clear, easy to understand information to parents so that they could base their educational decisions on the best information available about the overall academic performance of schools and districts/charter holders.” And in Utah, A-F proponents contend: “With this important accountability system in place, Utah is empowering everyone—whether school administrators, parents, classroom teachers or citizens—to make informed choices and to identify ways to strengthen and improve all of our schools for the benefit of every student in Utah” (Utah State Senate, 2013). School report cards, proponents suggest, “give schools a tool to encourage more parental and community involvement.” Such involvement is assumed important because “schools with higher levels of parent and community involvement have a better chance of succeeding” (Utah State Senate, 2013). Making an explicit link to the Florida system, Utah’s school grading website prominently features a quote from Jeb Bush—“what gets measured gets done”—and provides other rationales that reference Florida. The Indiana Department of Education (n.d.) suggests that “giving schools letter grades for their performance—just as we do for our students— ensures parents, students, educators and communities understand how their schools are performing.” They write further that “Indiana’s A through F grading system gives parents, students, educators and communities a clear and concise assessment of how well their schools are doing.” The West Virginia Department of Education (n.d.) echoes Indiana: “giving schools letter grades for their performance—just as we do for our students – ensures parents, students, educators and communities understand how their schools are performing.” Furthermore, “West Virginia’s A-F school grading system gives parents, students, educators and communities clear and concise information on how well their schools are Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 6 doing.” Private organizations such as Michelle Rhee’s Students First, Jeb Bush’s Foundation for Excellence in Education, and the American Legislative Exchange Council (ALEC) have added significant voices to the echo chamber, advocating for the creation of more such A-F accountability systems. Students First (2013a) had, until it merged with 50CAN, assigned A-F grades and GPA scores to states based on the extent to which they “empower parents,” “elevate the teaching profession,” and “spend wisely and govern well,” which the organization took to require, among other policies, assigning A-F grades to all K-12 schools. Students First writes: “A simple, common sense solution is to provide families with easy-to-understand annual school report cards, much like parents already receive about their kids’ performance” (2013b, p. 1). ALEC has also endorsed A-F letter grades. Describing the adoption of letter grades in North Carolina, ALEC contends that A-F grades are “a crucial step toward increasing transparency in the system”; such grades, one ALEC report argues, describe school performance “on a universally understood scale” (Ladner & Myslinski, 2014, p. 2). It appears, then, that the chorus in favor of A-F systems seems to be singing the same refrain: A-F systems supposedly are clear, concise systems that let everyone know how schools are doing and encourage parents to be involved in school choices and systems. Embedded in these claims, however, are several assumptions that need to be closely examined. One assumption is that these systems accurately and adequately measure what they purport to measure (school quality) and that they actually advance goals they purport to advance (parental empowerment, democratic engagement and citizenship, and so on). Another assumption is that fostering the democratic aims of education need not be among the considerations that go into designing accountability systems and assessing their validity. The following segments provide a close examination of these assumptions, finding them questionable at best. The Validity of School Report Cards as a Measure of School Quality Do state A-F school grades serve as valid indicators of school quality? Space limitations do not permit a description of each of the 16 state systems. To be sure, there are differences among state plans (see Table 1 below for detail on individual state systems). Despite their proliferation and variation, there has been relatively little credible research on how far these state systems validly measure school quality. What is known comes primarily from a set of papers produced by university researchers in Oklahoma at The Oklahoma Center for Education Policy and The Center for Education Research and Evaluation. These reports raise substantial doubts about the validity of the Oklahoma A-F system as a measure. To our knowledge, these papers provide the best and most rigorous examinations of the validity of A-F school grading systems as a measure of school quality to date, and so we rely heavily on them in this analysis. We found that all state A-F school grading systems share four pivotal features with Oklahoma’s: (1) school quality is summarized in a single composite letter grade8 on (2) a 5-point categorical scale (3) using proficiency levels to measure academic achievement. And (4): A-F school report cards are composite scores of unmediated outcomes. This fourth feature implicitly assumes that the school itself is primarily, if not exclusively, responsible for student performance. Because 8 The Ohio accountability system does not yet assign single composite letter grades to express school quality. Single letter grades are now being considered. Now, the Ohio A-F system assigns a variety of letter grades to schools for different outcomes: achievement, progress, gap closing, graduation rate, and so on. See: Ohio Department of Education (2016). Neglecting Democracy in Education Policy 7 the four features are, indeed, shared across all state A-F systems, the findings from Oklahoma provide a source of criticisms that generalize relatively straightforwardly across other state systems. Questions about and criticisms of each component follow. A Single Composite Grade A single composite score as an index of school qualities is a dubious proposition. It is by no means clear what a single grade can mean across such a diverse array of criteria – achievement, attendance rates, dropout rates, advanced class offerings, and so on (see Table 1 for an illustration of the range of possible criteria). Little, if any, attention is paid to how to justify combining the diverse components of each grade to render a value on the A-F scale. For example, in addition to whether or not to include attendance as a criterion, policymakers have to decide how heavily to weight it if they do: 10%? 20%? Should improvement in achievement levels be calculated, or should only raw achievement scores be included? The selection and weighting of criteria seem to have no basis other than the seat-of-the pants intuitions of policymakers woefully lacking in technical knowledge and skills. Five-Point Scale A-F grades exemplify a crude categorical scale. This produces considerable imprecision. Schools with the same grade are represented as equivalent when they can differ substantially. Within the five categories differences are rendered invisible, and there is no way of knowing if the difference between an “F” and a “D” is of the same magnitude as the difference between a “D” and an “C,” or if the difference between a “C” and “B” is of the same magnitude as the difference between a “B” and an “A.” But the problem goes deeper than simply imprecise scaling. Successfully remedying the problem of the imprecision of the A-F scale assumes that the grades are potentially intelligible, if imprecise, indicators of school quality, which is by no means evident. The numerical intervals of computed composite scores that are translated into the various grades, like the weighing of the various criteria that go into the computations, have no firmer basis than unprofessional intuition. The fundamental problem here, that a more precise scale cannot remedy, is the assumption, discussed in (1), that a single composite score for school quality is meaningful. Proficiency Level as Measures of Academic Achievement The Oklahoma findings reveal serious problems of imprecision and lack of interpretability associated with the use of proficiency levels to represent the academic achievement component of school grades. Thirty-three percent of Oklahoma school grades are based on student achievement values. However, the numerical test scores are grouped into only four proficiency levels: unsatisfactory, limited knowledge, proficient, and advanced. It is these calculated proficiency levels that are used in the grading formula—and also in calculations of academic growth, weighted at 34% in the grading formula. The procedure of converting original test score data to proficiency levels and using the new proficiency data to produce values for achievement and growth introduces unnecessary imprecision because it “amounts to throwing away information about examinee test performance” (Oklahoma Center for Education Policy & Center for Educational Research and Evaluation [OCEP & CERE], 2013a, p. 12) and thereby masks otherwise detectable differences in student academic performance within proficiency levels (Dean Ho, 2008). Such conflating of data muddies its interpretation. Empirical analysis of Oklahoma school grades revealed, for example, that there were practically no differences in average science and reading scores among “A,” “B,” and “C” schools. Students in “C” schools had higher average science scores than students in “B” schools. And students in “F” schools appeared to have had higher average reading and math achievement than students in “D” schools. Further, certain schools Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 8 with lower letter grades performed better in mathematics than schools with higher letter grades (OCEP & CERE, 2013b, p. 12-14). Here it may be asked: “If a letter grade, which is based primarily on standardized test scores,9 does not necessarily tell us anything about school differences in reading, math, and science outcomes, what does it tell us” (OCEP & CERE, 2013b, p. 13)? The answer here seems to be that it tells us very little or nothing. To be meaningful, the letter grade would need to represent a school’s performance pattern, but it turns out that within-school variation across subject areas, across grades, and across the academic year fluctuates a great deal. Thus, it is never clear what an “A” is or what an “F” indicates (OCEP & CERE, 2013b, p. 5). A-F School Report Cards as Composite Scores of Unmediated Outcomes The findings of the celebrated Coleman Report (Coleman et al., 1966), produced 50 years ago, have proved to be impressively robust: schools account for a remarkably small amount of the variance in student achievement scores (perceived as remarkably small in the mid-1960s) (Borman & Dowling, 2010). Credible empirical research continues to show that school effects typically account for less than 30% of student academic performance (Nye, Konstantopoulos, Hedges, 2004; Rockoff, 2004; Rowan, Correnti, Miller, 2002). Using only student academic performance and other isolated outcome measures to assign A-F school grades is, then, confusing—or even deceptive—because it ignores and obscures many important factors that contribute to school performance. Letter grades ignore, for example, the well-documented correlation between socioeconomic status and attendance and graduation rates (OCEP & CERE, 2013a, p. 5) and they attribute academic proficiency changes directly to schools that students attended only most recently (OCEP & CERE, 2013a, p. 15). The “primary assumption of the A-F accountability system, that student test scores can be dissected and manipulated into valid indicators of school performance, is simply false” (OCEP & CERE, 2013b, p. 8). Two more recent papers examining the Oklahoma A-F system, produced by the same Oklahoma researchers, corroborate these concerns about the validity of school report cards as a measure of school quality. The papers document a number of flaws in the Oklahoma letter grades. The researchers find, for example, that the Oklahoma letter grades tend to hide, rather than reveal, achievement gaps. They write: “minority and FRL students in the lower ranking schools outperformed their minority and FRL peers in higher ranked schools… Further, FRL students in the lowest performing schools actually had higher average achievement than their FRL peers in the highest ranked schools” (Adams et al., 2016a, p. 15). More generally, they doubt the “informational significance” of A-F letter grades – their ability to validly measure and express school quality. They write: “After removing achievement variance attributed to factors unrelated to teaching or school effectiveness, letter grades were unable to differentiate schools by average student achievement… Informational significance is lost on grades that hide achievement variance within and between schools, making any diagnostic and improvement use of A-F grades ineffectual” (Adams et al., 2016b, p. 23). In sum, they find that “school grades do not accurately represent achievement patterns within schools, nor are they suitable for distinguishing between higher performing and lower performing schools” (Adams et al., 2016a, p. 19). Despite such weaknesses, A-F school report cards are one among many school 9 At the time of the report, 33% of the Oklahoma grade was based on status achievement scores and another 34% is based on student growth as a function achievement scores. In all states for which we were able to determine the weightings, achievement scores are heavily weighed, typically accounting for a least 50% and often more. See Table 1 below for more detail. Neglecting Democracy in Education Policy 9 accountability systems spawned by No Child Left Behind’s mania for assessment. State after state claims that school grades are intuitive and easy for parents and the public to understand, since they are analogous with subject matter grades, with which virtually everyone is familiar. School grades are thus touted as providing valuable information to parents in their decision-making about schools, facilitating increased and more effective participation on their part, and ultimately fostering school improvement. These are largely claims about the validity of A-F school grading as a policy instrument, the topic of the next section. However, we make the preliminary observation here that it is unlikely that such grading systems can accomplish purported policy objectives if they fail on the prerequisite of validity—if they do not in fact accurately measure school quality. And they do in fact fail: as we show above, they do not and cannot provide an accurate assessment of school quality. Although there is some evidence that parents do, indeed, find school report cards useful in evaluating schools, especially when presented with appealing graphics (Mikulecky & Christie, 2014, p. 9-13), this is a case in which the perceived “face validity” of school report cards—the intuitive perception of validity—surely goes awry. “If [an A-F grading system] seems easy to understand, it is only because the use of a single indicator to represent something complex is familiar. We are used to letter grades. A truly comprehensive evaluation system is best not boiled down to a single value because it masks the very complexity it is trying to capture” (OCEP & CERE, 2013a, p. 18). The formulas by which school report cards are computed are often not readily available, and are inscrutably byzantine in any case. It would require a very atypical parent, indeed, to understand what the grades mean, particularly when it is by no means clear that they have any coherent meaning at all. One final observation about the validity of A-F school grades as a measure of school quality: to our knowledge, no state A-F system includes among its criteria democratic citizenship, the ability to engage in democratic dialogue with diverse others, and other public and civic educational outcomes.10 How far can a letter grade that makes no mention of democratic citizenship validly measure school quality in a democratic society? In sum, there are very strong reasons to reject the validity of A-F school grading systems, as currently conceived and implemented, as a measure of school quality. But the problems that beset A-F school grading systems apply not just to current systems. There are no technical fixes: the single summary evaluation on a crude five-point scale is irremediably flawed. The Validity of School Report Cards as a Policy Instrument The question of validity as a policy instrument of A-F grading systems is the question of how far such systems succeed in fulfilling proponents’ stated aims. Above, we detailed evidence of an “echo chamber,” where rationales for A-F school grading systems were similar, or indeed identical, across the states. We identified three rationales commonly articulated by proponents: (1) A-F school grades provide “simple” and “common sense” information to parents and communities about the education of their children. (2) By providing such information, A-F school grades encourage and empower citizens, parents, teachers, and administrators to participate in and take rational control of decisions about schooling. (3) A-F school grading systems work to improve schools to everyone’s benefit—as enabled and fostered by the realization of rationales (1) and (2). We argue that there are good reasons to doubt each of these rationales. 10 For detailed information about state accountability systems, see Table 1 below. Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 10 Rationale 1—letter grades provide parents and communities with clear information about school performance—is undermined thoroughly by the analysis of the previous section. However, simple and common sense school report cards may appear to the untrained eye, a modicum of technical analysis reveals them to be patently invalid representations of school quality. As previously observed, it follows that because school report cards are invalid as a representation of school quality, so must be policy instruments based upon them. The invalidity of school report cards as a representation of school quality leaves rationale one adrift, anchored in nothing. Like Rationale 1, Rationale 2—A-F school grades encourage and empower citizens, parents, teachers, and administrators to participate in and take rational control over decisions about schooling—finds its warrant in no more than common sense, apparently, for supporters cite no empirical research in its defense. We found little empirical research that speaks directly to the issue. We did find, however, a small set of recent studies on the general relationship between state accountability systems and parents’ and citizens’ attitudes toward government, their political participation, and their involvement in the education of their children. When the findings of these studies are extrapolated to school report card systems, they undermine the claim that A-F grading empowers stakeholders. Specifically, one study found that “parents residing in states with more developed assessment systems express significantly lower trust in government, substantially decreased confidence in government efficacy, and much more negative attitudes about their children’s schools” (Rhodes, 2015, p. 3). Accountability policies “demobilize parents by excluding them from key educational decisions and enmeshing their children’s schools in a punitive testing context that elicits parental anxiety and dissatisfaction” (p. 3). Significantly, parents in these states were less likely to participate substantively in the education of their children. When parents are alienated from democratic deliberation about public schooling, as they are in an A-F environment, they come to hold negative attitudes about schools in particular and government generally; in this way, they are actually separated from substantial democratic involvement with schools. Thus, rather than enhancing parental participation, more highly developed accountability systems, such as those exemplified by A-F school grading systems, actually suppressed it.11 Another recent study found very little evidence that the school performance information made available through school report cards in Ohio has been used by voters as they vote for school board members or by school board members as they make decisions about staffing. Indeed, the study finds “no evidence that voters act on these state or federal performance designations nor that school boards respond to them when making staffing decisions” (Kogan, Lavertu, & Peskowitz, 2016, p. 658). More generally, the study “indicates that despite the wide dissemination of simple and clear performance information, there is little evidence that electoral pressure served as a mechanism that motivated school board members to improve the quality of public education in Ohio” (p. 659). The study undermines the foundation of Rationale 2: if no evidence can be found that citizens and elected officials reliably use school performance information made available by school report cards, report cards cannot be said to empower citizens and elected officials to participate in and take 11 A-F school grading systems meet many of Rhodes’ criteria for determining which accountability systems count as “highly developed” and thereby suppress parental participation. These highly developed accountability systems include: (1) school ratings to measure school performance, (2) a statewide student identification system, allowing the state to link student test scores with schools or teachers, (3) rewards for high-performing or improving schools, (4) assistance to low-performing schools, and (5) sanctions for lower- performing schools. Hence Rhodes’ arguments apply broadly to A-F systems. Neglecting Democracy in Education Policy 11 rational control over decisions about schooling. Rationale 3—A-F school grading systems work to improve schools to everyone’s benefit (as enabled and fostered by the realization of rationales 1 and 2)—fails along with the others because of the cumulative relationship it bears to them. There are still further problems with this claim. As observed previously, the factors incorporated into A-F school report cards are confined to student academic performance and other outcome measures in isolation from the social, cultural, and economic context and from the policies, practices, and level of resources of schools. This is the source of two significant problems. First, confining evaluation criteria to student academic performance and other outcome measures in isolation from the social, cultural, and economic context and from policies, practices, and resources of schools is unfair to teachers, administrators, students and others: it holds them fully accountable for outcomes that they have limited power to produce. Two of the cardinal requirements for fairly implementing high-stakes testing are: (1) that all students are taught in conditions that provide a fair opportunity to learn test material, and (2) that the validity of reporting categories (proficiency levels, for example, or A-F grades) be established (American Educational Research Association, 2000). Neither of these requirements is met by school report card systems. The issue of fairness to those being held accountable is particularly germane to bureaucratic accountability, where rewards and sanctions follow directly from the report card evaluations and are assumed to be drivers of improvement. The so-called theory of action underlying bureaucratic accountability may be questioned (Lee & Reeves, 2012; National Research Council, 2011). Citing a recent white paper authored by an impressive group of educational testing policy scholars (Baker et al., 2010), the Oklahoma researchers contend that “it is a myth to think that using student test scores to punish or reward schools is a driver of improvement” (OCEP & CERE, 2013b, p. 27). In the view of these researchers, failure to improve academic outcomes emerges not from individual actors’ failings, but rather from lack of necessary resources. Given that A-F letter grades and consequent interventions in Oklahoma do not meaningfully address profound differences in capacity and school resources, there is little reason to believe that they will strengthen schools. The second significant problem with confining evaluation criteria to student academic performance and other outcome measures in isolation is that it precludes the capacity to produce the formative knowledge needed to improve performance on desired outcomes. In collapsing information from a limited number of outcome measures, grading plans divert attention from how school policies, practices, and resources interact with out-of-school factors and the characteristics of diverse students to produce (or fail to produce) desired educational outcomes. The focus on isolated outcomes, combined with the crude summary evaluations that grades on an A-F scale provide, undermines the claim that A-F grading systems function in general to improve schools. In fact, they are particularly ill -suited to address group-based gaps in achievement. In Oklahoma, for example, A-F letter grades tended to obscure, rather than reveal, within-school achievement gaps. Schools marked “A” and “B” were found to be least effective for minority students and students receiving free or reduced-price lunch (FRL) (OCEP & CERE, 2013b, p. 27). As stated before, FRL students attending “D” and “F” schools had better average math, reading, and science scores than FRL students in “A” and “B” schools. The measure of school quality embedded in the Oklahoma A-F system is blind to achievement gaps. Rather than making them visible and thus allowing communities and policymakers to address them, letter grades in this case have rendered them invisible, subsuming them into differences between schools. Almost all state plans include achievement growth as a general criterion in addition to achievement growth in the lowest quartile as a distinct criterion. Growth measures serve as a way of controlling for the influence of different student characteristics by measuring the difference between student achievement at the beginning and the end of a given period of time, on the presumption Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 12 that what happens in schools causes whatever difference exist. But this is hardly sufficient to overcome the problems associated with an exclusive focus on school outcomes: It neglects the role of social, cultural, and economic factors outside of schools, as well as of the policies, practices and resources of schools—all of which play a significant role in producing those outcomes. Before proceeding, we consider studies that have found that A-F accountability systems have driven limited school improvement. Examining letter grades in Florida and New York City, these studies find, in sum, that receiving an “F” grade boosts student achievement as measured by test scores, but that no other letter grade promotes school improvement. These studies typically suggest that school improvement associated with receipt of an F grade is spurred on by the “shaming effect” of school report cards. One study finds that schools in New York City “receiving a failing grade realized positive effects in English the first year of the sanction” but found “no evidence that receiving letter grades other than F had positive effects” (Winters & Cowen, 2012, p. 313). Against expectations, the results of receiving a D-grade “appear to been negative, not just in year 1 but in the second year as well” (p. 326). Another study of New York City finds that “summary letter grades drove improvements in student test scores in New York City schools that received an F grade” but that the “magnitude of the effect did appear to drop over time” (Winters, 2016, p. 9). Yet another study of New York City found that “the new accountability system put in place in New York City had important effects in the months that followed its launch in the fall of 2007. Math and English test scores improved in schools that received very low accountability grades” (Rockoff & Turner, 2010, p. 145-146). Finally, a study of the Florida A-F letter grade system found that “schools receiving an ‘F’ grade are more likely to focus on low-performing students, lengthen the amount of time devoted to instruction, adopt different ways to organize the day and learning environment of the students and teachers, increase resources available to teachers, and decrease principal control” (Rouse, Hannaway, Goldhaber, & Figlio, 2013, p. 275). While these papers provide support for school letter grades in a limited range, we remain deeply skeptical of A-F systems. First, these studies presume that A-F letter grades are clear and meaningful measures of school quality to begin with. As noted above, there is good reason to doubt that letter grades validly measure and express school quality. Second, the positive effects of A-F letter grades are relatively minor, impacting only certain schools receiving F grades, and still fall well short of the educational benefits promised by their proponents. Indeed, these positive effects may well be outweighed by the negative consequences documented above. Third, as noted above, report cards neglect the bulk of the factors that account for student achievement – effects beyond the walls (and control) of schools. For this reason, A-F systems may well distract citizens and elected officials alike from democratic discussion about these out-of-school effects, including poverty and socioeconomic status. Fourth, A-F systems presume that the conception of schooling and achievement embedded within them is suitable for democratic society, which is by no means clear. We say more about this fourth concern below. In summary, there are strong reasons to doubt that A-F school grades fulfill the aims articulated by their proponents and are valid as a policy instrument. Their neglect of contextual features, and of the policies, practices, and resources of schools, renders them ill-suited to drive school improvement. Rather than working to empower parents and community members in a way that promotes school involvement, they are more likely to alienate parents from democratic participation in the education of their children. The Validity of School Report Cards as a Democratic Assessment Framework Neglecting Democracy in Education Policy 13 Even if A-F school grades proved valid as a measure of school quality and valid as a policy instrument – which they do not – there are still strong reasons to hold that they are invalid as a democratic assessment framework. They are unsuited to guide schooling in democratic society for (at least) three reasons: first, they are blind to democratic educational outcomes; second, they impose a (neoliberal) conception of schooling with little apparent consideration of the range of competing educational and social visions; and, third, with anti-democratic consequences, they appear to presume that some “pure” conception of schooling and school quality, insulated from the political and ethical values of researchers, policymakers, and citizens, can be discovered and used to drive educational improvement. We detail each of these concerns below. Neglecting Democratic Educational Outcomes A-F systems appear to ignore entirely the fundamental place of schooling in preparing democratic citizens to engage in collaborative democratic deliberation. They are blind to democracy and democratic citizenship. No state A-F system measures directly the educational outcomes required to foster an effective democratic citizenry: civic engagement, the ability to engage with diverse others in authentic deliberation, understanding beliefs to be revisable and indeed revising them in light of contradictory evidence, working to maintain the conditions of democratic society, and so on. The general educational vision contained in A-F systems neglects, and undermines by crowding out, the role of schools in cultivating in students the prerequisite for democratic deliberation: democratic character, which includes the knowledge, abilities, and dispositions needed for effective participation in democratic politics. Michele Moses and John Rogers argue that democratic citizens must develop both capacities for and commitments to democratic deliberation, such as listening, weighing evidence, communicating with people from diverse backgrounds, and thinking critically about, rather than merely accordance with, authority (2013, p. 207-216). Except tangentially, no difference between “A” and “F” schools can tell us whether schools succeed in preparing students to be good democratic citizens or not. Schools that are granted “A” letter grades in existing accountability systems could be meeting these democratic educational ends considerably less well than schools receiving lower grades. Post-NCLB accountability systems, which include A-F school grades, have driven a narrowing of the curriculum away from democratic educational outcomes, especially away from the curricular content necessary for cultivating the democratic character (Meens & Howe, 2015). The intense focus on content knowledge, particularly English and mathematics, created by accountability systems has significantly limited attention to other subjects and goals, including democratic outcomes (Robelon, 2011). There is little reason to believe that A-F systems will promote, without substantial revision, democratic education. Certainly they are not aimed directly at cultivating “critical habits of the mind and the inclination to deliberate and debate conscientiously on matters of social importance” which are central to democratic character (Howe & Meens, 2012, p. 12). A-F systems are thus invalid as a democratic framework: they do little to promote democratic educational ends and indeed risk crowding these ends out of schooling. That A-F systems do not promote democratic education is not some abstract concern. Much hangs on whether or not all students, especially those who belong to historically marginalized groups, are given the tools necessary for participating in democratic politics. In democratic society, these students should be provided the abilities and knowledge for protesting the unjust circumstances into which they have been thrown, for giving voice to their experiences and making those voices forceful in democratic politics. Otherwise, their experiences and voices are denied, subsumed into dominant and narrow representations of how schools and society ought to be organized. And they are too often forced to comply with these dominant representations even as these representations diminish their own experiences and force them into alienating social and Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 14 economic positions. Any accountability system that fails to recognize the responsibility to cultivate the democratic character might well be said to help maintain existing injustice along lines of social class, gender, race, sexual orientation, and so on. To deny these historically marginalized groups the very tools necessary for participating in democratic politics is to collaborate in the process of consciously reproducing the highly unequal status quo. In this way, existing A-F systems are complicit in maintaining the existing social order and, consequently, the power and status of those who benefit from contemporary power arrangements. There is another side of this coin. When A-F systems neglect democratic educational outcomes, the problem is not only that historically marginalized groups are denied the tools needed for active democratic participation. A further, and less documented, problem is that academic, social, political, and economic elites are educated to be what Elizabeth Anderson (2010) calls “democratically incompetent.” They too are denied the tools needed for robust democratic citizenship. While they have little trouble dominating political life, the elite are nonetheless incompetent, practicing an impoverished form of democratic citizenship at best: they are typically unresponsive to the needs and aspirations of a large swath of fellow citizens, and instead govern in their own image and, typically, to their own benefit. It is apparent that “certain kinds of knowledge, as well as ignorance, exist at both ends of the hierarchy of advantage” (Howe, 2015, p. 198). But school report cards do little, or nothing, to promote robust democratic citizenship at either end of the spectrum of power. In their neglect of democratic educational outcomes, then, A-F systems doubly exacerbate democratic inequality and consequent social and economic inequality. Education policy that neglects democracy and democratic citizenship is not merely blemished; it is thoroughly wrong-headed from the start. Democratic values should not be seen as optional in education research and policy, one among many sets of values that might be promoted. Instead they should be seen as foundational, threaded into the fabric of good research and policy. No other institution is better situated to promote democratic citizenship than public schooling. Democracy is flimsy, no more than a pattern of behavior among citizens supported by institutions themselves constituted by patterns of behavior. Neglected in educational activity, policy, and research, it can wane. A-F systems, and education policy in general, cannot be properly evaluated in isolation from these normative considerations about the role of education in promoting and sustaining democracy. Imposing (Neoliberal) Conceptions of Schooling and School Quality Though in democracy citizens should be invited into deliberation about schooling, A-F systems appear to impose a particular conception of schooling and school quality with little or no consideration of competing educational and social visions. Questions about the validity of school report cards as measure of school quality and as a policy instrument, cannot be—should not be— abstracted from the broader normative discussion about the place of education within a robust democracy. Typically, however, there is little or no public deliberation about which specific outcomes need to be incorporated into assessment systems. For example, while such outcomes as job preparation commonly are promoted, there is little discourse about why such preparation is essential, how it is best defined, or how the need for such a practical outcome might be balanced with others—like preparation for participation in active citizenship. Criteria reflect particular political commitments, and they currently are being imposed with little reflection on the range of possible educational and social values. In contrast, in a democratic society the question of how schools ought to be structured should be subject to ongoing democratic deliberation (Gutmann, 1999). Implementation of particular visions should be open to revision as new reasons and contexts evolve. Proponents of the A-F systems claim they produce democratic engagement as a matter of course, as when, for Neglecting Democracy in Education Policy 15 example, Indiana policymakers state: “The greatest benefit of the A through F school grading system is heightened community awareness and increased dialogue and action among education stakeholders” (Indiana Department of Education, n.d.) And yet, existing evidence suggests that A-F systems conversely tend to stifle democratic control over educational structures. But the problem is not only that A-F systems presume, and thereby impose, a conception of schooling and schooling quality. A further concern is that the presumed view is undesirable for democratic society, rooted in neoliberalism. We say more about neoliberalism below, before describing how report cards tend to promote a distinctly neoliberal conception of schooling. For almost forty years, neoliberalism has been the ascendant political and economic framework, remaking political and economic life (Burgin, 2012; Mirowski & Phlewe, 2009; Stedman Jones, 2012). It has shaped education research and education researchers alike. The core of neoliberalism is the conviction that free markets should be spread to more and more domains of human life. For the neoliberal, the market is the best and most efficient mechanism for producing and distributing goods (Harvey, 2005; Peck, 2013). It is seen, further, as happily compatible with individual human freedom.12 The neoliberal “targets institutions and activities which lie outside of the market, such as universities, households, public administrations and trade unions… in order to bring them inside the market through acts of privatization” (Davies, 2014, p. 310.) Only through the extension of the market can efficiency and individual freedom be achieved. The development of neoliberalism as a distinct political framework began, roughly, in the 1920s and 1930s (Stedman Jones, 2012). The early neoliberal intellectuals aimed to “reconstruct a neo-liberalism that remained true to the classical liberal commitment to individual liberty” (p. 3). They feared that individual liberty, and classical liberalism broadly, were threatened not only by spreading fascism and totalitarianism, but also by New Deal liberalism, British social democracy, and Keynesian economic theory and policy. They converged on the central neoliberal position: among all economic alternatives, the free market most reliably secures individual freedom by denying any individual or group centralized authority over economic structures. Later neoliberal intellectuals refined neoliberalism, developing a more mature and coherent political framework centered on more radical advocacy of free market reform, deregulation and privatization, and monetarism. They became, in particular, more and more suspicious of any intervention into the free market. For example, Milton Friedman’s Capitalism and Freedom “presented the market as the means both to deliver social goods and to deliver the ends, the good life itself” (Stedman Jones, 2012, p. 8). Friedman writes: “there is an intimate connection between economics and politics, that only certain combinations of political and economic arrangements are possible… in particular, a society which is socialist cannot also be democratic, in the sense of guaranteeing individual freedom” (1962, p. 8). The widespread implementation, and eventual dominance, of neoliberalism began, roughly, in 1980 (Stedman Jones, 2012). The energy crisis, the debt crises, and “stagflation” during the 1970s created the economic, political, and ideological conditions in which neoliberal principles – fiscal discipline and austerity, privatization, deregulation, market reform, and more – seemed reasonable economic alternatives to reigning New Deal and Great Society liberalism and British social democracy. Neoliberal economic policy was adopted by the International Monetary Fund (IMF), the World Bank (WB), the World Trade Organization (WTO), the European Union (EU), and in the North American Free Trade Agreement (NAFTA). The infamous “structural adjustment” programs, administered by the IMF and the WB, spread free market economic policy throughout the world. Despite substantial challenge, especially recently during the “Great Recession,” the neoliberal 12 For an example of such an argument, see: Hayek (2007). Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 16 framework has proven durable. It remains the dominant organizing principle in social and economic life. Unlike its ancestor classical liberalism, neoliberalism is an active force. It works to create the conditions – social, political, economic, and ideological – needed for the proper functioning of free markets. Rather than the classical liberal imperative to clear space for individual self-determination, it seeks to construct individuals with the knowledge, skills, and dispositions needed for proper interaction with those markets. Unlike classical liberalism, neoliberalism has strengthened rather than weakened state control and monitoring over human life: Whereas classical liberalism represents a negative conception of state power in that the individual was to be taken as an object to be freed from the interventions of the state, neo-liberalism has come to represent a positive conception of the state’s role in creating the appropriate market by providing the conditions, laws and institutions necessary for its operation. In classical liberalism the individual is characterized as having an autonomous human nature and can practice freedom. In neo-liberalism the state seeks to create an individual who is an enterprising and competitive entrepreneur. (Olssen, 1992, p. 340) It is no coincidence that the widespread implementation of neoliberal policy beginning in 1980 corresponds neatly with a significant shift in education policy that began, roughly, with the publication of A Nation at Risk and culminated with the accountability systems spawned by No Child Left Behind, including school report card systems, which are but a new variation on the same general theme. During this period, education policy shifted away from the “equity regime,” in which the federal government played a narrow role in education typically confined to promoting equal educational opportunity, to a broader and more activist new policy regime, in which the federal government seeks to improve education through punitive accountability systems (McGuinn, 2005). At the heart of this new regime is the punitive neoliberal “audit culture,” which calls for constant monitoring and assessment of schools, along with associated rewards and punishments, intended to drive educational improvement. Alongside the rise of neoliberalism, we have witnessed a rapid proliferation of auditing, i.e., the use of business derived concepts of independent supervision to measure and evaluate performance by public agencies and public employees, from civil servants and school teachers to university lecturers and doctors: environmental audit, value for money audit, management audit, forensic audit, data audit, intellectual property audit, medical audit, teaching audit and technology audit emerged and, to varying degrees, acquired institutional stability and acceptance… Very few people have been left untouched by these developments. (Leys, 2003, p. 70) A-F systems exemplify neoliberal audit culture: they seek to drive educational improvement by auditing schools and rewarding or punishing them according to audit results. And, we argue below, they exemplify the activist neoliberal drive to create and maintain the individual and institutional conditions required for free markets: they tend to promote a distinctly neoliberal view of schooling that seeks to cultivate individuals with the needed skills, knowledge, and dispositions to navigate and sustain market society. The neoliberal view of schooling can be characterized by two central tenets. First, schooling should be economically-oriented and prepare students to properly interface with markets. Markets safeguard individual freedom and promote efficiency. Schooling is one of the central institutions for sustaining markets: it can, and should, cultivate individuals with the skills, knowledge, and beliefs necessary for the proper functioning of markets. Second, where possible, the provision of schooling Neglecting Democracy in Education Policy 17 should be privatized and put on the market, allowing for market competition that will promote quality schooling chosen by consumers and wither away shoddy schooling not chosen by consumers. Schools should not be “artificially” sustained if they cannot survive on the market. For the neoliberal, such tinkering with the market would be ethically and practically concerning, undermining the potential of the market to safeguard individual freedom and promote efficiency in the production, distribution, and consumption of schooling. A-F systems appear consistent with, if not outright supportive of, the first tenet of the neoliberal view of schooling: schooling should be economically oriented, training students to properly participate in free market life. These systems commonly conflate education and education for economic ends. For example, consider the rationales given for A-F school grades in a presentation produced by the Louisiana Department of Education (DoE). The Louisiana DoE (2013) contends “American education outcomes are not competitive internationally.” Reports that many other countries have outperformed the US educationally, the department suggests, have substantive economic consequences: “there is substantial cost to our country and our state associated with lower educational outcomes. Had the US closed the international achievement gap by 1998, the GDP could have been $1.3 trillion to $2.3 trillion higher in 2008.” The department notes, further, “Louisiana graduates will struggle to compete for jobs” because of inadequate school outcomes. Most new jobs, they write, will require education after high school. A-F school grades are taken to be a part of the solution to both of these (economic) problems. We find very similar discussion in other states. Some, but not all, A-F systems are generally supportive of the second tenet of the neoliberal view of schooling: privatizing the provision of schooling, such that market competition will promote quality schooling and eliminate poor schooling. Whether an A-F system supports the second tenet of neoliberal schooling depends on the accountability rewards and punishments associated with letter grades. Recall, for example, the Indiana Choice Scholarship Program, which provides eligible students with state funding for partial or full tuition costs at participating choice schools, including religiously affiliated schools. Recall too the Florida Opportunity Scholarship Program, which allows students who have attended schools earning either one “F” or three consecutive years of “D” grades to exit and enroll in higher-performing public schools within their district or any other district in the state, provided space is available. Whenever these “choice schools” are managed by corporations or non-profit organizations, A-F systems exemplify neoliberal privatization. Both report card systems move in the direction of, but do not fully endorse, the second tenet: while neither call for the general privatization of the provision of schooling, both embrace the view that the market can drive educational improvement, allowing parents and students to make choices about leaving one particular school for another, supporting schools that are selected and pressuring those that are not. Here and elsewhere, we find little to no discussion of non-market educational outcomes— cultivating, for example, good democratic citizens or ensuring that students have studied and worked with a diverse set of fellow citizens. In A-F systems, the neoliberal view of schooling (and especially its first tenet) typically crowds out democratic educational outcomes. To be clear, our worry is not that schools promote labor market skills, which can be a legitimate educational aim when held in proper balance with other educational aims. Rather, it is that A-F report cards, being as they are blind to explicit democratic educational outcomes, risk crowding out education for robust democratic citizenship. A-F systems that promote the neoliberal view of schooling to the detriment of democratic educational outcomes are undesirable in democratic society. Presuming “Pure” Conceptions of Schooling and School Quality More broadly, report cards systems appear to presume that some “pure,” or at least broadly uncontroversial, conception of schooling and school quality can found. They seek a conception of Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 18 schooling and school quality insulated from enduring moral-political reflection among citizens over education that can be used to drive school improvement. But there can be no such pure conception: any legitimate view of schooling in democratic society will emerge from deliberation among citizens, shot through with the values and aspirations of those citizens (Howe, 2009). In presuming that pure conception, they allow a particular view of schooling, and associated moral and political values, to sneak in without scrutiny. Beneath the illusion of a value-neutral conception of schooling and school quality, they typically covertly promote an undesirable neoliberal view of schooling. Consider two “domains” of questions in education research. In the technical domain, education researchers inquire into how far educational intervention X promotes educational outcome Y. The technical domain is the province of education researchers who possess the technical skills needed to answer technical questions. It is the domain, for example, of the statistician who draws on statistical methods to estimate the effects of class size reduction in a school district. In the normative domain, education researchers inquire into how far educational outcome Y is desirable, how far it conforms to the demands of robust democratic society. The normative domain is the jurisdiction of democratic citizens generally. Questions that fall in the normative domain should be subject to continued deliberation among researchers and citizens. (To be clear, the normative and technical domains are not cleanly separable in education research and policy: normative considerations will inevitably permeate the technical domain, while technical considerations will inevitably permeate the normative domain. We say more about this below.) Embedded inevitably within school report cards is some conception of schooling and school quality. To be meaningful, A-F letter grades must contain normative content. They must adopt some position, shot through with values, about the proper function of schooling in democratic society. They must adopt some view of what measures should be used to indicate fulfillment, or not, of that function. Here we inhabit the normative domain, the realm of democratic citizens. Normative views about school quality and the legitimate aims of schooling should be subject to deliberation among citizens. To presume or to impose some view of schooling and school quality, without deliberation, is to run afoul of democracy. In the case of A-F systems, education researchers and policymakers appear to have proceeded directly to the technical domain, skipping over the normative domain. They have presumed the particular (inevitably normative) conceptions of schooling and school quality embedded in report card systems, rather than holding them open to deliberation and weighing them against the range of competing educational views. It is presented as pure, or at least uncontroversial, insulated from ongoing moral-political reflection about education. A-F letter grades are presented as pure or uncontroversial measures of school quality. But there can be no pure conception of schooling and school quality. Any conception of schooling will be laden with political values because schooling must always work toward some end. Any legitimate view of schooling in democratic society will emerge from deliberation among citizens, thoroughly saturated by the values of those citizens. The presumption of a pure conception of schooling is anti-democratic, alienating citizens from deliberation. In leaping over the normative domain and into the technical domain, researchers and policymakers have neglected the foundational role of democratic values in education research and policy. Advocates of A-F systems might maintain that they should skip over the normative domain. They can, and should, bracket political and ethical values from their research. To insert their own values and aspirations into school letter grades would be anti-democratic. Instead, political and ethical questions – say, “what are the legitimate goals of schooling in democratic society?” and “what is a legitimate conception of school quality?” – are relegated to policymakers, who are democratically accountable to citizens. Education researchers should seek only to answer technical questions using their technical expertise, which will be governed only by epistemic and technical concerns and not Neglecting Democracy in Education Policy 19 contaminated by political considerations. They seek only to produce technical knowledge – how far does educational intervention X promote educational outcome Y – to be given to policymakers who will use that knowledge in deliberation.13 But this view is flawed. While education researchers and policymakers should, out of respect for democracy, remain vigilant about covertly embedding their values into A-F systems, this strategy backfires. Education research and policy, in general, cannot be insulated from political and ethical considerations (Howe, 2009; Putnam, 2002). Any conception of school quality, some version of which must be presumed in A-F systems, will be loaded with normative considerations. Failing to disclose that answers to question within the technical domain will be shot through with moral and political values will bias deliberation among policymakers and citizens, silently promoting those views. Instead of revealing the moral and political values that inevitably permeate A-F systems, it masks them, shielding them from deliberation and criticism. In sum, report card systems neglect the priority of democracy to education research and policy. They are unlikely to promote democratic educational outcomes. Rather than inviting citizens to deliberate about the host of possible educational and social visions that could be embedded in A- F letter grades, they appear to impose particular (typically neoliberal) conceptions of schooling and school quality. And they presume, wrongly, that pure, or at least broadly uncontroversial, conceptions of schooling and school quality can found and used to drive school improvement. In doing so, they often covertly promote particular values and particular views of schooling and school quality, which are shielded from deliberation and scrutiny. We find that report card systems are invalid as a democratic assessment framework. Because they are democratically invalid, A-F systems cannot be remediated with technical fixes. They are flawed normatively, beyond the reach of technical tinkering – they violate the general requirement for schooling in democratic society to promote democracy. They may well be irredeemably flawed, at least without a substantial consideration of the role of schooling and school accountability in democratic society. Conclusion and Recommendations We endorse three recommendations advanced by the Oklahoma researchers. First, policymakers should eliminate “the single grade, which cannot be composed without adding together unlike elements and promoting confusion and misunderstanding” (OCEP & CERE, 2013a, p. 6). Second, policymakers should develop “a report card format that uses multiple school indicators that more adequately reflect a school performance profile” (p. 6). Third, policymakers should enlist the services of assessment and evaluation experts in designing school accountability systems. While we find these recommendations sound, we believe that alone they are too narrow, that they fail to take into consideration the need to consider the role and responsibilities of an educational system within a democratic society. As stated above, these technical fixes alone cannot remedy the deeper democratic defects in report card systems. Therefore, we add our own recommendations to those above, noting we believe these are relevant not only to A-F grading 13 For a well-known defense of this view, and one connected to neoliberalism, see Friedman (1953). Friedman maintains that positive economics, as distinct from normative economics, “is in principle independent of any particular ethical position or normative judgments… [It] can be an ‘objective’ science, in precisely the same sense as any of the physical sciences” (p. 2). He contends that political and ethical values can, and should, be filtered from positive economics. Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 20 systems but to all school accountability systems. Given the above discussion, we recommend that in determining accountability systems for schools, policymakers should enable democratic deliberation over the many possible purposes of schooling in a democratic society before determining assessment criteria. The indicators of “school quality” must be determined through authentic conversation, reflecting the voices and experiences of all members of our democratic society—not just the narrow vision of policymakers. We recommend further that policymakers should ensure that accountability systems promote, rather than neglect or inhibit, the formation of democratic character—which must be consciously cultivated. While democratic outcomes may not be the only legitimate goal for public schools, they surely should be counted among the most essential. Unless these modifications can be made, rendering A-F systems valid as a democratic assessment framework, we recommend that they be abandoned as irredeemable. References Adams, C. M., Forsyth P. B., Ware, J., Mwavita, M., Barnes, L. L., & Khojasteh, J. (2016a). An Empirical Test of Oklahoma’s A-F School Grades. Education Policy Analysis Archives 24(4), 1- 29. Adams, C. M., Forsyth, P. B., Ware, J., & Mwavita, M. (2016b). The informational significance of A- F school accountability grades. Teachers College Record, 118(7), 1-31. American Educational Research Association (2000). Position statement on high-stakes testing. Washington, D.C.: Author. Retrieved from http://www.aera.net/About- AERA/AERA-Rules-Policies/Association-Policies/Position-Statement-on-High-Stakes- Testing. Anderson, E. (2010). The imperative of integration. Princeton, NJ: Princeton University. Arizona Department of Education (2013). The A-F letter grade system. Phoenix: Author. Retrieved from http://www.azed.gov/research-evaluation/files/2013/08/2013-a-f-letter- grades-guide-for-parents.pdf. Baker, E. L., Barton, P. E., Darling-Hammond, L., Haertel, E., Ladd, H. F., Linn, R. L., . . . Shepard, L. A. (2010). Problems with the use of student test scores to evaluate teachers. Washington, D.C.: The Economic Policy Institute. Borman, G., & Dowling, M. (2010). Schools and inequality: A multilevel analysis of Coleman’s Equality of Educational Opportunity data. Teachers College Record, 112(5), 1201- 1246. Burgin, A. (2012). The great persuasion: Reinventing free markets since the Depression. Cambridge, MA: Harvard University Press. Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Wood, A. M., Weinfeld, F. D., & York, R. L. (1966). Equality of educational opportunity. Washington D.C.: U.S. Department of Health, Education & Welfare. Davies, W. (2014). Neoliberalism: A bibliographic review. Theory, Culture, & Society, 31(7), 309-317. Dean Ho, A. (2008). The problem with ‘proficiency’: Limitations of statistics and policy under No Child Left Behind. Educational Researcher, 37(6), 351-360. Dillon, S. (2006, Jan. 6). Florida Supreme Court blocks school vouchers. The New York Times. Retrieved from http://www.nytimes.com/2006/01/06/us/florida-supreme-court-blocks- school-vouchers.html. Figlio, D., & Lucas, M. (2004). What’s in a grade? School report cards and the housing Neglecting Democracy in Education Policy 21 market. The American Economic Review, 94(3), 591-604. Florida Department of Education. (n.d.). Opportunity scholarship program. Tallahassee, FL: Author. Retrieved from http://www.fldoe.org/schools/school-choice/k-12-scholarship- programs/osp/. Foundation for Florida’s Future. (n.d.). School grades Q and A. Tallahassee, FL: Author. Retrieved from http://www.afloridapromise.org/Pages/Florida_Formula/Facts_on _the_FCAT_and_Floridas_Path_to_Success/School_Grades_Q_and_A.aspx. Friedman, M. (1953). Essays in positive economics. Chicago: University of Chicago Press. Friedman, M. (1962). Capitalism and freedom. Chicago: University of Chicago Press. Gutmann, A. (1999). Democratic education. Princeton, NJ: Princeton University Press. Gutmann, A., & Thompson, D. (2004). Why deliberative democracy? Princeton, NJ: Princeton University Press. Harvey, D. (2005). A brief history of neoliberalism. Oxford, UK: Oxford University Press. Hayek, F. (2007). The road to serfdom: Texts and documents – the definitive edition. Chicago: University of Chicago Press. Howe, K. R. (2015). The meritocratic conception of educational equality: Ideal theory run amuck. Educational Theory, 65(2), 183-201. Howe, K. R. (2009). Positivist dogmas, rhetoric, and the education science question. Educational Researcher, 38(6), 428-440. Howe, K. R., & Meens, D. E. (2012). How recent education reforms undermine local school governance and democratic education. Boulder, CO: National Education Policy Center. Indiana Department of Education. (2016, Nov. 17). Participating choice schools 2016-2017. Indianapolis: Author. Retrieved from http://www.doe.in.gov/choice/participating-schools- 2016-17. Indiana Department of Education (n.d.). A-F school accountability FAQ. Indianapolis: Author. Retrieved from http://www.svcs.k12.in.us/Downloads/revised-f-faq- 101712.pdf. Kogan, V., Lavertu, S., & Peskowitz, Z. (2016). Do school report cards produce accountability through the ballot box? Journal of Policy Analysis and Management, 35(3), 639-661. Ladner, M., & Myslinksi, D.J. (2014). Report card on American education. Arlington, VA: American Legislative Exchange Council. Lee, J., & Reeves, T. (2012). Revisiting the impact of NCLB high-stakes school accountability, capacity, and resources: State NAEP 1990 – 2009 reading and math achievement gaps and trends. Educational Evaluation and Policy Analysis, 34(2), 209-231. Leys, C. (2003). Market-driven politics: Neoliberal democracy and the public interest. New York: Verso. Longino, H. E. (1990). Science as social knowledge: Values and objectivity in scientific inquiry. Princeton, NJ: Princeton University Press. Louisiana Department of Education. (2013). Louisiana believes: Letter grade results Fall 2013. Baton Rouge: Author. Retrieved from http://www.louisianabelieves.com/ docs/test-results/10-24-13-report-card-presentation-(thursday).pdf?sfvrsn=4. Maine Department of Education. (n.d.). Questions and answers. Augusta, ME: Author. Retrieved from https://maine.gov/doe/schoolreportcards/resources/faq.html. McGuinn, P.J. (2005). No Child Left Behind and the transformation of federal education policy, 1965-2005. Lawrence, KS: University of Kansas Press. Meens, D. E., & Howe, K. R. (2014). NCLB and its wake: Bad news for democracy. Teachers College Record, 117(6), 1-44. Mikulecky, M, & Christie, K. (2014). Rating states, grading schools: What parents and experts say Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 22 states should consider to make school accountability systems meaningful. Denver: Education Commission of the States. Mirowski, P., & Phlewe, D. (2009). The road from Mont Pelerin: The making of the neoliberal thought collective. Cambridge, MA: Harvard University Press. Moses, M. S., & Rodgers, J. Enhancing a nation’s democracy through equitable schools. In P. Carter & K. Welner (Eds.), Closing the opportunity gap: What America must do to give every child an even chance (pp. 207-216). New York: Oxford University Press. National Research Council. (2011). Incentives and test-based accountability in education. Washington, D.C.: The National Academies Press. Nye, B., Konstantopoulos, S., & Hedges, L. V (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26(3), 237-257. Ohio Department of Education. (2016, May 18). Understanding Ohio’s report card. Columbus, OH: Author. Retrieved from http://education.ohio.gov/getattachment/ Topics/Data/Report-Card-Resources/Sections/General-Report-Card-Information/A-F- Report-Card.pdf.aspx. The Oklahoma Center for Education Policy and the Center for Educational Research and Evaluation. (2013a). An examination of the Oklahoma State Department of Education’s report card. Normal and Stillwater, OK: OCEP and CERE. The Oklahoma Center for Education Policy and the Center for Educational Research and Evaluation. (2013b). Oklahoma school grades: Hiding “poor” achievement. Normal and Stillwater, OK: OCEP and CERE. Olssen, M. (1996). In defense of the welfare state and of publicly provided education. Journal of Education Policy, 11(3), 337-362. Peck, J. (2013). Constructions of neoliberal reason. Oxford, UK: Oxford University Press. Putnam, H. (2002). The collapse of the fact/value dichotomy and other essays. Cambridge, MA: Harvard University Press. Rhodes, J. H. (2015). Learning citizenship? How state education reforms affect parents’ political attitudes and behavior. Political Behavior, 37, 181-220. Robelon, E. (2011). Most teachers see the curriculum narrow, survey finds.” Education Week. Retrieved from http://blogs.edweek.org/edweek/curriculum/2011/12/ most_teachers_see_the_curricul.html. Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. The American Economic Review, 94(2), 247-252. Rockoff, J. E., & Turner, L. J. (2010). Short-run impacts of accountability on school quality. American Economic Journal, 2(4), 119-147. Rouse, C. E., Hannaway, J., Goldhaber, D., & Figlio, D. (2013). Feeling the Florida Heat? How low- performing schools respond to voucher and accountability pressure. American Economic Journal, 5(2), 251-281. Rowan, B., Correnti, R., & Miller, R. J. (2002). What large-scale, survey research tells us about teacher effects on student achievement: Insights from the Prospects study of elementary schools. Teachers College Record, 104(8), 1525-1567. Stedman Jones, D. (2012). Masters of the universe: Hayek, Friedman, and the birth of neoliberal politics. Princeton, NJ: Princeton University Press. Students First. (2013a). State of education: State policy report card 2013. Sacramento, CA: Author. Students First. (2013b). Empowering parents with data: School report cards. Sacramento, CA: Author. Utah State Senate. (2013). Utah school grading. Salt Lake City: Author. Retrieved from http://utahschoolgrading.com/faq/. Neglecting Democracy in Education Policy 23 West Virginia Department of Education. (n.d.). West Virginia’s A through F school grading system: FAQs. Charleston, W.V.: Author. Retrieved from https://wvde.state.wv.us/federal- programs/resources/documents/9-A-FFAQ.pdf. Winters, M. A. (2016). Grading schools promotes accountability and improvement. New York: Manhattan Institute. Winters, M. A., & Cowen, J. M. (2012). Grading New York: Accountability and student proficiency in America’s largest school district. Educational Evaluation and Policy 34(3), 313- 327. Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 24 Appendix Table 1. State Report Card Accountability Systems Information gathered from state Department of Education websites. Each state Department of Education website was reviewed in order to find details about individual A-F school report card accountability systems. We have indicated where information could not be found. State Measured Student Achievement Scale Formula/Weighting Alabama (Final report card letter grades will be assigned December 2017.) Assessment Scores/Student Achievement Growth/Academic Progress Local Indicators Graduation Rate Attendance Alabama Plan 2020 Program Reviews Proficiency Scale (4 Level): Level I Level II Level III Level IV Weighting not readily available. Arizona (Letter grades were temporarily halted in 2014. They will resume in 2017 in a different form.) Assessment Scores/Student Achievement Achievement Gap Closure Growth/Academic Progress ELL Achievement Gains Academic Progress of Lowest Quartile Dropout Rate Graduation Rate Proficiency Scale (4 Level): Fall Far Below Standards Approaches Standards Meets Standards Exceeds Standards Elementary and Middle Schools: 50% Growth Score (25% growth of all students + 25% growth of students in lowest performing quartile) 50% Composite Score (% passing AIMS tests, % ELL students reclassified, falls far below reduction) High Schools: 50% Growth Score (25% growth of all students + 25% growth of students in lowest performing quartile) 50% Composite Score (% passing AIMS tests, % ELL students reclassified, graduation rate, dropout rate) Arkansas Assessment Scores/Student Achievement Growth/Academic Progress Graduation Rate Achievement Gap Closure Graduation Gap Closure Proficiency Scale (5 Level): Did Not Meet Partially Met Approaching Grade Level Met Grade Level Exceeded Grade Level Weighting not readily available. Neglecting Democracy in Education Policy 25 State Measured Student Achievement Scale Formula/Weighting Florida Assessment Scores/Student Achievement Growth/Academic Progress Academic Progress of Lowest Quartile Graduation Rate Advanced Placement Scores International Baccalaureate Exam Scores Dual Enrollment Credits Industry Certifications Proficiency Scale (5 Level): Level 1 Level 2 Level 3 Level 4 Level 5 ELA Achievement – 100 points Mathematics Achievement – 100 points Science Achievement – 100 points Social Studies Achievement – 100 points ELA Learning Gains – 100 points Mathematics Learning Gains – 100 points ELA Learning Gains of Lowest Performing Quartile – 100 points Mathematics Gains of Lowest Performing Quartile – 100 points Acceleration Component for Middle Schools (high school course enrollment and industry certification) – 100 points Graduation Rate for High Schools – 100 points College and Career Acceleration for High Schools – 100 points Georgia Assessment Scores/Student Achievement ACT/SAT Scores Advanced Placement Scores International Baccalaureate Exam Scores Graduation Rate Growth/Academic Progress Achievement Gap Closure Innovative Instructional Practices Career-Related Outcomes Performance of Economically Disadvantaged Students Performance of Students with Disabilities Performance of English Language Learners Proficiency Scale (4 Level): Beginning Learners Developing Learners Proficient Learners Distinguished Learners 110 Points Possible: Achievement – 50 points Progress – 40 points Achievement Gap – 10 points Challenge Points – 10 points Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 26 State Measured Student Achievement Scale Formula/Weighting Indiana Assessment Scores/Student Achievement Growth/Academic Progress Graduation Rate Advanced Placement Scores International Baccalaureate Exam Scores Dual Enrollment Credits Industry Certifications Proficiency Scale (3 Level): Not Proficient Proficient Highly Proficient Elementary/Middle Schools: 50% - Performance 50% - Growth High Schools: 20% - Performance 20% - Growth 60% - Multiple Measures (Graduation Rate, AP Scores, IB Scores, Dual Enrollment Credits, Industry Certifications) Louisiana Assessment Scores/Student Achievement Growth/Academic Progress ACT Scores Graduation Rate Advanced Placement Scores International Baccalaureate Exam Scores Proficiency Scale (5 Level): Unsatisfactory Approaching Basic Basic Mastery Advanced Elementary Schools: 100% - Evidence of Learning Middle Schools: 95% - Evidence of Learning 5% - High School Credits High Schools: 25% - End of Course Assessments 25% - ACT Scores 25% - Graduation Index (AP Scores, IB Scores, etc.) 25% - Graduation Rate Maine Assessment Scores/Student Achievement Growth/Academic Progress Graduation Rate Proficiency Scale (4 Level): Substantially Below Proficient Partially Proficient Proficient Proficient with Distinction Elementary/Middle Schools: 50% - Proficiency 50% - Growth High Schools: 40% - Proficiency 40% - Growth 20% - Graduation Rate Mississippi Assessment Scores/Student Achievement Growth/Academic Progress Graduation Rate Advanced Placement Scores International Baccalaureate Exam Scores Dual Enrollment Credits Proficiency Scale (5 Level) – Minimal Basic Passing Proficient Advanced Elementary/Middle Schools – 700 points possible: Reading Proficiency – 100 points Reading Growth All Students – 100 points Reading Growth Lowest 25% - 100 points Math Proficiency – 100 points Math Growth All Students – 100 points Math Growth Lowest 25% - 100 points Science Proficiency – 100 points Neglecting Democracy in Education Policy 27 State Measured Student Achievement Scale Formula/Weighting Mississippi Achievement Gap Closure High Schools – 1000 points possible: Reading Proficiency – 100 points Reading Growth All Students – 100 points Reading Growth Lowest 25% - 100 points Math Proficiency – 100 points Math Growth All Students – 100 points Math Growth Lowest 25% - 100 points Science Proficiency – 50 points U.S. History Proficiency – 50 points Graduation Rate – 200 points Acceleration (AP, IB, Dual Credit) Proficiency/Participation – 50 points College Readiness Rate (ACT) – 50 points New Mexico Assessment Scores/Student Achievement Growth/Academic Progress SAT/ACT Scores Dual Enrollment Credits Vocational Certifications Achievement Gap Closure School Climate Proficiency Scale – levels not readily available. Elementary/Middle Schools – 100 points possible: Proficiency – 20 points Growth – 20 points School Growth – 10 points Growth of Higher Performing Students – 20 points Growth of Lowest Performing Students – 20 points Opportunity to Learn (Attendance, Classroom Surveys) – 10 points High Schools – 100 points possible: Proficiency – 15 points Growth – 15 points School Growth – 10 points Growth of Higher Performing Students – 10 points Growth of Lowest Performing Students – 10 points Opportunity to Learn (Attendance, Classroom Surveys) – 8 points Graduation – 17 points Career and College Readiness – 15 points North Carolina Assessment Scores/Student Achievement Growth/Academic Progress ACT/SAT Scores Graduation Rate Proficiency Scale (5 Level) – Level 1 (Limited Command) Level 2 (Partial Command) Achievement – 80% Growth – 20% Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 28 State Measured Student Achievement Scale Formula/Weighting North Carolina College Enrollment College Course Completion Advanced Placement Scores Specialized Course Enrollment Level 3 (Sufficient Command) Level 4 (Solid Command) Level 5 (Superior Command) Ohio Assessment Scores/Student Achievement Growth/Academic Progress Four Year/Five Year Graduation Rates Achievement Gap Closure Annual Measurable Objectives (AMOs) SAT/ACT Scores Industry-Recognized Credentials Honors Diplomas Awarded Advanced Placement Scores International Baccalaureate Exam Scores Dual Enrollment Credits Proficiency Scale – levels not readily available. Letter grades assigned for several different categories: 1. Achievement 2. Progress 3. Graduation Rate 4. Gap Closing 5. K-3 Literacy 6. Prepared for Success Oklahoma Assessment Scores/Student Achievement Growth/Academic Progress Attendance Rate Graduation Rate ACT/SAT Scores Dropout Rate Advance Coursework Participation and Performance Proficiency Scale (4 level): Unsatisfactory Limited Knowledge Proficient Advanced Student Performance – 50% Overall Student Growth – 25% Bottom 25% Growth – 25% Neglecting Democracy in Education Policy 29 State Measured Student Achievement Scale Formula/Weighting Texas (For the 2015- 2016 school year, schools received letter grades for “domains” 1-4. A single letter A-F system using all five domains will go into effect for the 2017-2018 school year.) Assessment Scores/Student Achievement Growth/Academic Progress Achievement Gap Closure Attendance Rate & Chronic Absenteeism Dropout Rate Graduation Rate SAT/ACT Scores Advanced Placement Scores International Baccalaureate Exam Scores Dual Enrollment Credits Industry Certifications Military Enlistment Career/Technical Education Coursework Proficiency Scale (4 Level): Does Not Meet Approaches Meets Masters Domain 1 – Student Achievement – 35%* Domain 2 – Student Progress – 35%* Domain 3 – Closing Performance Gaps – 20% Domain 4 – Postsecondary Readiness – 35% Domain 5 – Community and Student Engagement – 10% * Weighting does not sum to 100% because schools use the better of their Domain 1 and Domain 2 scores. Utah Assessment Scores/Student Achievement Growth/Academic Progress Graduation Rate ACT Scores Proficiency Scale (4 level): Below Proficient Approaching Proficient Proficient Highly Proficient Elementary and Middle Schools – 600 points possible: % Proficient ELA – 100 points % Proficient Math – 100 points % Proficient Science – 100 points Growth of all students – 150 points Growth of below proficient students – 150 points High Schools – 900 points possible: % Proficient ELA – 100 points % Proficient Math – 100 points % Proficient Science – 100 points Growth of all students – 150 points Growth of below proficient students – 150 points Graduation Rate – 150 points ACT Achievement – 150 points Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 30 State Measured Student Achievement Scale Formula/Weighting West Virginia Assessment Scores/Student Achievement Growth/Academic Progress Attendance Rate Graduation Rate Reduction of Students at Risk for Dropping Out Advanced Placement Scores International Baccalaureate Exam Scores Dual Enrollment Credits Career/Technical Education Coursework Proficiency Scale – levels not readily available. Elementary Schools – 1200 points possible: ELA Proficiency – 175 points Math Proficiency – 175 points ELA Observed Growth – 100 points Math Observed Growth – 100 points ELA Adequate Growth – 100 points Math Adequate Growth – 100 points ELA Improvement of Lowest 25% - 100 points Math Improvement of Lowest 25% - 100 points 3rd Grade Reading Proficiency – 50 points At-Risk Subgroup Reduction – 100 points Attendance – 100 points Middle Schools – 1200 points possible: ELA Proficiency – 175 points Math Proficiency – 175 points ELA Observed Growth – 100 points Math Observed Growth – 100 points ELA Adequate Growth – 100 points Math Adequate Growth – 100 points ELA Improvement of Lowest 25% - 100 points Math Improvement of Lowest 25% - 100 points 8th Grade Math Proficiency – 50 points At-Risk Subgroup Reduction – 100 points Attendance – 100 points High Schools – 1500 points possible: ELA Proficiency – 250 points Math Proficiency – 250 points ELA Observed Growth – 100 points Math Observed Growth – 100 points ELA Adequate Growth – 100 points Math Adequate Growth – 100 points ELA Improvement of Lowest 25% - 100 points Math Improvement of Lowest 25% - 100 points Graduation Rate – 150 points College and Career Ready Indicators – 150 points At-Risk Subgroup Reduction – 50 points Attendance – 50 points Neglecting Democracy in Education Policy 31 About the Authors Kevin Murray University of Colorado Boulder kevin.murray-1@colorado.edu Kevin Murray earned a Ph.D. from the School of Education at the University of Colorado Boulder. He studies education policy, curriculum theory, and philosophy of education. Kenneth R. Howe University of Colorado Boulder ken.howe@colorado.edu Kenneth R. Howe is Professor Emeritus in the Educational Foundations, Policy, and Practice program area, and Associate Dean for Graduate Studies, School of Education, University of Colorado at Boulder. Professor Howe has published widely in his specializations in education policy, professional ethics, and philosophy of education. POLICY COLLECTION National Center for Education Policy – EPAA/AAPE education policy analysis archives Volume 25 Number 109 October 23, 2017 ISSN 1068-2341 Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for non- commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by the author(s) or EPAA. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), QUALIS A2 (Brazil), SCImago Journal Rank; SCOPUS, SOCOLAR (China). Please send errata notes to Audrey Amrein-Beardsley at audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. mailto:kevin.murray-1@colorado.edu mailto:ken.howe@colorado.edu http://www.doaj.org/ mailto:audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 32 education policy analysis archives editorial board Lead Editor: Audrey Amrein-Beardsley (Arizona State University) Editor Consultor: Gustavo E. Fischman (Arizona State University) Associate Editors: David Carlson, Lauren Harris, Margarita Jimenez-Silva, Eugene Judson, Mirka Koro- Ljungberg, Scott Marley, Jeanne M. Powers, Iveta Silova, Maria Teresa Tatto (Arizona State University) Cristina Alfaro San Diego State University Gene V Glass Arizona State University Gloria M. Rodriguez University of California, Davis Gary Anderson New York University Ronald Glass University of California, Santa Cruz R. Anthony Rolle University of Houston Michael W. Apple University of Wisconsin, Madison Jacob P. K. Gross University of Louisville A. G. Rud Washington State University Jeff Bale OISE, University of Toronto, Canada Eric M. Haas WestEd Patricia Sánchez University of University of Texas, San Antonio Aaron Bevanot SUNY Albany Julian Vasquez Heilig California State University, Sacramento Janelle Scott University of California, Berkeley David C. Berliner Arizona State University Kimberly Kappler Hewitt University of North Carolina Greensboro Jack Schneider College of the Holy Cross Henry Braun Boston College Aimee Howley Ohio University Noah Sobe Loyola University Casey Cobb University of Connecticut Steve Klees University of Maryland Nelly P. Stromquist University of Maryland Arnold Danzig San Jose State University Jaekyung Lee SUNY Buffalo Benjamin Superfine University of Illinois, Chicago Linda Darling-Hammond Stanford University Jessica Nina Lester Indiana University Adai Tefera Virginia Commonwealth University Elizabeth H. DeBray University of Georgia Amanda E. Lewis University of Illinois, Chicago Tina Trujillo University of California, Berkeley Chad d'Entremont Rennie Center for Education Research & Policy Chad R. Lochmiller Indiana University Federico R. Waitoller University of Illinois, Chicago John Diamond University of Wisconsin, Madison Christopher Lubienski Indiana University Larisa Warhol University of Connecticut Matthew Di Carlo Albert Shanker Institute Sarah Lubienski Indiana University John Weathers University of Colorado, Colorado Springs Sherman Dorn Arizona State University William J. Mathis University of Colorado, Boulder Kevin Welner University of Colorado, Boulder Michael J. Dumas University of California, Berkeley Michele S. Moses University of Colorado, Boulder Terrence G. Wiley Center for Applied Linguistics Kathy Escamilla University of Colorado, Boulder Julianne Moss Deakin University, Australia John Willinsky Stanford University Melissa Lynn Freeman Adams State College Sharon Nichols University of Texas, San Antonio Jennifer R. Wolgemuth University of South Florida Rachael Gabriel University of Connecticut Eric Parsons University of Missouri-Columbia Kyo Yamashiro Claremont Graduate University Amy Garrett Dikkers University of North Carolina, Wilmington Susan L. Robertson Bristol University, UK Neglecting Democracy in Education Policy 33 archivos analíticos de políticas educativas consejo editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editores Asociados: Armando Alcántara Santuario (Universidad Nacional Autónoma de México), Jason Beech (Universidad de San Andrés), Ezequiel Gomez Caride (Pontificia Universidad Católica Argentina), Antonio Luzon (Universidad de Granada), José Luis Ramírez Romero (Universidad Autónoma de Sonora, México) Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Ana María García de Fanelli Centro de Estudios de Estado y Sociedad (CEDES) CONICET, Argentina Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Miguel Ángel Arias Ortega Universidad Autónoma de la Ciudad de México Juan Carlos González Faraco Universidad de Huelva, España José Gregorio Rodríguez Universidad Nacional de Colombia, Colombia Xavier Besalú Costa Universitat de Girona, España María Clemente Linuesa Universidad de Salamanca, España Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Xavier Bonal Sarro Universidad Autónoma de Barcelona, España Jaume Martínez Bonafé Universitat de València, España José Luis San Fabián Maroto Universidad de Oviedo, España Antonio Bolívar Boitia Universidad de Granada, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Jurjo Torres Santomé, Universidad de la Coruña, España José Joaquín Brunner Universidad Diego Portales, Chile María Guadalupe Olivier Tellez, Universidad Pedagógica Nacional, México Yengny Marisol Silva Laya Universidad Iberoamericana, México Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Miguel Pereyra Universidad de Granada, España Ernesto Treviño Ronzón Universidad Veracruzana, México Gabriela de la Cruz Flores Universidad Nacional Autónoma de México Mónica Pini Universidad Nacional de San Martín, Argentina Ernesto Treviño Villarreal Universidad Diego Portales Santiago, Chile Marco Antonio Delgado Fuentes Universidad Iberoamericana, México Omar Orlando Pulido Chaves Instituto para la Investigación Educativa y el Desarrollo Pedagógico (IDEP) Antoni Verger Planells Universidad Autónoma de Barcelona, España Inés Dussel, DIE-CINVESTAV, México Paula Razquin Universidad de San Andrés, Argentina Catalina Wainerman Universidad de San Andrés, Argentina Pedro Flores Crespo Universidad Iberoamericana, México José Ignacio Rivas Flores Universidad de Málaga, España Juan Carlos Yáñez Velazco Universidad de Colima, México javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/819') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/820') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/4276') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1609') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/825') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/797') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/823') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/798') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/555') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/814') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/2703') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/801') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/826') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/802') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/816') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/3264') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/804') Education Policy Analysis Archives Vol. 25 No. 109 POLICY COLLECTION 34 arquivos analíticos de políticas educativas conselho editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editoras Associadas: Geovana Mendonça Lunardi Mendes (Universidade do Estado de Santa Catarina), Marcia Pletsch, Sandra Regina Sales (Universidade Federal Rural do Rio de Janeiro) Almerindo Afonso Universidade do Minho Portugal Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil José Augusto Pacheco Universidade do Minho, Portugal Rosanna Maria Barros Sá Universidade do Algarve Portugal Regina Célia Linhares Hostins Universidade do Vale do Itajaí, Brasil Jane Paiva Universidade do Estado do Rio de Janeiro, Brasil Maria Helena Bonilla Universidade Federal da Bahia Brasil Alfredo Macedo Gomes Universidade Federal de Pernambuco Brasil Paulo Alberto Santos Vieira Universidade do Estado de Mato Grosso, Brasil Rosa Maria Bueno Fischer Universidade Federal do Rio Grande do Sul, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Fabiany de Cássia Tavares Silva Universidade Federal do Mato Grosso do Sul, Brasil Alice Casimiro Lopes Universidade do Estado do Rio de Janeiro, Brasil Jader Janer Moreira Lopes Universidade Federal Fluminense e Universidade Federal de Juiz de Fora, Brasil António Teodoro Universidade Lusófona Portugal Suzana Feldens Schwertner Centro Universitário Univates Brasil Debora Nunes Universidade Federal do Rio Grande do Norte, Brasil Lílian do Valle Universidade do Estado do Rio de Janeiro, Brasil Flávia Miller Naethe Motta Universidade Federal Rural do Rio de Janeiro, Brasil Alda Junqueira Marin Pontifícia Universidade Católica de São Paulo, Brasil Alfredo Veiga-Neto Universidade Federal do Rio Grande do Sul, Brasil Dalila Andrade Oliveira Universidade Federal de Minas Gerais, Brasil