NEPC Review: Reassessing ESSA Implementation: An Equity Analysis of School Accountability Systems (EdTrust, September 2024) NEPC Review: Reassessing ESSA Implementation: An Equity Analysis of School Accountability Systems (EdTrust, September 2024) Reviewed by: Derek Gottlieb University of Northern Colorado November 2024 National Education Policy Center School of Education University of Colorado Boulder nepc.colorado.edu http://nepc.colorado.edu The National Education Policy Center (NEPC), a university research center housed at the University of Colorado Boulder School of Education, sponsors research, produces policy briefs, and publishes expert third-party reviews of think tank reports. NEPC publications are written in accessible language and are intended for a broad audience that includes academic experts, policymakers, the media, and the general public. Our mission is to provide high-quality information in support of democratic deliberation about education policy. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence and support a multiracial society that is inclusive, kind, and just. Visit us at: http://nepc.colorado.edu http://nepc.colorado.edu/review/essa Acknowledgements NEPC Staff Faith Boninger Publications Manager Francesca López Academic Editor Elaine Duggan Production Design Alex Molnar NEPC Director Kevin Welner NEPC Director Suggested Citation: Gottlieb, D. (2024). NEPC review: Reassessing ESSA implementation: An equity analysis of school accountability systems. Boulder, CO: National Education Policy Center. Retrieved [date] from https://nepc.colorado.edu/review/essa Funding: This review was made possible in part by funding from GREAT LAKES CENTER For Education Research & Practice the Great Lakes Center for Educational Research and Practice. This work is licensed under a Creative Commons Attribution-NonCommercial- NoDerivatives 4.0 International License. This publication is provided free of cost to NEPC’s readers, who may make non-commercial use of it as long as NEPC and its author(s) are credited as the source. For inquiries about commercial use, please contact NEPC at nepc@colorado.edu. 2 of 10 https://nepc.colorado.edu/review/essa http://www.greatlakescenter.org mailto:nepc%40colorado.edu?subject= https://creativecommons.org/licenses/by-nc-nd/4.0/ https://creativecommons.org/licenses/by-nc-nd/4.0/ http://nepc.colorado.edu NEPC Review: Reassessing ESSA Implementation: An Equity Analysis of School Accountability Systems (EdTrust, September 2024) Reviewed by: Derek Gottlieb University of Northern Colorado November 2024 Summary EdTrust recently published a report using an “equity analysis” to critique states’ account- ability plans, recommending several steps states might take to improve their systems. De- spite the language of “equity” and attention to “asset-based” framings of educational data, the vision of what high-quality accountability structures would look like and would do sim- ply recycles the naïve hopes that fueled the original push for NCLB. It calls for publicizing information that we have seen to be incomplete or gamed. It also calls for surgically tar- geting resources that we have seen to be chronically inadequate. These approaches did not achieve systemic equity or excellence when packaged as No Child Left Behind and similar policies. But this report, which relies heavily on indicators and tools that EdTrust has devel- oped or compiled itself, does not see the historical record of failure as a reason to abandon the approach. In critiquing state ESSA plans, it offers nothing more original than the very same strategy that gave rise to ESSA in the first place. http://nepc.colorado.edu/review/essa 3 of 10 NEPC Review: Reassessing ESSA Implementation: An Equity Analysis of School Accountability Systems (EdTrust, September 2024) Reviewed by: Derek Gottlieb University of Northern Colorado November 2024 I. Introduction When the Every Student Succeeds Act (ESSA) was signed into law in 2015, it produced a great deal of anxiety among a certain bloc of education reformers. ESSA broke with its pre- decessor, the No Child Left Behind Act (NCLB), by returning some discretion to state offi- cials when it came to deciding how to measure and report school quality. States were encour- aged and allowed to use “multiple measures” of educational quality, to factor in measures of “growth” as well as “achievement,” and to incorporate a non-academic outcome measure as well, such as “school climate” or “student attendance.”1 To many, this change represented an abdication of the federal government’s watchdog role in education. At the time, the Lead- ership Conference Education Fund’s Liz King said, “The lesson of the civil rights movement and community is that the federal government is the defender of vulnerable children and we are worried that with new state and local authority, vulnerable children are going to be at risk.”2 Would states backslide when it came to the high expectations and rigorous strictures of the NCLB era? EdTrust’s newly released report surveys the most recent state accountability plans and finds them wanting in exactly the ways that ESSA’s early critics worried about. Reassessing ESSA Implementation: An Equity Analysis of School Accountability Systems, by Nicholas Munyan-Penney, Abigail Jones, and Shayna Levitan, claims that states are not doing enough to guarantee equitable schooling for all children.3 It proposes a set of recommendations that would shore up the weaknesses it pinpoints. In order to get on board with its analysis and recommendations, however, one would have to accept the report’s idiosyncratic definition of “equity,” gloss over the many failures and perverse outcomes of federal educational oversight across the decades, and ultimately place http://nepc.colorado.edu/review/essa 4 of 10 an implausible amount of faith in the power of rules alone—unsupported by resources—to transform the educational destinies of our most vulnerable children. ESSA came about because lawmakers had learned some hard lessons from the NCLB era. This report asks us to learn them all over again. II. Findings and Conclusions of the Report The report finds many state accountability plans wanting. Mainly, the report says, state accountability plans are not doing enough on matters of equity—their requirements for sub- group reporting are lax, their methods of balancing growth aims with achievement aims are inconsistent, their public-facing reports are insufficiently user-friendly, and their criteria for evaluating critical school-level progress are often nonexistent. Also, according to the report, states have only “a mixed record” when it comes to “center- ing equity” in their accountability plans. The report finds that states often omit subgroup performance from school ratings, measure student growth in suboptimal ways, and fail to connect troubled schools to state and federal resources. Additionally, it finds that state reporting systems have lots of room to improve, with very few states allowing users to easily compare the performance of similar schools and demo- graphics, and even fewer offering or enabling an asset-based framing of the data. And lastly, states are failing to take advantage of federal school-improvement provi- sions—regulations that encourage states to attend to resourcing inequities, track “identi- fied” schools’ progress toward an improved status, and set criteria according to which such schools can “exit” their designation. The report’s solution to these issues is to make ESSA more like NCLB in terms of the focus of its regulations and the role of the federal government. The report would like to see states examine student outcomes at a more granular level to ensure that children have equitable access to education, and it would like to see states leverage data to identify trouble spots and target resources accordingly. Quite specifically, the report suggests that states can improve by: 1. Designing and using indicators in their state accountability systems that keep student learning front and center. 2. Developing school ratings that reflect how schools are serving all groups of students. 3. Establishing criteria that honestly identify which schools need to take steps to improve overall or for one or more student groups. 4. Reporting disaggregated accountability information in a way that is understandable, easily accessible, and widely available. 5. Providing meaningful support to schools that need to improve. http://nepc.colorado.edu/review/essa 5 of 10 III. The Report’s Rationale for Its Findings and Conclusions The report lays out the overarching rationale in a section entitled “Why We Need Federal School Accountability.”4 It is a single paragraph, but it plays all the hits. Education in the United States has historically been less than fully accessible to students experiencing vari- ous forms of marginalization, and federal regulations have been the primary tool for making universal access tohigh-quality education a reality. In particular, the report asserts that “the purpose” of public education consists in prepara- tion for participation in democratic self-governance and success in college and/or career. It declares that “all students can succeed when provided with the resources and support to succeed.” And importantly, though this is less explicit, the report leans almost exclusively on the role of information in empowering stakeholders to move institutions in particular direc- tions. That final piece undergirds the implicit and explicit evaluations of state ESSA plans, as well as the report’s summative recommendations. IV. The Report’s Use of Research Literature The report includes 39 references in total, only five of which were published in peer-re- viewed academic journals. Another four references cite other EdTrust publications. And while a handful of citations point to datasets or documents from government offices at the state or federal level, the bulk of this report’s literature base is drawn from media reports and studies published by think tanks or research institutes of one kind or another. Notably absent from the references section are any of the many strands of literature that (1) question the validity of “student learning” measures in contexts where resourcing hangs on the results,5 (2) draw connections between high-stakes federal monitoring and educational decay,6 and—particularly baffling in an equity-driven report—(3) underscore the relation- ship between dynamics of policy “targeting” and the stigmatization of marginalized student subgroups and the schools that serve them.7 The report starts with a particular view of the solution—more information, more transpar- ently, more accessibly—and it holds state ESSA plans up to the light of that solution. None of the research literature cited in the report says that this solution is the right one, or even grapples with critiques or alternatives. V. Review of the Report’s Methods The “methodology” description in the report covers data collection and internal validation. The reader is left to infer that each state’s ESSA plan was then compared to EdTrust’s own “equity principles for accountability” in order to generate a state-by-state analysis.8 http://nepc.colorado.edu/review/essa 6 of 10 VI. Review of the Validity of the Findings and Conclusions It is simply impossible to say that this report’s findings—that state accountability plans could be better at ensuring equity in particular ways—are “valid.” This is so for several reasons. First, the definition of equity in the report is neither explicitly stated nor argued for. This is a glaring omission, given recent research on the term’s slippery uses in official documents.9 One has to read between the lines of the report—and follow links to earlier EdTrust publica- tions—in order to discern what “equity” means in this context. The concept seems to involve (a) the academic performance of student subgroups, (b) the identification of schools need- ing support based on subgroup performance, (c) the targeting of resources to those schools, and (d) the monitoring and reporting of progress. But the way these factors reveal or affect equity is entirely ambiguous. The report utters the platitude, not infrequent in the equity literature, that “all students can succeed when pro- vided with the resources and supports to achieve.” Does this mean that equity will have been achieved when performance gaps among subgroups no longer exist? Or does it mean that equity will have been achieved when resource distribution is weighted and targeted correctly according to subgroup needs, presumably as revealed by performance gaps? The platitude suggests, in circular fashion, that there is no difference or conflict between an output-fo- cused concept of equity and an input-focused one: When inputs are equitably distributed, “success” will be, too. But there very plainly is a difference, and this report seems to take an outcome-side stance. The height of the report’s attention to resources pertains to federal school improvement funds—which is a 7% set-aside of Title I funding. In Wisconsin, where I live, Title I funding represents roughly 2% of the state’s education budget. So school improvement funds repre- sent a small cup from an even smaller pot of funding. The total amount of school-improve- ment funding available in Wisconsin as a whole would not even plug the gap in my own suburban school district’s operating budget. Whether it is true or not to say that all students can learn if given adequate resources, it is simply moot in the face of shamefully inadequate resources, and no degree of precision in monitoring or subgroup identification or targeting will affect that side of the equation. And that really gives the lie to the entire approach here, which ought to have been discredit- ed beyond redeeming by the very policy journey that took us from NCLB through Race to the Top and ultimately to ESSA. The report’s emphasis on monitoring and reporting—on data systems and user accessibility—reflects an almost Naderite faith in the power of consumers or interest groups to bring about institutional change through public pressure.10 If states make it easy to see gaps in subgroup performance, and if states exercise their duty to inform the public of progress, and if states fully leverage federal resources through better target- ing, then . . . what? Equity? The disappearance of outcome gaps? We already tried this ap- proach under NCLB—stringent measuring and reporting requirements, without substantial new investment of resources—and in the places where the results were not merely farcical, they were nightmarish in their unintended consequences.11 It is mind-boggling to see a new report stubbornly refusing to learn from the past, except to conveniently note that “equity” has supplanted “equality” in political currency. http://nepc.colorado.edu/review/essa 7 of 10 Also, in critiquing the report’s validity, I would be remiss to pass over the report’s fatally ambivalent approach to measuring outcomes and framing data, which also seems to have some relation to its concept of “equity”—though without an explicit definition, the precise relation remains less than clear. On the one hand, the report calls for states to use clear indicators of achievement, growth, and progress, pegged to specific stages of improvement and criteria for exiting “identified” status. And on the other hand, it wants states to adopt “asset-based” framings of the data in their reports in order to provide “a clear understanding of what accountability data says about schools’ strengths and areas of growth and the pur- pose and uses of this data.”12 The former approach, once more reminiscent of NCLB and Race to the Top, holds that while “school quality” or “student learning” might take different forms, the concepts eventual- ly converge on a single criterion against which every student and institution can be com- pared. And the latter approach, calling for asset-based framings, radically undermines that assumption by acknowledging that different forms of success or learning might represent incommensurable or even contradictory concepts of success—and that stakeholders might value these things differently. If “asset-based” data-framing is not supposed to be merely symbolic, then it is an acknowl- edgment that public schools perform multiple kinds of functions and produce multiple kinds of outcomes, all of which are legitimately valuable to various stakeholders in varying ways. Given the report’s emphasis on measuring, monitoring, reporting, and acting upon very specific indicators of school success, however, it is difficult to imagine that the report takes the substantive implications of acknowledging deviations from a standard as “assets” very seriously. VII. Usefulness of the Report for Guidance of Policy and Practice This report closely examines every state accountability plan, compiles a set of best practices, and provides states a highly specific to-do list. There is little reason to believe that these recommendations provide states with a useful road map, however—particularly in terms of achieving any definition of equity. Without substantially increasing the resources to be targeted for distribution, there are real and hard limits on the productive value of monitoring and reporting, no matter how good or robust our measurements are. It is one thing to have imagined, in the late 1990s, that federal accountability policy alone could transform public education across the country to finally make good on national promises of equal opportunity. It is quite another thing to pitch a wonkier version of the same approach three decades later, with so many tweaks and nuances and consistently underwhelming “successes” in our rearview mirror. At a certain point, faith in the idea that if state and federal governments could only compel schools to know better and try harder—in spite of the deafening consensus on the relation- ship between resources and school success13—becomes positively irresponsible. http://nepc.colorado.edu/review/essa 8 of 10 http://nepc.colorado.edu/review/essa Notes and References 1 Schneider, J. & Gottlieb, D. (2021). In praise of ordinary measures: The present limits and future possibilities of educational accountability. Educational Theory, 71(4), 455-473. 2 Huetteman, E. & Rich, M. (December 3, 2015). House restores local education control in revising no child left behind. New York Times, A25. 3 Munyan-Penney, N., Jones, A., & Levitan, S. (2024, September). Reassessing ESSA implementation: An equity analysis of school accountability system. Commonwealth Foundation. Retrieved September 30, 2024, from https://edtrust.org/rti/reassessing-essa-implementation-an-equity-analysis/ 4 Munyan-Penney, N., Jones, A., & Levitan, S. (2024, September). Reassessing ESSA implementation: An equity analysis of school accountability system (section: “Why We Need an Equity Analysis“). Commonwealth Foundation. Retrieved September 30, 2024, from https://edtrust.org/rti/reassessing-essa-implementation- an-equity-analysis/#why-we 5 Koretz, D. (2017). The testing charade. University of Chicago Press. Koretz, D. (2008). Measuring up. Harvard University Press. 6 Ewing, E. (2018). Ghosts in the schoolyard. University of Chicago Press. Shange, S. (2019). Progressive dystopia. Duke University Press. Kraft, M.A., Brunner, E.J., Dougherty, S.M., & Schwegman, D.J. (2020). Teacher accountability reforms and the supply and quality of new teachers. Journal of Public Economics, 188,(104212). 7 Jabbar, H., Daramola, E.J., Marsh, J.A., Enoch-Stevens, T., Alonso, J., & Allbright, T.N. (2022). Social construction is racial construction: Examining the target populations in school-choice policies. American Journal of Education, 128(3), 487-518. Singer, J. (2024). Quantification and the homogeneous representation of urban students in school choice research and politics. Race Ethnicity and Education, 27(4), 495-518. 8 EdTrust (2019, December 9). 5 things to advance equity in state accountability systems, school improvement, and reporting. Fact Sheet. Retrieved October 10, 2024, from https://edtrust.org/resource/5-things-to- advance-equity-in-state-accountability-systems-school-improvement-and-reporting/ 9 Levinson, M., Geron, T., & Brighouse, H. (2022). Conceptions of educational equity. AERA Open, 8. Retrieved November 11, 2024, from https://doi.org/10.1177/23328584221121344 10 Sabin, P. (2021). Public citizens: The attack on big government and the remaking of American liberalism. W.W. Norton & Company. 11 Koretz, D. (2017). The testing charade. University of Chicago Press. Au, W. & Ferrare, J.J. (Eds.). (2015). Mapping corporate education reform: Power and policy networks in the neoliberal state. Routledge. 12 Munyan-Penney, N., Jones, A., & Levitan, S. (2024, September). Reassessing ESSA implementation: An equity analysis of school accountability systems (section: “Our Vision for School Accountability Systems”). Commonwealth Foundation. Retrieved September 30, 2024, from https://edtrust.org/rti/reassessing-essa- implementation-an-equity-analysis/#vision 13 Rothstein, J. & Schanzenbach, D.W. (2022). Does money still matter? Attainment and earnings effects of post- 1990 school finance reforms. Journal of Labor Economics, 40(S1), S141-S178. 9 of 10 https://edtrust.org/rti/reassessing-essa-implementation-an-equity-analysis/ https://edtrust.org/resource/5-things-to-advance-equity-in-state-accountability-systems-school-improvement-and-reporting/ https://edtrust.org/resource/5-things-to-advance-equity-in-state-accountability-systems-school-improvement-and-reporting/ https://doi.org/10.1177/23328584221121344 https://edtrust.org/rti/reassessing-essa https://edtrust.org/rti/reassessing-essa-implementation Jackson, C.K. & Mackevicius, C.L. (2024). What impacts can we expect from school spending policy? Evidence from evaluations in the United States. American Economic Journal: Applied Economics, 16(1), 412-446. Jackson, C.K. (2018). Does school spending matter? The new literature on an old question. NBER Working Paper 25368. Retrieved November 11, 2024, from https://doi.org/10.3386/w25368 http://nepc.colorado.edu/review/essa 10 of 10 https://doi.org/10.3386/w25368