amea_2010.indd Association of Mexican American American Educators (AMAE) Journal © 2010 Dollars for Sense: Assessing Achievement Gaps in Arkansas in the Context of Substantial Funding Increases Joshua H. Barnett Arizona State University Nathan C. Jensen and Gary W. Ritter University of Arkansas Forty-four of the fi fty states have experienced school fi nance lawsuits as a consequence of funding gaps between rich and poor districts (Rebell, 2001). In some states, such as Kentucky, lawsuits brought about by property poor districts have resulted in a statewide remedy that reforms the property tax system and seeks to provide every child with an adequate education. In other states, such as New Jersey, lawsuits brought about by poor districts resulted in legal decisions and legislative responses that attempt to match the lowest wealth districts with the spending levels of the highest wealth districts. Throughout the litigation across the country and the respective judicial and legislative responses to those cases, the motivation for the cases is uniform – some individuals within the state contend that their students are being presented inferior educational opportunities compared to other students in the state. As a result of this inferior opportunity, legal action is used to leverage more resources for students (Monk, Pijanowski, & Hussain, 1997; Monk, Roellke, & Brent, 1996; Odden, Monk, Nakib, & Picus, 1995; Odden & Picus, 1992). However, these conversations are generally about all of the students in a particular district or school being underfunded. This fi nance conversation is occurring at the same time that the achievement gap debate continues to occur across the nation (Education Trust, 2005; Hertert, Busch, & Odden, 1994), especially in light of the 55th anniversary of the Brown v Board decision and the inauguration of America’s fi rst minority president. While achievement gap conversations can quickly expand to include every group of students, we focus on actual impact of policy decisions targeted to decrease the achievement gap in one state by providing categorical funding for certain students. Thus, in this paper, we respond to the nexus of the school fi nance discussion and achievement gap discussion by examining the performance trends for Hispanic and low income students on the National Assessment of Education Progress (NAEP), the Arkansas Benchmark, and the ACT. This analysis allows us to examine whether increases in student performance, or the narrowing of achievement gaps, followed these targeted increases in fi nancial resources. Specifi cally, we examine two related, but different, achievement gaps: the white-Hispanic gap and the gap between those students eligible for free and reduced lunch and those students not eligible. Before responding to these questions, we briefl y present the context of our data to explain the school fi nance situation in the state of interest. The Policy Context: The School Finance Climate in Arkansas For Arkansas, the school fi nance legal challenges began in 1983, when the Arkansas Supreme Court initially found the state’s school funding system unconstitutional under the equal protection clause of the state constitution (Dupree v Alma School District No. 30, 651 S.W.2d 90). The court found “no legitimate state purpose” and “no rational relationship to educational needs” in the state’s method of fi nancing public schools. The state responded with minor changes to the fi nance formula, but no substantive changes were required by the court or implemented by the state for nearly a decade. In May 2001, an Arkansas trial court declared the state’s education funding system “inequitable and inadequate” under the state constitution and requested an adequacy study be conducted (Lake View School District No. 25 v. Huckabee, No. 1992-5318). In November 2002, the state Supreme Court affi rmed the lower 39 Association of Mexican American American Educators (AMAE) Journal © 2010 court’s fi nding and gave the state until January 1, 2004, to improve the system (Lake View School District , No. 25 of Phillips County , et al. v. Mike Huckabee, Governor of the State of Arkansas, et al. No 01-836). In response to the Supreme Court ruling, Arkansas increased the total state appropriation for elementary and secondary education by $400 million to $1.84 billion—a 24 percent increase over the previous year. Additionally, the state now provided categorical funding for students with alternative learning environments, English language learners, free and reduced lunch students, as well as providing professional development money for teachers and special appropriations for facilities, debt service, student growth, catastrophic occurrences, and isolated districts. Not with standing these increases, the state decided that it would make no further increases to public education funding for the following year, 2004-05, which led to further litigation. Ultimately, the Supreme Court declared that the state had neglected its obligation to adequately fund public education (Lake View Sch. Dist. No. 25 v. Huckabee, 355 Ark. 617, 142 S.W.3d 643) and that the state “grossly underfunded” education. After further increases in funding, the Court decided in May 2007 that the state had met its constitutional mandate and closed the case. However, several questions remain in the background to the constitutionality discussion of Arkansas’ educational spending. First, although the emotional appeals of district offi cials seemed to sway the Supreme Court, all of the discussion centered on district level allocation. There was an attempt to provide targeted resources to the highest need students in these decisions; however, the state has not followed up to measure these impacts. Therefore, we determined that such an investigation was necessary to discover ultimately if these additional resources are positively impacting students. This work is important because each state has the constitutional responsibility to educate students and ensure that all students are given an equal educational opportunity. Most states have faced lawsuits contending that they have not provided all students with an equal opportunity. One often sought answer, especially in Arkansas, to the threat of more litigation was to increase district level resources with the idea that more money will make the system more adequate and equitable. However, we believe that Arkansans, and the citizens of other states, are more concerned with whether those resources are reaching the students and resulting in student achievement improvements. With the policy stage set, we now turn our attention to examining the methods, results, and implications of this work. Research Questions & Methodology Student Performance and Achievement Gaps Research Question: How has student performance changed over a fi ve year period from 2003 to 2008 as more resources have been put into the education system? A. Has the white-Hispanic gap decreased on the NAEP, the Arkansas Benchmark, and the ACT examinations? B. Has the poverty gap decreased on the NAEP? To respond to these questions, we assessed the extent to which these targeted increases in resources were followed by improvements in academic performance for the targeted groups of students. We assessed academic performance on three indicators of achievement: the National Assessment of Educational Progress (NAEP) exam, the Arkansas Benchmark exam, and the ACT exam. The NAEP is administered to a national sample of students in grades 4, 8, and 12, and assesses student performance in four major subject areas: reading, mathematics, writing, and science. The Arkansas Benchmark exam, which is administered to students in grades 3-8 in April of each school year, assesses student performance in the areas of math and literacy. Finally, the ACT (formerly referred to as the American College Testing program) measures college readiness in four areas: English, mathematics, reading, and science (with an optional writing test). For the NAEP and the Benchmark Exam, student performance is reported in four different categories: 40 Dollars for Sense Association of Mexican American American Educators (AMAE) Journal © 2010 Below basic, basic, profi cient, or advanced. For these comparisons, we choose to focus on the percentage of Hispanic and white students scoring in the profi cient and advanced range. For the ACT, we report average scores compare the change in white and Hispanic scores. The purpose for these comparisons was to determine if the gap in achievement levels between these two groups of students was narrowing, which would lend support to the effi cacy of increased fi nancial resources on a per-pupil basis for minority students. NAEP scores for low and high income students (as measured by free and reduced lunch eligibility) were also compared to determine if the achievement gap between these groups of students has narrowed since 2003. However, performance on the Arkansas Benchmark exam for FRL eligible and non eligible students was not included in this report due to the differences by how students are categorized. For example, on the Benchmark, achievement levels are not reported for non eligible students; data are only reported for FRL eligible students. Similarly, ACT scores for FRL eligible students are not available. Testing data for the NAEP exam were obtained directly from the NAEP website. Data for the Arkansas Benchmark exam were obtained from two different websites: The National Offi ce for Research on Measurement and Evaluation Systems (NORMES), and the Arkansas Department of Education (ADE). A breakdown of achievement levels by student sub-groups on the Arkansas Benchmark exam for the 2008 school year was only available on the ADE website. Therefore, for continuity purposes, ADE data on the Benchmark exam were used for 2006-2008, and NORMES data was used for 2004 and 2005. ACT data were also obtained from the ADE website. Not included within the analysis are the number of test takers for each examination. NAEP does not report this information, and the ACT data are reported by the districts to the state and no statewide database of test takers is maintained. The state testing offi ce does provide summary reports for the benchmark exams by grade and district; however, the number of test takers is not consistently maintained at the state level. Through our efforts to ascertain the impact of the additional categorical funding on achievement, information was also gathered and recommendations were offered (discussed in the conclusions section) regarding why these, and other, data should be maintained by the state department of education For all three exams, achievement gaps were compared starting with the 2003-04 academic year, which allowed for a comparison prior to and after the implementation of the categorical funding (which began in the Fall of 2004). After 2003-04, data for every available school year was incorporated into this report. In an effort to maintain consistency with NAEP, we examined only the 4th and 8th grade achievement levels for the Arkansas Benchmark exam, as those are the only grade levels tested on the NAEP exam. By using these grades, as well as including ACT results, we were able to compare achievement gaps at the elementary, middle, and high school level. Additionally, we were able to base our fi ndings on multiple examinations, rather than only a state or nationally administered exam. Furthermore, we contend that each of these tests (NAEP, Benchmark, and ACT) may have methodological and cultural issues, but we believe our analysis is strengthened by the triangulation approach to respond to our research question regarding the impact of additional resources to targeted populations. In the following section, we provide a comprehensive assessment of changes in student performance and achievement gaps during a fi ve year period following the infusion of additional resources (2003 through 2008). 11. http://nces.ed.gov/nationsreportcard/ 41 Dollars for Sense Association of Mexican American American Educators (AMAE) Journal © 2010 Results Before looking at the student performance data, two previous points need to be reiterated. First, test data is limited. In Arkansas, as in nearly all other states, policymakers can opt to change the statewide test offered to students. For example, Arkansas students took the Stanford Achievement Test, Ninth Edition (SAT-9) prior to 2003, took the Iowa Test of Basic Skills until 2008, and students from 2008 until the next change will take the Stanford Achievement Test, Tenth Edition (SAT-10). Therefore, while the Arkansas exams are useful, they are Arkansas specifi c and change over time. For this reason, we also examined data from consistent and national assessments such as the NAEP and ACT standardized college entrance exam. The second important point to reiterate is that the achievement gap data below are discussed in terms of percentage of students scoring profi cient or advanced. We recognize that using this measure comes with some limitations, and using scaled scores might be preferable; however, it is used in this paper for two key reasons. One, profi cient and advanced percentages are recognizable and discussed by school offi cials and policymakers. That is, these numbers are generally not confusing or created by complex statistical formulas that need expansive explanations. The purpose of this paper is to explore the trends in resources alongside trends in performance; therefore, we use the most straightforward data possible to encourage a straightforward discussion with school offi cials and policymakers. Two, both the NAEP and Arkansas Benchmark exams provide percentages of students scoring profi cient and advanced data, which means we can be consistent across tests. These two reasons led us to employ a comparative percentages analytic strategy. More sophisticated analysis could and have been employed for this work; however, this research responds directly and, we contend, straightforwardly to the research question regarding the infl uence of additional resources on achievement with easy to digest fi ndings. The remainder of this section explores the achievement gaps between Hispanic and white students and FRL and non-FRL students. White-Hispanic Achievement Gap Outlined in Table 1 are the percentages of Hispanic and white students that perform at the profi cient or advanced level on NAEP exams. Again, white students have shown consistent improvement in math since 2003 in both the 4th and 8th grade. However, Hispanic students demonstrated growth from 2003 to 2005, but then regressed in 2007. That trend is refl ected in the math performance gap, which narrowed in 2005 (16 percentage points for 4th grade, 13 percentage points for 8th grade), and then grew wider in 2007 (24 percentage points for 4th grade, 23 percentage points for 8th grade). Hispanic student performance in reading follows a pattern similar to math performance for 4th graders, where growth is observed between 2003 and 2005, with a decline in performance in 2007. However, from 2003 to 2005, 8th grade student performance showed a signifi cant decline (12 percentage points), with only slight improvement in 2007. Because white student performance in reading remained relatively stable, the reading performance gap varies from year to year, although the current performance gap is wider across both grades than it was in 2003. Compared to the national average, Arkansas’ white-Hispanic achievement gap is smaller in all four comparisons of data from the NAEP – 4th and 8th grade math and reading. 42 Dollars for Sense Association of Mexican American American Educators (AMAE) Journal © 2010 Table 1 Comparison of White & Hispanic Student NAEP Performance from 2003-2007 Math Reading 2003 2005 2007 2003 2005 2007 4th Grade Hispanic % Profi cient & Advanced 15% 25% 22% 18% 21% 16% White % Profi cient & Advanced 34% 41% 46% 35% 37% 36% Arkansas White-Hispanic Gap -19% -16% -24% -17% -16% -20% US Average Gap -27% -28% -29% -25% -24% -25% 8th Grade Hispanic % Profi cient & Advanced 7% 15% 8% 25% 13% 15% White % Profi cient & Advanced 24% 28% 31% 33% 32% 32% Arkansas White-Hispanic Gap -17% -13% -23% -8% -19% -17% US Average Gap -25% -24% -26% -25% -23% -24% The comparison of Hispanic and white students on the Benchmark exam, as shown in Table 2, reveals similar performance trends for students in 8th grade in both math and reading, as well as students in 4th grade reading. In the 8th grade, both Hispanic and white students have demonstrated consistent improvement for both math and reading. While there was a decline in performance from 2006 to 2007 in 8th grade reading, students in 8th grade math have made steady progress in each of the previous fi ve years. However, because the white students in 8th grade still outperform Hispanic students in both subjects, the achievement gap between the two groups has persisted (19 percentage points in math, 21 percentage points in reading). The performance trends for students in 4th grade have shown less stability from 2004 to 2008. In math and reading, both student groups declined after 2004, increased after 2005, and then decreased again in 2007 (with the lone exception of white students in math). However, since 2004, white students have shown an increase of 8 percentage points in the profi cient to advanced range, compared to an increase of only 3 percentage points for Hispanic students. As a result, this achievement gap has widened in the last fi ve years, with only a recent narrowing occurring between 2007 and 2008. A comparison of the average ACT performance for Hispanic and white students is highlighted in Table 3. Since 2003, the average score for Hispanic students has largely remained unchanged, decreasing from 18.9 in 2003 to 18.8 in 2007. As noted earlier, in the same time period, the average ACT score for white students has risen by 0.3 points. As a result, the achievement gap between the two student groups has widened from 2.2 points in 2003 to 2.6 in 2007. In summary, the three areas evaluated (the NAEP, Arkansas Benchmark, and ACT), the achievement gap has either remained stable or widened since 2003. While there are a number of instances of Hispanic students showing increases in performance levels, white students continue to demonstrate higher levels of achievement, which has led to the persistence of the achievement gap. These trends also challenge the effi cacy of the funding increases, and support the need for further action to be taken to ensure that the educational needs of Hispanic students in Arkansas are being met. 43 Dollars for Sense Association of Mexican American American Educators (AMAE) Journal © 2010 Table 2 Comparison of White & Hispanic Student Benchmark Performance from 2004-2007 Math Reading 2004 2005 2006 2007 2008 2004 2005 2006 2007 2008 4th Grade Hispanic % Profi cient & Advanced 64% 47% 57% 55% 67% 64% 44% 52% 44% 54% White % Profi cient & Advanced 74% 59% 68% 74% 82% 76% 60% 69% 67% 75% Arkansas White-Hispanic Gap -10% -12% -11% -19% -15% -12% -16% -17% -23% -21% 8th Grade Hispanic % Profi cient & Advanced 25% 25% 32% 39% 46% 40% 46% 57% 50% 54% White % Profi cient & Advanced 41% 43% 53% 57% 65% 62% 66% 73% 71% 75% Arkansas White-Hispanic Gap -16% -18% -21% -18% -19% -22% -20% -16% -21% -21% Table 3 Comparison of Hispanic & White ACT Performance 2003-2007 2003 2004 2005 2006 2007 Average Hispanic ACT Score 18.9 18.6 18.6 18.9 18.8 Average White ACT Score 21.1 21.2 21.2 21.5 21.4 Arkansas White-Hispanic Gap -2.2 -2.6 -2.6 -2.6 -2.6 Poverty Achievement Gap The fi nal analysis of performance trends compares students eligible for free and reduced lunch to students not eligible for the program, as outlined in Table 4. In math, eligible students in both 4th and 8th grade have shown consistent improvement in performance on NAEP examinations from 2003 to 2007. However, in reading, those same students have steadily declined in performance since 2003, with decreases of 3 percentage points in 4th grade and 4 percentage points in 8th grade. The performance trends for non-eligible students have shown consistent improvement since 2003 across both subjects and grade levels. As a result, the performance gap between FRL non-eligible and eligible students consistently widened each year, with the largest gap evident in 4th grade math performance. Compared to the national average, Arkansas’ poverty gap is smaller in all four comparisons of data from the NAEP – 4th and 8th grade math and reading. Table 4 Comparison of FRL Eligible & Non-Eligible Student NAEP Performance from 2003-2007 Math Reading 2003 2005 2007 2003 2005 2007 4th Grade FRL Eligible % Profi cient & Advanced 18% 22% 24% 20% 19% 17% Non-Eligible % Profi cient & Advanced 37% 48% 54% 39% 43% 44% Arkansas Poverty Gap -19% -26% -30% -19% -24% -27% US Average Gap -30% -31% -31% -26% -27% -27% 8th Grade FRL Eligible % Profi cient & Advanced 12% 13% 14% 19% 16% 15% Non-Eligible % Profi cient & Advanced 25% 30% 35% 34% 35% 36% Arkansas Poverty Gap -13% -17% -21% -15% -19% -21% US Average Gap -26% -26% -27% -24% -23% -24% 44 Dollars for Sense Association of Mexican American American Educators (AMAE) Journal © 2010 The performance trends on the NAEP exam highlight the discrepancy between FRL eligible and non- eligible students. In the four NAEP exams, the achievement gap has widened by a sizable margin since 2003 as a result of diminishing achievement levels in reading for eligible students, or greater performance increases by those students not FRL eligible. When taken in context with the performance of Hispanic students, it appears that improvements for key sub-groups have not followed the targeted increase in fi nancial resources. Gap Summary Table 5 presents a summary of the achievement gaps for each of the previously analyzed student sub- groups. The fi gures refl ect the change in achievement levels from the fi rst available testing period prior to the increase in targeted fi nancial resources to levels of student achievement fi ve years after the infusion of resources. In this table, a negative fi gure denotes an achievement gap that is growing wider, whereas a positive fi gure refl ects that the gap has narrowed. In total, the achievement gap widened by more than 1 percentage point in 12 of the 13 student achievement comparisons. Table 5 Summary of Achievement Gaps among Arkansas Student Sub-Groups White-Hispanic Achievement Gap FRL/Non-FRL Achievement Gap NAEP 4th Grade Math -5 pts. -11 pts. 4th Grade Reading -3 pts. -8 pts. 8th Grade Math -6 pts. -8 pts. 8th Grade Reading -9 pts. -6 pts. Arkansas Benchmark 4th Grade Math -5 pts. * 4th Grade Reading -9 pts. * 8th Grade Math -3 pts. * 8th Grade Reading +1 pts. * ACT -0.4 pts. * Note. NAEP and Arkansas Benchmark achievement gaps represent percentage point differences of profi cient and advanced scores. The ACT achievement gap refl ects differences in composite scores. *Data not available As discussed previously, the motivation for using three different tests, including the NAEP and statewide exam, was to triangulate the story. We want to know if the achievement gaps are changing. The results of our analyses presented in Table 5 (above) show that the NAEP and Arkansas statewide exam are generally consistent – with only 8th grade reading white-Hispanic gap presenting positive results. Conclusions Arkansas policymakers have achieved a great deal since 2003, increasing overall funding substantially statewide, particularly in districts with high percentages of disadvantaged students. At the same time, Arkansas has made considerable strides in improving the educational opportunities for all students. For instance, the number of Advanced Placement courses offered to students have steadily risen across the state. Furthermore, the state recently implemented Smart Core, a rigorous secondary level curriculum with the ambitious goal of ensuring that all high school graduates are prepared for higher education. As if to reinforce such improvement, former U.S. Secretary of Education Margaret Spellings praised Arkansas and Massachusetts as the two states leading the way in setting new standards in their respective educational systems. 45 Dollars for Sense Association of Mexican American American Educators (AMAE) Journal © 2010 Nevertheless, it is also apparent that disadvantaged students are still not receiving the academic support they need. The fact that performance increases have not boomeranged funding increases is disappointing, but not altogether surprising. While we realize that long term changes take time to take hold, we also fear that the targeted funding is not being employed effectively for the targeted students. Indeed, current data provided by the state do not allow us to examine whether targeted funds reached specifi c students in a district because public school funding data are only available at a district level. State policies also do not require districts to track how those resources are being used within the district or school, which hinders intelligent evaluation of such an investment. So, while we know that certain districts are receiving more resources, we cannot say that the schools – much less the students – are truly receiving these additional resources. We encourage state policymakers to reconsider the reporting of school expenditures at a school level, which would allow an exploration of where resources are being used effectively and ineffectively. Perhaps the fi rst step to ensuring that these targeted resources are being used to help the students most in need is to hold schools and districts more accountable for the ways in which this money is being spent. In this way, policymakers can ascertain whether or not these resources are actually reaching the target students, and determine if changes need to be made to how these resources are allocated. Further, and perhaps more importantly, by holding schools more accountable (and ensuring transparency in how these resources are used), policymakers can begin to identify the schools that are the most effective at educating these minority and low income students. Once these schools are identifi ed, policymakers may well encourage other schools to implement similar programs or strategies to help replicate the success demonstrated by these schools. By building on this success, these additional resources can be used to more effectively reach the targeted student populations. In the end, Arkansas policymakers should feel encouraged, yet unsatisfi ed, by their funding reform efforts. Indeed, Arkansas’ attainment of educational adequacy should be hailed as a long overdue achievement but should not be viewed as an ending point. Much work remains. Too many of our high school graduates require remediation when they reach college. Fewer than one in four 8th grade students scored at profi cient or above in the most recent administration of the NAEP. Most importantly, the analyses presented here emphasize that stubborn gaps in achievement persist between groups of students across the state, even with the additional resources in place. It is imperative that policymakers and educators fi nd effective ways to use the newly allocated resources to help all students, including Hispanic and FRL-eligible students, meet challenging standards, rather than be content simply to provide such resources. References American Federation of Teachers. Statewide average salary fi gures taken from Table I-7 State Rankings by 2001-02 Average Teacher Salary Adjusted by the 2001 AFT Interstate Cost of Living Index from Nelson and Drown, Survey and Analysis of Teacher Salary Trends 2002. www.aft.org/research Arkansas Department of Education. Retrieved July 27, 2008, from http://arkansased.org/ Dupree v. Alma School District No. 30, 651 S.W.2d 90 Hertert, L., Busch, C., & Odden, A. (1994). School fi nancing inequities among the states: The problem from a national perspective. Journal of Education Finance 19: 231-255. Education Trust. The funding gap 2005: Low income and minority students shortchanged by most states. http://www2.edtrust.org/NR/rdonlyres/31D276EF-72E1-458A-8C71-E3D262A4C91E/0/FundingGap2005.pdf Lake View School Dist. No. 25 v. Huckabee, 10 S.W.3d 892 (Ark. 2000). Lake View School Dist. No. 25 v. Huckabee, 355 Ark. 617, 142 S.W.3d 643. (Ark. 2004). Monk, D. H., Pijanowski, J. C., & Hussain, S. (1997). How and where the education dollar is spent. Financing Schools 7: 51-63. Monk, D. H., Roellke, C. F., & Brent, B. O. (1996). What education dollars buy: An examination of resource allocation patterns in New York state public school systems. Final report prepared for the Finance Center of the Consortium for Policy Research in Education, University of Wisconsin-Madison. National Center for Education Statistics. (2005). Revenues and expenditures by public school districts: School year 2002-03. U.S. Department of Education Institute of Sciences NCES 2006-312. 46 Dollars for Sense Association of Mexican American American Educators (AMAE) Journal © 2010 National Offi ce for Research on Measurement and Evaluation Systems. Retrieved July 25, 2008, from http://normes.uark.edu/ Nation’s Report Card. Retrieved August 1, 2008, from http://nationsreportcard.gov/ Odden, A. R., Monk, D. H., Nakib, Y., & Picus, L.O. (1995). The study of the education dollar: No Academy Awards and no fi scal smoking guns. Phi Delta Kappan 77:161–68. Odden, A. R., & Picus, L. O. (1992). School fi nance: A policy perspective. New York, NY: McGraw-Hill. Rebell, M. A. (2001). Education adequacy, democracy and the courts. In Achieving high educational standards for all: Conference summary, edited by T. Ready, C. Edley, Jr., & C. E. Snow. Washington, D.C.: National Research Council and National Academy Press. U.S. Department of Education, National Center for Education Statistics. (2003). Revenues and expenditures for public elementary and secondary education: School year 2000-01. 47 Dollars for Sense