EPAA Vol. 8 No. 42 Camilli: Texas Gains on NAEP: Points of Li... 1 of 5 Volume 8 Number 42 August 21, 2000 ISSN 1068-2341 A peer-reviewed scholarly electronic journal Editor: Gene V Glass, College of Education Arizona State University Copyright 2000, the EDUCATION POLICY ANALYSIS ARCHIVES. Permission is hereby granted to copy any article if EPAA is credited and copies are not sold. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education. Texas Gains on NAEP: Points of Light? Gregory Camilli Rutgers, The State University of New Jersey Related articles: Klein et al.: Vol. 8 No. 49 Haney: Vol. 8 No. 41 Abstract: The 1992-1996 gain in mathematics scores on NAEP from 4th to 8th grades in Texas is placed in perspective. The "miracle" in Texas looks much like the median elsewhere. Of 35 states and two districts (Guam and D.C.), the 52-point gain of Texas was good enough to earn Texas a rank of 17th or about the 46th percentile. Taking into consideration the wealth of states, Texas stands in the middle of the pack—no worse than most other states in delivering educational services to students. Haney (2000) examined a number of aspects of the Texas record of educational progress. This brief response concerns one particular indicator: the 1992-1996 gain in mathematics scores from 4th to 8th grades as measured by the National Assessment of Educational Progress (NAEP). In terms of the NAEP scales scores—not the achievement level percentages—the Texas gain from 1992-1996 was about 49 points. In any metric, this represents a sizable gain. In order to give some perspective to this accomplishment, it is customary to compare states. Implicitly, the rationale for doing so 2 of 5 is that some states do better than others, and through a process of competition and selection the level educational level of students can be bootstrapped. Since the Texas gain was the largest of any state, it could be argued that there is much merit in its methods and efficiencies. However, Haney raised a number of questions about whether this was a gain in achievement or whether it could be attributed to a large degree to changed in grade retention and dropout rates. There is a study on the 4th-8th grade mathematics gains that Haney did not consider which is relevant to this point. The Math cohort study by Barton et al (1998) estimated gains in math for a cohort of students in 4th grade who attended 8th grade four years later. To those who look to statistics to support the educational record of Texas (and to those who would take credit for the miracle), there is good news and bad news in this study. First, the good news. In the cohort study, Texas students gained about 52 points from 4th to 8th grade. Thus, unless students are retained in the 4th and 5th-8th grades disproportionately, there can be little question that the NAEP scores have gone up substantially. (Haney shows that for grades 2-8, the transition ratios are uniform. Questions arise in the 9th-10th grade transition.) But in regard to a comparison among states, the miracle in Texas looks much like the median elsewhere. Of 35 states and two districts (Guam and D.C.), the 52-point gain of Texas was good enough to earn Texas a rank of 17th or about the 46th percentile. Though Texas outranked four other states by less than one point, it should also be mentioned that six states outranked Texas by less than one point. This latter finding brings up a central point in the NAEP mathematics results for 1992 and 1996. In fact, the states are pretty well bunched up in the middle. In terms of statistical significance, Texas is different only from Guam (with a 40-point gain), and is not significant from Nebraska (ranked 1st with a 57-point gain). Was there a miracle in NAEP gains from 1992 to 1996 in Texas? The answer very clearly is no. Texas was average. One more simple representation helps to illustrate this latter point. In Figure 1, the state cohort gains are plotted against median state income (average across 1995-1997). Though a slight linear trend is evident (with Arizona and Hawaii being negative outliers), the story is relatively clear once more. With respect to wealth, which is one of the most reliable predictors of achievement, Texas stands in the middle of the pack—that is, no worse than most other states in delivering educational services to all students. Certainly, there is no criticism that can be leveled against Texas that cannot also be leveled against others states. However, within a paradigm that promotes healthy competition among states as a means of developing effective education policy, the points of light in Texas are not beacons. 3 of 5 Figure 1. 1992-1996 NAEP cohort gains in mathematics plotted against median family income. About the Author Gregory Camilli Email: camilli@rci.rutgers.edu Gregory Camilli is a professor in the Rutgers Graduate School of Education, and former Chair of the Department of Educational Psychology. His interests include measurement, program evaluation, and policy issues regarding student assessment. Dr. Camilli teaches courses in statistics and psychometrics, structural equation modeling, and meta-analysis. His research interets include efficacy studies of Head Start, implementation variability in cooperative learning and technology, and factors related to differential item functioning. References Haney, W. (2000). The Myth of the Texas Miracle in Education. Education Policy Analysis Archives, 8(41). Available online at http://epaa.asu.edu/epaa/v8n41/ (August 20, 2000) Barton, P. and Coley, R. (1998). Growth in School: Achievement Gains From the Fourth to the Eighth Grade. Princeton, NJ: Educational Testing Service. Copyright 2000 by the Education Policy Analysis Archives 4 of 5 The World Wide Web address for the Education Policy Analysis Archives is epaa.asu.edu General questions about appropriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 85287-0211. (602-965-9644). The Commentary Editor is Casey D. Cobb: casey.cobb@unh.edu . EPAA Editorial Board Michael W. Apple University of Wisconsin Greg Camilli Rutgers University John Covaleskie Northern Michigan University Alan Davis University of Colorado, Denver Sherman Dorn University of South Florida Mark E. Fetler California Commission on Teacher Credentialing Richard Garlikov hmwkhelp@scott.net Thomas F. Green Syracuse University Alison I. Griffith York University Arlen Gullickson Western Michigan University Ernest R. House University of Colorado Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Calgary Daniel Kallós Umeå University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Dewayne Matthews Western Interstate Commission for Higher Education William McInerney Purdue University Mary McKeown-Moak MGT of America (Austin, TX) Les McLean University of Toronto Susan Bobbitt Nolen University of Washington Anne L. Pemberton apembert@pen.k12.va.us Hugh G. Petrie SUNY Buffalo Richard C. Richardson New York University Anthony G. Rud Jr. Purdue University Dennis Sayers Ann Leavenworth Center for Accelerated Learning Jay D. Scribner University of Texas at Austin Michael Scriven scriven@aol.com Robert E. Stake University of Illinois—UC Robert Stonehill U.S. Department of Education David D. Williams Brigham Young University EPAA Spanish Language Editorial Board 5 of 5 Associate Editor for Spanish Language Roberto Rodríguez Gómez Universidad Nacional Autónoma de México roberto@servidor.unam.mx Adrián Acosta (México) Universidad de Guadalajara adrianacosta@compuserve.com J. Félix Angulo Rasco (Spain) Universidad de Cádiz felix.angulo@uca.es Teresa Bracho (México) Centro de Investigación y Docencia Económica-CIDE bracho dis1.cide.mx Alejandro Canales (México) Universidad Nacional Autónoma de México canalesa@servidor.unam.mx Ursula Casanova (U.S.A.) Arizona State University casanova@asu.edu José Contreras Domingo Universitat de Barcelona Jose.Contreras@doe.d5.ub.es Erwin Epstein (U.S.A.) Loyola University of Chicago Eepstein@luc.edu Josué González (U.S.A.) Arizona State University josue@asu.edu Rollin Kent (México) Departamento de Investigación Educativa-DIE/CINVESTAV rkent@gemtel.com.mx kentr@data.net.mx María Beatriz Luce (Brazil) Universidad Federal de Rio Grande do Sul-UFRGS lucemb@orion.ufrgs.br Javier Mendoza Rojas (México) Universidad Nacional Autónoma de México javiermr@servidor.unam.mx Marcela Mollis (Argentina) Universidad de Buenos Aires mmollis@filo.uba.ar Humberto Muñoz García (México) Universidad Nacional Autónoma de México humberto@servidor.unam.mx Angel Ignacio Pérez Gómez (Spain) Universidad de Málaga aiperez@uma.es Daniel Schugurensky (Argentina-Canadá) OISE/UT, Canada dschugurensky@oise.utoronto.ca Simon Schwartzman (Brazil) Fundação Instituto Brasileiro e Geografia e Estatística simon@openlink.com.br Jurjo Torres Santomé (Spain) Universidad de A Coruña jurjo@udc.es Carlos Alberto Torres (U.S.A.) University of California, Los Angeles torres@gseisucla.edu