TOWARDS AN OBJECTIVE EVALUATION OF TEACHER PERFORMANCE Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for non- commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-nd/2.5/. All other uses must be approved by the author(s) or EPAA. EPAA is published jointly by the Mary Lou Fulton College of Education at Arizona State University and the University of South Florida College of Education. Articles are indexed by H.W. Wilson & Co. Comment on this article at http://epaa.info/wordpress; please errata notes to Sherman Dorn (epaa-editor@shermandorn.com). EDUCATION POLICY ANALYSIS ARCHIVES A peer-reviewed scholarly journal Editor: Sherman Dorn College of Education University of South Florida Volume 14 Number 16 June 16, 2006 ISSN 1068–2341 Perceptions of the Impact of Accountability on the Role of Principals James E. Lyons, PhD Bob Algozzine, PhD University of North Carolina at Charlotte Citation: Lyons, J. E., & Algozzine, B. (2006). Perceptions of the impact of accountability on the role of principals. Education Policy Analysis Archives, 14(16). Retrieved [date] from http://epaa.asu.edu/epaa/v14n16/. Abstract Calls for accountability in America’s schools have created increased responsibilities for educational leaders. In this article, we describe and discuss a study of elementary, middle, and high school principals’ perceptions of the state-wide educational accountability program in North Carolina. The respondents indicated that the state’s accountability program has had its greatest impact on how they monitored student achievement, aligned the curriculum to the testing program, provided student remedial or tutorial opportunities, assigned teachers to grades levels or subjects, and protected instructional time. Views of some components, such as measures of school effectiveness, school safety standards, expectations and promotion standards for students, and financial bonuses received by staff members in schools that meet expected achievement standards, were viewed favorably. In contrast, the No Child Left Behind Adequate Yearly Progress (AYP) requirement (incorporated into the state’s accountability program), testing requirements for Limited English Proficiency students and special education students, the sanctions applied to schools that do not meet expected growth, and the school status designation labels that are applied to schools based upon student achievement were perceived more negatively. The predictable and unpredictable outcomes of a mandated accountability program on the perceptions (and behavior) of school principals create important considerations which are discussed for policy-makers and other professionals dealing with standards-based reform. Education Policy Analysis Archives Vol. 14 No. 16 2 Opiniones del impacto de la rendición de cuentas en los roles de directores/as de escuelas Resumen Las propuestas para aumentar la de rendición de cuentas (accountability) en las escuelas de América han incrementado las responsabilidades de los directores/as de escuelas. En este artículo, describimos y discutimos un estudio de las opiniones de los/as directores/as de escuelas secundarias del estado de Carolina del Norte. Los respuestas indican que el programa de rendición de cuentas del estado ha tenido su impacto más grande en: cómo supervisar los logros de los estudiantes, alinear los planes de estudios con las evaluaciones, implementar tutorías y oportunidades para estudiantes con dificultades de aprendizaje, asignar profesores de acuerdo a temáticas y/o grados-niveles, y proteger los tiempos dedicados a la instrucción. Acciones como mediciones de la eficacia de las escuelas, establecimiento de estándares de seguridad, estándares y expectativas sobre el aprendizaje para los estudiantes, y estímulos financieros para los miembros del equipo de las escuelas que alcanzaron los estándares apropiados, recibieron evaluaciones positivas. En contraste, el nivel anual de progreso adecuado (Adequate Yearly Progress AYP) de ley federal Sin abandonar ningún niño (The No Child Left Behind) incorporado en el programa de la responsabilidad del estado), los requisitos de evaluación de los estudiantes que están aprendiendo inglés como segundo idioma y de educación especial, las sanciones que se aplicaron a las escuelas que no obtuvieron los logros previstos, y la designación de estatus a las escuelas basadas en los logros académicos de los estudiantes fueron percibidos negativamente. Finalmente pensando en aquellas personas que participan en la toma de decisiones y otros profesionales que se ocupan de reformas basadas en estándares, se discuten los resultados previsibles e imprevisibles de un programa de rendición de cuentas (accountability) implementado mandatoriamente, sobre las opiniones (y comportamientos) de los/las directores/as de escuelas. Keywords: Accountability; principals; leadership roles and responsibilities; reform. Introduction Educational reforms and concerns about outcomes and accountability have been changing what goes on in schools for the past twenty years. Goals, objectives, directives, roles, responsibilities, programs, and activities have been targeted for sweeping changes; educational leaders have driven and been driven by all the “movements.” The insistence by policymakers and politicians that educators be held accountable for student outcomes has resulted in dramatic shifts in the responsibilities of department chairs, principals, and superintendents (Duke, Grogan, Tucker, & Heinecke, 2003; Goldberg, 2004, 2005; Ladd & Zelli, 2002; McGhee & Nelson, 2005; Tucker & Codding, 2002b). For example, principals today, particularly those in public schools, have myriad and varied leadership and managerial responsibilities as they carry out their roles. None of the many expectations are more important than that of chief educational accountability officer. Perhaps more than any time in history, student academic achievement is now the raison d’être of schools. As noted by Elmore, Abelmann et al. (cited in Ladd, 1996), the emphasis in terms of the principal’s role has now shifted from being accountable for money and other resources to being accountable for student outcomes and achievement. Perceptions of the Impact of Accountability on the Role of Principals 3 Hanson (2003) argues that instructional accountability has now been placed at the institutional (i.e., the individual school) level and the process of grading schools and rewarding and punishing them for performance has become commonplace. He further notes that “low performing schools that cannot find a way to pump up their tests scores might be ‘reconstituted’ by being taken over by the state or having administrators and/or teachers replaced—essentially sending the whole lot to the institutional guillotine” (p. 36). Yet, Hanson argues, schools are a reflection of their communities and that a failing school is often a symptom of a failing community (e.g., lack of proper health care, unemployment, street violence, or inadequate parent education). This presents a major dilemma, as some researchers such as Cuban (1988) and Hanson have observed that the vast majority of principals devote most of their attention to managing the school and pupil control, particularly in view of the post-Columbine heightened concerns about school security and safety. Given the many and often conflicting role demands under which principals work, it is little wonder that fewer candidates are applying for these positions and some writers are questioning why anyone would want to be a principal (Tucker & Codding, 2002b). Further, difficulties in finding educational leaders could not come at a worst time based on the importance of principals in creating effective schools (Cusick, 2003; Ladd & Zelli, 2002; Olsen, 1999). The emphasis on new roles and their relationship to the heightened focus on accountability were the target for this study. North Carolina’s Accountability Program North Carolina has received considerable national attention for its state-wide public education accountability program (Ladd & Zelli, 2002). Established in 1995 by the General Assembly and initiated in the 1996–1997 school year, this program commonly known as the ABCs of Public Education was established to foster, support and monitor student academic achievement in all local school districts. The program was initially implemented in the elementary grades, and the high school component was added during the 1997–1998 school year. This initiative which moved accountability from the district to the school level was designed to more quickly identify students performing below grade level so that intervention strategies might be timely employed. The program includes Student Accountability Standards called “gateways” for promotion at grades 3, 5, and 8, which require that students demonstrate grade level performance in reading, mathematics, and writing. Each year, elementary students in grades 3 through 8 take multiple-choice, end of grade tests that are aligned with the state’s standard course of study in reading and mathematics. Writing tests are also taken by students in grades 4 and 7. End-of-course tests are taken by high school students in 11 subject areas, including Algebra I, Algebra II, Geometry, English I, English II, U. S. History, Physical Science, Biology, Chemistry, Physics, and Economic, Legal and Political Systems. High school students are expected to meet local district and state graduation requirements and, effective 2005, to successfully pass an exit examination. A comprehensive overview of the general accountability standards is available online from the North Carolina Department of Public Instruction (http://www.ncpublicschools.org). The program is grounded in six components: a clearly articulated curriculum with content standards, agreement on the subject-matter content to be included, a set of valid and reliable indicators of progress and mastery, indicators reflective of how successful schools are at increasing student learning, a system of incentives and rewards to encourage improvement, and, a system of sanctions or directions for low-performing schools (Ladd & Zelli, 2002). The three basic purposes of the program were to provide local school accountability, emphasize mastery of basic skills, and promote as much local school decision making as possible. Since its inception, the program has been modified and improved to better portray school performance and to ensure that its measures are as http://www.ncpublicschools.org/ Education Policy Analysis Archives Vol. 14 No. 16 4 fair and accurate as possible. The 2002–2003 school year marked the seventh year of the ABCs program for K–8 grades and the sixth year for high schools. The effort was specifically designed to essentially end social promotion of students who do not perform at grade level on the tests. When students do not initially pass the exams, they are provided two rounds of re-testing and a formal review process; however, final decisions on promotions are made by building principals. Students who are not promoted to the next grade must receive extra help in smaller classes or additional instructional opportunities. They also may be provided a personalized education plan that outlines the intervention strategies to be undertaken. A description of this aspect of the ABCs is available from the North Carolina Department of Public Instruction’s Student Accountability Standards Reports website (http://www.ncpublicschools.org/accountability/reporting/sasr/2005). Ladd and Zelli (2002) summarized responses of a sample of North Carolina principals to key features of the state’s accountability framework, including “the state’s goals, the nature of incentives, the impact on the working environment of principals, and the actions they took in response to the accountability system” (p. 502). They found that the program was perceived by elementary school leaders to be a powerful tool for changing behavior in intended (i.e., increased emphasis on reading and math) and unintended ways (i.e., decreased emphasis on other content areas of instruction). Little other research has been completed regarding principals’ perceptions of their changing roles related to the impact of accountability. In this regard, we surveyed North Carolina principals to determine their perceptions of the state-wide accountability system and how it has affected their role as principal. Since the program has been operational sufficiently long for principals to have had substantial experience with it, we believed that valuable knowledge could be gained about the program by comparing information from a sample of principals representing elementary, middle and high schools. We also were interested in identifying instructional leadership responsibilities that the principals deemed most important in terms of their impact on student achievement. Method A survey was used to solicit principals’ perceptions on the key components of the state’s accountability program. To generate comprehensive, in-depth, and useful data, it was necessary to obtain responses from the subjects on an individual basis through a questionnaire. To ensure that participants would be representative of leadership within the state, principals were chosen from large urban, medium size, and small school districts. To the extent possible, districts from which the respondents were selected represented each of the three regions of the state, which includes the eastern region, central/piedmont region, and western region. Since large urban districts are only located in two of the three regions of the state, respondents from these districts necessarily came from only two regions. Participants Responses were received from 45 of 90 principals within two weeks of the first mailing (a return rate of 50%). To assure the respondents’ anonymity and confidentiality, no system was developed to follow-up with non-respondents. Therefore, no follow-up mailing or contact was possible with those who did not respond to the initial mailing. Should a second mailing have been made to non-respondents, a greater response rate would likely have been achieved; however, representativeness relative to the target population within the state was judged to be acceptable when compared to state figures (North Carolina Public Schools, 2003). http://www.ncpublicschools.org/accountability/reporting/sasr/2005 Perceptions of the Impact of Accountability on the Role of Principals 5 Of those principals who responded, 10 (22%) were elementary principals, 17 (38%) worked in middle schools, and 18 (40%) were high school principals. The gender distribution of respondents was similar to that of principals in the state: 23 (51%) of the respondents were male and 22 (49%) were female. The majority of elementary principals were female, the majority of the high school principals were male, and the numbers of male (9/20%) and female (8/18%) middle school principals responding were essentially the same. The masters’ degree is the highest degree held by the vast majority (56%) of the respondents. Twenty-six (58%) of the principals had less than ten years in the position, 14 (31%) had between 10 and 20 years of experience, and 5 (11%) were in the position for more than 20 years. The elementary principals’ schools averaged between 300 and 899 students, the middle schools averaged between 500 and 1,099, and the high school student enrollments ranged between 1,100 and over 1,500. Only one elementary principal responded who was in a school with less than 300 students. Thirty-two (71%) of the schools had between 21% and 59% of their students on free or reduced lunch (a proxy for the general social-economic status of the school). Only eight schools had less than 20% (4) or more than 80% (4) of the students on special lunch status. The respondents’ schools were selected to be representative of the varied school systems within the state. Procedures A questionnaire was mailed to a stratified random sample consisting of 10–12% of the elementary, middle, and high school principals in each of the selected districts. The population of principals in the state was divided into strata (groups) to ensure that principals from all across the state would be included. Through a random procedure, 25 school districts were chosen from which the respondents were selected. A letter was sent to each superintendent in these districts explaining the purpose of the study and requesting his/her written permission to sample 10–12% of the principals in the district. Of the superintendents contacted, 22 (88%) responded and provided written permission. Two superintendents did not respond and one superintendent elected not to provide permission. To solicit data from the principals, a three-part questionnaire (available from senior author) was developed. In the first section, 13 items (see Table 1) were used to assess principals’ levels of support (1=Strongly Oppose to 5=Strongly Support) for the key components of the state’s accountability program (e.g., emphasis on student testing and performance, school report cards, special testing requirements). The items were compiled from Department of Public Instruction documents defining North Carolina’s ABC’s program. In the second section, principals were asked to provide their perceptions of the level of influence (1=Little to 5=Substantial) of the state’s accountability program on 17 specific instruction leadership responsibilities (e.g., selecting teachers, monitoring instruction, obtaining needed resources). These items (see Table 2) were compiled from lists of professional standards and expectations represented in recent literature (Interstate School Leadership Licensure Consortium, School Leadership for the 21st Century Initiative). In the third section, principals provided demographic information to support description and generalization of the findings. A letter explaining the study and asking subjects to participate along with the questionnaire and a stamped return envelope was mailed to the 90 principals in the fall of the school year. They were asked to complete the questionnaires and return them within ten days. Education Policy Analysis Archives Vol. 14 No. 16 6 Design and Data Analysis The study was designed to illustrate perceptions of principals regarding key aspects of accountability and its impact on their roles. Content and face validity of the questionnaire were grounded in conformance with underlying documents defining constructs being measured and Cronbach coefficient alpha internal consistency estimates for responses on both sections of the questionnaire were high (rx x =.85 and .92). Responses were available from principals with varied demographic characteristics (e.g., school levels, gender, degrees held, years of experience, size of schools, and percentage of students on free or reduced lunch). Summaries and comparisons of perceptions were completed for different groups of principals (e.g., elementary, middle, and high school, male and female) and for the combined sample of respondents. Table 1 Summary Statistics for Principals Perceptions of the Components ABC Accountability Program Strongly Oppose Moderately Support Strongly Support 1 5 School Level North Carolina’s ABCs Accountability Components Elementary Middle High All F- ratioa 1. The measures of school effectiveness: High Student Performance; Safe, Orderly and Caring Schools; and Quality Teachers 4.20 0.92 4.38 0.96 4.17 1.10 4.24 0.99 0.18 2. Student Achievement Standards: a. Standards for Promotion at Grades 3, 5, & 8 b. Performance on 10 High School Subjects c. Computer Skills and High School Exit Exam 4.20 0.92 4.25 0.89 4.00 0.76 4.00 0.55 3.75 0.45 3.64 0.84 4.17 0.86 3.72 1.23 3.17 1.58 4.12 0.77 3.84 0.99 3.50 1.24 0.25 0.89 1.42 3. Emphasis on Student Testing/Performance 3.50 1.35 3.71 0.83 3.33 1.14 3.50 1.09 0.47 4. School Status Designation Labels: (School of Excellence, School of Distinction, School of Progress, No Recognition) 3.60 1.35 3.43 0.83 2.82 1.25 3.21 1.18 1.77 5. % of Students Required for Testing: a. 98% of eligible students-Elementary b. 95% of eligible students-High Schools 3.50 1.27 3.79 0.97 3.33 1.41 3.52 1.23 0.52 6. Sanctions for Schools That Do Not Meet Expected Growth (e.g., negative publicity, threat or assignment of assistance team) 2.30 0.95 2.86 1.03 2.44 1.25 2.55 1.11 0.87 7. Financial Bonus for Staff Members in Schools That Meet Student Achievement 4.50 0.53 4.14 0.95 4.06 1.14 4.20 0.95 0.69 Perceptions of the Impact of Accountability on the Role of Principals 7 Strongly Oppose Moderately Support Strongly Support 1 5 School Level North Carolina’s ABCs Accountability Components Elementary Middle High All F- ratioa Expectations 8. NC School Report Card (Web site Showing Individual School Performance on ABCs) 3.30 1.06 3.86 0.86 3.67 1.19 3.64 1.06 0.81 9. Testing Requirements for Exceptional Studentsb 2.10 1.20 2.00 1.30 2.33 1.03 2.17 1.15 0.34 Testing Requirements for Limited English Proficiency Studentsb 2.10 0.74 2.14 1.17 2.11 1.02 2.12 0.99 0.01 10. Safety Standards/Expectations for Schools 3.80 1.03 4.29 0.91 4.39 1.29 4.21 1.12 0.94 11. Intervention Expectations for Students Not Meeting Student Accountability Standards 4.00 0.67 4.07 0.73 3.94 1.16 4.00 0.91 0.70 12. Expectations for Schools to Meet Adequate Yearly Progress (All or Nothing) b 2.10 1.20 1.64 1.15 1.78 1.31 1.78 1.21 0.41 aReflects overall Analysis of Variance for differences in means across levels of school; p > .05; boverall rating for item was less favorable (i.e., below midpoint on scale). The second line in each cell is the standard deviation. Results No significant differences were found for any of the demographic variable comparisons. Responses were similar across school levels, gender, degrees held, years experience, size of schools, and percentage of students on free or reduced lunch. While these outcomes may be due to an absence of statistical power to detect differences, the sample was comparable to that represented in similar research (Ladd & Zelli, 2002, N=64) and more varied (i.e., elementary, middle, and high school vs. elementary school only).To illustrate the degree of similarity in responses evident across groups, perceptions are reported for elementary, middle, and high school principals as well as the combined group of respondents. Principals’ Perceptions of the ABCs Program Components Means and standard deviations for principals’ ratings reflective of the degree to which they support the primary components of the ABCs program are presented in Table 1 in the order they appeared on the survey. The top five program components supported by ratings across all of the respondents included the following: The Measures of School Effectiveness: High Student Education Policy Analysis Archives Vol. 14 No. 16 8 Performance, Safe Orderly and Caring Schools, and Quality Teachers (Item 1: M=4.24, SD=0.99); Safety Standards/Expectations for Students (Item 11: M=4.21, SD=1.12); Financial Bonus for Staff members in Schools that Meet Student Achievement Expectations (Item 7: M=4.20, SD=0.95); Student Achievement Standards: Promotion Standards at Grades 3, 5, & 8 (Item 2a: M=4.12, SD=0.77); and Intervention Expectations for Students Not Meeting Student Accountability Standards (Item 12: M=4.00, SD=0.91). Support for the ideals inherent in accountability appears to be similarly strong among principals. In contrast, practices associated with holding schools to high standards received less overall support: School Status Designation Labels (Item 4: M=3.21, SD=1.18); Sanctions for Schools That Do Not Meet Expected Growth (Item 6: M=2.55, SD=1.11; Testing Requirements for Exceptional Students (Item 9: M=2.17, SD=1.15); Testing Requirements for Limited English Proficiency Students (Item 10: M=2.12, SD=0.99); and, Expectations for Schools to Meet Adequate Yearly Progress (Item 13: M=1.78, SD=1.21). Influence of ABCs Program on Principals’ Instructional Leadership Responsibilities Perceptions of the degree to which the ABCs accountability program has affected how principals perform their instructional leadership responsibilities are presented in Table 2 in the order they appeared on the survey. The responsibilities that the respondents rated as most influenced by the program included the following: Monitoring Student Achievement (Item 7: M=4.69, SD=0.75); Aligning the Curriculum to the Testing Program (Item 9: M=4.62, SD=0.63); Providing Remedial and/or Tutorial Opportunities (Item 10: M=4.52, SD=0.63); Assigning Teachers to Grade Levels or Subjects (Item 2: M=4.45, SD=0.89); and Protecting Instructional Time (Item 5: M=4.43, SD=0.99). Responsibilities related to specific and frequent activities were reportedly more influenced that those with more general or global aspects of school leadership. For example, the instructional leadership responsibilities that the respondents rated as least influenced by the ABCs student accountability program included the following: Dealing with Parent Stress (Item 16: M=3.50, SD=1.15); Dealing with Student Stress (Item 15: M=3.50, SD=1.23); Evaluating Teachers (Item 3: M=3.93, SD=1.00); Obtaining Needed Resources (Item 11: M=4.19, SD=0.74); and, Dealing with Teacher Stress (Item 13: M=4.07, SD=1.02). More favored components of ABCs accountability and leadership responsibilities less likely to be affected by them were those with universal promise and high general degrees of acceptability (e.g., high achievement standards, supporting teachers and parents, obtaining resources). General expectations (e.g., numbers of students requiring testing, web site reporting) were moderately supported and general leadership responsibilities (e.g., assigning teachers, developing school schedule) were seen as moderately influenced by accountability demands. Less favored components of ABCs accountability and leadership responsibilities more likely to be affected by them were those requiring more direct action and typically low degrees of acceptability (e.g., supporting classroom instruction, meeting requirements, and dealing with sanctions). Perceptions of the Impact of Accountability on the Role of Principals 9 Table 2 Summary Statistics for Perceptions of the Influence of ABCs on Instructional Leadership Responsibilities Little Influence Moderate Influencea Substantial Influence 1 5 School Level Instructional Leadership Responsibilities Elementary Middle High All F-ratiob 1. Selecting Teachers 3.80 1.32 4.43 0.85 4.33 0.77 4.24 0.96 1.44 2. Assigning Teachers to Grade Levels or Subjects 4.20 1.40 4.64 0.50 4.44 0.78 4.45 0.89 0.72 3. Evaluating Teachers 3.60 1.17 4.21 0.70 3.89 1.08 3.93 1.00 1.14 4. Developing the School Schedule 4.00 1.33 4.21 0.80 4.17 1.15 4.14 1.17 0.12 5. Protecting Instructional Time 4.20 1.32 4.86 0.36 4.22 1.06 4.43 0.99 2.07 6. Monitoring Instruction 4.10 1.29 4.71 0.47 4.28 0.67 4.38 0.82 1.96 7. Monitoring Student Achievement 4.30 1.34 4.93 0.27 4.72 0.46 4.69 0.75 2.21 8. Encouraging/Promoting Staff Development 4.10 1.29 4.43 0.51 4.28 0.67 4.29 0.81 0.48 9. Aligning the Curriculum to Testing Program 4.60 0.70 4.43 0.65 4.78 0.62 4.62 0.63 1.26 10. Providing Remedial and/or Tutorial Opportunities 4.60 0.70 4.50 0.65 4.50 0.62 4.52 0.63 0.09 11. Obtaining New or Different Instructional Materials 4.40 0.84 4.14 0.77 4.11 0.68 4.19 0.74 0.52 12. Focusing More on Underachieving Students 4.50 0.71 4.43 0.65 4.28 0.83 4.38 0.73 0.33 13. Dealing with Teacher Stress 4.40 0.84 3.86 1.03 4.06 1.11 4.07 1.02 0.82 14. Feeling More Principal Stress 4.20 1.32 4.00 1.04 4.17 1.20 4.12 1.15 0.11 15. Dealing with Student Stress 3.70 1.34 3.57 0.85 3.33 1.24 3.50 1.23 0.37 16. Dealing with Parent/Guardian Stress 3.70 1.25 3.50 1.02 3.39 1.24 3.50 1.15 0.23 17. Obtaining Needed Resources 3.90 1.45 4.07 0.92 4.00 0.84 4.00 1.01 0.80 aOverall ratings for all items were above midpoint on the scale; breflects overall Analysis of Variance for differences in means across levels of school; p> .05. The second line in each cell is the standard deviation. Education Policy Analysis Archives Vol. 14 No. 16 10 Discussion Concerns about accountability have cast education into a damage control mode and educational leaders as well as those who prepare them bear considerable burden and blame as criticisms remain strong or escalate and efforts to salvage the system become ancient history (Berliner & Biddle, 1995; Bracey, 1997; Cotton, 2003; Goldberg, 2004, 2005; Hill, 2002; Kelley & Peterson, 2002; McGhee & Nelson, 2005; Stein, & Gewirtzman, 2003). In this regard, Tucker & Codding (2002b) paint a vivid picture: Why would anyone want the job of principal? Many school principals we know have the look these days of the proverbial deer caught in the headlights. Almost overnight, it seems, they have been caught in the high beams of the burgeoning accountability movement. (p. 1) As never before, the education enterprise is being challenged by the public and its federal, state, and local agents to improve student achievement. The federal government is putting pressure on representatives of the states who are pressing local governing bodies and superintendents to raise school performance. Of course, local leaders are tightening the vise by calling on principals to “make a difference” in their schools and the job becomes nearly undoable as the limits of power settle far above the individuals charged with change: Imagine that you are the principal, this person who is being asked to produce great improvements in student achievement. You cannot select your staff. You cannot fire anyone who is already on your staff. You cannot award or withhold a bonus from anyone. Seniority rights for teachers means that overnight, you can lose people you have made an enormous investment in and have them replaced by people who couldn’t care less about your agenda. You may have little control over the instructional materials that are used. Someone else controls the training agenda. Someone else controls how all but a small amount of your regular budget is spent. Someone else controls how the federal program money will be spent. Some people who work in your school report directly to people in the central office rather than to you. In some systems, you do not even have the right to assign teachers to classes because teachers’ seniority rights govern assignment. Yet despite all this, if your students do not make progress on the state accountability measures, your school is likely to be put on a public list of low performing schools. If performance does not improve, your school could be closed, the faculty disbanded, and you fired. You will be held responsible for the whole mess. (Tucker & Codding, 2002a, p. 6–7) While statements like these are not grounded in research or empirical evidence, they illustrate the context in which perceptions of educational leaders are often formed and provide a backdrop for considering the many factors that drive accountability or lack thereof in America’s educational system. In this regard, “surprisingly little is known about the impact of school-based accountability systems” (Ladd & Zelli, 2002, p. 494). Our findings were based on perceptions of elementary, middle, and high school principals. Professionals with less than ten years in the position responded to the survey in greater numbers. High school principals tended to have more years of experience than elementary and middle school principals. Over half of the high school principals had more than 10 years of experience in the position. Our outcomes suggest that there are differences in the acceptability and effects of our state’s accountability program, one of the oldest and most heralded in the country. Respondents reported five favorable components of the program: the measures of school effectiveness that form the basis for the program-high student performance; safe, orderly and caring schools; and quality teachers; safety standards/expectations for students; financial bonus for Perceptions of the Impact of Accountability on the Role of Principals 11 staff members in schools that meet student achievement expectations; student achievement standards: promotion standards at grades 3, 5, & 8; and intervention expectations for students not meeting student accountability standards. These areas of support were predictable and bear strong witness to the “nurturing, supportive, even ministerial” goals, ideals, and objectives that bring many successful teachers and professionals to the principalship (Graseck, 2005, p. 373). Unfavorable components were more representative of the dangers on the “high-stakes” side of program and their implications for school leaders: the expectation for schools to meet adequate yearly progress (required under NCLB), the testing requirements for limited English proficiency students, the testing requirements for exceptional students, the sanctions for schools that do not meet expected growth, and the school status designation labels assigned to schools based upon student academic achievement as measured by test scores. These findings were also predictable in light of continuing reactions from educators across the nation as NCLB is fully implemented and drawn from recent research in Texas (another state recognized for its accountability reputation): Regardless of prior success, principals may be removed from their positions solely as a result of accountability test scores; test scores trump all. Principals who serve as leaders of schools with diverse student populations may be especially vulnerable to removal; clearly, the risk of serving the at-risk is real. The educational tradition of defining people by test scores is particularly disturbing when consequences of failing to meet standards are viewed as “irreparable” by those who fall from grace. (McGhee & Nelson, 2005, pp. 370–372) Similarly, principals believe that their accountability program has had differential influence on important instructional leadership responsibilities. They indicated that the ABCs program had the most influence on monitoring student achievement, aligning the curriculum to the testing, providing remedial and/or tutorial opportunities, assigning teachers to grade levels or subjects, and protecting instructional time. In contrast, the instructional leadership practices that the principals believed were least influenced included dealing with student, teacher, and parent stress, evaluating teachers, and obtaining needed resources. Clearly, the focus of the latest grand drive for making schools better is correctly placed: teachers and teaching (and all its accoutrements) are at the core of improved student achievement. Of course, the difference between dreams (what and why) and reality (how) is the block. Taken as a whole, the most troublesome component of accountability as directed by NCLB requirements is measuring annual yearly progress (AYP) to identify schools in need of improvement, a practice with concerns for professional organizations, academics, and the general public as well (AYP status, 2004; National Education Association, 2004; Policy implications, 2004; Popham, 2004; Weaver, 2004). The arbitrary, unyielding nature of the index and its reliance on simplistic, single-measure notions of performance with varying technical adequacy create disillusionment, discomfort, and dilemmas that are difficult to overcome. When schools may fail to meet AYP by trivial degrees (e.g., one-tenth of a point), concerns are justified. When indicators are influenced by disproportionate numbers of students with special needs (e.g., at-risk, limited English proficiency, or disabilities) and special considerations or exemptions are needed but not forthcoming, concerns are justified. When consequences grounded in these concerns are punitive and powerful (e.g., negative media publicity, threat of outside assistance teams, administrative “reassignment), concerns are justified. Education Policy Analysis Archives Vol. 14 No. 16 12 Implications for Improving Research, Policy, and Practice Recently, Gibson, Caldeira, and Spence (2003) studied the legitimacy of the U.S. Supreme Court using a survey of confidence in the leaders as regarded by the American people. Grounding discussions about policy and improvement in practices in outcomes from surveys is common practice and polling public and professional attitudes has been a key component of efforts to improve public schools for some time (Elam & Brodinsky, 1989; Gallup, 1970; Langdon & Vesper, 2000; Rose & Gallup, 2002, 2003, 2004). Our findings reflect less than positive general attitudes of educational leaders toward the acceptability and effects of a well-known and long-standing accountability program; these outcomes parallel those reflected in other research on opinions about educational reforms (Ladd & Zelli, 2002; McGhee & Nelson, 2005; Tucker & Codding, 2002a, 2002b). So what can be done about all this shame, blame, and discontent? Clearly, developing curriculum content standards in critical areas of the curriculum, developing valid and reliable indicators of performance across the standards, and developing incentives and sanctions for favorable and unfavorable achievement are sound education fundamentals and educational leaders have a clear mandate to ensure that they are happening with fidelity. Additionally, Tucker and Codding (2002a, b) discuss broader assignations required for governments or agencies creating the work by setting the rules: Opportunities for funding research for effective leadership programs and methods should be available on a broader basis than has been the case in education. America’s finest companies plow considerable money into product development—most would not operate without it. They also have access to funds from many sources to help them do their work. This is not the case in education; but, “something has to give,” if the past is to serve as more than a promise of continued failure to progress. Efforts to prepare school leaders to lead and manage today’s schools with increasing pressure to produce steady gains in student performance requires attention from personnel preparation professionals. Opportunities for blending professional development provided in colleges and universities with the day-to-day “business” that goes on in local school districts have real merit here and deserve attention, acceptance, and fiscal support. In addition to providing “laboratories” for the development of effective practices in new and seasoned leaders, schools district leaders should also be asked to play a greater role in deciding who receives training, what the form and content of the training should be, and how it can be linked to work being done in their schools. The collaboration should also include mentoring and monitoring as programs strive to prepare more effective school leaders; and, adversarial relationships between colleges, universities, and school districts are clearly obstructions to be eliminated. Opportunities for funding higher education’s consumers need to be expanded. This means that money for training goes to those being trained, not those doing the training. Again, the giants of business and industry recognized the importance of “continuing education” (although seldom referring to it as such) and the availability of leadership and management training has produced important allegiances and powerful outcomes in the private sector. All things considered, accepting the sins of the past and not repeating them appears the best, but most formidable, course of action. Who Would Want This Job? Why would anyone want this job? As the mandate pile grows and the outcome package shrinks, it is easy to forget why smart people enter education and stay with it. Despite all the mess, Perceptions of the Impact of Accountability on the Role of Principals 13 the joys of listening, supporting, and inspiring teachers and others to improve the lives of children of all ages is one of the most rewarding jobs on the planet. The outcomes of our study suggest that it is a job being taken seriously by educational leaders charged with increasing responsibilities for higher and higher levels of performance and accountability. References AYP status. (2004, December 8). Education Week, 24, S6. Berliner, D. C. & Biddle, B. (1995). The manufactured crisis: myths, fraud, and the attack on America's public schools. New York, Addison-Wesley. Bracey, G. W. (1997). Setting the record straight : responses to misconceptions about public education in the United States. Alexandria, Va., Association for Supervision and Curriculum Development. Carnegie Forum on Education and the Economy. (1986). A nation prepared: Teachers for the twenty-first century. New York: Author Cooley, V. E. & Shen, J. (2003, March). School accountability and professional job responsibilities: A perspective from secondary principals. NASSP Bulletin, 87(634), 10– 25. Cotton, K. (2003). Principals and student achievement: What the research says. Alexandria, VA: Association for Supervision and Curriculum Development. Cuban, L. (1988). The managerial imperative and the practice of leadership in schools. Albany, State University of New York Press. Cusick, P. (2003, May 14). The principalship? No thanks. Education Week 22(36), 34, 44. DeMoss, K. (2002). Leadership styles and high-stakes testing: Principals make a difference. Education and Urban Society 35(1), 111–132. Doherty, K. M. & Skinner, R. A. (2003, January). Introduction: State of the states. Education Week 22(17), 75–76, 78. Duke, D. L., Grogan, M., Tucker, P. D., & Heinecke, W. F. (Eds.). (2003). Educational leadership in an age of accountability. Albany, NY: State University of New York Press. Elam, S., & Brodinsky, B. (Eds.). (1989). The Gallup/Phi Delta Kappa polls of attitudes toward the public schools, 1969–1988: A 20-year compilation and educational history. Bloomington, IN: Phi Delta Kappa. Gallup, G. H. (1970). The Public’s Attitudes toward the Public Schools: Second Annual Survey. Phi Delta Kappan, 52, 97–112. Education Policy Analysis Archives Vol. 14 No. 16 14 Gibson, J. L., Caldeira, G. A., & Spence, L. K. (2003). Measuring attitudes toward the United States Supreme Court. American Journal of Political Science, 47, 354–367. Goldberg, M. (2004). The test mess. Phi Delta Kappan, 85, 361–366. Goldberg, M. (2005). Test mess 2: Are we doing better a year later? Phi Delta Kappan, 86, 389– 395. Graseck, P. (2005). Where's the ministry in administration? Attending to the souls of our schools. Phi Delta Kappan, 86, 373–378. Hanson, E. M. (1991). Educational administration and organizational behavior (3rd ed.). Boston, Allyn and Bacon. Hanson, E. M. (2003). Educational administration and organizational behavior (5th ed.). Boston, Allyn and Bacon. Hill, P. W. (2002). What principals need to know about teaching and learning. In M. S. Tucker & J. B. Codding (Eds.). The principal challenge: Leading and managing schools in an era of accountability (pp. 43–76). Hoboken, NJ: Jossey-Bass/Wiley. Holmes Group (1986). Tomorrow's teachers: A report of the Holmes Group. East Lansing, MI: Author Hull, H. R. W. (2003, June). Will the NCLB help rural schools? State Government News, 15–17. Hurwitz, N. & Hurwitz, S. (2000, June). Tests that count. American School Board Journal, 187(1), 20–25. Kelley, C., & Peterson, K. D. (2002). The work of principals and their preparation: Addressing critical needs in the 21st century. In M. S. Tucker & J. B. Codding (Eds.). The principal challenge: Leading and managing schools in an era of accountability (pp. 247–312). Hoboken, NJ: Jossey-Bass/Wiley. Kowalski, T. J. (2003). Contemporary school administration : An introduction. Boston, Allyn and Bacon. Ladd, H. F. (Ed.). (1996) Holding schools accountable: Performance-based reform in education. Washington, DC: Brookings Institution. Ladd, H. F. & Zelli, A. (2002). School-based accountability in North Carolina: The responses of school principals. Educational Administration Quarterly, 38, 494–529. Langdon, C. A., & Vesper, N. (2000). The Sixth Phi Delta Kappa poll of teachers’ attitudes toward the public schools. Phi Delta Kappan, 81, 607–611. Lissitz, R. W. & Schaffer, W. D. (2002). Assessment in educational reform: Both means and ends. Boston: Allyn and Bacon. Perceptions of the Impact of Accountability on the Role of Principals 15 McGhee, M. W., & Nelson, S. W. (2005). Sacrificing leaders, villainizing leadership: How educational accountability policies impair school leadership. Phi Delta Kappan, 86, 367– 372. National Commission on Excellence in Education (1983). A nation at risk. Washington, DC, Government Printing Office. National Education Association. (2004, November). Test and punish. NEA Today, 23, 10–12, retrieved June 13, 2006, from http://www.nea.org/neatoday/0411/upfront.html. National Governors Association. (1991). Time for results: The governors' 1991 report on education. Washington, DC: Author. North Carolina Public Schools. (2003). North Carolina public schools statistical profile. Raleigh: Author. Retrieved June 13, 2006, from http://www.ncpublicschools.org/fbs/stats/statprofile03.pdf. Olsen, L. (2000, April 5). Worries of a standards "backlash" grow. Education Week, 19(30), 1,12–13. Olsen, L. (1999, March 3). Demand for principal growing, but candidates aren't applying. Education Week, 18(25), 1,20–22. Ornstein, A. C. (1991). Reforming American schools: The role of the States. NASSP Bulletin, 75(537). Policy implications of the 36th annual Phi Delta Kappa/Gallup poll. (2004). Phi Delta Kappan, 86, 53–56. Popham, W. J. (2004). Tawdry tests and AYP. Educational Leadership, 62, 85–86. Rose, L. C., & Gallup, A. M. (2002). The 34th annual Phi Delta Kappa/Gallup poll of the public’s attitudes toward the public schools. Phi Delta Kappan, 84, 41–46, 51–56. Rose, L. C., & Gallup, A. M. (2003). The 35th annual Phi Delta Kappa/Gallup poll of the public’s attitudes toward the public schools. Phi Delta Kappan, 85, 41–52. Rose, L. C., & Gallup, A. M. (2004). The 36th annual Phi Delta Kappa/Gallup poll of the public’s attitudes toward the public schools. Phi Delta Kappan, 86, 41–52. Spring, J. (2004). American education. New York, McGraw Hill. Stein, S. J., & Gewirtzman, L. (2003). Principal training on the ground: Ensuring highly qualified leadership. Portsmouth, NH: Heinemann. Tucker, M. S., & Codding, J. B. (2002a). Preparing principals in the age of accountability. In M. S. Tucker & J. B. Codding (Eds.). The principal challenge: Leading and managing schools in an era of accountability (pp. 1–40). Hoboken, NJ: Jossey-Bass/Wiley. http://www.nea.org/neatoday/0411/upfront.html http://www.ncpublicschools.org/fbs/stats/statprofile03.pdf Education Policy Analysis Archives Vol. 14 No. 16 16 Tucker, M. S., & Codding, J. B. (Eds.) (2002b). The principal challenge: Leading and managing schools in an era of accountability. Hoboken, NJ: Jossey-Bass/Wiley. Weaver, R. (2004, May 11–24). No: NCLB’s excessive reliance on testing is unrealistic, arbitrary, and frequently unfair. Insight on the News, 20, 47–48. About the authors James E. Lyons Bob Algozzine Email: jelyons@email.uncc.edu James E. Lyons (Ph. D., Ohio State University) is a Professor in the Department of Educational Leadership. Bob Algozzine (Ph. D., Penn State University) is a Professor in the Department of Educational Leadership and Co-Director of the Behavior and Reading Improvement Center at the University of North Carolina at Charlotte. Perceptions of the Impact of Accountability on the Role of Principals 17 EDUCATION POLICY ANALYSIS ARCHIVES http://epaa.asu.edu Editor: Sherman Dorn, University of South Florida Production Assistant: Chris Murrell, Arizona State University General questions about appropriateness of topics or particular articles may be addressed to the Editor, Sherman Dorn, epaa-editor@shermandorn.com. Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Robert Bickel Marshall University Gregory Camilli Rutgers University Casey Cobb University of Connecticut Linda Darling-Hammond Stanford University Gunapala Edirisooriya Youngstown State University Mark E. Fetler California Commission on Teacher Credentialing Gustavo E. Fischman Arizona State Univeristy Richard Garlikov Birmingham, Alabama Gene V Glass Arizona State University Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Ohio University William Hunter University of Ontario Institute of Technology Daniel Kallós Umeå University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Berkeley Michele Moses Arizona State University Anthony G. Rud Jr. Purdue University Michael Scriven Western Michigan University Terrence G. Wiley Arizona State University John Willinsky University of British Columbia http://epaa.asu.edu/epaa/board/apple.html http://epaa.asu.edu/epaa/board/berliner.html http://epaa.asu.edu/epaa/board/camilli.html http://epaa.asu.edu/epaa/board/darling-hammond.html http://epaa.asu.edu/epaa/board/fetler.html http://epaa.asu.edu/epaa/board/fischman.html http://epaa.asu.edu/epaa/board/garlikov.html http://epaa.asu.edu/epaa/board/green.html http://epaa.asu.edu/epaa/board/howleya.html http://epaa.asu.edu/epaa/board/howleyc.html http://epaa.asu.edu/epaa/board/hunter.html http://epaa.asu.edu/epaa/board/kallos.html http://epaa.asu.edu/epaa/board/levin.html http://epaa.asu.edu/epaa/board/mauhs.html http://epaa.asu.edu/epaa/board/mclean.html http://epaa.asu.edu/epaa/board/mintrop.html http://epaa.asu.edu/epaa/board/moses.html http://epaa.asu.edu/epaa/board/rud.html http://epaa.asu.edu/epaa/board/scriven.html http://epaa.asu.edu/epaa/board/wiley.html http://epaa.asu.edu/epaa/board/willinsky.html Education Policy Analysis Archives Vol. 14 No. 16 18 EDUCATION POLICY ANALYSIS ARCHIVES English-language Graduate-Student Editorial Board Noga Admon New York University Jessica Allen University of Colorado Cheryl Aman University of British Columbia Anne Black University of Connecticut Marisa Cannata Michigan State University Chad d'Entremont Teachers College Columbia University Carol Da Silva Harvard University Tara Donahue Michigan State University Camille Farrington University of Illinois Chicago Chris Frey Indiana University Amy Garrett Dikkers University of Minnesota Misty Ginicola Yale University Jake Gross Indiana University Hee Kyung Hong Loyola University Chicago Jennifer Lloyd University of British Columbia Heather Lord Yale University Shereeza Mohammed Florida Atlantic University Ben Superfine University of Michigan John Weathers University of Pennsylvania Kyo Yamashiro University of California Los Angeles Perceptions of the Impact of Accountability on the Role of Principals 19 Archivos Analíticos de Políticas Educativas Associate Editors Gustavo E. Fischman & Pablo Gentili Arizona State University & Universidade do Estado do Rio de Janeiro Founding Associate Editor for Spanish Language (1998—2003) Roberto Rodríguez Gómez Editorial Board Hugo Aboites Universidad Autónoma Metropolitana-Xochimilco Adrián Acosta Universidad de Guadalajara México Claudio Almonacid Avila Universidad Metropolitana de Ciencias de la Educación, Chile Dalila Andrade de Oliveira Universidade Federal de Minas Gerais, Belo Horizonte, Brasil Alejandra Birgin Ministerio de Educación, Argentina Teresa Bracho Centro de Investigación y Docencia Económica-CIDE Alejandro Canales Universidad Nacional Autónoma de México Ursula Casanova Arizona State University, Tempe, Arizona Sigfredo Chiroque Instituto de Pedagogía Popular, Perú Erwin Epstein Loyola University, Chicago, Illinois Mariano Fernández Enguita Universidad de Salamanca. España Gaudêncio Frigotto Universidade Estadual do Rio de Janeiro, Brasil Rollin Kent Universidad Autónoma de Puebla. Puebla, México Walter Kohan Universidade Estadual do Rio de Janeiro, Brasil Roberto Leher Universidade Estadual do Rio de Janeiro, Brasil Daniel C. Levy University at Albany, SUNY, Albany, New York Nilma Limo Gomes Universidade Federal de Minas Gerais, Belo Horizonte Pia Lindquist Wong California State University, Sacramento, California María Loreto Egaña Programa Interdisciplinario de Investigación en Educación Mariano Narodowski Universidad Torcuato Di Tella, Argentina Iolanda de Oliveira Universidade Federal Fluminense, Brasil Grover Pango Foro Latinoamericano de Políticas Educativas, Perú Vanilda Paiva Universidade Estadual Do Rio De Janeiro, Brasil Miguel Pereira Catedratico Universidad de Granada, España Angel Ignacio Pérez Gómez Universidad de Málaga Mónica Pini Universidad Nacional de San Martin, Argentina Romualdo Portella do Oliveira Universidade de São Paulo Diana Rhoten Social Science Research Council, New York, New York José Gimeno Sacristán Universidad de Valencia, España Daniel Schugurensky Ontario Institute for Studies in Education, Canada Susan Street Centro de Investigaciones y Estudios Superiores en Antropologia Social Occidente, Guadalajara, México Nelly P. Stromquist University of Southern California, Los Angeles, California Daniel Suarez Laboratorio de Politicas Publicas-Universidad de Buenos Aires, Argentina Antonio Teodoro Universidade Lusófona Lisboa, Carlos A. Torres UCLA Jurjo Torres Santomé Universidad de la Coruña, España http://epaa.asu.edu/epaa/board/acosta.html http://epaa.asu.edu/epaa/board/bracho.html http://epaa.asu.edu/epaa/board/canales.html http://epaa.asu.edu/epaa/board/casanova.html http://epaa.asu.edu/epaa/board/epstein.html http://epaa.asu.edu/epaa/board/kent.html http://epaa.asu.edu/epaa/board/schugur.html http://epaa.asu.edu/epaa/board/torres2.html http://epaa.asu.edu/epaa/board/torres.html Introduction North Carolina’s Accountability Program Method Participants Procedures Design and Data Analysis Table 1 Summary Statistics for Principals Perceptions of the Components ABC Accountability Program Results Principals’ Perceptions of the ABCs Program Components Influence of ABCs Program on Principals’ Instructional Leadership Responsibilities Summary Statistics for Perceptions of the Influence of ABCs on Instructional Leadership Responsibilities Discussion Implications for Improving Research, Policy, and Practice Who Would Want This Job? References Production Assistant: Chris Murrell, Arizona State University