TOWARDS AN OBJECTIVE EVALUATION OF TEACHER PERFORMANCE Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 22/7/2015 Facebook: /EPAAA Revisions received: 9/12/2016 Twitter: @epaa_aape Accepted: 16/12/2016 education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 24 Number 87 August 8, 2016 ISSN 1068-2341 The Invisible Schism: Teachers’ and Administrators’ Differing Perceptions of Education Reforms Sarah Melvoin Bridich University of Denver United States Citation: Bridich, S. (2015). The invisible schism: Teachers’ and administrators’ differing perceptions of education reforms. Education Policy Analysis Archives, 24(87). http://dx.doi.org/10.14507/epaa.24.2192 Abstract: This study examined teachers’ and administrators’ perceptions of education reforms, focusing on a state legislated education bill that altered teacher evaluations. A mixed-method design, including an electronic survey, was used to gather perceptions of Colorado Senate Bill 10-191: Great Teachers and Leaders Act from teachers and administrators in the Rockies School District (RSD), as well as these two groups’ general perceptions of teacher evaluations, education reforms, and change. Results revealed that teachers collectively hold similar views of education reforms, as do administrators; however, how each group perceives these elements of education policy and reform differs significantly. Both teachers and administrators believed that their groups see education reforms similarly, yet these groups had statistically significant differences on more than half of the survey questions. Qualitative data, in the form of open-ended responses to survey questions and semi-formal interviews, corroborated these findings. The two groups were unaware that their perceptions vary on critical issues related to the successful implementation of this education reform. This perception gap raises the questions of whether and how they can work together as reform implementation moves forward, and whether and how they can collectively support student learning as each group envisions, regardless of the policy itself. http://epaa.asu.edu/ojs/ http://dx.doi.org/10.14507/epaa.24.2192 Education Policy Analysis Archives Vol. 24 No. 87 2 Keywords: education reform; education policy; teacher perceptions; teacher evaluations; teacher effectiveness; teacher quality; administrator perceptions El cisma invisible: Las percepciones diferentes de la reforma educativa por parte de profesores y administradores Resumo: Este estudio examinó las percepciones de profesores y administradores sobre la reforma educativa, enfocado en una iniciativa legislada por el estado que cambió las evaluaciones de profesores. Se usó un disenso de métodos mixtos, incluyendo una encuesta electrónica, para obtener percepciones de Colorado Senate Bill 10-191: Great Teachers and Leaders Act de profesores y administradores en el Rockies School District (RSD), junto con las percepciones generales de estos dos grupos sobre la evaluación de profesores, reformas educativas y cambios. Los resultados mostraron que los profesores colectivamente tienen perspectivas de la reforma educativa similares a las de los administradores; sin embargo, la manera en que cada grupo percibe estos elementos de la política educativa y reforma varía significativamente. Los profesores al igual que los administradores creyeron que sus grupos tienen perspectivas similares de las reformas educativas; sin embargo, estos grupos tuvieron diferencias estadísticamente significativas en más de la mitad de las preguntas de la encuesta. Los datos cualitativos, en forma de respuestas abiertas a las preguntas de la encuesta y entrevistas semi-formales, corroboraron estos resultados. Los dos grupos no se dieron cuenta que sus percepciones varían en cuestiones criticas relacionadas a la implantación exitosa de esta reforma educativa. Esta brecha en percepción genera cuestiones de si y como se pueden cooperar mientras que procede la reforma, y también sí y como pueden apoyar el aprendizaje del estudiante colectivamente en la manera que cada grupo se imagine, a pesar de la política en sí. Palabras-claves: reforma educativa; política educativa; percepciones del profesor; evaluación del profesor; eficaz del profesor; calidad del profesor; percepciones del administrador O cisma invisível: Professores e administradores em diferentes percepções de reformas edu-cacionais Resumo: Este estudo examinou as percepções de professores e administradores em reformas de educação, focando em um estado de lei de legislação educacional que alterou avaliações de professores. Um design de método misto, inclu-indo uma pesquisa eletrônica, foi utilizada para reunir percepções do "Colorado Senate Bill” 10-191: Grandes Professores e Líderes agem de professores e administradores no Distrito Escolar Rockies (RSD), assim como essas percepções ge-rais em dois grupos de avaliações de professores, reformas na educação, e mudança. Os resultados revelaram que os professores têm, coletivamen -te, opiniões semelhantes de reformas da edu-cação, assim como administradores; No entanto, como cada grupo percebe estes elementos da política e reforma de educação diferem significa-tivamente. Ambos professores e administra-dores acreditavam que os seus grupos veem reformas de educação da mesma forma, ainda assim esses grupos tiveram diferenças estatica -mente significativas em mais da metade das perguntas da pesquisa. Os dados qualitativos, sob a forma de respostas abertas para questões de pesquisa e entrevistas semi-formais, colaboraram para estas conclusões. Os dois grupos não estavam cientes de que suas per-cepções variam em questões críticas relacionadas com a implementação bem sucedida desta reforma da educação. Essa diferença de per -cepção levanta as questões de saber se e como eles podem trabalhar juntos em como a imple -mentação da The invisible schism 3 reforma avança, e se e como eles podem coletivamente apoiar a aprendizagem do estudant e como cada grupo prevê, independentemente da própria política. Palavras-chave: reforma educacional, política de educação; percepções de professores; avaliações de professores; eficácia de professores; quali -dade de professores; percepções de administradores Introduction From state assessments to the achievement gap to teacher evaluations, belief that the next public education reform will finally usher in needed changes has dominated education discussions over the past five decades (Graham, 2005; Ravitch, 2010; Sarason, 1990; Sizer, 1992; Tyack & Cuban, 1995; Wagner, 2010). Yet the history of these modern education reforms waxes high on failed efforts (Cuban, 1998; Hargreaves & Shirley, 2008; Sarason, 1990; Tyack & Cuban, 1995; Tyack & Tobin, 1993; Wagner, 2010). The lack of teacher and administrator input in the design of reform efforts, at both the legislative and implementation levels, is viewed as one of the primary reasons various reforms have been unsuccessful (Berry, 2010; Hargreaves & Shirley, 2008; Odden, 1991; Sarason, 1990; Tyack & Cuban, 1995). As Hargreaves and Shirley note (2008), “[e]ducation leaders and teachers are the ultimate arbiters of change. The classroom door is the portal to reform or the raised drawbridge that holds it at bay” (pp. 59-60). Thus, regardless of the intentions behind a reform at the legislative level, reforms that fail to incorporate teachers’ and administrators’ beliefs and perspectives are often doomed at the implementation level. The lack of teacher and administrator input into state and national policies may play into the troubled and cyclical nature of reform efforts, but another implementation factor is also at work: the schism between teachers’ and administrators’ understandings of how education reforms are perceived within individual schools. This mixed-methods study explores how teachers and administrators1 within a single school district perceived education reforms generally, and the anticipated implementation of a new statewide reform of teacher evaluations specifically. While the two groups shared common understandings of some elements, they differed significantly in other areas. Yet both teachers and administrators believed their two groups were largely similar in collective perceptions. The reality that teachers and administrators believe their thinking is aligned, though it differs in actuality, indicates a meaningful perceptions gap between the groups that is not currently recognized. Moreover, if this gap in perceptions is not known, then it is also not addressed. The classroom door may be the portal to change, but what if the understanding of how change happens varies on either side of that door? In theory, teachers and administrators are working jointly to support students’ academic growth and overall well-being. Despite their good intentions, this study finds such collaboration may not be happening in practice. Moreover, the power structure within schools is such that administrators control what elements of the school will be discussed with the faculty and what will be implemented without discussion. This unequal power structure contributes to failed reform efforts because, as this study illustrates, teachers and administrators have divergent perceptions of education reforms and no clear means to confront these differences in order to move forward collectively to best support students. The purpose of this article is to explore the impact of teachers’ and administrators’ differing perceptions of education reforms within schools. The perception gaps found in this study carry implementation implications at both the school and district levels. If reforms are to be implemented 1 All “administrators” referred to in this paper are in-building school administrators, rather than district administrators, unless stated otherwise. Education Policy Analysis Archives Vol. 24 No. 87 4 with fidelity, administrators will need to create school cultures where teachers and administrators talk openly about their perceptions and work together to enact reforms within their schools. Teacher Evaluations in the Age of Accountability Students from the United States are consistently failing to compete internationally (Berliner, 2005; Dillon, 2010; Freidman, 2011). One organized effort to address this was The No Child Left Behind Act of 2001, which ushered in annual standardized testing as a means of increasing accountability for all students’ learning. This heightened attention on assessments was furthered by the United States Department of Education’s 2009 “Race to the Top” (RTTT) competition. In President Obama’s words, the guiding theory behind RTTT was that it was “time to stop talking about education reform and time to start doing it” (U.S. Department of Education, 2009, p. 1). One of the many elements addressed in RTTT was teacher evaluation. Although numerous out-of-school elements impact student learning and testing (Paul, 2012), it is widely understood that, within schools, teachers are the single most important element (Chetty, Friedman, & Rockoff, 2012; Johnson, 2012; Kristof, 2012; Weisberg, Sexton, & Sanders, 2009). Recent research supports a direct link between teacher quality and student learning (Johnson, 2009; Kane & Staiger, 2012; Rockoff, 2004; Sass, 2008; Weisberg et al., 2009). At the same time, the current teacher evaluation systems are not working (Kane & Staiger, 2012; Meyer, 2010a; Weisberg et al., 2009). Under current teacher evaluations, most teachers are rated as exceptional (Kane & Staiger, 2012; Weisberg, et al., 2009), yet students repeatedly perform far below the “exceptional” level on state, national and international assessments (Dillon, 2010; Friedman, 2011). This disparity between teacher ratings and student performance prompted many states to begin reexamining their teacher evaluation process. The RTTT application furthered this reexamination because Requirement D: Great Teachers and Leaders called for measuring student growth and connecting this student growth to teachers and principals through annual evaluations (U.S. Department of Education Executive Summary, 2009). Along with many other states, Colorado was inspired to revamp its teacher evaluation process because of the national competition to receive RTTT funds. Although Colorado did not receive RTTT money, Colorado did pass the highly controversial Senate Bill 10-191: Great Teachers and Leaders Act (SB 191) in dramatic fashion in May 2010 (Barnum, 2012; Haley, 2010; Meyer, 2010b; Meyer, Bartel, & Fender, 2010; Pena & Zeller, 2010). As a result, teacher evaluations are conducted annually for all teachers, 50% of a teacher’s evaluation is linked to student learning growth data as defined by the district,2 and teacher tenure is eradicated. As leading sponsor State Senator Mike Johnston explained it, the theory behind the law was that this more extensive teacher evaluation process would improve teaching practices and students’ learning simultaneously (Pena & Zeller, 2010). Power within Schools The way in which the new teacher evaluations were developed, and the intention behind them—to improve student learning through the revision of teacher evaluations and increased accountability for teachers—highlights issues of power and voice within the public education 2 When SB 191 became law, 50% of a teacher’s evaluation rating was linked to student assessment performance data, as measured by the Colorado Growth Model, but this subsequently was changed to student performance data as defined by the district and still, as of this article’s writing in 2015, had the “hold harmless” option phrase. The invisible schism 5 system. From a critical social theory perspective, elements of power and voice are interwoven into the fabric of SB 191 at multiple levels. First, politicians wield social capital and political power to create a new top-down plan to improve public education, such as a bill that determines what factors will conclusively measure teacher effectiveness. The decision to use formative assessments for 50% of the new teacher evaluations includes with it the implied value of standardized assessments in determining student learning. Indeed, under this new evaluation system, students’ performances on a test are deemed a greater measure of teacher effectiveness than whether a teacher inspires learning, is able to connect with students and keep them engaged, or is able to support a student’s overall development. Teachers and administrators were relatively voiceless in the legislative process of SB 191 (Meyer, 2010c), despite their critical and direct roles in the implementation of the new measures. A second issue of power and voice occurs in how SB 191 reinforces the status quo power structures within schools. Schools have delineated power structures, with administrators having power over teachers. SB 191’s elimination of teacher tenure further exacerbates the power differences by increasing the role of evaluations for a teacher’s job security and, therefore, increasing the imbalance of power between the two groups. While teachers and administrators are both integral parts of actualized reforms (Hall & Hord, 1984; McLaughlin, 1987; Sarason, 1990; Spillane, 1999; Tyack & Cuban, 1995), this increasingly stratified power structure has potential to impact negatively teachers’ inclination to use their voices within their school, for speaking one’s thoughts could carry higher consequences in the future. The ability of teachers and administrators to hold frank conversations on many things, including teacher evaluations and implementing new reforms, is potentially reduced through this bill. This imbalance in power between teachers and administrators may impact implementation efforts as well. Despite the high level of change implied by legislation, actual change is “ultimately a problem of the smallest unit…What is actually delivered or provided for under the aegis of a policy depends finally on the individual at the end of the line, or the ‘street level bureaucrat’” (McLaughlin, 1987, p. 174). In this case, the street level bureaucrat is the teacher, whose power within school is reduced as a result of SB 191. Moreover, if policy success is dependent upon “local capacity and will” (McLaughlin, 1987, p. 172), then successful implementation of education policies is dependent upon education’s local change agents, teachers and administrators, both having the capacity and will to implement stated policies with fidelity (Hall & Hord, 1984; Sarason, 1990; Tyack & Cuban, 1995). Factors that impact an educator’s will or motivation, such as the belief that mandated changes will— or will not—positively impact pedagogical practices and student learning, can have a large effect on the outcome of implemented legislation. Research Design This mixed-method study explored how teachers and administrators in one Colorado school district perceived a number of education reform elements, including the upcoming implementation of SB 191. The research questions for this study were: How do Colorado’s teachers and administrators perceive their respective roles in the implementation of SB 191’s new teacher evaluations? Do Colorado’s teachers’ and administrators’ responses vary by demographic variables? How do the responses of Colorado’s teachers and administrators compare? Based on these questions, the researcher designed a 40 question electronic questionnaire, with 38 questions using the Likert-type scale in six sections and with each section concluding with an open-response option (see Appendix A for the complete list of survey questions). The six categories, each with sub- questions, were: (1) current teacher evaluations; (2) ideal teacher evaluations; (3) education reform, in general; (4) SB 191, teacher evaluations; (5) SB 191, in general; and (6) change. The Likert-type scale Education Policy Analysis Archives Vol. 24 No. 87 6 was used to capture teachers’ and administrators’ levels of agreement with the questions, with values assigned as: 1 (Completely Disagree), 2 (Generally Disagree), 3 (Generally Agree), and 4 (Completely Agree), and questions directly related to SB 191 having the additional option of 5 (Not Applicable). Demographics questions came after these content questions. Following the quantitative collection and analysis, there were six semi-structured interviews conducted using the purposive sampling method. The researcher worked with one of the district’s assistant superintendents to select teacher and administrator interviewees who would be best able to help interpret the initial quantitative data results. The assistant superintendent recommended 11 teachers and administrators who he deemed were exemplary in their fields and could add significant insights to the findings; ultimately, six individuals participated in the interview process (see Appendix B for interview questions). The six individuals came from six different schools. Four were teachers and two were administrators. There was one male teacher and one male administrator, and the rest were female. The triangulation of the survey and interview data provided more holistic insights into the research questions than either approach could have done individually. For the purposes of this article, the district will be referred to as the Rockies School District (RSD). RSD is one of the larger and more diverse school districts in Colorado with over 25,000 students and more than 3,000 teachers and administrators (see Table 1). RSD was not one of the 27 school districts in Colorado piloting the new teacher evaluation system; consequently, SB 191 had not yet been implemented in RSD during the time of the study. Table 1 Ethnicity by survey groups and within RSD for 2012-13 school year (percentages) Asian African American Hispanic/ Latino American Indian or Pacific Islander Unknown/ Missing White Surveyed Teachers3 1% 2% 4% 2% 10% 81% Surveyed Administrators 0% 11% 13% 0% 15% 61% Licensed Personnel in District 2% 3% 5% 0% 3% 87% All teachers and administrators received the electronic survey (n = 3699) through their work email accounts. The result was a collective response rate of 18% (N = 653), though it should be noted that the teacher response rate was 16% (n = 589) and the administrator response rate was 63% (n = 64). The gender and school level demographics of responders closely matched the demographics of RSD’s entire teaching and administrative population (see Table 2). In the original study (Bridich, 2013), Colorado policymakers were also surveyed; however, due to the small number of state legislators who completed the survey on behalf of policymakers, their findings are excluded from this article. 3 The “Teacher” category includes classroom teachers plus everyone else in the district’s “Other Licensed Professional” category; however, 99% of non-administrator respondents identified as teachers and so the collective group is simply called “teachers.” The invisible schism 7 Table 2 Gender and School Level Demographics for RSD and Survey Participants RSD Demographics Survey Participant Demographics School level RSD Label Female Male Count Female Male Count Elementary 88% 12% 1673 88% 12% 257 ADMIN 61% 39% 44 71% 29% 21 LICENSED 88% 12% 1528 89% 11% 234 Mental HEALTH 89% 11% 57 100% 0% 1 NURSE 100% 0% 54 100% 0% 1 Middle 71% 29% 786 76% 24% 127 ADMIN 45% 55% 29 50% 50% 14 LICENSED 71% 29% 730 80% 20% 113 Mental HEALTH 76% 24% 17 0% 0% 0 NURSE 100% 0% 17 0% 0% 0 High 61% 39% 1065 60% 40% 156 ADMIN 48% 52% 27 57% 43% 14 LICENSED 61% 39% 1009 60% 40% 141 Mental HEALTH 83% 17% 18 0% 100% 1 NURSE 100% 0% 11 0% 0% 0 District Wide ADMIN 54% 46% 102 61% 39% 49 LICENSED 76% 24% 3412 78% 22% 488 Mental HEALTH 83% 17% 113 50% 50% 2 NURSE 100% 0% 72 100% 0% 1 TOTAL1 76% 24% 3699 Sub- total 77% 23% 540 Count 415 125 Missing 113 TOTAL2 653 1 Includes Entire Population in Study. 2 Includes every survey participant who marked "teacher" or "administrator" in the first question, even if participant did not complete demographic information at end of survey. Education Policy Analysis Archives Vol. 24 No. 87 8 Data Analyses Data analysis took place in two stages. For the survey instrument, the researcher analyzed all quantitative data using IBM SPSS to calculate descriptive statistics for each question, along with frequencies, Pearson chi-square, and one-way ANOVAs with the Scheffé and Tukey HSD post-hoc tests. For all tests, p = .05. Data from the 533 open-ended responses and the transcribed interviews were first reviewed with ATLAS.ti using the constant comparative method, and second were manually coded for themes. The survey data were run eight times through Pearson chi-square analysis tests. One analysis compared collective results from the teachers and administrators. Five explored sub-groups of teachers by the following elements: school level, gender, type of teacher (classroom teacher of a state-tested subject, classroom teacher of a non state-tested subject, Other Licensed Professional), teacher license type (Non-Professional, either Initial or Alternative, or Professional), and ethnicity (minority or white). Two analyses compared administrators, first by school level and second by gender. Survey results by teachers and administrators resulted in markedly different responses, with 60% of the questions resulting in statistically significant differences between the two groups. Occasionally the majority of both teachers and administrators indicated similar perceptions, but the disparity in percentages agreeing resulted in statistically different results. The survey data analysis by the five sub-groupings of teachers and the two sub-groupings of administrators illustrated that, overall, teachers collectively and administrators collectively had similar responses to the questions in all six categories. When statistically differences occurred within the teacher sub-groupings, the ANOVA results illustrated that high school teachers most frequently differed from their peers. Findings Triangulation of the survey and interview data illustrated that teachers and administrators had areas of common perceptions and of meaningful divergence. Teachers and administrators collectively desired to improve student learning, to use teacher evaluations as meaningful teaching tools for educators, and to be actively involved in the changes happening at their schools. Additionally, the majority of both groups believed that teachers and administrators perceived education reforms similarly. However, teachers’ and administrators’ perceptions differed significantly in how teacher evaluations are used in practice versus in theory, how education reforms are implemented at their schools, and in the anticipated value of SB 191’s newly legislated teacher evaluations. These findings are excerpted from a larger dissertation study (Bridich, 2013). Areas of Commonality Teachers and administrators shared common perceptions on broad questions related to education reforms, teacher evaluations, and changes within schools. To begin, teachers and administrators believed they understand education reforms similarly. A majority of teachers (n = 531, 68%) and administrators (n = 56, 66%) believed that teachers and administrators “have the same perceptions about what is needed to improve public education” (Q 22)(for additional data on The invisible schism 9 each question, see Appendix C). Many teachers (n = 531, 55%) and most administrators (n = 56, 75%) believe education reforms improve student learning (Q 17).4 Teachers (n = 542, 94%) and administrators (n = 58, 97%) believe teacher evaluations can help improve teaching (Q 12) and can positively impact student learning (teachers: n = 542, 93%; administrators: n = 58, 97%) (Q 13). Some participants shared that their evaluations were “meaningful” and “thought-provoking.” One teacher interviewee, Shelley (all names pseudonyms), believed that teacher evaluations existed to support teacher growth. She appreciated an evaluation system that used a coaching model, one that “makes you [teachers] better.” The administrators interviewed concurred that evaluations were a critical tool in helping teachers grow. Teachers (n = 538, 96%) and administrators (n = 58, 100%) also collectively “view change as an opportunity for growth” (Q 34). The majority of participants (teachers: n = 538, 98%; administrators, n = 58, 100%) expressed the desire to be involved directly with the changes at their school (Q36). One participant succinctly stated “[w]e’re humans… we like to be in the loop.” Another participant shared “[a]s a learner, and it should be the expectation that all teachers are learners, I navigate change best when given the opportunity to take some ownership of change.” Teachers (n = 538, 94%) and administrators (n = 58, 93%) also agree that there is a “right and wrong way” to bring new policies into the school (Q 38). Interviewees articulated that the right way to conduct change was to include teachers in the process and the wrong way was for administrators to simply announce the changes. “I think the right way, and the way that we’re trying to do it, is to talk about it as a team approach. We’re going to figure it out together.” As one survey participant noted, “[e]ngaging teachers as partners in the change is critical.” Areas of Divergence Despite the areas of convergence, teachers and administrators strongly diverged over more specific elements of teacher evaluations, the potential for SB 191 to positively influence student learning, and the act of implementing change within schools. Teacher evaluations. Although teachers and administrators shared some similar perceptions on teacher evaluations, they demonstrated profound disagreements about particulars. Teachers (n = 547, 44%, p = .003) did not believe the outgoing evaluations provided thorough reviews of teachers’ abilities, yet many administrators (n = 59, 68%) felt that they did (Q 9). Survey participants criticized the evaluations for being superficial, just a “mere snapshot” or a “dog and pony show,” since they believed predetermined and infrequent classroom visits did not present a true picture of an individual teacher’s abilities. Teachers (n = 542, 59%, p = .013) were also uncertain about the idea of using teacher evaluations to separate strong teachers from weak teachers while administrators (n = 58, 76%) largely supported this (Q 11). Teachers articulated they were uncomfortable when evaluations have this divisive element because the evaluations themselves were too subjective. “Growth and feedback is dependent upon WHO is evaluating you” (capitals in original survey response). One interviewee shared that “[s]ome administrators definitely overestimate their ability” to evaluate. Another teacher interviewee noted that the eradication of tenure, as will happen when SB 191 is fully implemented, was perceived as “scary” because it will require ongoing job evaluation. She further articulated: “[t]eachers are feeling like every year we have to prove ourselves as talented at what we do. And there’s something scary about that.” One 4 A majority of teachers and administrators both believe that education reforms improve student learning; however, this question resulted in a statistically significant difference because of the difference in proportion of responses within the two groups. Education Policy Analysis Archives Vol. 24 No. 87 10 administrator interviewed was also concerned about the new teacher evaluations separating strong from weak teachers, but for reasons opposite to those stated by the teachers. Andrew shared his fear that the process for eliminating bad teachers under SB 191 would backfire and actually make it harder to fire them. Andrew believed that replacing tenure with the new system—two consecutive years of ineffective ratings lead to a year of probationary status before the potential for dismissal— would actually be “more restrictive” than the previous practice and turn it into a “game that teachers will understand how to play.” Role of state assessments in teacher evaluations. There was also a large gap in teachers’ and administrators’ perceptions of what assessment data should be included in teacher evaluations, with significant disagreement over the use of state assessments in teacher evaluations. Most teachers (n = 542, 31%, p < .001) did not agree with linking students’ performances on state assessments to teacher evaluations, yet administrators (n = 58, 72%) did (Q 15). One participant shared that “[t]eachers should only be evaluated on the procedures and practices within their control.” Many were concerned that Colorado’s statewide tests were being developed concurrently with the implementation phase of SB 191, resulting in teachers being evaluated on something that did not yet exist and for which they could not knowingly prepare students. As Kate stated, “[t]he problem with the statewide data, from a teacher’s point of view, is that it is essentially mystery data. We never really see the questions or how they were graded; all we see are very vague standards.” This concern was corroborated by another interviewee who stated, “we don’t trust the test, yet… they present to us that they [test questions] will be thinking questions, high-level thinking rather than content- knowledge based, yet they [the tests] will be graded by the computer. Right there is the disconnect in our minds.” A second problem teachers had with statewide assessments was students’ lack of accountability to the test. In Colorado, state assessment results have no bearing on a student’s academic record and, therefore, students have no incentive to perform well. Jake shared that “the students have a very limited buy-in to it, other than what their intrinsic motivation might be to do well in the first place.” Students’ lack of incentive could have large implications for teachers, though, as Kate revealed in a personal example: “I had a student last year who was an honors student—he was very, very bright. He was angry that his mom had visited school during TCAP and so he got an unsatisfactory.” She went on to note that this one student’s result negatively affected her overall rating. Although the administrators acknowledged the teachers’ responses, they asserted that using students’ results on statewide testing to inform evaluations was appropriate and valuable. Administrators, Andrew shared, were “used to having data be part of our evaluation process,” so this addition was fine with him. Moreover, Andrew noted that previously he had struggled with getting teachers to understand the importance of state testing, and so he actually welcomed this change: One of the challenges that I have, that any principal probably has, is making sure that there is ownership of those scores at the teacher level. I think that an administrator welcomes this idea [of using state assessment data] because it will help us with teachers having buy-in for students’ performance on standardized tests because it’s also going to affect them directly. Anna also expressed general support for the connection between statewide assessment data and teacher evaluations. She stated that the alignment of statewide assessments to both the common core and to teacher evaluations will be a “win-win for kids.” The invisible schism 11 Senate Bill 191 (SB 191). Few teachers (n =524, 14%, p < .001) supported SB 191 during its development (Q 28) or were pleased with its final version (n =524, 14%, p < .001) (Q 29), yet at least half of the administrators (n = 54, 50% and 56% respectively) felt positively about both. Many survey participants echoed this participant’s sentiments: “This bill has been completely rushed, mishandled, focuses an unfair and disturbing lens on teachers, and puts an unfair burden on administrators.” Yet another participant shared this concern: “the government, whether at the state or federal level, often [creates reforms that] lead to unintended consequences (like increases in paperwork, inefficiencies in getting resources to the right people, etc.), and have little result on changing education.” Many participants shared that they understood SB 191 to be, officially, about students, but unofficially to be a referendum on teachers. Jake stated “Senate Bill 191 is about punishing teachers who don’t get students to grow.” Only 36% of teachers (n = 524, p < .001) believed “the reforms coming out of SB 191 have a chance to improve Colorado’s students’ learning in a meaningful way,” yet 83% of administrators (n = 54) thought this was possible (Q 33). The administrators interviewed felt that SB 191 was a positive change for education. As administrator Anna noted, SB 191 “is changing the playing field completely in education.” She did recognize, though, that this change could make teachers temporarily uncomfortable. Despite most teachers’ (n = 538, 98%, p = .053) desire to be a part of the change efforts in their school (Q 36), few felt that this was happening with the introduction of the new evaluation system. The interviewees each noted that teachers in their schools were generally aware that there would be a new evaluation system next year, but few were familiar with the law or directly involved in its implementation. One participant stated that he/she “felt like a puppet” when it came to school change. Other participants wrote: “[e]ducators working at schools have received close to ZERO information about how SB 191 is to be implemented” (capitals in original); “[s]adly, I know nothing about this bill;” “I don’t really have any information other than the information in the newspaper;” and “I have learned absolutely nothing about SB 191 from my school district.” One participant wrote that “there was an increased feeling of fear” due to the “lack of information that we have” about the new teacher evaluations. Polly stated that “the doors have been open, but only informally” for all teachers to participate in the implementation of the new evaluations. Most teachers, Polly believed, were “just hearing about it [the new evaluations]. It is so passive for the teachers, most of the teachers are like, ‘oh, here it comes.’” Teachers felt excluded from the early implementation efforts, yet the administrators’ interviewed believed they were openly including teachers in the process. Anna stated that there was “no hidden agenda” and all the information she was presenting to her faculty members was available in handouts and online for additional reading later. Both Anna and Andrew talked about the benefits of the pilot program during the 2012-2013 school year, namely that the process required one teacher from each school in the district to be directly involved. Anna noted that these teachers have already been trained in the new evaluation, which “helps teachers feel like they are a part of the process.” Anna further explained that she was doing everything possible to “make the process transparent” and to ensure “teachers feel like they are a part of the process.” Andrew shared that he had devoted an entire professional development session to explaining the upcoming teacher evaluation changes and that, in regard to feedback, “100% have been given that opportunity.” However, Andrew later stated that the percentage of teachers who had been actively involved in the pilot process and might be familiar enough with the new evaluations to ask tough questions, versus those who had only cursory knowledge of the new evaluations, was less than five percent. For this reason, he said: “I would say that it probably is fair to say that the majority of the teachers feel like they’re not going to have a direct impact” on the implementation of the new teacher evaluations at his school. Education Policy Analysis Archives Vol. 24 No. 87 12 Implications RSD’s teachers and administrators collectively held a few general perceptions about education reforms, teacher evaluations, and change within schools. Yet their perceptions diverged significantly and meaningfully with respect to more specific elements of each. Despite these differences, though, both groups shared the belief that their two groups held similar perceptions of what is needed to improve public education. The major problem is that they do not. This invisible schism yields implications for policy implementation at the school and district levels. Given the inherent power structure within schools, administrators will need to own the task of addressing these different perceptions. Administrators set the agenda for faculty meetings and determine professional development programming, and the data from this study indicates that the varied perceptions on education reforms between teachers and administrators must be explored in a meaningful way. To do this, administrators will have to challenge the traditional hierarchical school structure to create an open school culture where candid conversations (Catmull, 2014) can occur regularly. It will not be enough merely to acknowledge divergent perceptions, though. The underlying ideas must be unpacked; the varied mental models, the “deeply ingrained assumptions, generalizations or even pictures and images that influence how we understand our world and take action” (Senge, 2000, p. 8), between teachers and administrators must be examined. Teachers have been criticized for being “resistant to change” (Spillane, 1999, p. 165) or a “drag on reform” (Tyack & Cuban, 1995, p. 132), yet this study shows they are willing and eager to be included in the change process (Q 36). Beyond creating an open school culture, administrators will need to empower teachers to become change agents themselves (Hall & Hord, 1984; McLaughlin, 1987). Administrators must invite teachers to join the front lines of change within schools and give them opportunities to act upon their ideas of what will enable mandated changes most effectively to support student learning. It could appear that these recommendations would go against administrators’ own interests by upending the power dynamics within schools, for such open conversations would likely invite criticism and challenges. But that perspective is short-sighted. If student learning is administrators’ driving focus, then it behooves them to make this invisible schism visible and a concrete focus for change within the school. District leadership will also be essential. District administrators will need to give schools space to ensure that they are true “learning organizations,” places where “people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured… and where people are continually learning how to learn together” (Senge, 2000, p. 3). They must encourage this disruption to the traditional power structures within schools (Sarason, 1990). They will need to provide their building administrators guidance on facilitating difficult conversations as well as support for the inclusion of teachers’ voices in implementation at a greater level than has existed thus far. They must ensure that the “local capacity and will” (McLaughlin, 1987, p. 172) of teachers and administrators is ready and available to do the tough work—and then be prepared to honor the changes recommended by the teachers and administrators as implementation begins. These recommendations are not for the faint of heart. They compel administrators to challenge the existing power structure within their schools and to embrace a new model of implementation. They call for administrators to increase the time and energy they spend working with teachers to enact new policies. They ask districts to be flexible. Yet they just might provide an increased opportunity for reforms to be implemented with fidelity and for students to thrive in schools. The invisible schism 13 Conclusion As school districts in Colorado and around the country continue to explore how best to improve student learning, it is imperative that the two groups most directly involved in implementing the changes—teachers and administrators—communicate consistently and openly about the various elements of the new policies. The results from this mixed-methods study expose a seemingly invisible but dangerous chasm between teachers and administrators that merits attention. The reality of teachers’ and administrators’ differing perceptions, combined with their shared faith that they hold similar perceptions, reveals a problematic gap in communication that could hurt the effectiveness and implementation efforts of many types of reforms. Without open dialogue, it is likely that any policy—however well-intended, well-written, or well-crafted—will be challenged to improve student learning. Education policies designed without teacher and administrator input have led far too often to failed reform efforts. By contrast, direct communication between teachers and administrators about a policy and how collectively to implement it could provide a critical and often missing piece of the puzzle for successful change. References Barnum, M. (2012, August 26). Blueprints for teacher evaluations. The Denver Post, 4D. Berliner, D. C. (2005). Our impoverished view of educational reform. Teachers College Record. http://www.tcrecord.org/content.asp?contentid=12106 Berry, B. (2010). Getting "real" about teaching effectiveness and teacher retention. Journal of Curriculum and Instruction, 4(1), 1. http://dx.doi.org/10.3776/joci.2010.v4n1p1-15 Bridich, S. M. (2013). Perceptions surrounding the implementation of Colorado Senate Bill 10-191's new teacher evaluations. (Doctoral Dissertation), University of Denver. Briggs, D., & Domingue, B. (2011). Due diligence and the evaluation of teachers: A review of the value-added analysis underlying the effectiveness rankings of Los Angeles unified school district teachers by the Los Angeles Times. Boulder, CO: National Education Policy Center. Brill, S. (2011). Class warfare: Inside the fight to fix America’s schools. New York: Simon and Schuster. Catmull, E. (2014). Creativity, Inc.: Overcoming the unseen forces that stand in the way of true inspiration. New York: Random House. Chetty, R., Friedman, J. N., & Rockoff, J. E. (2011). The long-term impacts of teachers: Teacher value-added and student outcomes in adulthood. (No. 17699). Cambridge, MA: National Bureau of Economic Research. http://dx.doi.org/10.3386/w17699 Colorado Department of Education, Educator Effectiveness. (2012). Rubric for evaluating Colorado's teachers. Retrieved April 19, 2013, from http://www.cde.state.co.us/EducatorEffectiveness/downloads/SMES%20- %20Teacher/RubricforEvaluatingCOTeacherFINAL.pdf Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five approaches. Thousand Oaks, CA: Sage. Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage. Cuban, L. (1998). How schools change reforms: Redefining reform success and failure. Teachers College Record, 99(3), 453-477. Dillon, S. (2010, December 7). Top test scores from Shanghai stun educators. New York Times. http://www.tcrecord.org/content.asp?contentid=12106 http://dx.doi.org/10.3776/joci.2010.v4n1p1-15 http://dx.doi.org/10.3386/w17699 http://www.cde.state.co.us/EducatorEffectiveness/downloads/SMES%20-%20Teacher/RubricforEvaluatingCOTeacherFINAL.pdf http://www.cde.state.co.us/EducatorEffectiveness/downloads/SMES%20-%20Teacher/RubricforEvaluatingCOTeacherFINAL.pdf Education Policy Analysis Archives Vol. 24 No. 87 14 Fowler, F. J. (1995). In Susan McElroy (Ed.), Improving survey questions: Design and evaluation. Thousand Oaks, CA: Sage. Fowler, F. J. (2009). Survey research methods (4th ed.). Thousand Oaks, CA: Sage. http://dx.doi.org/10.4135/9781452230184 Friedman, H. H., & Amoo, T. (1999). Rating the ratings Scale. Journal of Marketing Management, 9(3), 114-123. Friedman, T. L. (2011, November 19). How about better parents? New York Times. Graham, P. A. (2005). Schooling America: How the public schools meet the nation's changing needs. New York: Oxford University Press. Haley, D. (2010). Democrats gone wild. The Denver Post. http://www.denverpost.com/2010/05/13/haley-democrats-gone-wild/ Hall, G. E., & Hord, S., M. (1987). Change in schools: Facilitating the process. Albany, NY: State University of New York Press. Hargreaves, A., & Shirley, D. (2008). The fourth way. Educational Leadership, 66(2), 56-61. Johnson, S. M. (2009). How best to add value? strike a balance between the individual and the organization in school reform. (No. 249). Washington, D.C.: Economic Policy Institute. Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains. Seattle, WA: Bill and Melinda Gates Foundation. McLaughlin, M. W. (1987). Learning from experience: Lessons from policy implementation. Educational Evaluation and Policy Analysis, 9(2), 171-178. http://dx.doi.org/10.3102/01623737009002171 Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA: Jossey-Bass. Meyer, J. P. (2010a, February 19). DPS teacher evaluations don't mirror quality, group says. The Denver Post. http://www.denverpost.com/2010/02/18/dps-teacher-evaluations-dont-mirror- quality-group-says/ Meyer, J. P. (2010b, April 24). Colorado teacher bill ignites firestorm of support, opposition. The Denver Post. http://www.denverpost.com/2010/04/24/colorado-teacher-bill-ignites- firestorm-of-support-opposition/ Meyer, J. P. (2010c, May 13). “Positive model'' for states weighing teacher reforms. The Denver Post. http://w ww.denverpost.com/2010/05/12/positive-model-for-states-weighing-teacher- reforms/ Odden, A.R. (1991). The evolution of education policy implementation. In A. Odden (ed) Education policy implementation. Albany, NY: State University of New York Press. Ooms, A. (2011, July 25). A tipping point for democrats on education. The Denver Post. http://www.denverpost.com/2011/07/22/guest-commentary-a-tipping-point-for- democrats-on-education/ Paul, A. M. (2012). It’s not me, it’s you. New York Times. http://www.nytimes.com/2012/10/07/opinion/sunday/intelligence-and-the-stereotype- threat.html Pena, F., & Zeller, L. H. (2010, May 7). Pass the teacher effectiveness bill. The Denver Post. http://www.denverpost.com/2010/05/06/pass-the-teacher-effectiveness-bill/ Ravitch, D. (2010). Death and life of the great American school system: How testing and choice are undermining education. New York: Basic Books. Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. The American Economic Review, 94(2), 247-252. http://dx.doi.org/10.1257/0002828041302244 http://dx.doi.org/10.4135/9781452230184 http://www.denverpost.com/2010/05/13/haley-democrats-gone-wild/ http://dx.doi.org/10.3102/01623737009002171 http://www.denverpost.com/2010/02/18/dps-teacher-evaluations-dont-mirror-quality-group-says/ http://www.denverpost.com/2010/02/18/dps-teacher-evaluations-dont-mirror-quality-group-says/ http://www.denverpost.com/2010/04/24/colorado-teacher-bill-ignites-firestorm-of-support-opposition/ http://www.denverpost.com/2010/04/24/colorado-teacher-bill-ignites-firestorm-of-support-opposition/ http://w/ http://www.denverpost.com/2011/07/22/guest-commentary-a-tipping-point-for-democrats-on-education/ http://www.denverpost.com/2011/07/22/guest-commentary-a-tipping-point-for-democrats-on-education/ http://www.nytimes.com/2012/10/07/opinion/sunday/intelligence-and-the-stereotype-threat.html http://www.nytimes.com/2012/10/07/opinion/sunday/intelligence-and-the-stereotype-threat.html http://www.denverpost.com/2010/05/06/pass-the-teacher-effectiveness-bill/ http://dx.doi.org/10.1257/0002828041302244 The invisible schism 15 Sarason, S. (1990). The predictable failure of education reform: Can we change course before it’s too late?. San Francisco, CA: Jossey-Bass. Sass, T. R. (2008). The stability of value-added measures of teacher quality and implications for teacher compensation policy. National Center for Analysis of Longitudinal Data in Education Research, (4), 1-7. Senate Bill 10-191: Great teachers and leaders act, 191, 1: 22-9-102. (2010). http://www.cde.state.co.us/cdedepcom/download/pdf/SB10-191.pdf Senge, P. (2006). The fifth discipline: The art and practice of the learning organization. New York: Doubleday. Sizer, T. (1992). Horace's compromise: The dilemma of the American high school. Boston: Houghton Mifflin. Spillane, J. P. (1999). External reform initiatives and teachers' efforts to reconstruct their practice: The mediating role of teachers' zones of enactment. Journal of Curriculum Studies, 31(2), 143- 175. http://dx.doi.org/10.1080/002202799183205 Tyack, D., & Cuban, L. (1995). Tinkering toward utopia: Reflections on a century of public school reform. Cambridge, MA: Harvard University Press. Tyack, D., & Tobin, W. (1993). The “grammar” of schooling: Why has it been so hard to change? American Educational Research Journal, 31(3), 453-479. http://dx.doi.org/10.3102/00028312031003453 U. S. Department of Education. (2009). Race to the top program executive summary. https://www2.ed.gov/programs/racetothetop/executive-summary.pdf University of Wisconsin-Madison. Qualtrics survey hosting service, survey best practices. Retrieved January 5, 2013, from http://survey.wisc.edu/BestPractices.html Wagner, T. (2010). The global achievement gap: Why even our best schools don't teach the new survival skills our children need - and what we can do about it. New York: Basic Books. Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness. New York: The New Teacher Project. Wright, S. P., Horn, S. P., & Sanders, W. L. (1997). Teacher and classroom content effects on student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in Education, 11, 57-67. http://dx.doi.org/10.1023/A:1007999204543 http://www.cde.state.co.us/cdedepcom/download/pdf/SB10-191.pdf http://dx.doi.org/10.1080/002202799183205 http://dx.doi.org/10.3102/00028312031003453 https://www2.ed.gov/programs/racetothetop/executive-summary.pdf http://survey.wisc.edu/BestPractices.html http://dx.doi.org/10.1023/A:1007999204543 Education Policy Analysis Archives Vol. 24 No. 87 16 Appendix A: Teacher and Administrator Survey Questions In the survey, questions 1-38 were completed using a Likert-type scale to capture teachers’ and administrators’ levels of agreement with the questions, with values assigned as: 1 (Completely Disagree), 2 (Generally Disagree), 3 (Generally Agree) to 4 (Completely Agree). Questions directly related to SB 191 had the additional option of 5 (Not Applicable). Questions 39 and 40 had different answer choices. The answer choices for question 39 were: 1 (Teachers), 2 (Building Administrators, 3 (District Personnel), 4 (Policymakers (i.e., legislators, lobbyists, etc.), 5 (Other), and 6 (Unsure). The answer choices for question 40 were: 1 (Students), 2 (Teachers), 3 (Building Administrators), and 4 (Policymakers). Q # Question Current Teacher Evaluations 1 Current teacher evaluation system improved your teaching. 2 Current teacher evaluation system has provided me with meaningful feedback. 3 Current teacher evaluation system has made me a better teacher overall. 4 Current teacher evaluation system encourages professional growth for teachers. 5 Current teacher evaluation system is linked to student learning. 6 Current teacher evaluation system is able to assess teachers’ overall teaching abilities. 7 Current teacher evaluation system is a high stress process for teachers. 8 Current teacher evaluation system is a high stress process for administrators. 9 Current teacher evaluation system allows for thorough reviews of teachers’ overall teaching abilities. 10 Current teacher evaluation system allows for thoughtful reviews of teachers’ overall teaching abilities. Ideal Teacher Evaluations 11 Ideal teacher evaluations distinguish strong teachers from weak teachers. 12 Ideal teacher evaluations help teachers become better teachers. 13 Ideal teacher evaluations have the ability to improve student learning. 14 Ideal teacher evaluations are linked to student assessment data of any form. 15 Ideal teacher evaluations are linked to student assessment data by the state department of education (i.e., CSAPs). 16 Ideal TE are linked to student assessment data created by the teacher. Education Reform 17 Education reforms improve student learning. 18 Education reforms view teachers as knowledgeable professionals. 19 Education reforms view building administrators as knowledgeable professionals. 20 Policymakers, teachers have the same perceptions about what is needed to improve public education. 21 Policymakers, building admins have the same perceptions about what is needed to improve public education. The invisible schism 17 22 Teachers, building admins have the same perceptions about what is needed to improve public education. Senate Bill 191 23 I will play an active role in the implementation of my school’s new teacher evaluation system. 24 Teachers at my school believe it will be beneficial to the teachers’ professional practice at my school. 25 Teachers at my school believe it will be beneficial to students at my school. 26 I have a strong understanding of the legislation SB 191. 27 I was highly involved in the development of SB 191 during its implementation phase. 28 I supported SB 191 during its development. 29 I was pleased with the final version of SB 191 as it was passed. 30 The media portrayed teachers fairly during the bill’s passage. 31 The media portrayed administrators fairly during the bill’s passage. 32 The media portrayed policymakers (i.e.,legislators, lobbyists) fairly during the bill’s passage. 33 The reforms coming out of SB 191 have a chance to improve Colorado’s students’ learning in a meaningful way. Change 34 I view change as an opportunity for growth. 35 I respond well when policies change at work. 36 I respond well to change at work when I am involved with creating the change. 37 I respond well to change at work when informed of what new policies will be and I must learn the new policies. 38 There is a right way and a wrong way to introduce new policies in a school. Other 39 Which group has primary responsibility for successful implementation of new teacher evaluations? 40 Central focus of SB 191? Education Policy Analysis Archives Vol. 24 No. 87 18 Appendix B: Interview Questions 1. Seventy-nine percent of teachers and administrators (but 56% of policymakers) support the linking of student assessment data created by teachers to teacher evaluations. But 69% of teachers do NOT support the linking of statewide assessment data (such as TCAP) to teacher evaluations while 72% of administrators and 55% of policymakers are okay with it. Why do you think this is the case? 2. The majority of teachers and administrators believe both that building administrators have “primary responsibility” for the successful implementation of SB 191 and that the central focus of the SB 191 reform was teachers. How do you reconcile this? 3. Over 92% of everyone who took the survey (RSD teachers and administrators, and state policymakers) agreed that the ideal teacher evaluation system will “help teachers become better teachers.” But only 59% of all teachers (compared to 76% of administrators and 78% of policymakers) agreed that an ideal teacher evaluation system will distinguish strong teachers from weak teachers. Why do you think this is? 4. The overwhelming majority of teachers, administrators, and policymakers agree that both policymakers and teachers, and policymakers and building administrators, do not have the same perceptions about what is needed for education reform. These same groups also agreed, collectively at approximately 67%, that teachers and building administrators do have similar perceptions. Yet the data from this study indicate that teachers and administrators disagree far more than teachers and policymakers or building admin and policymakers. What do you make of this finding? 5. Over 93% of everyone who took the survey agreed that ideal teacher evaluations have the potential to improve student learning. Yet only 55% of teachers, compared to 83% of administrators and 67% of policymakers, believe that SB 191 has a chance to improve student learning. What do you think may explain this gap? 6. Similarly, 55% of both policymakers and teachers believe that education reforms can improve student learning, but almost 75% of administrators believe this is possible. What do you think explains this gap in perceptions? 7. Over 60% of all administrators believe the current teacher evaluation process is both thorough and thoughtful, but only 44% of teachers think it is thorough and 55% think it is thoughtful. Why do you think this gap exists between teachers and administrators? Also, why might teachers feel that the current process is more thoughtful than thorough? 8. Almost everyone who took the survey agrees (over 93%) that there is, in fact, a right way and a wrong way to introduce new policies in a school. What do you think is the right way and/or wrong way at your school? How might this knowledge apply to the introduction of new teacher evaluations in RSD next year? The invisible schism 19 9. Similarly, over 97% of teachers “respond well to work when involved with creating the change,” compared to 82% who respond well to changes at work “when informed of the new changes,” which is still a large percentage. Yet only 46% of teachers believe they will play an active role in the implementation of the new teacher evaluation system in their school. What do you think of this anticipated low rate of participation for teachers in the implementation of the new teacher evaluation system? Particularly in light of the data about change preferences? 10. The following data applies to teachers: a. 50% stated that they did not have a strong understanding of the bill; b. 70% that they did not like the bill in its final version; c. 55% that the bill does not have a chance of improving student learning. What do you make of this data collection? 11. (Time permitting) Anything else you would like to add? Education Policy Analysis Archives Vol. 24 No. 87 20 Appendix 3: Selected Survey Results # Question Teachers Admin Value df p- value N1 4 Current teacher evaluation system encourages professional growth for teachers. 70% 90% 14.089 3 .003 606 5 Current teacher evaluation system is linked to student learning. 70% 80% 3.975 3 .264 606 6 Current teacher evaluation system is able to assess teachers’ overall teaching abilities. 62% 86% 13.935 3 .003 606 9 Current teacher evaluation system allows for thorough reviews of teachers’ overall teaching abilities. 44% 68% 14.267 3 .003 606 11 Ideal teacher evaluations distinguish strong teachers from weak teachers. 59% 76% 16.071 6 .013 600 12 Ideal teacher evaluations help teachers become better teachers. 94% 97% 6.057 6 .417 600 13 Ideal teacher evaluations have the ability to improve student learning. 93% 97% 9.363 6 .154 600 15 Ideal teacher evaluations are linked to student assessment data by the state department of education (i.e., CSAPs). 31% 72% 47.296 6 .000 600 17 Education reforms improve student learning. 55% 75% 44.739 6 .000 587 22 Teachers, building admins have the same perceptions about what is needed to improve public education. 68% 66% 8.476 6 .205 587 28 I supported SB 191 during its development. 14% 50% 85.960 8 .000 578 29 I was pleased with the final version of SB 191 as it was passed. 14% 56% 77.199 8 .000 578 33 The reforms coming out of SB 191 have a chance to improve Colorado’s students’ learning in a meaningful way. 36% 83% 62.702 8 .000 578 34 I view change as an opportunity for growth. 96% 100% 17.552 6 .007 596 35 I respond well to change at work when informed of what new policies will be and I must learn the new policies. 82% 97% 25.191 6 .000 596 The invisible schism 21 36 I respond well to change at work when I am involved with creating the change. 98% 100% 9.351 6 .053 596 38 There is a right way and a wrong way to introduce new policies in a school. 94% 93% 4.232 6 .645 596 1 Counts are an average for Teachers and Administrators. For all questions, at least one cell had an expected count less than five, which may have impacted significance. Education Policy Analysis Archives Vol. 24 No. 87 22 About the Author Sarah Melvoin Bridich University of Denver sbridich@gmail.com Dr. Bridich is a graduate of the University of Denver’s Educational Leadership and Policy Studies program. Her research interests include state and national policy implementation challenges at the local level, school leadership, teacher evaluations, teacher voice, charter and innovation schools, change leadership, and the history of education reforms. Sarah currently is the lead researcher on a multi-school case study exploring the role of the school leader in transitioning from theory to practice in new high schools in the Denver area. education policy analysis archives Volume 24 Number 87 August 8, 2016 ISSN 1068-2341 Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for non- commercial purposes only, and no alteration or transformation is made in th e work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by the author(s) or EPAA. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), QUALIS A2 (Brazil), SCImago Journal Rank; SCOPUS, SOCOLAR (China). Please contribute commentaries at http://epaa.info/wordpress/ and send errata notes to Audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. mailto:sbridich@gmail.com http://www.doaj.org/ mailto:Audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE The invisible schism 23 education policy analysis archives editorial board Lead Editor: Audrey Amrein-Beardsley (Arizona State University) Editor Consultor: Gustavo E. Fischman (Arizona State University) Associate Editors: Sherman Dorn, David R. Garcia, Eugene Judson, Jeanne M. Powers (Arizona State University) Cristina Alfaro San Diego State University Ronald Glass University of California, Santa Cruz R. Anthony Rolle University of Houston Gary Anderson New York University Jacob P. K. Gross University of Louisville A. G. Rud Washington State University Michael W. Apple University of Wisconsin, Madison Eric M. Haas WestEd Patricia Sánchez University of University of Texas, San Antonio Jeff Bale OISE, University of Toronto, Canada Julian Vasquez Heilig California State University, Sacramento Janelle Scott University of California, Berkeley Aaron Bevanot SUNY Albany Kimberly Kappler Hewitt University of North Carolina Greensboro Jack Schneider College of the Holy Cross David C. Berliner Arizona State University Aimee Howley Ohio University Noah Sobe Loyola University Henry Braun Boston College Steve Klees University of Maryland Nelly P. Stromquist University of Maryland Casey Cobb University of Connecticut Jaekyung Lee SUNY Buffalo Benjamin Superfine University of Illinois, Chicago Arnold Danzig San Jose State University Jessica Nina Lester Indiana University Maria Teresa Tatto Michigan State University Linda Darling-Hammond Stanford University Amanda E. Lewis University of Illinois, Chicago Adai Tefera Virginia Commonwealth University Elizabeth H. DeBray University of Georgia Chad R. Lochmiller Indiana University Tina Trujillo University of California, Berkeley Chad d'Entremont Rennie Center for Education Research & Policy Christopher Lubienski University of Illinois, Urbana-Champaign Federico R. Waitoller University of Illinois, Chicago John Diamond University of Wisconsin, Madison Sarah Lubienski University of Illinois, Urbana-Champaign Larisa Warhol University of Connecticut Matthew Di Carlo Albert Shanker Institute William J. Mathis University of Colorado, Boulder John Weathers University of Colorado, Colorado Springs Michael J. Dumas University of California, Berkeley Michele S. Moses University of Colorado, Boulder Kevin Welner University of Colorado, Boulder Kathy Escamilla University of Colorado, Boulder Julianne Moss Deakin University, Australia Terrence G. Wiley Center for Applied Linguistics Melissa Lynn Freeman Adams State College Sharon Nichols University of Texas, San Antonio John Willinsky Stanford University Rachael Gabriel University of Connecticut Eric Parsons University of Missouri-Columbia Jennifer R. Wolgemuth University of South Florida Amy Garrett Dikkers University of North Carolina, Wilmington Susan L. Robertson Bristol University, UK Kyo Yamashiro Claremont Graduate University Gene V Glass Arizona State University Gloria M. Rodriguez University of California, Davis Education Policy Analysis Archives Vol. 24 No. 87 24 archivos analíticos de políticas educativas consejo editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editores Asociados: Armando Alcántara Santuario (Universidad Nacional Autónoma de México), Jason Beech, (Universidad de San Andrés), Ezequiel Gomez Caride, Pontificia Universidad Católica Argentina), Antonio Luzon, Universidad de Granada Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Juan Carlos González Faraco Universidad de Huelva, España Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Miguel Ángel Arias Ortega Universidad Autónoma de la Ciudad de México María Clemente Linuesa Universidad de Salamanca, España José Gregorio Rodríguez Universidad Nacional de Colombia, Colombia Xavier Besalú Costa Universitat de Girona, España Jaume Martínez Bonafé Universitat de València, España Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Xavier Bonal Sarro Universidad Autónoma de Barcelona, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México José Luis San Fabián Maroto Universidad de Oviedo, España Antonio Bolívar Boitia Universidad de Granada, España María Guadalupe Olivier Tellez, Universidad Pedagógica Nacional, México Jurjo Torres Santomé, Universidad de la Coruña, España José Joaquín Brunner Universidad Diego Portales, Chile Miguel Pereyra Universidad de Granada, España Yengny Marisol Silva Laya Universidad Iberoamericana, México Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Mónica Pini Universidad Nacional de San Martín, Argentina Juan Carlos Tedesco Universidad Nacional de San Martín, Argentina Gabriela de la Cruz Flores Universidad Nacional Autónoma de México Omar Orlando Pulido Chaves Instituto para la Investigación Educativa y el Desarrollo Pedagógico (IDEP) Ernesto Treviño Ronzón Universidad Veracruzana, México Marco Antonio Delgado Fuentes Universidad Iberoamericana, México José Luis Ramírez Romero Universidad Autónoma de Sonora, México Ernesto Treviño Villarreal Universidad Diego Portales Santiago, Chile Inés Dussel, DIE-CINVESTAV, México Paula Razquin Universidad de San Andrés, Argentina Antoni Verger Planells Universidad Autónoma de Barcelona, España Pedro Flores Crespo Universidad Iberoamericana, México José Ignacio Rivas Flores Universidad de Málaga, España Catalina Wainerman Universidad de San Andrés, Argentina Ana María García de Fanelli Centro de Estudios de Estado y Sociedad (CEDES) CONICET, Argentina Juan Carlos Yáñez Velazco Universidad de Colima, México javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/819') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/820') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/4276') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1609') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/825') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/797') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/555') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/823') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/798') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/814') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/801') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1617') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/2703') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/802') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/816') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/826') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/804') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/3264') The invisible schism 25 arquivos analíticos de políticas educativas conselho editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editoras Associadas: Geovana Mendonça Lunardi Mendes (Universidade do Estado de Santa Catarina), Marcia Pletsch, Sandra Regina Sales (Universidade Federal Rural do Rio de Janeiro) Almerindo Afonso Universidade do Minho Portugal Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil José Augusto Pacheco Universidade do Minho, Portugal Rosanna Maria Barros Sá Universidade do Algarve Portugal Regina Célia Linhares Hostins Universidade do Vale do Itajaí, Brasil Jane Paiva Universidade do Estado do Rio de Janeiro, Brasil Maria Helena Bonilla Universidade Federal da Bahia Brasil Alfredo Macedo Gomes Universidade Federal de Pernambuco Brasil Paulo Alberto Santos Vieira Universidade do Estado de Mato Grosso, Brasil Rosa Maria Bueno Fischer Universidade Federal do Rio Grande do Sul, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Fabiany de Cássia Tavares Silva Universidade Federal do Mato Grosso do Sul, Brasil Alice Casimiro Lopes Universidade do Estado do Rio de Janeiro, Brasil Jader Janer Moreira Lopes Universidade Federal Fluminense e Universidade Federal de Juiz de Fora, Brasil António Teodoro Universidade Lusófona Portugal Suzana Feldens Schwertner Centro Universitário Univates Brasil Debora Nunes Universidade Federal do Rio Grande do Norte, Brasil Lílian do Valle Universidade do Estado do Rio de Janeiro, Brasil Flávia Miller Naethe Motta Universidade Federal Rural do Rio de Janeiro, Brasil Alda Junqueira Marin Pontifícia Universidade Católica de São Paulo, Brasil Alfredo Veiga-Neto Universidade Federal do Rio Grande do Sul, Brasil Dalila Andrade Oliveira Universidade Federal de Minas Gerais, Brasil