CIE Template 1 Volume 18, Number 1 April 20, 2015 ISSN 1099-839X Connecticut’s Educator Evaluation Process: The Concerns and Experiences of Educators in Two Pilot Districts Diana J. LaRocco 1 and Beth Ann Loveland Sennett 3 University of Hartford Deborah S. Wheeler 2 Upper Dublin School District This article provides select results from a study designed to explore and describe school personnel’s concerns about implementing Connecticut’s newly developed model of a system of educator evaluation and development. Data were gathered from school personnel in two public school districts involved in a statewide pilot of the model system. To examine change in educators’ concerns over time, study volunteers completed an Internet-based survey twice: in the beginning of the school year and at the end of the school year. A subsample of survey respondents also participated in a focus group at the end of the pilot school year. Results revealed a shift in educators’ concerns about the new educator evaluation system, moving from a focus on self and tasks to ideas about how to improve the process. At the same time, the level of commitment to implementing the new system as experienced in the pilot decreased somewhat. Keywords: Teacher concerns; teacher evaluation systems; change; pilot Background and Rationale Despite countless U.S. federal and state school- reform efforts over more than four decades, significant national concern about the academic achievement gap between subgroups of students remains. On the one hand, it is understood that student’s academic achievement is a complex issue with factors related to the individual child, families, communities, schools, and teacher quality each playing a role. On the other hand, federal mandates such as the Elementary and Secondary Education Act of 1965, and its most recent reauthorization, the landmark No Child Left Behind (NCLB) Act of 2001 (NCLB, 2002), squarely place responsibility for closing the achievement gap on school district personnel. The Congressional intent behind NCLB is to make certain that all students learn (McDonnell, 2005; Reichbach, 2004; The Thomas B. Fordham Foundation, 2001). Among other provisions, the law mandates that schools provide highly qualified teachers for every classroom and attempts to assess school districts’ progress toward the implementation of comprehensive school reform efforts through an emphasis on accountability. Having a teacher that is considered highly qualified under NCLB (2002) is only a beginning and does not guarantee in any way the teacher’s classroom performance would lead to improvements in student achievement. Ongoing educator supervision and evaluation systems that provide a direct connection between teacher practice, professional development, and improvement in educational outcomes for all students must be in place (Guskey, 1986; Peterson, 2000; Stronge, 2006). A combination of recent state and federal policy initiatives have begun to address the issue of educator supervision, and this has resulted in a significant number of states passing teacher evaluation legislation designed to address educator effectiveness based in part on student learning (Mead, 2012). Among these states is Connecticut, where in 2012, the state legislature passed a Current Issues in Education Vol. 18 No. 1 2 law that mandated fundamental changes in how teacher and school administrator evaluations are carried out. The provisions of Connecticut’s law, which are explained in more detail later, delineate key components of educator evaluation with the aim of improving the quality of teaching, school administration, and by extension student achievement. Before the law’s passage, the criteria for and the ways in which teachers and administrators were evaluated varied widely among Connecticut’s 164 public school districts. Implementing the law’s requirements represented a significant change from past practice. For the change to be enacted successfully educators must be authentically engaged in the process and have their concerns considered and addressed (Hall & Hord, 2011; Wagner et al., 2006; Waugh & Punch, 1987). This paper describes the results of an exploratory study involving school personnel working in two public school districts that had participated jointly in a statewide pilot of Connecticut’s newly mandated educator evaluation system. The investigation focused expressly on the phenomenon of educators’ concerns related to implementing the innovation in its initial stages. Teacher Supervision and Evaluation Danielson and McGreal (2000) described teacher evaluation as serving two primary purposes (a) to make decisions that lead to particular consequences (e.g., determine a teachers’ worthiness or merit) and (b) to enhance professional skills (e.g., the teacher is learning a new instructional strategy). The former is summative, generally externally imposed, uniformly applied, and intended to assess all teachers on the same criteria (Glickman, Gordon, & Ross-Gordan, 2004). The latter is formative and aims to provide teachers with the opportunity to learn about themselves as teachers and is intended to further their instructional practice (Glickman et al., 2004; Marzano, Frontier, & Livingston, 2011). Historically, supervision and evaluation in America’s public schools existed to control standards and improve instruction, with a focus on judging the work of the teacher rather than on student achievement (Burnham, 1976). Throughout the 1800s, large and multi-layered school systems began to develop in urban areas, leading to “a demand for teachers who held expertise in specific disciplines and for administrators who could assume increasingly complex roles” (Marzano et al., 2011, p. 13). In subsequent decades, teaching began to be viewed as a multifaceted enterprise that warranted ongoing feedback. During the 1960s, “the concept of supervision as democratic, cooperative, and creative guided the practice” (Burnham, 1976, p. 303). The notion of teachers as professionals who desired to improve themselves and their practice emerged, as did curriculum development and professional development courses as methods for improving instruction. Building on the idea that teachers are professionals who want to expand their expertise, the concept of clinical supervision, with classroom observation followed by a conference between teacher and observer, quickly gained importance during the mid- 20th century (Marzano et al., 2011). In the 1980s, Hunter (1983) popularized a seven-step model described as mastery teaching and provided a framework for teacher and supervisor to collaboratively analyze specific elements of an observed lesson and determine effectiveness of the teacher’s practice. Hunter’s model was based on lesson design analysis and a common language of instruction that supported deeper reflection on instructional practice. Hunter’s model became “the content of the preconference, observation, and postconference” (Marzano et al., 2011, p. 20). Contemporaneously, Glatthorn (1984) posited that teachers’ individual needs for professional growth should become part of the supervisory process, thereby providing a degree of ownership over their own skill development. Likewise, other scholars (Glickman, 1985; McGreal, 1983) began to promote the idea of a continuum of supervisory formats to be implemented based on teacher expertise and experience. This conception provided flexibility in placing teachers in differentiated supervisory stages based on performance (McGreal, 1983). In addition, improvement of instruction was the focus of both the supervisory process and professional development of teachers (Glickman, 1985). Despite this momentum in advancing teacher evaluation practices, findings from the RAND study (Wise, Darling-Hammond, Tyson-Bernstein, & McLaughlin, 1984) indicated that supervisory practices were not considered by teachers to be sufficiently standardized to support improvement of pedagogy. Administrators had neither adequate training nor the ability or determination to effectively implement teacher evaluations. Feedback was not well accepted by teachers, and evaluation practices lacked a uniform approach. To this time, agreement on which teacher competencies to evaluate was absent as was consensus on process for doing so. Charlotte Danielson’s 1996 work, Enhancing Professional Practice: A Framework for Teaching, is often credited for confronting both of these issues. The model, which articulates four domains of teaching responsibility—content, planning and preparation; instruction; the classroom environment; and professional responsibilities—provides evaluators with “precise language to distinguish teacher’s performance from one level to other” (Danielson, 2007, p. 27). Thus, teachers know the standards upon which they will be evaluated and are expected to reflect. The efficacy of Danielson’s work has been validated through the Consortium on Chicago School Research and the Measures of Effective Teaching (MET) Study, funded by the Gates Foundation. Findings of the MET study indicate that Danielson’s framework has predictive validity (Griffin, Connecticut’s Educator Evaluation Process: The Concerns and Experiences of Educators in Two Pilot Districts 3 2013). A key element of the successful use of the model in evaluation is the training of evaluators to use the framework with fidelity. Danielson’s work forms the basis for the pilot of Connecticut’s system for educator evaluation and development. Connecticut’s System for Educator Evaluation and Development Connecticut’s current system for educator evaluation and development has its beginnings in 1986 and with the adoption of the Education Enhancement Act. This act spurred a significant and long lasting process of change in Connecticut’s system of public education with a subsequent agenda of high standards for students and professional educators (Sergi, 2001). Briefly, Connecticut’s teachers were required to meet standards for entry into a teacher preparation program, demonstrate professional knowledge, and meet a new set of professional development standards. The trajectory of this change was expanded in 1999 with the publication by the Connecticut State Department of Education of Connecticut’s Commitment to Excellence in Teaching: The Second Generation. Three policy documents served as the framework for the state’s teacher evaluation process, most importantly Connecticut’s Common Core of Teaching (Connecticut State Department of Education [CSDE], 1999). The Common Core of Teaching was and remains linked by state law and regulations to teacher preparation, induction, and teacher evaluation. This set of professional teaching standards delineates the knowledge, skills, and dispositions expected of all Connecticut’s teachers. At that time, school districts throughout the state were directed to design and implement teacher evaluation processes and professional development plans aligned with the Common Core of Teaching. Nevertheless, as with other aspects of Connecticut’s 164 public school systems, each district’s plan for teacher evaluation was unique to the district. For example, the frequency with which a teacher or administrator would be observed could vary as did the foci of observations. In other words, there was not a statewide or uniform approach to the educator evaluation process. Although districts would routinely modify their teacher evaluation and professional development systems based on current research in subsequent years, no additional formal restructuring of educator evaluation and professional development was put in place until May, 2012, when Connecticut was awarded a waiver from NCLB (2002) requirements (Connecticut Coalition for Achievement Now, 2012). At the same time, the state legislature passed Public Law 12-116, An Act Concerning Educational Reform (2012). Among other provisions, Section 51(a) of the act mandates that the superintendent of each local or regional board of education annually evaluate each teacher in accordance with guidelines established by the Connecticut State Board of Education. The law went into effect July 1, 2012. An interagency workgroup, the Performance Evaluation Advisory Council (PEAC), was created and authorized to develop the framework for a model teacher-evaluation system based on the recently revised Common Core of Teaching (CSDE, 2010). PEAC’s work led to the development of Connecticut Guidelines for Educator Evaluation and Support (CSDE, 2012a), which are aligned with relevant provisions of Public Law 12-116 (2012) and to which all districts are expected to adhere. The State Board of Education unanimously approved the guidelines in June 2012. Concomitantly, the Connecticut State Department of Education (CSDE, 2012b) developed a model System for Educator Evaluation and Development (SEED) that schools and district could choose to adopt to meet the requirements of Public Law 12-116 (2012). The SEED model is based on Danielson’ (2007) rubrics for measuring professional performance and Connecticut’s Common Core of Teaching (CSDE, 2010). Ten school districts or consortiums of districts were selected to pilot the SEED model during the 2012-2013 school year. Under the SEED model, 45% of a teacher’s annual evaluation rating is awarded based purely on student-achievement data resultant from standardized test performance. Forty percent of the summative rating is linked directly to professional performance, tied to standards presented in the Connecticut’s Common Core of Teaching (CSDE, 2010) and Danielson’s (2007) framework and rated through observations of practice by the supervisor. Five percent of the summative rating is determined through either school-wide achievement indicators or student feedback. The remaining 10% of the teacher’s summative rating is based on parent feedback gathered through a survey. All in all enacting the SEED model represented a significant departure from past practice for teachers and administrators in Connecticut. Description of the SEED Model Pilot To establish the context within which the SEED pilot and the study described herein took place a description of the pilot year is provided in this section. On April 5, 2012, the Connecticut State Department of Education issued an invitation and application (S. Pryor, personal communication, 2012) to all Connecticut school districts to become one of ten pilot sites for the SEED program. The criteria indicated for selection of the pilot site districts included: district size, geographical representation, district designation as urban, suburban or rural, and consortia of small districts working collaboratively with other small districts. Discussion of this opportunity between the superintendents of District A and District B led to a shared application to become a consortium pilot site. Each superintendent wanted their district to gain the benefits provided through participation in the pilot that would afford state funding for all required faculty training, mandated calibration of evaluators, and implementation Current Issues in Education Vol. 18 No. 1 4 of the required parent and faculty survey administrations. In addition, participation would provide a year for practice and mastery of the nuances of the new evaluation system by their faculty before the statewide implementation scheduled for the academic year 2013- 2014. Finally, the superintendents perceived the districts would benefit from active collaboration with the State Department of Education leading to enhanced recognition and stature within the state. The rationale for submission of a collaborative application was threefold. First, these school districts are located in the rural northwestern region of the state, an area that tends to be excluded from representation in major State Department of Education initiatives. Second, the combined enrollment of about 2,000 students and employment of about 230 certified staff members (Strategic School Profiles, 2011-2012) would provide a viable number of participants for the pilot activities that could not be realized independently. Finally, these adjacent districts presented diverse structures, thus providing input from educators in both a small community-based, high-achieving comprehensive PK-12 district and a small regional PK-12 district comprised of three non-contiguous member towns whose high school is also a regional vocational agriculture education site. These districts have an established history of sharing resources and collaborating on initiatives that strongly supported the decision to apply as a consortium for inclusion in the state pilot. On July 3, 2012, following the approval of their joint application, the superintendents of District A and District B participated in an informational session held by the State Department of Education to introduce the pilot program. Table 1 presents the professional development activities conducted during the 2012-2013 school year by agencies contracted by the State Department of Education to provide training in both teacher and administrator evaluation to personnel in the pilot districts. Evaluators from both districts were provided access to Teachscape, a web-based program that trained and calibrated the skills of evaluators. Teachscape used Danielson’s (2007) rubric for measuring professional practice to appraise the skills of administrators in evaluating teaching practice through watching videos of lessons and rating the observed performance. My Learning Plan OASYS, a web-based observation and appraisal management system, was made available to pilot districts for use in maintaining records of annual goals, observations, and mid-year reflective conversations. In addition, My Learning Plan calculated summative ratings through application of the stored data to the SEED rating-matrix. Support for pilot districts in administering and interpreting response data from parent, faculty, and student surveys was provided through Panorama Education, a survey platform built specifically for K-12 education. Table 1 SEED Training Sessions for Teacher and Administrator Evaluation Agency Dates Participants /Topics Teacher Evaluation Education Connection 8/16/12 Teacher Evaluators from Districts A and B Introduction to teacher evaluation: overview of the 45%, 40%, 10% and 5% components of the model Education Connection 8/17/12 Teacher Evaluators from Districts A and B 45% Student Learning Objectives (SLO A & B) Education Connection 8/23/12 Superintendents from Districts A and B meet with Educational Consultant to review implementation status Education Connection 8/30/12 Certified staff from Districts A and B Teacher Evaluation Orientation Education Connection 9/10/12 Teacher Evaluators from Districts A and B 40% Teacher Practice components: Danielson Rubric and Summative Matrix Overview of Parent and Student Related Indicators Education Connection 1/18/13 Superintendents from Districts A and B meet with Educational Consultant to review implementation status Administrator Evaluation Connecticut Association of Schools 9/13/12 Administrator Evaluators from Districts A and B Introduction to administration evaluation: overview of the 45%, 40%, 10% and 5% components of the model Connecticut Association of Schools 9/25/12 Administrator Evaluators from Districts A and B 45% Teacher Effectiveness Outcomes, 10% and 5% Stakeholder Feedback Connecticut Association of Schools 10/1/12 Administrator Evaluators from Districts A and B 45% Student Learning: State Tests and Local Measures Connecticut Association of Schools 10/9/12 Administrator Evaluators from Districts A and B 40% Principal Performance and Practice Connecticut Association of Schools 10/17/12 Administrator Evaluators from Districts A and B Debrief 40% Principal Performance & Data System Connecticut Association of Schools 11/6/12 All administrators from Districts A and B Overview of Administrator Evaluation Model Components Connecticut’s Educator Evaluation Process: The Concerns and Experiences of Educators in Two Pilot Districts 5 Each district maintained participation in the pilot program throughout the school year and successfully evaluated all certified faculty below the rank of superintendent through application of the summative evaluation rubric. District personnel holding Connecticut certification as intermediate supervisors completed all teacher evaluations in both districts; there were no complementary evaluators trained or used. In District A, all teacher evaluators were principals or central office administrators. District B permitted teachers serving as department chairpersons as well as principals and central office administrators to evaluate teachers. At the conclusion of the pilot school year, each district reported that they completed the goal-setting process by the extended date of November 15, and the summative rating process by June 30. Neither district met the target of conducting six formal observations with every certified teacher. This was similar to other districts participating in the pilot. In sum, Connecticut’s Public Law 12-116 (2012) is an example of a public policy that aims to improve the quality of teaching and by extension student achievement. Similar federal polices and other comprehensive school reform efforts often require significant investment in changing the way educators approach their work and the need to implement innovations not only at the organizational level but also at the individual teacher level. Connecticut’s SEED is representative of a summative teacher evaluation process, with formative components, and this is a fundamental change in how the state’s school districts have evaluated educators. Implementation of innovation requires active participation and significant commitment on the part of both teachers and administrators. Given what is known about innovation implementation and the change process, a study eliciting public school personnel’s concerns about implementing Connecticut’s newly developed educator evaluation process at the beginning and end of the pilot year seemed warranted. Guiding Framework: Change and Innovation Implementation The theoretical underpinnings for this study are rooted in change theory (Demers, 2007) as it applies to the implementation of educational innovation and reform efforts (Fullan, 1993, 1999, 2000, 2001, 2003, 2006; Hall & Hord, 2006; Hord, Rutherford, Huling-Austin, & Hall, 2006; Wagner et al., 2006). Authentic educational change “desired or not, represents a serious personal and collective experience characterized by ambivalence and uncertainty” (Fullan, 2007, p. 23). If the results of the change process are successful, it can result in “a sense of mastery, accomplishment, and professional growth” (Fullan, 2007, p. 23). Observers of the change process have long contended that educational leaders must understand that the adoption and successful implementation of any innovation begins at the individual level (Fullan, 1985; Hall & Hord, 2006; Hord et al., 2006; Wagner et al., 2006). Moreover, success depends on “how school district central offices create and implement supports for change” (Honig, Copland, Rainey, Lorton, & Newton, 2010, p. 5). Implementing an educational innovation, such Connecticut’s new educator evaluation process, is accompanied inevitably by new learning and a need to rethink and adjust current practice. Such change often elicits concerns, anxiety, and uncertainty. Research has shown that when teachers’ concerns pertaining to implementing educational innovations are identified and addressed individuals are more apt to implement related practices (Burstein, Sears, Wilcoxen, Cabello, & Spagna, 2004; McLeskey & Waldron, 2002; Pedron & Evans, 1990; van den Berg, Sleegers, Geijsel, & Vandenberghe, 2000; Wade, Welch, & Jensen, 1994). Through understanding and addressing individual concerns, a school and personnel can better provide support with implementation of the innovation (Hall & Hord, 2006). As such, the framework that guided this investigation is Hall and Hord’s (2006, 2011) Concerns- Based Adoption Model (CBAM). The CBAM is a practical, evidence-based model (Anderson, 1997) focused on describing, measuring, and explaining the process of change experienced by both individuals and groups of educators attempting to implement an innovation. An innovation is something that is new to the individual and can encompass any combination of beliefs, understandings, behaviors, materials, instructional strategies, a curriculum, or a program. In the study described herein, the specific innovation under consideration was public school personnel’s concerns about implementing Connecticut’s newly developed model of a system of educator evaluation and development. The CBAM places particular emphasis on the diverse and unique meanings that individuals assign to the change and acknowledges that implementing any innovation is a highly personal experience. At the same time, it is understood generally that most individuals go through predictable stages of change, each of which is characterized by questions and concerns about the innovation as it unfolds. Concerns are not necessarily fears, anxieties, or worries, but rather “the composite representation of the feelings, preoccupation, thought, and consideration given to a particular issue or task” (Hall & Hord, 2006, p. 138). In this model, change and implementing an innovation are one and the same. The CBAM (Hall & Hord, 2006, 2011) encompasses six elements that interact dynamically as implementation of an innovation and the associated changes unfold. The elements are (a) external environment, (b) organizational culture, (c) resource system, (d) change facilitator, (e) probes, and (f) interventions. The external environment encompasses the historical, social, economic, and political milieu within which a school operates. It includes many and varied Current Issues in Education Vol. 18 No. 1 6 sources of externally imposed change that are outside of a school’s immediate locus of control. Notably, federal, state, local, and other externally imposed mandates quite often set the change agenda for schools (Hall & Hord, 2011), as was the case in this study. A school’s culture permeates the organization; is socially constructed; and encompasses shared knowledge, beliefs, and experiences (Hall & Hord, 2011). The culture is evidenced in how a school looks; the rituals and celebrations that take place; the stories shared; and the ways in which school personnel, students, families, and the community interact inside and outside of the school (Lindahl, 2006; Wagner & Masden-Copas, 2002). The resource system includes, among other means of support, materials (e.g., books, articles, manuals), time, and expert knowledge of the innovation available to those who will be implementing the change (Hall & Hord, 2011). Those with expert knowledge might include principals, teachers, staff developers, instructional coaches, or consultants external to the school. The change facilitator can be a single person, or several persons, whose primary role is to support implementers. Change facilitators are acutely aware of both the positive and negative, or dysfunctional, aspects of a school’s culture and maintain their focus on continuous improvement of teaching and learning (Hall & Hord, 2011). They draw on the resource system to ensure achieve implementation of and fidelity to the innovation. Probes are the formal and informal methods the change facilitator uses to assess an individual’s or group’s needs and concerns and identify personalized intervention strategies based on this assessment. Interventions are actions (e.g., sharing an book about the innovation) or events (e.g., rescheduling a meeting due to illness) that can affect the implementation of an either positively or negatively (Hall & Hord, 2011). The interventions emerge from an ongoing process assessment and feedback, are planned, systematic, and tailored to specific needs of individuals, groups, or the organization as a whole. Related to probes, the CBAM also includes three diagnostic dimensions: (a) Stages of Concern (SoC), Levels of Use (LoU), and Innovation Configurations (IC). SoC comprises seven developmental stages of concern that cluster into three areas, self, task, and impact (Hord et al., 2006). LoU covers six characterizations of distinctly different observable, patterns of individual behavior related to using the innovation. Finally, IC, or IC Maps depict a continuum of ways in which individuals might operationalize the components and dimensions of an innovation. The change facilitator is the individual who has responsibility for overseeing the innovation, building a supportive context, and helping individuals (Hall & Hord, 2006, 2011). While this is quite often the principal, Hall and Hord (2006) suggest that the “optimal arrangement for an organization is to have a leadership team” (p. 269). In the CBAM, the team would use both informal and formal tools to gather diagnostic information about how they might use resources and intervene to support individuals as they implement the innovation. More specifically, diagnosis would include: administering the SoC Survey (George, Hall & Stiegelbauer, 2006) to understand individual concerns related to the change process, developing LoU (Hall, Dirksen, & George, 2008) profiles to determine what individuals or groups are doing related to the innovation, and constructing and using IC maps to document and assess fidelity of implementation (Hord, Stiegelbauer, Hall, & George, 2008). The focus of this study is public school teachers’ and administrators’ concerns about implementing the newly developed system of educator evaluation and development. The SoC dimension of the CBAM (Hall & Hord, 2006, 2011) served to define the main variables of interest-educators’ concerns. Table 2 provides a summary of the CBAM stages of concern. Table 2 CBAM Stages of Concern (Hall & Hord, 2011) Categories Stage of Concern Stage Description Impact 6. Refocusing The individual is thinking about broader benefits of the innovation, including the possibility of introducing major changes or replacement of the innovations. 5. Collaboration The individual focuses on coordinating and collaborating with colleagues to improve the use of the innovation. 4. Consequence The individual focuses attention on the students affected by the innovation within his or her immediate sphere of influence. Task 3. Management The individual has shifted focus to the various processes and tasks required by the innovation. The focus is on efficiency, managing, and scheduling. Self 2.Personal The individual is uncertain about his or her ability to meet the requirements of the innovation, as well as his or her role in the innovation. 1. Informational The individual has a general awareness of the innovation and has an interest learning more. The interest is focused on substantive aspects of the innovation, not on his or her role in the innovation. Unrelated 0. Unconcerned The individual shows little or no concern about the innovation. Connecticut’s Educator Evaluation Process: The Concerns and Experiences of Educators in Two Pilot Districts 7 Methodology This exploratory study employed two methods of data collection: an Internet-based survey and a focus group discussion. To ensure participant confidentiality, the second author, who was the superintendent in one of the districts at the time of the study, was not involved in any way with the data collection process. The research- developed survey was administered online, in the beginning and the end of the pilot school year. Individuals completing the spring administration had an opportunity to self-select to participate in a 90-minute focus group discussion. The focus group was conducted at the end of the pilot school year. The paper is focused on change in school personnel’s concerns over time. Both teachers and administrators comprised the sample because the phenomenon of interest is concerns related to implementation of an innovation (i.e., SEED) that required reciprocal engagement, new learning, and significant commitment and the part of both groups. Therefore, the results presented are those from the 55 educators (i.e., teachers and administrators), working in two of the pilot districts, who completed both the fall and spring administrations of the survey and the subsample of eight individuals who self-selected to participate in the focus group discussion. Survey Description, Administration, and Data Analysis The recommendations of Dillman, Smyth, and Christian (2009) were followed for the design and administration of the survey. To confirm the clarity of the survey language and directions and try out the entire survey process (e.g., software functionality, time to complete the survey), a small-scale pilot was conducted with four individuals working in districts other than the ones under study. No modifications were made based on the pilot test. A copy of the tool is in the Appendix. The survey had four sections. For Sections I - III respondents were asked to read each statement and check the number on a 7-point Likert scale that best represented the degree to which each statement was true of them. Section I contained 7-items designed to measure concerns associated with the educational innovation (George et al., 2006; Hall & Hord, 2006, 2011). The items were in the form of statements that described concerns that a respondent could have regarding their involvement with the newly developed educator evaluation process. Section II of the survey contained 8-items that gauged participants’ overall experiences with and opinions about teacher evaluation generally. Section III contained 5- items that queried respondents about their commitment to key components of the educator evaluation system. Section IV of the survey contained five items that collected basic demographic information about each participant. The spring administration contained an additional item that provided respondents an opportunity to self-select to participate in a 90-minute focus group interview, which is described later. In the fall, study participants were recruited from the population of 229 Pre-K to 12 school personnel (e.g., teachers and administrators) working in two of the pilot districts. The superintendent of each district provided the lead author with an Excel file that contained names and school e-mail addresses. A series of six e-mails contacts were made to maximize the response rate (Dillman et al., 2009). The e-mails were sent through the online survey program. They contained information about the purpose of the study, a link to the informed consent form, and a link to the survey, which was a unique URL that was automatically created for each potential respondent. This URL was encrypted with SSL for added security during survey taking. The linked informed consent form was used because the online e-mail system did not allow attachments. Seventy-six individuals took part in the first administration of the survey (fall). Given the purpose of the study was to look at change in educators’ concerns over time, only these individuals were recruited to participate in the follow-up, end of pilot survey administration (spring). The procedures for the spring administration of the survey were the same as those of the fall. In all, 55 individuals participated in both the fall and spring administrations. Table 3 contains demographic information about this sample. As might be expected, the overwhelming majority of participants were teachers. The numbers of participants per district and school level were relatively similar. Table 3 Sample Characteristics (N = 55) District n % A 24 43.6 B 31 56.4 Highest Level of Education Completed Bachelor’s Degree 7 12.7 Master’s Degree 31 56.4 Sixth Year Degree 17 30.9 Years of Experience in Education 5 Years or less 6 10.9 6-10 Years 11 20.0 11-15 Years 8 14.5 16-20 Years 16 29.1 21-25 Years 6 10.9 26 Years or more 8 14.5 Position Held Teacher 45 81.8 Support Personnel (e.g., counselor, social worker) 3 5.5 Administrator (building or central office) 7 12.7 School Elementary 25 45.5 Secondary 30 54.5 Current Issues in Education Vol. 18 No. 1 8 The raw survey data were downloaded online to a Comma Separated Value (.CSV) file and imported into Microsoft® Excel®. Data were inspected and imported into SPSS for further analysis. Scoring the instrument involved assigning numerical values to each of the response choices. The survey data were analyzed using descriptive statistics; frequencies and percentages were calculated. A paired-samples t test was used to compare differences between beginning and end of pilot year responses to the survey (Field, 2013). To ensure that spurious significant differences were not found, alpha was set at .0001. Using a smaller alpha level lessens the chance of a Type 1 error (Trochim, 2005). The small initial level for alpha required neither a Bonferroni nor Sidak correction, as the chance of finding one or more significant differences in 20 tests was 0.002 (0.2%). Focus Group Description and Data Analysis Data were also collected using a focus group interview, which is a qualitative method for obtaining participants’ perceptions on a defined topic of interest in an open, nonthreatening environment (Krueger & Casey, 2008). As noted earlier, the spring administration of the survey contained an item inviting respondents to participate in a focus group. Thirteen individuals expressed an interest; of those, eight people self-selected to participate—four from each district. There were three administrators and five teachers. To maintain confidentiality of the focus group participants, no additional information was collected. The focus group was held at central non-school location and lasted 90 minutes. To ensure participant confidentiality, the second author was not present during the focus group. Using Krueger and Casey’s (2008) focus group recommendations as a guide, the lead author facilitated the discussion, and it was structured as follows: (a) welcome, (b) overview of the topic, (c) review of ground rules, and (d) the questions. During the welcome, participants were thanked for attending, the purpose of the discussion was reviewed, and ground rules were covered. The questions framed the conversation were as follows. 1. What are your thoughts and feelings today about your personal experiences with any part of the pilot process? 2. What were your initial thoughts and feelings about the educator evaluation process? 3. Thinking about the future and moving forward, what are your preferences and hopes for the educator evaluation process? Each participant was invited in turn to give input and express opinions to ensure that varying points of view were obtained. Active listening techniques were employed and follow-up questions primarily took the form of elaboration probes, to elicit more information, and clarifying probes, to check of understanding and clear up any confusion (Rubin & Rubin, 2012). Two graduate students recorded the participants’ responses. Detailed notes were taken in real-time using word processing software and a laptop computer. Notes were also recorded on chart paper. Procedures commonly used in qualitative research were used to analyze the focus group responses (Marshall & Rossman, 2011; Merriam, 2009; Corbin & Strauss, 2008). To enhance credibility, the data were examined and reexamined several times throughout the analysis. First, the authors independently analyzed the focus group data. Sections of text (e.g., words, phrases) were marked and coded (i.e., labeled), by hand or using word processing features including comment and highlight. With each reading, themes, categories, and corresponding coding were refined and modified as necessary. Then, the independent data analyses were compared and consensus was reached on the themes. Next, the results of the survey are presented, followed by the results of the focus group discussion. The paper concludes with an overall discussion of the results, study limitations, and suggestions for action and future research. Survey Results Analysis of the fall survey data for the 55 respondents that are of interest in this study reveals that their concerns about the educator evaluation process were largely centered on self and tasks, as delineated in the CBAM (Hall & Hord, 2011). The items with the highest mean response in the fall were informational, personal, and management 6.02 (SD = 1.35), 6.29 (SD = 1.18), and 6.18 (SD = 1.32), respectively. At the self-stage of concern, individuals have more of an impersonal interest in learning about “substantive aspects of the innovation, such as general characteristics, effects, and requirements for use” (George et al., 2006, p. 8). Personal concerns suggest uneasiness about the innovation and its consequence for the individual, and management concerns are related to the time and effort involved in implementation (George et al., 2006). Using an alpha level of .0001, a paired-samples t test was conducted to evaluate whether participants’ fall and spring concerns differed significantly. The spring mean was significantly lower for informational concerns (M = 4.40, SD = 2.00) than was the fall mean (M = 6.02, SD = 1.35), with t(54) = 4.46, p < .01, d = .36, 99.99% CI [.09, 3.14]. Likewise, the spring mean was significantly lower for personal concerns (M = 5.31, SD = 1.73) than was the fall mean (M = 6.29, SD = 1.18), with t(54) = 3.66, p = .001, d = .66, 99.99% CI [.14, 2.11]. These trends in the data suggest participants were not as concerned in spring with learning more about SEED or its consequences for them as they had been in the fall. With regard to refocusing efforts, the spring mean was significantly higher (M = 5.16, SD = 1.66) than was the fall mean (M = 4.11, SD = 1.99), with t(54) = - 3.30, p = .002, d = -0.57, 99.99% CI [-2.40, .29]. This trend indicates participants were moving away from Connecticut’s Educator Evaluation Process: The Concerns and Experiences of Educators in Two Pilot Districts 9 concerns about task and focusing on impact concerns, perhaps developing ideas about how the innovation might be improved or considering a process that could replace it (George et al., 2006). Figure 1 provides a fall to spring comparison of participants’ stage of concern regarding implementing SEED. Figure 1. Fall to spring comparison of participants’ stage of concern regarding implementing SEED. * p < .01, ** p = .001, *** p = .002 A paired-samples t test was also conducted to evaluate whether participants’ fall and spring levels of commitment to implementing SEED differed significantly. With regard to commitment, the spring mean (M = 4.27, SD = 1.90) was significantly lower than was the fall mean (M = 5.11, SD = 1.75), with t(54) = 2.41, p = .019, d = .46, 99.99% CI [-.62, 2.29]. Concerning participants’ beliefs about the feasibility of SEED, the spring mean (M = 2.65, SD = 1.60) was significantly lower than was the fall mean (M = 3.51, SD = 1.75), with t(54) = 2.67, p = .01, d = .46, 99.99% CI [- .49, 2.20]. Likewise, the spring mean for the importance of implementing SEED (M = 3.29, SD = 1.94) was significantly lower than was the fall mean (M = 4.00, SD = 1.89), with t(54) = 2.05, p = .045, d = .37, 99.99% CI [- .74, 2.16]. Overall, these trends in the data suggest that participants’ commitment to implementing SEED had diminished over the pilot year. Figure 2 provides a fall to spring comparison of participants’ level of commitment to implementing SEED. Figure 2. Fall to spring comparison of participants’ level of commitment to implementing SEED. *p = .019, **p = .01, ***p = .045 Finally, participants’ general experiences with teacher evaluation and opinions about SEED did not change significantly over the pilot years, as can be seen in Figure 3. The results might be an artifact of the newness of the innovation. They might also be a product of the timeframe within which data were collected; a year in the life of an innovation might be insufficient to experience any impact. Figure 3. Fall to spring comparison of participants’ general experiences with teacher evaluation and opinions about implementing SEED. In sum, trends in the differences between the results of the fall and spring survey administration appear to show that participating educators were moving from a focus on self and task concerns about the SEED process to impact concerns. At the same time, their commitment to implementing SEED had waned. The results are what might be expected given SEED was just launched; and the trends in the data are in keeping with the CBAM (Hall & Hord, 2011). In the early stages of implementing an innovation, Hall and Hord (2011) contended, concerns of an inexperienced implementer will quite often center on informational needs, the consequences of the innovation for the individual, and management of logistics and tasks. As time goes on and the individual becomes more adept at implementing the innovation, concerns can shift to thoughts about how to improve the innovation or approaches that might be even better (Hall & Hord, 2011). Focus Group Results Similar to the results of the survey, analysis of the focus group data revealed that participants’ initial concerns centered on self and managing tasks, as well as seeming disorganization at the state-level. Focus group participants described several valuable outcomes of having participated in the SEED pilot. These included opportunities to reflect on practice, collaborate with colleagues, and improve practice. Upon completion of the pilot and although concerns remained, individuals described how they were focused on improving the evaluation process to meet local-level needs. The opinions expressed by the focus group participants were consistent across the District A and District B and the teachers and administrators in attendance. Current Issues in Education Vol. 18 No. 1 10 Initial Concerns Focus group participants described concerns experienced at the beginning of the pilot year that centered on self and a general sense of unease, disorganization at the state-level, and the specific tasks related to implementation. The educators spoke of “alarm and worry on the part of teachers” (P2) and “being nervous about next year and how that will go” (P3). One person described approaching the pilot with skepticism, “I didn’t like the idea at first [and] thought it would be silly. I thought, ‘if you can write a good goal and figure out how to do that, you will be fine” (P1). The timing of the pilot was troubling for some: “To kick off the school year and do this at the same time was tough. The timing piece was hard. . . . It might have made more sense to start later . . . more time to digest, if we had started later” (P8). Another participant expressed “The practice was overwhelming. Moving to five areas of professional development with new forms, new policies, and new procedures. That was crazy” (P3). Tied to their accounts of general uneasiness with implementing the new process were descriptions of a seeming lack of preparation or organization at the state- level and a need for better information. One person said, “It was a bumpy start. They kept re-writing what we were doing straight through to November” (P7). Some focus group participants identified the lived experience surrounding the state’s training for the pilot participants as a source of distress. One individual recalled: Training was incredibly frustrating . . . the trainers were trained just before we were. The state wasn’t ready to launch. They were piloting the plane while learning how to pilot the plane. From having heard the committee on many occasions, I know that they were ramming it through no matter what. (P9) Similar to the results of the survey, task concerns described by focus group participants included how the process would affect participants personally and those related to managing the process. For many, “The goal- setting process in and of itself caused anxiety” (P3). Under the new evaluation system, teachers are expected to engage with their principals (or their designees) in a goal setting conference. Notably, state guidelines require that at least two of the three required goals be focused on student learning, and they account for 45% of a teacher’s annual evaluation. The student growth goals are to be clearly tied to student achievement and school or district priorities. Teachers must write the student growth goals as student learning outcomes, which are long-term, broad academic statements that describe an expectation for student improvement. Indicators of academic growth and development are used to measure student-learning outcomes, and these are written as SMART goals (i.e., Specific and strategic, Measurable, Aligned and attainable, Time bound. One person expressed uneasiness about the percentages they were using in their SMART goals because they seemed arbitrary: [The] problem was that we came up with some number for SMART goals—70%, 56%, or whatever percent . . . . We should be working for 100%, but don’t want to be penalized for not getting there . . . . We don’t want to encourage a low benchmark; what’s a reasonable number? (P5) Another individual explained how unrealistic it was to set student growth goals that applied to all students: Problems with goals led to the program being taken less seriously. Some goals had to apply to all students, which is impossible in high school. I wrote for the majority of my students. Expectations did not fit reality. This is the divide between goals and the reality of teaching. (P6) Still another person talked about the challenge of aligning student growth goals with the professional, or teacher growth goals they were expected to write: “We were so focused on [the] goal selected for the [student growth] goal. I spent weeks trying to make sure that the teaching goal reflected on my professional goal” (P2). Management of the number of observations and the administrative paperwork were also cited as components of the system that worried the participants initially and throughout the pilot. One individual shared, “The administrative piece is overwhelming, and way too many forms. We are ‘formed’ to death” (P1). Someone else reported, “The pre-observation form took a lot of time . . . caused anxiety . . . and I’m not sure it’s worth it” (P2). Structural expectations of the system were described as elements needing to be overcome rather than accommodated. A participant pronounced, “It was overly ambitious to think that people would get through six observations” (P6). Time was noted as a factor that contributed to some participants’ initial distress. One person shared, “[For] evaluators, far more time [was] required to talk through, to explain, and reassure people— one session after another” (P5). Valuable Outcomes Derived from Participation Despite initial reservations, focus group participants clearly identified benefits and the valuable outcomes derived from having participated in the pilot process. Highly regarded were opportunities to collaborate and give input on the new process. One attendee shared, “I was glad they [the State Department of Education] gave us a chance to modify [the district’s process]” (P4); and another said, “My district engaged us in discussion about [the pilot]” (P5). The return to Connecticut’s Common Core of Teaching was described as helpful because, in one participant’s words, “There has been a lack of value placed on the value of the content being communicated. Need to have subject matter count for something” (P6). Connecticut’s Educator Evaluation Process: The Concerns and Experiences of Educators in Two Pilot Districts 11 As the pilot year progressed, participants’ concerns were described as shifting away from self and managing tasks to their professional responsibility and how the goal setting process was affecting students. One participant commented, “Teachers personalized this. This is the goal. These are the kids. My work will influence this, and this is what we are going to do” (P9). Overwhelmingly, participants described how the pilot process afforded them with opportunities to reflect on professional practice, examine their teaching, and improve. The process was credited with providing new opportunities to work with colleagues and collaborate. As one participant shared, “Lots of good conversation around goal-setting . . . opportunities to discus pedagogy, to be in the classroom, to coach.” (P7). One person recounted how participating in the pilot “forced reflection where there was none . . . right now I am thinking about the notion of reflection . . .where there wasn’t enough, now there is more” (P1). Another individual explained, “We have looked at practice under a magnifying glass. At the time, it was hard. Now that it’s done, I am glad . . . This did cause me to try to change my practice in a positive way” (P2). Still another stated, “The whole point is that we need to know. We are using evaluations to improve and support teaching” (P3). Remaining Concerns Expressed benefits aside, doubt continued for some as to the value of the new system as experienced in the pilot. As one participant related, “I think it was not of much value. Because of disorganization, there was a lack of clarity. We went through the motions. So in the here and now I feel the same as I did at the beginning” (P5). Some spoke about what they viewed as insignificant change from the old evaluation system to the new. One individual explained, “It was not that much different from what we were doing at our schools. The SMART goals I set were similar to those I had been setting” (P6). Another individual shared perspective on what was lacking in the new system, “Teaching is a balance of science and art. The art component needs attention. This program does not consider the art piece, too. Data cannot account for all elements of teaching” (P7). This final comment seems to allude to a move towards refocusing efforts on how the SEED process could be improved, as was the case in the survey. Discussion This paper provides a look at public school personnel’s concerns about implementing Connecticut’s new system of educator evaluation and development in two pilot districts. Both the survey and the focus group results revealed a shift in district personnel’s concerns from an initial focus on self and tasks to refocusing efforts on local improvements to the educator evaluation process. Focus group participants described the demands of implementing the new program as overwhelming at the beginning of the pilot year. Consistent with the survey findings, the initial concerns mirrored those of inexperienced users, as described by Hall and Hord (2011). The initial concerns centered on self— learning about the innovation and acquiring the personal capacity for implementing the innovation (Hall & Hord, 2011). Focus group participants were also concerned about tasks, primarily those related to the individual’s role in and ability to manage, schedule, perform, or meet the requirements of the innovation (Hord & Hall, 2011). As one example, at the beginning of the implementation of the pilot program, focus group participants’ concerns revolved around the uncertainty and challenges associated with the novelty of the SEED process. The challenges were described as seemingly insurmountable, and participants expressed having felt personally overwhelmed. They shared how they struggled with the tasks of setting goals and managing the volume of observations and paperwork. These results are compatible with previous research suggesting the early stages of implementing and innovation can be fraught with intense concerns related to the need for information and worries about how the change will affect an individual personally (Burstein et al., 2004; McLeskey & Waldron, 2002; Pedron & Evans, 1990; van den Berg et al., 2000; Wade et al., 1994). The results of the spring survey and findings from the focus group are suggestive of participants’ movement toward impact concerns (Hall & Hord, 2011) and how they could improve upon the SEED process. Focus group participants described how they experienced process benefits as the pilot progressed. These included opportunities to reflect on practice, have collegial conversation, and improve their teaching. These benefits echo what other researchers have found. Teachers appreciate and are best supported through formative discourse and opportunities for reflection (Blase & Blase, 2000; Peterson & Comeaux, 1990; Ponticell & Zepeda, 2004; Stiggins & Bridgeford, 1985), and such practices can propel changes in how teachers approach their work (Vetter, 2012). At the same time, commitment to implementing SEED as experienced in the pilot diminished. Notably, by the end of the pilot year, data from both the survey and the focus group suggest that school personnel were refocusing their efforts on improvement of the evaluation process by effective management of expectations, streamlining reporting, and clarifying areas of the process that they described as vague. At the refocusing stage, individuals think about broader benefits of the innovation, including the possibility of introducing major changes or replacing the innovation entirely (Hall & Hord, 2011). In sum, neither district fully adopted the SEED process because it could not be implemented fully and with fidelity given the limitations of time, money, and personnel. Instead, educators in each district have used the feedback obtained through the pilot process and this Current Issues in Education Vol. 18 No. 1 12 study to refine and develop local versions of the teacher evaluation process. Gathering data about initial and ongoing concerns during a pilot of any innovation can aid school and district administrators with identifying areas for professional learning related to the innovations. These kinds of information can also provide insights into how to refine the innovation to meet local needs. Limitations and Recommendations for Future Research As with most research, this investigation has several limitations. First, the study was conducted in two districts among the state’s 164, the sample size for both the survey and the focus group were small, and individuals self-selected to participate. These facts limit the generalizability of the results to other settings. Another researcher should conduct a similar study in districts that have chosen to implement the SEED model. The results would add to an understanding of educator concerns about that process specifically and the Connecticut Guidelines for Educator Evaluation and Support (CSDE, 2012a) more generally. Second, a research-developed survey was employed in this study. While the survey provided a profile of participants’ stages of concern, additional research should be conducted to refine and validate the survey. Finally, other researchers should continue to examine the evolution of teacher evaluations systems in Connecticut and across the country to determine the components that best support teachers’ instructional improvement and lead to improved academic outcomes for all students. References An Act Concerning Educational Reform. Connecticut Public Law 12-116. (2012). Anderson, S. E. (1997). Understanding teacher change: Revisiting the concerns based adoption model. Curriculum Inquiry, 27(3), 331-367. doi: 10.1111/0362-6784.00057 Blase, J., & Blase, J. (2000). Effective instructional leadership: Teachers' perspectives on how principals promote teaching and learning in schools. Journal of Educational Administration, 38(2), 130-141. Burnham, R. M. (1976). Instructional supervision: Past, present and future perspectives. Theory into Practice, 15(4), 301-305. doi: 10.1080/00405847609542647 Burstein, N., Sears, S., Wilcoxen, A., Cabello, B., & Spagna, M. (2004). Moving toward inclusive practices. Remedial and Special Education, 25(2), 104-116. doi: 10.1177/07419325040250020501 Connecticut Coalition for Achievement Now. (2012, May 29). Connecticut granted NCLB waiver. Retrieved from http://www.conncan.org/media- room/blog/2012-05-connecticut-granted-nclb- waiver Connecticut State Department of Education. (1999). Common core of teaching. Retrieved from http://www.sde.ct.gov/sde/lib/sde/PDF/Curriculu m/Curriculum_Root_Web_Folder/ccteach_all.pd f Connecticut State Department of Education. (2010). 2010 common core of teaching: Foundational skills. Retrieved from http://www.sde.ct.gov/sde/lib/sde/pdf/educatorst andards/board_approved_cct_2-3-2010.pdf Connecticut State Department of Education. (2012a). Connecticut guidelines for educator evaluation. Retrieved from http://www.connecticutseed.org/wp- content/uploads/2012/09/Adopted_PEAC_Guide lines_for_Teacher_Evaluation.pdf Connecticut State Department of Education. (2012b). Connecticut’s system for educator evaluation and development (SEED). Retrieved from http://www.connecticutseed.org/?page_id=449 Corbin, A., & Strauss, J. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). Thousand Oaks, CA: Sage. Danielson, C. (1996). Enhancing professional practice: A framework for teaching. Alexandria, VA: Association for Supervision & Curriculum Development. Danielson, C. (2007). Enhancing professional practice: A framework for teaching (2nd ed.). Alexandria, VA: Association for Supervision & Curriculum Development. Danielson, C., & McGreal, T. L. (2000). Teacher evaluation to enhance professional practice. Alexandria, VA: Association for Supervision & Curriculum Development. Demers, C. (2007). Organizational change theory: A synthesis. Thousand Oaks, CA: Sage. Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). Hoboken, NJ: Wiley. Elementary and Secondary Education Act of 1965, PL 89- 10, 20 U.S.C. §6301 et seq. (1965). Field, A. (2013). Discovering statistics using IBM SPSS Statistics (4th ed.). Los Angeles, CA: Sage. Fullan, M. (1985). Change processes and strategies at the local level. Elementary School Journal, 85(3), 390-421. Fullan, M. (1993). Change forces: Probing the depths of educational reform. Bristol, PA: Falmer Press. Fullan, M. (1999). Change forces: The sequel. New York, NY: Falmer Press. http://www.conncan.org/media-room/blog/2012-05-connecticut-granted-nclb-waiver http://www.conncan.org/media-room/blog/2012-05-connecticut-granted-nclb-waiver http://www.conncan.org/media-room/blog/2012-05-connecticut-granted-nclb-waiver http://www.sde.ct.gov/sde/lib/sde/PDF/Curriculum/Curriculum_Root_Web_Folder/ccteach_all.pdf http://www.sde.ct.gov/sde/lib/sde/PDF/Curriculum/Curriculum_Root_Web_Folder/ccteach_all.pdf http://www.sde.ct.gov/sde/lib/sde/PDF/Curriculum/Curriculum_Root_Web_Folder/ccteach_all.pdf http://www.sde.ct.gov/sde/lib/sde/pdf/educatorstandards/board_approved_cct_2-3-2010.pdf http://www.sde.ct.gov/sde/lib/sde/pdf/educatorstandards/board_approved_cct_2-3-2010.pdf http://www.connecticutseed.org/wp-content/uploads/2012/09/Adopted_PEAC_Guidelines_for_Teacher_Evaluation.pdf http://www.connecticutseed.org/wp-content/uploads/2012/09/Adopted_PEAC_Guidelines_for_Teacher_Evaluation.pdf http://www.connecticutseed.org/wp-content/uploads/2012/09/Adopted_PEAC_Guidelines_for_Teacher_Evaluation.pdf http://www.connecticutseed.org/?page_id=449 Connecticut’s Educator Evaluation Process: The Concerns and Experiences of Educators in Two Pilot Districts 13 Fullan, M. (2000). Educational leadership. San Francisco, CA: Jossey-Bass. Fullan, M. (2001). Leading in a culture of change. San Francisco, CA: Jossey-Bass. Fullan, M. (2003). Change forces with a vengeance. New York, NY: Routledge Falmer. Fullan, M. (2006). Turnaround leadership. San Francisco, CA: Jossey-Bass. Fullan, M. (2007). The new meaning of educational change (4th ed.). New York, NY: Teachers College Press. George, A. A., Hall. G. E., & Stiegelbauer, S. M. (2006). Measuring implementation in schools: The stages of concern survey. Austin, TX: Southwest Educational Development Laboratory. Glatthorn. A. A. (1984). Differentiated supervision. Alexandria, VA: Association for Supervision & Curriculum Development. Glickman, C. D. (1985). Supervision of instruction: A developmental approach. Newton, MA: Allyn and Bacon. Glickman, C. D., Gordon, S. P., & Ross-Gordan, J. M. (2004). The basic guide to supervision and instructional leadership. New York, NY: Pearson. Griffin, L. (2013). Charlotte Danielson on teacher evaluation and quality. School Administrator, 1(70), 27-31. Guskey, T. R. (1986). Staff development and the process of teacher change. Educational Researcher, 15(5), 5-12. doi: 10.3102/0013189X015005005 Hall, G. E., Dirksen, D. J., & George, A. A. (2006). Measuring implementation in schools: Levels of use. Austin, TX. Southwest Educational Development Laboratory. Hall, G. E., & Hord, S. M. (2006). Implementing change: Patterns, principles, and potholes (2nd ed.). Boston, MA: Pearson Education. Hall, G. E., & Hord, S. M. (2011). Implementing change: Patterns, principles, and potholes (3rd ed.). Boston, MA: Pearson Education. Honig, M. I., Copland, M. A., Rainey, L., Lorton, J. A., & Newton, M. (2010). Central office transformation for district-wide teaching and learning improvement. Center for the Study of Teaching and Policy, University of Washington. Retrieved from http://depts.washington.edu/ctpmail/PDFs/S2- CentralAdmin-04-2010.pdf Hord, S. M., Rutherford, W. L., Huling, L., & Hall, G. E. (2006). Taking charge of change. Austin, TX: Southwest Educational Development Laboratory. Hord, S. M., Stiegelbauer, S. M., Hall, G. E., & George, A. A. (2008). Measuring implementation in schools: Innovation configurations. Austin, TX: Southwest Educational Development Laboratory. Hunter, M. C. (1983). Mastery teaching. El Segundo, CA: TIP Publications. Krueger, R. A., & Casey, M. A. (2008). Focus groups: A practical guide for applied research (4th ed.). Thousand Oaks, CA: Sage. Lindahl, R. (2006). The role of organizational climate and culture in the school improvement process: A review of the knowledge base. Retrieved from http://cnx.org/content/m13465/1.1/ Marshall, C., & Rossman, G. B. (2011). Designing qualitative research (5th ed.). Thousand Oaks, CA: Sage. Marzano, R. J., Frontier, T., & Livingston, D. (2011). Effective supervision: Supporting the art and science of teaching. Alexandria, VA: Association for Supervision & Curriculum Development. McDonnell, L. M. (2005). No Child Left Behind and the federal role in education: Evolution or revolution? Peabody Journal of Education, 80(2), 19-38. doi: 10.1207/S15327930pje8002_2 McGreal, T. (1983). Successful teacher evaluation. Alexandria, VA: Association for Supervision & Curriculum Development. McLeskey, J., & Waldron, N. L. (2002). Inclusion and school change: Teacher perceptions regarding curricular and instructional adaptations. Teacher Education and Special Education, 25(1), 41-54. doi: 10.1177/088840640202500106 Mead, S. (2012). Recent state action on teacher Effectiveness: What’s in state laws and regulations? Retrieved from http://bellwethereducation.org/sites/default/files/ RSA-Teacher-Effectiveness.pdf Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA: Jossey-Bass. No Child Left Behind Act of 2001, Pub. L. No. 107-110, § 115, Stat. 1425. (2002). Pedron, N. A., & Evans, S. B. (1990). Modifying classroom teachers’ acceptance of the consulting teacher model. Journal of Educational and Psychological Consultation, 1(2), 189-200. doi: 10.1207/s1532768xjepc0102_6 Peterson, K. D. (2000). Teacher evaluation: A comprehensive guide to new directions and practices. Thousand Oaks, CA: Corwin Press. Peterson, P. L., & Comeaux, M. A. (1990). Evaluating the systems: Teachers perspectives on teacher evaluation. Educational Evaluation and Policy Analysis, 12(1), 3-24. doi: 10.3102/01623737012001003 Ponticell, J. A., & Zepeda, S. J. (2004). Confronting well- learned lessons in supervision and evaluation. NASSP Bulletin, 88(639), 43-59. doi: 10.1177/01926350408863905 http://depts.washington.edu/ctpmail/PDFs/S2-CentralAdmin-04-2010.pdf http://depts.washington.edu/ctpmail/PDFs/S2-CentralAdmin-04-2010.pdf http://cnx.org/content/m13465/1.1/ http://bellwethereducation.org/sites/default/files/RSA-Teacher-Effectiveness.pdf http://bellwethereducation.org/sites/default/files/RSA-Teacher-Effectiveness.pdf Current Issues in Education Vol. 18 No. 1 14 Reichbach, A. M. (2004). Power behind the promise: Enforcing No Child Left Behind to improve education. The Boston College Law Review, 45(3), 667-704. Rubin, H. J., & Rubin, I. S. (2012). Qualitative interviewing: The art of hearing data (3rd ed.). Thousand Oaks, CA: Sage. Sergi, T. S. (2001). Praxis test recommendations. Retrieved from http://www.sde.ct.gov/sde/lib/sde/PDF/Cert/prax is/praxis_test_analysis.pdf Stiggins, R. J., & Bridgeford, N. J. (1985). Performance assessment for teacher development. Educational Evaluation and Policy Analysis, 7(1), 85-97. Strategic School Profiles. (2011-2012). Connecticut State Department of Education, CEDAR. Retrieved from http://sdeportal.ct.gov/Cedar/WEB/Researchand Reports/SSPReports.aspx Stronge, H. (Ed.). (2006). Evaluating teaching: A guide to current thinking and best practice (2nd ed.). Thousand Oaks, CA: Corwin Press. The Thomas B. Fordham Institute. (2002). No Child Left Behind: What will it take? Retrieved from http://edexcellence.net/publications/nclbwillittak e.html Trochim, W. M. K. (2005). Research methods: The concise knowledge base. Cincinnati, OH: Atomic Dog Publishing. van den Berg, R., Sleegers, P., Geijsel, F., & Vandenberghe, R. (2000). Implementation of an innovation: Meeting the concerns of teachers. Studies in Educational Evaluation, 26(4), 331- 350. doi: 10.1016/S0191-491X(00)00022-5 Vetter, A. (2012). Teachers as architects of transformation: The change process of an elementary-school teacher in a practitioner research group. Teacher Education Quarterly, 39(1), 27-49. Wade, S. E., Welch, M., & Jensen, J. B. (1994). Teacher receptivity to collaboration: Levels of interest, types of concern, and school characteristics as variables contributing to successful implementation. Journal of Educational and Psychological Consultation, 5(3), 177-209. doi: 10.1207/s1532768xjepc0503_1 Wagner, C., & Masden-Copas, P. (2002). An audit of the culture starts with two handy words. Journal of Staff Development, 23(3), 42-53. Wagner, T., Kegan, R., Lahey, L. L., Lemons, R. W., Garnier, J. Helsing, D., Howell, A., & Thurber Rasmussen, H. (2006). Change leadership: A practical guide to transforming schools. San Francisco, CA: Jossey Bass. Waugh, R. F., & Punch, K. F. (1987). Teacher receptivity to system wide change in the implementation stage. Review of Educational Research, 57(3), 237-254. doi: 10.3102/00346543057003237 Wise, A. E., Darling-Hammond, L., Tyson-Bernstein, H., & McLaughlin, M. W. (1984). Teacher evaluation: A study of effective practices (R- 3139-NIE). Santa Monica, CA: The RAND Corporation. http://www.sde.ct.gov/sde/lib/sde/PDF/Cert/praxis/praxis_test_analysis.pdf http://www.sde.ct.gov/sde/lib/sde/PDF/Cert/praxis/praxis_test_analysis.pdf http://sdeportal.ct.gov/Cedar/WEB/ResearchandReports/SSPReports.aspx http://sdeportal.ct.gov/Cedar/WEB/ResearchandReports/SSPReports.aspx http://edexcellence.net/publications/nclbwillittake.html http://edexcellence.net/publications/nclbwillittake.html Connecticut’s Educator Evaluation Process: The Concerns and Experiences of Educators in Two Pilot Districts 15 Appendix Connecticut’s New Teacher Evaluation Process: School Personnel’s Concerns and Opinions Survey © LaRocco 2012 Section I: Your concerns about the new teacher evaluation process The statements that follow are typical expressions of how school personnel might be feeling and or what they might be thinking as they take on a new practice such as the new teacher evaluation process. Read each statement. For each, please check the number the best reflects the degree to which the statement is true. 1, I am not concerned about the new teacher evaluation process. 1 2 3 4 5 6 7 Not True Very True 2. I would like to know more about the new teacher evaluation process. 1 2 3 4 5 6 7 Not True Very True 3. I want to know how the new teacher evaluation process will affect me. 1 2 3 4 5 6 7 Not True Very True 4. I am concerned with managing the new teacher evaluation process in practice. 1 2 3 4 5 6 7 Not True Very True 5. I want to know how the new teacher evaluation process will affect students. 1 2 3 4 5 6 7 Not True Very True 6. I have joined forces with colleagues so the new teacher evaluation process will have a collective impact on teaching and learning. 1 2 3 4 5 6 7 Not True Very True 7. I have ideas about modifications to the new teacher evaluation process that might make it work even better. 1 2 3 4 5 6 7 Not True Very True Current Issues in Education Vol. 18 No. 1 16 Section II: Your experiences with and opinions about teacher evaluation generally As you read each statement consider the entire evaluation process including planning for evaluation, observations, or other procedures and feedback. For each of the statements please check the number the best reflects the degree to which the statement is true. 1. My most recent evaluation experience was very high quality. 1 2 3 4 5 6 7 Not True Very True 2. My most recent evaluation experience had a strong impact leading to changes in my professional practice. 1 2 3 4 5 6 7 Not True Very True 3. I believe that a teacher's evaluation should include multiple indicators of student academic growth and development, including state standardized tests. 1 2 3 4 5 6 7 Not True Very True 4. I believe that a teacher's evaluation should include classroom observations. 1 2 3 4 5 6 7 Not True Very True 5 I believe that a teacher's evaluation should include whole-school student learning indicators. 1 2 3 4 5 6 7 Not True Very True 6. I believe that a teacher's evaluation should include parent feedback. 1 2 3 4 5 6 7 Not True Very True 7. I believe that a teacher's evaluation should include peer feedback. 1 2 3 4 5 6 7 Not True Very True 8. I believe that a teacher's evaluation should include student feedback. 1 2 3 4 5 6 7 Not True Very True Connecticut’s Educator Evaluation Process: The Concerns and Experiences of Educators in Two Pilot Districts 17 Section III: Level of commitment to the new teacher evaluation As you read each statement consider how committed you are to the new teacher evaluation process. For each of the statements please check the number the best reflects the degree to which the statement is true. 1. I am strongly in favor of the new teacher evaluation process. 7 6 5 4 3 2 1 Very True Not True 2. I am very enthusiastic about the new teacher evaluation process. 7 6 5 4 3 2 1 Very True Not True 3. I am committed to implementing the new teacher evaluation process. 7 6 5 4 3 2 1 Very True Not True 4. I believe it is feasible to implement the new teacher evaluation process. 7 6 5 4 3 2 1 Very True Not True 5. I believe it is important to implement the new teacher evaluation process. 7 6 5 4 3 2 1 Very True Not True Current Issues in Education Vol. 18 No. 1 18 Section IV: Information about you Please respond to each of the following items. 1. Highest level of education completed (Select one only.) Bachelor’s Degree Master’s Degree Sixth Year Degree Doctoral Degree Other: __________________________ 2. This is my first year working in this district. 5 Years or less 6-10 Years 3. Years of experience in education (Select one only.) 5 Years or less 6-10 Years 11-15 Years 16-20 Years 21-25 Years 26 Years or more 4. Location in which you currently work (Select one only.) Elementary School B1 Elementary School B2 Elementary School B3 High School B Elementary School A Intermediate School A High School A Other: __________________________ 5. Current position (Select one only.) Teacher Support Personnel (e.g., counselor, social worker) Administrator Other: __________________________ Connecticut’s Educator Evaluation Process: The Concerns and Experiences of Educators in Two Pilot Districts 19 Article Citation LaRocco, D. J., Wheeler, D. S., & Loveland Sennett, B. A. (2015). Connecticut’s educator evaluation process: The concerns and experiences of educators in two pilot districts. Current Issues in Education, 18(1). Retrieved from http://cie.asu.edu/ojs/index.php/cieatasu/article/view/1318 Author Notes Diana J. LaRocco, Ed.D. University of Hartford - College of Education, Nursing and Health Professions, Institute for Translational Research 200 Bloomfield Avenue, West Hartford, CT 06117 dlarocco@hartford.edu Dr. Diana LaRocco is faculty in the Doctoral Program in Educational Leadership at the University of Hartford. In addition to supervising doctoral-level research, she teaches research courses. Before coming to the University of Hartford, she was the Director of Research and Grants for the now Connecticut Department Developmental Services. She worked in the field of developmental disabilities for over 29 years and has extensive experience as a special education teacher and program administrator working with infants and toddlers with disabilities and their families in Connecticut's Birth to Three System. Dr. LaRocco directs the University-charted Center for Leadership in Education and Health within the Institute for Translational Research at the University of Hartford. She is also the University Coordinator of Assessment. Deborah S. Wheeler, Ed.D. Upper Dublin School District 1580 Fort Washington Avenue, Maple Glen, PA 19002 Deborah is the superintendent of schools for Upper Dublin School District in Pennsylvania. She has extensive experience as a special education teacher and administrator. Beth Ann Loveland Sennett, MA University of Hartford, College of Education, Nursing and Health Professions 200 Bloomfield Avenue, West Hartford, CT 06117 Beth Ann is a doctoral candidate and private educational consultant and tutor. She has extensive experience as a special education teacher and administrator. Manuscript received: 10/26/2013 Revisions received: 12/10/2014 Accepted: 01/19/2015 http://cie.asu.edu/ojs/index.php/cieatasu/article/view/1318 mailto:dlarocco@hartford.edu Current Issues in Education Vol. 18 No. 1 20 Volume 18, Number 1 April 20, 2015 ISSN 1099-839X Authors hold the copyright to articles published in Current Issues in Education. Requests to reprint CIE articles in other journals should be addressed to the author. Reprints should credit CIE as the original publisher and include the URL of the CIE publication. Permission is hereby granted to copy any article, provided CIE is credited and copies are not sold. Editorial Team Executive Editor Constantin Schreiber Assistant Executive Editor Anna Montana Cirell Niels Piepgrass Authentications Editors Tray J. Geiger Layout Editors Constantin Schreiber Copy Editors/Proofreaders Lucinda Watson Section Editors Earl Aguilera RikkyLynn Archibeque Evelyn Concepcion Baca Tray J. Geiger Darlene Michelle Gonzales Megan Hoelting Dani Kachorsky Laura Beth Kelly Tome Martinez Priyanka Parekh Kevin J. Raso Faculty Advisors Dr. Gustavo E. Fischman Dr. Jeanne M. Powers