Exploring candidate interaction with a high stakes teacher performance assessment: The KPTP and the Kansas laboratory Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 9/11/2019 Facebook: /EPAAA Revisions received: 2/11/2020 Twitter: @epaa_aape Accepted: 2/12/2020 education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 28 Number 138 September 21, 2020 ISSN 1068-2341 Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment: The KPTP and the Kansas Laboratory Connor K. Warner University of Utah United States Citation: Warner, C. K. (2020). Exploring candidate interaction with a high stakes teacher performance assessment: The KPTP and the Kansas laboratory. Education Policy Analysis Archives, 28(138). https://doi.org/10.14507/epaa.28.5037 Abstract: This qualitative study explores the experiences of a cohort of pre-service teachers completing a high-stakes teacher performance assessment (HSTPA), the Kansas Performance Teaching Portfolio (KPTP), during their final year of teacher preparation. The inquiry asks whether the act of completing the assessment modified candidate conceptualizations of good teaching, and, if so, in what ways. Data were gathered via in- depth interviews and content analysis, and data were analyzed via constant comparison. The study found that completing the KPTP was having some impact upon participant conceptions of good teaching, prodding them to broaden their understanding of the work of teachers to include not just dispositional and relational aspects of teaching, but elements of technical teaching practice. The article concludes with recommendations for policy, research, and practice. Keywords: performance assessment; teacher education; teacher education policy; teacher portfolio assessments; teaching; teacher licensure assessments http://epaa.asu.edu/ojs/ https://doi.org/10.14507/epaa.28.5037 Education Policy Analysis Archives Vol. 28 No. 138 2 Explorando la interacción del candidato con una evaluación de high stakes de desempeño docente: El KPTP y el laboratorio de Kansas Resumen: Este estudio cualitativo explora las experiencias de una cohorte de maestros en formación que completaron una evaluación de high stakes de desempeño docente (HSTPA), el Kansas Performance Teaching Portfolio (KPTP), durante su último año de preparación docente. La investigación pregunta si el acto de completar la evaluación modificó las conceptualizaciones de los candidatos sobre una buena enseñanza y, de ser así, de qué manera. Los datos se recopilaron mediante entrevistas en profundidad y análisis de contenido, y los datos se analizaron mediante una comparación constante. El estudio encontró que completar el KPTP estaba teniendo algún impacto en las concepciones de los participantes sobre la buena enseñanza, incitándolos a ampliar su comprensión del trabajo de los maestros para incluir no solo los aspectos disposicionales y relacionales de la enseñanza, sino también elementos de la práctica técnica de la enseñanza. El artículo concluye con recomendaciones de política, investigación y práctica. Palabras-clave: evaluación del desempeño; formación docente; política de formación docente; evaluaciones del portafolio de maestro s; enseñanza; evaluaciones de licenciatura de maestros Explorando a interação do candidato com uma avaliação de high stakes de desempenho do professor: O KPTP e o laboratório do Kansas Resumo: Este estudo qualitativo explora as experiências de uma coorte de professores em formação que completam uma avaliação de high stakes de desempenho do professor (HSTPA), o Kansas Performance Teaching Portfolio (KPTP), durante seu último ano de preparação de professores. A investigação pergunta se o ato de completar a avaliação modificou as conceituações do candidato de bom ensino e, em caso afirmativo, de que maneiras. Os dados foram coletados por meio de entrevistas em profundidade e análise de conteúdo, e os dados foram analisados por meio de comparação constante. O estudo descobriu que a conclusão do KPTP estava tendo algum impacto sobre as concep ções dos participantes de bom ensino, estimulando-os a ampliar sua compreensão do trabalho dos professores para incluir não apenas aspectos disposicionais e relacionais do ensino, mas elementos da prática de ensino técnico. O artigo conclui com recomendações para políticas, pesquisas e práticas. Palavras-chave: avaliação de desempenho; formação de professor; política de formação de professores; avaliações de portfólio de professores; ensino; avaliações de licenciamento de professores Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 3 Introduction In a famous dissenting opinion for a 1932 United States Supreme Court case, Justice Louis Brandeis argued It is one of the happy incidents of the federal system that a single courageous State may, if its citizens choose, serve as a laboratory; and try novel social and economic experiments without risk to the rest of the country. (Gardner, 1996, p. 475) This argument, drawn from Federalist #56 (Madison, 1788/1952) has become known as the states- as-laboratories metaphor, and has been interpreted to mean that the inherent sovereignty of state governments allows them to experiment with a variety of legal and policy solutions from which other states and the federal government can watch and benefit (Kronk Warner, 2015). Education policy in the United States has historically been representative of this metaphor, as the history of local and state control of education has created a multifaceted local and regional system of educational experimentation (Organisation for Economic Co-operation and Development, 2013). High-stakes teacher performance assessments (HSTPAs) likely exemplify the states-as-laboratories metaphor in action in the field of education. HSTPAs are a particular kind of summative, external performance assessment used by institutions or entire states to determine whether teacher candidates will be eligible for teacher licensure. The high-stakes part of the acronym is in reference to the fact that failure to pass the assessment will render a candidate ineligible for licensure, regardless of that candidate’s successful completion of other licensure requirements, such as university coursework or student teaching (Wilson et al., 2014). HSTPAs are increasingly being adopted across the United States, both at the institution and state levels, to serve as gatekeeping exams into the teaching profession (Knight et al., 2014). The most common of these, the educative Teacher Performance Assessment (edTPA) (Stanford Center for Assessment, Learning, and Equity, 2017), has been adopted by educator preparation programs (EPPs) in 40 states, with several states requiring that all teachers pass the exam to receive certification (American Association of Colleges for Teacher Education, n.d.). The rise of the edTPA has led to the creation and adoption of rival exams by both states and educational testing companies. As mandated HSTPAs have become the norm rather than the exception across the country, the teacher education professoriate has grappled to come to terms with them. The edTPA, at least partially due to its ubiquity, has proved particularly polarizing, with one camp arguing that the edTPA has the potential to bring badly needed coherence and rigor to teacher preparation and the other arguing that the edTPA decontextualizes the work of teaching and narrows the teacher preparation curriculum. While the debate rages on, this paper posits that an essential component of that debate ought to be a consideration of HSTPAs more broadly, particularly those that have been in consequential use far longer than the edTPA. One such long-running HSTPA is the Kansas Performance Teaching Portfolio (KPTP), which has served a gatekeeping role for teacher licensure in that state since 2008. As states and institutions across the country move rapidly toward adoption of some form of HSTPA to serve as gatekeepers to the teaching profession, it is concerning that, beyond the literature on the edTPA’s precursor, the Performance Assessment for California Teachers (PACT) and the nascent literature on the edTPA, there is little peer-reviewed research on other HSTPAs like the KPTP or the California Teaching Performance Assessment (CalTPA) that actually have a longer history of consequential use (California Commission on Teacher Credentialing, 2015; Kansas Division of Learning Services, 2015). To return to Brandeis’ metaphor, these states have been Education Policy Analysis Archives Vol. 28 No. 138 4 running assessment experiments in their laboratories, but the nation has not benefitted from the results. Knowledge generated in the Kansas laboratory—the state-specific policy solutions that the state of Kansas has developed—is particularly poorly disseminated. Specifically, literature studying the state of Kansas’ HSTPA, the KPTP, is scarce. Despite the fact that the KPTP has been in existence for almost two decades—it was first developed at the turn of the millennium and since 2008 has become a consequential component of teacher licensure—it is effectively absent from the scholarly literature on HSTPAs (Myers et al., 2016). At the time of writing, a search for KPTP on Google Scholar produced 495 possible hits. Unfortunately, not one was related to the Kansas Performance Teaching Portfolio. Follow-up searches with various combinations of “Kansas,” “Performance,” “Teaching,” and “Portfolio” produced similar results. A few articles mentioned the KPTP in passing as an example of a performance assessment (Glasgow et al., 2014; Greenberg & Walsh, 2012; Holdheide et al., 2010; Potemski et al., 2011) and one noted it as part of the context of teacher education in Kansas (Ellsworth et al., 2014). Though the Educational Testing Service (ETS) provided training and consulting during the development of the exam and the establishment of scoring procedures, the KPTP itself remains the property of the Kansas State Department of Education (KSDE), meaning that no outside entity has control over the content of the exam or of the candidate submissions (Myers et al., 2016). The KPTP is scored by teachers and teacher education faculty from institutions within the state of Kansas, who are trained by KSDE staff. Though certainly not the personal connection between evaluator and candidate that some critics of the edTPA have called for (Dover & Schultz, 2016; Greenblatt & O’Hara, 2015), scoring of the KPTP by Kansas teacher educators rather than by outside contractors or by teacher educators from other states, allows, at least in theory, for scorers to hold a deeper understanding of the context of education in the state and to be more responsive to contextual elements within candidate submissions (Nelson, 2016). Like the designers of the edTPA, the State of Kansas was concerned with limiting bias in the scoring process. Though KPTP scorers are drawn from the ranks of Kansas teachers and teacher educators, in order to increase objectivity and external accountability, scorers cannot evaluate submissions from their own teacher preparation institutions (Nelson, 2016). Lastly, the KPTP costs far less than either the edTPA, $300, or the second most common national performance assessment, the Praxis Performance Assessment for Teachers (PPAT), $275 (ETS, 2017). Several scholars have argued that the cost burden of nationally marketed HSTPAs will serve as a deterrent to many interested in entering the teaching profession, particularly individuals from lower socioeconomic and culturally and linguistically diverse backgrounds (Margolis & Doring, 2013; Greenblatt, 2018). Because the KPTP remains the intellectual property of the state, and KSDE provides administrative support for the exam, the only direct cost for candidates taking the KPTP is a scoring fee of $60 to $100, which is commonly rolled into their tuition and fees for student teaching (University of Kansas, 2016; Wichita State University, 2016). Given these unique characteristics, it seems likely that study of the KPTP can meaningfully add to the national conversation regarding HSTPAs. This study is an attempt to begin that work by examining the experiences of a cohort of secondary English teacher candidates navigating the completion of the KPTP during their final year of coursework in a teacher preparation program at a university in the state of Kansas (hereafter referred to as Center University). A major claim of those championing HSTPAs for gatekeeping of teacher licensure over other assessment formats is that the act of completing an HSTPA is, in and of itself, a powerful learning experience (Butler, 2015; Chung, 2008; Darling-Hammond & Hyler, 2013; Hollins, 2012; Stillman et al., 2015). In particular, supporters believe that HSTPAs will help articulate a “shared vision of competence for new Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 5 teachers” (Hollins, 2012, p.1) and help teacher candidates internalize that vision. To explore whether the KPTP can achieve those goals, this study was guided by the following questions: 1) Does the process of completing an HSTPA modify candidate conceptualizations of the work of teaching or what it means to be a good teacher? 2) If the process of completing the HSTPA does modify candidate conceptualizations, in what ways does it do so? Literature Review This study informs the scholarly knowledge-base within two strands of literature: literature relating to the design and implementation of HSTPAs, and literature exploring conceptualizations of good teaching and their importance to teaching practice and candidate learning. HSTPAs Advocates of HSTPAs have long called for the development and implementation of “a bar- like exam” to gatekeep entrance into the teaching profession (Mehta & Doctor, 2013, p. 8). These groups and individuals look to the implementation of such exams in the fields of medicine and law at the beginning of the 20th Century as central to the professionalization of those fields and the increased quality of practice that came with that professionalization (Darling-Hammond & Hyler, 2013). Most literature supporting the adoption of HSTPAs is predicated on the argument that rigorous gateway assessments can help to professionalize teaching by defining the essential competencies of good teaching. The jointly maintained American Association of Colleges for Teacher Education (AACTE) and edTPA (SCALE, 2016) website argued that the edTPA recommended passing score should serve as a bare minimum standard of competency agreed upon by teacher education professionals, but that individual states might choose to impose an even higher mandatory score for licensure. Butler (2015) maintained that the level of detail and rigor of completing the edTPA would help prospective teachers understand the depth of knowledge and skill required for successful teaching, and should increase the respect given to the teaching profession. Likewise, Hollins (2012) contended that the edTPA was a promising contribution to professional teaching because it was one of the first assessments to truly measure candidate ability to manage the complexities of teaching practice, not just their knowledge of teaching. Other proponents of HSPTAs have emphasized their potential for providing program improvement data to teacher preparation programs (Bastian et al., 2018; Darling-Hammond, 2010, Miller et al., 2015; Peck & McDonald, 2013; Peck et al., 2014) and for creating opportunities for faculty collaboration and intellectual engagement (Lachuk & Koellner, 2015; Peck et al., 2010; Warner et al., 2020). Lastly, Ahmed (2019) argued that the mandate of an HSTPA could be leveraged by pre-service teachers to resist scripted curricula, and, therefore, better serve the students in their field experiences. However, HSTPAs have also been questioned and criticized both by scholars and practitioners in teacher education. General concerns about HSTPAs include questions about their role in narrowing teacher education curricula (Dover & Schultz, 2016; Madeloni & Gorlewski, 2013), in redefining academic disciplines in ways that may not coincide with the disciplinary consensus (Hochstetler & McBee Orzulak, 2019; Warner, 2019), and in minimizing the importance of culturally responsive and critical multicultural education (Kleyn et al., 2015; Liu & Milman, 2013; Madeloni & Gorlewski, 2013; Picower & Marshall, 2016; Tuck & Gorlewski, 2016). As the most prevalent HSTPA, the edTPA has also critiqued for its connections to Pearson, a for profit corporation; for its significant cost to teacher candidates; for its use of external scorers; for its role in shifting the locus Education Policy Analysis Archives Vol. 28 No. 138 6 of educational control from the local or state levels to the national level (Greenblatt, 2018; Greenblatt & O’Hara, 2015; Margolis & Doring, 2013). The Shared Ancestry of HSTPAs While they differ in many details, HSTPAs, from the edTPA to the PPAT to the KPTP, share similar conceptualizations of the work of teaching, enshrined in the architecture of the assessments and derived from common ancestry. The family tree of today’s HSTPAs begins with the portfolio assessment component of National Board for Professional Teaching Standards (NBPTS) certification (Cochran-Smith & Power, 2010, Pecheone et al., 2013). The NBPTS was constituted in 1987 to both develop standards for accomplished teaching and to create an assessment and certification system that would allow for identification and recognition of accomplished teaching (Serafini, 2002). It began certifying teachers in 1994. The NBPTS portfolio consists of three sections: 1) Differentiation in instruction, which is composed of selected student work samples demonstrating learning growth over time, plus written discussion of instructional decision-making that helped facilitate that growth; 2) Teaching practice and learning environment, composed of video recordings of teaching accompanied by written analysis of teacher engagement with students and impact upon learning; and 3) Effective and reflective practitioner, requires gathering and analyzing student data and demonstrating the ability to use that data collaboratively with families and other stakeholders to impact student learning (NBPTS, 2017). Current HSTPAs tend to have similar architecture, requiring candidates to assemble evidence in the form of student work samples, videos of practice, and student data, then analyze and explicate that evidence by responding in writing to specific prompts. As shown in Table 1 below, generally, HSTPAs center upon similar competencies: planning instruction based upon knowledge of learners and learning environments, engaging in and explicating that instruction, and analyzing and reflecting upon the efficacy of that instruction based upon data and evidence (ETS, 2017; Missouri Department of Elementary and Secondary Education, 2017; Nelson, 2016; SCALE, 2016). Table 1 Shared Architecture of High-Stakes Teacher Performance Assessments by Task and Competency Competency edTPA Task PPAT Task KPTP Task Planning instruction based upon knowledge of learners and learning environments Task 1 Tasks 1 & 3 Tasks 1 & 2 Engaging in and explicating that instruction Task 2 Task 4 Task 3 Analyzing and reflecting upon the efficacy of that instruction based upon data and evidence Task 3 Task 2 Task 4 Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 7 In her analysis of the edTPA, Sato (2014) argued that teaching within the assessment is conceptualized as an executive process in which the teacher is responsible for selecting worthwhile learning goals for his or her students and then orchestrating learning activities for the group of students in an effort to support them toward achieving those goals. (p. 429) Requiring comparable tasks, processes, and skills, HSTPAs seem to conceptualize teaching in similar ways, as a technical process of observation, planning, enacting, assessing, and reflecting. While this is a fairly common way of defining teaching, it is not the only way to do so. Conceptualizations of Teaching Several scholars have explored the tension created by competing and seemingly contradictory definitions of teaching that have shaped educational discourse and practice (Chen et al., 2012; Jackson, 1986; Pillen et al., 2013; Wang et al., 2011, Warner, 2016). Lefstein (2005) provided a succinct overview of this tension in his analysis of conflicting visions of technical and personal teaching. In the former, teaching is conceptualized as a “a technical method, which can be ‘identified, disseminated and universally adopted’” (Lefstein, p. 334). In the latter, teaching is “viewed primarily as a relationship, which is constituted by how teachers and students interact holistically” (p. 347). Lefstein noted that the two sides of this dichotomy were frequently considered to be in conflict with each other, but believed that a more constructive approach would be to explore ways to address both conceptualizations. He argued, Purely technical teaching would lead to inflexible, unthinking, insensitive and amoral educational activity. An absolutist personal approach would deny teachers essential technical tools and important scientific insights, thereby confining them to the horizons of their own experience. Either vision, embraced exclusively and in its totality, becomes a dangerous hallucination. (Lefstein, p. 347) While Lefstein described ways that such a synthesis might occur, at present, much discourse about teaching is still firmly divided along ideological lines (Cochran -Smith et al., 2013; Sato, 2014; Lewis & Young, 2013; Wan g et al., 2010). At least part of the conflict over HSTPAs discussed in the previous section is rooted in this very ideological disagreement over the nature of teaching itself. If good teaching is primarily defined as a holistic development of personal and transformative relationships, then no series of standardized tasks is likely to be able to capture its complexity or measure its impact. That is, if good teaching is personal, then HSTPAs are not measuring good teaching. On the other hand, if good teaching is a technical process, then a standardized performance assessment is a perfectly reasonable way to measure teaching competence. If Lefstein is correct, however, and good teaching is a synthesis of the technical and the personal, then a truly valid assessment of teaching competence would need to capture that synthesis. Theoretical Framework Assessment is conceptualized as fulfilling multiple purposes within the scholarly literature and the field of education at large. Within the field of teacher education, assessment tends to be either conceptualized as assessment of learning or assessment for learning (DeLuca & Volante, 2016). Assessment of learning is the older, more traditional approach, in which assessment tasks are Education Policy Analysis Archives Vol. 28 No. 138 8 summative and evaluative, intended to allow the administrators of the assessment to measure the achievement of individuals at the end of a learning experience. Assessment for learning is a newer construct that envisions assessment as an ongoing process in which formative diagnostic feedback from assessments allows for the adjustment of instruction throughout the learning cycle. In such a conceptualization, the act of taking an assessment itself is understood to be a learning activity. Performance assessments, particularly the portfolio-based performance assessments upon which the HSPTAs were ultimately based, were initially conceptualized as formative experiences (Shulman, 1998). In fact, Shulman’s work on portfolio assessment at Stanford in the 1980s eventually formed the intellectual basis of the NBPTS portfolio and, from there, the underpinnings of today’s HSTPAs. Shulman (1998) argued that, that “whatever its effectiveness as an assessment form, the portfolio approach provided dynamite educational experiences” (p. 31). The claim of many proponents of HSTPAs is that they have the power to serve both as formative learning events and summative assessments of candidate mastery (Margolis & Doring, 2013; Meuwissen & Choppin, 2015; Peck et al., 2014). That is, proponents claim that HSTPAs are both assessments of learning and assessments for learning. As detailed earlier in this paper, HSTPAs are certainly being used in teacher education programs across the United States as summative assessments of learning. This study was not intended to interrogate the validity of using HSTPs in that manner, though recent studies trouble and complicate the use of HSTPAs and the edTPA in particular for making predictions of teaching effectiveness (Choppin & Meeuwissen, 2017; Gitomer et al., 2019; Goldhaber et al., 2017). Instead, this study was intended to interrogate the assertion that HSTPAs can be assessments for learning by interrogating the experiences of a cohort of preservice teachers as they completed an HSTPA to determine whether the process of completing the HSTPA was shaping their understanding of the work of teaching. Methods This study is based upon a larger research endeavor to understand the experiences of a group of teacher candidates navigating their final year of professional teacher education coursework. To explore my research questions regarding the potential of HSTPAs to modify candidate conceptualizations of the work of teaching, I designed a qualitative instrumental case study, where a “particular case is examined mainly to provide insight into an issue” (Stake, 2005, p. 445). Building such a picture required a mixture of qualitative methods including semi-structured interviewing and content analysis (Marshall & Rossman, 2006). As much as possible, the findings of this study have been reported in the words of my participants in order to faithfully portray their beliefs, feelings, and tensions. Participant responses should be interpreted not as objective reality, a construct that the paradigm informing this study rejects as essentially unknowable (Lincoln & Guba, 1985), but as accurate representations of reality as they perceive it, and which might be perceived quite differently by other stakeholders such as policymakers or teacher educators. Context This study took place within the English teacher education program in the School of Education at a university in Kansas, Center University (pseudonym). Primary focus was on the experiences of a cohort of teacher candidates who graduated in the Spring Semester of 2015. These 13 pre-service teachers were navigating their final year of coursework during the time of this study. Prior to study initiation, all participants had completed the majority of their general education and English content courses, as well as approximately 30 hours of Education coursework. At study Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 9 initiation, during the Fall Semester of 2014, all participants were enrolled in an advanced English methods course and an advanced teaching practicum. Upon successful completion of these courses, teacher candidates student-taught full-time in the Spring of 2015. Participants Participant selection for this study was based upon the concept of purposeful sampling (Lincoln & Guba, 1985). Since this study was not intended to produce contextless generalization, random sampling was not appropriate. Instead, sampling decisions were made in order to achieve maximum variation in information and perspective (Lincoln & Guba, 1985). First, I was particularly interested in the constructions and experiences of secondary pre- service teachers. The experience of secondary teachers is unique because they are often thought to be content experts first and teachers second; this impacts their understandings of what it means to be a good teacher and their sense of who they are as teachers (Cooper & He, 2012). Second, I focused on secondary English pre-service teachers because in addition to the tension discussed with all secondary teachers above, the experiences of English teachers are unique in that they “are often seen as responsible for the affective and attitudinal education of secondary students in ways that other content area teachers are not” (Hochstetler, 2011, p. 259). This emphasis on emotional work may have a significant impact on pre-service teachers’ understanding of what good English teaching looks like. Therefore, the initial participants for this study were the 13 students who made up the majority of the cohort of pre-service English teachers at Center University in the 2014-2015 academic year. The cohort was relatively homogenous, with most students having grown up and attended school in the Midwest. Twelve of the students were female, and only one was male. All were undergraduates, and most were traditional college students, in their fourth year out of high school at the time of the study. Researcher’s Role As a methods instructor, field experience supervisor, and researcher, I assumed many roles in the context of this study; this presented both advantages and limitations (Yin, 1994). I was the instructor for the methods course that all of these students took during the fall semester of 2014, and I was also the university supervisor for 4 of the 13 participants in their advanced teaching practicum. During the Spring of 2015 I was the university supervisor for nine of these teacher candidates while they student taught. Being so deeply embedded in the context under study offered the kind of “firsthand involvement” and full “immersion” that are essential elements of qualitative research (Marshall & Rossman, 2006, p. 100). Throughout the course of the study, I interacted with these participants on an almost daily basis from August 2014 through May 2015, giving me the “prolonged engagement” (p. 301) that Lincoln & Guba (1985) argued was at the heart of rigorous and trustworthy qualitative research. However, there were some drawbacks to my position as well. Several of my roles required me to evaluate my participants, which established an inherent hierarchy of power between us, and which held the potential to influence the ways in which participants responded to my inquiries. I attempted to mitigate these effects in a variety of ways. First, as with any ethical research, my participants were recruited voluntarily through a process of informed consent, and no member of the cohort was pressured to participate if he or she preferred not to do so. Second, the vast majority of the work that my participants did on the KPTP actually occurred within their spring seminar, a course with which I was not involved in any way. Though I was supervising some of these participants during student teaching, I was not the instructor of record for student teaching, as a Education Policy Analysis Archives Vol. 28 No. 138 10 clinical faculty member recorded final student teaching grades for based upon input from the university supervisor and the clinical supervisor of each student teacher. Lastly, my study focused on pre-service interactions with the KPTP, an instrument that was designed and scored not by me, but by readers outside of my institution. Clearly communicating these facts to the participants as part of the informed consent process helped minimize the impact of my status as an evaluator upon their responses to my inquiry. Data Sources I utilized two major data sources for this study. The first, and primary data sources were group and individual semi-structured interviews, which occurred at specific points throughout participants’ interactions with the KPTP. I drew additional data from documents collected within the context of the teacher education program at Center University, with the most important of these being completed participant KPTP tasks. The KPTP is composed of four tasks 1) Contextual Information and Learning Environment Factors, 2) Designing Instruction, 3) Teaching and Learning, and 4) Reflection and Professionalism. For task 1, candidates gather demographic, personal, and academic information about the students within their student teaching placements, and use that information to develop profiles of an entire class, a subgroup of students within that class, and two individual focus students. In KPTP task 2, candidates use draw upon the profiles developed in task 1 to design a unit of instruction for their students, including discussion of differentiation in instruction and assessment for the subgroup identified in task 1, and for the individual focus students. For task 3, candidates are intended to teach the unit they designed in task 2, keep a daily log of observations about their instruction, compile and analyze assessment results from the unit, video themselves teaching several of the lessons in the unit, watch those videos, and critically evaluate their own instruction. They are then supposed to use those data sources in task 4 to help them reflect on the efficacy of their instructional practice and to plan future professional development. Data Collection Data collection occurred in four phases over the course of one academic year. Each phase involved interviews and content analysis of participant-generated documents. The first phase, which occurred prior to candidates’ introduction to the KPTP was intended to help me understand how participants initially thought about teaching and teachers. The second phase of the inquiry was timed to capture tensions and responses in participants’ initial interactions with the KPTP at the beginning of their student teaching semester. During the third phase, participants were asked to reflect back on their responses in the previous two phases, and to summarize their experience completing the KPTP. Finally, the fourth phase was designed as a formal member check. Phase 1 During Phase 1, which took place during the fall semester of 2014, I sought to gain information about my participants’ constructions of good teaching and teachers. The data sources in this phase were collected prior to participant exposure or interaction with the KPTP, which provided me with a baseline for comparison as I moved into the later phases of the inquiry. The information I gathered in this phase helped me to understand the issues and questions that I should explore in the following phases. Participants were asked to respond to the essay prompt, “From your perspective, what is good English teaching?” I collected and read these essays, performing a content analysis on them. The content analysis involved unitizing data from the essays and then coding each unit that made any sort of statement about good teaching with a descriptive code (Skrtic, 1985). This was the beginning of the codebook used throughout the project. Based upon this Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 11 content analysis, I divided the cohort into four focus groups made up of participants who shared similar initial constructions of good teaching. Next I interviewed each group to gain further information and elaboration about the constructions of good teaching that these participants held. These interviews lasted approximately one hour per group, and involved questions like, Tell me about the best teaching you think you’ve done, and highlight what specifically within that example qualifies it as good teaching and Tell me about the best teacher(s) you’ve had or seen. What were these teachers doing specifically that made them good teachers? I transcribed these interviews and then unitized and coded the data, using the codes developed in the first part of Phase 1 and adding new codes as necessary. Table 2 in the Findings section of this paper provides the complete code book from Phase 1 of the project, and the number of data units for each code. Phase 2 Phase 2, which took place at the beginning of spring semester of 2015, involved analysis of participants’ initial interactions with the KPTP. During this phase, I performed a content analysis of participant KPTP tasks 1 and 2 looking for elements relating to participant constructions of good teaching. I then compared the constructions emerging from tasks 1 and 2 with the initial constructions established in phase 1 to determine whether changes in construction were occurring. Finally, I interviewed participants individually using tasks 1 and 2 as stimulus documents to gain further information and elaboration about emerging changes or tensions. Individual interviews lasted approximately 45 minutes, and included questions like In our previous interview, you described good teaching as relevant and caring. Do you feel like this view of good teaching is reflected in your KPTP Tasks 1 & 2? If so, can you show me where? and If you were a stranger reading your KPTP tasks, what kind of teacher would you imagine had written them, and can you show me what parts of your KPTP specifically would lead you to think that? Phase 3 Phase 3 took place toward the end of the spring 2015 semester, as participants finished their KPTP submissions. First I performed a content analysis of KPTP tasks 3 and 4 and compared constructions of good teaching emerging from these documents with those established in phases 1 and 2, to determine whether changes in construction were occurring. Then I interviewed participants individually using tasks 3 and 4 as stimulus documents to gain further information or elaboration about these changes and tensions. Individual interviews lasted approximately 45 minutes, and included questions like Where in your KPTP is your best teaching reflected? and Did the need to do the KPTP change how you taught in your field placement? Phase 4 Phase 4 took place at the very end of spring semester 2015. At this point, I compiled a case report, which I then provided to a member check group drawn from the ranks of the original participants, an essential component for the trustworthiness of my study (Erlandson et al., 1993; Lincoln & Guba, 1985; Skrtic, 1985). Five of the thirteen participants elected to participate in the member check. After these participants had read the report, they were brought together to discuss whether the constructions represented in the case report matched their own constructions of the situation. Following the member check, I then revised the case report based upon the data gathered during the member check itself. Data Analysis Data collection occurred in four phases over the course of one academic year. Each phase involved interviews and content analysis of participant-generated documents. The first phase, which Education Policy Analysis Archives Vol. 28 No. 138 12 occurred prior to candidates’ introduction to the KPTP was intended to help me understand how participants initially thought about teaching and teachers. Inductive Analysis Initial data analysis for this study was inductive rather than deductive (Lincoln & Guba, 1985). My primary analysis processes were constant comparison and open-coding, allowing themes to emerge from the data (Glaser & Strauss, 1967). This process allowed me to delve into the many interconnected layers of participant constructions of good teaching and understandings of the KPTP without initially imposing pre-determined categories. Deductive Analysis Once themes and categories were established through open coding in the initial inductive analysis, those themes were then analyzed deductively through the lenses of scholarship on definitions of good teaching (Patton, 2002). In particular, as participant constructions of good teaching became increasingly complex in the later phases of inquiry, the multifaceted framework of good teaching exemplified by the InTASC standards (Council of Chief State School Officers, 2013) became a useful tool for organizing and interrogating participant constructions of good teaching. Since the InTASC framework forms the construction of good teaching that guided development of the KPTP, comparing participant articulations of good teaching with categories of the InTASC framework helped to determine the degree to which the process of completing the KPTP shaped participant constructions of good teaching. Trustworthiness In order to ensure that my research was both rigorous and ethical, this study followed the guidelines for meeting trustworthiness criteria established by Lincoln & Guba (1985): credibility, transferability, dependability, and confirmability. Credibility refers to the believability of the study, which is demonstrated via persistent observation, triangulation, and in-process and summative member checks. Persistent observation means “identify[ing] those characteristics and elements in the situation that are most relevant to the problem or issue being pursued and focusing on them in detail” (Lincoln & Guba, 1985, p. 304). Constant comparison and emergent design allowed me to meet this criterion. I triangulated my data collection by collecting at a variety of points in time (the four phases previously described) and by collecting through various methods and sources (group interviews, individual interviews, documents). Member checks were built into the study throughout, and, as detailed above, the final phase of the study was formal summative member check. Transferability refers to whether or not the readers of a study are able to determine the degree to which the results are applicable to their own contexts. Purposeful sampling, as described earlier in this section, and detailed description allow readers to make transferability judgements. Dependability denotes the consistency by which a researcher follows established research practices within the naturalistic paradigm. This can be demonstrated by the keeping of complete records of research procedures, referred to as a dependability audit trail and the review of those records by an external party. Lastly, confirmability signifies that the interpretations made by the researcher can be linked directly to the raw data collected in the study. Like dependability, this is best demonstrated by a review of interpretation and data by an external party, a process referred to as a confirmability audit. To meet the requirements of dependability and confirmability audits, I employed the peer debriefer audit recommended by Erlandson et al. (1993). A peer debriefer is someone who is of the same academic rank and background, with whom the researcher can meet periodically throughout Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 13 the research process to discuss emerging analysis and design, and who can give constructive criticism at a variety of points in the process, offering alternate perspectives on data and serving as a “devil’s advocate” (Lincoln & Guba, 1985, p. 309). I recruited a peer debriefer, of similar academic rank, familiar with the teacher preparation program at Center University, but not with the particular participants in this study, and who had training in qualitative research generally and naturalistic inquiry specifically. We met on a monthly basis to discuss my research process, analysis, and decision-making, and both of us kept “written records of each encounter, partly for the sake of the audit trail…and partly for reference by [me] as [I] later [sought] to establish just why the inquiry emerged as it did” (Lincoln & Guba, 1985, p. 309). This continuous record of reflection and decision helped to ensure that each step of the emerging design was documented and rationalized, and allowed my peer debriefer to audit and question my interpretations. Findings The following section details the changes in participant conceptualizations of good teaching, both because and in spite of their experiences with the KPTP. Confirming what Chong (2011) argued, participants were entering their field experiences “with clear images of what teaching entails and how they [saw] themselves as teachers” (p. 220). However, this inquiry revealed that completing the KPTP was having some impact upon participant conceptions of good teaching, prodding them to broaden their understanding of the work of teachers to include not just dispositional and relational aspects of teaching, but elements of technical teaching practice as well. While early participant responses centered on the dispositional and emotional work of teaching, teacher knowledge of students, and the ability to use that knowledge to motivate and to form trusting relationships, as the pre-service teachers moved through the process of completing their KPTPs, their descriptions of good teaching became more complex. As one participant described, the KPTP influenced her by making transparent “how much work it takes to do it [teaching] and every little thing that goes into teaching and… It made me reflect on how I do all [these different things], but I didn't realize that I did [them]. That was kind of cool.” Her comment indicates that through the process of completing the KPTP, she was able to view teaching through a different lens than she had before, a lens that was more analytic than intuitive, more atomistic than holistic. To a greater or lesser degree, almost all participants experienced a similar shift in awareness, with corresponding implications for their conceptions of good teaching. In the initial phase of this inquiry, participants had been confident in their abilities to distill good teaching into a relatively small number of dispositional traits, like being playful and caring, and interpersonal skills, like empathizing and nurturing. By the later phases, however, good teaching seemed much more complex to them. As one participant said, “I think there are a lot of things that go into being a good teacher, and you kind of have to have all of them [to be a good teacher].” The sections that follow are organized based around the common themes that demonstrate the evolution of participant understandings of good teaching throughout the study. The first theme, personal teaching, dominated participant descriptions of good teaching in the early phases of the inquiry. As the study progressed, participants increasingly emphasized the second theme, technical teaching, in their documents and interview responses. The third theme, flexibility and adaptability, represents the understanding of good teaching that a few participants seemed to have constructed by the end of the study in order to reconcile their dedication to personal teaching with their increasing appreciation for technical teaching. Within the sections on each theme, salient quotes from participants are included to illustrate particular elements of the themes. Each quote was selected as representative of ideas voiced by multiple participants. Education Policy Analysis Archives Vol. 28 No. 138 14 Personal Teaching Relevant, fun, and caring—these are the words that participants used most often to describe good teaching, forming the first emergent theme in this inquiry, that of good teaching as personal teaching. At the beginning of the study, prior to interaction with the KPTP, as a whole, the cohort seemed to agree that the primary characteristics of good teaching were engaging students in fun and interesting experiences, establishing relevance of content and activities, and developing personal connections and relationships with students. Table 2 below details all codes relating to definitions of good teaching from Phase 1, the number of data units relating to each code, and the number of distinct participants whose construction of good teaching included material labeled with that code. Table 2 Participant Initial Constructions of Good Teaching by Number of Units and Participant Descriptive Code Technical/ Personal/ Either # of Data Units # of Participants Establish relevance Personal 28 12 Make learning fun/engaging Personal 24 9 Personal relationships Personal 21 9 Passion about subject/teaching Personal 10 4 Connecting to other disciplines Technical 9 6 Promote student agency Personal 9 7 Differentiation Either 7 8 Cultural responsivity Personal 7 5 Belief in/validation of all students Personal 6 4 High expectations Personal 6 5 Collaborative learning Either 4 4 Content expertise Technical 4 5 Critical thinking Either 3 3 Safe learning environment Either 3 4 Active learning Either 2 2 Encouraging creativity Either 2 3 Self-discovery Personal 2 2 Involving all students Either 2 2 Knowledge transfer Technical 2 2 Dialogue between student and teacher Either 2 2 Assessment cycle Technical 1 1 Levels of questioning Technical 1 2 Use of technology Technical 1 1 Material selection Technical 1 1 Discipline/authority Technical 1 1 Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 15 As Table 2 makes clear, the three top codes, all key elements of personal teaching, were each between two and three times more common than the most frequent code relating to technical teaching. All but one of the participants described establishing relevance as an essential element of good teaching at some point in Phase 1, versus only half that number for the top technical teaching code. Early on, one participant explained that good teaching is “about making sure your students are engaged in the lessons, and that you choose materials that will interest your students….In my [best] lesson [what made it good] was that everyone got out of their seats and everyone participated and was involved, mostly because it was fun.” Another participant praised one of her former teachers who “used to wear like hats for different subjects…like, this is going to require critical thinking, so I'm going to put on my hard hat…it’s like kinda cool. You know, like, teachers wear many hats…it was fun.” The majority of participants at some point identified fun as an essential quality of good teaching, particularly at the beginning of the inquiry. Along with fun, relevance was an important quality of good teaching for these participants. According to one participant, “An effective English teacher makes the literature and literacies relevant and applicable to students' past, present, and future lives.” Another agreed, stating “it is imperative that English teachers make reading and writing relevant to students.” One of the first important lessons that one participant felt she had learned in her pre-student teaching field experiences was that it was necessary “to try and get my students thinking about how issues in the novel might match up to issues in the world we live in today. Once my students were able to see how the novel could still connect to their own lives, they became more interested in hearing what I had to teach them.” Participants argued that learning could not happen unless students were able to connect to the material, and that good teachers knew their students well enough to help them make those connections. In order to know students well enough to facilitate connections, these participants believed that good teachers must truly care about their students. Throughout the study, participants used the words care and connections to describe the best teachers they had observed and the best teaching they personally had done. In her Phase 1 interview, one participant told the following story to illustrate this quality: My homeroom teacher, I was a competitive dancer, and we were somehow in the same city over a weekend. She had like a wedding and I had a competition, and she took time out of her weekend to come and watch me, and she took pictures and put them on the board…It was really cool. Another participant at the beginning of the study simply stated that the most important part of a “teacher's job [is] to make meaningful connections with students.” A third participant indicated that the best teacher she had ever observed “was always going out of her way to make sure that there was a personal connection” between her and her students. Across the board, these participants argued that good teachers were able ensure that students, in the words of one participant, did not feel like “little boxes to be checked.” Technical Teaching As the study progressed, a second theme, technical teaching, emerged from participant discussions of good teaching. In particular, participants began to emphasize the importance of backwards design, detailed planning, and classroom assessment. Table 3 below details all codes relating to definitions of good teaching from unitized data gathered in Phases 2 and 3, the number Education Policy Analysis Archives Vol. 28 No. 138 16 of data units relating to each code, and the number of distinct participants whose construction of good teaching included material labeled with that code. Table 3 Participant Constructions of Good Teaching at End of Phase 3 by Number of Units and Participant Descriptive Code Technical/ Personal/ Either # of Data Units # of Participants Make learning fun/engaging Personal 111 12 Assessment cycle Technical 40 12 Detailed planning Technical 37 10 Establish objectives/expectations Technical 27 9 Differentiation Either 25 6 Purposeful grouping Technical 24 8 Dialogue between student and teacher Either 24 6 High expectations Personal 20 5 Scaffolding Technical 18 7 Establish relevance Personal 18 8 Safe learning environment Either 15 9 Personal relationships Personal 15 6 Time on task Technical 14 5 Promote student agency Personal 12 5 Learning strategy instruction Technical 8 4 Cultural responsivity Personal 7 4 Competition Either 4 2 Activity sequencing Technical 4 3 Multiple modalities Either 3 3 Graphic organizers Either 3 2 Involving all students Either 2 2 Test preparation Technical 2 1 Discipline/authority Technical 1 1 Self-discovery Personal 1 1 Clarity Technical 1 1 Coverage Technical 1 1 Content expertise Technical 1 1 Note: Top three codes by frequency from Phase 1 of the study are represented with bold-face to highlight shift in relative emphasis over the course of the inquiry. While Table 3 makes clear that participants continued to believe that good teaching had to be fun and engaging, with data units coded to that description being almost three times more common than the next most frequent code, the other two elements of personal teaching emphasized in Phase 1 shifted in relative emphasis. In contrast, the assessment cycle, only mentioned by a single Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 17 participant in Phase 1, was the second most frequent code by Phase 3, with all but one participant indicating that a cycle of assessment was essential to good teaching. Backwards Design In Phases 2 and 3, participants focused heavily upon a cycle of establishing objectives, creating assessments to measure those objectives, planning instruction to teach those objectives, delivering that instruction, administering the assessments, and then determining the next set of objectives based upon the data gathered from the assessment, very similar to the process of instructional design or backwards design described by Wiggins & McTighe (2005). In Phase 1, however, only a single participant emphasized the importance of assessment within her description of good teaching. She described participating in a unit in her practicum placement, the field experience that pre-service teachers at Center University complete during the semester prior to student teaching, where she and her cooperating teacher analyzed student writing samples at the beginning of the unit to determine specific communication needs and then “created writing, reading, and speaking assignments that could augment what they needed to work on.” Task 2 of the KPTP, however, requires pre-service teachers to respond to prompts and design their lessons within a framework of pre-assessment, instruction, and post-assessment. One participant explained this cycle in the following way: “you've got to start with where you want them to end up. Which is another way of thinking about how everything you do should be working toward the product or assessment you want them to be able to perform on at the end of the unit.” A number of participants indicated that they had not previously conceptualized the work of teaching in this cyclic way. Another participant said the KPTP helped her to plan a whole unit and really do a lot of backwards design, considering standards and the needs of your students…it just really makes you think through very thoroughly all of those little details that should be considered in a unit with your students in the content. A third participant noted that the KPTP “made you lay out the objectives; made you lay out the pre- assessment; made you lay out everything bit by bit by bit,” and that this was a more systematic process of thinking about instruction than she would probably have considered prior to completing the KPTP. A different participant explained that the KPTP reinforced for her the importance of “always matching up the objectives to the standards to that summative assessment and then building everything in between…[because students] will get something from [the lesson] if we are not [planning instruction that way], but they will get so much more from it if we are really following that formula.” Another participant indicated that the KPTP “has shown me that in order to plan, whether I am doing a 50 page plan for a unit or not, in order to plan effectively and teach effectively you do have to put a lot of thought into it” before getting up in front of the class or even selecting the activities that will be enacted. Detailed Planning The KPTP definitely played a role in developing participant understanding of and belief in the importance of detailed instructional planning. When asked where in the KPTP they were able to demonstrate good teaching, participants almost invariably pointed to the lesson plans themselves as providing the best opportunity within the KPTP to demonstrate what they considered good teaching. One participant noted that the lesson plans were “probably the one part where you could see that I know kind of what I'm doing” and that “Being organized and planned” was essential to good teaching. Another agreed, saying, Education Policy Analysis Archives Vol. 28 No. 138 18 I really did like Task 2, the lesson planning, where you had to lay it all out and…what standards applied to it….I think that definitely shows good teaching, as far as keeping your formative assessments, summative assessments for every single lesson that you do tied to your standard. A third participant echoed these thoughts, noting that the KPTP really helped her to see the importance of “making sure that [the lesson] is tied to standards from which you derive main objectives” and then “planning with the assessment in mind…[by] thinking of the end of your lesson or unit before you plan anything, [and asking yourself] ‘what do you want the outcome [of the lesson] to be?’” These comments indicate a growing awareness of a conception good teaching discussed in the literature as organized, sequenced, and systematic (e.g. Hunter & Russell, 1981). Assessment Participants also emphasized the KPTP’s role in helping them explore assessment as a larger process including pre-assessments, formative assessments, and summative assessments. One participant noted that the KPTP “got me thinking about assessment more just because that was so prominent in the [KPTP]…I already formatively assess[ed] a lot in my classroom, but pre-assessing, I really didn't think about it until the [KPTP]….I will definitely pre-assess…more [in the future] and continue formative assessment and summative assessment.” Another agreed, saying “I am a little more cognizant of being intentional with assessment…because it is so emphasized in the [KPTP]…it sort of reaffirmed that you have to give multiple assessments, which you work from [and] learn from. [You need to] view them as assessments of your own teaching and [as well as] what kids are learning.” A third participant came to realize the importance of planning and establishing “clear expectations [for assessments] because when I didn't have those you could tell, and the kids were like, ‘I don't know what I'm doing.’ So, clear expectations—having a rubric or telling them what you are going to be grading something with” is an important part of good teaching. Flexibility and Adaptability The final theme of flexibility and adaptability emerged as a means for a few participants to reconcile their conviction in the importance of the first theme, personal teaching, with their growing awareness of the significance of the second theme, technical teaching. Good teaching, then, according to these participants, meant not just having a solid plan of instruction and assessment, but possessing the skills and habits of mind necessary to change that plan in real time depending upon student responses and needs. Toward the end of the study, one noted that good teaching was “being able to adapt, being flexible.” Another agreed, saying, “[Good teaching is] so much improvisation, really. You never know what students are going to say, or what direction you go. I think being able to think on your feet is definitely important. That was one of the components of good teaching. I think the ability to do that and to be flexible is important.” The two skills of writing concrete plans and of adapting instruction “on the fly” might seem to be unrelated, but for these participants, there was a definite connection. One participant explained this connection, saying “I took spring break to plan through the end of my time. I think doing that helped me adjust what I needed change on the fly because of assemblies and things like that. I was able to get rid of this and do this instead. Instead of scrambling last minute.” Another argued “the necessity of being flexible in instruction. I think that when I am putting it down on paper, [I] really do see how much things [may] need to change [depending on student need].” Later in the interview she reiterated that good teaching really did require “that flexibility—having a really specific plan, but having that specific and structured plan for the purpose of being able to change it Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 19 in live time if you need to [for your students].” In this sense, flexibility and adaptability seemed to be key attributes that allowed participants to form a bridge between their initial conceptions of good teaching as relevant, fun, and caring, and their new appreciation for the technical skills of planning instruction and assessment that are so prevalent within the KPTP. Discussion This section highlights the shift in participant conceptualization of teaching as a result of interaction with the KPTP, which I argue contributes and advances the dialogue around the role of HSTPAs in teacher education. During the first phase of the study, participants had described good teaching as primarily an interpersonal activity, highlighting the development of teacher-student relationships and teacher knowledge of students. These more technical elements of teaching, while missing from Phase 1 discussions of good teaching, rivaled the more personal elements of relevance, fun, and care in Phase 2 and 3 discussions. It is important to note that the only shared experiences that all participants had between Phase 1 and Phases 2 and 3 were beginning student teaching and completing their KPTPs. They had finished their university coursework by the end of Phase 1, so they encountered no new emphasis on technical teaching or backward design there. Working just with their mentor teachers during student teaching also cannot explain this observed shift in participant emphasis, as the planning processes of mentor teachers varied significantly from participant to participant. Some mentor teachers developed detailed lesson plans with associated assessments, while others just moved methodically through district-adopted textbooks and pre-packaged curricula. Still others were satisfied with only a few notes about topic and activity jotted on a weekly calendar. Given such wide variation in student teaching experience, it seems likely that the common involvement with the KPTP was responsible for shifting participant recognition of elements like backward design that are essential to successful KPTP completion. On the whole, the experience of completing the KPTP influenced this study’s participants to shift the criterion by which they judged the efficacy of their practice. Prior to undertaking the KPTP, participants judged the efficacy of teaching by the quality of the relationships produced within the practice—relationships between the teacher and the student and between the students and the content. Care was important because it positively impacted the relationship between student and teacher. Relevance was important because it positively impacted the relationship between the student and the subject matter. Fun was important because it positively impacted both relationships. The relationships were themselves important because of their potential to spark transformative experiences (Jackson, 1986). Working with the KPTP, however, shifted the criterion that participants referenced when judging good teaching, from transformative relationships to direct gains in student learning. These results suggest that supporters of HSTPAs may be correct that the act of completing such an assessment can indeed serve as a learning experience to help candidates internalize a common vision of teaching (Butler, 2015; Darling-Hammond & Hyler, 2013; Hollins, 2012; Stillman et al., 2015). The results of this study also suggest that critics of HSPTAs may be correct—the act of completing high-stakes, task-based assessments centered upon analyzing the planning, assessing, and enacting of learning experiences may be encouraging a narrowing of candidate focus from the personal and relational to more technical, process-product approaches (Dover & Schultz, 2016; Madeloni & Gorlewski, 2013). The candidates in this study, all of whom to a greater or lesser extent began the study viewing good teaching as firmly rooted in Lefstein’s (2005) personal perspective Education Policy Analysis Archives Vol. 28 No. 138 20 came to appreciate the opposing technical perspective. They credited the act of completing the KPTP as forcing them to undertake that intellectual work. This is not to say that any participants indicated that they enjoyed completing the KPTP. In general, they found the process onerous, and the task-based writing repetitive. The many separate prompts stymied their abilities to craft cohesive narratives, which left them feeling that their final products did not completely capture the nuances of their teaching practice. Despite developing a new appreciation for the technical aspects of teaching, they expressed frustration that the personal aspects they still saw as so very important were not allocated at least equal space within the mandated assessment structure. Only a few participants seemed to synthesize the personal teaching they brought to the KPTP with the technical teaching enshrined within the KPTP to form a new understanding of good teaching as flexible and adaptable. Limitations Since it was conceptualized as a naturalistic inquiry into the experiences of particular participants in a particular context, the study was never intended to be generalizable—though insights from the study may be transferable to other contexts (Skrtic, 1985). Results of this study should not be interpreted as being universally representative of the experiences of all pre-service teachers completing high-stakes teacher portfolio assessments. Rather, this study was intended to provide readers with access to the experiences of these 13 pre-service teachers at Center University as they attempted to navigate their final year of teacher preparation and the concurrent HSTPA. The participants in this study were all prospective English teachers, a discipline that previous scholars have recognized as being particularly concerned with affective dimensions of teaching (Hochstetler, 2011). The experiences of teacher candidates from other certification areas might very well be different. I invite readers to reflect critically and consider whether the findings of this study may provide insight into their own contexts. Implications for Policy State-specific HSTPAs are inherently more contextually responsive than their nationally- standardized counterparts. While the task architecture of the KPTP, for example, is similar to that of national HSTPAs, as described earlier in this paper, the procedures and processes for implementing and scoring it are determined by the Kansas Division of Learning Services rather than by an external testing company like ETS or Pearson. The state collaborates with university-based teacher licensure programs to recruit scorers familiar with the strengths and challenges of teaching in the different regions of Kansas. It sets a cut score based upon a target of proficiency negotiated with educational stakeholders across the state. Most importantly, the KPTP includes a process for remediation for those candidates who fail to meet the required level of proficiency (Nelson, 2016). Using the scores and justification statements provided by the KPTP scorers, teacher education faculty are allowed to meet with candidates whose submissions fail to meet the cut scores and establish a remediation plan for them. This plan can include the assignation of additional learning experiences prior to the rewriting of the previous KPTP or the development of an entirely new KPTP submission. The revised or new submission is then scored by trained KPTP scorers (Nelson, 2016). While nationally- normed HSTPAs allow resubmission of tasks, candidates are only provided with numerical scores, not commentary, and resubmission requires the payment of a significant fee. Policymakers interested in mandating HSTPAs for teacher licensure may consider the results of this study when deciding whether to adopt a national assessment like the edTPA or PPAT or to develop a state or local exam like the KPTP. If the primary motivation of such policymakers is to ensure that a unified vision of competent teaching practice is codified and understood by teacher Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 21 candidates prior to their reception of licensure, then this study suggests that a cheaper and more contextually responsive HSTPA may fulfil that purpose. Implications for Practice This study suggests that completing an HSTPA is a learning experience in and of itself. That learning experience centers on the understanding of good teaching upon which the HSTPA is based. Teacher educators may wish to consider the degree of compatibility between the view of good teaching that drives their particular educator preparation programs and the view of good teaching guiding any HSTPA that their candidates may be required to complete, in order to either leverage or mitigate that learning experience. Implications for Research Participant critiques of the instrument and the process of completing the assessment raise additional questions about whether the learning that candidates achieved was worth the time and frustration of completing the assessment, and whether that learning will result in lasting changes to teaching practice. Additionally, this study raises questions about what kind of identity tensions may have resulted for candidates as they undertook the heavy intellectual work of modifying deeply held conceptualizations of teaching during the process of completing a high-stakes gateway exam, the results of which would determine whether they were able to enter the profession at all. Conclusion Both supporters and critics of HSTPAs have focused on their potential to reify the conceptualization of teaching held by teacher candidates. Supporters see this potential as an advantage, arguing that a wide-spread unified vision of teaching is an essential element in the professionalization of the teaching workforce and the improvement of education nationwide. Critics also see HSTPAs as vehicles for defining teaching, but worry that the definitions encapsulated in HSTPAs like the edTPA and the KPTP are overly technical, ignoring what they see as equally or more important personal and political aspects of teaching. This study suggests that both camps may be at least partially correct. This research also suggests that, at least one of the goals of supporters of the edTPA, the development and diffusion of a unified vision of teaching with “student learning in the center of a three-step teaching cycle: planning, instruction, and assessment” (Sato, 2014) can be achieved with an HSTPA like the KPTP that is cheaper and more responsive to local contexts. Further research will be necessary to determine both the strength of and longevity of the learning experience created by the KPTP and the degree and impact of conceptual tension engendered by the KPTP. Acknowledgements The author would like to acknowledge the contribution of Dr. Heidi Hallman, University of Kansas Department of Curriculum & Teaching, for her mentorship during the completion of this study. In addition, he would like to thank Dr. Joe O’Brien, University of Kansas Department of Curriculum & Teaching, whose recommendation provided the initial spark for this inquiry. Education Policy Analysis Archives Vol. 28 No. 138 22 References Ahmed, K. S. (2019). Against the script with the edTPA: Preservice teachers utilize performance assessment to teach outside scripted curriculum. Urban Education, 1-31. https://doi.org/10.1177/0042085919873689 American Association of Colleges of Teacher Education (AACTE). (n.d.). edTPA FAQ. http://edtpa.aacte.org/faq Bastian, K. C., Lys, D., & Pan, Y. (2018). A framework for improvement: Analyzing performance-assessment scores for evidence-based teacher preparation program reforms. Journal of Teacher Education, 69(5), 448-462. https://doi.org/10.1177/0022487118755700 Butler, M. (2015). Three ways edTPA prepared me for the classroom. Educational Horizons, (February/March), 30. https://doi.org/10.1177/0013175X15570862 California Commission on Teacher Credentialing. (2015). California teaching performance assessment. Author. Chen, J., Brown, G. T. L., Hattie, J. A. C., & Millward, P. (2012). Teachers’ conceptions of excellent teaching and its relationships to self-reported teaching practices. Teaching and Teacher Education, 28(7), 936–947. https://doi.org/10.1016/j.tate.2012.04.006 Choppin, J. & Meuwissen, K. (2017). Threats to validity in the edTPA video component. Action in Teacher Education, 39(1), 39-53. https://doi.org/10.1080/01626620.2016.1245638 Chong, S. (2011). Development of teachers’ professional identities: From pre-service to their first year as novice teachers. KEDI Journal of Educational Policy, 8(2), 219–233. Chung, R.R. (2008). Beyond assessment: Performance assessments in teacher education. Teacher Education Quarterly, 35(1), 8-28. Cochran-Smith, M., Piazza, P., & Power, C. (2013). The politics of accountability: Assessing teacher education in the United States. The Educational Forum, 77(1), 6–27. https://doi.org/10.1080/00131725.2013.739015 Cochran-Smith, M., & Power, C. (2010). New directions for teacher preparation. Educational Leadership, (May), 6–13. Cooper, J., & He, Y. (2012). Journey of “becoming”: Secondary teacher candidates’ concerns and struggles. Issues in Teacher Education, 21(1), 89–108. Council of Chief State School Officers. (2013). InTASC model core teaching standards and learning progressions for teachers 1.0: A resource for ongoing teacher development . Author. Darling-Hammond, L. (2010). Evaluating teacher effectiveness: How teacher performance assessments can measure and improve teaching. Center for American Progress. Darling-Hammond, L., & Hyler, M. E. (2013). The role of performance assessment in developing teaching as a profession. Rethinking Schools, 27(4), 10–15. DeLuca, C., & Volante, L. (2016). Assessment for learning in teacher education programs: Navigating the juxtaposition of theory and praxis. Journal of the International Society for Teacher Education, 20(1), 19-31. Dover, A. G., & Schultz, B. D. (2016). Troubling the edTPA: Illusions of Objectivity and Rigor. The Educational Forum, 80(1), 95–106. https://doi.org/10.1080/00131725.2015.1102368 Educational Testing Service (ETS). (2017). The PPAT assessment task requirements. https://www.ets.org/ppa/test-takers/teachers/build-submit/requirements/ Ellsworth, T. M., O’Brien, J., & Moore, J. (2014). College councils for the social studies  : How to support our newest colleagues. Social Education, 78(2), 80–83. Erlandson, D., Skipper, B. L. B., Allen, S. D. S., & Harris, E. L. (1993). Doing naturalistic inquiry: A guide to methods. Sage. https://doi.org/10.1177/0042085919873689 http://edtpa.aacte.org/faq https://doi.org/10.1177/0022487118755700 https://doi.org/10.1177/0013175X15570862 https://doi.org/10.1016/j.tate.2012.04.006 https://doi.org/10.1080/01626620.2016.1245638 https://doi.org/10.1080/00131725.2013.739015 https://doi.org/10.1080/00131725.2015.1102368 https://www.ets.org/ppa/test-takers/teachers/build-submit/requirements/ Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 23 Gardner, J.A. (1996). The “states as laboratories” metaphor in state constitutional law. Valparaiso University Law Journal, 30(2), 475-491. Gitomer, D. H., Martínez, J. F., Battey, D., & Hyland, N. E. (2019). Assessing the Assessment: Evidence of Reliability and Validity in the edTPA. American Educational Research Journal, 2- 29. https://doi.org/10.3102/0002831219890608 Glaser, B. G., & Strauss, A. (1967). The discovery of grounded theory. Aldine. Glasgow, J., Jenkins, S., & Gillespie, K. (2014). Electronic observation: 21st century teacher education. The Advocate, 22(1), 22–29. https://doi.org/10.4148/2637-4552.1071 Goldhaber, D., Cowan, J., & Theobald, R. (2017). Evaluating prospective teachers: Testing the predictive validity of the edTPA. Journal of Teacher Education, 68(4), 377-393. https://doi.org/10.1177/0022487117702582 Greenberg, J., & Walsh, K. (2012). What teacher preparation programs teach about K-12 assessment: a review. National Council on Teacher Quality. Greenblatt, D. (2018). Neoliberalism and teacher certification. Policy Futures in Education, 16(6), 804-827. https://doi.org/10.1177/1478210318771827 Greenblatt, D., & O’Hara, K. E. (2015). Buyer beware : Lessons learned from edTPA in New York State. Thought & Action, (Summer), 57–68. Hochstetler, S. (2011). Focus on identity development: A proposal for addressing English teacher attrition. The Clearing House, 84, 256–259. https://doi.org/10.1080/00098655.2011.590552 Hochstetler, S., & Orzulak, M. J. M. (2019). Writing problems and promises in standardized teacher performance assessment. In H. Hallman, K. Pastore-Capuana, & D. Pasternak Possibilities, challenges, and changes in English teacher education today: Exploring identity and professionalization (pp. 57-71). Rowman & Littlefield. Holdheide, L. R., Goe, L., Croft, A., & Reschly, D. J. (2010). Challenges in evaluating special education teachers and English language learner specialists: Research and policy brief . National Comprehensive Center for Teacher Quality. Hollins, E. (2012). Finally, a truly promising new approach to improvin g teacher preparation. http://edtpa.aacte.org/wp-content/uploads/2012/09/Hollins-edTPA.pdf Hunter, M., & Russell, D. (1981). Planning for effective instruction: Lesson design. In Increasing your teaching effectiveness (pp. 63-69). The Learning Institute. Jackson, P. (1986). The practice of teaching. Teachers College Press. Kansas Division of Learning Services. (2015). Kansas performance teaching portfolio. Author. Kleyn, T., Lopez, D., & Makar, C. (2015). “What about bilingualism? A critical reflection on the edTPA with teachers of emergent bilinguals. Bilingual Research Journal, 38(1), 88-106. https://doi.org/10.1080/15235882.2015.1017029 Knight, S. L., Lloyd, G. M., Arbaugh, F., Gamson, D., McDonald, S. P., Nolan, J., & Whitney, . E. (2014). Performance assessment of teaching: Implications for teacher education. Journal of Teacher Education, 65(5), 372–374. https://doi.org/10.1177/0022487114550475 Kronk Warner, E. A. (2015). Justice Brandeis and Indian Country: Lessons from the tribal environmental laboratory. Arizona State Law Journal, 47(3), 857–916. https://doi.org/10.2139/ssrn.2489211 Lachuk, A. J., & Koellner, K. (2015). Performance-based assessment for certification: Insights from edTPA implementation. Language Arts, 93(2), 84-95. Lefstein, A. (2005). Thinking about the technical and the personal in teaching. Cambridge Journal of Education, 35(3), 333–356. https://doi.org/10.1080/03057640500319040 https://doi.org/10.3102/0002831219890608 https://doi.org/10.4148/2637-4552.1071 https://doi.org/10.1177/0022487117702582 https://doi.org/10.1177/1478210318771827 https://doi.org/10.1080/00098655.2011.590552 http://edtpa.aacte.org/wp-content/uploads/2012/09/Hollins-edTPA.pdf https://doi.org/10.1080/15235882.2015.1017029 https://doi.org/10.1177/0022487114550475 https://doi.org/10.2139/ssrn.2489211 https://doi.org/10.1080/03057640500319040 Education Policy Analysis Archives Vol. 28 No. 138 24 Lewis, W. D., & Young, T. V. (2013). The politics of accountability: teacher education policy. Educational Policy, 27(2), 190–216. https://doi.org/10.1177/0895904812472725 Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Sage. Liu, L. B., & Milman, N. B. (2013). Year one implications of a teacher performance assessment's impact on multicultural education across a secondary education teacher preparation program. Action in Teacher Education, 35(2), 125-142. https://doi.org/10.1080/01626620.2013.775971 Madeloni, B., & Gorlewski, J. (2013). Wrong answer to the wrong question: Why we need critical teacher education, not standardization. Rethinking Schools, 27(4). http://www.rethinkingschools.org/archive/27_04/27_04_madeloni-gorlewski.shtml Madison, J. (1952). Federalist 56. In R.M. Hutchins (Ed.), Great books of the western world: American state papers, the Federalist, J.S. Mill (Vol. 43, pp. 174-176). Encyclopedia Britannica, Inc. (Original work published 1788) Margolis, J., & Doring, A. (2013). National assessments for student teachers: Documenting teaching readiness to the tipping point. Action in Teacher Education, 35(4), 272–285. https://doi.org/10.1080/01626620.2013.827602 Marshall, C., & Rossman, G. B. (2006). Designing qualitative research (4th ed.). Sage. Mehta, J., & Doctor, J. (2013). Raising the bar for teaching. Phi Delta Kappan, 94(7), 8–13. https://doi.org/10.1177/003172171309400703 Meuwissen, K. W., & Choppin, J. M. (2015). Preservice teachers’ adaptions to tensions associated with the edTPA during its early implementation in New York and Washington states. Education Policy Analysis Archiver, 23(103), 1-25. https://doi.org/10.14507/epaa.v23.2078 Miller, M., Carroll, D., Jancic, M., & Markworth, K. (2015). Developing a culture of learning around the edTPA: one university’s journey. New Educator, 11(1), 37-59. https://doi.org/10.1080/1547688X.2014.966401 Missouri Department of Elementary and Secondary Education (MoDESE). (2017). Task requirements for the Missouri Pre-Service Teacher Assessment (MoPTA). https://www.mega.ets.org/test-takers/mopta/build-submit-tasks/requirements/ Myers, S., Chmidling, C., & Williams, J. (2016). Higher Education I nformation and Procedures. http://www.ksde.org/Agency/Division-of-Learning-Services/Teacher-Licensure-and- Accreditation/Postsecondary/Higher-Ed-Faculty-Resources/Higher-Education- Resources-TLA National Board for Professional Teaching Standards (NBPTS). (2017). Guide to National Board Certification (version 2.1). http://www.nbpts.org/wp- content/uploads/Guide_to_NB_Certification.pdf Nelson, N. (2016). Kansas performance teaching portfolio implementation guidelines. Kansas State Department of Education. Organisation for Economic Co-operation and Development (OECD). (2013). Teachers for the 21st century: Using evaluation to improve teaching. Author. Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Sage. Peck, C., Gallucci, C., & Sloan, T. (2010). Negotiating implementation of high -stakes performance assessment policies in teacher education: From compliance to inquiry. Journal of Teacher Education, 61(5), 451-463. https://doi.org/10.1177/0022487109354520 Peck, C. A., & McDonald, M. (2013). Creating “cultures of evidence” in teacher education: Context, policy and practice in three high data use programs. The New Educator, 9(1), 12- 28. https://doi.org/10.1080/1547688X.2013.751312 https://doi.org/10.1177/0895904812472725 https://doi.org/10.1080/01626620.2013.775971 http://www.rethinkingschools.org/archive/27_04/27_04_madeloni-gorlewski.shtml https://doi.org/10.1080/01626620.2013.827602 https://doi.org/10.1177/003172171309400703 https://doi.org/10.14507/epaa.v23.2078 https://doi.org/10.1080/1547688X.2014.966401 https://www.mega.ets.org/test-takers/mopta/build-submit-tasks/requirements/ http://www.ksde.org/Agency/Division-of-Learning-Services/Teacher-Licensure-and-Accreditation/Postsecondary/Higher-Ed-Faculty-Resources/Higher-Education-Resources-TLA http://www.ksde.org/Agency/Division-of-Learning-Services/Teacher-Licensure-and-Accreditation/Postsecondary/Higher-Ed-Faculty-Resources/Higher-Education-Resources-TLA http://www.ksde.org/Agency/Division-of-Learning-Services/Teacher-Licensure-and-Accreditation/Postsecondary/Higher-Ed-Faculty-Resources/Higher-Education-Resources-TLA http://www.nbpts.org/wp-content/uploads/Guide_to_NB_Certification.pdf http://www.nbpts.org/wp-content/uploads/Guide_to_NB_Certification.pdf https://doi.org/10.1177/0022487109354520 https://doi.org/10.1080/1547688X.2013.751312 Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 25 Peck, C. A., Singer-Gabella, M., Sloan, T., & Lin, S. (2014). Driving blind: Why we need standardized performance assessment in teacher education. Journal of Curriculum and Instruction, 8(1), 8-30. https://doi.org/10.3776/joci.2014.v8n1p8-30 Pecheone, R. L., Shear, B., Whittaker, A., & Darling-Hammond, L. (2013). 2013 edTPA field test: Summary report. Stanford Center for Assessment, Learning, and Equity. Picower, B., & Marshall, A.M. (2016). “Run like hell” to “Look before you leap”: Teacher educators’ responses to preparing teachers for diversity and social justice in the wake of edTPA. In J. Carter J. & H. Lochte (Eds.) Teacher performance assessment and accountability reforms (pp. 189-212). Palgrave Macmillan. Pillen, M. T., Den Brok, P. J., & Beijaard, D. (2013). Profiles and change in beginning teachers’ professional identity tensions. Teaching and Teacher Education, 34(1), 86–97. https://doi.org/10.1016/j.tate.2013.04.003 Potemski, A., Baral, M., & Meyer, C. (2011). Alternative measures of teacher performance: policy-to- practice brief. National Comprehensive Center for Teacher Quality. Reagan, E. M., Schram, T., McCurdy, K., Chang, T. H., & Evans, C. M. (2016). Politics of policy: Assessing the implementation, impact, and evolution of the Performance Assessment for California Teachers (PACT) and the edTPA. Education Policy Analysis Archives, 24(9), 1–27. https://doi.org/10.14507/epaa.24.2176 Sato, M. (2014). What is the underlying conception of teaching of the edTPA? Journal of Teacher Education, 65(5), 421–434. https://doi.org/10.1177/0022487114542518 Serafini, F. (2002). Possibilities and challenges: The National Board for Professional Teaching Standards. Journal of Teacher Education, 53(4), 316–327. https://doi.org/10.1177/0022487102053004004 Shulman, L. (1998). Teacher portfolios: A theoretical activity. In N. Lyons (Ed.), With portfolio in hand: Validating the new teacher professionalism (pp. 23–37). Teachers College Press. Skrtic, T. (1985). Doing naturalistic research into educational organizations. In Organizational theory and inquiry: the paradigm revolution (pp. 185–220). Sage. Stake, R. E. (2005). Qualitative case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 443–466). Sage. Stanford Center for Assessment, Learning, & Equity (SCALE) . (2017). edTPA. https://www.edtpa.com Stillman, J., Ragusa, G., & Whittaker, A. (2015). Teacher performance assessment: Readiness for professional practice. In E. R. Hollins (Ed.), Rethinking field experiences in preservice teacher preparation (pp. 171–201). Routledge. Tuck, E. & Gorlewski, J. (2016). Racist ordering, settler colonialism, and edTPA. Educational Policy, 30(1), 197-217. https://doi.org/10.1177/0895904815616483 University of Kansas. (2016). School of Education student teaching handbook. Author. Wang, J., Lin, E., Spalding, E., Klecka, C. L., & Odell, S. J. (2011). Quality teaching and teacher education: a kaleidoscope of notions. Journal of Teacher Education, 62(4), 331–338. https://doi.org/10.1177/0022487111409551 Wang, J., Odell, S. J., Klecka, C. L., Spalding, E., & Lin, E. (2010). Understanding teacher education reform. Journal of Teacher Education, 61(5), 395–402. https://doi.org/10.1177/0022487110384219 Warner, C. K. (2019). Response to Chapter 4: Writing problems and promises in standardized teacher performance assessment. In H. Hallman, K. Pastore-Capuana, & D. Pasternak Possibilities, challenges, and changes in English teacher education today: Exploring identity and professionalization (pp. 71-76). Rowman & Littlefield. https://doi.org/10.3776/joci.2014.v8n1p8-30 https://doi.org/10.1016/j.tate.2013.04.003 https://doi.org/10.14507/epaa.24.2176 https://doi.org/10.1177/0022487114542518 https://doi.org/10.1177/0022487102053004004 https://www.edtpa.com/ https://doi.org/10.1177/0895904815616483 https://doi.org/10.1177/0022487111409551 https://doi.org/10.1177/0022487110384219 Education Policy Analysis Archives Vol. 28 No. 138 26 Warner, C. K. (2016). Contested definitions of excellent teaching: An ana lysis of the discourse of quality. Journal of Thought, 50(1), 20-36. Warner, C. K., Bell, C. V., McHatton, P. A., & Atiles, J. (2020). Navigating mandates and working toward coherence: Our journey with a high -stakes teacher performance assessment. The Educational Forum, 84(3), 226-239. https://doi.org/10.1080/00131725.2020.1739182 Wilson, M., Hallam, P., Pecheone, R. L., & Moss, P. A. (2014). Evaluating the validity of portfolio assessments for licensure decisions. Education Policy Analysis Archives, 22(6), 1– 30. Wichita State University. (2016). Teacher education student handbook. Author. About the Author Connor K. Warner University of Utah connor.warner@utah.edu ORCID https://orcid.org/0000-0002-2676-1436 Connor K. Warner, Ph.D., is an Assistant Professor in the Urban Institute for Teacher Education at the University of Utah. His scholarship focuses on teacher preparation program design, teacher education policy, and teacher candidate assessment. education policy analysis archives Volume 28 Number 138 September 21, 2020 ISSN 1068-2341 Readers are free to copy, display, distribute, and adapt this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, the changes are identified, and the same license applies to the derivative work. More detai ls of this Creative Commons license are available at https://creativecommons.org/licenses/by-sa/4.0/. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), QUALIS A1 (Brazil), SCImago Journal Rank, SCOPUS, SOCOLAR (China). Please send errata notes to Audrey Amrein-Beardsley at audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. https://doi.org/10.1080/00131725.2020.1739182 mailto:connor.warner@utah.edu https://orcid.org/0000-0002-2676-1436 https://creativecommons.org/licenses/by-sa/4.0/ http://www.doaj.org/ mailto:audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 27 education policy analysis archives editorial board Lead Editor: Audrey Amrein-Beardsley (Arizona State University) Editor Consultor: Gustavo E. Fischman (Arizona State University) Associate Editors: Melanie Bertrand, David Carlson, Lauren Harris, Danah Henriksen, Eugene Judson, Mirka Koro-Ljungberg, Daniel Liou, Scott Marley, Molly Ott, Iveta Silova (Arizona State University) Madelaine Adelman Arizona State University Amy Garrett Dikkers University of North Carolina, Wilmington Gloria M. Rodriguez University of California, Davis Cristina Alfaro San Diego State University Gene V Glass Arizona State University R. Anthony Rolle University of Houston Gary Anderson New York University Ronald Glass University of California, Santa Cruz A. G. Rud Washington State University Michael W. Apple University of Wisconsin, Madison Jacob P. K. Gross University of Louisville Patricia Sánchez University of University of Texas, San Antonio Jeff Bale University of Toronto, Canada Eric M. Haas WestEd Janelle Scott University of California, Berkeley Aaron Benavot SUNY Albany Julian Vasquez Heilig California State University, Sacramento Jack Schneider University of Massachusetts Lowell David C. Berliner Arizona State University Henry Braun Boston College Kimberly Kappler Hewitt University of North Carolina Greensboro Noah Sobe Loyola University Casey Cobb University of Connecticut Aimee Howley Ohio University Nelly P. Stromquist University of Maryland Arnold Danzig San Jose State University Steve Klees University of Maryland Jaekyung Lee SUNY Buffalo Benjamin Superfine University of Illinois, Chicago Linda Darling-Hammond Stanford University Jessica Nina Lester Indiana University Adai Tefera Virginia Commonwealth University Elizabeth H. DeBray University of Georgia Amanda E. Lewis University of Illinois, Chicago A. Chris Torres Michigan State University David E. DeMatthews University of Texas at Austin Chad R. Lochmiller Indiana University Tina Trujillo University of California, Berkeley Chad d'Entremont Rennie Center for Education Research & Policy Christopher Lubienski Indiana University Federico R. Waitoller University of Illinois, Chicago John Diamond University of Wisconsin, Madison Sarah Lubienski Indiana University Larisa Warhol University of Connecticut Matthew Di Carlo Albert Shanker Institute William J. Mathis University of Colorado, Boulder John Weathers University of Colorado, Colorado Springs Sherman Dorn Arizona State University Michele S. Moses University of Colorado, Boulder Kevin Welner University of Colorado, Boulder Michael J. Dumas University of California, Berkeley Julianne Moss Deakin University, Australia Terrence G. Wiley Center for Applied Linguistics Kathy Escamilla University of Colorado, Boulder Sharon Nichols University of Texas, San Antonio John Willinsky Stanford University Yariv Feniger Ben-Gurion University of the Negev Eric Parsons University of Missouri-Columbia Jennifer R. Wolgemuth University of South Florida Melissa Lynn Freeman Adams State College Amanda U. Potterton University of Kentucky Kyo Yamashiro Claremont Graduate University Rachael Gabriel University of Connecticut Susan L. Robertson Bristol University Miri Yemini Tel Aviv University, Israel Education Policy Analysis Archives Vol. 28 No. 138 28 arquivos analíticos de políticas educativas conselho editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editoras Coordenadores: Marcia Pletsch, Sandra Regina Sales (Universidade Federal Rural do Rio de Janeiro) Editores Associadas: Andréa Barbosa Gouveia (Universidade Federal do Paraná), Kaizo Iwakami Beltrao, (EBAPE/FGVl), Sheizi Calheira de Freitas (Federal University of Bahia), Maria Margarida Machado, (Federal University of Goiás / Universidade Federal de Goiás), Gilberto José Miranda, (Universidade Federal de Uberlândia, Brazil), Maria Lúcia Rodrigues Muller (Universidade Federal de Mato Grosso e Science) Almerindo Afonso Universidade do Minho Portugal Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil José Augusto Pacheco Universidade do Minho, Portugal Rosanna Maria Barros Sá Universidade do Algarve Portugal Regina Célia Linhares Hostins Universidade do Vale do Itajaí, Brasil Jane Paiva Universidade do Estado do Rio de Janeiro, Brasil Maria Helena Bonilla Universidade Federal da Bahia Brasil Alfredo Macedo Gomes Universidade Federal de Pernambuco Brasil Paulo Alberto Santos Vieira Universidade do Estado de Mato Grosso, Brasil Rosa Maria Bueno Fischer Universidade Federal do Rio Grande do Sul, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Fabiany de Cássia Tavares Silva Universidade Federal do Mato Grosso do Sul, Brasil Alice Casimiro Lopes Universidade do Estado do Rio de Janeiro, Brasil Jader Janer Moreira Lopes Universidade Federal Fluminense e Universidade Federal de Juiz de Fora, Brasil António Teodoro Universidade Lusófona Portugal Suzana Feldens Schwertner Centro Universitário Univates Brasil Debora Nunes Universidade Federal do Rio Grande do Norte, Brasil Lílian do Valle Universidade do Estado do Rio de Janeiro, Brasil Geovana Mendonça Lunardi Mendes Universidade do Estado de Santa Catarina Alda Junqueira Marin Pontifícia Universidade Católica de São Paulo, Brasil Alfredo Veiga-Neto Universidade Federal do Rio Grande do Sul, Brasil Flávia Miller Naethe Motta Universidade Federal Rural do Rio de Janeiro, Brasil Dalila Andrade Oliveira Universidade Federal de Minas Gerais, Brasil Exploring Candidate Interaction with a High Stakes Teacher Performance Assessment 29 archivos analíticos de políticas educativas consejo editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Coordinador (Español / Latinoamérica): Ignacio Barrenechea, Axel Rivas (Universidad de San Andrés Editor Coordinador (Español / Norteamérica): Armando Alcántara Santuario (Universidad Nacional Autónoma de México) Editor Coordinador (Español / España): Antonio Luzon (Universidad de Granada) Editores Asociados: Felicitas Acosta (Universidad Nacional de General Sarmiento), Jason Beech ( Universidad de San Andrés), Angelica Buendia, (Metropolitan Autonomous University), Alejandra Falabella (Universidad Alberto Hurtado, Chile), Veronica Gottau (Universidad Torcuato Di Tella), Carolina Guzmán-Valenzuela (Universidade de Chile), Cesar Lorenzo Rodriguez Uribe (Universidad Marista de Guadalajara María Teresa Martín Palomo (University of Almería), María Fernández Mellizo-Soto (Universidad Complutense de Madrid), Tiburcio Moreno (Autonomous Metropolitan University-Cuajimalpa Unit), José Luis Ramírez, (Universidad de Sonora), Maria Veronica Santelices (Pontificia Universidad Católica de Chile) Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Ana María García de Fanelli Centro de Estudios de Estado y Sociedad (CEDES) CONICET, Argentina Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Miguel Ángel Arias Ortega Universidad Autónoma de la Ciudad de México Juan Carlos González Faraco Universidad de Huelva, España José Gregorio Rodríguez Universidad Nacional de Colombia, Colombia Xavier Besalú Costa Universitat de Girona, España María Clemente Linuesa Universidad de Salamanca, España Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Xavier Bonal Sarro Universidad Autónoma de Barcelona, España Jaume Martínez Bonafé Universitat de València, España José Luis San Fabián Maroto Universidad de Oviedo, España Antonio Bolívar Boitia Universidad de Granada, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Jurjo Torres Santomé, Universidad de la Coruña, España José Joaquín Brunner Universidad Diego Portales, Chile María Guadalupe Olivier Tellez, Universidad Pedagógica Nacional, México Yengny Marisol Silva Laya Universidad Iberoamericana, México Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Miguel Pereyra Universidad de Granada, España Ernesto Treviño Ronzón Universidad Veracruzana, México Gabriela de la Cruz Flores Universidad Nacional Autónoma de México Mónica Pini Universidad Nacional de San Martín, Argentina Ernesto Treviño Villarreal Universidad Diego Portales Santiago, Chile Marco Antonio Delgado Fuentes Universidad Iberoamericana, México Omar Orlando Pulido Chaves Instituto para la Investigación Educativa y el Desarrollo Pedagógico (IDEP) Antoni Verger Planells Universidad Autónoma de Barcelona, España Inés Dussel, DIE-CINVESTAV, México José Ignacio Rivas Flores Universidad de Málaga, España Catalina Wainerman Universidad de San Andrés, Argentina Pedro Flores Crespo Universidad Iberoamericana, México Juan Carlos Yáñez Velazco Universidad de Colima, México javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/819') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/820') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/4276') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1609') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/825') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/797') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/823') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/798') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/555') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/814') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/2703') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/801') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/826') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/802') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/3264') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/804')