TOWARDS AN OBJECTIVE EVALUATION OF TEACHER PERFORMANCE Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 5/4/2018 Facebook: /EPAAA Revisions received: 9/23/2018 Twitter: @epaa_aape Accepted: 9/23/2018 education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 26 Number 142 November 5, 2018 ISSN 1068-2341 What’s the Purpose? Educators’ Perceptions and Use of a State-Mandated Kindergarten Entry Assessment Hattie Harvey & Kathryn Ohle University of Alaska Anchorage United States Citation: Harvey, H., & Ohle, K. (2018). What’s the purpose? Educators’ perceptions and use of a state-mandated kindergarten entry assessment. Education Policy Analysis Archives, 26(142). http://dx.doi.org/10.14507/epaa.26.3877 Abstract: Educator’s perceptions and use of a state-mandated kindergarten entry assessment (KEA), the Alaska Developmental Profile (ADP), were investigated using a mixed -methods approach with 233 educators representing 23 districts in the state of Alaska. Educators reported inconsistencies in the administration, implementation, perceptions, and use of the ADP. These inconsistencies were connected to an unclear understanding of the purpose for the ADP, a finding that most likely reflects the compliance model of those administering state - mandated assessments instead of educators seeing these tools as useful for instructional decision-making. Results suggest policy makers should strive to ensure all stakeholders have a clear understanding of the purpose for a state-mandated KEA and that systems are in place to ensure reliability and validity of the data. Teachers are urged to become familiar in using formative, observation-based assessment approaches and consider how data gathered from a KEA can provide meaningful information for instructio nal planning. Administrators are encouraged to provide the professional development and support needed to engage in data - http://epaa.asu.edu/ojs/ http://dx.doi.org/10.14507/epaa.26.3877 Education Policy Analysis Archives Vol. 26 No. 142 2 driven decision making. Finally, researchers are reminded of their role in helping conduct research that examines the processes and impact of state-mandated assessments. Keywords: state-mandated assessments; kindergarten entry assessment; educator perceptions; early childhood educational policy ¿Cuál es el propósito? Percepciones de educadores y el uso de una evaluación Resumen: Percepciones del maestro y el uso de una evaluación de entrada para el jardín de infancia mandada por el estado (KEA) solicitada por el estado, el Alaska Developmental Profile (ADP) fueron encuestados usando un enfoque enfoque mixto con 233 educadores representando a 23 distritos en el estado de Alaska. Los educadores relataron inconsistencias en la administración, implementación, percepciones y uso del ADP. Estas inconsistencias estaban conectados a una comprensión clara de la finalidad de la ADP, encontrando que probablemente refleja el modelo de cumplimiento de los que administran comentarios ordenados por el estado, en lugar de los educadores consideran estas útiles herramientas para la toma de decisiones de instrucción. Los resultados sugieren que los responsa bles políticos deben esforzarse para asegurar que todos los interesados tengan una comprensión clara de la finalidad de un KEA requerido por el estado y que los sistemas están en su lugar para asegurar la fiabilidad y validez de los datos. Los maestros se les anima a familiarizarse con el uso de enfoques de evaluación formativa y en base a la observación y considerar cómo los datos recogidos de un KEA pueden proporcionar información útil para la planificación de la instrucción. Se alienta a los administradores a proporcionar el desarrollo profesional y el soporte necesario para involucrarse en la toma de decisiones basada en datos. Y, por último, los investigadores se les recuerda su papel en ayudar a realizar investigaciones para examinar los procesos y e l impacto de las evaluaciones requeridas por el estado. Palabras-clave: evaluaciones autorizadas por el Estado; evaluación de entrada de jardín de infancia; percepciones educativas; política educativa de la primera infancia Qual é o propósito? Percepções de educadores e o uso de uma avaliação Resumo: As percepções do educador e o uso de uma avaliação de entrada no jardim de infância (KEA) solicitada pelo estado, o Alaska Developmental Profile (ADP), foram pesquisados usando uma abordagem de abordagem mista com 233 educadores representando 23 distritos no estado do Alasca. Educadores relataram inconsistências na administração, implementação, percepções e uso do ADP. Essas inconsistências estavam conectadas a uma compreensão pouco clara do propósito do ADP, ao descobrir que provavelmente reflete o modelo de conformidade daqueles que administram avaliações ordenadas pelo estado, em vez de os educadores considerarem essas ferramentas úteis para a tomada de decisões instru cionais. Os resultados sugerem que os formuladores de políticas devem se esforçar para garantir que todas as partes interessadas tenham uma compreensão clara da finalidade de uma KEA exigida pelo estado e que os sistemas estejam implementados para garantir a confiabilidade e a validade dos dados. Os professores são encorajados a se familiarizarem com o uso de abordagens de avaliação formativa e baseada em observação e consideram como os dados coletados de um KEA podem fornecer informações significativas para o planejamento instrucional. Os administradores são encorajados a fornecer o desenvolvimento profissional e o suporte necessário para se envolver na tomada de decisão baseada em dados. Finalmente, os pesquisadores são lembrados de seu papel em ajudar a c onduzir pesquisas que examinem os processos e o impacto das avaliações exigidas pelo estado. Palavras-chave: avaliações mandatadas pelo estado; avaliação de entrada de jardim de infância; percepções educadoras; política educacional da primeira infância Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 3 What’s the Purpose? Educators’ Perceptions and Use of a State -mandated Kindergarten Entry Assessment “I guess I’m not sure…. what is the purpose of this assessment?” Despite the movement toward more curriculum-aligned assessments at district and school levels, state-mandated assessments remain a part of accountability systems. In the United States, the use of kindergarten entry assessments (KEAs) has significantly grown over the past five years with 35 out of 50 states either implementing or piloting a KEA with seven other states in development stages (Pierson, 2018). Although KEAs are less likely to be directly linked with a state’s accountability system under the Every Student Succeeds Act ([ESSA] 2015), many state educational agencies have adopted or created a KEA, and in some cases these are state-mandated (Pierson, 2018). This is due in large part to an increase in federal funding directed toward enhancing states’ comprehensive early childhood systems along with the evidence supporting the substantial longitudinal benefits of high-quality early care and educational experiences (Lehrl, Kluczniok, & Rossbach, 2016; McCoy et al., 2017). Between the Enhanced Assessment Grants and the Race to the Top Early Learning Challenge (RTT-ELC) grants, the U.S. Department of Education has invested more than $1 billion to the development of early childhood assessment systems, with much of the monies directed toward supporting KEA development and enhancement. Generally, a KEA is intended to document a child’s developmental skills and behaviors that have been shown to predict long-term student success (U.S. Department of Education, 2011, 2014). Yet despite this rapid growth in KEAs, there remain multiple complexities regarding the uniformity, utility, and validity of these assessments (Goldstein & Flake, 2016; Pierson, 2018). State KEAs vary greatly in terms of the instruments used and the types of learning that are assessed (Ackerman, 2018; Pierson, 2018). Perhaps these complexities are confounded by the lack of coalescence around a single definition of a KEA (Ackerman, 2018) and the contributing factor of state controlled K-12 education (Gottfried, Stecher, Hoover, & Cross, 2011). Nonetheless, the movement toward aligning early childhood state systems with K-12 education is promising and the potential benefits of a KEA as part of this system are critical (Goldstein & Flake, 2016). As a former kindergarten teacher and a former early childhood school psychologist, we began our investigation from a practical perspective wondering how information from a state- mandated assessment, in this case our state’s KEA, was used to inform instructional planning within the classroom setting. In a comprehensive review of research, Young and Kim (2010) identified a disconnect between state-mandated assessments and how those data help teachers make formative, instructional decisions. As a kindergarten teacher, I was purposeful in gathering data that helped to inform my instruction and I used observation to understand children’s development; as an early childhood school psychologist, I was cognizant of gathering data in an authentic way to best demonstrate a child’s skills and abilities in order to link to appropriate developmental goals. As teacher educators, we believe that assessment information gathered by educators should be useful and purposeful. Through informal conversations about the KEA used in our own state, we were perplexed at the discrepancies being described and we began to investigate this very question—how do educators’ perceive and use a state-mandated KEA in Alaska? This question responds to a call for inquiry about how KEA data inform teachers’ instruction (Ackerman, 2018) as well as a call to better understand how accountability policies from state-mandated assessments are useful to teachers for instructional decision-making (Young & Kim, 2010). In this literature review we explore the current complexities of a KEA, including definitions and purpose; outline our theoretical framework focused on the purpose of assessment; describe the Education Policy Analysis Archives Vol. 26 No. 142 4 KEA used in Alaska; and summarize research on using educator perceptions to understand practices related to educational policy. Kindergarten Entry Assessments Definitions and purpose(s). Across the educational community and within professional organizations the definition and purpose of a KEA is not as transparent as one might think. It is somewhat difficult to disaggregate defining a KEA from its purpose, as they are inextricably linked. For example, the Enhanced Assessment Grant defined a KEA as a measure “...provid[ing], at kindergarten entry, valid and reliable information on each child’s learning and development across the essential domains of school readiness…[and] would be used to support educators in providing effective learning opportunities to every child, and help close achievement gaps” (U.S. Department of Education, 2013). In the RTT-ELC grants, the U.S. Department of Education (2011) defined a KEA as: Administered to children during the first few months of their entering kindergarten; covers all Essential Domains of School Readiness; conforms with recommendations of the National Research Council (2008) reports on early childhood; valid and reliability for its intended purposes and for the target populations and is aligned to the State’s early learning and development standards...and results of the assessment should be used to inform efforts to close the school readiness gap at kindergarten and to inform instruction in the early elementary grades (p. 155). Other national organizations and policy groups reference a KEA specifically as a measure to better understand children’s development. For example, the Build Initiative (2018) described a KEA as providing a snapshot on children’s development and is a “process [that is] an organized way to learn what children know and are able to do, including their disposition toward learning, when they enter kindergarten and/or at other times” (p. 1). Regenstein, Connors, Romero-Jurado, & Weiner (2017) emphasized the importance of a KEA as a measure to provide information about multiple areas of a child’s development including social and emotional, and subsequently can be used to improve quality teaching by providing information for early learning supports. According to the National Association of Education for Young Children (2003), assessments that are used with young children should be connected to beneficial purposes, including to make sound decisions about teaching and learning, to identify significant concerns that may require focused intervention for individual children, and to help programs improve their educational and developmental interventions. Indeed there appears to be consensus that KEAs are in fact an assessment used to gather information about a child’s development. And while the exact developmental skills and domains vary across states, KEAs typically comprise a combination of the developmental domains including: cognition and general knowledge (to include early mathematics and early scientific development), approaches to learning, language and literacy, physical well-being and motor development (to include adaptive skills), and social-emotional development (Krasnoff, 2015; Regenstein et al., 2017; U.S. Department of Education, 2011; 2014). While this emulates a whole-child perspective, and one grounded in early childhood developmental theory, there remains a need to deepen our understanding of how educators perceive and use a KEA, particularly when considering the underlying educational discourse that data should be used to make informed decisions on all levels - classroom, school, district, state, and national (e.g., Mandinach, 2012; Young & Kim, 2010). In considering this discourse, we applied the overarching theoretical framework focused on the purpose of assessment. Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 5 Purposes of assessment. The attempt to delineate the purpose of any one assessment (including a KEA) is complex, multi-dimensional, and contextually-driven. In a review of the literature, Goldstein and Flake (2016) identified nine potential purposes of assessment, which include to summarize learning and development, inform instruction, improve outcomes for children, communicate with families, collect results-based accountability data, evaluate resource allocation and measure return on investments in funding, improve program quality, improve teacher quality, and/or maintain longitudinal data systems. Mandinach (2012) emphasized that assessment is the foundation of data-based decision making and “pertains to the systematic collection, analysis, examination, and interpretation of data to inform practice and policy in educational settings” (p. 1). Others have identified the purpose of assessments to align more closely to the National Association for the Education of Young Children (NAEYC; 2009) position statement on assessment noted earlier. For example, Fiore (2012) defined assessment as “a process that includes and/or results in products that inform decisions about children and curriculum” (p. 5). A similar definition is posed by Grisham-Brown and Pretti-Frontzack (2011, p. 17), who state that “assessment is defined as gathering information for the purposes of making decisions.” Yet despite some commonalities in defining the purpose of assessment, there is evidence that educators and educational organizations have an inconsistent understanding of the role of assessments in instructional decision-making (Wininger & Norman, 2005; Young & Kim, 2010). For example, Young and Kim (2010) summarized an abundance of research noting the vast array of assessments used (i.e., student portfolios, running records, spontaneous performance-based assessment), with less use of curriculum published or standardized tests but little consistency in how educators reported using the data for instructional planning. Hattie and Timperley (2007) offered three questions to help define the purpose of assessment for educators: “Where am I going? How am I going? and Where to Next?” (p. 88), all relevant questions to consider in the context of a KEA. Alaska’s Kindergarten Entry Assessment: The Alaska Developmental Profile In Alaska, the first state-mandated kindergarten entry assessment was initially introduced in state statute in 1997. This came about following the Quality Schools Initiative, which required a developmental profile be completed for each child entering kindergarten as part of a comprehensive system of student assessments (Fenton, 2001). Created by local educators, committees, and consultants, the Alaska Developmental Profile (ADP) was first used in 2000 and later revised in 2007. According to the ADP Implementation Guide (Alaska Department of Education and Early Development [DEED], 2016a), the purpose of the ADP is to “identify, record and summarize the skills and behaviors students demonstrate at the beginning of their kindergarten year, based on teacher observations” (p. 6). It is not intended to be used to determine if a child is ready to enter kindergarten, as in Alaska the only determinant for entering kindergarten is that children are five. The ADP is a tool used to measure children’s skills and abilities in five domains: 1) Physical Health and Well-Being; 2) Social and Emotional Development; 3) Approaches to Learning; 4) Cognition and General Knowledge; and 5) Communication, Language, and Literacy. Each domain is comprised of 2 goals (with the exception of Communication and Language which has 5 goals) and a set of 3-5 indicators for each goal that describe expected observable behaviors or skills. For example, under the domain of Communication, Language and Literacy one goal states Uses Receptive Communication Skills; this is followed by four indicators including, listening to others in group conversations and discussions, responds to a request, attends to book reading/story telling for at least five minutes, and understands prepositions in a simple commands. Teachers are expected to observe a child’s skills and behaviors over the first six to eight weeks of school and then rate each child on each goal using a three-point scale (1 = consistently demonstrates [80% of more of the time], 2 = progressing [inconsistent], or 3 = does not demonstrate [20% of less of the time]) (see Appendix A for an example of the ADP Education Policy Analysis Archives Vol. 26 No. 142 6 Recording Form for Classroom Use). Teachers input their final ratings into the state’s online reporting system by a specified date. The results then are disaggregated by demographic variables for comparison, enabling stakeholders to “identify areas most in need of intervention, track the results of investments, and monitor trends over time” (Alaska DEED, 2016b, p. 47). Despite the relatively long history of the ADP, little documentation exists regarding the perceptions, use, or impact of the ADP on administrators, teachers, students, or policy makers, providing impetus for the current study. Consequently, it may be relevant to consider that “Educational change depends on what teachers do and think – it’s as simple and complex as that” (Fullan, 2007, p. 129). Educator Perceptions of Assessment Engaging teachers in efforts to understand the collective expertise in schools is an approach that has long been seen as viable (Kaniuka, 2009). This assumption was also informed by the premise that “Policy is largely what practitioners perceive it to be rather than some external document or legislation” (Jennings, 1996, p.15). Over the last 20 years, numerous studies have looked at educator perceptions around state-mandated assessments, and more recently around other informal assessment measures such as curriculum-based assessments and KEAs. In regards to state- mandated assessments, many have documented concerns around a number of issues: questionable quality and reliability of tests (Franklin & Snow-Gerono, 2007; Public Education Network, 2007); an increase in time spent on teaching test-taking skills (Darling-Hammond & Rustique-Forrester, 2005); negative effects on teacher and student motivation (Hagge & Waltman, 2007; Powell, Higgins, Aram, & Freed, 2009); less creativity and innovation in instruction (Nadelson et al., 2012; Smith & Kovacs, 2011); and increased pressure and stress on teachers regarding the need to improve test scores (Fitzgerald, 2008). Alternatively, some educator perceptions are more favorable toward low- stakes assessment measures, such as curriculum-based assessments and authentic assessment measures (Bagnato, Goins, Pretti-Frontczak, & Neisworth, 2014), or informal assessment measures (Harrison, 2005; Nadelson, Pluska, Moorcroft, Jeffrey, & Woodard, 2014). Some recent research conducted on teachers’ perceptions of state-mandated KEAs indicates they do not feel it informs their instruction (Schachter, Strang, & Piasta, 2017a, 2017b) or that they are unclear about the purpose (Butts, 2013). As documented in the evaluation of the pilot study conducted in Oregon, “many principals, teachers, and assessors did not have a clear understanding of the primary purpose of the statewide kindergarten assessments and how this is different from the formative assessments already being implemented by teachers for instructional purposes” (Furrer & Green, 2013, p. 10). Documenting educators’ perceptions related to state educational policies is seen as a viable approach to understanding “real-world” classroom practices (Cohen & Ball, 1990; Hattie & Timperley, 2007; Jennings, 1996), and in this particular case, how a state’s mandated KEA is understood and used. Research Question and Rationale We sought to better understand educators’ perceptions and practices in using the ADP, Alaska’s KEA, and in particular if and how they use the data. Recently, Ackerman (2018) called for future research to examine the extent to which kindergarten teachers are actually using KEA data to inform their instruction, and noted that using a case-study approach may “...be helpful for generating lessons learned and thus inform other states’ KEA development and implementation efforts” (p. 20). Further, if KEAs have the utility to be used for instructional purposes to support children’s learning and development (Regenstein et al., 2017), then teachers and schools will be better positioned to meet the needs of their students, an educational discourse we both support and believe. For the current study we developed the following research question: how do educators’ perceive and use a state-mandated KEA in Alaska? Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 7 Method Participants Participants were recruited through electronic invitation (see Procedures section for details). The majority of the participants were kindergarten teachers (n=127), followed by elementary school principals (n=52), preschool teachers (n=39), and district-level administrators (n=15) for a total sample of 233 participants. We included preschool teachers in the sample because in some districts, public preschool teachers use the ADP. We also included school and district administrators, for as educational leaders, they are held accountable for implementing state-mandated assessments and guiding data-driven decision making. The total sample represented a range of teaching experience: 27% of the participants having 1-3 years of experience, 25% having 4-7 years of experience, and 48% having more than 7 years of experience. Participants’ educational levels ranged from associates degree (2%) to bachelor’s degree only (44%) to master’s degree or higher (64%). Table 1 presents participants’ background information including years of experience, highest degree obtained, bachelor’s educational degree by the total sample and the interview sample. Table 1 Demographic Information Demographic Total Sample Interview Sample Position Type Preschool Teacher 18 1 Kindergarten 127 23 School Principal 52 2 District-Level Administrator 39 4 Years in Position Less than 1 26 3 3-4 64 4 4-7 59 5 8-11 28 3 >11 54 14 Highest Degree CDA 4 0 BA 78 3 MA or PhD 150 27 BA Educational Training BA Early Childhood 20 2 BA Elementary 86 9 BA Other 3 0 Education Policy Analysis Archives Vol. 26 No. 142 8 Note. For total sample N = 231. For interview sample, n = 30. For the school type, n = 26 as the four district-level administrators did not identify with a school type. Of the total 233, 30 participants consented to participate in a follow-up interview, which included one preschool teacher, 23 kindergarten teachers, and six administrators, of whom four identified as district level administrators and two as school principals. The majority (47%) reported having worked in their position for over 11 years, and 90% had earned a master’s or doctoral degree. These participants represented 8 districts, of which 15 were from the largest school district. Although exact response rates were not available, the representation of participants across districts provides confidence in a reasonably representative sample. The participants in this study represented 23 out of the 54 districts in the state of Alaska, of which those 23 account for 79% of the students served in the state in 2015-2016. Note that district sizes range from 48,370 students to 14 students and the two largest districts (representing over ½ of the student population in the state) accounted for 53% of the participants, and the remaining 19 districts (representing 30% of students in the state) accounted for the remaining 47% of participants. Data Collection Instruments ADP practices survey. Participants completed an online survey using Qualtrics, designed by the researchers in collaboration with two school district administrators from the largest school district in Alaska, whom in particular wondered about possible group differences in perceptions. The survey questions were informed by the literature on teacher practices using assessment data (Lesaux & Marietta, 2012; Young & Kim, 2010). The initial questionnaire (included a demographic section plus 18 questions) was tested for face validity and content validity through the use of three content review experts from one school district (Director of Curriculum and Instruction, Director of Preschool Programs, and one Kindergarten teacher). The review experts were instructed about the purpose of the questionnaire and were asked a series of questions to determine face validity (e.g., are the questions clear and do they accurately relate the intent for which you should be answering the questions?). To determine the content validity, the content experts were asked if additional sections should be added to sufficiently address the topic, and if there were recommendations for any item deletion. All three of the content review experts reported a clear understanding of the directions and a clear understanding of the purpose of the study. Two of the three reviewers suggested the removal of two closed-ended items to increase readability and reduce redundancy and all three suggested removal of one open-ended question to reduce redundancy and increase clarity. The final questionnaire included ten closed-ended questions with either a 1-3 scale response (yes, maybe, no) or a disagree/agree response (e.g., Does the ADP impact your instructional decisions?), five open-ended questions (e.g., If yes, how does the ADP impact your instructional decisions?), and a solicitation to voluntarily participate in a follow-up interview (see Appendix B for the full survey). Of note is that for the purpose of this article, not all statemensts about the ADP are included in the results section (e.g., the ADP is a tool used by the state, the ADP provides a report for school administrator and district level administration) as they were specifically requested by the collaborating school district partners. Interview. A semi-structured interview (see Appendix C) was developed to allow for further clarifications regarding statements around how the stakeholders perceived the ADP, when it was administered, and how the results impacted instruction in comparison to other assessments given throughout the year. This served, in part, as a validity check of the responses given to survey questions (Schuman, 1970). Anecdotally, many were asked follow-up questions in relation to their beliefs about kindergarten readiness in relation to the ADP, as this line of questioning often naturally emerged. Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 9 Procedure Following approval from the Human Subjects Research Committee at the researcher's’ institution, an email invitation with an electronic link to the survey was distributed to all superintendents in 54 school districts in the state of Alaska who were asked to forward the survey to elementary school principals, kindergarten teachers and preschool teachers (when applicable). Following the completion of the surveys, one of the principal researchers and a trained research assistant contacted all participants who noted in the last question of the survey a desire to participate in a follow-up interview, which resulted in 30 interviews. The interviews were conducted in person, if located within a reasonable driving distance, and via phone for those located farther than driving distance. The interviews, which ranged from 10 minutes to 55 minutes, were audio-recorded, transcribed, and summarized. Additionally, the researchers wrote personal memos after the interviews were conducted or listened to, to record their perceptions of what the salient points were, which served as the start of the analysis process (Glesne, 2006). These reflective memos were compared to ensure both researchers identified similar key findings that emerged from each interview. Qualitative Data Analysis A qualitative approach where key themes and patterns were identified through the use of coding was used with the open-ended survey questions and interviews (Coffey & Atkinson, 1996; Glesne 2006). For the open-ended survey questions, the responses were first organized into one document and then read numerous times by both researchers. Notes were written to help identify the significant thoughts, ideas, and patterns that emerged, as is characteristic of open-coding schemes, and differences in perceived levels of importance were reconciled between the researchers. The notes were used to generate a list of codes (e.g., how the ADP was being used, the ADP shows growth, and educators who felt the ADP only looks at where students are coming in). With the interview data, both researchers wrote personal memos following each interview (for the researcher who did not conduct the interview, the audio recording was used). The personal memos were reviewed and compared between the two researchers for consistency, and once differences were reconciled both researchers used an open-coding system on the memos, allowing the significant thoughts, ideas, and patterns to emerge. While at times the codes seemed to match those that emerged from the survey data, there were also new ones that emerged. For example, a number of codes around data collection procedures emerged (i.e., time of data collection, educators who collected data during delayed entry, and educators who collected data longitudinally). The notes and codes that emerged from both sources of data were then compared and contrasted, which enabled the most prolific ideas to emerge within and across individual responses and data sources (Miles & Huberman, 1984). This resulted in 39 detailed codes, which were selectively applied to the surveys and interview transcripts by both researchers. Through discussion and consensus the selective codes were then reduced and separated into twelve themes based on related commonalities, which then fell under five larger, more inclusive, categories, with which our results are organized. Ensuring credibility. To ensure the research was conducted with rigor, efforts were made to ensure credibility which included clarified researcher bias, which promoted transferability by sharing sufficient detail of the results and the contexts in which they were gathered; multiple methods of data collected, which created dependability; and conducted an external audit with Education Policy Analysis Archives Vol. 26 No. 142 10 researchers from Regional Educational Laboratory Northwest. Finally, to promote confirmability we triangulated the data, engaged in member checking, and used peer debriefing. Quantitative Data Analysis Using the participant sample, the survey was tested for internal reliability using item analysis, which revealed one factor loading for the closed-ended questions. Using data from the participant sample, the Cronbach's alpha resulted in a strong internal consistency ( = .74). The closed-ended questions were analyzed using SPSS, which included descriptive statistics and Pearson Chi-Square correlations to determine if any demographic group differences exist. Results First, we present the findings that emerged from survey and interview data which provide a more in-depth understanding of teachers’ perceptions and practices as they relate to the ADP. These emerged from the categories that best connected to the research question, including inconsistencies in educator perceptions of ADP, how the ADP is used, how data are collected, and educator concerns about the ADP (see Table 2 for more details). Results that fell into the fifth category, Approaches to Teaching Kindergarten, are further explored in a separate publication (Ohle & Harvey, 2017). Second, we present results from an exploratory analysis to determine if there were any differences in perceptions and use based on position, educational background training, or years of experience that might illuminate, explain, or add to the findings. Psydonyms are used to protect participant identify and any information which could potentially link a participant to a district is removed. Results Regarding Educators Perceptions and Practices In looking at the significant thoughts, ideas, and patterns that emerged through the open- ended survey questions and interviews, four major categories became apparent including: 1) inconsistencies in how the ADP is used, 2) how data is collected, 3) concerns about the ADP, and 4) the ADP as an assessment tool. Inconsistencies in how the ADP is used. In reviewing the results, it became clear there was a lack of consistency in the perceptions of the ADP and how it is used. The responses from the binary closed-ended questions revealed mixed findings regarding participants’ use of the ADP. Forty-two percent of participants (n = 97) reported that the ADP impacts their instructional decisions, while 52% (n = 123) reported that it does not (question #2 in Appendix B). Thirty-two percent (n = 75) reported that the ADP impacts their students, while 56% (n = 130) reported that it does not (question #3 in Appendix B). Clearly there was much variety within educators’ perceptions in regards to whether it impacted their instruction and their students. Using responses from seven statements about the ADP in which participants selected to either agree or not agree, a lack of consistency emerged regarding educators ‘perceptions around how the ADP is used. As seen in Figure 1, less than half of the participants rated that the ADP provides information that informs instruction (n = 73), helps build relationships with students (n = 51), or utilizes a developmentally appropriate method (n = 120). Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 11 Figure 1. Statements about the ADP. For each statement, participants were given the option to select “agree” or “disagree” These conflicting viewpoints appeared in the interviews as well, as one administrator wondered out loud, What we know is where students often times come in, is a very different place than after nine weeks of instruction. And after nine weeks of being in the environment and building relationships with adults and peers, there’s so much growth that’s occurred during the first nine weeks….Kids often come in looking very different than after they did after a month, but if we make judgments about the information they demonstrate, you really need to look at how important it is to give children time within their environment. It may be a little bit about growth, but it’s really about how they come in. (Betsy, 1/29/16) The lack of consistency in educator perceptions emerged within the analysis of the open-ended survey questions as well, as some educators identified it as a tool that helped them gather data in a developmentally appropriate method, others used it to create a baseline, and for others, used it to inform their instruction and identify small groups. As one participant commented, “It gives me baseline data about my students’ school readiness and thus helps determine where I will begin with my instruction with each child”. However, many saw the ADP simply as a mandate and something they did to “check a box”. This impacted whether they saw the ADP as being helpful. For those that felt many of the elements on the ADP were things they were already looking at, they did not feel it was helpful but instead took away from their instructional time because, “I really don’t need the Alaska Developmental Profile to help me assess my students’ skills”. Conversely, some educators felt it was very helpful, as it reminded them to look at less obvious developmental domains (like gross motor skills), helped facilitate relationship-building, or allowed them to see things that helped them identify students who might need extra support or interventions. As one participant wrote, Education Policy Analysis Archives Vol. 26 No. 142 12 It is a wonderful, developmentally appropriate way to get a quick picture of where children are as they enter kindergarten. It allows children the opportunity to meet their teacher, see their classroom for the first time, address obvious screening issues, talk with parents about developmental concerns and preschool history and decrease the level of anxiety felt by our youngest learners. I believe it is extremely beneficial to all concerned parties. This participant was not alone, as others also vocalized that they wanted to use the ADP, despite these issues, because the ADP was the only tool used to reflect the whole child and in particular, social-emotional and motor development. As Chrissy said, “It is a really helpful tool that keeps us mindful of the more holistic student” (5/18/16). Mindy also reflected that view, “ADP looks at the whole child... I think if they want to know all about the whole child, then they can look at the ADP and work from there as far as developing all their skills” (1/27/16). This was important because: Whether you’re a brand-new teacher coming in or you’re a teacher that’s been teaching for a long time and you’re being handed that tool, you really have to set your mind to looking at the child as a whole child rather than just looking at them and breaking them down into scores or language arts or math. That’s what I like about it, it really gets you right off of the bat at the beginning of the year looking at children from so many different domains. There’s so much that makes up a child’s development. (Tabitha, 5/18/16) How data is collected. The inconsistency in how the ADP was viewed may have translated into differing opinions and approaches about how to prepare for and actually collect the data. While there are instructions on how to gather information for ADP on the Alaska DEED website, not everyone seemed aware of that fact. As Teresa commented in an interview, We’ve never done any kind of training whatsoever, it’s just one of those things that they just sort of throw at us, and so we don’t even know if there is a training for it. We don’t know how it’s important, or if it’s not important, because we don’t know what they do with it. At least my colleagues at this school do not. (1/19/16) Those who knew there were directions shared differing views of how closely they followed them. This seemed to play out first in how they approached the training on how to gather information for the ADP. Some followed the webinar, others went off what was sent in a reminder email, and others simply asked their more experienced colleagues how they did it. As one administrator commented, “When a new teacher comes into the district, there’s no training around the ADP. There’s maybe a teacher in the building that can help support, but depending on who those teachers are may have a very different approach to how they implement the ADP” (Betsy, 1/29/16). This difference in approach could be due to the fact that, “...with 66 different principals you get 66 different messages right?” (Roger, 2/14/16). They also followed varying timelines and approached data collection differently. Particularly in the largest district, many educators talked about using their “ADP kit”, which they would take out during the first week of school, their “Profile Week” or late-start week. Kindergarten students would come in at varying times following a delayed entry schedule to meet with their teacher, at which point the teacher would “do the Profile on them” using the kit. They would then document the results based on what they saw that first day and then set it aside until the portal opened two months later and they could log their results. The use of the kit was so common- place that when discussing results with a district administrator, the administrator knew exactly what it was, commenting, “This is back from the good old days when we created our own…that’s what we used to do!” (Roger, 2/14/16). This was in sharp contrast to those teachers who collected data Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 13 through informal observations at multiple points in time over the first six to eight weeks and then, either looking over their notes or simply reflecting back on what they thought they saw, came up with a score. “I usually have a checklist of things that I’ve seen. I’ll have some notes. Most of it’s in my head, but if I’m like ‘Oh gosh, that kid cannot catch a ball,’ I’ll write that down because that’s not something that’s in front of me all the time” (Krislyn, 5/18/16). Concerns about the ADP. These differences in approach appeared to be connected to concerns around the ADP, much of which was related to a disagreement on what the ADP was supposed to measure—was it measuring exactly where the students come in on day one or what they are capable of learning once they are comfortable, six to eight weeks in? Was it measuring the growth that occurred over those first two months? Were the results reflective of their past experiences or what they were doing or learning now? Additionally, others felt that regardless of what it might assess, there was just too much assessing and that the ADP just added to that issue. As one participant put it, Kindergarten has so much testing right now. It is hard to find time to actually teach. We are required to provide up to seven different one-on-one tests for AIMSweb three times a year, 10 HM [Houghton Mifflin] reading tests, 13 math assessments, up to 40 tests for report card testing each quarter, plus progress monitoring each week. WE DO NOT NEED ANYMORE TESTING (survey response) The feeling that too much testing is occurring could be related to the fact that many educators felt they had to use other assessment tools because they had concerns about the ADP itself. Many commented that the ADP felt too subjective. As Janelle commented, “My biggest objective is that it is so subjective. There’s no consistent way to measure anything on there. It’s all your opinion”; she later also commented that they found big differences year to year depending on who the kindergarten teacher was and how familiar they were with the kids (6/25/16). This concern about the level of subjectivity was also reinforced when a teacher exclaimed that she was embarrassed after doing it the first year and seeing others’ results, “It looked like the rest of the state was scoring it a different way! We went, ‘Oh, whatever we’re doing here…we don’t want to be the lone ranger here!’ We just want to look like everybody else” (Janessa, 5/18/16). Participants also worried that the test was not norm-referenced and had issues related to its validity. They found the lack of inter-rater reliability to be a problem, for as Melody commented, “If you want accurate information, you have to make sure everyone is doing it the same way with the same criteria, rather than just sentences with no data points in them” (1/26/16). As Becca put it, “If we’re not using it and it’s not valid data, then we don’t want a skewed picture of what’s happening in early childhood. So, why are we doing it? And if we want to [use it], how do we make it authentic, valid data?” (1/15/16). Other concerns cited were that the ADP did not show growth, was not used to make instructional decisions, or was just a one-time assessment. There were also multiple comments about the scores being too broad. As one kindergarten teacher commented, even though she had two kindergarten students with special needs who received the same score, they were miles apart in their development. This was problematic because she really wanted a good tool that would give her more holistic data on her students, as did Wanda, a special education preschool teacher who commented, What I find in my job is that there are no really good screeners that we can use beyond the really basic ones like the ASQ... I’ve just struggled in this position trying to figure out what to use to determine where the kids are and if they’re really delayed or not - regardless of what the reason is. Regardless if it is the lack of exposure or Education Policy Analysis Archives Vol. 26 No. 142 14 whether they really need special education services, there are still not very many good screening tools out there. I was hoping this was one we could use. (1/29/16) While there was some considerable variety in many of the responses, there were also some areas in which participants seemed to be consistent—mostly in stating their concerns about what the ADP was for or how it was used. Most had no idea how the state used the data and 22 out of 30 interviewees openly commented that they, “had no idea what the ADP was used for” (Elinor, 6/25/16). “I always wondered when I’m typing things in what use the state makes of the information. Do they want the health information? Is it just going in the database where nobody looks at it? They don’t tell us” (Nancy, 1/25/16). Jeanette wished they would receive more information about it, for I’m a huge fan of assessment or surveys or any kind of data collected for a reason. I like to be sure that there’s an intention there and (know) what the goal is. What kind of information are we looking for, why do we need this, and what kind of things are we going to do once we know what the information says? (Jeanette, 5/18/16) A large number of participants also seemed to think they should be receiving something in return from the state, after entering in their data, commenting that they would not do the ADP if given a choice because “I do not receive any feedback or findings based on the information I enter”, “I don’t know who looks at the information, if anyone”, and “It seems it’s just one more piece of ‘paperwork’ and I’m honestly not sure what the information is even used for” (survey responses). As an administrator commented, “One teacher said to me last year ‘I don’t even know what they do with these results! They just go away and I never see them again,’ and I think we have a yearning to hear what happens when you compile them and what the results are. We want to hear back” (Tabitha, 5/18/16). These feelings reflect the disjunction in understanding of not just what the ADP is for but for whom it is for. The ADP as an assessment. Given the perceived lack of clarity on what it measured or what it was used for, many educators were more comfortable pointing out other assessments and tools they used outside of the ADP, including both optional and self-selected informal tools as well as district-mandated tools. Many mentioned using the AIMSweb, an often district-mandated tool that is used for universal screening, progress monitoring, and uses one-minute assessments to measure a variety of discrete skills, both in math and reading, across grades K-12. They also referenced using tools like the Measures of Academic Progress (MAPS) test, Developmental Reading Inventory (DRI), Developmental Indicators for the Assessment of Learning (DIAL), Peabody Picture Vocabulary Test (PPVT), Dynamic Indicators of Basic Early Literacy Skills (DIBELS), Teaching Strategies Gold (TSG), Children’s Progress Academic Assessment (CPAA), Alaska Measures of Progress (AMP), World-Class Instructional Design and Assessment (WIDA), Cengage created assessments, or a number of other curriculum-based assessments, teacher-created formative assessments, or related rubrics. In these cases, they knew the purpose of the assessments, what they measured, and how to use the results, which differentiated them from the ADP. Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 15 Table 2 Qualitative Codes, Themes, and Categories Emerging Selective Codes Themes Category Educators who do not submit ADP results with student growth ADP shows growth How the ADP is being used Educators who felt the ADP is looking at growth Educators who felt the ADP is only looking at where the students are coming in Educators who submit ADP results with student growth Educators who felt the ADP was helpful Use of the ADP Educators who felt the ADP was unhelpful Educators who felt the ADP helped build relationships How the ADP impacts teachers Educators who felt the ADP impacted their instruction Educators who felt the ADP helps identify students needing early interventions/extra support Educators who collected data during delayed entry Time of data collection Data collection procedures Educators who collected data longitudinally Educators who felt they were already collecting information on the ADP Approaches to data collection Educators who gathered ADP data through remembering observational experiences Educators who used note-taking strategies to collect information for the ADP Educators referencing other people to guide ADP implementation Training Educators referencing the directions to guide ADP implementation Educators who did not receive training on the ADP Educators who did receive training on the ADP Education Policy Analysis Archives Vol. 26 No. 142 16 Table 2 cont. Qualitative Codes, Themes, and Categories Emerging Selective Codes Themes Category Educators who compared the ADP to the alignment of other standards Suggestions about the ADP Concerns and suggestions about the ADP Educators who shared they would like the ADP to measure growth/include a post-test Educators who thought ADP categories were too broad/not concrete Educators uncertain how the ADP data was being used Concerns about the ADP Educators who voiced concern over an abundance of testing Educators who voiced concerns about reliability, validity, and/or consistency Educators who referred to the ADP as a screening, tool, assessment, or snapshot Type of assessment/tool Perceptions about the ADP as an assessment Educators who felt the ADP was a developmentally appropriate tool Educators who compared the ADP to other assessments Comparison to other assessments Educators who held negative AIMSweb perceptions Educators who held positive AIMSweb perceptions Educators who described preschool experiences influencing their perception of kindergarten-readiness Kindergarten readiness Approaches to teaching kindergarten Educators who discussed an academically-based perception of kindergarten-readiness Educators who discussed a “come as you are” perception of kindergarten-readiness Educators who discussed a social-emotional based perception of kindergarten-readiness Educators who perceived kindergarten-readiness as self-regulation skills Educators referring to the importance of family communication and relationship-building Teaching philosophy Educators who voiced discrepancy between teaching philosophy and practice Educators with a maturationist perspective Educators with an environmentalist perspective Educators with constructivist perspective Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 17 Exploratory Analysis on Group Differences Although not one of the primary research questions, we conducted an exploratory analysis on group differences given the differences that emerged from our quantitative closed-ended questions. In order to understand if educators’ perceptions and use of the ADP differed based on position, educational background training, or years of experience, a series of Pearson chi-square tests were conducted using the categorical responses from the closed-ended questions. No differences emerged between preschool and kindergarten teachers on whether the ADP impacted their instruction or their students (question #2 in Appendix B). However, a significant difference emerged in whether they thought the ADP was a developmentally appropriate method of gathering data, χ2 (1, 137) = 3.68, p < .05, whereas kindergarten teachers were 3.8 times more likely to agree with the statement than preschool teachers (question #5). Additionally, a statistically significant difference also emerged between participants years of experience and whether they thought the ADP was a developmentally appropriate method of gathering data, χ2 (1, 206) = 14.7, p < .001, with those over 11 years of experience being more than two times more likely to agree with the statement. Surprisingly, no differences emerged between those with a degree in early childhood versus those with a degree in elementary education on any of the three categorical variables examined. A number of potential reasons for the resulting analyses can be explained through information gathered from the interviews. In particular, the methods and procedures in which information for the ADP is gathered (i.e., what some describe or perceive as “administration”) is vastly different (e.g., over time, single point in time, prescribed materials versus play-based experiences, etc), which may in fact lead to different perceptions and use. It is also important to note that these group differences should be interpreted with caution as they are based on single-test items. Discussion With growing attention on using data to inform practice, whether at the classroom, building, district or state level, it is critical to understand the role of state-mandated KEAs at the various stakeholder levels in order to facilitate improved data informed practices (Ackerman, 2018). Further, given the history of funding sources linked to state assessment systems (Fitzgerald, 2008; Thomas & Brady, 2005), the inconsistencies in how educators’ use assessment data for instructional decision making (Mandinach, 2012; Young & Kim, 2010) and what we know about the impact of teacher perceptions (Kaniuka, 2009), it is timely to consider how these systems, and in this case a state- mandated KEA, could be better interwoven for the benefit of students, educators, and administrators. Using the framework of Jennings (1996), we therefore sought to examine educators’ perceptions and practices with using the Alaska Developmental Profile (ADP), a state-mandated kindergarten entry assessment (KEA). Using both survey and interview data, we were able to explore educator’ perceptions of this tool, and in particular if and how they use the data. Our findings from both the survey and interview data revealed that there were numerous variations in educators’ perceptions of and practices with using the ADP, including how data was collected to inform the ADP, how the data from the ADP are used, and how the ADP compares to other assessments, in addition to a multiple of concerns about the ADP raised by participants. Quantitative results revealed that less than half of the participants (42%) used the ADP information to inform instruction and 32% believed that the information gathered has an impact on their students. Further, less than half of the participants believed that the ADP is a developmentally appropriate method for gathering data, although it is intended to be used as an observational tool. Although the quantitative results noted are in reference to single answer items, they reveal a level of Education Policy Analysis Archives Vol. 26 No. 142 18 inconsistencies in educator perceptions, further evidenced by the findings from the survey and interview data. Given these inconsistencies in educator perceptions across our coded categories, it appeared that many of the responses were tied to educators not understanding the purpose of the ADP. Therefore, we explore how variations in educators’ understanding of the purpose may help to explain our findings and provide implications and context for future work. Perceptions and Practices Related to Purpose Given that the ADP is defined by the Alaska Department of Education and Early Development (2016b) as a tool used to “identify, record and summarize the skills and behaviors students demonstrate at the beginning of their kindergarten year, based on teacher observations” (p. 6), we did not specifically include a survey or interview question asking what is the purpose of the ADP? However, as we examined the data, educators’ understanding, or perhaps misunderstanding, of purpose emerged as a prominent theme. Educators’ beliefs about the purpose varied in that some believed the data from the ADP is solely used to report to the state; some believed the ADP to be a tool to help inform instruction, plan for grouping, or provide a baseline measure; others expressed they believed the data from the ADP is used to support public preschool funding; and still others reported that they had “no idea what the ADP is used for” (Elinor, 6/25/16). Yet these findings are consistent with prior research, in that teachers often report an inconsistent understanding of the role of assessments in instructional decision-making (Wininger & Norman, 2005; Young & Kim, 2010), or how data from a state-mandated assessment are linked to teacher decision-making (Ingram, Louis, & Schroeder, 2004). Further yet, some recent research on educators’ perceptions of KEAs reveal inconsistencies in how KEA data are used (Schachter, Strang, & Piasta, 2017a, 2017b) or understood (Wilson, 2015). With the variations in educators’ beliefs about the purpose it may be relevant to consider how one’s understanding of the purpose is connected to how they use the information from the ADP (i.e., inform instruction, report to the state, build relationships with students) and simultaneously the methods in which they collect data to inform the ADP. For example, educators who described that the ADP helps to inform their instruction, guide groupings, guide decisions for supports for individual children, or provide ideas for strategies in specific developmental domains all describe uses that fall under the purpose of summarizing learning and development for informing decisions about teaching and learning (Fiore, 2012; Goldstein & Flake, 2016; NAEYC, 2009). When operating from this perspective, these educators valued the data and used it in a meaningful way, and subsequently their methods of data collection were more representative of a developmentally appropriate method (i.e., observations in natural settings or play-based contexts, collected over time). For those who reported that the ADP was not used to inform instruction or build relationships, their understanding of the purpose aligned more to the idea that the ADP is for providing a snapshot of children’s skills and abilities when they enter kindergarten and is for reporting to a state accountability system. When operating from this perspective, teachers tended to put less value in its use and therefore influenced the amount of time spent on gathering the information to report on the ADP (i.e., one point in time, “testing” situation, using an “ADP created kit”). These differences in perceptions and practices speak to the need for the clarity in the purpose, as many have argued that KEAs must have a clear purpose for stakeholders and policy makers, otherwise the potential for misuse and misinterpretation is at stake, particularly if linked to high-stakes decisions (Goldstein & Flake, 2016; Regenstein et al., 2017). In a recent review of 50 state’s education agency websites, Pierson (2018) identified that most states use KEAs for at least one of the following purposes: 1) to inform classroom instruction, curriculum planning, and professional development needs, 2) to identify students in need of specialized supports or interventions, or 3) to provide a statewide snapshot of what children know Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 19 when they enter kindergarten, monitor changes over subsequent kindergarten cohorts, and inform public policy and public investments in early childhood. Other policy reports and briefs also suggest an array of possible purposes similar to Pierson’s (2018) findings (Ackerman, 2018; Krasnoff, 2015; Regenstein et al., 2017). For example, in a recent policy report, Regenstein et al. (2017) suggest that KEA results should be used to improve teaching and learning at the school level and can be valuable to policy makers in order to help strengthen state early childhood systems and effectively target resources. Yet in our study, educators’ reported discrepancies in their understandings of the purpose and their use of the ADP, along with variations in how they collected the data. So why does so much variation exist in educator’s perceptions and practices of the ADP? One explanation may be tied to the work of Jennings (1996), who described educational policy to be influenced by teacher’s experiences, perceptions and beliefs, thus suggesting that policy implementation is linked to teachers’ prior knowledge and what they may learn from it. Similarly, Cizek, Fitzgerald, and Rachor (1996) found that teachers hold and implement individual assessment policies that reflect their own values and beliefs about teaching. This was found to be true based on a number of interviewees who related their own belief about assessment to how they were using the ADP. For example, Mindy believed that assessment with kindergarteners should reflect a more authentic approach and thus viewed the ADP as a tool to help look at the whole child and provide information about the child’s areas of strengths and weaknesses. Historically speaking, teachers are often unsure of what policies are, what they are really saying, and consequently, how they should affect their teaching (Darling-Hammond, 1990). They may unknowingly subscribe to programs/methods/policies that are counter to their own values and beliefs about what is best for children without even knowing it (Edmondson, 2001). In a seminal study by Cohen and Ball (1990), they found from interviews with teachers about policy issues, that they can be untroubled by the juxtapositions often found in policies. They noted, “Many of the teachers whom we observed did change their practice in response to the new policy, but the frame for those changes was the pedagogy that had been pressed by the older policies” (p. 334). Additionally, it has been argued that innovations that are more closely aligned with current practice are more readily implemented (Rogers, 2003). Although we did not find differences between educators’ training backgrounds and how they viewed or used the ADP, another explanation plausible to consider is educators’ understanding of and practices with developmentally appropriate early childhood assessment. For example, some educators referenced the importance of the ADP as their only tool to assess the whole child, as Tabitha (5/18/16) stated “It makes you look at the child as a whole, and how the child is functioning in all areas of development, and [how] those areas of development are working together for the child to be successful.” Whereas others reported that the ADP does not provide information that they can directly link to their kindergarten standards, or use to make instructional decisions. Underlying these opposing viewpoints may be differences in their understanding and application of assessment practices with young children. Goldstein and Flake (2016) suggest that training for both administrators and teachers should focus on developments in the field of learning and development in young children as well as assessment literacy. Our findings regarding teachers’ variations in perceptions and practices point to the relevancy and importance of gathering information from teachers in relation to assessment practices, particularly if there are underlying presumed uses or purposes for the assessment. We mistakenly had assumed that since the ADP webinar training appeared to provide clear directions on how to gather and use the data plus had a relatively long history of using it (>15 years), we did not need to include questions about how they gathered data for the ADP on the survey and further yet if they understood the purpose. Yet once we started analyzing the data, it became clear there was an inordinate amount of inconsistency in how educators were approaching the ADP, which speaks Education Policy Analysis Archives Vol. 26 No. 142 20 back to Jennings’ (1996) finding that, “Policy is largely what practitioners perceive it to be rather than some external document or legislation” (p. 15). Implications Our study provides a number of contributions to the literature and its implications for the use of KEAs as it relates to teachers, administrators, policy makers, and researchers. First and foremost, stakeholders at all levels must have a clear and communicated purpose of the assessment so that the information being reported and gathered can be used in a meaningful way and align to the intended purpose. For teachers working with young children, it is critical to be familiar and versed in using formative, observation-based, assessment in natural contexts and settings (NAEYC, 2009). Our findings indicate that for kindergarten teachers in our sample, variation exists in how they gather assessment data for the ADP and what they view as useful assessment information. Given that NAEYC (2009) advocates for assessment practices that occur within natural contexts and environments, are used to understand and improve learning, and should include multiple sources of evidence over time, teachers should be competent in incorporating these assessment practices. This may require both pre-service educator preparation programs and ongoing professional development to support teachers using assessment data to improve instruction. Additionally, we hope that teachers will empower themselves in becoming more active in using the data and responding back to state-mandated assessments like the ADP. The participants in this study had been doing the ADP for up to 16 years, many of whom had no knowledge what the information was used for, which puts them in a position where they are forced to blindly go along with something that might not benefit their instruction and/or students; many thought the “results” were what the state might send back to them, as opposed to realizing that the data they collected was the most valuable component and could inform their instruction; and finally, many felt they were having to “do the ADP on the students”, which depending on their school, may have included “testing” their students on their first day of kindergarten. For some, this lead to feelings of resentment, frustration, anxiety, or powerlessness. While there are always “unintended consequences” (Marshall & Gerstl-Pepin, 2006), one’s first interaction with schooling should be empowering, not the opposite. For administrators at both the school and district levels, it is imperative that they are able to provide the professional development and ongoing support for teachers and school personnel to engage in data-driven decision making and in particular, make use of information gathered from a KEA if that is the purported purpose. Simply asking teachers, how can I help you with this, how is this data informing your instruction, or what did you learn about your students that we should know about would go a long way in helping teachers consider how a state-mandated assessment might benefit them and their students. They too, must be knowledgeable of developmentally appropriate assessment practices in order to support and validate teachers’ gathering assessment data through observation- and play-based opportunities, especially since teachers are often looking to them for guidance on how to do things a certain way. In our discussions post data-collection about how teachers might gather data for the ADP using naturalistic observations (as opposed to sitting children down with a kit on day one), teachers often asked, “But would my principal actually allow me to do that?” Teachers want guidance and support, and administrators are in a position to offer both. They also have the power to help create consistency in how state-mandated assessments are completed and used, a role that is not being fulfilled. For policy makers, it becomes clear that those involved in the development, implementation and analysis of state KEAs must agree on a common purpose. Regenstein et al. (2017) suggest that although these assessments can provide valuable information to educators and policymakers, states must be cautious that KEAs are not linked to accountability systems, and in particular the Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 21 measurement of teacher effectiveness as these practices have potentially serious negative consequences. At the state level, KEAs can provide valuable data about how children are doing when they enter kindergarten, thus providing the opportunity for states to be thoughtful in early childhood investment and use the data to help inform regional and local targeting of resources. Given the longitudinal evidence linking children’s skills at kindergarten entry with later school achievement (Duncan et al., 2007; Pagani, Fitzpatrick, Archambault, & Janosz, 2010) and the potential use of KEAs as outlined by Regenstein et al. (2017), policy makers should also be cognizant of the need for supporting KEA development and review so that it can meet rigorous standards of reliability and validity (Ackerman, 2018; Goldstein & Flake, 2016). Without this important consideration, the use of the results may be nebulous and unreliable. Policy makers should also be aware that educators want evidence that a) these assessments are valid and reliable, and b) the data are being used to inform decision-making at the state level. Many of our participants were happy to make time to collect and share the data in exchange for knowing that it was being used for good, whether it be to advocate for more publicly-funded high quality preschools or to inform community organizations on whether their interventions seemed to be having an effect on kindergarten readiness. The exchange of information must go both ways. For researchers, it may be important to examine teachers’ practices with the implementation and use of other mandated assessments, whether at the state, district or school level, as similar results regarding the variation in teachers’ practices may be found with other assessments, even despite assessments that seemingly have a clear purpose and implementation procedures (e.g., curriculum-based assessments like AIMSweb). Researchers play an invaluable role by conducting research that examines both the processes and impact of state-mandated assessments by working with educational agencies who are charged with the development of policy, while at the same time, giving previously silenced stakeholders a voice. In this study, we found that teachers had never been asked, and consequently, never shared, their perceptions of the ADP. As a result, few realized how inconsistent data was collected or used, two issues that challenge the utility and reliability of the assessment. As Melody commented, “If you want accurate information you have to make sure everyone is doing it the same way with the same criteria” (1/26/16). Recommendations like these can be voiced and shared for the common good when researchers take the initiative to start and facilitate the conversation. Limitations Several limitations must be acknowledged that hinder the generalizability of these results. First this sample relied on a convenience sample, precluding generalization of the study findings to a broader population. Given this limitation, the representation of participants across districts as described in the methods section provides confidence in a reasonably representative sample. Second, although the survey was vetted for content validity, a pilot study using the survey was not completed thus decreasing the internal reliability and opportunity for modifications. Third, this study relied on teachers’ perceptions, opinions, and attitudes related to the ADP, without us properly vetting whether they understood what the ADP was first. Fourth, there were no discussions where the responders were encouraged to explore their past experiences, current values, or how the current policy is or is not like past efforts they have adopted, all of which often affect their understanding and reaction to other policies like this one. As Cuban (1988) has found when thinking about policy changes, one must specifically deal with altering the normal, values, and beliefs that guide schools and this was not done in this study. Finally, another limitation to this study is that participants were only prompted to think about the ADP. However, as Linda Darling-Hammond (1990) has pointed out, policies do not operate within a vacuum and a number of other policies could have been in a Education Policy Analysis Archives Vol. 26 No. 142 22 state of implementation that might have impacted their perspective. Thus, there is risk in generalizing these results to the other current efforts related to kindergarten entry assessments. Conclusion It is critical that stakeholders at all levels have a clear and common understanding of the purpose of a state-mandated assessment kindergarten entry assessment (KEA), and in what ways the data are being used and by whom. With the passing of Every Student Succeeds Act, there is a growing emphasis on the need for data-driven decision making which requires a “...fundamental philosophical shift from data for compliance to the principles of data for continuous improvement” (Mandinach, 2012, p. 72). This also shifts the notions from assessment for learning rather than simply of learning. With KEAs, states must be prepared to facilitate and support the professional development needs at multiple levels, including district and school administrators and teachers in order to ensure that the resulting information is used in a purposeful and valid way that ultimately helps to improve student outcomes. As stated by Regenstein et al. (2017), “As states continue to refine their accountability and assessment policies, it is important not to misuse KEA results in ways that draw incorrect conclusion from results and potentially discredit a tool that can help support instructional improvement” (p. 2). So often educators go straight to the “doing”—the writing, the creating, the assessing, the implementing, the analyzing—before fully articulating the purpose or reason driving the action. As summarized by Teresa (1/19/16) “I think that it would help if we knew something more….If we knew there was a purpose behind it, it would make it better.” References Ackerman, D. J. (2018). Real world compromises: Policy and practice impacts of kindergarten entry assessment- related validity and reliability challenges (ETS Research Report Series No. RR-18-13). Princeton, NJ: ETS. https://doi.org/10.1002/ets2.12201 Alaska Department of Education and Early Development. (2016a). Alaska Developmental Profile Implementation Guide. Retrieved from https://education.alaska.gov/tls/Assessments/DevelopmentalProfile/Fall2016/Implementa tionGuide_July2016.pdf. Alaska Department of Education and Early Development. (2016b). Kindergarten development profile training webx. Retreived from https://stateofalaska.webex.com/mw3000/mywebex/nbrshared.do. Bagnato, S. J., Goins, D. D., Pretti-Frontczak, K., & Neisworth, J. T. (2014). Authentic assessment as “best practice” for early childhood intervention: National consumer social validity research. Topics in Early Childhood Special Education, 35, 116-129. https://doi.org/10.1177/0271121414523652 Build Initiative. (2018). Kindergarten entry assessment - KEA. Retrieved from http://www.buildinitiative.org/TheIssues/EarlyLearning/StandardsAssessment/KEA.aspx Butts, R. (2013). Recommendations of the Washington Kindergarten Inventory of Developing Skills (WaKIDS) workgroup. Olympia, WA: Office of Superintendent of Public Instruction Early Learning Office. Cizek, G., Fitzgerald, S., & Rachor, R. (1996). Teachers’ assessment practices: Preparation, isolation, and the kitchen sink. Educational Assessment, 3, 159–179. https://doi.org/10.1207/s15326977ea0302_3 Coffey, A., & Atkinson, P. (1996). Making sense of qualitative data: Complementary research strategies. Thousand Oaks, CA: Sage. Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 23 Cohen, D. K. & Ball, D. L. (1990). Relations between policy and practice: A commentary. Educational Evaluation and Policy Analysis, 12, 331-338. https://doi.org/10.3102/01623737012003331 Cuban, L. (1988). A fundamental puzzle for school reform. Phi Delta Kappan, 70, 341-344. Darling-Hammond, L. (1990). Instructional policy into practice: “The power of the bottom over the top”. Educational Evaluation and Policy Analysis, 12, 339-347. Darling-Hammond, L., & Rustique-Forrester, E. (2005). The consequences of student testing for teaching and teacher quality. Yearbook of the National Society for the Study of Education, 104, 289-319. https://doi.org/10.1111/j.1744-7984.2005.00034.x Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., & Japel, C. (2007). School readiness and later achievement. Developmental Psychology, 43, 1428–1446. http://dx.doi.org/10.1037/0012-1649.43.6.1428 Every Student Succeeds Act of 2015, Pub. L. No. 114-95 § 114 Stat. 1177 (2015-2016). Fenton, R. (2001). Anchorage kindergarten profile: Implementing the Alaska kindergarten developmental profile. Proceeding for the annual meeting of the American Educational Research Association. Retrieved from https://eric.ed.gov/?id=ED453256. Fiore, L. B. (2012). Assessment of young children. New York, NY: Taylor & Francis. Fitzgerald, J. (February, 2008). No Child Left Behind: the teachers' voice. Minnesota 2020. Retrieved from http://www.mn2020.org/assets/uploads/article/nclb.pdf. Franklin, C. A., & Snow-Gerono, J. L. (2007). Perceptions of teaching in an environment of standardized testing: voices from the field. The Researcher, 21, 2-21. Fullan, M. (2007). The new meaning of educational change (4th ed.). New York: Teachers College Press. Furrer, C., & Green, B. L. (2013). Oregon Kindergarten Assessment fall 2012 pilot process evaluation. Portland, OR: Center for Improvement of Child and Family Services, Portland State University. Glesne, C. (2006). Becoming qualitative researchers: An introduction. Boston, MA: Pearson/Allyn & Bacon. Goldstein, J., & Flake, J. K. (2016). Towards a framework for the validation of early childhood assessment systems. Educational Assessment, Evaluation, and Accountability, 28, 273-293. http://dx.doi.org/10.1007/s11092-015-9231-8 Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing young children in inclusive settings: The blended practices approach. Baltimore, MD: Paul Brookes. Hagge, S. L., & Waltman, K. (2007). Teacher perceptions of NCLB: A multi-year study. In Proceedings of the Annual Meeting of the Iowa Educational Research and Evaluation Association. Retrieved from http://www.ierea.org/2007Conference/Teacher_Perceptions_Final.pdf. Harrison, C. (2005). Teachers developing assessment for learning: Mapping teacher change. Teacher Development, 9, 255-264. https://doi.org/10.1080/13664530500200264 Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77, 81-112. https://doi.org/10.3102/003465430298487 Ingram, D., Louis, K. S., & Schroeder, R. G. (2004). Accountability policies and teacher decision- making: Barriers to the use of data to improve practice. Teachers’ College Record, 106, 1258- 1287. https://doi.org/10.1111/j.1467-9620.2004.00379.x Jennings, N. E. (1996). Interpreting policy in real classrooms: Case studies of state reform and teacher practice. New York, NY: Teachers College Press. Kaniuka, T. S. (2009). NCLB, school-based instructional policy and decision-making: a proposed research agenda. College Student Journal, 43, 787-799. Lehrl, S., Kluczniok, K., & Rossbach, H. (2016). Longer-term associations of preschool education: The predictive role of preschool quality for the development of mathematical skills through Education Policy Analysis Archives Vol. 26 No. 142 24 elementary school. Early Childhood Research Quarterly, 36, 475-488. http://dx.doi.org/:10.1016/j.ecresq.2016.01.013. Lesaux, N. K., & Marietta, S. H. (2012). Making assessment matter: Using test results to differentiate reading instruction. New York: Guilford Press. Loesch-Griffin, D., Christiansen, E., Everts, J., Englund, L., & Ferrara, M. (2014). Silver State Kindergarten Inventory Development Statewide (SSKIDS) pilot evaluation: Findings from Nevada’s users of the Teaching Strategies GOLD (TSG) assessment tool. Virginia City, NV: Turning Point, Inc. Mandinach, E. B. (2012) A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist, 47, 71-85. http://dx.doi.org/10.1080/00461520.2012.667064 Marshall, C., & Gerstl-Pepin, C. I. (2005). Re-framing educational politics for social justice. Pearson College Division. McCoy, D. C., Yoshikawa, H., Ziol-Guest, K. M., Duncan, G. J., Schindler, H. S., Magnunson, K., Yang, Rui, Koepp, A., & Shonkoff, J. P. (2017). Impacts of early childhood on medium- and long-term educational outcomes. Educational Researcher, 46, 474-487. http://dx.doi.org/10.3102/0013189X17737739. Miles, M., & Huberman, M. (1984). Qualitative data analysis: A sourcebook of new methods. Beverly Hills, CA: Sage. Nadelson, L., Briggs, P., Bubak, K., Fuller, M., Hammons, D., & Sass, M. (2012). The tension between teacher accountability and flexibility: The paradox of standards-based reform. Teacher Education and Practice, 25, 196-220. Nadelson, L.S., Pluska, H., Moorcroft, S., Annie, J., & Woodard, S. (Fall 2014). Educators’ perceptions and knowledge of the Common Core. Issues in Teacher Education, 23, 47-66. National Association of Education for Young Children. (2009). Where we stand on curriculum, assessment, and program evaluation. Retrieved from: http://www.naeyc.org/files/naeyc/file/positions/StandCurrAss.pdf National Association of Education for Young Children,& National Association of Early Childhood Specialists in State Departments of Education. (2003). Early childhood curriculum, assessment, and program evaluation - building an effective, accountable system in programs for children birth through age 8. Retrieved from: http://www.naeyc.org/files/naeyc/file/positions/pscape.pdf. National Research Council. (2008). Early childhood assessment: What, why, and how. Washington, DC: National Academies Press. Ohle, K., & Harvey, H. (2017). Educators’ perceptions of school readiness within the content of a kindergarten entry assessment in Alaska. Early Child Development and Care. http://dx.doi.org/10.1080/03004430.2017.1417855. Pagani, L. S., Fitzpatrick, C., Archambault, I., & Janosz, M. (2010). School readiness and later achievement: A French Canadian replication and extension. Developmental Psychology, 46, 984- 994. http://dx.doi.org/10.1037/a0018881. Pierson, A. (2018). Exploring state-by-state definitions of kindergarten readiness to support informed policy making. Retrieved from https://ies.ed.gov/ncee/edLabs/regions/northwest/blog/kindergarten-readiness.asp. Powell, D., Higgins, H.J., Aram, R., & Freed, A. (2009). Impact of NCLB on curriculum and instruction in rural schools. The Rural Educator, 31, 19-28. Public Education Network. (2007, July). Open to the public: How communities, parents, and students assess the impact of the NCLB Act 2004-2007, the realities left behind. Retrieved from http://files.eric.ed.gov/fulltext/ED498400.pdf. Regenstein, E., Connors, M., Romero-Jurado, R., & Weiner, J. (2017, February). Uses and misuses of kindergarten readiness assessment results (Policy Conversation No. 6). Retrieved from the Ounce Educators’ Perceptions and Use of a State-mandated Kindergarten Entry Assessment 25 of Prevention Fund: https://www.theounce.org/wpcontent/uploads/2017/03/PolicyConversationKRA2017.pdf Rogers, E. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press. Schachter, R. E., Strang, T. M., & Piasta, S. B. (2017a). Teachers’ experiences with a state- mandated kindergarten readiness assessment. Early Years. http://dx.doi.org/10.1080/09575146.1017.129777. Schachter, R. E., Strang, T. M., & Piasta, S. B. (2017b). Teachers’ perspective on the Kindergarten Readiness Assessment in year 2: Easier to administer but what role can it play in instruction? Columbus, OH: The Schoenbaum Family Center (SFC) and Crane Center for Early Childhood Research and Policy (CCEC), College of Education and Human Ecology, The Ohio State University. Smith, J. M., & Kovacs, P. E. (2011). The impact of standards-based reform on teachers: the case of NCLB. Teachers, and Teaching: Theory and Practice, 17, 201-225. https://doi.org/10.1080/13540602.2011.539802 Thomas, J. Y., & Brady, K. P. (2005). The elementary and secondary education act at 40: Equity, accountability, and the evolving federal role in public education. Review of Research in Education, 29, 51-67. http://dx.doi.org/10.3102/0091732X029001051. U.S. Department of Education. (2011). Race to the Top – Early Learning Challenge application for initial funding (CFDA No. 84.412). Washington, DC: Author. U.S. Department of Education and U.S. Department of Health and Human Services. (2014). The Race to the Top - Early Learning Challenge Year Two Progress Report. Washington, DC: Author. Wininger, S., & Norman, A. (2005). Teacher candidates’ exposure to formative assessment in educational psychology textbooks: A content analysis. Educational Assessment, 10, 19-37. https://doi.org/10.1207/s15326977ea1001_2 Young, V. M., & Kim, D. H. (2010). Using assessments for instructional improvement: A literature review. Education Policy Analysis Archives, 18. http://dx.doi.org/10.14507/epaa.v18n19.2010 Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 5/4/2018 Facebook: /EPAAA Revisions received: 9/23/2018 Twitter: @epaa_aape Accepted: 9/23/2018 Appendix A Alaska Developmental Profile Recording Form for Classroom Use Rating* Category Definition 2 Consistently Demonstrates Student demonstrates the indicated skills or behaviors on a consistent basis (80% or more of the time). Students should be given this rating if they are generally able to demonstrate these skills most of the time. Students are not required to successfully demonstrate each skill and behavior all of the time to receive this rating. 1 Progressing Student demonstrates the indicated skills or behaviors on an inconsistent basis. Students should be given this rating if they demonstrate the indicated skills or behaviors on an inconsistent basis OR if they are unable to consistently demonstrate most of the indicated skills and behaviors (i.e., for students who demonstrate only some of the indicated skills or behaviors consistently). 0 Does Not Demonstrate Student does not demonstrate the indicated skills or behaviors (20% or less of the time). Students should be given this rating if they are generally unable to successfully demonstrate these skills most of the time. Domain Presch. Physical Well-Being, Health, and Motor Development Social and Emotional Development Goal And Indicators Directions: Place an ‘X’ in the category that most appropriately reflects this student’s development with respect to each italicized goal, as defined by the bulleted list of indicators. Attend ed Pre- school Demonstrates strength and coordination of large motor muscles  Runs with an even gait and with few falls  Maintains balance while bending, twisting, or stretching  Moves body into position to catch a ball, then throws the ball in the right direction  Kicks large ball to a given point with some accuracy  Able to alternate weight and feet while skipping or using stairs Demonstrates strength and coordination of small motor muscles  Cuts, draws, glues with materials provided  Copies several letters or shapes  Able to manipulate clothing fasteners (e.g., buttons, snaps, Velcro, zippers)  Puts together and pulls apart manipulatives (e.g., blocks, beads, cubes) appropriately Participates positively in group activities  Follows simple rules of participation in group activities  Participates cooperatively in large and small group activities  Invents and/or sets up activities that include more than one child  Is sometimes part of the audience, as well as an active participant in group events Regulates their feelings and impulses  Expresses self in safe and appropriate ways (e.g., expresses anger or sadness without fights)  Shows ability to control impulses, with guidance  Seeks peaceful resolution to conflict  Modifies behavior and expression of emotions for different environments (e.g., library, recess, hallway)  Stops and listens to instructions before starting an activity Student name Y/N 2 1 0 2 1 0 2 1 0 2 1 0 http://epaa.asu.edu/ojs/ Appendix B ADP Practices Survey (Completed Online) Thank you for participating in this survey. The purpose of this survey is to obtain perceptions about the Alaska Developmental Profile, which was created and is required by the state of Alaska, and to understand how the assessment informs or affects instructional practices. Those doing this research are not associated or affiliated with the Alaska Developmental Profile in any way; they simply want to understand how it is implemented and used by stakeholders. This survey should take between five and ten minutes, is being collected anonymously, and will remain confidential. However, at the end, you will be asked if you are interested in participating in a follow-up interview to help the researchers further understand how stakeholders view the Alaska Developmental Profile and the effect the Alaska Developmental Profile has on instruction, if any. Any contact information you provide will remain confidential. Demographics To help the researchers understand who is participating, please answer the following questions: 1. What is your position? ___ Preschool Teacher ___ Kindergarten Teacher ___ School Principal ___ District Level Administrator 2. How long have you been in your position? ___ Less than a year ___ Between 1-3 years ___ Between 4-7 years ___ Between 8-11 years ___ Over 11 years 3. What is your educational background? (check all that apply) ___ CDA or Associate’s degree ___ Bachelor’s degree in early childhood education ___ Bachelor’s degree in elementary education ___ Master’s or Doctoral degree ___ Other, please describe: _________________________________ 4. If you are school-based, what type of school are you teaching in? ___ Title I ___ Non Title I 5. What school district do you work in? _______________________ Survey Questions Educators’ perceptions and use of a state-mandated kindergarten entry assessment 6. How are you involved with the Alaska Developmental Profile? 7. Does the Alaska Developmental Profile have an impact on your instructional decisions? ____ Yes ____ No If yes, how so? 8. Does the Alaska Developmental Profile have an impact on your students? ____ Yes ____ No If yes, how so? 9. What assessments do you complete with your students? 10. How does the Alaska Developmental Profile compare to those assessments in a) the method of administration and b) the results? 11. Please rate whether you agree with the following statements. ___ Yes ___ No The ADP is an assessment/tool required and used by the state ___ Yes ___ No The ADP helps you gather data on where your students are developmentally ___ Yes ___ No The ADP provides information that informs teacher’s instruction ___ Yes ___ No The ADP helps teachers build relationships with their students ___ Yes ___ No The ADP utilizes a developmentally appropriate method for gathering data ___ Yes ___ No The ADP can provide reports to administrator and district level administration ___ Yes ___ No The ADP measures children’s cognitive, physical, social, & emotional growth 12. If not required by the state, would you choose to complete or use the Alaska Developmental Profile? ____ Yes ___ Maybe ____ No Why? 13. Are there any other thoughts or suggestions you would like to share about the Alaska Developmental Profile? 14. Would you be interested in participating in an interview about this subject? ____ Yes ____ No If yes, please leave your name and contact information (phone and e-mail address) below. Thank you so much for your time! We appreciate your assistance in helping us understand more about the Alaska Developmental Profile! Education Policy Analysis Archives Vol. 26 No. 142 Appendix C Interview Questions Teacher Interview Protocol Before beginning, participants will be reminded of the elements in the consent form, including the fact that participation is voluntary and refusal to participate will not penalize them in any way; they may skip any questions they prefer not to answer; and they may stop at any point. They will also be asked to confirm it is okay to audiotape the interview. 1. How would you describe the Alaska Developmental Profile? How do you feel about it? 2. How are you involved with the Alaska Developmental Profile? How long have you been involved with it? In what capacity? How were you trained for that role? 3. What types of resources do you need to do the Alaska Developmental Profile? Do you have access to them? How long do you think you spend completing the Profile? 4. Does the Alaska Developmental Profile have an impact on your instructional decisions? If yes, how so? Do you view these as positive, negative, or neither? 5. Does the Alaska Developmental Profile have an impact on your students? If yes, how so? Do you view these as positive, negative, or neither? 6. How does the Alaska Developmental Profile compare to other assessments you administer throughout the year? 7. Are there any other thoughts or suggestions you would like to share about the Alaska Developmental Profile when it comes to how it is implemented and/or how the data is shared or used? Administrator Interview Protocols Before beginning, participants will be reminded of the elements in the consent form, including the fact that participation is voluntary and refusal to participate will not penalize them in any way; they may skip any questions they prefer not to answer; and they may stop at any point. They will also be asked to confirm it is okay to audiotape the interview. 1. How would you describe the Alaska Developmental Profile? How do you feel about it? 2. How are you involved with the Alaska Developmental Profile? How long have you been involved with it? In what capacity? How were you trained for that role? 3. What types of resources do you devote to the Alaska Developmental Profile? How else do you support your teachers as they complete it? 4. Does the Alaska Developmental Profile have an impact on your interactions with the teachers who are administering it? If yes, how so? Do you view these as positive, negative, or neither? 5. Does the Alaska Developmental Profile have an impact on decisions you make about students? If yes, how so? Do you view these as positive, negative, or neither? 6. How does the Alaska Developmental Profile compare to other assessments you require, encourage, or see administered throughout the year? 7. Are there any other thoughts or suggestions you would like to share about the Alaska Developmental Profile when it comes to how it is implemented and/or how the data is shared or used? Educators’ perceptions and use of a state-mandated kindergarten entry assessment About the Authors Hattie Harvey University of Alaska Anchorage haharvey@alaska.edu http://orcid.org/0000-0002-9869-0868 Dr. Hattie Harvey is an Assistant Professor in the Early Childhood Special Education program, School of Education at the University of Alaska Anchorage. Dr. Harvey’s research focuses on assessment to information instructional and curricular planning, and social -emotional interventions for young children with challenging behavior s. Kathryn Ohle University of Alaska Anchorage kohle@alaska.edu http://orcid.org/0000-0001-5599-1452 Dr. Kathyn Ohle is an Associate Professor in the Early Childhood program, School of Eduation at the University of Alaska Anchorage. Dr. Ohle’s research focuses on early childhood teacher’s intentional teaching and assessment practices, particularly in relation to early literacy. education policy analysis archives Volume 26 Number 142 November 5, 2018 ISSN 1068-2341 Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for non- commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by the author(s) or EPAA. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), QUALIS A1 (Brazil), PubMed, Redalyc, SCImago Journal Rank, SCOPUS, SOCOLAR (China). Please send errata notes to Audrey Amrein-Beardsley at audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. mailto:haharvey@alaska.edu http://orcid.org/0000-0002-9869-0868 mailto:kohle@alaska.edu http://orcid.org/0000-0001-5599-1452 http://www.doaj.org/ mailto:audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE Education Policy Analysis Archives Vol. 26 No. 142 education policy analysis archives editorial board Lead Editor: Audrey Amrein-Beardsley (Arizona State University) Editor Consultor: Gustavo E. Fischman (Arizona State University) Associate Editors: David Carlson, Lauren Harris, Eugene Judson, Mirka Koro-Ljungberg, Scott Marley, Molly Ott, Iveta Silova (Arizona State University) Cristina Alfaro San Diego State University Amy Garrett Dikkers University of North Carolina, Wilmington Susan L. Robertson Bristol University Gary Anderson New York University Gene V Glass Arizona State University Gloria M. Rodriguez University of California, Davis Michael W. Apple University of Wisconsin, Madison Ronald Glass University of California, Santa Cruz R. Anthony Rolle University of Houston Jeff Bale OISE, University of Toronto, Canada Jacob P. K. Gross University of Louisville A. G. Rud Washington State University Aaron Bevanot SUNY Albany Eric M. Haas WestEd Patricia Sánchez University of University of Texas, San Antonio David C. Berliner Arizona State University Julian Vasquez Heilig California State University, Sacramento Janelle Scott University of California, Berkeley Henry Braun Boston College Kimberly Kappler Hewitt University of North Carolina Greensboro Jack Schneider University of Massachusetts Lowell Casey Cobb University of Connecticut Aimee Howley Ohio University Noah Sobe Loyola University Arnold Danzig San Jose State University Steve Klees University of Maryland Jaekyung Lee SUNY Buffalo Nelly P. Stromquist University of Maryland Linda Darling-Hammond Stanford University Jessica Nina Lester Indiana University Benjamin Superfine University of Illinois, Chicago Elizabeth H. DeBray University of Georgia Amanda E. Lewis University of Illinois, Chicago Adai Tefera Virginia Commonwealth University Chad d'Entremont Rennie Center for Education Research & Policy Chad R. Lochmiller Indiana University Tina Trujillo University of California, Berkeley John Diamond University of Wisconsin, Madison Christopher Lubienski Indiana University Federico R. Waitoller University of Illinois, Chicago Matthew Di Carlo Albert Shanker Institute Sarah Lubienski Indiana University Larisa Warhol University of Connecticut Sherman Dorn Arizona State University William J. Mathis University of Colorado, Boulder John Weathers University of Colorado, Colorado Springs Michael J. Dumas University of California, Berkeley Michele S. Moses University of Colorado, Boulder Kevin Welner University of Colorado, Boulder Kathy Escamilla University of Colorado, Boulder Julianne Moss Deakin University, Australia Terrence G. Wiley Center for Applied Linguistics Yariv Feniger Ben-Gurion University of the Negev Sharon Nichols University of Texas, San Antonio John Willinsky Stanford University Melissa Lynn Freeman Adams State College Eric Parsons University of Missouri-Columbia Jennifer R. Wolgemuth University of South Florida Rachael Gabriel University of Connecticut Amanda U. Potterton University of Kentucky Kyo Yamashiro Claremont Graduate University Educators’ perceptions and use of a state-mandated kindergarten entry assessment archivos analíticos de políticas educativas consejo editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editores Asociados: Armando Alcántara Santuario (Universidad Nacional Autónoma de México), Jason Beech, (Universidad de San Andrés), Angelica Buendia, (Metropolitan Autonomous University), Ezequiel Gomez Caride, (Pontificia Universidad Católica Argentina), Antonio Luzon, (Universidad de Granada), José Luis Ramírez, Universidad de Sonora), Paula Razquin (Universidad de San Andrés) Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Ana María García de Fanelli Centro de Estudios de Estado y Sociedad (CEDES) CONICET, Argentina Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Miguel Ángel Arias Ortega Universidad Autónoma de la Ciudad de México Juan Carlos González Faraco Universidad de Huelva, España José Gregorio Rodríguez Universidad Nacional de Colombia, Colombia Xavier Besalú Costa Universitat de Girona, España María Clemente Linuesa Universidad de Salamanca, España Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Xavier Bonal Sarro Universidad Autónoma de Barcelona, España Jaume Martínez Bonafé Universitat de València, España José Luis San Fabián Maroto Universidad de Oviedo, España Antonio Bolívar Boitia Universidad de Granada, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Jurjo Torres Santomé, Universidad de la Coruña, España José Joaquín Brunner Universidad Diego Portales, Chile María Guadalupe Olivier Tellez, Universidad Pedagógica Nacional, México Yengny Marisol Silva Laya Universidad Iberoamericana, México Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Miguel Pereyra Universidad de Granada, España Ernesto Treviño Ronzón Universidad Veracruzana, México Gabriela de la Cruz Flores Universidad Nacional Autónoma de México Mónica Pini Universidad Nacional de San Martín, Argentina Ernesto Treviño Villarreal Universidad Diego Portales Santiago, Chile Marco Antonio Delgado Fuentes Universidad Iberoamericana, México Omar Orlando Pulido Chaves Instituto para la Investigación Educativa y el Desarrollo Pedagógico (IDEP) Antoni Verger Planells Universidad Autónoma de Barcelona, España Inés Dussel, DIE-CINVESTAV, México José Ignacio Rivas Flores Universidad de Málaga, España Catalina Wainerman Universidad de San Andrés, Argentina Pedro Flores Crespo Universidad Iberoamericana, México Juan Carlos Yáñez Velazco Universidad de Colima, México javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/816') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/819') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/820') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/4276') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1609') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/825') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/797') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/823') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/798') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/555') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/814') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/2703') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/801') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/826') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/802') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/3264') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/804') Education Policy Analysis Archives Vol. 26 No. 142 arquivos analíticos de políticas educativas conselho editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editoras Associadas: Kaizo Iwakami Beltrao, (Brazilian School of Public and Private Management - EBAPE/FGV, Brazil), Geovana Mendonça Lunardi Mendes (Universidade do Estado de Santa Catarina), Gilberto José Miranda, (Universidade Federal de Uberlândia, Brazil), Marcia Pletsch, Sandra Regina Sales (Universidade Federal Rural do Rio de Janeiro) Almerindo Afonso Universidade do Minho Portugal Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil José Augusto Pacheco Universidade do Minho, Portugal Rosanna Maria Barros Sá Universidade do Algarve Portugal Regina Célia Linhares Hostins Universidade do Vale do Itajaí, Brasil Jane Paiva Universidade do Estado do Rio de Janeiro, Brasil Maria Helena Bonilla Universidade Federal da Bahia Brasil Alfredo Macedo Gomes Universidade Federal de Pernambuco Brasil Paulo Alberto Santos Vieira Universidade do Estado de Mato Grosso, Brasil Rosa Maria Bueno Fischer Universidade Federal do Rio Grande do Sul, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Fabiany de Cássia Tavares Silva Universidade Federal do Mato Grosso do Sul, Brasil Alice Casimiro Lopes Universidade do Estado do Rio de Janeiro, Brasil Jader Janer Moreira Lopes Universidade Federal Fluminense e Universidade Federal de Juiz de Fora, Brasil António Teodoro Universidade Lusófona Portugal Suzana Feldens Schwertner Centro Universitário Univates Brasil Debora Nunes Universidade Federal do Rio Grande do Norte, Brasil Lílian do Valle Universidade do Estado do Rio de Janeiro, Brasil Flávia Miller Naethe Motta Universidade Federal Rural do Rio de Janeiro, Brasil Alda Junqueira Marin Pontifícia Universidade Católica de São Paulo, Brasil Alfredo Veiga-Neto Universidade Federal do Rio Grande do Sul, Brasil Dalila Andrade Oliveira Universidade Federal de Minas Gerais, Brasil