Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 1 Volume 21, Issue 1 April 29, 2020 ISSN 1099-839X Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Karee E. Dunn, University of Tennessee Denise T. Airola, University of Arkansas Takuya Hayakawa, University of Tennessee Abstract: In this study, teacher education students’ concerns, sense of efficacy, and anxiety related to the future use of data to drive educational decision-making were explored. In alignment with prior research with practicing teachers, this sample of pre-service teachers reported concerns (thoughts, preoccupations, and feelings) that indicate they are not interested in engaging in data driven decision-making (DDDM). Moreover, they had a low sense of efficacy for DDDM and high levels of anxiety for DDDM; further indicating that they are unlikely to adopt DDDM practices. We explain these results, but we go further and propose a new way of talking about data that may mitigate some of these concerns. Specifically, we propose a new paradigm for evidence-based practice in which teacher experience and intuition are deemed of equal import with data. We propose anchored judgment as an integrated decision-making model in which the intersection of teacher experience, teacher intuition, and classroom data creates the context for optimal instructional decision-making. This model is based on established research about effective decision-making in psychology, medicine, and business, and may help support the international educational mandate for DDDM. Keywords: Teacher Efficacy; Anxiety; Concerns; Data Driven Decision-Making; Anchored Judgment; Evidence-Based Practice Citation: Dunn, K. E., Airola, D. T., & Hayakawa, T. (2020). Pre-service teacher’s efficacy, anxiety, and concerns about data and the new idea of anchored judgment. Current Issues in Education, 21(1). Retrieved from http://cie.asu.edu/ojs/index.php/cieatasu/article/view/1856 Accepted: 02/05/2020 http://cie.asu.edu/ojs/index.php/cieatasu/article/view/1856 Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 2 Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Previously, our research has focused on teacher and pre-service teacher engagement in Data Driven Decision-Making (DDDM) or evidence-based practice. We have explored teachers’ and pre-service teachers’ affective and cognitive responses to the concept and to targeted training for DDDM. In this study, we focus on the critical and captive population of pre-service teachers. Specifically, we examine pre-service teacher efficacy, anxiety, and concerns related to DDDM after being introduced to a new approach to DDDM, which we will describe in this paper, anchored judgment. First, we would like to define teacher DDDM as the systematic collection of a variety of data from multiple sources, ranging from teacher observations and other behavioral data to teacher- made assessments and standardized assessments, in order to guide instructional decision-making to help students learn (Choppin, 2002; Marsh, Pane, & Hamilton, 2006; von Geel, Keuning, Vissher, & Fox, 2016) and for continuous instructional improvement (Lai & Schildkamp, 2013). Classroom-level DDDM involves identifying patterns of understanding that reveal students’ strengths and weaknesses relative to learning objectives and goals in order to refine the selection and planning of instruction and interventions to facilitate achievement of teaching and learning goals (Dunn, Airola, Lo, & Garrison, 2013a, 2013b; Walker, Reeves, & Smith, 2016). We further propose, that at this level of DDDM, DDDM teachers use learning and assessment activities to produce classroom data, close to the learning and the learner, to uncover students’ conceptual understanding and develop competencies specific to learning objectives which may be discrete content standards or more complex and interdependent competencies across multiple subject areas. Hypothetically, teachers then use these comprehensive data to assist in designing learning experiences to respond instructionally to students’ needs, as revealed by the data, and to inform stakeholders, parents, educational partners, and students about learning progress and needs. While this definition captures what classroom level DDDM includes, the research reveals that teachers do not struggle to use classroom generated data, ranging from student behavior to teacher-made assessments (Wachen, Harrison, & Cohen-Vogel, 2018). Instead, teachers often struggle to integrate the standardized data available to them throughout the school year to their instructional decision-making and planning (Athanases, Bennet, & Wahleithner, 2013; Hoogland et al., 2016; Reeves & Chiang, 2017; von der Embse, Schoeman, Kilgers, Wicoff, & Bowler, 2017). We believe that the current conception of DDDM needs to be more fully explored in both pre- and in-service teachers. Thus, the research portion of this paper, which examines pre-service teacher efficacy, anxiety, and concerns related to DDDM, focuses on pre-service teachers’ understanding of standardized assessment and proposes a framework for developing anchored judgment as a means of integrating this definition with different components of how teachers really make decisions in the classroom, relying on data, experience, and intuition. Teachers’ sense of efficacy for DDDM, or DDDM efficacy, is a teacher’s beliefs in his or her ability to successfully engage in using standardized assessment data to guide instructional decision-making in order to bring about positive student outcomes (Dunn et al., 2013b). Conversely, DDDM anxiety reflects a teacher’s “trepidation, tension, and apprehension” related to engaging in DDDM (Dunn et al., 2013b, p. 87). Concerns were defined as one’s thoughts and feelings about an instructional innovation (George, Hall, & Stiegelbauer, 2006), in this case DDDM. Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 3 We also sought to explore the influence of pre-service teacher concerns and anxiety regarding DDDM on their sense of efficacy for DDDM. This study focused on DDDM in the United States (US); however, the movement to DDDM and resistance to that movement are educational change challenges experienced around the globe (Dantow & Hubbard, 2016; Marsh et al., 2006) In the US, the 2001 No Child Left Behind (NCLB) Act was in place 15 years, placing emphasis on the results of standardized assessments as the primary indicator of student achievement (Koretz & Hamilton, 2006) and increased the use of standardized assessment results in high stakes accountability at the student, school, and national level. Subsequently, NCLB was replaced by the Every Student Succeeds Act (ESSA) of 2015. ESSA provided states with some flexibility in developing systems of accountability allowing the integration of other forms of data beyond assessment results. States were given some latitude in setting long-erm goals and measurable progress expectations. Additionally, states were able to create systems of meaningful differentiation that could include achievement, progress in achievement, school quality and student success indicators, and for high schools, the four-year adjusted cohort graduation rate. In the area of school improvement, ESSA extended beyond the requirements of NCLB to use evidence-based practice and collect data to evaluate the impact of the practice for the children in the school. This requires the school to select improvement strategies (evidence-based practices) from well- designed studies, to provide a well-formed rationale for the efficacy of the practice in the local school, and to assess the local efficacy of the practice in the classroom and school (ESSA, 2015). While these mandates exist at the policy level, we know many practicing teachers still hold strong reservations about the role of standardized assessment and DDDM in their profession and in their schools (Airola & Dunn, 2011, von der Embse, Schoemann, Kilgus, Wicoff, & Bowler, 2017). One message shared by teachers is that they feel their professional training, knowledge, and experiences as educators are placed in an inferior position to standardized data (Airola & Dunn, 2011). Recently, a teacher shared they felt “bullied by data,” noting that financial consequences attached to standardized assessment data and related negative media coverage heavily contributed to this sense of being disempowered and abused. However, from our perspective pertinent data extend far beyond standardized assessment data to include comprehensive information from a wide array of sources, which may include sources such as teacher observation, parent perspectives, behavioral data, standardized assessment, classroom assessment, exhibitions of learning, and more. The final purpose of this article is to propose a new way to talk about DDDM. We propose that the human, teacher-factor, i.e., the professional educator, must regain a central role in our discussions about teachers, students, and data. Thus, we will share the idea of anchored judgment, which borrows emphasis on experience and intuition in decision-making processes from other fields like business and healthcare. Anchored judgment will be explored in the context of the discussion of findings as a possible alternative for re-empowering teachers in the context of the data era of K-12 education. Review of the Literature Regardless of the decades of empirical evidence that support the assertion that classroom level DDDM results in positive student outcomes (e.g., Carlson, Borman, & Robinson, 2011; Edmonds, 1979; Evans, 2009; Fuller & Johnson, 2001; Lai & McNaughton, 2013; von Geel et al., 2016; Wayman, Midgley, & Stringfield, 2006; Weber, 1971) and nearly two decades after the push to incorporate DDDM in U.S. classrooms (Dunn et al., 2013a, 2013b; NCLB, 2001; Reeves & Honig, 2015) and abroad (Faber & Visscher, 2014; Schildkamp, Karbautzki, & Vanhoof, 2013; Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 4 van Geel et al., 2016; Vanhoof & Schildkamp, 2014), these practices are still not common place nor fully realized as intended (Dunn et al., 2013a; U.S. Department of Education, 2010). Many practicing teachers openly report that they primarily do not use data because they feel underprepared, and in some cases unprepared, to do so (Athanases et al., 2013; Hoogland et al., 2016; Reeves & Chiang, 2017; von der Embse et al., 2017). In their study, Wachen and his colleagues (2018) found “few teachers were able to articulate an ability to bridge the divide between using data to identify students in need of help and using data to modify instruction” (p. 296). Reeves and Chiang (2017) recently stated, “The current status of teacher capacity and teacher learning for data use have thus led to calls for enhanced attention to data use during pre-service teacher education” (p. 156). Notably, in a review of DDDM research performed prior to their focus group study on teachers and DDDM, Hoogland and her colleagues (2016) found teachers are often still lacking in data literacy and skills, critical components of teacher data use in the classroom. The teachers in their focus groups recognized DDDM is important to teaching; but, also shared that they recognize they sorely lack the requisite skills for successful classroom DDDM. Teachers also report that using data remains challenging (Athanases et al., 2013) and anxiety provoking (Dunn et al., 2013a, 2013b; Piro, Dunlap, & Shutt, 2013). Prenger and Schildkamp (2018) found that teachers’ perceptions of data and their abilities in relation to data are predictive of the use of data to guide instructional decision-making and, ultimately, the quality of their instruction. Perhaps more importantly, they found that professional development that targeted these psychological variables increased this effective data use. When training targeted teacher beliefs and attitudes towards data, teachers were significantly more likely to use data. Generally, teachers report a sense of being extremely underprepared to integrate all the standardized data in their decision-making processes with their teacher made classroom assessments (von der Embse, et al., 2017). When reviewing the requisite skills for effective data use, one teacher more poignantly stated, “When I read all of these competences, if I match them all, I should wear a t-shirt that says ‘super teacher’! That teacher does not exist” (Hoogland et al., 2016, p. 382). Reeves and Chiang (2017) emphasize that to address these contraindications of teacher readiness for classroom data use requires targeted training in pre-service teacher preparation (Mandinach & Jimerson, 2016; Reeves & Chiang, 2017). Further, this type of targeted training has improved data use in practicing teachers (Pringer & Schildkamp, 2018); thus, it is important that we begin to gain a better understanding of these variables in our pre-service teachers, to address these in teacher education programs directly, and subsequently, to better prepare them for data use in their future classrooms. The developmental trajectory of DDDM in education and the catalysts driving those changes (i.e., related policy and public perception) have not always aligned with teacher interests or pre-service education. Teachers and data have a long history that became increasingly contentious in the NCLB era (von der Embse et al., 2016). During these decades, standardization of assessment for the purpose of providing comparability among groups began to play an increasingly important role in our schools for such purposes as measuring student progress and achievement, evaluating teacher performance, and determining school- and teacher-level consequences for achievement outcomes. NCLB (2001) created a federal, systemic model that elevated the stakes of standardized assessment data use for rewarding and punishing schools with various consequences, including fiscal ones. Across this history, teachers became increasingly skeptical of the tests used to gauge student achievement and frequently perceived themselves as punished by standardized assessments and DDDM-based reforms (von der Embse et al., 2017). In addition, teacher education programs have only relatively recently begun to incorporate explicit Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 5 responsibility within programs for providing training to prepare future teachers to use data in the instructional process (Council of Chief State School Officers [CCSSO], 2011; Council for the Accreditation of Educator Preparation [CAEP], 2014). Thus, many teachers report feeling underprepared to engage with data (Reeves, 2016; Volante & Fazio, 2007; Walker et al., 2017; Wayman, 2005). Individuals are less likely to engage in practices or actions for which they lack confidence or efficacy (Bandura, 1997; Bruce, Esmonde, Ross, Dookie, & Beatty, 2010; Straub, 2009). Efficacy and anxiety are malleable variables that respond well to training (Bandura, 1997), and, when improved in teachers, research supports increased engagement in the trained area (Reeves, Summers, & Grove, 2016; Tschannen-Moran & Hoy, 2001), in this case DDDM (Airola & Dunn, 2011; Dunn et al., 2013a). In addition, when teacher concerns about an innovation are addressed, the teacher is also more likely to implement innovation related practices (George et al., 2006). Given that research supports that the effective use of data by teachers to differentiate instruction leads to positive student outcomes, it is important to better understand teachers and pre- service teachers’ beliefs and abilities related to DDDM and to identify approaches to increasing their efficacy and competencies in DDDM. To do so, it is critical to have effective tools to assess the likelihood a teacher may engage in DDDM. Previous research has validated the 3D-MEA Inventory (Dunn et al., 2013b; Walker et al., 2016) and the SoCQ as it relates to DDDM (Airola & Dunn, 2011; Dunn et al., 2013a; George et al., 2006) with in-service teachers. Research has also established a relationship between DDDM efficacy and concerns (Dunn et al., 2013a). However, no research to date has validated these measures on pre-service teachers. The current study explored the validity of the 3D-MEA Inventory and the SoCQ for DDDM with pre-service teachers, and the relationship of efficacy, anxiety, and concerns to one another in order to better understand where pre-service teachers are with regard to DDDM and to recommend next steps to helping them become effective and active data-driven decision makers in their future classrooms. 3D-MEA Inventory Dunn and her colleagues (2013b) created the 3D-MEA to assess practicing teachers’ sense of efficacy and anxiety as they pertain to DDDM. The 3D-MEA consisted of 20-items, which assessed four dimensions of teachers’ sense of efficacy for DDDM and one dimension of DDDM anxiety using a 5-point Likert scale. Dunn et al. (2013b) originally validated the instrument with a sample of 1,728 teachers from 193 Oregon schools. The participants were engaged, at varying levels, in a statewide DDDM professional development program. A split-half sampling method was utilized to explore and subsequently confirm the factor structure (n1 and n2 = 864). Participants were randomly allocated to the two subsamples. The Exploratory Factor Analysis (EFA) supported a five-factor model. The five resultant scales were Efficacy for Data Identification and Access (Identification, 3-items), Efficacy for Data Technology (Technology, 3-items), Efficacy for Data Analysis and Interpretation (Interpretation, 3-items), Efficacy for Application of Data to Instruction (Application, 6 items) and DDDM Anxiety (5-items). Confirmatory Factor Analysis (CFA) was used to test the data model fit, and the maximal reliability with 95% confidence interval was calculated to provide a robust reliability estimate (Hancock & Mueller, 2001; Raykov, 2007). The CFA results supported the validity of the measure according to Hu and Bentler’s (1999) standards for data model fit. Cronbach’s alpha for the scales ranged from .81 to .92; thus, the scales were deemed reliable. More recently, Walker and his colleagues (2016) confirmed the validity of the internal structure and reliability of the 3D-MEA Inventory with a sample of Midwestern teachers (n = 365). While the measure has been validated Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 6 with practicing teachers, no research was found that validated the 3D-MEA with pre-service teachers. The Identification Scale was designed to assess practicing teachers’ perception of their ability to identify, access, and collect suitable reports needed to connect data to their instructional practice (Dunn et al., 2013b). The Technology Scale was developed to assess teachers’ perceptions of their ability to use the available technology tools to access information for DDDM. The Interpretation Scale assessed teachers’ judgment of their ability to analyze and interpret basic components of student performance data. The Application Scale measures teachers’ perceptions of their ability to connect what data reveals to their instructional decision-making in order to tailor instruction for increased student learning. The DDDM Anxiety scale was designed to assess teachers’ “sense of trepidation, tension, and apprehension” as it pertains to their ability to understand and engage in DDDM (Dunn, 2013b, p. 93). Based on the work of Tschannen-Moran and Hoy (2001) with the Teacher Sense of Efficacy Scale (TSES), it was hypothesized that the inexperience of pre-service teachers with the nuances of classroom level DDDM would likely result in the four efficacy scales collapsing into one general efficacy scale for DDDM in lieu of four distinct scales. SoCQ Also of interest in this study was the SoCQ. Hall, Wallace, and Dossett (1973) proposed the Concerns Based Adoption Model (CBAM) as an explanation of the processes or stages teachers experience as they are asked and trained to use new practices or innovations. To assess and understand this process, Hall and his colleagues (1979) developed the SoCQ, which has been used by teacher educators and researchers alike for nearly 40 years (e.g., Cicchelli & Baecher, 1989; Dunn et al., 2013a, 2013b; Gabby, Avargil, Herscovitz, & Dori, 2017; Shotsberger & Crawford, 1999; Yan & Deng, 2018). The CBAM proposed three levels of concern that incorporate seven stages of concern measured by the SoCQ. The first level, Self Concerns, focus on the needs of the teacher and the impact of the innovation on the teacher. The first three stages of concern fall under Self Concerns (Stages 0-2). In Stage 0, or the Unconcerned Stage, the teacher may either be unaware of the innovation or lack any interest or worry related to the innovation. In Stage 1, Informational Concerns, the teacher expresses awareness of and interest in the innovation. In Stage 2, Personal Concerns, the individual is concerned about how the innovation may impact him or her, manifesting concerns about personal ability and demands of the innovation (George et al., 2006). Task Concerns comprise the second level of concerns and consists of only one stage, Stage 3 or Management Concerns. In this stage, the individual expresses concerns related to the logistics and efficient use of resources, as they relate to the innovation. The final level is Impact Concerns, which includes Stages 4, 5, and 6. These stages focus on how the innovation will impact students and colleagues. In Stage 4, Consequence Concerns, the individual is concerned about how the innovation may impact student achievement outcomes. In Stage 5 or Collaboration Concerns, the individual expresses concerns related to working with others in order to implement the innovation. In Stage 6, Refocusing Concerns, the individual possesses concerns regarding how to improve or modify the innovation and how to facilitate co-workers use of the innovation (George et al., 2006). Efficacy, Anxiety, and Concerns About DDDM The majority of the research on educators and DDDM focus on their skills and a great deal of capital has been channeled into training these skills and assessing the same (Means, Chen, DeBarger, & Padilla, 2011; von der Embse et al., 2016), but to date, this has done little to produce Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 7 teachers that are more skilled in DDDM (Dunn et al., 2013b; Deluca & Bellara, 2013). While teacher beliefs powerfully influence their practice, currently, there is a dearth of literature pertaining to teachers’ beliefs and perceptions of themselves and DDDM (Dunn et al., 2013b). The purpose of this study is to add to this nascent body of literature that explores teachers’ perceptions of their abilities, anxiety, and concerns about DDDM. The limited literature on the topic is reviewed below. Bettesworth, Alonzo, and Duesbery’s (2008) conducted a qualitative study in which they found participating practicing teachers possessed low levels of confidence in their ability to engage in DDDM. The U.S. Department of Education (2010) reported similar findings. They shared that teachers had low levels of confidence in their DDDM abilities, which decreased the likelihood they would engage in DDDM. Airola and Dunn (2011) used the 3D-MEA Inventory and reported the teachers in their sample held low levels of efficacy for DDDM. Efficacy and anxiety are often inversely related (Bandura, 1988; Gresham, 2009; Hoffman, 2010). More specifically, researchers reported that as teachers’ general sense of efficacy for teaching increases, stress decreases, and vice versa (Greenwood, Olejnik, & Parkay, 1990; Schwarzer & Hallum, 2008). In the 3D-MEA Inventory validation studies, the researchers reported an inverse relationship between DDDM efficacy and DDDM anxiety (Dunn et al., 2013b; Walker et al., 2016). Further, Dunn and her co-researchers (2013) used structural equation modeling to support their hypothesis that teachers’ anxiety influenced teachers’ DDDM efficacy, which subsequently, influenced their concerns about DDDM. No research was found that explored these variables in the critical population of pre-service teachers who are in the ideal arena in which to receive effective DDDM training that addresses not only skills, but also belief sets that drive the adoption of DDDM and the overcoming of obstacles to implementation of new DDDM practices. The current study seeks to fill this gap. Purpose of the Study In this study, we sought to complete the following tasks. First, we aimed to validate the 3D-MEA Inventory with pre-service teachers. Second, we pursued testing a valid and reliable version of the SoCQ for pre-service teachers regarding DDDM. Next, we addressed the following research question— do concerns, as measured by the DDDM SoCQ, and anxiety, as measured by the 3D-MEA Inventory, relate to a teacher’s DDDM efficacy, as measured by the 3D-MEA Inventory? Finally, we wanted to introduce the concept of anchored judgment, a new way of thinking about DDDM that may empower teachers in the current data-driven educational paradigm to reprise their professional judgment as a valued skill to use in DDDM, perhaps a first step in mending the bridge between educational reformers and researchers to educational practitioners, particularly teachers (Dunn, 2015). Method This section describes the validation of the 3D-MEA Inventory and the SoCQ regarding DDDM on pre-service teachers. Specifically, the methods used to identify the latent structure of the two measures and to assess internal consistency of each measure are described. In addition, multiple linear regression was used to identify the variance explained in DDDM efficacy by DDDM anxiety and concerns regarding DDDM and will be discussed. Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 8 Participants Respondents were 602 pre-service teachers enrolled in a Master’s of Arts in Teaching (MAT) degree program (a five-year degree program) at a large mid-southern university in an urban city. In this MAT program, students earn a baccalaureate degree from the College of Arts and Sciences and a minor in Elementary, Middle School, or Secondary Education from the College of Education at the end of the fourth year. During the fifth year, students enroll in a one-year internship to complete requirements for a teaching license and to earn graduate credits toward their master’s degree. The majority of participants were under the age of 24 (n = 503), 52 participants were aged 25 to 30 years, and the remaining 47 respondents were over the age of 31. As with many teacher education programs, the sample was primarily female (85%, n = 512) and Caucasian (91%, n = 548). The remaining ethnic breakdown of the sample was as follows, 20 were African American, two were Asian, eight were Hispanic, five were Native American, six categorized as Other, and seven participants selected, “I’d Rather Not Identify”. The majority of participants reported wanting to teach at the elementary school level (60%, n = 359), the remaining plan to teach middle school (n = 62) and high school (n = 179). Of those who reported being trained to teach something other than general elementary curriculum, participants reported wanting to teach art (n = 31), English (n = 60), Foreign Language (n = 30), Mathematics (n = 21), Technology (n = 1), or Special Education (n = 211). Procedure Students enrolled in an introductory applied educational psychology course required of all degree-seeking and licensure-seeking teacher education students were asked to participate in the study prior to completing the DDDM unit for the course. Responses were required for the course, but students could opt to not have their data included in the study. A total of 637 students were asked to complete the surveys and participate in the study. A total of 602 completed the survey and submitted a signed informed consent document, a participation rate of 95%. Measures Along with demographic items, two self-report survey instruments were used for the purposes of this study. The 3D-MEA is a relatively new measure, which has been validated, but requires further scrutiny. The SoCQ has been used for nearly 40 years; however, overtime both social change and methodological practice change require a re-examination of the factor structure of the SoCQ (George et al., 2006; Hall, George, & Rutherford, 1979). Thus, this measure also necessitates fresh evaluation. 3D-MEA Inventory The full 3D-MEA Inventory was used for this study. It consisted of 20-items and used the following 5-point Likert response format— 1 = Strongly Disagree, 2 = Disagree, 3 = Neither Agree nor Disagree, 4 = Agree, and 5 = Strongly Agree. The Identification Scale consisted of 3- items. An example item from this scale was, “I am confident in my ability to access state assessment results for my students.” The Technology Scale also consisted of 3-items. An example item from this scale was, “I am confident that I can use my district’s data analysis technology to access standard reports.” The Interpretation Scale consisted of 3-items. An example item from this scale was, “I am confident in my ability to understand assessment reports.” The Application Scale consisted of 6-items. An example item was, “I am confident that I can use assessment data to Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 9 provide targeted feedback to students about their performance or progress.” The DDDM Anxiety Scale included 5-items, which measured a respondent’s self-judgment of his or her sense of apprehension and tension related to their ability to successfully engage in DDDM. An example item was, “I am intimidated by the task of interpreting students’ state level standardized assessments.” SoCQ The SoCQ has been widely used to identify the intensity of a stage of concern related to the adoption of an innovation- an educational practice or set of practices (George et al., 2006). It consists of seven scales coinciding with each of the seven concern stages. Five statements comprised each of the seven scales in the original model. Respondents are asked to select their degree of concern based on an 8-point Likert type scale, which describes how much each statement reflects the respondent’s current feelings. For each of the 35-items, the respondent selects one value that indicates how he or she feels about the statement as well as the magnitude of that feeling (0 = “Irrelevant”, 1 to 2 = “Not true of me now,” 3 to 5 = “Somewhat true of me now,” 6 to 7 = “Very true of me now”). Participants in the current study responded to all 35-items. George and his colleagues recommend placing the name of the target innovation in place of the word innovation in the survey. Thus, innovation was replaced with DDDM in survey items for this study. The SoCQ was designed to represent varying types of concerns that teachers have when they are introduced to an educational innovation, begin to use it, and then move to more mature perspectives along with increased confidence in the use of the innovation (Negrete, 2004). “I have very limited knowledge about the innovation.” is an example of a low-level concern (George et al., 2006, p. 81). A higher score on higher-level concerns is indicative of an innovation user profile whereas a higher score on a lower level concern is indicative of a non-user profile. A high score on this suggests the individual is more likely to be a user whereas a low score indicates the respondent is not likely engaging in the innovation. An example of a higher-level concern item is, “I would like to excite my students about their part in this approach.” (George et al., 2006, p. 81). A higher score in this case indicates likely innovation engagement, and a low score would suggest the respondent is less likely to use the innovation in the classroom. For clinical use, the scores on each scale may be used to produce a profile in which the relative position of each scale’s score to the other scale scores are used to help interpret the likelihood of adopting the new innovation and to create targeted training (George et al., 2006). The SoCQ was originally validated with a group of teachers working towards adopting team teaching practices (n = 830). A sub-sample (n = 132) of this group participated in a test-retest of the instrument over a two-week period (Hall et al., 1979). Cronbach’s alpha was used to establish reliability. For the seven stages or scales, alpha coefficients ranged from .64 to .83. The test-retest sub-sample correlation ranged from .65 to .84, indicating acceptable levels of internal consistency for each of the seven stages or scales (Hall et al., 1979). George et al. (2006) validated the SoCQ using intercorrelation matrices, interview data regarding expert judgments of concerns, and confirmation of expected group differences and changes across time. The SoCQ has been used successfully in numerous studies over the course of nearly four decades (see George et al., 2006 for a detailed presentation of examples). While the SoCQ has been used with pre-service teachers (i.e., Dunn, 2016; Reeves, 2017), it is important to explore the latent factor structure for this population. No published research was Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 10 found that completed the task of assessing the stability of the factor structure in pre-service teachers. Analysis To address the first two research purposes, EFA was used to explore the factor structure and validity of the 3D-MEA and the SoCQ with pre-service teachers. Prior to conducting the factor analysis, Kaiser-Meyer-Olkin (KMO) and Bartlett’s test of sphericity were examined in order to determine if the use of EFA was appropriate with the dataset. EFA is deemed appropriate if the KMO approaches 1.0 (> 0.80 is excellent) and the Bartlett’s test is significant at the 0.05 level (Hair, Anderson, Tatham, & Black, 1995). Hair and colleagues (1995) suggested that a sample size of 100 to 400 is adequate; thus, the sample size of 602 was deemed appropriate for EFA analysis. Principal-axis factoring (PAF) with an oblique rotation was employed to study the latent constructs of the two measures. PAF is recommended as the appropriate extraction method when the assumption of multivariate normality may be violated (Costello & Osborne, 2005). When variables are expected to correlate with one another, an oblique rotation is recommended. The number of factors were not constrained in order to allow the factorial structure to emerge through the analysis. A scree test and a parallel analysis were conducted in accordance with Reise, Walker, and Comrey’s (2000) recommendation. The third research purpose was to answer the following question—Do concerns, as measured by the SoCQ, and anxiety, as measured by the 3D-MEA Inventory, predict a significant portion of the variance in DDDM efficacy, as measured by the 3D-MEA Inventory? To address this question the data were analyzed using multiple linear regression. The DDDM efficacy score was entered as the dependent variable, and concerns and anxiety were entered as the independent or predictor variables. The significance and size of the coefficient of determination were examined to determine if the independent variables had a significant influence on DDDM efficacy. Additionally, the magnitude of impact for each independent variable was examined and interpreted. Results 3D-MEA EFA The means and standard deviations from the 3D-MEA are presented in Table 1. EFA was deemed appropriate as the KMO was less than one at .90 and the Bartlett’s test of sphericity was significant (p < .05). The initial EFA results included a scree test that did not clearly support a two- or three-factor model. Thus, two-, three-, and four-factor (plus and minus one) models were explored. The parallel analysis supported a two-factor model over both the original four-factor model and the three-factor model. In the three-factor model, items loaded substantially on more than one factor and the third factor had an insufficient number of items per factor with only two- items. Items were considered to meaningfully load on a factor if the factor was 0.40 or greater and less than 0.40 on the other (Comrey & Lee, 1992; Floyd & Widaman, 1995). Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 11 Table 1 Item content and descriptive statistics Item M SD Skew Kurtosis 1. I am confident in my ability to access state assessment results for my students. 2.78 1.05 0.10 -0.90 2. I am confident that I know what types of data or reports I need to assess group performance. 2.48 0.92 0.45 -0.27 3. I am confident that I know what types of data or reports I need to assess student performance. 2.67 0.98 0.29 -0.71 4. I am confident I can use the tools provided by my district’s data technology system to retrieve charts, tables, or graphs for analysis 2.93 1.04 -0.07 -0.96 5. I am confident I can use the tools provided by my district’s data technology system to filter students into different groups for analysis. 2.80 1.02 0.12 -0.83 6. I am confident that I can use my district’s data analysis technology to access standard reports. 2.78 1.03 0.07 -0.93 7. I am confident in my ability to understand assessment reports. 3.22 0.96 -0.48 -0.42 8. I am confident in my ability to interpret student performance from a scaled score. 3.10 0.96 -0.38 -0.72 9. I am confident in my ability to interpret subtest or strand scores to determine student strengths and weaknesses in a content area. 3.20 0.98 -0.44 -0.69 10. I am confident that I can use data to identify students with special learning needs. 3.14 1.02 -0.27 -0.74 11. I am confident that I can use data to identify gaps in student understanding of curricular concepts. 3.16 0.94 -0.47 -0.66 12. I am confident that I can use assessment data to provide targeted feedback to students about their performance or progress. 3.20 0.93 -0.56 -0.48 13. I am confident that I can use assessment data to identify gaps in my instructional curriculum. 3.16 0.97 -0.40 -0.71 14. I am confident that I can use data to group students with similar learning needs for instruction. 3.31 0.91 -0.66 -0.24 15. I am confident in my ability to use data to guide my selection of targeted interventions for gaps in my students’ understanding. 2.90 0.95 0.01 -0.68 16. I am intimidated by statistics.* 2.89 1.12 0.09 -0.93 17. I am intimidated by the task of interpreting students’ state level standardized assessment data.* 3.06 1.01 -0.89 -0.96 18. I am concerned that I will feel or look “dumb” when it comes to data driven decision-making.* 2.95 1.07 0.10 -0.88 19. I am intimidated by my district’s data retrieval technology.* 3.10 0.83 0.02 0.05 20. I am intimidated by the process of connecting data analysis to my instructional practice.* 3.07 0.97 -0.12 -0.69 Note: Items originally published by Dunn et al. (2013, p. 92). *Reverse coded items. Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 12 The efficacy scales collapsed into one scale, the DDDM Efficacy Scale (15-items), and the anxiety items remained on the second scale, DDDM Anxiety Scale (5-items). Table 2 presents the factor loadings and communalities from the two-factor solution. The two latent factors accounted for 100% of the variance as compared to 93% with three factors. These factors were identified as the following scales of the measure (see Table 2). The DDDM Efficacy Scale assessed pre-service teachers’ perceptions of their abilities to successfully engage in DDDM to guide instructional decision-making. The DDDM Anxiety Scale assessed teachers’ self-judgment of their trepidation, tension, and apprehension related to their ability to successfully engage in DDDM (Dunn et al., 2013a, 2013b). Reliability (Cronbach’s Alpha) was calculated for each of the two scales, DDDM Efficacy Scale (0.93) and DDDM Anxiety Scale (0.86). These correlation coefficients indicated that the pre-service teacher version of the 3D-MEA Inventory (P3D-MEA) exhibited strong internal consistency. Table 2 Exploratory factor analysis two-factor solution: Oblimin rotation Item F1 F2 Factor loadings 12 .79 15 .76 13 .75 11 .73 9 .72 14 .72 8 .71 7 .69 6 .69 5 .67 4 .66 10 .65 3 .62 2 .62 1 .61 17 .80 18 .76 20 .75 19 .73 16 .67 Factor Correlation F1 F2 -0.16 Note: Items originally published by Dunn et al. (2013, p. 92). *Reverse coded items. SoCQ EFA EFA was deemed appropriate as the KMO was less than one at .90 and the Bartlett’s test of sphericity was significant (p < .05). The initial EFA results included a scree test supported a one-factor model consisting of nine items. Items were considered to meaningfully load on a factor Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 13 if the factor was 0.40 or greater and less than 0.40 on the other (Comrey & Lee, 1992; Floyd & Widaman, 1995). The means and standard deviations of the retained SoCQ items are presented in Table 3. Table 3 presents the factor loadings from the one-factor solution. The two latent factors accounted for 100% of the variance as compared to 93% with two factors. For pre-service teachers without practical experiences, it is common for teaching related scales to collapse into one factor (Tschannen-Moran & Hoy, 2001). This factor was identified as the Pre-Service Data Concerns scale. This factor assessed pre-service teachers’ concerns, or thoughts, preoccupations, and desires, about learning more about DDDM and how to use it. Cronbach’s Alpha was calculated to be 0.89 for this scale, indicating the scale has strong internal consistency. Table 3 SoCQ item descriptive statistics and factor loading Original SoCQ Item No. M SD Skew Kurtosis Factor Loading 10. 4.58 2.19 -0.27 -0.96 0.69 14. 4.81 1.83 -0.40 -0.33 0.62 15. 5.33 1.89 -0.75 0.62 0.71 19. 5.14 1.94 -0.42 -0.48 0.56 26. 5.42 1.80 -0.66 0.13 0.64 27. 4.54 2.00 -0.27 -0.67 0.75 28. 5.28 1.92 -0.57 -0.22 0.69 33. 5.44 1.96 -0.72 -0.06 0.70 35. 5.48 2.08 -0.72 -0.29 0.65 Note: Items available in George et al. (2006). Multiple Linear Regression Multiple linear regression was used to determine if concerns regarding DDDM and DDDM anxiety significantly explain the variance in pre-service teachers’ sense of efficacy regarding DDDM. The DDDM efficacy score was entered as the dependent variable, and DDDM concerns and anxiety were entered as the independent variables. The means, standard deviations, and correlations among all the variables are presented in Table 4. Table 4 Correlation, means, and standard deviations for DDDM Concerns, DDDM Anxiety, and DDDM efficacy DDDM Scale 1 2 3 M SD (1) Concerns 1.00 46.00 12.99 (2) Anxiety 0.15 1.00 15.07 4.03 (3) Efficacy 0.17 -0.19 1.00 44.01 10.51 The two independent variables were entered simultaneously into the regression model. Preliminary examination of the results indicated no extreme multicollinearity in the data (variance inflation factors were less than five), and the assumptions underlying the application of multiple linear regression (i.e., independence, normality, heteroscedasticity, and linearity) were met. The Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 14 regression results indicated the set of independent variables significantly explained 7.7% of the variance in pre-service teacher DDDM Efficacy (F(2, 600) = 21.28; p < .001). The order of influence of the variables was DDDM Anxiety (t = -4.89; p < .001) and DDDM Concerns (t = 4.39; p < .001). Discussion Using DDDM in the classroom is an imperative for current and future teachers, supported by research literature as good instructional practice (Carlson et al., 2011; Lai & McNaughton, 2013; van Geel et al., 2016), required by credentialing agencies (CAEP, 2014; CCSSO, 2011; Means et al., 2011), and mandated by leaders at the federal, state, district, and school level (NCLB, 2001; Walker et al., 2016) and globally (Faber & Visscher, 2014; Vanhoof & Schildkamp, 2014). Yet supporting, requiring, and mandating DDDM has not led to tremendous integration of DDDM in classroom teaching practice. One argument with empirical support is that even when teachers are trained to use data, negative perceptions of standardized assessment, anxiety related to DDDM, concerns about its effect on students, schools, and personal evaluations, and doubts in their skills and abilities to engage in DDDM remain as significant obstacles to DDDM reform efforts gaining traction in classrooms (Dunn et al., 2013a, 2013b; Reeves et al., 2016; Volante & Fazio, 2007). While some researchers have helped better prepare teachers to use data (Lai & McNaughton, 2013; van Geel et al., 2016; Vanhoof & Schildkamp, 2014), the majority of teachers find many challenges to utilizing standardized data in their instructional planning (Athanases et al., 2013; Hoogland et al., 2016; Reeves & Chiang, 2017; von der Embse et al., 2017, Wachen et al., 2018). Reeves and his colleagues have done significant work connecting teachers’ DDDM efficacy to the use of data to drive instruction. Specifically, when teachers’ sense of efficacy is low, they found teachers are less likely to ground decision-making with data (Reeves et al., 2016; Walker et al., 2016). As teacher education is an ideal arena in which to prepare future teachers in DDDM by bolstering their related knowledge, skills, and efficacy, it is imperative that we have the necessary tools to understand these variables and continue to expand the small body of literature related to this population (Deluca & Bellara, 2013; Reeves, 2017; Reeves & Honig, 2015; Reeves et al., 2016). This study found the 3D-MEA to be a valid and reliable tool for assessing teachers sense of efficacy and anxiety, but the four efficacy scales collapsed into one DDDM Efficacy scale. This finding aligns with the expectation of the researchers and the previous work of Tschannen-Moran and Hoy (2001) in which the TSES’ subscales collapsed into one efficacy scale for pre-service teachers. We propose, as did Tschannen-Moran and Hoy, that pre-service teachers’ inexperience with the nuances of classroom practice are less able to distinguish their perceptions of their abilities in an idiosyncratic way, but they are more than capable of evaluating their overall abilities to teach. While Dunn and her colleagues (2013a) and Walker and his colleagues (2016) have validated the 3DM-EA Inventory with practicing teachers, this was the first study to validate the measure with pre-service teachers. The DDDM Efficacy scale assessed pre-service teacher’s self- evaluation of his or her ability to successfully engage in DDDM to bring about positive student outcomes. The DDDM Anxiety scale assessed pre-service teachers’ sense of “trepidation, tension, and apprehension” related to engaging in DDDM (Dunn et al., 2013b, p. 87). The EFA findings for the SoCQ were similar and interesting. We propose that the seven scales collapsed into one scale due to lack of experience with the unique facets of classroom practices. Moreover, many items did not load onto any one factor clearly because the pre-service teachers’ do not yet understand the related aspects assessed or are even aware of the related issues. Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 15 For example, this sample of teachers did not respond in a consistent pattern to SoCQ items such as, “I would like to know the effect of DDDM on my professional status.” Perhaps this was because these students are not even considering their status in the profession, as they have not yet formally joined the profession. However, they did respond in a patterned fashion to, “I am concerned about evaluating my impact on students”. A likely explanation to why this item loaded onto the DDDM Concerns scale is that a primary focus of teacher preparation course work pertains to how to best help students learn and grow. The validation of the DDDM Concerns scale for pre-service teachers adds an important tool in better understanding how likely these future teachers are to engage in DDDM. Understanding concerns not only helps us understand how likely these future teachers are to use an innovation, it also provides invaluable information regarding how to tailor instruction to address concerns and improve the likelihood of innovation engagement (Dunn, 2016; Dunn et al., 2013a; Gabby, Avargil, Herscovitz, & Dori, 2017; George et al., 2006; Shotsberger & Crawford, 1999; Yan & Deng, 2018). The DDDM Concerns scale assessed pre-service teachers’ thoughts, preoccupations, and desires pertaining to learning more about DDDM and how to successfully engage in DDDM in their future classrooms. The findings of this study not only validate measures of pre-service teachers’ efficacy, anxiety, and concerns, but also reveal this sample of pre-service teachers did possess relatively low levels of DDDM efficacy (M = 15.07), high levels of DDDM anxiety (M = 15.07, lower scores indicate greater anxiety), and only a moderate level of concerns about learning more about DDDM (M = 46.00). While the examination of pre-service teachers’ concerns about DDDM is a unique addition to the literature, the findings of low DDDM efficacy and high DDDM anxiety aligns with pervious work with in-service teachers (Dunn et al., 2013a, 2013b; Walker et al., 2016). Finally, the findings of this study revealed DDDM anxiety and concerns are predictive of pre-service teachers’ DDDM efficacy, which Reeves and his colleagues (2016) found to predict teacher engagement in DDDM. Dunn and her colleagues (2013) reported similar findings in which structural equation modeling revealed teachers’ DDDM anxiety was predictive of their DDDM concerns, and subsequently, concerns were predictive of their DDDM efficacy. The current study reveals that anxiety and concerns are significantly predictive of DDDM efficacy, but to a lesser degree than with practicing teachers. This may be because of the aforementioned lack of experience and nuanced understanding of classroom teaching. Yet, these results do indicate it is important that teacher educators identify and address pre-service teacher DDDM anxiety and concerns in order to improve their DDDM efficacy, which has been connected to teacher use of data to anchor decision-making (Reeves et al., 2016). In order to decrease anxiety, address concerns, increase efficacy, and ultimately increase the likelihood pre-service teachers’ will become data driven teachers, we would like to recommend a new direction in DDDM that empowers teachers to ground decision-making with data, but to also value their professional training and experience, coupled with their intuition. Teachers have been disempowered by extreme external oversight and control in their classroom, and advocates of DDDM often further downgrade the role of what a teacher knows in decision-making. The assessment industry, seeking to make their products more desirable, have evolved their standardized reports of student achievement to include more information in a variety of ways with the intent of increasing the utility of the standardized assessments at the classroom level and easing the burden of interpretation and valid use. These efforts increase the complexity of reporting, necessarily, which also impacts teachers’ efficacy and anxiety in use. As a result, DDDM is a phrase that often unintentionally causes a visceral reaction in teachers, making its four- Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 16 letter acronym exceedingly unpopular. Thus, we suggest a new approach to DDDM labeled anchored judgment in which ideal judgment occurs at the intersection of intuition, professional experience, and data, (See Figure 1), an approach supported by and used in research in other fields such as medicine and business. Figure 1. Anchored Judgment - intersection of teacher intuition, teacher experience, and available data A number of studies from various fields have compared purely intuitive models to purely statistical models and found the former lacking in accuracy (e.g., Dawes, Faust, & Meehl, 1989; Whitecotton, Sanders, & Norris, 1998). Such dichotomous perspectives of intuition have proven less than useful in other fields such as management (Blattberg & Hoch, 1990; Dane & Pratt, 2007) and healthcare (Robert, Tilley, & Peterson, 2014; Whitecotton et al., 1998). Researchers in these fields have found, in information-rich contexts, a combination of quantitative data and intuition is the most powerful decision-making approach, as it is driven by experience, qualitative information, and quantifiable data. Anchored Judgment What if we reconceptualized DDDM from the classroom up rather than the summative assessment down? Qualitative information in the classroom which arises from demonstrations of learning through teacher and/or student designed learning tasks and assessments, results in directly observable data, specific to each learner’s progress and/or attainment of concepts, skills, and competencies in one or more learning progressions. These qualitative data, which are distinctly different from data obtained through standardized assessments, inform the answers to questions about students’ learning within the moment of students’ attempts to learn. These learning and assessment activities result in student-produced evidence that helps to uncover the extent to which each student has developed conceptual understanding, as well as demonstrated sufficient evidence of attaining competencies specific to each student’s learning progression(s). Teachers then use this more visible, student-centered set of data to assist in designing learning experiences to respond to students’ needs, as revealed by the data, and to engage students and their parents in discussions about learning progress. This leads to more question-informed data Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 17 collection to identify supports needed to make progress or to assist in integrating across learning progressions. It also leads to student-contributed ideas for how to meet those needs or what new challenges or steps the student may be ready to take on next. These qualitative, concrete data become a critical part of developing anchored judgment through a combination of experience, intuition, and reflection on these question- and learning- informed data. Data from standardized sources can be used to assess the veracity of the teachers’ inferences based on the directly observable data and to calibrate the content and cognitive rigor of the tasks they are designing for student learning and assessment. We assert that this iterative process of reflection on inferences drawn from the concrete classroom data, inserted and integrated into the DDDM process, over time, results in teachers learning how to calibrate their experience and intuition with more standardized data –leading to anchored judgment. And, as they anchor their judgment about student learning in both qualitative and quantitative sources of data the likelihood of changed practice in response to what they learn from DDDM is increased. Anchored judgment as new way to frame DDDM pre-service and in-service teachers has the potential to build their self-efficacy for using classroom data to inform instruction. Effectiveness of anchored judgment is expected to grow with more experience and professional development. When embedded in a data-rich environment, anchored judgment, or the coupling of intuition, experience, and more quantifiable data, increases the accuracy of judgment (Blattberg & Hoch, 1990; Robert et al., 2014; Whitecotton et al., 1998). We define teacher intuition as a coalescence of extant knowledge and affect, grounded in current observations of class and individual-level unique behaviors and patterns of behaviors that may guide in-the-moment decision-making and inform strategic planning. Teacher professional experience includes the wide array of past, stored experiences related to teaching, ranging from experiences as students, experiences in teaching, and professional development. While we do not naively expect pre-service teachers to be highly skilled, we do propose that approaching DDDM from this perspective with pre-service or in-service teachers should illicit less anxiety and quell concerns, while providing a paradigm that facilitates rather than inhibits teachers’ sense of efficacy. Teachers often feel pressured to focus on standardized test outcomes for DDDM and rightly believe that tests provide incomplete and, sometimes, inaccurate descriptions of their students (Ingram, Louis, & Schroeder, 2004; Pandina Scot, Callahan, & Urquhart, 2008). Moreover, teachers and future teachers have seen data misused to force an innovation through, to hide facts, or to misrepresent reality, making it “difficult to treat facts as their friends” (Ingram et al., 2004, p. 1276). Thus, statistics can be used to mislead and teachers have felt the sting of these numerical violations. However, intuition is not always accurate. Evidence suggests statistical models are sometimes more accurate than individual intuition when directly compared (Dawes et al., 1989; Miller & Ireland, 2005; Whitecotton et al., 1998), but teachers are not blindly unaware of the fallibility of their intuition (Ingram et al., 2015; Johansson & Kroksmark, 2004). If numbers can be misused or fail to capture the entire spectrum of information and intuition is often less accurate than numbers, on what should teachers depend? We propose the answer is finding an appropriate system of checks and balances between intuition and quantitative data dependent upon contextual variables. Yanniv and Hogarth (1993) found that when individuals have an information rich context, like the K-12 classroom, in which to make decisions, they outperformed solely statistical models. In education research, Soland (2013) found teachers made similarly accurate predictions of dropout rates when compared to statistical models because they relied on their judgments of students’ academic tenacity based on daily observations. Research Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 18 from other fields and the limited findings in education suggest the model of anchored judgment has the potential to improve teacher perspectives of DDDM, teacher decision-making, and student outcomes. Limitations and Future Research The current study was limited in three ways. First, the sample was a convenience sample. Future research should look to validate these measures using random sampling. In addition, the participants were all from one mid-southern university. The measure should be validated using random sampling from a variety of institutions using different teacher education programming. For example, at the institution studied, students earn a five-year master’s degree, not a four-year undergraduate degree in education. Finally, participants in this study were relatively homogeneous, being primarily Caucasian females. Thus, future validation work with the current measures should seek a more heterogeneous sample of pre-service teachers. Future researchers should also consider investigating the efficacy of the anchored judgment model of DDDM instructional model for students. While the benefit of similar models has been supported in other fields such as business (Dane & Pratt, 2007; Miller & Ireland, 2005) and nursing (Crumlish & Kelly, 2009; Dinesh, 2008; Falzer & German, 2009; McGrath, 2008; Robert et al., 2014; Welsh & Lyons, 2001), it has not been tested widely in education (Åsvoll, 2012; Soland, 2013). Additionally, future research should explore, if this model is more readily acceptable to in- service and pre-service teachers as compared to a pure DDDM instructional model. Finally, future research should replicate Prenger and Schildkamp’s (2018) study exploring the impact of targeted training on teacher psychological reactions to data and their use of data. The profile should extend to include concerns, and the model of anchored judgment should be included in the curriculum to explore if this model moves the psychological profile toward that of an effective teacher data-user profile. Conclusions The current pre-service teacher findings of high anxiety and concerns with low DDDM efficacy levels that require improvement to indicate the likelihood that we have prepared them to engage in DDDM align with past research on in-service teachers (Bettesworth et al., 2008; Dunn et al., 2013a, 2013b; U.S. Department of Education, 2010). Additionally, the results suggest that DDDM anxiety and concerns are predictive of pre-service teachers’ DDDM efficacy, which has been found to predict engagement in DDDM (Reeves et al., 2016; Ringer & Schildkamp, 2018). This pattern was also similar to previous research findings in practicing teachers (Dunn et al., 2013). There is much more we need to learn about pre-service teachers and DDDM preparation. The two validated measures should be useful to those who wish to expand our understanding and to those who wish to use data to drive their DDDM instruction in teacher education courses. Further, it is important that we scientifically study the impact of targeted intervention on these variables in pre-service teachers and test if this type of training increases new teacher use of data in the classroom, as Pringer and Schildkamp (2018) found such training increased teacher data use and instructional quality. In summation, there is a great deal more we need to learn about pre-service teachers and DDDM, but as we see patterns of anxiety and concerns manifesting in similar patterns to practicing teachers, we propose that it is time we consider a new approach to DDDM. After nearly two decades of heavy emphasis on evidence-based practice in education with little traction gained, perhaps, we should move from a solely data-based approach to one that empowers teachers to Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 19 value their professional training and experience but also to anchor these decisions to the multitude of available student data. By no means do we suggest DDDM should be forgotten, just incorporated into a new paradigm of anchored judgment. As often happens in educations iterative reform process, the pendulum has swung from teacher-driven to data-driven across the past 60 years. Perhaps in the new millennium it should come to a rest at anchored judgment, a more centered approach on this continuum. The ESSA (2015) specifies requirements for the use of evidence-based practices. Under ESSA, state and local education agencies that plan to use their federal education dollars to support activities or interventions for struggling students must justify their selection of interventions based on a tiered, strength-of-evidence model. This creates a need for pre-service and in-service teachers to engage in new or deeper learning of statistical information and research design in order to evaluate the possibility of a particular intervention or activity and the impact of any implemented practice for their students. Anchored judgment has the potential to help teachers reduce anxiety and concerns related to using data and other statistical information when situating new learning within their professional experience and current work context. References Athanases, S. Z., Bennett, L. H., & Wahleithner, J. M. (2013). Fostering data literacy through preservice teacher inquiry in English language arts. The Teacher Educator, 48(1), 8–28. Airola, D. T., & Dunn, K. E. (2011). Oregon DATA project final evaluation report. Fayetteville, AR: Next Level Evaluation, Inc. Åsvoll, H. (2012). Perspectives on reflection and intuition in teacher practice: a comparison and possible integration of the cognitive constructivist and the Dreyfusian intuitive perspectives. Reflective Practice: International and Multidisciplinary Perspectives, 13, 789-804. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215. Bandura, A. (1988). Self-efficacy conception of anxiety. Anxiety Research, 1, 77-98. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman and Company. Bettesworth, L. R., Alonzo, J., & Duesbery, L. (2008). Swimming in the depths: Educators’ ongoing effective use of data to guide decision making. In T. J. Kowalski & T. J. Lasley (Eds.), Handbook on data-based decision making in education (pp. 286-303). New York, NY: Routledge. Blattberg, R. C., & Hoch, S. J. (1990). Database models and managerial intuition: 50% model + 50% manager. Proceedings of the National Academy of Sciences, 36, 887-899. Bruce, C. D., Esmonde, I., Ross, J., Dookie, L., & Beatty, R. (2010). The effects of sustained classroom-embedded teacher professional learning on teacher efficacy and related student achievement. Teaching and Teacher Education, 26(8), 1598-1608. Carlson, D., Borman, G., & Robinson, M. (2011). A multistate district-level cluster randomized trial of the impact of data-driven reform on reading and mathematics achievement. Educational Evaluation and Policy Analysis, 33, 378-398. Choppin, J. (2002, April). Data use in practice: Examples from the school level. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Cicchelli, T., & Baecher, R. (1989). Microcomputers in the classroom: Focusing on teacher Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 20 concerns. Educational Research Quarterly, 13(1), 37-46. Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis. New York: Academic Press. Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(7), 173-178. Retrieved from http://pareonline.net/getvn.asp?v=10&n=7 Council for the Accreditation of Educator Preparation. (2014). Strategic plan. Retrieved from http://caepnet.org/about/strategicplan/ Council of Chief State School Officers. (2011). InTASC model core teaching standards 2011. Washington, D.C: Council of Chief State School Officers. Retrieved from: http://www.ccsso.org/resources/publications/intasc_model_core_teaching_standards_201 1_ms_word_version.html Crumlish, N. & Kelly, B. D. (2009) How psychiatrists think. Advances in Psychiatric Treatment, 15(1), 72-79. Dane, E., & Pratt, M. G. (2007). Exploring intuition and its role in managerial decision making. Academy of Management Review, 32(1), 33-54. Dantow, A., & Hubbard, L. (2016). Teacher capacity for and beliefs about data-driven decision making: A literature review of international research. Journal of Educational Change, 17(1), 7-28. Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668-1674. Deluca, C., & Bellara, A. (2013). The current state of assessment education: Aligning policy, standards, and teacher education curriculum. Journal of Teacher Education, 64, 356-372. Dinesh, B. (2008) Decision-making in psychiatry: what can we learn? Acta Psychiatrica Scandinavica, 118(1), 1-3. Dunn, K. E. (2015). The scientist-practitioner: A Boulder Model for education. International Journal of Adult Vocational Education and Technology, 6, 40-45. Dunn, K. E. (2016). Educational psychology’s instructional challenge: Pre-Service Teachers’ Concerns Regarding Data Driven Decision-Making Practices. Psychology Learning and Teaching, 15, 31-33. Dunn, K. E. (2015). The scientist-practitioner: A Boulder Model for education. International Journal of Adult Vocational Education and Technology, 6, 40-45. Dunn, K. E., Airola, D. T., Lo, W., & Garrison, M. (2013a). Becoming data driven: The influence of teachers’ sense of efficacy on concerns related to data-driven decision making. The Journal of Experimental Education, 81, 222-241. Dunn, K. E., Airola, D. T., Lo, W., & Garrison, M. (2013b). What teachers think about what they can do with data: The development and validation of the Data Driven Decision-Making Efficacy Survey. Contemporary Educational Psychology, 38(1), 87-98. Edmonds, R. (1979). Effective schools for the urban poor. Educational Leadership, 37, 15-27. Evans, A. (2009). No Child Left Behind and the quest for educational equality: The role of teachers’ collective sense of efficacy. Leadership and Policy in Schools, 8, 64-91. Every Student Succeeds Act, 114 U.S.C. § 1177 (2015). Faber, J. M., & Visscher, A. J. (2014). Digitale leerlingvolgsystemen: Een review van de effecten op leerprestaties. 4W. Retrieved from http://www.kennishnet.nl/onderzoek/alle- onderzoeken/digitale-leerlingbolgystemen-een-review-van-de-effecten-op-leerprestaties/ Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 21 Falzer, P. R., & Garman, M. D. (2009). A conditional model of evidence-based decision making. Journal of Evaluation in Clinical Practice, 15, 1142-1151. Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286-299. Fuller, E. J., & Johnson, J. F. (2001). Can state accountability systems drive improvements in school performance for children of color and children from low-income homes? Education and Urban Society, 33, 260-283. Gabby, S., Avargil, S., Herscovitz, O., & Dori, Y. J. (2017). The case of middle and high school chemistry teachers implementing technology: Using the concerns-based adoption model to assess change processes. Chemistry Educational Research and Practice, 18, 214-222. George, A. A., Hall, G. E., & Stiegelbauer, S. M. (2006). Measuring implementation in schools: The Stages of Concern Questionnaire. Austin, TX: Southeast Educational Development Laboratory. Greenwood, G. E., Olejnik, S. F., & Parkway, F. W. (1990). Relationships between four teacher efficacy belief patterns and selected teacher characteristics. Journal of Research and Development in Education, 23, 102-106. Gresham, G. (2009). An examination of mathematics teacher efficacy and mathematics anxiety in elementary pre-service teachers. Journal of Classroom Interaction, 44(2), 22-38. Hair, J. F. J., Anderson, R. E., Tatham, R. L., & Black, W. C. (1995). Multivariate data analysis (4th ed.). Saddle River, NJ: Prentice Hall. Hall, G. E., George, A. A., & Rutherford, W. L. (1979). Measuring Stages of Concern about the innovation: A manual for use of the SoC Questionnaire. Austin: University of Texas. Hall, G. E., Wallace, R. C., & Dossett, W. A. (1973). A developmental conceptualization of the adoption process within educational institutions. Austin, TX: Research and Development Center for Teacher Education, The University of Texas. Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, S. H. C. du Toit, & D. Sorbom (Eds.), Structural equation modeling: Past and present. A Festschrift in honor of Karl G. Joreskog (pp.195-261). Chicago: Scientific Software International. Hoffman, B. (2010). “I can, but I’m afraid to try”: The role of self-efficacy beliefs and mathematics anxiety in mathematics problem-solving efficiency. Learning and Individual Differences, 20(3), 276-283. Hoogland, I., Schilkamp, K., van der Kleij, F., Heitnik, M., Kippers, M., Veldkamp, B., & Dijkstra, A. M. (2016). Prerequisites for data-based decision making in the classroom: Research evidence and practical illustrations. Teaching and Teacher Education, 60, 377- 386. Hu, L., & Bentler, P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 10, 435- 455. Ingram, D., Louis, K. S., & Schroeder, R. G. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers College Record, 106, 1258-1287. Johansson, T., & Kroksmark, T. (2004). Teachers’ intuition-in-action: How teachers experience action. Reflective Practice, 5, 357-381. Koretz, D. M., & Hamilton, L. S. (2006). Testing for accountability in K-12. In R. I. Brennan (Ed.), Educational measurement (4th ed., pp. 531-578). Westport, CT: American Council Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 22 on Education/Praeger. Lai, M. K., & McNaughton, S. (2013). Analysis and discussion of classroom and achievement data to raise student achievement. In K. Schildkamp, M. K. Lai, & L. Earl (Eds.), Data- based decision making in education: Challenges and opportunities (pp. 9-21). New York, NY: Springer. Lai, M. K., & Schildkamp, K. (2013). Data-based decision making: An overview. In K. Schildkamp, M. K. Lai, & L. Earl (Eds.), Data-based decision making in education: Challenges and opportunities (pp. 9-21). New York, NY: Springer. Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision-making in education. Santa Monica, CA: RAND Corporation. McGrath, J. M. (2008). Nursing intuition and avoiding the cascade to a crisis. Journal of Perinatal and Neonatal Nursing, 22, 181-182. Means, B., Chen, E., DeBarger, A., & Padilla, C. (2011). Teachers’ ability to use data to inform instruction: Challenges and supports. Washington, D.C: U.S. Department of Education Office of Panning, Evaluation, and Policy Development. Miller, C. C., & Ireland, R. D. (2005). Intuition in strategic decision making: Friend or foe in the fast-paced 21st century? Engineering Management Review, 33(2), 19-30. National Commission of Excellence in Education. (1983). A nation at risk: The imperative for educational reform (ED 226 006). Washington, DC: U.S. Government Printing Office. Negrete, E. G. (2004). Faculty concerns and perceptions of mandated educational change: An exploratory study. (Doctoral dissertation, University of Texas at Austin). Dissertation Abstracts International, 11A, 4127. No Child Left Behind Act, 20 U.S.C. § 6319 (2001). Pandina Scot, T., Callahan, C. M., & Urquhart, J. (2008). Paint-by-number teachers and cookie cutter students: The unintended effects of high-stakes testing on the education of gifted students. Roeper Review, 31, 40-52. Piro, J. S., Dunlap, K., & Shutt, T. (2014). A collaborative Data Chat: Teaching summative assessment data use in pre-service teacher education. Cogent Education, 1(1), 968409. Prenger, R. & Schildkamp, K. (2018). Data-based decision making for teacher and student learning: A psychological perspective on the role of the teacher. Educational Psychology, 38, 734-752. Raykov, T. (2007). Evaluation of criterion validity for scales with congeneric measures. Measurement and Evaluation in Counseling and Development, 40, 110-118. Reeves, T. D. (2017). Pre-service teachers’ data use opportunities during student teaching. Teaching and Teacher Education, 63, 263-273. Reeves, T. D., & Chiang, J. L. (2017). Building pre-service teacher capacity to use external assessment data: An intervention study. The Teacher Educator, 52, 155-172. Reeves, T. D., & Honig, S. L. (2015). A classroom data literacy intervention for pre-service teachers. Teaching and Teacher Education, 50, 90-101. Reeves, T. D., Summers, K. H., & Grove, E. (2016). Examining the landscape of teacher learning for data use: The case of Illinois. Cogent Education, 3(1). Reise, S. P., Waller, N. G., & Comprey, A. L. (2000). Factor analysis and scale revision. Psychological Assessment, 12, 287-297. Robert. R. R., Tilley, D. S., & Peterson, S. (2014). A power in clinical nursing practice: Concept analysis on nursing intuition. Clinical Practice, 23, 343-349. Schildkamp, K., Karbautzki, L., & Vanhoof, J. (2013). Exploring data use practices around Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 23 Europe: Identifying enablers and barriers. Studies in Educational Evaluation, 42, 15-24. Schwarzer, R., & Hallum, S. (2008). Perceived teacher self-efficacy as a predictor of job stress and burnout: Mediation analyses. Applied Psychology, 57, 152-171. Shotsberger, P. G., & Crawford, A. R. (1999). On the elusive nature of measuring teacher change: An examination of the Stages of Concern Questionnaire. Evaluation and Research in Education, 13(1), 3-17. Soland, J. (2013). Predicting high school graduation and college enrollment: Comparing early warning indicator data and teacher intuition. Journal of Education for Students Placed at Risk, 18, 233-262. Straub, E. T. (2009). Understanding technology adoption: Theory and future directions for informal learning. Review of Educational Research, 79, 625-649. Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17, 783-805. U.S. Department of Education, Office of Planning, Evaluation, and Policy Development. (2010). Evaluation of evidence-based practice in online learning: A meta-analysis and review of online learning studies. Washington, D.C: U.S. Department of Education. van Geel, M., Keuning, T., Visscher, A. J., & Fox, J. P. (2016). Assessing the effects of a school- wide data-based decision-making intervention on student achievement growth in primary schools. American Educational Research Journal, 53, 360-394. Vanhoof, J., & Schildkamp, K. (2014). From ‘professional development for data use’ to ‘data sue for professional development’. Studies in Educational Evaluation, 42,1-4. Volante, L., & Fazio, X. (2007). Exploring teacher candidates’ assessment literacy: Implications for teacher education reform and professional development. Canadian Journal of Education, 30, 749-770. von der Embse, N. P., Schoemann, A. M., Kilgus, S. P., Wicoff, M., & Bowler, M. (2017). The influence of test-based accountability policies on teacher stress and instructional practices: A moderated mediation model. Educational Psychology, 37, 312-331. Wachen, J., Harrison, C., & Cohen-Vogel, L. (2018). Data use as instructional reform. Exploring educators’ reports of classroom practice. Leadership and Policy in Schools, 17, 296-325. Walker, D. A., Reeves, T. D., & Smith, T. J. (2016). Confirmation of the Data-Driven Decision- Making Efficacy and Anxiety Inventory’s score factor structure among teachers. Journal of Psychoeducational Assessments. Retrieved from http://journals.sagepub.com/doi/abs/10.1177/0734282916682905?journalCode=jpaa Wayman, J. C. (2005). Involving teachers in data-driven decision-making: Using computer data systems to support teacher inquiry and reflection. Journal of Education for Students Placed at Risk, 10, 295-308. Wayman, J. C., Midgley, S., & Stringfield, S. (2006, April). Leadership for data-based decision- making collaborative educator teams. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. Weber, G. (1971). Inner city children can be taught to read: Four successful schools (Occasional paper no. 18). Washington, DC: Council for Basic Education. Welsh, I., & Lyons, C. M. (2001). Evidence-based care and the case for intuition and tacit knowledge in clinical assessment and decision making in mental health nursing practice: n empirical contribution to the debate. Journal of Psychiatric Mental Health Nursing, 8, 299-305. Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 24 Whitecotton, S. M., Sanders, D. E., & Norris, K. B. (1998). Improving predictive accuracy with a combination of human intuition and mechanical decision aids. Organizational Behavior and Human Decision Processes, 76, 325-348. Yan, T., & Deng, M. (2018). Regular education teachers’ concerns on inclusive education in China from the perspective of concerns-based adoption model. International Journal of Inclusive Education, 1-21. Retrieved from http://www.tandfonline.com/doi/abs/10.1080/13603116.2018.1435741 Yanniv, I., & Hogarth, R. M. (1993). Judgmental versus statistical prediction: Information asymmetry and combination rules. Psychological Science, 4, 58-62 Author Notes Karee E. Dunn University of Tennessee kareedunn@utk.edu Denise T. Airola University of Arkansas dairola@uark.edu Takuya Hayakawa University of Tennessee thayakaw@vols.utk.edu mailto:kareedunn@utk.edu mailto:dairola@uark.edu mailto:thayakaw@vols.utk.edu Pre-Service Teacher’s Efficacy, Anxiety, and Concerns about Data and the New Idea of Anchored Judgment Current Issues in Education, 21(1) 25 Volume 21, Issue 1 April 29, 2020 ISSN 1099-839X Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Current Issues in Education (CIE), it is distributed for non-commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by the author(s) or CIE. Requests to reprint CIE articles in other journals should be addressed to the author. Reprints should credit CIE as the original publisher and include the URL of the CIE publication. CIE is published by the Mary Lou Fulton Teachers College at Arizona State University. Editorial Team Consulting Editor Neelakshi Tewari Executive Editor Marina Basu Section Editors L&I – Renee Bhatti-Klug LLT – Anani Vasquez EPE – Ivonne Lujano Vilchis Review Board Blair Stamper Melissa Warr Monica Kessel Helene Shapiro Sarah Salinas Faculty Advisors Josephine Marsh Leigh Wolf