Microsoft Word - 1163-5014-3-LE_final.doc 1 Volume 16, Number 1 April 26, 2013 ISSN 1099-839X Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice Jo Beth Jimerson and Marla W. McGhee Texas Christian University In order to learn more about the ways in which educators in various roles construe “data” and “data use,” we conducted a study in a small school district in central Texas, collecting survey data from n=154 educators in August 2012. Analyses revealed that while all educators reported using some form of evidence to inform practice, the terms used to describe that evidence varied. Further, more teacher participants attached mixed connotations to the terms, as compared to district leaders and campus leaders. Teachers whose survey responses suggested broader, more improvement-oriented mental models of data use reported slightly higher levels of commitment to data-informed practice. We review models of data-rich collaborative inquiry that provide approaches similar to “data- driven decision making” but which may avoid accountability- and compliance-laden language that appears to heighten anxiety among some teachers. Keywords: data use; data-informed decision making; data driven decision making, school accountability, inquiry in schools Data needs a facelift. It needs a new name that people are excited about. Let’s not call it “data” anymore. Let’s call it “Party Time” and then everybody would be excited about it. It gets a bad rap. People don’t want to talk about data sometimes, you know. It’s useful to look at these numbers. But I think that people get overwhelmed and tired of hearing that word. (Middle school teacher in Jimerson, 2011, p. 291) In what may be a paradox of the current “age of accountability,” research suggests that teachers often embrace the underlying concept of educational data use (i.e., learning more about students’ needs) yet describe feelings of mistrust, uncertainty, or reticence toward formal “data use” (Earl & Fullan, 2003; Jimerson & Wayman, 2012; Louis, Leithwood, Wahlstrom, & Anderson, 2010). For some, this may be a matter of practicality: data systems are often difficult to navigate, and can contribute to frustration in a teacher’s already- busy day (Wayman & Cho, 2009; Wayman & Stringfield, 2006). For others, reticence may trace to prior experiences of the misuse or abuse of data by leaders (Earl & Fullan, 2003). In this paper, we address this issue by examining the meanings teachers and school leaders attach to the terms “data” and “data use,” and the consequences those differences in meaning have for teacher commitment to improvement efforts. To frame this exploration, we contend that teachers in the United States have always used educational data. Even in iconic “one room schoolhouses,” teachers gave assignments, used assessments, and issued grades and promotions accordingly. What makes the current context different is an ever-increasing pressure from state and federal accountability policies requiring more formal and systematic decision-making steeped in data use (Honig & Venkateswaran, 2012; Mandinach, 2012). In this environment, teachers are pressed to use data more Current Issues in Education Vol. 16 No. 1 2 frequently and in more complex ways than ever before. However, teachers do not act in a vacuum—quality teaching is affected by the quality of leadership in the school (Anderson, Leithwood, & Strauss, 2010; Datnow, Park, & Wohlstetter, 2007; Louis et al., 2010). We, therefore, should not expect robust data use among teachers to be the norm in contexts where school leaders lack the ability to build and sustain capacity for such. Or, as Wayman, Cho, and Johnston (2007) succinctly put it, “Data use lives and dies in the principal’s office” (p. 55). Because the role of principals and district leaders in building and sustaining data use is key, we think it important to consider how the words and actions of these leaders may influence how teachers think about data and data use. Unfortunately, research on leadership for data use suggests that principals (who oft transition to district leadership) are often ill-equipped to effectively lead for data use (Means, Padilla, DeBarger, & Bakia, 2009; Wayman, Jimerson, & Cho, 2012; Wayman, Spring, Lemke, & Lehr, 2012). We also find that research on leadership for data use frequently focuses on structural or systemic supports for data use (e.g., Kerr, Marsh, Ikomoto, Darilek, & Barney, 2006; Wayman, Jimerson, & Cho, 2012; Wayman Spring et al., 2012) rather than deep examinations of how language or modeling by leaders hinders or encourages teacher engagement in data use. One pathway to examining this intersection of leader modeling with teacher engagement involves the consideration of alternative models for talking about educational data. At present, much research orients toward the “data-driven decision making” (DDDM) model (Mandinach, Honey, & Light, 2006; Marsh, McCombs, & Martorell, 2010; Marsh, Pane, & Hamilton, 2006). Because the roots of DDDM privilege quantifiable outcomes that enable system-level comparisons, we should not be surprised that DDDM typically results in a high degree of attention to achievement outcomes in the form of accountability exams. These data permit not only the tracking of progress, but for comparisons across time, classrooms, student groups, and systems (Marsh et al., 2006). However, teacher needs orient toward classroom- level data focused on specific students, rather than systems (Means et al., 2009; Schildkamp & Kuiper, 2010). Such needs fit well with models of inquiry that make use of evidence without privileging broad-scale data and without relying on the accountability-laden verbiage that has become associated with “DDDM.” For example, literature parallel to research on “data use” yet focused on teacher decision-making frequently references action research, professional learning communities, and collaborative inquiry (Calhoun, 1994; Copland, 2003; Tucker, 2010). These models share characteristics of data- driven practice (e.g., the importance of question formation to drive investigation and the use of multiple streams of data to inform conclusions), but aim for a fine-grained, highly contextualized perspective. To that end, we wondered whether a mismatch exists between DDDM models and how teachers think about their roles and about the day-to-day practice of teaching. We further wondered whether an alternative model for data-informed inquiry might better fit teachers’ beliefs and philosophies about practice. To learn more about the ways in which educators think about “data use,” and thereby to ascertain whether non-“DDDM” models may provide educational data use with the “facelift” suggested by the middle school teacher in our epigraph, we conducted a study to examine the ways in which teachers and school leaders conceptualize “data” and “data use.” The study was guided by two questions: (1) What meanings do educators attach to “data use?” and (2) How are particular ways of thinking about “data use” related to teacher investment in data-informed practice? Data Use and Related Models: Background and Context Teachers have long been users of data to make professional decisions about their work. Over the past twenty-five years, however, accountability policies and high stakes consequences have colored the ways teachers and administrators view and use data at the classroom, school, and district levels. In several popular models for continuous improvement (other than DDDM), data is central to decision making and can powerfully impact practice while avoiding a hyper-focus on mandated state or federal testing. The use of these models and perceptions and attitudes toward data use rely heavily on guidance and modeling by campus leaders. In the following sections, we outline the research surrounding these issues. We explore the rise of the accountability movement and how associated pressures have affected the ways in which educators understand “data use,” describe data-rich inquiry practices, and discuss the critical role leadership plays in facilitating constructive models for data use. Data Use as a Centerpiece for Accountability “Reform” Where the classroom teacher had once been the primary individual responsible for assessing and documenting a student’s progress over time (typically via the individual report card), the late 20th century brought a dramatic shift in thinking concerning educational data collection, use, and reporting. In the state of Texas, for example, comprehensive reform meant sweeping educational changes—a statewide curriculum framework, criterion-referenced assessments, and public reporting of outcomes and information. A facet of the Texas model that made it unique was a focus on individual student results rather than data reported in the aggregate. “Since education systems have traditionally served some, but not all, students well, a focus on the aggregate level would Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice 3 probably not result in meaningful change. An explicit focus on each student was necessary” (Nelson, McGhee, Slater, & Meno, 2007, p. 703). While others in the United States were just beginning to ponder the differences in academic performance across various student groups, Texas was creating and installing a data-rich reform model specifically focused on addressing educational inequities in its public schools (Nelson et al., 2007). Although originally well-intentioned, the Texas model evolved over the years, producing a series of unintended consequences (Booher-Jennings, 2005; Holme, 2013; Holme, Carkum, & Snodgrass Rangel, in press). Beyond informing school improvement efforts at a systemic level, the Texas model, deeply rooted in high- stakes testing, publicly ranks and sorts schools into four categories, from “academically unacceptable” to “exemplary.” In many contexts, the pursuit of higher test scores (and thus higher ratings) effected a narrowing of the curriculum to focus on tested subjects, a shift from diagnostic to summative testing, and “gaming” practices aimed at mitigating or masking the performance of particular student groups (Holme, 2013; Valenzuela, Fuller, & Vasquez Heilig, 2006; Vasquez Heilig & Darling-Hammond, 2008). Despite intense criticism leveled at the altered Texas system, it nonetheless became a model for the nation under President George W. Bush in the form of the federal No Child Left Behind (NCLB) Act signed into law in 2002. State and federal accountability systems tied to funding and high profile consequences for not achieving Annual Yearly Progress or prescribed performance targets can and do dominate data discourse (Colyvas, 2012). And, once large-scale measurement systems or performance metrics of this type are made public, they are challenging to substantively change or undo, even if not widely embraced (Colyvas, 2012; Goren, 2012). Additionally, the NCLB Act and initiatives related to the law frequently refer to “scientifically-based research” and “scientifically-based evidence,” practices grounded in replicable experimental or quasi- experimental studies. This stance privileges controlled trials and quantifiable data. Because the policy dialogue around “scientifically-based” or “research-based” practice is heavily oriented toward large-scale, quantifiable data— or numbers—many practitioners have come to equate “data-driven” practice with the use of large-scale data sets that carry a mantle of legitimacy from the state and federal government. Ergo, “data” or “data use,” to many educators, means results of mandated test scores and district endorsed benchmark examinations, as these datasets are highly valued by administrative decision makers. More often than not, however, these results arrive to teachers in formats that are not particularly helpful and far too late in the academic year to actually inform instructional decisions. As Tucker (2010) notes: in education, state and district officials want data that shows broad trends so they can assess a school’s or a district’s overall effectiveness. (This is accountability data.) Teachers want additional information, such as results from classroom assessments that may track weekly progress. (p. 3) Herein lies the problem: Even where educators are intentional about the importance of using multiple measures of data to inform decision-making, “DDDM” is often so inextricably entangled with NCLB and state accountability policies that the overall effect is one of refocused attention to system-level, broad-scale data that enables comparisons across context (e.g., Mandinach, 2012; Marsh et al., 2006; Marsh et al., 2010). While DDDM-models themselves have great potential for positive application and for catalyzing subsequent improvement at multiple levels, the foregrounding of test scores and broad-scale data may have the unintended effect of seeming less-than-relevant to teachers yearning for timely student- and classroom- specific data (Ingram, Louis, & Schroeder, 2004; Firestone & Gonzales, 2007; Marsh et al., 2006; Valli & Buese, 2007). That is, the DDDM model itself is not as problematic as the (mis)understanding among educators that DDDM equates to monitoring test scores and determining how to raise test scores, rather than using multiple forms of data as clues to how best meet each child’s complex and changing needs. It is within this context that we ask: “Has the term ‘data-driven’ become too shopworn to recapture? Is it time to replace it with a term that conveys the same intent but which casts a broader conceptual net?” In the next section, we examine models of data-informed inquiry we perceive as up to this task in that they promote robust and broad use of multiple data streams while eschewing language inextricably tied to accountability testing. These models—action research and collaborative inquiry —may hold promise for those seeking to promote data use in ways that create greater buy-in from teachers and more effectively address the needs of learners. Data Use as Nested in Inquiry The research on data use is replete with models that set out multi-phase cycles. Supovitz (2010) suggested a four-part framework for knowledge-based organizational learning that examined how district-level leaders used data through phases of data capture, meaning-making, information-sharing, and knowledge codification. The Data Wise process (Boudett, City, & Murnane, 2005) moves participants through an eight-step process that involves preparation, inquiry, and action based on “digging deep” into sources of student data. For our exploration of the literature, we wanted to focus on Current Issues in Education Vol. 16 No. 1 4 models that may have, to this point, been noted only tangentially in the data use literature, but which situate data use at the heart of teaching and learning (i.e., classroom or school level) and which emphasize an improvement-orientation rather than one that focuses primarily on “accountability” or “DDDM.” Action research. One process that places data use at the center of systematic continuous improvement is the action research cycle. This inquiry model is ideal for the educational setting in that it can be adapted to any context and used to study larger, schoolwide issues as well as smaller areas of focus or concern. The presence (and importance) of data is pervasive throughout the action research process. As illustrated in Figure 1, solid arrows represent primary movement through the action research steps whereas the dashed lines indicate the common practice of returning to data sources again and again throughout the cycle for additional information, clarification, and refinement. Several facets of action research make its treatment of data unique as compared with other potential data use strategies. In step two of the process, data collection, Calhoun (1994) suggested that both on-site, school information (internal data) and professional literature and research (external data) be collected, organized, analyzed, and interpreted. This means that as we move to the action-taking phase, “we combine and apply” (Calhoun, 1994, p. 89) what we are discovering from studying on-site or school-based data and compare it with our findings in the professional literature and research. In the words of Calhoun (1994), “We use the knowledge gained from these collective studies to select innovations and develop initiatives that have the best chance of improving student learning” (p. 89). This quality of the action research cycle could help plug critical gaps in educator data use, as studies suggest that school and district leaders frequently neglect to search for external data or to thoughtfully attend to external data or research in actual decision-making processes (Coburn, Honig, & Stein, 2009; Coburn & Talbert, 2006). Additionally, because the action research process is a systematic approach to data use, problem solving, and solution generation, it requires triangulation in regard to data sources. Triangulation, or, “the act of bringing more than one source of data to bear on a single point” (Marshall & Rossman, 1999, p. 146) helps to create a more vivid and accurate picture of the circumstances at hand (Glanz, 2003; Supovitz & Klein, 2003; Wayman & Stringfield, 2006). It also enhances the validity and credibility of the work (Hendricks, 2009; Shen & Cooley, 2008). Using multiple data sources is essential both in clarifying the focus area or research question as well as corroborating findings or outcomes. Related to accessing and using a variety of data sources, Calhoun (1994) further suggested that action researchers not overlook existing archival data that may be immediately accessible. Grades, attendance records, promotion and retention figures, enrollments in special programs, demographic data and the like are generally readily available, require little work to acquire and provide “ an immediate picture of the educational climate of the school” (Calhoun, 1994, p. 53). Collaborative inquiry. Tucker (2010) spotlights a model that is both data-focused and team-oriented in its approach. In the Collaborative Inquiry Model, (see Figure 2) teams of educators, made up of teachers and administrators, focus specifically on a small group of struggling learners. The inquiry team examines student work, related assessment data, and instructional approaches used with students. The team then constructs Figure 1. The Action Research Cycle. (Source: Calhoun, 1994, p. 2) Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice 5 theory of action as to why the students are not prospering academically. In this model, as in action research, ongoing data collection, reflection, and analysis are central to the work. These cyclical processes are in effect “sandwiched” between a readiness procedure and collective recognition and celebration of accomplished work. And, like action research, the inquiry cycles include resources and information external to the classroom and/or school that may further inform the area of focus. Moreover, the collaborative inquiry model can be used to address highly contextualized classroom-level instructional issues as well as general schoolwide matters. The Principal: A Linchpin in Successful Data Use? In both models, educators are placed in proactive roles as they establish questions around problems of practice and collect data appropriate to those questions, rather than simply reacting to accountability demands or examining available data. Models such as the action research cycle and collaborative inquiry are flexible in practice, yet offer parameters and expectations for data use. In both models, the prominent role of data is clear as those engaged in inquiry collect, analyze, and make meaning of information vital for decision making and improvement. However, by highlighting a process of collaborative inquiry to inform practice, rather than being “data-driven,” both models avoid unnecessary entanglement with the language of NCLB or state-level accountability mandates. Both models situate the use of data within an improvement-oriented teaching and learning context. While the inquiry models presented above can be operationalized without central office intervention or support, in reality, schools exist within systems and rely heavily on these systems for resources and services, including matters related to meaningful data use. When the practice of data use can be viewed as a system-wide endeavor, we acknowledge the fundamental ways in which different levels of the system are inextricably linked and dependent upon one another (Coburn & Turner, 2012; Wayman, Cho, Jimerson, & Spikes, 2012). Honig and Venkateswaran (2012) submit that, “central office staff matter to school-level evidence use, and to some extent school staff are important participants in central office evidence-use processes” (p. 216). No one person is the sole provider of leadership at the campus level (Copland, 2003; Park & Datnow, 2009; Wayman Spring et al., 2012); however, campus leaders are uniquely situated to provide consistent instructional support for classroom teachers. This includes collecting, analyzing, and using data as “…it is naïve to believe that teachers will use assessment data to inform instruction without the coaching and support they need to begin the process” (Buhle & Blachowicz, 2008, p. 45). Although support for teachers’ data use may start with a Figure 2. NYC Schools Collaborative Inquiry Model. (Source: Tucker, 2010) Current Issues in Education Vol. 16 No. 1 6 series of district-sponsored activities or meetings with district data coaches, the principal is the individual who can sustain a focus on data use in practice (Datnow et al., 2007; Wayman et al., 2007). “Understanding the practice of data use not only can help us explain the outcomes of data use but also provides insight into when and under what conditions data use acts as a productive pathway to educational improvement and when it does not” (Coburn & Turner, 2012, p. 100). Developing and maintaining a vibrant vision for learning includes understanding and using data in ethical and appropriate ways. By the very nature of their work, principals are deeply knowledgeable about their schools and communities, and understand the school’s areas of strengths as well as targets for improvement. Principals serve as role models for those on their campuses and help to establish and shape the working culture and atmosphere across the school (Louis et al., 2010; Wayman Spring et al., 2012). Likewise, principals are generally quick to seek assistance from district-level staff members when help is needed. As Goren (2012) asserts: Data means different things to different people. School practitioners can learn from central office administrators and central office staff can learn from school practitioners. And indeed, what is necessary at all levels is the presence of individuals with the capacity to interpret, understand, and broker the information for appropriate use. (p. 235) At the campus level, we believe this person to be the principal. No individual located at the school site is better suited to represent the school and its unique needs or to serve in the role of “information broker.” What remains unclear is how the choices school leaders make in how they model data use—through word and deed— encourages or dissuades teachers from investing the time and energy needed to engage in data-informed practice. Theoretical Framework Our approach to this study was informed broadly by organizational theory and specifically by Morgan’s (2006) description of complexity theory and Senge’s (2006) concept of “mental models.” In describing how organizations function and change, Morgan (2006) noted that organizations are characterized by inherent complexity due to the multiple actors, programs, procedures, and internal/external pressures that are at constant interplay. Morgan asserted that order emerges from this complexity because the systems “get caught in tensions…falling under the influence of different attractors” that help define context and establish norms for actors in the system (p. 254). Attractors can cement norms and patterns in an organization, or disrupt norms and patterns, pushing a system to change (for better or worse). Senge (2006) took a slightly different approach in describing how leaders might catalyze change, and focused on learning in terms of “mental models,” or “deeply ingrained assumptions, generalizations, or even pictures or images that influence how we understand the world and how we take action” (p. 8). He noted that in a learning organization, members are committed to unearthing the mental models held throughout the system, to holding these up to new questioning and evidence, and to reforming those mental models as needed. We considered how these theories apply to current thinking about data use. Viewing the research through this theoretical lens gave rise to three assumptions that helped us consider the intersection of data use and professional learning about data use: 1. Educators hold particular “mental models” about data use—i.e., what they think data use is about and how they believe data ought to be used. 2. How educators engage in data use is affected by their “mental models.” 3. Because mental models and patterns of organizational behavior are malleable, the ways in which educators frame and engage in data use is likewise malleable. Method The present study draws from an ongoing project in Pence School District1, a small district located in central Texas. The broader project is driven by two overarching purposes: (1) to learn about teacher perceptions and attitudes toward data use; and (2) to provide the participating district with information to support planning and improvement initiatives. In this section, we provide information on the context of the study district and describe our procedures for data collection and analyses pertinent to this study. Study Context Pence School District was selected in part as a sample of convenience (access was eased by the desire of district leaders to obtain useful information to support ongoing improvement initiatives) and in part because the district’s performance under current Texas accountability system measures suggested room for improvement in terms of data use. To wit, the district as a whole was rated “Academically Acceptable”2 for the 2010-2011 school year, though two of the four campuses in the district were rated “Academically Unacceptable.” __________________________________________________________________________________________________ 1 Pseudonym 2 Under current accountability system structures, schools in Texas may be rated “Exemplary,” “Recognized,” “Academically Acceptable,” or “Academically Unacceptable.” Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice 7 Table 1 Selected Longitudinal Data, Texas Assessment of Knowledge and Skills (TAKS). Source: Texas Education Agency Academic Excellence Indicator System Exam/Population 2003 Pass Rate 2011 Pass Rate District change, 2003-11 TAKS All tests taken All students African-American White 41% 28% 47% 60% 48% 71% +19% +20% +24% TAKS Reading/ELA Economically Disadvantaged Hispanic 59% 55% 79% 77% +20% +22% TAKS Mathematics African-American White 40% 60% 62% 79% +22% +19% TAKS Science All Students Hispanic 29% 13% 73% 72% +44% +59% Also, the small size of the district (approximately 3,000 students) and number of campuses (four) provided an appropriate setting for an early study of the issues addressed in this paper. To understand the context of Pence Schools in relation to data use, it is important to examine how the district has changed over the last several years. In the not- distant past, Pence would have been considered a rural district on the outskirts of a city home to a regional university. Twenty years ago, Pence educators worked with a student population that was predominantly white and working class (i.e., 25% of students were nonwhite and just under 40% of students were considered eligible for free-or reduced price meals). Today, those demographics have shifted considerably, as the outer rings of the nearby city have expanded: Pence educators in 20113 worked with a population predominantly composed of students of color and a mix of working-class and poverty-level homes (60% of students are Latino or African-American and nearly 80% qualify for free or reduced price meals). In brief, Pence educators are striving to meet the needs of a rapidly changing community. Due in part to these changing demographics, and in part to increasingly rigorous accountability exams and policy requirements,4 Pence educators have worked diligently to effect improvements in teaching and learning. State-level data on attendance, state-mandated exams, and dropout rates evidence steady progress between 1994-2002 (when TAAS was in effect) followed by an overall drop in performance and subsequent improvement with the inauguration of the more-difficult TAKS exams (see Table 1 for data to illustrate this trend under TAKS). Despite marked improvements in student achievement outcomes over the past decade, the rate of progress has not kept pace with increasing accountability requirements under NCLB and the Texas accountability system. As a result, district leaders have become even more “data-driven,” both in efforts to be strategic about improvements in teaching and learning and in response to requirements under the Texas accountability system and under NCLB. While achievement data over time suggest that Pence educators are making headway in their improvement efforts, we also think it important to note that the pressure to escape accountability sanctions weighed heavily on Pence educators’ minds. For example, in an early interview, one district leader detailed how, because of accountability sanctions from the state and federal government, district and campus leaders were being required to implement eight different “improvement plans”: ____________________________________________________________________________________________________ 3 The most recent data available via state-level Academic Excellence Indicator System reporting. 4 In the last 20 years, Texas has moved from the Texas Assessment of Academic Skills, or TAAS, through the Texas Assessment of Knowledge and skills (TAKS), and now to the State of Texas Assessments of Academic Readiness (STAAR). Each iteration has been more rigorous than the past, and high-stakes (i.e., promotion in certain grades and graduation from high school) are linked to various exams. programs monitoring system. Current Issues in Education Vol. 16 No. 1 8 each of the four campuses had a state-required Campus Improvement Plan, the district had a required District Improvement Plan, each of the campuses tagged as “Academically Unacceptable” were required to work with a Professional Service Provider to implement additional improvement plans to address state and AYP compliance issues, and the district had a separate required improvement plan associated with a state-level special Additionally, one campus was “close to becoming” Academically Unacceptable in the coming year, so district leaders were preemptively working with to address concerns at that campus via another improvement plan. As one leader told us, so many required “improvement plans” created a context of “…putting out fires. And what you’re doing is basically charting, OK, which [report] is due when? Which deadline is what? It becomes a logistics nightmare.” As to whether data use was more a compliance- or improvement issue, the leader was emphatic in wanting data use to be about improvement— not just about testing. But, the leader noted: As far as I’m concerned, it’s a compliance issue because what it does, it doesn’t give you time to do the other things. What I tell the people that struggle with all of this is, ‘What they’re doing is making us do what the exemplary districts were already doing—it’s just they got to do it their way because they were proactive in doing it, whereas we have to do it the bureaucratic way.’ Terms It is important to define two terms we used in framing this study. In line with Wayman Cho et al. (2012), we consider “data” to be any information that can assist educators in knowing more about students and their needs. Further, we consider data to be information which has the potential to be codified and shared, even though, in practice, not all data are formally codified. In line with Schildkamp and Kuiper (2010), we take “data use” to be a process of “systematically analyzing existing data sources within the school, applying outcomes of analyses to innovate teaching, curricula, and school performance, and, implementing (e.g., genuine improvement actions) and evaluating these innovations” (p. 482). Instrument To inform our research questions, we obtained an overview of the perceptions and attitudes of district and school personnel via the Survey of Data Use & Professional Learning (Jimerson, 2012) a 66-item instrument which included Likert-style items as well as open-ended response items. Prior research (Jimerson, 2011) was used to inform initial construction of the instrument, and measures were taken to strengthen content validity and to assure stability of items over time. To wit, in June and July 2012, 12 EC-12 practitioners (four teachers, four campus principals, and four school district administrators) engaged in a series of cognitive interviews (see Desimone & LeFloch, 2004) to refine the instrument in terms of wording, clarity, and ordering of items. As a result of that process, the instrument was revised. To ascertain the reliability of items on the survey, a panel of n=31 practicing EC-12 educators from a variety of roles were administered the survey twice, 10-14 days apart. Items were analyzed via bivariate correlation; only items that correlated at statistically significant levels (p<.05) were included in analyses for the purposes of this study. Procedure The instrument was distributed online to all Pence educators during a two-week window in August, 2012. Table 2 denotes overall response rate as well as response rates by job role. Although most survey items were designed to enable comparisons via the use of reported levels of agreement on Likert-scale items, all personnel were invited to respond to several open-ended items as well. These responses provided an additional layer to our analysis and to our ability to interpret the data as a whole. All personnel in Pence ISD were invited to complete the survey electronically. Response rates are reflected in Table 2. Measures Likert-type items were used to inform quantitative analyses. Open-ended items from the online survey were used to inform qualitative analyses. Educational role was used as a category by which to compare educators on aspects of data use. Role was Table 2 Study Participation and Survey Response Rates, by Role Survey Data N Response Rate District-level Leaders 12 88.9% Campus-level Leaders (Principals, Asst. Principals) 11 93.3% Professional Support (e.g., Inst. Coaches, Specialists) 13 92.9% Teachers 118 81.4% Total 154 83.5% Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice 9 evaluated by a survey question that asked, “How would you best classify your current job?” Resulting categories for comparison included district-level (i.e., central office) leaders, campus-level leaders (i.e., principals and assistant principals), “professional support,” a category comprised of certified professionals who acted in a support role (e.g., counselors, librarians, instructional coaches), and teachers. A limitation that we acknowledge is the small cell size for persons in non-teaching categories; we interpreted analyses for these categories with caution, though we worked to bolster our interpretations of data via triangulation with open-ended survey responses. Scales. Individual survey items were set on a 5- point Likert scale (1=strongly disagree; 2=disagree; 3=neither agree nor disagree; 4=agree; 5=strongly agree). We used three scales in the analyses pertinent to this paper. The Construal of Data scale was comprised of three items that assessed perceptions regarding whether educators used elements beyond intuition or hunches to inform instruction. This scale included the items: “I use a variety of data to inform my teaching and/or daily practice”; “I use a variety of evidence to inform my teaching and/or daily practice”; and “I use a variety of information to inform my teaching and/or daily practice.” The alpha reliability of this scale was 0.85. The Beneficence of Data scale comprised four items that assessed the participant’s perception that data use holds promise for positive good as pertains to teaching and learning efforts. This scale included the items: “Data use helps me make informed decisions”; “Data use helps me make ethical decisions”; “Data use benefits educators and students”; and “Data use is about continuous improvement in the classroom.” The alpha reliability of this scale was .78. The Data Anxiety scale was comprised of three items that assessed participants’ level of concern regarding possible misuse or abuse of data. This scale included the items: “I worry that data will be used to shame or punish teachers at my school”; “I am concerned that data use will increase unhealthy competition among educators at my campus”; and “I am concerned that data use will increase unhealthy competition among educators in my district.” The alpha reliability of this scale was .88. Analyses To inform our research questions, we examined survey data to determine whether our intended scales were reliable, and to what degree. SPSS was used for all statistical analyses. We selected three scales (see Table 3) that we interpreted as pertinent to informing our research questions. We created a composite variable for each scale, and examined scale means by role (Appendix B contains an index of variables used in the course of this study). This comparison helped us understand some general patterns by role in terms of how respondents perceived “data” and “data use.” We used a one-way ANOVA to examine differences among scale means by role. To further illuminate this analysis, we analyzed responses to two open-ended response items: “When you hear you are going to be involved in ‘data use’ or will be ‘working with data,’ what comes to mind?” and “Please describe your last learning opportunity (formal or informal) that related in any way to data use.” Responses were categorized by role and coding was facilitated by Atlas.ti software. To strengthen the validity of our analysis of open-ended survey data, we began by following the suggestions for anchoring early codes in existing research (Miles & Huberman, 1994). We then followed the systematic procedures outlined by Merriam (2009) to guide thematic coding and the eventual merging of codes into themes. We also note that throughout this process, we worked to triangulate developing themes by examining our emerging coding scheme in light of other survey data and in light of project notes and clarifying conversations with district personnel (see Appendix A for the final coding scheme for our analyses). To examine the relationship among teacher investment in data use and ways of thinking about data use, we conducted three analyses. First, we used a one- way ANOVA to examine differences in the means by role on the “Beneficence of Data” scale. For all analyses, the threshold for statistical significance was established at p<.05. Next, we used the item “If no one (on my team/in my department) wanted to use data, I would still make the effort to do so” as a proxy for commitment to data use. We recoded this variable to isolate two groups: those who reported “high” levels of commitment (i.e., those who responded “agree” or “strongly agree”) and those who reported “low/lukewarm” levels of commitment (all other responses). We recoded the Beneficence scale to isolate three groups: Those who attributed “Low” beneficence to data, those who attributed “Moderate” beneficence to data, and those who attributed “High” beneficence to data. Focusing on the survey responses of teachers, we conducted a Pearson’s chi-square analysis to examine the relationship between reported level of commitment and the degree to which teachers attached beneficent characteristics to data use. Last, we used another Pearson’s chi-square analysis to examine the relationship between reported level of commitment to data use and the degree to which teachers considered data use to be “all about accountability ratings.” To accomplish this, we leveraged the survey item, “Data use is all about accountability ratings” and recoded the responses into three categories: “Low” agreement, “Moderate” agreement, and “High” agreement. Because we calculated multiple chi-squares in the course of the analyses, we used the Bonferroni Current Issues in Education Vol. 16 No. 1 10 adjustment in order to maintain the error rate with a level of significance of p<.05. Results In what follows, we present our findings in two sections: (1) Meanings of “data” and “data use” and (2) Teacher investment in data-informed practice. Meanings of “Data” and “Data Use” Upon beginning the study, one working hypothesis was that educators would construe “data” as conceptually different from “evidence” or “information,” that the term in and of itself would be construed more negatively than would “evidence” or “information,” and that these differences in construal might present a barrier to efforts to promote teacher buy-in to data-informed practice. However, analysis of survey data indicates that this was not the case among Pence educators. In fact, the alpha reliability of the “Construal” scale suggests that the underlying concept is that educators—regardless of role— use something to inform practice, and that this something is more than mere instinct or hunch. We learned that some practitioners felt comfortable referring to this “something” as “data,” whereas others preferred “evidence” or “information,” but most respondents— across job roles—either “agreed” or “strongly agreed” that they used at least one of these three to inform daily practice (see Table 3). To further examine these differences among scale means, we conducted a one-way ANOVA, and observed no statistically significant differences among scale means by job role on the “Construal” scale F(3, 146)=1.99, p>.05. We similarly observed no statistically significant differences among scale means by job role on the “Data Anxiety” scale F(3, 146) =1.92, p>.05. However, we did observe statistically significant differences among scale means by job role on the “Beneficence of Data Scale” F(3, 147) = 3.24, p<.05: Here, post-hoc tests revealed a difference between the means of central office leaders and teachers of .52 (with 95% certainty that the true mean difference fell between .05 and .99). We initially interpreted these analyses to suggest that Pence educators are generally more alike than not across job roles when it comes to perceptions and attitudes about data use. First, scale means tended toward the positive end for educators in all job roles as most reported the use of something other than intuition or hunches to inform practice. Second, educators in Pence tended towards neutrality or slight disagreement on the data anxiety scale (i.e., worry that data would be misused or would catalyze unhealthy competition among educators). And, third, educators in Pence tended to agree with statements that asserted a positive potential for data use to support teaching and learning efforts. Although scale data suggested only slight differences among roles in how educators in Pence construed the elements used in informing practice (see Table 3), open-ended survey data suggested more stark divides. This may suggest that some teachers responded to Likert-style items in ways they thought were expected of them as professional educators, rather than in ways that more accurately reflected their perceptions, but it could also be that perceptions on data use are complex, and only within the formats that allowed for greater depth of answers (i.e., open-ended survey items and interviews or focus groups) were disparities revealed. Table 4 presents a breakdown of the content of comments by role. District-level leaders tended to be uniformly positive about the potential of data to inform instruction for improvement purposes. When asked what “comes to mind” regarding the terms “data use” or “working with data,” none of the 11 respondents produced negative models, related a negative experience, and only one commented about accountability testing, specifically. However, we note that when asked about recent professional learning experiences related to data use, four of the 11 district leaders cited examples that related to the use of accountability exams, accountability data, or the use of data to address AYP or state accountability concerns. Table 3 Selected Scale Means and Standard Deviations by Role Role Construal α=.85 Data Anxiety α=.88 Beneficence of Data Use α=.78 District-level Leaders M= 4.45, SD=.52 M= 2.09, SD=.93 M= 4.43, SD=.53 Campus-level Leaders M= 4.00, SD=.50 M= 2.57, SD=.67 M= 4.05, SD=.54 Professional Support M= 3.95, SD=.38 M= 2.82, SD=.90 M= 3.79, SD=.76 Teachers M= 3.97, SD=.70 M= 2.77, SD=.95 M= 3.92, SD=.56 Overall M= 4.00, SD=.66 M= 2.71, SD=.94 M= 3.95, SD=.58 Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice 11 Table 4 Frequencies of Coded Text (Open-Ended Survey Items) by Role   Role N of participants offering any comments Comments suggesting a negative/frustrated perspective toward data/data use Comments suggesting a positive/embracing attitude toward data/ data use Comments linked to state testing or accountability requirements District Leaders 11 0 7 5 Campus Leaders 6 2 2 3 Professional Support 12 5 2 7 Teachers 102 37 24 28 Total 131 44 35 35 Responses from campus-level leaders were mixed. Some linked the terms “data use” and “working with data” to curriculum planning, and to looking at problem areas and areas of strength. One participant noted that the terms engendered both “excitement and frustration.” And, a few more negatively connoted phrases crept in at the campus leader level: “Not so much in the ‘fun’ department!” and “Pouring through lots of figures and numbers” were cited as immediate thoughts upon hearing one was going to be “involved in data use.” This suggests that for some, “data use” was associated with “numbers” and quantifiable measures. In response to the item asking about prior data-related professional learning experiences, several campus leaders cited accountability-oriented activities: “We learned how to spot holes in instruction using TAKS data,” noted one, while another noted a recent “data dig” in which departments examined “…what objectives were the lowest and highest on the most recent TAKS/EOC.” Persons in professional support roles were mixed in how they construed “data use,” and we observed an increase in negative or skeptical responses. Of the 12 persons who offered comments, seven referenced state testing requirements. These were couched in less-than- glowing terms (“Confusing numbers and over testing of students,” “A very long and arduous meeting. Lots of things that have to be interpreted by someone who knows how to read the data”). One stated that when working with data, “I get stressed and lack confidence.” The trajectory from uniformly positive and predominantly improvement-oriented models for data use to decidedly more mixed perspectives reached an apex with teachers. To be sure, many teachers were generally positive about data use. These teachers spoke to the ability of data to inform improvement efforts at the campus and classroom levels. Comments included “That we will look at data to better understand why students [perform] at certain levels and how we can use that information to address areas of need”; “Trying to find where the weaknesses and strengths are. Where can we improve?”; and “I want to immediately know exactly how my students have performed so that I know what my strengths and weaknesses as a teacher are.” However, many comments were skeptical. For example, one teacher said the term brought to mind, “Hopefully, useful information that clearly shows us strengths and weaknesses of our students to guide instruction… A fear that we will be gathering so much data that we lose sight of the fact that we need to teach the objectives being tested.” Other teachers noted outright frustration or hostility toward data use. One noted that data use was “tiresome” and another admitted, “Sometimes I worry about data-about my own students [data] being viewed by fellow coworkers.” Another said what came to mind is “Fear…of looking at the data and feeling like a failure.” One admitted, “I feel like it will be bad because our scores are low. I never look forward to going to look at data” while yet another offered a one-word response: “punishment.” Another noted that when told she would be “involved in ‘data use’ or ‘working with data’”: I think of columns of numbers that do not really have a true understanding of the student or of their performance. I think that sometimes we get so focused on numbers that we forget about the individuals behind those numbers. We need to remember that this was one test given in a few hours on one day of that child's life. They are inundated with “real world” issues every day, and to assess a child's learning based on a multiple choice test given with so many restrictions does not, in my opinion, give us an accurate assessment of what that child has learned. We note that of these comments, tinged by frustration or hostility, several were focused on state testing and accountability issues rather than “data use” in and of itself; however, as the question posed specified “data use,” not “accountability testing,” we found the number of responses among teachers that defaulted to an accountability-oriented model telling. In sum, analysis of Likert-type items suggested that all educators (regardless of role) used various elements to inform practice. Further, these data suggest that to a large degree, educators (across roles) in Pence ISD reported that data use has some positive attributes Current Issues in Education Vol. 16 No. 1 12 and offers a degree of promise in terms of informing teaching and learning efforts. However, when we examined open-ended survey data, we found that there are nuanced meanings associated with the terms “data” and “data use” that dredge up negative associations and accountability-dominated themes among teachers at greater rates than for district-level leaders. Investment in Data-Informed Practice Our second guiding question focused on the intersection of teacher commitment to or investment in data-informed practice and various mental models of data use among teachers. To explore this issue, we focused on teacher responses and conducted three analyses. Commitment to data use and beneficence. The first analysis examined the relationship among level of beneficence attributed to data use and reported commitment to data use among teachers. Reported commitment to data use among teachers differed by the degree of beneficence attached to data use (see Table 5). We observed that teachers who attributed higher levels of beneficence to data use were overrepresented among those who reported high commitment to data use (see Table 5). Of the 116 teachers surveyed, 64 attached a high level of beneficence to data use, and of those, 53 (82.8%) reported a high level of commitment to data use. This suggests that among Pence teachers, those who attach positive intent to data use also perceive data use to be worthwhile enough to warrant investment of time and effort. Commitment to data use and accountability- dominant orientation. The second analysis examined the relationship among level of agreement that data use is “all about accountability ratings” and reported commitment to data use among teachers. We found that reported commitment to data use differed by the degree to which teacher responses suggested an accountability-dominant orientation towards data use (see Table 6). Upon closer examination, we observed that persons whose responses suggested they did not equate “data use” strictly with “accountability ratings” were overrepresented among those who reported high commitment to data use. Of the 39 teachers in our sample whose responses indicated a low or non-accountability-dominant orientation for data use, 27 (87.2%) reported high commitment to data use, whereas only five (14.3%) of the teachers who reported lower levels of commitment to data use indicated a non- accountability dominant orientation. This suggests that among Pence teachers, those who reject a “data use equals accountability ratings” model also perceive data use to be an enterprise worthy of commitment. Table 5 Degree of Beneficence Attributed to Data Use by Reported Commitment to Data Use Reported Commitment to Data Use Pearson Chi-Square Test Low-Lukewarm High X2(df= 2) p Level of Beneficence attributed to data use N % N % 11.67 .003 Low (N=3) 1 33.3% 2 66.7% Moderate (N=49) 23 46.9% 26 53.1% High (N=64) 11 17.2% 53 82.8% Table 6 Accountability Orientation by Reported Commitment to Data Use Reported Commitment to Data Use Pearson Chi-Square Test Low-Lukewarm High X2 (df= 2) p Degree of Accountability Orientation N % N % Non-accountability orientation (N=39) 5 12.8% 34 87.2% 9.58 .008 Neutral orientation (N=38) 17 44.7% 21 55.3% Accountability orientation (N=39) 13 33.3% 26 66.7% Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice 13 Table 7 Anxiety About Data Use by Reported Experience of Data Misuse/Abuse Reported Anxiety about Data Use Pearson Chi-Square Low-No Anxiety Moderate- High Anxiety X2 (df= 2) p Reported personal experience with data misuse or abuse N % N % Has never experienced data misuse/abuse (N=40) 40 100.0% 0 0.0% 51.21 .000 Neutral (N=9) 7 77.8% 2 22.2% Has experienced data misuse/abuse in one or more work contexts (N=13) 13 26.5% 36 73.5% Anxiety about data use and prior bad acts. Last, we examined the relationship among level of anxiety over data use reported by teachers and whether teachers reported having worked in one or more schools where data were used to “shame or punish” teachers. To this end, we created a two-level variable from the Data Anxiety scale. Persons who scored 2.5 or lower were considered to evidence low levels of anxiety about data use. Persons who scored 3.5 or higher were considered to evidence moderate to high levels of anxiety about data use. For this analysis we excluded those whose responses indicated neutrality in terms of anxiety over use (i.e., those who scored between 2.5 and 3.5). We found that reported anxiety about data use differed by whether teachers reported having worked in a school where data were misused/abused (see Table 7). Upon closer examination, we observed that teachers who reported having worked in one or more contexts where data were used to shame or punish teachers were overrepresented among those who scored at levels of moderate to high anxiety. Of 98 teachers, 38 reported moderate or high levels of anxiety about data use, and of these, 36 (94.7%) reported having worked in a context of data misuse/abuse. Conversely, of the 40 teachers who reported having never worked in a context of data misuse/abuse, none scored at levels of moderate to high anxiety about data use. From these findings we drew two overarching conclusions. First, Pence teachers who have experienced misuse or abuse of data are significantly more likely to report anxiety or concern about data use. Second, prior knowledge or experience with data use has consequences for how teachers think about and model “data use.” We note that our survey item did not inquire as to whether these teachers had experienced data misuse/abuse in Pence, but whether they had worked in “one or more contexts” over the course of their careers. This suggests that even if current principals do not engage in destructive patterns of data use, they may have to engage in “make up” work to negate the prior bad acts of leaders in other schools or districts. Discussion This study was not conceived in an attempt to generalize findings to all public schools, but to describe and provide insight into the context of data use within Pence ISD. However, we do think our findings may prove useful to leaders in other contexts as they consider factors that may impact their efforts to improve data-informed practice. To that end, in this section we ask, “What broader lessons might we take from the situation in Pence ISD, as a starting point for further research and discussion?” When we examine our findings through the lens of our theoretical framework, we contend that there are three such lessons: (1) Educators hold nuanced differences in how they think about and respond to the terms “data” and “data use”; (2) Leaders have an opportunity to reframe mental models of data use to engender increased commitment to data use among teachers; and (3) Because different school contexts are comprised of actors with varying conceptions of “data” and “data use,” the onus is on school leaders to facilitate the co-construction of shared mental models for data use. Nuanced Differences in Mental Models for Data Use Educators across Pence ISD appear to grasp the expectation that they should use evidence to inform what happens in the classroom. And, they appear to have taken ownership of this expectation to a large degree. Even in the face of some anxieties about data use, a substantial number of educators across job roles in Pence ISD report using a variety of data to inform efforts related to teaching and learning, though some may refer to “evidence” or “information” rather than “data” in describing what they indeed use. The mental models associated with the terms “data” and “data use” ranged from positive (i.e., the Current Issues in Education Vol. 16 No. 1 14 central office leader who was “thrilled” by data use) through neutral to negative (i.e., those who associated the terms with “punishment,” said the terms evoked the thought, “yuck,” or admitted that, “I feel like it will be bad because our scores are low. I never look forward to going to look at data”). For some educators, these models may be triggered by a lack of confidence in skills relating to data use. This fits with existing research that suggests that although data use has been part of the education landscape for decades, teachers and school leaders still struggle to make sense of copious amounts of data in ways that inform practice (Earl & Fullan, 2003; Kerr, et al., 2006; Means, Chen, DeBarger, & Padilla, 2011; Wayman et al., 2007). Additionally, as suggested by the open-ended survey items, many teachers seem to equate “data use” with the use of accountability test scores. In this case, mental models may be skewed by prior bad experiences with leaders who misused data in some way: Many educators in Pence ISD reported having worked in one or more contexts where they perceived data were used to shame or punish teachers. These findings align with research documenting data misuse/abuse by some school leaders (Earl & Fullan, 2003; Ingram et al., 2004; Valli & Buese, 2007). We note that while accountability systems predicated largely upon the results of exams may indeed bring needed attention to patterns of underachievement or to needed improvements in teaching, our concern with an over-identification of “data use” with “test scores” and the use or misuse of these particular types of data is twofold. First, where teachers’ mental models equate “data use” with “test scores,” they may unwittingly lock attention onto standardized exam data to the exclusion of other, valuable data (e.g., classroom-based assignments, quizzes, portfolios, parent feedback) in the course of practice. Research suggests that a key component of effective data use is leading teachers away from this narrowed mindset and encouraging a broader construal of “data”—one that is inclusive of multiple elements (Coburn & Turner, 2012; Schildkamp & Kuiper, 2010; Spillane & Miele, 2007). Second, when teachers are overly anxious about how data may be used, they may fall into routinized data use that serves to shore up short term gains, but at the expense of broader improvements that necessitate contexts of trust and sharing among colleagues. Such patterns of threat rigidity have been explored in connection with school reform efforts (Daly, 2009; Olsen & Sexton, 2009; Spillane & Miele, 2007). Senge’s (2006) work with mental models suggests that diverging mental models are not particularly problematic for an organization if the actors in the organization are committed to carefully examining the varying models with an aim toward creating some shared understandings from which to operate. Whatever the cause of these diverging mental models in Pence ISD, teachers seemed to be mixed in terms of how they referenced “data” and “data use”: Some embraced the term “data” (as well as ‘information’ and ‘evidence’), and others embraced the concept but seem suspicious of or hostile to the terms “data” and “data use.” Although we are encouraged that so many teachers embraced the underlying concept of using multiple elements to inform instruction, we posit that in leaving these mental models unexamined, school leaders and teachers may unwittingly talk past one another in planning and decision-making. Simply engaging in sporadic discussions about data will not alleviate all issues related to the complexities inherent in using data well. Still, we think that if leaders would engage teachers in ongoing dialogues about student needs vis-à-vis collaborative inquiry or action research models, both parties could take a positive step towards improved collaborative data use. Senge’s work (2006) suggests that such dialogues are essential in getting members of an organization to create authentic and shared goals. He notes: The purpose of a dialogue is to go beyond any one individual’s understanding. … individuals gain insights that simply could not be achieved individually. … In dialogue, a group explores complex difficult issues from many points of view. Individuals suspend their assumptions but they communicate their assumptions freely. The result is a free exploration that brings to the surface the full depth of people’s experience and thought, and yet can move beyond their individual views. (pp. 223-224) Specific to data use, Park and Datnow (2009) found that purposes for data use are actually co-created in a nuanced give-and-take among leaders and teachers. That is, by engaging in dialogue on what “data” means and how “data use” looks in practice, the mental models of all parties are likely to change. Reframing Mental Models for Data Use Because the terms “data” and “data use” evoke negative or anxiety-producing mental models for a good number of teachers in Pence ISD, leaders could opt to reframe data use using a related model (e.g., action research or collaborative inquiry), hopefully eschewing the negative connotations with “DDDM” or “data use” while increasing buy-in to a data-informed process. Here we build on Morgan’s (2006) notion of using disruptors to “jar” the attractors around which organizational patterns of action or thinking are oriented. If we operate on the assumption that, for some teachers, prior experiences with data misuse/abuse cause them to associate the terms “data” and “data use” with accountability exams and broad scale assessments (to the detriment of including other forms of data), then we open up the possibility that responsible, constructive modeling of data over time by existing leaders can broaden the construal of “data use” Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice 15 for these teachers (see Kerr et al., 2007; Spillane & Miele, 2007). Efforts to broaden the scope of what “data use” entails may also avoid what Daly (2009) termed “rigid response”—or the tendency to respond in limited and generally non-innovative ways to accountability pressures, with less than optimal results. We noted that survey data indicated that teachers with higher reported commitment to data-informed practice tended to more strongly associate data use with continuous improvement, to credit data use with supporting informed and ethical decisions, and to push back on the notion that data use is all about accountability ratings. By working to reframe data use as beneficial to teaching and learning, and by refraining from misusing and abusing data, leaders may be able to increase the numbers of teachers who view themselves as committed to data-informed practice. In a similar vein, leaders who shift the focus from what they are monitoring to what they are modeling may be able to “jar” the mental models from models that privilege data use to those that highlight collaborative inquiry with data use being a critical component. In other words, instead of “leading data use” by compiling or highlighting reports and presenting test score data in faculty meetings, leaders might shift toward active engagement in collaborative groups with teachers and teacher leaders. In this way, the “leader” becomes the “co-learner,” examining multiple forms of data, asking questions, brainstorming instructional solutions, and evaluating outcomes. This type of positive leader modeling has been linked routinely to effective data use in the literature (Copland, 2003; Datnow et al., 2007; Kerr et al., 2007; Louis et al., 2010; Park & Datnow, 2009; Wayman Spring et al., 2012). To effect better data use, leaders must make the turn from simply managing (i.e., monitoring and requiring reports) to modeling (working alongside teachers to address problems of practice). Principals and district leaders in Pence ISD might thus opt to subsume “data use” in another model— i.e., “collaborative inquiry” or “action research”—rather than focusing on convincing teachers to be involved in “data-driven” practice. These models (as noted in our review of the literature) get to the heart of the concept itself: they promote the use of multiple measures of data (thereby getting away from “data” being understood only as “test scores”). They prioritize collaboration around data (which helps create a culture of shared rather than isolated practice). And, these models elevate data as a resource to inform the work of localized and continuous improvement driven by professionals, rather than identify “data use” with strict monitoring and compliance-driven activities that can mutate into data misuse/abuse. Building Shared Mental Models for Data Use As one of our priorities in undertaking this study in Pence ISD was to identify ways that school and district leaders could leverage professional learning to support improved data use, we also note the value of educators from all job roles spending time in dialogue with an aim of creating common understandings related to data use. The importance of devoting time and effort to uncovering assumptions about data use and building toward common understandings about data (e.g., “What counts as ‘data’?” “What data shall we use, and to what ends?” “How will we collaborate around data?”) appears frequently in research on effective data use (Hamilton et al., 2009; Park & Datnow, 2009; Wayman et al., 2007; Wayman, Jimerson, & Cho, 2012). Similar processes for building shared mental models appear in literature on professional learning communities (DuFour, Eaker, & DuFour, 2005; Hord, 1998; Thessin & Starr, 2011). Given our findings that suggest overlapping yet nuanced construals of “data” and “data use” among educators in Pence ISD, our stance is that district leaders and principals might be well-served by applying Senge’s (2006) work on both mental models and shared vision to data use efforts in more explicit fashion. The alternative to this intentionality in practice is to assume that educators come to the table with similar understandings, biases, and perspectives related to data and data use, and neither our research in this context, nor other research in the broader data use literature, suggests that this is the case. Conclusion In Pence ISD, educators across job roles reported using various elements to inform daily practice; however, analysis of survey data (including open-ended items) revealed nuanced differences in the terms they used to describe these elements. Further, we found a range of construals attached to educators’ mental models for data use: Some models seemed rooted in constructive, positive construals of “data use” while others seemed to tightly couple “data use” with accountability and compliance activities. Teachers who attached more positive attributes to data use also reported increased commitment to using data to inform practice. Educators in all roles are pressed to use more and more formal data in the pursuit of improved student outcomes: We suggest that school leaders might actually increase teacher ownership of and commitment to data-informed practice by, ironically, downplaying “data use” or “DDDM” and highlighting inquiry-oriented models that avoid accountability-laden language in favor of a tight focus on student-centered problems of practice. References Anderson, S., Leithwood, K., & Strauss, T. (2010). Leading data use in schools: Organizational conditions and practices at the school and district levels. Leadership and Policy in Schools, 9(3), 292-327, DOI: 10.1080/15700761003731492/ Booher-Jennings, J. (2005). Below the bubble: “Educational triage” and the Texas accountability system. American Educational Current Issues in Education Vol. 16 No. 1 16 Research Journal 42(2), 231-268. Boudett, K. P., City, E. A., & Murnane, R. J. (Eds.). (2005). Data Wise: A step-by-step guide to using assessment results to improve teaching and learning. Cambridge, MA: Harvard Education Press. Buhle, R., & Blachowicz, C. L. Z. (2009). The assessment double play. Educational Leadership 66(4), 42- 46. Calhoun, E. (1994). How to use action research in the self-renewing school. Alexandria: ASCD Coburn, C. E., Honig, M. I., & Stein, M. K. (2009). What’s the evidence on districts’ use of evidence? In J. D. Bransford, D. J. Stipek, N. J. Vye, L. M. Gomez, and D. Lam (Eds.), The role of research in educational improvement (67-87). Cambridge: Harvard Education Press. Coburn, C. E., & Talbert, J. E. (2006). Conceptions of evidence use in school districts: Mapping the terrain. American Journal of Education 112 (August), 469-495. Coburn, C. E., & Turner, E. O. (2012). The practice of data use: An introduction. American Journal of Education, 118 (2), 99-111. Colyvas, J. A. (2012). Performance Metrics as Formal Structures and through the Lens of Social Mechanisms: When Do They Work and How Do They Influence? American Journal of Education, 118(2), 167-197. Copland, M. (2003) Leadership of inquiry: Building and sustaining capacity for school improvement. Educational Evaluation and Policy Analysis 25(4), 375-395. Daly, A. (2009). Rigid response in an age of accountability: The potential of leadership and trust. Educational Administration Quarterly 45(2), 168-216. Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with data: How high-performing school systems use data to improve instruction for elementary students. Los Angeles: University of Southern California. Desimone, L. M., & LeFloch, K. C. (2004). Are we asking the right questions? Using cognitive interviews to improve surveys in education research. Educational Evaluation and Policy Analysis 26(1) 1-22. DuFour, R., Eaker, R., & DuFour, R. (2005). Recurring themes of professional learning communities and the assumptions they challenge. In R. DuFour, R. Eaker, and R. DuFour (Eds.), On Common Ground: The Power of Professional Learning Communities (pp. 7-29). Bloomington, IN: Solution Tree. Earl, L., & Fullan, M. (2003). Using data in leadership for learning. Cambridge Journal of Education, 33(3), 383-394. Firestone, W. A., & Gonzalez, R. A. (2007). Culture and processes affecting data use in schools. In P. A. Moss (Ed.), Evidence and decision making (pp. 132-154). Malden, MA: Blackwell. Glanz, J. (2003). Action research: An educational leader’s guide to school improvement (2nd ed.). Norwood, MA: Christopher-Gordon. Goren, P. (2012). Data, data, and more data—What’s an educator to do? American Journal of Education, 118(2), 233-237. Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. (2009). Using student achievement data to support instructional decision making (NCEE 2009- 4067). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/publications/practice guides/. Hendricks, C. (2009). Improving schools through action research: A comprehensive guide for educators (2nd ed.). Upper Saddle River, NJ: Pearson. Holme, J. J. (2013). Exit strategies: How low-performing high schools respond to high school exit examination requirements. Teachers College Record 115(1). Accessed 3/13/13 at http://www.tcrecord.org/library ID Number 16736. Holme, J. J., Carkum, R., & Snodgrass Rangel, V. (in press). High pressure reform: Examining urban schools’ response to multiple school choice policies. The Urban Review, 1-30. Honig, M. I., & Venkateswaran, N. (2012). School— central office relationships in evidence use as a systems problem. American Journal of Education, 118(2), 199-222. Hord, S. M. (1998). Creating a professional learning community: Cottonwood Creek School. Issues about Change 6(2), pp. 1–8. Ingram, D., Louis, K. S., & Schroeder, R. G. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers College Record 106(6), 1258- 1287. Jimerson, J. B. (2011). “Doing data”: Addressing capacity for data use through professional learning. Unpublished dissertation. Austin, TX: The University of Texas at Austin. Jimerson, J. B. (2012). Survey of Data Use & Professional Learning. Unpublished document. Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice 17 Jimerson, J. B., & Wayman, J. C. (2012). Branding educational data use through professional learning: Findings from a study in three school districts. Paper presented at the 2012 Annual Meeting of the American Educational Research Association, Vancouver, British Columbia. Kerr, K. I., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education 112(August), 496-520. Louis, K. S., Leithwood, K., Wahlstrom, K. L., & Anderson, S. E. (2010, July). Learning from Leadership: Investigating the Links to Improved Student Learning (Final Report of Research Findings). University of Minnesota and University of Toronto. Mandinach, E. B. (2012). A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist 47(2), 71-85. Mandinach, E. B., Honey, M., & Light, D. (2006). A theoretical framework for data-driven decision making. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. Retrieved 12/8/2012 from www.cct.edc.org Marsh, J. A., McCombs, J. S., & Martorell, F. (2010). How instructional coaches support data-driven decision making. Educational Policy 24(6), 872- 907. Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education. RAND. Retrieved 12/17/2012 from http://www.rand.org/content/dam/rand/pubs/occa sional_papers/2006/RAND_OP170.pdf. Marshall, C., & Rossman, G. B. (1999). Designing qualitative research (3rd ed.). Newbury Park, CA: Sage. Means, B., Chen, E., DeBarger, A., & Padilla, C. (2011). Teachers’ ability to use data to inform instruction. Washington, D.C.: U.S. Department of Education Office of Planning, Evaluation, and Policy Development. Means, B., Padilla, C., DeBarger, A., & Bakia, M. (2009). Implementing data-informed decision-making in schools—Teacher access, supports and use. Washington, D.C.: U.S. Department of Education Office of Planning, Evaluation and Policy Development. Merriam, C. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA: 2009. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage. Morgan, G. (2006). Images of organization. Thousand Oaks, CA: Sage Publications. Nelson, S., McGhee, M. W., Slater, C., & Meno, L. (2007), Fulfilling the promise of educational accountability, Phi Delta Kappan, 88(9), 702- 709. Olsen, B., & Sexton, D. (2009). Threat rigidity, school reform, and how teachers view their work inside current education policy contexts. American Educational Research Journal 46(1), 9-44. Park, V., & Datnow, A. (2009). Co-constructing distributed leadership: District and school connections in data-driven decision-making. School Leadership and Management, 29(5), 477- 494. Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and teacher education, 26, 482–496. Senge, P. M. (2006). The fifth discipline: The art & practice of the learning organization (Rev). New York: Currency Doubleday. Shen, J., & Cooley, V. E. (2008). Critical issues in using data for decision-making. International Journal of Leadership in Education, 11(3), 319-329. Spillane, J. P., & Miele, D. B. (2007). Evidence in practice: A framing of the terrain. In P.A. Moss (Ed), Evidence and Decision Making: 106th Yearbook of the National Society for the Study of Education (46-73). Supovitz, J. (2010). Knowledge-based organizational learning for instructional improvement. In Hargreaves, A., Lieberman, A., Fullan, M., & Hopkins, D. (Eds.). Second international handbook of educational change (pp. 707-723). New York: Springer. Supovitz, J. A., & Klein, V. (2003). Mapping a course for improved student learning: How innovative schools systematically use student performance data to guide improvement. Philadelphia: Consortium for Policy Research in Education- University of Pennsylvania. Thessin, R. A., & Starr, J. P. (2011). Supporting the growth of effective professional learning communities districtwide. Phi Delta Kappan 92(6): 48-54. Tucker, B. (2010). Putting data into practice: Lessons from New York City. Washington DC: Education Sector Reports. Valenzuela, A., Fuller, E., & Vasquez Heilig, J. (2006). The disappearance of high school English language learners from Texas high schools. Williams Review, 1(1), 166-200. Valli, L., & Buese, D. (2007). The changing roles of teachers in an era of high-stakes accountability. Current Issues in Education Vol. 16 No. 1 18 American Educational Research Journal 44(3) 519-558. Vasquez Heilig, J., & Darling-Hammond, L. (2008). Accountability Texas-style: The progress and learning of urban minority students in a high- stakes testing context. Educational Evaluation and Policy Analysis. 30(2), 75-110. Wayman, J. C., & Cho, V. (2009). Preparing educators to effectively use student data systems. In Kowalski, T. J., & Lasley, T. J. (Eds.), Handbook on data-based decision-making in education (pp. 89-104). New York: Routledge. Wayman, J. C., Cho, V., Jimerson, J. B., & Spikes, D. (2012).District-wide effects on data use in the classroom. Education Policy Analysis Archives 20(25). Retrieved from http://epaa.asu.edu/ojs/article/view/979. Wayman, J. C., Cho, V., & Johnston, M. T. (2007). The data-informed district: A district-wide evaluation of data use in the Natrona County School District. Austin: The University of Texas. Wayman, J. C., Jimerson, J. B., & Cho, V. (2012). Organizational considerations in establishing the Data-Informed District. School Effectiveness and School Improvement 23(2): 159-178. Wayman, J. C., Spring, S. D., & Lemke, M. A., Lehr, M. D. (2012, April). Using data to inform practice: effective principal leadership strategies. Paper presented at the 2012 Annual Meeting of the American Educational Research Association, Vancouver, Canada. Wayman, J. C., & Stringfield, S. (2006). Technology- supported involvement of entire faculties in examination of student data for instructional improvement. American Journal of Education, 112(August), 549-571. Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice 19 Appendix A Coding scheme for open-ended survey items Prior bad acts (PBA) Reference to negative experiences with data, examples or narratives of data misuse or abuse, or assertions of first-hand experiences with data being used to shame or punish teachers or schools. Rationale I Comments associated with data use being primarily purposed for continuous improvement, classroom/teaching improvement, or schoolwide improvement initiatives. Rationale A/C Comments associated with data use being primarily purposed for fulfilling compliance- oriented requirements, for meeting accountability standards (with no mention of improvement efforts), of “teaching to the test” or of goal orientations regarding accountability ratings (state or federal) Positive Leader Modeling Comments or narratives/examples regarding learning about data use through modeling of another leader—learning about data use in a way that created a motivation to emulate the principal/campus leader. Negative Leader Modeling Comments or narratives/examples regarding learning about data use via non-example of a leader—i.e., witnessing a type of data use that repulsed or repelled the educator from wanting to use data in similar ways. Positive/Embracing Attitude Comments that reflect enjoyment of data use, embracing of data for general good, or a valuing of the data use process Negative/Frustrated Attitude Comments that reflect a resistance to data use, a dislike or reticence toward the process, or a pushback against using data to inform instruction Genesis Comments that describe where a teacher or leader’s mental model for data use originated (e.g., leader, teacher or leader prep program, self-study) Definitions Delineations among the words “data,” “evidence” and “information” Testing Any references to testing or state tests following a question that asked about “data use” Current Issues in Education Vol. 16 No. 1 20 Appendix B Variables Index Construal Composite variable composed of the three items on the “Construal of Data” scale, range from 1 (Strongly Disagree to 5 (Strongly Agree) • I use a variety of data to inform my teaching and/or daily practice. • I use a variety of information to inform my teaching and/or daily practice. • I use a variety of evidence to inform my teaching and/or daily practice. Beneficence Composite variable composed of four items on the “Beneficence of Data” scale; range from 1 (Strongly Disagree to 5 (Strongly Agree) • Data use helps me make informed decisions. • Data use helps me make ethical decisions. • Data use benefits educators and students. • Data use is about continuous improvement in the classroom. Anxiety Composite variable composed of three items on the “Data Anxiety” scale, range from 1 (Strongly Disagree to 5 (Strongly Agree) • I worry that data will be used to shame or punish teachers at my school. • I am concerned that data use will increase unhealthy competition among educators at my campus. • I am concerned that data use will increase unhealthy competition among educators in my district. Commitment to Data Use 2-level variable generated by recoding responses to the question, “If no one (on my team, in my department) wanted to use data, I would still make the effort to do so). • 1=”Low/Lukewarm commitment” included responses of 1 (Strongly Disagree), 2 (Disagree) and 3 (Neither agree nor disagree) • 2=”High commitment” included responses of 4 (Agree) and 5 (Strongly Agree) Rounded Beneficence 3-level variable generated by rounding the composite variable “Beneficence” • 1=”Low Beneficence” (responses ranging from 1-2.99) • 2=”Moderate Beneficence” (responses ranging from 3.00-3.99) • 3=”High Beneficence” (responses ranging from 4.0-5.0) Accountability Orientation 3-level variable generated by recoding responses to the item, “Data use is all about accountability ratings.” • 1=”Non-accountability oriented) (responses of “Strongly disagree” or “Disagree”) • 2=”Accountability neutral” (responses of “Neither agree nor disagree”) • 3=”Accountability oriented” (responses of “Agree” “Strongly Agree”) Anxiety Level 2-level variable generated by recoding the composite variable “Data Anxiety” • 1=Low/No Anxiety (1-2.75) • 2=Moderate/High Anxiety (3.25-5) • Responses between 2.75 and 3.25 were excluded in the creation of this variable because those represented scores near the “neither agree nor disagree”—or neutral—midpoint of the scale. Reported Data Misuse/Abuse 3-level variable generated by recoding the item, “I have worked in one or more contexts where data were used to shame or punish teachers” • 1=Not experienced data use/misuse (responses of “Strongly Disagree” and “Disagree”) • 2=Neutral (responses of “Neither agree nor disagree”) • 3=Experienced data misuse/abuse (responses of “Agree” and “Strongly Agree) Leading Inquiry in Schools: Examining Mental Models of Data-Informed Practice 21 Article Citation Jimerson J.B., & McGhee, M. W. (2013). Leading inquiry in schools: Examining mental models of data-informed practice. Current Issues in Education, 16(1). Retrieved from http://cie.asu.edu/ojs/index.php/cieatasu/article/view/1163 Author Notes Jo Beth Jimerson, Ph.D. Texas Christian University TCU Box 297900 Fort Worth, Texas 76912 817-257-6777 j.jimerson@tcu.edu Jo Beth Jimerson serves as an Assistant Professor of Educational Leadership at Texas Christian University, where she works with aspiring school leaders and studies issues related to educational data use and school improvement efforts. Marla W. McGhee, Ph.D. Texas Christian University TCU Box 297900 Fort Worth, Texas 76912 817-257-6777 m.mcghee@tcu.edu Marla McGhee is an Associate Professor of Educational Leadership at Texas Christian University in Ft. Worth, Texas where she focuses her work on action research for school improvement, instructional leadership and supervision, and leadership practices that promote information literacy and effective school library media programs. Acknowledgements: The authors wish to thank Judy Groulx and Annie Nguyen for their support and input on earlier drafts of this paper, as well as the anonymous reviewers who provided constructive suggestions in the review process. We also thank the educators of “Pence Schools” for the time they so graciously gave to participate in this study. Manuscript received: 01/08/2013 Revisions received: 03/27/2013 Accepted: 04/04/2013 Current Issues in Education Vol. 16 No. 1 22 Volume 16, Number 1 April 26, 2013 ISSN 1099-839X Authors hold the copyright to articles published in Current Issues in Education. Requests to reprint CIE articles in other journals should be addressed to the author. Reprints should credit CIE as the original publisher and include the URL of the CIE publication. Permission is hereby granted to copy any article, provided CIE is credited and copies are not sold. Editorial Team Executive Editors Melinda A. Hollis Rory Schmitt Assistant Executive Editors Laura Busby Elizabeth Reyes Layout Editors Bonnie Mazza Elizabeth Reyes Recruitment Editor Hillary Andrelchik Copy Editor/Proofreader Lucinda Watson Authentications Editor Lisa Lacy Technical Consultant Andrew J. Thomas Section Editors Hillary Andrelchik Michelle Crowley Ayfer Gokalp Darlene Michelle Gonzales Courtney Hart David Isaac Hernandez-Saca Sultan Kilinc Younsu Kim Linda S. Krecker Carol Masser Bonnie Mazza Constantin Schreiber Faculty Advisors Dr. Gustavo E. Fischman Dr. Jeanne M. Powers