EPAAv20n25 Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 07/11/2011 Facebook: /EPAAAAPE Revisions received: 03/15/2012 Twitter: @epaa_aape Accepted: 05/29/2012 education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 20 Number 25 August 27th, 2012 ISSN 1068-2341 District-Wide Effects on Data Use in the Classroom Jeffrey C. Wayman University of Texas at Austin Vincent Cho Boston College Jo Beth Jimerson Texas Christian University & Daniel D. Spikes University of Texas at Austin Citation: Wayman, J. C., Cho, V., Jimerson, J. B., Spikes, D. D. (2012). District-Wide Effects on Data Use in the Classroom. Education Policy Analysis Archives, 20 (25). Retrieved [date], from http://epaa.asu.edu/ojs/article/view/979 Abstract: In the present study, an examination is conducted in three school districts of how data are used to improve classroom practice. In doing so, we explore the effects that attitudes toward data, principal leadership, and computer data systems have on how data are used to affect classroom practice. Findings indicate that educators are ambivalent about data: they see how data could support classroom practice, but their data use operates in the presence of numerous barriers. Many of these barriers are due to principal leadership and computer data systems; these barriers often have negative effects on attitudes toward data and disrupt the progression from using data to inform classroom practice. It is hypothesized that many of these barriers can be removed through effective district policies to improve structures and supports for using data. Keywords: Data use; data-based decision making; educational reform. epaa aape E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 2 Efectos generales en los distritos escolares del uso de datos en el aula Resumen: En el presente estudio, se examinan como tres distritos escolares usan datos para mejorar la práctica docente. Se exploraron que efectos tienen las actitudes hacia los datos, el liderazgo de los directores/as, y los sistemas de procesamiento de datos para modificar prácticas en el aula. Los resultados indican que los educadores son ambivalentes acerca de los datos: ven cómo los datos podrían apoyar prácticas en el aula, pero el uso de datos funciona en presencia de numerosas barreras. Muchas de estas barreras se deben al liderazgo de directores/as y a los sistemas de procesamiento de datos informáticos. Estas barreras suelen tener efectos negativos en las actitudes hacia los datos e interrumpen la incorporación de datos para mejorar prácticas en el aula. Se formula la hipótesis de que muchas de estas barreras pueden ser removidas a través de políticas distritales eficaces que mejoren las estructuras y brinden apoyos para el uso de datos. Palabras clave: uso de información; toma de decisiones basadas en datos; reforma educativa. Efeitos gerais nos distritos escolares do uso de dados em sala de aula Resumo: Neste estudo, examinamos como três distritos escolares utilizaram dados para melhorar a prática docente. Foram explorados os efeitos de atitudes em relação aos dados, a liderança dos/as diretores/as, e os sistemas de processamento de dados para alterar as práticas de sala de aula. Os resultados indicam que os educadores são ambivalentes respeito a o uso dos dados: observam como os dados podem apoiar as práticas de sala de aula, mas o uso de dados se faz na presença de muitas barreiras. Muitas dessas barreiras são devidas a liderança dos/as diretores/as e aos sistemas e processamento de dados informáticos. Essas barreiras tendem a ter efeitos negativos nas atitudes com o uso dos dados e interrompem os processos de incorporar dados para melhorar as práticas de sala de aula. Se formula a hipótese de que que muitas dessas barreiras podem ser removidas através de políticas distritais eficazes que melhoram as estruturas e fornecendo suporte para o uso de dados. Palavras-chave: uso de informações; toma decisões com base em dados; reforma educacional. Introduction During the last 10 years, the field of education has witnessed a substantial increase in studies which examine how educators may use student data to help improve their practice. This research has shown a number of factors that facilitate classroom data use, such as collaboration, principal leadership, personnel supports, and effective technology (e.g., Anderson, Leithwood, & Strauss, 2010; Wayman & Stringfield, 2006; Copland, 2003; Lachat & Smith, 2005; Marsh, McCombs, & Martorell, 2010). Much of the research on educational data use has focused on case studies and empirical descriptions. Thus, there are few established causal links between student achievement and the use of data (Hamilton, Halverson, Jackson, Mandinach, Supovitz & Wayman, 2009). This notwithstanding, there is reason to believe that the effective use of data may improve schooling. For instance, the effective use of data is often cited as part of more general school improvement initiatives (Ingram, Louis, & Schroeder, 2004; Stringfield & Datnow, 2002), teachers often report changes in practice based on data use (Wayman & Stringfield, 2006; Datnow, Park, & Wohlstetter, 2007), and studies are emerging that statistically correlate student achievement to interim assessment administration (Wayman, Shaw & Cho, 2011; Carlson, Borman, & Robinson, 2011; May & Robinson, 2007). D istrict-W ide E ffects on D ata U se in the C lassroom 3 Still, a set of scalable, effective data use practices remains elusive. Knowledge of effective data practice has largely been created by studying contexts chosen for exemplary conditions (e.g., Wayman & Stringfield, 2006; Datnow et al., 2007; Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006), but studies in more typical contexts reveal persistent problems with using data (Wayman, Cho & Johnston, 2007; Wayman, Cho, & Shaw, 2009a; Valli & Buese, 2007; Young, 2006). Further, these studies are often narrow in scope; few studies have examined data use throughout an entire district (Wayman, et al., 2007) or across multiple districts (Anderson et al., 2010; Datnow et al., 2007). Thus, there remains a need for further study of how data use at all levels of a district affect classroom practice, and in contexts not chosen for their proclivity in using data. The present study responds to that need through an examination of three districts of differing size and context, none of which were chosen for their excellence in using data. Accordingly, the goal of this study is to examine how practices at every level of the district affect the use of data in the classroom. We focus our research questions on four areas that prior research suggests may be particularly important: (1) How do educators commonly use data? (2) What are educators’ attitudes toward using data? (3) How do principals lead faculty in using data? (4) How well do computer data systems support educators in using data? Research on Educational Data Use Educational research has noted the wide variety of factors that influence educational data use. In the present study, we examine four areas that arise frequently in research: attitudes toward data, principal leadership, and computer data systems. In this section, we provide a brief overview of research in each of these four areas, followed by a model that conceptualizes how these factors work together to promote effective data use. Educators’ use of data. Research on educational data use notes that uses vary by role, but has focused primarily on how teachers and principals use data. For instance, research in exemplary settings has shown teachers using a variety of student-level data to group students, re-group students, and adjust instruction based on data (Wayman & Stringfield, 2006; Datnow et al., 2007; Lachat & Smith, 2005). Research in exemplary settings has shown principals using both student- and building-level data to make policy decisions and support faculty (Copland, 2003; Datnow et al., 2007). Research has provided little detail on data uses of central office staff and instructional coaches, with the exception of studies such as Honig and Coburn (2008) and Marsh et al., (2010). Educators’ attitudes toward data. Educators in rich data-using contexts often report that educators have positive attitudes toward data when supported by a culture of data use (Wayman & Stringfield, 2006; Copland, 2003; Datnow et al., 2007; Lachat & Smith, 2005; Knapp et al., 2006). Educators in these studies typically believed that using data helped them improve practice and resulted in improvements in student learning. Educators in other contexts often display more tempered attitudes. For instance, teachers have sometimes been shown to be suspicious of data initiatives, often separating data use from their own judgment (Ingram et al., 2004; Valli & Buese, 2007; Young, 2006). Educators in all roles are sometimes hesitant to use data for fear that it will require a large time investment and little practical return (Wayman, et al., 2009a). Principal leadership. Successful school-based data initiatives are almost always marked by principals who are employing practices such as setting clear expectations for data use, involving entire faculties, and making time for teachers to use data (Wayman & Stringfield, 2006; Copland, 2003; Datnow et al., 2007; Halverson, Prichett, & Watson, 2007; Knapp, Swinnerton, Copland, & Monpas-Huber, 2006; Supovitz & Klein, 2003). On the other hand, studies of contexts not known for data use describe a wide range of principal involvement (Anderson et al., 2010; Wayman et al., E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 4 2007; Young, 2006) – some principals in these studies were effectively leading faculty in data use, but most were not. Those who were not were often characterized by these studies as either disinterested in data use or lacking a set of strategies that could foster effective faculty data use. Computer data systems. Nearly every district in the United States has some form of computer system for managing student data (Means, Padilla, DeBarger, & Bakia, 2010). When implemented effectively, these systems have been shown to facilitate many facets of educator data use, such as collaboration and rapid turnaround of data (Wayman & Stringfield, 2006; Lachat & Smith, 2005; Long, Rivas, Light, & Mandinach, 2008). Other research chronicles the struggles of educators unsupported by effective data systems. These studies have shown computer systems often lack integration, are inefficient, and are hard to use (Means et al., 2010; Wayman et al., 2007; Wayman et al., 2009a). Conceptual framework. Many scholars have presented models to describe how the use of data may improve education (e.g., Copland, 2003; Mandinach, Honey, Light, & Brunner, 2008; Supovitz, 2010). While the details of these models differ, they all share the same core logic: data use provides information that educators may employ to change practice. The research base on educational data use shows mixed results in the application of this model. While some studies show this progression to happen successfully (Wayman & Stringfield, 2006; Datnow et al., 2007; Lachat & Smith, 2005), others show that inappropriate use of data has actually hindered educational practice (Wayman, et al., 2009a; Vasquez Heilig & Darling-Hammond, 2008; Valli & Buese, 2007). Prior knowledge, information, and other elements may influence how certain data are noticed, prioritized, and used (Alavi & Leidner, 2001; Coburn, Honig, & Stein, 2009; Tuomi, 1999). Thus, we posit that the progression from data to knowledge to practice is influenced by a variety of elements. The present study focused on three such elements: attitudes toward data, principal leadership, and computer data systems. Figure 1 provides a graphic representation of our logic: Figure 1. Conceptual Framework of Data Use at the Building Level. In this conceptualization, educator use of data provides information that influences classroom practice. However, educator uses of data are influenced by an educator’s attitude toward data, principal leadership, and the access they gain to data through data systems. It is important to distinguish data from information. Data are the raw inputs (e.g., student test scores or teacher observations) that educators may access about their students; information is the processed outcome of these data. Thus, educators access data from their computer data system and process it into information through their uses of data. This information is used to make changes to classroom practice. Uses of Data Information Attitudes toward data Computer data systems Principal leadership Classroom Practice D istrict-W ide E ffects on D ata U se in the C lassroom 5 While this is a building-level model, it is deliberately not role-specific, allowing for roles such as central office or instructional support to influence how practice occurs in the classroom. For example, how an instructional coach’s uses of data influences teachers’ classroom practice will be influenced by the coach’s attitude toward data, the leadership of campus principal(s) and the data systems that provide access. It is also important to note that the model does not assume that data use leads to improved practice – instead, the model allows for the possibility that these or other influences might result in data use that leads to more or less effective practice. Method Introduction The present study is drawn from a larger three-year project designed to help three school districts improve their use of data by employing a systemic focus called “the Data-Informed District (Wayman, 2010; Wayman et al., 2007). It is important to define three terms used in this project. First, we use a broad and encompassing definition of data, meaning anything that helps educators know more about their students (e.g., formal assessments, tests, quizzes, and student background data). Second, data use means the actions in which educators engage as they collect these data, organize and analyze them, and draw meaning from them to inform practice. Third, we often use the term effective data use to distinguish between data use practices that benefit educators in their practice (and which thus benefits student learning) from other data use practices that have been shown to actually hinder educational work (Wayman et al., 2007; Earl & Fullan, 2003; Valli & Buese, 2007; Young, 2006). Data were collected in three districts in Texas1 during the 2009-2010 school year. Districts were not selected for their success at using data; in fact, district leaders volunteered for this study to improve their districts’ data use. Boyer School District was a district of approximately 8,000 students that mostly served a non-Latino White population,2 less than five percent of who were economically disadvantaged. Gibson School District was a district of approximately 25,000 students of various ethnic backgrounds,3 half of whom were economically disadvantaged. Musial School District was a district of approximately 45,000 students of various ethnic backgrounds,4 a third of whom were economically disadvantaged. Districts varied in their student achievement: in a typical year, the percent of students meeting standards on the state exam was consistently greater than 95% in Boyer, approximately 75% in Gibson, and approximately 85% in Musial. The overall state rate was typically about 82%. Procedure We employed mixed-methods in conducting this study. Phone and in-person interviews were conducted with individuals, site visits were made to schools to conduct educator focus groups, and a confidential online survey was made available to all educators in each study district. In the following sections, we describe our procedures for collecting the qualitative and quantitative samples. Qualitative sample. Qualitative data were collected through individual interviews and focus groups. These were conducted using a semi-structured protocol that focused discussion on ways data were used and accessed, specific data systems employed, and wishes for future data use. All qualitative interviews were recorded and subsequently transcribed for analysis. 1 Pseudonyms are used for each district. 2 80% non-Latino White, 10% Latino. 3 40% Latino, 30% non-Latino White, 20% African American. 4 50% non-Latino White, 25% Latino, 10% African American. E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 6 At the central office level, individuals were identified through a review of central office positions. This list then was discussed with our primary district contacts to ensure proper coverage. Additionally, many interviewees were asked to suggest other individuals to interview. Central office educators were interviewed by telephone or in person. At the building level, teachers, principals, and other building staff participated through focus groups conducted during site visits to 19 campuses. These schools were chosen to be representative of each district in terms of size, location, and socioeconomic makeup (see Table 1). Table 1 Study Campuses, By District and Level Elementary School Middle School High School Total Boyer 2 1 1 4 Gibson 4 2 1 7 Musial 5 2 1 8 Study total 11 5 3 19 During school site visits, two focus groups were conducted in each school. One focus group consisted of the principal and individuals they chose who were familiar with data use in their school (e.g., assistant principals or instructional support staff). On the same day, a teacher focus group was conducted in that school. Teacher focus groups included three to five teachers who were selected by the principal from a randomly generated list of seven to nine teachers.5 High schools had many more teachers on staff than other schools; to ensure that we fully understood data use in the high schools, we conducted two teacher focus groups at those sites. The qualitative sample consisted of 197 total participants. Table 2 provides a description of this sample, disaggregated by educational role and district. Table 2 Study Participation, By Role and District Survey Data Interview Data N N Boyer Teachers 284 22 Campus Administrators 13 11 Central Office 3 6 Instructional Support 21 12 Boyer Total 321 51 5 This procedure created randomness in selecting teachers, but allowed principals latitude needed to collect a group of teachers at the same time (e.g., finding coverage). D istrict-W ide E ffects on D ata U se in the C lassroom 7 Table 2 (continued) Survey Data Interview Data N N Gibson Teachers 1117 30 Campus Administrators 62 18 Central Office 34 11 Instructional Support 82 6 Gibson Total 1295 65 Musial Teachers 1215 37 Campus Administrators 52 17 Central Office 72 16 Instructional Support 146 11 Musial Total 1485 81 Study total 3,101 197 Quantitative sample. Quantitative data were collected by administering the Survey of Educator Data Use (Wayman, Cho, & Shaw, 2009b), a 67-item instrument assessing a variety of factors, including attitudes toward data use, support for data use, instructional practices, technology, and specific ways in which data were used by the respondent. The survey was given online and made available to all educators throughout each district. Participants were not allowed to leave any items blank, so there were no missing survey data. Survey response rates were 50% in Boyer, 62% in Gibson, and 41% in Musial. The quantitative sample consisted of 3,101 individuals across the three districts. Table 2 provides a description of this sample, disaggregated by educational role and district. Measures Comparison categories. Educational role and district experience were each used to compare educators on aspects of data use. We defined four roles to be used for comparison: (1) campus administrators (principals and assistant principals), (2) central office staff, (3) instructional support staff (campus staff such as counselors, school psychologists, and instructional coaches), and (4) teachers. On the survey, district experience was collapsed into four categories: (1) 5 years or less, (2) 6 – 11 years, (3) 11 – 20 years, and (4) 20 years or more. Data use questions. Selected survey items were singled out to help describe how district educators used data. One block of 14 questions asked how often participants engaged in specific data uses, such as identifying individual students who need remedial assistance, setting school improvement goals, and evaluating district achievement trends and performance. These items were set on a four-point Likert scale with the following options: less than once a month, once or twice a month, weekly or almost weekly, and a few times a week. Each response option was numbered 1 – 4, with 1 corresponding to less than once a month. Another block of questions asked how often participants used specific computer systems in their district. These items were set on the same four-point Likert scale as the questions above. Four scales measuring attitudes and uses of data were formed from groups of survey items. The individual survey items within these groups asked how much the respondent agreed with a statement, offering the following options: strongly disagree, somewhat disagree, somewhat agree, and strongly E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 8 agree. Each response option was numbered 1 – 4, with 1 corresponding to strongly disagree. To create each scale, responses were averaged across the group of items in that scale. Scales thus ranged from one to four. These scales have been shown to be valid and reliable in other samples (Wayman et al., 2007; Wayman et al., 2009a). The Attitudes Toward Data scale was a four-item scale that asked participants whether they liked data, found it useful, and whether it helped them. The alpha reliability of this scale ranged from 0.838 to 0.8936. The Data's Effectiveness for Pedagogy scale consisted of five items that asked about the contributions that data can make for improving educational practice (e.g., helping to plan instruction, reveal new insights, or identify learning goals). The alpha reliability of this scale ranged from 0.930 to 0.949. The Principal Leadership scale consisted of five items that asked how much participants agreed that their principal or assistant principal(s) supported data use (e.g., encouraged it as a tool to support teaching, made training available, were examples of effective data users). The alpha reliability of this scale ranged from 0.909 to 0.910. The Computer Data Systems scale consisted of four items asking about computer systems, such as whether the participant’s systems were easy to use or provided ample data. The alpha reliability of this scale ranged from 0.943 to 0.951 Analyses Qualitative analyses followed methodology suggested by Miles and Huberman (1984). Drawing upon prior research on educational data use, an a priori list of potential analytic themes was generated. As qualitative data collection progressed, these themes were updated and refined during research team meetings. This collaborative and inductive process resulted in a conceptually coherent set of themes that was used for coding interviews and focus groups. The research team used this set of themes to code participant responses. Themes were examined by role and school level to identify emergent patterns and explanations regarding educator data use. Quantitative analyses were conducted as follows: for the block of 14 questions that asked about uses of data, mean responses were ranked for each role. This produced a ranked list of the frequency of data use for each of the 14 questions. For the four survey scales, ANOVAs were used to compare means responses by role. Significance was set at the 0.05 level and effect sizes (partial eta-squared) were computed. When role was statistically significant, Tukey post-hoc tests were performed to identify significant mean differences. Due to space restrictions, simultaneous confidence intervals for mean differences were computed but are not presented in this paper. For the questions that asked about frequency of computer use, we classified each system as an assessment system, student information system, data warehouse, or “other” system and dichotomized responses into weekly use or less. Percentages of weekly use were presented by role for each type of system. Results Besides differences in demographics, size, and economic makeup, our districts also presented diverse contexts in the ways data were approached and used. In the Boyer school districts, “data use” was viewed by most educators as the examination of state test data. Since almost every Boyer student passed the state test, many educators believed the state tests were irrelevant to their work – and thus, so was data use. Consequently, efforts to use data in Boyer were confined to small groups of interested educators. The district was attempting to implement some procedures around formative assessments, but was having trouble building momentum. 6 For each scale, alpha reliability was calculated specific to each district’s survey results. D istrict-W ide E ffects on D ata U se in the C lassroom 9 In Gibson, state test performance was strongly emphasized, but as part of a larger, curriculum-based initiative. Curriculum was divided into segments and a locally-built benchmark examination was given to students at the end of each segment. These examinations were tied to curriculum but were also intended to prepare students for performance on state tests. In Musial, there was a very strong focus on state test performance. District leadership used a variety of ways to communicate the importance of state test performance. Musial also implemented a set of locally-built assessments tied to state exams. Teachers were expected to examine these periodic assessments to improve instruction, with an eye toward performance on state tests. Musial had recently hired a central office administrator whose role was to support data use throughout the district with a strong focus on working with building-level educators on using data to improve instruction. In the following, we provide analysis of how these districts used data in their varying contexts. Sections are provided corresponding to our four research questions: (1) uses of data, (2) attitudes toward data, (3) principal leadership for data use, and (4) computer data systems. Uses of Data The ways data were used varied by role in all three districts. We use the following sections to provide descriptions of how data were used in four roles: central office educators, instructional support specialists, campus administrators, and teachers. Central office educators.7 Central office educators, in line with their responsibilities for large numbers of students and teachers, tended to use data for monitoring district and campus progress, providing feedback to campus personnel, and in support of broad-scale campus efforts. Many of these uses were centered on attempts to help building-level educators improve practice. For example, Gibson and Musial central office educators provided feedback to campuses about particular goals; campus personnel were expected to use this feedback to make adjustments in areas identified for improvement. As another example, central office educators in Boyer performed item analysis on behalf of teachers and helped inform departments and teachers about academic areas in which various grade levels needed attention. Instructional support specialists. Despite a variety of titles and intended responsibilities, the ways that instructional support specialists used data were consistent across the three districts. Persons in instructional support roles indicated that they used data in three main ways. First, they used data to identify and help teachers address the needs of individual students. Tables 3 through 5 show that instructional support personnel in both Musial and Boyer ranked the use of data to identify student learning first (M=2.91 and M=3.14, respectively); in Gibson, it was a close second (M=2.70). E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 10 Table 3 Respondent Rankings of Data Uses in Boyer, By Role Teachers Campus Administrators Instructional Support 1. Tailor instruction to individual student needs (2.46) 1. Develop recommendations for tutoring or other educational services for students (2.62) 1. Identify learning needs of students who are struggling (3.14) 2. Identify learning needs of students who are struggling (2.46) 2. Identify learning needs of students who are struggling (2.46) 2. Set learning goals for individual students (3.05) 3. Identify instructional content to use in class (2.33) 3. Form small groups of students for targeted instruction (2.38) 3. Discuss student progress or instructional strategies with other educators (2.95) 4. Set learning goals for individual students (2.31) 4. Meet with a specialist about data - e.g., instructional coach (2.31) 4. Develop recommendations for tutoring or other educational services for students (2.81) 5. Discuss student progress or instructional strategies with other educators (2.28) 5. Discuss student progress or instructional strategies with other educators (2.23) 5. Assign or reassign students to classes or groups (2.71) 6. Form small groups of students for targeted instruction (2.23) 6. Tailor instruction to individual student needs (2.23) 6. Tailor instruction to individual student needs (2.71) 7. Develop recommendations for tutoring or other educational services for students (2.12) 7. Set learning goals for individual students (2.23) 7. Identify learning needs of students who are not struggling (2.62) 8. Assign or reassign students to classes or groups (2.08) 8. Assign or reassign students to classes or groups (2.15) 8. Discuss data with a parent (2.52) 9. Identify learning needs of students who are not struggling (2.06) 9. Discuss data with a parent (2.15) 9. Identify instructional content to use in class (2.48) 10. Discuss data with a parent (1.81) 10. Discuss data with a student (1.92) 10. Form small groups of students for targeted instruction (2.48) 11. Discuss data with a student (1.79) 11. Choose which parents to contact (1.85) 11. Meet with a specialist about data - e.g., instructional coach (2.33) 12. Choose which parents to contact (1.76) 12. Identify learning needs of students who are not struggling (1.85) 12. Choose which parents to contact (2.24) 13. Meet with a specialist about data - e.g., instructional coach (1.75) 13. Interact with your principal about data use (2.10) 14. Interact with your principal about data use (1.45) 14. Discuss data with a student (2.10) Note. Mean response is shown in parentheses and only uses specific to the role are included. Note. Central office is not included because few uses applied to that specific role. Note. Teacher: n=284. Campus administrator: n=13. Instructional support: n=21. D istrict-W ide E ffects on D ata U se in the C lassroom 11 Table 4 Respondent Rankings of Data Uses in Gibson, By Role Teachers Campus Administrators Instructional Support 1. Identify learning needs of students who are struggling (2.72) 1. Identify learning needs of students who are struggling (3.16) 1. Discuss student progress or instructional strategies with other educators (2.74) 2. Tailor instruction to individual student needs (2.68) 2. Discuss student progress or instructional strategies with other educators (3.13) 2. Identify learning needs of students who are struggling (2.70) 3. Identify instructional content to use in class (2.60) 3. Develop recommendations for tutoring or other educational services for students (3.03) 3. Tailor instruction to individual student needs (2.65) 4. Form small groups of students for targeted instruction (2.51) 4. Tailor instruction to individual student needs (2.90) 4. Set learning goals for individual students (2.55) 5. Set learning goals for individual students (2.50) 5. Meet with a specialist about data - e.g., instructional coach (2.84) 5. Develop recommendations for tutoring or other educational services for students (2.55) 6. Develop recommendations for tutoring or other educational services for students (2.47) 6. Set learning goals for individual students (2.73) 6. Identify instructional content to use in class (2.50) 7. Discuss student progress or instructional strategies with other educators (2.45) 7. Form small groups of students for targeted instruction (2.69) 7. Assign or reassign students to classes or groups (2.39) 8. Assign or reassign students to classes or groups (2.35) 8. Discuss data with a parent (2.65) 8. Interact with your principal about data use (2.34) 9. Identify learning needs of students who are not struggling (2.28) 9. Assign or reassign students to classes or groups (2.61) 9. Form small groups of students for targeted instruction (2.34) 10. Discuss data with a student (2.12) 10. Discuss data with a student (2.58) 10. Meet with a specialist about data - e.g., instructional coach (2.22) 11. Choose which parents to contact (1.91) 11. Choose which parents to contact (2.50) 11. Identify learning needs of students who are not struggling (2.12) 12. Meet with a specialist about data - e.g., instructional coach (1.83) 12. Identify learning needs of students who are not struggling (2.29) 12. Discuss data with a student (2.12) 13. Discuss data with a parent (1.83) 13. Discuss data with a parent (2.04) 14. Interact with your principal about data use (1.66) 14. Choose which parents to contact (1.94) Note. Mean response is shown in parentheses and only uses specific to the role are included. Note. Central office is not included because few uses applied to that specific role. Note. Teacher: n=1117. Campus administrator: n=62. Instructional support: n=82. Note. Mean response is shown in parentheses and only uses specific to the role are included. Note. Central office is not included because few uses applied to that specific role. Note. Teacher: n=1117. Campus administrator: n=62. Instructional support: n=82. E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 12 Table 5 Respondent Rankings of Data Uses in Musial By Role Teachers Campus Administrators Instructional Support 1. Identify learning needs of students who are struggling (2.80) 1. Identify learning needs of students who are struggling (3.37) 1. Identify learning needs of students who are struggling (2.91) 2. Tailor instruction to individual student needs (2.76) 2. Develop recommendations for tutoring or other educational services for students (3.31) 2. Discuss student progress or instructional strategies with other educators (2.81) 3. Identify instructional content to use in class (2.62) 3. Discuss student progress or instructional strategies with other educators (3.29) 3. Develop recommendations for tutoring or other educational services for students (2.73) 4. Discuss student progress or instructional strategies with other educators (2.57) 4. Set learning goals for individual students (3.17) 4. Tailor instruction to individual student needs (2.66) 5. Set learning goals for individual students (2.54) 5. Discuss data with a parent (3.10) 5. Set learning goals for individual students (2.66) 6. Form small groups of students for targeted instruction (2.54) 6. Tailor instruction to individual student needs (3.02) 6. Interact with your principal about data use (2.32) 7. Develop recommendations for tutoring or other educational services for students (2.53) 7. Discuss data with a student (2.98) 7. Discuss data with a student (2.29) 8. Assign or reassign students to classes or groups (2.38) 8. Form small groups of students for targeted instruction (2.98) 8. Assign or reassign students to classes or groups (2.23) 9. Identify learning needs of students who are not struggling (2.38) 9. Choose which parents to contact (2.98) 9. Form small groups of students for targeted instruction (2.22) 10. Discuss data with a student (2.25) 10. Assign or reassign students to classes or groups (2.96) 10. Discuss data with a parent (2.21) 11. Choose which parents to contact (2.25) 11. Meet with a specialist about data - e.g., instructional coach (2.94) 11. Identify instructional content to use in class (2.10) 12. Discuss data with a parent (1.92) 12. Identify learning needs of students who are not struggling (2.81) 12. Meet with a specialist about data - e.g., instructional coach (2.10) 13. Interact with your principal about data use (1.78) 13. Identify learning needs of students who are not struggling (2.02) 14. Meet with a specialist about data - e.g., instructional coach (1.78) 14. Choose which parents to contact (1.92) Note. Mean response is shown in parentheses and only uses specific to the role are included. Note. Central office is not included because few uses applied to that specific role. Note. Teacher: n=1215. Campus administrator: n=52. Instructional support: n=146. D istrict-W ide E ffects on D ata U se in the C lassroom 13 Second, instructional support personnel used data to collaborate with teachers. Interview data revealed that instructional support personnel used a variety of data (e.g., primary reading data, math inventories) to aid teachers in forming small groups of students or to help teachers prepare lessons that target specific skills. Further, Tables 3 through 5 show that discussing student progress or instructional strategies ranked high in survey data: it was the highest-ranking survey variable in Gibson (M=2.74), second for Musial (M=2.81), and third for Boyer (M=2.95). Third, interview data suggested that instructional support personnel used data to help teachers reflect on practice. This included both monitoring and diagnosing of aggregated groups, as well as intervention and support with individual teachers and students. Campus administrators. In each district, our data showed that campus administrators often focused their data use on struggling students. This issue came up first and frequently in nearly every administrator focus group. On the survey, campus administrators ranked the use of data for identifying the needs of struggling students and for developing recommendations for intervention as the top two most frequent uses of data (see Tables 3 through 5). Administrators reported far less frequent use of data to identify the learning needs of students who were performing adequately or beyond, ranking this use last among all survey options in each district (see Tables 3 through 5). In addition, administrators also reported some frequent non-student based issues to which they applied data. One such use involved teacher feedback and evaluation efforts: administrators in Gibson and Musial described classroom “walkthroughs” and how they collected and reported data from this process. Administrators in each study district also described using various data to gauge the fidelity of curriculum implementation (i.e., whether teachers were on schedule and assessing the rigor of their teaching strategies). Teachers. Across the districts, teachers reported a variety of uses of data, including using data to help struggling students, group and regroup for instruction, reteach particular concepts and skills, and adjust instruction. Similar to administrators, teachers in interviews often focused on using data to support struggling students. This use also ranked first or second for surveyed teachers in each district (see Tables 3 through 5). Teacher comments less frequently focused on the needs of students who were performing adequately or who were excelling in the classroom. Such use of data ranked ninth among teachers in all three districts (see Tables 3 through 5). Teachers also discussed various ways that they used data to change how or what they taught; Tables 3 through 5 show these items rank highly. In interviews, this often related to grouping, such as forming instructional groups of students or deciding what to teach to the entire group. Teacher comments in each district indicated that their attention to groups occurred infrequently during the year, such as at the start of quarters. Although surveyed teachers reported frequently using data to adjust instruction for individual students (see Tables 3 through 5), we heard little mention of this in focus groups. We also found that teachers only rarely talked about using data as a centerpiece of meetings with instructional support personnel or with campus administrators. Such use of data ranked near the bottom for surveyed teachers in each district (see Tables 3 through 5). In focus groups, teachers either did not mention this type of data use or described it as happening infrequently. Attitudes Toward Data Survey and interview data revealed that participants were generally positive about data and its potential, even in the face of consistently present barriers. These barriers made many educators ambivalent about data – they saw value in data, but were hesitant because of hardships and other concerns. In the following two sections, we present results describing positive attitudes held by participants, followed by results that outline the barriers that temper these attitudes. E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 14 Positive attitudes. In each district, we found educators to be generally positive about data and what it could do for their practice. Participants in all roles averaged at least 3.00 on the Attitudes toward Data and Data’s Effectiveness for Pedagogy scales, with some roles approaching the maximum of 4.00. In interviews, participants often spoke positively of data and their potential. While positive, teachers often displayed more skepticism about data than did those in other roles. Table 6 shows that teachers’ views were significantly different from those in other roles on the two scales. Table 7 shows that in each district, teachers ranked significantly lower on the two scales (excepting Musial central office administrators, who were similar to teachers on both scales). Campus administrators and instructional support personnel consistently ranked high, if not significantly so. In fact, campus administrators in Gibson and Musial averaged near the maximum on the Effectiveness scale. Educators in all roles showed slightly more optimism about the effectiveness of data than in their personal attitudes toward using it. Table 6 One-way ANOVAs for Survey Scales by Role, for Each District df F p Eta-squared Boyer Attitudes Toward Data 2, 315 9.91 0.00 0.06 Data’s Effectiveness for Pedagogy 2, 315 8.48 0.00 0.05 Principal Leadership 2, 315 2.59 0.08 0.02 Computer Data Systems 2, 315 0.14 0.88 0.00 Gibson Attitudes Toward Data 3, 1291 35.74 0.00 0.08 Data’s Effectiveness for Pedagogy 3, 1291 25.90 0.00 0.06 Principal Leadership 3, 1291 8.13 0.00 0.02 Computer Data Systems 3, 1291 8.80 0.00 0.02 Musial Attitudes Toward Data 3, 1484 36.17 0.00 0.07 Data’s Effectiveness for Pedagogy 3, 1484 20.54 0.00 0.04 Principal Leadership 3, 1484 31.70 0.00 0.06 Computer Data Systems 3, 1484 0.69 0.56 0.00 Note. Independent variable is role. Note. Central Office administrators are not included for Boyer because of insufficient response. D istrict-W ide E ffects on D ata U se in the C lassroom 15 Table 7 Means of Survey Scales, Disaggregated by District and Role Attitudes Toward Data Data’s Effectiveness for Pedagogy Principal Leadership Computer Data Systems Boyer Teachers 3.05ca,is 3.34is 2.97 3.11 Campus Administrators 3.44t 3.68 2.89 3.02 Central Office N/A N/A N/A N/A Instructional Support 3.56t 3.82t 3.30 3.13 Gibson Teachers 3.13co,ca,is 3.42co,ca,is 3.21co,is 3.07ca,co Campus Administrators 3.71t 3.92t 3.35co 2.83is,t Central Office 3.71t 3.84t 2.86t,ca,is 2.57is,t Instructional Support 3.53t 3.69t 3.41co,t 3.12ca,co Musial Teachers 3.12ca,is 3.45ca,is 3.26ca,co,is 2.94 Campus Administrators 3.84t,is 3.90t,co 3.53t,co 3.02 Central Office 3.03ca,is 3.47ca,is 2.62t,ca,is 3.00 Instructional Support 3.50t,ca,is 3.73t,co 3.45t,co 3.02 Note. Means are not presented for Boyer Central Office because of insufficient response. Note. Significant pairwise role differences are noted by superscripts within the table. t = teachers, ca = campus administrators, co = central office, and is = instructional support Interview data captures the details of these positive attitudes, which differed by district. Educators in Gibson and Musial had greater exposure to data and showed attitudes more similar to each other than to Boyer educators. Throughout Gibson and Musial, participants who were positive about data saw it as a way to support professional judgment. For them, data contributed to instruction (e.g., providing feedback about individual students, lessons, programs, or learning issues) and was an important part of reflecting collaboratively about issues of mutual concern. A Musial teacher described this attitude thusly: “Say you administer a common assessment, you go to team meeting and talk about it: the strengths you are seeing, the weaknesses. ‘[We discuss] How can we change our instruction to make this concept more clear?’” In comparison, Boyer educators described their positive attitudes more vaguely than educators in Gibson and Musial. Boyer educators spoke generally, as evidenced by this teacher: “Working with another grade level teacher [on student data] before the school year is valuable. I get to learn what helps certain students.” Further, Boyer administrators typically focused their positive attitudes not on themselves, but on benefits for other educators. For example, central office administrators valued data as a tool for teachers and campus administrators, but they rarely reported using data in their own work. Similarly, campus administrators were positive about data but focused on the work of teachers, not themselves. Barriers tempering positive attitudes. The positive attitudes above were often tempered by day-to- day difficulties in using data, such as problems with computer systems, lack of time to reflect on data, and the labor-intensiveness of using data. In line with survey results, teachers in all three E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 16 districts described these barriers with more negativity than did educators in other roles. In Gibson and Musial, where data were in more frequent use, educators described barriers in greater detail and breadth than did Boyer educators. Teachers in Gibson and Musial expressed concerns about the role that data was playing in their district; these concerns were not seen in Boyer. For example, some Musial teachers felt that data were being used to inappropriately compare or encourage unnecessary competition among campuses. One said, “We are pressured to meet standards, pressured by the data to meet standards, absolutely. They make it clear that that’s very published and very public.” While teachers were most vocal about data barriers, non-teachers (campus administrators, central office educators, and instructional support educators) also shared these concerns. For example, non-teachers in Gibson were especially concerned about their lack of integrated computer data systems. They were concerned about challenges in accessing the right data and in sometimes having to rely on others to get data for them. As another example, non-teachers in Musial also expressed concerns about the difficulties of data access. Additionally, they worried about the kinds of conclusions that might be drawn from data, such as data only serving to confirm expectations, rather than expanding knowledge. A few also felt that data they had personally collected were more informative to decision making. Non-teachers in Boyer were concerned about two issues. One was lack of time – particularly, how labor intensive data use could be due to the lack of integration in their data systems. The other was a general concern that some educators might undervalue data’s role in improving practice since the district already had high levels of student achievement. Principal Leadership for Data Use Principals across the districts seemed to hold the benefits of data use in high regard (see Attitudes Toward Data above). Further, Table 7 suggests that participants across all three districts were generally positive about principal leadership for data use, as indicated by averages on the Principal Leadership scale. These views varied by role in Gibson and Musial (see Table 6), where more expectations were placed on principals to use data. In both districts, principals and their instructional support staff scored significantly higher on the Principal Leadership scale than did teachers and central office administrators. Central office administrators in both districts scored significantly lower on this scale than all other roles. Nevertheless, qualitative data show that faculty struggles with data use were often connected to the leadership of their principals. These data show considerable variation in principals’ leadership behaviors for data use. Further, we observed principals in Gibson and Musial more involved in leading for data use than were principals in Boyer. In the following two sections, we present detail on the positive and negative leadership approaches observed. Positive leadership strategies. A few principals in our study had established structures that promoted regular, consistent data use in their schools. With the exception of one principal in Boyer, these principals worked in Gibson or Musial, and were more common at the elementary than secondary level. A few principals were particularly active in developing robust collaborative routines. Not only did they support teacher-to-teacher collaboration, they also worked directly with teachers on data-related activities and used a collaborative, collegial style in setting expectations and plans for using data. Their teachers reported planning with their administrators at team meetings and described their administrators as committed to communicating with teachers about data. An exchange between Musial focus group teachers described this perspective: Teacher 1: He’ll come and sit down with a team. Teacher 2: He’ll pare it back for you. D istrict-W ide E ffects on D ata U se in the C lassroom 17 Teacher 1: Yeah, he’ll sit down and say, “Well, let’s talk about this and how it works.” He wants to know how we’re doing things, what’s the growth, he’s very serious about that. Similarly, a handful of campuses stood as examples for how administrators and teachers could be in strong agreement about how and why data can contribute to improving schooling. Notably, we heard talk at these campuses that suggested data use was a substantive part of everyday work, fostered by leaders and sustained by the interactions and activities of faculty. Examples of other structures included explicit time set aside for faculty to examine data use, varied forms of collaboration (e.g., grade-level teams, vertical teams, interdisciplinary teams), and instructional coaches trained to support teachers in using data. Further, principals that appeared to be more skilled at using data were better able to help faculty learn to use data systems, lead discussions on the meaning of data, and promote effective questioning. Negative leadership strategies. Unfortunately, many more principals were employing few of these strategies, or were using strategies that had negative effects. In these schools, teachers directly expressed frustration with their administrators’ attempts to lead for data use – their negativity often was expressed first toward data explicitly, but frequently transitioned toward comments critical of their principals’ leadership behaviors. Many teachers were critical of the data skills of their principals and some said their principals did not engage in any significant data analysis. One Gibson teacher described how this affected faculty data use: There are leaders on this campus who don’t know how to analyze the data. And so you have pockets of teachers who really good at using and analyzing data, and then you have pockets of teachers who just follow along with what those folks are doing, but they have no idea how to access any of their data. Some principals and faculty described a directive, “top down” approach to leading the use of data. Many of these campuses were quite involved in using data, but there was little evidence of collegiality. In fact, teachers at these campuses more frequently spoke of data use as a monitoring exercise, as this Gibson teacher did: “They want ‘proof’ that we’re analyzing data – they don’t just let me go analyze my data. I don’t need to write anything down, I need to absorb it. In my opinion, they want stuff written down so it’s proof that I’ve looked at the data.” We also heard of leadership behaviors that seemed punitive in nature. For instance, teachers at one Musial campus said that their principal posted lists of teachers whose failure rates were greater than ten percent. Another example came from a Gibson campus, where teachers reported being called in to discuss data with their administrators when assessment scores were not up to par; these teachers reported feeling that the tone of these discussions was punitive. Our data revealed few instances in any district of principals structuring time for faculty to examine and reflect upon data. There was no evidence that principals were limited by district policies in their ability to provide time for data. In fact, there were instances where district policy enabled faculty collaborative time (e.g., PLCs, common planning time), but principals rarely specified that their faculties spend that time using data. As a consequence, it was common for teachers at all three levels to describe their use of data occurring before school, after school, at lunch, or on their conference period. Further, there were few examples where principals provided structure for how the time set for using data could be spent (e.g., guiding questions, analysis protocols). Teachers often reported mixed feelings about their interactions with administrators. In Gibson and Boyer, some teachers worried that administrators did not respect other demands on teachers’ time. This was especially so for teachers who were asked to perform burdensome and E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 18 redundant activities, such as hand copying data in preparation for meetings or creating paper records for administrators as “evidence” of collaboration. In Musial some teachers were concerned that collaborative data events were more about creating “lockstep” and predetermined practices, rather than about innovating solutions for students. Finally, we were struck by the disconnect we observed between teachers and principals at many schools in each district. At some schools, it felt as if administrators and teachers were talking about two completely different schools. For example, we frequently encountered schools where administrators cited clearly articulated expectations for using data, but teachers were unable to articulate these same expectations. As another example, administrators in one Gibson school described themselves and their faculty as avid users of data and provided many examples and artifacts showing how their school used data. However, teachers at this school were negative toward data, reported using it infrequently, and were suspicious of their principal’s motives around using data. Computer Data Systems We defined a “data system” as any computer-based tool that helped educators examine student data. Under this definition, the data revealed a diversity of systems employed in each district: 39 distinct systems were found in Boyer, 58 were found in Gibson, and 68 were found in Musial. While the variety of systems was large, many provided data that was fed into to three general types of systems: (1) student information systems (SISs), which typically handle day-to-day student information, such as scheduling, course grades, and demographic information; (2) assessment systems, which rapidly organize test results; and (3) data warehouses, which integrate data from a variety of systems and provide a comprehensive, longitudinal view of student performance. In the following, we first provide three sections presenting an overview of data systems in each district. Next, we offer a section that describes the uses of these systems, followed by a section outlining issues in integrating these systems. Finally, we provide a section that outlines educator attitudes toward these systems. Systems in Boyer. Of the study districts, Boyer was the least advanced in its data technologies. Their SIS was not technically an SIS, but an electronic gradebook built to handle grades and other student data. Boyer employed two separate assessment systems: one geared toward collecting, analyzing, and reporting district- or state-level test results (e.g., district benchmarks and state achievement tests), and the other geared toward providing regular, formative assessment data to teachers. Boyer had no data warehouse to integrate various systems. Systems in Gibson. Gibson employed an SIS for day-to-day information, as well as an assessment system for providing assessment data. There existed a data warehouse in Gibson, but it was not user-friendly in the ways that it integrated or presented data; consequently, the data warehouse typically was used only to organize and access district- and state-level tests. Systems in Musial. Musial also employed an SIS for day-to-day information and an assessment system for providing assessment data. Although Musial technically had a data warehouse, it did not exist for many educators in a practical sense: due to site license issues, most educators (and all teachers) did not have direct access to the data warehouse. The data warehouse provided reports that pulled data from many disparate systems, but access problems had created bottlenecks in accessing these reports. Further, when reports arrived, they typically were in the form of a large Access or Excel file. System uses. Table 8 shows the number and percent of educators that reported weekly use of their district’s various systems. In each district, the SIS was the most prevalently used system. D istrict-W ide E ffects on D ata U se in the C lassroom 19 Otherwise, these numbers reflect the diversity of technological contexts in the three districts – no clear patterns were apparent across district or role. Table 8 Number and Percent of Participants Using Data Systems Weekly or More, Disaggregated by District and Role SIS Assessment System Data Warehouse Other System Boyer Teachers 258 (91%) 64 (23%) N/A 49 (17%) Campus Administrators 11 (85%) 3 (23%) N/A 2 (15%) Central Office N/A N/A N/A N/A Instructional Support 14 (67%) 3 (14%) N/A 5 (24%) Gibson Teachers 479 (43%) 161 (14%) 138 (12%) 177 (16%) Campus Administrators 46 (74%) 10 (16%) 30 (48%) 33 (53%) Central Office 13 (38%) 4 (12%) 11 (32%) 14 (41%) Instructional Support 27 (33%) 17 (21%) 30 (37%) 18 (22%) Musial Teachers 1097 (90%) 120 (10%) 54 (4%) 211 (17%) Campus Administrators 34 (65%) 25 (48%) 20 (39%) 25 (48%) Central Office 22 (31%) 7 (10%) 13 (18%) 31 (43%) Instructional Support 47 (32%) 35 (24%) 13 (9%) 57 (39%) Note: Percent of participants reporting weekly or more use given in parentheses. Note. Boyer does not have a data warehouse Note. Figures are not presented for Boyer Central Office because of insufficient response. Although the distinctions between SISs, assessment systems, and data warehouses may seem clear, we found that districts sometimes attempted to “stretch” the intended use of their systems to meet locally desired ends. Thus, systems might be used in ways for which they were not necessarily designed. For example, the Boyer electronic gradebook was being used by central office members as an SIS and doubled for some teachers as both an SIS and gradebook. The Gibson data warehouse provided another example: although it could provide data integration, it typically was applied as an assessment system. We heard some instances of data systems facilitating work, but much more commonly heard how technical limitations suppressed data use. For example, many Musial educators described the “information overload” they felt with the large Excel files or stacks of printouts they received. Gibson educators often experienced a slow turnaround time for feedback regarding district-based assessments. Boyer educators rarely described using their systems in ways that were not compliance oriented (e.g., state achievement reports or student documentation). System integration. In every district, many educators described the desire for easy to access, integrated information about students. They wanted the various systems in their district to connect in a way that offered one-stop access to all data, as described by a Musial principal: “This district wants to be data driven. They value it. They have tons of data, but not one good solid system.” E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 20 District administrators in each district recognized this problem and each district was addressing it in a different way. Boyer administrators were considering a data warehouse and saw participating in our study as one way in which they might prepare themselves for the “next level” in data system implementation. Gibson was developing plans to obtain a more dynamic data warehouse that could provide more refined, comprehensive, and timely views on students. Musial was working to expand licenses to allow teachers access to the data warehouse. This work included developing a user-friendly system interface, as well as tasking a central office administrator to help ensure that educators’ interests were represented in the new interface. Educators in each district were going to great lengths to compensate for lack of integration. Several educators in each district described using self-built and maintained systems (e.g., Excel, Access, or paper and pencil) in order to aggregate and analyze their students’ data. Although more work-intensive and less powerful than the output from an integrated system, the resultant analyses were often described as useful. A Gibson assistant principal described one such self-built system used in his school: Every teacher has a folder and a corresponding spreadsheet that has every single one of their students on it. Each student has their ethnicity next to their name, along with the class period that they’re in. And what this allows the teacher to do is enter the data after an assessment is taken, and the teacher will then be able to sort through this to see how students in different classes did by ethnicity and (standard). And then they’ll be able to target their intervention by specific (standard) in a special intervention period, where the students go to a particular intervention based upon need. So it’s a quick way for teachers to have an at-a-glance look at how the kids are doing on what they’re teaching in the classroom. We uncovered these self-designed systems at every level of every district, sometimes shared across teams or buildings via email or a school common drive. In some cases, “power users” were assigned the task of querying, assembling, and distributing these makeshift approaches to integrating data. Attitudes toward data systems. Table 6 shows that scores on the Computer Data Systems scale only differed by role in Gibson. Table 7 shows that teachers and instructional support staff in Gibson perceived their data systems more positively than did campus administrators and central office personnel. Interview data in all three districts revealed stronger negative attitudes than did survey data. Typically, educators in all three districts were highly frustrated with the amount of time it took to access and analyze data, and most of their frustration resulted from a lack of system integration. Lacking an integrated data system, educators in each district reported that the time burdens of generating and compiling data left little time to actually use the data. For example, educators in each district reported spending too much time pulling together data from disparate systems and some reported hand-copying data to ready it for use. This was especially so in Gibson and Musial, where some described using their time to fill out forms that could otherwise have been pre-populated by computer data systems. One Gibson teacher described one such example, saying, “We took the [data system] printout and copied the same information in a watered-down format onto another sheet of paper … yes, it divided things I was strong in and weak in, but I could have done that on the printout.” Educators in every district saw such labor-intensive routines as untenable in light of today’s advancements in technology. D istrict-W ide E ffects on D ata U se in the C lassroom 21 Discussion The goal of the present study was to examine how practices at every level of the district affect the use of data in the classroom. We addressed this goal by exploring uses of data, attitudes toward data, principal leadership, and computer data systems. Educators were ambivalent in their attitudes toward data – they often could see how using data could improve practice, but were negative about the barriers that made data use hard for them. Many of these barriers related to principal leadership and data systems. Some principals in this study were shown to be employing multiple leadership strategies to facilitate faculty data use; in those schools, attitudes were good and data supported educational practice. Unfortunately, more principals were employing few such strategies; in those schools, educators reported negative attitudes, and had difficulty using data to improve practice. Results also showed a range of hardships and wasted time caused by inefficient computer data systems. Most importantly, these factors together affected how data use translated into knowledge and practice. Fortunately, many of these barriers are ones that can be handled structurally. Formal district policies for using data were largely absent from our results, so we suggest that districts can support and facilitate classroom practice by attending to data-related district policy. That is, districts can improve how data inform practice if leaders are intentional about writing and codifying district policy to alleviate the barriers found in the present study. In the following sections, we provide discussion of five areas that district leaders might consider when creating policy: context, attitudes, principal leadership, computer data systems, and focus on all learners. Context Context can be important in considering district data use policies. However, our results suggest that, regardless of context, districts may be dealing often with similar policy issues. We identified a few findings that were considerably different from district to district; other differences existed only in details specific to each district. One finding from this study depended strongly on context: the interaction of context and policy environment seemed to affect how data were used. Gibson and Musial were average- achieving districts looking to improve, so there was a stronger response to accountability ratings in these districts than in Boyer. The attention given to accountability ratings in Gibson and Musial sometimes seemed to distract principals and other leaders from the bigger picture of teaching and learning. Educators in the high-achieving Boyer district did not view their students’ achievement in terms of accountability. Still, accountability had an effect on data use in Boyer, because many educators saw data use as accountability-related and thus believed that data use did not apply to them. Thus, the accountability system created little motivation in Boyer to engage in data use. For most other findings, context mattered only in detail. For instance, regardless of district, we found that educator attitudes toward data were often ambivalent because of barriers, but the barriers themselves often depended on district. As another example, all districts had integration problems with their data systems, but the responses to this problem varied by district. Attitudes Attitudes toward data have been shown by prior research to be important: in exemplary contexts, studies have shown attitudes toward data to be positive and educators believed that data use positively supported their practice (Wayman & Stringfield, 2006; Datnow et al., 2007; Lachat & Smith, 2005). However, studies in these contexts rarely present the daily barriers shown in our data. Our data showed attitudes toward data were remarkably positive, even in the face of these barriers. Positive perceptions of data’s potential seemed to be what kept educators using data, because many E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 22 thought data could help them in their practice. Consequently, educators sometimes even constructed time-consuming workarounds in order to use data. This notwithstanding, it must not be ignored that educator attitudes in our study often were negative toward what data use did to them. At first glance, this seemed related to role: survey data indicated that teachers were consistently more negative than administrators or support staff about using data. However, in triangulating survey data with interview data, we came to interpret these attitudes as related to everyday practice, not role: attitudes toward data were mostly related to the barriers and problems educators faced. We surmise that teachers were typically more negative than other roles because the nature of how they worked with data made them subject to more barriers. To contrast, instructional support staff used data frequently, but were often more positive. Instructional support jobs often carried natural structures that enabled them time and other supports, so it is possible instructional support staff did not face the day-to-day barriers that teachers did. It is reasonable to conclude that if the activities associated with data make it harder for one to do one’s job, then one would be understandably negative about data – regardless of role. In considering how attitudes might shape district policy, there is a bright spot in these results: attitudes seem to be influenced by structural barriers and these structures can be affected by district policy. District policies can be written to improve a number of complaints found in our study, such as time to use data, learning opportunities, expectations for data use that align with improved practice rather than accountability compliance, principal leadership, and data systems. Principals Numerous studies have suggested that the principal is important to the success of a data initiative (e.g., Wayman & Stringfield, 2006; Copland, 2003; Datnow et al., 2007). In the present study, the connection between a principal and faculty appeared to greatly affect how data were used; this connection seemed to depend on the strategies employed by the principal. In schools where principals intentionally employed strategies to support data use to improve student learning, teachers typically reported better attitudes and more effective data use. In schools where principals employed fewer or negative strategies, attitudes were worse and data use was reported to be less effective. In these schools, teachers were often left to deal with data on their own – not just in terms of the meaning they drew from it and subsequent change in practice, but also in structures, such as time and data access. District policies can address principal leadership strategies in a number of ways. Some are relatively simple. For instance, policies could provide for professional development that helps principals learn how to use data-related leadership strategies. Or, policies could contain a list of the types of strategies that district leaders believe may be most effective in the district context (see Wayman, Spring, Lemke, & Lehr (2012) for a research-based inventory of such strategies). Policies such as these may seem obvious, but we saw no evidence that any of our districts had addressed principal leadership in any structured way. While these policies are an important first step, the current study provides evidence that simple policies such as these – those that only mandate action – are not sufficient. Policies must additionally provide elaboration and support for how these actions may be implemented. For example, our findings highlighted district policies that provided building-level educators time to collaborate (e.g., PLCs, common planning time). Despite these structures, principals rarely encouraged or required this time to be allocated to analyzing data. In this case, principals were not limited by district policy in providing faculty time to use data, but were not fully supported by this policy because it only created collaborative time. Thus, even in the presence of these policies, many D istrict-W ide E ffects on D ata U se in the C lassroom 23 teachers still reported insufficient time to use data. These policies would have been more effective with regard to data use if they had been written to also provide support for what went on in that time. For instance, a policy could require that this time be dedicated to data use a certain number of days per week and could provide guidance regarding key problems to work on. Computer Data Systems Results of this study, like others (e.g., Wayman & Stringfield, 2006; Lachat & Smith, 2005; Means et al., 2010), make it clear that educators need integrated, one-stop access to the data they want to use. Integration was so important to the educators in the present study that many improvised time-consuming methods to connect various pieces of data. The results of this study additionally showed how ineffective data systems affected attitudes toward data and the progression from data to information to practice. Clearly, any district that chooses to make data an important part of their educational experience should enable some way to integrate data so that educators have one-stop access to all the data that are available. But our findings show that the use of these systems is often more complex than merely enabling the hardware. For instance, users and districts were sometimes applying these systems in ways different than envisioned by system designers. Cho and Wayman (2012) suggested a reason for this: system use is dependent on the sense that the user makes of it. They further suggest that preconceptions about data systems, data, and their utility for practice serve as a lens for determining what is meaningful about a system. . Thus, educators may have certain expectations, potential uses, or kinds of practices in mind before accessing data. This may promote some system features as worthy of use or adaptation, while obscuring the significance of others. Consequently, district policies for data systems should be twofold: Besides providing the means for providing one-stop, integrated access, policies must also attend to how the system will fit into the everyday work of the educator. We posit that data systems should be at the center of every district policy about data use. For instance, a policy dealing with professional learning should outline how the data system will be used to support skills educators gain during professional learning sessions. Or, a policy supporting principal time structures should also describe how the data system may be used to make use of this time more productive. Focus on All Learners Our results showed data use being focused primarily on struggling learners. We believe this is probably the case in most districts – while prior research has not explicitly identified or quantified this result as in the current study, almost all of the uses of data described in prior research involve struggling learners (e.g., Anderson et al., 2010; Wayman & Stringfield, 2006; Datnow et al., 2007; Lachat & Smith, 2005; Marsh et al., 2010; Young, 2006). Focusing on struggling learners is an absolute necessity; this fact has been underscored in the current accountability climate. However, we also caution against narrowing data use to focus on one group of students, because narrowing toward one group does not fit how schools work. Instead, we suggest that a data initiative will be more successful for struggling and non-struggling students alike if it is focused on formative opportunities for every student throughout the district. District data policies should be written to describe how this focus on all students will raise the achievement of struggling learners through increased differentiation or other sound practices. Conclusion Most of the educators in our study wanted data to be used to support classroom practice. Unfortunately, they faced barriers and problems that sometimes made this hard to do. Our personal E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 24 experience and our reading of the literature indicate that many districts across the United States are in the same situation as our study districts. While we found that the influences on data use were numerous and complex, we also believe that many influences can be made positive by effective district policy. Unfortunately, our districts had very few – if any – policies related to data use. On the bright side, some of these policies can be easily written. Further, a little bit of policy may go a long way toward improving conditions around data use. If districts are to write effective policy, they must keep in mind that, at the heart, the findings of this study speak to how data use fits (or does not fit) into the everyday work of education. Thus, it is likely that district policies grounded in compliance will not be effective, but policies that react to and support the ways that educators work will be effective. We believe that the effective use of data can be a strong contributor to educational practice. However, we believe this hypothesis cannot be confirmed until contexts are created where a variety of factors support data use district-wide. This study has presented new information toward making these supports a reality and we hope to see such supports become commonplace in the coming years. References Alavi, M., & Leidner, D. E. (2001). Review: Knowledge management and knowledge management systems: Conceptual foundations and research issues. MIS Quarterly, 25(1), 107-136. Anderson, S., Leithwood, K., & Strauss, T. (2010). Leading data use in schools: Organizational conditions and practices at the school and district levels. Leadership and Policy in Schools, 9(3), 292 – 327, DOI: 10.1080/15700761003731492. Carlson, D., Borman, G. D., & Robinson, M. (2011). A multi-state district-level cluster randomized trial of the impact of data-driven reform on reading and mathematics achievement. Educational Evaluation and Policy Analysis 33(3), 378398. Cho, V. & Wayman, J. C. (2012, April). Districts’ efforts for data use and computer systems. Paper presented at the 2012 Annual Meeting of the American Educational Research Association, Vancouver, Canada. Coburn, C.E., Honig, M.I., and Stein, M.K. (2009). What’s the evidence on districts’ use of evidence? In J. D. Bransford, D.J. Stipek, N.J. Vye, L.M. Gomez, and D. Lam (Eds.), The role of research in educational improvement (67-87). Cambridge: Harvard Education Press. Copland, M. A. (2003). Leadership of inquiry: Building and sustaining capacity for school improvement. Educational Evaluation and Policy Analysis, 25, 375–395. Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with data: How high-performing school systems use data to improve instruction for elementary students. Los Angeles: University of Southern California, Rossier School of Education, Center on Educational Governance. Earl, L. & Fullan, M. (2003). Using data in leadership for learning. Cambridge Journal of Education, 33(3), 383-394. Halverson, R., Prichett, R. B., & Watson, J. G. (2007, May). Formative feedback systems and The New Instructional Leadership. Retrieved June 30, 2011 from http://www.wcer.wisc.edu/publications/workingPapers/Working_Paper_No_2007_03.pdf Hamilton, L., Halverson, R., Jackson, S. S., Mandinach, E., Supovitz, J. A., Wayman, J. C. (2009). Using student achievement data to support instructional decision making. Washington D.C.: Institute of Education Sciences and the National Center for Education Evaluation. D istrict-W ide E ffects on D ata U se in the C lassroom 25 Honig, M. I. & Coburn, C. (2008). Evidence-Based Decision Making in School District Central Offices. Educational Policy, 22(4), 578-608. Ingram, D., Louis, K. S., & Schroeder, R. G. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers College Record, 106, 1258– 1287. Kerr, K. A., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education, 112(4), 496–520. Knapp, M. S., Swinnerton, J. A., Copland, M. A., & Monpas-Huber, J. (2006). Data-informed leadership in education. Seattle, WA: Center for the Study of Teaching and Policy, University of Washington. Lachat, M. A., & Smith, S. (2005). Practices that support data use in urban high schools. Journal of Education for Students Placed At Risk, 10(3), 333–349. Long, L., Rivas, L. M., Light, D., & Mandinach, E. B. (2008). The evolution of a homegrown data warehouse: TUSDstats. In Mandinach, E. B., & Honey, M. (Eds.), Data-driven school improvement (pp. 209-232). New York, Teachers College Press. Mandinach, E. B., Honey, M., Light, D., Brunner, C. (2008). A conceptual framework for data- driven decision making. In Mandinach, E. B., and Honey, M. (Eds.), Data-driven school improvement. New York: Teachers College Press. Marsh, J. A., McCombs, J. S., Martorell, F. (2010). How instructional coaches support data-driven decision making: Policy implementation and effects in Florida middle schools. Educational Policy, 24(872). May, H. & Robinson, M. A. (2007). A randomized evaluation of Ohio’s Personalized Assessment Reporting System (PARS). Madison, WI: Consortium for Policy Research in Education. Means, B., Padilla, C., DeBarger, A., and Bakia, M. (2010). Implementing data-informed decision-making in schools—Teacher access, supports and use. Washington, D.C.: U.S. Department of Education Office of Planning, Evaluation and Policy Development. Retrieved February 27, 2012 from www.ed.gov/about/offices/list/opepd/ppss/reports.html. Miles, M., & Huberman, A. M. (1984). Qualitative data analysis: A sourcebook of new methods. Beverly Hills, CA: Sage. Stringfield, S., & Datnow, A. (2002). Systemic supports for schools serving students placed at risk. In Stringfield, S., & Land, D. (Eds.) (2002). Educating at-risk students. Chicago: National Society for the Study of Education. Supovitz, J. A. (2010). Knowledge-based organizational learning for instructional improvement. In Hargreaves, A., Lieberman, A., Fullan, M., & Hopkins, D. (Eds.). Second international handbook of educational change (pp. 707-723). New York: Springer. Supovitz, J. A., & Klein, V. (2003). Mapping a course for improved student learning: How innovative schools systematically use student performance data to guide improvement. Philadelphia: Consortium for Policy Research in Education. Tuomi, I. (1999). Data is more than knowledge: Implications of the reversed hierarchy for knowledge management and organizational memory. Journal of Management Information Systems, 16(3), 103. Valli, L., & Buese, D. (2007). The changing roles of teachers in an era of high-stakes accountability. American Educational Research Journal, 44(3), 519–558. Vasquez Heilig, J. & Darling-Hammond, L. (2008). Accountability Texas-style: The progress and learning of urban minority students in a high-stakes testing context. Educational Evaluation and Policy Analysis. 30(2), 75-110 E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 26 Wayman, J. C. (2010, May). The Data-Informed District: A preliminary framework. Paper presented at the 2010 Annual Meeting of the American Educational Research Association, Denver CO. Wayman, J. C., Spring, S. D., & Lemke, M. A., Lehr, M. D. (2012, April). Using data to inform practice: effective principal leadership strategies. Paper presented at the 2012 Annual Meeting of the American Educational Research Association, Vancouver, Canada. Wayman, J. C., Cho, V., & Johnston, M. T. (2007). The Data-Informed District: A district-wide evaluation of data use in the Natrona County School District. Retrieved February 1, 2010 from http://edadmin.edb.utexas.edu/datause Wayman, J. C., Cho, V., & Shaw, S. M. (2009a). First-year results from an efficacy study of the Acuity data system. Austin: Authors. Wayman, J. C., Cho, V., & Shaw, S. (2009b). Survey of Educator Data Use. Unpublished document. Wayman, J. C., Shaw, S. M., & Cho, V (2011). Second-year results from an efficacy study of the Acuity data system. Austin: Authors. Wayman, J. C., & Stringfield, S. (2006). Technology-supported involvement of entire faculties in examination of student data for instructional improvement. American Journal of Education, 112(4), 549–571. Young, V. M. (2006). Teachers’ use of data: Loose coupling, agenda setting, and team norms. American Journal of Education, 112(4), 521–548. D istrict-W ide E ffects on D ata U se in the C lassroom 27 About the Authors Jeffrey C. Wayman University of Texas at Austin Email: jwayman@austin.utexas.edu Jeff Wayman is a faculty member in the Department of Educational Administration at the University of Texas at Austin. His teaching and research interests focus on the effective use of data for school improvement. Dr. Wayman's research on data-based decision-making includes efficient structures for creating data-informed school districts, software that delivers student data to educators, effective leadership for data use, and systemic supports that enable widespread teacher use of student data. Vincent Cho Boston College Email: vincent.cho@bc.edu Vincent Cho is an assistant professor at Boston College in the Dept. of Educational Leadership and Higher Education. Cho studies the technologies, leadership practices, and district policies that support data use. His aim is to help schools and educators make the most out of their knowledge about students. Jo Beth Jimerson Texas Christian University Email: j.jimerson@tcu.edu Jo Beth Jimerson is an Assistant Professor of Educational Leadership at Texas Christian University in Fort Worth, Texas, where she works with aspiring educational leaders through principal and superintendent preparation programs. Her research focuses on professional learning, data use, and the development of effective campus and district leadership practices. Daniel D. Spikes University of Texas at Austin Email: dspikes@austin.utexas.edu Daniel Spikes is a doctoral student in the Educational Policy & Planning program at The University of Texas at Austin and is currently employed as the Assistant Director for Pre- College Academic Readiness Programs with the Division of Diversity and Community Engagement. His research interests include the achievement gap, data-based decision making, school-tracking policies, social justice, and anti-racist leadership. E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 28 Acknowledgement The authors would like to thank The Spencer Foundation for funding the project from which this article comes. The authors would also like to express our thanks to Virginia Snodgrass Rangel, Meghan Lehr, Melinda Lemke, and Stephen Spring for their thoughtful assistance. We also wish to extend special thanks and admiration to all of the educators in our three study districts, not only for their assistance in our project, but for their commitment to education. education policy analysis archives Volume 20 Number 25 August 27th , 2012 ISSN 1068-2341 Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for non- commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by the author(s) or EPAA. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, H.W. WILSON & Co., QUALIS A2 (Brazil), SCImago Journal Rank; SCOPUS, SOCOLAR (China). Please contribute commentaries at http://epaa.info/wordpress/ and send errata notes to Gustavo E. Fischman fischman@asu.edu D istrict-W ide E ffects on D ata U se in the C lassroom 29 education policy analysis archives editorial board Editor Gustavo E. Fischman (Arizona State University) Associate Editors: David R. Garcia (Arizona State University) Rick Mintrop, (University of California, Berkeley) Jeanne M. Powers (Arizona State University) Jessica Allen University of Colorado, Boulder Christopher Lubienski University of Illinois, Urbana- Champaign Gary Anderson New York University Sarah Lubienski University of Illinois, Urbana- Champaign Michael W. Apple University of Wisconsin, Madison Samuel R. Lucas University of California, Berkeley Angela Arzubiaga Arizona State University Maria Martinez-Coslo University of Texas, Arlington David C. Berliner Arizona State University William Mathis University of Colorado, Boulder Robert Bickel Marshall University Tristan McCowan Institute of Education, London Henry Braun Boston College Heinrich Mintrop University of California, Berkeley Eric Camburn University of Wisconsin, Madison Michele S. Moses University of Colorado, Boulder Wendy C. Chi* University of Colorado, Boulder Julianne Moss University of Melbourne Casey Cobb University of Connecticut Sharon Nichols University of Texas, San Antonio Arnold Danzig Arizona State University Noga O'Connor University of Iowa Antonia Darder University of Illinois, Urbana- Champaign João Paraskveva University of Massachusetts, Dartmouth Linda Darling-Hammond Stanford University Laurence Parker University of Illinois, Urbana- Champaign Chad d'Entremont Strategies for Children Susan L. Robertson Bristol University John Diamond Harvard University John Rogers University of California, Los Angeles Tara Donahue Learning Point Associates A. G. Rud Purdue University Sherman Dorn University of South Florida Felicia C. Sanders The Pennsylvania State University Christopher Joseph Frey Bowling Green State University Janelle Scott University of California, Berkeley Melissa Lynn Freeman* Adams State College Kimberly Scott Arizona State University Amy Garrett Dikkers University of Minnesota Dorothy Shipps Baruch College/CUNY Gene V Glass Arizona State University Maria Teresa Tatto Michigan State University Ronald Glass University of California, Santa Cruz Larisa Warhol University of Connecticut Harvey Goldstein Bristol University Cally Waite Social Science Research Council Jacob P. K. Gross Indiana University John Weathers University of Colorado, Colorado Springs Eric M. Haas WestEd Kevin Welner University of Colorado, Boulder Kimberly Joy Howard* University of Southern California Ed Wiley University of Colorado, Boulder Aimee Howley Ohio University Terrence G. Wiley Arizona State University Craig Howley Ohio University John Willinsky Stanford University Steve Klees University of Maryland Kyo Yamashiro University of California, Los Angeles Jaekyung Lee SUNY Buffalo * Members of the New Scholars Board E ducation P olicy A nalysis A rchives V ol. 20 N o. 25 30 archivos analíticos de políticas educativas consejo editorial Editor: Gustavo E. Fischman (Arizona State University) Editores. Asociados Alejandro Canales (UNAM) y Jesús Romero Morante (Universidad de Cantabria) Armando Alcántara Santuario Instituto de Investigaciones sobre la Universidad y la Educación, UNAM México Fanni Muñoz Pontificia Universidad Católica de Perú Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Imanol Ordorika Instituto de Investigaciones Economicas – UNAM, México Pilar Arnaiz Sánchez Universidad de Murcia, España Maria Cristina Parra Sandoval Universidad de Zulia, Venezuela Xavier Besalú Costa Universitat de Girona, España Miguel A. Pereyra Universidad de Granada, España Jose Joaquin Brunner Universidad Diego Portales, Chile Monica Pini Universidad Nacional de San Martín, Argentina Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Paula Razquin UNESCO, Francia María Caridad García Universidad Católica del Norte, Chile Ignacio Rivas Flores Universidad de Málaga, España Raimundo Cuesta Fernández IES Fray Luis de León, España Daniel Schugurensky Universidad de Toronto-Ontario Institute of Studies in Education, Canadá Marco Antonio Delgado Fuentes Universidad Iberoamericana, México Orlando Pulido Chaves Universidad Pedagógica Nacional, Colombia Inés Dussel FLACSO, Argentina José Gregorio Rodríguez Universidad Nacional de Colombia Rafael Feito Alonso Universidad Complutense de Madrid, España Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Pedro Flores Crespo Universidad Iberoamericana, México Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM México Verónica García Martínez Universidad Juárez Autónoma de Tabasco, México José Luis San Fabián Maroto Universidad de Oviedo, España Francisco F. García Pérez Universidad de Sevilla, España Yengny Marisol Silva Laya Universidad Iberoamericana, México Edna Luna Serrano Universidad Autónoma de Baja California, México Aida Terrón Bañuelos Universidad de Oviedo, España Alma Maldonado Departamento de Investigaciones Educativas, Centro de Investigación y de Estudios Avanzados, México Jurjo Torres Santomé Universidad de la Coruña, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM México Antoni Verger Planells University of Amsterdam, Holanda José Felipe Martínez Fernández University of California Los Angeles, USA Mario Yapu Universidad Para la Investigación Estratégica, Bolivia D istrict-W ide E ffects on D ata U se in the C lassroom 31 arquivos analíticos de políticas educativas conselho editorial Editor: Gustavo E. Fischman (Arizona State University) Editores Associados: Rosa Maria Bueno Fisher e Luis A. Gandin (Universidade Federal do Rio Grande do Sul) Dalila Andrade de Oliveira Universidade Federal de Minas Gerais, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Paulo Carrano Universidade Federal Fluminense, Brasil Luciano Mendes de Faria Filho Universidade Federal de Minas Gerais, Brasil Alicia Maria Catalano de Bonamino Pontificia Universidade Católica-Rio, Brasil Lia Raquel Moreira Oliveira Universidade do Minho, Portugal Fabiana de Amorim Marcello Universidade Luterana do Brasil, Canoas, Brasil Belmira Oliveira Bueno Universidade de São Paulo, Brasil Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil António Teodoro Universidade Lusófona, Portugal Gaudêncio Frigotto Universidade do Estado do Rio de Janeiro, Brasil Pia L. Wong California State University Sacramento, U.S.A Alfredo M Gomes Universidade Federal de Pernambuco, Brasil Sandra Regina Sales Universidade Federal Rural do Rio de Janeiro, Brasil Petronilha Beatriz Gonçalves e Silva Universidade Federal de São Carlos, Brasil Elba Siqueira Sá Barreto Fundação Carlos Chagas, Brasil Nadja Herman Pontificia Universidade Católica –Rio Grande do Sul, Brasil Manuela Terrasêca Universidade do Porto, Portugal José Machado Pais Instituto de Ciências Sociais da Universidade de Lisboa, Portugal Robert Verhine Universidade Federal da Bahia, Brasil Wenceslao Machado de Oliveira Jr. Universidade Estadual de Campinas, Brasil Antônio A. S. Zuin Universidade Federal de São Carlos, Brasil