Characteristics of the Opt-Out Movement: Early Evidence for Colorado Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 12/8/2018 Facebook: /EPAAA Revisions received: 12/2/2018 Twitter: @epaa_aape Accepted: 12/3/2018 education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 27 Number 33 April 1, 2019 ISSN 1068-2341 Characteristics of the Opt-Out Movement: Early Evidence for Colorado Grant Clayton Andrea J. Bingham & Gregory B. Ecks Univesity of Colorado – Colorado Springs United States Citation: Clayton, G., Bingham, A. J., & Ecks, G. B. (2019). Characteristics of the opt-out movement: Early evidence for Colorado. Education Policy Analysis Archives, 27(33). http://dx.doi.org/10.14507/epaa.27.4126 Abstract: Testing and accountability measures have continued to expand since the passage of the No Child Left Behind Act in 2001. In addition to school and district accountability, student test scores increasingly formed the foundation of teacher performance metrics. State participation rates exceeded the 95% minimum prescribed by law despite increasing opposition to many testing requirements. However, the rollout of the Common Core aligned PARCC tests in 2015 marked the start of a backlash against state mandated testing. The movement, commonly called opt-out, encouraged families not to participate in required tests. We use pooled OLS regression on statewide panel data from Colorado schools to examine school-level characteristics in one of the states with the largest declines in test participation. We find the prevalence of opt-out is largest in charter schools, suburban and rural areas, higher performing schools, and schools with a higher proportion of White students. Keywords: Accountability; education reform; politics of education http://epaa.asu.edu/ojs/ http://dx.doi.org/10.14507/epaa.27.4126 Education Policy Analysis Archives Vol. 27 No. 33 2 Características del movimiento de “opt-out”: Evidencia temprana para Colorado Resumen: Las medidas de evaluación y rendición de cuentas han continuado expandiéndose desde la aprobación del Acta de No Child Left Behind en 2001. Además de la responsabilidad de la escuela y el distrito, los puntajes de las pruebas de los estudiantes formaron cada vez más la base de las métricas de rendimiento de los maestros. Las tasas de participación del estado superaron el mínimo del 95% prescrito por la ley a pesar de la creciente oposición a muchos requisitos de prueba. Sin embargo, el lanzamiento de las pruebas PARCC alineadas con Common Core en 2015 marcó el inicio de una reacción violenta contra las pruebas exigidas por el estado. El movimiento, comúnmente llamado de exclusión, alentó a las familias a no participar en las pruebas requeridas. Usamos la regresión OLS agrupada en los datos del panel estatal de las escuelas de Colorado para examinar las características a nivel escolar en uno de los estados con los mayores descensos en la participación en las pruebas. Encontramos que la prevalencia de la exclusión voluntaria es mayor en las escuelas charter, áreas suburbanas y rurales, escuelas con mayor rendimiento y escuelas con una mayor proporción de estudiantes blancos. Palabras clave: Rendición de cuentas; Reforma educativa; política de la educación Características do movimento de “opt-out”: Evidencia temprana para Colorado Resumo: As medidas de teste e rendição de contas continuaram a se expandir desde a aprovação da Lei de No Child Left Behind, de 2001. Além da responsabilidade escolar e distrital, os resultados dos testes dos alunos formaram cada vez mais a base das métricas de desempenho dos professores. As taxas de participação do Estado excederam o mínimo de 95% prescrito pela lei, apesar da crescente oposição a muitos requisitos de teste. No entanto, a implementação dos testes PARCC alinhados ao Common Core em 2015 marcou o início de uma reação contra os testes obrigatórios do Estado. O movimento, comumente chamado de opt-out, encorajou as famílias a não participarem dos testes exigidos. Usamos a regressão OLS agrupada em dados de painel estaduais de escolas do Colorado para examinar as características do nível escolar em um dos estados com os maiores declínios na participação em testes. A prevalência de opt-out é maior em escolas charter, suburbanas e rurais, escolas de melhor desempenho e escolas com maior proporção de estudantes brancos. Palavras-chave: Rendição de contas; reforma educacional; política de educação Introduction For better or worse, standardized testing is one of the foundations of the modern accountability-driven educational system in the United States. Countless policies across the nation hold schools, and increasingly, teachers, accountable for student performance on such exams. Standardized test results are widely reported in the media and are seen by parents and policymakers as a reflection of school quality. A major assumption underlying the use of standardized tests is that the group of students taking the exams represents the student body within the school or school system. It is for that reason that the No Child Left Behind Act required 95% participation rates on mandated tests. Typically, schools have achieved and exceeded this threshold (CDE, 2015; Chingos, 2015). Recently, however, a movement in which students do not take the exams has cropped up in some areas of the country. Starting in 2015, a number of locations began to experience significant levels of non- participation – an event that coincided with the implementation of the Common Core aligned PARCC tests for states such as Colorado. Typically known as “opting-out,” this non-participation has been relatively unexamined in terms of the types of students and schools directly involved. Characteristics of the Opt-Out Movement 3 Nationally, opt-out rates have varied dramatically with California experiencing a relatively modest 3%, while New York and Colorado had 10% and 20% rates, respectively, in English Language Arts (Bennett, 2016). To date, few studies examine the student level characteristics of schools where participation in opt-out was prevalent. Colorado represents an excellent context for a detailed investigation of the opt-out phenomenon. First, the state experienced a dramatic increase in opt-out rates after the start of the Common Core aligned tests. Between the 2011 and 2014 school years, no district in the state fell below the required 95% participation rate. In 2015, however, participation rates in six districts fell to as low as 60% participation, depending on the grade level and test (CDE, 2015; Gorski, 2015b). Second, the state simultaneously adopted a new teacher accountability system utilizing student-level test data for high-stakes employment decisions. Finally, Colorado continues to be a national leader in opting-out, calling into question accountability requirements that utilize test scores for teachers, schools, and federal measures. Characteristics that predict changes in the rate of opt-out are informative to policy considerations in a number of ways. If opt-out is evenly or randomly distributed across schools, then there is little cause for concern about the quality of inferences on state-level assessments or teacher evaluation metrics. If, however, there are systematic differences between those who opt-out and those who participate, then those inferences become suspect. For example, highly-involved parents and/or parents with high levels of cultural capital – class-based resources that can be used to garner social profits (Bourdieu, 1986) – and knowledge about schooling could withdraw their students from what they see as extraneous or excessive state-level assessments. This is consistent with research suggesting that privileged students and their parents – those with higher levels of cultural capital – are more likely to ask for exceptions (Lareau & Calarco, 2012). Other parents, with less knowledge about the mechanics of testing or those who defer to their children’s educators, may be less likely to withdraw their students from testing. Because cultural capital has been shown by some to be associated with educational outcomes (e.g. DiMaggio & Mohr, 1985), these scenarios could skew school-level results. Further, should opt-out be significantly more likely in lower performing schools, state and federal education policy makers should view these changes with the same suspicion as widespread test cheating schemes. Finally, systematic differences across racial/ethnic lines should raise questions regarding active or passive discrimination on the part of teachers or school leaders. To explore opt-out, we examined changes in grade-level participation rates on state- mandated ELA tests in grades 3-9 in Colorado after the spread of the opt-out movement in 2015. The primary objective of this study is to explore whether particular school characteristics are associated with a decline in test participation. Specifically, our research question asks: Are there school-level characteristics that predict opt-out behavior in state mandated exams? We found that the percentage of opt-out is largest in suburban and rural areas, higher performing public schools, and schools with a higher proportion of White students. Background High-stakes testing and widespread federal incentives to states regarding accountability measures can largely be traced to the second Bush administration’s reauthorization of the Elementary and Secondary Education Act (ESEA) known as No Child Left Behind (NCLB). NCLB was clearly the largest expansion of ESEA and, for the first time, included required annual assessments for all students. Schools and districts from every state were now accountable to federal student performance mandates that aligned assessments to state content standards. Annual assessments in ELA were required for all students in grades 3-8 and once during high school Education Policy Analysis Archives Vol. 27 No. 33 4 (Egalite, Fusarelli, & Fusarelli, 2017; McGuinn, 2016). In an attempt to guarantee educational equity, states were required to report the level of student proficiency as measured by adequate yearly progress (AYP) for all relevant student subgroups, including gender, race/ethnicity, poverty, disability, and English language status (Haertel & Herman, 2005). The Obama administration’s Race to the Top program further incentivized federal priorities including adoption of the Common Core and linking student test performance to teacher accountability (U.S. Department of Education, 2009). In late 2015, the Obama administration passed the Every Student Succeeds Act (ESSA), the ninth major reauthorization of ESEA. Addressing concerns from states, districts, and schools, the act removed the AYP measure; however, the focus on rigorous content standards and the use of annual assessments established in NCLB remained unchanged. The new law returned a number of accountability decisions to the state and local level and provided greater flexibility with performance level indicators and school improvement interventions. The reauthorization also maintained a focus on the idea of educational equity through assessment, requiring the reporting of student subgroup performance on two new student subgroups: gender and migrant status (National Conference of State Legislatures, 2015). While there were some noticeable differences between ESSA and NCLB, using assessments to attempt to improve student achievement in underperforming schools remained a fundamental focus of federal education policy (Egalite et al., 2017; McGuinn, 2016). Central to the increase in accountability measures was a high participation rate in state mandated testing. NCLB required, and ESSA retained, the threshold of 95% participation in large part to ensure valid inferences about the results for both the school as a whole and reported subgroups (Spellings, 2007). The failure to achieve high participation rates has been shown to negatively affect the ability of policymakers to achieve valid interpretations and correctly identify low performing schools (Baker & Linn, 2002; Betebenner & Linn, 2010). The issue of the diminished participation and the degree to which scores reflected the total population was further exacerbated when student-level test performance was used as part of the measure of teacher efficacy in systems such as Colorado’s. The Opt-Out Movement Overall, limited research exists on the opt-out movement. Existing studies present a complex narrative. For example, a New York study using district-level data found a positive relationship between wealth/test scores and opting-out (Chingos, 2015). After controlling for free/reduced lunch, however, the relationship changed direction, showing that districts with lower scores had higher levels of opting-out (Chingos, 2015). There are other accounts, from the popular media and other sources, that suggest that most opt-outs are representative of a limited number of highly involved, affluent families in a limited number of schools (Gorski, 2015b; Robertson, 2015). This is supported by some existing research. A non-representative, online national survey, for example, found that activists in the opt-out movement tend to be “highly educated, white, married, politically liberal parent(s) whose children attend public school and whose household median income is well above the national average” (Pizmony-Levy & Saraisky, 2016, p. 6). Opt-out participants were well informed and represented groups that frequently test well – only a fraction of the respondents expressed concern about test performance or school quality as a reason for opting- out. Indeed, in Pizmony-Levy and Saraisky’s (2016) study, many opt-out participants indicated that their primary reason for opting-out was opposition to the use of tests to evaluate teachers. Others were concerned about how testing promotes rote memorization over critical thinking, and about growing privatization and the role of corporations in schools. In short, this research, and the Characteristics of the Opt-Out Movement 5 accompanying media accounts, suggest that parents with knowledge about schooling and educational systems and the perceived power to question those systems – in other words, parents with high levels of cultural capital – are more empowered to opt-out. Still, some states – like New York and Colorado – have experienced relatively high rates of opt-out, suggesting the phenomenon may be more widespread and complex than a limited group of suburban districts. Other related research suggests that for schools facing federal and state sanctions, such as school closure or change to charter status, opting-out could present an opportunity to artificially increase test scores in ways similar to cheating (Amrein-Beardsley, Berliner, & Rideau, 2010; Jacob & Levitt, 2002). For example, Amrein-Beardsley, Berliner, & Rideau (2010), found that a significant number of surveyed teachers had at least heard of another teacher helping students in some way on state-mandated tests. Relating this to the practice of opting-out, one teacher in Amrein-Beardsley and colleagues’ study reported that she/he encouraged a student to opt-out, stating: The student did not speak the language so it was very unfair to take a test that one could not read. So I said stay home in the morning and come after lunch. I allowed the student to do the math section because he could do the problems. (p. 15) Though there has been little research as to how widespread this issue is, and we could only find one example of this in a published study, this conduct warrants attention. While cheating artificially raises scores of select students, opting-out removes potentially lower scoring students from the testing pool entirely. Teachers and school leaders could be actively or passively encouraging students to test based on their perceived ability to meet achievement standards (Figlio, 2006; Freeman, 2015), which could mean discrimination along racial/ethnic lines and/or socioeconomic status. In this way, selectively encouraging a few students to opt-out could achieve similar aims as more complicated cheating programs. Indeed, researchers using Ohio data demonstrated that school rankings could be swayed by a change in the participation of as few as 11 students in a single school (Beaver, Westmaas, & Sludden, 2015). If this motivation is a primary reason for opting-out, we should expect schools with lower accountability ratings to experience higher levels of opting-out. There is also the possibility that opting-out could reflect regional differences in the perceived utility of school accountability systems. Rural, urban, and suburban schools differ on many of the metrics that reflect geographic differences. This includes wide disparities in base funding, passing rates for bond initiatives, and different rates of attendance to higher education (Bowers, Metzger, & Militello, 2010; Byun, Meece, & Irvin, 2012; Gebert, Calkin, & Schuster, 2004; Reyes & Rodriguez, 2004). It should not be surprising that they respond to accountability measures uniquely based on the local context. This could extend to opt-out, though there has been no research to date that parses out these differences. Finally, opting-out could be reflective of a broader, more complex social movement. According to Pizmony-Levy and Saraisky (2016), parents engaging in opting-out are taking part in collective action to “pursue their objectives mainly through non-institutional means of disruption of accustomed practices (e.g. standardized tests)” (p. 10). If a broader social movement is the motivation for opting-out, then timing of opt-out would be expected to coincide with a critical event such as the implementation of the Common Core aligned PARCC tests or new statewide accountability measures for teachers (Klandermans, 1984). As a result, we would expect wider variation across types of students and schools engaging in opting-out, as the motivations are not well captured through school-level covariates. In short, the research on opting-out is scant. Existing studies indicate that White, affluent, educated parents may be at the forefront of the opt-out movement, in opposition to high levels of testing, and the use of high stakes testing for teacher evaluation (Pizmony-Levy & Saraisky, 2016). Education Policy Analysis Archives Vol. 27 No. 33 6 There is also the potential that lower-performing students could be encouraged to opt-out in order to raise mandated test scores. The gaps in the literature leave many questions to be answered: does opt-out vary across racial/ethnic lines? By socioeconomic status? By school-type? Turnaround status? Opt-out could be happening at different levels for specific sub-groups and school contexts. Framing the Opt-Out Movement: Cultural Capital The research outlined above suggests that the opt-out movement could be a reaction by families with the knowledge and perceived power to opt-out of testing without fear of repercussions. Research also indicates that opt-out may be a form of protest, with parents using opt-out to object to increased testing requirements, privatization, and how the tests are used, among other concerns (Pizmony-Levy & Saraisky, 2016). The idea of parents choosing to opt-out to help their children, or to protest testing and accountability could be seen as a manifestation of parents’ cultural capital. In this paper, we link cultural capital (Bourdieu, 1977; 1986) to this idea of increased parent knowledge about schooling and empowerment to question school practices, and use cultural capital as an explanatory framework for our findings. According to Bourdieu (1973), cultural capital is the internalization of culture and traditions, usually passed on by the family. Cultural capital is acquired over time and is present in knowledge about school, appropriate attitudes and beliefs, personal style, and linguistic competence sanctified by the dominant culture (Lamont & Lareau, 1988). It is class-based, with the more privileged classes able to use their internalized resources and knowledge in exchange for benefits (Bourdieu, 1986). Cultural capital is also institutionalized as credentials and qualifications that can be used for influence and economic gain. In the education system, cultural capital (or perceived lack of cultural capital) is how social inequality and class structure are perpetuated. In education, cultural capital also manifests in how parents interact with schools and educators – often, parents with high levels of cultural capital feel empowered to ask for exceptions, and they use their knowledge and resources to provide their children with educational advantages (Lareau & Calarco, 2012). Privileged parents also pass on internalized knowledge about appropriate school behavior to their children. Those students who know how to act in school – according to primarily White, middle-class norms – are better able to operate within that system. Scholars applying theories of cultural capital have also found that middle- and upper-class parents pass on notions of social class that provide advantages and encourage entitlement (Lareau, 2002). Indeed, students’ advantages and the entitlement propagated by parents’ levels of cultural capital may ultimately influence student performance – some research has also shown that cultural capital is related to student outcomes (DiMaggio & Mohr, 1985). In relation to opting-out, cultural capital may manifest in two ways: parents’ decision to opt- out in order to provide advantages to their children (e.g. to avoid “testing fatigue”), or the decision to opt-out in order to protest the very concept of testing (e.g. to object to using tests to evaluate teachers). In both scenarios, parents with higher degrees of cultural capital may have the knowledge and access to information about opting-out and feel sufficiently empowered to remove their children from required testing because they do not fear the repercussions of doing so. Parents with more cultural capital use their resources to gain advantages for their students in their interactions with teachers and schools (Lareau, 2011). It stands to reason then that if parents feel that their students are not gaining anything from high stakes testing requirements, or if they feel that the increased levels of testing are actively taking something away from their children, parents with high levels of cultural capital may try to assist their children by opting-out. Further, their experience with federal accountability measures reflects an increase in test preparation and narrowing of the curriculum to more accurately align with state and national standards such as the Common Core. Characteristics of the Opt-Out Movement 7 Thus, they may want to opt their students out of testing to protest the idea of “teaching to the test.” These parents’ social class and cultural capital may also allow them to feel entitled to opt-out to protest increased testing requirements, using tests to evaluate teachers, or the increased level of privatization in schools. Cultural capital thus provides an explanatory framework within which to situate our findings on opt-out. Colorado Context We focus specifically on Colorado in our exploration of the opt-out movement. We do so because Colorado presents an important case study of the type of schools and communities that have thus far been affected by the opt-out movement in some way. The state is geographically and politically diverse with a strong history of local control in education. It was one of three states, along with New York and Rhode Island, to experience large scale opt-out on state testing in 2015 (Bennett, 2016). Colorado fits squarely within a national movement wherein a majority of Americans were aware of opt-out as it related to standardized tests and the Common Core, specifically (Pizmony-Levy & Cosman, 2017). The implementation of the new PARCC tests in 2015 marked a change in the incremental increase in statewide testing and accountability measures. This change was characterized by an alignment to the Common Core and by an increased number of testing days. Moreover, this occurred against the backdrop of the baseline year of the new – and hotly contested – teacher accountability system, which required using student growth measures for 50% of the teachers’ total score (CDE, 2015; Johnston, 2014). This perceived structural shift in the education landscape prompted increased criticism from a number of stakeholders, including parents and school leaders. Criticism was widespread both nationally (Bennet, 2016; Fairbanks, 2015; Taylor & Rich, 2015) and locally (Gorksi, 2015a, 2015b); community leaders from local and national groups such as United to Counter the Core and United Opt Out encouraged parents to opt-out of testing (Gorksi, 2015b; Robertson, 2015). Policymakers in Colorado did attempt to respond to an apparent groundswell against the new tests. In January 2015, the Colorado State Board of Education (SBE) voted to allow local districts to opt-out of portions of state mandated testing – including the new PARCC assessments. This action subsequently called into question if the SBE had the statutory authority to release districts from testing and culminated in the State Attorney General issuing an opinion stating the SBE did not have the legal authority to grant such waivers (State of Colorado, 2015). Further complicating matters, the Commissioner of the Colorado Department of Education (CDE) issued a statement in February 2015 that districts would not be penalized for having participation rates fall below the required 95% (CDE, 2015), although it was silent on the impact on school performance rankings. The United States Department of Education, however, later rejected Colorado’s request and stated districts would not be exempt from participation requirements (Deslile, 2015). Colorado’s statewide school performance framework (SPF) places an emphasis on five key performance indicators: academic growth, academic achievement, academic growth gaps, postsecondary workforce readiness, and participation. Each performance indicator is rated to form an overall school performance ranking in order of overall achievement: performance, improvement, priority improvement, and turnaround. Schools placed on turnaround status must submit annual plans to the state and face additional monitoring requirements for a period of three years. Turnaround schools that fail to meet performance indicators are required to move to at least priority improvement and may be closed or reconstituted as charters (CDE, 2015). By the spring of 2016, Education Policy Analysis Archives Vol. 27 No. 33 8 the bottom two categories, turnaround and priority improvement, equaled approximately 9% of schools in the state. As they entered the 2015 testing window, schools had limited information as to the exact requirements for participation and specifically how the state would take opt-out into account in the school performance frameworks. For those schools in the turnaround and priority improvement categories, systematic non-participation held the simultaneous possibility of changing achievement rankings—up or down depending on which students failed to participate—and possible sanctions for failing to meet the 95% participation threshold. Though declining participation rates in some Colorado schools is well known, prior to this study, we have little detailed information about the characteristics of the schools where the opt-out phenomenon is most prevalent. Is the decline in participation widespread across the state, or is it centralized in a few particular schools that receive attention? Have participation rates dropped similarly in urban, rural, and suburban settings? Is opt-out most prevalent in low- or high- performing schools? The purpose of this study is to address such questions and more clearly document the relationship between observed characteristics and declines in test participation. In so doing, we aim to examine the overall pattern of relationships between school characteristics and test participation rates. Formally we ask: Are there school-level characteristics that predict opt-out behavior in state mandated exams? Sample and Methods We examined the participation rate of public traditional and charter schools in Colorado schools for state mandated testing for English language arts (ELA) from the 2012 testing cycle through 2016. This was accomplished by assembling a number of publicly available datasets provided by CDE and the National Center for Education Statistics (NCES) into a single panel. From CDE, we obtained school performance framework (SPF) scores, enrollment characteristics, test participation in ELA and math, and school level characteristics. We merged these data with NCES urbanicity classification collapsed to the levels of urban, suburban, and rural. This panel resulted in all public schools in Colorado. We then restricted our sample to traditional public schools and charter schools—excluding sites dedicated to special education, vocational, online, alternative education, or closed within the 2012-2016 period— leaving a final sample of 96% of public schools in Colorado. This yielded 1,049 elementary, 545 middle, and 390 high schools. Descriptive data is presented in table 1. CDE suspended calculations of SPF during the transition to PARCC testing in 2015 and did not release new rankings until spring 2017. This meant all schools retained their 2014 ranking during the entire opt-out period. Schools may have, however, responded to suspending the SPF differently based on their baseline ranking. Characteristics of the Opt-Out Movement 9 Table 1 Descriptive Statistics of the Sample 3rd 4th 5th 6th 7th 8th 9th Urbanicity Rural 0.40 0.40 0.40 0.44 0.51 0.51 0.57 (0.49) (0.49) (0.49) (0.50) (0.50) (0.50) (0.50) Suburban 0.36 0.36 0.37 0.38 0.28 0.28 0.24 (0.48) (0.48) (0.48) (0.49) (0.45) (0.45) (0.43) Urban 0.23 0.24 0.24 0.18 0.21 0.21 0.18 (0.43) (0.43) (0.43) (0.38) (0.41) (0.41) (0.39) Race/Ethnicity Native American 0.01 0.01 0.01 0.01 0.01 0.01 0.01 (0.23) (0.23) (0.23) (0.20) (0.21) (0.20) (0.20) Asian 0.03 0.03 0.03 0.03 0.02 0.02 0.02 (0.03) (0.03) (0.03) (0.03) (0.03) (0.03) (0.03) Hispanic 0.33 0.33 0.33 0.31 0.32 0.32 0.31 (0.25) (0.25) (0.25) (0.25) (0.25) (0.25) (0.25) African-American 0.04 0.04 0.04 0.33 0.37 0.37 0.36 (0.07) (0.07) (0.07) (0.07) (0.07) (0.07) (0.07) Mutli-Racial 0.04 0.04 0.04 0.03 0.03 0.03 0.03 (0.03) (0.03) (0.03) (0.02) (0.02) (0.02) (0.02) White 0.56 0.56 0.56 0.59 0.58 0.58 0.59 (0.27) (0.27) (0.27) (0.26) (0.27) (0.27) (0.27) School Performance Framework Category Turnaround 0.03 0.03 0.03 0.03 0.03 0.03 0.02 (0.17) (0.17) (0.17) (0.17) (0.17) (0.17) (0.14) Priority Improvement 0.07 0.07 0.07 0.06 0.07 0.07 0.04 (0.26) (0.26) (0.26) (0.24) (0.25) (0.26) (0.20) Improvement 0.2 0.2 0.18 0.19 0.19 0.19 0.16 (0.40) (0.40) (0.40) (0.40) (0.40) (0.40) (0.40) Performance 0.68 0.68 0.70 0.68 0.67 0.67 0.70 (0.47) (0.47) (0.47) (0.47) (0.47) (0.47) (0.45) School Type Charter 0.13 (.33) 0.13 (.33) 0.13 (.33) 0.22 (.41) 0.25 (.43) 0.25 (.43) 0.16 (.37) (0.33) (0.33) (0.33) (0.41) (0.43) (0.43) (0.37) Total Number of Schools 1049 1049 1042 675 545 544 390 Mean (SD) Education Policy Analysis Archives Vol. 27 No. 33 10 In this study, we present results on the tested grades 3-9 in ELA. Although not reported here, results are similar when we look at the math exams. Our primary analysis was a pooled OLS regression model to measure the relationship between observed school characteristics and participation rates on state mandated tests in ELA. Formally, we used OLS to estimate: Participation rateit= β0+ β2X +β3 (X * opt-out years) + β4 opt out years+ε (1) where Participation rate is the percentage participation in ELA in year t for grade i; X is a vector of school characteristics including school racial and ethnic composition with White as the reference; charter; urbanicity with urban as the reference group; SPF with Performance as the reference; and percentage free/reduced lunch (FRL); opt-out years indicates the year is 2015 or later; ε is a robust term clustered at the school level; and β’s are parameters to be estimated. Our primary interest was the interaction between selected observed characteristics and the opt-out years indicator although we present results for baseline years for comparison and descriptive purposes. To examine various motivations for opting-out, we present a series of variations on Model (1). The first, presented in Table 3, is a pooled OLS of the baseline years 2012-2014. Next, we present a series of models interacted on a different set of covariates theorized to motivate opting out with the opt-out period. These interactions include urbanicity (Table 4), race/ethnicity (Table 5), SPF (Table 6), charter and proportion of FRL students (Table 7). All models met the assumptions for OLS including linearity, multivariate normality, and homoscedasticity. To further prevent issues with homoscedasticity, we used the robust standard errors command in Stata 14 (StataCorp, 2014). To explore concerns of multicollinearity, we conducted a variance inflation factor (VIF) tests for each model utilizing a VIF cutpoint of 10 to identify issues of multicollinearity (Mertler & Vannatta, 2013). Results We first set out to describe the basic patterns of participation rates during the time periods under consideration. Consistent with expectations from media reports, there was a substantial decline in the participation rates on Colorado’s state standardized tests in the 2015 school year with a rebound in 2016 that did not, however, return to previous levels. We present basic descriptive characteristics of the sample for ELA in Table 2. The table shows that while there was a decline in participation in all grade levels, the drop was sharpest in the middle and high school grades. In the ninth grade, participation dropped from 98% in 2014 to 75% in 2015. The remainder of the table illustrates differences in the changes of participation rates by SPF and urbanicity. The aggregate decline in participation, not controlling for any other factors, was similar across school performance ratings. However, it does appear that the decline in overall participation in 2015 was larger in rural schools than in the urban or suburban schools, although this gap closed the following year. Characteristics of the Opt-Out Movement 11 Table 2 Statewide Assessment Participation Rates in ELA By Grade Baseline Years Opt-out Years 2012 2013 2014 2015 2016 3rd 99.59 (1.33) 99.54 (1.51) 99.60 (1.57) 94.95 (8.78) 96.10 (6.33) 4th 99.67 (0.90) 99.66 (1.14) 99.60 (1.09) 94.68 (9.72) 95.64 (7.04) 5th 99.34 (4.71) 99.54 (3.42) 99.57 (1.31) 94.36 (10.16) 95.02 (7.87) 6th 99.53 (1.11) 99.30 (5.49) 99.46 (1.08) 92.31 (12.39) 92.99 (10.30) 7th 99.59 (0.90) 99.40 (2.02) 99.51 (0.93) 89.26 (14.90) 91.11 (12.40) 8th 99.15 (3.04) 99.04 (4.77) 99.25 (2.28) 86.74 (17.17) 88.00 (14.66) 9th 99.33 (4.18) 98.09 (4.50) 98.18 (5.0) 74.71 (24.16) 81.29 (20.31) School Performance Rating Performance 99.23 (2.47) 99.15 (3.98) 99.31 (1.53) 91.49 (14.37) 95.24 (8.02) Improvement 99.23 (2.47) 99.15 (3.98) 99.32 (1.53) 91.48 (14.37) 95.25 (8.03) Priority Improvement 98.96 (4.23) 99.31 (1.60) 99.14 (3.13) 91.99 (15.36) 97.49 (4.15) Turnaround 99.18 (2.75) 97.38 (11.57) 98.06 (6.59) 91.70 (12.35) 96.19 (6.07) Urbanicity Urban 99.41 (1.57) 99.24 (4.32) 99.36 (1.20) 91.36 (14.67) 95.11 (8.86) Rural 99.38 (3.16) 99.42 (3.05) 99.49 (1.67) 85.26 (21.82) 92.27 (11.44) Suburban 99.29 (3.12) 99.32 (3.12) 99.27 (2.31) 89.67 (17.53) 92.70 (11.97) Note: Mean participation rate (Standard Deviation) We descriptively illustrate the relationship between SPF and participation rate in Figure 1. The figure shows testing participation rates for ELA in grades 5, 7, and 9 in 2016 for schools of increasing school quality as measured under the accountability system without any statistical controls. The horizontal axis is the percentage of possible points on the SPF, which is the metric used to assign the categorical rating, and each symbol is a unique school/grade combination. The downward slope of the fitted lines shows the negative relationship between higher performing schools and the participation rate for each grade. Also apparent in the figure is the exceptionally high rate of opt-out for ninth graders — with many schools falling below 60%—and the degree of slope on the fit line relative two the other grades. Education Policy Analysis Archives Vol. 27 No. 33 12 Figure 1. Percentage participation in ELA for grades 5, 7, and 9 in ELA 2016 by overall SPF points earned. Baseline Characteristics We now turn to a basic regression model meant to describe the overall pattern of relationships between school characteristics and test participation rates accounting for other observed characteristics of the school. We first consider the relationship between school characteristics and participation rates prior to the advent of the opt-out movement. Table 3 reports the results from the pooled OLS regressions where the dependent variable is the participation rate for the school on the ELA exam and the independent variables include the full vector of school characteristics in Model 1. In order to assess the relationships between these variables and participation rates at the baseline, the regression in Table 3 includes only observations from years 2012, 2013, and 2014. 0 2 0 4 0 6 0 8 0 1 0 0 P a rt ic ip a ti o n R a te 0 20 40 60 80 100 SPF Percentage Earned 5th Grade Fitted values 7th Grade Fitted values 9th Grade Fitted values ELA 2016 Characteristics of the Opt-Out Movement 13 Table 3 Regression results for ELA for Baseline Years ELA 3rd 4th 5th 6th 7th 8th 9th Urbanicity Rural 0.075 0.250* -1.630 1.429 0.045 0.305 0.033 (-0.149) (0.148) (1.062) (1.347) (0.164) (0.211) (0.529) Suburban 0.061 0.293** -0.930 1.634 -0.076 -0.297 -0.643 (-0.135) (0.132) (0.700) (1.750) (0.149) (0.211) (0.557) Race/Ethnicity Native American 0.000 0.001 -0.018 0.053 -0.052 -0.252** -0.551** (-0.025) (0.014) (0.046) (0.094) (0.052) (0.110) (0.257) Asian -0.012 -0.002 -0.005 0.082* 0.031*** 0.054** 0.009 (-0.010) (0.009) (0.018) (0.047) (0.010) (0.027) (0.065) Hispanic 0.003 0.004 -0.017 0.002 0.005 0.002 -0.008 (-0.004) (0.003) (0.013) (0.007) (0.004) (0.008) (0.027) African American -0.022* 0.006 -0.010 -0.049 -0.005 -0.002 -0.012 (-0.012) (0.005) (0.032) (0.050) (0.007) (0.013) (0.034) Multi-Racial 0.014 -0.011 -0.030 -0.001 0.002 -0.061 -0.123 (-0.021) (0.015) (0.058) (0.061) (0.023) (0.073) (0.173) School Performance Rating Turnaround -0.407 -0.281 -1.240 -0.537 -0.719 -1.732 -7.880 (-0.558) (0.236) (1.801) (0.520) (0.461) (1.070) (4.862) Priority Improvement 0.053 -0.267** 0.352 -0.150 -0.211 -1.421* -2.488* (-0.105) (0.123) (0.289) (0.436) (0.140) (0.770) (1.279) Improvement -0.020 -0.152** 0.070 -0.415 -0.105 -0.459** -1.579** (-0.107) (0.066) (0.151) (0.576) (0.088) (0.217) (0.664) Charter -0.041 0.034 0.067 -0.310 0.075 0.530*** 1.699** (0.144) (0.061) (0.156) (0.258) (0.079) (0.188) (0.714) % FRL -0.004 -0.005* 0.008 -0.001 -0.005 0.007 0.023 (0.003) (0.002) (0.008) (0.010) (0.004) (0.009) (0.020) Observations 2,780 2,786 2,768 1,675 1,321 1,277 912 R-squared 0.071 0.104 0.081 0.056 0.230 0.161 0.265 *** p<0.01, ** p<0.05, * p<0.1 The upper panel of Table 3 compares participation rates prior to the start of the opt-out movement. Suburban and rural schools participated at slightly different rates than did the reference group urban, although the differences were rarely statistically or practically significant with differences in participation ranging from .033% in the ninth grade to -1.63% in the fifth grade. In baseline years, urbanicity was a significant predictor in the fourth grade with rural and urban schools participating at approximately 0.25% higher rates that suburban schools. Education Policy Analysis Archives Vol. 27 No. 33 14 The second panel of Table 3 compares the relative proportion of school populations with White as the reference group. In baseline years, Race/ethnicity was not a strong predictor of participation in elementary grades. In the eighth and ninth grades, schools with higher proportions of Native American students participating at lower rates of -0.252 and -0.551 respectively. Schools with higher proportions of Asian had slightly higher participation rates of 0.082 in the sixth and 0.031 in the seventh. The third panel of Table 3 compares baseline characteristics by school quality as measured by the state’s School Performance Framework (SPF). Prior to opt-out, lower SPF ratings frequently corresponded to decreased participation. Schools at the lowest ranking—turnaround—had some of the largest differences in participation relative to the top ranking comparison group—performance. The results were not statistically significant for turnaround schools although by the ninth grade the difference was a practically significant 7.88% lower participation. Similarly, priority improvement and improvement category schools showed increasingly larger differences in participation in later grades with significant differences in the eighth and ninth grades with priority improvement schools have 2.488% lower participation levels. Finally, charter status reflected greater variation by grade with 0.034% higher participation in the fourth grade and 1.699% higher in the ninth although results were only significant in the eighth and ninth grades. Increases in the proportion of FRL were rarely a significant predictor of test participation prior to the opt-out movement. Opt-out Years Next, we present the primary regression models that look at the differences in the relationship between the observed characteristics and participation rates before and after the opt-out phenomenon. Each table reports the estimates of interest from regression models by grade level that include interactions of particular categories. For space considerations, the table reports only results from the coefficients for the main effect and the interaction between the identified school characteristic and the opt-out period. Again, these are not fully interacted models. The models differ by the variables that are interacted with the 2015 and 2016 time periods when the opt-out movement appears to have begun. Table 4 reports the interaction results for urbanicity with the results relative to the control group, urban schools. Compared to urban schools, rural and suburban schools participated at higher rates prior to the start of opt-out. When interacted with the opt-out period, however, the results change. The rate of participation for rural schools consistently and significantly decreased relative to urban schools during the opt-out period. The interaction of rural* opt-out shows the change in participation rates ranged from -1.491% in the third grade to a high of -4.602% in the seventh grade. A similar picture emerges for suburban schools starting in the fourth grade. Participation rates consistently decreased in suburban school from the sixth through ninth grades with ninth graders experiencing an 8% decline relative to their urban peers. This finding is consistent with the common perception that largely parents and students outside of the cities drive the opt-out phenomenon. Characteristics of the Opt-Out Movement 15 Table 4 Regression results for Urbanicity VARIABLES 3rd 4th 5th 6th 7th 8th 9th Rural 0.715*** 0.890*** 0.704* 1.184*** 2.035*** 2.943*** 1.959* VIF 3.97 (0.180) (0.199) (0.404) (0.340) (0.452) (0.592) (1.167) Suburban 0.457*** 0.640*** 0.566* 0.892*** 1.563*** 2.000*** 2.275** VIF 3.48 (0.138) (0.136) (0.302) (0.272) (0.309) (0.451) (1.008) Rural* Opt Out -1.491*** -2.380*** -2.342*** -4.037*** -4.602*** -4.472*** -3.583* VIF 3.48 (0.451) (0.482) (0.554) (0.817) (0.962) (1.181) (2.023) Suburban* Opt Out 0.155 -0.859** -0.766 -2.185*** -3.467*** -4.177*** -8.151*** VIF 3.68 (0.385) (0.406) (0.480) (0.755) (0.973) (1.295) (2.310) Opt Out -4.148*** -3.882*** -4.180*** -5.024*** -7.293*** -9.691*** -19.18*** -0.319 (0.330) (0.397) (0.617) (0.706) (0.955) (1.768) Observations 4,707 4,702 4,671 2,846 2,246 2,191 1,556 R-squared 0.171 0.183 0.161 0.203 0.274 0.310 0.403 *** p<0.01, ** p<0.05, * p<0.1 Robust standard errors in parentheses Reference Group (Urban) Table 5 follows the same pattern, presenting the main effects of racial/ethnic composition of schools, opt-out and their interaction. The key outcome of interest is the interaction term showing the change in the effect of race/ethnicity during the opt-out time period. Before opt-out began there were few significant or practical differences based on race/ethnicity relative to the comparison group, White students. After opt-out began, increases in the percentage of Native American students consistently corresponded with a decline in participation rates although results were only significant in high school with a -1.182% change. More notable, however, is the result showing schools with higher proportions of Hispanic and African-American students relative to the proportion of White students, experienced increases in participation rates. Schools with larger proportions of African-American students showed positive and significant increases in participation during the opt-out time period. The effect was smallest in the third and fourth grades at 0.064% increase and highest in the ninth grade at 0.422% increase in participation. Schools with higher proportions of Hispanic students followed a similar patter with third grade showing the smallest increase at 0.046% and eighth grade showing the highest relative change at 0.178%. Education Policy Analysis Archives Vol. 27 No. 33 16 Table 5 Regression results for Race/Ethnicity 3rd 4th 5th 6th 7th 8th 9th Native 0.015 0.020 0.017 0.025 0.038* 0.003 -0.059 VIF 1.93 (-0.024) (0.013) (0.020) (0.028) (0.023) (0.028) (0.037) Asian -0.036** -0.042*** -0.035* 0.016 0.020 0.050* 0.290** VIF 2.06 (-0.015) (0.016) (0.019) (0.022) (0.026) (0.030) (0.113) Hispanic 0.013*** 0.008* 0.003 0.018** 0.020** 0.024** 0.052*** VIF 4.12 (-0.004) (0.005) (0.006) (0.008) (0.009) (0.011) (0.019) Black (0.013) 0.018** 0.0065 -0.011 0.020 0.034 0.058 VIF 2.66 (-0.011) (0.009) (0.023) (0.035) (0.017) (0.022) (0.041) Multi (-0.007) -0.036** -0.008 -0.009 0.014 0.029 -0.046 VIF 2.81 (-0.018) (0.016) (0.030) (0.035) (0.039) (0.056) (0.138) Opt Out -6.360*** -8.129*** -8.361*** - 10.92*** - 14.71*** - 17.67*** -21.81*** (-0.744) (0.798) (0.812) (1.086) (1.525) (1.610) (2.405) Native* Opt Out -0.045 -0.048 -0.086 -0.330* -0.243 -0.271 -1.182** VIF 2.07 (-0.092) (0.0954) (0.122) (0.188) (0.250) (0.342) (0.545) Asian* Opt Out 0.138*** 0.128** 0.0966 0.0907 0.180* 0.0167 -0.653* VIF 2.61 (-0.045) (0.054) (0.062) (0.085) (0.106) (0.144) (0.340) Black* Opt Out 0.064*** 0.064*** 0.080*** 0.148*** 0.238*** 0.293*** 0.422*** VIF 2.47 (-0.018) (0.018) (0.023) (0.047) (0.040) (0.059) (0.099) Multi* Opt Out -0.059 0.0336 -0.0559 -0.0411 -0.382 -0.527* -1.684*** VIF 2.81 (-0.099) (0.105) (0.117) (0.176) (0.254) (0.275) (0.556) Hispanic* Opt Out 0.046*** 0.078*** 0.091*** 0.104*** 0.141*** 0.178*** 0.176*** VIF 5.11 (-0.009) (0.009) (0.010) (0.014) (0.019) (0.020 (0.034) *** p<0.01, ** p<0.05, * p<0.1 Robust standard errors in parentheses Reference Group (White) Table 6 shows interaction results for the relationship between SPF—the school’s accountability rating— and participation rates. Prior to opt-out lower ranked schools consistently had lower participation rates relative to the top ranked Performance category. During the opt-out period, those results reverse sign with lower ranked schools showing an increase in participation rates relative to top ranked schools. The difference by school performance ranking was most stark in the middle and high school grades. For instance, controlling for other factors, the participation rates of turnaround and improvement schools increased by 13.51% and 10.27% in the eighth and ninth grades respectively. These regression results square with basic descriptive statistics that do not include statistical adjustments. The participation rates on the eighth grade ELA exam in 2014 and 2015, respectively, was 97% and 94% in turnaround schools, 99% and 91% in priority improvement schools, but 99% and 86% in performance schools. Note that there was a decline in participation in each of these school types at the time that opt-out began. This decline was just much larger in the higher rated schools than in the lower rated schools. Characteristics of the Opt-Out Movement 17 Table 6 Regression results for School Performance Factor 3rd 4th 5th 6th 7th 8th 9th Improvement -0.241* -0.644*** -0.535*** -0.904** -0.961*** -1.736*** -2.600*** VIF 2.03 (0.129) (0.119) (0.174) (0.429) (0.210) (0.290) (0.751) Priority -0.409*** -1.006*** -0.829*** -1.049*** -1.780*** -3.226*** -3.864*** VIF 1.88 (0.151) (0.191) (0.228) (0.343) (0.337) (0.654) (1.238) Turnaround -1.175** -1.373*** -2.553 -2.303*** -2.904*** -5.205*** -12.90*** VIF 1.96 (0.564) (0.300) (1.621) (0.424) (0.530) (1.024) (4.273) Opt Out -4.872*** -5.539*** -5.758*** -8.248*** -11.69*** -14.50*** -25.14*** (0.256) (0.277) (0.300) (0.472) (0.645) (0.710) (1.162) Turnaround* Opt Out 0.154 1.624* 3.218* 2.536*** 5.414*** 9.029*** 13.51** VIF 1.90 (1.276) (0.858) (1.860) (0.974) (1.231) (1.250) (6.012) Priority* Opt Out 1.543*** 2.778*** 2.191*** 3.571*** 5.458*** 5.533*** 6.318 VIF 1.78 (0.411) (0.348) (0.436) (0.658) (1.021) (1.791) (4.385) Improvement* Opt Out 0.876** 1.193** 0.804 2.246** 3.285*** 3.680*** 10.27*** VIF 2.09 (0.428) (0.507) (0.535) (0.903) (1.066) (1.255) (1.672) Observations 4,707 4,702 4,671 2,846 2,246 2,191 1,556 R-squared 0.168 0.181 0.158 0.198 0.275 0.312 0.409 *** p<0.01, ** p<0.05, * p<0.1 Robust standard errors in parentheses Reference Group (Performance) Table 7 shows results for charter schools and those with higher rates of poverty as measured by participation in the Free/Reduced price lunches (FRL). During the opt-out period, increasing percentages of FRL students corresponded to significantly increased participation rates. The effect was smallest in the third grade at 0.021% increase and largest in the ninth grade at 0.178% increase. The main effect for charter schools was a consistent decrease in participation. During opt-out, charter elementary schools had larger drops in participation relative to traditional public schools with the most pronounced change in the third and fourth grades at -2.87% and -2.73% respectively. Curiously, this effect changes sign in later grades with charter schools showing an increase in participation relative to traditional public schools of 3.26% in the eighth grade. Education Policy Analysis Archives Vol. 27 No. 33 18 Table 7 Regression Results for Charter and Percentage Free/Reduced Lunch 3rd 4th 5th 6th 7th 8th 9th Charter -0.472*** -0.502*** -1.428*** -1.215*** - 1.449*** - 1.261*** -1.817** VIF 1.9 (0.151) (0.116) (0.496) (0.322) (0.205) (0.265) (0.724) %FRL -0.038*** -0.051*** -0.058*** -0.080*** - 0.129*** - 0.152*** - 0.233*** VIF 4.46 (0.004) (0.005) (0.005) (0.010) (0.011) (0.013) (0.024) Opt Out -5.074*** -6.376*** -7.241*** -10.03*** - 15.38*** - 19.80*** - 29.28*** (0.272) (0.332) (0.368) (0.606) (0.870) (1.071) (1.581) Charter* Opt Out -2.870*** -2.733*** -1.620 -0.261 2.244** 3.260** 3.173 VIF 1.96 (0.876) (0.967) (1.116) (0.987) (1.052) (1.276) (2.393) %FRL* Opt Out 0.021*** 0.042*** 0.053*** 0.070*** 0.114*** 0.155*** 0.178*** VIF 2.78 (0.005) (0.005) (0.006) (0.011) (0.013) (0.017) (0.035) Observations 4,707 4,702 4,671 2,846 2,246 2,191 1,556 R-squared 0.178 0.193 0.169 0.205 0.287 0.331 0.409 *** p<0.01, ** p<0.05, * p<0.1 Robust standard errors in parentheses Robustness Tests Tables 3 through 6 present results for a specific source of motivation for opting-out— urbanicity, race/ethnicity, SPF, charter, or Free/Reduced price lunches. They do not, however, present a fully interacted model, which we present in Table 8. The fully interacted model acts as robustness test of model specification. The limited interaction models in the previous tables do not include the full vector of school characteristics and their interaction in order to concentrate attention on individual sources of motivation for opt-out. The results for the fully interacted model do reveal changes when including the full vector of covariates and their interactions. In most instances, interactions for the opt-out period lose their significance and reduce in magnitude. In some instances, the coefficients change direction as well. This reveals the model is not robust to model specifications and there may be other sources of variance that are not identified in this model. Further, a VIF test revealed an unacceptably high VIF for Free/reduced price lunch participation (r =0.60, VIF12.09) and the variable was removed from the final model. The fully interacted model in Table 8 presents a more complex picture as many of the interactions lose significance. There were some consistent results across model specifications. Notably, charter participation rates showed similar, significant declines in participation in the elementary grades across both the limited and fully interacted models. In the fully interacted model, charter status corresponded to 3.334% decline in participation in the third grade and remained significant—although smaller—through the sixth grade with a 1.714% decline. Similarly, increasing proportions of Black and Hispanic students continued to correspond with significant increases in participation across all grades. In the fully interacted model, increases in the proportion of Black students corresponded to a significant increase in participation all grades ranging from 0.064% in the Characteristics of the Opt-Out Movement 19 third grade to 0.422% in the ninth grade. Similarly, increases in the proportion of Hispanic students corresponded to significant increases in participation across all grades. In this case of Hispanic students the point estimates in Table 5 and Table 8 remained closer than for other races/ethnicities. Results for SPF were not as robust in the fully interacted model. In Table 5, the limited interaction model showed consistent and significant effects in grades six through nine for turnaround schools and all grades for priority improvement and improvement schools. In contrast, the fully interacted model in Table 8 shows inconsistent effects for SPF with the exception of Improvement schools in the ninth grade still showing a 6.86% increase in participation relative to the highest ranked reference group (performance). Similarly, urbanicity was no longer significant with the exception of suburban third graders in the fully interacted model at 1.426% increase in participation. Table 8 Regression Results for Fully Interacted Model 3rd 4th 5th 6th 7th 8th 9th Improvement -0.046 -0.200*** 0.139 -0.366 -0.116 -0.303** -1.185** 0.095 0.066 0.159 0.424 0.072 0.129 0.575 Priority 0.017 -0.265** 0.222 -0.112 -0.187 -0.952 -1.584 0.089 0.116 0.186 0.274 0.143 0.592 1.206 Turnaround -0.456 -0.263 -1.099 -0.596* -0.657 -2.128** -9.192* 0.531 0.230 1.598 0.358 0.400 1.032 4.782 Native -0.010 0.005 -0.001 0.006 0.014 -0.012 -0.002 0.020 0.009 0.022 0.019 0.010 0.024 0.033 Asian -0.008 -0.018* -0.012 0.0415*** 0.034** 0.044** 0.242** 0.012 0.011 0.014 0.014 0.015 0.021 0.099 Hispanic 0.000 0.000 -0.007 0.006* 0.001 0.009** 0.008 0.001 0.001 0.004 0.004 0.002 0.005 0.009 Black -0.020** -0.001 -0.012 -0.031 -0.002 0.008 0.025 0.009 0.004 0.027 0.034 0.006 0.009 0.023 Multi 0.011 -0.009 0.006 0.0439* 0.024 -0.007 -0.106 0.014 0.011 0.033 0.023 0.018 0.039 0.115 Charter -0.051 -0.002 -0.915* -0.100 0.152** 0.409*** -0.802 0.136 0.101 0.492 0.353 0.069 0.113 0.516 Opt Out - 6.134*** -7.045*** - 8.027*** -9.234*** - 14.90*** - 20.13*** - 24.95*** 0.983 1.066 1.269 1.682 2.389 2.667 4.383 Rural -0.003 0.058 -0.285 -0.006 0.001 0.078 -0.468 0.087 0.081 0.492 0.174 0.116 0.165 0.472 Suburban -0.103 0.035 -0.141 -0.020 0.121 -0.117 0.112 0.090 0.068 0.375 0.139 0.114 0.170 0.547 Charter* Opt Out - 3.334*** -3.302*** -2.303** -1.714* 0.551 1.359 3.425 VIF=1.95 0.885 0.975 1.121 1.040 1.051 1.277 2.458 Turnaround* Opt Out -1.341 -0.704 -0.129 -2.075* -0.555 0.263 2.447 VIF=2.40 1.382 1.035 1.983 1.090 1.444 1.625 7.291 Improvement* Opt Out -0.0872 -0.358 -1.327** 0.0387 0.560 -0.204 6.860*** VIF=2.44 0.519 0.606 0.633 1.031 1.110 1.300 2.049 Priority* Opt Out -0.166 0.288 -1.133* 0.333 1.322 0.170 -0.643 Education Policy Analysis Archives Vol. 27 No. 33 20 Table 8 cont. Regression Results for Fully Interacted Model VIF=2.11 0.539 0.515 0.616 0.868 1.098 1.972 4.693 Rural* Opt Out 0.0471 -0.592 -0.103 -1.339 0.242 2.063 3.458 VIF=6.25 0.570 0.590 0.834 1.011 1.374 1.616 2.948 Suburban* Opt Out 1.426*** 0.507 0.919 0.0988 0.120 0.832 -1.864 VIF=5.88 0.507 0.524 0.686 0.924 1.227 1.448 2.705 Native* Opt Out -0.0254 -0.0457 -0.0752 -0.320* -0.238 -0.263 -1.402** VIF=2.19 0.101 0.104 0.132 0.189 0.252 0.346 0.569 Asian* Opt Out 0.0967** 0.0937* 0.0528 0.0464 0.170 0.0400 -0.456 VIF=2.79 0.046 0.055 0.061 0.086 0.107 0.150 0.380 Black* Opt Out 0.113*** 0.0864*** 0.111*** 0.165*** 0.249*** 0.329*** 0.441*** VIF=3.43 0.024 0.023 0.035 0.053 0.044 0.064 0.117 Multi* Opt Out -0.100 -0.0322 -0.0921 -0.161 -0.382 -0.405 -1.428** VIF=7.27 0.106 0.111 0.130 0.189 0.283 0.301 0.611 Hispanic* Opt Out 0.049*** 0.076*** 0.099*** 0.104*** 0.149*** 0.197*** 0.189*** VIF=6.77 0.012 0.013 0.014 0.019 0.023 0.026 0.044 Observations 4,707 4,702 4,671 2,846 2,246 2,191 1,556 R-squared 0.189 0.209 0.188 0.222 0.309 0.353 0.442 *** p<0.01, ** p<0.05, * p<0.1 Robust standard errors in parentheses As a final test of robustness, we present Table 9 in which we computed a falsification test with all the same models using 2013 as the interaction year. Since 2013 is prior to opt-out we would not expect to see any of the same results. None of the individual 2013 interaction models for urbanicity, race/ethnicity, SPF, charter, or Free/Reduced price lunches were significant. For simplicity we present only the fully interacted model of the falsification test. In the fully interacted model, the only consistently significant result is for Hispanic students, who showed a decline in participation rates relative to the other baseline years. Table 9 Regression Results for Falsification Test 3rd 4th 5th 6th 7th 8th 9th Improvement 0.082 -0.482 -0.386 -0.238 -0.228 -1.001 0.213 (0.315) (0.386) (0.401) (0.568) (0.672) (0.769) (1.185) Priority 0.006 -0.341 -0.304 -0.164 -0.507 -2.854** - 5.142** (0.291) (0.283) (0.350) (0.465) (0.646) (1.417) (2.353) Turnaround -0.996 -0.653 -0.646 -1.602*** -1.787** -3.004** -6.315 (0.769) (0.544) (0.831) (0.585) (0.826) (1.202) (4.886) Native -0.029 -0.016 -0.088 -0.151 -0.124 -0.208 -0.422* (0.063) (0.069) (0.084) (0.139) (0.174) (0.237) (0.245) Asian 0.019 0.017 0.025 0.096** 0.120* 0.118 -0.010 (0.026) (0.032) (0.036) (0.047) (0.062) (0.091) (0.223) Hispanic 0.032*** 0.041*** 0.033** 0.066*** 0.066*** 0.0761*** 0.0562* (0.012) (0.013) (0.014) (0.018) (0.023) (0.023) (0.034) Black 0.023 0.033** 0.0131 0.05* 0.101*** 0.098** 0.048 Characteristics of the Opt-Out Movement 21 Table 9 cont. Regression Results for Falsification Test (0.017) (0.015) (0.028) (0.028) (0.033) (0.045) (0.074) Multiracial -0.069 -0.022 -0.080 -0.050 -0.165 -0.189 -0.455 (0.071) (0.074) (0.090) (0.109) (0.145) (0.164) (0.342) Charter -1.580*** -1.652*** -2.723*** -1.123** 0.482 0.951 2.449 (0.536) (0.584) (0.766) (0.531) (0.604) (0.804) (1.515) Rural -0.462 -0.212 -0.875 -0.564 0.615 1.372 1.365 (0.315) (0.315) (0.565) (0.477) (0.693) (0.879) (1.741) Suburban 0.163 0.243 -0.126 0.325 0.858 0.531 -0.233 (0.275) (0.269) (0.413) (0.430) (0.640) (0.779) (1.546) Free/Reduced -0.018** -0.013 -0.007 -0.026 -0.001 0.028 0.084** (0.009) (0.009) (0.011) (0.017) (0.024) (0.026) (0.038) 2013 -0.014** -0.494*** -0.748 -1.654* -1.461*** -1.349** -2.039* (0.183) (0.180) (0.483) (0889) (0.398) (0.555) (1.039) Rural* 2013 0.598* 0.169 1.510* 0.465 -0.392 -1.086 -1.756 (0.345) (0.349) (0.901) (0.599) (0.729) (0.898) (1.922) Suburban* 2013 -0.222 -0.271 0.745 -0.640 -0.701 -0.893 -0.884 (0.304) (0.296) (0.764) (0.547) (0.657) (0.826) (1.714) Turnaround* 2013 0.889 0.005 -5.198 1.041 0.827 2.553* -0.051 (0.872) (0.891) (5.800) (1.170) (1.447) (1.460) (5.131) Improvement* 2013 -0.215 0.0960 0.235 -0.595 0.170 0.977 -1.333 (0.392) (0.408) (0.416) (1.380) (0.684) (0.782) (1.575) Priority* 2013 -0.082 0.110 0.305 0.235 -0.002 2.439* 3.838 (0.346) (0.335) (0.407) (0.775) (0.713) (1.419) (2.474) Native* 2013 -0.0303 0.0320 0.0956 0.170 0.142 0.160 0.402 (0.072) (0.071) (0.085) (0.142) (0.176) (0.242) (0.253) Asian* 2013 -0.008 -0.047 -0.042 -0.078 -0.110* -0.094 0.116 (0.032) (0.034) (0.040) (0.062) (0.065) (0.100) (0.243) Black* 2013 -0.047** -0.032* 0.022 -0.148 -0.120*** -0.109** -0.073 (0.020) (0.017) (0.055) (0.101) (0.035) (0.045) (0.079) Hispanic* 2013 -0.029** -0.039*** -0.047*** -0.061*** -0.063*** -0.077*** -0.068* (0.013) (0.014) (0.017) (0.019) (0.024) (0.024) (0.038) Multiracial* 2013 0.121 0.00159 0.0869 0.0949 0.178 0.205 0.241 (0.077) (0.077) (0.093) (0.116) (0.150) (0.167) (0.385) Charter* 2013 1.720*** 1.615*** 2.693*** 0.877 -0.163 -0.450 -2.456 (0.542) (0.580) (0.762) (1.105) (0.618) (0.829) (1.649) Free/Reduced *2013 0.019** 0.010 0.017 0.035 0.005 -0.022 -0.073* (0.010) (0.010) (0.012) (0.022) (0.025) (0.026) (0.040) Observations 3,756 3,751 3,725 2,273 1,788 1,748 1,239 R-squared 0.179 0.182 0.158 0.195 0.293 0.318 0.449 *** p<0.01, ** p<0.05, * p<0.1 Robust standard errors in parentheses Education Policy Analysis Archives Vol. 27 No. 33 22 Unfortunately, results are not fully robust to changes in model specification. There are, however, significant predictors of changes in participation rates that remain across specifications. Notably, charter schools in elementary grades corresponded with large decreases in participation. Increases in the proportion of Black and Hispanic students consistently showed relative increases in participation and was robust to changes in model specifications. Discussion and Conclusion The opt-out movement has the potential to be one of the most important phenomena in education policy today. Test data is widely used by states, districts, and schools in order to find areas of excellence and weakness in public schools and can influence critical factors including school rankings, program evaluation, and merit pay. However, for these tests to be informative, they must present information representative of the students enrolled in the schools. The opt-out movement threatens the ability of tests to serve that task. Our results extend the literature beyond the previous examinations of New York (Bennett, 2016; Chingos, 2015) through a systematic analysis of a state not previously published, Colorado. Further, ours is the first study, of which we are aware, that employs panel data instead of a cross- sectional design. We illustrate the type of schools that have experienced changes in the proportion of students sitting for standardized exams. We find that opt-out is a widespread phenomenon within Colorado. However, the change in test participation was largest in suburban and rural areas, higher performing schools, schools with a higher proportion of White students, schools with lower proportions of students participating in Free/Reduced price lunches, and elementary students in charter schools. These results are at least consistent with the popular press narrative that opt-out is largely a response from parents whose children are likely to do well on the exams. For charter schools, it is not suprising that parents who were engaged in school choice were also more likely to make an affirmative decision not to take part in state required testing. One of the motivations behind the 95% participation requirement was to ensure the validity of inferences behind exam results. School level results do not appear to show systematic non- participation in the types of schools most likely to benefit from by actively encouraging parents and students to opt-out — those lowest in SPF rankings. Instead, the results seem to support a narrative that opt-out is largely driven by high-achieving schools. This could be due to a perceived lack of benefit from school accountability mechanisms and/or a reaction to what is perceived to be onerous levels of testing (Gorski, 2015b; Pizmony-Levy & Saraisky, 2016; Robertson, 2015). These results can also be explained, at least partially, by theories of cultural capital – more specifically, by differences in cultural capital reflective of social class. As noted earlier, families with more cultural capital perceive and interact with schools differently (Lareau, 1987; Lareau & Horvat, 1999). More affluent families express differing levels of expectations from schools and generally have higher levels of involvement, often irrespective of race (Lareau, 1987; Lareau & Horvat, 1999). Because of this increased involvement, parents may recognize that students’ participation in standardized testing is not mandatory. Indeed, such parents may view school accountability measures and testing requirements as something being done to other types of schools, not the schools where their children attend. These parents may feel empowered to use their privileged position to protest high stakes testing – they may have concerns about over-testing and increased privatization, or they may reject the concept of using tests to hold teachers and schools accountable. This aligns with the (limited) existing research. It is also reflected in the more than two thirds of opt- out participants who gave their children’s schools an A or B rating and also indicated that their Characteristics of the Opt-Out Movement 23 primary reasons for opting-out was opposition to the use of tests to evaluate teachers (Pizmony- Levy & Saraisky, 2016). Parents with higher levels of cultural capital may also use their privilege to provide their students with advantages (Lareau & Calarco, 2012), which in this case could include helping their children avoid “testing fatigue” or over-testing. Highly-informed parents with higher levels of cultural capital may be more concerned about over-testing or the perceived lack of utility in testing and may take steps to focus their students’ energy on more important tests. For example, although participation rates for PARCC testing in high schools declined during this time, there was not a similar decline in participation in Advanced Placement (AP) tests. From 2014 to 2016 the number of participating schools in offering AP classes increased by 31% and the number of AP examinations administered increased by 9% (College Board, 2016). In the Pizmony-Levy and Saraisky survey (2016), 27% of respondents listed “standardized tests take away too much time” as a reason for opting-out (p. 22). However, this does not appear to apply to AP tests, suggesting that parents may see these tests as more useful and more beneficial to students than high stakes accountability testing since AP tests reflect college bound behaviors. As a result, parents with higher degrees of cultural capital may see value in AP, but they may opt-out their students from the “extraneous” state tests. Conversely, scholars have consistently shown that parents with less cultural capital may be less informed about educational processes more generally and are thus more likely to defer to teacher and school decisions and processes (Lareau, 1987; Lareau & Horvat, 1999). For example, parents with lower levels of cultural capital may be more likely to defer to school procedures even if they are aware they can withdraw their students from testing and may be less likely to know that opting-out is an option in the first place. In other words, these parents may feel that they do not know as much about education and/or what is educationally best for their children as the teachers and administrators at their children’s schools and may thus refrain from opting-out. Alternatively, perhaps due to less involvement in the day-to-day processes of the school, these parents may not even know that opting-out is an option (something schools are unlikely to widely advertise, given the high stakes nature of testing). Thus, parents who opt-out must have outside knowledge of schooling and testing processes. Pizmony-Levy and Saraisky (2016) frame the opt-out movement in similar terms identifying it as social movement with necessary conditions of resources, changes in political structures, and communication of demands to achieve success. Two of the key elements they identify are “opposition to the Common Core” and opposition to the “the growing role of corporations in schools” (p. 22), both of which require higher levels of knowledge about school processes and schooling in general. The results of this paper are highly relevant for the policy conversation and for the general understanding of the nature and potential impacts of the opt-out movement, at least as it has been applied so far. Given that the idea of high stakes testing is reliant on those tests being representative of each school’s student population, higher levels of opt-out from any group is cause for concern. Colorado is one of a few states to have experienced a large and sustained decrease in test participation starting in 2015 (Bennett, 2016). There is little reason to believe this movement would not spread to other states. Indeed, in 2016, the U.S. Department of Education warned 11 states about their falling participation rates (Pizmony-Levy, 2018). This study presents Colorado as an informative case study as well as a quantitative perspective to the growing body of research into the opt-out phenomenon. A single state failing to meet federal accountability requirements presents an interesting test for decentralization efforts of the Department of Education under the current administration. From the perspective of those who want to continue to use testing and accountability as a way to hold ineffective schools responsible for poor performance, our finding that opt-out is much Education Policy Analysis Archives Vol. 27 No. 33 24 larger in schools that had previously received high performance ratings is somewhat comforting. However, the fact that participation rates do decline in lower performing schools, just not as much as in high-performing schools, suggests the possibility of broader policy implications. The examples of teachers and principals in lower performing schools responding to pressures of accountability are numerous (Blinder, 2015; Jacob & Levitt, 2002) and relaxing participation requirements may invite schools facing sanctions to encourage strategic opting-out. Our work utilizes school-level data, which captures many of the important school and community effects. It may, however, mask many important characteristics associated with opting- out such as prior student achievement and the relationship between demographic characteristics on the student level. Further research using student-level data is needed in order to assess which students within schools are more likely to opt-out. Such an analysis would be in a better position to understand whether there is any strategic opt-out according to prior student performance. Alternatively, a finding more consistent with the results in this paper may support the conclusion that opt-out is more likely among higher performing students that see the test as an unnecessary burden. References Amrein-Beardsley, A., Berliner, D. C., & Rideau, S. (2010). Cheating in the first, second, and third degree: Educators' responses to high-stakes testing. Education Policy Analysis Archives, 18, 14. http://dx.doi.org/10.14507/epaa.v18n14.2010 Baker, E. L., & Linn, R. L. (2002). Validity issues for accountability systems. CSE Technical Report 585 (Vol. 97). Los Angeles: CRESST. Beaver, J. K., Westmass, L., & Sludden, J. (2014). The potential effects of opting out of state tests in Pennsylvania. (Report September 201). Retrieved from Pacer Research for Action: http://www.researchforaction.org/wp-content/uploads/2014/09/Research-for-Action- Opt-Out-Policy-Brief-September-2014.pdf Bennett, R. E. (2016). Opt out: An examination of issues (Research Report No. RR-16-13). Princeton, NJ: Educational Testing Service. Betebenner, D. W., & Linn, R. L. (2010). Growth in student achievement: Issues of measurement, longitudinal data analysis, and accountability. Princeton, NJ: Educational Testing Service. Blinder, A. (2015, April 1). Atlanta educators convicted in school cheating scandal. The New York Times. Retrieved from https://www.nytimes.com/2015/04/02/us/verdict-reached-in- atlanta-school-testing-trial.html Bourdieu, P. (1973). Cultural reproduction and social reproduction. In R. Brown (Ed.), Knowledge, education, and cultural change: Papers in the sociology of education (pp. 71-112). London: Tavistock Publications Limited. Bourdieu, P. (1977). Outline of a Theory of Practice (Vol. 16). United Kingdom: Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511812507 Bourdieu, P. (1986). The forms of capital. In J. G. Richardson (Ed.), Handbook of theory and research for the sociology of education (pp. 241-258). New York: Greenwood Press. Bowers, A. J., Metzger, S. A., & Militello, M. (2010). Knowing the odds: Parameters that predict passing or failing school district bonds. Educational Policy, 24(2), 398-420. http://dx.doi.org/10.1177/0895904808330169 Byun, S. Y., Meece, J. L., & Irvin, M. J. (2012). Rural-nonrural disparities in postsecondary educational attainment revisited. American Educational Research Journal, 49(3), 412-437. http://dx.doi.org/10.3102/0002831211416344 Characteristics of the Opt-Out Movement 25 Chingos, M. (2015). Who opts out of state tests?(Research Report No. 113). Retrieved from Brookings Institution website: http://www.brookings.edu/research/papers/2015/06/18-chalkboard- who-opts-out-chingos College Board. (2016). AP archived data 2016 Colorado state report. Retrieved from: https://research.collegeboard.org/programs/ap/data/archived/ap-2016 Colorado Department of Education. (2015, February). Parent refusal information. Retrieved from: http://www.cde.state.co.us/communications/parentrefusal Deslile, D. S. (2015) (Letter to The Honorable Robert Hammond, Colorado Commissioner of Education.) Retrieved from http://www2.ed.gov/policy/eseaflex/secretary-letters/co4.html DiMaggio, P., & Mohr, J. (1985). Cultural capital, educational attainment, and marital selection. American Journal of Sociology, 90(6), 1231-1261. http://dx.doi.org/10.1086/228209 Egalite, A. J., Fusarelli, L. D., & Fusarelli, B. C. (2017). Will decentralization affect educational inequity? The Every Student Succeeds Act. Educational Administration Quarterly, 53(5), 757- 781. http://dx.doi.org/10.1177/0013161X17735869 Fairbanks, A. M. (2015, January 8). Teachers to the test. The Atlantic. Retrieved from http://www.theatlantic.com/education/archive/2015/01/teachers-to-the-test/384306/ Figlio, D.N. (2006). Testing, crime and punishment. Journal of Public Economics, 90(4), 837- 851. http://dx.doi.org/10.1016/j.jpubeco.2005.01.003 Freeman, E. (2015). Falsification, annual targets, and errant leadership: media portrayal of the Atlanta test-cheating scandal. Educational Policy, 29(7), 1012-1052. http://dx.doi.org/10.1177/0895904814531649 Gebert, K. M., Calkin, D. E., & Schuster, E. G. (2004). The Secure Rural Schools Act of 2000: Does it make rural schools secure? Journal of Education Finance, 30(2), 176-186. Gorski, E. (2015a, January 8). Colorado Board of Education votes to give districts opt-out on tests. The Denver Post. Retrieved from https://www.denverpost.com/2015/01/08/colorado- board-of-education-votes-to-give-districts-opt-out-on-tests/ Gorski, E. (2015b, February 25). “Opt-out” movement: Just say no to new Colorado assessments. The Denver Post. Retrieved fromhttps://www.denverpost.com/2015/02/28/opt-out- movement-just-say-no-to-new-colo-assessments/ Haertel, E., & Herman, J. (2005). A historical perspective on validity arguments for accountability testing. CSE Report 654 (Vol. 104). Los Angeles. http://dx.doi.org/10.1111/j.1744-7984.2005.00023.x Jacob, B.A., & Levitt, S.D. (2002). Rotten Apples: An investigation of the prevelence and predictors of teacher cheating. Boston, MA: National Bureau of Economic Research. Johnston, M. (2014, March 6). Giving Colorado’s teacher evaluation bill time to succeed. The Denver Post. Retrieved from http://www.denverpost.com/2014/03/06/giving-colorados-teacher- evaluation-bill-time-to-succeed/ Klandermans, B. (1984). Mobilization and participation: Social-psychological expansions of resource mobilization theory. American Sociological Review, 49(5), 583-600. http://dx.doi.org/10.2307/2095417 Lamont, M., & Lareau, A. (1988). Cultural capital: Allusions, gaps and glissandos in recent theoretical developments. Sociological Theory, 153-168. http://dx.doi.org/10.2307/202113 Lareau, A. (1987). Social class differences in family-school relationships: The importance of cultural capital. Sociology of Education, 2(60), 73-85. http://dx.doi.org/10.2307/2112583 Lareau, A. (2002). Invisible inequality: Social class and childrearing in black families and white families. American Sociological Review, 5(67), 747-776. http://dx.doi.org/10.2307/3088916 Lareau, A. (2011). Unequal childhoods: Class, race, and family life. Berkeley: University of California Press. Education Policy Analysis Archives Vol. 27 No. 33 26 Lareau, A., & Calarco, J. M. (2012). Class, cultural capital, and institutions: The case of families and schools. In S. T. Fiske & H. R Markus (Eds.),The secret handshake: Social class divides in face-to-face encounters. New York: Russell Sage Foundation. Lareau, A., & Horvat, E. M. (1999). Moments of social inclusion and exclusion race, class, and cultural capital in family-school relationships. Sociology of Education, 37-53. http://dx.doi.org/10.2307/2673185 McGuinn, P. (2016). From No Child Left Behind to the Every Student Succeeds Act: Federalism and the education legacy of the Obama administration. Publius: The Journal of American Federalism, 46(3), 392-415. http://dx.doi.org/10.1093/publius/pjw014 Mertler, C., & Vannatta, R. (2013). Advanced and multivariate statistical methods (5th ed.). Glendale, CA: Pyrczak. National Conference of State Legislatures. (2015). Summary of the Every Student Succeeds Act, legislation reauthorizing The Elementary and Secondary Education Act. Retrieved at http://www.ncsl.org/documents/educ/ESSA_summary_NCSL.pdf Pizmony-Levy, O. (2018, October 18). Opinion: The opt-out movement is gaining ground, quietly. The Hechinger Report. Retrieved from: https://hechingerreport.org/opinion-the-opt-out- movement-is-gaining-ground-quietly/. Pizmony‐Levy, O., & Cosman, B. (2017). How Americans view the Opt Out movement. Research Report. New York: Teachers College, Columbia University. Pizmony‐Levy, O., & Saraisky, N. (2016). Who opts out and why? Results from a national survey on opting out of standardized tests. Research Report. New York: Teachers College, Columbia University. Reyes, A. H., & Rodriguez, G. M. (2004). School finance: Raising questions for urban schools. Education & Urban Society, 37(1), 3-21. http://dx.doi.org/10.1177/0013124504268103 Robertson, P. (2015). Testing opt-out/Refusal guide for: Colorado. Retrieved from United Opt Out: http://unitedoptout.com/wp-content/uploads/2014/04/October-13-2015-Colorado-Opt- Out-Guide.pdf Spellings, M. (2007). Building on the results : A blueprint for strengthening the No Child Left Behind Act. U.S. Departement of Education. Retrieved from : www.ed.gov/policy/elsec/leg/nclb/buildingonresults.html. StataCorp. (2015). Stata 14 Base Reference Manual. College Station, TX: Stata Press. State of Colorado, Office of the Attorney General. (2015). Formal Opinion of Cynthia H. Coffman, Attorney General (No. 15-03). Denver, CO: Office of the Attorney General. Retrieved from http://www2.cde.state.co.us/artemis/lawserials/law112internet/law112153internet.pdf Taylor, K., & Rich, M. (2015, April 20). Teachers’ unions fight standardized testing, and find diverse allies. The New York Times.Retrieved from https://www.nytimes.com/2015/04/21/education/teachers-unions-reasserting-themselves- with-push-against-standardized-testing.html U.S. Department of Education. (2009, November). Race to the Top program executive summary. Washington, D.C.: Author. Retrieved January 20, 2017 from https://www2.ed.gov/programs/racetothetop/executive-summary.pdf Characteristics of the Opt-Out Movement 27 About the Authors Grant Clayton University of Colorado – Colorado Springs gclayto2@uccs.edu Grant Clayton is an Assistant Professor in the Department of Teaching and Learnin g at the University of Colorado – Colorado Springs. His primary research interests are in education policy, concurrent enrollment, and teacher preparation. Andrea J. Bingham University of Colorado – Colorado Springs abingham@uccs.edu Andrea J. Bingham is an Assistant Professor in the Department of Leadership, Research, and Foundations at the University of Colorado – Colorado Springs. Her research addresses organizational change and instructional reform. Gregory B. Ecks University of Colorado – Colorado Springs gregory.ecks@d11.org Gregory B. Ecks is an Adjunct Professor in the Department of Teaching and Learning at the University of Colorado – Colorado Springs and serves fulltime as Director Student Support & Engagement at Colorado Springs District 11. His research interest is in student engagement including attendance and discipline. education policy analysis archives Volume 27 Number 33 April 1, 2019 ISSN 1068-2341 Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for non- commercial purposes only, and no alteration or transformation is made in th e work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by the author(s) or EPAA. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), QUALIS A1 (Brazil), SCImago Journal Rank, SCOPUS, SOCOLAR (China). Please send errata notes to Audrey Amrein-Beardsley at audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. mailto:gclayto2@uccs.edu mailto:abingham@uccs.edu mailto:gregory.ecks@d11.org http://www.doaj.org/ mailto:audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE Education Policy Analysis Archives Vol. 27 No. 33 28 education policy analysis archives editorial board Lead Editor: Audrey Amrein-Beardsley (Arizona State University) Editor Consultor: Gustavo E. Fischman (Arizona State University) Associate Editors: David Carlson, Lauren Harris, Eugene Judson, Mirka Koro-Ljungberg, Scott Marley, Molly Ott, Iveta Silova (Arizona State University) Cristina Alfaro San Diego State University Amy Garrett Dikkers University of North Carolina, Wilmington Susan L. Robertson Bristol University Gary Anderson New York University Gene V Glass Arizona State University Gloria M. Rodriguez University of California, Davis Michael W. Apple University of Wisconsin, Madison Ronald Glass University of California, Santa Cruz R. Anthony Rolle University of Houston Jeff Bale University of Toronto, Canada Jacob P. K. Gross University of Louisville A. G. Rud Washington State University Aaron Bevanot SUNY Albany Eric M. Haas WestEd Patricia Sánchez University of University of Texas, San Antonio David C. Berliner Arizona State University Julian Vasquez Heilig California State University, Sacramento Janelle Scott University of California, Berkeley Henry Braun Boston College Kimberly Kappler Hewitt University of North Carolina Greensboro Jack Schneider University of Massachusetts Lowell Casey Cobb University of Connecticut Aimee Howley Ohio University Noah Sobe Loyola University Arnold Danzig San Jose State University Steve Klees University of Maryland Jaekyung Lee SUNY Buffalo Nelly P. Stromquist University of Maryland Linda Darling-Hammond Stanford University Jessica Nina Lester Indiana University Benjamin Superfine University of Illinois, Chicago Elizabeth H. DeBray University of Georgia Amanda E. Lewis University of Illinois, Chicago Adai Tefera Virginia Commonwealth University David E. DeMatthews University of Texas at Austin Chad R. Lochmiller Indiana University A. Chris Torres Michigan State University Chad d'Entremont Rennie Center for Education Research & Policy Christopher Lubienski Indiana University Tina Trujillo University of California, Berkeley John Diamond University of Wisconsin, Madison Sarah Lubienski Indiana University Federico R. Waitoller University of Illinois, Chicago Matthew Di Carlo Albert Shanker Institute William J. Mathis University of Colorado, Boulder Larisa Warhol University of Connecticut Sherman Dorn Arizona State University Michele S. Moses University of Colorado, Boulder John Weathers University of Colorado, Colorado Springs Michael J. Dumas University of California, Berkeley Julianne Moss Deakin University, Australia Kevin Welner University of Colorado, Boulder Kathy Escamilla University ofColorado, Boulder Sharon Nichols University of Texas, San Antonio Terrence G. Wiley Center for Applied Linguistics Yariv Feniger Ben-Gurion University of the Negev Eric Parsons University of Missouri-Columbia John Willinsky Stanford University Melissa Lynn Freeman Adams State College Amanda U. Potterton University of Kentucky Jennifer R. Wolgemuth University of South Florida Rachael Gabriel University of Connecticut Kyo Yamashiro Claremont Graduate University Characteristics of the Opt-Out Movement 29 archivos analíticos de políticas educativas consejo editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editores Asociados: Armando Alcántara Santuario (Universidad Nacional Autónoma de México), Angelica Buendia, (Metropolitan Autonomous University), Alejandra Falabella (Universidad Alberto Hurtado, Chile), Antonio Luzon, (Universidad de Granada), José Luis Ramírez, (Universidad de Sonora), Paula Razquin (Universidad de San Andrés), Maria Alejandra Tejada-Gómez (Pontificia Universidad Javeriana, Colombia) Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Ana María García de Fanelli Centro de Estudios de Estado y Sociedad (CEDES) CONICET, Argentina Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Miguel Ángel Arias Ortega Universidad Autónoma de la Ciudad de México Juan Carlos González Faraco Universidad de Huelva, España José Gregorio Rodríguez Universidad Nacional de Colombia, Colombia Xavier Besalú Costa Universitat de Girona, España María Clemente Linuesa Universidad de Salamanca, España Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Xavier Bonal Sarro Universidad Autónoma de Barcelona, España Jaume Martínez Bonafé Universitat de València, España José Luis San Fabián Maroto Universidad de Oviedo, España Antonio Bolívar Boitia Universidad de Granada, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Jurjo Torres Santomé, Universidad de la Coruña, España José Joaquín Brunner Universidad Diego Portales, Chile María Guadalupe Olivier Tellez, Universidad Pedagógica Nacional, México Yengny Marisol Silva Laya Universidad Iberoamericana, México Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Miguel Pereyra Universidad de Granada, España Ernesto Treviño Ronzón Universidad Veracruzana, México Gabriela de la Cruz Flores Universidad Nacional Autónoma de México Mónica Pini Universidad Nacional de San Martín, Argentina Ernesto Treviño Villarreal Universidad Diego Portales Santiago, Chile Marco Antonio Delgado Fuentes Universidad Iberoamericana, México Omar Orlando Pulido Chaves Instituto para la Investigación Educativa y el Desarrollo Pedagógico (IDEP) Antoni Verger Planells Universidad Autónoma de Barcelona, España Inés Dussel, DIE-CINVESTAV, México Paula Razquin Universidad de San Andrés, Argentina Catalina Wainerman Universidad de San Andrés, Argentina Pedro Flores Crespo Universidad Iberoamericana, México José Ignacio Rivas Flores Universidad de Málaga, España Juan Carlos Yáñez Velazco Universidad de Colima, México javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/816') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/819') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/820') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/4276') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1609') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/825') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/797') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/823') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/798') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/555') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/814') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/2703') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/801') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/826') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/802') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/816') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/3264') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/804') Education Policy Analysis Archives Vol. 27 No. 33 30 arquivos analíticos de políticas educativas conselho editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editoras Associadas: Kaizo Iwakami Beltrao, (Brazilian School of Public and Private Management - EBAPE/FGV, Brazil), Geovana Mendonça Lunardi Mendes (Universidade do Estado de Santa Catarina), Gilberto José Miranda, (Universidade Federal de Uberlândia, Brazil), Marcia Pletsch, Sandra Regina Sales (Universidade Federal Rural do Rio de Janeiro) Almerindo Afonso Universidade do Minho Portugal Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil José Augusto Pacheco Universidade do Minho, Portugal Rosanna Maria Barros Sá Universidade do Algarve Portugal Regina Célia Linhares Hostins Universidade do Vale do Itajaí, Brasil Jane Paiva Universidade do Estado do Rio de Janeiro, Brasil Maria Helena Bonilla Universidade Federal da Bahia Brasil Alfredo Macedo Gomes Universidade Federal de Pernambuco Brasil Paulo Alberto Santos Vieira Universidade do Estado de Mato Grosso, Brasil Rosa Maria Bueno Fischer Universidade Federal do Rio Grande do Sul, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Fabiany de Cássia Tavares Silva Universidade Federal do Mato Grosso do Sul, Brasil Alice Casimiro Lopes Universidade do Estado do Rio de Janeiro, Brasil Jader Janer Moreira Lopes Universidade Federal Fluminense e Universidade Federal de Juiz de Fora, Brasil António Teodoro Universidade Lusófona Portugal Suzana Feldens Schwertner Centro Universitário Univates Brasil Debora Nunes Universidade Federal do Rio Grande do Norte, Brasil Lílian do Valle Universidade do Estado do Rio de Janeiro, Brasil Flávia Miller Naethe Motta Universidade Federal Rural do Rio de Janeiro, Brasil Alda Junqueira Marin Pontifícia Universidade Católica de São Paulo, Brasil Alfredo Veiga-Neto Universidade Federal do Rio Grande do Sul, Brasil Dalila Andrade Oliveira Universidade Federal de Minas Gerais, Brasil