Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 9/14/2017 Facebook: /EPAAA Revisions received: 8/20/2018 Twitter: @epaa_aape Accepted: 8/27/2018 SPECIAL ISSUE Rethinking Education Policy and Methodology in a Post-truth Era education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 26 Number 150 November 19, 2018 ISSN 1068-2341 Scientifically Based Research in a Post-Truth Era Jessica Van Cleave Mars Hill University United States Citation: Van Cleave, J. (2018). Scientifically based research in a post-truth era. Education Policy Analysis Archives, 26(150). http://dx.doi.org/10.14507/epaa.26.3392 This article is part of the Special Issue, Rethinking Education Policy and Methodology in a Post-truth Era guest edited by Jennifer R. Wolgemuth, Mirka Koro-Ljungberg, Travis M. Marn, Anthony J. Onwuegbuzie, and Shaun M. Dougherty. Abstract: In this article, I explore the tension between the current political context in which science needs defending against anti-intellectualism and outright assaults on evidence as a means of decision-making on the one hand and the overzealous scientism that can result from backlash against a perceived lack of rigor in various f orms of inquiry. To do so, I return to the emergence of the discourse of scientifically based research (SBR) in education and the debates surrounding it (2002-2013), which have implications for how and why educational researchers would advocate for science and what that advocacy might do. Specifically, I argue that we must have a science that does not allow alternative facts and politically expedient truth claims while still allowing science to be flexible, responsive, and theoretically informed. I conclude by advocating for theoretically informed activism and non-innocent science. http://epaa.asu.edu/ojs/ http://dx.doi.org/10.14507/epaa.26.3392 Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 2 Keywords: scientifically based research; non-innocent science; governmentality; regime of truth Investigación con base científica en una era posverdad Resumen: En este artículo, explotan la tensión entre el actual contexto político en que la ciencia necesita defenderse contra el antiintetismo y los asaltos directos a la evidencia como un medio de toma de decisión, por un lado, y el cientificismo excesivamente celoso que puede resultar de la reacción contraria falta de rigor en varias formas de investigación. Para ello, vuelvo al surgimiento del discurso de la investigación científicamente fundamentada (SBR) en la educación y en los debates que la rodean (2002 -2013), que tienen implicaciones sobre cómo y por qué. hacer. Específicamente, argumento que debemos tener una ciencia que no permita hechos alternativos y reivindicaciones de verdad políticamente expeditas, mientras que todavía permite que la ciencia sea flexible, responsiva y teóricamente informada. Concluyo defendiendo el activismo teóricamente informado y la ciencia no inocente. Palabras-clave: investigación con base científica; ciencia no inocente; gubernamentalidad ; régimen de verdad Pesquisa cientificamente fundamentada em uma era pós-verdad Resumo: Neste artigo, eu exploro a tensão entre o atual contexto político em que a ciência precisa se defender contra o antiintelectualismo e assaltos diretos à evidência como um meio de tomada de decisão, por um lado, e o cientificismo excessivamente zeloso que pode resultar da reação contrária falta de rigor em várias formas de investigação. Para tanto, volto ao surgimento do discurso da pesquisa cientificamente fundamentada (SBR) na educação e nos debates que a cercam (2002-2013), que têm implicações sobre como e por quê. fazer. Especificamente, argumento que devemos ter uma ciência que não permita fatos alternativos e reivindicações de verdade politicamente expeditas, enquanto ainda permite que a ciência seja flexível, responsiva e teoricamente informada. Concluo defendendo o ativismo teoricamente informado e a ciência não inocente. Palavras-chave: investigação com base científica; ciencia no inocente; gubernamentalidad; regimen de verdad Scientifically Based Research in a Post-Truth Era The word science frequently is used as a value-free and neutral term, as if there is a general understanding of what is meant by it, and as if it is, by its nature, a contributor to the common good. However, science, like any other term, is bound up in power/knowledge relations that privilege some definitions and enactments over others. Science has long been a contested term. Indeed, in the wake of the United States presidential election of 2016, science became one of the many loci of resistance to the so-called “alternative facts” administration that had just been installed.1. The need 1 The Union of Concerned Scientists (2017) put together a report that, among other things, detailed attacks on science that had occurred in the first 6 months of the Trump administration. In the executive summary to that report, they highlighted the following: The administration has shown a blatant disregard for scientific facts and evidence, appointing officials with a track record of misrepresenting scientific information, overruling the recommendations of scientists on exposure to toxic pesticides, removing scientific information from agency websites, and dismissing independent science advisors. Aided and abetted by Congress, the Scientifically based research in a post -truth era 3 to support and defend science, which was perceived as being under attack and in need of defending, spurred a march on Washington, DC in April of 2017.2 This need was further bolstered by a Facebook group boasting more than half a million members that “champions robustly funded and publicly communicated science that serves the common good” (About, March for Science, n.d.) and now the March for Science Organization (www.marchforscience.com), which “work[s] toward a future where science is fully embraced in public life and policy” and includes as part of its goals “promoting evidence-based policies.” Science is good. Full stop. However, what’s missing from these conversations, mission statements, and goals, vehement and necessary though they are, is whose science we are defending and advocating, and whose, by omission, we are not.3 My goal is not to claim that there is not a diversity of sciences and scientific approaches represented by those who associate with the March for Science. Indeed, many who marched and many who support the organization come from similar perspectives to my own as well as a wide range of others. Rather, I contend that in using the word “science” without operationalizing it and by championing the idea of science that serves the “common good,” the March for Science as an organization puts forth a public face of a unitary science.4 In this article, I explore the tension between the current political context in which science needs defending against anti-intellectualism and outright assaults on evidence as a means of decision-making on the one hand and the overzealous scientism that can result from backlash against a perceived lack of rigor in various forms of inquiry. Indeed, in our attempts to rescue and defend science, we must be wary of restricting what science can be and do by narrowing our definitions in order to create a banner to stand behind.5 Responding to questions about scientists’ (and others) role as advocates for science requires an examination of science as a non-neutral concept. Such an examination is important because questions of science in educational research methodology and policy are fraught with issues of truth, power, and knowledge that demand complex responses. Educational research, for example, experienced the paradigm wars of the 1980s in which objectivist-positivist science was rejected in favor of interpretivist and critical approaches among others (Gage, 1989) that were seen as more appropriate to address the needs of society and schooling. Those paradigm wars, sometimes referred administration has delayed or eviscerated science-based rules that safeguard the American people, from protecting workers from toxic work environments to helping communities prepare for the impacts of climate change. Moreover, President Trump and his administration have created a hostile environment for federal government scientists, making it more difficult for these individuals to meet their job duties and responsibilities and engendering fear about discussing their work. (pp. 2-3) 2 The March for Science was held again on April 14, 2018, though with much smaller crowds and participation. 3 In a context in which the new head of NASA is a climate change denier and does not recognize the value of expanding human knowledge (Lusher, 2018), it also seems reasonable to wonder if we have the luxury of allowing such exclusions to matter. 4 The March for Science is not the only organization that is actively defending science. However, given its relatively high public profile, it serves as a useful example for this article. 5 I struggle with wondering if the need to defend science isn’t too important right now to question it. Encouraging researchers to think about who defines science and how might undercut the value of a strong defense. In some ways, the election constituted a crisis of confidence for me about the value of my work altogether. How can I defend alternative ways of knowing and being in the world when I still wonder if any of it matters…or at least if it matters enough in this moment when bodies are on the line, when the environment is on the line, when the policies of the administration explicitly reject using science in decision- making? See, for example, Biesecker and Borenstein (2018, April 24). http://www.marchforscience.com/ Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 4 to as the science wars, continued through the 1990s, and skirmishes erupt even now as researchers grounded in various paradigms jockey for position and funding. Specifically, I return to the emergence of the discourse of scientifically based research (SBR) in education and the debates surrounding it (2002-2013), which reignited the ongoing conflict between quantitative and qualitative, biological science and social science, and hard science and soft science because we have a lot to learn from it. SBR as truth still pervades educational research and policy, producing subjects, institutions, and practices that maintain and perpetuate that truth despite the fact that the debates have died down in recent years. The goal of revisiting SBR is not to discredit any one definition of science in informing the politics and methodology of educational research. Rather, the goal is to highlight that calls to rally around narrow definitions of science might make some researchers—specifically qualitative researchers whose work is grounded in critical and post theories—complicit in their own exclusion from what constitutes science and high-quality research. Much of the SBR debates resulted from the return of positivist social science that claims that an external reality exists waiting to be discovered; that human beings are capable of understanding and describing that reality accurately; and that multiple observers of the same phenomenon can agree on what they see. These debates have implications for how and why we, as educational researchers, would advocate for science and what that advocacy might do. I argue that we must broaden our advocacy efforts to include conversations about which science and science for whom. In other words, as educational researchers, we need to be careful not to establish too narrow a definition of science at the same time that we desperately need a definition to rally behind. We must have a science that doesn’t allow for alternative facts and for claiming truth because of individual beliefs or political expediency while still allowing for science to be flexible, responsive, and theoretically informed. Shifting Science Science has a rich history of questioning itself and refusing narrowing. Thomas Kuhn (1962/1996), for example, in his seminal text The Structure of Scientific Revolutions, argued that how we understand and define science is always shifting. He claimed that there is no singular truth to science but that the idea of science is determined in any given moment by the consensus of the scientific community. Kuhn used the term paradigm to describe such consensus. Through changes in thinking, technological developments, cutting-edge research, and other means, scientific beliefs change over time. When a rival paradigm becomes strong enough to challenge the current, established paradigm, a paradigm shift may occur. Kuhn believed that science is always temporal and contextual, described by the values and culture of the historical period. Further, he recognized that only a certain set of intellectual options, technological tools, terminology, and so forth are available to science at any given time, which always restrict what it is possible to think and know. Consequently, the goal of science according to Kuhn is not to find a universal truth but to create a theory or model that accounts for the greatest number of observations and phenomena in a given context. Moreover, inquiry that falls outside of norms and established procedures is absolutely necessary for science to continue to be relevant and useful. Kuhn (1962/1996) also critiqued the concept objectivity and argued that because science is conducted by humans, it is always subjective. The development of scientific measurement tools was an attempt to remove human error from observation. However, when both the observer and observed are human or a human phenomenon, subjectivity is always present. Importantly, much that relates to human beings simply cannot be measured objectively. Scientifically based research in a post -truth era 5 The notion of shifts in the norms of inquiry necessitating a shift in the understanding of what constitutes science is not relegated to the natural sciences. Foucault (1977/1980), for example, asserted that in all manner of knowledge production, “what has emerged ...is a sense of the increasing vulnerability to criticism of things, institutions, practices, discourses. A certain fragility has been discovered in the very bedrock of existence—even, and perhaps above all, in those aspects of it that are most familiar, most solid and most intimately related to our bodies and to our everyday behavior” (p. 80). Indeed, the expansion and pursuit of scientific understanding has also been the work of qualitative researchers who found that existing notions of science and inquiry were inadequate for meeting the needs of society. Lincoln and Guba (1985) believed that qualitative research constituted a new perspective on “the question of what there is that can be known and how one can go about knowing it” (p. 7). They explained that positivist inquiry had become insufficient for answering questions that continued to be raised in education and educational research and wrote, cracks have begun to appear in science’s magnificent edifice as new “facts” are uncovered with which the old paradigm cannot deal or explain. Normal science in the Kuhnian sense is becoming more and more difficult to sustain. Serious challenges are being mounted from the perspective of alternative paradigms that suggest new and different answers. (p. 7) Those alternative paradigms also enable new and different questions. Lather (2004) pointed to 30 years of “the social critique of science” (p. 17) in educational research as well as the constantly changing definitions and understandings of science as one field of discourse among others. In short, the question of science has been rigorously and intensely explored in various contexts for decades. The danger, then, is in an over reliance of the idea of science and its trappings—the generalization of positivist assumptions, values, and definitions across all inquiry—which results in what some have called scientism. Scientism Coined by Hayek (1952), scientism describes the belief that the methods of the natural sciences can be applied to the human sciences. Hayek (1952) explained that “[t]he methods which scientists or men fascinated by the natural sciences have so often tried to force upon the social sciences were not necessarily those which the scientists in fact followed in their own field, but rather those which they believed that they employed” (p. 22). In other words, the perception of what constitutes science and scientific practice rather than the actual practice of science in the natural sciences produces a “slavish imitation of the method and language of Science” (p. 24). According to Widdowson (2011), scientism “began as a label for the doctrine that truth is fixed, a priori and universal; that inductive science is the only means to its discovery and certainty is a realistic outcome” (para. 3). Further, Margolis (2003) argued that scientism had been unable to adequately respond to its critics and specifically maintained that scientism’s “refusal to admit the viability and reasonableness of a constructivist and historicist reading of science and practical life” signaled the end of “deceptions by which to prolong scientism’s hegemony” (p. 13). Haack (2003) agreed, stating that “[s]cientism is an exaggerated kind of deference towards science, an excessive readiness to accept as authoritative any claim made by the sciences, and to dismiss every kind of criticism of science or its practitioners as anti-scientific prejudice” (pp. 17-18). Sorrell (1991) noted that “the belief that science, especially natural science, is much the most valuable part of human learning— much the most valuable part because it is much the most authoritative, or serious, or beneficial” (p. 1) is major tenet of scientism. Because of that, social science research that does not emulate the Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 6 methods of the natural sciences is often not accepted as being scientific. Ruccio and Amariglio (2003) explained that “the growth of scientific knowledge” of the last three centuries “has been accompanied by an elaborate philosophical defense of a variety of exclusionary practices by which those deemed to be untrained in or unreceptive to such science are shunted aside or even denied opportunities to speak (since they are considered to be the voice of unreason)” (p. 42). Here, rationality is a feature only of scientific thought, and all other thought and the knowledge produced by such thoughts have no claims to reason. Further, scientism is “science’s belief in itself” (Habermas, 1968/1971, p. 4), which means that science is seen as the only way to produce knowledge; science is knowledge. Stenmark (2001) noted that “[w]hat is characteristic of Scientism is that it works with a narrow definition of science” (p. 5). Conversely, those critical of scientism adhere to the belief that there are multiple truths and multiple ways of knowing and producing knowledge, many of which become subjugated knowledges in the face of scientism. Haack (2003) explained that because there is no singular method of scientific inquiry, the so- called scientific method is a myth maintained by a scientistic culture. The myth of the scientific method enables an almost religious fervor in science that is both fascinating and perplexing. Scientism installs a singular notion science into all inquiry, regardless of domain, in an effort to make a science of everything, including education, as was notably shown in the debates around scientifically based research. Scientifically Based Research in Education Tracing SBR back to the 1890s, Beghetto (2003) and others (e.g., Franco, 2007; Smith, 2003; St. Pierre, 2006) argued that the term was the federal government’s tactic for transforming a “soft” social science (education) into a “hard” science, like physics, in effect, making education a science. Beghetto (2003) wrote that the government’s involvement in defining science for education could “be traced from the Cooperative Research Act of 1954 to the creation of the National Institute of Education in the early 1970s, which was later subsumed by the Office of Educational Research and Improvement (OERI)” (para. 2). However, scientifically based research as a concept emerged between 1999 and 2002 when the U.S. Congress passed three pieces of federal legislation that defined scientifically based research in education. The original description of SBR first appeared in REA, was repeated in NCLB, and then was reinforced and expanded by ESRA. I have included below the complete definition of SBR from each of the three federal laws as well as a brief discussion of the changes in the definition of SBR in the three laws. The purpose of REA was to provide early literacy intervention for all students in order to identify deficits, improve reading skills, and increase teacher expertise in the area of early literacy “through the use of scientifically based reading research” (Title VIII, 1999). In order to enforce the mandate of scientifically based research, it was necessary to define SBR.6 Offering up a definition 6 REA defined SBR as follows: The term “scientifically based reading research”— (A) means the application of rigorous, systematic, and objective procedures to obtain valid knowledge relevant to reading development, reading instruction, and reading difficulties; and (B) shall include research that— (i) employs systematic, empirical methods that draw on observation or experiment; (ii) involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn; Scientifically based research in a post -truth era 7 that seemed overly general and confused, scientifically based research began its emergence with REA. Subsequently, according to Baez and Boyles (2009), in the summer of 2000, draft legislation was introduced “by United States Representative Mike Castle (R-Del) that pertained to the reauthorization of The Office of Educational Research and Improvement” (p. 6). Called “The Castle Bill,” this legislation included a definition of scientifically based research that would determine the allocation of federal dollars for educational research. Eisenhart and Towne (2003) explained that “The Castle Bill” proposed standards for “scientifically based quantitative” and “scientifically based qualitative” research (p. 32). The reauthorization process for OERI continued for some time, and The Castle Bill was revised and redrafted multiple times, sparking debate about the nature of scientific educational research and the need to define it. Although the definitions proposed in The Castle Bill did not ultimately make it into law, they influenced the conversation about the federal influence on educational research. SBR itself did not have much impact until the passage of NCLB, which was the far overdue reauthorization of the Elementary and Secondary Education Act that had expired in 1997. NCLB was passed by The House of Representatives in May of 2001, passed by the Senate in June of 2001, and signed into law on January 8, 2002 by President George W. Bush. It lifted the definition of SBR from The Reading Excellence Act of 1999 (REA) and expanded it to include preferences for certain kinds of research methodologies.7 It has been reported that the term “scientifically based research” appears in NCLB 111 times (Neuman, 2002; Shavelson & Towne, 2002), representing unprecedented federal legislation of research methodology. Described by Beghetto (2003) as “the most sweeping reform of the Elementary and Secondary Education Act (ESEA) since it was enacted in 1965, [NCLB] redefines the federal role in K-12 education” (para 2). The definition of SBR in NCLB includes conceptions of reliability and validity that are not appropriate for qualitative research, and further, the definition privileges a peculiar “what works” mentality that gives (iii) relies on measurements or observational methods that provide valid data across evaluators and observers and across multiple measurements and observations; and (iv) has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review. (Title VIII, 1999) 7 NCLB defined SBR as follows: The term “scientifically based research”— (A) means research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs; and (B) includes research that— (i) employs systematic, empirical methods that draw on observation or experiment; (ii) involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn; (iii) relies on measurements or observational methods that provide reliable and valid data across evaluators and observers, across multiple measurements and observations, and across studies by the same or different investigators; (iv) is evaluated using experimental or quasi-experimental designs in which individuals, entities, programs, or activities are assigned to different conditions and with appropriate controls to evaluate the effects of the condition of interest, with a preference for random-assignment experiments, or other designs to the extent that those designs contain within-condition or across-condition controls; (v) ensures that experimental studies are presented in sufficient detail and clarity to allow for replication or, at a minimum, offer the opportunity to build systematically on their findings; and (vi) has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review. (Title IX, 2001) Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 8 preference to causal research that uses “random-assignment experiments” (Title IX, 2001) and replicability. As Walters, Lareau, and Ranis (2009) commented, “[i]n effect, NCLB gave the federal government an unusual degree of authority for setting standards for what constitutes good science and an unusual degree of control over the conduct of ‘science’’(p. 6). SBR functionally became federal policy, as a variety of federal (e.g., U.S. Institute of Education Sciences) and national (e.g., NRC) institutions and professional associations (e.g., American Educational Research Association) took it up and variously enforced it. In 2002 November, The Education Sciences Reform Act (ESRA) became law and the research arm of the Department of Education, OERI, became the new Institute of Education Sciences (IES), whose very title emphasized the importance of science in education. Baez and Boyles (2009) argued that ESRA was particularly important because it was the “first explicit attempt to establish a science for educational research,” whereas REA and NCLB dealt with issues of education at large. Consequently, ESRA and its definition of SBR disciplined the field of educational research, solidifying the discourse of SBR in documentary and juridical form. ESRA used the definition of SBR from NCLB, applied it specifically to educational research, and mandated preferred methodologies and research designs that would constitute high-quality science.8 ESRA placed a high 8 ESRA defined SBR as follows: The term ‘‘scientifically based research standards’’ means research standards that— (i) apply rigorous, systematic, and objective methodology to obtain reliable and valid knowledge relevant to education activities and programs; and (ii) present findings and make claims that are appropriate to and supported by the methods that have been employed. (B) The term includes, appropriate to the research being conducted— (i) employing systematic, empirical methods that draw on observation or experiment; (ii) involving data analyses that are adequate to support the general findings; (iii) relying on measurements or observational methods that provide reliable data; (iv) making claims of causal relationships only in random assignment experiments or other designs (to the extent such designs substantially eliminate plausible competing explanations for the obtained results); (v) ensuring that studies and methods are presented in sufficient detail and clarity to allow for replication or, at a minimum, to offer the opportunity to build systematically on the findings of the research; (vi) obtaining acceptance by a peer-reviewed journal or approval by a panel of independent experts through a comparably rigorous, objective, and scientific review; and (vii) using research designs and methods appropriate to the research question posed. (19) SCIENTIFICALLY VALID EDUCATION EVALUATION.—The term ‘‘scientifically valid education evaluation’’ means an evaluation that— (A) adheres to the highest possible standards of quality with respect to research design and statistical analysis; (B) provides an adequate description of the programs evaluated and, to the extent possible, examines the relationship between program implementation and program impacts; (C) provides an analysis of the results achieved by the program with respect to its projected effects; (D) employs experimental designs using random assignment, when feasible, and other research methodologies that allow for the strongest possible causal inferences when random assignment is not feasible; and (E) may study program implementation through a combination of scientifically valid and reliable methods. (20) SCIENTIFICALLY VALID RESEARCH.—The term ‘‘scientifically valid research’’ includes applied research, basic research, and field-initiated research in which the rationale, design, and interpretation are soundly developed in accordance with scientifically based research standards. (Title I, 2002) Scientifically based research in a post -truth era 9 priority on the randomized control trial (RCT),9 explaining that it was the only adequate method for producing causal inferences and thereby implying that causal research is superior in solving educational problems. The discussion of SBR within those three federal laws began “a whole new ‘régime’ in discourse and forms of knowledge” (Foucault, 1980b, p. 112). The Emergence of a Definition What was immediately obvious in looking across the three pieces of legislation is that the definition of SBR was expanded in each succeeding law. In fact, in REA, the definition was 111 words long; in NCLB, it was just over 200 words; and in ESRA, the definition topped 330 words. However, what was not explicitly stated is how the definitions in the laws were developed. Eisenhart and Towne (2003) researched the genesis of the definition of SBR and documented their findings. They reported that Robert Sweet, then professional staff member for the majority members of the House Education and Workforce Committee, was tasked with creating a definition of SBR to be included in REA. According to Eisenhart and Towne, To craft this definition, which would be the starting point for all of the now numerous definitions that appear in major federal education laws, Sweet visited the websites of several DC-based research institutions (including the NRC, although this took place well before the committee was convened to produce SRE [NRC, 2002]), consulted with numerous university-based researchers (primarily with backgrounds in cognitive psychology), and shared drafts with these researchers (he estimates approximately 20-25 of them). The language that emerged from the several-months- long process was inserted into REA (1999), and passed without fanfare. (2003, p. 32) Of particular note is that the researchers Sweet consulted by and large had backgrounds in cognitive psychology, which uses positivism, so it would follow that their definition of science would be positivist. Research grounded in other epistemologies would have different methodologies, different standards, and therefore, different definitions of science. Importantly, each definition of SBR also included a reference to observational methods, though the intended meaning the term in each instance is unclear. In the SBR definitions, observational methods are set in opposition to experimental methods (i.e., each law states “empirical methods that draw on observation or experiment” Title IX, 2001), seeming to signify that observation stands for any methods that are not experimental. This may be a gesture towards the inclusion of qualitative methodology,10 but those who conduct qualitative research likely would have used different language. The definitions of SBR in the three pieces of legislation completely ignore interviewing, which is arguably as important, if not more important, than observation in many qualitative research studies. Thus, the way the word “observation” is used in the laws does not make sense and indicates 9 RCT is an experimental design in which a sample is randomly selected and assigned to both control groups and experimental groups. An intervention is applied to the experimental group, and data is collected based on the effect of the intervention. 10 It is also possible that “observational methods” are a reference to standardized observation protocols that commonly exist in experimental research and single-subject design research. However, the use of observational methods, and specifically observation protocols, is commonly addressed in qualitative research textbooks (e.g., Creswell, 1998; Fraenkel & Wallen, 2003; Patton, 2002). Because this term can be taken up in both communities, its use in federal definitions of SBR is both confusing and problematic. Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 10 a lack of knowledge about research methodologies that are not experimental. Further, the definition of SBR in REA stipulated that data must remain valid “across...observers and across...observations.” In other words, high quality studies must be replicable, which is neither possible nor desirable in qualitative methodologies as each observer necessarily notices and describes different things, all of which are “correct” and “valid” though different. Moreover, multiple observers do not and cannot agree on what they see because as Patton (2002) explained, “what people ‘see’ is highly dependent on their interests, biases, and backgrounds. Our culture shapes what we see, our early childhood socialization forms how we look at the world, and our value systems tell us how to interpret what passes before our eyes” (p. 260). Qualitative studies, then, cannot be replicated because each is specific to a particular site, to particular participants, and to a particular researcher. In other words, qualitative research deals with “particular problems and must deal with local conditions that limit generalizations and theory building” (Berliner, 2002, p.18). Observational data in qualitative research cannot be replicated across observers or observations nor does qualitative inquiry aim for such replication. The language used to define SBR demonstrated little understanding of qualitative research practices and purposes. What is clear is that the insistence on “rigorous scientific standards” (U.S. House, 2002, p. 2) led to inadequate definitions of science that had a real material effect on who could receive federal funding for educational research. Instead of responding to the confusion created by the definition of SBR in REA, NCLB and ESRA continued and elaborated that confusion. It is also important to understand that the experimental method privileged by SBR is produced in and through the discourse and theories of positivism, not outside theory as is sometimes implied. Because the federal government both supported and promoted a positivist science in educational research to the exclusion of other sciences described in other theoretical frameworks, the range of possible sciences recognized as valuable by the federal government was greatly diminished. The government promoted the standardization of methodology as a move towards some “common good,” with particular meanings of “science,” “standards,” “quality,” and “evidence” that are only thinkable within positivism. As a result, SBR had real, material effects on educational practice, policy, and research and on people—students, classroom teachers, educational researchers, policy makers—such as preference for research funding, proliferation of standardized testing, and so forth. Unsurprisingly, some of the loudest resistance [against SBR] at that time came from qualitative researchers whose work was deemed by the SBR contingent as interesting, but not scientific and those whose work was grounded in postmodern theories, labeled as “an extreme epistemological perspective” (p. 25, fn). SBR effectively mandated positivist science, privileging experimental methodologies and excluding the diverse research methodologies and theories that qualitative researchers use. That exclusion made it difficult, if not impossible, for qualitative researchers to address the complexities of social life in schools and society that we have long since agreed require such diverse perspectives. Indeed, as Denzin and Lincoln (2011) argue, qualitative researchers are engaged in a “perpetual resistance against attempts to impose a single, umbrella-like paradigm” (p. xiii) as they seek to “make sense of the terrifying conditions that define daily life” (p. xii). Nonetheless, SBR became so pervasive at all levels of education that it began to function as truth—normal and natural to the extent that it became invisible and beyond question, making capitulation or critique the only available options, both of which reinforce its truth. Assumptions about the poor quality of educational research were the rationale for the imposition of federal definitions of science and methodology as described above. Those same assumptions were cited as the impetus for the National Research Council’s reports about scientific Scientifically based research in a post -truth era 11 research in education, and furthermore, those assumptions drove the American Educational Research Association (AERA) to write and publish standards for reporting research in its journals, draft its own definition of scientifically based research, and convene annual meetings around the issue of quality in educational research. AERA’s efforts forever impacted what constitutes social science research in terms of publication in their journals. Although there are many other publication venues, and indeed, critical qualitative and post-qualitative research has been proliferating in some circles over the last decade, it is meaningful that institutions imbued with power continue to operate on a model established by SBR. For example, even the International Congress of Qualitative Inquiry, which is certainly a space that does not privilege randomized control trials, took up the notion of quality in qualitative research as a theme for the conference during the SBR era. So although SBR does not prevent other kind of inquiry from taking place in lots of areas, even some with institutional support, I argue it still shapes that inquiry, and more specifically, how and to what extent that inquiry is capable of shaping policy and practice in education. Definition Goes Underground Somewhat ironically, Russ Whitehurst, former head of the Institute of Education Sciences, which mandated a hierarchy of methodological quality that favored randomized control trials as the gold standard, now thinks that the manner in which SBR was installed was inappropriate. He said, "[o]ne error of NCLB was demanding the use of [scientifically based research] when it didn't exist, thus debasing the currency. Another was getting Congress into the business of defining SBR in education—research methodologists they are not" (Sparks, 2015, para 10). Whitehurst continued, "[p]erhaps these were justified at the time because the state of education research was really awful. But neither makes sense today" (Sparks, 2015, para 10). NCLB notably mentioned SBR more than 111 times, but its more recent reauthorization, the Every Child Succeeds Act (ESSA), doesn’t mention SBR at all. Indeed, it simply calls for educational interventions to be evidenced-based, which seems at first blush like a significant backpedal from previous iterations of science in education. However, the shift in language does little more than to push the conversation underground while maintaining the privileging of a narrow definition of science. Rather than the definition and enforcement of this privileging occurring at the federal level, the decisions are now relegated to the state level, as Whitehurst notes above.11 Still, the assertion that the state of educational research was “awful” and has now been fixed because of SBR is a testament to its enduring truth value. SBR became instrumental in the practices of government institutions,12 requiring education researchers and others who wish to conduct so-called scientific research to discipline themselves according to the rationality of the discourse of SBR. The practice of SBR has persisted and expanded and at present, SBR is a discourse that claims to be science and disqualifies other ways of knowing. In short, SBR defined the truth of high quality research in education. Taking a deeper dive into how “evidence based” is defined in ESSA highlights that enduring truth value. Sparks (2015) explains that the definition is “buried in the bowels of the bill” and describes tiered levels of evidence, modeled on the Investing in Innovation grant program: 11 There is no small amount of irony in Whitehurst now making these claims about the implementation of SBR after NCLB, as he was a primary architect of how much of that implementation took place as the then- head of the Institute of Education Sciences. Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 12  Strong evidence: includes at least one well-designed and -implemented experimental study, meaning a randomized controlled trial.  Moderate evidence: includes at least one well-designed and -implemented quasi- experimental study, like a regression discontinuity analysis.  Promising evidence: includes at least one well-designed and implemented correlational study that controls for selection bias (para. 4-5). That is, even now, though SBR is no longer in common parlance, its vestiges remain. Same song, different verse, as it were, though it remains unclear what impacts this legislation will have. Concurrently, there is a response in the educational research community to the post-truth era and the need to defend science that raises additional questions. For example, the theme of the 2019 Annual Meeting of AERA is “Leveraging Education Research in a “Post-Truth” Era: Multimodal Narratives to Democratize Evidence.” The call for submissions asserts that “we see daily examples of policy issues—from climate change to immigration—in which appeals by powerful leaders to personal beliefs and emotions hold more sway than objective facts and evidence” (AERA, 2018, p. 1) and calls on its members to propose papers and sessions that defend the work that we do as educational researchers, which it contends can and should be done by collaborating across epistemological and methodological divides. It reads: These cross-boundary collaborations do not imply that we should ignore the methodological, disciplinary, and epistemological disagreements in our field. They signify our different understandings of whose truth matters, what warrants reliable evidence, and whether or not objectivity exists. These differences are significant, and, hopefully, they force each of us to think more deeply about our research. But at this moment in history, we must also work together across some of these divides to learn from each other and bring comprehensive, systematic evidence to bear on critical issues in educational policy and practice. (AERA, 2018, p. 1) Speaking across methodological divides was a common theme in the debates surrounding SBR. Maxwell (2004) believed our responsibility is to try to speak across this difference, though that was often impossible in the SBR debates because many scholars concluded that some divides exist across which we cannot speak and that those in privileged methodological and epistemological positions can easily erase other perspectives in such efforts. Fear of erasing difference persisted, not only because erasure is a violent act, but also because a critique that erases or dismisses difference and dissent can create an epistemological holocaust where each side is determined to eliminate the other. Rather than eliminate other descriptions, Foucault (1980b) suggested situating “historically how the effects of truth are produced within discourses [like science] which themselves are neither true nor false” (p. 118). Privileging the randomized control trials, for example, excludes other methodologies, and one might ask whether such exclusion should be the goal of science. In juxtaposing words to create terms like “evidence-based narratives” or positing that there is a “marginalization of empirical research,” the AERA call for proposals recreates the kind of epistemological and methodological confusion that occurred in the definitions of SBR in federal legislation. The response to the post- truth era and the defense of science must surely take a more theoretically informed approach rather than becoming a reinforcement of problematic rejections and complicity that only further empower damaging discourses. Scientifically based research in a post -truth era 13 Implications What types of knowledge do you want to disqualify in the very instant of your demand: ‘Is it a science’? Which speaking, discoursing subjects—which subjects of experience and knowledge—do you then want to ‘diminish’ when you say: ‘I who conduct this discourse am conducting a scientific discourse, and I am a scientist’? (Foucault, 1980, p. 85). The point here is not to establish any definition, iteration, or practice of science as evil, wrong, or in error. Rather, the point is to demonstrate how one definition of science can be taken up as the truth, limiting ways of knowing that can contribute to the theory, research, policy, and practice of education or any other field. Moreover, as Foucault noted, what is often found at the beginning is not intentional, rational deliberation but chance, accidents, petty politics, disagreements among factions, and so on, such as was the case with the emergence of SBR. Thus, today’s truthful knowledge is not based on a firm foundation but on contingency. As Kuhn established, the truth of things changes over time as different power/knowledge formations emerge. Definitions of science often appear to be, but are not, totalities. In any epistemology, descriptions of concepts such as science become so ingrained in common parlance and practice that they are no longer questioned. They become impervious to critique because in their journey to becoming true, they became invisible. As Foucault (1980a) wrote, “the attempt to think in terms of a totality has in fact proved a hindrance to research” (p. 81). The goal, then, is not attempt to produce another truth of science—some would claim that the concept has been used so differently in so many different discourses that it has effectively been stripped of all meaning. Instead, the goal is to understand how one definition of science can come to be seen as the truth. According to Cullenberg, Amariglio, and Ruccio (2001), the idea of “agreement” (voluntary, forced, and every combination in between) in science is what actually needs to be understood and investigated. Further, those theories that often succeed at any given moment in time in shaping a field of thought are either bound to more general social institutions and patterns of status, wealth, and power, or are able to hegemonize the field by “normalizing” the conditions under which that theory arises, and maybe both. As Feyerband (2010) explained: there are no singularly exceptional methods that are productive of science, and even that actual scientific “progress” is a result of scientists’ refusal to codify and obediently follow any philosophically prescribed road toward truth. (pp. 28-29) Indeed, such critiques call into question the sacred and singular nature of science in important ways that cannot be forgotten, even when the political moment calls for a fierce defense of science. Consequently, in our efforts to advocate for science in the current moment, part of that advocacy is to “struggle against the coercion of a theoretical, unitary, formal and scientific discourse” (Foucault, 1980a, p. 85). Or in Manning’s (2016) words, the goal is to break the cycle of negative critique that is “deeply complicit with the way things are” and that can only be “for or against” (p. 203). Since the 2016 election, the March for Science and even more broadly the term science itself, runs the risk of becoming a similar kind of regime of truth to SBR. You are either for or against the monolithic notion of science, belying its contingent, multiple, and contested history and creating a tension for those who would like to be champions for science. The March for Science is an important and necessary reaction formation to the post-truth era. However, it must guard against falling victim to the same unitary traps that produced SBR. For example, there is use-value in the March for Science becoming a non-profit organization. But by doing so, it must establish a mission and goals, it must define itself. One of the results of that institutionalization of this resistance and Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 14 counter-protest has been to use language like “science” without operationalizing it, to champion “evidence-based” practices, and to advocate science for the “common good.” These words, though seemingly unproblematic, are not value-free, and they are not without a complex and wrought history. As a result of becoming an organization, then, there is now an institution, there are subjects, there are documents, and so forth that help to, because of bureaucratic and practical requirements, solidify and codify what is meant when the March for Science is discussed. That is not to say that the March for Science is not inclusive, nor is it an indication that the March for Science has in mind the same kind of methodological hierarchy that proponents of SBR wrote into policy and law. Certainly, there are some proponents of the March for Science who have those perspectives, but they are not the only ones to exist. However, in order to be able to continue to function, to respond with increasing urgency to the anti-intellectual and anti-science policies and actions of the Trump administration as they produce the United States in the post-truth era, this pro-science movement cannot afford to fall into the same traps. It cannot afford to risk losing any allies. My argument is not that this is already happening (though some do feel excluded by this organization or wary of marching for a science that does not include them) but rather that, in our urgency to defend the scientific work that protects our environment, we may fail to practice a theoretically informed activism that is inclusive and capacious. Advocacy for a rigorous, open, shifting science instead of a search for a unitary banner to stand behind might free educational researchers to think research and advocacy differently this time. Non-Innocent Science Because SBR was repeated until it had become invisible, the center of that structure had become fixed, both reinforcing it as a structure and limiting the play within that structure. We risk limiting the play of science now if we do not heed the lessons of SBR and recognize how fraught this all is. As Foucault, (1976/1978) wrote, the “general design or institutional crystallization is embodied in the state apparatus, in the formulation of the law, in the various [political] hegemonies” (p. 93). That means that as structures emerge and are maintained and perpetuated to the point that they become institutionalized, it is necessary to look more closely at the structure and recognize it as a description rather than a truth. The acknowledgement and deconstruction of SBR (or any definition of science) as a structure, for example, freed educational researchers to think research differently. Foucault (1988) saw it as his responsibility “to show people that they are much freer than they feel, that people accept as truth, as evidence, some themes which have been built up at a certain moment during history, and that this so-called evidence can be criticized and destroyed” (p. 10). Consequently, it is against any solid definition or discourse of science that educational researchers must think because “[t]hought is freedom in relation to what one does” (Foucault, 1984a, p. 388). The kind of freedom offered by a structure such as SBR, the seemingly value-free science put forth by the March for Science Organization, or the confused call for proposals from AERA is not liberty because there is only a certain range of options for thought and action within a discourse. On the contrary, Foucault (1984b) claimed, “liberty is a practice” and it “must be exercised ... it can never be inherent in the structure of things to guarantee freedom. The guarantee of freedom is freedom” (p. 245). If no singular definition of science (or its defense) is real or true, then a practice of freedom for educational researchers would be not to respond at all, either through resistance or capitulation. After recognizing any definition of science as one of many possible descriptions, the next step for social scientists is to enact social science as they describe it—not necessarily to take up the dominant, normalized social science as if it’s real and true and good. In fact, what has become Scientifically based research in a post -truth era 15 conventional should always be critiqued. As Butler (1993) explained, the “failure of certain kinds of ideals is itself mobilizing, vitalizing, expansive, inaugurative of the new, productive of possibilities” (p. 7). That is not to say that some new form of social science can guarantee high quality science. Any claims to science produce another structure, another grid of intelligibility through which to make sense of the world, excluding some practices and processes, ontologies, epistemologies and methodologies while privileging others. That structure, too, will fail if it has no play, if it is not malleable, if we are so dogmatic about its enactment that “we’ve forgotten we made it up” (St. Pierre, 2011, p. 613). Rather, I argue that we might view the unraveling of the truth claims of any scientific discourse as an invitation to produce theoretically-informed activism and non-innocent science. One need not lay claim to the label of science to the exclusion of all other ways of knowing. Instead, I advocate for non-innocent science, a science that claims its allegiances--epistemological, ontological, methodological, political, and otherwise—rather than claiming to be value-free. I wonder what might happen if we began to ask, and take seriously the question, what kind of science are we producing when we do research? Research findings about language education or dropout rates or teacher retention are useful, but research becomings about what kind of science was produced in the doing of research is the “work of thought” for which Foucault advocated. Science that does not claim its allegiances represents itself as innocent. That is to say that science that claims to be atheoretical, value-free, and objective assumes itself to be harmless, blameless, and above suspicion because it is self-evident. Indeed, Howe (2003) wrote that “epistemological bias is more deep-seated, subtle, and unconscious than the kind of bias that characterizes, say, partisan political wrangling” (p. 100). Likewise, The March for Science, particularly in its online presence, appears to support the common good without explicitly recognizing that there are many sciences and many ways of speaking science. And AERA erases the marginalized groups and methodologies it claims to champion by taking up the problematic language of the science that produced that marginalization. But science that does not claim its allegiances is dangerous, just as everything is dangerous. Epistemology is never self-evident. And, as Foucault (1983) said, “if everything is dangerous, then we always have something to do” (pp. 231-232). The non-innocent science I suggest we enact is an ontological move to be with research differently. It is not just in multiplicity that high quality science is found—more science and more kinds of science are good, but they are not the point—rather, it is in the enactment of social science that looks different from itself, claims its allegiances, and actively seeks out what we cannot yet think. This is an argument for the ongoing rupture of science until it becomes unrecognizable. Every enactment of research is an opportunity to rethink science. I am not interested in what science is because defining science in one way or another simply serves to produce it as a monolithic concept. The non-innocent science I describe above, conversely, proposes science as always already imbricated with philosophy, which means that science always looks different from itself because, as Foucault noted, power and knowledge work together to produce the truth of something. This is not to say that the question of how science is defined and who defines it is not important. But as Butler (1995) reminds me, “[t]he key question of whether or not a position is right, coherent, or interesting is, in this case, less informative than why it is we come to occupy and defend the territory that we do, what it promises us, from what it promises to protect us” (pp. 127-128). Freedom from monolithic science gives us permission to do research in unconventional ways so that we can “produce different knowledge and produce knowledge differently” (St. Pierre, 1997, p. 175) in addition to being differently in the world and enacting different forms of activism. Rather than the traditional move to produce findings relevant to a specific field or content area, I wonder what might happen if we began to ask, and take seriously the question, what kind of Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 16 science are we producing when we do research? Research findings about language education or dropout rates or teacher retention are useful, but research becomings about what kind of science was produced in the doing of research is the “work of thought” to which Foucault (1984/1985) referred: As to those for whom to work hard, to begin and begin again, to attempt and be mistaken, to go back and rework everything from top to bottom, and still find reason to hesitate from one step to the next—as to those, in short, for whom to work in the midst of uncertainty and apprehension is tantamount to failure, all I can say is that clearly we are not from the same planet. (p. 7) If science comes from somewhere—situated in time and space and place and values and ontologies and epistemologies and…and…and—then our job as advocates becomes to support inclusive rather than exclusive science. If we do not want to limit what we can think and do and become as educational researchers, then we must be open to any and all methodologies, including those that we haven’t been able to imagine yet—the paradigm to come. References American Educational Research Association. (2018). AERA 2019 Annual Meeting Call for Submissions: Leveraging Education Research in a “Post-Truth” Era: Multimodal Narratives to Democratize Evidence. Washington, DC: AERA. Baez, B., & Boyles, D. (2009). The politics of inquiry: Education research and the “culture of science.” Albany, NY: State University of New York Press. Beghetto, R. (2003). Scientifically based research (ERIC Digest 167). Eugene, OR: ERIC Clearinghouse on Educational Management. (ERIC Document Reproduction Service, No. ED474304). Berliner, D. (2002). Educational research: The hardest science of all. Educational Researcher 31(8), 18- 20. https://doi.org/10.3102/0013189X031008018 Biesecker, M., & Borenstein, S. (2018, April 24). EPA chief signs proposal limiting science used in decisions. AP News. Retrieved from https://www.apnews.com/f25e1975b1ea4417a5995cc2b8d87a8e/EPA-chief-signs- proposal-limiting-science-used-in-decisions. Butler, J. (1993). Poststructuralism and postmarxism. Diacritics, 23(4), 3-11. https://doi.org/10.2307/465304 Butler, J. (1995). For a careful reading. In S. Benhabib, J. Butler, D. Cornell & N. Fraser (Eds.). Feminist contentions: A philosophical exchange (pp. 127–143). New York: Routledge. Butler, J. (2004). Precarious Life: The powers of mourning and violence. Brooklyn, NY: Verso. Congressional Record-House. (2002). Congressional Record: Proceedings and Debates of the 107th Congress, Second Session, V. 148, Pt. 15, October 10, 2002 to November 8, 2002. Washington, DC: United States Government Printing Office. Cullenberg, S., Amariglio, J., & Ruccio, D. F. (2001). Introduction. In S. Cullenberg, J. Amariglio, & D. F. Ruccio (Eds.), Postmodernism, economics and knowledge (pp. 3-57). London: Routledge. https://doi.org/10.4324/9780203410707.pt1 Education Sciences Reform Act of 2002. Pub. Law No. 107-279 (2002). Eisenhart, M., & Towne, L. (2003). Contestation and change in national policy on “scientifically based” education research. Educational Researcher, 32(7), 31-38. https://doi.org/10.4324/9780203410707.pt1 Feyerabend, P. (2010). Against method. New York: Verso Books. (Original work published 1975). Scientifically based research in a post -truth era 17 Foucault, M. (1978). The history of sexuality: An introduction, Volume I. (R. Hurley, Trans.). New York: Vintage Books. (Original work published 1976). Foucault, M. (1980a). Two lectures. In Gordon, C. (Ed.). Power/knowledge: Selected interviews and other writings, 1972-1977 (pp. 78-108). New York: Pantheon. Foucault, M. (1980b). Truth and power. In Gordon, C. (Ed.). Power/knowledge: Selected interviews and other writings, 1972-1977 (pp. 109-133). New York: Pantheon. Foucault, M. (1983). On the genealogy of ethics: An overview of work in progress. In H.L. Dreyfus & P. Rabinow (Eds.), Michel Foucault: Beyond structuralism and hermeneutics (2nd ed.) (pp. 229- 252). Chicago: The University of Chicago Press. Foucault, M. (1984a). Polemics, politics, and problematizations: An interview. In Rabinow, P. (Ed.). The Foucault reader (pp. 381-390). New York: Pantheon Books. Foucault, M. (1984b). Space, knowledge, and power. In Rabinow, P. (Ed.). The Foucault Reader. New York: Pantheon Books. Foucault, M. (1985). The history of sexuality, Vol. 2: The use of pleasure. (R. Hurley, Trans.). New York: Vintage Books. (Original work published 1984). Foucault, M. (1988). The concern for truth. In L. Kritzman (Ed.). Politics, philosophy, culture: Interviews and other writings 1977-1984 (pp. 255-267). New York: Routledge. Franco, S. (2007). Reauthorization of NCLB: Time to reconsider the scientifically based research requirement. Third Education Group Review/Essays, 3(6,) 1-12. Gage, N. L. (1989). The paradigm wars and their aftermath: A “historical” sketch of research on teaching since 1989. Educational Researcher, 18(7), 4-10. Haack, S. (2003). Defending science within reason: Between scientism and cynicism. Amherst, NY: Prometheus Books. Habermas, J. (1971). Knowledge and human interests. Boston: Beacon Press. (Original work published 1968) Hacking, I. (1983). Representing and intervening: Introductory topics in the philosophy of natural science. Cambridge: Cambridge University Press. https://doi.org/10.4324/9780203410707.pt1 Hayek, F. A. (1952). The counter-revolution of science: Studies on the abuse of reason. Glencoe, IL: The Free Press. Howe, K. (2003). Closing methodological divides: Toward democratic educational research. Dordrecht, the Netherlands: Kluwer Academic Press. Kuhn, T. S. (1996). The structure of scientific revolutions (3rd ed.). Chicago, IL: University of Chicago Press. (Original work published 1962). https://doi.org/10.7208/chicago/9780226458106.001.0001 Lather, P. A. (1996). Troubling clarity: The politics of accessible language. Harvard Educational Review, 66(3), 525-545. https://doi.org/10.17763/haer.66.3.6qxv1p081102560g Lather, P. (2000). Drawing the line at angels: Working the ruins of feminist ethnography. In E. E St. Pierre & W. Pillow (Eds.). Working the ruins: Feminist poststructural theory and methods in education (pp. 284-312). New York: Routledge. Lather, P. (2004). This IS your father's paradigm: Government intrusion and the case of qualitative research in education. Qualitative Inquiry, 10(1), 15-34. https://doi.org/10.1177/1077800403256154 Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Beverly Hill, CA: Sage. Lusher, A. (2018, April 20). Trump's new Nasa chief Jim Bridenstine a 'climate change denier' who could make 'terrifying' decisions, US senators warn. The Independent. Retrieved from https://www.independent.co.uk/news/world/americas/us-politics/james-bridenstine-nasa- climate-change-denier-donald-trump-ignoring-science-terrifying-launch-a8313966.html. https://www.independent.co.uk/news/world/americas/us-politics/james-bridenstine-nasa-climate-change-denier-donald-trump-ignoring-science-terrifying-launch-a8313966.html https://www.independent.co.uk/news/world/americas/us-politics/james-bridenstine-nasa-climate-change-denier-donald-trump-ignoring-science-terrifying-launch-a8313966.html Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 18 Manning, E. (2016). The minor gesture. Durham, NC: Duke University Press. https://doi.org/10.1215/9780822374411 March for Science. (n.d.) About. www.marchforscience.com Margolis, J. (2003). The unraveling of scientism: American philosophy at the end of the twentieth-century. Ithaca, NY: Cornell University Press. Muller, J. Z. (January 29, 2009). Our epistemological depression. The American: The Journal of the American Enterprise Institute, 1-6. National Research Council. (2002). Scientific research in education (R. J. Shavelson & L. Towne, Eds.; Committee on Scientific Principles for Education Research). Washington, DC: National Academies Press. Neuman, S. (2002). Welcome and introduction. Paper presented at The Use of Scientifically Based Research in Education Working Group Conference, Washington D.C. No Child Left Behind Act of 2001. Pub. Law No. 107-110 (2001). Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage. Pickering, M. (1993). August Comte, An intellectual biography, Volume I. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511527975 Reading Excellence Act of 1999. Pub. Law No. 105-277 (1999). Ruccio, D. F., & Amariglio, J. (2003). Postmodern moments in modern economics. Princeton, NJ: Princeton University Press. Ryder, M. (2005). Scientism. Encyclopedia of Science Technology and Ethics (3rd ed.). Detroit: MacMillan Reference Books. Retrieved from http://carbon.ucdenver.edu/~mryder/scientism_este.html. Shavelson, R. J., & Towne, L. (Eds.). (2002). Scientific research in education. Washington, DC: National Research Council, National Academy Press. Silberstein, L. J. (2002). Postzionism: A critique of Zionist discourse (Part 2): The controversies surrounding the conflict of national memory and identity. The Palestine-Israel Journal of Politics, Economics, and Culture, 9(3). Retrieved from http://www.pij.org/details.php?id=135 Smith, A. (2003). Scientifically based research and evidence-based education: A federal policy context. Research & Practice for Persons with Severe Disabilities, 28(3), 126-132. https://doi.org/10.2511/rpsd.28.3.126 Sparks, S. (2015, December 4). What role will research play in ESSA? Education Week. Retrieved from http://blogs.edweek.org/edweek/inside-school- research/2015/12/essa_waves_at_tiered_evidence.html?print=1 Stenmark, M. (2001). Scientism; Science, ethics, and religion. Aldershot, England: Ashgate. St. Pierre, E. A. (1997). Methodology in the fold and the irruption of transgressive data. International Journal of Qualitative Studies in Education, 10(2), pp. 175-189. https://doi.org/10.1080/095183997237278 St. Pierre, E. A. (2006). Scientifically based research in education: Epistemology and ethics. Adult Education Quarterly, 56(4), 239-266. https://doi.org/10.1177/0741713606289025 St. Pierre, E. A. (2011). Post qualitative research: The critique and the coming after. In N. Denzin & Y. Lincoln (Eds.), Sage handbook of qualitative research (4th ed.; pp. 611-626). Thousand Oaks, CA: Sage. St. Pierre, E .A. (2012). Another postmodern report on knowledge: Positivism and its others. International Journal of Leadership in Education, 15(4), 483-503. https://doi.org/10.1080/13603124.2012.696710 http://www.marchforscience.com/ http://blogs.edweek.org/edweek/inside-school-research/2015/12/essa_waves_at_tiered_evidence.html?print=1 http://blogs.edweek.org/edweek/inside-school-research/2015/12/essa_waves_at_tiered_evidence.html?print=1 Scientifically based research in a post -truth era 19 Union of Concerned Scientists. (2017). Sidelining science since day one: How the Trump Administration has harmed public health and safety in its first six months. Washington, DC: The Center for Science and Democracy. U.S. House Subcommittee on Education Reform. (2002). Hearings on the reauthorization of the Office of Educational Research and Improvement. 107th Cong., 2nd sess. Washington, DC: U.S. Government Printing Office. Walters, P., Laureau, A., & Ranis, S. (2009). Education research on trial: The search for rigor and the promotion of randomized studies. New York: Routledge. https://doi.org/10.4324/9780203928684 Widdowson, W. (December 4, 2011). What is “scientism?” [Web log post]. Retrieved from http://whyevolutionistrue.wordpress.com/2011/12/04/what-is-scientism-a-guest-post/ About the Author Jessica Van Cleave Mars Hill University jvancleave@mhu.edu https://orcid.org/0000-0002-2531-3965 Jessica Van Cleave is an associate professor of education at Mars Hill University. Her research explores the intersection of poststructural theories, (post)qualitative inquiry, and educational policy. About the Guest Editors Jennifer R. Wolgemuth University of South Florida jrwolgemuth@usf.edu Jennifer R. Wolgemuth is an Associate Professor in Measurement and Research at the University of South Florida. Her research focuses on the socio-politics of social science research. Her work illuminates and disrupts categorical accounts of the contexts, ethics, and outcomes of social science research, including their personal and social impacts on researchers, participants, and those who shepherd research evidence into policy and practice. Mirka Koro-Ljungberg Arizona State University Mirka.Koro-Ljungberg@asu.edu Mirka Koro-Ljungberg (Ph.D., University of Helsinki) is a Professor of qualitative research at the Arizona State University. Her scholarship operates in the intersection of methodology, philosophy, and socio-cultural critique and her work aims to contribute to methodological knowledge, experimentation, and theoretical development across various traditions associated with qualitative research. She has published in various qualitative and educational journals and she is the author of Reconceptualizing qualitative research: Methodologies without methodology (2016) published by SAGE and co- editor of Disrupting data in qualitative inquiry: Entanglements with the Post-Critical and Post-Anthropocentric (2017) by Peter Lang. mailto:jvancleave@mhu.edu https://orcid.org/0000-0002-2531-3965 mailto:jrwolgemuth@usf.edu mailto:Mirka.Koro-Ljungberg@asu.edu Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 20 Travis M. Marn Southern Connecticut State University Marnt1@southernct.edu Travis M. Marn (Ph.D., University of South Florida) is a Professor in the Curriculum & Learning Department at Southern Connecticut State University where he teaches child development and educational psychology. His scholarship focuses on philosophically informed empirical accounts of performative identities, employing new materialism and posthumanism in psychological research, and contributing to the development of qualita tive and post-qualitative methods and methodologies. Anthony J. Onwuegbuzie Sam Houston State University tonyonwuegbuzie@aol.com Anthony J. Onwuegbuzie is Professor in the Department of Educational Leadership at Sam Houston State University, where he teaches doctoral-level courses in qualitative research, quantitative research, and mixed research. Further, he is a Distinguished Visiting Professor at the University of Johannesburg and an Honorary Professor at the University of South Africa. His research areas primarily involve social and behavioral science topics, including disadvantaged and under-served populations such as minorities, children living in war zones, students with special needs, and juvenile delinquents. Also, he has conducted numerous research studies on factors that predict educational achievement at the primary, secondary, and tertiary levels. Additionally, he writes extensively on qualitative, quantitative, and mixed methodological topics applicable to multiple disciplines within the field of the social and behavioral sciences. Shaun M. Dougherty Vanderbilt University shaun.dougherty@vanderbilt.edu Shaun M. Dougherty is an Associate Professor of Public Policy and Education, Peabody College of Education & Human Development, Vanderbilt University. His work focuses on applied quantitative analysis of education policies and programs, equity, and career and technical education. mailto:Marnt1@southernct.edu mailto:tonyonwuegbuzie@aol.com mailto:shaun.dougherty@vanderbilt.edu Scientifically based research in a post -truth era 21 SPECIAL ISSUE Rethinking Education Policy and Methodology in a Post-truth Era education policy analysis archives Volume 26 Number 150 November 19, 2018 ISSN 1068-2341 Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for non- commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by the author(s) or EPAA. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), PubMed, QUALIS A1 (Brazil), REDALyC, SCImago Journal Rank; SCOPUS, SOCOLAR (China). Please send errata notes to Audrey Amrein-Beardsley at Audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. http://www.doaj.org/ mailto:Audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 22 education policy analysis archives editorial board Lead Editor: Audrey Amrein-Beardsley (Arizona State University) Editor Consultor: Gustavo E. Fischman (Arizona State University) Associate Editors: David Carlson, Lauren Harris, Eugene Judson, Mirka Koro-Ljungberg, Scott Marley, Molly Ott, Iveta Silova (Arizona State University) Cristina Alfaro San Diego State University Amy Garrett Dikkers University of North Carolina, Wilmington Susan L. Robertson Bristol University Gary Anderson New York University Gene V Glass Arizona State University Gloria M. Rodriguez University of California, Davis Michael W. Apple University of Wisconsin, Madison Ronald Glass University of California, Santa Cruz R. Anthony Rolle University of Houston Jeff Bale OISE, University of Toronto, Canada Jacob P. K. Gross University of Louisville A. G. Rud Washington State University Aaron Bevanot SUNY Albany Eric M. Haas WestEd Patricia Sánchez University of University of Texas, San Antonio David C. Berliner Arizona State University Julian Vasquez Heilig California State University, Sacramento Janelle Scott University of California, Berkeley Henry Braun Boston College Kimberly Kappler Hewitt University of North Carolina Greensboro Jack Schneider University of Massachusetts Lowell Casey Cobb University of Connecticut Aimee Howley Ohio University Noah Sobe Loyola University Arnold Danzig San Jose State University Steve Klees University of Maryland Jaekyung Lee SUNY Buffalo Nelly P. Stromquist University of Maryland Linda Darling-Hammond Stanford University Jessica Nina Lester Indiana University Benjamin Superfine University of Illinois, Chicago Elizabeth H. DeBray University of Georgia Amanda E. Lewis University of Illinois, Chicago Adai Tefera Virginia Commonwealth University Chad d'Entremont Rennie Center for Education Research & Policy Chad R. Lochmiller Indiana University Tina Trujillo University of California, Berkeley John Diamond University of Wisconsin, Madison Christopher Lubienski Indiana University Federico R. Waitoller University of Illinois, Chicago Matthew Di Carlo Albert Shanker Institute Sarah Lubienski Indiana University Larisa Warhol University of Connecticut Sherman Dorn Arizona State University William J. Mathis University of Colorado, Boulder John Weathers University of Colorado, Colorado Springs Michael J. Dumas University of California, Berkeley Michele S. Moses University of Colorado, Boulder Kevin Welner University of Colorado, Boulder Kathy Escamilla University of Colorado, Boulder Julianne Moss Deakin University, Australia Terrence G. Wiley Center for Applied Linguistics Yariv Feniger Ben-Gurion University of the Negev Sharon Nichols University of Texas, San Antonio John Willinsky Stanford University Melissa Lynn Freeman Adams State College Eric Parsons University of Missouri-Columbia Jennifer R. Wolgemuth University of South Florida Rachael Gabriel University of Connecticut Amanda U. Potterton University of Kentucky Kyo Yamashiro Claremont Graduate University Scientifically based research in a post -truth era 23 archivos analíticos de políticas educativas consejo editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editores Asociados: Armando Alcántara Santuario (Universidad Nacional Autónoma de México). Angelica Buendia, (Metropolitan Autonomous University), Ezequiel Gomez Caride, (Pontificia Universidad Católica Argentina), Antonio Luzon, (Universidad de Granada), José Luis Ramírez, Universidad de Sonora), Paula Razquin (Universidad de San Andrés) Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Ana María García de Fanelli Centro de Estudios de Estado y Sociedad (CEDES) CONICET, Argentina Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Miguel Ángel Arias Ortega Universidad Autónoma de la Ciudad de México Juan Carlos González Faraco Universidad de Huelva, España José Gregorio Rodríguez Universidad Nacional de Colombia, Colombia Xavier Besalú Costa Universitat de Girona, España María Clemente Linuesa Universidad de Salamanca, España Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Xavier Bonal Sarro Universidad Autónoma de Barcelona, España Jaume Martínez Bonafé Universitat de València, España José Luis San Fabián Maroto Universidad de Oviedo, España Antonio Bolívar Boitia Universidad de Granada, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Jurjo Torres Santomé, Universidad de la Coruña, España José Joaquín Brunner Universidad Diego Portales, Chile María Guadalupe Olivier Tellez, Universidad Pedagógica Nacional, México Yengny Marisol Silva Laya Universidad Iberoamericana, México Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Miguel Pereyra Universidad de Granada, España Ernesto Treviño Ronzón Universidad Veracruzana, México Gabriela de la Cruz Flores Universidad Nacional Autónoma de México Mónica Pini Universidad Nacional de San Martín, Argentina Ernesto Treviño Villarreal Universidad Diego Portales Santiago, Chile Marco Antonio Delgado Fuentes Universidad Iberoamericana, México Omar Orlando Pulido Chaves Instituto para la Investigación Educativa y el Desarrollo Pedagógico (IDEP) Antoni Verger Planells Universidad Autónoma de Barcelona, España Inés Dussel, DIE-CINVESTAV, México José Ignacio Rivas Flores Universidad de Málaga, España Catalina Wainerman Universidad de San Andrés, Argentina Pedro Flores Crespo Universidad Iberoamericana, México Juan Carlos Yáñez Velazco Universidad de Colima, México javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/816') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/819') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/820') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/4276') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1609') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/825') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/797') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/823') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/798') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/555') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/814') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/2703') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/801') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/826') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/802') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/3264') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/804') Education Policy Analysis Archives Vol. 26 No. 150 SPECIAL ISSUE 24 arquivos analíticos de políticas educativas conselho editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editoras Associadas: Kaizo Iwakami Beltrao, (Brazilian School of Public and Private Management - EBAPE/FGV, Brazil), Geovana Mendonça Lunardi Mendes (Universidade do Estado de Santa Catarina), Gilberto José Miranda, (Universidade Federal de Uberlândia, Brazil), Marcia Pletsch, Sandra Regina Sales (Universidade Federal Rural do Rio de Janeiro) Almerindo Afonso Universidade do Minho Portugal Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil José Augusto Pacheco Universidade do Minho, Portugal Rosanna Maria Barros Sá Universidade do Algarve Portugal Regina Célia Linhares Hostins Universidade do Vale do Itajaí, Brasil Jane Paiva Universidade do Estado do Rio de Janeiro, Brasil Maria Helena Bonilla Universidade Federal da Bahia Brasil Alfredo Macedo Gomes Universidade Federal de Pernambuco Brasil Paulo Alberto Santos Vieira Universidade do Estado de Mato Grosso, Brasil Rosa Maria Bueno Fischer Universidade Federal do Rio Grande do Sul, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Fabiany de Cássia Tavares Silva Universidade Federal do Mato Grosso do Sul, Brasil Alice Casimiro Lopes Universidade do Estado do Rio de Janeiro, Brasil Jader Janer Moreira Lopes Universidade Federal Fluminense e Universidade Federal de Juiz de Fora, Brasil António Teodoro Universidade Lusófona Portugal Suzana Feldens Schwertner Centro Universitário Univates Brasil Debora Nunes Universidade Federal do Rio Grande do Norte, Brasil Lílian do Valle Universidade do Estado do Rio de Janeiro, Brasil Flávia Miller Naethe Motta Universidade Federal Rural do Rio de Janeiro, Brasil Alda Junqueira Marin Pontifícia Universidade Católica de São Paulo, Brasil Alfredo Veiga-Neto Universidade Federal do Rio Grande do Sul, Brasil Dalila Andrade Oliveira Universidade Federal de Minas Gerais, Brasil