6 May 2016, Vol. 8, No. 1 AJHPE Evidence-based practice (EBP) is positioned as an inherent good in the medical and clinical literature, and not without reason. It relies on the integration of research evidence, clinical expertise and patient preferences,[1] and has become a foundation on which health systems are built and improved. There is however, a growing body of literature that takes a more critical stance towards EBP, especially when practitioners make assumptions about what constitutes ‘the evidence’ and how the data informing that evidence are gathered.[2,3] The evidence upon which EBP is premised is usually derived from experimental research conducted in professional disciplines that are firmly rooted in the positivist paradigm; the research method most closely associated with this is the randomised controlled trial (RCT). RCTs are quantitative, controlled experiments in which the effect of an intervention can be determined more objectively than by observational studies.[4] There is no doubt that the method has utility in determining cause-effect relationships between medical treatments and patient outcomes, making it a powerful design for intervention studies with the objective of determining the influence of one variable on another.[5] In an educational context it may initially seem reasonable to expect that an experimental design could determine the effect of a teaching intervention that aims to improve student learning. The argument is that by using randomisation to average out the differences between students, one would be able to demonstrate which teaching and learning strategies lead to the largest effect sizes. These data, presumed to be free of subjective interpretation, could then inform policies that drive the implementation of effective teaching interventions.[6] However, if we assume that the evidence gathered via experimental research provides insight into an objective reality, we must take a position on teaching and learning that is at odds with our best explanations for how learning happens. Therefore, if we want to use RCTs in educational research, we must assume that there is a cause-effect relationship in the teaching and learning interaction that can be objectively measured. In this article we argue that RCTs are an inappropriate design choice for educational research because they force one to assume ontological and epistemological positions that are at odds with theoretically informed perspectives of learning. RCTs in educational research We begin by highlighting the biased way in which RCTs are positioned relative to other forms of research endeavours, explicit in the language employed by RCT proponents. Goldacre[7] suggests that ‘Evidence-based interventions in teaching could … replace the current system where untested methods are passed to teachers through a variety of often dubious outlets’, and ‘We need a slow revolution that puts evidence at the heart of teaching’. Torgerson[8] asserts that RCTs are the ultimate expression of evaluative research, referring to ‘the importance and supremacy of the RCT’, and expresses concern that educational research tends to rely on ‘manifestly inferior’ qualitative methods. It seems clear that those who most strongly advocate the use of RCTs in education have an inherent bias against other methods of data collection, strongly positioning themselves within a positivist interpretation of reality. This does not mean that RCTs and other forms of experimental research are not valuable tools in the repertoire of the researcher; randomisation is rightly considered an appropriate design choice in clinical trials. By controlling for the influence of all other variables between groups – through the assumed equal distribution of those differences in a large enough randomised sample – any differences in outcome can be more confidently attributed to the intervention.[5] As the RCT is a powerful tool successfully used in medical and clinical research, some have suggested that it should therefore underpin all ‘good’ research regardless of context’.[9] However, one cannot assume that RCTs can provide more – and better – evidence, which inevitably leads to improvements in education.[6] RCT proponents hope that these trials can do for educational researchers what they have done for medical researchers, i.e. provide clear-cut answers around the relative benefits of one intervention over another.[7] RCTs are presented as a gold standard, able to determine ‘the truth’ by simplifying and Randomised controlled trials (RCTs) are a valued research method in evidence-based practice in medical and clinical settings because they are associated with a particular ontological and epistemological perspective that is situated within a positivist world view. It assumes that environments and variables can be controlled to establish cause-effect relationships. However, current theories of learning suggest that knowledge is socially constructed, and that learning occurs in open systems that cannot be controlled and manipulated as would be required in a RCT. They recognise the importance and influence of context on learning, which positivist research paradigms specifically aim to counter. We argue that RCTs are inappropriate in education research because they force one to take up ontological and epistemological positions in a technical rationalist framework, which is at odds with current learning theory. Afr J Health Professions Educ 2016;8(1):6-8. DOI:10.7196/AJHPE.2016.v8i1.683 Randomised controlled trials in educational research: Ontological and epistemological limitations M Rowe,1 PhD; C Oltmann,2 PhD 1 Department of Physiotherapy, Faculty of Community and Health Sciences, University of the Western Cape, Cape Town, South Africa 2 Division of Pharmacy Practice, Faculty of Pharmacy, Rhodes University, Grahamstown, South Africa Corresponding author: M Rowe (michael@realmdigital.co.za) Short Report May 2016, Vol. 8, No. 1 AJHPE 7 generalising the complex social interactions of the educational context.[10] For example, Goldacre[7] suggests that by ‘… collecting better evidence about what works best, and establishing a culture where this evidence is used as a matter of routine, we can improve outcomes’. By positioning the RCT as the best way to collect ‘better evidence about what works best’, it has been suggested as a means by which educational practice can be improved, as it generates absolute facts about an existing reality that is objectively measured.[10] By choosing the RCT as a method of gathering data, the researcher is taking a stance within a framework that describes what they believe about what it means to know something in the world. The way researchers make decisions about which methodologies are useful is determined in part by their ontological and epistemological perspectives. A research methodology is not simply a neutral plan for designing a systematic inquiry, but is instead informed by a theoretical perspective. The selection of a research method is therefore a proxy for expressing a belief about what it means to know and our attempts to better understand what we know.[11] Therefore, our beliefs around our ways of knowing in the world influence how we choose to investigate them.[10] According to Grix,[12] research is best done by establishing a relationship between what a researcher thinks can be researched (the ontological position), what we believe can be known about it (the epistemological position), and how to go about acquiring it (the methodological approach). Thus, the influence of the ontological and epistemological position on what and how a topic is investigated is clear.[12] As researchers we are required to make explicit claims about how we view reality (ontology) and what constitutes knowledge (epistemology),[13] because these perspectives have a significant influence on the methodology chosen and ultimately on the outcomes of the research.[14] Educational and clinical contexts differ It may initially seem as if clinical and educational contexts are similar and that the processes in both are therefore susceptible to the same methods of investigation. However, there are fundamental differences that make it difficult to see how experimental methods are appropriate in the process of evaluating learning. While it could be argued that both clinical and educational contexts represent complex (open) systems,[8] this typically conflates the clinical trial, where all variables are carefully controlled, with a health system. While health systems are complex environments, it is clear that RCTs are not used to investigate complete health systems. Instead, their objective is to achieve generalisable simplicity by holding all else equal and determining the effect of a single variable, the outcome of which is applicable across a variety of contexts.[10] Therefore, the RCT represents an attempt to create an ‘artificially closed system’[15] whereby the relationships between variables in controlled, non-complex contexts are determined.[10] This is appropriate in the positivist ontology and epistemology and is therefore the reason that the RCT is an appropriate method of gathering data in clinical research. However, it is not possible to create controlled, non-complex educational environments that enable cross-context predictions.[10] Randomisation does not control for other sources of variation and confounding factors that are likely to be found in educational contexts. These include for example, factors that lead to differences in studying methods, changes in learner motivation, and effects of other, non-intervention experiences that occur during the implementation of the intervention.[5] It is impossible to create a closed education system, even an ‘artificially closed system’,[15] and therefore almost impossible to identify how much of the intervention the learner actually ‘receives’, or to determine what the learner does with what the teacher provides.[6] The factors that influence learning outcomes cannot be recreated in different contexts, or even in the same context at different times.[5] Education therefore exists in an open system, and even if we attempt to reduce and limit change and variation – internally and externally – we will not be able to determine causality.[15] RCTs assume that every implementation of the intervention is the same, that everyone receiving the intervention will be affected in the same way, and that giving and receiving the intervention is divorced from the reality of the individual personalities and institutional contexts of the participants. In educational research it is difficult to tightly control variables and blind subjects in ways that are ethical or feasible, and one cannot ‘apply curriculum daily’ in the same way that one can prescribe medication.[5] It is therefore impossible to control for confounding variables in learning environments, making the attempt to use RCTs in this context ‘hopelessly flawed’.[10] The beliefs of the researcher around the context in which the research will take place clearly inform the choice of method used to gather data, which in turn informs the outcome of the study. By conflating a clinical trial with research that seeks to determine the effectiveness of a teaching intervention on student learning, RCT proponents ignore the fact that clinical and educational contexts are fundamentally different and that these differences require different methods of gathering data. We now present an argument demonstrating how beliefs around knowledge and the nature of reality may contradict our understanding of how learning happens. Beliefs around reality and knowledge If one believes in an objective reality that is separate from the people conducting research into that reality, one is more likely to view knowledge as a quantity of something that is to be accumulated.[16] The belief that knowledge is separate from knowers and that it can be transmitted to others suggests that a positivist approach to educational research should be considered.[17] The focus is likely to be on using reliable and valid tools to collect quantitative data about the learning intervention that is regarded as value free.[18] Positivist research maintains that knowledge is objective, that it involves hypothesis testing and identifies causality.[19] In this understanding of the world, learning is focused on the teacher and concerned largely with the transfer of information to the learner.[11] Therefore, a positivist perspective on the nature of reality is most likely associated with a method of teaching and learning that considers knowledge as something that can be transferred between people. An alternative perspective on the nature of reality is that it is interpretive, subjective and different for each person.[20] In this context, knowledge is socially constructed by individuals interacting with each other and the world.[17] Thus, interpretive perspectives of reality involve an attempt to understand phenomena from an individual’s perspective,[10] and consequently recognise that in certain contexts it is not possible to determine causality. Instead, interpretive research focuses on trying to understand and explain reality from the unique vantage point of individuals.[21] If knowledge is socially constructed and experienced differ- ently by individuals in different contexts, it is a conception of reality that is fundamentally at odds with positivism and therefore unlikely to be explained with positivist research methods. There are many theories that seek to explain how learning happens. Regardless of which theory one adopts, they all describe an interpretation and negotiation taking place between an individual and their unique Short Report 8 May 2016, Vol. 8, No. 1 AJHPE Short Report context. This understanding of learning recognises that it influences and is in turn influenced by the context[22] in a complex relationship between knowledge, the knower and knowing.[23] There is therefore a distinction between con ceptions of research that are atomistic and orientated towards external products with the intention of producing outcomes, and conceptions that are holistic and analytical with an orientation towards internal process- es, where the intention is to understand.[11] If the purpose of educational research is to understand learning in a holistic way, rather than to control and predict for certain predetermined outcomes, then interpretive research methodologies may be more appro- priate than experimental methods such as RCTs.[9] One suggestion to move the discussion forward is for researchers to avoid the ‘methodological tribalism’ that takes up so much of our attention and to embrace a pragmatic approach to research, where we use the tool that is the best fit for the problem we are addressing.[24] Conclusion RCTs are not neutral methods of gathering unbiased data that describe an objective reality. They are positioned within paradigms that come with certain ontological and epistemological perspectives about the world, which seem to be at odds with ontological and epistemological perspectives of learning. The RCT requires the researcher to adopt a particular stance in terms of their beliefs about the world, which is in contrast with our understanding about how learning happens. To use RCTs in educational research, we are forced to assume a positivist view of the world in which the learning context must be simplified and controlled, rather than accepting the complexity and inherent subjectivity of the nature of learning, and working in it. The use of the RCT in educational research is therefore a flawed design choice, as it betrays a flawed understanding of the nature of learning. It requires us to accept that there exists a set of ‘correct conditions’, and that by controlling for the ‘right’ variables we are able to control learning in the same way that we control clinical trials. The problem with RCTs in educational research is therefore not only that they may be ineffective and impractical when determining the value of a learning intervention, but that they also force us to take up ontological and epistemological positions in a technical rationalist framework that perceives the world as having a single truth, which is inconsistent with a real understanding of learning. References 1. Sackett DL, Rosenberg WMC, Gray JAM, Richardson WS. Evidence based medicine: What it is and what it isn’t. BMJ 1996;312:71-72. 2. Biesta GJJ. Why ‘what works’ still won’t work: From evidence-based education to value-based education. Studies in Philosophy and Education 2010;29(5):491-503. 3. Dimitriadis G. Revisiting the question of evidence. Cultural Studies – Critical Methodologies 2008;8(1):3-14. 4. Chalmers TC, Smith H Jr, Blackburn B, et al. A method for assessing the quality of a randomised control trial. Control Clin Trials 1981;2(1):31-49. 5. Sullivan GM. Getting off the ‘gold standard’: Randomized controlled trials and education research. J Grad Med Educ 2011;3(3):285-289. 6. Norman G. RCT = results confounded and trivial: The perils of grand educational experiments. Med Educ 2003;37:582-584. 7. Goldacre B. Building evidence into education. Department for Education, London. 2013. https://www.gov.uk/ government/news/building-evidence-into-education (accessed 6 January 2016). 8. Torgerson CJ. Educational research and randomised trials. Med Educ 2002;36:1002-1003. 9. Denzin NK. The elephant in the living room: Or extending the conversation about the politics of evidence. Qualitative Research 2009;9(2):139-160. 10. Regehr G. It’s NOT rocket science: Rethinking our metaphors for research in health professions education. Med Educ 2010;44:31-39. [http://dx.doi.org/10.1111/j.1365-2923.2009.03418.x] 11. Brew A. Teaching and research: New relationships and their implications for inquiry-based teaching and learning in higher education. Higher Education Research and Development 2003;22:3-18. 12. Grix J. The Foundations of Research. London: Palgrave Macmillan, 2004. 13. Maton K, Moore R. Social Realism, Knowledge and the Sociology of Education – Coalitions of the Mind. London: Continuum International Publishing Group, 2010:1-13. 14. Jackson E. Choosing a methodology: Philosophical underpinning. Practitioner Research in Higher Education Journal 2013;7(1):49-62. 15. Danermark B, Ekstrom M, Jakobsen L, Karlsson JC. Explaining Society: Critical Realism in the Social Sciences. London: Routledge, 2002. 16. Mourad RP Jr. Postmodern Philosophical Critique and the Pursuit of Knowledge in Higher Education. Westport, CT: Bergin and Garvey, 1997. 17. Crotty M. The Foundations of Social Research. London: Sage, 1998. 18. House E. Realism in research. Educ Res 1991;20(6):2-25. 19. Creswell JW. Research Design: Qualitative and Mixed Methods Approaches. London: Sage, 2009. 20. Guba EG, Lincoln YS. Competing paradigms in qualitative research. In: Denzin NK, Lincoln YS, eds. Handbook of Qualitative Research. London: Sage, 1994;105-117. 21. Cohen L, Manion L, Morrison K. Research Methods in Education. 6th ed. London: Routledge, 2007. 22. Marton F, Hounsell D, Entwistle N. The Experience of Learning, 2nd ed. Edinburgh: Scottish Academic Press, 1997. 23. Maton K. Knowledge-knower structures in intellectual and educational fields. In: Christie F, Martin J, eds. Language, Knowledge and Pedagogy: Functional Linguistics and Sociological Perspectives. London: Continuum, 2007:87-108. 24. Lamont M, Swidler A. Methodological pluralism and the possibilities and limits of interviewing. Qualitative Sociology 2014;37:153-171.