Australasian Journal of Educational Technology, 2023, 39(2). 98 Defining an effective approach to blended learning in higher education: A systematic review Shaun McCarthy, Edward Palmer University of Adelaide Blended learning has enjoyed wide acceptance as a teaching and learning approach in higher education, but its use and understanding commonly fail to extend across all levels of blending. At the institutional level, challenges still exist in aligning a blended learning approach with core university priorities. Often, there is a focus on the provisions of technology tools and associated training; however, there is less emphasis on the development of frameworks that support an institutional-level approach to blended teaching and learning and ways that these can be effectively measured. This paper analyses previous work undertaken in the field of blended learning and looks to build on the literature by defining an effective approach to adoption using conceptual clarity, blended frameworks and institutional-level implementation of blended learning as a framework to describe effective use within higher education. Implications for practice or policy: • University decision makers should define an institutional approach to blended learning and foster a common understanding of what success will look like. • Institutional strategy must carefully consider the multifaceted roles of students, academics and administrators within blended learning. • Blended learning adoption should be measured using criteria and descriptive standards to evaluate framework implementation. Keywords: blended learning, hybrid learning, blended learning framework, blended learning implementation, blended learning evaluation, higher education, systematic review Introduction Blended learning has, for decades, been heralded as a transformative teaching and learning approach and its use has become increasingly more popular in international higher education as technological approaches improve and universities look for better ways to engage with learners (Alammary et al., 2014; Garrison & Kanuka, 2004; Mirriahi et al., 2015). Indeed, the impact of the COVID-19 pandemic has further increased the emphasis on this teaching and learning strategy, and as a result, this trend is expected to continue (Singh, 2021). To this end, Norberg et al. (2011 p. 207) suggested that blended learning is effectively “the new normal”, influenced by the technology used as part of our everyday lives and has been evolving since the invention of the printing press. Although seemingly well established and simple in principle, the concept of blended learning continues to create debate due to its underlying complexity (Garrison & Vaughan, 2008). Others have also suggested that the ever-changing technology landscape makes the establishment of functional approaches to blended learning a moving target, due to teaching activities that change in composition over time (Caner, 2012; Lim et al., 2019; Oliver & Trigwell, 2005). Blended learning is transformative in the way it is used to reorder and rethink the teaching and learning dynamic (Garrison & Kanuka, 2004) by combining both classroom and online elements to convey information to learners. The combined benefit of blending helps to negate the shortcomings of traditional face-to-face (FTF) learning, by way of flexible delivery and personalised learning options (Medina, 2018), as well as mitigating the lack of true teacher-student interaction observed in fully online e-learning (Bernard et al., 2014). Research has shown that where educators have designed their teaching to maximise student engagement through a blended learning approach, participants have demonstrated better learning outcomes and higher performance when compared to purely FTF or online learning delivery methods alone (Alammary et al., 2014; Bernard et al., 2014; Caner, 2012; Garrison & Kanuka, 2004; Lim et al., 2019; Norberg et al., 2011). Although these results are significant, blended learning is yet Australasian Journal of Educational Technology, 2023, 39(2). 99 to be fully integrated in most higher education institutions (Graham et al., 2013; Smith & Hill, 2019). This may be partially due to the fact that blended learning as an approach largely remains conceptual and lacks a true definition (Moskal et al., 2013; Torrisi-Steele & Drew, 2013). Blended learning has become an umbrella term for a variety of overlapping approaches such as e-learning, hybrid learning, mixed-mode delivery, problem-based learning and flipped classroom (Smith & Hill, 2019). And although this variability in approaches offers significant benefits in terms of delivery options, the same variability makes it difficult for universities to define a framework for institutional implementation (Mirriahi et al., 2015; Moskal et al., 2013; Porter & Graham, 2016). At the institutional level, challenges still exist in aligning a blended learning approach with core university priorities. Often, there is a focus on the provisions of technology tools and associated training; however, there is less emphasis on the development of pedagogical approaches to blended teaching and learning (Norberg et al., 2011). This lack of true understanding is also observed in research associated with blended learning adoption (Porter et al., 2014; Smith & Hill, 2019) and the resulting frameworks that support the way blending occurs across different levels (Adekola et al., 2017; Bonk & Graham, 2012; Pima et al., 2018). In their systematic review exploring the scholarship of blended learning within higher education, Anthony et al. (2020) also noted the paucity of studies investigating the theoretical foundation of blended learning implementation. Although rigorous, their review did not use a theoretical model for evaluation, nor did it use the Web of Science, Scopus or ERIC in the database search. Smith and Hill (2019) also looked to define the nature of blended learning through its depiction in research. Although they too contributed to the development of a theoretical foundation of blended learning, their review was based on articles published over a 5-year period and guided by prior research more so than by a true systematic approach. This paper is therefore a timely contribution to the review of scholarship within the field of blended learning over the last 2 decades. It takes a systematic approach to review significant online libraries and build on current literature, with the goal of defining effective use of blended learning within higher education. Conceptually, blended learning is often associated with the philosophical worldview of constructivism and the associated research methods follow either a grounded theory approach to develop a hypothesis based on the existing understanding of blended learning or, less often, an ethnographic approach in order to collect data associated with the behavioural patterns and actions of research participants (i.e., students) over a period of time (Creswell & Creswell, 2018). In terms of developing an appropriate conceptual framework (Green, 2014) for use in evaluating blended learning approaches, this study has referenced prior work to uncover themes that are common within the field of higher education (Anthony et al., 2020; Halverson et al., 2014; Pima et al., 2018; Smith & Hill, 2019). A visual representation of this conceptual framework is presented in Figure 1, where an institutional blended learning approach is defined through the development of conceptual clarity, an understanding of blended frameworks and mechanisms that drive institutional-level implementation of blended learning. The following research questions underpin the model: (1) Conceptual clarity: How should we define blended learning within higher education? (2) Blended frameworks: What is the most effective blended learning framework? (3) Implementation: How should blended learning be implemented by universities? Australasian Journal of Educational Technology, 2023, 39(2). 100 Figure 1. Conceptual framework for blended learning in higher education By defining a conceptual framework, we create a lens through which existing literature can be reviewed. This framework helps to shape the criteria for article selection and for article exclusion. The overarching goal of this investigation was to define an effective approach to blended learning in higher education, as such the conceptual framework looks to define conceptual clarity, blended frameworks and methods associated with institutional-level implementation of blended learning to evaluate approaches in current literature. Methods An extended literature search was undertaken using the Web of Science, SCOPUS and ERIC databases to identify existing research through a broad analysis, using a systematic approach. The search terms were based on the conceptual framework presented in this study, with particular emphasis around blended learning in higher education, and are consistent with Pima et al. (2018). This framework also follows the sub-themes of instructional design identified in the work of both Pima et al. (2018) and Halverson et al. (2014). The key search terms for this activity were “blended learning higher education” OR “blended learning framework” OR “blended learning design” in the title AND “higher education” within the abstract or keywords. The search was conducted in January of 2022 to enable a complete search of literature published in the 20-year period from 2001 to 2021. Reference lists from articles were also reviewed to identify additional resources that did not appear in the literature search but had been regularly referenced by the final group of selected articles. Finally, the first 200 records returned by a Google Scholar search for the search term ‘allintitle:"higher education" "blended learning"’ was added to the collection of titles to validate the final list of literature and reduce the chance that something of significance had been omitted. Eligibility criteria The search process resulted in 1,423 records containing 497 duplicates which were subsequently removed. After excluding papers based on title and abstract and using the established eligibility criteria listed in Table 1, 90 articles remained. This literature was reviewed in full to arrive at the final database of articles used in this study (n = 23). An additional five publications were included based on literature Blended learning in higher education Conceptual clarity Blended frameworks Implementati on Australasian Journal of Educational Technology, 2023, 39(2). 101 that was regularly referenced in the final group. An overview of the search protocol is presented in Figure 2 and is based on the recommendations of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Page et al., 2021). Figure 2. Overview of the systematic review process based on the PRISMA The eligibility criteria used in this review was significant in shaping the final group of articles selected for this study, in particular the exclusion criteria used to omit articles that did not fit the intended themes of the investigation, these were (a) studies addressing blended learning within specific disciplines, (b) studies that focused on specific pedagogical approaches or tools, (c) use of specific learning management systems, (d) blended approaches that originate from a pre-existing online approach and (e) implementation of blended learning in response to COVID-19. These exclusions were implemented to assist the identification of articles that addressed blended learning from a holistic perspective, originating from a more traditional teacher-led approach, using models and frameworks that are largely technology Australasian Journal of Educational Technology, 2023, 39(2). 102 agnostic. We reviewed the screened records separately, creating a final list; we then compared and discussed the identified articles to reach final agreement. Table 1 Summary of eligibility criteria Includes Excludes Studies that employ a blended learning framework, overarching design model or theory Use of blended learning within specific disciplines (e.g., engineering, medicine) Institutional-level implementation Specific course design, pedagogical approaches, or tools (e.g., flipped classroom, mobile learning) Targets blended learning approaches within higher education Use of specific learning management system(s) English language articles published from 2001 to 2021 Blended learning originating from a pre-existing online approach (e.g., massive open online courses) Full-text literature from peer-reviewed journals or academic books Implementation strategies in response to COVID-19 Quality assessment A critical appraisal was undertaken to determine the individual research validity, level of bias and relevance of each identified study. An assessment checklist was developed based on the key questions presented in Young and Solomon (2009), with a total of 10 criteria designed to evaluate the quality of all studies selected for this review (N = 28). We undertook a quality assessment separately using the checklist shown in Table 2. Assessment questions were measured based on a point scale where 2 points was equal to an answer of “yes”, 0 points was equal to an answer of “no”, and 1 point was equal to “partially”. Where a score was not applicable, for example, where data was not collected, no score was given. Studies could therefore have a maximum score of 20 and a minimum of 0. The independent scoring of each study was compared to reach agreement and is presented in Table 3. The final scores were normalised to account for criteria that was not applicable and represented as a percentage, where a higher overall score indicates that the study was of high validity and relevance. The final results show a range of 70% to 95% indicating that the quality of the selected studies was high overall and appropriate for this study. Table 2 Quality assessment criteria Questions Criteria Q1 Does the study employ a blended learning framework or model or theory? Q2 Does the study target institutional-level blended learning approaches within higher education? Q3 Does the study contribute to a theoretical foundation of blended learning? Q4 Are the research aims plainly stated? Q5 Is the study design appropriate? Q6 Are data collection and analysis well described? Q7 Are the findings credible and clearly discussed? Q8 Are the implications clearly stated? Q9 Do the stated limitations have any impact on the result? Q10 Are there any conflicts of interest? Australasian Journal of Educational Technology, 2023, 39(2). 103 Table 3 Results of quality assessment Study Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 % Garrison & Kanuka (2004) 2 2 2 2 2 1 1 2 88% Oliver & Trigwell (2005) 2 2 2 1 1 0 2 2 0 2 70% Bliuc et al. (2007) 1 2 2 2 1 1 2 1 2 2 80% Vaughan (2007) 1 2 2 0 1 2 2 1 2 72% Garrison & Vaughan (2008) 2 2 2 2 2 2 1 2 94% Huang et al. (2008) 2 2 1 1 2 2 1 2 81% Norberg et al. (2011) 1 2 1 2 2 2 2 2 88% Bonk & Graham (2012) 2 2 2 1 1 1 2 2 1 2 80% Caner (2012) 1 2 2 1 1 1 2 2 2 2 80% Graham et al. (2013) 2 2 2 2 2 2 2 1 2 94% Chaudhri & Gallant (2013) 1 2 2 2 2 1 2 2 2 2 90% Moskal et al. (2013) 1 2 2 2 2 2 2 2 1 2 90% Torrisi-Steele & Drew (2013) 2 2 2 1 2 2 2 2 1 2 90% Alammary et al. (2014) 2 2 2 2 2 2 2 2 1 2 95% Bernard et al. (2014) 1 2 2 2 2 2 2 2 1 2 90% Porter et al. (2014) 2 2 2 2 2 2 2 2 1 2 95% Halverson et al. (2014) 1 2 1 2 2 2 2 1 1 2 80% Mirriahi et al. (2015) 1 2 2 2 2 2 2 2 1 2 90% Wang et al. (2015) 1 2 2 2 2 2 2 2 2 2 95% Porter & Graham (2016) 2 2 2 2 2 2 2 2 1 2 95% Adekola et al. (2017) 2 2 2 2 2 2 2 2 1 2 95% Vaughan et al. (2017) 1 2 2 2 1 1 2 2 1 2 80% Galvis (2018) 2 2 2 2 2 2 2 2 1 2 95% Medina (2018) 0 2 2 1 2 2 2 2 1 2 80% Pima et al. (2018) 2 2 1 2 2 2 2 1 2 89% Lim et al. (2019) 2 2 2 2 1 2 2 1 2 89% Smith & Hill (2019) 2 2 2 2 2 2 2 2 1 2 95% Anthony et al. (2020) 0 2 2 2 2 2 2 2 1 2 85% Results We each coded the identified articles into themes associated with the conceptual framework separately. In many cases, the literature contained overlapping themes. Where this was identified, we discussed the best fit for each study and placed each in the most relevant grouping based on the primary intention of the research. For example, the paper by Graham et al. (2013) straddles the themes of blended frameworks and institutional implementation; however, the goal of the paper was to identify the key issues associated with institutional adoption of blended learning to guide administrators; it was therefore associated with the institutional implementation grouping. Similarly, the thematic analysis carried out by Halverson et al. (2014) touched on all themes; however, the aim of the study was to uncover the breadth of scholarship across the field of blended learning; it was therefore associated with the conceptual clarity grouping. The result of this thematic grouping is shown in Table 4 below. It is evident that there is a large body of work associated with conceptual clarity emphasising blended teaching models, but fewer studies that address effective blended learning frameworks and institutional implementation within higher education. Australasian Journal of Educational Technology, 2023, 39(2). 104 Table 4 Thematic grouping of identified articles Conceptual clarity Blended frameworks Institutional implementation Oliver & Trigwell (2005) Garrison & Vaughan (2008) Garrison & Kanuka (2004) Bliuc et al. (2007) Mirriahi et al. (2015) Vaughan (2007) Huang et al. (2008) Wang et al. (2015) Chaudhri & Gallant (2013) Norberg et al. (2011) Adekola et al. (2017) Graham et al. (2013) Bonk & Graham (2012) Lim et al. (2019) Moskal et al. (2013) Caner (2012) Porter et al. (2014) Torrisi-Steele & Drew (2013) Porter & Graham (2016) Alammary et al. (2014) Vaughan et al. (2017) Bernard et al. (2014) Galvis (2018) Halverson et al. (2014) Anthony et al. (2020) Medina (2018) Pima et al. (2018) Smith & Hill (2019) The distribution of articles published by theme over time is illustrated in Figure 3. The timeline suggests that the bulk of the work identified has largely been published within the last decade, with only six articles published prior to 2011. The greatest concentration of publications appeared in the period from 2012 to 2015. The emphasis on studies associated with conceptual clarity is also visible here, showing an increasing number of publications leading up to 2018, a trend also identified by Pima et al. (2018) in their thematic study. Literature associated with blended learning frameworks appear more sparsely across the years, while those addressing the implementation of blended learning within higher education are relatively consistent; however, the numbers do not appear to be growing. These results support the findings of previous investigations (Halverson et al., 2014; Pima et al., 2018; Smith & Hill, 2019), where blended models associated with instructional design are more common than scholarship addressing institutional frameworks or implementation of blended learning within higher education. Figure 3. Publications by thematic grouping from 2001 to 2021 Figure 4 details the number of publications broken down by study type, with the most popular approach identified as literature reviews followed by case studies. The grouping of these studies was consistent with the findings of Smith and Hill (2019), who found a greater number of empirical than non-empirical or combined studies, although the majority took an inferential approach and had a relatively small sample size. The geographical distribution of publications was established by identifying the institutional location of the first author or the publication location. The concept of blended learning is international, at least as a principle, with literature identified in this investigation originating from 12 countries. The majority of publications came from the United States of America (n = 7), followed by Australia (n = 5), Canada (n = 4), Australasian Journal of Educational Technology, 2023, 39(2). 105 the United Kingdom (n = 3), Columbia (n = 2), then China, Hong Kong, the United Arab Emirates, Norway, Tanzania, Turkey and Sweden (n = 1). This distribution of work is consistent with similar studies (Anthony et al., 2020; Smith & Hill, 2019), although Anthony et al. (2020) identified Malaysia as the primary country of publication in their investigation into blended learning adoption and implementation. Figure 4. Distribution of study research methods Discussion How should we define blended learning within higher education? The ability to define blended learning in any real way has been problematic for many academics due to the ambiguity in the way it is both viewed and implemented (Bernard et al., 2014; Oliver & Trigwell, 2005; Smith & Hill, 2019). The inconsistencies exist at several levels and include aspects associated with the intended purpose and pedagogical approaches, the ratio of online and FTF time, the role of technology and the role of the teacher. Despite this, blended learning has enjoyed wide acceptance as a teaching and learning approach in higher education, but a lack of common understanding and subsequent selection of the most appropriate design approach is problematic for teachers with little prior experience (Alammary et al., 2014). Many studies investigating blended learning cite the early work of Garrison and Kanuka (2004, p. 96) as a fundamental reference used to described blended learning: “the thoughtful integration of classroom FTF learning experiences with online learning experiences”. Although this definition captures the essence of blended learning, it remains simplistic; Smith and Hill (2019) argued that for this reason blended learning remains relatively specialised and not entirely well understood. Subsequent studies have suggested the additional need for a thoughtful pedagogical approach (Alammary et al., 2014) as well as greater consideration in terms of optimising achievement of the learning objectives (Huang et al., 2008). Oliver and Trigwell (2005) made an argument suggesting that the term blended learning should be abandoned completely as the associated work describes the teacher’s perspective, not that of the student. They also suggested that traditional FTF learning also has no true definition and, in fact, looks quite different for academics (and students) who have begun their journey in higher education during the age of the Internet, a view also shared by Norberg et al. (2011) and Caner (2012). Oliver and Trigwell (2005, p. 21) lobbied for variation theory as an alternative approach and suggested that the term blended pedagogies would be more fitting to describe the teaching methodology. Although these points are well made, the term blended learning is well established, and it is therefore better to build on this understanding, rather than argue against it. Australasian Journal of Educational Technology, 2023, 39(2). 106 Bonk and Graham (2012, p. 7) provided a model for the development of an effective description, citing the “four dimensions of interaction” required for effective blended learning; these are space, time, fidelity (senses) and humanness. Despite the widespread citation of this publication, these aspects are not commonly used to form the basis of a comprehensive definition of blended learning. This literature review showed blended learning is often defined using very broad terms, covering a range of different learning modes, or by very specific terms that aim to provide more rigidity around the approach. For example, the following definition from Medina (2018, p. 43) is incredibly broad: Blended learning is viewed as a pedagogical approach that offers educational communities the opportunity to customise their learning using synchronous and asynchronous delivery modes to increase levels of interaction among the agents involved (generally, students and faculty). While open to interpretation, this description largely addresses the space and time aspect of blending but tells us little about the roles of the teacher and student (humanness) or technology (fidelity); it also provides little in the way of an implementable approach. In contrast to this, Bernard et al. (2014, p. 94) posited that: Blended learning is the combination of face-to-face and online learning outside of class, where the latter does not exceed 50% of the course time. Face-to-face classroom time therefore can be greater than 50%. Defined blending ratios have been cited by others (Bliuc et al., 2007; Caner, 2012), and although this approach specifically addresses time and space aspects, it fails to consider the respective roles of teacher and student as well as technology fidelity or purpose of the blend. In fact, the ambiguity of blended learning suggests that attempting to define a ratio is unlikely to lend itself to a scalable framework. Many other researchers have been thoughtful about the role of technology in blended learning in terms of space and time, but only a few have incorporated the changing role of the teacher and student in the teaching and learning dynamic (Medina, 2018; Norberg et al., 2011; Pima et al., 2018) or aspects associated with individual student learning (Alammary et al., 2014; Mirriahi et al., 2015). Given the array of interpretive differences, many researchers have suggested that an exact definition of blended learning is not required (Lim et al., 2019; Mirriahi et al., 2015; Norberg et al., 2011; Oliver & Trigwell, 2005), and that it is reasonable for there to be a more general understanding of what blended learning is and what it sets out to achieve (Moskal et al., 2013). Given this level of acceptance of blended learning as an approach, there is flexibility for institutions to define this in their own way, developing a common language and basis from which stakeholders can work (Moskal et al., 2013; Smith & Hill, 2019; Vaughan, 2007). In addition to the dimensions of interaction, Bonk and Graham (2012, p. 7) also cited the four levels of blended learning: the activity level, the course level, the programme level and the institutional level. This has been used to describe the way blending can occur in terms of the student experience, but as Caner (2012) pointed out, it can also be used to reference levels of understanding. A common definition at the institutional level creates a foundation from which a blended learning framework can be established (Bliuc et al., 2007; Smith & Hill, 2019). This definition may also extend to programme- level blending or provide guidance for contextualisation, where the goal is to ensure a consistent student experience across their learning journey. At the course and activity level, the definition of blending can be “left open” (Galvis, 2018, p. 12) for teachers to interpret as they see fit, in order to develop a learning experience that meets the needs of the curriculum through the use of effective learning spaces and technology that support opportunities to create key teaching moments between the teacher and the student. As such, an appropriate institutional definition might consist of the following: Blended learning involves the thoughtful and ongoing development of curriculum, maximising the effectiveness of the teacher as a facilitator of knowledge and enabling student learning where, when and how they are best able to receive it. Australasian Journal of Educational Technology, 2023, 39(2). 107 Although this definition remains broad, it is appropriate at the institutional level. It references the need for thoughtful integration of teaching approaches, calls out the role of teachers as expert facilitators and acknowledges that students need effective mechanisms in the form of space, time and technology to participate effectively in the teaching and learning process. What is the most effective framework associated with blended learning in higher education? An institutional framework needs to account for a range of factors, including: learner needs, aspects of effective curriculum development and delivery, technology advantages, support mechanisms, organisational preparedness, stakeholder roles, risk factors and processes for continuous improvement (Adekola et al., 2017; Garrison & Vaughan, 2008; Lim et al., 2019; Mirriahi et al., 2015; Pima et al., 2018). Of the five articles associated with the blended learning frameworks theme, only two referenced examples and provided commentary around alternative frameworks that have been associated with blended learning. Adekola et al. (2017) cited the octagonal framework developed by Khan (2005) and subsequently refined this in their work noting that Khan’s framework was based solely around the development of e-learning and did not address infrastructure needs. Although partially relevant, Khan (2005) was not identified in this systematic review due to the specific nature of the search criteria and was not subsequently added to the final list of titles because the focus of the study was biased towards E-learning and not blended learning, therefore it did not meet the inclusion criteria. Adekola et al. (2017) also highlighted the work carried out by Graham et al. (2013), who detailed a blended learning adoption model but noted that their framework did not consider the views of all stakeholders, in particular, teachers and students. The adoption framework presented by Graham et al. (2013) has been included in this study, but is associated with the blended learning implementation theme. This is because their proposed framework looks to analyse the degree to which an institution has adopted and implemented policies associated with blended learning rather than offering a framework that addresses the requirements to support blended learning in the first instance. Wang et al. (2015) identified four alternative frameworks, including an earlier reference to the Khan (2005) octagonal framework. Two others were associated more closely with design models, focusing on instructional and course design approaches; however, the last was an early reference to the community of inquiry (CoI) framework discussed in the popular book authored by Garrison and Vaughan (2008). Despite this popularity, Wang et al. (2015) argued that the CoI framework was not specifically designed to address blended learning, but was instead originally developed to examine computer-mediated online learning. Interestingly, the complex adaptive blended learning system framework proposed by Wang et al. (2015 p. 382) is also not rooted in blended learning, but instead originates from physics, chemistry and mathmatics where it was used to learn about “dynamic and non-linear systems such as neural systems, ecologies, galaxies, and social systems”. In many ways, the establishment of an effective blended learning framework poses similar difficulties to the pursuit of conceptual clarity. There are conflicting views regarding the requirements of a framework and the articulation of these across the four blending levels (activity, course, programme, institutional) outlined by Bonk and Graham (2012) has yet to be described effectively, based on the literature reviewed in this study. Table 5 presents a combined matrix showing the framework factors presented by the literature identified in this systematic review. These factors were identified directly from the individual frameworks presented in each of the studies and from their expanded discussions. Two of the five titles described frameworks that address blended learning at the activity, course and programme level, while three (Adekola et al., 2017; Lim et al., 2019; Wang et al., 2015) suggested they have addressed blended learning from an institutional level. Reviewing the table, it is clear that the framework presented by Lim et al. (2019) is the most comprehensive in terms of considering the combined factors. Although they make a sound case for placing the curriculum (learning) at the centre of their framework, they have neglected the ethical and legal factors (accessibility, equity of access, copyright compliance) highlighted by Adekola et al. (2017) and more significantly the role of the student as co-creators identified by Mirriahi et al. (2015), Wang et al. (2015) and Adekola et al. (2017). Australasian Journal of Educational Technology, 2023, 39(2). 108 Table 5 Blended learning framework factors identified from existing literature Title Garrison & Vaughan (2008) Mirriahi et al. (2015) Wang et al. (2015) Adekola et al. (2017) Lim et al. (2019) Blending level Activity course and programme Activity course and programme Institutional Institutional Institutional Curriculum design x x x Teaching strategy and pedagogy x x x x x Technology integration x x x x x Student role x x x Learning support x x x Teacher role x x x x Teacher support x x x Institution role x x x Vision and policy x x Partnerships x x Physical infrastructure x x Ethical and legal x Evaluation & research x Each of the studies associated with the framework theme also discussed the implementation of the frameworks they offered. Adekola et al. (2017) discussed organisational preparedness for change, citing narratives around stakeholder roles and themes associated with a change management process. In addition to their blended learning framework, they also proposed a transitioning framework comprised of change agents, institutional considerations, organisational preparedness for change and stakeholder roles. Wang et al. (2015) demonstrated the use of their framework to evaluate blended learning research and practice. They suggested that making the institution a subsystem of their framework assists with elevating blending from course to institutional level. They also found that the relationship between teacher and institution was significant in terms of redefining their role to facilitator, adviser and promoter of learning and supporting this through professional development. Although they offered a smaller number of framework factors that did not address institutional-level blending, Mirriahi et al. (2015) did propose criteria indicators and standards to define quality of practice, thereby enabling opportunities for improvement and identification of professional development opportunities. The CoI framework developed by Garrison and Vaughan (2008) was the least prescriptive in terms of suggesting implementation methods, proposing the use of plenary sessions and workshops as vehicles to raise the profile of blended learning. In contrast to this, Lim et al. (2019) provided detailed strategic dimensions intended to complement each of the themes within their framework and support institutional implementation. Australasian Journal of Educational Technology, 2023, 39(2). 109 Figure 5. An institutional-level blended learning framework The review of literature associated with the framework theme suggests that there is clear difference between frameworks addressing blending at the activity, course and programme level and those that that address blending at the institutional level. Just like conceptual clarity, an institutional-level understanding requires a framework that is broader and provides an effective platform to support flexible delivery. Those that do not address institutional-level blended learning are effectively teaching models, used to guide the way specific subject matter is taught. As such, an institutional framework should be considered the space in which blended learning can take place, providing a framework with associated criteria and standards by which academic practice can be measured. The most effective blended learning framework would therefore address the full range of factors identified in this study; this should be supported by an institutional-level understanding of what each factor means and how these will be supported by all stakeholders (i.e., students, teachers, administrators). A conceptual illustration of an institutional-level framework informed by this review is presented in Figure 5. How should blended learning be implemented within universities? Although it is recognised that there is no single approach to blended learning, there is a desire from higher education institutions to seek a pathway that enables them to reshape their programme and course design, transform teaching methods and develop a blend that meets a broader learning and teaching need in the form of student access and tailored learning (Galvis, 2018; Smith & Hill, 2019). A strategy that addresses the institution as a whole is required for implementation at broad scale; however, difficulties lie in how this finds its way down to the underpinning course-level pedagogy and teaching activities undertaken inside and outside of the classroom (Graham et al., 2013). Although 10 studies were identified and aligned with the blended learning implementation theme in this review, not all of these had a clear focus on institutional leadership and decision making. Four studies referenced the same institutional adoption framework (Galvis, 2018; Graham et al., 2013; Porter et al., 2014; Porter & Graham, 2016), while two others targeted administrative considerations for successful implementation of blended learning (Vaughan, 2007; Vaughan et al., 2017). The remaining studies referenced a mix of framework factors with implementation and adoption recommendations that were Australasian Journal of Educational Technology, 2023, 39(2). 110 split across blending levels (activity, course, programme and institution). This was particularly evident in the systematic review carried out by Anthony et al. (2020), who investigated constructs and factors affecting blended learning uptake by students, lecturers and administrators, comparing the popularity of adoption models largely centred around the use of technology in teaching practices. The early work of Garrison and Kanuka (2004) also discussed implementation across the four blending levels and had many similar themes to the later work of Graham et al. (2013) when addressing organisational and leadership issues. Moskal et al. (2013) offered insight into the need for common language and highlighted the organisational benefits of blended learning in terms of scalability beyond the institution, arguing that the missing ingredient for implementation is often mid-level capacity needed within faculty to drive support at the course and programme level. Chaudhri and Gallant (2013, pp. 83–84) highlighted the potential for multiple approaches to implementation of blended learning, stating that there was no set recipe, but noted five critical factors required to support implementation: (1) selection of the right model of blended learning (2) redesign of curriculum to align with blended learning (3) redesign of assessment tools (4) training of academic staff (5) implementation of information technology infrastructure. These factors have many similarities to the strategy, structure, support adoption model proposed by Graham et al. (2013) and strengthens their argument for this thematic approach to implementation of blended learning. Although common themes have been identified in this review, it is also evident that there is no consistent approach to institutional implementation and the research surrounding it. Indeed Graham et al. (2013) reported that there was little data available to show how well blended learning has been integrated in higher education, largely due to the way it is defined and subsequently measured. The ability to clearly define what blended learning means, within an institution at the very least, is an important factor (Porter et al., 2014), and as Vaughan et al. (2017, p. 112) pointed out in their case study of evidence-based practice, “without a universal definition of blended learning there is no shared language by which the education field can describe the phenomena or address its opportunities and challenges”. Although the results of this review suggest that a universal definition is challenging, it does reinforce the importance of an institutional-level understanding of blended learning, detailing what it sets out to achieve and how adoption and effectiveness can be measured. Despite this mixed approach to implementation, it is clear that a means of measuring framework adoption is required to determine effectiveness and maturity within an institution. The work of Graham et al. (2013) is highly referenced throughout the literature identified in this study, and their 3-stage adoption framework (Table 6) clearly identifies levels of blended learning implementation as well as key strategy, structure and support issues that should be addressed at the institutional level (Table 7). Table 6 Implementation stages for blended learning adoption (adapted from Graham et al., 2013, p. 7) Stage Description Stage 1: Awareness/exploration No institutional blended learning strategy, limited awareness and support for individual faculty exploring ways in which they may employ blended learning techniques in their classes Stage 2: Adoption/early implementation Institutional adoption of blended learning strategy and experimentation with new policies and practices to support its implementation Stage 3: Mature implementation/growth Well-established blended learning strategies, structure and support that are integral to university operations Australasian Journal of Educational Technology, 2023, 39(2). 111 Graham et al.’s (2013) 3-stage adoption framework was tested by Porter et al. (2014) and subsequently expanded by Porter and Graham (2016) using Rogers’ diffusion of innovations theory to develop a numerical approach that evaluates levels of blended learning uptake within an institution. Although useful, many of the criteria used to calculate the uptake scores (i.e., reducing in class time, recording lectures, using online quizzing and exams) could be, in part, a result of institutional efficiencies and reactions to address student accessibility rather than the result of a well-implemented and purposefully designed approach to blended learning. Porter and Graham (2016) also suggested that early majority adopters respond to compelling evidence; however, their measurement approach largely targeted functions carried out at the activity and course level of blended learning associated with the support aspect in the Graham et al. (2013) adoption framework. Their findings suggested that sufficient infrastructure and support were key drivers of adoption; however, the relationship between these measures and the strategy and structure themes are not quite as clear and therefore may not provide a sufficient level of evidence to sway the early majority or provide an effective measure of adoption across all three adoption themes. Table 7 Implementation categories for blended learning adoption (adapted from Graham et al., 2013, p. 7) Theme Description Strategy Issues relating to the overall design of blended learning, such as definition of blended learning, forms of advocacy, degree of implementation, purposes of blended learning, and policies surrounding it Structure Issues relating to the technological, pedagogical, and administrative framework facilitating the blended learning environment, including governance, models, scheduling structures, and evaluation Support Issues relating to the manner in which an institution facilitates the implementation and maintenance of its blended learning design, incorporating technical support, pedagogical support, and faculty incentives Galvis (2018, p. 29) also emphasised the requirement for faculty support identified by Porter and Graham (2016) and proposed that institutional adoption is intrinsically related to adoption by faculty and neither is effective without the other. In terms of measuring the maturity of blended learning, they proposed a 6- point approach using a radar graph which provides a useful way to visualise the results; however, their evaluation primarily targets course and programme-level blending. Although they did not provide a full evaluation model, Galvis (2018) did also point out the benefit of an institutional educational model that provides clarity around expectations and responsibilities related to the implementation of blended learning, suggesting that this enables the development of indicators that can be used for evaluation. This concept of using criteria and descriptive standards to evaluate framework implementation was also highlighted as an important missing factor by Mirriahi et al. (2015) in their framework, suggesting that this provides opportunities to benchmark academic practice in a consistent way. Although there has been a good deal of research undertaken since the work of Graham et al. (2013), their thematic and staged approach continues to provide an appropriate level of rigour around the implementation and adoption of blended learning in higher education. Despite this prescriptive approach, there remains a need to tailor this to the needs of individual institutions, with specific reference to the factors that comprise an effective blended learning framework in association with an institutional-level definition that is developed through effective engagement with key stakeholders. Conclusion Although blended learning is well documented, its understanding is varied and use within higher education does not always result in successful outcomes for the student, teacher or institution. This paper has used a conceptual framework comprising conceptual clarity, blended frameworks and institutional- level implementation of blended learning to evaluate current literature with the overarching goal of defining an effective approach in higher education. Australasian Journal of Educational Technology, 2023, 39(2). 112 In practice, the final application of blended learning is an individual endeavour and relies heavily on implementation by the teacher who is ultimately the subject matter expert, along with those directly supporting them to carry out robust educational design. As a result, any institutional strategy should carefully consider the multifaceted roles of students, academics and administrators and their ongoing importance in the teaching and learning dynamic to develop an institutional definition that provides both support and flexibility. To this end, it may be most effective to focus less on the ratios of blended learning used and more on the purpose and roles (humanness), technology intervention or lack thereof (fidelity), including when and how they are used (space and time) in order to describe their direct impact on individual student learning and develop a community approach to blended learning. To implement this effectively, an institutional strategy requires a vision and associated policy that enables the development of a definition and framework that fosters a common understanding of what success will look like. This vision needs to be championed at all levels to facilitate a cultural shift that clearly acknowledges the combined framework factors driving the teaching and learning process. From a practical standpoint, this acknowledgement requires the development of a significant and evolving structure that supports a holistic approach promoting teacher development and student support, tools and infrastructure, technology, and pedagogy, as well as an effective way to measure adoption. This structure helps to create an environment that encourages the greatest level of uptake and informs the development of a systematic approach that continues to influence the future direction of blended learning within an institution. If executed well a successful institutional framework has the ability to facilitate blended learning adoption that results in a paradigm shift altering the very nature of teaching, enabling students to learn in a manner that best suits their needs and provides flexibility and scalability opportunities for the institution. Opportunities exist for research to investigate the effectiveness of the combined institutional-level blended learning framework presented in this study and to establish descriptive criteria that can be used to evaluate the maturity and effectiveness of implementation within higher education. Limitations Although some articles were added to the review due to regular citation, others may have been omitted. This is largely a factor of implementing the search strategy used in this study; however, further analysis could be undertaken to identify additional work that informs the refinement of framework factors and indicators that can be used for evaluation of adoption. Author contributions Author 1: Conceptualisation, Investigation, Analysis, Writing – original draft, Writing – review and editing; Author 2: Data curation, Analysis, Writing – review and editing. References Adekola, J., Dale, V. H. M., & Gardiner, K. (2017). Development of an institutional framework to guide transitions into enhanced blended learning in higher education. Research in Learning Technology, 25. https://doi.org/10.25304/rlt.v25.1973 Alammary, A., Sheard, J., & Carbone, A. (2014). Blended learning in higher education: Three different design approaches. Australasian Journal of Educational Technology, 30(4), 440–454. https://doi.org/10.14742/AJET.693 Anthony, B., Kamaludin, A., Romli, A., Raffei, A. F. M., Phon, D. N. A. L. E., Abdullah, A., & Ming, G. L. (2020). Blended learning adoption and implementation in higher education: A theoretical and systematic review. Technology, Knowledge and Learning, 27, 531–578. https://doi.org/10.1007/S10758-020-09477-Z https://doi.org/10.25304/rlt.v25.1973 https://doi.org/10.14742/AJET.693 https://doi.org/10.1007/S10758-020-09477-Z Australasian Journal of Educational Technology, 2023, 39(2). 113 Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014). A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Journal of Computing in Higher Education, 26(1), 87–122. https://doi.org/10.1007/s12528-013-9077-3 Bliuc, A. M., Goodyear, P., & Ellis, R. A. (2007). Research focus and methodological choices in studies into students’ experiences of blended learning in higher education. The Internet and Higher Education, 10(4), 231–244. https://doi.org/10.1016/J.IHEDUC.2007.08.001 Bonk, C. J., & Graham, C. R. (2012). The handbook of blended learning: Global perspectives, local designs. John Wiley & Sons. Caner, M. (2012). The definition of blended learning in higher education. In P. Anastasiades (Ed.), Blended learning environments for adults: Evaluations and frameworks (pp. 19–34). IGI Global. https://doi.org/10.4018/978-1-4666-0939-6.ch002 Chaudhri, A. A., & Gallant, M. (2013). Critical success factors for the implementation of blended learning in higher education a case study from New Zealand. In Proceedings of the 2013 International Conference on Current Trends in Information Technology (pp. 82–85). IEEE. https://doi.org/10.1109/CTIT.2013.6749482 Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed method (5th ed.). SAGE Publications. Galvis, Á. H. (2018). Supporting decision-making processes on blended learning in higher education: Literature and good practices review. International Journal of Educational Technology in Higher Education, 15(1), 1–38. https://doi.org/10.1186/S41239-018-0106-1 Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education, 7(2), 95–105. https://doi.org/10.1016/J.IHEDUC.2004.02.001 Garrison, D. R., & Vaughan, N. D. (2008). Blended learning in higher education: Framework, principles, and guidelines. John Wiley & Sons. https://doi.org/10.1002/9781118269558 Graham, C. R., Woodfield, W., & Harrison, J. B. (2013). A framework for institutional adoption and implementation of blended learning in higher education. The Internet and Higher Education, 18, 4– 14. https://doi.org/10.1016/J.IHEDUC.2012.09.003 Green, H. E. (2014). Use of theoretical and conceptual frameworks in qualitative research. Nurse Researcher, 21(6), 34–38. https://doi.org/10.7748/nr.21.6.34.e1252 Halverson, L. R., Graham, C. R., Spring, K. J., Drysdale, J. S., & Henrie, C. R. (2014). A thematic analysis of the most highly cited scholarship in the first decade of blended learning research. The Internet and Higher Education, 20, 20–34. https://doi.org/10.1016/J.IHEDUC.2013.09.004 Huang, R., Ma, D., & Zhang, H. (2008). Towards a design theory of blended learning curriculum. In J. Fong, R. Kwan, & L. Wang (Eds.), Lecture notes in computer science: Vol. 5169. Hybrid learning and education (pp. 66–78). Springer. https://doi.org/10.1007/978-3-540-85170-7_6 Khan, B. H. (Ed.). (2005). Managing e-learning: Design, delivery, implementation and evaluation. IGI Global. Lim, C. P., Wang, T., & Graham, C. (2019). Driving, sustaining and scaling up blended learning practices in higher education institutions: A proposed framework. Innovation and Education, 1(1), 1–12. https://doi.org/10.1186/S42862-019-0002-0 Medina, L. C. (2018). Blended learning: Deficits and prospects in higher education. Australasian Journal of Educational Technology, 34(1). https://doi.org/10.14742/AJET.3100 Mirriahi, N., Alonzo, D., & Fox, B. (2015). A blended learning framework for curriculum design and professional development. Research in Learning Technology, 23. https://doi.org/10.3402/rlt.v23.28451 Moskal, P., Dziuban, C., & Hartman, J. (2013). Blended learning: A dangerous idea? The Internet and Higher Education, 18, 15–23. https://doi.org/10.1016/J.IHEDUC.2012.12.001 Norberg, A., Dziuban, C. D., & Moskal, P. D. (2011). A time-based blended learning model. On the Horizon, 19(3), 207–216. https://doi.org/10.1108/10748121111163913 Oliver, M., & Trigwell, K. (2005). Can ‘blended learning’ be redeemed? E-learning and Digital Media, 2(1), 17–26. https://doi.org/10.2304/ELEA.2005.2.1.17 Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson , A., Lalu, https://doi.org/10.1007/s12528-013-9077-3 https://doi.org/10.1016/J.IHEDUC.2007.08.001 https://doi.org/10.4018/978-1-4666-0939-6.ch002 https://doi.org/10.1109/CTIT.2013.6749482 https://doi.org/10.1186/S41239-018-0106-1 https://doi.org/10.1016/J.IHEDUC.2004.02.001' https://doi.org/10.1002/9781118269558 https://doi.org/10.1016/J.IHEDUC.2012.09.003 https://doi.org/10.7748/nr.21.6.34.e1252 https://doi.org/10.1016/J.IHEDUC.2013.09.004 https://doi.org/10.1007/978-3-540-85170-7_6 https://doi.org/10.1186/S42862-019-0002-0 https://doi.org/10.14742/AJET.3100 https://doi.org/10.3402/rlt.v23.28451 https://doi.org/10.1016/J.IHEDUC.2012.12.001 https://doi.org/10.1108/10748121111163913 https://doi.org/10.2304/ELEA.2005.2.1.17 Australasian Journal of Educational Technology, 2023, 39(2). 114 M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S.,… Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Systematic Reviews, 10, Article 89. https://doi.org/10.1186/S13643-021-01626-4 Pima, J. M., Odetayo, M., Iqbal, R., & Sedoyeka, E. (2018). A thematic review of blended learning in higher education. International Journal of Mobile and Blended Learning, 10(1), Article 1. https://doi.org/10.4018/IJMBL.2018010101 Porter, W. W., & Graham, C. R. (2016). Institutional drivers and barriers to faculty adoption of blended learning in higher education. British Journal of Educational Technology, 47(4), 748–762. https://doi.org/10.1111/BJET.12269 Porter, W. W., Graham, C. R., Spring, K. A., & Welch, K. R. (2014). Blended learning in higher education: Institutional adoption and implementation. Computers & Education, 75, 185–195. https://doi.org/10.1016/J.COMPEDU.2014.02.011 Singh, H. (2021). Building effective blended learning programs. In B. H. Khan, S. Affouneh, S. H. Salha, & Z. N. Khlaif (Eds.), Challenges and opportunities for the global implementation of e-learning frameworks (pp. 15–23). IGI Global. https://doi.org/10.4018/978-1-7998-7607-6.CH002 Smith, K., & Hill, J. (2019). Defining the nature of blended learning through its depiction in current research. Higher Education Research and Development, 38(2), 383–397). https://doi.org/10.1080/07294360.2018.1517732 Torrisi-Steele, G., & Drew, S. (2013). The literature landscape of blended learning in higher education: The need for better understanding of academic blended practice. International Journal for Academic Development 18(4), 371–383. https://doi.org/10.1080/1360144X.2013.786720 Vaughan, N. (2007). Perspectives on blended learning in higher education. International Journal on E- Learning, 6(1), 81–94 http://www.learntechlib.org/p/6310/ Vaughan, N., Reali, A., Stenbom, S., van Vuuren, M. J., & MacDonald, D. (2017). Blended learning from design to evaluation: International case studies of evidence-based practice. Online Learning, 21(3), 103–114. https://doi.org/10.24059/OLJ.V21I3.1252 Wang, Y., Han, X., & Yang, J. (2015). Revisiting the blended learning literature: Using a complex adaptive systems framework. Journal of Educational Technology & Society, 18(2), 380–393. https://www.jstor.org/stable/jeductechsoci.18.2.380 Young, J., Solomon, M. (2009). How to critically appraise an article. Nature Clinical Practice Gastroenterology & Hepatology, 6(2), 82–91. https://doi.org/10.1038/ncpgasthep1331 Corresponding author: Shaun McCarthy, shaun.mccarthy@adelaide.edu.au Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. Please cite as: McCarthy, S., & Palmer, E. (2023). Defining an effective approach to blended learning in higher education: A systematic review. Australasian Journal of Educational Technology, 39(2), 98- 114. https://doi.org/10.14742/ajet.8489 https://doi.org/10.1186/S13643-021-01626-4 https://doi.org/10.4018/IJMBL.2018010101 https://doi.org/10.1111/BJET.12269 https://doi.org/10.1016/J.COMPEDU.2014.02.011 https://doi.org/10.4018/978-1-7998-7607-6.CH002 https://doi.org/10.1080/07294360.2018.1517732 https://doi.org/10.1080/1360144X.2013.786720 http://www.learntechlib.org/p/6310/ https://doi.org/10.24059/OLJ.V21I3.1252 https://www.jstor.org/stable/jeductechsoci.18.2.380 https://doi.org/10.1038/ncpgasthep1331 mailto:shaun.mccarthy@adelaide.edu.au https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.8489 Introduction Methods Eligibility criteria Quality assessment Results Discussion How should we define blended learning within higher education? What is the most effective framework associated with blended learning in higher education? How should blended learning be implemented within universities? Conclusion Limitations Author contributions References