Australasian Journal of Educational Technology, 2019, 35(5). 29 Blended learning in large enrolment courses: Student perceptions across four different instructional models Ron Owston York University Dennis York University of Guelph Taru Malhotra York University Drawing on data from five large enrolment introductory courses in a public university, we compared students’ perceptions of blended learning on design, interaction, learning, and satisfaction in four different blended models. The models, which were the result of a course redesign initiative, had different combinations of face-to-face lectures, online sessions, and small group tutorial classes. Our findings suggest that students perceived courses with fully online lectures and in-class tutorials most positively on design and overall satisfaction, while those enrolled in courses with in-class lectures and in-class tutorials, supplemented by online discussions, felt most positively about interaction. Students perceived learning in the former courses more favourably than the latter, however the differences were not statistically significant. The least preferred model overall was the one that had in-class lectures and tutorials that alternated weekly between in-class and online sessions. Introduction At a time when public post-secondary institutions in most countries are facing funding cutbacks and rising class sizes, there is a need to rethink the design of large enrolment lecture courses (Bates, 2015). The redesign of courses using blended learning is an option that offers the possibilities of increased student satisfaction, improved learning and retention, and better utilisation of classroom space, while maintaining student face-to-face contact with peers and the instructor (Dziuban, Hartman, Juge, Moskal, & Sorg, 2006). Twigg (2000) recommended that for institutions to realise the greatest impact from their investment, they should focus course redesign efforts on large enrolment introductory courses that most students take. By doing so a redesign initiative will impact the greatest number of students. This study is situated at a public university whose school of fine arts heeded Twigg’s advice and redesigned five large enrolment introductory academic courses. The faculty decided to redesign the courses using four different blended learning models that had different combinations of face-to-face lectures, online sessions, and small group tutorial classes. Previously, all courses contained all components fully face-to-face. Our research goal was to compare students’ perceptions of their experiences in the courses across the four models, and to assess the strengths and limitations of the models for large enrolment introductory courses from the student perspective. Blended learning, in this study, is broadly conceived as the thoughtful and complementary integration of face-to-face and online technologies (Garrison & Vaughan, 2008). This definition is pedagogically helpful as it does not restrict blended learning to be a specific portion of time spent online as does the widely-cited Allen, Seaman, and Garrett (2007) definition, nor does it reference a specific blended model such as the flipped classroom. Although student perception of blended learning is a well-researched topic (Drysdale, Graham, Spring, & Halverson, 2013), we could not identify any published research comparing student perceptions across various blended models within the same academic discipline. Researchers to date have focused on: (a) comparing student perceptions of blended learning to those of traditional lectures or to fully online courses (e.g., Larson & Sung, 2009); (b) exploring perceptions of blended courses across a wide variety of disciplines and academic levels (e.g., Dziuban et al., 2006); (c) examining student perceptions in flipped classrooms (e.g., Ramnanan & Pound, 2017); (d) studying the relationship of perceptions to achievement (e.g., López-Pérez, Pérez-López, & Rodríguez-Ariza, 2011); or (e) investigating perceptions of proportion of time spent online in a blended course (e.g., Owston & York, 2018). This study responds to calls for deeper and more nuanced research about the design characteristics of blended learning that promote Australasian Journal of Educational Technology, 2019, 35(5). 30 student success, rather than research that compares blended learning to other modes of instruction (Bernard, Borokhovski, Schmid, Tamin, & Abrami, 2014; Zhao, Lei, Yan, Lai, & Tan, 2005). Our investigation of student perceptions in large enrolment courses is also of significance because of the unique pedagogical challenges these courses pose. There is no agreed upon quantitative definition in the literature of what constitutes a large enrolment class, as largeness depends much upon the academic nature and goals of the course. For example, a fine arts studio course would be considered large with 30 students, whereas an introductory psychology course with that many students would be judged as small. We concur with Maringe and Sing’s (2014) definition of largeness as “any class where the numbers of students pose both perceived and real challenges in the delivery of quality and equal learning opportunities to all students in that classroom” (p. 763). Not surprisingly, as class sizes increase there tends to be a reduction in the amount and intensity of interactions among students and students with the instructor. This in turn can lead to passivity and anonymity, decreasing engagement with course content, higher levels of absenteeism and dwindling attendance especially toward the end of semester, noise and distraction as students arrive late and leave early from class, more off-task behavior, and overall low student satisfaction (Mulryan-Kyne, 2010). Despite these seemingly insurmountable difficulties, lectures still have a valuable role to play in higher education and should not be disparaged as they have been by some critics (Friesen, 2011). French and Kennedy (2017) argue that lectures can provide an overarching view of and bring structure to a subject; allow the lecturer the opportunity to build a complex argument over time and model how an expert approaches a topic; motivate, stimulate, and challenge students; promote listening and note-taking skills; be cost-effective and efficient for teaching at scale; and develop a shared communal sense of understanding. They see integration of interactive learning opportunities, such as tutorial classes or online activities, either in lieu of some of, or in addition to, the lecture time, to mitigate many of the shortcomings of the lecture. Therefore, the study of various combinations of face-to-face and online lectures and tutorials can contribute to our understanding of the role blended learning can play in redesigning traditional large enrolment courses. Conceptual framework Our study is framed by four key dimensions of student perceptions of blended learning that are fundamental to the development of blended courses: design, interaction, learning, and student satisfaction of blended learning in higher education. These four factors are discussed next. Design of blended courses Blended course designers are typically faced with three unique instructional dilemmas: (a) what learning activities are suitable for online and face-to-face components; (b) what is the relationship between the face- to-face and online components; and (c) how course time is distributed between online and face-to-face components of the course (McGee & Reis, 2012). Beyond these, all other instructional decisions for designing blended courses are essentially the same as those necessary for designing effective fully face-to- face or fully online courses. Current research suggests that the nature of the learning activities is more important to students than their delivery mode (Banerjee, 2011; Manwaring, Larsen, Graham, Henrie, & Halverson, 2017). A growing body of literature shows that students tend to prefer activities that offer choices and promote social interaction. Students also appear to value online lectures, problem-solving exercises, the use of various tools, and online discussions (Bueno-Alastuey & Lopez-Perez, 2014; Hung & Chou, 2015; Wai & Seng, 2015). On the other hand, students feel less engaged when they have trouble downloading lectures, use intricate tools that restrict their online participation, or the instructor’s presence is not established or is lacking (Wai & Seng, 2015). Once the learning activity is designed (or selected) in alignment with the learning objectives of the course, the designer determines which mode of delivery fits best to support student learning (Garrison & Vaughan, 2008; McGee & Reis, 2012; Stein & Graham, 2014). When the modes of delivery are integrated purposefully, students tend to perceive value in the blended course and recognise the advantages of both face-to-face and online activities (Gerbic, 2010). For example, Gerbic noted that online asynchronous discussions allowed more time for thoughtful and higher-quality dialogue and enabled them to influence the direction of the conversation. At the same time, students endorsed the opportunity to connect with their peers in the classroom environment. On the other hand, when the relationship between the two modes was Australasian Journal of Educational Technology, 2019, 35(5). 31 weak, students reported that online discussions became isolated and irrelevant to their learning. The recent study by Herbert, Velan, Pryor, and Kumar (2017) demonstrated that students were highly satisfied with their blended learning experience when interactive face-to-face sessions were complemented with synchronous video sessions. Students also reported in this study that they perceived online activities were meaningful and engaging. Little research has been done on the proportion of time to be allocated to each mode and its benefits. A few studies (Asarta & Schmidt, 2015; Farley, Jain, & Thomson, 2011) reported that students opt to attend about half of the face-to-face classes, when they are given the choice of attending lectures or accessing lecture recordings online. More recently, Owston and York (2018) found a small but significantly more positive perception of student experiences in the courses, with 36% to 50% time spent online over those with lower proportions online. Interaction in blended courses A growing body of literature shows that increased interaction has become a critical success factor in blended courses. Moore (1989) classified learner interaction into three categories: student-to-student (S-S), student to teacher (S-T), and student to content (S-C). Since then researchers have sought to explore the effectiveness of these three types of interaction and found that interaction is contextually grounded in such factors as field of study, course organisation, student experience, course interaction expectations, and level of instructor’s facilitation (Castaño-Muñoz, Duart, & Sancho-Vinuesa, 2014; Means, Toyama, Murphy, & Baki, 2013). In a meta-analysis of 74 individual studies, Bernard et al. (2009) found that all three types of interaction had a significant impact on academic achievement with an overall weighted average effect size of 0.38. Both S-C and S-S interactions were equally strong compared to S-T interaction suggesting that the latter interaction was “less effective, more difficult to implement consistently, or [they] provided less added value than either S-S or S-C” (p. 1259). Later work by Castaño-Muñoz et al. (2014) found that the purposeful integration of online discussions and collaborative project work tend to increase significantly both S-S and S-T interactions. Similarly, Kurucay and Inan (2017) found that students engaged in group projects scored significantly higher on graded assignments than students who interacted with the learning material independently. Other research explored interaction in relation to students’ perceived learning and satisfaction. While several researchers (Kuo, Walker, Schroder, & Belland, 2014; Kurucay & Inan, 2017; Sher, 2009) found that both S-S and S-T interactions were significantly related to perceived learning and satisfaction, Chang and Smith (2008) found that all three types of interactions were significant predictors of satisfaction. The variations in the outcomes of these studies may be caused by subject matter, instructional design decisions, and the level of facilitation required for both online and face-to-face learning contexts (Dennen, 2005; McGee & Reis, 2012). As blended pedagogy matured as a field, Gerbic (2010) expanded upon the concept of interaction to develop a framework that builds upon the strengths of both face-to-face and online interactions: (a) presence or absence of visual and aural cues, (b) synchronous and asynchronous timing, and (c) speech- and text-based communication. For example, in-class time may be used to introduce students to online activities to ensure they fully grasp the instructional directions they need to follow. Further, the instructor can prioritise the role of face-to-face small groups to build their confidence and establish social presence, resulting in increased interaction during online activities. On the other hand, online time can be utilised to prepare students for more effective discussions in the classroom. This approach to interaction seemed to be a balance between social activity and instructor’s presence and scaffolding in both online and face-to-face environments (Garrison, Anderson, & Archer, 2001; Palinscar, 1998). Learning in blended courses Undergraduate students tend to perceive that they learn better in blended courses than in face-to-face courses across a variety of undergraduate subject areas and class sizes (Castle & McGuire, 2010; Herbert et al., 2017; Melton, Bland, & Chopak-Foss, 2009; Owston, York, & Murtha, 2013). After analysing 4,038 student self-assessments of learning, Castle and McGuire (2010) found that undergraduate students preferred onsite courses that had online components, that is, blended courses. Melton et al. (2009) reported that students in general health courses scored significantly higher on a learning self-assessment scale than Australasian Journal of Educational Technology, 2019, 35(5). 32 those in traditional courses (N = 177). Two of the above cited studies also specifically mentioned large enrolment classes. A large enrolment (N = 264) pathology course studied by Herbert et al. (2017) required students to spend 50% of course time working on online modules. Students strongly agreed on an end of course survey that the modules promoted deeper understanding, made learning more efficient, motivated them to learn about the topic, and individualised their learning. Owston et al. (2013) found that students in 20 different large undergraduate courses in a wide range of subjects preferred learning in the blended mode over other traditional courses they had taken. Students’ beliefs about learning are supported by six independent meta-analyses that have yielded a median effect size of 0.37 favouring blended learning (Bernard et al., 2014; Çirak Kurt, Yildirim, & Cücük 2018; Means et al., 2013; Spanjers et al., 2015; Vo, Zhu, & Diep, 2017; Zhao et al., 2005). These studies, summarised in Table 1, include 583 individual effect sizes, and indicate a range from a small to large effects according to Cohen’s (1988) criterion. Table 1 Summary of meta-analytic studies of blended learning achievement effect sizes Study Effect size (Hedges g+ or as indicated) No. of effect sizes included in study (k) Notes Bernard et al. (2014) .33 117 Comparison of blended to face-to- face Çirak Kurt et al. (2018) 1.04* 32 Comparison of blended to face-to- face Means et al. (2013) .35 50 Comparison of blended to face-to- face Spanjers et al. (2015) .34 30 Comparison of blended to face-to- face with objective performance measures Vo et al. (2017) .39 51 Overall effect of STEM and non- STEM courses combined compared to face-to-face Zhao (2005) .49* 51 Compared to face-to-face when 60% - 80% of course online * Cohen’s d (considered equivalent to Hedges g+ for samples k > 20) From the above studies three moderating variables emerged that are of interest to the present study. Vo et al. (2017) found a larger effect size for STEM courses (g+ = .496) compared to non-STEM courses (g+ = .210), which suggests that STEM disciplines may be more amenable to blended learning. Spanjers et al. (2015) reported that objective end-of-course assessment measures yielded a larger effect size (g+ = .34) than subjective measures (g+ = .27). Lastly, proportion of time spent online appears to be related to achievement. Zhao et al. (2005) found that students in courses with 60% to 80% of time spent online achieved higher than those who spent more time online. Both Bernard et al. (2014) and Means et al. (2013) included this variable in their analyses but did not find it to be a statistically significant factor, although they reported that a larger proportion of time spent online in blended courses approached significance. More recently, Owston and York (2018) found that students performed better in blended courses when 36% to 50% of time was spent online versus lower proportions of online time. In addition to these studies, meta-analytic research on flipped classrooms in the health professions has yielded a significant effect size of 0.33 over traditional classroom learning (Hew & Kwan, 2018). Moreover, large scale longitudinal research has found higher success and lower withdrawal rates for blended learning than face-to-face or fully online courses (Moskal, Dziuban, & Hartman, 2013). This research was conducted over a span of 16 years and involved more than 1 million students at the University of Central Florida enrolled in a variety of academic programs across campus. Australasian Journal of Educational Technology, 2019, 35(5). 33 Student satisfaction in blended courses Research indicates that students generally prefer blended over face-to-face and fully online courses. A recent EDUCAUSE Center for Applied Research survey of undergraduate students indicated that almost three-quarters of students prefer a mix of online and face-to-face components, while the remainder are about equally split between preferring face-to-face and fully online courses (Brooks, 2016). This trend has remained consistent over the 4 years that the US national survey has been conducted. Other studies found similar results where over 70% of students were satisfied with their blended courses (Dziuban et al., 2006; Owston, Garrison, & Cook, 2006). They noted that flexibility, convenience, reduced travel time, and face- to-face interaction were most appealing features of blended learning. In their meta-analysis of 30 studies, Spanjers et al. (2015) found that students slightly favoured blended formats with a small average effect size (g+ = .11, p < .05). The body of evidence suggests that students often rank the blended delivery of the course significantly higher over its face-to-face or web-enhanced counterpart on satisfaction, teaching effectiveness, and their overall course expectations (Forte & Root, 2011; Kumrow, 2007; Vernadakis, Giannousi, Tsitskari, Antoniou, & Kioumourtzoglou, 2012). Lim, Morris, and Kupritz (2007) found that students were clearer on the method of learning in the blended course as it allows for opportunities to seek clarification about course requirements, though some studies (e.g., Larson & Sung, 2009) were inconclusive whether the course delivery methods influence student satisfaction. More recently, researchers have examined whether there is a relationship between satisfaction and other measures of learning, such as achievement and disposition. They found that high achievers tend to view blended learning more positively than low achievers (Owston et al., 2013), and upper-year students, compared to first-year students, appear to be more engaged in blended courses (Vargas-Madriz & Nocente, 2016). At the same time, undergraduate students, on the whole, favour blended learning more than graduate students (Castle & McGuire, 2010). Other research suggests that positive satisfaction with blended learning is more likely to nurture a positive attitude towards learning (López-Pérez et al., 2011). Research questions The goal of this study was to investigate, from the perspective of students, four different blended learning instructional models in large enrolment undergraduate classes with different combinations of face-to-face lectures, online sessions, and small group tutorial classes. Four key factors were chosen to compare the models: design, interaction, learning, and student satisfaction. From the above overview of research on these factors several trends are evident. Designers face challenges of how to divide course components between online and face-to-face, how online and face-to-face activities can be integrated, and what proportion of time of a course should be spent online. Decisions about these factors appear to be mediated by intended course learning outcomes and course content area. Research recognises the need for student- student, student-instructor, and student-content interactions to be present in a blended course, although there is no consensus on their relative importance. There is consensus that students tend to learn modestly better in blended courses compared to traditional lectures, however, students in STEM fields appear to perform better than those in non-STEM fields, and there may be a need to have a substantial portion of time spent online for the benefits to accrue. Lastly, students in blended courses appear on average to be more satisfied overall, nevertheless high performing students and undergraduate students tend to show higher satisfaction. Our specific research questions were: 1. How do student perceptions of four blended models, with different combinations of face-to-face lectures, online sessions, and small group tutorial classes, compare on design, interaction, learning, and overall satisfaction? 2. What are the strengths and limitations of the four models for large enrolment blended introductory courses? Australasian Journal of Educational Technology, 2019, 35(5). 34 Method Setting The study took place over a 3-year period at a large public comprehensive university located in a major urban city in Canada. The student body is very culturally and racially diverse, and a significant number of students are first generation in their family to attend university. The vast majority of students commute to campus, and approximately half of them work part time during the academic year. Undergraduate humanities and social sciences programs enrol the majority of the university’s students. The university’s school of fine arts embarked on a redesign initiative for five of its large (~300 enrolment) freshman introductory academic courses for non-majors: art, dance, film, music, and theatre. Art and dance were offered twice over the 3-year period of study, while theatre, music, and dance were offered three times. Fine arts students were required to take any two of the five courses that were not in their major area of study. Before the redesign courses were offered with a 2-hour lecture and a 1-hour tutorial class weekly for one semester. Four different blended models were used in the redesigned courses. 1. Blend CLTW (class lectures/tutorials/web-enhanced) had the normal in-class lectures and tutorials but the course was enhanced by online discussions. There was no reduction of face-to-face time under this model. The instructor retained the structure of the traditional course format and used Moodle to supplement in-class sessions with additional online activities to enhance students’ understanding of key concepts and to increase interaction among the participants. Additionally, Moodle was utilised to build a repository of course documents, reference materials, and complementary resources for students to support their learning. 2. Blend CLOT (class lectures/online tutorials) had in-class lectures with online tutorial classes. Students attended a weekly 2-hour lecture delivered by the instructor in the face-to-face format and participated in a fully online tutorial session facilitated by the teaching assistants (TAs). Students were also able to obtain help at informal drop-in sessions every other week. 3. Blend OLCT (online lectures/in-class tutorials) had asynchronous online lectures with in-class tutorials. Students viewed a weekly 2-hour online lecture delivered by the instructor via Moodle and attended a 1-hour face-to-face tutorial facilitated by TAs every week. 4. Blend CLHT (class lectures/hybrid tutorials) had in-class lectures and hybrid online/in-class tutorials. Students attended a weekly 2-hour lecture delivered by the instructor in the face-to-face format and participated in a 1-hour TA-guided tutorial that alternated weekly between face-to-face and online with Moodle. In-class lectures were given in a large hall that accommodated all students. The instructor who taught online, although an experienced lecturer, had little previous experience with teaching in this medium, but had participated in several faculty workshops on the topic. Face-to-face tutorial classes had 25 to 30 students in smaller classrooms. All tutorial classes were led by TAs, all of whom were graduate students with little or no previous teaching experience. The focus of the five courses was on different urban fine art practices: visual arts, film, theatre, dance, and music. Student assignments involved a mixture of research and writing assignments, as well as creative and analytical work. At the weekly class meetings, the instructor supported lectures with audio-visual presentations, including film screenings as well as discussions with invited guests. TA helped students further explore the course material via online or face-to-face discussions, readings, film analyses and reviews, and various assignments. TA attended lectures (or view the online lectures in one course), however the instructor had no direct involvement in the tutorial sessions. Out-of-class experiential learning activities were essential to all the courses and were part of course grade assessment. For instance, the experiential activities included: (a) visiting art events, institutions, and architecture in the city; (b) watching assigned films online at students’ own convenience; or (c) attendance at theatrical performances. The course instructors clearly stated the expectations with regards to attendance of face-to-face sessions and participation in the online activities. For instance, in one of the courses, the course syllabus stated that “being part of an intellectual community requires that you attend both lectures and tutorials regularly (whether online or in person), read required readings in advance and with care, and involve yourself in discussions in ways that will help you and other students to learn.” All online components made use of the Australasian Journal of Educational Technology, 2019, 35(5). 35 Moodle course management system. In addition, a few courses also used quizzes, Twitter, and other technologies to promote student engagement with the content. Instruments Students were given a questionnaire based on Owston et al. (2013) that contained 20 Likert scale questions relating to four factors about their experience in their course: design (Q1 to Q5), interaction (Q6 to Q11), learning (Q12 to Q16), and satisfaction (Q17 to Q20). Participants responded on a 5-point scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree) to statements about the four factors relative to other traditional lecture courses they had taken. Cronbach’s alpha for the scales were 0.8, 0.9, 0.7, and 0.7 for design, interaction, learning, and satisfaction respectively, which are generally considered as acceptable levels (Tavakol & Dennick, 2011). The study was approved by the university’s research ethics board and students voluntarily consented to participate. Administration of the questionnaire occurred in class toward the end of the semester, and all students in attendance at that time agreed to participate. A total of 2081 valid responses were obtained which represented a 54% response rate. The number of responses for each blend was: CLTW = 261, CLOT = 691, OLCT = 969, and CLHT = 160. Data analysis For each of the four scales – design, interaction, learning, and satisfaction – a multivariate analyses of variance (MANOVA) was carried out, with scale questions as dependent variables and the Blend model as the between subjects’ factor. If significant differences between the Blend model were found, follow-up analyses of variance (ANOVAs) and post-hoc analyses were conducted to determine where the differences occurred. Prior to undertaking the analyses, tests were undertaken to determine whether statistical assumptions were violated. All data were analysed using the IBM Statistical Package for Social Sciences (SPSS) version 24. Study limitations Several limitations to this study are evident which may affect interpretation of the results. Courses were taught by different instructors and each tutorial class had different TAs. Although the faculty were experienced educators, the TAs had little, if any, prior teaching experience. Our comparisons of the blend learning models do not take these variables into consideration. Even though the five courses we investigated were organised around a common theme of art in the city, the academic content was different for each course. Lastly, available resources did not permit us to observe during lectures or study online interactions, so we are not able assess how effectively interactions were facilitated. Results Tables 2 through 5 show the mean differences, standard errors, significance of the differences, and the 95% confidence intervals of each of four scales. The MANOVA tests indicated significant differences for the Blend model for each scale: design, Pillai’s Trace V = .040, F(15, 4959) = 4.42, p < .001, partial eta squared ƞ2 = .013; interaction, V = .070, F(18, 5889) = 7.84, p < .001, ƞ2 = .023; learning, V = .045, F(15, 4890) = 4.92, p < .001, ƞ2 = .015; and satisfaction, V = .104, F(12, 6024) = 8.11, p < .001, ƞ2 = .035. Since all MANOVAs were significant, follow-up ANOVAs were conducted for each scale question. For design, significant differences for the Blend model were found for all five questions (Q1 to Q5) p < .05, and partial eta squared, ƞ2, ranged from .015 to .017; for interaction significant differences for the Blend model were found for all six questions (Q6 to Q11) p < .001 and ƞ2 ranged from .019 to .027; for learning significant differences for the Blend model were found for three of the five questions (Q12 to Q16) p < .016 and ƞ2 ranged from .005 to .016; and for satisfaction significant differences for the Blend model were found for all four questions (Q17 to Q20) p < .015 and ƞ2 ranged from .005 to .077. Thus, out of the 20 questions, the Blend model was not significant for only two learning variables: Q12 (I was more engaged in the course), and Q15 (Course helped me develop better communication skills). According to Richardson (2011), partial eta squared values of .01, .06, and .14 represent small, medium, and large effects respectively, hence effect sizes ranged from very small to medium. Australasian Journal of Educational Technology, 2019, 35(5). 36 Research question 1 To address the first research question, post hoc analyses were done for the significant variables using the Games-Howell test, as homogeneity of variance could not be assumed. Results of these analyses are given in Tables 2 to 5 for the design, interaction, learning, and satisfaction questions respectively, which are summarised next. Design questions Post hoc analyses of the design questions given in Table 2 indicate that 14 significant differences were found. Students in Blend OLCT rated all five questions significantly higher than Blend CLHT, and more favourably than Blend CLOT on four out of five questions: Q1 (online and face-to-face course components enhanced each other), Q2 (find information easily on Moodle), Q3 (online resources useful), and Q4 (clear course expectations). Other significant differences found were: Blend CLTW higher than Blend CLOT on Q1 and higher than Blend CLHT on Q2, Q4, and Q5 (reliable online technology); and Blend CLOT higher than Blend CLHT on Q5. Thus, overall Blend OLCT was rated the highest by students on design questions followed by Blend CLTW, while Blend CLHT was the lowest. Moreover, Blend OLCT was rated significantly higher than the other blends on Q1, which concerns the important design feature of blending of the online and face-to-face components. Table 2 Post hoc multiple comparisons for design questions Dependent variable Blend format (I) Blend format (J) Mean difference (I-J) Std error Sig. 95% Confidence interval Lower bound Upper bound Q1. The online and face- to-face components of this course enhanced each other. CLTW CLOT .277* .100 .03 .018 .537 OLCT -.093 .082 .663 -.305 .117 CLHT .246 .127 .213 -.081 .575 CLOT OLCT -.371* .079 0 -.575 -.168 CLHT -.031 .125 .995 -.354 .292 OLCT CLHT .340* .110 .013 .053 .627 Q2. I was able to find course information easily at the Moodle site. CLTW CLOT .149 .096 .407 -.099 .398 OLCT -.079 .077 .736 -.278 .120 CLHT .362* .121 .016 .048 .677 CLOT OLCT -.228* .075 .014 -.424 -.033 CLHT .213 .120 .294 -.099 .525 OLCT CLHT .441* .106 0 .167 .716 Q3. The online resources at the Moodle site were useful. CLTW CLOT .182 .090 .182 -.050 .414 OLCT -.081 .069 .646 -.26 .097 CLHT .279 .112 .066 -.012 .571 CLOT OLCT -.263* .072 .002 -.449 -.076 CLHT .097 .114 .83 -.199 .394 OLCT CLHT .360* .099 .002 .103 .618 Q4. The course expectations were clearly communicated. CLTW CLOT .082 .086 .776 -.140 .305 OLCT -.115 .067 .323 -.289 .059 CLHT .307* .118 .05 .000 .615 CLOT OLCT -.197* .071 .03 -.382 -.013 CLHT .225 .121 .248 -.088 .538 OLCT CLHT .423* .108 .001 .141 .704 Australasian Journal of Educational Technology, 2019, 35(5). 37 Note. The error term is mean square (error) = 1.262. * The mean difference is significant at the .05 level. Interaction questions For interaction, 22 significant differences were found as shown in Table 3. Thirteen of these significant differences were due to Blend CLTW students rating interaction questions more positively than the other blends. On three of those questions – Q7 (amount of interactions with other students, Q8 (quality of interactions with other students), and Q9 (I felt connected to other students) – Blend CLTW was significantly higher than all of the other blends. Blend OLCT students rated six questions significantly higher than Blend CLOT and three questions higher than Blend CLHT. In no other blend did students rate interaction questions higher than Blends CLTW or OLCT. Table 3 Post hoc multiple comparisons for interaction questions Dependent variable Blend format (I) Blend format (J) Mean difference (I-J) Std error Sig. 95% Confidence interval Lower bound Upper bound Q6. I was more likely to ask questions in this course. CLTW CLOT .144 .080 .269 -.061 .349 OLCT -.160 .073 .128 -.349 .029 CLHT .380* .109 .003 .098 .664 CLOT OLCT -.305* .057 .000 -.452 -.157 CLHT .237 .100 .084 -.021 .494 OLCT CLHT .541* .095 .000 .296 .786 Q7. The amount of my interaction with other students in this course increased. CLTW CLOT .580* .083 .000 .366 .794 OLCT .378* .077 .000 .179 .577 CLHT .577* .124 .000 .256 .899 CLOT OLCT -.202* .061 .005 -.358 -.046 CLHT -.003 .115 1.000 -.300 .295 OLCT CLHT .199 .111 .276 -.088 .486 Q8. the quality of my interaction with other students in this course was better. CLTW CLOT .551* .078 .000 .350 .753 OLCT .310* .073 .000 .122 .498 CLHT .343* .119 .023 .034 .651 CLOT OLCT -.241* .057 .000 -.388 -.094 CLHT -.209 .110 .236 -.495 .077 OLCT CLHT .032 .107 .990 -.245 .309 Q5. The technology used for online portions of this course was reliable. CLTW CLOT .025 .092 .993 -.214 .264 OLCT -.040 .076 .951 -.237 .156 CLHT .447* .121 .002 .133 .762 CLOT OLCT -.065 .073 .808 -.255 .124 CLHT .422* .120 .003 .112 .733 OLCT CLHT .488* .107 0 .209 .767 Australasian Journal of Educational Technology, 2019, 35(5). 38 Q9. I felt connected to other students in this course. CLTW CLOT .469* .082 .000 .258 .680 OLCT .270* .076 .003 .073 .467 CLHT .519* .118 .000 .215 .822 CLOT OLCT -.199* .057 .003 -.347 -.052 CLHT .050 .106 .966 -.226 .325 OLCT CLHT .249 .102 .073 -.016 .513 Q10. The amount of my interaction with the instructor in this course increased. CLTW CLOT .402* .088 .000 .175 .628 OLCT .014 .084 .998 -.203 .230 CLHT .477* .124 .001 .157 .796 CLOT OLCT -.388* .061 .000 -.545 -.231 CLHT .075 .109 .903 -.209 .358 OLCT CLHT .463* .106 .000 .187 .738 Q11. The quality of my interaction with the instructor in this course was better. CLTW CLOT .265* .085 .010 .047 .483 OLCT -.076 .080 .778 -.283 .130 CLHT .300 .121 .067 -.014 .614 CLOT OLCT -.341* .062 .000 -.500 -.182 CLHT .035 .110 .989 -.250 .321 OLCT CLHT .376* .107 .003 .099 .653 Note. The error term is mean square (error) = 1.414. * The mean difference is significant at the .05 level. Learning questions The post hoc analysis given in Table 4 indicates that of the three significant differences found, Blend OLCT students perceived learning significantly higher than Blend CLOT students on two questions – Q13 (increased interest in material) and Q14 (improved understanding) – and higher than Blend CLHT on Q14. Blend OLCT Students rated all three learning questions higher than Blend CLTW, however the differences were not significant. Table 4 Post hoc multiple comparisons for learning questions Dependent variable Blend format (I) Blend format (J) Mean difference (I-J) Std error Sig. 95% Confidence interval Lower bound Upper bound Q13. Taking this course increased my interest in the material. CLTW CLOT .195 .097 .182 -.054 .444 OLCT -.168 .079 .146 -.371 .036 CLHT .049 .120 .978 -.262 .360 CLOT OLCT -.363* .075 .000 -.557 -.169 CLHT -.146 .118 .603 -.452 .159 OLCT CLHT .217 .104 .164 -.054 .487 Australasian Journal of Educational Technology, 2019, 35(5). 39 Q14. This course improved my understanding of key concepts. CLTW CLOT .094 .087 .703 -.130 .317 OLCT -.169 .069 .067 -.346 .008 CLHT .154 .108 .488 -.126 .434 CLOT OLCT -.263* .069 .001 -.439 -.086 CLHT .060 .108 .945 -.220 .340 OLCT CLHT .323* .094 .004 .078 .568 Q16. I had more opportunities in this course to reflect on what I have learned. CLTW CLOT -.218 .093 .093 -.459 .023 OLCT -.163 .072 .105 -.348 .022 CLHT -.298 .116 .051 -.597 .001 CLOT OLCT .054 .077 .894 -.143 .252 CLHT -.080 .119 .906 -.387 .226 OLCT CLHT -.135 .103 .556 -.401 .131 Notes. Q12 and Q15 are omitted because no significant overall effect for Blend was found for these questions. The error term is mean square (error) = 2.465. * The mean difference is significant at the .05 level. Satisfaction questions Table 5 indicates that 12 significant differences were found for satisfaction. Blend OLCT was rated higher by students than the other three blends on Q17 (I was satisfied) and on Q19 (allowed more flexibility); students also rated Blend OLCT higher than Blend CLHT on Q18 (I would take another course). On Q19 (more flexibility in personal schedule) and Q20 (reduced travel time to campus), Blend CLOT was higher than Blend CLTW, and on Q20, Blend CLHT was higher than Blend CLTW. Hence, Blend OLCT was rated most favourably overall by students on satisfaction. Table 5 Post hoc multiple comparisons for satisfaction questions Dependent variable Blend format (I) Blend format (J) Mean difference (I-J) Std error Sig. 95% Confidence interval Lower bound Upper bound Q17. Overall I am satisfied with this course. CLTW CLOT -.044 .077 .942 -.243 .155 OLCT -.229* .071 .008 -.413 -.045 CLHT .074 .120 .927 -.237 .385 CLOT CLTW .044 .077 .942 -.155 .243 OLCT -.185* .058 .008 -.335 -.035 OLCT CLHT .303* .109 .030 .021 .586 Q18. Given the opportunity I would take another course in the future that has both online and face-to-face components. CLTW CLOT -.011 .086 .999 -.233 .212 OLCT -.120 .081 .447 -.329 .088 CLHT .207 .133 .405 -.137 .552 CLOT OLCT -.110 .064 .321 -.275 .056 CLHT .218 .124 .295 -.103 .538 OLCT CLHT .328* .120 .035 .016 .639 Australasian Journal of Educational Technology, 2019, 35(5). 40 Q19. This course allowed me to have more flexibility in my personal schedule. CLTW CLOT -.838* .087 .000 -1.062 -.613 OLCT -1.123* .077 .000 -1.321 -.924 CLHT -.790* .124 .000 -1.112 -.469 CLOT OLCT -.285* .066 .000 -.454 -.115 CLHT .048 .118 .978 -.257 .352 OLCT CLHT .332* .111 .016 .046 .619 Q20. This course allowed me to reduce my total travel time to campus each week. CLTW CLOT -.903* .102 .000 -1.165 -.642 OLCT -.783* .095 .000 -1.028 -.538 CLHT -.736* .153 .000 -1.131 -.341 CLOT OLCT .120 .082 .457 -.090 .331 CLHT .167 .145 .656 -.208 .543 OLCT CLHT .047 .140 .987 -.317 .411 Note. The error term is mean square (error) = 1.162. * The mean difference is significant at the .05 level. Research question 2 Research question 2 was to assess the strengths and limitations of the four models. The assessment was done by synthesising the above results. Table 6 provides a summary of the findings given in Tables 2, 3, 4, and 5. It shows the number of times each blend was significantly higher on questions than other blends for each factor. The data suggest that students clearly preferred Blend OLCT on design and satisfaction compared to the other blends. As can be seen in the table, this blend was significantly higher than the other blends 9 and 8 times on design and satisfaction respectively. With regard to learning, students preferred Blend OLCT as it was significantly higher than other blends 3 times; however, when referring back to Table 5, one can see that Blend OLCT was significantly higher than Blends CLOT and CLHT, but not significantly higher than Blend CLTW. On interaction Blend CLTW was highest followed by Blend OLCT. In summary, Blend OLCT, which had online lectures and face-to-face tutorials was the most preferred, while Blend CLTW, which had online discussions and web resources that supplemented the normal in- class lectures and tutorials, was second most preferred. Blend CLOT, which had in-class lectures and online tutorials, was third by only a narrow margin behind Blend CLTW. Blend CLHT, with lectures and hybrid tutorials, appeared to be the least desirable model according, as it was not significantly higher than any other blend on any questions. Table 6 Number of times a blend was significantly higher than another blend for each factor Blend Factor Design Interaction Learning Satisfaction CLHT 0 0 0 0 CLOT 1 0 0 0 CLTW 4 13 0 4 OLCT 9 8 3 8 Total number of significant differences 14 22 3 12 Discussion and conclusions Traditionally, higher education institutions have integrated tutorial classes into large enrolment lecture courses to improve student learning of lecture content (Ramsden, 2003). As more institutions adopt blended learning for traditional lecture courses, the question arises about how to redesign these courses using the blended model. Our research contributes to the understanding of this issue from the perspective of student preferences. To date, the literature has not addressed the question of what combinations of online and face- Australasian Journal of Educational Technology, 2019, 35(5). 41 to-face lectures, online sessions, and tutorials students prefer. We compared four different blended models in redesigned, large enrolment fine arts courses that had previously used a traditional mix of weekly lectures and tutorials. The comparison was made on four key factors – design, interaction, learning, and satisfaction. Our finding that courses with online lectures and in-class tutorials (Blend OLCT) were rated significantly higher than the other three models on design and satisfaction, and higher than two out of three other models on learning was perhaps not too surprising. Blend OLCT was the only one studied with fully online lectures. Given the large size of the courses in this study, students normally sat passively in a tiered lecture hall listening to their instructor and watching multimedia presentations on the screen at the front of the class. Placing essentially the same content online, with instructor narration, as was done in this study, would have allowed students to more carefully study and review content, something that was not available to them in the traditional version of the courses. At the same time, students had the opportunity to discuss the content with and receive individual assistance from TAs during tutorial classes. Thus, Blend OLCT became somewhat like a traditional course that includes lecture capture technology. Research has shown that students highly value lecture capture courses, as they can review course content multiple times and view classes that they missed, while at the same time, maintain interaction with their instructor and peers (Karnad, 2013; Soong, Chan, Cheers, & Hu, 2006; Traphagan, Kucsera, & Kishi, 2009). Blend OLCT could also be construed of as a form of flipped classroom, although the amount of time spent in tutorials was only 1 hour per week instead of the normal 5 hours. Likewise, students appear to be overwhelmingly in favour of flipped classrooms despite some misgivings about the amount of work required with this model (Ramnanan & Pound, 2017). Thus, our finding about students preferring Blend OLCT is consistent with research about student perceptions in related areas. That students in the traditional web-enhanced courses (Blend CLTW) rated interaction significantly higher than Blend OLCT can be explained quite readily. We believe that this was because students in Blend CLTW were able to interact with the TAs in the same way as the Blend OLCT students, however they were also able to interact with the TAs online in the supplementary discussions. In other words, Blend CLTW students simply had more opportunities to interact with peers, the instructor, and TAs. Our research also bears out what others have found about interaction: namely that students prefer to learn socially, and whether it is interaction with peers or the instructor does not seem to matter as much as the fact that there are many occasions to converse (Chang & Smith, 2008; Gerbic, 2010; Sher, 2009). The finding that Blend OLCT was slightly higher than Blend CLTW on learning, but not significantly so, may be because students in Blend CLTW spent more time on task than the other blends, hence students might have felt that they simply learned more. Means et al. (2013) speculated that time on task was one of the reasons students tend to learn more in blended courses. An interesting finding was that overall preferences for Blend CLTW, which had in-class lectures and tutorials, were only slightly higher than Blend CLOT, which had in-class lectures but online tutorials, with only two exceptions. First, as noted above, Blend CLTW students rated interaction higher than all other modes. Second, on two questions about freedom in their personal schedules and reduced commute to campus, Blend CLOT students were higher than Blend CLTW. This finding is consistent with one of the often-cited advantages of blended learning about the convenience it provides students (Moskal et al., 2013). Other than these two exceptions, what is evident is that students did not indicate a clear preference for either online or face-to-face tutorial sessions. Courses with lectures and hybrid tutorials (Blend CLHT) were the least preferred overall. Students in this group did not rate any questions higher than students in other groups. This was likely due to the blended nature of the tutorials. Students in Blend CLHT alternated between meeting in person for tutorials one week, followed by online tutorials the other week. It seems as though this course format did not provide meaningful flexibility and convenience to students, nor did sufficient benefit accrue from half of their time being spent in either fully online tutorials or fully face-to-face tutorials. We conclude from this study that, relative to the other blending models, courses with online lectures and in-class tutorials (Blend OLCT) were most preferred overall. This study was focused on large enrolment courses, so the current findings infer that institutions may be able to free up large lecture halls for other purposes if they adopt this model, as suggested by Dziuban et al. (2006). An implication for further research is to assess whether students in this model are likely to achieve higher than other blend designs. There is some promising evidence that they may. Blend OLCT had an equivalent of two-thirds of the course online, Australasian Journal of Educational Technology, 2019, 35(5). 42 one third face-to-face. Zhao et al. (2005) found that courses with proportions of time online of this magnitude led to higher student achievement, however it should be noted that their meta-analysis was undertaken when online technologies were not as advanced as today. Moreover, Bernard et al. (2014) and Means et al. (2013) both speculated that when greater than half of a blended course is online, performance will be higher than when proportionally less time is spent online. Hence, we recommend that future research be conducted to examine student achievement in blended courses where lectures are online and face-to- face tutorial sessions are held. We further recommend that studies on blended learning should take into account what happens in the online and face-to-face sessions, and how the two components are integrated. This can be accomplished through observing in classrooms, interviewing instructors, and analysing online interactions. Data of this nature can then be used to help explain any observed differences in student preferences and performance. References Allen, I. E., Seaman, J., & Garrett, R. (2007). Blending in: The extent and promise of blended education in the United States. Needham, MA: Sloan Consortium. Retrieved from http://files.eric.ed.gov/fulltext/ED529930.pdf Asarta, C. J., & Schmidt, J. R. (2015). The choice of reduced seat time in a blended course. Internet and Higher Education, 27, 24-31. https://doi.org/10.1016/j.iheduc.2015.04.006 Banerjee, G. (2011). Blended environments: Learning effectiveness and student satisfaction at a small college in transition. Journal of Asynchronous Learning Networks, 15(1), 8-19. Retrieved from https://eric.ed.gov/?id=EJ918215 Bates, A. W. (2015). Teaching in a digital age: Guidelines for designing teaching and learning. Burnaby, BC: Simon Fraser University. Retrieved from https://www.tonybates.ca/teaching-in-a-digital-age/ Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., Tamim, R. M., Surkes, M. A., & Bethel, E. C. (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research, 79(3), 1243–1289. https://doi.org/10.3102/0034654309333844 Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014). A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Journal of Computing in Higher Education, 26(1), 87-122. https://doi.org/10.1007/s12528-013-9077-3 Brooks, D. C. (2016). ECAR study of undergraduate students and information technology 2016. Louisville, CO: ECAR. Retrieved from https://library.educause.edu/resources/2016/6/~/media/files/library/2016/10/ers1605.pdf Bueno-Alastuey, M. C., & López Pérez, M. V. (2014). Evaluation of a blended learning language course: Students’ perceptions of appropriateness for the development of skills and language areas. Computer Assisted Language Learning, 27(6), 509-527. https://doi.org/10.1080/09588221.2013.770037 Castaño-Muñoz, J., Duart, J. M., & Sancho-Vinuesa, T. (2014). The internet in face-to-face higher education: Can interactive learning improve academic achievement? British Journal of Educational Technology, 45(1), 149-159. https://doi.org/10.1111/bjet.12007 Castle, S. R., & McGuire, C. J. (2010). An analysis of student self-assessment of online, blended, and face-to-face learning environments: Implications for sustainable education delivery. International Education Studies, 3(3), 36-40. https://doi.org/10.5539/ies.v3n3p36 Chang, S. H. H., & Smith, R. A. (2008). Effectiveness of personal interaction in a learner-centered paradigm distance education class based on student satisfaction. Journal of Research on Technology in Education, 40(4), 407-426. https://doi.org/10.1080/15391523.2008.10782514 Çirak Kurt, S., Yildirim, İ., & Cücük, E. (2018). The effects of blended learning on student achievement: A meta-analysis study. Hacettepe University Journal of Education,33(3), 776-802. https://doi.org/10.16986/HUJE.2017034685 Cohen, J. (1988). Statistical power analyses for the social sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Dennen, V. P. (2005). From message posting to learning dialogues: Factors affecting learner participation in asynchronous discussion. Distance Education, 26(1), 127-148. https://doi.org/10.1080/01587910500081376 Drysdale, J. S., Graham, C. R., Spring, K. J., & Halverson, L. R. (2013). An analysis of research trends in dissertations and theses studying blended learning. Internet and Higher Education, 17, 90-100. https://doi.org/10.1016/j.iheduc.2012.11.003 http://files.eric.ed.gov/fulltext/ED529930.pdf https://doi.org/10.1016/j.iheduc.2015.04.006 https://eric.ed.gov/?id=EJ918215 https://www.tonybates.ca/teaching-in-a-digital-age/ https://doi.org/10.3102/0034654309333844 https://doi.org/10.1007/s12528-013-9077-3 https://library.educause.edu/resources/2016/6/%7E/media/files/library/2016/10/ers1605.pdf https://doi.org/10.1080/09588221.2013.770037 https://doi.org/10.1111/bjet.12007 https://doi.org/10.5539/ies.v3n3p36 https://doi.org/10.1080/15391523.2008.10782514 https://doi.org/10.16986/HUJE.2017034685 https://doi.org/10.1080/01587910500081376 https://doi.org/10.1016/j.iheduc.2012.11.003 Australasian Journal of Educational Technology, 2019, 35(5). 43 Dziuban, C., Hartman, J., Juge, F., Moskal, P., & Sorg, S. (2006). Blended learning enters the mainstream. In C. J. Bonk and C.R. Graham (Eds.), Handbook of blended learning: Global perspectives, local designs (pp. 195-206). San Francisco, CA: Pfeiffer. Farley, A., Jain, A., & Thomson, D. (2011). Blended learning in finance: Comparing student perceptions of lectures, tutorials and online learning environments across different year levels. Economic Papers: Journal of Applied Economics and Policy, 30(1), 99-108. https://doi.org/10.1111/j.1759- 3441.2010.00094.x Forte, J. A., & Root, V. (2011) To ITV or not to ITV: A comparison of hybrid and web-enhanced approaches to teaching a macro-course in human behavior in the social environment. Journal of Human Behavior in the Social Environment, 21(1), 82-96. https://doi.org/10.1080/10911359.2011.535732 French, S., & Kennedy, G. (2017). Reassessing the value of university lectures. Teaching in Higher Education, 22(6), 639-654. https://doi.org/10.1080/13562517.2016.1273213 Friesen, N. (2011). The lecture as a transmedial pedagogical form: A historical analysis. Educational Researcher, 40(3), 95-102. https://doi.org/10.3102/0013189X11404603 Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7-23. https://doi.org/10.1080/08923640109527071 Garrison, D. R., & Vaughan, N. D. (2008). Blended learning in higher education: Framework, principles, and guidelines. San Francisco, CA: Jossey-Bass. Gerbic, P. (2010). Getting the blend right in new learning environments: A complementary approach to online discussions. Education and Information Technologies, 15(2), 125-137. https://doi.org/10.1007/s10639-009-9100-5 Herbert, C., Velan, G. M., Pryor, W. M., & Kumar, R. K. (2017). A model for the use of blended learning in large group teaching sessions. BMC Medical Education, 17(1), 1-11. https://doi.org/10.1186/s12909-017-1057-2 Hew, K. F., & Lo, C. K. (2018). Flipped classroom improves student learning in health professions education: A meta-analysis. BMC Medical Education, 18(38), 1-12. https://doi.org/10.1186/s12909- 018-1144-z Hung, M. L., & Chou, C. (2015). Students’ perceptions of instructors’ roles in blended and online learning environments: A comparative study. Computers & Education, 81, 315-325. https://doi.org/10.1016/j.compedu.2014.10.022 Karnad, A. (2013). Student use of recorded lectures: A report reviewing recent research into the use of lecture capture technology in higher education, and its impact on teaching methods and attendance. London: London School of Economics and Political Science. Retrieved from http://eprints.lse.ac.uk/50929/1/Karnad_Student_use_recorded_2013_author.pdf Kumrow, D. E. (2007). Evidence-based strategies of graduate students to achieve success in a hybrid web-based course. Journal of Nursing Education, 46(3), 140–145. Kuo, Y. C., Walker, A. E., Schroder, K. E., & Belland, B. R. (2014). Interaction, internet self-efficacy, and self-regulated learning as predictors of student satisfaction in online education courses. Internet and Higher Education, 20, 35-50. https://doi.org/10.1016/j.iheduc.2013.10.001 Kurucay, M., & Inan, F. A. (2017). Examining the effects of learner-learner interactions on satisfaction and learning in an online undergraduate course. Computers & Education, 115, 20-37. https://doi.org/10.1016/j.compedu.2017.06.010 Larson, D. K., & Sung, C. H. (2009). Comparing student performance: Online versus blended versus face-to-face. Journal of Asynchronous Learning Networks, 13(1), 31-42. Retrieved from https://files.eric.ed.gov/fulltext/EJ837556.pdf Lim, D. H., Morris, M. L., & Kupritz, V. W. (2007). Online vs. blended learning: Differences in instructional outcomes and learner satisfaction. Journal of Asynchronous Learning Networks, 11(2), 27-42. Retrieved from https://files.eric.ed.gov/fulltext/EJ842695.pdf López-Pérez, M. V., Pérez-López, M. C., & Rodríguez-Ariza, L. (2011). Blended learning in higher education: Students’ perceptions and their relation to outcomes. Computers & Education, 56(3), 818- 826. https://doi.org/10.1016/j.compedu.2010.10.023 Manwaring, K. C., Larsen, R., Graham, C. R., Henrie, C. R., & Halverson, L. R. (2017). Investigating student engagement in blended learning settings using experience sampling and structural equation modeling. Internet and Higher Education, 35, 21-33. https://doi.org/10.1016/j.iheduc.2017.06.002 https://doi.org/10.1111/j.1759-3441.2010.00094.x https://doi.org/10.1111/j.1759-3441.2010.00094.x https://doi.org/10.1080/10911359.2011.535732 https://doi.org/10.1080/13562517.2016.1273213 https://doi.org/10.3102/0013189X11404603 https://doi.org/10.1080/08923640109527071 https://doi.org/10.1007/s10639-009-9100-5 https://doi.org/10.1186/s12909-017-1057-2 https://doi.org/10.1186/s12909-018-1144-z https://doi.org/10.1186/s12909-018-1144-z http://eprints.lse.ac.uk/50929/1/Karnad_Student_use_recorded_2013_author.pdf https://doi.org/10.1016/j.iheduc.2013.10.001 https://doi.org/10.1016/j.compedu.2017.06.010 https://files.eric.ed.gov/fulltext/EJ837556.pdf https://files.eric.ed.gov/fulltext/EJ842695.pdf https://doi.org/10.1016/j.compedu.2010.10.023 https://doi.org/10.1016/j.iheduc.2017.06.002 Australasian Journal of Educational Technology, 2019, 35(5). 44 Maringe, F., & Sing, N. (2014). Teaching large classes in an increasingly internationalising higher education environment: Pedagogical, quality and equity issues. Higher Education, 67(6), 761-782. https://doi.org/10.1007/s10734-013-9710-0 McGee, P., & Reis, A. (2012). Blended course design: A synthesis of best practices. Journal of Asynchronous Learning Networks, 16(4), 7-22. Retrieved from https://files.eric.ed.gov/fulltext/EJ982678.pdf Means, B., Toyama, Y., Murphy, R. F., & Baki, M. (2013). The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record, 115(3), 1-47. Retrieved from https://www.sri.com/work/publications/effectiveness-online-and-blended-learning- meta-analysis-empirical-literature Melton, B. F., Bland, H. W., & Chopak-Foss, J. (2009). Achievement and satisfaction in blended learning versus traditional general health course designs. International Journal for the Scholarship of Teaching and Learning, 3(1), 1-13. Retrieved from https://digitalcommons.georgiasouthern.edu/cgi/viewcontent.cgi?article=1155&context=ij-sotl Mulryan-Kyne, C. (2010). Teaching large classes at college and university level: Challenges and opportunities. Teaching in Higher Education, 15(2), 175-185. https://doi.org/10.1080/13562511003620001 Moskal, P., Dziuban, C., & Hartman, J. (2013). Blended learning: A dangerous idea? Internet & Higher Education, 18, 15-23. https://doi.org/10.1016/j.iheduc.2012.12.001 Moore, M. G. (1989). Editorial: Three types of interaction. American Journal of Distance Education, 3(2), 1-7. https://doi.org/10.1080/08923648909526659 Owston, R. D., Garrison, D. R., & Cook, K. (2006). Blended learning at Canadian universities: Issues and practices. In C. J. Bonk, & C. R. Graham (Eds.), The handbook of blended learning: Global perspectives, local designs (pp. 338–350). San Francisco, CA: Pfeiffer. Owston, R. D., & York, D. N. (2018). The nagging question when designing blended courses: Does the proportion of time devoted to online activities matter? Internet and Higher Education, 36, 22-32. https://doi.org/10.1016/j.iheduc.2017.09.001 Owston, R. D., York, D. N., & Murtha, S. (2013). Student perceptions and achievement in a university blended learning strategic initiative. Internet and Higher Education, 18, 38-46. https://doi.org/10.1016/j.iheduc.2012.12.003 Palincsar, A. S. (1998). Social constructivist perspectives on teaching and learning. Annual Review of Psychology, 49(1), 345-375. Ramnanan, C. J., & Pound, L. D. (2017). Advances in medical education and practice: Student perceptions of the flipped classroom. Advances in Medical Education and Practice, 8, 63–73. https://doi.org/10.2147/AMEP.S109037 Ramsden, P. (2003). Learning to teach in higher education (2nd ed.). New York, NY: Routledge. Richardson, J. T. E. (2011). Eta squared and partial eta squared as measures of effect size in educational research. Educational Research Review, 6(2), 135–147. https://doi.org/10.1016/j.edurev.2010.12.001 Sher, A. (2009). Assessing the relationship of student-instructor and student-student interaction to student learning and satisfaction in web-based online learning environment. Journal of Interactive Online Learning, 8(2), 102-120. Soong, S. K. A., Chan, L. K., Cheers, C., & Hu, C. (2006). Impact of video recorded lectures among students. In L. Markauskaite, P. Goodyear, & P. Reimann (Eds.), Proceedings of the 23rd Annual ASCILITE Conference (pp. 789–793). Sydney: Sydney University Press. Retrieved from http://www.ascilite.org/conferences/sydney06/proceeding/pdf_papers/p179.pdf Spanjers, I. A. E., Könings, K. D., Leppink, J., Verstegen, D. M. L., de Jong, N., Jeroen Katarzyna, C., J. G., & Merriënboer, V. (2015). The promised land of blended learning: Quizzes as a moderator. Educational Research Review, 15, 59-74. https://doi.org/10.1016/j.edurev.2015.05.001 Stein, J., & Graham, C. R. (2014). Essentials for blended learning: A standards-based guide. New York, NY: Routledge. Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach's alpha. International Journal of Medical Education, 2, 53-55. https://doi.org/10.5116/ijme.4dfb.8dfd Traphagan, T., Kucsera, J. V., & Kishi, K. (2009). Impact of class lecture webcasting on attendance and learning. Educational Technology Research & Development, 58(1), 19–37. https://doi.org/10.1007/s11423-009-9128-7 Twigg, C. A. (2000). Course readiness criteria: Identifying targets of opportunity for large-scale redesign. EDUCAUSE Review, 35(3), 41-49. Retrieved from http://www.thencat.org/Articles/Course_Redesign.pdf https://doi.org/10.1007/s10734-013-9710-0 https://files.eric.ed.gov/fulltext/EJ982678.pdf https://www.sri.com/work/publications/effectiveness-online-and-blended-learning-meta-analysis-empirical-literature https://www.sri.com/work/publications/effectiveness-online-and-blended-learning-meta-analysis-empirical-literature https://digitalcommons.georgiasouthern.edu/cgi/viewcontent.cgi?article=1155&context=ij-sotl https://doi.org/10.1080/13562511003620001 https://doi.org/10.1016/j.iheduc.2012.12.001 https://doi.org/10.1080/08923648909526659 https://doi.org/10.1016/j.iheduc.2017.09.001 https://doi.org/10.1016/j.iheduc.2012.12.003 https://doi.org/10.2147/AMEP.S109037 https://doi.org/10.1016/j.edurev.2010.12.001 http://www.ascilite.org/conferences/sydney06/proceeding/pdf_papers/p179.pdf https://doi.org/10.1016/j.edurev.2015.05.001 https://doi.org/10.5116/ijme.4dfb.8dfd https://doi.org/10.1007/s11423-009-9128-7 http://www.thencat.org/Articles/Course_Redesign.pdf Australasian Journal of Educational Technology, 2019, 35(5). 45 Wai, C. C., & Seng, E. L. K. (2015). Measuring the effectiveness of blended learning environment: A case study in Malaysia. Education and Information Technologies, 20(3), 429-443. https://doi.org/10.1007/s10639-013-9293-5 Vernadakis, N., Giannousi, M., Tsitskari, E., Antoniou, P., & Kioumourtzoglou, S. (2012). Comparison of student satisfaction between traditional and blended technology course offerings in physical education. Turkish Online Journal of Distance Education, 13(1), 137-147. Retrieved from https://files.eric.ed.gov/fulltext/EJ976936.pdf Vargas-Madriz, L. F., & Nocente, N. (2016). Student engagement and satisfaction between different undergraduate blended learning courses. Proceedings of E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2016, 1443-1448. Chesapeake, VA: Association for the Advancement of Computing in Education. Retrieved from https://www.learntechlib.org/p/174090 Vo, M. H., Zhu, C., & Diep, A. N. (2017). The effect of blended learning on student performance at course-level in higher education: A meta-analysis. Studies in Educational Evaluation, 53, 17-28. https://doi.org/10.1016/j.stueduc.2017.01.002 Zhao, Y., Lei, J., Yan, B., Lai, C., & Tan, H. S. (2005). What makes the difference? A practical analysis of research on the effectiveness of distance education. Teachers College Record, 107(8), 1836-1884. Retrieved from https://eric.ed.gov/?id=EJ694412 Corresponding author: Ron Owston, rowston@edu.yorku.ca Australasian Journal of Educational Technology © 2019. Please cite as: Owston, R., York, D., & Malhotra, T. (2019). Blended learning in large enrolment courses: Student perceptions across four different instructional models. Australasian Journal of Educational Technology, 35(5), 29-45. https://doi.org/10.14742/ajet.4310 https://doi.org/10.1007/s10639-013-9293-5 https://files.eric.ed.gov/fulltext/EJ976936.pdf https://www.learntechlib.org/p/174090 https://doi.org/10.1016/j.stueduc.2017.01.002 https://eric.ed.gov/?id=EJ694412 mailto:rowston@edu.yorku.ca https://doi.org/10.14742/ajet.4310 Introduction Conceptual framework Design of blended courses Interaction in blended courses Learning in blended courses Student satisfaction in blended courses Research questions Method Setting Instruments Data analysis Study limitations Results Research question 1 Research question 2 Discussion and conclusions References