Australasian Journal of Educational Technology, 2021, 37(4). 51 Flipping a classroom with a three-stage collaborative instructional model (3-CI) for graduate students Feng-Kuang Chiang, Zhenhua Wu Department of Educational Technology, College of Education, Shanghai Normal University, China The flipped classroom is an innovative and increasingly popular pedagogical approach in higher education. It emphasises student learning responsibility, deeper learning, differentiated instruction and more efficient use of class time. However, despite its increasing popularity across disciplines, few studies have elaborated on strategies for implementing a flipped classroom beyond its essential elements. The present study thus proposed a three- stage collaborative instructional model (3-CI), an extension of the classic flipped classroom model. A case study approach was adopted to investigate 3-CI’s effectiveness through students’ perceptions in Research Methods in Educational Technology, with 29 graduate students. Results show that 3-CI increases participants’ satisfaction, engagement and collaboration. Furthermore, 3-CI design strategies, which emphasise collaboration and student-centredness, can help college educators to incorporate the flipped approach into their teaching practice. Implications for practice or policy: • Instructor-student collaborative partnerships in the flipped classroom contribute to students’ deeper engagement and understanding. • Instructors must be cognisant of their more important role in the flipped classroom in providing structural support to the students, which includes maintaining mutual interaction, building a social learning environment and facilitating knowledge construction. • Instructors need to pay attention to the quality, length and content of lecture videos and reading material; quizzes after watching the video can check and reinforce students’ understanding. Keywords: flipped classroom, graduate students, collaboration, instructional model, student perceptions Introduction With the growing sophistication of the 21st century workplace, more is demanded from the education system, and higher education, in particular, is facing heightened scrutiny with regard to programme effectiveness and student learning outcomes (Elken & Tellmann, 2019; Jacob & Gokbel, 2017). Studies on cognition and education have noted significant learning improvements when students are actively engaged in the learning process and are required to exercise higher-order thinking, such as reasoning, critical thinking and problem-solving (L. Chen et al., 2015; Mayer, 2002). Student-centred learning environments, which encourage such active learning, can improve student engagement, satisfaction and achievement. In particular, the flipped classroom is a student-centred pedagogical approach that fosters student responsibility towards their learning, promotes student–teacher interactions and provides more opportunities for cooperation (Arnold-Garza, 2014; Chiang & Chen, 2017; O’Flaherty & Phillips, 2015; Zaka et al., 2019); as such, it has been receiving increased attention (Bond, 2020). The flipped classroom is defined as students learning lecture material, which is provided digitally as well as in print form, before class, and class time is reallocated for homework and practical application of knowledge (Van Alten, Phielix, Janssen, & Kester, 2019). The flipped classroom was popularised by two high school chemistry teachers in Colorado (Bergmann & Sams, 2012), and it has since been widely used in primary and secondary education (Arnold-Garza, 2014; Bond, 2020; O’Flaherty & Phillips, 2015). However, recent years have witnessed a strong growth of interest in the application of flipped classroom in higher education and a surge of publications (Lundin et al., 2018; Van Alten et al., 2019). Colleges have embraced flipped learning as it promotes active learning and increases students’ engagement with the material (Karabulut-Ilgu et al., 2018). The use of digital technology in the flipped classroom provides students with varied ways to access course content, including instructor pre-recorded videos, massive open Australasian Journal of Educational Technology, 2021, 37(4). 52 online courses platforms, YouTube clips or TED talks, before coming to the class (K. Chen & Chuang, 2016; Gilboy et al., 2014; Mason et al., 2013). Audio narration with screen sharing is also suitable for presenting the thought processes underlying complex concepts (Zaka et al., 2019). For students, worksheets and online quizzes can be used as formative assessment to address their misunderstanding and provide instant feedback, thus increasing the effectiveness of the pedagogy (Van Alten et al., 2019). Pre-class reading material, whether printed or electronic, is usually assigned as a crucial supplementary learning resource (Chiang & Chen, 2017, Lundin et al., 2018; Van Alten et al., 2019). The aim of the pedagogical arrangement of students receiving preliminary exposure to material prior to class is to achieve the most efficient use of limited class time, allowing for greater focus on more engaging teaching and learning elements (Arnold-Garza, 2014; Kim et al., 2014). Specifically, by moving the traditional passive lecture to outside class time, teachers are able to spend more class time clarifying and analysing concepts, monitoring student performance, providing personalised feedback and organising activities that involve higher-order thinking and the practical application of knowledge (Arnold-Garza, 2014; Boucher et al., 2013). The flipped classroom thus improves learning outcomes through differentiated instruction and greater student–teacher interaction (Bond, 2020; Kim et al., 2014; Strayer, 2012), and deeper learning happens when students are working collaboratively to generate meaning. The flipped classroom has been applied in all major academic domains, including humanities, social science, natural science and formal science (Van Alten et al., 2019), with STEM (engineering, computer science mathematics) and health education (medical, pharmaceutical, nursing) being influential as implementation and research contexts. (Betihavas et al., 2016; F. Chen et al., 2017; Karabulut-Ilgu et al., 2018; Lundin et al., 2018). It is often regarded as a “promising” pedagogical approach (F. Chen et al., 2017, p. 591; Van Alten et al., 2019, p. 2) and studies have presented evidence of improved student engagement, satisfaction and performance in the flipped classroom (Bond, 2020; O’Flaherty & Phillips, 2015; Seery, 2015). Bond (2020) proposed a student engagement model and concluded that more than 90% of flipped classroom literature in schools reports at least one aspect of behavioural, affective or cognitive engagement. With regard to learning outcomes and student satisfaction, the effect is not as significant on the whole and there is great variation in effect size between studies (Betihavas, et al., 2016; Karabulut-Ilgu et al., 2018; Van Alten et al., 2019). In some flipped classrooms, students achieved significantly better learning indications of performance, while in others, they performed even worse than in a regular classroom. In the same vein, some students displayed very positive attitudes, and others adopted negative views in different studies. A major implication arising from these studies is that careful attention should be paid to the design and implementation of flipped learning, so as to fully exploit the benefits of the pedagogy (Karabulut-Ilgu et al., 2018; Van Alten et al., 2019). As O’Flaherty and Phillips (2015) pointed out, although college educators recognise the value of the flipped classroom, some teachers lack a full understanding of the flipped pedagogy and are short of effective ways to translate the concept into practice, thus limiting the potential of this pedagogical innovation to facilitate active learning. The design of the flipped classroom does not involve a simple reversal of content delivery order between lectures and homework, nor the mere replacement of lectures with videos. Instead, it requires a complete redesign of the curriculum and student learning experiences, in which instructors must develop a more student-centred teaching process that coherently links learning activities before, during and after class. More research on flipped classroom design to strengthen the theory underlying the flipped classroom and its implementation in higher education is needed (Kim et al., 2014). Such effort is illustrated by a few studies: Bergmann and Sams (2012) proposed several technology-related design considerations; Brame (2013) suggested four design principles for the flipped classroom; Kim et al. (2014) extended Brame’s (2013) suggestions and proposed a design framework and nine design principles based on their exploratory research of three undergraduate flipped classrooms; and Van Alten et al. (2019) offered several general guideline for increasing the effectiveness of a flipped classroom. However, few studies have detailed, beyond the essential elements, how the flipped classroom can be innovatively designed to improve the student learning experience (Kim et al., 2014; O’Flaherty & Phillips, 2015). Because of this scarcity of detailed and empirically validated implementation strategies, educators have little support in converting the conceptual framework of the flipped classroom into specifically planned learning sequences; this inhibits the effective implementation of the flipped classroom in higher education. Australasian Journal of Educational Technology, 2021, 37(4). 53 Learning outcomes in the flipped classroom are highly dependent on the opportunities available for greater interactions between students (Van Alten et al., 2019). Although asynchronism is an oft-cited advantage of the flipped classroom – where students can learn independently at their own time and pace before class – it places more responsibility on students to learn from lectures, demonstrate understanding in the classroom and perform project-based tasks. However, these increased demands on learning responsibility may overwhelm students and lead to disengagement when they perceive a flipped learning environment as providing them with limited structural support for their learning (Bond, 2020; Strayer, 2012). Group work is believed to be an effective way to lessen students’ burden and afford them much-needed help (Arnold- Garza, 2014; Bond, 2020). Collaboration in group-based learning enables students to hold each other accountable through the examination, evaluation and application of their understanding and that of their peers both before and during class (Davidson & Major, 2014). Before coming to the classroom, students can work in small groups and use peer support to collaboratively prepare for class; during in-class sessions, individual contributions are woven together for problem-solving and the application of knowledge (Falcione et al., 2019; Zaka et al., 2019). Studies have provided evidence that well-constructed collaborations in flipped learning increase student engagement, satisfaction and achievement (Bond, 2020; K. Chen & Chuang, 2016; L. Chen et al., 2015; Chiang & Chen, 2017; Foldnes, 2016; Van Alten et al., 2019; Zaka et al., 2019). Another affordance of the flipped learning approach is the opportunity for instructors to collaborate with students in designing the curriculum, which is regarded as a principle of good scholarship of teaching and learning (Acai et al., 2017). The instructor–student partnership is defined as a shared and reciprocal process in which both parties work together to design learning, implement curriculum and achieve shared goals (Acai et al., 2017; Ouyang et al., 2020). Past decades have seen a variety of forms of instructor-student partnership implementation, such as the learning-community approach, students as collaborators and the student-faculty partnership, which aims to transform a traditional teacher-centred instruction into student- centred learning (Ouyang et al., 2020). Such instructor–student partnerships, if properly designed, have great potential to create a win-win relationship between all parties involved and to greatly benefit learning and teaching in the flipped classroom. Considering this need for greater empirical and design-based research on flipped classroom implementation in higher education, particularly the importance of integrating student–student and student–teacher collaboration into flipped classroom design, we adopted a self-developed three-stage collaborative instructional model (3-CI). The 3-CI model is a revision of the modified flipped classroom instructional model, which has been tested in some graduate courses in design-based studies (Chiang & Chen, 2017; Chiang & Liu, 2020); these studies have been conducted over 6 years. However, more research is required on the application of the 3-CI model in higher education to validate its effectiveness and benefits. This study’s results can help teachers in higher education to better incorporate flipped pedagogy into their routine teaching practice. The research questions of this study were as follows: (1) How do students perceive the 3-CI model? (2) What are the strengths and weakness of the 3-CI model? Method This study used a case study approach to investigate the effectiveness and features of the 3-CI model through students’ perceptions in a one-semester long graduate course, Research Methodologies in Educational Technologies. Both qualitative and quantitative data were collected at the end of the course, and participants’ perceptions of the pedagogical approach were obtained through a questionnaire. Semi- structured interviews were also conducted. The quantitative data in the questionnaire were analysed using SPSS software to address the research question of how students perceive the 3-CI model and the qualitative data, including answers to the semi-open questions in the questionnaire and the interview transcript, were analysed by NVivo for data triangulation and to obtain a more complete understanding of students’ experiences of flipped learning with the application of the 3-CI model. Australasian Journal of Educational Technology, 2021, 37(4). 54 Participants The study was conducted at a university in Shanghai. Participants were graduate students (N = 29) who took the aforementioned research methods course in the autumn semester of the 2019 academic year. All participants (20 female and nine male) were majoring in educational technology. The study was approved by the Academic Office at Shanghai Normal University, and we strictly followed the ethics guidelines for data collection and analysis, such as anonymity and voluntary participation. Prior to the study, we informed the students about the research design and obtained consent from all the participants, and they could opt out the study anytime during the process. All but one participant responded to the questionnaire, and six students agreed to participate in interviews on a voluntary basis. Approximately half the participants reported having experiences with learning in flipped classrooms, but none of them had experience learning with the application of the 3-CI model. One of us taught the course. 3-CI instructional model and flipped classroom implementation Research Methodologies for Educational Technologies is a core module for the university’s graduate students majoring in educational technology. The course covers key concepts and major qualitative and quantitative methodologies in educational technology research, specifically in research design, sampling and data collection and analysis. The course aims to equip students with the skills necessary for conducting rigorous research in educational technology. The 3-CI model used is aligned with the curriculum objective of developing students’ 21st-century skills, such as collaboration, critical thinking and problem solving. The 3-CI instructional model is a modified flipped classroom model in which the learning activities are organised into three stages: pre-class, in-class and after-class. Throughout the learning and teaching process, student -ed in the learning design. In the 3semphasi areteacher collaboration –student and student– I model, students form work groups and group members are encouraged, based on what they have C laboratively discuss the material in depth, develop their conceptual prepared individually, to col in ,CI model-teacher partnership is essential to the 3–understanding and achieve a shared goal. Student making. The three -which students are invited to participate in curriculum implementation and decision ation of learning steacher interaction, which greatly aids the reali–learning stages involve intensive student objectives. Figure 1. 3-CI model (revised from Chiang & Chen, 2017, pp. 3–4; Chiang & Liu, 2020, p. 49) Australasian Journal of Educational Technology, 2021, 37(4). 55 Students were assigned to nine work groups of three to four students each. Whilst students were free to choose their own group members, every group was required to include one male student to reduce possible gender bias, as female participants greatly outnumbered male participants. The work groups in this course assumed two roles: a teaching group and a learning group. Every group had the opportunity to serve as a teaching group once during the semester. The teaching group adopted the role of a teacher and was in charge of content delivery during in-class sessions, when the other groups were learning groups. The work groups took turns serving as a teaching group or learning group. The online platform ChaoXing (http://i.mooc.chaoxing.com), which is a popular online learning platform in China, was used as a virtual learning space to assist instruction, provide learning resources and promote student-teacher interaction. Before each lesson, students watched videos and learned from other course materials, such as slides, research papers and book chapters posted by the instructor on the learning platform. In addition to these basic learning assignments, the teaching group members worked collaboratively on preparing the lecture for the next classroom session. They studied the learning materials, searched online resources for supplementary information, discussed their preparatory work with the instructor and made improvements to the design of the learning activity. Members of the learning group studied the materials without having to prepare the lesson for the following class, and they learned independently and in groups. Students could post their questions on the learning platform and respond to questions posted by their peers. At this stage, the instructor guided the teaching group on their teaching preparation and design of content delivery, in addition to responding to questions posted on the learning platform. The weekly in-class session lasted two and a half hours and was divided into two parts. The first part was a student-led lecture in which the teaching group presented a topic and led other learning activities, such as question and answer and group discussions. The learning groups attended the lecture and were free to ask questions and make comments on the presentation. The second part was a teacher-led discussion and reflection. The instructor reviewed the teaching group’s presentation, raised questions regarding the content and design and stimulated deeper thinking by posing thought-provoking problems. After class, the teaching groups further improved their lecture design according to the feedback from in- class sessions; after that, they uploaded the finalised lectures and archived them in the learning platform as material for students in the upcoming year. The instructor was able to continue interacting with the students on the platform if they still had problems. Furthermore, students were required to finish another two assignments. For the first, each student was required to critically review an academic paper, where the review report must include a précis, a checking of the citations and references of the paper and an analysis of the strengths and weaknesses of the paper. For the second, every group had to apply a certain research methodology (e.g., case study, quasi-experimental study, design-based research or content analysis) to conduct an empirical study in the field of educational technology and write a research report. Research proposals had a midterm submission deadline, and the final research papers were required to be submitted before the end of the course. The instructor communicated frequently with the students to direct the design and execution of their research. The final research reports were graded through a combination of self- evaluation, peer evaluation and instructor evaluation. The final course grade was weighted as follows: 15% for participation (discussion and video watching), 25% for the individual academic paper review, 20% for the group research plan and 40% for the final group empirical research report. The 3-CI-based course implementation is detailed in Table 1. http://i.mooc.chaoxing.com/ Australasian Journal of Educational Technology, 2021, 37(4). 56 Table 1 Curriculum implementation of the 3-CI model Pre-class In-class After-class Instructor Provide students with learning materials, such as videos, slides, research papers and book chapters. Review the student-led lecture. Interact with students on the learning platform to strengthen their understanding after lectures. Answer questions that students post as they are learning. Raise questions regarding the content and design of students’ presentations. Guide students in their empirical research. Guide teaching groups’ preparation of the lecture and design of the learning activity. Stimulate deeper thought in students through thought-provoking problems. Students Teaching group Go over the materials provided by the instructor, independently and in groups. Present the lecture and lead other teaching activities, such as question and answer and group discussions. Reflect on and improve the lecture design. Express thoughts regarding the learning material. Finish the research paper review assignment individually. Prepare for the lecture. Conduct an empirical study in groups. Learning group Go over the materials provided by the instructor, independently and in groups. Listen to the lectures given by the teaching groups. Finish the research paper review assignment individually. Express thoughts and questions about the learning material. Provide feedback on the teaching group’s lecture. Conduct an empirical study in groups. Between groups Answer questions and communicate ideas with each other on the learning platform. Interact in the classroom and communicate ideas regarding the design and content of the lecture. Interact with each other and offer feedback on the empirical study. Instruments A 31-item questionnaire, which was a modification of the questionnaires of Chiang and Chen (2017) and Chiang and Liu (2020), was used in the present study. The questionnaire captured students’ perception of their learning experience in the 3-CI model. The questionnaire comprised four parts. The first part consisted of 22 questions in four dimensions: pre-class activities; in-class activities; after-class activities and overall experiences. These questions were scored on a Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). The second part comprised one question in which students ranked the course’s major teaching and learning activities by how helpful they were. The third part consisted of three scoring items, that is, students scored how well they performed, how well their group members performed and how well the course was designed and implemented. The scores ranged from 0 to 100. The last part consisted of five semi-open questions that prompted students to detail their learning experiences during the course. Specifically, the questions addressed (a) students’ overall perception of the 3-CI instructional model, (b) students’ favourite and least-favourite part of the curriculum, (c) students’ overall perception of the flipped classroom, and (d) additional comments on the instructional model and curriculum design. These semi-open questions were also utilised as the interview protocol in subsequent interviews to further solicit students’ opinions. Since all the participants are Chinese, the questionnaire and interview were developed and conducted in Chinese language. Australasian Journal of Educational Technology, 2021, 37(4). 57 Data analysis Quantitative data and qualitative data were collected and analysed separately, and then the data were compared and examined for consistency. The quantitative data from the Likert-scale questions, the ranking question and the scoring questions in the questionnaire were computed and analysed using SPSS (version 25). Descriptive statistics were run for students’ perception of the 3-CI model. Audio recordings of the interviews were transcribed, and all the qualitative data, including the interview transcriptions and answers to the semi-open questions were input into NVivo 11 software for coding and analysis. The constant comparative method (Glaser & Strauss, 2017), which involves an iterative process of open coding, was employed in analysing qualitative data. To begin with, two coders independently analysed a couple of transcripts and developed an initial coding scheme, based on the occurrence and frequency across data; then they compared, discussed and modified the two coding schemes to establish the final categories and themes for further analysis. After the coding structure were fully defined, they continued testing against the data using a categorised thematic frame while paying attention to variations. They independently coded each transcript and answer. Finally, they discussed and reviewed all the analysis results until all the discrepancies were settled on. This process ensured reliability and validity of the analysis. Results Students’ perception of their learning experience The data from the 22 Likert-scale questions revealed overall student satisfaction with the 3-CI curriculum design. Specifically, many agreed that the learning activities strengthened their understanding and cultivated their research skills (mean = 5.34, maximum of 7). Students were most satisfied with the in-class learning activities (mean = 5.57), and pre-class activities (mean = 4.97) ranked last. Student satisfaction with three stages and overall design are presented in Table 2. Table 2 Satisfaction with the three stages and overall design of the 3-CI model Ranking Stage Mean Standard deviation 1 In-class 5.57 1.12 2 After-class 5.43 1.17 3 Overall 5.38 1.07 4 Pre-class 4.97 1.27 The highest-scored 10 items with regard to students’ perceptions of their learning experience are presented in Table 3 and the lowest-scored three items are presented in Table 4. The top two highest-ranked items favoured by students in the flipped classroom were “The instructor’s explanations and comments after student-led lectures were highly beneficial for my understanding of the topic” (mean = 5.96) and “The empirical study report assignment was highly beneficial for my learning” (mean = 5.93). Australasian Journal of Educational Technology, 2021, 37(4). 58 Table 3 Ten highest-scored items in students’ perception of their learning experience Ranking Item Mean SD Stage 1 The instructor’s explanations and comments after student-led lectures were highly beneficial for my understanding of the topic. 5.96 0.96 In-class 2 The empirical study report assignment was highly beneficial for my learning. 5.93 0.90 After-class 3 I am very satisfied with the instructor’s teaching. 5.90 0.99 In-class 4 The classroom quizzes were helpful for verifying and reinforcing my understanding of concepts. 5.79 1.03 In-class 5 The curriculum enabled good student–teacher interaction during in-class sessions. 5.71 1.12 In-class 6 The curriculum enabled good student–student interaction during in-class sessions. 5.50 1.04 In-class 7 Since I learned the material, I have been able to apply my knowledge of research methodologies to new contexts and accomplish knowledge migration. 5.46 1.23 After-class 8 The assessment of students’ learning in this course is objective and comprehensive. 5.46 1.07 Overall 9 I am very satisfied with the opportunities for student–student interactions. 5.40 1.07 Overall 10 I can always get the assistance I need from the instructor after class. 5.39 1.37 After-class Table 4 Three lowest-scored aspects in students’ evaluation of their experiences Ranking Item Mean SD Stage 22 I am always ready to watch the videos posted by the instructor on the learning platform. 4.07 1.07 Pre-class 21 I am satisfied with the online learning platform. 4.78 1.07 Overall 20 The student-led lectures are highly beneficial for my learning. 4.89 1.20 In-class Responses to ranking questions were consistent with those to the Likert-scale questions. Students ranked the most rewarding parts of the course to be, in order, the instructor’s explanations and comments during in-class activities, the final empirical study and the classroom quizzes. The least rewarding parts, from the least rewarding one, were videos on the learning platform, classroom interactions using the whiteboard (students wrote answers on small whiteboards in response to the teacher’s questions), and the critical review of an academic research paper. Students rated the overall design and implementation of the curriculum the highest (mean = 93.11, maximum of 100), which demonstrates their perceived satisfaction with the flipped classroom application in this study, followed by their peers’ performance (mean = 89.18) and their own performance (mean = 81.71), both of which were also based on their perception of the learning experiences. All scores indicate good performance. According to the university’s grading scale, a score above 80 is good, and above 90 is excellent (Shanghai Normal University, 2016, pp. 29–32). The difference between the two scores suggests that students recognise the performance of their peers and believed that they could improve themselves further. These quantitative evaluations by students regarding their learning experiences, specific learning activities and their peers’ performance can be further elucidated through qualitative analysis. Australasian Journal of Educational Technology, 2021, 37(4). 59 Students’ perception of the curriculum design and implementation Thematic analysis was employed for both the transcribed interview responses and answers to the semi-open questions in the questionnaire. Data were pooled together, and codes were compared using NVivo software (version 11); three categories (the strengths, weaknesses and possible areas of improvement of the curriculum design) were then identified through the analysis. The subthemes are detailed in Table 5. Table 5 Students’ perceptions of the design and implementation of the curriculum and suggestions for improvement Perceptions and suggestions Frequency of mention Curriculum design strengths Overall design The curriculum design was novel and interesting. 12 The curriculum adopted a student-centred approach. 8 The learning experience was engaging because of the variety of learning activities. 9 Collaboration Group collaboration was highly beneficial for learning. 7 The discussion of concepts and group preparation of the student lecture helped us to strengthen our understanding. 5 Learning together afforded us different perspectives and promoted knowledge sharing and reflection. 4 Student–teacher collaboration and interaction facilitated learning. 15 Student-led lectures Having students prepare lectures and deliver them in class helped them to fully grasp the concepts. 8 The requirement to prepare and deliver lectures prompted students to take initiative in their learning. 5 Skill development The course developed students’ communication and collaboration skills. 4 The course made students better independent learners and researchers. 3 The course developed students’ presentation and academic writing skills. 3 Curriculum design weaknesses Video watching Some videos had poor visual quality (i.e., low resolution). 4 Some English-language videos had no Chinese subtitles, which made them difficult for the students to understand. 3 Learning pressure Too much content to learn. 5 Pre-class preparation was demanding, and the learning pace was too fast. 3 Student-led lectures Some student lectures were inadequate and may have presented concepts incorrectly. 2 Collaboration in lecture preparation was unsatisfactory. 2 Possible curriculum design improvements Video watching Improve the quality of videos. 6 Replace part of the English-language videos with Chinese- language videos or at least include Chinese subtitles. 6 Provide assessments, such as quizzes, to verify students’ understanding after they have watched the videos. 3 Reading material Add more research papers written in Chinese to supplement the English-language literature. 4 Grouping arrangements Group students later in the course after they have had more time to get to know each other. 2 Place no restrictions on grouping. 2 Australasian Journal of Educational Technology, 2021, 37(4). 60 Strengths of the curriculum design Most students credited the 3-CI curriculum design as being novel and interesting. The flipped classroom emphasising a student-centred approach and both student–student and student–teacher collaboration made the learning experience engaging and fruitful. Students also noted the benefits of group work in pre-class, in-class and after-class activities, in which students collaborated to deliver lectures and conduct empirical research. When the group members were learning concepts or discussing the lecture design, the differing perspectives they offered promoted greater understanding, sharing and reflection of the material. Students learned a lot from the members of their group with respect to learning content, approaches and attitudes. The students gave the following recounts during their interviews (the original quotes were Chinese, and we translated them into English): Overall, the curriculum design was good. Making preparations before coming to class was helpful for learning … Student-led lectures combined with the teacher’s explanations contributed to our understanding and retention of knowledge. The empirical study assignment allowed us to apply knowledge learned in the classroom to solve a real research question. The assessment approaches in this curriculum were comprehensive and objective. (Student 1) It was a completely new teaching approach to me, and I had never had such a learning experience before. The learning process was engaging, and I was really motivated to learn. (Student 13) When we worked together, everyone had their own opinions, and although these opinions sometimes varied or conflicting, we benefited from that. (Student 24) Excellent curriculum design! Excellent group activities! Excellent teaching presentation! (Student 18) Throughout the course, the instructor continually interacted with students, which facilitated their learning and promoted their understanding. Before class, the instructor supported students in their lecture design through social media or the online learning platform, sharing educational resources and giving specific suggestions. After students gave their lectures, the instructor commented on and provided explanations for the concepts being discussed, when necessary, in addition to posing questions and giving quizzes to verify and clarify students’ understanding of the material. Participants welcomed the instructor’s partnership with student presenters in delivering the content and perceived such partnership to be highly beneficial: What I liked most was group collaboration lectures and peer evaluations. It promoted knowledge sharing between group members and reflection on the concepts learned. (Student 12) When the members of the teaching group finished their presentation, the teacher summarised the content, verified our understanding, highlighted the focus of the lecture topic, and provided us with practical examples. I liked that! (Student 3) There was intensive interaction between students and the teacher and intensive interaction among the students. (Student 26) Having students design lectures and present them involved a considerable amount of preparation, including watching videos, reading, and looking up supplementary information, which promoted learning initiative in students. Furthermore, the preparation of lectures also required students to thoroughly grasp the material, which evidently deepened their understanding: What I liked most was the student-led lectures. It replaced the traditional approach of “the teacher talks and students listen” with “students lead and the teacher assists.” This made me feel like I was more active in learning. (Student 6) Australasian Journal of Educational Technology, 2021, 37(4). 61 Letting students deliver the lectures prepared them to be “specialists” of a certain kind of research methodology and honed their presentation skills. (Student 5) The curriculum’s cultivation of various abilities, such as in communication, collaboration, presentation, academic writing, independent learning and research skills, was also mentioned in the questionnaires and interviews: The curriculum design was wonderful. My teamwork, communication and exploration abilities were significantly improved. (Student 4) The curriculum improved my capabilities and promoted communication. It helped me think independently and reflect on what I had learned. (Student 2) Weaknesses in the curriculum design The qualitative data revealed several weaknesses in the design and implementation of the curriculum, most of which were related to pre-class activities. Such data elucidates why students’ perceptions were the most negative for the pre-class stage. The primary weakness was in the videos provided. The participants complained of poor visual quality in some videos and the lack of Chinese subtitles in English-language videos, which made them difficult to comprehend. These problems made students less interested in watching these videos: The videos’ visual quality needs to be improved. (Student 2) There were too many English videos. Sometimes it was difficult to understand them. (Student 20) The English videos that had no Chinese subtitles really dampened our learning interest. (Student 22) My English is not good. It was very hard to understand the English video without Chinese subtitles, so I had little interest in watching those videos. (Student 3) Flipped learning generally requires students to be active in their learning; this was especially true for the 3-CI model, in which students are required to deliver lectures. Thus, some students noted feeling under considerable pressure from the fast pace, wide range of tasks and heavy load of preparation work: There was too much to learn. It was hard for me to keep up. (Student 22) The learning pace was too fast. There was too much learning content for us to grasp. (Student 10) The learning pressure was a bit too much for me. I think it’s because I was not fully prepared before class. (Student 28) Although most of students favourably perceived the student-led lecture design, some students noted problems related to its implementation. Students had not previously attended a course similar to Research Methodologies in Educational Technologies, which covers many abstract concepts and teaches research skills at a level close to that used in actual research. Therefore, despite their best effort during preparation, students inevitably presented content inaccurately in some lectures. Some students were also dissatisfied with how their group collaborated: Some groups did not correctly present their concepts. It caused misunderstanding and affected my learning at the outset. (Student 21) I was not satisfied with the student-led lectures. The students’ presentation capabilities varied. I felt that I had to constantly adjust to different presentation styles, and it affected my learning efficiency. (Student 14) Australasian Journal of Educational Technology, 2021, 37(4). 62 We had problems with group collaboration. We were not familiar with each other when we formed the group, so problems emerged later during learning and collaboration. We did not communicate very well. (Student 15) Suggestions for improving the curriculum design Students offered suggestions for improving the curriculum design and implementation. These suggestions centred on the learning material and grouping approach. They suggested that the quality of all learning videos be checked and that low-resolution videos be replaced with high-quality ones. They also recommended that Chinese subtitles be included in English-language videos. Students also commented that exercises, such as short quizzes, should be provided after students have watched a video to reinforce their understanding. They also recommended the addition of more Chinese-language literature to the supplementary reading materials. As for the grouping approach, students suggested that the grouping should occur after the students have had more time to get to know each other, thus enabling them to select group members who they can easily work with. Students also expressed the desire that there be no restrictions on group composition (in this study, every group was required to include a male student): The videos could be shorter, be in Chinese, and be more interesting. (Student 3) The quality of the videos should be improved, and the instructor can provide a quiz after a video so that we can self-evaluate to verify our understanding of the concepts. (Student 19) Apart from English literature, I recommend including more Chinese literature. (Student 6) It would be great if the teacher would let us get to know each other better before grouping since we want to be in groups with people who share our interests and goals. (Student 15) Discussion The present study reports the application of a method for pedagogical innovation in the design and implementation of the 3-CI instructional model in a graduate course. Participants’ perceptions of 3-CI were investigated, and the study’s results indicate that the revised flipped model produced positive outcomes with respect to student satisfaction, engagement and improvement in collaboration. Firstly, the flipped curriculum design resulted in relatively high levels of student satisfaction, as indicated by the questionnaire responses. This finding is consistent with previous studies on flipped learning that have demonstrated that student satisfaction is significantly affected by motivation granted by the learning environment, a student- centred approach and the format and structure of learning materials (e.g., Critz & Wright, 2013; Forsey et al., 2013; Mason et al., 2013). Secondly, the participants reported increased engagement and participation. The curriculum design in this study provided students with multiple avenues for learning concepts, involved class time devoted to student-led lectures and required the completion of a research project. The learning process featured intensive student–student and student–teacher interactions, enhanced enjoyment in novel learning activities and positive self-perception and self-efficacy through the completion of complex research projects. It echoed Bond’s (2020) multifaceted engagement model to engage students behaviourally, affectively and cognitively. Finally, participants generally perceived improvements in their communication and collaboration skills due to the curriculum design, which emphasised group work. Students had to work together to prepare and present lectures, conduct empirical studies and write reports. The instructor deliberately employed a collaboration strategy throughout the learning process, which provided students with many opportunities to develop their collaboration skills. The 3-CI model emphasises the establishment of a student-centred environment and integration of active learning strategies in the curriculum design. Shea et al. (2012) reported that students were more engaged when granted greater autonomy and challenged with higher-order tasks. The curriculum in the present study employed a principle of learning by doing and assignments that required student to be responsible for their own learning. The tasks, such as student-led lectures, critical reviews of research papers and empirical research, helped students to self-regulate their learning process. Specifically, students practised setting project goals, formulating strategies, monitoring their progression and evaluating their achievements. An active learning environment motivated them to explore problems and help them engage with the material more meaningful. Australasian Journal of Educational Technology, 2021, 37(4). 63 Although an oft-cited feature of flipped learning is the shift of responsibility from instructors to students, the 3-CI model highlights the irreplaceable value that teachers have to offer. Kim et al. (2014) suggested that the teacher’s role in a flipped classroom is as active and important, if not even more so, than it is in a conventional classroom. A successful flipped classroom design requires the instructor to systematically design curriculum activities and guide students’ learning experiences. As seen in the questionnaire responses, aspects of the curriculum design favoured by students mostly related to the teacher’s direct involvement, such as the clarification and reinforcement of concepts in the classroom, guidance for students’ empirical research assignment and assistance provided to students through the application of their group work. Furthermore, students reported that they were most satisfied with the in-class stage and benefitted most from the teacher’s explanation of concepts, consistent with Van Alten’s (2019) finding that sustaining face-to-face time is critical to the success of a flipped classroom. As opposed to traditional large- group lectures, the teacher’s explanations in our study were usually in the form of micro-lectures and aimed at reviewing students’ performance, addressing students’ misunderstandings or knowledge gaps and posing questions to further stimulate their thinking. Kim et al. (2014) noted that students learn better and are more satisfied when the instructor is more involved, because the instructor “holds a lot of power in making the flip successful” (Arnold-Garza, 2014, p. 15). They should motivate students to engage with the content, provide them with multiple avenues for demonstrating their knowledge and constantly be available to students throughout the learning process. As its name suggests, 3-CI is defined by collaboration, and this study’s results demonstrate the benefits of this emphasis. In flipped learning, students must follow through on viewing the lectures, reviewing the material and developing their own understanding. These tasks may be challenging for some students, and they may become frustrated with the seemingly onerous demands placed on their time and attention. To address these concerns, the 3-CI model provides students with multiple opportunities to collaborate in pre- class, in-class and after-class activities. Thus, students can support each other and collaboratively complete a large project. Further, instructor–student collaboration is an important part of 3-CI; according to Ouyang et al., (2020), instructor–student collaboration constitutes an active and dynamic process that promotes students’ participation, engagement and attainment of shared goals. In pre-class and after-class activities, the instructor provided structured guidance to students as they prepared their lectures and undertook their empirical research, giving feedback on their learning problems through frequent interaction. During in- class sessions, the teacher partnered with the teaching group to deliver content and helped clarify students’ understanding and reinforce their knowledge. This extensive teaching presence motivated students to explore the content and reassured them that they were on the right track, which contributed greatly to their achievement of the learning goals. Educators implementing flipped learning should note the problems identified in the present study, which mostly related to the participants’ dissatisfaction with pre-class arrangements. Firstly, students’ learning interest was dampened by the poor visual quality of videos and lack of foreign-language support, and some students cited the pre-class videos as the least rewarding part of flipped learning. Owing to the unsatisfactory videos, some students did not engage deeply enough with the pre-class assignments to be adequately prepared for classroom activities, which contributed to their dissatisfaction. As literature has suggested that video quality (resolution, length, language, authorship) are influential in students’ perceptions (Akçayır & Akçayır, 2018; DeSantis et al., 2015), the instructor should select and organise learning materials with care, adjusting them on the basis of students’ levels of preparation and understanding of the content. Furthermore, for a better pre-class experience, instructors could employ interactive exercises such as after-video quizzes and online discussions to ensure that students complete the pre-class assignments and understand the concepts. Quizzes generally have a positive influence on learning outcomes due to the testing effect (Dirkx et al., 2014; McDaniel et al., 2007), and the meta-analysis conducted by Van Alten et al. (2019) has revealed that the inclusion of quizzes significantly affects the effectiveness of a flipped classroom in a positive way. Finally, as suggested by Arnold-Garza (2014), the instructor should clearly communicate to students their responsibilities and the course requirements, emphasising that the flipped classroom may be more demanding than a typical lecture-based one. Notably, student failure to engage with the learning materials before class is a risk in the flipped classroom, and such failure neutralises the benefits of subsequent in-class activities (Zaka et al., 2019). Australasian Journal of Educational Technology, 2021, 37(4). 64 Conclusions and implications Increasingly popular in higher education, the flipped classroom features a reversal of the order of course delivery and effective use of class time. Although college educators generally recognise the value of the flipped classroom in improving students’ learning experiences, they require support in translating the concept of the flipped classroom into its context-sensitive implementation. The present study demonstrated how 3-CI, a revised flipped classroom method, can be designed and implemented in a graduate class, specifically through the investigation of students’ perceptions of 3-CI and discussion of its features. The student-centred 3-CI model emphasises both student–student and student–teacher collaboration, and the results of this study suggest that 3-CI increases students’ satisfaction, engagement and collaboration. As the findings of the present study further the understanding of flipped classroom, several practical implications can be drawn from the pedagogical innovation effort in this research. Firstly, the setting up of an environment emphasising collaboration can help students better cope with the challenges presented by flipped learning. Although students may vary in their comfort level with collaboration, they generally benefit from the enhanced support from collaborative activities, both with their peers and the instructor. Secondly, educators should be well aware of the increased important role they play in the flipped classroom. The essential elements that contribute to a successful flipped learning, such as student engagement, learning autonomy and efficient use of class time, hinge on the instructor’s meticulous design of activities with the aim of providing structural support to the students. Lastly, students’ preparedness prior to the classroom is crucial, so that the instructor should pay more attention to the pre-class learning material to ensure student engagement and completion of the tasks. We advise that the quality, length and content of the video and reading material presented be carefully examined to best suit the students’ need, and that quizzes be provided after the video or reading material as a feasible way to check and reinforce student understanding. Limitations and future directions This study, however, has several limitations. The first is the potentially limited generalisability of the findings because the participants were graduate students. Specifically, Strayer (2012) noted that flipped learning may be more beneficial to more advanced students. Therefore, whether 3-CI is also effective for undergraduate students remains unclear and should be clarified by future studies. Second, the findings pertain to student perceptions and not student performance. Therefore, although students held favourable perceptions of 3-CI, we cannot conclude that 3-CI improved their learning performance. Therefore, future studies could use a quasi-experimental method to determine whether 3-CI improves academic achievement. Finally, because the instructor was also a researcher in the study, subjective bias may have been present, possibly and unintentionally affecting participants’ attitudes. To minimise such possible influence, the instructor was not involved in data collection and analysis, and external researchers were invited to conduct the survey and interview and analyse the data. However, it is preferable that 3-CI be replicated by other instructors in the future and thus enhance the interpretation of the findings. Despite these limitations, the current study provides insight into the design and implementation of the flipped pedagogical approach. It extends essential flipped classroom elements to propose an actionable model that can serve as a reference to college educators during curriculum renewal. Further efforts by scholars and educators can strengthen understanding of the value of flipped classroom pedagogy. Acknowledgements This work was supported by grants (307-AC9103-20-365205) and (304-AC9103-19-368407004) from Shanghai Normal University, China. Australasian Journal of Educational Technology, 2021, 37(4). 65 References Acai, A., Akesson, B., Allen, M., Chen, V., Mathany, C., McCollum, B., Spencer, J., & Verwoord, R. E. M. (2017). Success in student-faculty/staff SoTL partnerships: Motivations, challenges, power, and definitions. The Canadian Journal for the Scholarship of Teaching and Learning, 8(2), Article 8. https://doi.org/10.5206/cjsotl-rcacea.2017.2.8 Akçayır, G., & Akçayır, M. (2018). The flipped classroom: A review of its advantages and challenges. Computers & Education, 126, 334–345. https://doi.org/10.1016/j.compedu.2018.07.021 Arnold-Garza, S. (2014). The flipped classroom teaching model and its use for information literacy instruction. Communications in Information Literacy, 8(1), 7–22. https://doi.org/10.15760/comminfolit.2014.8.1.161 Bergmann, J., & Sams, A. (2012). Before you flip, consider this. Phi Delta Kappan, 94(2), 25–25. http://doi.org/10.1177/003172171209400206 Betihavas, V., Bridgman, H., Kornhaber, R., & Cross, M. (2016). The evidence for ‘flipping out’: A systematic review of the flipped classroom in nursing education. Nurse Education Today, 38(38), 15– 21. https://doi.org/10.1016/j.nedt.2015.12.010 Bond, M. (2020). Facilitating student engagement through the flipped learning approach in K–12: A systematic review. Computers & Education,151, 1–36. https://doi.org/10.1016/j.compedu.2020.103819 Boucher, B., Robertson, E., Wainner, R., & Sanders, B. (2013). “Flipping” Texas State University’s physical therapist musculoskeletal curriculum: Implementation of a hybrid learning model. Journal of Physical Therapy Education, 27(3), 72–77. https://doi.org/10.1097/00001416-201307000-00010 Brame, C., (2013). Flipping the classroom. Vanderbilt University Center for Teaching. http://cft.vanderbilt.edu/guides-sub-pages/flipping-the-classroom/ Chen, F., Lui, A. M., & Martinelli, S. M. (2017). A systematic review of the effectiveness of flipped classrooms in medical education. Medical Education, 51(6), 585–597. https://doi.org/ 10.1111/medu.13272 Chen, K., & Chuang, K. (2016). Building a cooperative learning environment in a flipped classroom. Academy of Educational Leadership Journal, 20(2), 8–15. https://www.abacademies.org/articles/aeljvol20no22016.pdf Chen, L., Chen, T., & Chen, N. (2015). Students’ perspectives of using cooperative learning in a flipped statistics classroom. Australasian Journal of Educational Technology, 31(6), 621–640. https://ajet.org.au/index.php/AJET/article/download/1876/1319 Chiang, F., & Chen, C. (2017). Modified flipped classroom instructional model in “learning sciences” course for graduate students. The Asia-Pacific Education Researcher, 26(1–2), 1–10. https://doi.org/10.1007/s40299-016-0321-2 Chiang, F., & Liu, Y. (2020). 3CI师生合作教学法运用在研究生课堂的基于设计研究 [Application of 3-CI model in science education for graduate students: Design-based research]. Research in Teaching, 43(1), 1–9. http://jxyj.ysu.edu.cn/#/digest?ArticleID=1977 Critz, C., & Wright, D. (2013). Using the flipped classroom in graduate nursing education. Nurse Educator, 38(5), 210–213. https://doi.org/10.1097/NNE.0b013e3182a0e56a Davidson, N., & Major, C. H. (2014). Boundary crossings: Cooperative learning, collaborative learning, and problem-based learning. Journal on Excellence in College Teaching, 25(3&4), 7–55. http://celt.muohio.edu/ject/fetch.php?id=592 DeSantis, J., Van Curen, R., Putsch, J., & Metzger, J. (2015). Do students learn more from a flip? An exploration of the efficacy of flipped and traditional lessons. Journal of Interactive Learning Research, 26(1), 39–63. https://doi.org/10.1074/jbc.M302713200 Dirkx, K. J. H., Kester, L., & Kirschner, P. A. (2014). The testing effect for learning principles and procedures from texts. The Journal of Educational Research, 107(5), 357–364. https://doi.org/10.1080/00220671.2013.823370 Elken, M., & Tellmann, S. M. (2019). Linking higher education and the world of work: Learning outcomes and intermediary organisations. Journal of Education and Work, 32(8), 678–692. https://doi.org/10.1080/13639080.2019.1696951 Falcione, S., Campbell, E., Mccollum, B., Chamberlain, J., Macias, M., Morsch, L. A., & Pinder, C. (2019). Emergence of different perspectives of success in collaborative learning. The Canadian Journal for the Scholarship of Teaching and Learning, 10(2), 1–21. https://doi.org/10.5206/cjsotl-rcacea.2019.2.8227 https://doi.org/10.5206/cjsotl-rcacea.2017.2.8 https://doi.org/10.1016/j.compedu.2018.07.021 https://doi.org/10.15760/comminfolit.2014.8.1.161 http://doi.org/10.1177/003172171209400206 https://doi.org/10.1016/j.nedt.2015.12.010 https://doi.org/10.1016/j.compedu.2020.103819 https://doi.org/10.1097/00001416-201307000-00010 http://cft.vanderbilt.edu/guides-sub-pages/flipping-the-classroom/ https://doi.org/%2010.1111/medu.13272 https://www.abacademies.org/articles/aeljvol20no22016.pdf https://ajet.org.au/index.php/AJET/article/download/1876/1319 https://doi.org/10.1007/s40299-016-0321-2 http://jxyj.ysu.edu.cn/#/digest?ArticleID=1977 https://doi.org/10.1097/NNE.0b013e3182a0e56a http://celt.muohio.edu/ject/fetch.php?id=592 https://doi.org/10.1074/jbc.M302713200 https://doi.org/10.1080/00220671.2013.823370 https://doi.org/10.1080/13639080.2019.1696951 https://doi.org/10.5206/cjsotl-rcacea.2019.2.8227 Australasian Journal of Educational Technology, 2021, 37(4). 66 Foldnes, N. (2016). The flipped classroom and cooperative learning: Evidence from a randomized experiment. Active Learning in Higher Education, 17(1), 39–49. https://doi.org/10.1177/1469787415616726 Forsey, M., Low, M., & Glance, D. (2013). Flipping the sociology classroom: Towards a practice of online pedagogy. Journal of Sociology, 49(4), 471–485. https://doi.org/10.1177/1440783313504059 Gilboy, M. B., Heinrichs, S., & Pazzaglia, G. (2014). Enhancing student engagement using the flipped classroom. Journal of Nutrition Education and Behavior, 47(1), 109–114. https://doi.org/10.1016/j.jneb.2014.08.008 Glaser, B., & Strauss, A. (2017). The discovery of grounded theory: Strategies for qualitative research (1st ed.). Routledge. https://doi.org/10.4324/9780203793206 Jacob, W. J., & Gokbel, V. (2017). Global higher education learning outcomes and financial trends: Comparative and innovative approaches. International Journal of Educational Development, 58(1), 5– 17. https://doi.org/10.1016/j.ijedudev.2017.03.001 Karabulut-Ilgu, A., Jaramillo Cherrez, N., & Jahren, C. T. (2018). A systematic review of research on the flipped learning method in engineering education. British Journal of Educational Technology, 49(3), 398–411. https://doi.org/10.1111/bjet.12548 Kim, M. K., Kim, S. M., Khera, O., & Getman, J. (2014). The experience of three flipped classrooms in an urban university: An exploration of design principles. Internet and Higher Education, 22, 37–50. https://doi.org/10.1016/j.iheduc.2014.04.003 Lundin, M., Bergviken Rensfeldt, A., Hillman, T., Lantz-Andersson, A., & Peterson, L. (2018). Higher education dominance and siloed knowledge: A systematic review of flipped classroom research. International Journal of Educational Technology in Higher Education, 15(20), 1–30. https://doi.org/10.1186/s41239-018-0101-6 Mason, G. S., Shuman, T. R., & Cook, K. E. (2013). Comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering course. IEEE Transactions on Education, 56(4), 430–435. https://doi.org/10.1109/TE.2013.2249066 Mayer, R. E. (2002). Rote versus meaningful learning. Journal of Theory into Practice, 41(4), 226–232. https://doi.org/10.1207/s15430421tip4104_4 McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007). Testing the testing effect in the classroom. European Journal of Cognitive Psychology, 19(4–5), 494–513. https://doi.org/10.1080/09541440701326154 O'Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. Internet and Higher Education, 25, 85–95. https://doi.org/10.1016/j.iheduc.2015.02.002 Ouyang, F., Chang, Y., Scharber, C., Jiao, P., Huang, T. (2020). Examining the instructor-student collaborative partnership in an online learning community course. Instructional Science, 48(3), 183– 204. https://doi.org/10.1007/s11251-020-09507-4 Seery, M. K. (2015). Flipped learning in higher education chemistry: Emerging trends and potential directions. Chemistry Education: Research and Practice, 16(4), 758–768. https://doi.org/10.1039/C5RP00136F Shanghai Normal University. (2016). 学生手册 [Student handbook]. http://jwc.shnu.edu.cn/_upload/article/files/a1/6f/f4c268fa49b5990934ba033af58d/1e1e2e1f-0bd9- 43b7-a405-6c0ba1caf90f.pdf Shea, P., Hayes, S., Smith, S. U., Vickers, J., Bidjerano, T., Pickett, A., et al. (2012). Learning presence: Additional research on a new conceptual element within the community of inquiry (CoI) framework. Internet and Higher Education, 15(2), 89–95. https://doi.org/10.1016/j.iheduc.2011.08.002 Strayer, J. (2012). How learning in an inverted classroom influences cooperation, innovation and task orientation. Learning Environments Research, 15(2), 171–193. https://doi.org/10.1007/s10984-012-9108-4 Van Alten, D. C. D., Phielix, C., Janssen, J., & Kester, L. (2019). Effects of flipping the classroom on learning outcomes and satisfaction: A meta-analysis. Educational Research Review, 28, 1–18. https://doi.org/10.1016/j.edurev.2019.05.003 Zaka, P., Fox, W. H., & Docherty, P. D. (2019). Student perspectives of independent and collaborative learning in a flipped foundational engineering course. Australasian Journal of Educational Technology, 35(5), 79–94. https://doi.org/10.14742/ajet.3804 https://doi.org/10.1177/1469787415616726 https://doi.org/10.1177/1440783313504059 https://doi.org/10.1016/j.jneb.2014.08.008 https://doi.org/10.4324/9780203793206 https://doi.org/10.1016/j.ijedudev.2017.03.001 https://doi.org/10.1111/bjet.12548 https://doi.org/10.1016/j.iheduc.2014.04.003 https://doi.org/10.1186/s41239-018-0101-6 https://doi.org/10.1109/TE.2013.2249066 https://doi.org/10.1207/s15430421tip4104_4 https://doi.org/10.1080/09541440701326154 https://doi.org/10.1016/j.iheduc.2015.02.002 https://doi.org/10.1007/s11251-020-09507-4 https://doi.org/10.1039/C5RP00136F http://jwc.shnu.edu.cn/_upload/article/files/a1/6f/f4c268fa49b5990934ba033af58d/1e1e2e1f-0bd9-43b7-a405-6c0ba1caf90f.pdf http://jwc.shnu.edu.cn/_upload/article/files/a1/6f/f4c268fa49b5990934ba033af58d/1e1e2e1f-0bd9-43b7-a405-6c0ba1caf90f.pdf https://doi.org/10.1016/j.iheduc.2011.08.002 https://doi.org/10.1007/s10984-012-9108-4 https://doi.org/10.1016/j.edurev.2019.05.003 https://doi.org/10.14742/ajet.3804 Australasian Journal of Educational Technology, 2021, 37(4). 67 Corresponding author: Zhenhua Wu, bluskye@shnu.edu.cn Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. Please cite as: Chiang, F.-K., & Wu, Z. (2021). Flipping a classroom with a three-stage collaborative instructional model (3-CI) for graduate students. Australasian Journal of Educational Technology, 37(4), 51-67. https://doi.org/10.14742/ajet.6330 mailto:bluskye@shnu.edu.cn https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.6330 Introduction Method Participants 3-CI instructional model and flipped classroom implementation Instruments Data analysis Results Students’ perception of their learning experience Students’ perception of the curriculum design and implementation Strengths of the curriculum design Weaknesses in the curriculum design Suggestions for improving the curriculum design Discussion Conclusions and implications Limitations and future directions Acknowledgements References