Australasian Journal of Educational Technology, 2019, 35(3). 1 Investigating the implementation of accredited massive online open courses (MOOCs) in higher education: The boon and the bane Xinghua Wang Nanyang Technological University, Singapore Allison. H. Hall University of Illinois at Chicago Qiyun Wang Nanyang Technological University, Singapore This study investigated a 3-year implementation of accredited massive online open courses (MOOCs) in a conventional university in China. Fifteen students and two staff were interviewed, and relevant policy documents were examined. Thematic analysis was utilised to analyse the data. The study found that students mostly took a wary stance towards accredited MOOCs, while the university supported them. Reasons for students’ favour and disfavour were explored. There was widespread online absenteeism, which was unique to accredited MOOCs. Moreover, accredited MOOCs were found to be completely independent of traditional onsite courses, making them difficult to form a synergy with the latter. Also, a distinct mismatch was identified between the course assessment system and the new course format. Furthermore, there were insufficient regulation, support, and incentives from the university in facilitating the implementation of accredited MOOCs, indicating a discrepancy between university policies and practice. Solutions for future improvement of accredited MOOCs were explored. This study will aid educational practitioners and policymakers in widening access to quality education by exploring effective solutions regarding integrating accredited MOOCs into conventional higher education. Introduction Since their initial appearance, massive online open courses (MOOCs) have generated substantial discussion and debate with regards to the opportunities and threats associated with them. Some see MOOCs as revolutionary and argue that they widen access to quality education and enhance social inclusion (Sandeen, 2013; Zhou, 2017), while more critical others view them as merely duplicating the passive instructionism of the twentieth century (Selwyn, Bulfin, & Pangrazio, 2015) and consider them a clever marketing trick by elite universities (Krause & Lowe, 2014) because of the various monetarisation and business models surrounding them. With the fast advancement of MOOCs and their potential influence in education, they have been enlisted in the modernisation agenda for many universities around the world and have been associated with such concepts as scalability and sustainability of higher education. Alongside this, there have been increasing discussions regarding the integration of MOOCs into conventional higher education and the recognition of academic credit associated with them (Kursun, 2016; Sandeen, 2013). Nonetheless, most such discussions are still at the conceptual level, with few reaching the practical level, primarily due to the paucity of actual cases of integration. In this empirical study, we investigated the integration of accredited MOOCs into the formal curriculum of a public university in China (the focal university hereafter). The MOOCs are provided by three commercial MOOC platforms, carry academic credit transferable to the focal university, and have been implemented at the university since September 2013. Furthermore, these MOOCs are obligatory for all students at the university. Fifteen students and two staff were interviewed, in conjunction with the examination of relevant university policy documents. Qualitative thematic analysis was utilised to explore the current status of the implementation of accredited MOOCs at the focal university, including students’ response to them, their role in existing curriculum, and the challenges and problems faced. The findings of this study provide empirical evidence for Australasian Journal of Educational Technology, 2019, 35(3). 2 the current discussion about integrating MOOCs into formal curricula of conventional higher education, assigning them academic credit towards degree programs, and informing educational researchers, practitioners, and policymakers in exploring more effective solutions on this topic. Literature review This section introduces the basics of MOOCs, describes extant studies investigating the integration of accredited MOOCs in conventional universities, and then presents the research questions that guided this study. Types of MOOCs Among the varied ways previous research has distinguished MOOCs, an influential one categorises them into cMOOCs and xMOOCs (Margaryan, Bianco, & Littlejohn, 2015; Smith & Eng, 2013) based on pedagogical model. cMOOCs, based on connectivism, are driven by the pedagogical principle of social learning. They represent the earlier version of MOOCs which emphasised distributed peer learning and constituted part of the open educational resources movement. Participants themselves are the primary content contributors. Mass communication and interaction among participants results in the formation of a learning community where knowledge is constructed and expertise built (Downes, 2013). xMOOCs, based on instructivism, are currently the prevalent category. They often use a knowledge transmission model and are embodied in the form of video recordings of conventional classroom lectures given by famous professors from leading universities (Universities UK, 2013). According to Hayes’ (2015) report on the quality of MOOCs, cMOOCs excel in creating mass social interaction among participants, but quality assurance and owning the rights to learning content are highly questionable. Despite the relatively high quality of learning content, xMOOCs often lack sufficient interaction. Nonetheless, the two types of MOOCs have substantially contributed to the development of higher education by widening the access for students and institutions to engage with each other in new ways (Universities UK, 2013). Integration of MOOCs into conventional institutions of higher education Some scholars see MOOC accreditation in conventional universities as the solution for such issues as high dropout rates and low course satisfaction. For instance, Hollands and Tirthali (2014) argued that providing formal course credit for MOOCs could increase students’ motivation to learn and enhance their commitment in completing the courses, thus decreasing high dropout rates. Nevertheless, many universities, as well as students, hold reservations on when and how MOOCs can be effectively integrated into conventional onsite curricula (Cross & Whitelock, 2017). Despite the increasing number of papers on MOOCs published in international journals in recent years, limited studies have closely examined the topic of MOOC accreditation (Kursun, 2016). Furthermore, extant discussions on this topic are largely at the conceptual level, with few being substantiated. Here, the current status of MOOCs’ integration into conventional universities is given, followed by discussions of why such integration is necessary and what the associated challenges are. Forms of integration in extant studies are then presented. Current status of MOOCs being integrated into conventional universities MOOCs provide a complementary means for students to attain academic credit in conventional higher education. An increasing number of universities are experimenting with varied forms of integrating MOOCs into the traditional curricula and credit-recognition models. Nevertheless, the universities providing MOOCs are normally not willing to offer their own academic credit for the courses for various reasons, including protecting the residential-campus experience. Although many MOOCs (e.g., Coursera and edX) provide certificates or badges for students successfully completing courses, very few of them have been accepted for academic credit or even for admission into conventional institutions of higher education (Kursun, 2016). Currently, there is still a considerable amount of work surrounding credit recognition for MOOCs and their Australasian Journal of Educational Technology, 2019, 35(3). 3 integration into formal education, including content licensing, assessment, and quality assurance (Fox, 2016), which cannot be accomplished in a short timeframe. Transition of MOOCs from informal education to formal education via credentialing MOOCs are not likely to replace conventional onsite universities (Haber, 2014), and may face a threat in the future “if they are not well-integrated in broader university strategies and do not establish their own role within the university offering” (Schuwer et al., 2015, p. 30). As such, integrating them into conventional universities to be a part of formal education is one way to keep them sustainable. However, the transition of MOOCs from informal education to formal education requires credentialing. Scholars concerned about MOOC accreditation worry that MOOCs may not have the same rigor in learning, assessment, and identity verification as conventional onsite courses (Hollands & Tirthali, 2014). Low quality MOOCs and poor instruction have been considered impediments to the transition of MOOCs from informal education to formal education (Schuwer et al., 2015). Furthermore, high dropout rates (Hew & Cheung, 2014), the lack of effective institutional strategies, and too much regulation (Schuwer et al., 2015) all tend to obstruct the integration of MOOCs into conventional universities. Forms of integration of MOOCs into conventional universities Extant studies exploring methods of integration are mainly from the perspective of blended learning (e.g., Bruff, Fisher, McEwen, & Smith, 2013; Israel, 2015). Bruff et al. (2013), for instance, proposed that the integration of MOOCs into conventional classrooms is affected by two elements: coupling and cohesion. Coupling is concerned with the dependency between online and onsite components of a hybrid course whilst cohesion means the relatedness of the overall course content. Moreover, Israel (2015) found that existing studies generally integrate MOOCs into conventional classrooms in two ways: (1) as replacement for conventional onsite courses, and (2) as supplementary resources for conventional onsite courses. Both ways take the form of blended learning. That is, MOOCs are blended with or into onsite course components. Nevertheless, these two ways of integration did not work out effectively for student learning (Israel, 2015). Furthermore, they required onsite instructors to invest substantial time and effort in re-designing MOOCs for effective use in blended learning (Griffiths, Chingos, Spies, & Mulhern, 2014). To provide empirical insight into MOOC accreditation and to contribute new knowledge about the integration of accredited MOOCs into conventional institutions of higher education, this study investigated a 3-year implementation of accredited MOOCs in the formal curriculum in a conventional university. To this end, this study aimed to answer the following research questions: 1. How did students respond to the accredited MOOCs? 2. How did accredited MOOCs reconcile with onsite courses in the focal university? 3. What were the possible challenges and problems arising during the implementation of accredited MOOCs? Methodology Research site This study was conducted at a public university in China. The focal university has approximately 35,000 full- time students. The accredited MOOCs were implemented for more than 3 years: September 2013 to December 2016 (the time of this study). Three MOOC platforms were used: ZhiHuiShu (http://www.zhihuishu.com/), ErYa (http://erya.mooc.chaoxing.com/), and Ulearning (http://www.ulearning.cn/), all of which have mobile apps devised for smartphones and tablets. The three MOOC platforms are for-profit and are developed by commercial organisations in collaboration with many universities in China. All claim on their websites that they bring together leading universities and educators across the country to provide education in a wide range of fields. The MOOCs on these platforms primarily consist of video lectures of varying lengths and exercises http://www.zhihuishu.com/ http://erya.mooc.chaoxing.com/ http://www.ulearning.cn/ Australasian Journal of Educational Technology, 2019, 35(3). 4 with very limited discussion forums. The universities validate and export the MOOCs. There is no assessment for prior learning, which is usually used to determine whether a student is eligible for joining certain MOOCs. The MOOCs that the focal university imports are credit-bearing courses lasting for one semester which is about 16 weeks. The academic credits are offered by the universities exporting MOOCs and are transferrable to the focal university. The accredited MOOCs have been obligatory for students admitted since September 2013. Students are required to finish four accredited MOOCs within 3 years, with each accounting for 2 academic credits. To strengthen liberal education, students from science disciplines such as physics, mathematics, and engineering are required to take at least two courses from humanities and social sciences, and vice versa. Currently, the courses registered for by students are all provided by other universities. The focal university declares in its policy documents regarding accredited MOOCs that it is committed to providing innovative and useful online courses that enable greater learning flexibility and better learning experience for students. The accredited MOOCs, according to the documents in the focal university, support self-paced learning, are academically rigorous, and require a similar investment of time and effort as the face- to-face courses. Participants This study only examined the third-year university students who had already finished the four online credit courses. Fifteen student participants in total were sampled purposively with their informed consent. Six of them were from disciplines of sciences, including mathematics and physics, while nine were from disciplines of social sciences, including foreign languages and sociology. To further strengthen the sample representativeness, 6 of the 15 students had high GPAs (3.7 and above), 6 had intermediate GPAs (3.0-3.7), and 3 had low GPAs (3.0 and below). These students, aged between 19 and 21, grew up with the fast advancement of information and communications technologies and generally have strong computer literacy (Zou & Huang, 2014). In addition, two staff involved in the implementation of accredited MOOCs in the focal university were interviewed. Research design and data collection We adopted a case study design to investigate the implementation of accredited MOOCs at the focal university, as this design could help us develop an in-depth, contextual, and holistic understanding of the implementation of and students’ experience of accredited MOOCs (Yin, 2009). To answer the research questions, three types of data were collected for this study: (1) two rounds of in-depth interviews with the students, (2) in-depth interviews with two university staff, and (3) policy documents about accredited MOOCs in the focal university. The multiple data sources enabled us to triangulate the data and further strengthen the reliability of the findings. In-depth interviews were utilised in this study: (a) to capture how interviewees view their world and the complexities of their individual perceptions and experiences of the topic being investigated (Patton, 2002), thus enabling us to examine in-depth their experiences and practices of utilising MOOCs; and (b) to reveal how their experiences shape and are shaped by the interpersonal, social, and institutional forces (Seidman, 2013). In-depth interviews were concurrent with data analyses with the aim of utilising initial findings from previous rounds of interviews to inform subsequent rounds of interviews and further data analyses. First, three students, one from mathematics and two from foreign languages were interviewed with three open- ended questions: What was your general experience with these accredited MOOCs? What were the challenges you encountered while using them? How were these challenges addressed? The aim was to get a general picture of students’ experience with accredited MOOCs and generate more fine-grained questions for the second round of interviews. Each interview lasted for about 1 hour. Second, based on the initial interviews, we formulated 16 questions for the second round of interviews. Three of the remaining 12 students were interviewed utilising the 16 questions, which were further revised and adapted Australasian Journal of Educational Technology, 2019, 35(3). 5 to 18 questions based on the interviews. In the end, the remaining nine students were interviewed with these 18 questions (Table 1). Each interview was carried out in a semi-structured form and lasted for about 90 minutes. Third, the interviews of the 15 students were pulled together and analysed using inductive thematic analysis. The research patterns extracted from student interviews were further validated and refined through interviewing the staff in charge of accredited MOOCs at the focal university. Five staff were approached while only two agreed to be interviewed. Each interview lasted for about 1 hour. Table 1 Interview questions Number Questions 1 What was your general experience with these accredited MOOCs? 2 What were the challenges you encountered while using them? 3 How were these challenges addressed? 4 Do you like or dislike these accredited MOOCs? Please indicate your reasons. 5 What do you think of these accredited MOOCs and their associated platforms? 6 What do you think of the relationships between these accredited MOOCs and other conventional courses in your university? 7 What do you think of the role of these accredited MOOCs in the existing curricula of your university? 8 What were the supportive strategies provided by your university in facilitating the implementation of these accredited MOOCs? What do you think of them? 9 How did your university manage the implementation of these accredited MOOCs? 10 What do you think of the current collaboration between your university and the providers of these accredited MOOCs? 11 How often did you use the mobile applications of these accredited MOOCs? 12 Do you like or dislike these mobile applications? Please indicate your reasons. 13 How often did you skip these accredited MOOCs? 14 What were the countermeasures devised by your university and the MOOC platforms to prevent you from skipping these accredited MOOCs? What do you think of these countermeasures? 15 How was your performance in these accredited MOOCs evaluated? 16 What do you think of the current evaluation strategies devised by your university and these accredited MOOCs? 17 What suggestions do you have for your university to further improve its management and supportive strategies in implementing accredited MOOCs? 18 What suggestions do you have for these accredited MOOCs and their associated platforms in improving their quality and usefulness in the future? Reflexivity According to the epistemological and methodological underpinnings of qualitative research, we as the researchers constitute an important instrument of the study (Creswell, 2007). It was thus essential for us to be reflexive throughout the research process. As part of our effort, we kept a reflective diary to record our thoughts on interviews, emerging ideas during our research activities, tentative interpretations of the collected data, and decisions and changes regarding this study (Creswell, 2007; Tracy, 2013). In keeping the diary, we were driven to reflect on our evolving understanding of our research. Furthermore, the research diary kept a chronological record of our study and thereby helped us remember important details later on (Duff, 2008). It thus enabled us to revisit earlier ideas and decisions and progressively refine various aspects of our study. As such, the research diary served as an audit trail of our research and strengthened the reliability of the research findings. Thus, the research diary, besides being a data source, also became part of the analysis and interpretation process itself when we reflected on new data and themes. Australasian Journal of Educational Technology, 2019, 35(3). 6 Data analysis Each natural sentence from the interviews and university’s policy documents was considered a unit of analysis as it has clear boundaries and is reliably identified (Yin, 2009). Thematic analysis was employed to identify themes or issues and their relationships (Creswell, 2007), and general principles informed by grounded theory (Charmas, 2006) were followed in analysing the data. The thematic analysis of this study comprised three phases: initial coding, focused coding, and axial coding. Despite the conceptual separation of the three phases, the whole thematic analysis process was recursive in that we went back and forth between different phases to modify and refine the extracted codes and themes. In the phase of initial coding, we coded the data line by line together and utilised the participants’ words as codes wherever possible. This process went on iteratively until new codes were difficult to identify. In the second phase, the data were re-coded by utilising the codes that were most relevant to the research purpose. We used the research question as a guide and went back and forth between the initial codes and the data. During the focused coding phase, some initial codes were dropped, split, modified, or combined, and new codes were added, based on which a smaller number of categories were built. For instance, the emergent understanding of the initial code “management of accredited MOOCs” helped us further split it into “insufficient support and regulation from the focal university” and “mismatch between the current course assessment and the new learning form.” In the third phase, which was axial coding, we sought to identify links among these emergent categories by relating them to one another utilising both inductive and deductive approaches. During axial coding, the authors and another researcher analysed the themes and data independently. We kept the themes on which our analyses concurred in principle. For the disputed themes, we went back to the codes and data, re- analysed them with a different perspective, and identified new themes together. We did this iteratively until all our disagreements were resolved. Results Students’ mostly wary stance versus the university’s support Overall, less than half of the interviewed students supported the current format of accredited MOOCs in their curricula, while the remaining were largely wary of them. Specifically, among the 15 student interviewees, 7 held a neutral attitude towards accredited MOOCs, 6 liked them, and 2 did not like them. This was in stark contrast to the university’s support of them. The focal university expected to use MOOCs as a way to reshape students’ learning experience by providing more flexibility and course choices. Furthermore, according to the two interviewed teachers, the focal university was planning to import new MOOCs and also launch new ones in the three MOOC platforms in response to the call of the central Ministry of Education (MOE) for sharing and widening access to quality resources via the form of MOOCs among universities in the country (MOE of The People’s Republic of China [MOE of PRC], 2015). Students favouring accredited MOOCs mostly were impressed by the following characteristics: (a) high-quality course content, (b) flexibility in learning, (c) ownership of learning, and (d) diversified curriculum. These courses are taught by famous professors mostly from elite universities. For me, I trust the quality of the course content. My school did not have many of the courses before. Their existence enriches our choice of course selection. (Student 14) I can set my own pace of learning. I really like this point. I do not have to worry about being late for class or miss some important content if I cannot show up in class occasionally. (Student 3) Based on interviews with the students, reasons leading to students’ neutral perspective or dislike of accredited MOOCs were mainly attributable to MOOCs themselves and internet infrastructure. They included: (a) low quality of demonstration lectures, (b) difficulty of changing courses once registering for, (c) poor design of the Australasian Journal of Educational Technology, 2019, 35(3). 7 video lecture length, (d) insufficient server capacities of MOOC platforms, (e) slow internet speed, (f) monotonous mode of content delivery, which was unengaging, and (g) lack of interaction. Most course’ demonstration lectures cannot effectively convey a general picture of them. They were not specifically made to summarise the whole course, but were mainly selected from the normal video lectures within the courses. What I abhor most is that I can hardly change courses once I registered for them due to both administrative constraints and the limited functions of MOOC platforms. (Student 5) The length of video lectures is not very standardized. Some can be as short as two or three minutes while others can be as long as one and a half hours. I find them (the long ones) very annoying (Student 7). What really made me angry is the low server capacities of these MOOC platforms, exacerbated by the slow internet speed. For instance, in the examination period, hundreds of thousands of students from many universities across the country joining the same course took the examination at the same time by logging in to the same MOOC platform. This brought an overwhelming burden to the MOOC platforms’ servers. I experienced system crashes two times and almost lost my scores. As far as I know, many other students had similar experiences. You can see many posts complaining this issue in online forums. (Student 1). These courses feature instructors’ monologue, very dull and unengaging. When I was watching video lectures, questions constantly popped up on the screen. I find them very distracting. No courses have tutors offline to answer my questions. Although the courses have online forums, very limited interactions happen. It makes me miss the traditional face-to-face courses. (Student 4). Widespread online absenteeism All the 15 student interviewees indicated that they had skipped accredited MOOCs deliberately. Skipping MOOCs here is quite different from skipping traditional onsite courses which require physical presence. It refers to the scenario where students log in to MOOCs and leave video lectures on without watching them. Notably, skipping MOOCs is characterised by cognitive and affective absence. At times, students found other people to help them skip courses by giving these people their accounts and passwords if certain courses were installed with anti-plagiarism functions. When all lessons of a course were released at one time, the online absenteeism would be fueled. Many students even turned on several video lectures from one or two courses at the same time. The loose design of examinations may have further encouraged students’ absenteeism. As indicated by one student (Student 10): “we do not have to watch through all videos so as to pass exams because they are mostly multiple/single-choice questions”. Another student (Student 12) added that: “I can search answers online using smartphone during examinations if I find some questions challenging.” Such a loose design for examinations may nullify the purpose of full engagement in and commitment to MOOCs. Most students acknowledged that they basically skipped one or two whole courses. For courses lasting for 16 weeks, the majority of the students disclosed that it took them 4 to 5 weeks to finish the courses by utilising various techniques for skipping courses. Although they considered skipping these courses intentionally to be unethical, they found it hard not to do so in view of the unengaging instruction and considerable amount of homework from other courses. Furthermore, the testimonies of several students (Students 1, 10, 11, and 12) suggest that skipping MOOCs may be a commonly accepted practice for most students, not only in the focal university, but also in many other universities. According to Student 12, the rampant absenteeism even bred a new business at the focal university and many other universities. Some students earned around 400 dollars a month by taking accredited MOOCs for others. This was echoed by the observation of other students (Students 4, 11, and 15), who indicated that there were Australasian Journal of Educational Technology, 2019, 35(3). 8 software development companies making a profit by developing add-ons to help students skip these courses without being caught. The focal university realised the widespread absenteeism a long time ago, but had limited power to monitor and regulate students’ online engagement. It primarily depended on the three MOOC platforms to detect and maintain students’ online attendance, which is not always reliable as many students simply leave videos on without watching them. Although one MOOC platform upgraded its system by treating 5 to 10 minutes’ inactivity as a null learning behavior and popping up warning messages or questions subsequently, many students turned to paying other students to take MOOCs for them. As Teacher 1 lamented: We have noticed this issue (online absenteeism) for a long time. Currently we took two countermeasures: (a) launch a series of punishment policies seeking to deter students from doing so; (b) require the cooperation from the course providers to develop effective anti-plagiarism functions. But considering the large student population and the constantly upgraded plagiarism software, these measures are not effective. Separation between accredited MOOCs and traditional onsite courses in the focal university According to most participants, accredited MOOCs paralleled traditional onsite courses at the university. There was neither connection nor relation between these two types of courses. Teacher 1 disclosed the reason behind such arrangement: Even though an ideal move would be connecting online courses with offline course to forge a fully integrated curriculum, but such combination will entail a major overhaul and redesign of our university’s entire curricula across all departments. This move is very costly in finance and talent, and cannot be completed within a short timeframe. Moreover, we lack high-profile administrators who have the determination and vision to do so. Teacher 2 added that simply adding accredited MOOCs into the university’s formal curriculum was the practice followed by many universities currently, and that it reduced the administrative costs. In addition, Teacher 1 believed that such a simple combination without deleting or modifying onsite courses in existing formal curricula would not involve laying off any staff or reshuffling departments. Mismatch between the conventional course assessment system and the new course format Both interviewed teachers indicated that the traditional examination system was still applied to evaluate students’ performance of online learning. The final examination scores of these MOOCs were utilised as the single criterion determining whether a student passed a course or not. They worried that the mismatch between the traditional course assessment system and the new learning format might greatly hinder the integration of MOOCs in formal educational curricula and students’ attitudes towards and acceptance of them. The two teachers admitted that there had not been any follow-up study investigating the effectiveness of current strategies used to implement these accredited MOOCs since their introduction three years ago. No improvement proposals or plans had been initiated. Insufficient regulation, support, and incentives from the focal university According to the two teachers, there was basically neither academic nor social support from the focal university in enhancing students’ engagement in accredited MOOCs. Relevant policies in the focal university only stressed the academic credit these courses carried, possible benefits associated with them, and students’ obligatory duty to attend the courses, without proposing any incentive strategies. Although one policy mentioned that each department should appoint at least one staff member in charge of the implementation of accredited MOOCs, most of the interviewed students indicated they did not know who was actually in charge of it. No specific information in the policy indicated whom to consult with when students encountered academic problems and Australasian Journal of Educational Technology, 2019, 35(3). 9 where and how to find support. Teacher 2 stated: “staff normally do not want to be responsible for the extra workload imposed on them”. Merely relying on students’ self-regulation to engage in the courses, without even guiding students to improve their self-regulation, would be unrealistic. As disclosed by several students (Students 3, 8, 10, and 13), some accredited MOOCs even foster their habit of procrastination. They bemoaned that without proper support, regulation, and human interaction, they tended to procrastinate until the end period of the courses and spend the last few weeks rushing through them. Students’ voices regarding the future improvement of accredited MOOCs Despite the mentioned drawbacks associated with accredited MOOCs, most of the interviewed students still wanted to keep such course form, particularly in view of the flexibility and convenience associated with them. Meanwhile, students were expecting these courses to be improved. Their proposals in this regard are summarised in the following: MOOCs and the associated platforms • Making quality demonstration lectures of MOOCs to effectively inform students’ course selection; • Enabling students to change courses easily within a timeframe, e.g., one or two weeks after the course selection; • Standardising video length of MOOCs into series of knowledge blocks of 10 to 20 minutes; • Enhancing server capabilities of MOOC platforms to keep students’ data safe; • Adding more interactive functions to increase students’ engagement; • Improving virtual learning communities to encourage peer learning; • Providing online/offline tutors or instructors to answer students’ queries and questions; • Releasing video lectures one by one rather than all of them at one time as one way to decrease online absenteeism; and • Restructuring the instructional formats to make MOOCs attractive, for instance, experimenting with game-based learning in some MOOCs as many students grew up influenced by the proliferation of online games; gamifying some MOOCs could bridge their past experience with MOOCs, thereby, increasing students’ acceptance of and engagement in MOOCs. The focal university • Upgrading internet speed, particularly the wireless internet speed, to provide students a smooth online learning experience; and • Providing a series of support to facilitate students’ engagement in and commitment to these courses, such as offering effective guidance regarding which courses to register for and how to effectively use them, routinely organising online and offline seminars to enhance interactions among students and between students and instructors, and providing incentives to enhance students’ motivation and engagement. Discussion and conclusion This study investigated a 3-year implementation of accredited MOOCs in a conventional university in China by interviewing 15 students and 2 staff and examining relevant university policy documents. All data were analysed using thematic analysis. From the interviewees’ feedback and the examination of the three MOOC platforms, the accredited MOOCs at the focal university were found to be mostly xMOOCs, which are often instructor-centered with limited human interaction. For learners studying MOOCs mostly based on their interest and personal demands, xMOOCs would provide a content-rich learning resource. However, when MOOCs are assigned academic credit and become mandatory for all students in a university, xMOOCs may not be very suitable. Other more interactive and content-rich forms of MOOCs are needed. Australasian Journal of Educational Technology, 2019, 35(3). 10 Regarding the first research question about students’ responses to accredited MOOCs in their curricula, students were found to mostly take a wary stance towards the current accredited MOOCs. Less than half of the interviewed students supported the current course format. Meanwhile, the focal university embraced accredited MOOCs and was seeking to further upgrade them. In this regard, this finding is different from the study of Cross and Whitelock (2017), which argued that not only universities but also students held reservations about the integration of MOOCs in their curricula and the associated credentialing. Some of the reasons for students’ favour and disfavour of accredited MOOCs have been reported by previous studies, for instance, flexibility in learning, unengaging instruction, and lack of interaction (e.g., Hew & Cheung, 2014; Margaryan et al., 2015). Other reasons may be new when accredited MOOCs become a part of formal curricula in conventional universities, for example, the lack of quality demonstration video lectures to inform students’ course selection, difficulty in changing courses after course selection, and insufficient server capacities of MOOC platforms during examination periods. Many reasons causing students’ disfavour of accredited MOOCs have to do with MOOCs themselves and their platforms. Despite growing up as the computer literate generation, the interviewed students still found it hard to be engaged in MOOCs due to the lack of social presence, interaction, and support in MOOC platforms. Thus, the priority of future work should start by enhancing the ease of use and the instructional design of courses. The focal university’s support of accredited MOOCs may be attributed to two factors. First, the focal university aimed to provide quality learning and teaching resources to its students while improving their learning experience through MOOCs. Second, unlike many Western universities which are managed by university boards of trustees and/or regents, both public and private universities in China are accountable to the central MOE. In view of the tension between the large student population and the limited quality learning resources in China, the central MOE has called for the development of MOOCs and welcomed the involvement of commercial organisations to collaborate with universities in developing and implementing quality MOOCs (MOE of PRC, 2013, 2015; Zhu, Li, & Liu, 2014). Thus, the focal university was liable to respond to the call of central MOE with action. Together with accreditation, students in the focal university were obligated to complete the courses. Otherwise, they could not graduate with sufficient credits. But when many of the courses were not engaging and lacked academic support, and when third-party tools were developed to aid students to cheat and effective regulations were not in place, widespread online absenteeism happened easily. All interviewed students, including those with high GPAs, skipped online courses intentionally, which would not have happened so rampantly in traditional onsite courses. Instead of bringing benefits for students, the accredited MOOCs even risked cultivating bad habits among them. This issue should attract policymakers’ attention. In fact, the issue of widespread online absenteeism is mainly caused by both the flawed accredited MOOCs and the ineffective management of the focal university, thus, the resolution of this issue should start by solving these two problems rather than launching new deterrent strategies punishing students who skip courses. As to the second research question regarding how accredited MOOCs reconciled with existing onsite course in the focal university, this study found that accredited MOOCs stood independently in the entire curricula and the two course formats were separate from each other, thereby leaving both intact. This is different from previous studies (e.g., Griffiths et al., 2014; Israel, 2015), which indicated that the integration of MOOCs in conventional universities mostly took the form of blended learning, involving the redesign and reconstruction of existing onsite courses. Nevertheless, in the focal university, there was basically no interaction between online and offline learning. Accredited MOOCs did not seek to enhance the effectiveness of conventional face-to-face courses, and vice versa. Even though such form of course integration enriched the focal university’s existing curricula and minimised the administrative costs, students’ learning burden was increased as they needed to take four more courses. In this regard, students’ complaints such as the lack of offline support and the issue of widespread online absenteeism can be partly attributed to this separate course arrangement. Australasian Journal of Educational Technology, 2019, 35(3). 11 In terms of the third research question about the challenges and problems encountered during the implementation of accredited MOOCs in the focal university, this study primarily summarised two themes: (1) a mismatch between the conventional course assessment system and the new course format; and (2) insufficient regulation, support, and incentives from the focal university. The question regarding how to effectively evaluate students’ performance in MOOCs has been perplexing for educational practitioners. It is still largely an unsolved problem and an ongoing research topic (Meek, Blakemore, & Marks, 2017; Sharrock, 2015). In whatever sense, utilising the final examination score of MOOCs as the sole criterion to measure students’ online performance is neither rigorous nor reliable. The learning process should be valued. Formative assessments should be used together with summative ones. Due to the large scale of MOOCs, it would be impossible to ask instructors or tutors of MOOCs to conduct all assessments. More rigorous technology-based assessments should be devised. Machine grading, peer grading, and expert scoring may be used together (Cooper & Sahami, 2013; Sandeen, 2013). Furthermore, technology- based assessment is preferred in combination with adaptive learning and feedback mechanisms to fully exploit the potential of MOOCs. Students of higher capabilities are automatically led to higher levels of course content based on the assessment outcome. Students of lower capabilities are automatically provided with more similar course content to strengthen their understanding of it. Overall, it would be inappropriate to manage accredited MOOCs using the same strategies being used to manage traditional face-to-face courses. If MOOCs are to deliver high-quality learning experience for students, new pedagogical designs and organisational mechanism are needed (Yuan & Powell, 2013). A fundamental transformation of them may not be possible within a short timeframe. Future effort may seek to first bridge MOOCs and traditional courses (Sandeen, 2013). This involves a deepened collaboration across universities importing and exporting MOOCs, which may facilitate the integration of high-quality learning and teaching resources scattered across different universities. In addition, a strong partnership among students, universities, and MOOC platforms should be built. Effective integration of MOOCs in formal higher education has the potential to bring positive disruption that could lead to the refined curriculum and policy of a university and subsequently to its improved competitiveness in comparison with its national and international counterparts. As the implementation of MOOCs in the focal university was basically a process from the top down, imposed by university administrators, the challenges and problems arising during the implementation of accredited MOOCs can be largely attributed to the hasty design and management of the focal university’s administrators regarding how to effectively integrate accredited MOOCs into the curricula university-wide. When the focal university started to introduce accredited MOOCs into its existing curricula in 2013, there were limited studies theoretically and, particularly, empirically exploring how to make MOOCs effective for student learning, let alone research on the integration of accredited MOOCs into conventional universities. Without theoretical guidance and practical examples to follow, it is not advisable to advance the agenda of accredited MOOCs at the university-wide level. Instead, it is preferable to experiment with several departments before moving on to the whole university. In this regard, the focal university did not make sufficient preparation work beforehand. Regarding the revision of accredited MOOCs, it should not simply come from a designer’s head. Designers of accredited MOOCs should listen carefully to their users’ needs, expectations, and problems encountered. In doing so, they increase the possibility of designing more engaging accredited MOOCs for students. Besides the solution strategies proposed by students in the section of results, four more should be considered: 1. Many instructors, including famous professors in leading universities, may not be used to teaching and organising learning activities in virtual platforms. They may have insufficient knowledge regarding the effective delivery of course content and management of virtual classes. In this regard, more future research in this area is needed and more professional development should be provided. 2. Authenticating and proctoring examinations should be strengthened to enhance the credibility of accredited MOOCs and to minimise/avoid plagiarism. MOOC platforms may try identity-verifying technologies, for instance, photo identification, key-stroke recognition, and biometric techniques. Australasian Journal of Educational Technology, 2019, 35(3). 12 Plagiarism detection software can be embedded in accredited MOOC platforms. As to onsite examinations, a controlled and proctored environment should be sustained. 3. Approaches to integrating MOOCs into universities’ formal curricula are as diverse as the universities themselves are. Each university is advised to explore its own way by learning from the existing best examples of other universities domestically and abroad, regularly evaluating the effectiveness of its approach, and timely refining it. 4. Currently, the accredited MOOCs in the three MOOC platforms are verified by the universities exporting these courses, which may lead people to question their trustworthiness and quality assurance. As accredited MOOCs have swelled in numbers in recent years, a third party should intervene to give an authentic evaluation and certification of the quality of these courses, in order to protect student interests and sustain high-quality higher education. For this, we can learn from the Quality Assurance Agency for Higher Education (QAA) in the UK and American Council on Education College Credit Recommendation Service (ACE CREDIT) in the US (Hayes, 2015; Sandeen, 2013). Nevertheless, several limitations of this study deserve proper attention. First, even though this study tried to include participants from different disciplines and academic backgrounds, the sample size should be increased to enhance the quality of the research findings. Second, the data from the three rounds of data collection may not be equivalent as different sets of semi-structured interview questions were used for each round. As a result, the research findings may be affected. Third, the data sources from the interviews with students and staff and the universality policy documents may not be sufficient to capture the implementation of accredited MOOCs in the focal university in its full complexity. Future research should consider more data sources, for instance, students’ reflection journals while they are studying the MOOCs and interviews with students and other stakeholders at critical moments in the implementation of accredited MOOCs. To conclude, accredited MOOCs in the focal university were both a boon and bane for students. The implementation of accredited MOOCs in the focal university leaves substantial room for improvement. The reasons behind students’ supportive attitudes towards accredited MOOCs, including high-quality course content, flexibility, and ownership of learning, and students’ expectations of keeping them in their curricula together pave the road for their future improvement. High-ranked administrators in universities may have a more important role to play during the process, as the implementation of accredited MOOCs in conventional higher education is basically a top-down process, instead of a bottom-up one. It asks for the vision, insight, consistency, and determination of policymakers. Acknowledgement The authors are deeply grateful to the anonymous reviewers and editors for their constructive and insightful feedback on our manuscript. References Bruff, D. O., Fisher, D. H., McEwen, K. E., & Smith, B. E. (2013). Wrapping a MOOC: Student perceptions of an experiment in blended learning. Journal of Online Learning and Teaching, 9(2), 187-199. http://jolt.merlot.org/vol9no2/bruff_0613.pdf Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. London: Sage. Cooper, S., & Sahami, M. (2013). Reflections on Stanford's MOOCs. Communications of the ACM, 56(2), 28- 30. Retrieved from https://dl.acm.org/citation.cfm?id=2408776.2408787&coll=portal&dl=ACM Creswell, J. (2007). Qualitative inquiry and research method: Choosing among five approaches (2nd ed.). Thousand Oaks, CA: Sage. Australasian Journal of Educational Technology, 2019, 35(3). 13 Cross, S., & Whitelock, D. (2017). Similarity and difference in fee-paying and no-fee learner expectations, interaction and reaction to learning in a massive open online course. Interactive Learning Environments, 25(4), 439-451. https://doi.org/10.1080/10494820.2016.1138312 Downes, S. (2013). The quality of massive open online courses. In B. H. Khan & M. Ally (Eds.), International handbook of e-learning (Vol. 1, pp. 65-77). UK: Routledge. https://www.routledgehandbooks.com/doi/10.4324/9781315760933.ch4 Duff, P. (2008). Case study research in applied linguistics. New York, NY: Lawrence Erlbaum. Fox, R. (2016). MOOC impact beyond innovation. In C. C. Ng, R. Fox, & M. Nakano (Eds.), Reforming learning and teaching in Asia-Pacific universities (pp. 159-172). Singapore: Springer. Griffiths, R. J., Chingos, M. M., Spies, R., & Mulhern, C. (2014). Adopting MOOCs on campus: A collaborative effort to test MOOCs on campuses of the university system of Maryland. Online Learning, 19(2), 7-21. http://dx.doi.org/10.24059/olj.v19i2.523 Hayes, S. (2015). MOOCs and quality: A review of the recent literature. Gloucester: The Quality Assurance Agency for Higher Education. Retrieved from http://publications.aston.ac.uk/26604/1/MOOCs_and_quality_a_review_of_the_recent_literature.pdf Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45-58. https://doi.org/10.1016/j.edurev.2014.05.001 Hollands, F. M., & Tirthali, D. (2014). MOOCs: Expectations and reality. Full report. New York, NY: Columbia University. Retrieved from https://files.eric.ed.gov/fulltext/ED547237.pdf Israel, M. J. (2015). Effectiveness of integrating MOOCs in traditional classrooms for undergraduate students. The International Review of Research in Open and Distributed Learning, 16(5), 102-118. https://doi.org/10.19173/irrodl.v16i5.2222 Krause, S. D., & Lowe, C. (2014). Invasion of the MOOCs: The promises and perils of massive open online courses. Anderson, SC: Parlor Press. Kursun, E. (2016). Does Formal Credit Work for MOOC-Like Learning Environments? The International Review of Research in Open and Distributed Learning, 17(3), 75-91. http://dx.doi.org/10.19173/irrodl.v16i5.2222 Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of massive open online courses (MOOCs). Computers & Education, 80, 77-83. https://doi.org/10.1016/j.compedu.2014.08.005 Meek, S. E., Blakemore, L., & Marks, L. (2017). Is peer review an appropriate form of assessment in a MOOC? Student participation and performance in formative peer review. Assessment & Evaluation in Higher Education, 42(6), 1000-1013. https://doi.org/10.1080/02602938.2016.1221052 Ministry of Education of The People’s Republic of China. (2013). Notice on the establishment of MOE Researech Center for Online Education. Retrieved from http://www.rcoe.edu.cn/?cat=1 Ministry of Education of The People’s Republic of China. (2015). On the development and management of online open courses in higher education institutions. Retrieved from http://old.moe.gov.cn/publicfiles/business/htmlfiles/moe/s7056/201504/186490.html Patton, M. Q. (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage. Sandeen, C. (2013). Integrating MOOCs into traditional higher education: The emerging “MOOC 3.0” era. Change: The Magazine of Higher Learning, 45(6), 34-39. https://doi.org/10.1080/00091383.2013.842103 Schuwer, R., Jaurena, I. G., Aydin, C. H., Costello, E., Dalsgaard, C., Brown, M., ... Teixeira, A. (2015). Opportunities and threats of the MOOC movement for higher education: The European perspective. The International Review of Research in Open and Distributed Learning, 16(6), 20-38. http://www.irrodl.org/index.php/irrodl/article/view/2153/3522 Seidman, I. (2013). Interviewing as qualitative research: A guide for researchers in education and the social sciences. New York, NY: Teachers College Press. Selwyn, N., Bulfin, S., & Pangrazio, L. (2015). Massive open online change? Exploring the discursive construction of the ‘MOOC’in newspapers. Higher Education Quarterly, 69(2), 175-192. https://doi.org/10.1111/hequ.12061 Sharrock, G. (2015). Making sense of the MOOCs debate. Journal of Higher Education Policy and Management, 37(5), 597-609. https://doi.org/10.1080/1360080X.2015.1079399 Australasian Journal of Educational Technology, 2019, 35(3). 14 Smith, B., & Eng, M. (2013). MOOCs: A learning journey. In S. K. S. Cheung, J. Fong, W. Fong, F. L. Wang, & L. F. Kwok (Eds.), Hybrid learning and continuing education (pp. 244-255). Berlin Heidelberg: Springer. Tracy, S. J. (2013). Qualitative research methods: Collecting evidence, crafting analysis, communicating impact. Malden, MA: Wiley-Blackwell. Universities UK. (2013). Massive open online courses: Higher education's digital moment? Retrieved from https://www.universitiesuk.ac.uk/policy-and-analysis/reports/Documents/2013/massive-open-online- courses.pdf Yin, R. K. (2009). Case study research: Design and methods (4th ed.). Los Angeles, CA: Sage. Yuan, L., & Powell, S. (2013). MOOCs and open education: Implications for higher education (A White Paper). Bolton: JISC CETIS. Retrieved from http://publications.cetis.org.uk/wp- content/uploads/2013/03/MOOCs-and-Open-Education.pdf Zhou, J. (2017). Exploring the factors affecting learners' continuance intention of MOOCs for online collaborative learning: An extended ECM perspective. Australasian Journal of Educational Technology, 33(5), 123-135. https://doi.org/10.14742/ajet.2914 Zhu, C., Li, G. L., & Liu, H. T. (2014). The developments of Chinese MOOC platforms. Journal of Higher Education Research, 37(2), 15-19. Retrieved from http://gdjyyjxb.nudt.edu.cn/ch/reader/create_pdf.aspx?file_no=20140203&flag=1&journal_id=gdjyyjxb& year_id=2014 Zou, T. Q., & Huang, L. L. (2014). A study on the characteristics of post 90s and the innovation of teaching models for them in the Internet era. China University Teaching, 1, 38-41. Retrieved from http://kns.cnki.net/KCMS/detail/detail.aspx?filename=jxcy201401009&dbname=CJFD&dbcode=CJFQ Corresponding author: Xinghua Wang, wangxinghua379@gmail.com Australasian Journal of Educational Technology © 2019. Please cite as: Wang, X. H., Hall, A. H., & Wang, Q. Y. (2019). Investigating the implementation of accredited MOOCs in higher education: The boon and the bane. Australasian Journal of Educational Technology, 35(3), 1-14. https://doi.org/10.14742/ajet.3896 mailto:wangxinghua379@gmail.com https://doi.org/10.14742/ajet.3896 Xinghua Wang Nanyang Technological University, Singapore Allison. H. Hall University of Illinois at Chicago Qiyun Wang Nanyang Technological University, Singapore Introduction Literature review Types of MOOCs Integration of MOOCs into conventional institutions of higher education Current status of MOOCs being integrated into conventional universities Transition of MOOCs from informal education to formal education via credentialing Forms of integration of MOOCs into conventional universities Methodology Research site Participants Research design and data collection Table 1 Reflexivity Data analysis Results Students’ mostly wary stance versus the university’s support Widespread online absenteeism Separation between accredited MOOCs and traditional onsite courses in the focal university Mismatch between the conventional course assessment system and the new course format Insufficient regulation, support, and incentives from the focal university Students’ voices regarding the future improvement of accredited MOOCs MOOCs and the associated platforms The focal university Discussion and conclusion Acknowledgement References Corresponding author: Xinghua Wang, wangxinghua379@gmail.com