International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 04, 2021 Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… The Relationship between Massive Online Open Courses (MOOCs) Content Design and Students’ Performance https://doi.org/10.3991/ijim.v15i04.20201 Norhidayah Mohamad (), Azrina Othman, Tan Siew Ying, Neerosha Rajah Universiti Teknikal Malaysia Melaka, Melaka, Malaysia norhidayah@utem.edu.my Norsamsinar Samsudin Universiti Pendidikan Sultan Idris, Tanjong Malim, Malaysia Abstract—Over the years, there has been tremendous growth in online so- cial networking which contributed to the revolution of higher education’s learn- ing environment. Massive Online Open Courses (MOOC) is now common in online education, which provides open networks of self-directed learning. MOOCs are referred as a continuation trend in innovation that initiated by learning from geographical distance and online. MOOCs is a current evolve- ment in higher learning institution in Malaysia with the aims to provide quality education for the students. However, there are some challenges in developing effective instructional design courses and retaining learners in MOOCs. There- fore, the purpose of this study is to discover the relationship between MOOC’s content design and students’ performances among UTeM’s engineering stu- dents. Through literature review, variables are identified such as courses content design, enhancement in teaching and learning and students’ performance. In this study, 373 samples have been collected and the data analysed using SPSS. The results revealed that there are significant relationships between courses materi- als, courses activities, and courses tools with students’ performance. It is be- lieved that this research paper will beneficial to higher education to improve MOOC’s content design in order to enhance the students’ performance in the future. Keywords—MOOC, MOOC’s Content Design, Students’ Performance, Higher Education, UTeM 1 Introduction In today’s high-tech and dynamic world, the learning environment among the learners around the world keeps changing and continuously improving with the ad- vancement of technology. The study by [1] noted that technology is a necessary in- strument for current education. The integration of technology and traditional learning has lead to the existence of online learning as well as the establishment of Massive 4 http://www.i-jim.org https://doi.org/10.3991/ijim.v15i04.20201 mailto:norhidayah@utem.edu.myorg Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… Online Open Courses (MOOCs). Currently, easy access to advanced technologies enables students to manage and monitor their learning sources [2]. MOOCs have become a national agenda in the higher education institution of Ma- laysia as it is addressed in the 11th Malaysian Plan (2016-2020). MOOCs have been launched by Universiti Teknikal Malaysia Melaka (UTeM) on 7th September 2015. In education, the main objective in learning is the outcome which is the students’ per- formance. It is because excellent academic achievement is not only a personal aim to become expert or knowledgeable people in a particular field, but also considered as the added value to retain a productive society. Hence, the relationship between MOOC’s content design and performance of students is focused on this research. Besides, studies on the relationship between MOOC’s content design and students’ performance are very limited. According to [3], there is a restricted number of studies that highlight the instructional design of MOOCs in recent years. A study by [4] no- ticed a little amount of students that able to complete a MOOC successfully which contributes to MOOCs issue. Besides, [5] found out that the dropout rate in MOOCs is quite high which encompassed around 90%. Furthermore, [6] has conducted a study about the reasons students sign up and drop the courses in MOOCs. The results showed that failure to understand content materials is one of the main causes of drop- out from MOOCs. Besides, [7] indicates that many MOOCs suffer from ineffectively implemented in- structional design principles into the courses. [8] reported that e-teaching is a faculty that requires design expertise and delivery skills. From the finding of [9], MOOCs face some challenges in ensuring the students accomplish the courses and retaining them in a particular course. Some integration of future MOOCs can be done by in- cluding some useful additional instruments to deliver interesting lecture courses and education tools in the learning platform. According to UTeM’s portal, there are six engineering faculties in total that have been offered by UTeM. Therefore, the sample size that has been selected in this pre- sent study is among engineering students at UTeM since UTeM focuses more on the engineering field and the major subjects that are being offered are Engineering sub- jects. As a result, this research is carried out to determine the effect of MOOCs’ con- tent design on the students’ performance. These research findings will reveal the overall students’ performance in utilising MOOCs platform for education purposes consequently provides some empirical data for future study. Further studies are re- quired to improve and enhance the current MOOC’s content design to ensure better students’ performance in higher education. 2 Literature Review 2.1 Courses content design Courses materials: According to [10], materials are essential in conducting teach- ing-learning activities as it can support learners with various learning patterns. For example, learning materials such as tutorial videos, electronic resources, electronic iJIM ‒ Vol. 15, No. 04, 2021 5 Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… books, and exercise sets are normally been implemented in MOOCs. [11] further added by stating MOOCs online classes do include a short tutorial video presentation, simulations, and online laboratories in combination with computer-graded exams and online forums that enable discussion of course content among learners. [12] found out that longer duration of educational videos is less engaging compared to shorter vide- os. However, there is only little significant relationship between time-consuming in viewing lecture videos and exam scores [13]. Besides, [14] reported that videos provide learners with flexibility and enable scalability of course content for MOOCs providers. MOOCs videos can be character- ized into several categories such as introductory videos with an explanation of the course and its objectives, explanation of courses content with animations and audio narration, the tutorial video provides to real learners, documentary-style video, dia- logue between tutors and others, and questions integration video [15]. Accordingly, videos providing flexibility in the learning process by allowing pause, repeat, or skip the videos which can reinforce student learning in MOOCs. Furthermore, [16] claimed that passive or declarative information is gained by stu- dents through reading, viewing or studying the video tutorials, lesson slides and other related mediums. The result has shown that the implementation of video lectures only contribute limited value to student learning. This finding was supported by the work of [17] stating readings (50%) and videos (40%) are the major supporting materials used in MOOCs; while 6% of people choose discussion forums as a helpful educa- tional resource. Hence, in open education platforms, pre-recorded videos are quite well- known and have gained positive evaluations, as a result, affect the quiz performance. Courses activities: Courses activities consist of quizzes, exams, assignments, and interactive activities is considered as active or interactive features that need students to be more active in participating in all these activities [15]. Their study revealed that student learning outcomes can be better enhanced by providing more interactive activ- ities. Learners that participate more in activities can gain knowledge more than learn- ers that viewing videos and studying slides. Their findings also showed that learning benefit from extra doing courses activities is six times more than those that extra watching or reading courses materials. The study by [18] showed that MOOCs allow participants to engage with non- graded quizzes which offer chances for students to check their understanding and knowledge. Then, they have options either to participate in a course certificate test of a particular course. There are various forms of assessment which consist of automated assessments (e.g., multiple-choice quiz), peer-assessment (e.g., students evaluate each other’s work), and self-assessment (students assess their work). Also, [19] stated that the purpose of using automated multiple-choice questions (MCQs) as one of the courses activities is to examine the knowledge and understand- ing gain by students throughout a course. However, research by [20] revealed that the development of MCQs may take a longer time and involved in difficulties in evaluat- ing a high-level of cognitive abilities. Furthermore, peer-assessment is suitable to be implemented when knowledge is incorporated into more complicated circumstances 6 http://www.i-jim.org Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… [21]. It is essential to develop assessments and scalable feedback, which reliable eval- uation process with usable feedback may help in student learning. 2.2 Enhancement in teaching and learning Courses tools: One of the MOOCs courses tools is the discussion forum. Accord- ing to [22] forums are defined as an environment for cooperation among students where they can interact and learn with each other. [23] explained that forums can facilitate learners in discussions with other members to develop new knowledge throughout a particular course. Besides, MOOC discussion forums can be referred to as spaces for the exchange of ideas in MOOCs which involves making original posts, reading posts, responding or commenting on posts [24]. MOOCs forums have many purposes and provide many benefits for the partici- pants such as students also able to verify their knowledge of a particular subject and discuss in solving some questions of a task in the forum [25,26]. MOOC forum is a good environment for peer assistance, in which all the questions from the students can be answered among themselves without involving any instructor [22]. Good course content discussions are provided by a discussion forum, which enables learners to review course materials and feedback from each other. [27] noted that those intelli- gent MOOCs users act as volunteer mentors in improving the interaction gap between students and instructors. In contrast, the study of [28] found that the discussion forum is only slightly important and seldom be used. 2.3 Students’ performance Students’ performance is consisting of different forms. The definition of perfor- mance can be modified based on the things that learners want to accomplish, such as high exam grades, an expanded social network, or several mixtures of distinct measures [29]. The study by [30] discovered that student’s behaviours in an educa- tional environment are considered as one of the forms of students’ performance. However, [4] noted that there are variables of overall participation of students in MOOCs activities, with some different styles of involvement based on their aims or objectives. In addition, [27] that user engagement during the course refers to the inter- actions with Video and Course Navigation elements among participants in MOOCs. It is therefore, the more capable and competent students can involve well and success- fully in completing the course. Another form of student performance is examination grade. According to [31], Graded Point Average (GPA) acts as an indicator that was typically used to evaluate the student’s academic performance. Most of the academic institutions required stu- dents to achieve a minimum GPA to pursue degree education. Several universities positioned 1.5 as the requirement of minimum GPA requirement. Students that achieve a GPA of 3.0 and above are considered as having excellent academic perfor- mance. Thus, GPA normally utilised by educational planners in accessing academic results. Faculty members can involve in developing strategies and supervise student performance progression to enhance their educational achievement. The work of [12] iJIM ‒ Vol. 15, No. 04, 2021 7 Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… noticed that the more courses materials a student read, the higher grades can be ob- tained due to more chances in learning and assessing information. Although there are several forms of students’ performance, however in this study, the students’ performance is only analysed in terms of capability in performing better outcomes in utilising MOOC’s content design in their learning environment. Several questions related to the students’ performance in performing better outcomes by im- plementing MOOC’s content design will be the dependent variable in this research. Other forms of students’ performance will not be included in this report analysis. 2.4 Underpinning theory Pedagogical theory: [32] in (1968) created a cognitive perspective of education (theory) in which stress on conceptual growth in learning from a meaningful, pro- found knowledge of specific presented ideas to shallow as well as rote memorization of ideas. [33] explained that when a person learns a particular concept in a meaningful manner, memory can remain for a longer period, able be apply to new issues and contexts, and enhance the capability to study other unrelated courses. Four situations contribute to the meaningful learning of concepts, such as i) when a concept is obvi- ously determined ii) when given clear examples of related concepts to reality, iii) when concepts are linked with current understanding, and iv) when learners have learning motivation in a particular topic. Well-designed course materials need to be done and provided at a suitable time in proper order. Previous work by [34] reported that ideas form a hierarchy needed to be in an appropriate sequence from simple to more abstract concepts. Educators should create a structural course content that allows learners in learning new knowledge in a hierarchically proper manner. However, the concepts that have been understood by students need to be identified by educators before designing a new course structure to enhance student’s learning. This is because too abstract or too quick a course structure will influence the learning process where students will easily be lost and get bored. Thus, instructors are required to select ap- propriate examples in guiding the students with the correct way to apply the concept in new circumstances. Research framework Fig. 1. Proposed Conceptual Framework 8 http://www.i-jim.org Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… According to Figure 1, there are several independent variables which consist of courses content design and enhancement in teaching and learning. The courses con- tent design is divided into two categories, which are courses materials and courses activities. The enhancement in teaching and learning is the courses tools. All these variables will be tested in this study to find out whether these MOOC’s content design will affect the students’ performance. Hypotheses: Based on the proposed research framework, there are some hypothe- ses have been constructed to solve the research questions and to achieve the research objectives as following: H1: There is a significant relationship between courses materials and students’ per- formance H2: There is a significant relationship between courses activities and students’ performance H3: There is a significant relationship between courses tools and students’ per- formance 3 Methodology In this research, a questionnaire survey was distributed which contains questions regarding variables to determine the relationship between MOOC’s content design on students’ performance. The questionnaire was designed into three parts that are Sec- tion A which consists of demographics information of respondents such as student ID, email, gender, faculty, and year of study. This is followed by the second part (Section B) about the independent variables where the researcher will focus on the MOOC’s content design which consists of courses content design (courses materials and cours- es activities) and enhancement in teaching and learning (courses tools). The last part of this questionnaire which is Section C is related to the dependent variables concen- trating on the performance of engineering students at UTeM. The researcher applied a 5-point Likert scale in this questionnaire to measure each answer from respondents. Stratified random sampling is chosen to run this research. UTeM has been selected as the location for this research. It is because this study is focusing on the engineering students that are currently studying at UTeM. The entire population which is the UTeM students are then divided into several strata according to their gender, faculty, and year of study. In 2019, the estimated population of UTeM’s students is 12,077 people. From the official website of UTeM, the population stated 10,738 undergradu- ate students and 1,339 postgraduate students. According to [35], the sample size is consists of 373 students due to the overall population that exceed than 10,000 peoples. UTeM established 6 engineering faculties which consist of Faculty of Electronic and Computer Engineering (FKEKK), Faculty of Electrical Engineering (FKE), Faculty of Mechanical Engineering (FKM), Faculty of Manufacturing Engineering (FKP), Facul- ty of Electrical and Electronic Engineering Technology (FTK), Faculty of Information and Communications Technology (FTMK). There are a total of 24 engineering cours- es have been offered by UTeM. However, engineering students from only four facul- ties (FKEKK, FTMK, FTK, and FKP) are selected as respondents in this research. iJIM ‒ Vol. 15, No. 04, 2021 9 Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… 4 Results 4.1 Pearson correlation Table 1. Correlations Courses Materials Courses Activities Courses Tools Students’ Performance Courses Materials Pearson Correlation 1 .881** .770** .865** Sig. (2-tailed) .000 .000 .000 N 373 373 373 373 Courses Activities Pearson Correlation .881** 1 .785** .858** Sig. (2-tailed) .000 .000 .000 N 373 373 373 Courses Tools Pearson Correlation .770** .785** 1 .819** Sig. (2-tailed) .000 .000 .000 N 373 373 373 Students’ Performance Pearson Correlation .865** .858** .819** 1 Sig. (2-tailed) .000 .000 .000 N 373 373 373 373 **Correlation is significant at the 0.01 level (2-tailed) Table 1 illustrates the findings of the the Pearson Correlation Coefficient Analysis for three interval-scale variables. Based on the result above, it is proven that all inde- pendent variables (courses materials, courses activities, courses tools) are positively and significantly associated with dependent variables (students’ performance). The correlation value between courses materials and students’ performance is the highest among other variables which constitute r = 0.865 and correlation is significant at the level of 0.000. This indicates that there is a high positive significant relationship between these two variables due to r > 0.7 and p < 0.5. Besides, the results showed that courses activities and students’ performance variables are significantly correlated due to r = 0.858 and significant at the level of 0.000. Furthermore, the correlation value between students’ performance and courses tools is 0.819 with a significant level of 0.000. This indicates that there is a high positive significant relationship as the r > 0.7, p < .05. From this analysis, it can be concluded that courses content design which includes courses materials and courses activities, as well as enhancement in teaching and learning such as courses tools, have a significant relationship with students’ performance due to their high correlation value between variables. This also indicated that courses materials, courses activities, and courses tools positively influence students’ performance. It is proven that the better the courses materials, courses activities and courses tools be implemented in MOOCs platform, the better the students’ performance that can be achieved. 10 http://www.i-jim.org Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… 5 Discussion 5.1 Objective 1: To investigate the relationship between courses content design and students’ performance H1: There is a significant relationship between courses materials and students’ performance. Based on the analysis in the previous section, it is proven that courses materials and students’ performance were significantly correlated, r (373) = .865, p < .05. Therefore, H1 is accepted in this study. This finding supported by [29] with the prov- en result of students involved in video-watching affects the students’ performance. From the study of [12], the more courses materials a student reads, the higher achievement can be obtained due to more chances in learning and assessing infor- mation. [13] reported that there is a significant relationship between time-consuming in viewing lecture videos and exam scores which contributes to these research find- ings. H2: There is a significant relationship between courses activities and students’ performance. From the analysis of the previous section, it is also proven that there is a significant relationship between courses activities and students’ performance as r > 0.7 as well as p < 0.05. Hence, H2 is also accepted. According to [16], student learning outcomes can be better enhanced by providing more courses activities such as quizzes, exams, assignment and interactive activities. The research findings by [36] which presents that weekly quizzes and peer assessment have shown significance relationship with students’ performance. It is further supported by previous research of [37] that found a strong positive relationship between the number of student activities and their final course grade. 5.2 Objective 2: To examine the relationship between enhancement in teaching and learning and students’ performance H3: There is a significant relationship between courses tools and students’ per- formance. Based on the correlation analysis, courses tools and students’ performance varia- bles were significantly correlated due to the p < 0.05 and r > 0.7. Thus, H3 is accepted in this study. This finding supports the work of [25] mentioned that students able to verify their knowledge of a particular subject and discuss in solving some questions of a task in the forum, which able to enhance the performance of students. Besides, [26] pointed out that students are making use of discussion forums to initiating conversa- tion and conduct their learning. Furthermore, [18] also concluded that MOOCs fo- rums act as a place in which students can enhance knowledge about others as well as about the course content. iJIM ‒ Vol. 15, No. 04, 2021 11 Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… 6 Conclusion This research intended to study the relationship between MOOC’s content design and students’ performance among engineering students at UTeM, Melaka. The results stated hypotheses (H1, H2, and H3) are accepted which indicates that three independ- ent variables (courses materials, courses activities, and courses tools) have a signifi- cant relationship with students’ performance. The findings of this research can act as guidelines and may benefit educators in enhancing the quality of MOOCs for their students. The educators will be more conscious of the main elements in MOOCs that influence students’ performance. Besides, the study materials in this research that gathered from various previous researches also can be taken as a source of reference to construct new courses or improve the existing courses. Hence, all the learners can utilise MOOCs courses in their education as well as improve their performances to achieve their goals. 7 Limitation and Future Research Lastly, three independent variables have been identified in this study, they are courses materials, courses activities, and courses tools. The results showed that there are high impacts of independent variables on the dependent variable which 82.1% of the variation of students’ performance is influenced by courses materials, courses activities, and courses tools. Thus, the conclusion has been drawn that there are re- maining 17.9% could affected students’ performance due to some other factors which are not considered in this study. Therefore, future researchers are suggested to include several other variables that affect students’ performance in implementing MOOCs which are not covered in this study. Hence, this can assist future researchers in in- specting and investigating more variables related to the study. More specific research also may be conducted by future researchers to explore more speculative explanations throughout the study. 8 Acknowledgement The corresponding author would like to thank Universiti Teknikal Malaysia Melaka (UTeM) and Centre for Technopreneurship Development (CTeD) for provid- ing a research grant (FRGS/1/2017/SS01/FPTT-CTED/P00347) for this research. 9 References [1] M. Fadzil, L. A. Latif and T.A.M.T.M. Azzman, "Moocs in Malaysia: A preliminary case study", 2015. [2] J. Mackness, S. Mak and R. Williams, "The ideals and reality of participating in a MOOC", in Proceedings of the 7th international conference on networked learning 2010, University of Lancaster, 2010, pp. 266-275. 12 http://www.i-jim.org Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… [3] J. Kay, P. Reimann, E. Diebold and B. Kummerfeld, "MOOCs: So many learners, so much potential…", IEEE Intelligent Systems, vol. 28, no. 3, pp. 70-77, 2013. https://doi.org/ 10.1109/MIS.2013.66 [4] D. Koller, A. Ng, C. Do and Z. Chen, "Retention and intention in massive open online courses: In Depth", Educause review, vol. 48, no. 3, pp. 62-63, 2013. [5] R. Rivard, "Measuring the MOOC Dropout Rate", Inside Higher Ed, vol 8, 2013. [6] K. F. Hew and W. S. Cheung, "Students' and instructors' use of massive open online cours- es (MOOCs): Motivations and challenges", Educational Research Review, vol. 12, no. 4, pp. 5-58, 2014. https://doi.org/10.1016/j.edurev.2014.05.001 [7] A. Margaryan, M. Bianco and A. Littlejohn, "Instructional quality of massive open online courses (MOOCs)", Computers and Education, vol. 80, pp. 77-83, 2015. https://doi.org/10. 1016/j.compedu.2014.08.005 [8] V. Salyers, L. Carter, A. Carter, S. Myers and P. Barrett, "The search for meaningful e- learning at Canadian universities: A multi-institutional research study", International Re- view of Research in Open and Distance Learning, vol. 15, no.6, pp.313-337, 2014. https://doi.org/10.19173/irrodl.v15i6.1713 [9] S. I. De Freitas, J. Morgan and D. Gibson, "Will MOOCs transform learning and teaching in higher education? Engagement and course retention in online learning provision", Brit- ish Journal of Educational Technology, vol. 46, no. 3, pp. 455-471, 2015. https://doi.org/10. 1111/bjet.12268 [10] B. F. Klimova and P. Poulova, "Impact of a form of online materials on the quality of edu- cation-a case study", International Journal of Digital Information and Wireless Communi- cations (IJDIWC), vol. 3, no.1, pp. 43-49, 2013. [11] M. B. Hoy, "MOOCs 101: An Introduction to Massive Open Online Courses", Medical Reference Services Quarterly, vol. 33, no. 1, pp. 85-91, 2014. https://doi.org/10.1080/ 02763869.2014.866490 [12] P. J. Guo and K. Reinecke, "Demographic differences in how students navigate through MOOCs", in Proceedings of the first ACM conference on Learning@ scale conference, pp. 21-30, 2014. https://doi.org/10.1145/2556325.2566247 [13] S. R. Emmons, R. P. Light and K. Börner, "MOOC visual analytics: Empowering students, teachers, researchers, and platform developers of massively open online courses", Journal of the Association for Information Science and Technology, vol. 68, no. 10, pp. 2350- 2363, 2017. https://doi.org/10.1002/asi.23852 [14] Y. C. Lee, W. C. Lin, F. Y. Cherng, H. C. Wang, C. Y. Sung and J. T. King, "Using Time- Anchored Peer Comments to Enhance Social Interaction in Online Educational Videos", In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Sys- tems, pp. 689-698, 2015. https://doi.org/10.1145/2702123.2702349 [15] N.P. Morris and J. Lambe, Studying a MOOC: a guide. Palgrave MacMillan, 2014. [16] K. R. Koedinger, E A. Mclaughlin, J. Kim, J. Z. Jia and N. L. Bier, "Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC", in Proceed- ings of the Second (2015) ACM Conference on Learning @ Scale, pp. 111-120, 2015. https://doi.org/10.1145/2724660.2724681 [17] M. Giannakos, M. L. Jaccheri and J. Krogstie, "Looking at MOOCs rapid growth through the lens of video-based learning research", International Journal of Emerging Technologies in Learning (IJET), vol. 9, no. 1, pp. 35-38, 2014. https://doi.org/10.3991/ijet.v9i1.3349 [18] F. Bonafini, C. Chae, E. Park and K. Jablokow, "How Much Does Student Engagement with Videos and Forums in a MOOC Affect Their Achievement?", Online Learning Jour- nal, vol. 21, no.4, pp.223-240, 2017. https://doi.org/10.24059/olj.v21i4.1270 iJIM ‒ Vol. 15, No. 04, 2021 13 https://doi.org/10.1109/MIS.2013.66 https://doi.org/10.1109/MIS.2013.66 https://doi.org/10.1016/j.edurev.2014.05.001 https://doi.org/10.1016/j.compedu.2014.08.005 https://doi.org/10.1016/j.compedu.2014.08.005 https://doi.org/10.19173/irrodl.v15i6.1713 https://doi.org/10.1111/bjet.12268 https://doi.org/10.1111/bjet.12268 https://doi.org/10.1080/02763869.2014.866490 https://doi.org/10.1080/02763869.2014.866490 https://doi.org/10.1145/2556325.2566247 https://doi.org/10.1002/asi.23852 https://doi.org/10.1145/2702123.2702349 https://doi.org/10.1145/2724660.2724681 https://doi.org/10.3991/ijet.v9i1.3349 https://doi.org/10.24059/olj.v21i4.1270 Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… [19] R. O'Toole, "Pedagogical strategies and technologies for peer assessment in Massively Open Online Courses (MOOCs)", University of Warwick, Coventry, 2013. Available: http://wrap.warwick.ac.uk/54602. [20] H.K. Suen, "Peer assessment for massive open online courses (MOOCs)", International Re-view of Research in Open and Distance Learning, vol. 15, no. 3, pp. 312-327, 2014. https://doi.org/10.19173/irrodl.v15i3.1680 [21] D. K. Comer, C. R. Clark and D. A. Canelas, "Writing to Learn and Learning to Write across the Disciplines: Peer-to-Peer Writing in Introductory-level MOOCs", The Interna- tional Re-view of Research in Open and Distributed Learning, vol. 15, no. 5, pp. 26-82, 2014. https://doi.org/10.19173/irrodl.v15i5.1850 [22] D. Coetzee, A. Fox, M. A. Hearst, and B. Hartmann, "Should your MOOC forum use a reputation system?", in Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, pp.1176-1187, 2014. https://doi.org/10. 1145/2531602.2531657 [23] M. Dubosson, and S. Emad, "The Forum Community, the Connectivist Element of an xMOOC", Universal Journal of Educational Research, vol. 3, no. 10, pp. 680-690, 2015. https://doi.org/10.13189/ujer.2015.031004 [24] A. Sharif and B. Magrill, "Discussion Forums in MOOCs", International Journal of Learn- ing, Teaching and Educational Research, vol. 12, no. 1, pp. 119-132, 2015. [25] J. R. Young, "Providers of Free MOOC's Now Charge Employers for Access to Student Data", The Chronicle of Higher Education. Available: https://www.chronicle.com /article/providers-of-free-moocs-now/136117 [26] A, Darabi, M.C. Arrastia, D.W. Nelson, T. Cornille, and X. Liang, "Cognitive presence in asynchronous online learning: A comparison of four discussion strategies", Journal of Computer Assisted Learning, vol. 27, no. 3, pp. 216-227, 2011. https://doi.org/10.1111/j. 1365-2729.2010.00392.x [27] C. Alario-Hoyos, M. Pérez-Sanagustín, C. Delgado-Kloos, M. Muñoz-Organero, and A. Rodríguez-De-Las-Heras, "Analysing the impact of built-in and external social tools in a MOOC on educational technologies", in European Conference on Technology Enhanced Learning, Springer, Berlin, Heidelberg, pp. 5-18, 2013. https://doi.org/10.1007/978-3-642- 40814-4_2 [28] M. Vitiello, S. Walk, V. Chang, R. Hernandez, D. Helic and C. Guetl, "MOOC dropouts: A multi-system classifier, in European Conference on Technology Enhanced Learning, Springer, Cham, pp. 300-314, 2017. https://doi.org/10.1007/978-3-319-66610-5_22 [29] C. G. Brinton, M. Chiang, S. Jain, H. Lam, Z. Liu and F.M.F. Wong, "Learning about so- cial learning in MOOCs: From statistical analysis to generative model", IEEE Transactions on Learning Technologies, vol. 7, no. 4, pp. 346-359, 2014. https://doi.org/10.1109/ TLT.2014.2337900 [30] S. T. Hijazi and S. M. M. Naqvi, "Factors Affecting Students' Performance: A Case of Pri- vate College", Bangladesh E-Journal of Sociology, vol. 3, no. 1, pp. 1-10, 2006. [31] S. S. Sansgiry, M. Bhosle and K. Sail, "Factors that affect academic performance among pharmacy students", American Journal of Pharmaceutical Education, vol. 70, no. 5, 2006. https://doi.org/10.5688/aj7005104 [32] D. P. Ausubel, "The psychology of meaningful learning; an introduction to school learn- ing", Grune and Stratton, 1968. [33] J. D. Novak, "Learning creating and using knowledge: Concept maps as facilitative tools in schools and corporations", Routledge, 2010. Available: https://doi.org/10.4324/97802 03862001. 14 http://www.i-jim.org http://wrap.warwick.ac.uk/54602 https://doi.org/10.19173/irrodl.v15i3.1680 https://doi.org/10.19173/irrodl.v15i5.1850 https://doi.org/10.1145/2531602.2531657 https://doi.org/10.1145/2531602.2531657 https://doi.org/10.13189/ujer.2015.031004 https://www.chronicle.com/article/providers-of-free-moocs-now/136117 https://www.chronicle.com/article/providers-of-free-moocs-now/136117 https://doi.org/10.1111/j.1365-2729.2010.00392.x https://doi.org/10.1111/j.1365-2729.2010.00392.x https://doi.org/10.1007/978-3-642-40814-4_2 https://doi.org/10.1007/978-3-642-40814-4_2 https://doi.org/10.1007/978-3-319-66610-5_22 https://doi.org/10.1109/TLT.2014.2337900 https://doi.org/10.1109/TLT.2014.2337900 https://doi.org/10.5688/aj7005104 https://doi.org/10.4324/9780203862001 https://doi.org/10.4324/9780203862001 Paper—The Relationship Between Massive Online Open Courses (MOOCs) Content Design and… [34] L. VanDamme, "The hierarchy of knowledge: The most neglected issue in education", The Objectivist Standard, vol. 1, no. 1, I, 2006. [35] R. V. Krejcie and D. W. Morgan, "Determining sample size for research activities", Educa- tional and psychological measurement, vol. 30, no. 3, pp. 607-610, 1970. https://doi.org/10.1177/001316447003000308 [36] W. Admiraal, B. Huisman and M. Van de Ven, Self- and Peer Assessment in Massive Open Online Courses", International Journal of Higher Education, vol. 3, no. 3, pp. 119- 128, 2014. [37] J. DeBoer, A. D. Ho, G. S. Stump and L. Breslow, "Changing 'course' reconceptualizing educational variables for Massive Open Online Courses", Educational Researcher, vol. 43, no. 2, pp. 74-84, 2014. https://doi.org/10.3102/0013189X14523038 10 Authors Dr. Norhidayah binti Mohamad is a senior lecturer at the Department of Tech- nopreneurship in Faculty of Technology Management and Technopreneurship, Uni- versiti Teknikal Malaysia Melaka (UTeM). Azrina binti Othman is a lecturer at the Department of Technology Management in Faculty of Technology Management and Technopreneurship, Universiti Teknikal Malaysia Melaka (UTeM). Dr. Norsamsinar binti Samsudin is a senior lecturer at the Department of Busi- ness Management and Entrepreneurship in Faculty of Management and Economics, Universiti Pendidikan Sultan Idris (UPSI). Tan Siew Ying is a final year student pursuing a Bachelor of Technopreneurship in Faculty of Technology Management and Technopreneurship, Universiti Teknikal Malaysia Melaka (UTeM). Neerosha a/p Rajah is a postgraduate student pursuing Masters of Research in En- trepreneurship in Faculty of Technology Management and Technopreneurship, Uni- versiti Teknikal Malaysia Melaka (UTeM). Article submitted 2020-12-01. Resubmitted 2021-01-07. Final acceptance 2021-01-08. Final version published as submitted by the authors. iJIM ‒ Vol. 15, No. 04, 2021 15 https://doi.org/10.1177/001316447003000308 https://doi.org/10.3102/0013189X14523038