International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 11, 2021 Paper—Success and Failure Aspects of LMS in E-Learning Systems Success and Failure Aspects of LMS in E-Learning Systems https://doi.org/10.3991/ijim.v15i11.20805 Abdulsalam K. Alhazmi () University of Science and Technology, Aden, Yemen abdulsalamkaed@gmail.com Athar Imtiaz Massey University, Palmerston North, New Zealand Fatima Al-Hammadi, Ezzadeen Kaed Interactive Language Centre, Kuala Lumpur, Malaysia Abstract—Learning management systems (LMS) and their associated tools have created value for higher education institutions worldwide by improving con- tent deliverability, accessibility, and retrievability. Nevertheless, many studies have criticized these systems for their teacher-centred approaches, limiting op- portunities for social and informal learning. After several decades of institutions using LMS, this research aims to discuss the success and failure aspects of these LMS from various perspectives, including structure, content, and support for emerging technologies. A thematic literature review and analysis, success and failure aspects are organized and verified by LMS experts. Alongside the aspects identified from the literature, these experts also noted several additional aspects of success and failure in LMS. Overall, the analysis results indicated the success aspects of LMS relate to seven key factors: single sign-on, learning management, content management, integration, security, tracking, and group management. Similarly, the failure aspects could be classified into eight factors: content crea- tion and sharing, communicative features, structure, learning engagement, assess- ment, user interfaces, social and informal learning, and mobile features. The im- plications of the research for education institutions, instructors, developers, and system providers and the resulting directions for future research are thus also dis- cussed. Keywords—E-learning, Learning Management System (LMS), Course Man- agement System (CMS), Online Learning, Emerging Technologies, Learning Outcomes, e-Assessment, Student Engagement 1 Introduction In recent years, there has been a growing demand for e-learning technologies, while the use of learning management systems (LMS) has been seen increasingly among all education institutes as these attempts to develop better management of various iJIM ‒ Vol. 15, No. 11, 2021 133 https://doi.org/10.3991/ijim.v15i11.20805 mailto:a.alhazmi@ust.edu Paper—Success and Failure Aspects of LMS in E-Learning Systems education-related tasks [1-8]. Each LMS aims to deliver learning to students based on the “anywhere, anytime“ concept [9],[10], while various features of specific LMS help with content management and facilitate increased teacher-teacher, teacher-learner, and learner-learner collaboration and communication. However, some studies have criti- cized LMS concerning transforming learning practices [1-4],[10-12] design structure, pedagogy [3], [13-15], usability and reliability [16],[17], and interactivity and design features in terms of supporting all system users [18-20]. Others have emphasized that LMS can limit opportunities for social and informal learning [3],[21],[22], restricting its potential to enhance teaching and learning [16]. This has been used to explain why students‘ participation in LMS activities has been one of the most significant challenges in rolling out these online environments [3], [20]. It has also been assumed that high levels of obligation can improve the level of students use of LMS [23]; however, stu- dents will also be more engaged more if they can organize their preferences and needs, contribute to content, and fully participate in learning activities [24-26]. In terms of student communication and interaction using LMS, some research also indicates that, in many educational institutions, the features related to administration within the LMS are used much more frequently than the interactive features and tools [13],[16],[20],[27]. Such use may be related to the design basis of many LMS, where the learner’s role is limited to receiving rather than creating or sharing information [3],[28]. Researchers have thus noted several directions for development that could contribute to greater use of LMS for improving teaching and learning, including pedagogical [3], [5],[29] and technological [3], [14],[15],[23],[30], improvements, and the incorporation of better assessment tools [4],[31]. However, to facilitate this, further pedagogical and technological development in educational institutions is required for ICT applications overall [32] and LMS in particular [3]. It can therefore be concluded that, while LMS is highly successful in supporting the administration of learning, it currently falls short in supporting learning itself [1],[3],[33]. This study thus aims to determine the relevant success and failure factors for LMS from the literature and examine this issue from a practical perspective. Generally, LMS is now used by numerous educational institutions worldwide, as- suming that such systems will transport the institutions involved to the next stage of e- learning. However, the research suggests that LMS does not generally meet LMS adop- tion expectations concerning motivating student learning and student engagement. This issue has been discussed since the first adoptions of these systems, though recent re- search has also reported the same concerns. The aim of this study is to help educational institutions develop their planning and evaluation of LMS better, highlight how LMS could be used to innovate teaching and improve students learning. An understanding of the success and failure aspects of LMS should help facilitate a critical evaluation of current LMS practices, supporting further innovative development and more effective implementation. 134 http://www.i-jim.org Paper—Success and Failure Aspects of LMS in E-Learning Systems 2 Methodology This paper aims to explore the success and failure factors of LMS based on the liter- ature and knowledge from experts in the field. A range of LMS-based literature was thus searched, and he extracted success and failure factors summarised. LMS contains many tools and features in common, despite being developed by different vendors, and such features remain the same despite the use of different proprietary names [34]. To investigate the success and failure factors of LMS, the present study examined such systems as broadly as possible, without taking into account vendor or LMS type (whether open-source or commercial system). Thus, two methods were applied to high- light LMS‘s success and failure, a review and an exploratory survey. Using the first method, related research articles were retrieved for all dates since the most popular LMS, Moodle, was launched, 2002 [35]. The literature searched included both peer- reviewed and related reports, such as EDUCAUSE reports. In line with the scope of this paper, a range of related keywords and phrases such as “success aspects LMS,“ “success and failure aspects of LMS,“ “success and unsuccess LMS,“ and “Why LMS fails“ were applied. Thematic analysis was then applied to identify key emerging themes [36]; as is typical with thematic analysis, data were thus summarised and orga- nized, rather than analyzed more deeply [37], with data from the selected papers being broadly divided into “success“ and “failure“ themes based on aspects such as content, communication, pedagogical, assessment, and learning outcomes. These themes were then reviewed by experts in LMS and e-learning, who were selected based on purposive sampling, which involves specifying the characteristics of the population of interest and selecting experienced individuals or groups knowledgeable about the research topic [38], [39]. This initially required identifying critical criteria and identifying those po- tential respondents who might meet those criteria [40]. A survey with both closed and open-ended questions was then distributed to the selected group to achieve this re- search‘s purpose. The selected experts received the questionnaire via email and were thus able to rate the classified themes easily. At the same time, the use of open questions allowed additional information to be gathered about the “success“ and “failure“ aspects of LMS by permitting the respondents to provide more detailed information [41]. 3 Review The review was organized in three main parts: LMS features and tools common to commercial and non-commercial LMS; success aspects of LMS; and failure aspects of LMS. All of these were identified based on related discussions in the literature. 3.1 LMS features and tools LMS features and tools vary slightly from one system to another across both open source and commercial implementations; however, many core features are standard across all systems [3]. An LMS requires students, teachers, and admin tools, which can thus be classified into three main groups: Content management, learning management, iJIM ‒ Vol. 15, No. 11, 2021 135 Paper—Success and Failure Aspects of LMS in E-Learning Systems and assessment management [42]. As shown in [34], LMS features can be categorized into the following aspects: Presenting Content Knowledge, Presenting Course Manage- ment Information, Searching for Information, and Facilitating Discussion. All LMS also supports various features and tools across systems, many of which similar to a greater or lesser extent across both open source and commercial implementations. As seen in [3], another classification attempt identifies standard features available in LMS, including event management, document management, forum management, learning track management, and assessment management. Alongside these standard features, other features and functions are subject to regular upgrades to ease teacher administra- tion and improve learner learning environments [43]. 3.2 LMS success aspects Categorizing the success aspects of LMS suggests that success factors are related to single sign-on, learning management, content management, integration, and security. Each of these is discussed in more detail below. Single sign-on: The main advantage of any LMS is that multiple learning tools are integrated into a single system [27], [28]. With a single sign-on, users can thus gain access to a variety of LMS tools, including course content, discussion boards, forums, chat rooms, quizzes, and grade books, thus integrating multiple features of academic life into a single system to provide a convenient environment to support various aca- demic purposes. This creates a “virtual learning environment“ wherein learners can schedule their learning and access related activities more efficiently [43]. Overall, sin- gle login LMS improves user experiences, allows easy management of various tools, and increases security. Learning management: One of the most successful aspects of LMS seen in their use as administrative tools [1], [3], [33], [44],. Instructors can store and manage course content, syllabuses, and presentations alongside making announcements, creating groups, assigning roles, tracking students’ activities, and organizing students‘ results. Thus, classes‘ administration became easier for instructors, mainly where the course content required for instructors to handle a course or a class are continuously updated with the LMS in terms of the course outline; e-content, such as notes and activities; questions banks; and assignments. Any new instructor assigned to teach the course that is given appropriate access privileges can also manage such content, assign learners, manage quizzes and exams, and track students‘ progress on assigned activities and on- going assignments, decreasing the risk of disruption. Content management: LMS is often maintained mainly as electronic document re- positories [8]. Instructors use LMS to upload course resources and materials, such as syllabuses, assignments, and readings, allowing these to be shared easily with students [45]. Students can thus access course materials regardless of where they are or the time of day. Although such resources are generally managed based on a centralized structure [10], the students can access and use various LMS features to access resources and download materials available online in the forms that suit them [4],[46]. Similarly, the instructor can easily manage teaching materials as required by the course and curricu- lum, updating, removing, and referencing additional resources and materials [45]. 136 http://www.i-jim.org Paper—Success and Failure Aspects of LMS in E-Learning Systems Integration: LMS can be integrated with other information systems in the academic environment [10], such as registration and library systems, to facilitate data and infor- mation transformation among relevant managerial and academic units, enhancing over- all performance. This also helps students more easily monitor their schooling require- ments. Security: Users, roles, and permissions represent the most basic security elements in LMS. LMS adoption is managed at an institutional level, and this can only be ac- cessed with authorization, thus making it secure at this level. LMS comes with sets of user roles and permissions to facilitate administrators managing users, allowing them to access the relevant information or perform specific tasks at different access levels. The authorized users are the enrolled students on the university’s courses, and the course instructors and course admin can thus manage the students‘ interactions and ac- cess to the course materials or discussions. As reported in [47], most respondents have strong positive perceptions of LMS security; however, as [48] points out, secured and manageable systems are prerequisites for LMS, as these are large systems with exten- sive stores of materials and resources. 3.3 LMS failure aspects Despite the wide popularity and adoption of LMS, several recent studies and reports have shown that while LMS is highly successful in enabling the administration of learn- ing, they are less helpful in enabling learning itself [1],[49],[50]. LMS tools are man- aged based on a teacher-centred structure which limits learning interactivity and en- gagement. A summary of the failure aspects commonly reported in the literature iden- tifies several key issues: Content creation and sharing, communicative feature design, teacher-centred structure, learner disengagement, and inflexible assessment features. Content creation and sharing: Features for content creation and sharing in LMSs are relatively inflexible, as teachers‘ and learners‘ roles are limited to uploading and downloading course materials, respectively. Although different types of information can be added to the course directly or linked with other external resources on the web through hyperlinks [34], the dominant rule of any LMS is as an electronic document repository [3],[8],[51]. While students using appropriate login details can access and download these materials throughout their courses, LMS tools should be more dynamic to allow faculty members to create interactive content aligned with learning outcomes at different levels. Students should also be involved in developing and giving feedback on course content and activities, with tools modelled on wikis and blogs available to enrich the content and motivate them to search for and contribute more knowledge un- der instructors‘ supervision. Communicative feature design: Many studies have confirmed that the collabora- tive and interactive features in LMS are poorly utilized and rarely used [3],[34],[50],[52]. Many studies have discussed this issue in terms of investigating why LMS is not effectively used for this purpose [26]. The main reason is often given as the teacher-centred LMS structure, which limits learners‘ roles and reduces the level of interactivity due to its centralized basis [3]. LMS should be designed to advance com- munication tools among students, offering chat or other group tools based on text, iJIM ‒ Vol. 15, No. 11, 2021 137 Paper—Success and Failure Aspects of LMS in E-Learning Systems audio, and video. This could help promote LMS acceptance among learners and in- crease engagement. It could also shift the use of LMS to a more student-centred model. Teacher-centred structure: Traditional teaching and learning processes depend on the one-way delivery of information. As an educational technology, LMS is supposed to help facilitate moving learning toward a more learner-centred approach, innovating pedagogical practices [3]. Morgan [49], however, reported that there is little empirical evidence that LMS improves pedagogy. This was confirmed by [11], which noted that learning and teaching have not changed; faculty members in the LMS environment are simply managing an old process in new ways. This teacher-centred approach results in the development of passive learning, where the learner‘s role is to receive rather than to interact and be actively involved [3]. Learner disengagement: Student engagement is the measurement of learning qual- ity at most education institutions [53]. It refers to “the amount of physical and psycho- logical energy that the student devotes to the academic experience“ [54]. This is im- portant, as the level of students‘ engagement in learning activities significantly impacts their rate of acquiring knowledge. LMS, as an e-learning technology, has the potential to improve student engagement. Nevertheless, researchers have questioned how LMS currently promote student motivation for learning [53], as an engaging student in LMS activities remains a significant challenge associated with these systems [23], [54], alongside the limited use of LMS by instructors [30],[44],[18]. Assessment: Assessment is an essential part of learning and significantly impacts student outcomes [42], [48]. However, most LMS tools for managing online assessment have been assessed by users as inflexible and difficult to use [23]. Using LMS for as- sessment is a process that is still in the early stages, and the current usage of assessment tools focuses on converting paper-based processes into computerized ones [10]. Align- ing assessment tools with learning outcomes and the different levels of cognitive do- main is another issue leading to LMS failure [42]. 4 Expert Evaluation of LMS Success and Failure Aspects The proposed aspects of success and failure outlined above were presented to several experts in the field. A group consisting of five LMS admin staff, 14 lecturers with ex- perience in using LMS to support teaching and learning processes, and two instructional designers was developed to rate all aspects and provide feedback. Respondent charac- teristics are presented in Table 1. Table 1. Research Sample Expert Specialisation No. of experts % Lecturer 14 66.7% LMS Admin 5 23.8% Instructional designers 2 9.5% All 21 100% 138 http://www.i-jim.org Paper—Success and Failure Aspects of LMS in E-Learning Systems 5 Aspect Rating Results The constructed aspects derived from the literature were presented to the LMS ex- perts detailed in section 4 for two purposes. The first was to get expert feedback on the suggested success and failure aspects of LMS, while the second was to rate these as- pects based on the experts‘ broad agreement with the proposed aspects’ effects on LMS success. Summaries of the experts’ feedback on the proposed aspects are presented in Tables 2 and 3. Table 2. LMS Success Aspects Rating Theme Rating Agree Somewhat agree Disagree 1 Single Sign-On 90.5% 9.5% - 2 Learning management 71.4% 23.8% 4.8% 3 Content management 85.7% 14.3% - 4 Integration 66.7% 23.8% 9.5% 5 Security 95.2% 4.8% - Table 3. LMS Failure Aspects Rating Theme Rating Agree Somewhat agree Disagree 1 Content creation and sharing 76.2% 19.0% 4.8% 2 Communicative feature design 85.7% 9.5% 4.8% 3 Teacher-centred structure 81.0% 14.3% 4.8% 4 Learner disengagement 71.4% 19.0% 9.5% 5 Inflexible assessment 85.7% 9.5% 4.8% As shown in Table 2, according to the experts’ feedback, “Security“ is the most ap- parent advantage of LMS, with 95.2% of the group selecting Agree and 4.8% selecting Somewhat agree for this aspect. Using LMS in a learning environment also provides several features that facilitate user management, which is another generally agreed upon successful LMS aspect. Usually, LMS provided by higher education institutions are also supported by their IT Services, which further supports their increased security and manageability aspects. The second most highly rated success aspect was “Single-Sign- On,“ with a good majority of the experts agreeing (90.5%) or somewhat agreeing (9.5%) that “single sign-on“ for LMS users facilitates access to a variety of tools to support teaching and learning. Around 86% of the experts also believed that an essential function of LMS was to store course contents, while 14.3% somewhat agreed to this. Concerning the “Learning management“ aspect, just under 72% agreed that LMS was successfully used for administrative purposes. The least highly rated success aspect of LMS was “Integration,“ with only 66.7% of the expert group agreeing with this as a factor for success, and 23.8% somewhat agreeing that this may be considered a success factor; almost 10% thus do not see this option as a successful aspect of LMS. Among the failure aspects, shown in Table 3, “Communicative feature design“ and “Inflexible assessment“ were the most noticeable failure aspects of LMS, with 85.7% iJIM ‒ Vol. 15, No. 11, 2021 139 Paper—Success and Failure Aspects of LMS in E-Learning Systems experts in agreement and 9.5% somewhat agreeing in each case. This indicates that 4.8% of the experts do not agree that the current communicative or assessment features of LMS should be considered as failure aspects. The second most highly rated failure aspect is “Teacher-centred structure,“ which refers to approaches taken when implementing LMS due to overreliance on instructors and course owners managing course content and coordinating course activities: 81% of experts agreed that this was a failure factor, while 14.3 are somewhat agreed. This was followed by “Content creation and sharing,“ with 76.2% agreeing to this being a failure aspect. The least agreed upon (71.4%) failure aspect was “Learner disengagement.“ 5.1 Emergent aspects In addition to the proposed aspects drawn from the literature review, several new aspects were synthesized from the experts‘ feedback. A summary of these emergent aspects is presented in Table 4; as shown, two success and three failure aspects emerged in this way. Some experts suggested that LMS provides the ability to track students’ access to learning materials within the system, offering reports on the number of users attempting to access and utilize specific content. Another advantage mentioned in this way was that LMS allows the management of class activities to create groups, assign tasks to class groups, and assign roles to group members such as “moderators,“ “ad- min,“ etc. Some LMS users are confused by cluttered GUIs and long screen flows, which was thus one of the reported failure aspects noted by the experts. Two other failure aspects that emerged were a lack of social learning features and limited mobile-based learning options. Table 4. New LMS aspects synthesized from experts‘ feedback. Theme Expert Reference S u c c e ss 1 Tracking Student Learning E3, E7, E16 2 Group Management E4, E12 3 F a il u re 4 Confusing User Interface E2, E9, E12, E16, E17, E20 5 Limited Personalised and Social Learning E1, E17, E18, E20 6 Mobile features E1, E3, E17, E18, E20 6 Discussion The entire thematic analysis results produced a grouping of seven success aspects of LMSs that featured single sign-on, learning management, content management, inte- gration, security, student learning tracking, and group management. The first five fac- tors were derived from the literature and confirmed by the experts, while the last two were derived from the experts’ feedback. The failure aspects were similarly organized into eight themes, five based on the literature, and three based on the experts’ feedback: 140 http://www.i-jim.org Paper—Success and Failure Aspects of LMS in E-Learning Systems content creation and sharing, communicative features, structure, learning disengage- ment, assessment, social and informal learning, and mobile features. Verification of LMS success and failure aspects by experts in the field thus revealed many essential findings. The most commonly acknowledged success aspects of LMS are content management and single sign-on. The experts also acknowledged the func- tion of LMS as electronic repositories for all course-related materials, producing results generally in line with the other studies that have reported LMS being extensively used for administrative purposes. Agreement with the use of LMS integrating with other in- formation systems such as library management systems was the lowest, potentially be- cause few common LMS have this feature by default. The failure aspects‘ ratings confirm three significant issues with LMS, highlighted in earlier studies. LMS does not promote two-way communication, as can be inferred from the strong disapproval of the communication features of LMS. Experts are also not satisfied with the assessment features and tools of LMS: most assessment modules in LMS include multiple-choice questions and similar restricted answer-type questions. Thus, aligning assessment with learning outcomes and cognitive thinking domains re- mains a significant challenge in these systems [42]. Technology-based pedagogy should seek to motivate learning experiences in ways that take account of individual differ- ences in learning. However, LMS generally facilitates a teacher-centred learning ap- proach. Content creation and sharing in LMS is thus still in a basic form, with limited LMS tools. Moreover, comparing LMS to social media and Web 2.0 technologies, LMS limits social and informal learning [21],[22], indicating that improvement is required from both technological and pedagogical perspectives. Learner disengagement is an- other highly rated failure factor, mainly related to the limited features available in LMS to support content sharing and student-student interaction [31], [55]. The success factors suggested by the experts, which were tracking learning and group management, highlight the fact that teachers and students prefer LMS that offer better insight into learning activities and promote student interest through enabling ac- tive learning in online groups. This helps them assess learning progress as well as plan out future course materials. As rated by the experts, the literature review-based failure aspects have a relationship with the failure aspects later suggested by the same experts. Confusing user interfaces and the need to click many times to perform a task reflect inadequate design concepts and lead to low user interest from both teacher and student perspectives. Student en- gagement can also be improved by integrating updated text, audio, and video features into LMS, thus improving both student–student communication and student-instructor communication. Developing a mobile version of LMS would help promote flexible learning, allowing proper “any place, any time“ access [56]. This is in line with recent studies showing that using LMS from mobile devices among university students is limited due to usa- bility issues [17]. As mobile-based applications for various purposes, including educa- tion, continue to increase rapidly [56],[57], further development of LMS to facilitate this is required. iJIM ‒ Vol. 15, No. 11, 2021 141 Paper—Success and Failure Aspects of LMS in E-Learning Systems 7 Implications and Future Research The results of this work help define the success and failure aspects of LMS imple- mentations in education institutions. Many education institutions have used LMS and have made great efforts and significant investments in commercial LMS with high ex- pectations of moving to a new e-learning environment. However, when education in- stitutions attempt implementation of such systems, the benefits of LMS are often not as expected despite the systems still offering considerable advantages. This study‘s findings should help education institutions with the planning and eval- uation of LMS based on the recognized advantages of these systems by defining the aspects that such systems are most likely to improve, particularly in e-learning. Looking at such systems as tools within a set of other essential requirements and comprehen- sively considering integrating additional tools to improve education and learning out- comes is essential. Thus, this study has implications for educational institutions in nar- rowing the gap between reality and expectations when institutions investigate investing in such systems as part of their continuing efforts to adopt effective e-learning practices. This study has several important practical implications for instructors supporting student engagement and involvement in course-related activities in LMS. Instructors must realize that pedagogy and technology are both essential to support student learn- ing. Lecturers must play a direct role in designing learning activities that encourage and provide peer communication and participation opportunities and provide creative and relevant academic content in the LMS, directly affecting overall student learning. For LMS vendors and application developers, this paper offers several valuable in- sights for innovating LMS by developing compatible mobile features, as well as by developing additional features to support peer communication such as audio and video chat and content creation and sharing, with messaging and notification settings drawn from the proven design concepts of various social networking tools. Perhaps more crit- ically, they may be advised to develop additional assessment features and applications to allow integration of assessments aligned with specific learning outcomes and cogni- tive levels of thinking to motivate and improve student learning, rather than simply facilitating management. Based on this study's discussions and findings, it is recommended that further re- search into the technological and pedagogical aspects of LMS is undertaken. However, the paper results help refine future studies on LMS to focus on trending aspects, includ- ing student engagement in LMS to improve student learning and learning outcomes and promote social and lifelong learning in LMS. A focus on incorporating AI tools in LMS systems to improve students’ experiences and learning is also advised. Integration of e- learning systems with new emerging technologies in education has to be considered [58], especially after the pandemic of Covid-19, which has enforced education intui- tions to utilize a variety of emerging online communication technologies [59]. 142 http://www.i-jim.org Paper—Success and Failure Aspects of LMS in E-Learning Systems 8 Conclusion LMS has developed wide popularity and use amongst higher education institutions. After decades of LMS use and an ongoing discussion about LMS implementation in support of the initiatives of e-learning, this paper presented an overview of the success and failure aspects of LMS based on the analysis of a literature review, as verified and supplemented by professionals in the field of e-learning. The success aspects emerged as single sign-on, learning management, content man- agement, integration, security, student learning tracking, and group management. In contrast, the failure aspects were classified as content creation and sharing, communi- cative features, structure, learning disengagement, assessment, social and informal learning, and mobile features. Thus, the most successful aspects were related to admin- istrative and management-related features and tools, while the failure aspects were re- lated to technological and pedagogical issues. The results of this paper should contribute to the development of a better understand- ing of LMS based on an equal acknowledgment of both the key advantages and the failure aspects of such systems by education institutions, instructors, LMS software providers, and the relevant developer community, which should position them to ad- dress the limitations better and to develop more practical implementations. Besides, highlighting the failure aspects of LMS presents valuable pedagogical and practical in- sights into targets for improvement for researchers and practitioners interested in e- learning applications in education institutions. The integration of advanced communication and content sharing features into LMS is urgently required to develop better acceptance among learners and increase their en- gagement. This could also shift LMS from being a teacher-centred environment by en- abling students to create and apply knowledge rather than simply receiving it. As mo- bile systems' adoption is now dominant among the population intended to use LMS, to truly promote “anytime, anywhere“ learning, and to motivate lifelong learning, it must also be a top priority to improve LMS systems’ support mobile-friendly interfaces. Considering the success and failure aspects of LMS throughout its now extended adoption period, better awareness of the related principles, as discussed in this paper, should contribute to both practical and theoretical improvement, better implementation, and continuous development of LMS. 9 References [1] N. M. and J. D. Malcolm Brown, “EDUCAUSE Review, vol. 50, no. 4,” Educ. Rev., vol. 50, no. 4to indeed “anytime, anywhere” learning trulyuse.edu/articles/2015/6/whats-next- for-the-lms. [2] S. D. Smith and J. Borreson Caruso, “The ECAR study of undergraduate students and infor- mation technology,” Educ. - Cent. Appl. Res., no. October, pp. 1-13, 2010. [3] A. K. Alhazmi and A. A. Rahman, “Why LMS failed to support student learning in higher education institutions,” 2012, https://doi.org/10.1109/IS3e.2012.6414943 [4] D. A. Back, F. Behringer, N. Haberstroh, J. P. Ehlers, K. Sostmann, and H. Peters, “Learning management system and e-learning tools: An experience of medical students’ usage and iJIM ‒ Vol. 15, No. 11, 2021 143 https://doi.org/10.1109/IS3e.2012.6414943 Paper—Success and Failure Aspects of LMS in E-Learning Systems expectations,” Int. J. Med. Educ., vol. 7, pp. 267-273, 2016. https://doi.org/10.511 6/ijme.57a5.f0f5 [5] M. Ouadoud, A. Nejjari, M. Y. Chkouri, and K. E. El Kadiri, “Educational modeling of a learning management system,” Proc. 2017 Int. Conf. Electr. Inf. Technol. ICEIT 2017, vol. 2018-Janua, pp. 1-6, 2018, https://doi.org/10.1109/EITech.2017.8255247 [6] J. Chipps, J. Kerr, P. Brysiewicz, and F. Walters, “A Survey of University Student�_ Per- ceptions of Learning Management Systems in a Low-Resource Setting Using a Technology Acceptance Model,” CIN Comput. Informatics, Nurs., vol. 33, p. 7Q_77, 2015. https://doi.org/10.1097/CIN.0000000000000123 [7] G. Ssekakubo, H. Suleman, and G. Marsden, “Designing mobile LMS interfaces: learners’ expectations and experiences,” Interact. Technol. Smart Educ., vol. 10, pp. 147-167, 2013. https://doi.org/10.1108/ITSE-12-2012-0031 [8] S. Papadakis, M. Kalogiannakis, E. Sifaki, and N. Vidakis, “Access Moodle Using Smart Mobile Phones. A Case Study in a Greek University,” 2018, pp. 376-385. https://doi.org/10.1007/978-3-319-76908-0_36 [9] D. K. Mah, “Learning Analytics and Digital Badges: Potential Impact on Student Retention in Higher Education,” Technol. Knowl. Learn., vol. 21, no. 3, pp. 285-305, 2016. https://doi.org/10.1007/s10758-016-9286-8 [10] J. Broadbent, “Academic success is about self-efficacy rather than frequency of use of the learning management system,” Australas. J. Educ. Technol., vol. 32, no. 4, pp. 38-49, 2016, https://doi.org/10.14742/ajet.2634 [11] G. Naveh, D. Tubin, and N. Pliskin, “Student LMS use and satisfaction in academic institu- tions: The organizational perspective,” Internet High. Educ., vol. 13, no. 3, pp. 127-133, 2010, https://doi.org/10.1016/j.iheduc.2010.02.004 [12] N. Adzharuddin, “Learning Management System (LMS) among University Students: Does It Work?” Int. J. e-Education, e-Business, e-Management e-Learning, vol. 3, no. 3, 2013, https://doi.org/10.7763/IJEEEE.2013.V3.233 [13] A. M. Apandi and A. Raman, “Factors Affecting Successful Implementation of Blended Learning at Higher Education International Journal of Instruction, Technology & Social Sci- ences Factors Affecting Successful implementation of Blended Learning at Higher Educa- tion To cite this articl,” Int. J. Instr. Technol. Soc. Sci., vol. 1(1), no. May, pp. 13-23, 2020. [14] N. N. M. Kasim and F. Khalid, “Choosing the right Learning Management System (LMS) for the higher education institution context: A systematic review,” Int. J. Emerg. Technol. Learn., vol. 11, no. 6, pp. 55-61, 2016. https://doi.org/10.3991/ijet.v11i06.5644 [15] A. Yuen, B. Fox, A. Sun, and L. Deng, “Course management systems in higher education: understanding student experiences,” Interact. Techn. Smart Edu., vol. 6, pp. 189-205, Aug. 2009. https://doi.org/10.1108/17415650911005393 [16] X. Hu, L. Lei, J. Li, N. Iseli-Chan, F. Siu, and S. Chu, “Access Moodle Using Mobile Phones: Student Usage and Perceptions,” 2016. https://doi.org/10.1007/978-981-10-0027- 0_10 [17] S. Papadakis, M. Kalogiannakis, E. Sifaki, and N. Vidakis, “Evaluating Moodle use via Smart Mobile Phones. A case study in a Greek University,” EAI Endorsed Trans. Creat. Technol., vol. 5, p. 156382, Jul. 2018. https://doi.org/10.4108/eai.10-4-2018.156382 [18] W. Alkarney and M. Albraithen, “Critical Success Factors for Learning Management Sys- tem Implementation: Non-Scientific Colleges Case A Conceptual Model,” 21st Saudi Com- put. Soc. Natl. Comput. Conf. NCC 2018, pp. 1-6, 2018. https://doi.org/10.1109/NCG.2018. 8592974 144 http://www.i-jim.org https://doi.org/10.5116/ijme.57a5.f0f5 https://doi.org/10.5116/ijme.57a5.f0f5 https://doi.org/10.1109/EITech.2017.8255247 https://doi.org/10.1097/CIN.0000000000000123 https://doi.org/10.1108/ITSE-12-2012-0031 https://doi.org/10.1007/978-3-319-76908-0_36 https://doi.org/10.1007/s10758-016-9286-8 https://doi.org/10.14742/ajet.2634 https://doi.org/10.1016/j.iheduc.2010.02.004 https://doi.org/10.7763/IJEEEE.2013.V3.233 https://doi.org/10.3991/ijet.v11i06.5644 https://doi.org/10.1108/17415650911005393 https://doi.org/10.1007/978-981-10-0027-0_10 https://doi.org/10.1007/978-981-10-0027-0_10 https://doi.org/10.4108/eai.10-4-2018.156382 https://doi.org/10.1109/NCG.2018.8592974 https://doi.org/10.1109/NCG.2018.8592974 Paper—Success and Failure Aspects of LMS in E-Learning Systems [19] M. N. Habib, W. Jamal, U. Khalil, and Z. Khan, “Transforming universities in interactive digital platform: case of city university of science and information technology,” Educ. Inf. Technol., 2020. https://doi.org/10.1007/s10639-020-10237-w [20] D. Kember, C. McNaught, F. C. Y. Chong, P. Lam, and K. F. Cheng, “Understanding the ways in which design features of educational websites impact upon student learning out- comes in blended learning environments,” Comput. Educ., vol. 55, no. 3, pp. 1183-1192, 2010. https://doi.org/10.1016/j.compedu.2010.05.015 [21] C. McLoughlin and M. Lee, “Personalised and self-regulated learning in the Web 2.0 era: International exemplars of innovative pedagogy using social software,” Australas. J. Educ. Technol., vol. 26, pp. 28-43, Mar. 2010. https://doi.org/10.14742/ajet.1100 [22] B. Chen and T. Bryer, “Investigating Instructional Strategies for Using Social Media in For- mal and Informal Learning,” Int. Rev. Res. Open Distance Learn., vol. 13, pp. 87-104, Jan. 2012. https://doi.org/10.19173/irrodl.v13i1.1027 [23] K. Maxwell and A. A. Angehrn, “A study of successful and unsuccessful online learning environment experiences,” Commun. Comput. Inf. Sci., vol. 111 CCIS, no. PART 1, pp. 375-382, 2010. https://doi.org/10.1007/978-3-642-16318-0_44 [24] A. Jafari, P. Mcgee, and C. Carmean, “Managing Courses Defining Learning: What Faculty, Students, and Administrators Want,” Educ. Rev., Jan. 2006. [25] C. Vrasidas, “Issues of pedagogy and design in e-learning systems,” 2004. https://doi.org/ 10.1145/967900.968086 [26] D. Zhang, J. L. Zhao, L. Zhou, and J. F. Nunamaker, “Can e-learning replace classroom learning?,” Commun. ACM, vol. 47, no. 5, pp. 75-79, 2004. https://doi.org/10.1145/9862 13.986216 [27] N. Fathema, D. Shannon, and M. Ross, “Expanding the Technology Acceptance Model (TAM) to Examine Faculty Use of Learning Management Systems (LMSs) In Higher Edu- cation Institutions.,” MERLOT J. Online Learn. Teach., vol. 11, no. 2, pp. 210-232, 2015. [28] K. Georgouli, I. Skalkidis, and P. Guerreiro, “A framework for adopting LMS to introduce e-learning in a traditional course,” Educ. Technol. Soc., vol. 11, no. 2, pp. 227-240, 2008. [29] S. Balula, J. A. Diniz, S. B. Dias, and J. A. Diniz, “International Forum of Educational Tech- nology & Society Towards an Enhanced Learning Management System for Blended Learn- ing in Higher Education Incorporating Distinct Learners’ Profiles Towards an Enhanced Learning Management System for Blended Learning ,” vol. 17, no. 1, 2014. [30] A. Alenezi, “Barriers to Participation in Learning Management Systems in Saudi Arabian Universities,” Educ. Res. Int., vol. 2018, 2018. https://doi.org/10.1155/2018/9085914 [31] A. K. Alhazmi, A. A. Rahman, and H. Zafar, “Conceptual model for the academic use of Social Networking Sites from student engagement perspective,” in IC3e 2014 - 2014 IEEE Conference on e-Learning, e-Management and e-Services, Apr. 2015, pp. 1-6. https://doi.org/10.1109/IC3e.2014.7081232 [32] M. Kalogiannakis and S. Papadakis, The Dual Form of Further Education of Educators in ict: Technological and Pedagogical Training. 2007. [33] M. Brown, “The NGDLE: We Are the Architects,” Educ. Rev., vol. 52, no. 4, 2017, [Online]. Available: https://er.educause.edu/articles/2017/7/the-ngdle-we-are-the-architects . [34] S. R. Malikowski, M. E. Thompson, and J. G. Theis, “A model for research into course management systems: Bridging technology and learning theory,” J. Educ. Comput. Res., vol. 36, no. 2, pp. 149-173, 2007. https://doi.org/10.2190/1002-1T50-27G2-H3V7 [35] A. Büchner, Moodle 3 Administration. Packt Publishing, 2016. [36] V. Braun and V. Clarke, “Using thematic analysis in psychology,” Qual. Res. Psychol., vol. 3, no. 2, pp. 77-101, Jan. 2006. https://doi.org/10.1191/1478088706qp063oa iJIM ‒ Vol. 15, No. 11, 2021 145 https://doi.org/10.1007/s10639-020-10237-w https://doi.org/10.1016/j.compedu.2010.05.015 https://doi.org/10.14742/ajet.1100 https://doi.org/10.19173/irrodl.v13i1.1027 https://doi.org/10.1007/978-3-642-16318-0_44 https://doi.org/10.1145/967900.968086 https://doi.org/10.1145/967900.968086 https://doi.org/10.1145/986213.986216 https://doi.org/10.1145/986213.986216 https://doi.org/10.1155/2018/9085914 https://doi.org/10.1109/IC3e.2014.7081232 https://er.educause.edu/articles/2017/7/the-ngdle-we-are-the-architects https://doi.org/10.2190/1002-1T50-27G2-H3V7 https://doi.org/10.1191/1478088706qp063oa Paper—Success and Failure Aspects of LMS in E-Learning Systems [37] M. Maguire and B. Delahunt, “Doing a thematic analysis: A practical, step-by-step guide for learning and teaching scholars.,” 2017. [38] L. Christensen, R. Johnson, and L. Turner, “Research Methods Design and Analysis,” 2010. [39] V. L. Creswell, J.W. and Plano Clark, Designing and Conducting Mixed Methods Research., 2nd ed. Los Angeles: SAGE Publications, 2011. [40] W. M. Trochim and J. Donnelly, “Research methods knowledge base,” 2001. [41] M. Allen, “The SAGE Encyclopaedia of Communication Research Methods.” Thousand Oaks, California, 2017. https://doi.org/10.4135/9781483381411 [42] A. K. Alhazmi, H. Zafar, and F. Al-Hammadi, “Framework for integrating outcome-based assessment in online assessment: Research in progress,” in Proceedings of the 2015 Science and Information Conference, SAI 2015, Sep. 2015, pp. 217-221. https://doi.org/10.1109/ SAI.2015.7237147 [43] H. Coates, R. James, and G. Baldwin, “A critical examination of the effects of learning man- agement systems on university teaching and learning,” Tert. Educ. Manag., vol. 11, no. 1, pp. 19-36, 2005. https://doi.org/10.1080/13583883.2005.9967137 [44] M. J. Asiri, R. B. Mahmud, K. Abu Bakar, and A. F. Bin Mohd Ayub, “Factors Influencing the Use of Learning Management System in Saudi Arabian Higher Education: A Theoretical Framework,” High. Educ. Stud., vol. 2, no. 2, pp. 125-137, 2012. https://doi.org/10.5539/ hes.v2n2p125 [45] D. Williams and A. Whiting, “Exploring the Relationship Between Student Engagement, Twitter, and a Learning Management System: A Study of Undergraduate Marketing Stu- dents,” Int. J. Teach. Learn. High. Educ., vol. 2016, pp. 302-313, Jan. 2017. [46] Y. J. Joo, N. Kim, and N. H. Kim, “Factors predicting online university students’ use of a mobile learning management system,” Educ. Technol. Res. Dev., vol. 64, no. 4, pp. 611- 630, 2016 https://doi.org/10.1007/s11423-016-9436-7 [47] M. F. Hilmi, S. Pawanchik, and Y. Mustapha, “Exploring security perception of learning management system (LMS) portal,” in 2011 3rd International Congress on Engineering Ed- ucation (ICEED), 2011, pp. 132-136. https://doi.org/10.1109/ICEED.2011.6235375 [48] N. Rjaibi, L. Rabai, A. Aissa, and M. Louadi, “Cyber Security Measurement in Depth for E- learning Systems,” Int. J. Adv. Res. Comput. Sci. Softw. Eng., vol. 2, no. 11, pp. 1-15, 2012, [Online]. Available: http://www.researchgate.net/publication/235770827_Cyber_Secu- rity_Measurement_in_Depth_for_E-learning_Systems/file/32bfe51366472ad9e0.pdf . [49] G. Morgan, “Faculty Use of Course Management Systems Factors Shaping CMS Use Initial Adoption of CMS,” Educ. Cent. Appl. Res., no. May, pp. 1-6, 2003. [50] C. Phelps and Y. Michea, “Learning Management Systems ‘Evaluation Focuses on Tech- nology Not Learning School of Health Information Sciences, University of Texas Health Science Center at Houston,” p. 2003, 2003. [51] S. Lonn and S. D. Teasley, “Saving time or innovating practice: Investigating perceptions and uses of Learning Management Systems,” Comput. Educ., vol. 53, no. 3, pp. 686-694, 2009. https://doi.org/10.1016/j.compedu.2009.04.008 [52] P. Danaher, J. Luck, and J. McConachie, “The stories that documents tell: changing tech- nology options from Blackboard, Webfuse and the content management system at Central Queensland University,” Stud. Learn. Eval. Innov. Dev., vol. 2, no. 1, pp. 34-43, 2005. [53] C. Beer, K. Clark, and D. Jones, “Indicators of engagement,” ASCILITE 2010 - Australas. Soc. Comput. Learn. Tert. Educ., pp. 75-86, Jan. 2010. [54] N. Sthapornnanon, R. Sakulbumrungsil, A. Theeraroungchaisri, and S. Watcharadam- rongkun, “Social constructivist learning environment in an online professional practice course,” Am. J. Pharm. Educ., vol. 73, no. 1, pp. 1-8, 2009. https://doi.org/10.5688/aj730110 146 http://www.i-jim.org https://doi.org/10.4135/9781483381411 https://doi.org/10.1109/SAI.2015.7237147 https://doi.org/10.1109/SAI.2015.7237147 https://doi.org/10.1080/13583883.2005.9967137 https://doi.org/10.5539/hes.v2n2p125 https://doi.org/10.5539/hes.v2n2p125 https://doi.org/10.1007/s11423-016-9436-7 https://doi.org/10.1109/ICEED.2011.6235375 http://www.researchgate.net/publication/235770827_Cyber_Security_Measurement_in_Depth_for_E-learning_Systems/file/32bfe51366472ad9e0.pdf http://www.researchgate.net/publication/235770827_Cyber_Security_Measurement_in_Depth_for_E-learning_Systems/file/32bfe51366472ad9e0.pdf https://doi.org/10.1016/j.compedu.2009.04.008 https://doi.org/10.5688/aj730110 Paper—Success and Failure Aspects of LMS in E-Learning Systems [55] A. K. Alhazmi and A. A. Rahman, “A framework for student engagement in social network- ing sites,” 2014. [56] H. Seyed Ebrahim, K. Ezzadeen, and A. K. Alhazmi, “Acquiring knowledge through mobile applications,” Int. J. Interact. Mob. Technol., vol. 9, no. 3, pp. 71-74, 2015. https://doi.org/ 10.3991/ijim.v9i3.4495 [57] A. Jalil, M. Beer, and P. Crowther, “Mobile learning in a seminar or workshop: A case study for evaluating MOBIlearn2 basic components and their application,” Bull. Tech. Comm. Learn. Technol., vol. 17, pp. 30-33, Dec. 2015. [58] Papadakis, St. (2020). Robots and Robotics Kits for Early Childhood and First School Age. International Journal of Interactive Mobile Technologies (iJIM), 14 (18), 34-56. https://doi.org/10.3991/ijim.v14i18.16631 [59] N. H. Al-Kumaim, F. Mohammed, N. A. Gazem, Y. Fazea, A. K. Alhazmi, and O. Dakkak, “Exploring the Impact of Transformation to Fully Online Learning During COVID-19 on Malaysian University Students’ Academic Life and Performance,” Int. J. Interact. Mob. Technol. (iJIM); Vol 15, No 05, Mar. 2021, [Online]. Available: https://www.online-jour- nals.org/index.php/i-jim/article/view/20203/8843 https://doi.org/10.3991/ijim.v15i05.2020 3 10 Authors Dr. Abdulsalam K. Alhazmi, is a Consultant and Researcher in Technology Appli- cations in e-learning and e-commerce systems. He is the Director of International Pro- grams and Cooperation at the University of Science and Technology, managing the International Joint and Dual Degree Programs with International Universities. He also works as a Consultant for a few Malaysian companies specializing in e-learning and smart systems applications. Dr. Athar Imtiaz, Lecturer at the School of Fundamental Sciences, Massey Univer- sity, Palmerston North, New Zealand. From 2012 onwards he has been involved in re- search. A good portion of his research has been in the education technologies, user experience, applied AI and data analytics. The focus has been e-assessment, robotics for learning programming and using natural user interfaces for learning. Dr. Fatima Al-Hammadi, Department of Instructional Technologies, Interactive Language Centre, Kuala Lumpur, Malaysia. Her research has been in the curriculum development, technology-enhanced learning, and e-assessment. Ezzadeen Kaed, LMS Manager at the Department of Instructional Technologies, Interactive Language Centre, Kuala Lumpur, Malaysia. Article submitted 2021-01-03. Resubmitted 2021-03-28. Final acceptance 2021-03-28. Final version pub- lished as submitted by the authors. iJIM ‒ Vol. 15, No. 11, 2021 147 https://doi.org/10.3991/ijim.v9i3.4495 https://doi.org/10.3991/ijim.v9i3.4495 https://doi.org/10.3991/ijim.v14i18.16631 https://www.online-journals.org/index.php/i-jim/article/view/20203/8843 https://www.online-journals.org/index.php/i-jim/article/view/20203/8843 https://doi.org/10.3991/ijim.v15i05.20203 https://doi.org/10.3991/ijim.v15i05.20203