International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 16, No. 06, 2022 Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… Framework for Gamification of Online Training Platforms for Employee Engagement Enhancement https://doi.org/10.3991/ijim.v16i06.28485 Abdullatif Saleh Alfaqiri(*), Siti Fadzilah Mat Noor, Noraidah Sahari Universiti Kebangsaan Malaysia, Selangor, Malaysia P90491@siswa.ukm.edu.my Abstract—Organizations have moved away from traditional classroom-based training to online training whereby employees can attend courses at their conve- nience either at home or in the workplace. This transition has been fuelled by lockdowns that governments worldwide have implemented to counter the spread of the COVID-19 pandemic, where employees are expected to work or attend training from home. However, several issues including lack of employee engage- ment hinder the effective delivery of online corporate training. In recent years, gamification has been identified as a solution to lack of engagement in educa- tion. Gamification involves the incorporation of game elements into non-game applications to encourage user engagement, where different combinations are used for different learning contexts. However, little research has been done to identify suitable game elements for the gamification of corporate online training. This study proposes a set of game elements to improve employee engagement with online training platforms. A step-by-step analysis of the literature led to the identification of nine potentially suitable game elements. These elements were evaluated by a group of experts selected from five well-known training providers in Saudi Arabia, who completed a semi-structured interview questionnaire about the usability, completeness, and usefulness of the elements. The result of the evaluation indicated that the identified game elements will be able to effectively and efficiently improve employee engagement. The set of nine game elements was then incorporated into a proposed framework for the gamification of online training platforms to enhance employee engagement. Keywords—online training, gamification, game elements, engagement, employee 1 Introduction In organizations generally, the human resource development department is respon- sible for improving employee performance and engagement. Conventionally, this is achieved by conducting on-the-job training which starts at induction and continues throughout an employee’s career through the provision of various trainings to improve skills. The adoption of technology in training platforms has propelled corporate train- ing to another level. In particular, the use of IT has shifted corporate training from iJIM ‒ Vol. 16, No. 06, 2022 159 https://doi.org/10.3991/ijim.v16i06.28485 mailto:P90491@siswa.ukm.edu.my Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… the classroom-based to the online mode [1]. This shift has accelerated due to the try- ing times currently being experienced by many organizations and communities due to the COVID-19 pandemic [2]. Organizations have opted for online training because it removes barriers associated with location and time, thereby allowing employees to attend training at any time from anywhere. However, one of the key issues hampering the success of online training is lack of engagement [3-6]. This issue of disengage- ment is attributed to a lack of employee enthusiasm and interest in online training [7]. Indeed, sustaining the engagement of employees in online training so that they immerse themselves in learning to obtain knowledge and skills has become an overwhelming challenge. There have been ongoing attempts by domain researchers [8, 9] to resolve this issue through the use of mobile learning [10], and multimedia in online training platforms [11]. However, it has been found that such solutions can only marginally mitigate the issue of disengagement in online training [5]. In recent years, gamification has emerged as an important tool for improving engagement in education, and in online learning spe- cifically [12]. Therefore, this study envisages that gamification will also be a solution to the issue of employee disengagement with online corporate training. According to the literature, “Gamification can be defined as the process of using game elements in a non-game context” [13] . However, the question of how can game elements be added to an online training platform to enhance employee engagement is yet to be answered [14, 15]. To the best of our knowledge, there is no single agreed-upon standard or best practice that provides strategies for gamifying online training platforms [16]. Thus, a framework for the gamification of online training platforms is lacking. In addition, there is a need to discover game elements that can have a positive impact on employee engagement during online training sessions [14, 16]. Therefore, the contribution of this study is twofold. First, a list of game elements is derived from an analysis of the existing literature. Subsequently, the list of game elements is evaluated for their suitability for the online corporate training context by a group of online training experts using a semi-structured interview questionnaire. Second, the identified game elements are incorporated into a proposed framework to examine employees’ acceptance towards gamification and the impact of gamification on employee engagement factors, particularly the behavioural, emotional, and cogni- tive engagement of employees. The framework is based on the Technology Acceptance Model (TAM). The rest of this paper is organized as follows: Section 2 provides some background on training and development in organizations and Section 3 focuses on online training in organizations. Next, Section 4 gives an overview of the use of gamification as a human resource development strategy, after which Section 5 presents the main theories and models used in gamification. Then, Section 6 presents the methodology employed to identify suitable game elements from the literature and Section 7 provides the results of the literature analysis and the list of the nine game elements identified. Next, Section 8 describes the expert evaluation of the identified game elements. Then, Section 9 discusses the results of their evaluation. This is followed by Section 10, which presents the proposed gamification framework, which is based on the TAM and incorporates the identified game elements. Section 11 concludes the paper. 160 http://www.i-jim.org Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… 2 Training and development in organizations The role of the human resource management department in an organization can be divided into three or four main tasks, of which training and development is one [17]. Training can be defined as a process of providing professional skills and qualifications to prepare individuals to perform specific job-related tasks [18]. Training can also be referred to as a process of adjusting the skills, knowledge, and attitudes of employees to improve the performance of organizations and working groups, as well as that of the employees themselves in terms of work processes [19]. In addition to its usage as an improvement tool, training can be used to provide new employees with the initial information they need to facilitate their integration into an organization and to bolster their knowledge and skills throughout their tenure. The learning achieved via the process of training can be attributed to interaction, observation, and joint experience with other trainees participating in the training [18]. These factors contribute to making the employee more capable of decision making and becoming more interested in learning from the real situations in their life so that they can solve problems that cannot be solved through the use of knowledge alone. The learning obtained via training also improves the behaviours of employees that are related to personal values and goals [17, 19]. The impact of the provision of training in an organization can be evidenced by several aspects, such as increased productivity, reduced cost to time of work ratio, improved efficacy, positive changes in attitude and behaviour, and increased awareness and knowledge [20]. However, the effectiveness of training in an organization can be hindered by a number of factors, such as not structuring the training programme in line with practice, lack of teaching experience of the trainer, and the quality of the resources used in the training. However, lack of engagement in training is the paramount reason for ineffective training outcomes [3, 4]. 3 Online training in organizations Online training can be described as the technology-based and systematic acquisition of skills, rules, concepts, and/or attitudes that increase the productivity of both the employee and the organization [21]. It can also be described as method that enables the trainer and the trainee to be in different locations yet still connect through the use of technology capabilities (tools, technology, and equipment) [22]. Online training approaches are especially important in organizations where employees work different shifts and where there are high turnover rates, both of which are organizational aspects that are especially evident in developing countries [23, 24]. As noted above, the effectiveness of online training is a key factor in developing employee skills because this mode of training can facilitate the transfer of acquired knowledge to the workplace. There exist several factors that may positively influence employees to gain knowledge when attending online training. Researchers have high- lighted that these factors include employee engagement, the work environment, and the design of the training intervention, among which employee engagement is seen as critical [25, 26]. On the other hand, there is also a risk of conflict in the context of iJIM ‒ Vol. 16, No. 06, 2022 161 Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… online training between individual learning needs and organizational needs. This poten- tial conflict stems from the fact that online training must properly support individual learning while meeting the organization’s skills requirements [27]. Achieving a balance between these two needs can be difficult because it may involve trying to engage an employee in gaining knowledge on issues that are of little use to him or her [28]. Stud- ies show that participatory behaviour affects the centrality and effectiveness of learn- ing, and these studies also emphasize the safety and cognitive presence in their work. Learning resources and energy are key issues in employee engagement [28-30]. In fact, the outcome of the training may be more dependent on the behaviour of the employee as compared to the nature of the work environment [28]. Therefore, when designing an online training course, it is important to design a train- ing style that is suitable for the needs of the individual [14], for example, using cogni- tive skills acquisition theory when designing an online training course for employees who need a specific skill. In cognitive skills acquisition theory it is posited that “learn- ers acquire skills in three stages: declarative knowledge, knowledge synthesis, and pro- cedural skills” [31]. In stage 1, “the instructions are stored in the learners’ memory and represented as a declarative network of facts with definitions and relationships” [31]. In stage 2, “knowledge compilation is the process of applying the newly acquired knowl- edge to the learners’ own problem domain through decision rules” [31]And stage 3 “is where these decision rules are permanently stored in the learners’ memory to become procedural skills” [31]. Online training should include practice examples with imme- diate feedback and it is vital to use concrete examples when, for example, illustrating concepts. Also, the use of modelling in online training, for example where an instructor demonstrates and guides actions before the learners’ try something themselves, leads to better learning outcomes as compared to reading instructions or lecture-based instruc- tions with the same content but without demonstrations. In online training, demon- stration and guidance are often done by combining text, audio, graphics, video, and animation, i.e., through the use of multimedia. Some of the literature on multimedia and learning deals with the human–computer interaction aspects of learning such as cognitive load and modality principles [32]. Gamification, i.e., the use of design elements for video games in non-gaming systems to increase the engagement and motivation of the user, also has the potential to increase the engagement of learners and to train employees to handle complex situations [33]. However, there are potential risks when implementing gamification. For example, the quality of the learning of tasks may suffer if the game aspect distracts the learner from the overall purpose of the task, or the novelty of the gamification could wear off, or the game element might not be perceived as challenging enough. Contextual aspects also play a role in the selection of gamification elements because workplaces have different cultures [34]. For example, the element of competition may affect the learning outcome in one context [35] but may not be preferred by learners in another context [34]. Here, prior experience of playing games and use of computers can influence engagement via gamification, where previous experience of games tends to lead to better engagement while lack of previous experience can create anxiety. This is an important issue because anxiety affects the learning process negatively and thus potentially also the level of employee engagement in online training. 162 http://www.i-jim.org Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… 4 Use of gamification as a human resource development strategy The concept of gamification has been used in human resource management for a number of reasons, including the attraction, training, development, and engagement, and retention of talented employees [36, 37]. Organizations use a variety of tools for the recruitment and selection of best-performing employees. The tools that are convention- ally used include cognitive ability and personality tests that enable employers to select the right employees for key job positions. However, advances in technology have led some organizations to utilize data analytics like reported in [38]. By applying the quan- titative and statistical techniques of data analytics, organizations can detect patterns to facilitate talent management, employee compensation, and employee health [36, 39]. As regards the specific uses of gamification in human resource management, this advanced technology is used in two areas: digital interviews and personnel voice pro- filing. As such, this means that gamification is replacing conventional human resource tools as a method for assessing employee competence [40]. Gamification is used to measure key performance indicators and personality traits for appraisal purposes [41]. That is, gamification is used to enhance conventional human resource management tools and offer solution to address the cumbersome recruitment process [41]. For exam- ple, an application called Pymetrics is used by some organizations to test job candidates by using different game elements to measure cognitive skills and emotional capacity [42]. The application can also be used for employee development, thereby allowing organizations to identify talent within the pool of existing employees. Another area of human resource management that would benefit from the urgent attention of research- ers is the gamification of training and development because it has been argued that with gamified training systems, organizations can achieve a better level of congruency between organizational goals and individual effectiveness [36]. 5 Theories and models used in gamification In the literature on gamification, researchers have highlighted the positive results of the use of gamification for improving engagement [43]. However, because gamifica- tion is still in its infancy, there is often a lack of a theoretical underpinning for some of the gamification proposals [44]. Consequently, recent studies have shown an interest in the use of theory in the gamification of various domains [45]. Likewise, several studies have used design models when gamifying systems. In this light, both theories and design models should be seen as an integral part of the process of gamification. For example, Self-Determination Theory (SDT) looks at the concept of improving motivation through the use of gamification [46]. The theory focuses on the improve- ment of both intrinsic and extrinsic motivation. According to SDT, everything between these two extremes is extrinsic motivation because a person experiencing a motivation would need some external motivation in order to engage in an activity and a person who is intrinsically motivated would not need an external motivator to engage in the activity [47]. iJIM ‒ Vol. 16, No. 06, 2022 163 Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… Flow theory is another theory employed in gamification for the achievement of the optimal level of engagement. In the context of gamification, flow theory focuses on the interplay between the challenge of an activity and the ability of the player when performing that activity. Flow theory is incorporated in gamification design in an effort to address the various skill levels users possess when performing a task or activity [48]. The TAM is one of the most cited theories in gamification and has been adopted by numerous studies. Adoption of new educational technology comes with new problems [49]. So, according to the TAM, the adoption of an IT system is determined by the user’s volition to use the system, which in turn is shaped by the user’s attitude towards the system [50, 51]. In recent research on the acceptance of e-learning in universities [52] and mobile e-banking [53], it was found that using gamification has a big impact on the acceptance of new technologies. Therefore, this theory is used by the current study to investigate employees’ acceptance of the use of gamification for online training. There are some other theories that have been used in gamification that are not described here because they are similar to the above-mentioned theories or rarely used. For example, the Attention, Relevance, Confidence, and Satisfaction (ARCS) motiva- tional design is essentially focused on the same concept addressed by SDT [12], while cognitive load theory is one of the less-used theories [54]. Other gamification-specific theories have been used in some cases, including the dynamic model for gamification of learning [55] and the theory of gamification learning [56], these are specific to student education rather than employee education, hence, are not thoroughly discussed in this paper. As regards the models that can be employed in the field of gamification, the Mechanics-Dynamics-Aesthetics (MDA) game design framework focuses on three phases of the gamification process: mechanics, dynamics, and aesthetics. The frame- work outlines the steps that need to be followed when gamifying a system. The mechan- ics phase focuses on the definition of the game elements, the dynamics phase involves the study of user interaction with the game elements, and the aesthetics phase inves- tigates the user experience and how the game elements improve user interaction [57]. Another design model is the Four-Player Model, which focuses on examining user engagement in activities involving game elements [14]. As the name suggests, the model contains four types of player: killers, who are players that engage by defeating other players in the completion of a task; achievers who engage by trying to finish tasks or quests to accomplish goals within the game; socializers who get engaged when interacting or collaborating with other players; and explorers who become engaged when they are immersed in the game environment and are exploring different aspects of the game [14]. The literature also includes design methods that have been proposed as guides that developers can follow for the gamification of online learning systems. Recent works of particular relevance to the current study include [12], [58], [59], and [60]. In [12], the proposed conceptual model uses gamification to embed motivational and engagement designs into e-learning. The model consists of relevant concepts from prior studies. The logical structure of the model provides a visual display of how game elements are linked to the learning context of e-learning. However, the aim of [12] is different from that of the current study in that the model in [12] was developed to try to improve student motivation and engagement in technical and vocational education and training. 164 http://www.i-jim.org Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… The focus of [58] also differs from that of the current study because the model in [58] does not consider online platforms alone; rather, it is directed at the gamification of blended e-learning where e-learning is used in support of traditional classroom-based learning. Likewise, the gamification approach presented in [59] is not the same as that of the current study because [59] uses mobile technology and card games to increase the learning effectiveness, motivation, and engagement of blind school students. Hence, the model proposed in [59] cannot be used among adults employed in organizations (the target population of the current study) and which expect their employees to meet key performance indicators. Finally, in [60], the researchers study the impact of gamification on the learning outcomes, engagement, and learning experience of students. The study concludes that gamification can have a positive impact on student engagement. Nevertheless, the study does not propose a specific strategy for the gamification steps or game elements that can improve student engagement. 6 Methodology The methodology employed in this study consisted of three stages. The first stage focused on the identification of game elements that can enhance the engagement of employees in online training. The elements were identified through an analysis of the literature, as described in Section 7. The second stage involved an evaluation of the ele- ments identified in the first stage of the methodology. The evaluation was achieved by requesting a group of experts to complete a semi-structured interview questionnaire on the completeness and suitability of the identified elements. The process and the results of the evaluation are described and discussed in Section 8 and Section 9, respectively. Finally, the third stage concerned the development of a framework based on TAM con- structs that could be used to evaluate whether the identified game elements can improve employee engagement when added to an online training platform. The proposed frame- work is presented in Section 10. 7 Identification of game elements To date, studies have used a plethora of game elements to achieve user engagement. However, there is a lack of agreement on the best game elements for user engagement, particularly the engagement of adult employees. Therefore the authors conducted an extensive review to identify the commonly used game elements that might have the potential to be employed for the engagement of employees in online training. First, a set of studies primarily on online learning gamification studies were reviewed. Then, those gamification studies with a focus on user engagement in online learning systems were shortlisted so that the most common game elements could be extracted. The shortlisted studies are shown in Table 1. iJIM ‒ Vol. 16, No. 06, 2022 165 Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… Table 1. Selected sources Source Title Year [61] Effectiveness of gamification in the engagement of students 2016 [62] The role of gamified e-quizzes on student learning and engagement: An interactive gamification solution for a formative assessment system 2020 [63] Developing a comprehensive engagement framework of gamification for reflective learning 2016 [64] Gamification design framework based on SDT for student motivation 2015 [65] Gamification framework model based on social engagement in e-learning 2.0 2014 [66] Gamification in e-learning: Introducing gamified design elements into e-learning systems 2015 [67] A gamified e-learning design model to promote and improve learning 2016 [68] An architecture of a gamified learning management system 2014 [69] The contribution of gamification on user engagement in a fully online course 2015 [70] Students’ evidential increase in learning using gamified learning environment 2018 The authors were able to extract 54 game elements from the selected sources. Then, the extracted game elements were subjected to a reduction process, where the game elements were compared to each other by analysing their specific descriptions as pro- vided in the respective sources. This process resulted in the identification of nine game elements that are commonly used for user engagement. It should be noted that none of the sources contain all of the extracted game elements. Table 2 shows which studies contain the nine identified elements and Table 3 provides definitions of these elements. Table 2. Common game elements used for engagement Element Selected Frameworks [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] Points • • • • • • • • • Levels • • • • • • • • • Challenges • • • • • • • • Rewards • • • • • Competition • • • • • • • Progress bar • • • • • Leader board • • • • • • • Badges • • • • • • Feedback • • • • 166 http://www.i-jim.org Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… Table 3. Description of the identified game elements Element Description Points Points serve as the metric for progress, and users receive them depending on the progress accomplished. The total points score serves to rank users in terms of higher to lower levels of engagement. Levels Each level in the training system corresponds to the study of one of the training modules covered in the study case. Challenges During training, users are required to overcome several challenges. Rewards Users receive positive rewards for completed activities and for user interactions. Competition Users compete among themselves, either individually against another user, or as a member of a team against another team. Progress bar At each level, the progress bar enables users to visualize how much progress they have made in terms of completed activities versus total activities. In a similar way, users can also visualize their overall progress in the online training system in terms of completed levels and user evolution. Leader board Users can see their performance in relation to other users on a leader board that shows the top best performers in the training system. Badges Users are awarded badges, depending on how well they perform in quizzes (gold, silver, or bronze medals), and missions (mission starter, mission legend, or mission killer), user interaction (active user), and returning to the training system (frequent user). Feedback Users receive feedback on every activity they complete so that they can evaluate their decisions and improve their learning throughout the online training programme. 8 Evaluation of the identified game elements Several steps were taken to evaluate the effectiveness of the identified game ele- ments. The evaluation was conducted to determine first, whether the elements would be suitable for the gamification of online training systems and second, to help inves- tigate whether gamified online training improves employee engagement as compared to non-gamified online training. In general, an evaluation can be either formative or summative. A formative evaluation assesses the artefact while it is still in its design and development stage, whereas a summative evaluation assesses the artefact after it is fully developed. In this study, the authors used a formative evaluation to evaluate the elements. Expert evaluation is one of the methods that can be used in a formative evaluation. This type of evaluation was selected for this study because it has been used by previous gamification studies and because it is familiar to the authors. The selection of appro- priate experts is a key part of conducting an expert evaluation. The experts must be equipped with the needed knowledge and experience and should be willing to partici- pate. Therefore, the experts were carefully selected according to the following criteria: the expert must have utilized gamification, have more than three years of experience with online training systems and be currently employed by a well-known Saudi Arabia training provider. iJIM ‒ Vol. 16, No. 06, 2022 167 Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… Based on the above inclusion criteria, the authors identified 15 gamification experts who were eligible to participate in the evaluation. They were invited to take part in the study via phone or email. Seven out of the 15 experts agreed to participate in the evaluation, two of whom withdrew in the middle of the process. The remaining five experts completed the evaluation. On average, the experts had approximately six years of gamification experience and most of them had more than 10 years of online training experience. The evaluation experts were sent a semi-structured interview questionnaire via Google Forms to enable them to give their views. The experts were asked to evaluate the nine game elements by answering “Yes or No” questions with respect to the follow- ing: the completeness of the list of game elements, accuracy of the descriptions of the elements, usefulness of the elements, and whether some elements are more important than others. The following questions were posed: Q1: Do you agree that the nine elements presented in the list cover most of the game elements available and required for the gamification of online training? (Completeness) Q2: Do you agree with the descriptions of the nine elements that are presented in the list? (Accuracy of description) Q3: Are there descriptions that need changes? If yes, what would you change in those descriptions? (Accuracy of description) Q4: Are there any missing game elements that you think would be important for gam- ifying online training? If yes, please mention them. (Completeness) Q5: Do you think any specific game elements should be given higher importance than others? If yes, please mention them. (Importance of some elements over others) Q6: Are there game elements that you think should be removed from the list? If yes, please mention them. (Importance of some elements over others) In addition, the experts were asked two questions to obtain their opinion on the usefulness of the game elements. The experts were asked to answer these questions by selecting a score on a five-point Likert-type scale, where points 1 to 5 indicated worst, bad, good, better, and best, respectively. The questions were structured as follows: Q7: On a scale of 1 (worst) to 5 (best), how would you rate the usefulness of the ele- ments? (Usefulness) Q8: On a scale of 1 (worst) to 5 (best), how would you rate the ease of understanding of the elements? (Usefulness) 9 Discussion The questions were ordered in a randomized way in each expert questionnaire to avoid ordering bias. In regard to the first five questions, the experts were consistent in their evaluation. They all agreed that the list of the elements was sufficient for the 168 http://www.i-jim.org Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… gamification of online training systems. One of the experts even said, “These elements are the core of all game elements.” Hence, the experts unanimously asserted that there was no need for any additional elements. With regard to the descriptions of the elements, apart from one expert who com- mented, “Yes of course I agree but you have to evaluate which of the nine elements has the most direct influence on the first three criteria such as ease of use, usefulness and [attitude] towards use”, all of the experts approved the descriptions as written in the questionnaire. Therefore, no changes were made to the descriptions. As regards the importance of some elements over others, apart from one expert who said that elements such as badges, points, levels, and leader board are more important for training systems, the other four experts agreed that there are no elements to which they could give more value as compared to the others in the list. Moreover, none of the experts removed any elements from the list. Given the above, the authors decided that the importance of badges, points, levels, and leader board would be considered during later stages of the study but the overall contents of the list were not changed because most of the experts agreed that the same value should be given to each of the elements listed in the questionnaire. Finally, in respect of the usefulness and the ease of understanding of the game ele- ments, these two aspects were scored by the experts on a five-point Likert scale. Use- fulness and ease of understanding had an average score of 4.2 and 5, respectively. This result shows that the experts considered that the elements were very useful and very easy to understand. 10 Proposed framework In addition to identifying the nine key game elements for the proposed framework, the authors also tried to identify the user engagement dimensions and indicators that could be used when investigating the engagement of employees in online training. Three engagement dimensions were found to satisfy the aims of this study because pre- vious studies had employed them for investigating the engagement of higher education students with gamified online learning systems. The three dimensions are cognitive, emotional, and behavioural. As regards the underpinning theory for the framework, the TAM was selected for this study because it simplifies and provides granular steps for evaluating employee acceptance of gamification. Figure 1 shows the proposed frame- work developed for this study. The framework consists of three main components: game elements, engagement dimensions, and TAM constructs. iJIM ‒ Vol. 16, No. 06, 2022 169 Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… Fig. 1. Proposed gamification framework The authors believe that the proposed framework will be effective for studying employee engagement when attending online training. The framework advocates that the ease of use of gamification and the perceived usefulness of gamification (which is also predisposed by ease of use of gamification) are the two factors that are affect employee attitude towards gamification. In order to evaluate the relevance of the frame- work, it was subjected to expert validation. 11 Conclusion Many studies have been done on the gamification of educational systems. However, less has been said about the gamification of online training. At least, there are no design models or game elements that have been empirically proposed in the literature. There- fore, this study conducted a literature analysis to identify a set of game elements that can be added to online training platforms to improve employee engagement. A total of nine elements were identified and then evaluated by five experts selected from training provider organizations in Saudi Arabia. The experts gave their opinion on the identified elements and agreed that the elements would improve engagement among employees attending online training. Now that the important game elements have been identified and evaluated by experts, our future work will focus on developing our proposed gamification frame- work that can help developers add these elements to different components of online training platforms. It is expected that the framework will focus on activities specifically associated with the process of gamification of online training platforms. In other words, the framework will detail which game element is suitable for which component of the training platform. Finally, the framework will be employed in the development of a gamified online training system to determine whether this type of online training can improve the engagement of employees. 170 http://www.i-jim.org Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… 12 Acknowledgements This work was supported by the Universiti Kebangsaan Malaysia/Faculty of Information Science and Technology—Research Center for Software Technology and Management—Multimedia Software and Usability Research. 13 References [1] Butler, A., et al., The use of mobile learning technologies for corporate training and devel- opment: A contextual framework, in Strategic corporate communication in the digital age. 2021, Emerald Publishing Limited. https://doi.org/10.1108/978-1-80071-264-520211007 [2] Karakose, T., R. Yirci, and S. Papadakis, Exploring the interrelationship between covid- 19 phobia, work–family conflict, family–work conflict, and life satisfaction among school administrators for advancing sustainable management. Sustainability, 2021. 13(15): p. 8654. https://doi.org/10.3390/su13158654 [3] Beinicke, A. and T. Bipp, Evaluating training outcomes in corporate e-learning and class- room training. Vocations and Learning, 2018. 11(3): p. 501–528. https://doi.org/10.1007/ s12186-018-9201-7 [4] Angelova, M., Corporate trainings–opportunities and challenges for employees and man- agers. Knowledge Society and Global Economy: p. 11. [5] Alfaqiri, A.S., S.F.M. Noor, and N.S. Ashaari, Exploring indicators of engagement: Appli- cations for gamification of online training systems. Periodicals of Engineering and Natural Sciences (PEN), 2020. 8(4): p. 2096–2106. [6] Alsubhi, M.A., N.S. Ashaari, and T.S.M.T. Wook, The challenge of increasing student engage- ment in e-Learning platforms in 2019 International Conference on Electrical Engineering and Informatics (ICEEI). 2019. IEEE. https://doi.org/10.1109/ICEEI47359.2019.8988908 [7] Ab Rahman, R., S. Ahmad, and U.R. Hashim, A study on gamification for higher educa- tion students’ engagement towards education 4.0, in Intelligent and Interactive Computing. 2019, Springer. p. 491–502. https://doi.org/10.1007/978-981-13-6031-2_5 [8] Doumanis, I., et al., The impact of multimodal collaborative virtual environments on learn- ing: A gamified online debate. Computers & Education, 2019. 130: p. 121–138. https://doi. org/10.1016/j.compedu.2018.09.017 [9] Kyewski, E. and N.C. Krämer, To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course. Computers & Education, 2018. 118: p. 25–37. https://doi.org/10.1016/j.compedu. 2017.11.006 [10] Papadakis, S., Advances in Mobile Learning Educational Research (AMLER): Mobile learning as an educational reform. Advances in Mobile Learning Educational Research, 2021. 1(1): p. 1–4. https://doi.org/10.25082/AMLER.2021.01.001 [11] Poultsakis, S., et al., The management of digital learning objects of natural sciences and digital experiment simulation tools by teachers. Advances in Mobile Learning Educational Research, 2021. 1(2): p. 58–71. https://doi.org/10.25082/AMLER.2021.02.002 [12] Jayalath, J. and V. Esichaikul, Gamification to enhance motivation and engagement in blended eLearning for technical and vocational education and training. Technology, Knowledge and Learning, 2020: p. 1–28. https://doi.org/10.1007/s10758-020-09466-2 [13] Alsubhi, M.A., N.S. Ashaari, and T.S.M.T. Wook, Design and evaluation of an engage- ment framework for e-Learning gamification. International Journal of Advanced Computer Science and Applications (IJACSA), 2021. 12(9): p. 411–417. https://doi.org/10.14569/ IJACSA.2021.0120947 iJIM ‒ Vol. 16, No. 06, 2022 171 https://doi.org/10.1108/978-1-80071-264-520211007 https://doi.org/10.3390/su13158654 https://doi.org/10.1007/s12186-018-9201-7 https://doi.org/10.1007/s12186-018-9201-7 https://doi.org/10.1109/ICEEI47359.2019.8988908 https://doi.org/10.1007/978-981-13-6031-2_5 https://doi.org/10.1016/j.compedu.2018.09.017 https://doi.org/10.1016/j.compedu.2018.09.017 https://doi.org/10.1016/j.compedu.2017.11.006 https://doi.org/10.1016/j.compedu.2017.11.006 https://doi.org/10.25082/AMLER.2021.01.001 https://doi.org/10.25082/AMLER.2021.02.002 https://doi.org/10.1007/s10758-020-09466-2 https://doi.org/10.14569/IJACSA.2021.0120947 https://doi.org/10.14569/IJACSA.2021.0120947 Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… [14] Iacono, S., M. Vallarino, and G. Vercelli, Gamification in corporate training to enhance engagement: An approach. International Journal of Emerging Technologies in Learning (iJET), 2020. 15(17): p. 69–84. https://doi.org/10.3991/ijet.v15i17.14207 [15] Guetzoian, E., Gamification strategies for higher education student worker training, in handbook of research on acquiring 21st century literacy skills through game-based learn- ing. 2022, IGI Global. p. 164–179. https://doi.org/10.4018/978-1-7998-7271-9.ch009 [16] Larson, K., Serious games and gamification in the corporate training environment: A literature review. TechTrends, 2020. 64(2): p. 319–328. https://doi.org/10.1007/ s11528-019-00446-7 [17] Bemelmans-Videc, M.L., B. Eriksen, and E.N. Goldenberg, Facilitating organizational learning: Human resource management and program evaluation, in Can Governments Learn? 2020, Routledge. p. 145–187. https://doi.org/10.4324/9780429336256-11 [18] Cascio, W.F., Training trends: Macro, micro, and policy issues. Human Resource Manage- ment Review, 2019. 29(2): p. 284–297. https://doi.org/10.1016/j.hrmr.2017.11.001 [19] Gil, E.P., et al., Training and development from the managements viewpoint: a case study in the food sector. Navus: Revista de Gestão e Tecnologia, 2020. 10(1): p. 1–14. https://doi. org/10.22279/navus.2020.v10.p01-14.1194 [20] Pudovina, A., et al., Corporate training: from planning to efficiency. Revista Inclusiones, 2020: p. 448–466. [21] Garcia, I., et al., The effects of game-based learning in the acquisition of “soft skills” on undergraduate software engineering courses: A systematic literature review. Computer Applications in Engineering Education, 2020. 28(5): p. 1327–1354. https://doi.org/10.1002/ cae.22304 [22] Scaduto, A., D. Lindsay, and D.S. Chiaburu, Leader influences on training effectiveness: motivation and outcome expectation processes. International Journal of Training and Devel- opment, 2008. 12(3): p. 158–170. https://doi.org/10.1111/j.1468-2419.2008.00303.x [23] Ali, S.M., et al., A framework for improving the engagement of medical practitioners in an e-training platform for tuberculosis care and prevention. Future Internet, 2019. 11(1): p. 6. https://doi.org/10.3390/fi11010006 [24] Karakose, T., et al., Science mapping of the global knowledge base on management, leader- ship, and administration related to COVID-19 for promoting the sustainability of scientific research. Sustainability, 2021. 13(17): p. 9631. https://doi.org/10.3390/su13179631 [25] Schöbel, S., et al. Gamification of online training and its relation to engagement and prob- lem-solving outcomes in Academy of management proceedings. 2019. Academy of Manage- ment Briarcliff Manor, NY 10510. https://doi.org/10.5465/AMBPP.2019.11949abstract [26] Karakose, T., H. Polat, and S. Papadakis, Examining teachers’ perspectives on school principals’ digital leadership roles and technology capabilities during the COVID-19 pan- demic. Sustainability, 2021. 13(23): p. 13448. https://doi.org/10.3390/su132313448 [27] Sharek, D. and E. Wiebe, Investigating real-time predictors of engagement: Implications for adaptive videogames and online training. International Journal of Gaming and Com- puter-Mediated Simulations (IJGCMS), 2015. 7(1): p. 20–37. https://doi.org/10.4018/ IJGCMS.2015010102 [28] Sitzmann, T. and J.M. Weinhardt, Training engagement theory: A multilevel perspective on the effectiveness of work-related training. Journal of Management, 2018. 44(2): p. 732–756. https://doi.org/10.1177/0149206315574596 [29] Wu, S., M.-A. Rizoiu, and L. Xie, Beyond views: Measuring and predicting engagement in online videos in Twelfth international AAAI conference on web and social media. 2018. [30] Wook, T.S.M.T. and N.S. Ashaari, An investigation of user engagement factors in E-learning for working adult learners in 2015 International Conference on Electrical Engineering and Informatics (ICEEI). 2015. IEEE. 172 http://www.i-jim.org https://doi.org/10.3991/ijet.v15i17.14207 https://doi.org/10.4018/978-1-7998-7271-9.ch009 https://doi.org/10.1007/s11528-019-00446-7 https://doi.org/10.1007/s11528-019-00446-7 https://doi.org/10.4324/9780429336256-11 https://doi.org/10.1016/j.hrmr.2017.11.001 https://doi.org/10.22279/navus.2020.v10.p01-14.1194 https://doi.org/10.22279/navus.2020.v10.p01-14.1194 https://doi.org/10.1002/cae.22304 https://doi.org/10.1002/cae.22304 https://doi.org/10.1111/j.1468-2419.2008.00303.x https://doi.org/10.3390/fi11010006 https://doi.org/10.3390/su13179631 https://doi.org/10.5465/AMBPP.2019.11949abstract https://doi.org/10.3390/su132313448 https://doi.org/10.4018/IJGCMS.2015010102 https://doi.org/10.4018/IJGCMS.2015010102 https://doi.org/10.1177/0149206315574596 Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… [31] DeKeyser, R., B. VanPatten, and J. Williams, Skill acquisition theory. Theories in second language acquisition: An introduction, 2007. 97113. https://doi.org/10.4324/9781410615299 [32] Gupta, S., Reducing computer anxiety in self-paced technology training. 2017. https://doi. org/10.24251/HICSS.2017.018 [33] Alsubhi, M.A., N. Sahari, and T.T. Wook, A conceptual engagement framework for gamified e-learning platform activities. International Journal of Emerging Technologies in Learning (iJET), 2020. 15(22): p. 4–23. https://doi.org/10.3991/ijet.v15i22.15443 [34] Schöbel, S., M. Söllner, and J.M. Leimeister, The agony of choice–analyzing user prefer- ences regarding gamification elements in learning management systems. 2016. https://doi. org/10.2139/ssrn.3159163 [35] Santhanam, R., D. Liu, and W.-C.M. Shen, Research note—gamification of technology- mediated training: not all competitions are the same. Information Systems Research, 2016. 27(2): p. 453–465. https://doi.org/10.1287/isre.2016.0630 [36] Murawski, L., Gamification in human resource management—status quo and quo vadis. German Journal of Human Resource Management, 2021. 35(3): p. 337–355. https://doi. org/10.1177/2397002220961796 [37] Nurtanto, M., et al., A review of gamification impact on student behavioral and learning outcomes. iJIM, 2021. 15(21): p. 23. https://doi.org/10.3991/ijim.v15i21.24381 [38] Vidakis, N., et al., In-game raw data collection and visualization in the context of the “Thi- melEdu” educational game in international conference on computer supported education. 2019. Springer. https://doi.org/10.1007/978-3-030-58459-7_30 [39] Ērgle, D. and I. Ludviga, Use of gamification in human resource management: Impact on engagement and satisfaction. in 10th international scientific conference, “business and management 2018”. 2018. https://doi.org/10.3846/bm.2018.45 [40] Marzi, G., et al., Gamification in human resource management webinar. 2020. [41] Silic, M., et al., The effects of a gamified human resource management system on job satis- faction and engagement. Human Resource Management Journal, 2020. 30(2): p. 260–277. https://doi.org/10.1111/1748-8583.12272 [42] Walls, P.J., Investing in human capital: achieving organizational competitiveness through gamification. ENDLESS: International Journal of Future Studies, 2021. 4(1): p. 1–7. https:// doi.org/10.54783/endless.v4i1.34 [43] Hamari, J., J. Koivisto, and H. Sarsa, Does gamification work?—A literature review of empirical studies on gamification. in 2014 47th Hawaii international conference on system sciences. 2014. IEEE. https://doi.org/10.1109/HICSS.2014.377 [44] Khaleel, F.L., et al., The architecture of dynamic gamification elements based learning con- tent. Journal of Convergence Information Technology, 2016. 11(3): p. 164–177. [45] Jamaludin, N.F., et al., Gamification design elements to enhance adolescent motivation in diagnosing depression. International Journal of Interactive Mobile Technologies, 2021. 15(10). https://doi.org/10.3991/ijim.v15i10.21137 [46] Botte, B., S. Bakkes, and R. Veltkamp, Motivation in gamification: constructing a cor- relation between gamification achievements and self-determination theory. in Inter- national conference on games and learning alliance. 2020. Springer. https://doi. org/10.1007/978-3-030-63464-3_15 [47] Ryan, R.M. and E.L. Deci, Intrinsic and extrinsic motivation from a self-determination the- ory perspective: definitions, theory, practices, and future directions. Contemporary Educa- tional Psychology, 2020. 61: p. 101860. https://doi.org/10.1016/j.cedpsych.2020.101860 [48] Vann, S.W. and A.A. Tawfik, Flow theory and learning experience design in gamified learn- ing environments. Learner and User Experience Research, 2020. iJIM ‒ Vol. 16, No. 06, 2022 173 https://doi.org/10.4324/9781410615299 https://doi.org/10.24251/HICSS.2017.018 https://doi.org/10.24251/HICSS.2017.018 https://doi.org/10.3991/ijet.v15i22.15443 https://doi.org/10.2139/ssrn.3159163 https://doi.org/10.2139/ssrn.3159163 https://doi.org/10.1287/isre.2016.0630 https://doi.org/10.1177/2397002220961796 https://doi.org/10.1177/2397002220961796 https://doi.org/10.3991/ijim.v15i21.24381 https://doi.org/10.1007/978-3-030-58459-7_30 https://doi.org/10.3846/bm.2018.45 https://doi.org/10.1111/1748-8583.12272 https://doi.org/10.54783/endless.v4i1.34 https://doi.org/10.54783/endless.v4i1.34 https://doi.org/10.1109/HICSS.2014.377 https://doi.org/10.3991/ijim.v15i10.21137 https://doi.org/10.1007/978-3-030-63464-3_15 https://doi.org/10.1007/978-3-030-63464-3_15 https://doi.org/10.1016/j.cedpsych.2020.101860 Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… [49] Flavin, M. and V. Quintero, UK higher education institutions’ technology-enhanced learn- ing strategies from the perspective of disruptive innovation. Research in Learning Technol- ogy, 2018. 26. https://doi.org/10.25304/rlt.v26.1987 [50] Davis, F.D., Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 1989: p. 319–340. https://doi.org/10.2307/249008 [51] Surendran, P., Technology acceptance model: a survey of literature. International Journal of Business and Social Research, 2012. 2(4): p. 175–178. [52] Fathema, N., D. Shannon, and M. Ross, Expanding the Technology Acceptance Model (TAM) to examine faculty use of Learning Management Systems (LMSs) in higher education institutions. Journal of Online Learning & Teaching, 2015. 11(2). [53] Baptista, G. and T. Oliveira, Why so serious? Gamification impact in the acceptance of mobile banking services. Internet Research, 2017. https://doi.org/10.1108/IntR-10-2015-0295 [54] Türel, Y.K., M. Turhan, and M. Turan, A multicultural examination of the dropout problem for e-Learning courses. Better e-Learning for Innovation in Education, 2017: p. 125. [55] Ding, L., C. Kim, and M. Orey, Studies of student engagement in gamified online dis- cussions. Computers & Education, 2017. 115: p. 126–142. https://doi.org/10.1016/j. compedu.2017.06.016 [56] Armstrong, M.B. and R.N. Landers, An evaluation of gamified training: Using narrative to improve reactions and learning. Simulation & Gaming, 2017. 48(4): p. 513–538. https://doi. org/10.1177/1046878117703749 [57] Putra, S.D. and V. Yasin, MDA framework approach for gamification-based elementary mathematics learning design. International Journal of Engineering, Science and Information Technology, 2021. 1(3): p. 35–39. https://doi.org/10.52088/ijesty.v1i3.83 [58] Behl, A., et al., Gamification and gigification: A multidimensional theoretical approach. Journal of Business Research, 2022. 139: p. 1378–1393. https://doi.org/10.1016/j.jbusres. 2021.09.023 [59] Sophie Chang, C.-H., et al., Design and evaluation of a multi-sensory scaffolding gamifica- tion science course with mobile technology for learners with total blindness. 2022. https:// doi.org/10.1016/j.chb.2021.107085 [60] Yang, K.C. and Y. Kang, The effectiveness of gamification on student engagement, learning outcomes, and learning experiences, in research anthology on developments in gamification and game-based learning. 2022, IGI Global. p. 1599–1618. https://doi.org/10.4018/978-1- 6684-3710-0.ch077 [61] da Rocha Seixas, L., A.S. Gomes, and I.J. de Melo Filho, Effectiveness of gamification in the engagement of students. Computers in Human Behavior, 2016. 58: p. 48–63. https://doi. org/10.1016/j.chb.2015.11.021 [62] Zainuddin, Z., et al., The role of gamified e-quizzes on student learning and engagement: an interactive gamification solution for a formative assessment system. Computers & Educa- tion, 2020. 145: p. 103729. https://doi.org/10.1016/j.compedu.2019.103729 [63] Silpasuwanchai, C., et al., Developing a comprehensive engagement framework of gami- fication for reflective learning in Proceedings of the 2016 ACM conference on designing interactive systems. 2016. ACM. https://doi.org/10.1145/2901790.2901836 [64] Lamprinou, D. and F. Paraskeva, Gamification design framework based on SDT for student motivation in 2015 International conference on Interactive Mobile Communication Technol- ogies and Learning (IMCL). 2015. IEEE. https://doi.org/10.1109/IMCTL.2015.7359631 [65] Wongso, O., Y. Rosmansyah, and Y. Bandung, Gamification framework model, based on social engagement in e-learning 2.0. in 2014 2nd International conference on technology, informatics, management, engineering & environment. 2014. IEEE. https://doi.org/10.1109/ TIME-E.2014.7011583 174 http://www.i-jim.org https://doi.org/10.25304/rlt.v26.1987 https://doi.org/10.2307/249008 https://doi.org/10.1108/IntR-10-2015-0295 https://doi.org/10.1016/j.compedu.2017.06.016 https://doi.org/10.1016/j.compedu.2017.06.016 https://doi.org/10.1177/1046878117703749 https://doi.org/10.1177/1046878117703749 https://doi.org/10.52088/ijesty.v1i3.83 https://doi.org/10.1016/j.jbusres.2021.09.023 https://doi.org/10.1016/j.jbusres.2021.09.023 https://doi.org/10.1016/j.chb.2021.107085 https://doi.org/10.1016/j.chb.2021.107085 https://doi.org/10.4018/978-1-6684-3710-0.ch077 https://doi.org/10.4018/978-1-6684-3710-0.ch077 https://doi.org/10.1016/j.chb.2015.11.021 https://doi.org/10.1016/j.chb.2015.11.021 https://doi.org/10.1016/j.compedu.2019.103729 https://doi.org/10.1145/2901790.2901836 https://doi.org/10.1109/IMCTL.2015.7359631 https://doi.org/10.1109/TIME-E.2014.7011583 https://doi.org/10.1109/TIME-E.2014.7011583 Paper—Framework for Gamification of Online Training Platforms for Employee Engagement… [66] Strmečki, D., A. Bernik, and D. Radošević, Gamification in e-Learning: introducing gam- ified design elements into e-learning systems. Journal of Computer Science, 2015. 11(12): p. 1108–1117. https://doi.org/10.3844/jcssp.2015.1108.1117 [67] Malas, R.I. and T. Hamtini, A gamified e-learning design model to promote and improve learning. Int Rev Comput Softw (IRECOS), 2016. 11: p. 8–19. https://doi.org/10.15866/ irecos.v11i1.7913 [68] Swacha, J., An architecture of a gamified learning management system in Interna- tional Conference on Web-Based Learning. 2014. Springer. https://doi.org/10.1007/ 978-3-319-13296-9_22 [69] Surendeleg, G., U. Tudevdagva, and Y.S. Kim, The contribution of gamification on user engagement in fully online course in Creativity in Intelligent, Technologies and Data Sci- ence: First Conference, CIT&DS 2015, Volgograd, Russia, September 15–17, 2015, Pro- ceedings. 2015. Springer. [70] Vanduhe, V., et al., Students’ evidential increase in learning using gamified learning envi- ronment in Proceedings of the future technologies conference. 2018. Springer. https://doi. org/10.1007/978-3-030-02686-8_82 14 Authors Abdullatif Saleh Alfaqiri is a Ph.D. candidate in the Faculty of Information Science & Technology, Universiti Kebangsaan Malaysia (UKM), Malaysia. He received his Bachelor’s degree in Computer Science from Tabuk University, Saudi Arabia and a Master’s degree in Computer Security at De Montfort University in the UK. His area of expertise is technologies in e-learning platforms and he is involved in research related to human–computer interaction and e-learning systems & platforms. Email: P90491@ siswa.ukm.edu.my Siti Fadzilah Mat Noor is a senior lecturer and a researcher at the Learning Technol- ogy and Human–Computer Interaction Research Lab Centre for Software Technology and Management, Faculty of Information Science & Technology, UKM, Malaysia. Her research interests include multimedia applications and e-learning technology. Email: fadzilah@ukm.edu.my Noraidah Sahari is an associate professor at the Centre for Software Technology and Management, Faculty of Information Science & Technology, UKM, Malaysia. Her research interests are multimedia applications, e-learning technology, interaction design and usability. Email: nsa@ukm.edu.my Article submitted 2021-11-30. Resubmitted 2022-01-13. Final acceptance 2022-01-22. Final version published as submitted by the authors. iJIM ‒ Vol. 16, No. 06, 2022 175 https://doi.org/10.3844/jcssp.2015.1108.1117 https://doi.org/10.15866/irecos.v11i1.7913 https://doi.org/10.15866/irecos.v11i1.7913 https://doi.org/10.1007/978-3-319-13296-9_22 https://doi.org/10.1007/978-3-319-13296-9_22 https://doi.org/10.1007/978-3-030-02686-8_82 https://doi.org/10.1007/978-3-030-02686-8_82 mailto:P90491@siswa.ukm.edu.my mailto:P90491@siswa.ukm.edu.my mailto:fadzilah@ukm.edu.my mailto:nsa@ukm.edu.my