Australasian Journal of Educational Technology, 2021, 37(4). 145 Students' preference for the use of gamification in virtual learning environments Julieth Katherin Acosta-Medina, Martha Liliana Torres-Barreto, Andrés Felipe Cárdenas-Parga Universidad Industrial de Santander Virtual education faces tremendous challenges such as lack of motivation and high dropout rates. However, one solution to this problem is the incorporation of digital pedagogical strategies based on gamification, which promote interest, facilitate the learning process, and contribute to reducing dropout. In this sense, we evaluated the student's preference for using the gamified tool Didactic City, by using a quantitative methodology and structural equation models. This preference included fundamental factors such as utility, knowledge, engagement, enjoyment, motivation, and ease of use. The results are expected to contribute to create a more informed decision-making process not only for video game designers but also for pedagogues, and educational managers who could perform game improvements, redesigns, inclusion, or exclusion of individual elements that would make students prefer to use gamification tools in the future. The results indicate that the students' utility, the enjoyment generated by the tool, and the improvement of knowledge are the critical factors in terms of preference for using gamified tools in virtual learning environments. Implications for policy and practice: • In virtual learning environments, higher education institutions and instructors can include gamified tools to create pleasant environments that increase students' motivation and facilitate learning. • Designers and developers of gamified educational tools should measure their acceptance, as this factor is decisive for fulfilling the purposes of these pedagogical tools. • Researchers could evidence more factors that influence the preference for the use of gamified educational tools. Keywords: gamification, preference for use, e-learning, virtual learning environments, structural equation modelling Introduction One of the main problems of higher education is the high dropout rate. In 2013, around a third of students in the whole world dropped out of college in their first school year (Organisation for Economic Co- operation and Development [OCDE], 2013). In Colombia, the dropout rate is higher in virtual academic programs (60%), compared to face-to-face programs (48%) (Ministerio de Educación Nacional de Colombia, 2017). Although there are multiple risk factors for dropouts, such as knowledge deficit, lack of resources, or poor communication within the classroom, it is worth noting that the common cause and/or consequence of these is low motivation to learn (González Castro et al., 2017). Even students who do not leave the educational system and manage to finish their careers report high boredom rates and state that the learning process is tedious and unattractive (Ochoa Sierra & Moya Pardo, 2018). They consider virtual learning environments as rigid and cold distant scenarios dedicated exclusively to sharing content and carrying out activities (Melo-Solarte & Díaz, 2018). However, new information technologies can play an essential role in the fight, countering these problems since they facilitate the implementation of strategies that promote interest, improve motivation, facilitate learning, and, consequently, reduce dropouts. Among these strategies, those that incorporate gamification stand out. Gamification is understood as an educational trend that improves learning environments to make them more pleasant (Filippou et al., 2018), increasing motivation, commitment, and academic performance (Acosta-Medina, Torres -Barreto, & Alvarez-Melgarejo, 2020; Toda et al., 2019) and therefore contributing positively to the development of teaching and learning processes (Lobo-Rueba et al., 2020). When using digital gamified tools, it is essential to measure the preference for their use since this factor acts as a determining catalyst to fulfill the tool's purposes. Furthermore, measuring this factor provides valuable information for the tool's designers, allowing them to correct or adjust the game in several aspects (Fu et al., 2009). Australasian Journal of Educational Technology, 2021, 37(4). 146 Within this context, the paper sought to answer the question: Which factors determine students' preference for the use of gamification in their virtual classrooms? To this end, we proposed an evaluation model that involved a set of factors such as ease of use, engagement, motivation, and knowledge, combining the unified theory of acceptance and use of technology (UTAUT) model (Venkatesh et al., 2003) with the fun-theory and the learning-theory (Filippou et al., 2018), and led us to a set of equations that evaluate the main factors affecting the student's preference for using a gamified tool, such as Didactic City. These results were useful to improve this gamified tool and to consider when designing new tools. The study reinforces previous findings in terms of students' preferences for use. It lays the foundations for developing future educational gamified tools, as results indicate the importance of the designers' pedagogical work, contributing to face the challenges in academia to develop efficient and effective innovative pedagogical strategies based on gamification. Theoretical framework The new generation of students, also known as digital natives (Prensky, 2013), net generation (Oblinger et al., 2005), millennials (Howe & Strauss, 2000), or gamer generation (Beck & Wade, 2004) have experienced a world with so much information and communication technologies, and are totally surrounded by social networks and online games. These students have acquired specific technical skills and require new pedagogical approaches (Bourgonjon et al., 2010). In this way, gamification emerges as a promising tool for teaching and learning processes and becomes attractive for twenty-first century students (Fotaris et al., 2016; Kocadere & Çağlar, 2018). Didactic City Among the gamified experiences, is Didactic City. This is a digital gamified tool available in a video game format, launched into the market in 2020. Its main objective is to reinforce the teaching and learning processes of citizenship competencies and reduce student desertion in virtual training programs. Its predominant mechanic is building. Its users assume the city-mayors' role and are asked to solve different civic education missions, to reinforcing written and cognitive citizenship skills (Paba-Medina et al., 2020). For each completed mission, the player is given a series of coins that they can use to acquire different buildings and build-up their city (Figure 1). Figure 1. Didactic City environment Didactic City has eight levels, each one increase in difficulty. There are new city maps to build, new available buildings, and more complicated civic education missions at each higher level. It has an artificial intelligence configuration with different bots that allow establishing a student-teacher contactability model Australasian Journal of Educational Technology, 2021, 37(4). 147 through different communication channels such as email, text messages, or phone calls, offering them additional academic material or facilitating the permanent accompaniment of the teacher. This occurs when a student is detected through a machine learning model (MLM) as a possible dropout from the educational system. The Didactic City MLM is based on two algorithms: decision trees and neural networks, and uses data from students related to the platform's interaction, such as class attendance, homework performed, and grades. Neural networks combine each possible state of the input data with each possible state of the data to be predicted. They generate variables used by the decision trees to classify the data in high, medium, and low ranges of motivation. These two algorithms were used because previous literature has shown that they have higher precision when forecasting the academic motivation of the student based on their behaviour in the learning management system (LMS) (Đurđević Babić, 2017). The MLM has an accuracy of 94.4% and uses students' data to be trained to reach better predictions continuously. When a student falls in a low range of motivation, they are considered as a high probability dropping-out- student. Consequently, the system generates new actions to try to raise their motivation again. It is based on the theory of self-determination, which states that students who are motivated in their school activities are more likely to stay in school, facilitating conceptual understanding and adaptation to the school environment (Deci et al., 1991). Likewise, in this study, student motivation was calculated using academic variables used in previous investigations, such as Cocea and Weibelzahl (2011) and Kularbphettong and Tongsiri (2012), who used variables such as score test, score class activity, number of test and grade to predict student motivation. For Didactic City's development, the C# programming language was used in the multiplatform video game engine Unity 3D. Also, for the programming of the machine learning model, the Python programming language was used. Both components are connected through REST API application programming interface rest with JSON coding functions to allow for storage and distribution of text, organised in a front-end and back-end infrastructure. On the front-end, there are direct user interactions with Didactic City, and on the back-end, there is the end-to-end information management to interact with the other components of the platform as the contactability module and the database. Evaluation of gamified tools Although there has been limited research on evaluating gamified educational tools, some studies provide guidelines for identifying the relationship between knowledge and game style. However, there are numerous learning styles, which makes the empirical application of this evaluation difficult (Prensky, 2001). On the other hand, heuristics have been used in other evaluations, which introduces the researcher's particular subjectivity. In this sense, the combination of both techniques can make the evaluation of gamified tools more effective (Fu et al., 2009). Additionally, previous research has proposed different models to determine the acceptance of technology in a particular environment. For example, the theory of reasoned action states that most actions are determined by the rational valuation made of expectations or results that may or may not be achieved. The theory of planned behaviour adds the variable of planned behaviour. The technology acceptance model (TAM) emerges by combining these two theories that assess perceived utility, ease of use, and enjoyment. The motivational model add two constructs to the previous variables: intrinsic motivation and extrinsic motivation. The model of personal computer utilisation also assesses four other constructs: prior knowledge and skills, ease of use, social factors, and enabling conditions. The theory of diffusion of innovation states that some people are more open than others to the adoption of an innovation, and they react very differently. Likewise, cognitive social theory confirms that human behaviour must be described in terms of the interaction between cognitive, behavioural, and environmental determinants (Fernández Morales et al., 2015). By considering the previous theories and models along with the expectation confirmation theory, emerges the UTAUT model, which assesses factors such as effort, the expectation of performance, ease, social influence, use, and intention of behaviour mediated by variables such as age, gender, and experience (Venkatesh et al., 2003). However, this model does not evaluate fundamental aspects of gamification, neither the characteristics of educational tools, therefore, it is pertinent to combine UTAUT with some Australasian Journal of Educational Technology, 2021, 37(4). 148 complementary aspects of the gaming experience, such as fun and learning. For example, the fun-model proposes that people's behaviours can be explained through their intrinsic psychological motivations such as enjoyment, immersion, engagement, gaming experience, and social interaction. On the other hand, the learning-model is used mainly to evaluate tools applied in education; and therefore, it evaluates constructs such as knowledge and utility (Filippou et al., 2018). Conceptual model In this research, we combine the UTAUT model (perceived utility, motivation or behavioural intention, ease of use), with the fun-theory (engagement and enjoyment) and the learning-theory (knowledge), and we added certain variables linked to the neural characteristics of Didactic City. Therefore, the factors considered are: (1) perceived utility, (2) knowledge, (3) engagement, (4) enjoyment, (5) motivation, and (6) ease of use. Each of these factors and the associated hypotheses are described below. 1. Perceived utility (USE). Is defined as the degree of performance improvement that an individual perceives when using a particular tool (Davis, 1989). This factor is one of the fundamental determinants for people's attitude towards a new system or technology (Van Der Heijden, 2004). For the case of learning tools, when students consider them relevant and useful, they are more likely to prefer to use them (Prensky, 2010). H1. The perceived utility positively affects the preference for use. 2. Knowledge (KNO). Students have a greater preference for learning tools that allow interaction, critical thinking, control, and experimentation (Prensky, 2001). When students perceive a gamified tool as useful in the classroom, they also believe it improves their knowledge since it helps them achieve their learning goals, resulting in students preferring its use (Filippou et al., 2018). H2. Knowledge positively affects the preference for use. H3. Knowledge positively affects the perceived utility. 3. Engagement (ENG). Engagement occurs when players get pleasure from interacting with the gamified tool's mechanics, exhibiting active participation with a high degree of attention and enthusiasm. When this occurs, the student is involved in the learning process, facilitating the acquisition of knowledge (Wichadee & Pattanapichet, 2018). This factor is conditioned by characteristics related to completing tasks and the probability of getting immediate feedback. If a gamified experience meets these conditions, students can be expected to find it useful (Filippou et al., 2018). H4. Engagement positively affects perceived utility. 4. Enjoyment (ENJ). Enjoyment is essential for gamification because it generates dopamine release in the brain, a neurotransmitter that increases motivation, attention, and therefore learning ability (Acosta-Medina, Torres-Barreto, Álvarez-Melgarejo, et al., 2020). This is an essential source of value for gamers, affecting preference for use rather than utility, as students can enjoy using the gamified system without seeing its possible contribution to performance. However, if the tool is useful but has no enjoyment, the preference of using it is very likely to be affected (Yang et al., 2017). H5. The enjoyment positively affects the preference for use. 5. Motivation (MOT). Refers to efforts that lead to meet objectives or satisfy needs. It is based on a willingness or intention of change. Motivation is driven by different scenarios (Su & Cheng, 2015). It is essential for meaningful learning (Prensky, 2013). Gamification can generate two types of motivations: extrinsic, generated by external factors such as tangible prizes and rewards associated with self-realisation and personal growth; and intrinsic motivation, which facilitates the student to have a successful learning process (Su, 2016). H6. Motivation positively affects engagement. H7. Motivation positively affects knowledge. 6. Ease of use (EoU). Ease of use is a critical component in the process of adopting technological systems and is defined as the lack of effort that a person perceives when using a particular system, therefore, it refers to simplicity, either in understanding, interaction, accessibility, or operation (Davis, 1989; Yang et al., 2017). If the gamified tool is easy to use, this allows students to focus their attention on interacting with it and using it for its intended purpose, not focusing on how to operate it (Van Der Heijden, 2004). Furthermore, a high-complicated system distracts users from their primary task, decreasing perceptions of knowledge improvement, engagement, and enjoyment (Filippou et al., 2018). Australasian Journal of Educational Technology, 2021, 37(4). 149 H8. Ease of use positively affects usability. H9. Ease of use positively affects engagement. H10. Ease of use positively affects knowledge. H11. Ease of use positively affects enjoyment. The joint model of hypotheses and relationships is presented in Figure 2. Figure 2. Conceptual model Research methodology The methodological approach used in this research is quantitative. Initially, based on the conceptual model proposed to measure the preference for the use of Didactic City, a questionnaire was designed to collect the students' perception and to test the hypotheses. This was done following the steps proposed by DeVellis, (1991): define the concept to be measured, create a list of factors to consider, establish items in each factor, determine the measurement scale, and choose a pilot sample to assess reliability and validity of the survey. For the last step, the following tests and indicators were used: content validity, Cronbach's alpha (α), compound reliability (CR), convergent validity (AVE), and item analysis. Subsequently, as part of this academic exercise, a sample of 256 virtual higher education students was asked to use the gamified tool Didactic City for 4 weeks. A simple random convenience sampling was used and include students from different Colombian universities who were enrolled in virtual training programs since these programs are the main beneficiaries and users of Didactic City. After completing specific missions within the tool, the players answered the questionnaire. There were 203 responses, representing a response rate of 79.3%. Additionally, each student who participated in the study was provided with informed consent to make it clear that their participation was voluntary, any data collected was deidentified, and that the data provided was purely for educational purposes. The ethical principles of justice, respect, autonomy, benevolence, and confidentiality were guaranteed. All these research procedures used in this study were approved by the Research and Extension Operating Committee of the university associated with this research. The collected information was analysed using a structural equation model (SEM). We examined the different predictors in the preference for the use of the gamified tool. We also used confirmatory factor analysis (CFA). This model was executed using IBM SPSS AMOS software following the steps proposed by Bagozzi and Yi (2012): (1) specification, (2) identification, (3) parameter estimation, in this case using the maximum likelihood estimation method (MLE), (4) fit evaluation, and (5) model re-specification. Finally, the results were analysed and compared with the findings from similar investigations found in the literature. Results and analysis Data collection instrument The questionnaire used for this research was divided into two parts. The first part included sociodemographic data (age, gender, previous experience with gamified tools). The second part contained Australasian Journal of Educational Technology, 2021, 37(4). 150 35 items assigned to the seven factors of study: perceived utility (4 items), knowledge (4 items), engagement (6 items), enjoyment (5 items), motivation (7 items), ease of use (6 items), and preference for its use (3 items). The surveyed students rated each item using a 5-point Likert scale (strongly disagree to strongly agree). These items were ordered randomly to avoid possible bias in the answers (Table 1). Table 1 Instrument used for data collection Variable Items Perceived utility (USE) Adapted from (Davis, 1989; Filippou et al., 2018) USE1 Using Didactic City increases my performance in citizenship competences USE2 Didactic City would help me to get better results in the SABER-PRO tests USE3 Didactic City is more effective compared to other teaching and learning strategies USE4 I think Didactic City is a useful tool Knowledge (KNO) Adapted from (Filippou et al., 2018; Fu et al., 2009) KNO1 Didactic City increases my knowledge in citizenship competencies KNO2 I use my previous knowledge in citizenship competences when I play Didactic City KNO3 I am interested in learning about citizenship competences with Didactic City KNO4 I am motivated to use the acquired knowledge of citizenship competences in daily life Engagement (ENG) Adapted from (Chen et al., 2018; Filippou et al., 2018) ENG1 The activities that I can do in Didactic City keep me interested all the time ENG2 Didactic City activities encourage me to use the tool more frequently ENG3 As I played, I was talking to myself in a loud voice ENG4 I am curious to play all the levels of Didactic City ENG5 I am not sure what I can find in the next level of Didactic City and that causes more interest ENG6 I felt the time passing by quickly when using Didactic City Enjoyment (ENJ) Adapted from (Filippou et al., 2018; Koivisto & Hamari, 2014) ENJ1 I feel happy when using Didactic City ENJ2 I feel bored when using Didactic City ENJ3 I feel tired when I finish using Didactic City ENJ4 I find the experience of using Didactic City very pleasant. ENJ5 I find the experience of using Didactic City very interesting Motivation (MOT) Adapted from (Chen et al., 2018) MOT1 I am motivated to complete all the levels MOT2 Didactic City makes I feel enthusiastic MOT3 When I get badges in Didactic City I feel good MOT4 The moral dilemmas of Didactic City called out my attention MOT5 The problem situations of Didactic City called out my attention MOT6 Didactic City completely got my attention MOT7 I feel curious about the SABER-PRO questions from Didactic City Ease of use (EoU) Adapted from (Filippou et al., 2018; Koivisto & Hamari, 2014; Van Der Heijden, 2004) EoU1 I easily understood how Didactic City works EoU2 Didactic City did not demand a greater mental effort to understand how it works EoU3 the interaction with Didactic City is clear and understandable EoU4 I consider it easy to level up in Didactic City EoU5 I find it easy to interact with Didactic City EoU6 I consider Didactic City easy to use. Use preference (PRF) Adapted from (Filippou et al., 2018) PRF1 If I could choose, I would choose a course where this tool was used PRF2 If I had to vote, I would vote to use Didactic City in virtual courses PRF3 I am enthusiastic about using Didactic City in virtual courses Australasian Journal of Educational Technology, 2021, 37(4). 151 Reliability and validity of the instrument We carried out a process of validation of our instrument as part of a pilot study, using the following steps. Expert evaluation of the instrument's content and validity During this process, 2 items of the ease of use factor (EoU2 and EoU5) and 3 of the motivation factors (M4, M5, and M7) were eliminated due to the incorrect interpretation questions. Application of a pilot test to 65 virtual students For the analysis, the records with missing values were discarded, Ten records were deleted. Cronbach's alpha was calculated to assess the questionnaire's reliability and stability, obtaining a value of less than 0.7. It was necessary to eliminate certain items in some factors (ENG3, ENG6, ENJ3, and K2). After this exercise, a Cronbach's alpha of 0.796 was obtained for the entire survey as a group and values greater than 0.733 for each factor separately. This showed that the scale developed had high internal consistency and reliability, since values higher than 0.7 are considered optimal (Hair et al., 2010). Assessment of compound reliability (CR) This calculation was used to evaluate the items' consistency in each factor and the complete questionnaire. All obtained values higher than 0.8 (Table 2). Table 2 Instrument reliability and validity Item Mean SD Mean SD Cronbach’s α CR AVE ENG1 3.93 0.74 3.89 0.92 0.73 0.84 0.57 ENG2 3.60 0.91 ENG4 4.25 0.91 ENG5 3.76 1.10 ENJ1 3.62 0.85 4.03 0.83 0.82 0.88 0.66 ENJ2 4.16 0.83 ENJ4 4.25 0.80 ENJ5 4.09 0.82 EoU1 3.58 1.15 3.59 1.10 0.82 0.89 0.67 EoU3 3.45 0.90 EoU4 3.53 1.30 EoU6 3.80 1.04 K1 4.33 0.79 4.22 0.77 0.78 0.87 0.70 K3 4.16 0.71 K4 4.16 0.79 M1 4.16 1.03 4.00 0.93 0.83 0.89 0.66 M2 3.73 0.85 M3 3.91 0.95 M6 4.22 0.88 P1 3.89 0.90 3.90 0.84 0.83 0.90 0.74 P2 3.98 0.85 P3 3.84 0.79 U1 4.04 0.86 3.95 0.93 0.76 0.89 0.59 U2 3.87 1.02 U3 3.73 1.03 Measurement of validity Australasian Journal of Educational Technology, 2021, 37(4). 152 This metric indicates whether a questionnaire can measure the investigation's object and to what extent the scale function is valid. For this study, convergent type validity was used. It refers to how the elements that make up a scale behave. For this purpose, the average variance extracted (AVE) was calculated, and metrics higher than 0.5 were obtained (Table 2). This indicates a good fit (Fornell & Larcker, 1981). Evaluation of the adequacy of the questions and determination of the items The item analysis used, included descriptive statistics (mean and standard deviation [SD]). The mean of the individual items indicated that all the questions on the scale were adequate since their deviation was within a range of ± 1.5 SD of the scale's general mean (Table 2). Structural equation model (SEM) We performed a descriptive analysis of participants´ sociodemographic data, taking into account that in the context of digital gamification, age, gender and previous experience with video games could eventually, affect gameplay and preference (Filippou et al., 2018; Koivisto & Hamari, 2014). It was found that the majority of the respondents (80.3%) were aged between 18 and 25. In addition, they did not have significant differences in terms of experience with video games, so they are considered homogeneous data. In this sense, sociodemographic factors were not considered for the final model, eliminating unnecessary complexity. We then performed CFA. CFA was viable in this case since the data presented a KMO index = 0.951 (an excellent sample adequacy) and the Barlett sphericity test confirmed that it was feasible to generate the factorial analysis. The CFA indicated that the first 7 factors, equivalent to the factors used in the model, explained 77.94% of the total variance of the data. Continuing with the SEM, there were a total of 7 constructs, 2 being exogenous variables (ease of use and motivation) and 5 endogenous variables (use preference, engagement, enjoyment, knowledge, and perceived utility), explained by 26 observed variables. Furthermore, the proposed model was overestimated (gl > 0), since the number of parameters to estimate (63) was less than the amount of element of the variance-covariance matrix (351). However, when estimating the parameters of the model using the MLE, it was found that none of the goodness-of-fit indicators satisfied the goals proposed in the literature, and for this reason it was necessary to re-specify of the conceptual model. It should be noted that, in this case, estimation of parameters with maximum likelihood was used because the communalities of the factorial analysis were greater than 0.6, which indicated the relevance of performing the bootstrap analysis with a re-sampling of 500 replications and 90% confidence intervals (Escobedo Portillo et al., 2016). For the re-specification of the model, two paths were followed. The first path was to eliminate the relationships between statistically insignificant factors (p-value > 0.01), resulting in H2: KNO → PRF, H9: ENG← EoU, H10: KNO ← EoU, and H11: ENJ←EoU being discarded. The second path corresponded to establishing new relationships between variables following the recommendations generated by the IBM SPSS AMOS software in the "Modification Indices" option, which sought to reduce the chi-square index, while always being properly supported theoretically. In this sense, considering the theory of creativity, which establishes that a state of mind is characterised by a focus of activities that cause great pleasure and intense intrinsic motivation (Csikszentmihalyi, 1990), we affirmed that when a system is easy to use, the user finds an intrinsic motivation that leads him to continue using it. Likewise, this type of motivation drives behaviours that are triggered by achievement, pleasure, satisfaction, or other positive feelings (Silva et al., 2019). Additionally, this theory also determines that the total immersion of the user in a specific activity (engagement), occurs thanks to high satisfaction, enjoyment and other factors related to the activity. Therefore, the following additional hypotheses are proposed: H12: Motivation positively affects ease of use. H13: Motivation positively affects enjoyment. H14: Engagement positively affects enjoyment. Australasian Journal of Educational Technology, 2021, 37(4). 153 After re-structuring the conceptual model, its parameters were estimated using the maximum likelihood (MLE) technique, and the final model was obtained (Figure 3). Figure 3. Re-specified model The model results were improved. Only two elements were not statistically significant (p-value > 0.01); the relationships between motivation → enjoyment, and engagement → enjoyment (Table 3). Table 3 Standardised model estimates Parameters Standardised estimation SE CR p-value MOT→KNO 0.805 0.071 11.785 0.000 MOT→ENG 1.02 0.063 15.893 0.000 EoU→USE 0.17 0.045 3.622 0.000 MOT→ENJ 0.429 0.359 1.128 0.259 USE→KNO 0.611 0.07 7.779 0.000 ENG→USE 0.303 0.071 4.019 0.000 USE→PRF 0.509 0.106 5.070 0.000 ENJ→PRF 0.459 0.102 4.663 0.000 ENG→ENJ 0.575 0.364 1.522 0.128 MOT→EoU 0.493 0.066 5.168 0.000 Furthermore, when evaluating the goodness-of-fit of the re-specified model, it is worth noticing that most indicators satisfied the goals established in theory (Table 4). When evaluating the model fit, we considered three parameters: (1) absolute appropriate measures that are responsible for determining the degree to which the general model predicts the correlation matrix (Escobedo Portillo et al., 2016), (2) measures of Australasian Journal of Educational Technology, 2021, 37(4). 154 incremental adjustment that compare the proposed model with an existing one where the correlations between the variables involved are null (Hu & Bentler, 1999), and (3) parsimony adjustment measures that relate the model fit quality with the number of coefficients necessary to achieve the level of adjustment (Rourke & Hatcher, 2013). Table 4 Goodness-of-fit model Indicators Conceptual model Re-specified model Goal Absolute adjustment measures Chi-square 933.443 520.443 Least possible Chi-square/gl 3.2411 1.964 < 3 Parsimony adjustment measures PRATIO 0.886 0.883 Near 1 PCFI 0.748 0.829 Near 1 PNFI 0.701 0.779 Near 1 AIC 1059.443 640.443 Least possible From these findings, we evaluated the hypotheses proposed, rejecting those that had insignificant factor loadings or those that were eliminated in the re-specified model (Table 5). Table 5 Hypothesis summary Hypothesis Result H1 The perceived utility positively affects the preference for use. Do not reject H2 Knowledge positively affects preference for use. Reject H3 Knowledge positively affects utility. Do not reject H4 Engagement positively affects profit. Do not reject H5 The enjoyment positively affects the preference for use Do not reject H6 Motivation positively affects engagement. Do not reject H7 Motivation positively affects knowledge. Do not reject H8 Ease of use positively affects utility. Do not reject H9 Ease of use positively affects engagement. Reject H10 Ease of use positively affects knowledge. Reject H11 Ease of use positively affects the enjoyment. Reject H12 Motivation and ease of use are correlated factors. Do not reject H13 Motivation positively affects enjoyment. Reject H14 Engagement positively affects enjoyment. Reject In , the motivation generated had a highly significant positive effect on engagement and knowledge. Furthermore, this factor was positively correlated with the ease of use of the system, as found by Su and Cheng (2015), who concluded that engagement occurs when people feel motivated by interacting with the mechanics of the gamified tool. When this occurs, and the tool is easy to use, there is a perfect scenario for learning. Likewise, when engagement occurs, students are trapped in the tool's narrative, increasing motivation towards learning (Filippou et al., 2018). Similarly, the utility perceived by users and the enjoyment generated by Didactic City had a highly significant positive effect on user preference. This was expected since these factors have high predictive values to explain the intention to use a technological system (Van Der Heijden, 2004). On the one hand, when students perceive that the gamified tool is useful because it tangibly helps them to meet their academic goals, there is a greater probability that they prefer to use it (Filippou et al., 2018). In this sense, any implementation of gamification in education needs to make sure that it directly supports students' learning aspects. On the other hand, enjoyment is an essential source of value for digital-tools users, as it directly influences the behaviour of continuing to use it (Yang et al., 2017). Likewise, the improvement of knowledge and the engagement has a positive and significant impact on utility. This happens since the usefulness of learning tools is measured in terms of improving knowledge; therefore, if students believe that the gamified tool helps them in their study processes, it allows them to learn things differently, and Australasian Journal of Educational Technology, 2021, 37(4). 155 they consider it useful. Likewise, when the tool generates engagement, the student considers it useful because it helps them to turn something tedious into something delightful (Davis et al., 2018). However, the ease of use of the Didactic City had a low effect on utility, and in this sense, results differed from those found in other studies (Davis et al., 2018; Van Der Heijden, 2004). However this could be explained because ease of use has the potential to influence people's behaviour towards a new technological system, however, this effect only lasts for a short period of time (Yang et al., 2017). Additionally, each factor's indirect impact on the preference for use was measured by multiplying the statistically significant coefficients of their respective paths (Table 6). These paths indicated that the perceived utility, the generated enjoyment, and the improvement of knowledge, in that order of importance, were critical factors for students' preference for using the gamified tool Didactic City in the classroom. For this reason, it is recommended that people who design and develop tools or gamified experiences place the focus on reinforcing these factors by showing users the usefulness of the tool, seeking to generate pleasure for students with their interaction with it, and facilitating the acquisition of knowledge. Therefore, in this case the utility has a significantly higher correlation with ease of use, which, is a strong determinant in the acceptance of a digital tool. Furthermore, it was confirmed that enjoyment is an essential source of value for students and makes it easier for them to get involved in their learning process (Yang et al., 2017). Table 6 Impact of each factor on use preference Factor Multiplication of coefficients Impact Use Preference Perceived utility 0.51 0.51 Enjoyment 0.46 0.46 Knowledge (0.61 * 0.51) 0.3111 Motivation (1.02 * 0.30 * 0.51) 0.15606 Engagement (0.30 * 0.51) 0.153 Easy to use (0.17 * 0.51) 0.0867 Conclusions According to our results, instead of traditional methods of learning, students prefer gamification in their virtual learning environments. Gamification generates comprehensive, immersive environments that facilitate the acquisition of knowledge and increase their motivation. Therefore, it is suggested that teachers and educational institutions become more immerse on the use of gamified digital strategies in their classrooms. In the gamified tool Didactic City, students' preference to use it is directly affected by the perception about the utility, the enjoyment generated by the tool, and the opportunity to acquire more knowledge thanks to it. However, gamification's enjoyment and usefulness decrease with its use (Koivisto & Hamari, 2014). Therefore, it is suggested that new elements be incorporated throughout the different levels of gamified tools so that users can experience multiple innovations throughout their experience. Regarding the model of structural equations, there is a robust relationship between motivation, knowledge, and engagement. This phenomenon occurs because motivation, being one of the main gamification elements, makes things more fun. Incorporating it into educational environments makes it easier for students to acquire knowledge, encourage them to become more involved in their academic processes. However, when analysing the results of each particular factor, deficiencies were observed in two dimensions. In engagement, some activities of the tool did not encourage continued use and in ease of use. The recommendation is for educational gamified tool design teams to focus more specifically on these constructs. It is worth highlighting the importance of establishing valid measurements to predict the acceptance of digital pedagogical strategies based on gamification and users' behaviour in the face of these technological systems, since it is necessary to know if students prefer gamified experiences over classical learning instruments. Additionally, it is important to know which variables affect their choice before evaluating the Australasian Journal of Educational Technology, 2021, 37(4). 156 impact of these strategies and the aspects that can be improved within the classroom, since use-preference is a decisive factor for technological tools. For future work, it is recommended to follow a line of research with a broader spectrum. We recommend using a larger sample with different sociodemographic characteristics to obtain more robust results. This recommendation is because in some cases, age directly affects most of the benefits of gamified strategies, mainly the ease of use of this type of technological tools. Among the limitations of this research is its exploratory nature, with confirmation bias, since the proposed model presents an acceptable fit for the collected data, but it has only been confirmed that it is one of the possible models that explain the preference for use. Additionally, Didactic City was tested for a limited time of 4 weeks, which does not allow for long-term analysis of the tool. Therefore, studies can be generated that seek to identify the impact of time of use of the resource on learning flow and the preference for such gamified systems. Acknowledgements Special thanks to the Universidad Industrial de Santander for the financial support received that allowed the materialisation of this project and to the General System of Royalties fund from the Antioquia Government Science, Technology and Innovation Department, administered through the Francisco José de Caldas Autonomous Heritage National Fund Financing for Science, Technology and Innovation – MINCIENCIAS. References Acosta-Medina, J. K., Torres-Barreto, M. L., & Alvarez-Melgarejo, M. (2020). Literature mapping about gamification in the teaching and learning processes. Revista ESPACIOS, 41(11), 26-39. https://www.revistaespacios.com/a20v41n11/a20v41n11p26.pdf Acosta-Medina, J. K., Torres-Barreto, M. L., Álvarez-Melgarejo, M., & Paba-Medina, M. C. (2020). Gamificación en el ámbito educativo: Un análisis bibliométrico. I+D Revista de Investigaciones, 15(1), 28–36. https://doi.org/10.33304/revinv.v15n1-2020003 Bagozzi, R. P., & Yi, Y. (2012). Specification, evaluation, and interpretation of structural equation models. Journal of the Academy of Marketing Science, 40(1), 8–34. https://doi.org/10.1007/s11747- 011-0278-x Beck, J. C., & Wade, M. (2004). Got game: How the gamer generation is reshaping business forever. Harvard Business School Press. Bourgonjon, J., Valcke, M., Soetaert, R., & Schellens, T. (2010). Students’ perceptions about the use of video games in the classroom. Computers & Education, 54(4), 1145–1156. https://doi.org/10.1016/j.compedu.2009.10.022 Chen, C. C., Huang, C. C., Gribbins, M., & Swan, K. (2018). Gamify online courses with tools built into your learning management system (Lms) to enhance self-determined and active learning. Online Learning Journal, 22(3), 41–54. https://doi.org/10.24059/olj.v22i3.1466 Cocea, M., & Weibelzahl, S. (2011). Disengagement detection in online learning: Validation studies and perspectives. IEEE Transactions on Learning Technologies, 4(2), 114–124. https://doi.org/10.1109/TLT.2010.14 Csikszentmihályi, M. (1990). The domain of creativity. In M. A. Runco, & R. S. Albert (Eds), Theories of creativity (Vol. 115, p. 190–212). Sage Publications Inc. Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly: Management Information Systems, 13(3), 319–339. https://doi.org/10.2307/249008 Davis, K., Sridharan, H., Koepke, L., Singh, S., & Boiko, R. (2018). Learning and engagement in a gamified course: Investigating the effects of student characteristics. Journal of Computer Assisted Learning, 34(5), 492–503. https://doi.org/10.1111/jcal.12254 Deci, E. L., Vallerand, R. J., Pelletier, L. G., & Ryan, R. M. (1991). Motivation and education: The self determination perspective. Educational Psychologist, 26(3–4), 325–346. https://doi.org/10.1080/00461520.1991.9653137 DeVellis, R. F. (1991). Scale development: Theory and applications. Applied social research methods series (Vol. 26). Sage Publications Inc. https://www.revistaespacios.com/a20v41n11/a20v41n11p26.pdf https://doi.org/10.33304/revinv.v15n1-2020003 https://doi.org/10.1007/s11747-011-0278-x https://doi.org/10.1007/s11747-011-0278-x https://doi.org/10.1016/j.compedu.2009.10.022 https://doi.org/10.24059/olj.v22i3.1466 https://doi.org/10.1109/TLT.2010.14 https://doi.org/10.2307/249008 https://doi.org/10.1111/jcal.12254 https://doi.org/10.1080/00461520.1991.9653137 Australasian Journal of Educational Technology, 2021, 37(4). 157 Đurđević Babić, I. (2017). Machine learning methods in predicting the student academic motivation. Croatian Operational Research Review, 8(2), 443–461. https://doi.org/10.17535/crorr.2017.0028 Escobedo Portillo, M. T., Hernández Gómez, J. A., Estebané Ortega, V., & Martínez Moreno, G. (2016). Modelos de ecuaciones estructurales: Características, fases, construcción, aplicación y resultados. Ciencia & Trabajo, 18(55), 16–22. https://doi.org/10.4067/S0718-24492016000100004 Fernández Morales, K., McAnally Salas, L., & Vallejo Casarín, A. (2015). Apropiación tecnológica: Una visión desde los modelos y las teorías que la explican. Perspectiva Educacional, 54(2), 109–125. https://doi.org/10.4151/07189729-Vol.54-Iss.2-Art.331 Filippou, J., Cheong, C., & Cheong, F. (2018). A model to investigate preference for use of gamification in a learning activity. Australasian Journal of Information Systems, 22, 1–23. https://doi.org/10.3127/ajis.v22i0.1397 Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39. https://doi.org/10.2307/3151312 Fotaris, P., Mastoras, T., Leinfellner, R., & Rosunally, Y. (2016). Climbing up the leaderboard: An empirical study of applying gamification techniques to a computer programming class. The Electronic Journal of E-Learning, 14(2), 94–110. Fu, F. L., Su, R. C., & Yu, S. C. (2009). EGameFlow: A scale to measure learners’ enjoyment of e- learning games. Computers and Education, 52(1), 101–112. https://doi.org/10.1016/j.compedu.2008.07.004 González Castro, Y., Manzano Durán, O., & Torres Zamudio, M. (2017). Riesgos de deserción en las universidades virtuales de Colombia, frente a las estrategias de retención. Libre Empresa, 14(2), 177– 197. https://doi.org/10.18041/1657-2815/libreempresa.2017v14n2.3038 Hair, J. F., Black, W. C., & Babin, B. J. (2010). Multivariate data analysis: A global perspective. Pearson Prentice Hall. Howe, N., & Strauss, W. (2000). Millennials rising: The next great generation. Vintage Books. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. Kocadere, S. A., & Çağlar, S. (2018). Gamification from player type perspective: A case study. Educational Technology and Society, 21(3), 12–22. https://www.jstor.org/stable/26458503 Koivisto, J., & Hamari, J. (2014). Demographic differences in perceived benefits from gamification. Computers in Human Behavior, 35, 179–188. https://doi.org/10.1016/j.chb.2014.03.007 Kularbphettong, K., & Tongsiri, C. (2012). Mining educational data to analyze the student motivation behavior. World Academy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering, 6(8), 102-1036. https://doi.org/10.5281/zenodo.1079926 Lobo-Rueba, M. Á., Paba-Medina, M. C., & Torres-Barreto, M. L. (2020). Análisis descriptivo de experiencias gamificadas para enseñanza y aprendizaje en educación superior en ingeniería. Revista ESPACIOS, 41(16), 21. Melo-Solarte, D. S., & Díaz, P. A. (2018). El aprendizaje afectivo y la gamificación en escenarios de educación virtual. Información Tecnológica, 29(3), 237–248. https://doi.org/10.4067/S0718- 07642018000300237 Ministerio de Educación Nacional de Colombia. (2017). Reporte sobre deserción y graduación en educación superior. Imprenta Nacional de Colombia. Oblinger, D., Oblinger, J. L., & Lippincott, J. K. (2005). Educating the net generation. Educause. Ochoa Sierra, L., & Moya Pardo, C. (2018). La evaluación docente universitaria: Retos y posibilidades. Folios, 49, 41–60. https://doi.org/10.17227/folios.49-9390 Organisation for Economic Co-operation and Development (2013). Education today. OECD Publishing. https://doi.org/10.1787/edu_today-2013-en Paba-Medina, M. C., Acosta-Medina, J. K., & Torres-Barreto, M. L. (2020). Priorización De Competencias Ciudadanas En Un Contexto Gamificado. Panorama, 14(27), 51–72. https://doi.org/10.15765/pnrm.v14i27.1522 Prensky, M. (2001). Digital game-based learning. McGraw-Hill. Prensky, M. (2010). Teaching digital natives: Partnering for real learning. SAGE Publications Inc. Prensky, M. (2013). Enseñar a nativos digitales (1st ed.). SM Ediciones. Rourke, N. O., & Hatcher, L. (2013). A step-by-step approach to using SAS for factor analysis and structural equation modeling. SAS Institute. https://doi.org/10.17535/crorr.2017.0028 https://doi.org/10.4067/S0718-24492016000100004 https://doi.org/10.4151/07189729-Vol.54-Iss.2-Art.331 https://doi.org/10.3127/ajis.v22i0.1397 https://doi.org/10.2307/3151312 https://doi.org/10.1016/j.compedu.2008.07.004 https://doi.org/10.18041/1657-2815/libreempresa.2017v14n2.3038 https://www.jstor.org/stable/26458503 https://doi.org/10.1016/j.chb.2014.03.007 https://doi.org/10.5281/zenodo.1079926 https://doi.org/10.4067/S0718-07642018000300237 https://doi.org/10.4067/S0718-07642018000300237 https://doi.org/10.17227/folios.49-9390 https://doi.org/10.1787/edu_today-2013-en https://doi.org/10.15765/pnrm.v14i27.1522 Australasian Journal of Educational Technology, 2021, 37(4). 158 Silva, R., Rodrigues, R., & Leal, C. (2019). Play it again: How game-based learning improves flow in accounting and marketing education. Accounting Education, 28(5), 484–507. https://doi.org/10.1080/09639284.2019.1647859 Su, C.-H. (2016). The effects of students’ motivation, cognitive load and learning anxiety in gamification software engineering education: A structural equation modeling study. Multimedia Tools and Applications, 75(16), 10013–10036. https://doi.org/10.1007/s11042-015-2799-7 Su, C. H., & Cheng, C. H. (2015). A mobile gamification learning system for improving the learning motivation and achievements. Journal of Computer Assisted Learning, 31(3), 268–286. https://doi.org/10.1111/jcal.12088 Toda, A. M., Palomino, P. T., Oliveira, W., Rodrigues, L., Klock, A. C. T., Gasparini, I., Cristea, A. I., & Isotani, S. (2019). How to gamify learning systems? An experience report using the design sprint method and a taxonomy for gamification elements in education. Educational Technology and Society, 22(3), 47–60. Van Der Heijden, H. (2004). User acceptance of hedonic information systems. MIS Quarterly, 28(4), 695–704. https://doi.org/10.2307/25148660 Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478. https://doi.org/10.2307/30036540 Wichadee, S., & Pattanapichet, F. (2018). Enhancement of performance and motivation through application of digital games in an English language class. Teaching English with Technology, 18(1), 77–92. Yang, Y., Asaad, Y., & Dwivedi, Y. (2017). Examining the impact of gamification on intention of engagement and brand attitude in the marketing context. Computers in Human Behavior, 73, 459–469. https://doi.org/10.1016/j.chb.2017.03.066 ____________________________________________________________________________________ Corresponding author: Julieth Katherin Acosta-Medina, katheacosta19@hotmail.com Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC- ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY- NC-ND 4.0. Please cite as: Acosta-Medina, J. K., Torres-Barreto, M. L., & Cárdenas-Parga, A. F. (2021). Students' preference for the use of gamification in virtual learning environments. Australasian Journal of Educational Technology, 37(4), 145-158. https://doi.org/10.14742/ajet.6512 https://doi.org/10.1080/09639284.2019.1647859 https://doi.org/10.1007/s11042-015-2799-7 https://doi.org/10.1111/jcal.12088 https://doi.org/10.2307/25148660 https://doi.org/10.2307/30036540 https://doi.org/10.1016/j.chb.2017.03.066 mailto:katheacosta19@hotmail.com https://creativecommons.org/licenses/by-nc-nd/4.0/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.6512 Introduction Theoretical framework Research methodology Results and analysis Conclusions Acknowledgements References