International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 08, 2021 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive Multimedia https://doi.org/10.3991/ijim.v15i08.19777 Mar’atus Sholikah (), Dwi Harsono Universitas Negeri Yogyakarta, Yogyakarta, Indonesia maratussholikah.2019@student.uny.ac.id Abstract—The closure of schools and universities nationwide and even al- most worldwide during the COVID-19 pandemic resulted in learning activities turning to distance learning. One of the increasing distance learning users is the mobile app. However, the problem educators experience when implementing learning by adopting m-learning applications is concerned about adequate stu- dent engagement or, in some cases, an inability to reach students fully. To that end, distance learning using m-learning has created questions about the im- portance of student engagement leading to understanding. Thus, this study in- vestigated the influence of m-learning adoption on student engagement through digital readiness and presented 16 items of instruments. Electronic question- naires collected data (N =89) from all masters and doctoral students in Indone- sia. Testing the instrument’s validity using confirmatory factor analysis, while reliability is measured using Cronbach’s alpha and composite reliability. The results are significant for digital readiness as a mediator in student perception of the influence of the adoption of m-learning innovation on student engagement. Although students positively perceive that the adoption of m-learning has a pos- itive effect, they must also have strong digital skills to complete their academic work. Besides, they must also be committed to fully engaged in learning activi- ties using m-learning— this finding provides practical implications for improv- ing effective and interactive online learning in college. Keywords—Students’ involvement, digital readiness, online learning, mobile learning, interactive multimedia 1 Introduction Covid-19 pandemic disrupts all sectors, including education. The implementation of lockdown and social distancing led to the suspension of face-to-face learning activ- ities. Therefore, the government provides online learning instruction rather than face- to-face learning. Face-to-face learning is considered ineffective in engaging students [1]. Thus, educational institutions turn to online learning to effectively deliver learn- ing materials [2], [3]. However, online learning is also an important issue because not all regions have adequate information and communication technology [4]. Besides, the teachers’ narrative notes reveal that student participation in online classes tends to be low due to a lack of parental supervision, internet access, resources, and teacher iJIM ‒ Vol. 15, No. 08, 2021 101 https://doi.org/10.3991/ijim.v15i08.19777 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… skills [5]. To do so, teachers must optimize digital tools and resources to create inter- active and affordable learning and teaching for students [6]. Thus, students will be encouraged to get involved in the learning process. Student involvement is a crucial component of effective learning because signifi- cant learning activities will improve student performance [7]. Previous research has found that based on survey results from 186 colleges, students engaged successfully when learning activities involve students with materials, educators, and other students [8]. The type of learning media may influence students’ perceptions of their engage- ment level [9]. One of the popular media that can increase student participation is mobile-based learning. This learning media has been implemented in Europe, using tablets and mobile devices to be the most popular learning media because it is effort- less to use [10] [11]. According to Kabali, mobile devices are an exciting learning medium because they are easy to use [12]. This device’s most important advantages are its screen, portability, and ease of use due to touch screen technology [13], [14]. Also, mobile learning is more mobile learning because students can use it anytime and anywhere [15]. Supported by the results of a survey from Common Sense Media in the United States shows that the ease of learning access using mobile devices resulted in this educational application began to be developed further as a teaching media to provide academic experiences for students [13], [16], [17]. To that end, the app is increasingly being adopted among developed and developing countries [14] by identifying the importance of developing educational applications that correspond to the current age and conditions. This educational application allows students to form their content [13]. The main factor influencing the adoption of digital technology devices as learn- ing tools in higher education is technology experience and readiness. Mobile technol- ogy, whether students believe it is easy to use, offers various essential advantages in education [18]. It means that students’ proficiency and readiness are indispensable in using mobile devices, as they will be aware of their course benefits. Previous research on technology and student engagement has found that digital technology utilization can encourage student engagement [19], [20]. In line with Bar- ak & Green, Henderson, Selwyn, & Aston, digital technology is becoming a central aspect of higher education, which inherently affects all aspects of the student experi- ence [21]–[23]. Therefore, using digital technology can conduct more incentive teach- ing-learning processes, improve self-management and student progress, increase par- ticipation and involvement in the learning process, and predict increased student en- gagement [24]–[27]. However, there is no guarantee that student involvement is act- ing as a result of technology use [28]. Supported by Tamim, Bernard, Borokhovski, Abrami, & Schmid found that the application of technology in education had only a tiny to moderate impact on student achievement [29]. Instead, good planning and pedagogy and the right tools are essential because technology can improve teaching development [30], [31]. However, significant technology cannot replace poor teach- ing. Furthermore, the interactivity and retention of learning using digital technology are usually lower than face-to-face learning [32]. Many students experience stress and frustration related to online education and difficulty completing tasks. Like Jaggars, 102 http://www.i-jim.org Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… the form of stress experienced in online learning is technical difficulties and internet networks with an online learning environment, thus reducing their commitment and learning engagement [33]. Coomey and Stephenson, in the Min Hu & Hao Li’s study, summarized 100 research reports and journal articles on online learning and conclud- ed that there are four characteristics of online learning, one of which is participation or engagement [34]. In face-to-face learning, educators can use several teaching methods and strategies during the learning process based on interactive learning. Learning will implement appropriately to ensure the effectiveness of education and the quality of learning. However, in an online learning environment, due to a lack of communication be- tween students and educators and some uncontrollable factors such as learning envi- ronments, information gaps, and learning time, educators cannot understand the level of student engagement. In comparison, active student involvement in online learning indicates that they can efficiently carry out online learning. Otherwise, no matter how good the learning material equals zero. Therefore, further research is needed to ana- lyse student involvement in online learning adaptation by paying attention to students’ digital readiness. Key identified issues include various educational challenges during the COVID-19 pandemic, how they use smart mobile devices as teaching media, and the influence of mobile learning on students and students’ digital readiness. This re- search can help educators understand student engagement, help students reflect on their behaviour, and increase their online learning process participation. After that, the research findings combined the proposed literature on adopting mobile learning with student involvement in online education mediated by student digital readiness. 2 Literature Review With positive support for online learning advancement at universities, students can effectively achieve successful online learning. This research aims to test the adoption of m-learning, digital readiness, and student involvement in universities’ online edu- cation contexts. Figure 1, in this section, illustrates this research model, which is the reason for the proposed hypothesis. This research suggests and tests the research model, consisting of three factors: m-learning, digital readiness, and student involve- ment. iJIM ‒ Vol. 15, No. 08, 2021 103 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… Fig. 1. The research model 2.1 Digital readiness and students’ involvement Digital readiness for students relates to knowledge, attitudes, and competencies to use digital technology to meet educational goals and higher education [35]. Student involvement in universities is likely to be enhanced by the adoption of digital tech- nology, as found by Salman & Abdul Aziz and Kim, Hong, & Song, that students’ digital readiness is related to the active application of technology for academic activi- ties [36], [37]. Digital readiness for students, according to Margaryan, Littlejohn, & Vojt, includes the use of digital, which defined as the skill to perform academic tasks, the development of digital media competencies through the participation and evalua- tion of digital culture, and the application of information literacy skills [38]. Kim, Hong, & Song research found that students in Korea are digital natives who may not be actively influential in digital technology for academic activities or associ- ate with digital literacy [37]. Students in universities currently show a gap between digital skills in informal contexts and formal learning [38]. Students’ digital readiness includes the use of digital-related skills in the academic field, the development of digital media capabilities through participation, and the application of information literacy skills and learning strategies. It can be one of the crucial links between m- learning experience and student engagement. So, this study developed the following hypothesis: Hypothesis 1 (H1): M-learning adoption is positively related to students’ involve- ment 2.2 M-learning, digital readiness, dan students’ involvement As mobile device technology has overgrown, many young people and adults be- come regular users. What is more, tablet and smartphone devices are now popular among socioeconomic classes [39]. Due to its affordability compared to other digital devices, it is increasingly being adopted in developed and developing countries [14]. 104 http://www.i-jim.org Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… Previous research has shown that the use of these mobile devices has penetrated the context of higher education [18]. Specifically, many researchers have traced the ef- fects of mobile device adoption on learning in higher education environments [40], [41]. Their findings suggest that mobile device learning apps improve critical thinking and student performance. Researchers focusing on higher education showed that mobile devices’ utilization in courses provides new potential for students to actively and effectively engage in the learning process [13]. Mobile device technology considers the most welcoming and interactive way for students to improve their conceptual and knowledge because they will learn to see the experience through various perspectives. The learning-based mobile device contributes to the educational process. This type of tablet technology tends to have small dimensions, portability and is suitable for students and very interesting and very effective in improving students’ accuracy in the learning process [13], [14]. It needs student digital readiness to achieve success in mobile device technology. Because without digital enthusiasm, students will find it challenging to understand the use of technology. Therefore, the hypotheses proposed in this study are: Hypothesis 2 (H2): M-learning adoption is positively related to digital readiness Hypothesis 3 (H3): Digital readiness is positively related to students’ involvement Hypothesis 4 (H4): The application of m-learning affects student engagement through digital readiness 3 Methodology 3.1 Data collection This study applied a quantitative method based on a survey. Data analysis on the relationship of variables in the research model employed the SmartPLS software to obtain the respondent’s results. The questionnaire consists of two main parts. The first section contains questions about demographic characteristics: gender, level of educa- tion, and regional origin. The second section includes closed-door questions about m- learning adoption, digital readiness, and student engagement. This study asks the experts related to the education field to review the instrument of the questionnaire’s contents. Based on feedback from experts, the item revises according to their advice. After the change, the next stage is testing the questionnaire on a sample of 30 students in Semarang. This testing aims to know for clarity and ease of use. The test results of this research instrument’s validity and reliability see in table 2. 3.2 Sample This study’s population consists of random students selected from 36 provinces in Indonesia studying at Yogyakarta State University. The study involved 179 students chosen, considering specific demographic variations such as gender, level of educa- iJIM ‒ Vol. 15, No. 08, 2021 105 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… tion, and regional origin. Of the initial samples, 20 declined, 70 refused to respond to the sent questionnaire, and 89 respond to the questionnaire. To that end, the final sample consisted of 89 students. Table 1. Data Respondent Attributes Classification Percent Gender Male 41.57 Female 58.43 Education S2 87.64 S3 12.36 Province Bengkulu 2.25 Special Region of Yogyakarta 24.72 Special Capital Region of Jakarta 3.37 West Java 5.62 Central Java 10.11 East Java 7.87 West Kalimantan 4.49 South Kalimantan 6.74 Central Kalimantan 4.49 North Kalimantan 5.62 East Nusa Tenggara 3.37 Papua 1.12 Riau 3.37 North Sulawesi 4.49 Central Sulawesi 8.99 West Sumatera 3.37 Surveys are conducted through self-managed online questionnaires using google form. Respondents voluntarily participated in the study by sending a message via WhatsApp to click on the link address. We used a closed questionnaire because it only presents questions and a choice of answers. The selection of solutions in this study uses a Likert scale (1: strongly disagree-5: strongly agree). In the questionnaire, the study also raised questions about the frequency of mobile device use in online learning implementation using the Likert scale (1: almost never-5: always). The re- sults showed that the frequency of mobile phone use for online learning activities was very high, with a median of 5. 3.3 Measurement The questionnaire was adapted from previous research developed by researchers to test mobile device-based learning adaptations, digital readiness, and their involvement in the survey. Student engagement was measured using a scale developed by Han- delsman, Briggs, Sullivan, and Towler [42]. The realities of student engagement demonstrated by Cronbach’s alpha score of 0.841 are the instrument’s reliability in a high category. 106 http://www.i-jim.org Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… M-learning instruments are measured through students’ evasion of their resources and abilities when engaging in m-learning [43]. Individual evaluation of skills and resources is an antecedent to adopting the them-learning component. Behavioral con- trol is a positive predictor of the intention to adopt m-learning. Three items adapted from Chu and Chen included the statements: “I have the necessary knowledge to use the university’s m-learning system,” “Using the them-learning system is entirely with- in my control,” and “I have the resources necessary to use m-learning applications.” The scale showed strong reliability in this study: Cronbach’s alpha equals 0.920. The digital readiness instrument took from Hong and Kim, which measures univer- sities’ digital competencies that students feel for academic involvement [35]. Digital readiness is considered necessary for student academic success. Such instruments’ reliability scale shows very strongly: Cronbach’s alpha equals 0.841. Table 2 shows the research instrument. Table 2. Measuring independent and dependent variables Variables Items Factor AVE CR α Student involvement (SI) SI1 I always study regularly 0.737 0.545 0.855 0.787 SI2 I always find ways to make learning interesting to me 0.750 SI3 I always l learning material when think- ing about the course between class meetings 0.854 SI4 I want to learn the material 0.740 SI5 Using m-learning is excellent fun 0.785 M-learning adoption (MA) MA1 Facilitate the implementation of the learning process 0.762 0.675 0.935 0.920 MA2 The quality of the learning process is improving 0.803 MA3 The learning process is more effective 0.834 MA4 Increase productivity in learning 0.843 MA5 It can be used in all areas of learning 0.859 MA6 Following the learning methods used 0.841 MA7 Following the style in learning activities 0.804 Digital readiness (DI) DI1 I can generate keywords to search for information for academic work. 0.845 0.681 0.894 0.841 DI2 I can communicate with classmates using real-time communication tools (e.g., video conferencing tools or messengers). 0.868 DI3 I can share everything, like my files with classmates, using online software. 0.870 DI4 I can collab with classmates using online software. 0.706 3.4 Data analysis Partial Least Squares-Structural Equation Modeling (PLS-SEM) was adopted to test the research model by empirically assessing structural models along with meas- urement models [44]. The study used SmartPLS 3.0 software [45] and utilized two iJIM ‒ Vol. 15, No. 08, 2021 107 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… evaluation approaches: measurement models and structural models [46]. PLS-SEM is used to test research models, and hypotheses using latent variables with several varia- bles observed using regression-based methods [47]. Also, PLS-SEM is more explora- tory by understanding the coefficients of absolute paths and dependent variances de- scribed by independent variables in the research model, rather than checking match goodness [48]. PLS-SEM is a more practical approach to developing theories with limited context. Assumption of multivariate normality, smaller sample size, and measurement scale compared to covariant modelling-based structure equations. For the model conformity index, the study adopted the Chin criteria [49]. For PLS-SEM, the data size must be at least ten times the number of constructs associated with a single endogenous depend- ent construction. There are three constructs in this study, then the minimum amount of data to apply PLS-SEM is 30 (10 x 3 constructs). So, the total size is 89 exceeds the recommended sample size. The fit model was measured by using the SRMR standard. The research model showed a value of 0.07 less than 0.08 [50]. The value indicates that the research mod- el corresponds to the data. In other words, the model conformity index indicates 0.07, which means that the value of a good fit model [47]. 4 Results Of the 150 questionnaires distributed, 89 questionnaires were returned and were el- igible for analysis. The overall respondent’s answer results almost all answered agreed from each question. It is indicated by the result each variable influences the other. Furthermore, this study’s data analysis uses the outer and structural models in SmartPLS. 4.1 Measurement model assessment The goodness of fit test is the first step to analysing measurement models using SmartPLS [51]. This test can be measured by looking at the Standardized Root Mean Square Residual index (SRMR). Judging by the test results of fit, estimated, and satu- rated models must meet the criteria of SRMR values below 0.08 to be acceptable [50]. In this study, the SRMR value was 0.077, meaning the model was appropriate. Alt- hough the use of the goodness of fit test in PLS-SEM is still not widely used in re- search, the assessment’s results are provided informatively in the test report [52]. In the next step, the measurement model assessment emphasizes the validity and relay test. Reliability testing is indicated by Cronbach’s’ alpha and composite reliabil- ity values. At the same time, validity tests can be measured using loading factor and average variance extracted (AVE) typically used to assess convergent validity. The recommended loading factor and Cronbach’s’ alpha value is ≥0.70 [53], and the CR and AVE values should be higher than 0.60 and 0.50 [44]. If an item’s value is less than that criteria, the model removed it from data analysis. 108 http://www.i-jim.org Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… Table 3. Reliability and analysis of convergent validity Variables Items Loading α CR AVE Student involvement (SI) S1 0.807 0.860 0.900 0.643 SI2 0.813 SI3 0.835 SI4 0.845 SI5 0.700 M-learning adoption (MA) MA1 0.746 0.913 0.931 0.659 MA2 0.805 MA3 0.839 MA4 0.856 MA5 0.819 MA6 0.845 MA7 0.761 Digital readiness (DI) DI1 0.875 0.856 0.903 0.699 DI2 0.852 DI3 0.799 DI4 0.818 Table 4. Discrimination Validity Variables Fornell-Larcker HTMT DI MA SE DI MA SE DI 0.836 MA 0.745 0.812 0.838 SE 0.772 0.829 0.802 0.896 0.836 Table 3 confirms that the loading factor value is more than 0.7, and the CR and AVE values also show higher than 0.60 and 0.50 [44], [53]. Meanwhile, table 4 shows that the test results of the discrimination validity using the heterotraite-mononitrate ratio (HTMT) and Fornell-Larcker, prove that the value obtained is less than 0.90, meaning the HTMT and Fornell-Larcker values are accepted. 4.2 Structural model assessment The next step, structural model analysis, aims to determine the hypothesis’s signif- icance. Table 5 will explain the R-square and R-square adjusted values. Table 5. R-square value result Variables R-square R-square adjusted Digital readiness 0.555 0.550 Student involvement 0.741 0.735 Table 5 proves that the digital readiness of students is 0.555. The variance value of student digital readiness is explained by the mobile learning overspan of 55.5%, while other variables are not part of the model. Additionally, the student engagement varia- iJIM ‒ Vol. 15, No. 08, 2021 109 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… ble’s R-square value indicates the number 0.741 or 74.1 influenced by the variables studied in this research model. This study used a path coefficient to test the hypothe- sis, as shown in table 6. The bootstrap analysis results prove that this study supports the significance of the relationship that has been hypothesized. Table 6. Summary of hypothetical test results Hypotheses β t-value p-value Decisions H1: MA → SI 0.571 6.728 0.000 Supported H2: DI→SI 0.347 3.785 0.000 Supported H3: MA → DI 0.745 15.919 0.000 Supported H4: MA→DI→ SI 0.258 3.573 0.000 Supported Hypothetical test results are listed in table 6, the results show that MA → SI (t=6,728; β=0.571); DI → SI (t = 3,785; β = 0.347); and MA→DI (t= 15,919; β = 0.745) had significant and positive correlation, so H1, H2, and H3 were supported in this study. Table 5 also shows that DI has positive partial mediation in the MAIN → SI path. Hence the hypothetical test results of mediation effects show (t= 3,573; β = 0.258) are significant and supported in this study. Fig. 2. PLS Model Of M-Learning Adoption of Students’ Involvement through Digital Readiness Figure 2 shows the path coefficient and significance level for each hypothesis, in- dicating that the hypothesis is supported at p < 0.05. The results prove that the adop- tion of m-learning (β=0.571, p < 0.01, supporting H1) has a significant favorable influence on students’ involvement. Adopting m-learning (β=0.347, p<0.01, support- ing H2) substantially influences digital readiness. Besides, digital readiness (β=0.745, p<0.01, keeping H3) positively affects students’ involvement. To test how students’ digital ability mediates m-learning adoption on students’ participation represented in table 6 and 2. Table 6 and figure 2 show that digital readiness (β=0.258, p<0.01, sup- porting H4) can mediate the influence of m-learning adoption on students’ involve- ment. 110 http://www.i-jim.org Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… 5 Discussion In this study, we tested our research model to reveal the relationship between m- learning adoption and students’ involvement. The finding indicates that the influence of m-learning adoption encourages student engagement. This research also examines the role of digital mediation readiness on m-learning adoption and student involve- ment in college settings. This study tries to contribute to increased student involvement by using m- learning. The findings suggest that the first hypothesis (H1), stating that m-learning adoption affects students’ participation, was supported in this study. It means that the better the use of m-learning adoption, the more student participation will also in- crease, and vice versa. The lowest item value in the them-learning adoption instru- ment is MA1: Facilitate in m-learning process’s implementation based on the analysis results. It proves that the ease of using m-learning is still not satisfying students. Therefore, the lecturer or educator should make guidelines or procedures of m- learning usage to be more detailed and precise to understand m-learning easily. This study’s conclusions are essential to know that the adoption of innovation m- learning can influence student involvement in online learning. Previous student partic- ipation in m-learning in higher education significantly affects their performance levels [54]. In common with Kuh, students’ efforts are intended for educational activities that can contribute to academic results [55]. The high level of relationships is signifi- cant due to students’ commitment or action to engage in defense activities, resulting in better academic performance [56]. In other words, m-learning and student engage- ment directly predict their achievements in educational activities. In the study results, m-learning can increase productivity in learning is the most significant factor that is 0.859. While m-learning facilitates the learning process’s implementation is the lowest factor, which is 0.746. Thus, the increasing productivity in learning contributes the most in explaining the advantages of m-learning. This research is consistent with Bennett & Bennett, showing that the main obstacles teach- ers face in using technology are not on the limitations of technological means and funds, but rather on the willingness to use technology and confidence in technology benefits [57]. The belief in technology benefits can be seen from the advantages of technology compared to conventional methods. Added by Carter & Bélanger, tech- nology influences students’ increasing motivation in learning [58]. The second hypothesis (H2) reads digital readiness affects students’ involvement, and the third hypothesis (H3) expressed with influential digital enthusiasm faced with the adoption of m-learning proved supported in this study. These findings are con- sistent with previous studies’ results, which to guide students towards productivity and better results in using m-learning, it is necessary to improve their readiness [59]. One of the essential objectives of higher education using m-learning as interactive multimedia is to make students more interactive in the learning process through stu- dents’ involvement [60]–[62]. To that effect, the adoption of m-learning plays a sig- nificant role. It means that students’ experience and readiness in adopting m-learning can contribute to student engagement. Besides, students are confident in their digital iJIM ‒ Vol. 15, No. 08, 2021 111 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… abilities for their academic work, meaning they have digital readiness, making them more likely to achieve better academic achievement. The student involvement instrument shows that the lowest item value is SI5: Using m-learning is tremendous fun. To do so, educators need to improve them-learning app to become a more enjoyable learning app. One way is to add the latest learning fea- tures such as podcasts and assignments through social media such as TikTok, Insta- gram, etc. Meanwhile, the lowest item value on digital readiness instruments lies in DI3: I can share everything, like my files with classmates, using online software. This way, educators can create additional data-sharing features for students so they can access all learning materials through m-learning. The fourth hypothesis is that m-learning innovation’s adoption affects student en- gagement through digital readiness, proven supported in this study. This study’s find- ings imply that students who actively adopt m-learning and have confidence still need to commit and strive to study the use of digital materials [37], [54]. Also, given that digital readiness mediates the relationship between the partial use of m-learning and student engagement, universities need to consider students’ digital competencies with increasing student academic engagement and success [63], [64]. Therefore, universi- ties should focus on supporting and ensuring students have an enriched experience using m-learning for their learning. When students are more involved in learning, they will benefit more from their learning activities. Conversely, when students are less engaged in learning, they will find it challenging to engage in learning and gain a little advantage in learning activi- ties. In learning using m-learning, whether students can participate in the learning process is the most critical factor to ensure learning effectiveness. Therefore, the es- sence of m-learning adoption is the continuous development of the student’s cognitive level and to achieve practical and interactive learning. Students need to participate in learning actively [34]. This study’s practical implications advise students to improve digital competencies to deepen their academic experience. Besides, universities need to provide training, direction, and support to students by regularly evaluating m-learning experiences and adoption rates for their involvement. Also, universities must recognize the need for technology integration in student learning and strive to integrate m-learning into the curriculum. In particular, integrated blended learning with learning needs proves to be a practical and interactive learning approach to improving learning outcomes, student achievement, student engagement, and academic satisfaction in higher education [65]–[68]. Theoretically, this research also provides additional insight into what fac- tors influence student involvement. In addition to learning media, it also affects stu- dents’ digital readiness. Therefore, this research contributes to adding theoretical studies in education, especially in selecting teaching media suitable for the current pandemic situation. 112 http://www.i-jim.org Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… 6 Conclusion The impact of school and college closures in the COVID-19 pandemic situation is the switch of all learning from face-to-face to distance learning. Many problems still found in distance learning implementation are students’ low involvement in the learn- ing process. Therefore, the adoption of m-learning becomes one of the latest learning media alternatives and by student conditions, where portability and mobility take precedence in the present. In the implementation of m-learning, the critical factor influencing the learning process’s success is students’ readiness and involvement in using digital technology, namely m-learning. Thus, this study examines the influence of m-learning adoption on student engagement by mediating digital readiness varia- bles. This study has four hypotheses, and all premises are consistently supporting in this study. Most students fully feel that m-learning is more interactive and flexible in fulfilling their learning materials. Because interactive multimedia-based m-learning offers a lot of material interaction with students, learning performance and student satisfaction can also improve. This finding can be a reference for improvements from the same application or design of similar interactive learning applications in higher education and non-educational contexts. The implementation of m-learning requires a technolo- gy, materials, and learning environment that can attract attention, impressive appear- ance, and have an inspiring appeal. M-learning technology can improve student cog- nition and support student cognition during covid-19. This research proves that creat- ing practical and interactive learning in the covid-19 pandemic can be done by utiliz- ing technology such as m-learning. The limitations of this finding are that the samples are still limited. However, the models come from all provinces in Indonesia. They are all enrolled in one Universitas Negeri Yogyakarta, one of the universities that implement digital-based learning and teaching. Besides, online polls are also a limitation in this study because polls’ filling relies heavily on respondents’ integrity and honesty. As a result, researchers cannot control the integrity of its contents. Meanwhile, this study also has a limitation of only investigating three constructs: m-learning adoption, digital readiness, and students’ involvement. Furthermore, it is vital to test additional antecedents in this study. For example, aspects of social inter- action should be more considered. Because technology can increase social interaction, social interaction is ultimately more important to students than technology. In turn, social interaction affects students’ ability to engage in learning [69-70]. The research model can also be expanded to antecedents to predict digital readiness, student en- gagement, and m-learning adoption. For example, they add ancestors to parental sup- port, student background, and technology adoption experiences. iJIM ‒ Vol. 15, No. 08, 2021 113 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… 7 References [1] G. M. Slavich and P. G. Zimbardo, "Transformational Teaching: Theoretical Underpin- nings, Basic Principles, and Core Methods," Educ. Psychol. Rev., vol. 24, no. 4, pp. 569- 608, 2012. https://doi.org/10.1007/s10648-012-9199-6 [2] W. Ali, "Online and Remote Learning in Higher Education Institutes: A Necessity in light of COVID-19 Pandemic," High. Educ. Stud., vol. 10, no. 3, p. 16, 2020. https://doi.org/ 10.5539/hes.v10n3p16 [3] S. J. Daniel, "Education and the COVID-19 pandemic," Prospects, vol. 49, no. 1-2, pp. 91- 96, 2020. https://doi.org/10.1007/s11125-020-09464-3 [4] H. D. Hermawan, N. Deswila, and D. N. Yunita, "Implementation of ICT in Education in Indonesia during 2004-2017," Proc. - 2018 Int. Symp. Educ. Technol. ISET 2018, no. July 2018, pp. 108-112, 2018. https://doi.org/10.1109/ISET.2018.00032 [5] R. C. Kalloo, B. Mitchell, and V. J. Kamalodeen, "Responding to the COVID-19 pandemic in Trinidad and Tobago: challenges and opportunities for teacher education," J. Educ. Teach., vol. 46, no. 4, pp. 452-462, 2020. https://doi.org/10.1080/02607476.2020.1800407 [6] B. Eickelmann and J. Gerick, "Lernen mit digitalen Medien," Langsam vermisse ich die Schule, pp. 153-162, 2020. https://doi.org/10.31244/9783830992318.09 [7] A. W. Chickering and S. C. Ehrmann, "Implementing the Seven Principles: Technology as Lever," AAHE Bulletin, no. October 1996, 1996. [8] M. D. Dixson, "Creating effective student engagement in online courses : What do students find engaging?" J. Scholarsh. Teach. Learn., vol. 10, no. 2, pp. 1-13, 2010. [9] S. L. Hines et al., "Engaging online students by activating ecological knowledge," Ecol. Evol., no. June, pp. 1-10, 2020. https://doi.org/10.22541/au.159355985.50556448 [10] M. Drolia, E. Sifaki, and S. Papadakis, "An Overview of Mobile Learning for Refugee Students : Juxtaposing Refugee Needs with Mobile Applications' Characteristics," Chal- lenges, vol. 11, no. 31, pp. 1-14, 2020. https://doi.org/10.3390/challe11020031 [11] G. Giannakopoulos, N. A. Tatlas, V. Giannakopoulos, A. Floros, and P. Katsoulis, "Acces- sible electronic games for blind children and young people," Br. J. Educ. Technol., vol. 49, no. 4, pp. 608-619, 2018. https://doi.org/10.1111/bjet.12628 [12] H. K. Kabali et al., "Exposure and use of mobile media devices by young children," Pedi- atrics, vol. 136, no. 6, pp. 1044-1050, 2015. https://doi.org/10.1542/peds.2015-2151 [13] S. Papadakis, and M. Kalogiannakis, "A research synthesis of the real value of self- proclaimed mobile educational applications for young children.," Mobile learning applica- tions in early childhood education, 1-19, 2020. https://doi.org/10.4018/978-1-7998-1486- 3.ch001 [14] S. Chaudron, R. Di Gioia, and M. Gemo, "Young children (0-8) and digital technology," Europe; EUR 29070, 2018. [15] N. Cavus and D. Ibrahim, "M-Learning: An experiment in using SMS to support learning new English language words," Br. J. Educ. Technol., vol. 40, no. 1, pp. 78-91, 2009. https://doi.org/10.1111/j.1467-8535.2007.00801.x [16] J. Godfrey and M. Reed, "App store after five years," ACT The App Association, 2013. [Online]. Available: https://actonline.org/wpcontent/uploads/2014/04/The-App-Store- After-Five-Years.pdf. [17] S. Papadakis, J. Vaiopoulou, M. Kalogiannakis, and D. Stamovlasis, "Developing and ex- ploring an evaluation tool for educational apps (ETEA) targeting kindergarten children," Sustain., vol. 12, no. 10, pp. 1-10, 2020. https://doi.org/10.3390/su12104201 114 http://www.i-jim.org https://doi.org/10.1007/s10648-012-9199-6 https://doi.org/10.5539/hes.v10n3p16 https://doi.org/10.5539/hes.v10n3p16 https://doi.org/10.1007/s11125-020-09464-3 https://doi.org/10.1109/ISET.2018.00032 https://doi.org/10.1080/02607476.2020.1800407 https://doi.org/10.31244/9783830992318.09 https://doi.org/10.22541/au.159355985.50556448 https://doi.org/10.3390/challe11020031 https://doi.org/10.1111/bjet.12628 https://doi.org/10.1542/peds.2015-2151 https://doi.org/10.4018/978-1-7998-1486-3.ch001 https://doi.org/10.4018/978-1-7998-1486-3.ch001 https://doi.org/10.1111/j.1467-8535.2007.00801.x https://actonline.org/wpcontent/uploads/2014/04/The-App-Store-After-Five-Years.pdf https://actonline.org/wpcontent/uploads/2014/04/The-App-Store-After-Five-Years.pdf https://doi.org/10.3390/su12104201 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… [18] S. Papadakis, N. Zaranis, and M. Kalogiannakis, "Parental involvement and attitudes to- wards young Greek children's mobile usage," Int. J. Child-Computer Interact., vol. 22, no. xxxx, p. 100144, 2019. https://doi.org/10.1016/j.ijcci.2019.100144 [19] P. S. D. Chen, A. D. Lambert, and K. R. Guidry, "Engaging online learners: The impact of Web-based learning technology on college student engagement," Comput. Educ., vol. 54, no. 4, pp. 1222-1232, 2010. https://doi.org/10.1016/j.compedu.2009.11.008 [20] N. Louwrens and M. Hartnett, "Student and teacher perceptions of online student engage- ment in an online middle school," J. Open, Flex. Distance Learn., vol. 19, no. 1, pp. 27-44, 2015. [21] M. Barak and G. Green, "Novice Researchers' Views About Online Ethics Education and the Instructional Design Components that May Foster Ethical Practice," Sci. Eng. Ethics, vol. 26, no. 3, pp. 1403-1421, 2020. https://doi.org/10.1007/s11948-019-00169-1 [22] M. Henderson, N. Selwyn, and R. Aston, "What works and why? Student perceptions of 'useful' digital technology in university teaching and learning," Stud. High. Educ., vol. 42, no. 8, pp. 1567-1579, 2017. https://doi.org/10.1080/03075079.2015.1007946 [23] N. Selwyn, "Digital downsides: exploring university students' negative engagements with digital technology," Teach. High. Educ., vol. 21, no. 8, pp. 1006-1021, 2016. https://doi.org/10.1080/13562517.2016.1213229 [24] Y. Alison and Ö. Delialioğlu, "The effect of authentic m-learning activities on student en- gagement and motivation," Br. J. Educ. Technol., vol. 50, no. 2, pp. 655-668, 2019. https://doi.org/10.1111/bjet.12559 [25] H. Bouta, S. Retalis, and F. Paraskeva, "Utilising a collaborative macro-script to enhance student engagement: A mixed-method study in a 3D virtual environment," Comput. Educ., vol. 58, no. 1, pp. 501-517, 2012. https://doi.org/10.1016/j.compedu.2011.08.031 [26] T. Rashid and H. M. Asghar, "Technology use, self-directed learning, student engagement and academic performance: Examining the interrelations," Comput. Human Behav., vol. 63, pp. 604-612, 2016. https://doi.org/10.1016/j.chb.2016.05.084 [27] J. Salaber, "Facilitating student engagement and collaboration in a large postgraduate course using wiki-based activities," Int. J. Manag. Educ., vol. 12, no. 2, pp. 115-126, 2014. https://doi.org/10.1016/j.ijme.2014.03.006 [28] J. Kirkwood, "Motivational factors in a push-pull theory of entrepreneurship," Gend. Manag., vol. 24, no. 5, pp. 346-364, 2009. https://doi.org/10.1108/17542410910968805 [29] R. M. Tamim, R. M. Bernard, E. Borokhovski, P. C. Abrami, and R. F. Schmid, "What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study," Rev. Educ. Res., vol. 81, no. 1, pp. 4-28, 2011. https://doi.org/10.3102/0034654310393361 [30] C. Englund, A. D. Olofsson, and L. Price, "Teaching with technology in higher education: understanding conceptual change and development in practice," High. Educ. Res. Dev., vol. 36, no. 1, pp. 73-87, 2017. https://doi.org/10.1080/07294360.2016.1171300 [31] S. Popenici, "Towards a New Vision for University Governance, Pedagogies, and Student Engagement," in The Student Engagement Handbook: Practice in Higher Education, no. December 2013, 2013, pp. 23-42. [32] B. Dietz-Uhler, A. Fisher, and A. Han, "Designing Online Courses to Promote Student Re- tention," J. Educ. Technol. Syst., vol. 36, no. 1, pp. 105-112, 2007. https://doi.org/ 10.2190/ET.36.1.g [33] S. S. Jaggars, "Choosing Between Online and Face-to-Face Courses: Community College Student Voices," Am. J. Distance Educ., vol. 28, no. 1, pp. 27-38, 2014. https://doi.org/ 10.1080/08923647.2014.867697 iJIM ‒ Vol. 15, No. 08, 2021 115 https://doi.org/10.1016/j.ijcci.2019.100144 https://doi.org/10.1016/j.compedu.2009.11.008 https://doi.org/10.1007/s11948-019-00169-1 https://doi.org/10.1080/03075079.2015.1007946 https://doi.org/10.1080/13562517.2016.1213229 https://doi.org/10.1111/bjet.12559 https://doi.org/10.1016/j.compedu.2011.08.031 https://doi.org/10.1016/j.chb.2016.05.084 https://doi.org/10.1016/j.ijme.2014.03.006 https://doi.org/10.1108/17542410910968805 https://doi.org/10.3102/0034654310393361 https://doi.org/10.1080/07294360.2016.1171300 https://doi.org/10.2190/ET.36.1.g https://doi.org/10.2190/ET.36.1.g https://doi.org/10.1080/08923647.2014.867697 https://doi.org/10.1080/08923647.2014.867697 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… [34] M. Hu and H. Li, "Student engagement in online learning: A review," in Proceedings - 2017 International Symposium on Educational Technology, ISET 2017, 2017, pp. 39-43. https://doi.org/10.1109/ISET.2017.17 [35] A. J. Hong and H. J. Kim, "College Students' Digital Readiness for Academic Engagement (DRAE) Scale: Scale Development and Validation," Asia-Pacific Educ. Res., vol. 27, no. 4, pp. 303-312, 2018. https://doi.org/10.1007/s40299-018-0387-0 [36] A. Salman and A. Abdul Aziz, "Evaluating user Readiness towards Digital Society: A Rasch Measurement Model Analysis," Procedia Comput. Sci., vol. 65, no. Iccmit, pp. 1154-1159, 2015. https://doi.org/10.1016/j.procs.2015.09.028 [37] H. J. Kim, A. J. Hong, and H. D. Song, "The roles of academic engagement and digital readiness in students' achievements in university e-learning environments," Int. J. Educ. Technol. High. Educ., vol. 16, no. 1, pp. 3-18, 2019. https://doi.org/10.1186/s41239-019- 0152-3 [38] A. Margaryan, A. Littlejohn, and G. Vojt, "Are digital natives a myth or reality? University students' use of digital technologies," Comput. Educ., vol. 56, no. 2, pp. 429-440, 2011. https://doi.org/10.1016/j.compedu.2010.09.004 [39] S. F. Griffith and D. H. Arnold, "Home learning in the new mobile age: Parent-child inter- actions during joint play with educational apps in the US," J. Child. Media, vol. 13, no. 1, pp. 1-19, 2019. https://doi.org/10.1080/17482798.2018.1489866 [40] M. Kalogiannakis, M. Ampartzaki, S. Papadakis, and E. Skaraki, "Teaching natural sci- ence concepts to young children with mobile devices and hands-on activities. A case study," Int. J. Teach. Case Stud., vol. 9, no. 2, p. 171, 2018. https://doi.org/10.1504/IJTCS.2018.090965 [41] T. A. Schmitt, "Current methodological considerations in exploratory and confirmatory factor analysis," J. Psychoeduc. Assess., vol. 29, no. 4, pp. 304-321, 2011. https://doi.org/ 10.1177/0734282911406653 [42] M. M. Handelsman, W. L. Briggs, N. Sullivan, and A. Towler, "A Measure of College Student Course Engagement," J. Educ. Res., vol. 98, no. 3, pp. 184-192, 2005. https://doi.org/10.3200/JOER.98.3.184-192 [43] T. H. Chu and Y. Y. Chen, "With Good We Become Good: Understanding e-learning adoption by theory of planned behavior and group influences," Comput. Educ., vol. 92-93, pp. 37-52, 2016. https://doi.org/10.1016/j.compedu.2015.09.013 [44] C. Fornell and D. F. Larcker, "Evaluating Structural Equation Models with Unobservable Variables and Measurement Error," J. Mark. Res., vol. 18, no. 1, p. 39, 1981. https://doi.org/10.2307/3151312 [45] C. M. Ringle, S. Wende, and J. M. Becker, "SmartPLS 3," 2015. [46] D. W. Gerbing and J. C. Anderson, "Monte Carlo evaluation of goodness of fit indices for structural equation models," Social. Methods Res., vol. 21, no. 2, pp. 132-160, 1992. https://doi.org/10.1177/0049124192021002002 [47] J. F. J. Hair, G. T. M. Hult, C. M. Ringle, and M. Sarstedt, A primer on partial least squares structural equation modeling (PLS-SEM), Second ed. SAGE, 2017. https://doi.org/10.15358/9783800653614 [48] S. Petter, "'Haters Gonna Hate': PLS and Information Systems Research," ACM SIGMIS Database DATABASE Adv. Inf. Syst., vol. 49, no. 2, pp. 10-13, 2018. https://doi.org/ 10.1145/3229335.3229337 [49] W. W. Chin, A. Gopal, and W. D. Salisbury, "Advancing the Theory of Adaptive Structu- ration: The Development of a Scale to Measure Faithfulness of Appropriation," Inf. Syst. Res., vol. 8, no. 4, pp. 342-367, 1997. https://doi.org/10.1287/isre.8.4.342 116 http://www.i-jim.org https://doi.org/10.1109/ISET.2017.17 https://doi.org/10.1007/s40299-018-0387-0 https://doi.org/10.1016/j.procs.2015.09.028 https://doi.org/10.1186/s41239-019-0152-3 https://doi.org/10.1186/s41239-019-0152-3 https://doi.org/10.1016/j.compedu.2010.09.004 https://doi.org/10.1080/17482798.2018.1489866 https://doi.org/10.1504/IJTCS.2018.090965 https://doi.org/10.1177/0734282911406653 https://doi.org/10.1177/0734282911406653 https://doi.org/10.3200/JOER.98.3.184-192 https://doi.org/10.1016/j.compedu.2015.09.013 https://doi.org/10.2307/3151312 https://doi.org/10.1177/0049124192021002002 https://doi.org/10.15358/9783800653614 https://doi.org/10.1145/3229335.3229337 https://doi.org/10.1145/3229335.3229337 https://doi.org/10.1287/isre.8.4.342 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… [50] L. Hu and P. M. Bentler, "Fit indices in covariance structure modeling: Sensitivity to under parameterized model misspecification.," Psychol. Methods, vol. 3, no. 4, pp. 424-453, 1998. https://doi.org/10.1037/1082-989X.3.4.424 [51] J. Henseler, G. Hubona, and P. A. Ray, "Using PLS path modeling in new technology re- search: updated guidelines," Ind. Manag. Data Syst., vol. 116, no. 1, pp. 2-20, 2016. https://doi.org/10.1108/IMDS-09-2015-0382 [52] J. F. Hair, T. Hult, C. M. Ringle, and M. Started, A primer on partial least squares path modeling (PLS-SEM), 2nd ed. Thousand Oaks, CA: SAGE, 2017. [53] J. C. Nunnally, I. H. Bernstein, and J. M. F. Berge, Psychometric theory. New York: McGraw Hill, 1967. [54] J. Davies and M. Graff, "Performance in e-learning: Online participation and student grades," Br. J. Educ. Technol., vol. 36, no. 4, pp. 657-663, 2005. https://doi.org/10.11 11/j.1467-8535.2005.00542.x [55] G. D. Kuh, " Assessing What Really Matters to Student Learning Inside The National Sur- vey of Student Engagement," Chang. Mag. High. Learn., vol. 33, no. 3, pp. 10-17, 2001. https://doi.org/10.1080/00091380109601795 [56] T. Rodgers, "Student Engagement in the E-Learning Process and the Impact on Their Grades," Int. J. cyber Soc. Educ., vol. 1, no. 2, pp. 143-156, 2008. [57] J. Bennett and L. Bennett, "A review of factors that influence the diffusion of innovation when structuring a faculty training program," Internet High. Educ., vol. 6, no. 1, pp. 53-63, 2003. https://doi.org/10.1016/S1096-7516(02)00161-6 [58] L. Carter and F. Bélanger, "The utilization of e-government services: Citizen trust, innova- tion and acceptance factors," Inf. Syst. J., vol. 15, no. 1, pp. 5-25, 2005. https://doi.org/ 10.1111/j.1365-2575.2005.00183.x [59] R. D. Axelson and A. Flick, "Defining Student Engagement," Chang. Mag. High. Learn., vol. 43, no. 1, pp. 38-43, 2010. https://doi.org/10.1080/00091383.2011.533096 [60] A. Ituma, "An evaluation of students' perceptions and engagement with e-learning compo- nents in a campus-based university," Act. Learn. High. Educ., vol. 12, no. 1, pp. 57-68, 2011. https://doi.org/10.1177/1469787410387722 [61] D. Morin, J. D. E. Thomas, and R. George Saadé, "Activities and Resources in Online Learning: From a Critical Thinking View," Proc. 2012 InSITE Conf., no. June 2014, pp. 597-602, 2012. https://doi.org/10.28945/1678 [62] R. G. Saadé, D. Morin, and J. D. E. Thomas, "Critical thinking in E-learning environ- ments," Comput. Human Behav., vol. 28, no. 5, pp. 1608-1617, 2012. https://doi.org/10.1016/j.chb.2012.03.025 [63] T. Blayne, "Reexamining digital-learning readiness in higher education: Positioning digital competencies as key factors and a profile application as a readiness tool," Int. J. E- Learning Corp. Gov. Heal. High. Educ., vol. 17, no. 4, pp. 425-451, 2018. [64] M. Parkes, S. Stein, and C. Reading, "Student preparedness for university e-learning envi- ronments," Internet High. Educ., vol. 25, pp. 1-10, 2015. https://doi.org/10.1016/j.iheduc. 2014.10.002 [65] J. M. Alducin-Ochoa and A. I. Vázquez-Martínez, "Hybrid Learning: An Effective Re- source in University Education?," Int. Educ. Stud., vol. 9, no. 8, p. 1, 2016. https://doi.org/10.5539/ies.v9n8p1 [66] M. V. López-Pérez, M. C. Pérez-López, and L. Rodríguez-Ariza, "Blended learning in higher education: Students' perceptions and their relation to outcomes," Comput. Educ., vol. 56, no. 3, pp. 818-826, 2011. https://doi.org/10.1016/j.compedu.2010.10.023 [67] M. V. López-Pérez, M. C. Pérez-López, L. Rodríguez-Ariza, and E. Argente-Linares, "The influence of the use of technology on student outcomes in a blended learning context," iJIM ‒ Vol. 15, No. 08, 2021 117 https://doi.org/10.1037/1082-989X.3.4.424 https://doi.org/10.1108/IMDS-09-2015-0382 https://doi.org/10.1111/j.1467-8535.2005.00542.x https://doi.org/10.1111/j.1467-8535.2005.00542.x https://doi.org/10.1080/00091380109601795 https://doi.org/10.1016/S1096-7516(02)00161-6 https://doi.org/10.1111/j.1365-2575.2005.00183.x https://doi.org/10.1111/j.1365-2575.2005.00183.x https://doi.org/10.1080/00091383.2011.533096 https://doi.org/10.1177/1469787410387722 https://doi.org/10.28945/1678 https://doi.org/10.1016/j.chb.2012.03.025 https://doi.org/10.1016/j.iheduc.2014.10.002 https://doi.org/10.1016/j.iheduc.2014.10.002 https://doi.org/10.5539/ies.v9n8p1 https://doi.org/10.1016/j.compedu.2010.10.023 Paper—Enhancing Student Involvement Based on Adoption Mobile Learning Innovation as Interactive… Educ. Technol. Res. Dev., vol. 61, no. 4, pp. 625-638, 2013. https://doi.org/10.1007/s11423-013-9303-8 [68] T. Lyons and M. M. Evans, "Blended Learning to Increase Student Satisfaction: An Ex- ploratory Study," Internet Ref. Serv. Q., vol. 18, no. 1, pp. 43-53, 2013. https://doi.org/10.1080/10875301.2013.800626 [69] M. Händel, M. Stephan, M. Gläser-Zikuda, B. Kopp, S. Bedenlier, and A. Ziegler, "Digital readiness and its effects on higher education student socio-emotional experiences in the context of COVID-19 pandemic," no. July, 2020. https://doi.org/10.31234/osf.io/b9pg7 [70] S. Papadakis, Μ., Kalogiannakis, Ε., Sifaki, and N. Vidakis, "Evaluating moodle use via smart mobile phones. A case study in a Greek university." EAI Endorsed Transactions on Creative Technologies, 5(16), 2018. https://doi.org/10.4108/eai.10-4-2018.156382 8 Authors Mar’atus Sholikah is a postgraduate student in Economics Education Department at Universitas Negeri Yogyakarta (maratussholikah.2019@student.uny.ac.id). Dwi Harsono is a lecturer at Universitas Negeri Yogyakarta. He is also chief of the public administration department (dwiharsono@uny.ac.id). Article submitted 2020-11-09. Resubmitted 2021-01-25. Final acceptance 2021-01-25. Final version published as submitted by the authors. 118 http://www.i-jim.org https://doi.org/10.1007/s11423-013-9303-8 https://doi.org/10.1080/10875301.2013.800626 https://doi.org/10.31234/osf.io/b9pg7 https://doi.org/10.4108/eai.10-4-2018.156382 mailto:maratussholikah.2019@student.uny.ac.id mailto:dwiharsono@uny.ac.id