International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 03, 2021 Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education Interactive Mobile Technologies on Civic Education Learning in Higher Education https://doi.org/10.3991/ijim.v15i03.17509 Muhammad Japar (), Yuyus Kardiman, Raharjo Universitas Negeri Jakarta, Jakarta, Indonesia mjapar@unj.ac.id Dini Nur Fadhillah Universitas Sebelas Maret, Surakarta, Indonesia Syifa Syarifa Universitas Negeri Jakarta, Jakarta, Indonesia Abstract—This study aims to describe the process of developing mobile learning in civic education in higher education. Data collection was carried out using the research and development (R&D) through the A.D.D.I.E. model. Based on the material and media expert’s judgment, the feasibility level of 91.33% and 91.67%, respectively, reflect the application was very feasible to use. In the small and field group trials, the application score was between 4.29 and 4.06, meaning that the application was well developed and feasible for civic education learning in higher education. Keywords—Mobile learning, civic education, social tolerance, higher education 1 Introduction Technology in the global era is currently developing very quickly in civic education learning [1]. One of the positive impacts of the emergence of technologies is to help civic education students advance awareness of their rights and obligations as citizens. In addition to that, the development of civic education learning and technology shows increasing students’ active participation through video games[2]. So, learning based on information technology can be connected with higher education pedagogy to advancing civic education learning [3]. According to Fagan [4], educational institutions are re- quired to be adaptive to the changing paradigm of education from traditional to con- ventional, which is a demand of global society using I.C.T. (Information Communica- tion and Technology) [5]. Based on Chiu et al. [6], students must find information sources, process, and build their knowledge when the lecture gives an illustration. Therefore, students need technology to get information. In Russia, civic education be- comes an essential part of changing the cultural image and awareness of students' atti- tudes towards their political attitudes through social media[7]. Instead, the development 84 http://www.i-jim.org https://doi.org/10.3991/ijim.v15i03.17509 mailto:mjapar@unj.ac.id Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education of technology in Israel increased civic empowerment in the educational environment through civic education learning[8]. Meanwhile, In England, civic education learning used smartphones and became a new cultural ecology in the learning process[9]. The operating system that is widely used today in smartphones is Android-based because it is open source. Android helped developers create applications for various kinds of smartphones[10]. Consequently, android has many applications that can help higher education students' in the learning process. In the digital era, civic education is essential to encourage students to be democratic people [11]. Based on Barber [12], learning democracy can be done through civic edu- cation learning amidst inequality to understand students in the middle of globalization. Therefore, in order for civic education to keep up with technology advances, it is nec- essary to develop an android-based mobile application into an innovative learning pro- cess [13]. So, Learning media is needed to deliver learning messages concerning mul- ticulturalism and social tolerance [14]. Likewise, British Columbia and Canada also promote human rights, multiculturalism, and national identity through civic education [15]. As we know, multiculturalism provides awareness to students that diversity makes them more tolerance in society [16]. Bangwayo-Skeete dan Zikhali [17] also shows that social tolerance can reduce the possibility of conflict between communities such as in Latin America and the Caribbean. However, the United States tried to develop social capital measures against social bonds in social tolerance without seeing any significant results [18]. Consequently, Flew and Iosifidis[19] shows that social tolerance can be done by introducing cultural values from local wisdom. In the technology era, civic education uses mobile application learning media to learn by themself [20] [21]. The concept of learning by Ugbaje [22] towards a pedagogical and sociotechnical integration using mobile learning provides many benefits for lecturers and students. On the other side, Kalogiannakis and Papadakis[23] showed how technology affected ed- ucators' increasing pedagogical training. However, the mobile application must always be developed to minimize existing limitations such as device performance, display screens, and storage capacity[24]. Smartphones also encourage students to reach their learning goals[25], mainly through interactive mobile technologies[26]. This research aims to prepare students to instill social tolerance to accept diversity and minimalize discrimination on religion, race, and ethnicity through interactive mobile technologies on civic education learning. 2 Literature Review 2.1 Mobile technologies for civic education learning Mobile learning is developed through an android application [27] in the form of a learning media product that can be downloaded via an Android-based smartphone. The use of smartphones as learning media also provides new experiences for students and makes it easier for students to learn because of its simple shape[5]. Nickerson, Rapanta, and Goby [28] explained that mobile learning as a learning medium is not only assessed on one side. Using mobile learning should motivate and stimulate students to remember iJIM ‒ Vol. 15, No. 03, 2021 85 Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education what they have learned and provide learning stimulation [29]. Thus, the use of an ap- plication as a learning medium must fulfill several criteria such as ease of navigation, contains cognition, knowledge, and presentation of information. Also, mobile learning has media integration criteria to attract learners[30]. There is a function as a whole, meaning that the program being developed must provide the learner's learning desired. Therefore, the mobile application can improve the quality of civic education learn- ing[31]. Mobile learning is related to learning through mobile wireless technology. Stu- dents have the right to access learning materials and information to improve their qual- ity of life regardless of where they live, their status, and their culture[29]. The develop- ment of mobile learning is motivated by the high-speed penetration of mobile devices. According to Kalogiannakis and Papadakis [32], mobile devices can be used as learning media. The use of mobile learning in teaching and learning activities spurs students to understand the material by improving their cognitive domains and increasing the affec- tive domain and interactive power in the psychomotor domain [33]. Mobile learning developed through an android-based application helps students un- derstand how to implement social tolerance in civic education[34]. Civic education stu- dents conduct analyses related to ethnic diversity. Developing mobile learning is long- life learning; students can be more active in the learning process [35]. In civic education there is the concept of tolerance also implies an attitude not to be insulting, not con- demning, not blaspheming, not feeling self-righteous, and not wanting to win alone in living together with other components that are different from our existence[36]. Be- cause living a tolerance can build togetherness. Civic education taught students how to be Tolerance and respect diversity [37]. In some instances, it connotes the attitude of passiveness towards something dislike, and often it means putting up with or enduring something disliked [38]. Tolerance can also mean terms of socio-cultural and religious contexts, which means attitudes and actions that prohibit discrimination against differ- ent groups or are not acceptable to the majority in a society [39]. Therefore, innovative learning media are needed to introduce cultural diversity to prospective civic education learning in higher education[40]. Students need to have the character of mutual respect through social tolerance in civic education learning. According to Lee and Park [41], mobile technologies can facilitate students’ civic education engagement, and learning innovation arises from a change in the learning paradigm [42]. Civic education can make students aware of the diversity and local culture of all ethnic groups. Thus, mobile learning to introduce civic education through local culture can provide direction for students to implement social tolerance in their life. Meanwhile, students can become the next generation of the nation with a high toler- ance for different customs and cultures. Therefore, this research focused on interactive mobile technologies on civic education learning in higher education. Based on Ingrams [43], civic behavior is transformed through mobile information and connectivity. So, mobile learning makes students more civically engaged[44]. 86 http://www.i-jim.org Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education 3 Method 3.1 Research design This study is an educational research and development (R & D) study that aims to develop mobile technologies in learning resources for social tolerance based on android in learning and learning theory courses [45]. Educational development research in- cludes the development process, product validation, and product testing. Researchers are trying to develop a product effectively used in preparing civic education teachers who have social tolerance through development research. This study is an Android- based mobile application as a learning resource for social tolerance education to im- prove pre-service civic education teachers' learning innovation and provide learning media. The A.D.D.I.E. development model developed by Dick and Carry in this study in the Analysis, Design, Development, Implementation, and Evaluation stages.[46]. 3.2 Research setting and data collection This research was conducted at the Civic Education Study Program, Universitas Negeri Jakarta. The university is one of the Educational Personnel Teaching Institutions in Jakarta, Indonesia. This research started in March 2020. Data collection was carried out using interview techniques and questionnaires. In this study, we have collaborated with Civic Education lectures and students for interviews. The results of the interview are used as material for consideration in the learning media development process. Also, the questionnaire is intended for instructional media experts and material experts. The validity is here to test the developed learning media's appropriateness and test its suita- bility with the material. The lecturers' results and students' responses gave a positive response, which was shown by the questionnaire's results. The data obtained from the results of the response questionnaire of lecturers and students are then analyzed using quantitative data to test the practicality of the product developed. [47]. 3.3 Data analysis The research data were analyzed qualitatively and quantitatively. Quantitative data analysis is used to analyze data collected from questionnaires. Quantitative data were obtained at the research stage of design validation and use trials. The value obtained for each of these steps can be obtained by using a questionnaire of data analysis of expert validity (expert lecturers of learning media and expert lecturers of civic education) and questionnaires for analyzing lecturers and students' responses to the use of mobile learn- ing media. The validity is here to test the developed learning media's feasibility and test its suitability with the material. Answers to expert validation questionnaires use a Likert scale; the measured variables are translated into variable indicators. The questionnaire test for the validation of learning media experts can be done by comparing the number of respondents' scores (∑) with the total ideal score (N). iJIM ‒ Vol. 15, No. 03, 2021 87 Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education The validation criteria used in the validity of media research are presented in the table. Table 1. The Percentage of Media Feasibility No Achievement Level Qualification 1 81-100% Highly Feasible 2 61-80% Feasible 3 41-60% Moderately Feasible 4 21-40% Less Feasible 5 <20% Unfeasible Meanwhile, qualitative descriptive analysis is used to process interviews, data from critical questionnaires, and suggestions by instructional media experts and material ex- perts. Data analysis techniques are used to classify information from qualitative data in responses, criticism, and suggestions for improvements and revisions of mobile learn- ing products. 3.4 Data collection techniques We were collecting data in this study using interview techniques, questionnaires, and tests. In this study, researchers interviewed lecturers in civic education courses using interview guidelines that contained several aspects, including learning media, learning methods, student characteristics, and learning facilities. The interviews' results were then considered in the media development process following students' and lecturers' needs. Besides, the questionnaire is aimed at media and material experts. Researchers work with educational technology lecturers to review the media. Also, the test is divided into two, namely, pre-test and post-test. The pre-test is given to students to determine their initial ability in learning material about the diversity of cultures around before they are taught using learning media. Simultaneously, the post-test is carried out to deter- mine student learning outcomes after they get material using learning media. 4 Results and Discussion 4.1 Mobile learning design Before carrying out the learning media design process, researchers observed the learning process of civic education courses. The interview was conducted with lecturers in civic education courses and class 2017 at the civic education study program, Univer- sitas Negeri Jakarta. The analysis includes several aspects such as curriculum, under- standing of social tolerance, prospective teachers' needs, the media used by lecturers, 88 http://www.i-jim.org Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education and teaching materials. Based on these data, civic education's learning process utilizes the lecture and discussion method with PowerPoint tools. In the civic education learning process, I usually divide the groups of 5 people each. Then, they will make papers and PowerPoint. After that, present it in class. Then it will be followed by a discussion. As a lecturer in civic education, ease of learning is needed. However, the lack of facilities makes it difficult to develop learning innovations (Lec- ture, 2020). As a student, I feel that the current learning process is not very innovative. Moreover, we will deal with students who understand much better-using technology. So that technology in the field of civic education is needed. (Student, 2020). The results of interviews conducted with students and lecturers showed that there was no thorough preparation in an innovative learning process. Research data also shows that civic education students doubt their ability to develop innovative learning media. It is because tolerance is considered a valuable asset for any community to create an open and humane society [48]. Based on the results of the analysis collected by re- searchers, lecturers need learning media that can help students understand civic educa- tion learning in cultural diversity material to implement it in an attitude of social toler- ance. Besides, mobile application increases learning motivation in civic education learning. The use of smartphones as an effort for the learning process at universities can increase student involvement in the classroom[33], [49]. The learning media developed in this study is an Android-based application called "T.O.L.E.R.A.N." and can be down- loaded via the Google Play Store. This application can improve the quality of students learning and is useful for improving civic education learning outcomes[29], [50], [51]. An android-based application has been developed to help civic education lectures in- troduce S.T.E.M. activities in the classroom and prepare them to teach digital literacy [32],[52]. In the next stage, the researcher conducted an application design. In this process, the layout was designed using CorelDRAW X8 and Canva's features. Meanwhile, the ap- plication features were developed by Atoz Createapss, a web-based mobile application maker. The design process consists of developing the assessment instrument and de- signing the "T.O.L.E.R.A.N." application. Described as follows: a. Developing an Assessment Instrument for the "T.O.L.E.R.A.N." Application: The "T.O.L.E.R.A.N." application assessment instrument is formulated in the form of a questionnaire to check the application's appropriateness. This instrument aims to collect data from material experts, media experts, and students. This instrument uses a Likert scale reference that includes 4 categories: Very Good, Good, Bad, and Very Bad. b. "T.O.L.E.R.A.N." Application Design: This application consists of several essential menus. On the Main Menu, the features are displayed under a definite theme. The "search for journals" menu provides a ref- erence for students to make it easier to analyze various research results related to civic education. When students click, search for journals, they are immediately di- rected to the Scientific Journal of the Democracy Forum belonging to the Citizenship Education Study Program, Universitas Negeri Jakarta. This journal publishes the lat- est research results on citizenship and tolerance. The menu "Tolerance Education" iJIM ‒ Vol. 15, No. 03, 2021 89 Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education includes learning models that instill social tolerance and interesting learning con- cepts for students in civic education learning. On the "Web Tolerance" menu, stu- dents can analyze the problems of intolerance in Indonesia. The cultural and tolerant living conditions in each area. The "Tolerance Gallery" menu is provided to make it easier for readers to see Indonesia's cultures. Also, carrying out traditional ceremo- nies in several areas is carried out in collaboration with each other even though they are of different religious backgrounds. The "menu of tolerance videos" offers users more interactive, equipped with animations and videos of implementing tolerance in cultural diversity. Detailed learning materials about social tolerance and cultural di- versity can be found by collecting interesting animated videos to watch and learn. The video can be accessed on the Video Tolerance menu. In short, the design of the "T.O.L.E.R.A.N." application above has considered several essential aspects, such as the use of exciting animation/photos to attract students' interest, provide complete information sources, and offer interactive use. This is relevant to the research results where the application developed is expected to increase social tolerance learning. 4.2 Validation of application Material experts and media experts carry out the "T.O.L.E.R.A.N." application val- idation. They are the lecturers at the Educational Technology and Civic Education, Uni- versitas Negeri Jakarta. Material experts assess the quality of content and social toler- ance learning objectives based on the material developed in the application and input their responses to the questionnaire designed using a 5-point scale. Also, they are en- couraged to provide suggestions and comments to improve the application. The follow- ing are the results of the material expert's validation. Table 2. Results of Material Expert Validation No Aspect Percentage 1 Content Quality and Purpose 90,21% 2 Application Program 89,90% 3 Relevance of Material 93,90% Table 2 shows that it is highly feasible for students and lecturers of civic education from the quality of the content and objectives. Also, the application program is consid- ered highly feasible. According to the results of the material expert's comments ex- plained: • The material provided by the "T.O.L.E.R.A.N." application is very interactive, then the animation contained in the learning video is easy to understand. Layout in the form of right-left alignment, understanding, process, and tolerance implementation is exciting and neat. Some of the cultures raised are very attractive to users. (material experts, 2020). Besides, the researcher validated media experts with the following results. 90 http://www.i-jim.org Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education Table 3. Results of Media Expert Validation No Aspect Percentage 1 Content Quality and Purpose 89,21% 2 Application Program 93,90% 3 Relevance of Material 91,90% Table 3 shows the media validation from these three aspects is highly feasible as a learning medium. Thus, based on material experts and the "T.O.L.E.R.A.N." applica- tion, lecturers and students are suitable for learning civic education in class. The ad- vancement of information technology presents its challenges for pre-service teachers in the learning process to utilize technology in inadequate education [4]. Besides, mobile learning's effective use can improve interactive learning and encourage an efficient learning process [53], [54]. After validating, the next researcher implemented the "T.O.L.E.R.A.N." application. To measure the effectiveness of this development's re- sults, researchers did not only measure learning outcomes through questionnaires at the small group trial stage and field test. Each question asked refers to the same indicator. The following are the implementation results. Table 4. Validation Results of Small Group and Field Group No Indicator Small-Group Field Group Scores Categories Scores Categories 1 Describe the structure and sequence of teach- ing tolerance 4,44 Good 4,03 Good 2 The learning process is varied 4,42 Good 4,11 Good 3 Ease of Use 4,5 Good 4,03 Good 4 Understand the diversity of Indonesian cul- ture 4,08 Good 3,71 Moderate 5 The effectiveness of navigation in the appli- cation 4,22 Good 3,44 Moderate 6 Provide motivation and attract attention to learn tolerance 4,11 Good 4.72 Good 7 Attractive and innovative learning media 4,29 Good 4,42 Good The "T.O.L.E.R.A.N." material and media aspects presented in Table 4 prove that this application is feasible, as indicated by a score ranging between 4.29 and 4.06. 5 Evaluation of Application At the evaluation stage, a final product revision ("T.O.L.E.R.A.N.") is carried out based on suggestions and input from students involved in large group trials. • I find learning through the "T.O.L.E.R.A.N." application very good. Because using a mobile application can add to my insight into how to teach correctly in class. (Stu- dents 2, 2020). I am very interested and happy to use the "T.O.L.E.R.A.N." application as a medium for learning tolerance in the classroom. Easy to use, the material is easy to understand to learn to manage situations in the classroom through the mobile application (Student 4, 2020). iJIM ‒ Vol. 15, No. 03, 2021 91 Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education Besides, students suggested that improvements need to be made to make the "T.O.L.E.R.A.N." device more attractive. Besides that, the animation, video, and ma- terial in "T.O.L.E.R.A.N." are following basic competencies' objectives. Based on stu- dent validation and responses, "T.O.L.E.R.A.N." provides an Android-based learning media that is very feasible for civic education. Through its development, this research confirms that learning activities that utilize technology can be carried out anywhere and anytime. In line with Chamorro et al.[55] a mobile application can be a recommendation on social context such as civic education learning. Besides that, the purpose of using application-based learning media can support collaboration in the learning process. Therefore, the potential for integrating technology and learning media is used to solve problems that arise due to a lack of facilities in learning activities. Meanwhile, this study's findings are relevant for increasing interactive learning and encouraging an ef- ficient learning process. [4], [53], [54]. 6 Conclusion "T.O.L.E.R.A.N." is an application based on mobile technologies that are used as a learning medium through a five-stage development process (A.D.D.I.E.)—namely, an- alyzing an early stage to see the needs of lecturers and students. Furthermore, design refers to making research instruments, designing "T.O.L.E.R.A.N." products, and up- loading them to the Google Play Store. Then, in the development process related to application development, conducting expert assessments, and revising. After making the revisions, the small and field group trials were carried out to process the "T.O.L.E.R.A.N." application due diligence. This development's final stage is an eval- uation that compares the testing phase results and concludes its feasibility. According to material and media experts, the feasibility level of 91.33% and 91.67% reflects high feasibility. "T.O.L.E.R.A.N." has been the subject of small group and field trials as a learning medium for civic education, and responses are expressed as dimensionless scores in the 4.0-4.6 range, meaning that this application qualifies for further testing on large groups. In the small and field group trials, using "T.O.L.E.R.A.N." obtained a score between 4.29 and 4.06, which confirms its high eligibility for social tolerance learning in civic education. The application developed in this study focuses on a specific topic of Civics learning, namely, social tolerance. Therefore, this application can be used as a civic education learning model on social tolerance material. Thus, civic education understands the the- ory related to being a good citizen, but it is also able to identify local wisdom that exists in the students' environment so that they can implement social tolerance in society. Be- sides, the successful use of mobile applications in learning is influenced by teacher literacy about technology. Thus, future research can investigate this issue. It can be concluded that the Android-based learning media created can be used to improve stu- dent learning outcomes. This learning media can help students understand the material through animations available on mobile learning applications. In addition to animation, learning media also provides learning videos and journal searches. Besides, lecturers and students can see the latest information related to social tolerance. This application 92 http://www.i-jim.org Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education also provides practice questions that are directly connected to the Google form to facil- itate students. As feedback, students will be shown scores and comments when working on all questions. 7 Acknowledgement The author would like to thank the Research and Community Service Institute of Universitas Negeri Jakarta for funding this research as part of a competitive research program in the 2020 fiscal year with contract number 9 / KOMP-UNJ/ LPPM /V/ 2020. 8 References [1] M. Kalogiannakis and S. Papadakis, “Combining mobile technologies in environmental ed- ucation: a Greek case study,” Int. J. Mob. Learn. Organ., vol. 11, no. 2, p. 108, 2017. [2] G. Dishon and Y. B. Kafai, “Connected civic gaming: rethinking the role of video games in civic education,” Interact. Learn. Environ., vol. 0, no. 0, pp. 1–12, 2019. https://doi.org/ 10.1080/10494820.2019.1704791 [3] S. Papadakis, J. Vaiopoulou, M. Kalogiannakis, and D. Stamovlasis, “Developing and ex- ploring an evaluation tool for educational apps (E.T.E.A.) targeting kindergarten children,” Sustain., vol. 12, no. 10, pp. 1–10, 2020. https://doi.org/10.3390/su12104201 [4] M. H. Fagan, “Factors Influencing Student Acceptance of Mobile Learning in Higher Edu- cation,” Comput. Sch., vol. 36, no. 2, pp. 105–121, 2019. [5] A. M. Sølvberg and M. Rismark, “Learning spaces in mobile learning environments,” Act. Learn. High. Educ., vol. 13, no. 1, pp. 23–33, 2012. [6] P. S. Chiu, Y. H. Pu, C. C. Kao, T. T. Wu, and Y. M. Huang, “An authentic learning based evaluation method for mobile learning in Higher Education,” Innov. Educ. Teach. Int., vol. 55, no. 3, pp. 336–347, 2018. https://doi.org/10.1080/14703297.2017.1417147 [7] A. Tolstenko, L. Baltovskij, and I. Radikov, “Chance of civic education in Russia,” SAGE Open, vol. 9, no. 3, 2019. https://doi.org/10.1177/2158244019859684 [8] T. Gamliel, “Education in civic participation: Children, seniors and the challenges of an intergenerational information and communications technology program,” New Media Soc., vol. 19, no. 9, pp. 1388–1405, 2016. https://doi.org/10.1177/1461444816639971 [9] J. Cook, N. Pachler, and B. Bachmair, “Ubiquitous mobility with mobile phones: A cultural ecology for mobile learning,” E-Learning Digit. Media, vol. 8, no. 3, pp. 181–196, 2011. https://doi.org/10.2304/elea.2011.8.3.181 [10] V. P. Dennen and S. Hao, “Intentionally mobile pedagogy: The M-COPE framework for mobile learning in higher education,” Technol. Pedagog. Educ., vol. 23, no. 3, pp. 397–419, 2014. https://doi.org/10.1080/1475939x.2014.943278 [11] J. Kahne, E. Hodgin, and E. Eidman-Aadahl, “Redesigning Civic Education for the Digital Age: Participatory Politics and the Pursuit of Democratic Engagement,” Theory Res. Soc. Educ., vol. 44, no. 1, pp. 1–35, 2016. https://doi.org/10.1080/00933104.2015.1132646 [12] B. R. Barber, “Can We Teach Civic Education and Service-Learning in a World of Pri- vatisation, Inequality, and Interdependence?,” J. Coll. Character, vol. 13, no. 1, pp. 37–41, 2012. https://doi.org/10.1515/jcc-2012-1869 iJIM ‒ Vol. 15, No. 03, 2021 93 https://doi.org/10.1080/10494820.2019.1704791 https://doi.org/10.1080/10494820.2019.1704791 https://doi.org/10.3390/su12104201 https://doi.org/10.1080/14703297.2017.1417147 https://doi.org/10.1177/2158244019859684 https://doi.org/10.1177/1461444816639971 https://doi.org/10.2304/elea.2011.8.3.181 https://doi.org/10.1080/1475939x.2014.943278 https://doi.org/10.1080/00933104.2015.1132646 https://doi.org/10.1515/jcc-2012-1869 Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education [13] M. Murdiono, E. N. Rahmawati, M. A. Aziz, and U. N. Yogyakarta, “Developing an An- droid-Based Mobile Application for Civic Education Learning,” Int. J. Interact. Mob. Tech- nol., vol. 14, no. 16, pp. 180–193, 2020. https://doi.org/10.3991/ijim.v14i16.14967 [14] A. Elboubekri, “The Intercultural Communicative Competence and Digital Education,” J. Educ. Technol. Syst., vol. 45, no. 4, pp. 520–545, 2017. [15] P. Bromley, “Multiculturalism and human rights in civic education: The case of British Co- lumbia, Canada,” Educ. Res., vol. 53, no. 2, pp. 151–164, 2011. https://doi.org/10.1080/ 00131881.2011.572363 [16] K. Dunn and S. P. Singh, “Pluralistic conditioning: Social tolerance and effective de-moc- racy,” Democratisation, vol. 21, no. 1, pp. 1–28, 2014. [17] P. F. Bangwayo-Skeete and P. Zikhali, “Explaining self-declared social tolerance for human diversity in Latin America and the caribbean,” Forum Soc. Econ., vol. 42, no. 2–3, pp. 181– 206, 2013. https://doi.org/10.1080/07360932.2012.714713 [18] J. Wise and R. Driskell, “Tolerance Within Community: Does Social Capital Affect Toler- ance?,” Soc. Indic. Res., vol. 134, no. 2, pp. 607–629, 2017. https://doi.org/10.1007/ s11205-016-1449-4 [19] T. Flew and P. Iosifidis, “Populism, globalisation and social media,” Int. Commun. Gaz., 2019. [20] C. Kustandi, D. N. Fadhillah, R. Situmorang, D. S. Prawiradilaga, and S. Hartati, "V.R. use in online learning for higher education in Indonesia," Int. J. Interact. Mob. Technol., 2020. https://doi.org/10.3991/ijim.v14i01.11337 [21] I. Ismail, S. N. Azizan, and T. Gunasegaran, “Mobile learning in Malaysian universi-ties: Are students ready?,” Int. J. Interact. Mob. Technol., vol. 10, no. 3, pp. 17–23, 2016. https:// doi.org/10.3991/ijim.v10i3.5316 [22] S. Okai-Ugbaje, “Towards a pedagogical and sociotechnical framework for the strategic in- tegration of mobile learning in higher education in low and middle income countries,” High. Educ. Res. Dev., vol. 0, no. 0, pp. 1–18, 2020. https://doi.org/10.1080/07294360. 2020.1773768 [23] M. Kalogiannakis and S. Papadakis, "The dual form of further education of educators in I.C.T.: technological and pedagogical training.," in Proceedings of the 8th International Con- ference On Computer Based Learning in Science, 2007, pp. 265–276. [24] M. Zhou and Z. Li, “Blended mobile learning in theatre arts classrooms in higher edu-cat- ion,” Innov. Educ. Teach. Int., vol. 56, no. 3, pp. 307–317, 2019. [25] S. Papadakis, M. Kalogiannakis, E. Sifaki, and N. Vidakis, “Evaluating Moodle use via Smart Mobile Phones. A case study in a Greek University,” E.A.I. Endorsed Trans. Creat. Technol., vol. 5, no. 16, 2018. https://doi.org/10.4108/eai.10-4-2018.156382 [26] A. I. Saroia and S. Gao, “Investigating university students’ intention to use mobile learning management systems in Sweden,” Innov. Educ. Teach. Int., vol. 56, no. 5, pp. 569–580, 2019. https://doi.org/10.1080/14703297.2018.1557068 [27] M. Wilson and D. U. Bolliger, “Mobile learning: Endless possibilities for allied health edu- cators,” J. Diagnostic Med. Sonogr., vol. 29, no. 5, pp. 220–224, 2013. https://doi.org/ 10.1177/8756479313503734 [28] C. Nickerson, C. Rapanta, and V. P. Goby, “Mobile or Not? Assessing the Instruction-al Value of Mobile Learning,” Bus. Prof. Commun. Q., vol. 80, no. 2, pp. 137–153, 2017. https://doi.org/10.1177/2329490616663707 [29] S. Papadakis, “Evaluating pre-service teachers’ acceptance of mobile devices with re-gards to their age and gender: A case study in Greece,” Int. J. Mob. Learn. Organ., vol. 12, no. 4, pp. 336–352, 2018. https://doi.org/10.1504/ijmlo.2018.10013372 94 http://www.i-jim.org https://doi.org/10.3991/ijim.v14i16.14967 https://doi.org/10.1080/00131881.2011.572363 https://doi.org/10.1080/00131881.2011.572363 https://doi.org/10.1080/07360932.2012.714713 https://doi.org/10.1007/s11205-016-1449-4 https://doi.org/10.1007/s11205-016-1449-4 https://doi.org/10.3991/ijim.v14i01.11337 https://doi.org/10.3991/ijim.v10i3.5316 https://doi.org/10.3991/ijim.v10i3.5316 https://doi.org/10.1080/07294360.2020.1773768 https://doi.org/10.1080/07294360.2020.1773768 https://doi.org/10.4108/eai.10-4-2018.156382 https://doi.org/10.1080/14703297.2018.1557068 https://doi.org/10.1177/8756479313503734 https://doi.org/10.1177/8756479313503734 https://doi.org/10.1177/2329490616663707 https://doi.org/10.1504/ijmlo.2018.10013372 Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education [30] C. W. J. Chen, “Mobile learning: Using application Auralbook to learn aural skills,” Int. J. Music Educ., vol. 33, no. 2, pp. 244–259, 2015. https://doi.org/10.1177/0255761414533308 [31] Z. Mushtaq and A. Wahid, “Revised approach for the prediction of functional size of mobile application,” Appl. Comput. Informatics, 2019. [32] M. Kalogiannakis and S. Papadakis, "The Use of Developmentally Mobile Applications for Preparing Pre-Service Teachers to Promote S.T.E.M. Activities in Preschool Class-rooms," in Mobile Learning Applications in Early Childhood Education, Hershey: I.G.I. Global, 2020, pp. 82–100. https://doi.org/10.4018/978-1-7998-1486-3.ch005 [33] T. Ahmad, “Student perceptions on using cell phones as learning tools,” P.S.U. Res. Rev., vol. 4, no. 1, pp. 25–43, 2020. [34] C. Sheng-Hung, “Course delivery and module learning via learning objects (knowledge map) in mobile learning environment,” Asian Assoc. Open Univ. J., vol. 7, no. 1, pp. 43– 54, 2012. https://doi.org/10.1108/aaouj-07-01-2012-b004 [35] B. R. Cook and A. Babon, “Active learning through online quizzes: better learning and less (busy) work,” J. Geogr. High. Educ., vol. 41, no. 1, pp. 24–38, 2017. https://doi.org/ 10.1080/03098265.2016.1185772 [36] E. Karliani, S. Kartadinata, U. S. Winataputra, and K. Komalasari, “Indonesian civic en- gagement among college students,” J. Hum. Behav. Soc. Environ., pp. 1–11, 2019. [37] M. Japar, K. Sahid, and D. N. Fadhillah, “Social tolerance in multi-religious states: A case study in cigugur society, Indonesia,” Int. J. Innov. Creat. Chang., vol. 7, no. 7, 2019. [38] M. Japar, Komarudin, D. N. Fadhillah, and S. Syarifa, “Mapping the balinese social toler- ance model (Bhinneka tunggal ika for strengthening unity in diversity),Mapeo del modelo de tolerancia social de bali (Bhin-neka tunggal ika para fortalecer la unidad en la di-ver- sidad),” Opcion, 2019. [39] B. Sulistiono, A. Yusuf, and I. Hidayat, “Local Wisdom in Muslim Social Community in Bali Province: A Study of Tolerance,” in 2nd International Conference on Culture and Lan- guage in Southeast Asia, 2018, vol. 302, pp. 56–59. https://doi.org/10.2991/icclas- 18.2019.15 [40] R. Kumar, S. A. Karabenick, J. H. Warnke, S. Hany, and N. Seay, “Culturally Inclu-sive and Responsive Curricular Learning Environments (CIRCLEs): An exploratory sequen-tial mixed-methods approach,” Contemp. Educ. Psychol., vol. 57, pp. 87–105, 2019. https://doi. org/10.1016/j.cedpsych.2018.10.005 [41] H. Lee and Y. J. Park, “Mobile use, personal values, and connectedness with civic life,” Soc. Sci. J., vol. 55, no. 1, pp. 52–61, 2018. [42] S. Cuban, "Transnational families, I.C.T.s and mobile learning," Int. J. Lifelong Educ., vol. 33, no. 6, pp. 737–754, 2014. https://doi.org/10.1080/02601370.2014.963182 [43] A. Ingrams, “Mobile phones, smartphones, and the transformation of civic behavior through mobile information and connectivity,” Gov. Inf. Q., vol. 32, no. 4, pp. 506–515, 2015. https://doi.org/10.1016/j.giq.2015.07.007 [44] B. Whitacre, “Fixed broadband or mobile: What makes us more civically engaged?,” Telemat. Informatics, vol. 34, no. 5, pp. 755–766, 2017. https://doi.org/10.1016/j.tele.2017. 02.006 [45] S. Isaac and W. B. Michael, Handbook in Research and Evaluation. San Diego, CA: Robert R. Knapp, 1971. [46] R. Maslowski and A. J. Visscher, “Formative evaluation in educational computing re-search and development,” J. Res. Comput. Educ., vol. 32, no. 2, pp. 239–239, 1999. [47] L. Retief, M. Potgieter, and M. Lutz, “The usefulness of the rasch model for the re-finement of likert scale questionnaires,” African J. Res. Math. Sci. Technol. Educ., vol. 17, no. 1–2, pp. 126–138, 2013. https://doi.org/10.1080/10288457.2013.828407 iJIM ‒ Vol. 15, No. 03, 2021 95 https://doi.org/10.1177/0255761414533308 https://doi.org/10.4018/978-1-7998-1486-3.ch005 https://doi.org/10.1108/aaouj-07-01-2012-b004 https://doi.org/10.1080/03098265.2016.1185772 https://doi.org/10.1080/03098265.2016.1185772 https://doi.org/10.2991/icclas-18.2019.15 https://doi.org/10.2991/icclas-18.2019.15 https://doi.org/10.1016/j.cedpsych.2018.10.005 https://doi.org/10.1016/j.cedpsych.2018.10.005 https://doi.org/10.1080/02601370.2014.963182 https://doi.org/10.1016/j.giq.2015.07.007 https://doi.org/10.1016/j.tele.2017.02.006 https://doi.org/10.1016/j.tele.2017.02.006 https://doi.org/10.1080/10288457.2013.828407 Paper—Interactive Mobile Technologies on Civic Education Learning in Higher Education [48] N. Berggren and T. Nilsson, “Globalization and the transmission of social values: The case of tolerance,” J. Comp. Econ., vol. 43, no. 2, pp. 371–389, 2015. [49] C. Gan, H. Li, and Y. Liu, “Understanding mobile learning adoption in higher educa-tion An empirical investigation in the context of the mobile library,” Electron. Libr., vol. 35, no. 5, pp. 846–860, 2017. https://doi.org/10.1108/el-04-2016-0093 [50] J. I. Holden, “Mobile inquiry-as-play in mathematics teacher education,” Horiz., vol. 24, no. 1, pp. 71–81, 2016. [51] L. L. Minicozzi, “iPads and pre-service teaching: exploring the use of iPads in k-2 class- rooms,” Int. J. Inf. Learn. Technol., vol. 35, no. 3, pp. 160–180, 2018. https://doi.org/10. 1108/ijilt-05-2017-0032 [52] C. Mitchell and J. D. Turner, “Only one ipad: Preparing pre-service teachers to teach digital literacies in under-resourced elementary schools,” Lit. Res. Pract. Eval., vol. 9, pp. 43–59, 2018. https://doi.org/10.1108/s2048-045820180000009004 [53] I. Lestari, A. Maksum, and C. Kustandi, “Mobile learning design models for State Uni-ver- sity of Jakarta, Indonesia,” Int. J. Interact. Mob. Technol., vol. 13, no. 9, pp. 152–171, 2019. https://doi.org/10.3991/ijim.v13i09.10987 [54] P. Dorouka, S. Papadakis, and M. Kalogiannakis, "Tablets and apps for promoting ro-botics, mathematics, S.T.E.M. education and literacy in early childhood education," Int. J. Mob. Learn. Organ., vol. 14, no. 2, pp. 255–274, 2020. https://doi.org/10.1504/ijmlo.2020. 10026334 [55] M. Kalogiannakis, G. Nirgianaki, and S. Papadakis, S. “Teaching magnetism to pre-school children: The effectiveness of picture story reading,” Early Childhood Education Journal, 46(5), 535-546, 2018. https://doi.org/10.1007/s10643-017-0884-4 [56] D. F. Chamorro-Vela et al., “Recommendation of Mobile Applications based on social and contextual user information,” Procedia Comput. Sci., vol. 110, no. 2016, pp. 236–241, 2017. 9 Authors Muhammad Japar is a Professor on Pancasila and Civic Education, Faculty of So- cial Science, Universitas Negeri Jakarta. He is also the vice dean of the finance and staffing department on Faculty of Social Science. Yuyus Kardiman is a Lecture on Pancasila and Civic Education, Faculty of Social Science, Universitas Negeri Jakarta. He often writes books related to civic education learning. Raharjo is a Lecture on Pancasila and Civic Education, Faculty of Social Science, Universitas Negeri Jakarta. Dini Nur Fadhillah is a Master Degree Student on Pancasila and Civic Education, Universitas Sebelas Maret. Syifa Syarifa is an Undergraduate Student on Pancasila and Civic Education, Fac- ulty of Social Science, Universitas Negeri Jakarta. Article submitted 2020-08-03. Resubmitted 2020-11-20. Final acceptance 2020-11-20. Final version pub- lished as submitted by the authors. 96 http://www.i-jim.org https://doi.org/10.1108/el-04-2016-0093 https://doi.org/10.1108/ijilt-05-2017-0032 https://doi.org/10.1108/ijilt-05-2017-0032 https://doi.org/10.1108/s2048-045820180000009004 https://doi.org/10.3991/ijim.v13i09.10987 https://doi.org/10.1504/ijmlo.2020.10026334 https://doi.org/10.1504/ijmlo.2020.10026334 https://doi.org/10.1007/s10643-017-0884-4