International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol  17 No  15 (2023) iJIM | Vol. 17 No. 15 (2023) International Journal of Interactive Mobile Technologies (iJIM) 103 iJIM | eISSN: 1865-7923 | Vol. 17 No. 15 (2023) | JIM International Journal of Interactive Mobile Technologies Sureephong, P., Chernbumroong, S., Sangamuang, S., Sirasakamol, O., Intawong, K.,Puritat, K. (2023). Enhancing Information Literacy for Spotting Fake News: A Study on the Efficacy of a Serious Game for M-Learning Across Different Age Groups. International Journal of Interactive Mobile Technologies (iJIM), 17(15), pp. 103–119. https://doi.org/10.3991/ijim.v17i15.40865 Article submitted 2023-04-26. Resubmitted 2023-06-12. Final acceptance 2023-06-12. Final version published as submitted by the authors. © 2023 by the authors of this article. Published under CC-BY. Online-Journals.org PAPER Enhancing Information Literacy for Spotting Fake News: A Study on the Efficacy of a Serious Game for M-Learning Across Different Age Groups ABSTRACT With so many online information sources in recent years, it has become increasingly difficult to determine if the content is based on facts, half-truths, or lies. As a result, the goal of this research is to propose a serious game design for learning to evaluate sources using the CRAAP test. In the game, players take on the role of librarians who must evaluate news from social media and newspapers, determine whether it is fake or true, and then inform the people of the city. During their efforts to make the correct decision, the players are able to observe and learn about the impact of fake news on the community and the city as a result of their deci- sions. To evaluate the game, we did a randomized online field study, including quantitative research based on pre-posttests involving 351 participants. The results revealed that using a serious game of “How to Spot Fake News” can improve the knowledge of information literacy needed to evaluate online sources of information. Finally, we provide preliminary evidence that gaming improves people’s ability to recognize and resist misinformation. KEYWORDS m-learning applications, life-long m-learning, information literacy, serious game, game- based learning 1 INTRODUCTION Online learning has evolved as a significant alternative and important techno- logical resource for learners and other individuals seeking knowledge and infor- mation. It relies on the internet or websites as the primary distribution method for material and interaction. Online learning offers multiple advantages, including increased access to resources, improved conditions for learning, and preparation for a knowledge-based society [1]. The outbreak of the COVID-19 pandemic has has- tened the adoption of online learning as a means of mitigating the virus spread. Pradorn Sureephong1, Suepphong Chernbumroong1, Sumalee Sangamuang1, Orasa Sirasakamol2, Kannikar Intawong1, Kitti Puritat1(*) 1Chiang Mai University, Chiang Mai, Thailand 2Rajamangala University of Technology Lanna, Chiang Mai, Thailand kitti.p@cmu.ac.th https://doi.org/10.3991/ijim.v17i15.40865 https://online-journals.org/index.php/i-jim https://online-journals.org/index.php/i-jim https://doi.org/10.3991/ijim.v17i15.40865 https://online-journals.org/ https://online-journals.org/ mailto:kitti.p@cmu.ac.th https://doi.org/10.3991/ijim.v17i15.40865 104 International Journal of Interactive Mobile Technologies (iJIM) iJIM | Vol. 17 No. 15 (2023) Sureephong et al. The World Health Organization declared COVID-19 a pandemic, leading to a sudden shift in the education system and widespread adoption of online teaching [2]. However, with the spread of online learning and the COVID-19 pandemic, the problem of info-emics has also emerged. Info-emics refers to the spread of misinfor- mation and fake news during the COVID-19 pandemic, which can be highly detri- mental as people may mistakenly believe false information and act on it [3]. During the COVID-19 pandemic, many fake cures and misleading claims about the virus were spread. For example, there were cases in which people in Israel suffered adverse con- sequences, including deaths and illnesses, after believing and following misinforma- tion on social media claiming that alcohol could cure COVID-19 [4]. Similarly, in China, herbal medicines were promoted as potential treatments for COVID-19 despite warn- ings from medical experts about their potential risks and lack of scientific evidence [5]. Other examples of misleading claims include fake remedies in Ireland, Brazil, Kenya, and North Africa, all lacking proper scientific evidence to support their effective- ness [6]. The aforementioned incidents emphasize the importance of addressing the issue of fake news and misinformation during the COVID-19 epidemic. The serious game “How to Spot Fake News” was created with funding from Thailand’s Safe and Creative Media Development Fund to counteract the spread of misinformation and improve information literacy. The fundamental goal of this project is to raise aware- ness and promote self-evaluation of fake news, as well as develop an understanding of the impact of fake news among students and the general population. To achieve these goals, the serious game focuses on improving resource evalu- ation abilities and training information literacy using the CRAAP test. It provides news in a variety of categories, including educational, economic, social, health, reli- gion and belief news. An experimental study was conducted to evaluate the game’s usefulness in improving learning outcomes and raising awareness of fake news. Finally, our research contributes to the development of successful strategies for com- bating fake news and promoting information literacy in the digital age. 2 RELATED WORK 2.1 Definition of “Fake News” and media literacy against disinformation The increase in fake news has created an urgent need for a clear definition of the term. Fake news refers to the intentional dissemination of inaccurate, deceptive, and false information disguised as trustworthy news [7]. It predominantly circulates on social media and mass media platforms and takes various forms, such as lies, distorted content, and unidentified sources. Tandoc et al [8] emphasize that fake news often imitates real news to establish credibility. It poses a significant threat as it deceives individuals by presenting misleading information and blurring the line between real and fake news [9]. To combat the spread of disinformation, media literacy plays a crucial role. Media literacy encompasses the ability of citizens to critically engage with, evaluate, and create information. It also includes understanding political, ideological, and commer- cial motives as well as hidden meanings [10]. The European Commission emphasizes the importance of critical thinking for media literacy, including the ability to access, understand, and judge contrasting features of media content while creating diverse communication contexts [11]. Efforts to improve media literacy include the develop- ment of reliable and credible competency tests. These tests, aligned with ACRL stan- dards, aim to assess students’ ability to analyze verified and credible information, critically evaluate opinion reports, and conduct effective database searches [12]. https://online-journals.org/index.php/i-jim iJIM | Vol. 17 No. 15 (2023) International Journal of Interactive Mobile Technologies (iJIM) 105 Enhancing Information Literacy for Spotting Fake News: A Study on the Efficacy of a Serious Game for M-Learning Across Different Age Groups Additionally, viewers of the television media should be able to critically evaluate its content because all kinds of media include hidden agendas and messages [13]. Developing analytical and critical skills, discovering trustworthy sources, and distin- guishing fake news from authentic information are vital in the face of technological advancements and information overload [14]. In Thailand, raising awareness of information and media literacy among young people is a priority. PRD Thailand, in collaboration with the Friedrich Naumann Foundation for Freedom and UNESCO, organized the “PRD Youth Camp on Media and Information Literacy” to foster creativity in media production, encourage fact-checking, responsible digital behavior, and the development of good digital habits, such as respon- sible content sharing and consumption on social media platforms. The camp coin- cided with the official launch of the Thai version of the UNESCO e-book, “Journalism Education and Training”, which equips educators and trainers with strategies to con- front false and misleading information through information and media literacy, social media verification, and combating online defamation. Furthermore, the Ministry of Digital Economy and Society in Thailand established the Anti-Fake News Center to combat disinformation by monitoring its spread on social media platforms [15]. 2.2 Evaluation of news and information literacy training With the proliferation of news, especially fake news, on social media platforms, consumers must be able to recognize and verify the credibility of the information they encounter. The CRAAP Model, developed by Sarah Blakeslee and librarians at California State University, provides a framework for evaluating news based on five criteria: currency, relevance, authority, accuracy, and purpose [16]. Similarly, Jim Kapoun’s checklist for website evaluation includes criteria such as objectivity, authority, currency, accuracy, and coverage [17]. However, some librarians, includ- ing Candice Dahl and Marc Meola, have raised concerns about the effectiveness of Kapoun’s criteria in evaluating sources [18], [19]. They propose alternatives, such as a contextual method that compares free websites with subscribed library con- tent, as well as verification, motive, goal, and comparison [18], [19]. LibGuides, a website-based application developed by librarians, offers a platform for organizing and promoting library collections, as well as resources for evaluating fake news [20]. Eric Novotny’s Fake News Guide within LibGuides covers various forms of fake news, including sarcasm, prejudice, bunk science, clickbait, and gossip mills [21]. Information literacy training has gained recognition in the workplace [22], emphasizing the importance of evaluating and verifying information when search- ing for and sharing news [23]. Barham and Kirton discuss an information skills course that focuses on specific products and services, general searching abilities, managing information overload, citing digital sources, and using search engines [25]. They emphasize the need for feedback and evaluation to improve skills and high- light the role of librarians in teaching [26]. Winterman, Skelton, and Abell note that many courses predominantly emphasize information searching, particularly online sources, while neglecting information organization, document management, email usage, and information production and presentation [24]. 2.3 Serious game and gamification of education The concepts of serious games and gamification have the potential to enhance learning outcomes in terms of motivation and learning performance [48], [50], https://online-journals.org/index.php/i-jim 106 International Journal of Interactive Mobile Technologies (iJIM) iJIM | Vol. 17 No. 15 (2023) Sureephong et al. making them valuable tools for more effective use in the education system [51]. The application of game-based approaches to address the needs of multi-learners [49] and promote news literacy has been utilized in academic libraries [27], [35]. Games offer a promising solution to tackle the lack of motivation and participation among youth in media literacy [28], [29] and news consumption [30], [31], [32]. Throughout history, various types of games, such as quizzes, crosswords, and numeral challenges, have been employed to capture reader attention and encourage news consumption [33], [34]. Games provide an engaging way to introduce news literacy, presenting journalistic values, practices, and rules in a tangible manner rather than as abstract concepts [33]. In fact, Bogost and colleagues [33] argue that news literacy and game design rely on similar skills, such as revealing inequality through transparency and understanding complex information systems. To engage young participants in news literacy games, a study group of six partic- ipants aged 10–14 years was formed. The study began with instructions and a game for participants to choose from six popular games. Ultimately, they chose Monopoly and Buffalo. The participants, who were divided into two teams, shared their expe- riences as game creators or players and as news consumers. Discussions revolved around topics such as fake news, gaming, and browsing habits. The goal was to uncover any misconceptions and assess the participants’ current understanding of fake news while presenting the study group’s learning objectives. During the study, participants learned about the importance of news literacy and the core principles of game development through online examples and resources. One of the games developed, called “Fakeopoly”, focused on the creation and spread of fake news and sheds light on how fake news spreads and how readers react after consuming and sharing it. However, another game called Lying Geese did not emphasize the cre- ation, dissemination, or consumption of fake news, potentially hindering a deeper understanding of these situations [36]. Notably, there is a mobile application called “FakeYou” that can be played online on a variety of devices, including tablets, mobile phones, and personal computers. FakeYou uses a gamified approach to improve consumers’ ability to identify fake news. It encourages players to critically examine the trustworthiness of headlines and provides an interactive experience by creating realistic fake headlines. The game employs a “learning by doing” technique in which players create fake head- lines and have to select the correct headline from three options. Only one headline has its origins in the newspaper article, while the other two are distractors [37]. In addition to FakeYou, another game called Factitious, developed by Grace & Hone [38], challenges players’ journalism skills. It employs two primary methods to pro- mote critical thinking regarding fake news. The first method focuses on examin- ing signs and indicators of fake news and urging the participants to think critically about the source and content of news fragments. The game emphasizes the gray area between real and fake news, enabling players to differentiate between the two types of content and developing curiosity both within and outside the classroom. The second method is acquiring information to combat fake news. The game’s goal is to survey an online audience of readers in order to acquire a better understanding of their perception of ambiguous news content. This feature makes it a valuable tool for clarifying the meaning behind news producers’ content [38]. Overall, integrating serious games and gamification into education, particu- larly in the context of news literacy, provides several benefits. These approaches have the potential to intrigue and encourage students while also improving crit- ical thinking abilities and creating a better knowledge of news consumption and media literacy. https://online-journals.org/index.php/i-jim iJIM | Vol. 17 No. 15 (2023) International Journal of Interactive Mobile Technologies (iJIM) 107 Enhancing Information Literacy for Spotting Fake News: A Study on the Efficacy of a Serious Game for M-Learning Across Different Age Groups 3 DEVELOPING A GAME TO TRAIN INFORMATION LITERACY FOR COMBATING FAKE NEWS In order to develop a game, we employed the MDA framework [39], [40] as the guideline for our approach in the design process. The original concept of MDA consists of three components of game design: mechanics (M), dynamics (D), and aesthetics (A). The key part of our core idea is to build engagement and replayability [41] in game design. In the game titled “How to Spot Fake News”, we designed the game to influence the replay value of players in order to provide many alternative scenarios of fake news affected by the player’s choice for continuous play value. In addition, we described each step in the development of our approach as follows: 3.1 Determining the game design concept To define the learning goal of our game design, we followed the framework of knowledge expert co-creation [42], brainstorming the subject matter experts in the field of information literacy who provide the learning objective for our game design. Essentially, we sought subject matter experts in information literacy, librarians, and game designers to collaborate in order to grasp the target player’s learning purpose and strike a balance between the learning goal and the fun component. To identify the learning objectives, we began by employing the CRAAP test [10] to validate the fake news taught in university information literacy courses. The target audience for our approach is university students with a basic level of information literacy who must play the complex gameplay mechanics. An example of our subject matter expert brainstorming is shown in Table 1. Table 1. Subject matter experts involved in the game design Subject Matter Expert Role of Knowledge Our Case Study Librarian, Lecturer Expert in information literacy as a librarian primarily concerned with both teaching and evaluating students’ sources Librarian and Lecturer in University Player A group of players who employ the system. The selection of participants for this game’s focus group Students, game tester Game designer Expert in applying Game Designer principles to the design, development, and assessment of educational games. Game designer 3.2 Defining the core game flow and mechanics The game’s key concept is that it gives players authority over the news that is pub- lished in Thailand’s National Library. The players take on the role of a librarian who is tasked with evaluating news from social media and newspapers, determining which is fake and which is not, and then informing the residents of the city. The players can then see and learn about the negative effects of fake news on the community and city. In terms of the game’s core loop and basic game mechanics, the primary tasks show the direction of the game and what the player will do subsequently. The primary loop and mechanics of our game are depicted in Figures 1 and 2. Our game’s main game loop consists of the following phases: walking from home to work, conversing with people and society, observing the community and city, walking to work, and evaluat- ing the source of news before publishing. Every step is described further below. https://online-journals.org/index.php/i-jim 108 International Journal of Interactive Mobile Technologies (iJIM) iJIM | Vol. 17 No. 15 (2023) Sureephong et al. Fig. 1. The activity flow of the core game loop 1. Walk from home to work: Through a simulation game, we designed our first activity flow to learn about the impact of fake news on the daily lives of people in the city. The player begins at home and travels to work through a rural area, hos- pital, city, and business community before arriving at the workplace. Furthermore, the player has a seven-day period before his or her performance is evaluated by the National Library of Thailand and a decision is made on whether or not the player gets fired from the position of librarian. 2. Conversation with people and the society: The game mechanics in this activity flow were developed for the player to learn through talking with people in society from every sector of the city to understand what people think about news, the moods of other people, and what happens to them. In addition, we included people who offered news literacy expertise based on the CRAAP test in every sector of the city, as shown in Figure 2. The purpose of these mechanics is to let the player understand the impact of fake news on people‘s views. These thoughts motivate people to take action, which can have both beneficial and harmful consequences. Fig. 2. Men in black who provide the knowledge of the CRAAP test https://online-journals.org/index.php/i-jim iJIM | Vol. 17 No. 15 (2023) International Journal of Interactive Mobile Technologies (iJIM) 109 Enhancing Information Literacy for Spotting Fake News: A Study on the Efficacy of a Serious Game for M-Learning Across Different Age Groups 3. Observe the community and city: The goal of this game mechanism is to imitate the influence of fake news and its direct impact on society in the city and everywhere users go based on the news published by the player. Similar to the pre- vious mechanics, the player is able to walk around the city and observe the social community to learn more about the widespread influence of fake news. However, these mechanics constitute the most significant notion, as they enable the player to become aware of the impact of fake news as well as the fact that even one false publishing of fake news can spread far and quickly, having a serious impact on the community, as shown in Figure 3. Fig. 3. An example of the situation in the city based on player verification in the news 4. Evaluating the source of news before publishing: This activity is the most significant game mechanism for users to learn how to utilize the CRAAP test to eval- uate news. In summary, the CRAAP test, as illustrated in Table 2, is a guideline for determining the source of news based on currency, relevance, authority, accuracy, and purpose. In terms of gameplay, each day, the player is tasked with proving the six news items on their table based on a range of categories. Before releasing an article of news to the residents of the city, the player must carefully evaluate and stamp which ones are fake and which are not. Figure 4 depicts the gameplay of this exercise. Fig. 4. (Continued) https://online-journals.org/index.php/i-jim 110 International Journal of Interactive Mobile Technologies (iJIM) iJIM | Vol. 17 No. 15 (2023) Sureephong et al. Fig. 4. Example of gameplay for players to verify the news Table 2. Subject matter experts involved in the game design CRAAP Test Evaluation Currency: The timeliness of the information – When was the news posted or published? – Is the news current or out of date on the topic? – Has the information been revised or updated? Relevance: The coverage and depth of the content and its importance for your needs – Have you looked at a variety of other sources before determining you will use this one? – Who is the intended audience? – Would you be comfortable using this source for a research paper? – Does the news relate to your topic or answer your question? Authority: The source of the information – Who wrote the news, and what are their qualifications? – Has the author published works in a more traditional format, like books or journal articles? – Is the author associated with an organization? – What does the URL reveal about the author or the source? Accuracy: The reliability and correctness of the information – Are there citations for the news provided so that you can tell where the news came from? – Has the news been reviewed or referred to? – Is the content of the news free of emotion? – Are there any spelling, grammar, or other typographical errors? Purpose: Reason the news exists. Presence of bias or prejudice – What is the purpose of the news? To inform? To teach? To sell? To entertain? To persuade? – Are there political, ideological, cultural, religious, institutional, or personal biases? – Is the information factual? Opinion? Propaganda? How to Spot Fake News was developed with the Unity game engine (https://unity. com/),which we built in two versions: PC desktop and Web-based versions (http:// lis.human.cmu.ac.th/fakenews). However, we also developed the server for collect- ing data and monitoring the behavior of players. The server was developed with PHP 7.0 and MySQL. https://online-journals.org/index.php/i-jim https://unity.com/ https://unity.com/ http://lis.human.cmu.ac.th/fakenews http://lis.human.cmu.ac.th/fakenews iJIM | Vol. 17 No. 15 (2023) International Journal of Interactive Mobile Technologies (iJIM) 111 Enhancing Information Literacy for Spotting Fake News: A Study on the Efficacy of a Serious Game for M-Learning Across Different Age Groups 4 PURPOSE OF THE STUDY AND RESEARCH QUESTIONS Thailand’s Safe and Creative Media Development Fund funded the serious game “How to Spot Fake News”. Furthermore, the funds for this project are intended to raise awareness, facilitate self-evaluation of fake news, and provide an understand- ing of the impact of fake news on students and the general public. As a result of this, a serious game based on the CRAAP test was developed to teach information literacy as well as improve source evaluation skills for combat- ing online information and identifying fake news. Furthermore, the serious game provided five different categories of news: educational news, economic news, social news, health news, and religion and belief news. To verify the effectiveness of the game, we looked at how it affected those learn- ing objectives as well as people’s knowledge of fake news. Additionally, the experi- ment investigated the following research questions: 1. Does the serious game (How to Spot Fake News) improve knowledge of informa- tion literacy, resulting in spotting fake news? (RQ1). 2. Does the serious game “How to Spot Fake News” improve the knowledge of infor- mation literacy to spot fake news in five categories: educational news, economic news, social news, health news, and religion and belief news? (RQ2). 3. Does the serious game “How to Spot Fake News” improve the knowledge of infor- mation literacy to spot fake news in all age groups of online participants? (RQ3). 5 RESEARCH METHODOLOGY In this study, we employed the research and development (R&D) method. The pro- cess of developing and producing a specific product while also assessing its effective- ness is referred to as research and development. This method normally serves two purposes: (1) developing a product and (2) evaluating its effectiveness. The objective of this study was to design and implement a serious game targeted at improving information literacy and detecting fake news. To evaluate the game’s effectiveness, we gathered quantitative data from 351 participants of various ages. This data was acquired to answer the three research questions. 5.1 Participants Participants were recruited through information on the official website of the University and Organization of Safe and Creative Media Development Fund. As such, we also promoted the official website on Facebook and Google Ads in order to provide information to anyone with an internet connection who could visit the game’s website and participate. At the beginning, the game asked permission for a voluntary in-game (pre-post) survey related to scientific research for a period of two months. From that method, we collected N = 351 responses over the one-month period that followed it, which was completed by pairing pre-post responses with the information used for scientific research. Please note that we only collected infor- mation from players who fully completed the gameplay and who agreed that their information could be used. The study was approved by the Chiang Mai University Research Ethic Committee (CMU REC No. 64/092). The demographics of the study participants are shown in Table 3. https://online-journals.org/index.php/i-jim 112 International Journal of Interactive Mobile Technologies (iJIM) iJIM | Vol. 17 No. 15 (2023) Sureephong et al. Table 3. Demographics of the online participants N Percent Gender Male 185 52.00% Female 171 48.00% Total 356 Age 20–29 210 58.98% 30–49 126 35.39% 40–49 20 5.61% Total 356 Education Bachelor 305 85.67% Master 51 14.32% Total 356 5.2 Procedure and measurement The online participants who were accepted as voluntary in-game participants for scientific research were asked to take the in-game module pre- and post-tests after fin- ishing the gameplay. The pre-posttests focused on the ability to use information literacy. The idea of a key dependent variable measured in the survey was the respondents’ ability to evaluate the news on social networks within five news categories: educational news, economic news, social news, health news, and religion and belief news. During the gameplay, participants were asked to verify whether the news was fake or not by providing six news stories per day in the game module and randomly selecting from the five news categories. Then, after the participants finished all levels of the gameplay mod- ule “How to Spot Fake News”, they were asked to take the same post-test as in the game module. Please note that the content of the game was created and collected based on real online news by the lecturer in the Department of Library and Information Science. 6 DATA ANALYSIS AND RESULTS 6.1 Results of the overall knowledge of information literacy regarding fake news spotting (RQ1) We obtained the data consisting of the scores of the online participants evaluat- ing the news within five news categories. We collected the data from the online par- ticipants, who evaluated forty news stories in five categories in total. The summary of the pre-test and post-test results based on the total scores of the news evaluation experiment is shown in Table 4. Additionally, the data from the pre-test and post-test were examined using the Kolmogorov-Smirnov (KS) test, which revealed that none of the data sets had a normal distribution. Thus, we applied the non-parametric method of the Wilcoxon signed rank test for the news evaluation skills with a paired sample using the SPSS program. In order to verify the RQ1, Table 5 shows a p-value < 0.001, which means that there was a highly significant difference in terms https://online-journals.org/index.php/i-jim iJIM | Vol. 17 No. 15 (2023) International Journal of Interactive Mobile Technologies (iJIM) 113 Enhancing Information Literacy for Spotting Fake News: A Study on the Efficacy of a Serious Game for M-Learning Across Different Age Groups of the statistical evidence to clarify that the serious game of “How to Spot Fake News” can improve the overall knowledge of information literacy in spotting fake news. Table 4. Results of the pre-test and post-tests on news evaluation skills Group N Pre-Test (SD) Post-Test (SD) Online Participants 356 (185 male / 171 female) 14.37 (4.23) 18.50 (4.41) Table 5. Verification results of t-test assumptions for the pre-test and post-test Mean Std. Deviation T-Value P-Value Pre-Post (Immediately) –4.12536 5.89 –13.102 0.000 6.2 Results of fake news spotting of five news categories (RQ2) There is a variety of news with different patterns of fake news, and in fact, people do not have equal information literacy for each type of news. Thus, to clarify the answer to RQ2, “Does the serious game ‘How to Spot Fake News’ improve the knowledge of infor- mation literacy to spot fake news in five categories: educational news, economic news, social news, health news, and religion and belief news?” we also separately collected the data from the pre-test and post-test based on each type of news. We considered using the non-parametric Wilcoxon signed rank test verification similar to RQ1 because the data didn’t show a normal distribution. Please note that the calculation of the pre-test is also based on the same method as for RQ1, but used separately for each type of news. The results of the pre-test and the post-test are shown in Figure 5 and Table 6, which show the p-value separately for each type of news. Based on the results in Table 7, education-, health-, religion and belief-related news have a p-value of 0.000, which shows a signifi- cant difference in improving knowledge of information literacy. On the other hand, the results of economic and social news have a p-value of .206 and .181, respectively, which shows no significant difference in improving knowledge of information literacy. Fig. 5. Box chart of the results for the pre-test and post-test on news evaluation skills for each news category https://online-journals.org/index.php/i-jim 114 International Journal of Interactive Mobile Technologies (iJIM) iJIM | Vol. 17 No. 15 (2023) Sureephong et al. Table 6. Results of the pre-test and post-test on news evaluation skills for each news category Group Type of News N Pre (SD) Post (SD) Online Participants Education 351 3.14 (2.03) 4.44 (1.71) Economy 351 3.11 (1.88) 3.31 (2.10) Social 351 3.11 (1.81) 3.31 (1.71) Health 351 2.47 (1.69) 3.70 (2.04) Religion & Belief 351 2.52 (1.66) 3.72 (1.94) Table 7. Results of the t-test assumptions for the pre-test and post-test Group Type of News Mean Std. Deviation T-value P-value Online Participants Education –1.29 2.67 –9.10 .000 (*sig) Economy –0.20 2.83 –1.33 .182 Social –0.20 2.36 –1.60 .110 Health –1.22 2.63 –8.75 .000 (*sig) Religion & Belief –1.19 2.69 –8.65 .000 (*sig) Note: *p-value less than 0.001. 6.3 Results of spotting fake news based on aging (RQ3) In a study by Allcott and Gentzkow [43], they discovered that the ability to dis- tinguish fake news headlines from true ones during the U.S. election increased with age. Similarly, we collected data on different age groups to address RQ3: “Does the serious game ‘How to Spot Fake News’ improve knowledge of information literacy to identify fake news in all age groups of online participants?” The experimental results consisted of a pre-test and a post-test, as presented in Tables 8 and 9, where the p-value was calculated separately for each age group. Based on the results shown in Table 9, the age groups of 20–29 and 30–39 exhib- ited a p-value of 0.000, indicating a significant difference in terms of improved knowledge of information literacy. However, the age group of 40–49 had a p-value of 0.176, suggesting that there was no significant difference in terms of statistical evi- dence to support the claim that the serious game can enhance knowledge of infor- mation literacy in this age group. Therefore, the serious game appears to be effective in improving knowledge of information literacy only in the age groups of 20–29 and 30–39. Table 8. Results of the pre-test and post-test of age groups Group Age N Pre (SD) Post (SD) Online Participants 20–29 210 13.77 (4.34) 19.32 (4.14) 30–39 116 15.17 (3.72) 17.21 (4.50) 40–49 25 15.80 (4.72) 17.60 (4.71) https://online-journals.org/index.php/i-jim iJIM | Vol. 17 No. 15 (2023) International Journal of Interactive Mobile Technologies (iJIM) 115 Enhancing Information Literacy for Spotting Fake News: A Study on the Efficacy of a Serious Game for M-Learning Across Different Age Groups Table 9. Results of the pre-test and post-test on news evaluation skills in various age ranges Group Age Mean Std. Deviation T-value P-value Online Participants 20–29 –5.55 5.54 –14.502 .000 (*sig) 30–39 –2.04 5.65 –3.891 .000 (*sig) 40–49 –1.80 6.45 –1.394 .176 Note: *p-value less than 0.001. 7 DISCUSSION AND FINDINGS The results of our study indicate that the serious game “How to Spot Fake News” can improve knowledge of information literacy regarding spotting fake news (RQ1). This serious game provides evidence for the efficacy of the tool in training information literacy, specifically related to fake news. Additionally, our results are consistent with the findings of previous studies related to the Bad News game [44] and [52], which significantly reduced the perceived reliability of news in online misinformation [46] and [47], and the serious game MathE [45], which also confirmed an improvement in a more critical attitude toward online information evaluation and judgment. However, we also collected information literacy data for every category of news (RQ2). Surprisingly, the results revealed a substantial impact only on the evaluation of education, health, and religion and belief-related news, but not on economic and social news, which the researchers had not anticipated. Based on these results, we determine that knowledge of the CRAAP test is insufficient for analyzing economic news as it requires economic and financial literacy. Similarly, we believe that social news is subject to certain biases, leading participants to swiftly evaluate and judge such news based on prior knowledge and life experiences. Regarding the effect of age groups (RQ3), we discovered that our approach can only improve understanding of information literacy in the age groups of 20–29 and 30–39, specifically in the categories of education, health, and religion and belief. However, it had no effect on the age group of 40–49 in any category. In conclusion, our approach supports previous research results, as it has been proven to be beneficial for improving information literacy related to spotting fake news for a wide range of age groups between 20–29 and 30–39, specifically in the cat- egories of education, health, and religion and belief. However, an alternate approach would be more appropriate for improving information literacy in the 40–49 age group. 8 CONCLUSIONS, LIMITATION AND FUTURE DIRECTIONS We proposed a serious game for improving the knowledge of information liter- acy by spotting fake news. The results show that there was a significant difference in terms of information literacy when judging the news on social media for online participants using the serious game titled “How to Spot Fake News”. In addition, the in-depth analysis of other results shows that the online participants aged 20–29 show a significant difference in information literacy regarding news evaluation based on CRAAP in every category of news. The age group of 30–39 shows a signifi- cant difference in information literacy regarding news evaluation based on CRAAP only in connection with the news categories of education, health, and religion and belief, but not for the categories economy and social news. In summary, this study https://online-journals.org/index.php/i-jim 116 International Journal of Interactive Mobile Technologies (iJIM) iJIM | Vol. 17 No. 15 (2023) Sureephong et al. shows that it is worth implementing a serious game applied to improving knowl- edge of information literacy for spotting fake news for online participants. This study was based on quantitative data collected from online participants in Thailand. In addition, the scope of this work was limited to a small sample size for each age group (20–29) (n = 351), 30–39 (n = 116), 40–49 (n = 25), and online partici- pants located in Thailand. In addition, the game interface in this research was only available in Thai, which means that for researchers not familiar with the Thai lan- guage, the interface and the study results need to be adjusted. Moreover, the context of the news was specified for Thailand, so this context could vary depending on the country in which it is applied. Thus, the results of the study cannot be generalized beyond the population and context specified in our approach. In the future, we plan to conduct further studies on the same topic with larger sam- ple sizes and other types of data in order to improve the overall stability of the data collection. Secondly, we plan to conduct a mixed-methods study to collect both qual- itative and quantitative information in order to understand the behavior of partici- pants judging various news categories and their awareness of the impact on the city. Finally, we plan to implement a game system in order to collect more information for analyzing participant behavior. For example, the average time spent judging the dif- ferent news categories, completing the game, or interacting with the people in the city. 9 ACKNOWLEDGMENT The authors would like to express their gratitude to Thailand’s Safe and Creative Media Development Fund for funding and this research was partially supported by Chiang Mai University. 10 REFERENCES [1] S. Appana, “A review of benefits and limitations of online learning in the context of the student, the instructor and the tenured faculty,” Int. J. E-Learn., vol. 7, no. 1, pp. 5–22, 2008. [2] S. Dhawan, “Online learning: A panacea in the time of COVID-19 crisis,” J. Educ. Technol. Syst., vol. 49, no. 1, pp. 5–22, 2020. https://doi.org/10.1177/0047239520934018 [3] P. Patwa et al., “Fighting an infodemic: COVID-19 fake news dataset,” arXiv [cs.CL], 2020. https://doi.org/10.1007/978-3-030-73696-5_3 [4] N. Karimi and J. Gambrell, “Hundreds die of poisoning in Iran as fake news suggests methanol cure for virus. Times of Israel,” Times of Israel, 2020. [5] Y. Yang, “Use of herbal drugs to treat COVID-19 should be with caution,” Lancet, vol. 395, no. 10238, pp. 1689–1690, 2020. https://doi.org/10.1016/S0140-6736(20)31143-0 [6] S. McDermott, “From baking soda to urine: How home remedies around the world were repackaged as Covid-19 ‘cures,’” TheJournal.ie, 16-Aug-2020. [7] L. Farmer, “News literacy and fake news curriculum: School librarians’ perceptions of pedagogical practices,” J. Media Lit. Educ., vol. 11, no. 3, pp. 1–11, 2019. https://doi. org/10.23860/JMLE-2019-11-3-1 [8] E. C. Tandoc Jr, Z. W. Lim, and R. Ling, “Defining ‘Fake News’: A typology of scholarly definitions,” Digit. Jl., vol. 6, no. 2, pp. 137–153, 2018. https://doi.org/10.1080/21670811. 2017.1360143 https://online-journals.org/index.php/i-jim https://doi.org/10.1177/0047239520934018 https://doi.org/10.1007/978-3-030-73696-5_3 https://doi.org/10.1016/S0140-6736(20)31143-0 https://doi.org/10.23860/JMLE-2019-11-3-1 https://doi.org/10.23860/JMLE-2019-11-3-1 https://doi.org/10.1080/21670811.2017.1360143 https://doi.org/10.1080/21670811.2017.1360143 iJIM | Vol. 17 No. 15 (2023) International Journal of Interactive Mobile Technologies (iJIM) 117 Enhancing Information Literacy for Spotting Fake News: A Study on the Efficacy of a Serious Game for M-Learning Across Different Age Groups [9] A. Kaplan, “‘Satire’ and ‘prank’ websites are being weaponized as fake news. Media Matters for America,” mediamatters.org, 27-Oct-2017. [Online]. Available: https://www. mediamatters.org/fake-news/satire-and-prank-websites-are-being-weaponized-fake- news [Accessed: 15-Feb-2023]. [10] P. Aufderheide, Media Literacy. A Report of the National Leadership Conference on Media Literacy. Washington, DC: Aspen Institute, Communications and Society Program, 1993. [11] T. Koltay, “The media and the literacies: Media literacy, information literacy, digital literacy,” Media Cult. Soc., vol. 33, no. 2, pp. 211–221, 2011. https://doi. org/10.1177/0163443710393382 [12] B. Boh Podgornik, D. Dolničar, A. Šorgo, and T. Bartol, “Development, testing, and valida- tion of an information literacy test (ILT) for higher education,” J. Assoc. Inf. Sci. Technol., vol. 67, no. 10, pp. 2420–2436, 2016. https://doi.org/10.1002/asi.23586 [13] T. İnan and T. Temur, “Examining media literacy levels of prospective teachers,” International Electronic Journal of Elementary Education, vol. 4, no. 2, pp. 269–285, 2017. [14] S. Andretta, Information Literacy: A Practitioner’s Guide. Witney, England: Chandos Publishing (Oxford), 2005. [15] The Government Public Relations Department, “PRD Thailand Promotes Awareness of Media and Information Literacy,” thailand.prd.go.th, 16-Feb-2020. [Online]. Available: https://thailand.prd.go.th/en/content/page/index/id/155893 [Accessed: 16-Feb-2023]. [16] S. Blakeslee, Evaluating Information: Applying the CRAAP Test. Chico, CA: Meriam Library, California State University, 2010. [17] J. Kapoun, “Teaching undergrads WEB evaluation: A guide for library instruction,” Coll. Res. Libr. News, vol. 59, no. 7, pp. 522–523, 2020. https://doi.org/10.5860/crln.59.7.522 [18] M. Meola, “Chucking the checklist: A contextual approach to teaching undergraduates web-site evaluation,” Portal, vol. 4, no. 3, pp. 331–344, 2004. https://doi.org/10.1353/ pla.2004.0055 [19] C. Dahl, “Undergraduate research in the public domain: The evaluation of non‐academic sources online,” Ref. Serv. Rev., vol. 37, no. 2, pp. 155–163, 2009. https://doi.org/10.1108/ 00907320910957198 [20] E. Bushhousen, “LibGuides,” J. Med. Libr. Assoc., vol. 97, no. 1, pp. 68–69, 2009. https://doi. org/10.3163/1536-5050.97.1.020 [21] E. Novotny, “FakeNews,” guides. libraries.psu.edu, 04-Mar-2017. [Online]. Available: https://libraries.psu.edu/search/all/fakenews [Accessed: 16-Feb-2023]. [22] B. Allan, “The many faces of information literacy,” Aust. Acad. Res. Libr., no. 10, pp. 1–5, 2002. [23] M. Secker, “A humanistic approach to information literacy training: The program at the Commonwealth Scientific and Industrial Research Organisation (CSIRO) J M Rendel Laboratory,” Rockhampton. University of New England: School of Professional Development and Leadership, 2002. [24] V. Winterman, V. Skelton, and A. Abell, “A new kind of worker,” Library and Information Update, vol. 2, no. 10, pp. 38–39, 2003. [25] L. Barham and J. Kirton, “Information literacy in a government department, Lifelong learning: Whose responsibility and what is your contribution?” in Proceedings of the Third International Lifelong Learning Conference, 2004. [26] I. McCallum and S. Quinn, “NSW Department of Agriculture. Library Services Client Survey 2000,” 2000. [27] B. Leenaraj, W. Arayaphan, K. Intawong, and K. Puritat, “A gamified mobile application for first-year student orientation to promote library services,” J. Libr.. Inf. Sci., vol. 55, no. 1, pp. 137–150, 2023. https://doi.org/10.1177/09610006211067273 [28] A. Donnelly and C. Craddock, “Information literacy at Unilever R&D,” Libr. Inf., vol. 1, 2002. https://online-journals.org/index.php/i-jim https://www.mediamatters.org/fake-news/satire-and-prank-websites-are-being-weaponized-fake-news https://www.mediamatters.org/fake-news/satire-and-prank-websites-are-being-weaponized-fake-news https://www.mediamatters.org/fake-news/satire-and-prank-websites-are-being-weaponized-fake-news https://doi.org/10.1177/0163443710393382 https://doi.org/10.1177/0163443710393382 https://doi.org/10.1002/asi.23586 https://thailand.prd.go.th/en/content/page/index/id/155893 https://doi.org/10.5860/crln.59.7.522 https://doi.org/10.1353/pla.2004.0055 https://doi.org/10.1353/pla.2004.0055 https://doi.org/10.1108/00907320910957198 https://doi.org/10.1108/00907320910957198 https://doi.org/10.3163/1536-5050.97.1.020 https://doi.org/10.3163/1536-5050.97.1.020 https://libraries.psu.edu/search/all/fakenews https://doi.org/10.1177/09610006211067273 118 International Journal of Interactive Mobile Technologies (iJIM) iJIM | Vol. 17 No. 15 (2023) Sureephong et al. [29] R. Hobbs, “Digital and media literacy: A plan of action. White Paper, Knight Foundation Commission on the Information Needs of Communities in a Democracy.” Washington, D.C.: Aspen Institute, 2010. [30] I. Campos and A. Sardo, “Teaching news literacy to children with digital games,” in Proceedings of the 17th ACM Conference on Interaction Design and Children, 2018. https://doi.org/10.1145/3202185.3210783 [31] E. Huang, “The causes of youths’ low news consumption and strategies for making youths happy news consumers,” Converg. Int. J. Res. New Media Technol., vol. 15, no. 1, pp. 105–122, 2009. https://doi.org/10.1177/1354856508097021 [32] M. B. Robb, “News and America’s kids: How young people perceive and are impacted by the news.” San Francisco, CA: Common Sense, 2017. [33] I. Bogost, S. Ferrari, and B. Schweizer, “News games: Journalism at Play.” London, England: MIT Press, 2012. [34] R. Ferrer-Conill, “Gamifying the news: exploring the introduction of game elements into digital journalism,” Karlstads Universitet, Sweden, 2018. [35] W. Arayaphan, O. Sirasakmol, W. Nadee, and K. Puritat, “Enhancing intrinsic motivation of librarian students using virtual reality for education in the context of culture heritage museums,” TEM J., pp. 620–630, 2022. https://doi.org/10.18421/TEM112-16 [36] I. Literat, Y. K. Chang, and S.-Y. Hsu, “Gamifying fake news: Engaging youth in the partic- ipatory design of news literacy games,” Converg. Int. J. Res. New Media Technol., vol. 26, no. 3, pp. 503–516, 2020. https://doi.org/10.1177/1354856520925732 [37] L. Clever et al., “FakeYou! – A gamified approach for building and evaluating resilience against fake news,” in Disinformation in Open Online Media, Cham: Springer International Publishing, 2020, pp. 218–232. https://doi.org/10.1007/978-3-030-61841-4_15 [38] L. Grace and B. Hone, “Factitious: Large scale computer game to fight fake news and improve news literacy,” in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 2019. https://doi.org/10.1145/3290607.3299046 [39] V. Aleven, E. Myers, M. Easterday, and A. Ogan, “Toward a framework for the analy- sis and design of educational games,” in 2010 Third IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning, 2010. https://doi.org/10.1109/ DIGITEL.2010.55 [40] K. Intawong and K. Puritat, “A framework of developing mobile gamification to improve user engagement of physical activity: A case study of location-based augmented reality mobile game for promoting physical health,” Int. J. Onl. Eng., vol. 17, no. 7, p. 100, 2021. https://doi.org/10.3991/ijoe.v17i07.22349 [41] R. Adellin, C. T. Khuan, and L. D. Gertrude, “Conceptual framework puzzle game with high replayability,” J. Phys. Conf. Ser., vol. 1228, no. 1, p. 012070, 2019. https://doi. org/10.1088/1742-6596/1228/1/012070 [42] P. Ariya, K. Puritat, and K. Intawong, “Knowledge expert co-creation-based conceptual framework for educational game,” in 2019 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT-NCON), 2019. https://doi. org/10.1109/ECTI-NCON.2019.8692289 [43] H. Allcott and M. Gentzkow, “Social media and fake news in the 2016 election,” J. Econ. Perspect., vol. 31, no. 2, pp. 211–236, 2017. https://doi.org/10.1257/jep.31.2.211 [44] J. Roozenbeek and S. van der Linden, “Fake news game confers psychological resis- tance against online misinformation,” Palgrave Commun., vol. 5, no. 1, 2019. https://doi. org/10.1057/s41599-019-0279-9 [45] A. Katsaounidou, L. Vrysis, R. Kotsakis, C. Dimoulas, and A. Veglis, “MAthE the game: A serious game for education and training in news verification,” Educ. Sci. (Basel), vol. 9, no. 2, p. 155, 2019. https://doi.org/10.3390/educsci9020155 https://online-journals.org/index.php/i-jim https://doi.org/10.1145/3202185.3210783 https://doi.org/10.1177/1354856508097021 https://doi.org/10.18421/TEM112-16 https://doi.org/10.1177/1354856520925732 https://doi.org/10.1007/978-3-030-61841-4_15 https://doi.org/10.1145/3290607.3299046 https://doi.org/10.1109/DIGITEL.2010.55 https://doi.org/10.1109/DIGITEL.2010.55 https://doi.org/10.3991/ijoe.v17i07.22349 https://doi.org/10.1088/1742-6596/1228/1/012070 https://doi.org/10.1088/1742-6596/1228/1/012070 https://doi.org/10.1109/ECTI-NCON.2019.8692289 https://doi.org/10.1109/ECTI-NCON.2019.8692289 https://doi.org/10.1257/jep.31.2.211 https://doi.org/10.1057/s41599-019-0279-9 https://doi.org/10.1057/s41599-019-0279-9 https://doi.org/10.3390/educsci9020155 iJIM | Vol. 17 No. 15 (2023) International Journal of Interactive Mobile Technologies (iJIM) 119 Enhancing Information Literacy for Spotting Fake News: A Study on the Efficacy of a Serious Game for M-Learning Across Different Age Groups [46] J. P. Forgas and R. East, “On being happy and gullible: Mood effects on skepticism and the detection of deception,” J. Exp. Soc. Psychol., vol. 44, no. 5, pp. 1362–1367, 2008. https:// doi.org/10.1016/j.jesp.2008.04.010 [47] N. C. Ebner et al., “Uncovering susceptibility risk to online deception in aging,” J. Gerontol. B Psychol. Sci. Soc. Sci., vol. 75, no. 3, pp. 522–533, 2020. https://doi.org/10.1093/ geronb/gby036 [48] A. K. Barianos, A. Papadakis, N. Vidakis, “Content manager for serious games: Theoretical framework and digital platform,” Adv. Mobile Learn. Educ. Res., vol. 2, no. 1, pp. 251–262, 2022. https://doi.org/10.25082/AMLER.2022.01.009 [49] F. Lazarinis et al., “An adaptable multi-learner serious game for learning cultural heritage,” Adv. Mobile Learn. Educ. Res., vol. 2, no. 1, pp. 201–215, 2022. https://doi. org/10.25082/AMLER.2022.01.004 [50] A.-I. Zourmpakis, M. Kalogiannakis, and S. Papadakis, “Chapter 5: A review of the litera- ture for designing and developing a framework for adaptive gamification in physics edu- cation,” in The International Handbook of Physics Education Research: Teaching Physics, M. F. Taşar and A. P. R. Heron, Eds. Melville, New York: AIP Publishing, 2023, pp. 1–26. https:// pubs.aip.org/books/monograph/160/The-International-Handbook-of-Physics-Education [51] M. Kalogiannakis, A. I. Zourmpakis, and S. Papadakis, “Education of preschool and ele- mentary teachers on the use of adaptive gamification in science education,” Int. J. Technol. Enhanc. Learn., vol. 14, no. 1, p. 1, 2022. https://doi.org/10.1504/IJTEL.2022.10044586 [52] A. Modirrousta-Galian and P. A. Higham, “Gamified inoculation interventions do not improve discrimination between true and fake news: Reanalyzing existing research with receiver operating characteristic analysis,” J. Exp. Psychol. Gen., 2023. https://doi. org/10.1037/xge0001395 11 AUTHORS Pradorn Sureephong is currently working as Assistant Professor in College of Arts, Media and Technology, Chiang Mai University. Ph.D. in Knowledge Management (Email: pradorn.s@cmu.ac.th). Suepphong Chernbumroong is currently working as Lecturer in College of Arts, Media and Technology, Chiang Mai University. Ph.D. in. Knowledge Management (Email: suepphong.c@cmu.ac.th). Sumalee Sangamuang is currently working as Lecturer in Computer Engineering Department, Faculty of Engineering, Chiang Mai University. Ph.D. in Computer Science (Email: sumalee.sa@cmu.ac.th). Orasa Sirasakamol is currently working as Lecturer in Electronics Engineering and Automatic Control Systems, Rajamangala University of Technology Lanna. Ph.D. in Kunming University of Science and Technology, China (Email: orsa_k@ rmutl.ac.th). Kannikar Intawong is currently working as Assistant Professor in Faculty of Public Health, Chiang Mai University. Ph.D. in Computer Science (Email: kannikar.i@ cmu.ac.th). Kitti Puritat is currently working as Assistant Professor in Department of library and information science, Faculty of Humanities, Chiang Mai University. Ph.D. in Knowledge management Chiang Mai University (Email: kitti.p@cmu.ac.th). https://online-journals.org/index.php/i-jim https://doi.org/10.1016/j.jesp.2008.04.010 https://doi.org/10.1016/j.jesp.2008.04.010 https://doi.org/10.1093/geronb/gby036 https://doi.org/10.1093/geronb/gby036 https://doi.org/10.25082/AMLER.2022.01.009 https://doi.org/10.25082/AMLER.2022.01.004 https://doi.org/10.25082/AMLER.2022.01.004 https://pubs.aip.org/books/monograph/160/The-International-Handbook-of-Physics-Education https://pubs.aip.org/books/monograph/160/The-International-Handbook-of-Physics-Education https://doi.org/10.1504/IJTEL.2022.10044586 https://doi.org/10.1037/xge0001395 https://doi.org/10.1037/xge0001395 mailto:pradorn.s@cmu.ac.th mailto:suepphong.c@cmu.ac.th mailto:sumalee.sa@cmu.ac.th mailto:orsa_k@rmutl.ac.th mailto:orsa_k@rmutl.ac.th mailto:kannikar.i@cmu.ac.th mailto:kannikar.i@cmu.ac.th mailto:kitti.p@cmu.ac.th