Fact-checking as digital media literacy in higher education ISSN: 1504-4831 Vol 18, No 1 (2022) https://doi.org/10.7577/seminar.4689 ©2022 (Siling Tekoniemi, Sirkku Kotilainen, Mari Maasilta, Kirsti Lempiäinen). This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. Fact-checking as digital media literacy in higher education Siling Tekoniemi Lapland University Email: siling.tekoniemi@gmail.com Sirkku Kotilainen Tampere university Email: sirkku.kotilainen@tuni.fi Mari Maasilta Lapland University Email: mari.maasilta@unilapland.fi Kirsti Lempiäinen Lapland University Email: kirsti.lempiäinen@unilapland.fi Abstract Information disorder is a growing phenomenon, and fact-checking has long been recognized as an effective practice to evaluate media. Still, knowledge about fact-checking is not coherent in higher education institutions, not even in the field of media education. The paper is based on a case study exploring the teaching of fact-checking in higher education as evaluation of an intercultural workshop online in the critical-pragmatic framework with mixed methods as part of ERASMUS+ project (2020 - 2022) titled as Fact Checking: European cooperation project on disinformation and fact-checking training. The case-based action research was actualized as a workshop online on fact-checking with the title “Information Disorder and Fact-checking” in master level for 10 international exchange students. Mixed methods approach produced rich data for thematic analysis. Key https://doi.org/10.7577/seminar.4689 mailto:siling.tekoniemi@gmail.com mailto:sirkku.kotilainen@tuni.fi mailto:mari.maasilta@unilapland.fi mailto:kirsti.lempiäinen@unilapland.fi Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 2 Vol. 18 – Issue 1 – 2022 findings focus on students’ learning outcomes, teaching practices and show participant’s challenges in facing unfamiliar media environments in the workshop. Findings highlight a need for developing fact-checking teaching in higher education as digital media literacy in a framework which integrates pragmatic with critical approach as digital design and other hands-on educational practices together with culture-based contextualization. Moreover, the study suggests that contents of digital media literacy need updating with fact-checking and algorithm-based communication for the recognition of technology as a counterpart in the organization of information disorder. Keywords: fact checking, digital media literacy, higher education, media education Introduction Disinformation spread over the digital social networks has been identified as a threat to democracies, economy, and to individuals. Threats such as fake news or conspiracy theories grasp communication and democracy and infect them with suspicion (e.g., Carlsson, 2021). This alters the confidence of the general public of the media and tends to undermine the freedom of press and the freedom of expression. As highlighted in the report published by the High Level Expert Group set up by the European Commission (2018), tackling misleading information requires a multidimensional approach and strategy because of its technological, legal, political and educational implications. The global economic market which drives on efficiency, adaptability and being open for free mobility of ideas and people also urges us to think about information, truth, and empowerment in our teaching (see Kreissl et al., 2015; Stein, Andreotti, Bruce & Suša, 2016; Teichler, 2019). Problems integrated to the global spread of disinformation have been mapped as information disorder (e.g., Wardle and Derakhshan, 2017) highlighting the needs for developing citizens’ skills in fact-checking as critical awareness of the differences in fake and trustworthy information. One can ask how to make that happen, for example, at schools, when the studies on fact-checking are in the evolving stage with growing demand for education for professionals like teachers (e.g., Hobbs, 2017b)? According to Hobbs (e.g.) knowledge about fact-checking is not coherent in higher education institutions, not even in the field of media education. This counts European universities as well as the disciplines giving fact checking teaching vary from country to country (Beaudreau & Frau- Meigs, 2021). For example, in Finland, all teacher education is academic producing masters. Master level media education programs include teaching of digital media literacies (e.g. Rasi, Ruokamo and Maasilta 2017) and, one can assume that they include teaching of disinformation as well. Still, academic papers mostly are lacking on how fake news and fact-checking are integrated into media education. Thus, the question is, how fact-checking could be taught as digital media literacy in higher education? Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 3 Vol. 18 – Issue 1 – 2022 This paper is discussing the question with the focus on education, namely media education which mostly concerns education, media, communication and information sciences. For example, as focal points, libraries in the Member States of EU are actively working to improve the overall level of literacy and inclusion in lifelong learning through their activities and services (for example: Boosting Digital Skills and Competencies for Librarians in Europe (https://www.biblio-project.eu/). This article is based on a case study in the joint European three-year ERASMUS+ project “Fact Checking: European cooperation project on disinformation and fact-checking training” (2020 - 2022). Nova University Lisbon served as coordinator of the project having eight partners from six countries including one Portuguese press association and seven universities. In Addition to Portugal, countries involved were Finland, France, Greece, Netherlands and Poland. Context of the project is higher education in Europe. Higher education is approached from the viewpoint of examination, so higher education institutions giving bachelors or master degrees are in the focus. These include universities and in some cases polytechnics that teach bachelors programs. Project’s mapping study (Beaudreau & Frau-Meigs, 2021) highlights that fact checking policies vary a lot at European universities. What is more, there are higher education colleges operating outside universities that teach fact checking in disciplines such as Journalism or Communication Science, but these are not studied in the project. The main aim of the ERASMUS+ “Fact-checking” project was to create a new Master’s program on fact-checking for European higher education to develop the skills of media professionals and media education teachers. The new master’s program designed by the end of the project gives a study basis for the emerging profession of fact-checkers. Forming the basic knowledge for the design of the new master program three studies were conducted including 1) a survey in European universities on fact-checking teaching followed by 2) a case-based action research as a workshop on fact-checking in a digital literacy master course in 2020. Moreover, 3) a survey among European media professionals was conducted. This article is focusing on the workshop case study, practically a master level course for international university students implemented by the Finnish Team of the ERASMUS+ project. Digital Media Literacy as Fact-checking Digital literacy encourages people to access, analyze, create, reflect, and act using digital tools (e.g., Ferrari, 2013; Buckingham, 2015). Fact-checking is described as the practice of seeking factual information in order to promote its truth value. (Miller, 2020). Thus, taking fact-checking as practice relates to teaching practices. Integrating fact-checking into Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 4 Vol. 18 – Issue 1 – 2022 digital literacy teaching practice may encourage learners to practice with digital fact- checking tools and to identify information in the digital platforms. This integration was the core task to explore in the workshop on fact-checking. Renee Hobbs (2017a) finds commonalities with digital literacy and media literacy as: a) critical analysis of messages, b) communication and advocacy, c) awareness online and d) balancing risks and opportunities in using media contents (e.g., Buckingham, 2015). According to Hobbs (2020) elements of mis, dis, and mal information belong to integrated digital media literacy of today with links to all of these commonalities. Understanding of fake information is, first of all, about critical reading abilities following with other commonalities (b-d). Julian Sefton-Green, Ola Erstad and Helen Nixon (2009) are mapping digital literacy in three different frameworks based on how communication and the user is understood. Transmission framework is based on information processing and the focus is on evaluating skills of the user. Pragmatic approach researchers are interested in practices and making sense of the users engagement online. Finally, the transformative model highlights critical thinking and mindsets in dialogue by the user (e.g.). Colin Lanskhear and Michel Knobel (2015) suggest a sociocultural framework for understanding digital media literacies as multiple and contextual literacies. Sociocultural framework is the base in this study as well in reflecting a workshop that was offered to international master students and realizes through the contextualized application of Renee Hobbs' AACRA model (2017). This model helps learners to: Access the media ecosystem and digital fact-checking tools; Analyze media content using fact-checking strategies and tools; Create fact-checking reports.; Reflect participants' attitudes and critical awareness toward media and take Action to solve problems related to their lives and careers (e.g.). There are many terms concerning false, misleading, and fake information (e.g., fake news, propaganda, misinformation, and disinformation). First, fake news is defined as “news articles that are intentionally and verifiably false, and could mislead readers” (Allcott & Gentzkow, 2017). Even if the term is commonly used in everyday speech, its definition is vague (Tandoc & al., 2017) and many researchers comment that, in many cases, it is not a correct expression (e.g., Hobbs, 2017b; Wardle & Derakshan, 2017; Carlsson, 2018). The term is inadequate because it is not able to capture the complex problem of disinformation, which may involve content that is not actually, or completely “fake” but fabricated information blended with facts. Besides, there are also information practices that do not belong to the category of “news”, but includes for instance “some forms of automated accounts used for astroturfing, networks of fake followers, fabricated or manipulated videos, targeted advertising, organized trolling, visual memes, and much more” (European Commission, 2018). Moreover, the term has been used misleadingly by some politicians and their supporters, who have labeled the media coverage and news they find disagreeable as ‘fake news’ (European Commission, 2018). Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 5 Vol. 18 – Issue 1 – 2022 There is considerable debate over the most effective way to address “fake news.” Some scholars argue that services like Facebook and Google are undeniably media platforms with a responsibility to flag false stories and even alter economic incentives for publishers (and Facebook has indeed started to take some action). Others argue the solution lies in teaching greater media literacy and “emotional skepticism” to the public. Unfortunately, none of the proposed solutions are easy, and their effectiveness remains largely untested and may even backfire. For example, Hobbs (2017b) suggests that educators pay more attention to the concepts of propaganda and disinformation. Propaganda is effective communication to affect people's emotions. It can be the most important and difficult term for students to understand because it is closely related to people’s daily lives and can appear in many forms (e.g., advertising, sponsored content, or political promotion). Some forms of propaganda, such as marketing or health communication are not always harmful; sometimes, they are beneficial. (Hobbs, Kanižaj, & Pereira, 2019.) Thus, Hobbs (2020) recommends that propaganda is a significant subject for teaching practices in media education because it is related to more complex situations in real life. In addition to fake news and propaganda, Wardle and Derakshan (2017) recommend the terms misinformation, disinformation, and malinformation as the three types of information disorder. Misinformation is defined as information that is false but not intentionally, it may be caused by, for example, a journalist’s mistake or error, while disinformation is by purpose designed, presented and promoted to cause harm or to gain financial profit. Malinformation is the term used for truthful information that is spread without permission or with the purpose to cause harm. Examples of malinformation may include leaks, harassment and hate speech. (Wardle & Derakshan, 2017.) Many researchers emphasize that it is important for media literacy educators to help learners expand their knowledge on these different terms and definitions, because the knowledge can help them to better analyze and evaluate different types of information, raise awareness of all forms of media, and analyze information from different perspectives (e.g., Hobbs, 2017b; Wardle & Derakshan, 2017). Moreover, European fact-checking experts, journalists, media specialists and pedagogues highlight the importance of empowering students with critical thinking and digital information literacy skills to resist mis- and disinformation and aim to activate students to verify their social media content as most of the youth get their daily news through social media and YouTube. According to the report Towards a Better Democracy (Rautiainen, 2019), there is a direct link between education and other factors/indicators (media freedom, trust, new forms of participation) that are key to preventing the spread of misinformation. The higher the media literacy index is, the greater the trust in public structures, the less likely it is to spread conspiracy theories and misinformation, and so on. The basic understanding is that the level of education, the state of the media, the level of trust in society and the spread of new forms of participation illustrate the existence of media literacy (e.g.). Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 6 Vol. 18 – Issue 1 – 2022 Several authors have recently been calling for the integration of media literacy with technology orientation as algorithm-based digital literacy or even artificial intelligence- based literacy for increasing the public awareness on information and contents in digital platforms (e.g. Valtonen et. al, 2019; Kotilainen & al., 2021). The most important is considered the critical adjustment of media practices. Computational thinking augments media literacy from content orientation also to critical assessment of platforms and services as critical digital literacy (Kotilainen et al., 2021). Critical focus is suggested to lay, for example, on these aspects in digital platforms (Valtonen et. al, 2019): user tracking, recommenders in the usage, dynamic content creation and reinforcement learning in platforms, attention engineering and content filtering curation during the usage of digital platforms. Tracking and recommenders are about how recommender systems are tracking personal data, dynamic content creation mean, for example, how followers, news and bots are able to carry out conversations. Considering attention engineering one should reflect on how the contents are tailored and in content filtering curation the focus should be on how social media is learning to curate content that the user prefers to use (e.g.). These are important integrations in the case of identifying fake information online as news or social media postings and platform communication in general. Thus, in higher education it is calling for new understanding with media educational pedagogies together with digital media literacies as well. The workshop on fact-checking was testing contents and teaching practices at a master level. Workshop on Fact-checking The case-based action study, Workshop online on fact-checking was conducted with the title “Information Disorder and Fact-checking”. It was part of the 5-ECTS public elective course on Digital Literacy. The workshop was organized online so it was possible to involve two universities that are 680 km apart: Tampere University in the southern part of Finland and Lapland University in the northern part of the country. The participants in this study were 10 international students, aged 22 to 35. The students attended the class in both universities together as nine females and one male. Participants' origins were Finland (6), Ukraine (1), China (1), Spain (1), and Belgium (1). Their study programs ranged from information sciences to education and to Nordic studies (Tekoniemi, 2021). Educators conduct action research with students in schools to improve teaching practice (Van der Stoep & Johnston, 2009). Thus, this practice-based approach was applied because of the possibility to seek theoretical understanding of integrating fact-checking in the scope of digital literacy education in practice (e.g.). Action research consists of four cycles: planning, acting, observing, and reflecting (Kemmis & McTaggart, 2005), and it is usually conducted in more than two rounds. Due to the limited time of the ERASMUS+ project with the need to test pedagogic methods, however, the present study is a case- based action study. It carries out one workshop in this study, and the second round will be Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 7 Vol. 18 – Issue 1 – 2022 organized later during the first author’s PhD study. The workshop followed action research cycles as planning the workshop and implementing it with the limits of existing curricula at the university. Special focus was in observing through qualitative methods including researcher’s diary and reflecting, i.e. analyzing the collected data (e.g. Kemmis & McTaggart, 2005). Implementation as research followed research ethical manners from the beginning throughout the process as, for example, informing about the study, collecting research consents from participants in this way making the implementation transparent for students, and their participation to the study voluntary-based with the possibility to withdraw from the study at any time (e.g., Shamoo and Resnik, 2009) It was selected to focus fact-checking knowledge and teaching activities through Renee Hobbs' AACRA model (2017a), helping participants to: a) access the media ecosystem and digital fact-checking tools. b) analyze media content using fact-checking strategies and tools, c) create fact-checking reports, d) reflect participants’ attitudes and critical awareness toward media, and e) take action to solve problems related to their lives and careers. Activities related to the themes were arranged according to this model to serve the learning goals. The goals of this short intensive-kind workshop mostly were as such: expanding the knowledge of fact-checking and identifying information disorder with fact-checking strategies together with accessing digital fact-checking tools. Moreover, solving problems through creation and production was a goal, for which to achieve the participants were required to design a Fact-Checking Curriculum according to the target audience they chose. The workshop consists of two sessions. The first session, lasting about 1.5 hours, focused on the introduction of the workshop and the participants’ knowledge of fact-checking. Session two, lasting another 1.5 hours, focused on the experience of fact-checking practice, the development of critical thinking, and the introduction of the assignments. After the workshop, learners continued with their independent learning process for more than 100 hours through work on the assignments. The case-based action study was organized to explore methodologically these sub questions as “core issues”: Core issue 1. What are the learning outcomes of the workshop? Core issue 2. How is the AACRA model working in the fact-checking teaching practice? Core issue 3. What are participants’ challenges? Responding to the three core issues, mixed data collection methods were planned. For example, there are datasets as the workshop assignments that aimed to find out the learning outcomes of the workshop (core issue 1) and explore the relations of the teaching practices and the AACRA cycles (core issue 2); and datasets (e.g. participant’s learning diaries and open questionnaire 1 and 2) looked into the results of learning outcomes (core Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 8 Vol. 18 – Issue 1 – 2022 issue 1) and participant’s challenges and expectations (core issue 3); and also datasets (e.g. researcher’s diaries and interviews with participants) focused on the reflection of the teaching and learning processes (core issue 2) and feedbacks (core issue 3). (see Table 1). Table 1 Data for Core Issues (Tekoniemi, 2021) Methods Participant’s fact-checking reports Participant’s designing productions of a fact- checking curriculum Participant’s learning diaries Open questionnaire 1, Open questionnaire y 2 Researcher’s diaries Interviews with participants Purposes Core issues 1 & 2 Core issues 1 & 2 Core issues 1 & 3 Core issues 1 & 3 Core issue 2 & 3 Core issues 2 & 3 Thematic methods are flexible for analyzing content under different structures (Braun & Clarke, 2006), thus the mixed data was analyzed thematically. The data were initially analyzed using different codes under the planned structure. Total of 45 initial codes were found through intensive reading of the collected data several times. Then, data was analyzed to find similarities and differences among different data sets and finally to create 11 themes. After the themes were formed, they were divided into three main categories as “Learning outcomes, Teaching and learning practices and Expectations, challenges and comments”, responding to the three core issues 1-3. Under each of these categories, there were two to six themes from the relative data source. For example, Participants’ Learning Diaries are designed to students’ learning outcomes and their challenges during the course (core issues 1 and 3), however, many participants wanted to discuss their experiences in teaching activities as well in this assignment, so the data were also serving as a minor source for evaluating teaching activities in the workshop i.e., responding to core issue 2. Findings Findings based on the workshop data are mapped as learning outcomes, teaching and learning activities and challenges based on thematic analysis described above (Tekoniemi, 2021). Learning outcomes form the main in-depth findings of the study and, they are mapped into five key findings as such: the 1) students’ expanding the knowledge of fact- checking and media ecosystem; 2) strengthening students’ critical thinking; 3) students enhancing their media analyzing skills and motivation; 4) Increased experiences on digital fact-checking tools; 5) encouraging participants to solve problems through creation and production. Findings of Teaching and Learning Activities show out the importance of Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 9 Vol. 18 – Issue 1 – 2022 students' own involvement to create and dwell into knowledge. Challenges mainly concern students’ experiencing new untrustful media environments in the workshop, i.e., which they usually avoid in their everyday media usage. As learning outcomes, first (1) students show out the expanding of their knowledge of fact- checking and the media ecosystem in several data sets (e.g., from participants' learning diaries, student 1,3,7, Finland). Before the workshop, most participants reported that they had neither any knowledge nor any study experience about the information disorder phenomenon. A few participants had a little understanding of the subject, but that knowledge was very limited. After the workshop, most participants' understanding was broadened and strengthened, and their knowledge was more systematic and deeper than before (e.g., from open questionnaire 1 and 2). Second (2) strengthening students’ critical thinking was shown when they had come across disinformation in their lives, and their reactions were “anger,” “frustrate,” and “stop believing” (e.g., from open questionnaire 1) Data showed that before the workshop, people seldom took action to report harmful information. After the workshop, however, participants tried to build up good attitudes towards media, and their critical awareness was strengthened. For example, participants described (from participants' learning diaries, student 4, Belgium) their different understanding of the proper attitudes when engaging with the media. Third (3) students enhancing their media analyzing skills and motivation to analyze showed out as identifying mis-, dis- and mal-information in the workshop since most participants showed great interest in analyzing media, and many of them actively shared their judgments based on their understanding. After the workshop, data showed that participants were able to consider several perspectives in media analysis, especially when working on their assignments (e.g. from participants' learning diaries, student 1,5, Finland). Fourth (4) students increasing their experiences in digital fact-checking tools was reported in multiple ways. For example, they had never before used any fact-checking tools, and some of them did not even know that there were any fact-checking tools in their countries. After the learning experience, however, most participants had practice in using digital tools and showed great interest in doing digital fact-checking. For example, some participants described their experiences using digital tools and tried to analyze some fact-checking tools after the workshop (e.g., from participants' learning diaries, student 1, Finland and open questionnaire 2). Fifth (5) encouraging participants to solve problems through creation and production was highlighted during the workshop. Participants reflected on the challenges of fact-checking worldwide and suggested different solutions to fight against disinformation. In the Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 10 Vol. 18 – Issue 1 – 2022 assignments, they also designed different fact-checking classes to help various target audiences. For example, to solve the problem of information disorder worldwide, many recommended that more training in fact-checking should be tailored for different groups in reaction to disinformation. Findings of Teaching and Learning Activities were mostly reported in data sets of participant’s fact-checking reports and participant’s own designs of fact-checking curricula. The main teaching activities adopted in this workshop included seminars and hands-on approaches. These activities were planned to integrate fact-checking into the framework of digital literacy education. For example, two activities as Producing Fact- Checking Reports and Designing a Fact-Checking Curriculum were implemented as hands- on pedagogies, helping participants interact in AACRA learning cycles in practice (Hobbs, 2017a). Many participants investigated to find much evidence to form their judgments, when producing fact-checking reports (e.g. from participants' fact-checking reports, student 8, Spain and student 10, Ukraine). And when designing a fact-checking curriculum, most participants were dedicated to solving problems by considering their target group, reviewing the fact-checking knowledge, and overcoming challenges in producing creative teaching plans (e.g. from participant's design of a fact-checking curriculum, student 4, Belgium). Other than the hands-on pedagogies, the seminar session was also essential to link fact-checking with digital literacy education in deepening participant’s fact-checking understanding based on their assignments. For example, many participants reviewed the knowledge and pondered fact-checking elements in producing teaching plans (e.g., from participant's designing production of a fact- checking curriculum, student 4, Belgium). These activities helped participants to become involved in AACRA cycles, especially through creation. The biggest challenges were reported mostly in interviews and participant’s learning diaries, as a worry about adapting to new media environments in this workshop. Some felt uncomfortable when engaging with unfamiliar media sources. For example, participants described that they usually limited their media access to trusted sources, as they felt uncomfortable accessing unfamiliar sources and tools in the workshop. Moreover, many of them claimed that they did not get enough training in fact-checking during their academic studies in general, and they suggested that training should be given as early as possible in the degree program. Discussion The case-based action study (Tekoniemi,2021) integrated fact-checking into media education teaching practices following the Hobbs (2017a; 2017b) AACRA model in master level. The learning outcomes indicate that the approach was helping participants access, analyze, create, reflect, and act in learning digital literacies (e.g.). In addition to the fulfillment of learning goals, the design of teaching activities to achieve these goals can be Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 11 Vol. 18 – Issue 1 – 2022 considered essential. Data shows that the workshop met most participants’ expectations of learning fact-checking strategies, methods, and tools. Furthermore, it also inspired them to use academic fact-checking tools in their study, and many learners also related the learning experience with their future careers. In all, the workshop was a good example of integrating fact-checking in media education. Considering university students’ learning expectations, it tailored content and material to promote participants’ critical digital literacies and increased students’ motivation to fact-checking practices in their everyday life. Thus, following Hobbs’ model (2017a; 2017b) in higher education with the adaptation to fact-checking seems to be working in a media education approach. For developing the model further out of this local context, pedagogic culture-based contextualization should be promoted in teaching activities from the perspectives of the participants, especially when they have intercultural backgrounds. That may increase the student’s motivation in developing their own workshops when returning their home countries. Moreover, the study identified some challenges for the international students participating in the workshop. For example, the limitation of access to unfamiliar media sources and tools, and the barriers of a digital learning environment were reported in the data. Thus, future workshops should pay more attention to the promotion of participants’ access to digital tools and multiple media environments based on the different cultures of participants. Moreover, they should emphasize the practice of different forms and types of the information disorder ecosystem following a similar intercultural manner and develop contextualization to cultures of origin of the students. When discussing the challenges of students to face new media platforms in the workshop raises the question from a technological perspective as well. Several authors have recently been calling for the integration of media literacy with technology orientation as algorithm- based digital literacy for increasing the public awareness on information and contents in digital platforms (e.g. Valtonen et. al, 2019; Kotilainen & al., 2021). The most important is considered the critical adjustment of media practices including fact-checking practices. Critical focus is suggested to lay, for example, on these aspects in digital platforms (Valtonen et. al, 2019): user tracking, recommenders in the usage, dynamic content creation and reinforcement learning in platforms, attention engineering and content filtering curation during the usage of digital platforms. Tracking and recommenders are about how recommender systems are tracking personal data, dynamic content creation mean, for example, how followers, news and bots are able to carry out conversations. Considering attention engineering one should reflect on how the contents are tailored and in content filtering curation the focus should be on how social media is learning to curate content that the user prefers to use (e.g.). Conclusion The objective of this study was to increase the understanding on how fact-checking is Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 12 Vol. 18 – Issue 1 – 2022 taught as digital media literacy in higher education. Recalling the history of media education, this study shows that fact-checking can be integrated in digital media literacy for enhancing the students' own involvement to create and dwell into knowledge in a short workshop to produce learning even in strengthening students’ critical thinking, even in a short-term workshop. This kind of learning experiences were possible in practice-based curriculum design and other critical reflection in seminars. In future, student’s task to design educational games as learning activities in master level could be considered. Findings highlight a need for developing fact-checking teaching in higher education as digital media literacy in a framework which integrates pragmatic with critical approach as digital design and other hands-on educational practices. The research reflects a high learning demand by university students for the training in fact- checking and digital media literacies. Thus, there is a need for studies in future to integrate fact-checking-related subjects, e.g., propaganda and disinformation. The study suggests that contents of digital media literacy need updating with algorithm-based communication for the recognition of technology as a counterpart in the organization of information disorder. Based on the case study there is a need for developing the field of fact-checking teaching further in higher education. In general, studies on digital media literacy mostly cover education, especially teacher education or information studies and, lacking other scientific fields. Thus, there is a need to broaden research on media education to more professional perspectives covering higher education. This can be concluded based on the case study as well, for example, as a workshop online on fact-checking in master level which included participants from several scientific fields. Case study followed Hobbs’ (2017a) AACRA model to help participants achieve digital media literacies through hands-on practices. It was tailored for international university students of different study backgrounds. Thus, this kind of workshop model online can be considered as an example when designing fact-checking courses in other contexts. Not a straightforward way, but to contextualize culture-based the workshop regarding contents and teaching activities based on, for example, available teaching resources or current themes on disinformation and participants’ backgrounds. References Allcott, H. & Gentzkow, M. (2017). Social Media and Fake News in the 2016 Election. Journal of Economic Perspectives, 31(2), 211–236. https://doi.org/10.1257/jep.31.2.211 Beaudreau, J. & Frau-Miegs, D. (2021). Executive Summary: Introducing fact-checking in the curriculum at university level. In D. Frau-Miegs & J. Beaudreau (Eds.), Mapping https://doi.org/10.1257/jep.31.2.211 Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 13 Vol. 18 – Issue 1 – 2022 Fact Checking and Disinformation in Europe. Erasmus + Fact Checking Project (pp. 3–11). Unpublished report. Paris: University of Sorbonne-Nouvelle. Buckingham, D. (2015). Defining digital literacy. What do young people need to know about digital media? Nordic Journal of Digital Literacy, 10(4), 21–34. https://doi.org/10.18261/ISSN1891-943X-2015-Jubileumsnummer-03 Carlsson, U. (ed.) (2018). Medie- och informationskunnighet i den digitala tidsåldern: En demokratifråga. Kartläggning, analys och reflektioner. [Media and Information Literacy in the Digital Age: A Question of Democracy]. Göteborg: Nordicom, University of Gothenburg. Carlsson, U. (2021) Foreword. In Frau-Meigs, D., Kotilainen, S., Pathak-Shelat, M., Hoechsmann, M., Poyntz, S. (eds.) The Handbook on Media Education Research (pp. xi-xviii). Hoboken, N.J.: Wiley Blackwell. Crowe, S., Cresswell, K., Robertson, A., Huby, G., Avery, A., & Sheikh, A. (2011). The Case Study Approach. BMC Med Res Methodol, 11(100). https://doi.org/10.1186/1471-2288-11-100 European Commission. (2018). A multi-dimensional approach to disinformation. Report of the independent High level Group on fake news and online disinformation. https://ec.europa.eu/digital-single-market/en/news/final-report-high-level-expert- group-fake-newsand-online-disinformation Ferrari Anusca, 2013. A Framework for Developing and Understanding Digital Competence in Europe. Hobbs, R. (2017a). Create to learn: introduction to digital literacy. Hoboken, N.J.: Wiley Blackwell. Hobbs, R. (2017b). Teaching and Learning in a Post-Truth World, Citizens in the Making. Educational Leadership, 75(3), 26–31. Hobbs, R. (2020). Media literacy in action: Questioning the media. https://doi.org/10.1002/9781118978238.ieml0063 Hobbs, R., Kanižaj, I. & Pereira, L. (2019). Digital Literacy and Propaganda. Media Studies, 10(19), 1–7. https://doi.org/10.1002/9781118978238.ieml0144 Kemmis, S. & McTaggart, R. (2005). Participatory Action Research: Communicative Action and the Public Sphere. In: Denzin, N. and Lincoln, Y., Eds., The Sage Handbook of Qualitative Research, 3rd Edition, Sage, Thousand Oaks, 559-603. Kreissl, K. & al. (2015). ‘Will Gender Equality Ever Fit in? Contested Discursive Spaces of University Reform.’ Gender and Education, 27(3), 221–238. https://doi.org/10.1080/09540253.2015.1028903 https://doi.org/10.18261/ISSN1891-943X-2015-Jubileumsnummer-03 https://doi.org/10.1186/1471-2288-11-100 https://ec.europa.eu/digital-single-market/en/news/final-report-high-level-expert-group-fake-newsand-online-disinformation https://ec.europa.eu/digital-single-market/en/news/final-report-high-level-expert-group-fake-newsand-online-disinformation https://doi.org/10.1002/9781118978238.ieml0063 https://doi.org/10.1002/9781118978238.ieml0144 https://doi.org/10.1080/09540253.2015.1028903 Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 14 Vol. 18 – Issue 1 – 2022 Kotilainen, S., Okkonen, J., Vuorio, J., Leisti, K. (2020): “Youth Media Education in the Age of Algorithm‐Driven Social Media, In Frau-Meigs, D., Kotilainen, S., Pathak- Shelat, S., Hoechsmann, M., Poyntz, S. R. (eds.) The Handbook of Media Education Research, pp. 131-140, Wiley-Blackwell. https://doi.org/10.1002/9781119166900.ch10 Lanskhear, C., Knobel, M. (2015) ‘Digital Literacy and Digital Literacies: Policy, Pedagogy and Research Considerations for Education.’ Nordic Journal of Digital Literacy, 2006-2016, 8 – 20. https://doi.org/10.18261/ISSN1891-943X-2015-Jubileumsnummer-02 Miller, I (2020). ‘Research Guides: Journalism: Fact-Checking Sites’. Eastern Washington University. Retrieved 17 December 2020. Ponce, O.A. & Pagan-Maldonado, N. (2015). Mixed Methods Research in Education: Capturing the Complexity of the Profession. International Journal of Educational Excellence, 1(1), 111–135. Rasi, P., Ruokamo, H. & Maasilta, M. (2017). Towards a Culturally Inclusive, Integrated and Transdisciplinary Media Education Curriculum: Case Study of an International MA Program at the University of Lapland. Journal of Media Literacy Education 9(1), 22-35. https://doi.org/10.23860/JMLE-2017-9-1-3 Rautiainen, M. (Ed.) (2019). Towards a better democracy: Council of Europe's competences for democratic culture in education and teaching. https://jyx.jyu.fi/bitstream/handle/123456789/63745/978-951-39-7734-4.pdf Seroen, J-D. (2019). European cooperation project on disinformation and fact checking training. Empowering current and future media and education professionals, to identify, prevent, and combat fake news spread over digital networks. ERASMUS+ KA2 project. Lisbon: Universidade Nova de Lisboa. Sefton-Green J, Nixon, H., Erstad, O. (2009) ’Reviewing approaches and perspectives on "digital literacy".’ Pedagogies: An International Journal, 4(2), 107–125. https://doi.org/10.1080/15544800902741556 Shamoo, A. E., & Resnik, D. B. (2009). Responsible Conduct of Research. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195368246.001.0001 Stein, S., Andreotti, V. Bruce, J. & Suša, R. (2016). ‘Towards Different Conversations about the Internationalization of Higher Education.’ Comparative and International Education/Éducation comparée et international, 45(1), art. 2. https://doi.org/10.5206/cie-eci.v45i1.9281 Tandoc Jr., E.C. & Zheng, W.L. & Ling, R. (2017). Defining “Fake News”: A typology of scholarly definitions. Digital Journalism, 7(3), 1–17. https://doi.org/10.1080/21670811.2017.1360143 https://doi.org/10.1002/9781119166900.ch10 https://doi.org/10.18261/ISSN1891-943X-2015-Jubileumsnummer-02 https://doi.org/10.23860/JMLE-2017-9-1-3 https://jyx.jyu.fi/bitstream/handle/123456789/63745/978-951-39-7734-4.pdf https://doi.org/10.1080/15544800902741556 https://doi.org/10.5206/cie-eci.v45i1.9281 https://doi.org/10.1080/21670811.2017.1360143 Fact-checking as digital media literacy in higher education Seminar.net - International journal of media, technology and lifelong learning 15 Vol. 18 – Issue 1 – 2022 Teichler, U. (2009). ‘Internationalisation of Higher Education: European Experiences.’ Asia Pacific Education Review, 10(1), 93–106. https://doi.org/10.1007/s12564-009-9002-7 Tekoniemi, S. (2021). Fact-checking Meets with Media Education: Evaluating A Workshop in Higher Education. Master Thesis in Digital Literacy Education. Tampere University, Finland. https://urn.fi/URN:NBN:fi:tuni-202106075736 Valtonen, T., Tedre, M., Mäkitalo, K., Vartiainen, H. (2019). Media Literacy education in the age of machine learning. Journal of Media Literacy Education, 11(2), 20–36. https://doi.org/10.23860/JMLE-2019-11-2-2 Wardle, C. & Derakshan, H. (2017). Information disorder: Toward an interdisciplinary framework for research and policy making. Council of Europe report. DGI(2017)09. VanderStoep, S. W., & Johnston, D. D. (2009). Research methods for everyday life : Blending qualitative and quantitative approaches. San Francisco, CA: Jossey-Bass. Yin, R. K. (2009). Case Study Research. Design and Methods (4th ed.). Los Angeles: Sage https://doi.org/10.1007/s12564-009-9002-7 https://urn.fi/URN:NBN:fi:tuni-202106075736 https://doi.org/10.23860/JMLE-2019-11-2-2 Fact-checking as digital media literacy in higher education Abstract Introduction Digital Media Literacy as Fact-checking Workshop on Fact-checking Findings Discussion Conclusion References