Bulletin of Social Informatics Theory and Application ISSN 2614-0047 Vol. 5, No. 2, September 2021, pp. 150-157 150 https:doi.org/10.31763/businta.v5i2.492 Development of a criticality scale related to hoaxes in social media Anjarie Dharmastuti a,1,*, Bambang Budi Wiyono b,2, Imanuel Hitipeuw a,3, Hetti Rahmawati a,4, Fitri Wahyuni c,5, Fressi Apriliyanti d,6 a Department of Educational.Psychology, Universitas Negeri Malang, Malang, Indonesia b Department of Educational Administration, Universitas Negeri Malang, Malang, Indonesia c Department of Country Guidance and Counseling, Universitas Negeri Malang, Malang, Indonesia d Kindergarten Al Wa'yi, Malang, Indonesia 1 dharmastuti.1601139@students.um.ac.id *; 2 bambang.budi.fip@um.ac.id; 3 imanuel.hitipeuw.fip@um.ac.id; 4 hetti.rahmawati.fppsi@um.ac.id; 5 fitri.wahyuni.1601119@students.um.ac.id; 6 apriliyanti.fresy123@gmail.com * corresponding author 1. Introduction The rapid development of social media technology has provided benefits for human life. Especially during the Covid 19 pandemic, the use of social media is known to have increased intensively, both for communication, information, and the learning process [1]–[3]. The existence of social media is also glorified as a symbol of openness and freedom of information [4]–[6]. Social media is identified as the most popular platform for online information dissemination [5]. Netizens are free to be actively involved in the flow of information itself. The phenomenon of sharing, tweeting, or retweeting on social media has now become part of the daily lives of internet users or what are often referred to as netizens [4]. The rapid behavior of sharing information online, causes the flow of information to become overloaded [7], [8]. This is as reported by Beam et al. [9], that there is a high correlation between information exposure and information sharing behavior. This abundance of information actually has an impact on increasing the pathology of the information itself [10]. Various false information, rumors, and hoaxes also spread instantly [11], [12]. The circulation of false information or what is more popularly known as hoax is currently a concern [13], [14]. Concern about hoaxes is increasing, along with the impact it brings. The impact of hoaxes is clearly very detrimental, including the proliferation of fake infodemics that trigger unrest in the midst of a pandemic [15], [16], confusion regarding vaccine rumors [12], causing slander and prejudice [17], heated political strife [18]; and also it creates feelings of alienation and cynicism [19], and threatens A R T I C L E I N F O A B S T R A C T Article history Received August 4, 2021 Revised August 24, 2021 Accepted September 1, 2021 The massive circulation of hoaxes on social media is currently a fairly complex problem in society. One simple way that is considered effective to reduce the rate of hoaxes is to not disseminate information without prior verification. Verification efforts with the aim of minimizing the impact of this hoax can be referred to as criticality. The term criticality is basically still quite new when compared to the term critical thinking. Criticality refers to the concept of critical thinking accompanied by a commitment to act based on socio-cultural values. The purpose of this study was to develop a criticality scale related to hoaxes on social media. The method used is quantitative involving 400 students from state universities in Malang City. The results of this study indicate that the criticality scale related to hoaxes developed has met the criteria of good validity and reliability. This is an open access article under the CC–BY-SA license. Keywords Criticality Hoax Social media Confirmatory factor analysis Reliability http://creativecommons.org/licenses/by-sa/4.0/ http://creativecommons.org/licenses/by-sa/4.0/ ISSN 2614-0047 Bulletin of Social Informatics Theory and Application 151 Vol. 5, No. 2, September 2021, pp. 150-157 Dharmastuti et.al (Development of a criticality scale related to hoaxes in social media) the welfare of society [20]. It can be said that hoaxes are one of the "dark sides" of the glittering sophistication of social media [21]. Stopping hoaxes is not an easy thing, it requires cooperation from various multidisciplinary or cross-sectors [22]. The social media platform has tried to use algorithms to minimize hoaxes, but these efforts are felt to be not optimal [23]. Therefore, there is an emphasis on the micro level, where every social media user is expected to be responsible for the circulation of hoaxes [24]–[26]. However, the circulation of hoaxes is also rapidly increasing due to user behavior, both intentional and unintentional [8]. Individuals are at the forefront of efforts to combat the circulation of hoaxes. The tendency of people to gather and tell stories without data is basically one of the causes of the massive circulation of hoaxes. This can be seen in the research results of Khan & Idris [8], which show that one of the predictors of the circulation of hoaxes in Indonesia is the lack of willingness and skills to verify information. Basically, this information verification skill cannot be separated from critical thinking skills [5], [10], [27]. Based on research results during the pandemic, it was also found that individuals who do not think critically are more likely to spread hoax information related to Covid 19 [26]. Critical thinking skills, in the end, become something much more valuable when faced with the increasingly uncontrolled phenomenon of hoax circulation [5], [8], [25], [27], [28]. Thinking critically before spreading information on social media is a form of individual effort in mitigating the circulation of hoaxes [29]. As expressed by one of the critical thinking experts, namely Paul Richard (1992) which states that critical thinking is a very important skill, especially when changes occur so quickly. A simple example of critical thinking by verifying is by questioning the credibility of the source or evidence of an information [30]. Unfortunately, the practice of critical thinking on social media is still low. This can be seen from the high number of cases of hoax spread [29]. Critical thinking is basically a higher-order mental function that is influenced by social factors and is carried out in a social context [31]. Critical thinking is not a competency that only exists in individual mentality, but is a way of life that must be understood from a socio-cultural perspective. The concept of critical thinking that emphasizes cognitive and socio-cultural perspectives is then called criticality (Davies, 2015). Davies uses the term criticality to describe critical thinking as a trait that includes thinking, reflecting, and following up with action or action. According to Davies [32], criticality is a skill concept that should be taught at the college level. Ironically, this uncritical behavior regarding information on social media is also found among academics in higher education [10], [33], [34]. In connection with the background of the problem, this research aims to develop a criticality scale related to hoaxes on social media. So far, there are still not many criticality scales related to hoaxes on social media, especially among college students. With the development of this scale, it is hoped that it can add to the literature related to similar scales. Given that this hoax problem is getting more and more worrying, it is hoped that the development of this scale can make a small contribution to efforts to minimize the circulation of hoaxes. 2. Criticality In particular, criticality in disseminating information on social media is a cognitive as well as social activity [4], [35]. Concern for the impact of hoaxes in society and the world must be the basis for critical thinking and following up with action [36]. Criticality related to information on social media can be applied by identifying evidence, sources, and cognitive biases that arise before deciding to share it [30], [37]. Criticality is generally emphasized at the level of higher education. This is also supported by the finding that high-level cognitive abilities develop optimally in late adolescence to adulthood [38]. Criticality requires individuals to be moved to do something based on the results of their critical thinking. Skills and dispositions are essential for critical thinking, they are not sufficient unless supplemented by action. This emphasis on action is because criticality has added a socio-cultural dimension to critical thinking which initially tends to only be at the cognitive level [32]. Criticality is 152 Bulletin of Social Informatics Theory and Application ISSN 2614-0047 Vol. 5, No. 2, September 2021, pp. 150-157 Dharmastuti et.al (Development of a criticality scale related to hoaxes in social media) basically a high-level mental function that is influenced by social factors and is carried out in a social context [31]. Criticality has become a highly valued skill in modern society [38]. This is because in the modern era, where information technology is developing rapidly, skills are needed to analyze information, evaluate its credibility and apply information appropriately [39]. Of course, by developing criticality skills, the community will benefit more from the various information currently available [40]. On the other hand, failure to critically analyze information will certainly have a negative impact [41]. 3. Method 3.1. Item generation The criticality scale used in this study was developed based on the critical thinking scale in understanding and responding to news (critical thinking performance in news) [30], [37]. The scale developed consists of 3 dimensions, namely (1) criticality related to messages (content), (2) criticality related to sources or data, (3) criticality related to bias. The answer to the statement of each item will be given a score of 1-5 with a Likert scale model. Variations of answer choices are very often, often, moderately, not often, very not often. The blueprint for the criticality scale can be seen in Table 1. Table.1 Blueprint Criticality Scale Dimention Indicator Item Tot. Item Fav Unfav Criticality about message Identify the truth of the information to be disseminated 1, 3 2, 4 4 Criticality about the source Identify sources/data information to be disseminated 6, 7 5, 8 4 Criticality about bias Identify the bias of the information to be disseminated 9, 11 10, 12 4 Total Item 12 3.2. Content Adequacy Assessment The results of the expert validity test were analyzed using the Aiken V method. Based on the Aiken V analysis, with 12 raters, a V value of 0.78 was obtained with a significance of 0.01 and a V value of 0.69 with a significance of 0.05. The results of the analysis with Aiken V showed the highest score was 0.95 and the lowest score was 0.78. Thus, it can be concluded that this instrument is valid in terms of face validity at a significance of 0.01. Based on the results of this expert validity test, it appears that none of the items were eliminated. 3.3. Questionnaire Administration, By conducting trials on 120 samples with ready-made instruments then followed up with scoring. The number of test subjects in this study were 120 students of state universities in the city of Malang, namely Brawijaya University, Malang State University, Maulana Malik Ibrahim State Islamic University. 3.4. Factor analysis This step was done by conducting a confirmatory test. Based on the results of the confirmatory analysis test presented in Fig. 1, it can be concluded that all indices have exceeded their cut off values. It appears that the p-value is 0.161 0.05, GFI and AGFI 0.90, CFI and TLI 0.95, and the RMSEA value 0.08. Based on the results of the confirmatory test, it is known that the constructed construct has met the feasibility standard. 3.5. Construct Validity Based on the results of the analysis of construct validity, it was found that from 12 items, three items were eliminated by using a minimum loading factor limit of 0.5. then get nine valid items. ISSN 2614-0047 Bulletin of Social Informatics Theory and Application 153 Vol. 5, No. 2, September 2021, pp. 150-157 Dharmastuti et.al (Development of a criticality scale related to hoaxes in social media) Fig. 1. Construct Validity 3.6. Internal Consistency Assessment This step was done by conducting a reliability test. The results of the Cronbach reliability test showed the results of 0.841 and the composite reliability test showed the results of 0.997, which means that the criticality construct can be said to be reliable. 4. Results and Discussion 4.1. Statistics and Data Analysis After rearranging the valid items, namely a number of nine items, a field test was carried out. This field test was conducted on 400 students of state universities in Malang, namely Brawijaya University, Malang State University, Maulana Malik Ibrahim State Islamic University. Sampling is done by probability sampling technique. Based on the results of the confirmatory analysis test presented in Fig. 1, it can be concluded that all indices are more than their cut off values. It appears that the p value is 0.866 0.05, GFI and AGFI 0.90, CFI and TLI 0.95, and the RMSEA value 0.08. Thus, it can be said that the unidimensionality of the criticality scale has been fulfilled. Of the 9 indicators, there are 6 indicators with a loading factor value of 0.5, in other words 6 indicators have met convergent validity. The three dimensions are proven to be able to contribute in explaining criticality by 79% (cri1), 83% (cri2), and 89% (cri3). The results of the Cronbach reliability test showed a result of 0.814 and the results of the composite reliability test showed a result of 0.898. 154 Bulletin of Social Informatics Theory and Application ISSN 2614-0047 Vol. 5, No. 2, September 2021, pp. 150-157 Dharmastuti et.al (Development of a criticality scale related to hoaxes in social media) Fig. 2. Statistics Analysis Based on the results of trials and field tests, it can be said that the criticality scale related to hoaxes on social media has met the appropriate criteria. In other words, this scale is a ready-to-use measuring tool to measure student criticality in responding to hoaxes on social media, because it has been tested both in terms of reliability and validity. The number of items on this scale is quite minimalist, only 6 items out of 12 items at the beginning of development. The criticality scale developed consists of three dimensions, namely 1) criticality related to messages (content), 2) criticality related to sources or data, 3) criticality related to bias. Each dimension is represented by two items, one favorable item and one unfavorable item. In full, the six valid items are presented in the Table 2. Table.2 Valid Items No Items Valid 1 I choose not to share information if I'm not sure if it's true 2 I spread information without thinking about the credibility of the content 3 I evaluate the veracity of the information source first, before spreading it 4 I do not consider the credibility of the data presented in the information that I will share 5 I reconsider my conclusions regarding the veracity of the information I will share 6 I consider information that does not match my beliefs to be false information The three dimensions on this scale represent the individual's tendency to respond to information. The first dimension is related to the criticality of messages or data from a message, this is very important considering that some information is made in a bombastic way so that it encourages individuals to get carried away by their emotions and just believe. The second dimension is related to the source of information, this becomes very important, given the tendency to just believe in an information if the sender is a person who has familiarity. Sources of information are also important regarding the credibility of information. The third dimension is related to the biases that often arise in individual thinking. Some information on social media triggers bias so that often the right and wrong judgment of an information is only based on personal beliefs or subjectivity. The three dimensions on the scale can be used as criticality parameters related to hoaxes on social media. Based on psychometric rules, the criticality scale related to hoaxes on social media can also be said to be good. Internal consistency with Cronbach's alpha and composite reliability of this scale have shown good results. The construct validity of this scale has also shown good results. In other words, ISSN 2614-0047 Bulletin of Social Informatics Theory and Application 155 Vol. 5, No. 2, September 2021, pp. 150-157 Dharmastuti et.al (Development of a criticality scale related to hoaxes in social media) this scale is able to measure the relationship between criticality and the behavior of spreading hoaxes on social media. The results of the confirmatory analysis also show that the developed criticality scale model shows a fairly good model fit index based on the goodness of fit criteria. The limitation of developing this scale is the disproportionate proportion of subjects between men and women. The majority (76%) of the research subjects were women. so that for the development of a better criticality scale in the future, it is necessary to arrange so that the number of male and female subjects becomes more proportional. The items on this criticality scale generally still look the same as the notion of critical thinking. The determination of the term criticality in this scale refers to the social aspect in the context of disseminating information on social media. Criticality in this case concerns the effect of actions on other people or society. Literally, basically thinking or deciding not to do something (not disseminating information) is also a behavior. The choice of the term criticality in this context also emphasizes the importance of consistency between thought and action. 5. Conclusion Psychometrically, the criticality scale related to hoaxes on social media developed in this study has met the good psychometric rules. In other words, this scale can be used for research purposes related to the criticality of college students in responding to hoaxes on social media. References [1] M. Drouin, B. T. McDaniel, J. Pater, and T. Toscos, “How Parents and Their Children Used Social Media and Technology at the Beginning of the COVID-19 Pandemic and Associations with Anxiety,” Cyberpsychology, Behav. Soc. Netw., vol. 23, no. 11, pp. 727–736, Nov. 2020, doi: 10.1089/cyber.2020.0284. [2] R. Sigurvinsdottir, I. E. Thorisdottir, and H. F. Gylfason, “The Impact of COVID-19 on Mental Health: The Role of Locus on Control and Internet Use,” Int. J. Environ. Res. Public Health, vol. 17, no. 19, p. 6985, Sep. 2020, doi: 10.3390/ijerph17196985. [3] B. B. Wiyono, A. Wedi, S. Wahyuni, and A. V. Valdez, “Use of Online Communication Media as A Student Learning Tool in College and its Effect on the Achievement of Students,” in 2020 6th International Conference on Education and Technology (ICET), Oct. 2020, pp. 186–191, doi: 10.1109/ICET51153.2020.9276562. [4] V. Karnowski, L. Leonhard, and A. S. Kümpel, “Why Users Share the News: A Theory of Reasoned Action-Based Study on the Antecedents of News-Sharing Behavior,” Commun. Res. Reports, vol. 35, no. 2, pp. 91–100, Mar. 2018, doi: 10.1080/08824096.2017.1379984. [5] K. Y. L. Ku, Q. Kong, Y. Song, L. Deng, Y. Kang, and A. Hu, “What predicts adolescents’ critical thinking about real-life news? The roles of social media news consumption and news media literacy,” Think. Ski. Creat., vol. 33, p. 100570, Sep. 2019, doi: 10.1016/j.tsc.2019.05.004. [6] O. Turel and H. Qahri-Saremi, “Explaining unplanned online media behaviors: Dual system theory models of impulsive use and swearing on social networking sites,” New Media Soc., vol. 20, no. 8, pp. 3050–3067, Aug. 2018, doi: 10.1177/1461444817740755. [7] A. Khan, S. Khan, S. Zia-ul-islam, and M. Khan, “Communication Skills of a Teacher and Its Role in the Development of the Students ’ Academic Success,” J. Educ. Pract., vol. 8, no. 1, pp. 18–21, 2017. [8] M. L. Khan and I. K. Idris, “Recognise misinformation and verify before sharing: a reasoned action and information literacy perspective,” Behav. Inf. Technol., vol. 38, no. 12, pp. 1194–1212, Dec. 2019, doi: 10.1080/0144929X.2019.1578828. [9] M. A. Beam, M. J. Hutchens, and J. D. Hmielowski, “Clicking vs. sharing: The relationship between online news behaviors and political knowledge,” Comput. Human Behav., vol. 59, pp. 215–220, Jun. 2016, doi: 10.1016/j.chb.2016.02.013. [10] A. P. Weiss, A. Alwan, E. P. Garcia, and J. Garcia, “Surveying fake news: Assessing university faculty’s fragmented definition of fake news and its impact on teaching critical thinking,” Int. J. Educ. Integr., vol. 16, no. 1, p. 1, Dec. 2020, doi: 10.1007/s40979-019-0049-x. 156 Bulletin of Social Informatics Theory and Application ISSN 2614-0047 Vol. 5, No. 2, September 2021, pp. 150-157 Dharmastuti et.al (Development of a criticality scale related to hoaxes in social media) [11] M. Tambuscio, D. F. M. Oliveira, G. L. Ciampaglia, and G. Ruffo, “Network segregation in a model of misinformation and fact-checking,” J. Comput. Soc. Sci., vol. 1, no. 2, pp. 261–275, Sep. 2018, doi: 10.1007/s42001-018-0018-9. [12] X. Xiao, Y. Su, and D. K. L. Lee, “Who Consumes New Media Content More Wisely? Examining Personality Factors, SNS Use, and New Media Literacy in the Era of Misinformation,” Soc. Media + Soc., vol. 7, no. 1, p. 205630512199063, Jan. 2021, doi: 10.1177/2056305121990635. [13] N. A. Karlova and K. E. Fisher, “A social diffusion model of misinformation and disinformation for understanding human information behaviour,” Inf. Res., vol. 18, no. 1, 2013. [14] K. P. K. Kumar and G. Geethakumari, “Detecting misinformation in online social networks using cognitive psychology,” Human-centric Comput. Inf. Sci., vol. 4, no. 1, p. 14, Dec. 2014, doi: 10.1186/s13673-014- 0014-x. [15] M. Montesi, “Understanding fake news during the Covid-19 health crisis from the perspective of information behaviour: The case of Spain,” J. Librariansh. Inf. Sci., vol. 53, no. 3, pp. 454–465, Sep. 2021, doi: 10.1177/0961000620949653. [16] S. O. Oyeyemi, E. Gabarron, and R. Wynn, “Ebola, Twitter, and misinformation: a dangerous combination?,” BMJ, vol. 349, no. oct14 5, pp. g6178–g6178, Oct. 2014, doi: 10.1136/bmj.g6178. [17] E. C. Tandoc, “The facts of fake news: A research review,” Sociol. Compass, vol. 13, no. 9, Sep. 2019, doi: 10.1111/soc4.12724. [18] H. Allcott and M. Gentzkow, “Social Media and Fake News in the 2016 Election,” J. Econ. Perspect., vol. 31, no. 2, pp. 211–236, May 2017, doi: 10.1257/jep.31.2.211. [19] M. Balmas, “When Fake News Becomes Real,” Communic. Res., vol. 41, no. 3, pp. 430–454, Apr. 2014, doi: 10.1177/0093650212453600. [20] E. Ferrara, “‘Manipulation and abuse on social media’ by Emilio Ferrara with Ching-man Au Yeung as coordinator,” ACM SIGWEB Newsl., no. Spring, pp. 1–9, Apr. 2015, doi: 10.1145/2749279.2749283. [21] C. V. Baccarella, T. F. Wagner, J. H. Kietzmann, and I. P. McCarthy, “Social media? It’s serious! Understanding the dark side of social media,” Eur. Manag. J., vol. 36, no. 4, pp. 431–438, Aug. 2018, doi: 10.1016/j.emj.2018.07.002. [22] D. M. J. Lazer et al., “The science of fake news,” Science (80-. )., vol. 359, no. 6380, pp. 1094–1096, Mar. 2018, doi: 10.1126/science.aao2998. [23] P. Pourghomi, F. Safieddine, W. Masri, and M. Dordevic, “How to stop spread of misinformation on social media: Facebook plans vs. right-click authenticate approach,” in 2017 International Conference on Engineering & MIS (ICEMIS), May 2017, pp. 1–8, doi: 10.1109/ICEMIS.2017.8272957. [24] M. Koohikamali and A. Sidorova, “Information Re-Sharing on Social Network Sites in the Age of Fake News,” Informing Sci. Int. J. an Emerg. Transdiscipl., vol. 20, pp. 215–235, 2017, doi: 10.28945/3871. [25] P. Lorenz-Spreen, S. Lewandowsky, C. R. Sunstein, and R. Hertwig, “How behavioural sciences can promote truth, autonomy and democratic discourse online,” Nat. Hum. Behav., vol. 4, no. 11, pp. 1102– 1109, Nov. 2020, doi: 10.1038/s41562-020-0889-7. [26] [26] M. L. Stanley, N. Barr, K. Peters, and P. Seli, “Analytic-thinking predicts hoax beliefs and helping behaviors in response to the COVID-19 pandemic,” Think. Reason., vol. 27, no. 3, pp. 464–477, Jul. 2021, doi: 10.1080/13546783.2020.1813806. [27] G. Pennycook and D. G. Rand, “Who falls for fake news? The roles of bullshit receptivity, overclaiming, familiarity, and analytic thinking,” J. Pers., vol. 88, no. 2, pp. 185–200, Apr. 2020, doi: 10.1111/jopy.12476. [28] S. Talwar, A. Dhir, D. Singh, G. S. Virk, and J. Salo, “Sharing of fake news on social media: Application of the honeycomb framework and the third-person effect hypothesis,” J. Retail. Consum. Serv., vol. 57, p. 102197, Nov. 2020, doi: 10.1016/j.jretconser.2020.102197. [29] A. M. Guess et al., “A digital media literacy intervention increases discernment between mainstream and false news in the United States and India,” Proc. Natl. Acad. Sci., vol. 117, no. 27, pp. 15536–15545, Jul. 2020, doi: 10.1073/pnas.1920498117. ISSN 2614-0047 Bulletin of Social Informatics Theory and Application 157 Vol. 5, No. 2, September 2021, pp. 150-157 Dharmastuti et.al (Development of a criticality scale related to hoaxes in social media) [30] A. Maksl, S. Ashley, and S. Craft, “Measuring news media literacy,” J. Media Lit. Educ., vol. 6, no. 3, pp. 29–45, 2015. [31] N. M. Lee, “Fake news, phishing, and fraud: a call for research on digital media literacy education beyond the classroom,” Commun. Educ., vol. 67, no. 4, pp. 460–466, Oct. 2018, doi: 10.1080/03634523.2018.1503313. [32] M. Davies, “A Model of Critical Thinking in Higher Education,” in Higher Education: Handbook of Theory and Research, 2015, pp. 41–92. [33] X. Chen, S.-C. J. Sin, Y.-L. Theng, and C. S. Lee, “Why Students Share Misinformation on Social Media: Motivation, Gender, and Study-level Differences,” J. Acad. Librariansh., vol. 41, no. 5, pp. 583–592, Sep. 2015, doi: 10.1016/j.acalib.2015.07.003. [34] J. McDougall, “Media Literacy versus Fake News,” Medijske Stud., vol. 10, no. 19, pp. 29–45, Oct. 2019, doi: 10.20901/ms.10.19.2. [35] J. Choi, “Why do people use news differently on SNSs? An investigation of the role of motivations, media repertoires, and technology cluster on citizens’ news-related activities,” Comput. Human Behav., vol. 54, pp. 249–256, Jan. 2016, doi: 10.1016/j.chb.2015.08.006. [36] G. Dunne, “Beyond critical thinking to critical being: Criticality in higher education and life,” Int. J. Educ. Res., vol. 71, pp. 86–99, 2015, doi: 10.1016/j.ijer.2015.03.003. [37] S. Ashley, A. Maksl, and S. Craft, “Developing a News Media Literacy Scale,” Journal. Mass Commun. Educ., vol. 68, no. 1, pp. 7–21, Mar. 2013, doi: 10.1177/1077695812469802. [38] M. R. Su and K. K. Shum, “The Moderating Effect of Mindfulness on the Mediated Relation Between Critical Thinking and Psychological Distress via Cognitive Distortions Among Adolescents,” Front. Psychol., vol. 10, Jun. 2019, doi: 10.3389/fpsyg.2019.01455. [39] H. A. Butler, “Halpern Critical Thinking Assessment Predicts Real-World Outcomes of Critical Thinking,” Appl. Cogn. Psychol., vol. 26, no. 5, pp. 721–729, Sep. 2012, doi: 10.1002/acp.2851. [40] C. P. Dwyer, M. J. Hogan, and I. Stewart, “An integrated critical thinking framework for the 21st century,” Think. Ski. Creat., vol. 12, pp. 43–52, Jun. 2014, doi: 10.1016/j.tsc.2013.12.004. [41] C. Noone, “Mindfulness and Critical Thinking: Structural Relations, Short-Term State Effects, and Long- Term Intervention Effects,” National University of Ireland, Galway, 2016.