DOI: 10.3303/CET24114161 Paper Received: 10 June 2024; Revised: 9 September 2024; Accepted: 13 November 2024 Please cite this article as: Németh R., Erdős F., Kovács K., 2024, Fake News Classification on Social Media in the Field of Environmental Engineering, Chemical Engineering Transactions, 114, 961-966 DOI:10.3303/CET24114161 CHEMICAL ENGINEERING TRANSACTIONS VOL. 114, 2024 A publication of The Italian Association of Chemical Engineering Online at www.cetjournal.it Guest Editors: Petar S. Varbanov, Min Zeng, Yee Van Fan, Xuechao Wang Copyright © 2024, AIDIC Servizi S.r.l. ISBN 979-12-81206-12-0; ISSN 2283-9216 Fake News Classification on Social Media in the Field of Environmental Engineering Richárd Németh, Ferenc Erdős*, Katalin Kovács Széchenyi István University, Department of Informatics, H-9026 Győr. Egyetem Sq. 1. erdosf@sze.hu Environmental engineering plays a crucial role in addressing the sustainability challenges of the environment and everyday life. These areas cover a wide range of aspects, from environmental impacts, efficient use of natural resources, waste management and recycling, and improving air and water quality to preventing, managing and mitigating environmental hazards and disasters. For example, climate change, global warming, pollutant emissions, etc., are frequently discussed on social media. Unfortunately, however, these platforms, as the primary means of mass information, provide scope for misinformation, manipulation and influencing opinions through dubious or outright fake news and posts. Despite the prevalence and growing volume of research on detecting fake news, there is a significant lack of quality publications focusing specifically on this environmental engineering-related area. In this paper, the authors aim to bridge this gap by developing a supervised machine- learning model for detecting fake news in this field of science. To build a robust model, a training dataset was constructed by using a specific dataset of labelled social media posts containing keywords from Chemical Engineering Transactions journal articles, with a particular focus on sustainability and environmental aspects. This research contributes to enhancing the integrity of environmental engineering in social media discourse by providing a tool for identifying and mitigating the spread of misinformation, promoting informed decision-making and critical thinking, and improving public awareness of environmental matters. 1. Introduction The impact of misinformation and disinformation in the field of environmental engineering should not be underestimated. Fake news about the environment, wellbeing and health is particularly sensitive to the individual, making it an excellent tool for manipulating public opinion and mobilising social groups. Although the detection of fake news and the development of prevention are frequently researched areas today, there is not much research in the literature focusing specifically on this. In this research, the authors reviewed more than 1500 articles from Chemical Engineering Transactions, resulting in a 50-word keyword list on environmental engineering. This list was applied to the tagged true and false news data downloaded from Kaggle (Subhadeep, 2022), resulting in a dataset containing entries on the topic. The work aims to perform a comparative analysis by combining task-appropriate learning methods, machine learning (ML) algorithms and good practices, using different methods of data cleaning, text processing, text analysis and hyperparameter optimization, resulting in a clear identification of the most suitable methodology for classifying fake news in the domain in terms of performance and efficiency. The primary measure of evaluation is the accuracy of prediction, but comparisons are also made using other mathematical-statistical metrics like precision, recall, F1 score and so on. 2. Artificial Intelligence – History, models and algorithms 2.1 The spread of Artificial Intelligence and its milestones The best-known and most-cited definition of Artificial Intelligence (AI) is that of AI researcher Demis Hassabis, who describes AI as „a science of making machines smart” (Hassabis, 2017). According to Andrew Ng, founder of DeepLearning.AI, „artificial intelligence is the new electricity” of our time (Mühlhoff, 2019). From a computer science perspective, AI is an emerging technology that uses machines to simulate human intelligence 961 (Investopedia, 2024). Whichever way you approach the question, it is undeniable that AI is here with us. But what events shaped it into what we know today? The first step on the long road to artificial intelligence is usually identified as Huxley's neuron model. It is also worth mentioning the first experiment to determine whether a machine can give human responses, the so-called Turing test (Saygin et al., 2000). The beginning of AI research goes back to John McCarthy, who came up with the new name in 1956 on the campus of Dartmouth College. The basic model for artificial neurons is Rosenblatt's elementary perceptron, introduced in 1957. The field's first comprehensive description of the basic problems and the search for a way forward was written by Minsky (1960). After an initial boom, there was a sharp decline in the 1970s and 1980s, which is referred to in the literature as the "MI winter". A combination of factors has led to a general disillusionment with AI research in the scientific community. Progress has proved to be too slow and cumbersome compared to expectations, resulting in a significant reduction in funding for AI research. Since the 1990s, there has been a new explosion in the field, mainly due to the spread of the internet and the increasing computing power of hardware components. In the development of neural networks inspired by the human brain, it was recognized that perception, thought and action are not independent of each other. The backpropagation algorithm, published in 1989, offered an efficient way to train complex, multi-layered networks and minimize errors in the output, enabling the potential of neural networks to be exploited. Sekhar and Meghana (2020), among others, have written a comprehensive paper on the detailed operation of the algorithm. Accelerated, exponential progress over the past nearly 30 y has produced a number of achievements that have turned heads around the world. In the late 2010s, the first large language models were released – based on the transformer architecture introduced in 2017. 2.2 Types of Artificial Intelligence and the most common neural network architectures At the dawn of artificial intelligence, AI-based solutions were much more focused on performing automated computational tasks, i.e. they were programmed to perform a single, typically computationally intensive task. These are collectively referred to as narrow or weak AI, and their main characteristic is that they are not capable of solving problems but are only suited to the specific target task in a specific, constrained environment. Narrow AI is not capable of making autonomous decisions and is not capable of independent thinking (Maggiolo, 2021). In contrast, so-called general or strong AI is capable of learning, can see through the complexity of the tasks at hand, can make rational decisions like humans, and can even outperform the human mind in any intellectual domain. In fact, a high level of powerful artificial intelligence that is at least as intelligent as a human in all respects is still unattainable. Even the most advanced current models cannot infer, understand or explain the processes and causes behind data (Butz, 2021). True general AI requires machine intelligence with capabilities that go beyond the human-brain-computer analogy. As mentioned earlier, AI research first gained momentum with the creation of perceptrons, and the first truly widespread model was the multilayer perceptron (MLP). This model allows the detection of patterns, correlations and relationships in data (Lumacad and Namoco, 2022), but does not do well with unstructured data such as text and images. Convolutional Neural Networks (CNNs) have been developed specifically for the analysis and processing of image data, modelled on the visual systems of living beings, and are effective mainly in the areas of nonlinear patterns, face and image recognition (Ghosh et al., 2020). So-called recurrent neural networks (RNN) are used for the analysis and processing of time-series data (mainly language sequences). The most popular models are LSTM (Long Short-term Memory) and GRU (Gated Recurrent Unit), and the most dominant model in the field of natural language processing is transformer networks. The latter, through its multi-head attention mechanism, is able to identify dependencies and relationships within texts, which leads to a better understanding of different contexts (Lu et al., 2020). These models “have heightened the machine's capability to understand, produce, and interact with human language in unprecedented ways, and (...) have infiltrated a range of sectors, including finance, healthcare, biology, and education, revolutionizing both traditional and emerging domains” (Li, 2024). So, they provide a framework for all the technologies (text generation, machine translation, sentiment and emotion analysis, and response systems) that can be used to identify fake news. 2.3 Machine learning algorithms used in the research Given the availability of labelled data for training the model, supervised learning (SL) methods were employed. SL is a “machine learning approach that leverages labelled data to educate a system in forecasting outcomes based on its training” (Alnuaimi and Albaldawi, 2024). It has two main categories: regression and classification. The research used some of the same ML algorithms as Hassan and Saeed (2023), as their paper proved to be the most comprehensive and thorough of the publications on two-class classification approaches. These algorithms are: 1) logistic regression, which is "a type of regression used for prediction, and (... ) can be considered as an extension of linear regression" (Alnuami and Albaldawi, 2024); 2) the decision forest, which is composed of different decision trees and works on the principle of majority voting, 3) the boosted decision tree, which "is a type of ensemble model that is mostly used to correct the flaws in earlier trees" (Hassan and Saeed, 962 2023) and 4) the neural network, which performs binary classification in a similar way to the way the human brain works. All of these models can be added as components in the Azure Machine Learning platform. 3. Types and characteristics of fake news Untrue or distorted information can be divided into several categories. The difference between them lies primarily in the intentionality and method of dissemination. According to Walters (2018), the common characteristics of this type of news are falsely published content, objectively false claims and newsworthiness. The most harmless type, misinformation, is incorrect, inaccurate information, or mistaken information that is not disseminated/spread primarily for manipulative purposes. Its main characteristic is that it is not intentionally misleading but is the result of a mistake or misunderstanding – the disseminator believes the information to be credible. The information may be erroneous or inaccurate, but the original intent is not to deceive or manipulate – rather, it is a "good faith mistake". This category includes press errors, journalistic errors due to journalistic negligence, rumours and any other unintentionally distorted/misrepresented news (Allcott and Gentkow, 2017). Disinformation is false/misleading information that is deliberately and knowingly disseminated, usually with the aim of influencing, manipulating or favouring a particular message/view. A marked difference from the previous category is that disinformation is also created and disseminated with the intention to deceive. According to FIIA, it is "used extensively (…) to refer to written or oral communication containing intentionally false, incomplete, or misleading information (frequently combined with true information), which seeks to deceive, misinform, and/or mislead the target” (Pynnöniemi and Rácz, 2016). Fake news is intentionally false/misleading information that is usually spread online. They "get a lot of attention on social media and are consumed by millions of people. They are used by politicians, particularly during elections” (Sadiku et al, 2018). The popular term fake news can mean clickbait headlines, politically motivated misinformation, conspiracy theories, and media outlets not favoured by the propagator or accidental press errors. Such news is artificially "manufactured", targeted at specific topics/individuals, and intended to influence, spread misconceptions, and manipulate public opinion. Whatever the type of fake news, it can be a way to shake people's trust. Most of the global problems affecting all of humanity are related to the environment. The latest advances in artificial intelligence are not yet being used to identify fake news in this domain. Our research aims to fill this gap. The aim was to create an ML-based model capable of identifying fake news specifically related to environmental engineering with high accuracy, which will later be used to predict the veracity of social media posts. In this chapter, the preparation of the dataset, the data cleaning method, the process of decomposing the training and test set, and the ML algorithms used are described. During this process, the authors mainly used Azure's Machine Learning cloud-based service platform, which provides researchers with a range of technologies. 3.1 Data set and data cleaning The set of training and validation data comes from the Fake and Real News Data dataset (Subhadeep, 2022), specially prepared to identify fake news. This dataset has its own limitations – primarily the social media context and the nature of the news on which the training was performed, and the interval processed determines the period over which the model is suitable for categorising posts. However, the global nature and use of news with diverse content greatly increases its usability. The set was initially filtered for entries containing the environmental engineering-related keywords that had been collected. For the keyword specification, abstracts from the 2022 volumes of the Chemical Engineering Transactions were analysed. Based on the analysis, Table 1 shows the selected keywords relevant to the field under examination and their frequency in the abstracts. Table 1: Most common keywords from Chemical Engineering Transactions 2022 Rank Keyword Occurrence Rank Keyword Occurrence 1 hydrogen 269 11 carbon dioxide 45 2 biomass 195 12 greenhouse 44 3 co2 156 13 climate change 39 4 wastewater 113 14 pollution 38 5 sustainable 91 15 recycling 32 6 renewable 76 16 flammable 25 7 explosion 71 17 dust 24 8 environment 68 18 corrosion 23 9 sustainability 52 19 covid 21 10 reactor 51 20 pandemic 15 963 This left 1,744 records out of the original 6,334 entries, which was plenty enough to train the models. As the REAL and FAKE class labels were already available, the model evaluation simply involved evaluating the results and comparing the prediction with the actual result. The first task was data cleaning. This involved fixing the tags that were incorrectly linked and merging the “title” field with the detailed text field. Empty fields and unreadable characters were then removed, and the delimiters were corrected to obtain a proper CSV file. Since the set was still not correct for Azure (its Summarize function identified 4 classes instead of 2 because of the incorrect structure), non-printable characters were cleaned. 3.2 Text processing For preprocessing, stopwords were removed, and lemmatization, lowercase conversion, sentence detection, number and special character filtering, duplicate character deletion and tokenization were performed. The pre- processed data was split into 80 % teaching and 20 % validation datasets. The dataset contained 2 columns and 1,744 records each in English. Two different methods for text processing and text attribute generation were tried. Feature hashing is “a common technique for handling out-of-dictionary vocabulary, and for creating a lookup table to find feature weights in constant time” (Fletcher et al., 2022), while the Extract N-gram from text procedure consists of creating a dictionary from the words of the cleaned dataset, according to the predefined breakdown (unigrams, bigrams and trigrams) (Microsoft, 2021). In the approaches tested, the trigrams-based N-gram text processing combined with the different machine learning algorithms yielded significantly more accurate prediction results compared to feature hashing. The focus shifted to utilizing this type of text processing in subsequent training processes. 3.3 Training the models During the training process efforts were made to train different models (Model1-5) using several ML algorithms, including two-class logistic regression (L2 regularization weight: 1.0), two-class decision forest (number of decision trees: 8; maximum depth of the decision trees: 32; bagging resampling), neural network (one hidden layer, nodes in the hidden layer: 100; learning rate: 0.1; learning iterations: 100) and two-class boosted decision tree (maximum number of leaves per tree 20; minimum number of samples per leaf node:10; learning rate:0.2; trees constructed: 100) . In some instances (Model 6 and Model 7), the process was extended with hyperparameter optimization (HPO): logistic regression (L2 regularization weights: 0.01; 0.1; 1.0), neural network (learning rates: 0.1; 0.2; 0.4; learning iterations: 20; 40; 80; 160). In the evaluation process, 10-fold Cross-validation was utilized to calculate the average values of the different metrics. 4. Results Table 2 shows the scored and evaluated models and components executed in Azure and their different metrics: Table 2: Prediction accuracy achieved using different components in Azure Component Model1 Model2 Model3 Model4 Model5 Model6 Model7 Data cleaning Yes Yes Yes Yes Yes Yes Yes Preprocessing Yes Yes Yes Yes Yes Yes Yes Split data Yes Yes Yes Yes Yes Yes Yes Text proc. F. hashing F. hashing F. hashing F. hashing N-gram N-gram N-gram Algorithm Logistic regression Decision forest Neural network Boosted Dec. Tree Logistic regression Neural network Logistic regression HPO No No No No No Yes Yes Accuracy 0.668 0.636 0.659 0.653 0.92 0.926 0.92 Precision 0.723 0.659 0.709 0.716 0.928 0.929 0.928 Recall 0.588 0.626 0.588 0.555 0.918 0.929 0.918 F1 Score 0.648 0.642 0.643 0.625 0.923 0.929 0.923 AUC 0.714 0.703 0.713 0.698 0.975 0.976 0.975 HPO is about finding the best combinations in order to find the best hyperparameter configuration (Hossain et al., 2021). Both the use of text processing (in this case Extract n-gram) and HPO positively improved the accuracy of the predictions, and the use of neural networks proved to be the most efficient algorithm. The combination of extract n-gram, neural network and hyperparameter optimization (Model 6) proved to be the most efficient among the 8 different approaches. The best trained model resulted in an accuracy of 0.926. 964 Figure 1: Metrics and confusion matrix of the best run The confusion matrix “is a contingency table that is used for describing the performance of a classifier/classification system when the truth is known“ (Yang and Berdine, 2017). As can be seen from the matrix, only 13-13 False Negative and False Positive classifications were made, so the model is quite efficient. This statement is also supported by the ROC curve below (Figure 2): Figure 2. ROC curve of the best approach The ROC (Receiver Operating Characteristics) curve “is a two-dimensional plot that illustrates how well a classifier system works as the discrimination cut-off value is changed over the range of the predictor variable. The x-axis or independent variable is the false positive rate for the predictive test. The y-axis or dependent variable is the true positive rate for the predictive test”. (Yang and Berdine, 2017). It can be used for visual comparison of the performance of different binary classification models. 5. Conclusions In conclusion, this study has effectively contributed to fake news classification on social media. As a novelty, environmental engineering-specific social media posts were utilised, and the comparative analysis of logistic regression, decision forest, boosted decision tree and neural network underlined the variations in how well the used machine learning models performed for an NLP-based two-class classification scenario in this specific domain. As a result, a model with high accuracy was built, which could be used for fake news classification in social media posts related to the environmental field. It can help environmental managers to understand the potential of social media as a primary source of news and help them to design their company's communications in a way that does not appear to be fake news to the receiving community. In addition, the model can be used to build an early warning monitoring system that continuously monitors potential environmental fake news on social media, which environmental officers can evaluate and refute in the appropriate forums if alarmed. In the long term, this will help to increase public trust in environmental measures and new technologies. However, the results are by no means universal. Our trained model has several potential limitations that can significantly impact the model's performance and generalizability when it is utilized on another dataset. The used training data is representative of only a portion of the real-world data, and for a specific time interval; thus, it may exhibit bias in its predictions. It may be unable to generalise well to social media posts outside this time horizon and used terminology. The analysis provides a deeper understanding of the problem of detecting fake news and outlines the potential for further improvements using more sophisticated networks and their combined methods. The next step in the research could be to combine the original data with other datasets and test other neural network architectures to increase efficiency. 965 References Allcott H., Gentkow M., 2017, Social Media and Fake News in the 2016 Election. Journal of Economic Perspectives,12, DOI: 10.3386/w23089. Alnuaimi A.F.A.H., Albaldawi T.H.K., 2024, An overview of machine learning classification techniques, BIO Web Conf., Fifth International Scientific Conference of Alkafeel University, 2024, 97, DOI: 10.1051/bioconf/20249700133. Butz M.V., 2021, Toward Strong AI. KI - Kunstliche Intelligenz, 35, 91. Fletcher S., Roegiest A., Hudek A.K., 2020, Hash the Universe: Differentially Private Text Extraction with Feature Hashing, 13 December 2022, PREPRINT (Version 1) available at Research Square. DOI: 10.21203/rs.3.rs-2363101/v1. Ghosh A., Sufian A., Sultana F., Chakrabarti A., De D., 2020, Fundamental Concepts of Convolutional Neural Network. In: Balas V., Kumar R., Srivastava R. (Eds.), Recent Trends and Advances in Artificial Intelligence and Internet of Things. Intelligent Systems Reference Library, 519–567, DOI: 10.1007/978-3-030-32644- 9_36. Hassabis D., 2017, Artificial intelligence: Chess match of the Century. Nature, 544(7651), 413, DOI: 10.1038/544413a. Hassan S.A., Saeed M.S., 2023, A Comparative Study Evaluated the Performance of Two-class Classification Algorithms in Machine Learning, Kurdistan Journal of Applied Research, 8(2), 46–50, DOI: 10.24017/science.2023.2.5. Hossain M.R., Timmer D., Moya H., 2021, Machine learning model optimization with hyperparameter tuning approach. 2021 International Conference on Advanced Engineering, , accessed 11.10.2024. Investopedia, 2024, What Is Artificial Intelligence (AI)? , accessed 19.06.2024. Li J., 2024, The evolution, applications, and future prospects of large language models: An in-depth overview. Applied and Computational Engineering, 34(1), 234, DOI: 10.54254/2755-2721/35/20230399. Lu S., Wang M., Liang S., Lin J., Wang Z., 2020, Hardware Accelerator for Multi-Head Attention and Position- Wise Feed-Forward in the Transformer. 2020 IEE 33rd International System-on-Chip Conference (COCC), Las Vegas, USA, DOI: 10.1109/socc49529.2020.9524802. Maggiolo G., 2021, Pigro Blog, History of AI: Deep Blue and Strong and Weak Artificial Intelligence. , accessed 19.06.2024. Microsoft, 2021, Extract N-Gram Features from Text component reference. < https://learn.microsoft.com/en- us/azure/machine-learning/component-reference/extract-n-gram-features-from-text?view=azureml-api- 2#create-a-new-n-gram-dictionary>, accessed 20.06.2024. Minsky M., 1960, Steps toward Artificial Intelligence. Proc. IRE49(1961), DOI: 10.1109/jrproc.1961.287775. Mühlhoff R., 2019, Human-aided artificial intelligence: Or, how to run large computations in human brains? Toward a media sociology of machine learning. New Media & Society, 22(10), 1872, DOI: 10.1177/1461444819885334. Pynnöniemi K., Rácz A. (eds.), 2016, Fog of Falsehood: Russian Strategy of Deception and the Conflict in Ukraine. The Finnish Institute of International Affairs, , accessed 11.10.2024 Sadiku M.N.O., Eze T.P., Musa S.M., 2018, Fake News and Misinformation. International Journal of Advances in Scientific Research and Engineering, 4 Saygin A, Cicekli I., Akman V., 2000, Turing Test: 50 Years Later, Minds and Machines 10(4), 463–464., DOI: https://doi.org/10.1007/978-94-010-0105-2_2 Sekhar Ch., Meghana P.S., 2020, A Study on Backpropagation in Artificial Neural Networks, Asia-Pacific Journal of Neural Networks and Its Applications, 4(5), 187–190, DOI: 10.31695/IJASRE. Subhadeep C., 2022, Fake and Real News Data [Data Set]. Kaggle, DOI: 10.34740/KAGGLE/DSV/3229106. Turner G., The Royal Institution: 10 AI milestones of the last 10 years. , accessed 20.06.2024. Walters R.M., 2018, How to Tell a Fake: Fighting Back Against Fake News on the Front Lines of Social Media. Texas Review of Law & Politics, 23(1), 111-179. Yang S., Berdine G., 2017, The receiver operating characteristics (ROC) curve, The Southwest Respiratory and Critical Care Chronicles, 5(19), 34-36, DOI: 10.12746/swrccc.v5i19.391. 966