A Heterogeneous Network-Based Positive and Unlabeled Learning Approach to Detect Fake News | Springer Nature Link (formerly SpringerLink) Skip to main content Advertisement Log in Menu Find a journal Publish with us Track your research Search Cart Home Intelligent Systems Conference paper A Heterogeneous Network-Based Positive and Unlabeled Learning Approach to Detect Fake News Conference paper First Online: 28 November 2021 pp 3–18 Cite this conference paper Access provided by University of Notre Dame Hesburgh Library Download book PDF Download book EPUB Intelligent Systems (BRACIS 2021) A Heterogeneous Network-Based Positive and Unlabeled Learning Approach to Detect Fake News Download book PDF Download book EPUB Mariana C. de Souza  ORCID: orcid.org/0000-0002-1746-841410, Bruno M. Nogueira  ORCID: orcid.org/0000-0002-2401-242311, Rafael G. Rossi  ORCID: orcid.org/0000-0001-8513-321312, Ricardo M. Marcacini  ORCID: orcid.org/0000-0002-2309-348710 & … Solange O. Rezende  ORCID: orcid.org/0000-0002-5233-763910  Show authors Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13074)) Included in the following conference series: Brazilian Conference on Intelligent Systems 1271 Accesses 8 Citations Abstract The dynamism of fake news evolution and dissemination plays a crucial role in influencing and confirming personal beliefs. To minimize the spread of disinformation approaches proposed in the literature, automatic fake news detection generally learns models through binary supervised algorithms considering textual and contextual information. However, labeling significant amounts of real news to build accurate classifiers is difficult and time-consuming due to their broad spectrum. Positive and unlabeled learning (PUL) can be a good alternative in this scenario. PUL algorithms learn models considering little labeled data of the interest class and use unlabeled data to increase classification performance. This paper proposes a heterogeneous network variant of the PU-LP algorithm, a PUL algorithm based on similarity networks. Our network incorporates different linguistic features to characterize fake news, such as representative terms, emotiveness, pausality, and average sentence size. Also, we considered two representations of the news to compute similarity: term frequency-inverse document frequency, and Doc2Vec, which creates fixed-sized document representations regardless of its length. We evaluated our approach in six datasets written in Portuguese or English, comparing its performance with a binary semi-supervised baseline algorithm, using two well-established label propagation algorithms: LPHN and GNetMine. The results indicate that PU-LP with heterogeneous networks can be competitive to binary semi-supervised learning. Also, linguistic features such as representative terms and pausality improved the classification performance, especially when there is a small amount of labeled news. Supported by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior [10662147/D], Fundação de Amparo à Pesquisa do Estado de São Paulo [2019/25010-5, 2019/07665-4], and Conselho Nacional de Desenvolvimento Científico e Tecnológico [426663/2018-7, 433082/2018-6, and 438017/2018-8]. Access provided by University of Notre Dame Hesburgh Library. Download conference paper PDF Similar content being viewed by others A network-based positive and unlabeled learning approach for fake news detection Article 18 November 2021 Fake News Detection Chapter © 2021 Positive-Unlabeled Learning for Sentiment Analysis with Adversarial Training Chapter © 2019 Explore related subjects Discover the latest articles, books and news in related subjects, suggested using machine learning. Categorization Heterodox Economics Learning algorithms Machine Learning Probabilistic data networks Self-serving bias 1 Introduction Detecting fake news is a challenging task since fake news constantly evolves, influencing the formation of the opinion of social groups as accepted [6]. To minimize the effects of disinformation provided by fake news, Machine Learning algorithms have been proposed in the literature, which learn classification models to discriminate true and fake content [4, 22]. The most common way to deal with this problem is to characterize the fake news detection as a supervised binary or multiclass classification problem [22]. However, labeling real and fake news to cover different subjects, sources, and falsity levels, as well as updating the classification model are costly processes [4]. All of these challenges drive the investigation of more appropriate models for representing and classifying news. Positive and Unlabeled Learning (PUL) algorithms can be good alternatives in this scenario [3]. PUL algorithms learn models considering little labeled data of the interest class and use the unlabeled data to increase classification performance [9]. Therefore, PUL eliminates the need to label a large number of news of the uninteresting classes. Due to the suitability of PUL to fake news detection scenario, in this paper, we proposed an approach based on the algorithm Positive and Unlabeled Learning by Label Propagation [9] (PU-LP), a network-based semi-supervised transductive learning algorithm. PU-LP has been little explored for text classification, despite its good performance in numerical datasets. PU-LP infers reliable sets of interest (fake news) and not interest (true news) using a small number of labeled data, Katz index and a k-Nearest Neighbor (k-NN) network. Then, a label propagation algorithm is used to label the remaining unlabeled objects as interest or not interest. Representing news using traditional models, i.e., Bag-of-Words or embeddings, may not be discriminative enough to distinguish fake and real content [17, 21]. Thus, an important challenge for automatic fake news detection is to assemble a set of features that efficiently characterize a false content in a dynamic scenario. This set may contain features about creator or spreader, the target victims, the news content and the social context [22]. In this work, we propose using content-based features in a domain-independent fake news detection approach based on the PU-LP algorithm. Our network use linguistic features such as representative terms, news emotiveness, average number of words per sentence, and pausality as network objects, thus making a heterogeneous network [12]. These features have proven to be relevant for news classification [13]. Also, the presence of different objects and relations in the network allows for representing different patterns on data [15]. To create the k-NN news network, we represent the text with Bag-of-Words (BoW) and Doc2Vec (D2V) models to compute the similarity between the news. To the best of our knowledge, heterogeneous networks and PUL have not been explored in the literature to detect fake news. We evaluated different combinations of features to measure the impact of these features in the classification performance. Our experiments used relatively small sets of labeled fake news: from 10% to 30% of the fake news set. The heterogeneous network was compared with the k-NN networks, used in the original proposal of PU-LP. Two Portuguese datasets and four English datasets were used, containing only one or more subjects. For the label propagation, we used two well-established algorithms: Label Propagation Through Heterogeneous Networks [12] and GNetMine [7]. We evaluated the algorithms considering the \(F_{1}\) measure of the fake news class. The results obtained the proposed approach were compared with a semi-supervised binary baseline, and demonstrate competitiveness even with half the news initially labeled and without the labeling of real news. This paper is organized as follows: Sects. 2 and 3 present related work. Section 4 describes the proposed approach. Section 5 presents experimental evaluation and discussions. Section 6 presents conclusions and future works. 2 Related Work Existing methods for news classification, in general, represent news in a structured way using traditional models such as BoW, word (or document) embeddings or networks [22]. However, these models are not discriminative enough to capture the nuances to distinguish fake from real news, requiring the investigation of textual and contextual features that can be incorporated into the representation model [17, 21, 22]. Contextual features are extracted from the news environment, such as post, source, and publisher related information [21]. Textual features involve lexical, syntactic and semantic aspects of the news content. Lexical features are related to frequency statistics of words, which can be done using n-gram models, able to identify, for example, the presence of vague, doubt or sensationalist expressions [4]. Syntactic features are related to the presence and frequency of Part-Of-Speech patterns, like subjects, verbs and adjectives [17]. At the semantic level, meanings are extracted from terms, where methods like Linguistic Inquiry and Word Count (LIWC) [10] can be used to estimate emotional, cognitive and structural components present in the written language. In [17], authors proposed representation models for the Fake.BR dataset that combine BoW and linguistic features, such as pausality, emotiveness, and uncertainty. The authors reached 96% of macro \(F_{1}\) considering two thirds of the data as training. In [18], the authors proposed a multimodal representation that combines text and images information. For each news, 124 textual features (81 of them with LIWC) and 43 image features are collected. The approach achieves 95% accuracy with Random Forest using approximately 7,000 news documents and 90% of data in the training set. Some works use networks as a representation model to detect false information. [13] proposes a knowledge network for fact-checking, in which the entry is a triple (subject, predicate, object). The network is constructed with information from Wikipedia, in which given a sentence, if it exists within the knowledge network, it is considered authentic. The approach achieves approximately 74% accuracy. [21] proposes a diffuse neural network graph for inferring the credibility of news, creators and subjects. The graph has approximately 14,000 news items, 3,600 creators and 152 subjects. The approach achieves the best overall performance in the binary classification compared to SVM, Deep Walk and a neural network. [11] proposes an Adversarial Active Learning Heterogeneous Graph Neural Network to detect fake news. A hierarchical attention mechanism is used to learn nodes’ representations, and a selector is responsible for consulting high-value candidates for active learning. The authors evaluate two datasets: the first containing 14,055 news, 3,634 creators and 152 subjects, and the second containing 182 news, 15,257 twitter users and 9 publishers. With 20% of real and fake news labeled, the algorithm reaches respectively 57% and 70% of macro \(F_{1}\). [19] proposes IARNet, an Information Aggregating and Reasoning Network to detect fake news on heterogeneous networks. The network contains the post source, comments and users as nodes and interactions between them as edges. The authors evaluated two datasets, Weibo and Fakeddit, and 70% of the labeled news for training. The algorithm achieved 96% accuracy. To avoid real news labeling efforts, [5] proposes the DCDistanceOCC, a One-class Learning (OCL) algorithm to identify false content. For each news item, linguistic characteristics are extracted, such as number of words per sentence and sentiment of the message. A new example is classified as the interest class if its distance to the class vector is below a threshold. The algorithm is executed using 90% fake news to train the model. The approach reaches an average \(F_1\) that ranges from 54% to 67%, especially considering the Fake.BR dataset. It is possible to observe that most of the existing work for fake news detection adopts binary supervised learning approaches, which requires labeling news considering the real and fake classes. Additionally, the broad spectrum of real news also makes the generation of a representative labeled training set difficult. On the other hand, the PUL algorithms, which mitigate the drawback of binary supervised learning algorithms in fake news detection, are scarce in this scenario. Also, the use of heterogeneous networks, which usually are more adequate in semi-supervised text classification [12], has not been explored in the literature for detecting fake news. Considering these gaps, in this paper we propose a PUL algorithm for fake-news detection, which is based on a heterogeneous network composed of news, terms, and linguistic features. Our approach uses only information that can be collected in the publication’s content, not depending on external information. The proposed approach is detailed in the next sections. 3 Positive and Unlabeled Learning Algorithms PUL algorithms learn models considering a set of labeled documents of an interest class and unlabeled documents to train a classifier (inductive semi-supervised learning) or to classify unlabeled known documents (transductive semi-supervised learning) [3]. The purpose of using unlabeled documents during learning is to improve classification performance. Besides, since the unlabeled documents are easy to collect and a user has to label only a few documents of the interest class, PUL has gained attention in the last years [3, 9]. The most common PUL approaches are those which perform learning in two stages. In the first stage, a set of non-interest documents is generated by extracting reliable outliers documents to be considered as non-interest class. In addition, the set of interest documents can also be increased with reliable interest documents. Once there are interest, non-interest, and unlabeled documents, a transductive learning algorithm is applied to infer the label to the remaining unlabeled documents in the second stage [3]. Generally, the PUL algorithms based on this framework include the application of self-training in some or all of the steps, which can be computationally costly and do not improve the classification performance [12]. We can observe the use of algorithms based on vector-space model, such as Rocchio or Expectation Maximization, also demonstrated lower performances than other approaches in text classification [12, 20]. Despite the benefits of network-based approaches in semi-supervised learning scenarios, there are few PUL proposals based on networks, such as Positive documents Enlarging PU Classifier (PE-PUC) [20] and Positive and Unlabeled Learning by Label Propagation (PU-LP) [9]. PE-PUC uses network representations only to increase the set of interest documents with reliable positive documents. The network is not used in the classification step. The remaining steps consider Bayesian classifiers, which have been shown to perform poorly when the labeled training data set is small [12]. The use of networks allows to extract patterns of the interest class even they are given by different distributions, densities on regions in the space [15]. As PU-LP is entirely network-based, achieving good performance in [9], we propose to use it for news classification. Next section describes the proposed approach. 4 Proposed Approach: PU-LP for Fake News Detection Considering that PUL algorithms can reduce labeling effort for news classification, the benefits of network-based representations for semi-supervised learning, and that adding extra information to the network can contribute to differentiating real and fake content, we propose an approach based on PU-LP algorithm applied to fake news detection. After PU-LP infer interest and non-interest news sets in the news network, we add relations (edges) between news, relevant terms and linguistic features making a heterogeneous network. Thus, a label propagation is applied to classify the unlabeled news. We compared different proposed heterogeneous networks with the traditional PU-LP algorithm considering only the news network. We use different parameters, label propagation algorithms, and representation models for unstructured data to assess the algorithms’ behavior. Figure 1 presents the proposed approach of PU-LP for semi-supervised fake news detection. The next subsections presents details of each stage of the proposed approach. Fig. 1. Proposed approach for detecting fake news based on the semi-supervised PU-LP algorithm. Circular nodes represent news, nodes with the letter t are representative unigrams and bigrams, nodes with the letter p correspond to pausality, e emotiveness, and s the average number of words per sentence. Full size image 4.1 News Collection and Representation Model Let \(\mathcal {D} = \{d_1, \ldots , d_l, d_{l+1}, \ldots , d_{l+u} \}\) be a news set and \(\mathcal {C} = \big \{\textit{interest},\)\( \textit{not interest}\big \}\), i.e., \(\mathcal {C} = \{\textit{fake}, \textit{real}\}\), be a set of class labels. The first l elements of \(\mathcal {D}\) are fake labeled news, composing the interest labeled set \(\mathcal {D}^{+}\). The remaining u elements are unlabeled news (fake and real), composing the set \(\mathcal {D}^{u}\), and \(u \gg l\) (Fig. 1 - Stage 1). The news dataset must be pre-processed, and a representation model, such as BoW or document embedding [1] must be adopted to transform news into structured data (Fig. 1 - Stage 2). Next, PU-LP builds an adjacency matrix with the complete set of examples, \(\mathcal {D}\), detailed below. 4.2 k-NN Matrix, Katz Index and Sets Extraction The representation model and a chosen distance metric are used to calculate an adjacency matrix. In this matrix, news with similar content has a low distance between them. The adjacency matrix is used as a basis for building a k-Nearest Neighbors (k-NN) matrix, called A, so that \(A_{i,j} = 1\) if the news \(d_j\) is one of the k nearest neighbors to the news \(d_i\), and \(A_{i,j} = 0\) otherwise (Fig. 1 - Stage 3). Through the k-NN matrix, a heterogeneous network \(\mathcal {N}\) is also created. The heterogeneous network can be defined as a triple \(\mathcal {N} = \langle \mathcal {O},\mathcal {R},\mathcal {W} \rangle \), in which \(\mathcal {O}\) is the set of objects, \(\mathcal {R}\) is the set of relations between objects, and \(\mathcal {W}\) is the set of weights of these relations. Given two pairs of objects \(o_i, o_j \in \mathcal {O}\), the relationship between them is \(r_{o_i,o_j}\) and the weight of a relation \(r_{o_i,o_j}\) is given by \(w_{o_i,o_j} \forall o_i, o_j \in \mathcal {O}\). In \(\mathcal {N}\), objects are news, the relationships between news are created according to the k-NN matrix, and the weights \(w_{o_i,o_j}\) are the cosine similarity between news. Although the cosine is a good local similarity measure, its performance as a global similarity measure is not good enough. To consider local paths in the news network that involve neighboring news in common, [9] propose using the Katz Index. The basic idea is that if two news have many neighbors in common, they are likely to be of the same class. Katz index is a global similarity measure that calculates the similarity between pairs of nodes considering all possible paths in a graph that connect them. Thus: $$\begin{aligned} sim(d_i,d_j) = \sum ^{\infty }_{h=1} \alpha ^h \cdot |\textit{path}_{d_i,d_j}^{}| = \alpha A_{i,j} + \alpha ^2 (A^2)_{i,j} + \alpha ^3 (A^3)_{i,j} + ..., \end{aligned}$$ (1) in which \(\alpha \) is a parameter that controls the influence of paths. With a very small \(\alpha \), the long paths contribute very little. In the Katz index, when \(\alpha < 1/\epsilon \), being \(\epsilon \) the biggest eigenvalue for the matrix A, Eq. 1 converges and can be calculated as follows: \(S = (I - \alpha A)^{-1} - I\), which I denotes the identity matrix and S has dimnesions \(|\mathcal {D}| \times |\mathcal {D}|\). Thus, \(S_{i,j} \in \mathbb {R}\) denotes the similarity between the nodes \(d_i\) and \(d_j\) according to Katz index (Fig. 1 - Stage 4). The labeled news in \(\mathcal {D}^{+}\) and the similarity matrix S are used to infer two sets: the set of reliable interest news RI and reliable non-interest news RN (Fig. 1 - Stage 5). The set RI contains news from set \(\mathcal {D}^{u}\) that are most similar to examples from \(\mathcal {D}^{+}\), and RN will contain news from \(\mathcal {D}^{u} - RI\) that are most dissimilar to the set \(\mathcal {D}^{+} \cup RI\). For the inference of the RI set, an iterative method is applied. The task is divided into m iterations. In each of the m iterations, the total number of reliable interest news that will be extracted is \((\lambda /m) \times |\mathcal {D}^{+}|\), which \(\lambda \) controls the set’s size. The news in \(\mathcal {D}^{u}\) are ranked according to their average similarities for all news in \(\mathcal {D}^{+}\) based on S. The \((\lambda /m)|\mathcal {D}^{+}|\) most similar news are taken from \(D^{u}\) and added into \(RI'\). At the end of each iteration, RI is incremented with the elements in \(RI'\) (\(RI \leftarrow RI \cup RI'\)) [9]. For the inference of the reliable non-interest set, news in \(\mathcal {D}^{u} - RI\) are ranked according to their average similarities (based on S) for all news in \(\mathcal {D}^{+} \cup RI\). The algorithm extracts the \(|\mathcal {D}^{+} \cup RI|\) most dissimilar news, forming the set RN. After getting the set RN, the sets \(\mathcal {D}^{+} \cup RI\), RN, and \(\mathcal {D}^{u} \leftarrow (\mathcal {D}^{u} - RI - RN\)) are used as input by label propagation algorithms based on transductive semi-supervised learning. More details about the algorithm can be seen in [9]. 4.3 Adding Features in the News Network Considering that the PU-LP algorithm is entirely based on networks, we propose adding new features as nodes into the news network \(\mathcal {N}\) that has been shown to be relevant in [2, 17] in news classification: incertainty = total number of modal verbs and passive voice non immediacy = total of singular first and second personal pronoun emotiveness (emo) = \(\frac{\text {total number of adjectives + total number of adverbs}}{\text {total number of nouns + total number of verbs}}\) pausality (pau) = \(\frac{\text {total number of punctuation marks}}{\text {total number of sentences}}\) average words sentence (avgws) = \(\frac{\text {total number of words}}{\text {total number of sentences}}\) Among the features considered, we compute for each dataset which ones had the highest correlation with the target attribute and appeared in more than one dataset. The characteristics chosen to be included in the network were: pausality, emotiveness, and average words per sentence. Table 1 shows the correlations values. The features are added to the news network \(\mathcal {N}\) as new unlabeled nodes \(f_j\), \(0