Frontiers in Business, Economics and Management ISSN: 2766-824X | Vol. 10, No. 1, 2023 58 A Review of Text Sentiment Analysis Methods and Applications Yuxin Jin1, 2, Kui Cheng2 and Xinjie Wang1, 2, Lecai Cai2, * 1 School of Automation and Information Engineering, Sichuan University of Science & Engineering, Yibin 644000, China 2 Sanjiang Artificial Intelligence and Robot Research Institute, Yibin 644000, China * Corresponding author Abstract: This paper reviews the application of natural language processing in sentiment analysis. Sentiment analysis is an important task aimed at automatically identifying and inferring sentiment tendencies and sentiment intensity in texts. This paper first introduces the application areas of sentiment analysis, including practical applications of text sentiment analysis. Then, text pre-processing techniques such as word separation, deactivation removal and punctuation processing are discussed. Then, feature extraction and representation methods are explored, including bag-of-words model, TF-IDF, word embedding and Word2Vec, attention mechanism and Transformer. In addition, methods for sentiment analysis, such as sentiment dictionaries and rule-based methods, traditional machine learning methods, and deep learning-based methods, are presented. Finally, the application areas of sentiment analysis are discussed and conclusions are given. The review in this paper will help readers understand the current status and development trend of natural language processing applications in sentiment analysis, as well as the advantages and disadvantages of different methods in sentiment analysis. Keywords: Natural Language Processing, Sentiment Analysis, Machine Learning, Deep Learning. 1. Introduction Natural Language Processing (NLP) is an important research direction in computer science and artificial intelligence, aiming to enable computers to understand, process and generate human language. Text sentiment analysis is an important task in NLP, which automatically identifies and inferred the sentiment tendency and intensity in text by extracting information from user opinions to obtain key sentiment information[1]. With the increase of text data such as social media and online comments, sentiment analysis is crucial to understand user needs, market dynamics and changes in public opinion[2]. 2. Applications in Text Sentiment Analysis 2.1. Text Pre-Processing Technology Before text sentiment analysis, text pre-processing is a key step that involves cleaning, normalizing and transforming the raw text data for subsequent feature extraction and analysis[3]. The specific text preprocessing process is shown in Figure 1. Figure 1. Data Pre-processing Flow Chart 2.1.1. Participle and Word Marking Phrase splitting slices a text sequence into words to understand the sentiment tendency of each word[4]. Commonly used word separation algorithms include lexical, rule-based and statistical approaches[5]. Chinese word separation is an important pre-processing step, and commonly used tools include jieba[6] and THULAC[7]. English can be simply separated by spaces. Lexical annotation determines lexical properties for words, such as nouns, verbs, adjectives, etc. Lexical annotation provides contextual information and helps to identify emotionally colored words. Commonly used tools are Hidden Markov Model, Maximum Entropy Model and Conditional Random Field[8][9][10][11][12][13][14]. 2.1.2. Deactivation and Punctuation Handling Deactivation and punctuation processing are common steps in text pre-processing, which aim to cleanse and purify text data by removing words and symbols that are irrelevant or distracting to sentiment analysis[15]. Deactivated words are words that appear frequently in the text but have no real meaning for the task of sentiment analysis, such as "of", "had", "is", and common pronouns, prepositions, conjunctions, etc. The presence of deactivated words adds to the text. The presence of deactivated words increases the complexity of text processing and may interfere with the results of sentiment analysis. In sentiment analysis, removing deactivators can reduce noise and redundant information, and improve the performance and effectiveness of sentiment analysis models[16]. The list of commonly used deactivated words can be obtained from open source web libraries or custom vocabularies, and the removal of deactivated words can be done by simple rule-based methods or based on a list of deactivated words[17]. In addition, various dictionaries exist in different domains, such as the U.S. Army Military Dictionary[18], the Attitude Dictionary, the Extended Sentiment Dictionary, and the Sentiment Dictionary for the book domain based on ultra-short reviews, developed based on terminological theory[19]. Punctuation processing refers to the removal of 59 punctuation marks from text or their processing[20]. Punctuation usually does not carry sentiment information in sentiment analysis, and it adds complexity to the text, which may interfere with the process of feature extraction and sentiment classification. Therefore, removing punctuation marks can simplify the structure of text and improve the accuracy of sentiment analysis. Common punctuation marks include periods, commas, exclamation marks, question marks, etc. 2.1.3. Morphological reduction and stemming Word form reduction and stem extraction in sentiment analysis serve to reduce words to their original forms or to extract their stems in order to reduce variations in the surface forms of words and to capture core meaning[21]. Morphological reduction reduces words to their basic form by considering information such as root, lexical nature and context. Stem extraction, on the other hand, extracts word stems and reduces the dimensionality of the feature space. This better captures the true meaning of words and reduces redundancy. Commonly used techniques include dictionary- based and rule-based approaches[22]. Word form reduction and stemming extraction provide more accurate and consistent feature representation, eliminate feature redundancy and noise, and improve model accuracy and ability to capture sentiment tendencies. However, appropriate tools need to be selected, tuned and optimized according to the task and dataset. 2.2. Feature Extraction and Representation Methods 2.2.1. Bag-of-words model and TF-IDF Feature extraction and representation methods are crucial in sentiment analysis, transforming raw text into computer- processable numerical features[23]. Bag-of-words model and TF-IDF are commonly used text feature extraction and representation methods for capturing the frequency and importance of words and applying them to sentiment analysis tasks[24]. The bag-of-words model treats text as a collection of words, ignoring order and grammatical structure and focusing only on the frequency of word occurrences[25]. The text is converted into a numerical representation by constructing a document-word matrix. The bag-of-words model assesses the importance of words by assuming that words with higher frequencies are given more weight in the representation. TF-IDF (word frequency-inverse document frequency) is a method that integrates word frequency and document frequency to evaluate the importance of words in text[26]. TF-IDF filters out common words and highlights the important words in a given document. Its formula is defined as follows:   , , log : i i j i i j ij k k j ij n tf n D idf j t d TF IDF f idf        (1) The advantages of the bag-of-words model and TF-IDF are their simplicity and ease of implementation, and their ability to effectively represent lexical information of text. They are suitable for sentiment analysis, revealing users' opinions and emotional tendencies, capturing the frequency and importance of keywords. However, they also have limitations, such as the inability to capture word order and contextual information, and ignoring semantic relationships. 2.2.2. Word embedding and Word2Vec Word embeddings and Word2Vec play an important role in sentiment analysis. Word embedding converts text into a dense vector representation that better captures the semantic information of words[27]. This continuous vector representation provides better feature representation and semantic similarity computation, which helps to identify sentiment tendencies and expressions. Using the pre-trained Word2Vec model can extract the contextual relationships of words and improve the performance of sentiment analysis models[28]. Word embedding maps words to a continuous vector space and provides rich feature representation by learning to capture the semantic relationships of words, making semantically similar words closer together in the vector space. It uses language models or neural networks to learn word embedding models on large-scale text data, making similarly meaningful words closer together in vector space. Word2Vec is a commonly used word embedding algorithm based on a shallow neural network model trained on large- scale text data to generate high-quality word vectors by predicting contextual information around words. It consists of a continuous bag-of-words model (CBOW) and a Skip-gram model, both of which generate word embedding vectors that capture the semantic features of words. Its network model diagram is as follows: Figure 2. CBOW and Skip-gram Models Word embeddings better capture the semantic information of words and usually perform well in sentiment analysis. word2Vec is an important feature extraction and representation method that captures semantic relations and contextual information of words and provides rich feature representation, thus improving the accuracy and effectiveness of sentiment analysis. 2.2.3. Attention mechanism and Transformer The Attention mechanism is an important feature representation, similar to the attention mechanism used by humans in understanding text[29]. It solves the parallelism and long sequence dependency problems in RNN and LSTM networks. The attention mechanism learns the correlation between different parts of a text and focuses on important pieces of information, assigning weights to different parts in a specific task. In text sentiment analysis, the attention mechanism improves model performance by focusing on important contextual information to better capture the semantic and sentiment expressions of the text and identify the words that contribute to sentiment classification. Transformer[30] model is a deep learning model based on 60 the attention mechanism, a sequence transformation model based entirely on attention that replaces the cyclic layer with a multi-headed self-attentive mechanism.Transformer model uses multiple self-attentive layers to encode the input sequence by simultaneously considering information at different locations in the sequence, capturing the dependencies between global and local, and improving the sentiment analysis accuracy and effectiveness. Compared with the traditional recurrent neural network (RNN), the Transformer model has parallel computing capability, improves training efficiency, and can handle longer text sequences, performing well in sentiment analysis tasks. In summary, feature extraction and representation methods play a key role in natural language processing sentiment analysis. Bag-of-words model and TF-IDF are used to capture lexical information, word embedding and Word2Vec are used to represent semantic relationships of words, and attention mechanism and Transformer model can focus on important text fragments and contextual information. The combined application of these methods can improve the accuracy and effectiveness of sentiment analysis and help analyze people's opinions, attitudes and emotions, etc. 2.3. Emotional Analysis Method 2.3.1. Emotional lexicon and rules-based approach The sentiment lexicon and rule-based approach is a commonly used sentiment classification technique. The method uses pre-constructed sentiment dictionaries and rules for sentiment classification, and the overall sentiment polarity is derived by matching and weighting the sentiment words in the text[31]. The sentiment lexicon contains vocabulary and corresponding sentiment tendencies, which can be constructed manually or obtained by automatic mining and machine learning. The process of sentiment lexicon analysis is as follows: word segmentation and lexical annotation segment the text into words or phrases and determine the lexical nature of each word. By matching the sentiment lexicon, the sentiment words present in the text are identified. The sentiment score of the text is calculated based on the number, position and sentiment polarity of the sentiment words. Common calculation methods include simple counting, weighted counting and rule-based matching[32]. Figure 3: Flow Chart of Sentiment Analysis Method Based on Sentiment Dictionary Rules can also be used to handle negation, degree adverbs and context dependencies. Negation words can change the sentiment polarity of subsequent sentiment words, and degree adverbs can adjust the weights of sentiment words to reflect sentiment intensity. Rule matching and sentiment adjustment can be performed according to the context dependency of sentiment expressions. English sentiment dictionaries are more maturely developed, and the common ones are SentiWordNet and General Inquirer Opinion Lexicon[33]. Chinese sentiment dictionaries include HowNet, a Zhiwang dictionary, NTUSD of National Taiwan University and Dalian University of Technology's Chinese Sentiment Vocabulary Ontology Library[1]. The method based on sentiment lexicon and rules has the advantages of simplicity, intuitiveness and interpretability. It does not require a large amount of training data and is computationally efficient. However, the method also has limitations. The quality and coverage of sentiment dictionaries directly affect the classification accuracy, and the lack of domain-specific or new vocabulary sentiment information may lead to classification errors[34]. In addition, the rule-based approach has difficulty capturing complex semantic and contextual relationships. 2.3.2. Traditional machine learning based approach In sentiment analysis, traditional machine learning-based approaches use labeled sentiment datasets for model training and prediction. The approach converts text features into numerical features and performs sentiment classification using classification algorithms, such as plain Bayes[35], support vector machines[36], decision trees[37] and random forests[38]. Feature selection and model tuning are key steps, where important features can be selected by information gain and mutual information, and parameters can be tuned using cross-validation, grid search, etc. Sentiment analysis methods based on traditional machine learning have the advantages of being explanatory and suitable for small-scale datasets. They can provide explanatory relationships between features and classification results, and can incorporate domain knowledge for feature engineering. However, the performance of traditional machine learning methods may be limited when dealing with large-scale datasets and complex contexts. In addition, feature engineering requires manual involvement and high requirements for domain knowledge. 2.3.3. Deep learning based sentiment analysis method Deep learning-based sentiment analysis methods use multilayer neural networks to learn abstract feature representations. Trained with large-scale datasets, these methods are able to automatically extract key semantic features of text and implement sentiment classification tasks. At the core of deep learning methods are neural network models, as shown in Figure 4. In sentiment analysis, commonly used neural network models include convolutional neural network (CNN)[39], recurrent neural network (RNN) and long and short term memory network (LSTM)[40], Transformer[30], etc. Figure 4. Neural Network Model CNN (Convolutional Neural Network) suffers from the lack of sequence modeling capability in sentiment analysis 61 and has limited ability to model text sequences due to the limitation of local perceptual fields and weight sharing to capture long-range dependencies. Figure 5. Convolutional Neural Networks On the other hand, RNN (Recurrent Neural Network) is able to retain contextual information and model sequence dependencies, but is prone to gradient disappearance and gradient explosion problems, and has limited memory capacity when dealing with long sequences, which affects the accuracy of sentiment classification[41]. Figure 6. Recurrent Neural Networks To overcome these problems, LSTM (long short-term memory network) introduces a gating mechanism and memory units to solve the gradient problem and long-term dependency problem and improve the performance of sentiment analysis. Figure 7. Long Short-Term Memory Networks However, the computational complexity of LSTM is high and still has limitations for very long text sequences. When LSTM is used for text sequences of more than 200 words, it still has more serious long-term since problem and generates higher errors. Transformer model is an emerging model after RNN and LTSM, which is widely used in text sentiment analysis. Based on the self-attention mechanism and the introduction of parallel computing capability, the processing and understanding of text sequences are more efficient, while the attention mechanism can solve the problem of long-distance dependence of sequences. The essence of the attention mechanism is to selectively filter a small amount of important information from a large amount of information and focus on this important information, ignoring the unimportant information[42]. By weighting the weights at different positions in the input sequence, it enables the model to focus more on the important parts when processing the sequence data. Its structure diagram is shown in Figure 8. Figure 8. Attention Mechanism Module In the Transformer model, the attention mechanism is applied to the Self-Attention mechanism (Self-Attention)[43]. The Self-Attention mechanism calculates the interactions of each position in the sequence with other positions and assigns a weight to each position based on the similarity. By weighting and summing the attention weights of each position, the contextual representation of each position and the degree of contribution can be obtained. The advantage of this attention mechanism is that it can fuse information from different positions in the sequence to establish global semantic associations and thus better understand the semantics and structure of the whole sequence. Figure 9. Self-Attention Mechanism Module The difference between the self-attention mechanism and the attention mechanism lies in the fact that Q, K, and V of the self-attention mechanism are homologous, i.e., they all come from the same word vector. Multi-head Self Attention[44] is an extended self-attentive mechanism model in Transformer, i.e., the superposition of 62 multiple multi-head self-attentive mechanisms constitutes the encoder-decoder structure in Transformer. It slices the input into eight independent attention heads to compute the self- attention in parallel, each with different K, V, and Q, which is equivalent to eight linear transformations of the original self-attention, so that different representation spaces can be learned. Transformer[30], proposed by Vaswani et al. is a neural network integrated with a multi-headed self-attentive mechanism. The model adopts an encoder-decoder structure, which solves the problems such as inability to compute in parallel with long distance dependence in RNN and LTSM. In sentiment analysis tasks, the Transformer model possesses an extremely powerful context modeling capability[45]. It can encode and decode text sequences, capture semantic information between keywords and contexts, and effectively construct dependencies in sequences. The overlay computation by multi-headed self-attentive mechanism enables the model to better understand the semantic and sentiment expressions in sentences and extract the features that contribute to sentiment classification. Meanwhile Transformer introduces operations such as residual module and batch normalization, which effectively avoid the problems of gradient disappearance and gradient explosion during network training, while improving the training efficiency. Its schematic diagram is shown in Figure 10. Figure 10. Transformer Module However, deep learning-based sentiment analysis methods also have some challenges. First, deep learning methods have a high demand for data and require a large amount of labeled data for model training. Second, deep learning models tend to be more complex and require more computational resources for training and inference, and also lack interpretability to explain the prediction results of the models. In summary, sentiment lexicon and rule-based methods are simple and intuitive, and are suitable for small-scale data and simple sentiment analysis tasks. Traditional machine learning methods have good explanations and interpretability and perform better on small data sets. Deep learning methods have achieved remarkable results on large-scale data and complex contexts by learning abstract semantic feature representations. Future research can explore the fusion of different methods to improve the accuracy and robustness of sentiment classification and combine domain knowledge and contextual information to achieve more refined and personalized sentiment analysis applications. 3. Text Sentiment Analysis Application Areas The large amount of user-generated text data on social media platforms contains rich sentiment information, and sentiment analysis can be used to analyze users' sentiment tendencies towards specific events, products or topics, thus providing insights about users' emotional attitudes. For example, in microblog sentiment analysis, based on sentiment lexicon, Yuqing Li et al. proposed a bilingual lexicon sentiment analysis method based on KNN algorithm[46], and Yanyan Zhao used the huge data volume of microblogs to build a mega sentiment lexicon[47], which has significantly improved the effect of sentiment classification on microblogs. Sentiment analysis can also help companies understand consumers' emotional attitudes and emotional needs towards brands, so that they can develop more targeted marketing strategies and brand management strategies. For example, Asghar[48] et al. produce sentiment dictionaries integrating emoticons and domain terms for hotel review data, and obtain hotel review sentiment for improving hotel facilities to meet people's needs. Min Peng[49] et al. use LDA topic model to analyze potential attributes of goods and calculate users' emotional tendency and user similarity to the attributes of goods, which enables merchants to better design more perfect goods according to customers' emotional preferences. Jinmeng Li combines sentiment analysis algorithms with eye- tracking devices[50], where users wear eye-tracking devices to derive the area of gaze on the text, and finally the sentiment value of the user's area of interest is derived from the algorithmic model to determine the user's preference for the content . 4. Conclusion This review provides a comprehensive overview of the application of natural language processing in text sentiment analysis. Sentiment features can be effectively extracted from text by text pre-processing techniques and feature extraction methods. Different sentiment classification algorithms and sentiment intensity analysis methods can help to understand the sentiment tendency and intensity in texts. Moreover, multilingual sentiment analysis and case studies in different application domains demonstrate the wide application of sentiment analysis. However, there are still challenges such as data scarcity, model generalization capability and interpretability that need to be addressed. Future research can combine multimodal data and emerging technologies to improve the accuracy and efficiency of sentiment analysis and provide more valuable information for decision making in various industries. Acknowledgment Fundamental project: Sichuan University of Science & Engineering Graduate Student Innovation Fund (Y2022141), Design and Application of Text Sentiment Analysis Model Based on Eye Tracking Core+ Eye Movement Instrument. 63 References [1] Zhong Jiawa, Liu Wei, Wang Sili et al. Review of Methods and Applications of Text Sentiment Analysis[J]. Data Analysis and Knowledge Discovery,2021,5(06):1-13. [2] Wang YJ, Zhu JQ, Wang ZM et al. Review of Applications of natural language processing in text sentiment analysis [J/OL]. Computer Applications:1-12 [2023-07-03]. [3] Zhao Jingsheng, Song Mengxue, Gao Xiang, et al. A study of text representation in natural language processing[J]. Journal of Software, 2021, 33(1): 102-128. [4] Li Yachao, Xiong Deyi, Zhang Min. A review of neural machine translation[J]. Journal of Computer Science, 2018, 41(12): 2734-2755. [5] Ao Sheng,Xu Lan,Ao Qingwen.Application of NLP Chinese word separation technology in bridge report data processing[J]. Traffic World,2020(17):3-5.DOI:10.16248/j.cnki.11- 3723/u.2020.17.001. [6] Wang YY, Dong GW. Research on the problems and countermeasures of netroots libraries based on jieba subtext[J]. Jiangsu Science and Technology Information,2023,40(06):35- 38. [7] Yin R, Wang Q, Li P, et al. Multi-granularity chinese word embedding[C]//Proceedings of the 2016 conference on empirical methods in natural language processing. 2016: 981- 986. [8] Zhao Pengfei,Zhao Chunjiang,Wu Huarui et al. BERT-based multi-feature fusion for named entity recognition in agriculture[J]. Journal of Agricultural Engineering, 2022, 38(03): 112-118. [9] Chen Zhiyan,Li Xiaojie,Zhu Shuhua et al. A two-way maximal matching word splitting method based on Hash structured dictionary[J]. Computer Science,2015,42(S2):49-54. [10] Li Ship. Statistics in Chinese word splitting[J]. China Statistics, 2020(10):34-35. [11] Liu, X.Y.Y., Sheng, Y.H., Qin, J.R., et al. A semantic matching method for spatio-temporal trajectories based on hidden Markov model[J]. Geography and Geographic Information Science,2023,39(03):1-6. [12] Liu, Yunzhong, Lin, Yaping, Chen, Zhiping. Hidden Markov model-based text information extraction[J]. Journal of System Simulation, 2004, 16(3): 507-510. [13] Li Ronglu, Wang Jianhui, Chen Xiaoyun, et al. Chinese text classification using maximum entropy model[D]. , 2005. [14] Zhou J.S., Dai X.Y., Yin Cunyan, et al. Automatic recognition of Chinese organization names based on cascaded conditional random field model[J]. Journal of Electronics, 2006, 34(5): 804. [15] Huang Bo, Liu Chuancai. Automatic Chinese text summarization based on weighted Textrank[J]. Application Research of Computers/Jisuanji Yingyong Yanjiu, 2020, 37(2). [16] Zhang, Liu, Wang, Liwei, Huang, Bo, et al. Sentiment classification model and experimental study of microblog comments based on word vectors with multi-scale convolutional neural network[J]. Library Intelligence Work, 2019, 63(18): 99. [17] Gong, Shuang-Shuang, Chen, Yu-Feng, Xu, Jin-An, et al. A multi-word expression extraction method for Chinese based on web text[J]. Journal of Shandong University (Science Edition), 2018, 53(9): 40-48. [18] Zheng Zheng,Wang Huadan,Hao Lihua. A study of the U.S. military language dictionary from a terminological perspective[J]. China Science and Technology Terminology, 2021, 23(03): 33-41. [19] Zhou Zhi,Wang Chunying,Zhu Jia-Li. Research on the construction of an emotion lexicon in book domain based on ultra-short reviews[J]. Intelligence Theory and Practice, 2021, 44 (09): 183-189+197.DOI:10.16353/j.cnki.1000- 7490.2021.09.026. [20] Liu P, Wang W, Qiu L, et al. CDCPP: (CDCPP: Cross-Domain Chinese Punctuation Prediction)[C]//Proceedings of the 19th Chinese National Conference on Computational Linguistics. 2020: 593-603. [21] Yi, Shun-Ming, Yi, Hao, Zhou, Guo-Dong. Research on Twitter sentiment classification method using sentiment feature vector[J]. Small Microcomputer Systems, 2016, 37(11): 2454-2458. [22] Zhang Huaping, Liu Qun. A coarse classification model for Chinese words based on the N shortest path method[J]. Chinese Journal of Information, 2002, 16(5): 1-7. [23] Tang Huifeng, Tan Songbo, Cheng Xueqi. A comparative study of Chinese sentiment classification techniques based on supervised learning [D]. , 2007. [24] Lu YH, Li YF. An improved TF-IDF algorithm for computing the weights of textual feature items[J]. Library Intelligence Work, 2013, 57(03): 90. [25] Han, Tong-Hui, Yang, Dong-Qiang, Ma, Hong-Wei. A method for constructing text feature representations using sentiment word statistics[J]. Application Research of Computers/Jisuanji Yingyong Yanjiu, 2019, 36(7). [26] Ramos J. Using tf-idf to determine word relevance in document queries[C]//Proceedings of the first instructional conference on machine learning. 2003, 242(1): 29-48. [27] Xue Yanfei, Mao Qirong, Zhang Jianming. A deep recommendation model incorporating contextual information [J]. Application Research of Computers/Jisuanji Yingyong Yanjiu, 2021, 38(4). [28] Rong X. word2vec parameter learning explained[J]. arXiv preprint arXiv:1411.2738, 2014. [29] Shi L, Wang Y, Cheng Y, et al. A review of attention mechanisms in natural language processing[J]. Data Analysis and Knowledge Discovery, 2020, 4(5): 1-14. Ren Huan, Wang Xuguang. A review of attention mechanisms[J]. Computer Applications, 2021, 41(S01): 1-6. [30] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30. [31] Chen L C, Lee C M, Chen M Y. Exploration of social media for sentiment analysis using deep learning[J]. Soft Computing, 2020, 24: 8187-8197. Pan D, Yuan J, Li L, et al. Deep neural network-based classification model for Sentiment Analysis [C]//2019 6th International Conference on Behavioral, Economic and Socio-Cultural Computing (BESC). IEEE, 2019: 1-4. [32] Zhang ZQ, Ye Q, Li YJ. A review of sentiment analysis of Internet product reviews[J]. Journal of Management Science, 2010, 13(006): 84-96. [33] Deng S, Sinha A P, Zhao H. Adapting sentiment lexicons to domain-specific social media texts[J]. Decision Support Systems, 2017, 94: 65-76. [34] Chen Yan-Fang, Li Zhi-Yu, Liang Zhuan, et al. A review of online social network rumor detection[J]. Journal of Computer Science, 2018, 41(7): 1648-1676. [35] Li Jingmei, Sun Lihua, Zhang Qiaorong, et al. A plain Bayesian classifier for text processing[J]. Journal of Harbin Engineering University, 2003, 24(1): 71-74. 64 [36] Xiao Z, Liu F, Li B. A semantic distance-based SVM sentiment classification method for Web reviews[J]. Computer Science, 2014, 41(9): 248-252. [37] Liu Gang, Zhang Weishi. Sentiment analysis of Internet users' evaluation based on decision tree[J]. Modern Computer: Midterm Journal, 2017 (11): 15-19. [38] Neethu M S, Rajasree R. Sentiment analysis in twitter using machine learning techniques[C]//2013 fourth international conference on computing, communications and networking technologies (ICCCNT). IEEE, 2013: 1-5. [39] O'Shea K, Nash R. An introduction to convolutional neural networks[J]. arXiv preprint arXiv:1511.08458, 2015. [40] Sherstinsky A. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network[J]. Physica D: Nonlinear Phenomena, 2020, 404: 132306. [41] Zhao YY, Wang ZY, Wang P, et al. A review of task-based dialogue systems[J]. Journal of Computer Science, 2020, 43(10): 1862-1896. [42] Shan, W.F., Li, C.Y., Chen, J., et al. Application of convolutional neural network and self-attentive mechanism to identify geomagnetic field disturbance events[J]. Seismic Geomagnetic Observation and Research, 2022, 43(5): 49-63. [43] Shaw P, Uszkoreit J, Vaswani A. Self-attention with relative position representations[J]. arXiv preprint arXiv:1803.02155, 2018. [44] Voita E, Talbot D, Moiseev F, et al. Analyzing multi-head self- attention: Specialized heads do the heavy lifting, the rest can be pruned[J]. arXiv preprint arXiv:1905.09418, 2019. [45] Chen Chen, Xia-Bing Zhou, Zhong-Qing Wang, et al. Conversational sentiment classification based on multi-party attention modeling[J]. Chinese Journal of Informatics, 2022, 36(12): 173-181. [46] Li Yuqing,Li Xin,Han Xu et al. A bilingual lexicon-based sentiment analysis method for micro-bodied categories[J]. Journal of Electronics,2016,44(09):2068-2073. [47] Zhao Yan-Yan,Qin Bing,Shi Qiu-Hui et al. Construction of large-scale sentiment lexicon and its application in sentiment classification[J]. Journal of Chinese Information, 2017, 31(02): 187-193. [48] Asghar M Z, Khan A, Ahmad S, et al. Lexicon-enhanced sentiment analysis framework using rule-based classification scheme[J]. PloS one, 2017, 12(2): e0171649. [49] Peng M, Xi J, Dai XY, et al. Collaborative filtering recommendation algorithm based on sentiment analysis and LDA topic model[J]. Chinese Journal of Information, 2017, 31(02):194-203. [50] Li Jinmeng. Research on natural language emotion analysis based on eye-tracking [D]. Beijing University of Posts and Telecommunications, 2021. DOI:10.26969/d.cnki.gbydu.2021.001688.