JURNAL RISET INFORMATIKA Vol. 2, No. 4 September 2020 P-ISSN: 2656-1743 |E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v2i4.162 1 The work is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License SENTIMENT ANALYSIS OF DIGITAL WALLET SERVICE USERS USING NAÏVE BAYES CLASSIFIER AND PARTICLE SWARM OPTIMIZATION Alvie Delia Cahyani1, Tati Mardiana2, Laela Kurniawati3 Information System1.3; Information Technology2 STMIK Nusa Mandiri1.3; Universitas Bina Sarana Informatika2 http://www.nusamandiri.ac.id1.3, http://www.bsi.ac.id2 E-mail : adeayu.aar@gmail.com1, tati.ttm@bsi.ac.id2, laela@nusamandiri.ac.id3 Abstrak— Layanan dompet digital memberikan banyak kemudahan dan keuntungan kepada para penggunanya. Namun, tidak semua pengguna layanan dompet digital memiliki opini yang positif terhadap layanan tersebut. Oleh karena itu, perusahaan layanan transportasi online perlu melakukan analisis untuk mengetahui sentimen masyarakat terhadap produknya. Metode Naïve Bayes Classifier merupakan metode yang sederhana, cepat, berakurasi tinggi, dan mempunyai performa yang cukup baik untuk melakukan klasifikasi data. Namun, metode Naïve Bayes Classifier mengasumsikan atributnya indepensi sehingga dapat menyebabkan akurasinya kurang optimal. Tujuan penelitian ini adalah mengoptimalkan metode Naïve Bayes classifier menggunakan Particle Swarm Optimization dalam klasifikasi polaritas percakapan layanan dompet digital. Penelitian ini menggunakan data yang dari Twitter sebanyak 490 data tweet. Hasil pengujian dengan confusion matrix dan kurva ROC menunjukkan peningkatan akurasi metode Naïve Bayes Classifier dompet digital Dana dari 60.00% menjadi 91.67% dan dompet digital iSaku dari 53.23% menjadi 85.00%. Hasil uji T-Test dan Anova menunjukkan hasil pengujian pada kedua metode klasifikasi memiliki perbedaan yang nyata dalam nilai accuracy. Kata Kunci: Dompet Digital; Twitter; Analisis Sentimen; Naïve Bayes Classifier; Particle Swarm Optimization. Abstract— Digital wallet services adequately provides many benefits to its users. However, not all users of digital wallet services obtain a positive opinion about the service. Therefore, online transportation service companies need to carry out an analysis to determine general sentiment towards their products. The Naïve Bayes Classifier method represents a method that is simple, fast, excellent accuracy and obtains a comparatively excellent performance for classifying data. However, the Naïve Bayes Classifier method assumes that the attributes are independent so that it can cause the accuracy to obtain less than optimal. This study aims to improve the accuracy of the Naive Bayes classification for the classification of public opinion on digital wallet services using Particle Swarm Optimization. This study manages data from Twitter as much as 490 tweet data. The test results with confusion matrix and ROC curves show an increase in the accuracy of the Naïve Bayes Classifier method for the Dana digital wallet from 60.00% to 91.67% and the iSaku digital wallet from 53.23% to 85.00%. The results of the T-Test and Anova test show that the test results of both classification methods provide significant differences in the accuracy value. Keywords: Digital Wallet; Twitter; Sentiment Analysis; Naïve Bayes Classifier; Particle Swarm Optimization. INTRODUCTION In recent years, the popularity of digital wallet services in Indonesia has experienced significant growth. Digital wallets represents electronic money that introduces people to easy, safe, and profitable cashless or non-cash payment methods (Aaputra, Didi Rosiyadi, Windu Gata, & Syepry Maulana Husain, 2019). Based on Bank Indonesia data, there was an increase in the number of electronic money transactions including digital wallets at the end of 2019 soaring 79.3% to 5.2 billion compared to 2018 of 2.9 billion transactions. Moreover, the total nominal value of electronic money transactions experienced a drastic increase of 208.5%, reaching145 trillion rupiahs. This figure has increased to 98 trillion rupiahs or almost three times compared to 2018 of 47 trillion rupiahs (Budiansyah, 2020). As of April 2020, 49 e-money companies have received official licenses from Bank Indonesia, including banks, technology, and communication companies (Indonesia, 2020). http://creativecommons.org/licenses/by-nc/4.0/ mailto:adeayu.aar@gmail.com1 mailto:tati.ttm@bsi.ac.id2 P-ISSN: 2656-1743 | E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v2i4.162 JURNAL RISET INFORMATIKA Vol. 2, No. 4 September 2020 2 The work is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License Not all digital wallet services acquire several active users. This occurs due to a lack of public trust in the security of these services, and they still consider this as something new. Based on the research results of iPrice Group and App Annie, the ten digital wallet applications that have most significant number of monthly active users in the second quarter of 2019 are GoPay, Ovo, Dana, LinkAja, Jenius, Go Mobile, iSaku, Sakuku, Doku, and Paytren, while the most downloaded digital wallet applications in the second quarter of 2019 are GoPay, Ovo, Dana, LinkAja, iSaku, Jenius, Go Mobile, Paytren, Sakuku, and Doku (Devita, 2019). Digital wallet services adequately provides many benefits and advantages to their users. However, not all users of digital wallet services obtain a positive opinion of the service. In this digital era, people can provide opinions about a service, product, politics, and other topics through social media like Twitter (Mahendrajaya, Buntoro, & Setyawan, 2019). The number of daily Twitter users jumped up in the 3rd quarter of 2019, daily users on Twitter increased by 17% to 145 million users. One of the countries experiencing the most massive growth in active daily Twitter users in Indonesia (Clinten, 2019). The number of tweets sent to Twitter is up to 500 million daily tweets and 200 billion tweets (Maulana, 2016). Companies need to do sentiment analysis to find out what the public thinks about their products or services (Pertiwi, 2019). Sentiment analysis can extract public opinion on certain topics, politics, products, or services contained in unstructured texts (Saidah & Mayary, 2020). The previous research related to the analysis of the sentiment of digital wallet service users, namely research conducted by Mahendrajaya, Buntoro, and Setyawan using the Lexicon Based and Support Vector Machine methods. The sentiment labeling of GoPay user comments from Twitter is 923 positive comments and 287 negative comments. The sentiment labeling implements the Lexicon Based method. This study implements the SVM method for classification of comparing two kernels. The linear kernel classification of comments as many as 1109 reviews got an accuracy value of 89.17% of the results. While the polynomial kernel with a comment classification of 1021 reviews got an accuracy value of 84.38%. The results of this study concluded that the classification of tweet data on Twitter for GoPay user comment reviews using the SVM method and linear kernel is good (Mahendrajaya et al., 2019). Based on the description above, the researchers researched analyzing the sentiment of users of the Dana and iSaku digital wallet services, because not many have researched these digital wallets. We humbly propose implementing Particle Swarm Optimization to improve the Naïve Bayes classifier method in the classification of public opinion on digital wallet services. This study aims to increase the accuracy value and AUC value in the sentiment analysis of digital wallet service users by adding attribute weights using PSO to the Naïve Bayes Classifier method. MATERIALS AND METHODS The stages carried out in this study used the Cross Standard Industry Process for Data Mining or CRISP-DM methodology, as follows (Pratama, Pradnyana, & Arthana, 2020): 1. Business Understanding Business Understanding is a stage in recognizing a business process in a company or organization in the form of business objectives, assessing conditions and field problems, and their needs. In this case, the business objectives being carried out are to find out the opinions given by users of the digital wallet services Dana and ISaku and apply the Naïve Bayes Classifier and PSO methods to analyze the sentiment of digital wallet service users. 2. Data Understanding Data Understanding is the stage of collecting data, understanding data, and identifying problems related to data quality. This study manages tweet data in Indonesian from 19 June 2020 to 27 June 2020. The data collection process is by crawling tweet data from social media Twitter using the twitter search operator on Rapid Miner 9.1. The keywords used are @danawallet and iSaku. The data was taken from 19 June 2020 to 27 June 2020. From the results of the crawling, the data obtained were 490 tweets consisting of 272 tweet keywords @danawallet and 218 tweets with iSaku keyword. 3. Data Preparation Data preparation is the stage of preparing data for the next data mining process by changing the data from unstructured text documents to structured data. The application of data preparation in analyzing the sentiment of users of the Dana and iSaku has several stages, including (Kurniawan & Susanto, 2019): a. Transform Case The process of changing all capital letters into lowercase letters contained in tweet data so that they are uniform. http://creativecommons.org/licenses/by-nc/4.0/ JURNAL RISET INFORMATIKA Vol. 2, No. 4 September 2020 P-ISSN: 2656-1743 |E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v2i4.162 3 The work is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License b. Tokenize The process of tokenization is to cut sentences into sections or words based on punctuation such as commas, periods, and other punctuation as needed. c. Stopword Removal This process stop removal is to eliminate unnecessary words. Omitting the word will not change the information in it. d. Stemming The process stemming is to remove the affixes contained in words so that all words return to their root words. 4. Modeling Modeling is the stage of processing data from preparation results to obtain the required information. This processing process uses the Naive Bayes Classifier method to predict the probability of class membership and the addition of attribute weighting (attribute weighting) using PSO to increase accuracy in the classification of the Naïve Bayes Classifier. Testing the proposed classification model uses the RapidMiner 9.1 application. We tested the classification model that was formed to measure the level of accuracy and AUC with randomly separated data with k-folds cross validation with k = 10. Measurement of accuracy using confusion matrix and AUC values using the ROC curve. 5. Evaluation Evaluation is to validate the formed model. This evaluation uses a confusion matrix and ROC curve and tests the significant difference between the weighting before and after weighting the attributes with PSO on the Naïve Bayes Classifier using the T-Test and ANOVA methods. Figure 1. Proposed Model RESULTS AND DISCUSSION This chapter will explain the results and discussion of the research on the sentiment analysis of digital wallet service users. 1. Business Understanding In this study, the business objectives carried out were to determine the opinions given by users of Dana and iSaku digital wallet services, apply the Naïve Bayes Classifier and PSO methods to analyze the sentiment of digital wallet service users, and find out the increase in the value of Accuracy and AUC value in sentiment analysis users of digital wallet services with additional weight attributes use PSO to the Naïve Bayes Classifier method. 2. Data Understanding Data understanding of the data collection stage in this study is to retrieve tweet data with the crawling method from Twitter social media using the Twitter search operator on Rapid Miner 9.1 and enter the data in the form of Ms. Excel to facilitate data processing. The data taken is only tweets in Indonesian. The keywords used are @danawallet and iSaku . The data was taken from 19 June 2020 to 27 June 2020. From the results of the crawling, the data obtained were 490 tweets consisting of 272 tweet keywords @danawallet and 218 tweets with iSaku keyword. The design of the tweet data crawling model is presented in Figure 2. Figure 2. Tweet Data Crawling Model Design The flow of the data collection process in Figure 2 is to search for tweet data on Twitter based on the entered keywords. The use of this keyword as a sentiment analysis dataset, removes duplicates from the dataset based on the specified attributes. The attribute in this research is text. Duplicate text represent the selected attribute has the same value in it. Select the attributes you want to display and delete other attributes and convert the obtained dataset into Ms file. Excel. http://creativecommons.org/licenses/by-nc/4.0/ P-ISSN: 2656-1743 | E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v2i4.162 JURNAL RISET INFORMATIKA Vol. 2, No. 4 September 2020 4 The work is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License 3. Data Preparation (Initial Data Processing) The data from the crawling that has been previously obtained is still raw data, where the data still contains a lot of noise and is unstructured. Therefore, a preprocessing stage is needed to remove noise in words and data to make it more structured to facilitate research. Preprocessing in this study uses the Gataframework which can be accessed through http://www.gataframework.com/textmining.Gataf ra-mework is an alternative in the pre-processing text to process words in Indonesian because in the RapidMiner application the dictionary to change Indonesian acronyms and stopwords are still not available (Hermanto, Mustopa, & Kuntoro, 2020). Data from preprocessing results using Gataframework will be used as a dataset for testing models in the RapidMiner application. Figure 3. Display of Gataframework Text Mining Tools 4. Modeling After carrying out the preprocessing stage, the next step is the classification process modeling process to determine a sentence as a member of the positive class or negative class based on the probability calculation value of the Naïve Bayes Classifier formula. If the probability value in a sentence having a positive class is greater than the negative class, then the sentence is included in the positive class. And if the probability of the sentence having a positive class is smaller than the negative class, then the sentence is included in the negative class. The overall results of the classification data can be seen in Table 1. Table 1. Amount of Data Class Positive and Negative Digital Wallets No Digital Wallets Positive Negative 1. Dana 152 120 2. iSaku 104 114 5. Evaluation Evaluation is done to validate the formed model. Evaluation is done by using the confusion matrix and ROC curve and testing the significant difference between before and after weighting the attributes with PSO in the Naïve Bayes Classifier using the T-Test and ANOVA methods. The results of the T-Test and Anova tests obtained a significant and more accurate model. a. Testing the Fund Sentiment Model using the Naive Bayes Classifier The first experiment was to test the classification model using the Naive Bayes Classifier method. The test design of the Fund sentiment classification model using the Naive Bayes Classifier method is presented in Figure 4. Figure 4. Design of Sentiment Classification Model Funds using the Naïve Bayes Classifier http://creativecommons.org/licenses/by-nc/4.0/ JURNAL RISET INFORMATIKA Vol. 2, No. 4 September 2020 P-ISSN: 2656-1743 |E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v2i4.162 5 The work is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License In Table 2, it is explained that the results of testing the 10 fold cross-validation model of Dana's sentiment were obtained as many as 272 data consisting of 152 positive data and 120 negative data. A total of 88 data were predicted to be class positive according to the positive class prediction and 64 data predicted class positive turned out to be in the negative class, 75 data were predicted to be class negative accordingly, namely included in the negative class prediction and as many as 45 data predicted to be class negative turns into a positive class. The results obtained by using the Naïve Bayes Classifier algorithm using Rapid Miner 9.1 get the Accuracy value = 60.00% and AUC = 0.599. Table 2. Results of the Confusion Matrix Fund Sentiment using the Naïve Bayes Classifier Accuracy : 60.00% +/- 11.53% (micro: 59.93%) True Positive True Negative Class Precision Pred. Positive 88 45 66.17% Pred. Negative 64 75 53.96% Class Recall 57.89% 62.50% Figure 5. ROC Fund Sentiment Curve using the Naïve Bayes Classifier b. Sentiment Model Testing iSaku uses the Naive Bayes Classifier The first experiment was to test the classification model using the Naive Bayes Classifier method. The test design of the sentiment classification model iSaku using the Naive Bayes Classifier method is presented in Figure 6. Figure 6. Design of Sentiment Classification Model iSaku uses the Naïve Bayes Classifier In Table 3, it is explained that the test results of the model 10 fold cross-validation sentiment iSaku obtained as many as 218 data consisting of 104 positive data and 114 negative data. A total of 30 data were predicted to be class negative according to the class negative prediction, and 84 data predicted class negative turned out to be in a positive class, 86 data were predicted to be class positive according to the class positive prediction, and 18 data predicted class positive turned out to be in the class positive. negative. The results obtained by using the Naïve Bayes Classifier algorithm using Rapid Miner 9.1 get the Accuracy value = 53.23% and AUC = 0.520. Table 3. Results of Confusion Matrix Sentiment iSaku uses the Naïve Bayes Classifier Accuracy : 53.23% +/- 5.51% (micro: 53.21%) True Negative True Positive Class Precision Pred. Negative 30 18 62.50% Pred. Positive 84 86 50.59% Class Recall 26.32% 82.69% Figure 7. ROC Sentiment Curve iSaku using the Naïve Bayes Classifier http://creativecommons.org/licenses/by-nc/4.0/ P-ISSN: 2656-1743 | E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v2i4.162 JURNAL RISET INFORMATIKA Vol. 2, No. 4 September 2020 6 The work is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License c. Testing the Fund Sentiment Model using the Naive Bayes Classifier and PSO The second experiment was to test the classification model using the Naive Bayes Classifier and PSO methods. The test design of the Fund sentiment classification model using the Naive Bayes Classifier and PSO methods is presented in Figure 8. Figure 8. Design of a Fund Sentiment Model using NBC + PSO Based on the results of trials changing the value of Inertia Weight on the PSO, the highest Accuracy and AUC values are located at Population Size 7, the Maximum Number of Generation value is 100, and the Inertia Weight value 0.2 produces an Accuracy value of 91.67% and an AUC value of 0.750. Table 4 explains that from a sample of 0.1 used in the classification with the Naïve Bayes Classifier and PSO, 14 of the Fund's digital wallet service users were predicted to be class positive, which was included in the positive class prediction and 1 data predicted class positive turned out to be in the negative class, 11 The data predicted in the negative class was by the negative class prediction, and 1 data predicted the negative class turned out to be in a positive class. The results obtained by using the Naïve Bayes Classifier algorithm and PSO using Rapid Miner 9.1 get an Accuracy value = 91.67% and AUC = 0.750. Table 4. Funds Sentiment Confusion Matrix Results using NBC + PSO Accuracy : 91.67% +/-17.08 % (micro: 92.59 %) True Positive True Negative Class Precision Pred. Positive 14 1 93.33% Pred. Negative 1 11 91.67% Class Recall 93.33% 91.67% Figure 9. ROC Fund Sentiment Curve using NBC + PSO d. Sentiment Model Testing iSaku uses the Naive Bayes Classifier and PSO The second experiment was to test the classification model using the Naive Bayes Classifier and PSO methods. The test design of the sentiment classification model iSaku using the Naive Bayes Classifier and PSO methods is presented in Figure 10. Figure 10. Design of Sentiment Model ISaku uses NBC + PSO Based on the results of trials changing the Inertia Weight value on the PSO, the highest Accuracy and AUC values are located at Population Size 7, the Maximum Number of Generation value is 100, and the Inertia Weight value 0.1 results in an Accuracy value of 85.00% and an AUC value of 0.800. Table 5 explains from a sample of 0.1 which is used in the classification with the Naïve Bayes Classifier and PSO users of the digital wallet service I. As much as 8 data is predicted to be a negative class according to that is included in the negative class prediction and 3 data predicted the negative class turns into the positive class, 10 The data was predicted to be class positive according to the positive class prediction, and as many as 0 data was predicted to be class positive, it turned out to http://creativecommons.org/licenses/by-nc/4.0/ JURNAL RISET INFORMATIKA Vol. 2, No. 4 September 2020 P-ISSN: 2656-1743 |E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v2i4.162 7 The work is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License be in the negative class. The results obtained by using the Naïve Bayes Classifier algorithm and PSO using Rapid Miner 9.1 get an Accuracy value = 85.00% and AUC = 0.800. Table 5. Results of Confusion Matrix Sentiment iSaku use NBC + PSO Accuracy : 85.00% +/- 22.91% (micro: 85.71%) True Negative True Positive Class Precision Pred. Negative 8 0 100.00% Pred. Positive 3 10 76.92% Class Recall 72.73% 100.00% Figure 11. ROC Sentiment Curve iSaku using NBC + PSO Based on the test results, an evaluation was carried out by observing the comparison of the Accuracy and AUC values of each method as presented in Table 6. The evaluation results showed that the Accuracy and AUC values of NBC + PSO were higher than single NBC. The use of PSO for attribute weights can increase the accuracy of the Naïve Bayes Classifier algorithm in analyzing the sentiment of digital wallet service users. Table 6. Comparison of Experimental Results using NBC and NBC + PSO Naïve Bayes Classifier Naïve Bayes Classifier dan PSO Accura cy AUC Accurac y AUC Dana 60.00 % 0.599 91.67% 0.750 ISaku 53.23 % 0.520 85.00% 0.800 Further evaluation was carried out by researchers to test the validity of the classification model performance using the T-Test and Anova test. e. T-Test and Anova test for Fund sentiment The T-Test and Anova test designs for Fund sentiment on the RapidMiner 9.1 application are presented in Figure 12. Figure 12. T-Test Model Design and Anova Model Classification of NBC with NBC + PSO Fund Sentiment The results of the T-Test are presented in Table 7 shows that the NBC + PSO method has a significant difference in value because it has a probability of <0.050, namely 0.000 on the Naive Bayes Classifier method. Table 7. T-Test Results of Naïve Bayes Classification Model with NBC + PSO Fund Sentiment T-Test Significance A B C 0.607 +/- 0.094 0.917 +/- 0.171 0.607 +/- 0.094 0.000 0.917 +/- 0.171 The Anova test results are presented in Table 8 showing that dk between groups (comparison) = 1, dk in residuals (denominator) = 18, and alpha = 0.050. Then the F table value is F0.050 (1.18) = 4.41. and F count = 25,245. The value of Fcount> Ftable = 25,245> 4.41, then the two classification methods tested have a significant difference in the Accuracy value. http://creativecommons.org/licenses/by-nc/4.0/ P-ISSN: 2656-1743 | E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v2i4.162 JURNAL RISET INFORMATIKA Vol. 2, No. 4 September 2020 8 The work is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License Table 8. ANOVA Test Results for Naïve Bayes Classification Model with NBC + PSO Fund Sentiment ANOVA Test Source Square Sums DF Mean Squares F Prob Between 0.480 1 0.480 25.245 0.000 Residuals 0.342 18 0.019 Total 0.823 19 The evaluation results show that NBC + PSO is a fairly good combination of methods in classifying data. The use of PSO for attribute weighting affects increasing the value of Accuracy in analyzing the sentiment of users of Dana's digital wallet services. f. T-Test and Anova test for sentiment iSaku The T-Test and Anova test design for sentiment iSaku in the RapidMiner 9.1 application are presented in Figure 13. Figure 13. Design of the T-Test and Anova Model Test for Classification of NBC with NBC + PSO Sentiment iSaku The results of the T-Test are presented in Table 9 shows that the NBC + PSO method has a significant difference in value because it has a probability of <0.050, namely 0.000 on the Naive Bayes Classifier method. Table 9. T-Test Results for Naïve Bayes Classification Model with NBC + PSO Sentiment iSaku T-Test Significance A B C 0.523 +/- 0.053 0.850 +/- 0.229 0.523 +/- 0.053 0.000 0.850 +/- 0.229 Anova test results are presented in Table 10 showing that dk between groups (comparison) = 1, dk in residual (denominator) = 18, and alpha = 0.050. Then the F table value is F0.050 (1.18) = 4.41. and F count = 19,377. The value of Fcount> Ftable = 19,377> 4.41, then the two classification methods tested have a significant difference in the Accuracy value. Table 10 Anova Test Results for Naïve Bayes Classification Model with NBC + PSO Sentiment iSaku ANOVA Test Source Square Sums DF Mean Squares F Prob Betwee n 0.536 1 0.536 19. 377 0.000 Residu als 0.497 18 0.028 Total 1.033 19 The evaluation results show that NBC + PSO is a fairly good combination of methods in classifying data. The use of PSO for attribute weighting affects increasing the value of Accuracy in the sentiment analysis of iSaku digital wallet service users. It can also be ignored that the digital wallet that has the best service is Dana because most of its users give a positive opinion on the digital wallet service. http://creativecommons.org/licenses/by-nc/4.0/ JURNAL RISET INFORMATIKA Vol. 2, No. 4 September 2020 P-ISSN: 2656-1743 |E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v2i4.162 9 The work is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License CONCLUSION The results of the classification of 490 tweet data using the Naive Bayes Classifier show that the Dana digital wallet has 152 positive sentiments and 120 negative sentiments, as well as the iSaku, has 104 positive sentiments and 114 negative sentiments. Testing tweet data using PSO can increase the Accuracy value and the AUC value. The increase was significant, previously the sentiment test using NBC only produced 60.00% and an AUC value of 0.599 after adding the PSO, the Accuracy value was 91.67% and the AUC value was 0.750, while for sentiment testing iSaku using NBC only produced 53.23% and a value AUC 0.520 after adding the PSO Accuracy value to 85.00% and 0.800 AUC value. Weighting the attributes using PSO has a major effect on the Accuracy results obtained, giving an increase of 31.67% for Fund sentiment and 31.77% for Sentiment ISaku. The results of the T-Test and Anova test show that the two classification methods tested have significant differences in the Accuracy value. NBC + PSO is a fairly good combination of methods in classifying data. The use of PSO for attribute weighting affects increasing the value of Accuracy in analyzing the sentiment of users of the Dana and ISaku digital wallet services. REFERENCE Aaputra, S. A., Didi Rosiyadi, Windu Gata, & Syepry Maulana Husain. (2019). Sentiment Analysis Analisis Sentimen E-Wallet Pada Google Play Menggunakan Algoritma Naive Bayes Berbasis Particle Swarm Optimization. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 3(3), 377–382. https://doi.org/10.29207/resti.v3i3.1118 Budiansyah, A. (2020). GoPay & OVO Cs Kian Populer, Transaksi Tembus Rp 145 T. Retrieved April 23, 2020, from CNBC Indonesia website: https://www.cnbcindonesia.com/tech/2020 0204112543-37-135041/gopay-ovo-cs-kian- populer-transaksi-tembus-rp-145-t Clinten, B. (2019). Pengguna Aktif Harian Twitter Indonesia Diklaim Terbanyak. Retrieved April 24, 2020, from Kompas website: https://tekno.kompas.com/read/2019/10/3 0/16062477/pengguna-aktif-harian-twitter- indonesia-diklaim-terbanyak Devita, V. D. (2019). Siapa Aplikasi E-wallet dengan Pengguna Terbanyak di Indonesia? Retrieved June 11, 2020, from iPrice Group website: https://iprice.co.id/trend/insights/e-wallet- terbaik-di-indonesia/ Hermanto, H., Mustopa, A., & Kuntoro, A. Y. (2020). Algoritma Klasifikasi Naive Bayes Dan Support Vector Machine Dalam Layanan Komplain Mahasiswa. JITK (Jurnal Ilmu Pengetahuan Dan Teknologi Komputer), 5(2), 211–220. https://doi.org/10.33480/jitk.v5i2.1181 Indonesia, B. (2020). Informasi Perizinan Penyelenggara dan Pendukung Jasa Sistem Pembayaran. Retrieved April 23, 2020, from Bank Indonesia website: https://www.bi.go.id/id/sistem- pembayaran/informasi-perizinan/uang- elektronik/penyelenggara- berizin/Contents/Default.aspx Kurniawan, I., & Susanto, A. (2019). Implementasi Metode K-Means dan Naïve Bayes Classifier untuk Analisis Sentimen Pemilihan Presiden (Pilpres) 2019. Eksplora Informatika, 9(1), 1– 10. https://doi.org/10.30864/eksplora.v9i1.237 Mahendrajaya, R., Buntoro, G. A., & Setyawan, M. B. (2019). Analisis Sentimen Pengguna Gopay Menggunakan Metode Lexicon Based Dan Support Vector Machine. Komputek, 3(2), 52. https://doi.org/10.24269/jkt.v3i2.270 Maulana, A. (2016). Twitter Rahasiakan Jumlah Pengguna di Indonesia. Retrieved April 24, 2020, from CNN Indonesia website: https://www.cnnindonesia.com/teknologi/2 0160322085045-185-118939/twitter- rahasiakan-jumlah-pengguna-di-indonesia Pertiwi, M. W. (2019). Analisis sentimen opini publik mengenai sarana dan transportasi mudik tahun 2019 pada twitter menggunakan algoritma naïve bayes, neural network, KNN dan SVM. Inti Nusa Mandiri, 14(1), 27–32. Pratama, K. A., Pradnyana, G. A., & Arthana, I. K. R. (2020). Pengembangan Sistem Cerdas Untuk Prediksi Daftar Kembali Mahasiswa Baru Dengan Metode Naive Bayes (Studi Kasus: Universitas Pendidikan Ganesha). Sintech Journal, 3(1), 22–34. Saidah, S., & Mayary, J. (2020). Analisis Sentimen Pengguna Twitter Terhadap Dompet http://creativecommons.org/licenses/by-nc/4.0/ P-ISSN: 2656-1743 | E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v2i4.162 JURNAL RISET INFORMATIKA Vol. 2, No. 4 September 2020 10 The work is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License Elektronik Dengan Metode Lexicon Based Dan K – Nearest Neighbor. Jurnal Ilmiah Informatika Komputer, 25(1). http://creativecommons.org/licenses/by-nc/4.0/