Journal of Applied Engineering and Technological Science Vol 4(1) 2022: 532-538 532 THE IMPLEMENTATION OF DEEP LEARNING TECHNIQUES IN DEVELOPING CONVERSATIONAL CHATBOT AS THE SOURCE OF VACCINATION INFORMATION Yuliska 1* , Nina Fadhilah Najwa 2 , Khairul Umam Syaliman 3 Department of Informatics Engineering, Politeknik Caltex Riau, Indonesia123 yuliska@pcr.ac.id Received : 18 November 2022, Revised: 14 December 2022, Accepted : 14 December 2022 *Corresponding Author ABSTRACT The Covid-19 pandemic has hit Indonesia for more than 2 years. To overcome Covid-19, Indonesian government implemented a vaccination program with a target of 70% of the population being vaccinated. However, the recorded population that has been vaccinated to reduce the risk of being exposed to Covid- 19 is still low. Several studies have stated that information and invitations to vaccines through mass media are considered insufficient to convince the population to vaccinate. Residents who are still unsure and do not even want to vaccinate need really comprehensive information from experts. To answer this problem, a chatbot that can replace experts in explaining everything related to vaccines can be one solution. This is evidenced by a study which states that the interaction between people who have not been vaccinated with a chatbot that explains about vaccination can reduce the level of doubt of the population about the vaccine by up to 20%. The purpose of this research is to build a chatbot using deep learning technique. Meanwhile, the deep learning technique used to build a conversational chatbot is the Multilayer Perceptron Network (MLP). Based on the result of our study, our chatbot can answer 83% questions correctly out of 30 questions. Keywords : Chatbot, Deep Learning, Vaccination 1. Introduction Until today, the Covid-19 Pandemic has been hitting Indonesia for more or less than two years. Over the past two years, 144.136 Indonesians have died due to exposure to Covid-19 (sehatnegeriku.kemkes.go.id, accessed on January 10, 2022). However, with the high death rate due to this pandemic, the number of people recorded as having been vaccinated to reduce the risk of exposure to Covid-19 still tends to be low, namely as many as 116 of Indonesia's 273.5 million people (kompas.com, accessed on January 8, 2022), or approximately 42% of the total population. Meanwhile, in Riau province, the number of people vaccinated with the second dose is also still relatively low, namely 49% (corona.riau.go.id, accessed on January 15, 2022). One of the reasons for the low number of people vaccinated is the doubt of a large part of the population about vaccination, both in terms of vaccine safety and effectiveness (Marwan, 2021). However, the residents' doubts were also accompanied by the residents' curiosity for detailed information about vaccination. Several studies state that information and invitations to vaccines through the mass media are insufficient to convince people to vaccinate. Residents who are still trying to decide whether to vaccinate need comprehensive information from experts, not just invitations via short messages in the mass media. Meanwhile, discussions with experts are challenging for residents who are against vaccination (Ndwandwe & Wiysonge, 2021; Aw et al., 2021; Andrews et al., 2022). To answer the problems above, chatbots that can replace experts in explaining everything related to vaccines can be one solution. Chatbots can offer private question-and-answer sessions (Personalized Q&A) so that information delivery is faster and more effective (Ma et al., 2021) (Jenneboer et al., 2022). This is proven by a study that states that the interactions between residents who have not been vaccinated and a chatbot that explains vaccinations can reduce the level of doubt among residents about vaccines by up to 20% (Zakirman & Rahayu, 2018). After interacting with chatbots, residents who were initially against vaccines have a more positive view of vaccines. Yuliska et al … Vol 4(1) 2022 : 532-538 533 Based on the problems above, in this study, a chatbot will be built using a deep learning technique. Meanwhile, the technique or method used to build a conversational chatbot is the Multilayer Perceptron Network (MLP) (Jehad & Yousif, 2021) (Rachmad Syulistyo & Meliana Agustin, 2022) (Dwi Sripamuji et al., n.d.). MLP is a deep learning network that has been widely used in text processing and classification. Conversational chatbots are chatbots that can generate words just like humans so that the interactions between chatbot users and chatbots become more lively (Desai & Shah, 2021). Hopefully, this chatbot can help residents obtain comprehensive, complete, and targeted vaccine information. So that residents have sufficient knowledge about vaccines and are confident to vaccinate (Yang et al., 2021; Yulita et al., 2021; Nuanmeesri et al., 2022). 2. Research Methodology In this study, all initial research procedures must have been carried out, such as a literature study which was carried out by collecting and reading, and understanding references related to the issue of Chatbot development as a forum for Vaccination information, as well as the method used, namely MLP (Li et al., 2021). After that, data collection is carried out, which is then followed by building a Chatbot. The research methodology is illustrated in Figure 1, with an explanation of each stage as follows: Fig. 1. Research Methodology Stage 1: Problem Identification At this stage, we contain a problem formulation so that we can understand and analyze the problems related to the low vaccination rate and can identify these problems. Stage 2: Determining Research Objectives At this stage, we look for the most appropriate solution to the problems found so that the solution becomes a goal in the research being conducted. Stage 3: Data Collection At this stage, we carry out the necessary data collection activities such as collecting questions that residents usually ask regarding vaccination, information in the form of answers to questions related to vaccination, which can be in the form of the type of vaccine, the ingredients contained in the vaccine, where the vaccine is, and some other important info. Stage 4: Creating a Deep Learning Model Yuliska et al … Vol 4(1) 2022 : 532-538 534 After data collection in stage 3, we will form a dataset that contains pairs of questions and answers that can be used to create deep learning models (MLP) to perform chat generation. Stage 5: Creating a Chatbot At this stage, we begin to implement the chat generation model to create a vaccination information chatbot. Stage 6: Testing At this stage, we will conduct tests on the WhatsApp Bot that was built to find out the quality of the Whatsapp Bot both from a technical perspective and its use for the population. Stage 7: Conclusion At this stage, we will draw conclusions based on the tests that have been carried out. Tabel 1 – The Example of Data Used in This Study Questions Answers Apakah Vaksin Drop Out? Vaksin Drop Out adalah sasaran yang belum mendapatkan vaksinasi primer lengkap sesuai jadwalnya Apakah ada batas waktu untuk melakukan vaksinasi booster? Tidak ada batas waktu. Apakah vaksin booster gratis? Vaksin booster gratis untuk semua Warga Negara Indonesia. Jenis vaksin COVID-19 apa saja yang akan digunakan di Indonesia? Sinovac, AstraZeneca, Sinopharm, Novavax, Moderna, Pfizer, Cansino, Sputnik V Siapa saja yang berhak mendapatkan vaksin booster? Semua Warga Negara Indonesia usia 18 Tahun ke atas berhak mendapatkan vaksin booster. Apa saja jenis vaksin yang dapat diberikan kepada ibu hamil? Semua jenis vaksin yang ada saat ini dapat diberikan pada ibu hamil dan menyusui Apakah menerima vaksin COVID-19 dapat mengubah DNA seseorang? Vaksin COVID-19 tidak dapat mengubah atau berinteraksi dengan DNA seseorang dalam cara apapun. Apakah vaksin itu obat? Vaksin bukanlah obat Kenapa vaksin booster diberikan setengah dosis? Karena imunogenesitas yang didapatkan setlah dibooster dengan setengah dosis tidak berbeda secara signifikan dengan booster dosis penuh Apakah paracetamol boleh dikonsumsi setelah vaksinasi COVID-19? Jika demam, menggigil atau pegal-pegal boleh minum paracetamol dan beristirahat. Pada rentang usia dan berapa dosis yang direkomendasikan untuk melakukan vaksinasi janssen? Peserta dengan umur 18 tahun dan lebih, dan direkomendasikan satu dosis 2.1. Data Used The data used in this study are question-and-answer data related to vaccination. Answers and questions are obtained from: a) Interview with health workers b) Vaccination information document c) Website covid-19.go.id d) Internet Data collection and processing were carried out for +/- 2 months until 89 pairs of answers and questions were finally collected. Table 1 describes an example of the data used in this study. The data in table 1 is then converted into data.json, as shown in Figure 2. 2.2. Model For Chat Generation The architecture of the Multiple Layer Perceptron (MLP) Model, as well as the stages carried out prior to the model training process used in this research, can be seen in Figure 3. Yuliska et al … Vol 4(1) 2022 : 532-538 535 Fig. 2. Questions and Answers in .json Fig. 3. Deep Learning Model Used As can be seen in Figure 3, the model consists of 3 layers, namely the input layer with a hidden size of 128. Then followed by a hidden layer with a hidden size of 64. Finally, there is an output layer with a hidden size of 89 topics, with an activation function the one used is softmax. 3. Results and Discussions In this section, we explain the training scheme we applied in creating the MLP model to select answers to questions asked by chatbot users and also explain the experimental results. 3.1. Training Scheme Before training, we used the NLTK library to tokenize and clean sentences (preprocessing). MLP is implemented using the Keras library. The hidden sizes that we used in the MLP model were 128, 64, and 89 (according to the number of topics), respectively. We used Relu activation function for the input layer and hidden layer, and softmax for the output layer. The dropout operation is also applied at a ratio of 0.5 in each layer. Finally, we used SGD optimizer (Ruder, 2016) with a learning rate of 10-2 and a batch size of 100. Yuliska et al … Vol 4(1) 2022 : 532-538 536 3.2. The Implementation of Chatbot Interface From the MLP model training scheme, according to the explanation above, a model accuracy of 98% is obtained. The training result model is then used to create a chatbot using Tkinter. Chatbots can be used as appropriate, where chatbots can answer questions related to vaccines according to the topics asked. Here is the chatbot interface: Fig. 4. Chatbot Interface 3.3. Chatbot Validity Testing To test the chatbot's validity, we randomly asked 30 questions related to vaccination. Of the 30 questions asked, the chatbot answered 83% of the questions correctly, as seen in the pie chart in figure 5. More complete information (18 out of 30 questions) regarding the questions asked in the validity test can be seen in table 2. From Figure 6 and Table 2, it can be seen that the chatbot can answer most of the questions related to vaccination correctly. It shows that the chatbot can function properly. Fig. 5. Validity Test Pie Chart Tabel 2 – Validity Testing (18 out of 30 Questions Asked) Questions Answers Results Apa manfaat vaksinasi lengkap? Vaksinasi lengkap dan booster mampu mengurangi risiko dirawat dan kematian Valid Apakah Vaksin Drop Out? Bagi sasaran yang mengalami drop out dalam waktu lebih dari enam bulan, maka vaksinasi primer nya dilanjutkan kembali sebanyak 2 kali dan vaksinasi dapat menggunakan jenis vaksin yang berbeda dari vaksin semula Not Valid Yuliska et al … Vol 4(1) 2022 : 532-538 537 Bagaimana bila saya terlambat lebih dari 3 bulan tidak melakukan vaksinasi booster? Anda bisa mendapatkan vaksin booster setelah 3 bulan mendapatkan vaksin dosis kedua Not Valid Apakah vaksin booster gratis? Vaksin booster gratis untuk semua Warga Negara Indonesia Valid Apakah saya bisa memilih jenis vaksin booster yang diterima? Masyarakat tidak dapat memilih jenis vaksinasi booster Valid Di mana tempat pelayanan vaksinasi COVID-19? Anda dapat cek lokasi dan jenis vaksin yang dibutuhkan melalui aplikasi PeduliLindungi Valid Siapa saja yang berhak mendapatkan vaksin booster? Kelompok prioritas penerima vaksin adalah penduduk yang berdomisili di Indonesia yang berusia lebih besar atau sama dengan 18 tahun Not Valid Apa saja jenis vaksin yang dapat diberikan kepada ibu hamil? Semua jenis vaksin yang ada saat ini dapat diberikan pada ibu hamil dan menyusui Valid Apakah menerima vaksin COVID-19 dapat membuat seseorang menjadi magnetic? Menerima vaksin COVID-19 tidak dapat membuat seseorang menjadi magnetik, termasuk pada lokasi penyuntikan yang biasanya dilakukan pada lengan atas Valid Apa itu vaksinasi? Vaksin bukanlah obat Not Valid Vaksin COVID-19 mana yang paling baik untuk saya? Semua vaksin yang disetujui oleh WHO telah terbukti aman dan efektif dalam melindungi penerimanya terhadap sakit berat yang ditimbulkan oleh COVID-19 Valid Berapa dosis dan interval yang dibutuhkan untuk vaksinasi COVID-19 agar terbentuk kekebalan yang optimal? Dosis dan cara pemberian harus sesuai dengan yang direkomendasikan untuk setiap jenis vaksin covid-19 Valid Apakah seseorang harus menjalani pemeriksaan kesehatan terlebih dahulu sebelum divaksinasi? Meskipun tidak diwajibkan, sasaran dapat memeriksakan terlebih dahulu kondisi kesehatannya sebelum datang ke tempat pelayanan vaksinasi COVID-19 Valid Apakah vaksin COVID-19 bisa diberikan bersamaan dengan jenis vaksin lainnya? Vaksin COVID-19 bisa diberikan bersama vaksin lain Valid Bagaimana jika vaksinasi COVID-19 diberikan kepada seseorang yang sedang berpuasa? Vaksin COVID-19 tetap aman untuk diberikan kepada seseorang yang sedang berpuasa Valid Apakah manfaat dari vaksin COVID-19? Vaksin COVID-19 bermanfaat untuk memberi perlindungan agar tidak tertular atau sakit berat akibat COVID-19 Valid Apakah vaksin COVID-19 nanti juga tersedia untuk anak-anak? Pengembangan vaksin untuk anak-anak masih direncanakan pada beberapa kandidat vaksin Valid Apa itu vaksin Moderna dan Pfizer-BioNTech? vaksin mRNA yang tidak mengandung virus hidup Valid 5. Conclusion In this study, we created a deep learning-based chatbot as a source of vaccination information. The deep learning technique used is Multiple Layer Perceptron (MLP). The data used to create the chatbot consists of 89 questions from 27 vaccination topics. MLP is trained to be able to answer questions related to vaccination and obtain 98% accuracy. Based on the validity testing results, the chatbot can answer 83% of the questions correctly out of 30. References Andrews, N., Stowe, J., Kirsebom, F., Toffa, S., Rickeard, T., Gallagher, E., ... & Lopez Bernal, J. (2022). Covid-19 vaccine effectiveness against the Omicron (B. 1.1. 529) variant. New England Journal of Medicine, 386(16), 1532-1546. Aw, J., Seng, J. J. B., Seah, S. S. Y., & Low, L. L. (2021). COVID-19 vaccine hesitancy—A scoping review of literature in high-income countries. Vaccines, 9(8), 900. Desai, M., & Shah, M. (2021). An anatomization on breast cancer detection and diagnosis employing multi-layer perceptron neural network (MLP) and Convolutional neural network (CNN). Clinical eHealth, 4, 1-11. Dwi Sripamuji, A., Ramadhanti, I., Riski Amalia, R., Saputra, J., & Prihatnowo, B. (n.d.). 2022. Penerapan Algoritma Support Vector Machine Dan Multi-Layer Perceptron Pada Yuliska et al … Vol 4(1) 2022 : 532-538 538 Klasifikasi Topik Berita. Jurnal Nasional Pendidikan Teknik Informatika, 11(2). https://doi.org/10.23887/janapati.v11i2.44151 Jehad, R., & Yousif, S. (2021). Classification of fake news using multi-layer perception. AIP Conference Proceedings, 2334. https://doi.org/10.1063/5.0042264 Jenneboer, L., Herrando, C., & Constantinides, E. (2022). The Impact of Chatbots on Customer Loyalty: A Systematic Literature Review. In Journal of Theoretical and Applied Electronic Commerce Research (Vol. 17, Issue 1, pp. 212–229). MDPI. https://doi.org/10.3390/jtaer17010011 Li, M., Achiluzzi, E., Al Georgy, M. F., & Kashef, R. (2021, October). Malicious Network Traffic Detection in IoT Environments Using A Multi-level Neural Network. In 2021 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) (pp. 169-175). IEEE. Ma, Z., Dou, Z., Zhu, Y., Zhong, H., & Wen, J. R. (2021). One Chatbot per Person: Creating Personalized Chatbots based on Implicit User Profiles. SIGIR 2021 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 555–564. https://doi.org/10.1145/3404835.3462828 Marwan. (2021). Peran vaksin penanganan pandemi COVID19. In Fakultas Kedokteran Universitas Mulawarman - RSU A. W. Sjahranie Samarinda (Vol. 1, Issue covid). Ndwandwe, D., & Wiysonge, C. S. (2021). COVID-19 vaccines. Current opinion in immunology, 71, 111-116. Nuanmeesri, S., Poomhiran, L., Chopvitayakun, S., & Kadmateekarun, P. (2022). Improving Dropout Forecasting during the COVID-19 Pandemic through Feature Selection and Multilayer Perceptron Neural Network. International Journal of Information and Education Technology, 12(9), 851-857. Rachmad Syulistyo, A., & Meliana Agustin, V. (2022). Predicting News Article Popularity with Multi Layer Perceptron Algorithm. Journal of Applied Intelligent System, 7(2), 193–205. Ruder, S. (2016). An overview of gradient descent optimization algorithms. http://arxiv.org/abs/1609.04747 Yang, F., Moayedi, H., & Mosavi, A. (2021). Predicting the degree of dissolved oxygen using three types of multi-layer perceptron-based artificial neural networks. Sustainability, 13(17), 9898. Yulita, I. N., Abdullah, A. S., Helen, A., Hadi, S., Sholahuddin, A., & Rejito, J. (2021). Comparison multi-layer perceptron and linear regression for time series prediction of novel coronavirus covid-19 data in West Java. In Journal of Physics: Conference Series (Vol. 1722, No. 1, p. 012021). IOP Publishing. Zakirman, Z., & Rahayu, C. (2018). Popularitas WhatsApp sebagai media komunikasi dan berbagi informasi akademik mahasiswa. Shaut Al-Maktabah : Jurnal Perpustakaan, Arsip Dan Dokumentasi, 10(1), 27–38. https://doi.org/10.15548/shaut.v10i1.7