JURNAL RISET INFORMATIKA Vol. 5, No. 3. June 2023 P-ISSN: 2656-1743 |E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v5i3.526 Accredited rank 4 (SINTA 4), excerpts from the decision of the DITJEN DIKTIRISTEK No. 230/E/KPT/2023 1 IMPLEMENTATION OF HYBRID METHOD IN TOURISM PLACE RECOMMENDATION SYSTEM BASED ON IMAGE FEATURES Steven Christ Pinantyo Arwidarasto-1*), Desti Fitriati-2 Department of Informatics Engineering Universitas Pancasila Jakarta, Indonesia1,2 stevenchristpa2001@gmail.com1, desti.fitriati@univpancasila.ac.id2 (*) Corresponding Author Abstrak Pada industri 4.0, terjadinya ledakan data tak terstruktur maupun berstruktur yang menghasilkan informasi pengetahuan yang sangat luas dan variatif. Tentunya, manusia tidak dapat mengolah banyak informasi dalam waktu cepat, yang membuat eksistensi sistem rekomendasi menjadi berarti. Sistem ini mempelajari informasi yang ada dan memberikan saran yang sesuai dengan apa yang diinginkan oleh pengguna. Dewasanya, banyak sistem rekomendasi lebih menitikberatkan pada penggunaan metode content-based filtering dimana hasil rekomendasi didasari berdasarkan kemiripan fitur dari konen yang disukai oleh pengguna. Hal ini membatasi variasi informasi yang relevan kepada pengguna. Selain itu, dalam konteks tempat wisata, banyak penelitian yang belum menggunakan data gambar yang dapat memuat banyak objek fitur dalam satu frame sebagai faktor penentu dalam memberikan rekomendasi. Hal ini membuat, dalam penelitian ini, penulis memproposalkan penambahan fitur gambar sebagai salah satu parameter penentu rekomendasi untuk mengetahui dampak penggunaan gambar pada performa model. Adapun performa terbaik yang didapat yaitu 0.364 menggunakan matrik RMSE menggunakan metode Hybrid Image. Kata kunci: industri, rekomendasi, kemiripan, gambar, variasi Abstract In the industrial 4.0 era, there is an explosion of unstructured and structured data that produces broad and varied knowledge information that humans cannot process quickly. This issue makes the existence of recommendation systems meaningful. This system studies the existing information and provides suggestions according to the user's will. In the past, many recommendation systems have focused more on content-based filtering methods where recommendation results are similar based on the features of the Content that match the user's personality. This method limits the variety of information that is relevant to users. In addition, in the context of tourist attractions, many studies have not used image data that can contain many objects in one frame as a determining factor in providing recommendations. Therefore, in this study, the authors propose to add image features as one of the parameters of the recommendation system to determine the impact of using image features on the model performance. The best performance obtained is 0.364 RMSE metric using the Hybrid Image method. Keywords: industrial, recommendation, similar, image, varied INTRODUCTION The rapid development of the times has made much information appear to be used as a basis for decision-making (Walek & Fojtik, 2020). This resulted in a huge amount of information that made the processing stage difficult for decision-makers to process existing information. In this case, the recommendation system exists to provide relevant decision suggestions for its users based on preferences and historical data (Geetha et al., 2018; Mohammadpour et al., 2019). The use of recommendation systems in everyday life, among others, in the creative economy and tourism industries. In this sector, the recommendation system provides advice and outreach to people who want or like natural, cultural, educational, or religious exploration. For creative economy and tourism industries, a recommendation system requires information such as tourist categories, place names, locations, and descriptions of tourist attractions to be used as features of the content representation, P-ISSN: 2656-1743 | E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v5i3.526 JURNAL RISET INFORMATIKA Vol. 5, No. 3. June 2023 Accredited rank 4 (SINTA 4), excerpts from the decision of the DITJEN DIKTIRISTEK No. 230/E/KPT/2023 2 which the majority of the features mentioned are text-based features. There are several approaches widely used to select relevant items, namely Collaborative filtering (C.F.) and Content-based filtering (CBF), where both approaches have their own merits. For instance, the C.F. approach suggests that Content is seen by other users with highly similar preferences based on historical records (Geetha et al., 2018; Ng, 2020). In contrast, the CBF approach offers Content with similar features to the Content users prefer (Ng, 2020). One of the C.F. algorithms widely used for recommending Content is Matrix Factorization (M.F.) which has proven to be one of the popular models in the Netflix Competition (Koren et al., 2009). The challenges faced include finding the reliability of the ratings given by users to items. This reliability refers to the relevance of the ratings on a user-item transaction based on the difference in ratings of items. Several studies conducted experiments to overcome the problem of rating relevance, one of which was to improve M.F. with a Bernoulli probability (Ortega et al., 2021). This research uses the Movielens dataset, which contains user rating data for several films. This method obtains an evaluation score using a pointwise matrix, namely, Mean Absolute Error (M.A.E.) of 0.093 (native) and 0.036 (enforced). Another study was conducted to optimize M.F. using benchmark data, namely the movie lens, which uses the Rating Centrality method to determine the reliability of the ratings given by the user (Wu et al., 2018). This method obtains an evaluation score with a pointwise matrix, namely RMSE (Root Mean Squared) of 0.898 at an alpha value of 80%. In contrast to the C.F. methods, several studies conducted CBF to recommend items. One of which uses the Content-based Image Retrieval (CBIR) method to retrieve relevant image data using V.G.G. 16 Deep Convolutional Network architecture to select the dominant features of an image into vectors which ranked based on the highest score of cosine similarity matrix (Rian et al., 2019). The metrics used in this study, namely F1 score and Precision, respectively, 73% and 89.6%. Looking at the capabilities of each approach, several studies then combined the C.F. and CBF approaches, known as the Hybrid method, to help the C.F. method find out the personalization context of the user (Context-aware recommendation model). In doing so, the combined effort could result in a more relevant selection of travel recommendations that draw on other users' histories and similar Content. An example of the Hybrid method combines the Deep Neural Network method and the C.F. method (using the Alternating Least Square (A.L.S.) algorithm) (Biswas & Liu, 2022). In this study, feature extraction was done using user-place representation to determine the relationship between both parameters and give relevant recommendations. This method obtains an RMSE score of 0.0106. Another example of research to optimize M.F. using the Deep Hybrid model architecture is combining M.F. and Neural Network (N.N.) using categorical and continuous features (Γ‡akΔ±r et al., 2019). This study used a listwise evaluation matrix, namely Normalized Discounted Cumulative Gain (NDCG) and Hit Ratio (H.R.), was used. The NDCG and H.R. scores obtained were 0.855 and 0.628 for recommendations based on items and 0.795 and 0.581 for user recommendations. Based on these studies, The Hybrid model uses personalized M.F. methods that utilize features from Content aline with users' preferences. In the context of the creative economy and tourism industries, the content feature that is more dominant to attract the attention of its users is information about the sights of these tourist attractions. This opens up opportunities for using data features that dominantly have information related to object sightseeing as a personalization method for its users. Regarding information about tourist sights, if the model processes text-based information, the user is limited by the semantic expression of the text. However, the presence of images can provide broader information about the atmosphere and conditions of the tourist destination to be visited. Therefore, utilizing images as an information source can be a solution for finding relevant tourist attractions. For example, a study optimized M.F. architecture by combining M.F. with Resnet50 to extract information from images (Elsayed et al., 2022). This research uses the Amazon Fashion dataset, which contains products for clothing needs. The model obtained an A.U.C. matrix evaluation with 100 negative samples for each user of 0.8250. Based on the literature studies that have been conducted, many recommendation system studies in the tourism sector still use text-based recommendations. Examples of research within the scope of Indonesia's creative economy and tourism industries use the SMART (Simple Multi-Attribute Rating Technique) method, which utilizes user preferences based on the most relevant attributes to users. This SMART method gives weight to each attribute and calculates the relevance score for each JURNAL RISET INFORMATIKA Vol. 5, No. 3. June 2023 P-ISSN: 2656-1743 |E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v5i3.526 Accredited rank 4 (SINTA 4), excerpts from the decision of the DITJEN DIKTIRISTEK No. 230/E/KPT/2023 3 sample. The model obtained an average accuracy of 83.3% (Sihotang et al., 2021). Other studies used the Hybrid method by combining C.F. and CBF approaches. The algorithm used for C.F. approaches is K-Nearest Neighbor (K.N.N.) to predict the rating value given by the user. In the case of the CBF approach, the features of the descriptive text features extracted by the TF-IDF method. The model obtained an M.A.E. result of 0.3678 with k neighbors of 25% (Lubis et al., 2020). Based on several studies, this research aims to find a suitable algorithm using the Hybrid method by combining the C.F. approach and Image Feature Extraction from CBIR. The C.F. approach with the M.F. algorithm was used to suggest tourist attractions to users from other users with similar preferences. Meanwhile, the Image Feature Extraction method performs the feature extraction process of tourist attractions images as additional data. RESEARCH METHODS The following figure 1 shows the propose method in this study. Figure 1. Proposed Method In Fig. 1, the proposed method has several stages. These stages will be carried out to gain a suitable method for recommending tourism to people. These stages consist of: a. Input Data, Tourism Transaction, and Tourism Image At this stage, user-item transaction data and images of each tourist spot will be input via the input layer. As for image data, the data will be converted into a vector. b. Fetch Features At this stage, user-item transaction data is processed by selecting only the features needed in the form of User ID features, Tourist Attraction I.D.s, and explicit ratings given by users. c. Normalization Next, the rating data is normalized to a value range of 0-1. The normalization used is the min-max normalization which uses the most significant value of the rating, which is 5, with the smallest value, 1. d. Resize Image Image data is processed by setting the image's dimension format (length and width) to 100 using the OpenCV Python package. e. Color Format Conversion The image is converted from the Blue Green Red (B.G.R.) color format to Red Green Blue (RGB). This is because the library used to read images uses OpenCV, which orders B.G.R. f. Feature Extraction Convolutional Neural Networks (CNN) Convolution Neural Network (CNN) is one of the implementations of artificial neural networks in image processing. CNN consists of several convolution blocks that take image vectors and perform adaptive learning of spatial features from images (Yamashita et al., 2018). CNN performs kernel striding operations to extract features from images at the convolution and pooling layers. After feature extraction, these features will be mapped to the fully connected layer as output results. The results of this output can be given to the regressor or classifier for later identifying suitable labels for the data. CNN has several well-known architectural implementations, including Resnet-50, VGG-16, VGG-19, InceptionV3, and Xception. In this study, the CNN architecture for feature extraction uses the Resnet-50 architecture. g. Embeddings Embeddings are a compression method to map the input feature results into vectors (Hrinchuk et al., 2019). In the recommendation system, embeddings are needed to get the latent vector User P-ISSN: 2656-1743 | E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v5i3.526 JURNAL RISET INFORMATIKA Vol. 5, No. 3. June 2023 Accredited rank 4 (SINTA 4), excerpts from the decision of the DITJEN DIKTIRISTEK No. 230/E/KPT/2023 4 ID and Tourism ID. The second vector combines the dot product multiplication operation to produce a factorized matrix. h. Matrix Factorization This method performs user-item object mapping by performing dot product multiplication operations based on the vector values generated by the user and item embeddings (Zhang, 2022). The use of Matrix Factorization also has the ability as Dimensionality Reduction because the dot product multiplication operation will only use vectors that already have values which can be seen in (1). R β‰ˆ 𝑃𝑇 . Q = Θ’ ............................................................... (1) (Zhang, 2022) Equation (1) shows that P is the user-feature matrix, and Q is the items-feature matrix. The dot product of two latent factors can predict the R rating of an item, which can be seen in (2). π‘Ÿπ‘–π‘— = 𝑝𝑖 𝑇 π‘žπ‘— = βˆ‘ π‘π‘–π‘˜ 𝑇 π‘žπ‘˜π‘— π‘˜ π‘˜=1 .......................................... (2) (Zhang, 2022) Equation (2) shows that r_ij is the predicted result of the dot product operation. The rating prediction results from the dot product operation will be compared with the actual results to obtain the loss value using the M.S.E. matrix, which can be seen in (3) and (4). 𝑒𝑖𝑗 2 = (π‘Ÿπ‘–π‘— βˆ’ ȓ𝑖𝑗 ) 2 = (π‘Ÿπ‘–π‘— βˆ’ βˆ‘ π‘π‘–π‘˜ π‘žπ‘—π‘˜ π‘˜ π‘˜=1 ) 2 ............... (3) πΏπ‘œπ‘ π‘ (𝑝, π‘ž) = βˆ‘ 𝑒𝑖𝑗 2 + πœ† (||π‘π‘–π‘˜ || 2 + ||π‘žπ‘—π‘˜ || 2 ) ....... (4) (Zhang, 2022) In equation (4), the loss function has Ξ» as regularization, which will be used as a penalty. The loss value will be used by gradient descent to minimize the difference between the predicted and actual values, which can be seen in (5), and (6). πœ•πΆ πœ•π‘π‘–π‘˜ = βˆ‘ 2(π‘Ÿπ‘–π‘— βˆ’ βˆ‘ 𝑝𝑖 π‘žπ‘— π‘˜ π‘˜=1 )𝑗 (βˆ’π‘žπ‘—π‘˜ ) + 2πœ†π‘π‘–π‘˜ ..... (5) πœ•πΆ πœ•π‘žπ‘—π‘˜ = βˆ‘ 2(π‘Ÿπ‘–π‘— βˆ’ βˆ‘ 𝑝𝑖 π‘žπ‘— π‘˜ π‘˜=1 )𝑗 (βˆ’π‘žπ‘–π‘˜ ) + 2πœ†π‘π‘—π‘˜ ..... (6) (Zhang, 2022) Equations (5) and (6) above will be used by the optimizer for learning to minimize the loss between the predicted results and the actual value. Data, Instruments, and Data Collection Techniques This study uses secondary data from 2021, namely Indonesian Tourism Dataset data, which has several unique users of 300 accounts, several tourist attractions (Tourism Sites) 437 spread across five major cities in Indonesia, and the total number of reservations along with ratings on a scale of 1-5 is 10,000 samples. This data is taken from a file in .csv format based on literature research. Figure 2. Distribution of Reservations User-based Based on Figure 2, it can be concluded that the data distribution is not normally distributed. Figure 3. Distribution of Ages Based on Figure 3, it can be concluded that the majority range of ages is between 27-30 Figure 4. Distribution of Ratings Based on Figure 4, it also can be concluded that most ratings are above 1. In addition to text data, image data are obtained through Google Images randomly. The sources of image data were also varied; the majority were obtained from Wikipedia, Pegipegi, Tokopedia, Traveloka, and Tiket. These images are used as a representation of tourist attractions, with a total of 437 images. RESULTS AND DISCUSSION In this study, several experiments were conducted using several architectures, including; Matrix Factorization, Neural Collaborative Filtering, Hybrid method using text features, Hybrid M.F. and N.C.F. methods with image features, and Hybrid MF- NCF method with image features. JURNAL RISET INFORMATIKA Vol. 5, No. 3. June 2023 P-ISSN: 2656-1743 |E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v5i3.526 Accredited rank 4 (SINTA 4), excerpts from the decision of the DITJEN DIKTIRISTEK No. 230/E/KPT/2023 5 Table 1. Algorithm Comparison Table Algoritma Metrics NDCG @5 Acc@5 MSE RMSE M.A.E. Matrix Factorizati on (M.F.) 38.4% 52.7% 0.653 0.808 0.649 Neural Collaborati ve Filtering (N.C.F.) 15.9% 51.8% 0.179 0.423 0.348 Hybrid Text Feature 42.8% 49.9% 0.160 0.400 0.333 Neural Collaborati ve Filtering Image 62.8% 52.1% 0.219 0.468 0.376 Matrix Factorizati on Image 28.2% 54.6% 0.158 0.398 0.332 Hybrid M.F. Image Recommen dation (Personaliz ed) 71.5% 53.8% 0.133 0.364 0.309 Hybrid N.C.F. Image Recommen dation (Personaliz ed) 85.8% 52.7% 0.133 0.364 0.311 Hybrid M.F. +NCF Image Recomme ndation (Personali zed) 60.2% 56.2% 0.142 0.376 0.317 Based on Table I, the results of the evaluation of the highest Indonesian Tourism Dataset data based on the regression matrix were obtained by the Hybrid M.F. Image Recommendation (Personalized) and N.C.F. Personalized methods, which had the lowest M.S.E., RMSE, and M.A.E. values compared to other methods. The regression matrix measures 'how close' the predicted results are to the actual results and becomes the main matrix. Table I shows that images as additional parameters for the model have a better average performance than those without image features. Furthermore, the additional personalization data from the categories of tourist attractions preferred by users increased model performance, as indicated by a lower average M.A.E. evaluation than the model without personalization. Based on the NDCG matrix, which is used to measure the quality of giving recommendations. Based on these scores, the highest model is N.C.F. Image Personalized, with a score of 85.8%. Based on the top-k accuracy matrix to get a slice of tourist attractions in the top recommendations for each user using the average value, the combined M.F. and N.C.F. Image Recommendation Personalized model obtained the highest average accuracy value of 56.2% compared to other methods. This result is due to the model's nature which combined the linear and non-linear models, resulting in broader regression for the model. These combinations acted as ensembled learning with the output of mean value between the two algorithms. Figure 5. Architecture of Hybrid Models (MF and N.C.F. Combined) P-ISSN: 2656-1743 | E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v5i3.526 JURNAL RISET INFORMATIKA Vol. 5, No. 3. June 2023 Accredited rank 4 (SINTA 4), excerpts from the decision of the DITJEN DIKTIRISTEK No. 230/E/KPT/2023 6 Based on Fig. 5, the hybrid models combined M.F. and N.C.F. models and averaged their outputs to gain the most relevant score. For the image feature extractor, the author used Resnet50. Figure 6. Set Up Personalization When a new user registers for the website, the website will redirect to the personalization setup page, where the user must choose four tourist attractions they like, which can be seen in Fig. 6. Figure 7. Home Page After setting up the personalization for a new user, the user will redirect to the home page, where the system provides top-5 recommendations based on the previous personalization setup or users' histories (old users), which can be seen in Fig. 7. Figure 8. Booking Page When the user plans to visit a tourist spot, the user can select the visit link to open the tourist place order form, which can be seen in Figure 8. CONCLUSIONS AND SUGGESTIONS Conclusion Based on the evaluation of the research that has been carried out on the proposed algorithms, the use of the Hybrid method with image and personalization features can provide tourist recommendations. Although, the evaluation of the M.S.E., RMSE, and M.A.E. regression matrices where the prediction results are not very satisfactory due to a significant distance from the actual value. In this case, the use of the Hybrid method is carried out by combining two examples of the implementation of the M.F. method, namely GMF and N.C.F., with the Resnet50 CNN architecture. In addition, this model is supported by user personalization in the form of tourist categories that each user likes. Based on the experimental results above, it was found that the Hybrid N.C.F. + M.F. Image (Personalized) model has an NDCG accuracy of JURNAL RISET INFORMATIKA Vol. 5, No. 3. June 2023 P-ISSN: 2656-1743 |E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v5i3.526 Accredited rank 4 (SINTA 4), excerpts from the decision of the DITJEN DIKTIRISTEK No. 230/E/KPT/2023 7 60.2% for the top-5 with an accuracy top-5 of 56.2%. Suggestion This recommendation system is not perfect and has room for improvement that can be improved further into a Context-awareness based recommendation system where recommendations are made based on users' online patterns, length of interaction with each site that provides information on certain tourist attractions, and also providing a matrix of calculating relevance scores provided by the user. This can positively impact the accuracy of scoring by users for each tourist spot. In addition, based on existing data, descriptions of tourist attractions can be processed by the Text Sequence Learning model to determine a suitable probability relationship between each description of tourist attractions. REFERENCES Biswas, P. K., & Liu, S. (2022). A hybrid recommender system for recommending smartphones to prospective customers. Expert Systems with Applications, 208(1). https://doi.org/10.1016/j.eswa.2022.11805 8 Γ‡akΔ±r, M., Γ–ΔŸΓΌdΓΌcΓΌ, Ş. G., & Tugay, R. (2019). A Deep Hybrid Model for Recommendation Systems. AI*IA 2019: AI*IA 2019 – Advances in Artificial Intelligence, 321–335. https://doi.org/10.1007/978-3-030-35166- 3_23 Elsayed, S., Brinkmeyer, L., & Schmidt-Thieme, L. (2022). End-to-end image-based fashion recommendation. Workshop on Recommender Systems in Fashion and Retail, 109–119. https://doi.org/10.1007/978-3-031-22192- 7_7 Geetha, G., Safa, M., Fancy, C., & Saranya, D. (2018). A Hybrid Approach using Collaborative filtering and Content based Filtering for Recommender System. Journal of Physics: Conference Series, 1000(1). https://doi.org/10.1088/1742- 6596/1000/1/012101 Hrinchuk, O., Khrulkov, V., Mirvakhabova, L., Orlova, E., & Oseledets, I. (2019). Tensorized Embedding Layers for Efficient Model Compression (pp. 1–13). arXiv. https://doi.org/10.48550/arXiv.1901.10787 Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), 30–37. https://doi.org/10.1109/MC.2009.263 Lubis, Y. I., Napitupulu, D. J., & Dharma, A. S. (2020). Implementasi Metode Hybrid Filtering (Collaborative dan Content-based) untuk Sistem Rekomendasi Pariwisata. 12th Conference on Information Technology and Electrical Engineering, 28–35. https://citee.ft.ugm.ac.id/2020/download51. php?f=TI-5 - Implementasi Metode Hybrid Filtering.pdf Mohammadpour, T., Bidgoli, A. M., Enayatifar, R., & Javadi, H. H. S. (2019). Efficient clustering in collaborative filtering recommender system: Hybrid method based on genetic algorithm and gravitational emulation local search algorithm. Genomics, 111(6), 1902–1912. https://doi.org/10.1016/J.YGENO.2019.01.0 01 Ng, Y. K. (2020). CBREC: A book recommendation system for children using the matrix factorisation and content-based filtering approaches. International Journal of Business Intelligence and Data Mining, 16(2), 129–149. https://doi.org/10.1504/IJBIDM.2020.10473 8 Ortega, F., Lara-Cabrera, R., GonzΓ‘lez-Prieto, Á., & Bobadilla, J. (2021). Providing reliability in recommender systems through Bernoulli matrix factorization. Information Sciences, 553, 110–128. https://doi.org/10.1016/j.ins.2020.12.001 Rian, Z., Christanti, V., & Hendryli, J. (2019). Content- Based Image Retrieval using Convolutional Neural Networks. Proceedings - 2019 IEEE International Conference on Signals and Systems, ICSigSys 2019, 1–7. https://doi.org/10.1109/ICSIGSYS.2019.881 1089 Walek, B., & Fojtik, V. (2020). A hybrid recommender system for recommending relevant movies using an expert system. Expert Systems with Applications, 158, 113452. https://doi.org/10.1016/J.ESWA.2020.11345 2 Wu, Z., Tian, H., Zhu, X., & Wang, S. (2018). Optimization Matrix Factorization Recommendation Algorithm Based on Rating Centrality (pp. 1–12). arXiv. https://doi.org/10.1007/978-3-319-93803- 5_11 Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018). Convolutional neural networks: an overview and application in radiology. Insights into Imaging, 9, 611–629. https://doi.org/10.1007/s13244-018-0639 Zhang, Y. (2022). An Introduction to Matrix P-ISSN: 2656-1743 | E-ISSN: 2656-1735 DOI: https://doi.org/10.34288/jri.v5i3.526 JURNAL RISET INFORMATIKA Vol. 5, No. 3. June 2023 Accredited rank 4 (SINTA 4), excerpts from the decision of the DITJEN DIKTIRISTEK No. 230/E/KPT/2023 8 factorization and Factorization Machines in Recommendation System, and Beyond (pp. 1– 15). arXiv. https://doi.org/10.48550/arXiv.2203.11026