INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL Online ISSN 1841-9844, ISSN-L 1841-9836, Volume: 17, Issue: 4, Month: August, Year: 2022 Article Number: 4568, https://doi.org/10.15837/ijccc.2022.4.4568 CCC Publications An Enhanced Neural Graph based Collaborative Filtering with Item Knowledge Graph Sangeetha M., Meera Devi Thiagarajan Sangeetha M. Department of Computer Science and Engineering, Kongu Engineering College, Erode, Tamilnadu, India Corresponding author: sangeethcse@gmail.com Meera Devi Thiagarajan Department of Electronics and Communication Engineering Kongu Engineering College Erode, Tamilnadu, India tmeeradevi@gmail.com Abstract Recommendation system is a process of filtering information to retain buyers on e-commerce sites or applications. It is used on all e-commerce sites, social media platform and multimedia platform. This recommendation is based on their own experience or experience between users. In recent days, the graph-based filtering techniques are used for the recommendation to improve the suggestions and for easy analysing. Neural graph based collaborative filtering is also one of the techniques used for recommendation system. It is implemented on the benchmark datasets like Yelp, Gowalla and Amazon books. This technique can suggest better recommendations as compared to the existing graph based or convolutional based networks. However, it requires higher processing time for convolutional neural network for performing limited suggestions. Hence, in this paper, an improved neural graph collaborative filtering is proposed. Here, the content-based filtering is performed before the collaborative filtering process. Then, the embedding layer will process on both the recommendations to provide a higher order relation between the users and items. As the suggestion is based on hybrid recommendation, the processing time of Convolutional neural network is reduced by reducing the number of epochs. Due to this, the final recommendation is not affected by the smaller number of epochs and also able to reduce its computational time. The whole process is realized in Python 3.6 under windows 10 environment on benchmark datasets Go Walla and Amazon books. Based on the comparison of recall and NDCG metric, the proposed neural graph-based filtering outperforms the collaborative filtering based on graph convolution neural network. Keywords: filtering techniques, neural network, graph convolution neural network https://doi.org/10.15837/ijccc.2022.4.4568 2 1 Introduction Recommender system is a process of filtering of information to retain the buyers in e-commerce sites or applications. It is used in all e-commerce sites like Amazon, flip kart, kindle, Kindle, and multimedia sites and social media sites. This recommendation is based their own experience or ex- perience between the users. Generally, the recommendation system is classified into three categories. The three categories are as follows: content-based filtering, collaborative based filtering and hybrid filtering. In a hybrid recommendation system, the results of both content and collaborative filtering are blended using weights or features to provide the user a final recommendation. Wu et al., (2014) utilized the fuzzy tree logic for the e-services platform first. Then, they utilized the fuzzy concept to improve the recommendation in E-learning applications. Here, the fuzzy tree is constructed for learn- ing activities and for the user profiles. Then, the matching algorithm is used to identify the similarities between these two trees. Based on the similarity, the recommendation is made for the user, which is suitable for their requirements. Luo et al., (2015) utilized the alternating direction method to improve the collaborative filtering process. The collaborative filtering-based recommendation system utilized the non-negative matrix factorization (NMF) for good representation of the target matrix. But this approach suffers from high complexity and computational cost. In order to solve these problems, a non-negative latent factor (ANLF) model based on the alternating direction method (ADM) proposed. The ADM process each single function to obtain higher convergence speed and lower complexity. It not only reduces the criteria but also improves the recommendation choices. Due to lower computational cost and complexity, this model can be realized in practical format. Song et al., (2016) utilized two main concepts in improving the recommendation choices in context-aware recommendation systems. The two concepts are cluster tree and click-behaviour. The cluster tree formed based on the items selected by the user. With these cluster trees, the recommendation system improved its choice by adding the user choices and new user arrivals. As the system updates its recommendations for every new arrival, the final item recommendation improved effectively in large datasets. Yang et al. (2017) utilized the combination of content and collaborative based filtering for suggesting the jobs to the user in the job portal sites. Here, the job suggestions are based on the statistical relation between the user and jobs. The functional gradient boosting is used for finding the relation between the items and user. Its performance is compared with Markov Process and Ackley technique based on accuracy, precision and recall. Mu (2018) surveyed the deep learning role in recommendation systems. The main advantage in deep learning technique is it can identify the relation between user and item effectively by analysing in non-linear and non-trivial mode. Here, it described the metrics and parameters used in recommendation system. Then, it described the deep learning role in recommendation process in terms of graph or data mining approach. Then, it concludes the survey with their challenges and its future scope. This survey gives a clear idea about the deep learning role in recommendation sys- tem. Liu et al., (2017) utilized the feedback mechanism to improve the recommendation choices in cloud storage system. As data in the cloud grows on a regular basis, recommendation algorithms struggle to keep up. To solve these problems, the feedback mechanism is utilized in the cloud ser- vices, to track the user behaviour and new arrival. The feedback role is to maximize the accuracy in recommendation choice. Ortega et al., (2018) analysed the collaborative filtering role in journal recommendation system. Here, the journal dataset collected from public available dataset. Then, the collected information processed to identify the dominant criteria for recommendations. Then, the dif- ferent collaborative filtering methods applied on the recommendation system to improve the choices. Among many collaborative filtering approaches, the probability matrix factorization provides the best results. Anandhan et al., (2018) surveyed the different recommendation system used in social media platforms like blog, multimedia content or forum sites. First, it describes the social media platform, where the recommendation system can be used. Then, it surveyed around 61 articles from 2011 to 2015 for recommendation process. Finally, it describes the challenges and its future remedy for the challenges. Zhang et al., (2018) utilized the transfer learning process to improve the recommendation choice and also to reduce the data sparsity problem. Here, the knowledge transfer takes place between the source and target domain. Because the target set has denser data than source and it also has same items with different rating. Hence, to improve the recommendation choice, the similarity between the user and items determined by aligning source and target domain features. Then, these aligned https://doi.org/10.15837/ijccc.2022.4.4568 3 feature information transfer between them using kernels. This knowledge transfer process helps the recommendation system to improve the user preferences. The paper is organized as follows: section 2 describes the recent techniques in recommendation system. The brief explanation of the proposed method given in section 3. Section 4 compares the proposed method performance with the existing techniques. Section 5 summarizes the paper and its future work. 2 Literature survey Iqbal et al., (2019) improved the user-item rating by selecting proper kernel for the recommenda- tion system. Here, the user-item rating improved by adding dominant factors that affect the rating of an item based on both user and item preferences. With this additional information, the recommenda- tion choice improved. To enhance the recommendation choice further, the kernel selects by analysing different kernel role in recommendation process. By proper kernel selection and contextual information about users and items improved the recommendation choices in benchmark datasets like movie lens dataset. Al-Ghuribhi and Noah (2019) utilized the different parameters that affect the recommenda- tion systems. Many recommendation systems utilise only the overall rating for the recommendation, which is not sufficient to improve the user’s preferences. Therefore, they utilized different factors like item features, user rating of an item and user choices analysed for recommendations. Due to the mul- tiple criterias, the dominant criteria determined through sentiment analysis. Based on the dominant criteria, the recommendation made, which improves the user choices. Yin et al. (2019) proposed an attention mechanism in the graph-based approach collaborative filtering recommender system. Here, the data sparsity problem is reduced by utilizing the attention mechanism in the learning of the graph neural network architecture. Chen et al., (2019) proposed an innovative clustering approach-based recommendation system. Here, first it constructs the bipartite graph between the user and items. Then, it extracts the features from it and project in single mode. But it suffers from the data sparsity problem. This data sparsity problem is overcome by using node2vec technology. The node2vec extracts the complex relationship in user or items. Based on this relationship, the clustering process performed using dynamic nearest neighbour technique. With the DNN and node2vec it improved the recommen- dation choices in larger dataset. Hors et al., (2019) designed a mobile application for recommending message for smoker to quit the smoking. First, it analysed the World Health Organization guidelines to determine the factors responsible for quitting smoking. Based on this feature, a behavioural model designed for user messages. Then, the recommendation system analysed using three different mod- els like user context, taxonomy and demographic variables. By combining these three models, the recommendation choice for the user improved in health recommendation system by incorporating the user behaviour and different contexts. Fernandez et al., (2019) proposed a recommendation system for higher education students to select their subjects. Here, the approach tested on the Spanish uni- versity dataset. Among many attributes like age, under graduate group, percentage and subjects, the subject provides a better recommendation choice for the students. Based on the subject criteria, the computer science group chosen as the optimal subject for the higher education, because it provides the student with high graduation rate. Ferreto et al., (2020) designed a recommendation model for suggesting physical activity in hypertension patients. Here, the data collected from the hypertension patients and three types of physical activities designed based on hypertension values. The dataset comprises thirty- two attributes and their roles in recommendation analysed using HyperRecSysPA. Based on this model, the physical activity prescribed for the patients. Then, the recommendations verified with the practioner, it observed that this model achieved 75Alshammari et al., (2019) designed a recommendation system based on black box testing model. Black box testing model analyse the recommendation choices in all attributes in the dataset. With this multiple analysis, the recommen- dation improved effectively but it lacks the semantic relationship between the choice and attributes. Therefore, a semantic relationship proposed between these attributes and choices, the recommendation improved effectively in movie lens dataset. Dau and Naomie (2019) utilized the sentiment analysis and attention-mechanism for the recommendation system in Amazon review and Yelp dataset. Here, the long short-term memory network used to extract the features and improve the attention process in the recommendations. This approach improved its accuracy in recommendation as compared to https://doi.org/10.15837/ijccc.2022.4.4568 4 the traditional mechanisms. Ngaffo et al., (2020) proposed a recommendation system using hybrid approach. The hybrid approach indicates it combines both the user and item-based recommendation. The Bayesian inference technique used for learning the recommendation and it shows that the recom- mendation improved by this hybrid approach. Based on the hybrid approach from Ngaffo et al., in this, an enhanced neural graph collaborative filtering proposed for the recommendation system. The recommendation system based on the three factors like user rating, item-based recommendation and similarity between the item rating. This hybrid approach improved the recommendation and requires less dropout embedding layer for processing. 3 Proposed Method In this section, the proposed method enhanced neural graph based collaborative filtering working explained. Here, three layers are responsible for the recommendation process. The three layers are as follows: Embedding layer, Propagation layer and Prediction layer. The simple diagram to describe the proposed method flow shown in the figure 1. Figure 1: : Flow of proposed enhanced NGCF https://doi.org/10.15837/ijccc.2022.4.4568 5 3.1 Embedding layer It is the first layer of the recommendation process. Here, the relation between the users (U) and items (I) and also the information about the items embedded from the dataset to propose a new choice for the user. Consider a dataset D comprising U users and I items and its relation between the users and relation between the items and the information regarding each item considered for making a recommendation process. It is given as follows 3.2 Propagation Layer In this layer, the strong relation between the user and items analysed by passing the information from embedding layer to multiple propagation layer with attention-based mechanism. Here, the single layer working explained and it is same for other higher order layers. 3.2.1 Working of first layer The relation between the user and items analysed with the help of three processes as in [21]. 1. Information construction 2. Attention process 3. Information aggregation. 3.2.2 Information construction The similarity between the user and items determined by sending the information from an item to user as follows The terms W1, W2 and W3 are the weight matrices to propagate the valuable information and it belongs to the dataset. In this, the three interactions were concentrated to improve the recommendation. The three interactions are item, interaction between the user and items and also between the entity relations between the items. Mostly, the item interactions only concentrate in other approaches, but in this, both users based and knowledge-based interactions were analysed by performing element wise multiplication process. The term P_UIIk is the decay factor for the propagation. Here, the prediction is based on the convolutional neural network. Due to this the decay factor is considered as Laplacian based normalized adjoint matrix. 3.2.3 Attention process In this process, the knowledge between the items analysed based on the user interactions by the information passing scheme. After the information passed from the items to user, the relation between the items analysed with the help of following formula Here, the terms E indicates the entity and it defined as the possible connections made by the items with the user. The subscripts S and E indicate the starting and ending point of the entity. The term R indicates the relation between the entities and the corresponding weight factor is indicated byWR. Based on this, the relation among the items is studied. This process is similar to the formation of EGO network in the graph. https://doi.org/10.15837/ijccc.2022.4.4568 6 3.2.4 Information Aggregation In this process, the information from all users is consolidated to define the user relation and also indicate the user-item relations. The output of the L embedding propagation layer is as follows: The above process is repeated for all the embedding layers and the final embedding layer output is obtained by aggregating the each embedding layer outputs. Due to the message passing and attention scheme, the relation between the user and items is enhanced as compared to the existing neural graph based collaborative filtering. 3.2.5 Prediction layer In this, the convolution neural network with Adam optimizer is used for the prediction process. The information gathered from the propagation layer is passed through different layers like weighted average, max pooling and and l2_normalization process for the final prediction for the target item. The model parameters were tuned by optimizing the Bayesian personalized Ranking loss. The final recommendation of an item is given as follows: The other tuning parameter for the convolutional neural network is the dropout ratio is 0.2, three embedding layers and one folding layer and 50 epochs with 0.01 learning rate for the training process with 64 layer as embedding size and 1024 as mini-batch size and 0.2 as node and message dropout ratio. Based on the above process, the recommendation for the user is proposed. This approach able to recommend the products effectively by identifying the stronger relation between the user and items using user-item graph and item knowledge. The implementation and discussion of the proposed method results with the other existing techniques is given in following section. 4 Experimental setup and result discussion 4.1 Software requirements The proposed method tested under windows 10 environment using python 3.6.5 with following packages numpy, scipy, scikit-learn and tensorflow. The proposed Enhanced neural graph based col- laborative filtering analysed on two benchmark datasets namely Go Walla and Amazon book dataset. 4.2 Dataset description Here, the traditional two benchmark datasets Go Walla and Amazon book dataset is used for the analysis. The description of the dataset is given in the table 1. Table 1: Dataset description https://doi.org/10.15837/ijccc.2022.4.4568 7 4.3 4.3 Performance Comparison of proposed Enhanced Neural graph based col- laborative filtering (E-NGCF) The proposed method performance compared with other graph based collaborative filtering like Neural graph based collaborative filtering (NGCF), graph-based convolution neural network using two metrics recall and NDCG. Here, NGCF implements the user-item embedding with the help of the embedding-based neural graph CNN. GCN, on the other hand, only uses first-order neighbours for the user item relationship. The comparison of the top 20 recommendation choices by proposed and existing method is shown in table 2. Table 2 shows that the proposed enhanced NGCF able to predict Table 2: Performance comparison the product recommendations accurately as compared to the other two existing algorithms for both the datasets. It achieved by adding the attention process in the propagation layer and also by finding the stronger relationship between the user and items using higher order embedding propagation process. 4.3.1 Performance analysis based on network parameters 4.3.2 Analysis based on Embedding Layers First, the performance of E-NGCF based on embedding layers is shown in the table 3. In both Table 3: E-NGF performance w.r.t. Embedding layers datasets, increasing the propagation layers results in improving the choice of recommendation only. Therefore, the three embedding layers are sufficient for recommender system. Because, adding more embedding layer may results in overfitting. 4.3.3 Analysis based on N-fold The term N-fold is used to split the adjoint matrix for the message propagation function. In existing NGCF, the N-fold value is 100 and it requires 1658.0 s time period for executing one epoch of training. But, in the proposed E-NGCF, it is reduced to 1, because the splitting of adjoint matrix is not effective after fold 1. Due to this it required only 991.6 s for one epoch and the time period for consecutive epochs is also reduced. Therefore, by reducing the N-fold, the computational time for the filtering process can be reduced. 4.3.4 Analysis based on the dropout ratios Dropout ratio is an important parameter in the network to prevent the network from overfitting and underfitting problem. The terms ND and MD in the following tables and figures indicates the https://doi.org/10.15837/ijccc.2022.4.4568 8 node dropout ratio (i.e., user dropout) factor and MD represents the message dropout factor in the propagation layers. Based on this, the performance of the proposed method is compared for the dropout ratios 0.0 to 0.8 with regular interval of 0.2. The proposed method E-NGCF is compared with the existing NGCF for both the datasets and it is shown in the figures and table. Table 4 shows Table 4: Go Walla comparison based on Drop out ratios Figure 2: :Drop out comparison for Gowalla that the proposed method is able to perform better even with the variations in the drop out ratio. Hence, the dropout ratio has lesser effects in recommendation process. But the optimal drop out ratio for the proposed method is 0.2. Similarly for the Amazon book dataset is shown in the table 5. In both the datasets, the proposed E-NGCF able to improve its recall value as compared to the existing NGCF. But the optimal dropout ratio is 0.2 and it remains same for both the methods. It is due to the embedding layer process. 4.3.5 Analysis based on the Epochs Due to the higher order connectivity and knowledge graph, the proposed method able to reduce its training loss to 40.03 for four epochs. For comparison, it requires 10 epochs to reach the same training loss by existing NGCF. Therefore, in all aspects like performance comparison and network parameters and computational time, the proposed method enhanced neural graph based collaborative filtering outperforms the existing neural graph based collaborative filtering. https://doi.org/10.15837/ijccc.2022.4.4568 9 Table 5: Amazon book comparison based on Drop out ratios Figure 3: :Drop out comparison for Amazon book 5 Conclusion The proposed method, Enhanced Neural Graph Collaborative Filtering, outperforms the existing NGCF technique by using the following information: user, user-item graph, and item-knowledge graph for the recommendation process. It also included the attention process in the propagation layer. It reduces the adjoining matrix split operation. The proposed approach improves its performance in the following parts by incorporating the aforementioned adjustments into the present strategy. It improved the choice of recommendation by having a high recall and NDCG as compared to the existing technique. It reduces the computational time for the training by reducing the splitting process. In the training phase, the minimal loss is achieved in only four epochs. Overall, the proposed modification improved the existing NGCF performance in all aspects, like choice of recommendation and processing time. Therefore, the proposed enhanced neural graph based collaborative filtering is the best recommendation system. In the future, the proposed method might be enhanced by using neural multiplication instead of dot multiplication in the message propagation process. References [1] Dhingra, S., & Kumar, D. (2019). A review of remotely sensed satellite image classification. International Journal of Electrical & Computer Engineering (2088-8708), 9(3). [2] Kumar, R., & Sharma, P. K. (2019). Classification Techniques for Object Detection in Remote Sensing Images. Available at SSRN 3356495. https://doi.org/10.15837/ijccc.2022.4.4568 10 [3] Das, P., & Pandey, V. (2019). Use of Logistic Regression in Land-Cover Classification with Moderate-Resolution Multispectral Data. Journal of the Indian Society of Remote Sensing, 47(8), 1443-1454. [4] Barde, I., Suryawanshi, N., Samarth, S., Bhure, T., Rehpade, N. N., & Balamwar, S. Object Based Classification Using Image Processing Techniques. [5] Borra, S., Thanki, R., & Dey, N. (2019). Satellite Image Classification. In Satellite Image Anal- ysis: Clustering and Classification (pp. 53-81). Springer, Singapore. [6] Tuba, E., Jovanovic, R., & Tuba, M. (2020). Multispectral Satellite Image Classification Based on Bare Bone Fireworks Algorithm. In Information and Communication Technology for Sustainable Development (pp. 305-313). Springer, Singapore. [7] Mohammed, M. A., Naji, T. A., & Abduljabbar, H. M. (2019). The effect of the activation functions on the classification accuracy of satellite image by artificial neural network. Energy Procedia, 157, 164-170. [8] Yu, L., Lan, J., Zeng, Y., & Zou, J. (2019). Comparison of Land Cover Types Classification Meth- ods Using Tiangong-2 Multispectral Image. In Proceedings of the Tiangong-2 Remote Sensing Application Conference (pp. 241-253). Springer, Singapore. [9] Ishii, T.; Nakamura, R.; Nakada, H.; Mochizuki, Y.; Ishikawa, H. (2015). Surface object recog- nition with CNN andSVM in Landsat 8 images. In Proceedings of the IEEE 14th IAPR Interna- tional Conference on MachineVision Applications (MVA), Tokyo, Japan, 18–22 May 2015; pp. 341–344. [10] Espinola, M., Piedra-Fernandez, J. A., Ayala, R., Iribarne, L., & Wang, J. Z, (2014). “Con- textual and Hierarchical Classification of Satellite Images Based on Cellular Automata”, IEEE Transactions on Geoscience and Remote Sensing, 53(2), 795–809.doi:10.1109/tgrs. [11] M. Espnola et al., (2010). Cellular automata applied in remote sensing to implement contextual pseudo-fuzzy classification, in Proc. 9th Int. Conf. ACRI, vol. 6350. [12] SarikaYadav, Imdad Rizvi, ShailajaKadam, Luis Iribarne, and James Z. Wang, (2015). Urban Tree Canopy Detection Using Object-Based Image Analysis for Very High Resolution Satellite Images IEEE Trans on geosciences and remote sensing. [13] F. S. Al-Ahmadi and A. S. Hames, (2009). , “Comparison of Four Classification Methods to Extract Land Use and Land Cover from Raw Satellite Images for Some Remote Arid Areas, Kingdom of Saudi Arabia”, JKAU; Earth Sci., Vol. 20 No.1, pp: 167-191 (A.D./1430 A.H.) [14] D. Menaka, L. Padmasuresh and S. SelvinPrem Kumar, 2015. "Classification of Multispectral Satellite Images using Sparse SVM Classifier", Indian Journal of Science and Technology, Vol. 8, No. 24. [15] DM.Längkvist, A. Kiselev, M. Alirezaie and A.Loutfi 2016. Classification and segmentation of satellite orthoimagery using convolutional neural networks. Remote Sensing, 8(4), 329. https://doi.org/10.15837/ijccc.2022.4.4568 11 Copyright ©2022 by the authors. Licensee Agora University, Oradea, Romania. This is an open access article distributed under the terms and conditions of the Creative Commons Attribution-NonCommercial 4.0 International License. Journal’s webpage: http://univagora.ro/jour/index.php/ijccc/ This journal is a member of, and subscribes to the principles of, the Committee on Publication Ethics (COPE). https://publicationethics.org/members/international-journal-computers-communications-and-control Cite this paper as: Sangeetha M; Thiagarajan M.-D. (2022). An Enhanced Neural Graph based Collaborative Filtering with Item Knowledge Graph, International Journal of Computers Communications & Control, 17(4), 4568, 2022. https://doi.org/10.15837/ijccc.2022.4.4568 Introduction Literature survey Proposed Method Embedding layer Propagation Layer Working of first layer Information construction Attention process Information Aggregation Prediction layer Experimental setup and result discussion Software requirements Dataset description 4.3 Performance Comparison of proposed Enhanced Neural graph based collaborative filtering (E-NGCF) Performance analysis based on network parameters Analysis based on Embedding Layers Analysis based on N-fold Analysis based on the dropout ratios Analysis based on the Epochs Conclusion