ISSN : 2693 6356 2022 | Vol 5 | Issue 5 Effective Use of Convolutional Neural Networks for Deep Learning in CBIR Lakshmeelavanya Alluri 1 , Hemalatha Dendukuri 2 Assistant Professor 1,2 , SRKR pharmacy College, Bhimavaram, Andhra Pradesh, India. Abstract Research into content-based image retrieval (CBIR) has attracted a great lot of attention. Users have a hard time coming up with the input since a CBIR system uses low-level observable characteristics, and the system itself doesn't provide satisfactory retrieval results. To boost the efficiency of the retrieval process in a CBIR system, it is crucial to investigate effective feature representations and suitable similarity metrics. The fundamental problem has been the semantic mismatch between the low-level picture pixels and the high-level semantics that people comprehend. Machine learning (ML) is one approach that has been investigated as a potential means of closing the semantic gap. In this research, we are motivated by the recent achievements of deep learning techniques in computer vision applications to take on an advanced deep learning technique called Convolutional Neural Network (CNN) in order to investigate its effects on the study of feature representations and similarity measures. In this research, we investigated how CNNs may be used to address classification and retrieval challenges. We decided to adapt the deep architecture for image retrieval by employing transfer learning. By calculating the Euclidean distance between each picture's feature vectors and our query image, we can determine which images are most similar to it in the dataset. These feature vectors were extracted from the retraining of the proposed CNN model's last-but-one fully connected layer. INTRODUCTION The popularity of search engines like Bing Image Search has skyrocketed in recent years. With the proliferation of image search engines like Gazopa's (Private Company) CBIR search engine, Imense Image Search Portal (Private Company), and Like.com (Private Company), picture retrieval has become a formidable challenge. The problems, restrictions, and time commitments inherent with metadata-based systems have increased curiosity in CBIR. Existing technology makes it easy to search through textual information, but it is impractical for humans to manually describe each image in a database, especially for very large databases or for images that will be generated automatically, like those from surveillance cameras. There are further problems, such as the possibility of missing photos whose descriptions use alternative comparable words. The miscategorization issue may be avoided using systems that classify photos according to semantic classes, such as "tiger" as a subclass of "animal," but the user will have to put in more work to find the images that might be tigers but are instead classified as "animals." To solve the image retrieval issue, or the challenge of searching for specific digital pictures inside vast databases, content-based image retrieval (CBIR) applies techniques for image collecting, preprocessing, analysis, representation, and comprehension. In contrast to more conventional methods, the CBIR system is built on the principles of concept-based image indexing (CBII) [1].The effectiveness of a CBIR system's retrieval function relies heavily on the accuracy with which features are represented and similarity measurements are taken. Despite the many methods proposed, this is still a difficult undertaking because of the disconnect between the image's pixels and the high-level semantics people experience. ML is a promising strategy for addressing this issue over the long run. When it comes to classification tasks and feature research, deep learning is a family of ML algorithms where several layers of data processing stages in hierarchical patterns are used [2]. The field of picture categorization has seen remarkable progress because to deep learning frameworks. There is a discrepancy between how photographs are categorized and how they are ranked in terms of similarity. Black boots, white boots, and dark-gray boots are all boots for the purposes of image classification, however when rating comparable photos, we often score the dark- gray boot higher than the white boot if the query image is a black boot. For data with a grid-like structure, such as picture data (a 2D grid of pixels), CNNs [2] are a specialized ANN. Simply said, convolutional neural networks (CNNs) are ANNs in which at least one layer replaces matrix multiplication with convolution. Convolution allows for parameter sharing, equivariant representations, and sparse interactions, all of which might aid in the development of a better ML system. This challenge is well-suited for the use of neural networks, and CNNs in particular are well- known for their capacity to learn forms, textures, and colors. Using a state-of-the-art deep learning system— CNNs for examining feature representations from image data—we conducted research on a deep learning architecture for CBIR systems. In general, we retrain the previously learned CNN model using our data. Next, the trained network is put to use to do two things: categorize objects into their correct classes and conduct a nearest-neighbors analysis to return the most comparable and relevant photos to the input image [3-4]. I. RELATED WORK The 1.2 million pictures in the ImageNet dataset were classified into 1000 categories using a deep CNN trained by Krizhevsky et al. [5]. The authors developed an eight-layer network, the first five of which were convolutional layers and the last three of which were completely linked. This network has been trained using two GTX 580 GPUs, each with 3GB of memory, since the memory limitations of a single GPU would prevent it from being trained with a larger dataset. Top-1 error rates of 37.5% and top-5 error rates of 17.0% were attained by the authors when they employed characteristics derived from the 7th layer to retrieve related images. However, Babenko et al. [6] recommended to compress the features using dimensionality reduction approach and achieved a decent performance due to the large dimensionality of CNN features and the inefficiency of similarity calculation between two 4096-dimensional vectors. For hash learning, researchers have turned to deep models. The revolutionary performance of image retrieval using deep learning was shown by Xia et al. [7], who introduced a supervised hashing approach to investigate binary hash codes. They have utilized a matrix decomposition approach to examine the codes being used to represent the data in a preliminary processing stage. However, in the event of enormous data, this step is essential due to the increased storage and processing needs. Using hashing-based approaches that project the high-dimensional features to low- dimensional feature space and construct the binary hash codes, Lin et al. [8] suggested an easy and effective supervised learning model for rapid image retrieval system. This method uses binary pattern matching techniques or Hamming distance calculation, both of which enhance search efficiency while drastically decreasing computing time. The authors say that whereas it takes 109.767ms to compute the Euclidean distance between two 4096- dimensional feature vectors, it only takes 0.113ms to compute the Hamming distance between two 128-bit binary codes. Adding more layers to the network and more neurons to each layer is the simplest technique to improve the performance of Deep Neural Networks (DNNs). Szegedy et al. [3] developed a deep CNN architecture called Inception that outperformed previous methods on the ImageNet dataset for both classification and detection. The efficiency with which this strategy utilizes networked computer resources is its major metric. The writers have expanded the network's reach and depth. The architectural tradeoffs were made with the Hebbian principle in mind. This architecture is useful for dramatically expanding the number of neurons at each stage without introducing a corresponding increase in computing complexity. Increases in both the breadth of each step and the total number of steps are now possible because to more efficient use of computer resources. using the goals of addressing clothing style categorization and related clothes retrieval, Chen et al. [9] investigated Deep Learning using CNNs. Transfer learning is used to simplify training by refining pre-trained structures on huge datasets. The model is built around a collection of deep networks, each of which was trained on a small subset of data due to the massive size of deep networks' parameters. On three apparel datasets, our strategy outperformed previous methods that use ML algorithms with shallow structure, with an 18% increase in accuracy seen on the biggest dataset including 80,000 photos.In their study, Khosla and Venkataraman [10] use a strategy wherein additional CNNs are trained on the shoe dataset and then used to categorize input shoe images into suitable shoe classes and to conduct the closest neighbors assessment to return K most similar shoes to the given input shoe picture. The closest matches to the input picture were returned using Caffe as the neural network architecture and the Euclidean distance measure. The average score was 4.12 out of 5, and the retrieval accuracy was 75.6%, both of which were attained using this method of determining the Euclidean distance between the features vectors of the pictures. For fine- grained image categorization using pretrained neural networks, Iliukovich-Strakovskaia et al. [11] proposed a 'Two Flow Model' in which the input picture is processed in two distinct ways. In the first process, the picture is transformed into a low-dimensional feature vector space using certain conventional dimensionality reduction algorithms, and then the most useful features are selected. The second route processes the picture using a previously trained CNN, which uses the features from the global pooling layer to refine the feature selection process. At last, a nonlinear classifier is fitted using a combined set of characteristics extracted from both flows. Using Inception_BN and Inception_21k as examples of pretrained deep neural networks, this method employs Random Forest for both feature selection and the final step of a nonlinear classifier. By comparison, Inception_21k achieved 69.3% accuracy on the global pooling layer, whereas Inception_BN deep neural network accuracy ranged from 55.5% to 68.0%. II. PROPOS ED IMPLEMENTATION This section describes proposed methodology which employs DConvNet for CBIR system. Working of CNN can be explained as follows: A 2- D convolutional layer applies sliding filters to the input. The layer convolves the input by moving the filters along the input vertically and horizontally and computing the dot product of the weights and the input, and then adding a bias term. A Re LU layer performs a threshold operation to each element of the input, where any value less than zero is set to zero. A max pooling layer performs down- sampling by dividing the input into rectangular pooling regions and computing the ma ximu m of each region. A fully connected layer multiplies the input by a weight matrix and then adds a bias vector. 3.1. DL-CNNFig. 1. Proposed DConvNet for CBIR system eigenvalue decomposition of every covariance matrix, which gives the list of eigenvalues or According to the facts, training and testing of DL-CNN involves in allowing every source image via a succession of convolution layers by a kernel or filter, rectified linear unit (Re LU), max pooling, fully connected layer and utilize SoftMax layer with classification layer to categorize the objects with probabilistic values ranging from [0,1]. Figure 1 discloses the architecture of DL-CNN that is utilized in proposed methodology for CBIR system for enhanced feature representation of word image over conventional retrieval systems. 3.2. Principal component analysis Principal component analysis is an approach of machine learning which is utilized to reduce the dimensionality. It utilizes simple operations of matrices from statistics and linear algebra to compute a projection of source data into the similar count or lesser dimensions. PCA can be thought of a projection approach where data with m-columns or features are projected into a subspace by m or even lesser columns while preserving the most vital part of source data. Let Ibe a source image matrix with a size of n * m and results in J which is a projection of I. The primary step is to compute the value of mean for every column. Next, the values in every column are centered by subtracting the value of mean column. Now, covariance of the centered matrix is computed. At last, compute theeigenvectors. These eigenvectors constitute the directions or components for the reduced subspace of J, whereas the peak amplitudes for the directions are represented by these eigenvectors. Now, these vectors can be sorted by the eigenvalues in descending order to render a ranking of elements or a xes of the new subspace for I. Genera lly, k eigenvectors will be selected which are referred principal components or features 3.3. Euclidean distance To evaluate distances between query word image 𝐼𝑞 and retrieved word images 𝐼𝑟, a metric must be defined. We need a measurement method to tell how the query and retrieved word images are similar (bit per bit). Therefore, we want a similarity measure where the distance value will be the number of similar bits in the considered images. Fig. 2. Illustration of Euclidean distance Fig. 3. Retrieved dog images using DConvNet CBIR system III. RES ULTS AND DISCUSS ION In this section we discussed the simulation results of CBIR system. The proposed algorithm has been tested with few databases and displayed the outputs in the below figures. Fig. 3 shows that retrieving images using proposed CBIR scheme. Similarly, proposed retrieval system has been shown in fig. 4 and 5 with different classification images. As a measure of performance, we have used two widely used metrics of Precision and Recall. Precision is a measure of ability of CBIR algorithm to retrieve only relevant images, while Recall decides the ability of CBIR algorithm to retrieve all relevant images as defined by eq. (1) and eq. (2) respectively. Fig. 4. Retrieved car images using DConvNet CBIR system Fig. 5. Retrieved bird images using DConvNet CBIR system IV. CONCLUS IONS Using DConvNet and PCA with pairwise hamming distance, this paper demonstrated a powerful CBIR system. The suggested CBIR system showed higher performance in simulations, with more relevant pictures being retrieved. Additionally, mAP and mAR are used to illustrate how the proposed CBIR system's performance compares to that of previously described CBIR systems. REFERENCES [1] Y. Liu, D. Zhang, G. Lu, and W.-Y. Ma, ―A survey of content-based image retrieval with high-level semantics,‖ Pattern recognition, vol. 40, no. 1, pp. 262– 282, 2007. [2] Y. Le Cun, Y. Bengio, and G. Hinton, ―Deep learning,‖ Nature, vol. 521, no. 7553, pp. 436– 444, 2015. [3] C. Szegedy,W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, ―Go ing deeper with convolutions,‖ in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9. [4] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, ―Rethinking the inception architecture for computer vision,‖ in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2818– 2826. [5] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ―Imagenet classification with deep convolutional neural networks,‖ in Advances in neural in formation processing systems, 2012, pp. 1097–1105. [6] A. Babenko, A. Slesarev, A. Chigorin, and V. Lempitsky, ―Neural codes for image retrieval,‖ in European conference on computer vision. Springer, 2014, pp. 584–599. [7] R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan, ―Supervised hashing for image retrieval via image representation learning.‖ in AAAI, vol. 1, 2014, p. 2. [8] K. Lin, H.-F. Yang, J.-H. Hsiao, and C.-S. Chen, ―Deep learning of binary hash codes for fast image retrieval,‖ in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2015, pp. 27– 35. [9] J.-C. Chen and C.-F. Liu, ― Visual-based deep learning for clothing from large database,‖ in Proceedings of the ASE Big Data & Social Informatics 2015. ACM, 2015, p. 42. [10] N. Khosla and V. Venkataraman, ― Building image-based shoe search using convolutional neural networks,‖ CS231n Course Project Reports, 2015. [11] A. Iliu kovich-Strakovskaia, A. Dral, and E. Dral, ― Using pre-trained models for fine- grained image classification in fashion field,‖ 2016. [12] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell, ―Decaf: A deep convolutional activation feature for generic visual recognition.‖ in Icml, vol. 32, 2014, pp. 647– 655. [13] G. Shrivakshan, C. Chandrasekar et al., ―A comparison of various edge detection techniques used in image processing,‖ IJCSI International Journal of Computer Science Issues, vol. 9, no. 5, pp. 272–276, 2012. [14] A. Maurya and R. Tiwari, ―A novel method of image restoration by using different types of filtering techniques,‖ International Journal of Engineering Science and Innovative Technology (IJESIT) Vo lume, vol. 3, 2014. [15] R. Kandwal, A. Kumar, and S. Bhargava, ―Review: existing image segmentation techniques,‖ International Journal of Advanced Research in Computer Science and Software Engineering, vol. 4, no. 4, 2014. [16] K. Roy and J. Mukherjee, ―Image similarity measure using color histogram, color coherence vector, and sobel method,‖ International Journal of Science and Research (IJSR), vol. 2, no. 1, pp. 538–543, 2013. [17] J. Shlens, ―Train your own image classifier with Inception in Tensor- Flow,‖ https://research.googleblog.com/2016/03/train- your-ownimageclassifier-with.html, 2016. [18] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. Le- Cun, ―Over feat: Integrated recognition, localization and detection using convolutional networks,‖ arXiv preprint arXiv:1312.6229, 2013. [19] P. Wu, S. C. Hoi, H. Xia, P. Zhao, D. Wang, and C. Miao, ―Online multimodal deep similarity learning with application to image retrieval,‖ in Proceedings of the 21st ACM international conference on Multimedia. ACM, 2013, pp. 153– 162. [20] S. Liu, Z. Song, G. Liu, C. Xu, H. Lu, and S. Yan, ―Street-to shop: Cross-scenario clothing retrieval via parts alignment and auxiliary set,‖ in Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2012, pp. 3330–3337. [21] K. Yamaguchi, M. H. Kiapour, L. E. Ortiz, and T. L. Berg, ―Retrieving similar styles to parse clothing,‖ IEEE transactions on pattern analysis and machine intelligence, vol. 37, no. 5, pp. 1028–1040, 2015. [22] J. Wan, P. Wu, S. C. Hoi, P. Zhao, X. Gao, D. Wang, Y. Zhang, and J. Li, ― Online learning to rank for content-based image retrieval,‖ 2015.