3___AITI#8538___105-117 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 An Integrated Approach towards Efficient Image Classification Using Deep CNN with Transfer Learning and PCA Rahul Sharma * , Amar Singh Department of Computer Applications, Lovely Professional University, Jalandhar, India Received 22 September 2021; received in revised form 21 October 2021; accepted 22 October 2021 DOI: https://doi.org/10.46604/aiti.2022.8538 Abstract In image processing, developing efficient, automated, and accurate techniques to classify images with varying intensity level, resolution, aspect ratio, orientation, contrast, sharpness, etc. is a challenging task. This study presents an integrated approach for image classification by employing transfer learning for feature selection and using principal component analysis (PCA) for feature reduction. The PCA algorithm is employed for reducing the dimensionality of the features extracted by the VGG16 model to obtain a handful of features for speeding up image reorganization. For multilayer perceptron classifiers, support vector machine (SVM) and random forest (RF) algorithms are used. The performance of the proposed approach is compared with other classifiers. The experimental results establish the supremacy of the VGG16-PCA-Multilayer perceptron model integrated approach and achieve a reorganization accuracy of 91.145%, 95.0%, 92.33%, and 98.59% on Fashion-MNIST dataset, ORL dataset of faces, corn leaf disease dataset, and rice leaf disease datasets, respectively. Keywords: dimensionality reduction, feature extraction, image recognition, PCA, transfer learning 1. Introduction Manual image recognition by experts suffers from different limitations, e.g., time-consuming, human error, biased opinion, non-availability of experts, etc. Nowadays, automated image recognition technology appears in many aspects of day-to-day tasks and plays a very important role in education, face recognition, medical disease diagnosis, agricultural activities, driverless cars, advertising, image restoration, etc. Automated solutions are very useful for amateurs as well as field experts. With the advances in technology, the widespread use of image capturing and processing devices enabled people to capture, share, search, and retrieve images. Image recognition helps identify and analyze the objects and use the learned knowledge for decision making. The objective of image recognition is to label a given image into class among a set of pre-defined classes. To accurately label an image, the extraction of useful image features is very important. Features represent the patterns of the object in the image and are used for recognition. There are different methods for feature extraction from images such as grayscale feature extraction, the use of mean pixel values of channels, and edge feature extraction. The grayscale feature extraction uses a single channel as an input, and the number of features will be the same as the number of pixels. Mean pixel values of channels generate a matrix using the pixel values from three channels. Thus, the number of features remains the same and the pixel values of all three channels are considered. The edge features extraction method extracts edges as features and uses that as the input for the model. Edges can be extracted by using different algorithms, e.g., Prewitt edge detection, Sobel edge detection, * Corresponding author. E-mail address: prof.sharma.rahul@gmail.com Tel.: +91-7006957906; Fax: +91-1922-234315 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 Laplacian edge detection, Canny edge detection, etc. Low-level features of an image can also be extracted by using different techniques, e.g., histogram of oriented gradients (HOG), generalized search tree (GIST), scale-invariant feature transform (SIFT), speeded up robust feature (SURF), etc [1]. Training image classification algorithms by using these hand-crafted features is time-consuming and requires technical expertise. The pretrained convolution neural network (CNN) models are trained using large, distributed, and standard datasets. The weights learned by the pretrained model can be reused to solve a similar problem. Transfer learning employing the pretrained CNN enables filters to extract valuable characteristics from images [2]. The CNN model has different layers such as the convolutional layer, pooling layer, dropout layers, non-linear layers, and fully connected layers. The feature maps of these layers can be viewed and give a distinct representation of the input image. The VGG16 architecture is employed to extract important features of images [3]. To improve the space and time complexity, principle component analysis (PCA) is applied as feature reduction technique. Deep learning techniques are successfully applied for solving the problems related to healthcare, agriculture, engineering, entertainment, music composition, advertising, signal processing, image recognition, robotics, image coloring, image captioning, etc. The application of machine learning techniques for solving real-time applications efficiently and accurately is gaining importance. Real-time applications often require handling huge data. Extracting significant features and reducing noise are required for improving computational speed and accuracy. Some of the current work in this field includes handling multi-sensor data for chatter detection using metaheuristic algorithms and recursive feature elimination [4], feature selection using fuzzy entropy measures with a similarity classifier [5], effective feature selection using enhanced PCA [6], artificial bee colony for feature selection of breast cancer data [7], etc. Recent studies have integrated deep learning with the Internet of Things (IoT) for performing real-time complex sensing and recognition tasks. IoT applications generate a huge volume of data that requires preprocessing and dimensionality reduction. Some of the tasks involving IoT and deep learning include reducing the energy consumption of IoT enabled devices [8], online automated monitoring of cyber-attacks [9], automatic online detection of defects in gas-insulated switchgear [10], anomaly prediction in IoT networks [11], real-time monitoring of agriculture fields [12], etc. The main objective of this study is to propose an efficient and accurate approach for image recognition based on the integrated approach. Some of the challenges faced by deep learning image classifiers are computationally expensive, energy-intensive, and time-consuming, and have high memory requirements. Thus, there is a need for scaling up the performance of image classifiers and overcoming the bottlenecks faced during model training and image classification. The proposed approach reduces the computational complexity and maintains high classification accuracy. The contribution of the work is summarized in the following points. (1) An integrated approach is proposed employing deep learning for feature extraction, PCA for feature reduction, and image classification module with multilayer perceptron, support vector machine (SVM), and random forest (RF). (2) The accuracy of multilayer perceptron, SVM, and RF classifiers is enhanced by tuning their hyperparameters using a grid search algorithm. (3) With the PCA reduced features, efficient utilization of computation resources is conducted with the improved computational efficiency of model training and testing. This study is organized as follow. Section 2 presents literature review. Section 3 introduces transfer learning, the PCA algorithm, and the proposed image classification approach. Section 4 presents the results and discussion. Finally, section 5 presents the conclusion. 106 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 2. Related Work Several techniques were proposed for image classification. Kaur et al. [13] explored various pretrained CNN models for the classification of magnetic resonance brain images. Their proposed pretrained deep CNN models were demonstrated. The authors explored 8 different pretrained CNN models out of which the AlexNet model illustrated best classification accuracy. Pires De Lima et al. [14] presented remote-sensing image classification using transfer learning. Their CNN models trained on diverse natural image datasets gave better results for remote-sensing image classification tasks. The study demonstrates that the pretrained model trained on a generalized dataset can be accurately applied to remote-sensing image classification. The results show that the pretrained models can easily be used for feature extraction of the unseen images related to different domains. Xue et al. [15] presented an ensemble learning strategy based on Inception-V3, Xception, VGG16, and ResNet-50 CNN models. The ensemble learning technique employed a weighted voting approach with an accuracy of 98.61%. The high computational requirements of the classifier which used four pretrained models as base learners are one of the limitations of their proposed approach. Garcia-Dominguez et al. [16] proposed an automatic optimum image feature selector and classification open-source tool “FrImCla” [16]. The tool based upon the statistical study automatically selects the feature extraction and classification techniques. For feature selection, the pretrained models as well as traditional methods are explored. Transfer learning techniques are popularly used for feature selection in computer vision. Sert and Boyacı [17] presented a feature fusion transfer learning-based approach for the recognition of sketch drawings. For improving the efficiency of feature reduction, PCA was proposed. The selected features were input to SVM for classification. The proposed approach was able to classify freehand sketches of the Sketchy dataset with an accuracy of 97.91%. Chen et al. [18] proposed an automated robot arm control program enabling human-robot interaction using the YOLOv4 algorithm. Eight different hand gestures were recorded in a controlled laboratory environment and image features were classified by employing a deep CNN. The proposed approach classified the images accurately and the recognized hand gestures were used to control robot arm movement. Khan et al. [19] presented an improved saliency-based segmentation technique for the extraction of the infected area of cucumber leaves. The deep neural networks VGG-VD-19 and VGG-S, M, F were employed for the extraction of useful features. Subsequently, the features obtained were fused and were used for image classification by SVM with an accuracy of 98.08%. Kaur et al. [20] employed the pretrained model AlexNet to identify the patients with Parkinson disease. The final layers of the model were modified and the transfer learning-based model was able to diagnose the disease with an accuracy of 89.23%. Li et al. [21] presented a shallow VGG16 neural network-based approach for plant disease detection. The block of the VGG16 pretrained model was used for the extraction of prominent image features. The PCA reduced features were input to SVM or RF algorithms. Experimental results illustrate that the simple shallow neural network and statistical machine learning algorithms are very useful for plant disease detection. Sujatha et al. [22] presented the analysis of different machine learning and pretrained deep learning algorithms for the classification of diagnosis of citrus plant diseases. The experimental work shows that deep learning algorithms performed better than machine learning algorithms for disease classification. The VGG16 model outperformed other classifiers (VGG19, Inception-V3, RF, stochastic gradient descent, and SVM). Behera et al. [23] presented papaya maturity stage prediction models using k-nearest neighbor (KNN), SVM, naïve Bayes, and deep learning neural networks (AlexNet, VGG16, VGG19, GoogleNet, ResNet50, etc). Both the VGG16 and VGG19 models were able to classify papaya images with an accuracy of 100%. The VGG19 model achieved 100% training accuracy in 10 epochs. Chiu et al. [24] proposed an efficient method for predicting breast cancer on the dataset obtained from University Hospital Care of Coimbra. For reducing the dimensionality of the dataset, the PCA algorithm was used. The features were extracted by using a transfer learning neural network and 107 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 classification by SVM. Loey et al. [25] presented a hybrid approach employing the pretrained CNN model ResNet50 for feature extraction of the images of real-world masked face dataset (RMFD), simulated masked face dataset (SMFD), and labelled faces in the wild (LFW) dataset. The extracted features were classified by SVM decision tree and ensemble learning techniques. Using the proposed approach, the SVM model classified RMFD with 99.64%, SMFD with 99.49%, and LWF with 100% accuracy. Bhagwat et al. [26] presented a review of traditional machine learning and deep learning techniques for early and accurate detection of plant diseases. Early detection of plant diseases is a challenging task that needs to be addressed for precision agriculture. Deep learning architectures illustrate their significance for important feature extraction and classification. Deep neural networks require large datasets for training. Collecting a large dataset is very time-consuming and expensive. Transfer learning using pretrained models (e.g., single-shot multibox detector, VGGNet, ResNet50, InceptionV2, GoogLeNet, MobileNet, etc.) is very useful to overcome these limitations and recognize the plant diseases with high accuracy. Training a deep learning model with large data of diverse examples is highly desirable to avoid over-fitting. Generally, the aim is to develop a model which generalizes well on the unseen test data by minimizing both the bias and variance. To solve a complex problem, the model needs to learn more parameters so that the data size for training increases. Transfer learning is commonly employed to solve computer vision problems with a small dataset, and the acquisition of more data is time-consuming or expensive. Ren et al. [27] demonstrated accurate image classification employing transfer learning algorithms on small or poor image datasets. The minimum data size requirements for the training transfer learning model were explored. The proposed model was able to be trained using small, poor-resolution, unfavorably cropped, or rotated images. Transfer learning algorithm was able to achieve above 75% accuracy with different data sizes. Weimann et al. [28] presented a transfer learning-based approach for the classification of atrial fibrillation. The CNN model was first trained on a large public dataset and then the weights learned were fine-tuned on a small dataset of atrial fibrillation. The proposed approach employing transfer learning improved the performance of the CNN model by 6.57%. Liu et al. [29] presented an automatic manufacturing defect detection system with limited training samples employing transfer learning. The proposed model was able to extract the discriminative features and was able to detect defects with an accuracy of 99% improving the model accuracy by 11%. Li et al. [30] proposed an approach demonstrating the usefulness of transfer learning to train deep learning models on small training data for the diagnosis of COVID-19. The transfer learning using the CheXNet model already trained for the classification of chest X-ray images was employed. The already learned knowledge was fine-tuned on the limited COVID-19 dataset. The proposed approach outperformed several other methods. The first few layers of a deep CNN model extract generic features, and the final layers extract the specific features of an image. Yosinski et al. [31] discussed the applicability of transferring features from the base task to the target task. The generalization performance of the model is enhanced by transferring features from the base layers. The performance is enhanced when the base task and target task are similar. Using the pre-learned weights for the neural network is better than initializing the weights randomly. Mahajan et al. [32] discussed an efficient plant species recognition model which used transfer learning for feature extraction and enhanced multiclass adaptive boosting for image classification. The open-source plant species dataset FLAVIA was used in the study. Using 10-fold cross-validation, an accuracy of 95.85% was achieved. Singh et al. [33] applied a novel nature-inspired multi-population parallel three-parent genetic algorithm (P3PGA) for the optimization of routing problems in wireless mesh networks. The P3PGA benchmarked tests on CEC-2014 tests and their application for finding minimum cost routes illustrate faster convergence time compared to other popular nature-inspired algorithms. The optimization techniques enhance the performance of image classification algorithms. Table 1 presents some of the recent studies on feature extraction, feature reduction, and classification. 108 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 Table 1 Description, technique, and limitation of recent studies Ref. Description Technique Limitation [34] Real-time accurate and efficient monitoring of water quality of water resources for decision making Long short-term memory recurrent neural networks (LSTM RNNs), PCA, linear discriminate analysis (LDA), and independent component analysis (ICA) for feature reduction Classification performance is not tested using deep learning classifiers. [35] Non-linear ensemble learning for improving feature extraction and forecasting the carbon price Deep learning, LSTM, RF, RNN, and bagging algorithm The proposed ensemble learning approach needs to select optimal models rather than random selection. [36] COVID-19 prediction by inspecting X-ray images Transfer learning feature extraction ResNet18, ResNet50, ResNet101, VGG16, VGG19, and SVM for classification with an accuracy of 94.7% The severity of the disease is not detected. [37] Detection of Alzheimer disease by classification of functional MRI images KNN, SVM, decision tree, LDA, RF, and CNN Limited dataset is used. [38] Detection of external defects in the tomatoes ResNet18, ResNet34, ResNet50, ResNet101, ResNet152, grid search with a high accuracy of 94.6% Deep autoencoders and one-class classifiers used for improving accuracy need to be explored. The classifiers need to be generalized. [39] Selection of optimum features for medical image classification Gray level co-occurrence matrices (GLCM), grey-level run length matrix (GLRLM), crow search optimization algorithm for feature selection, and deep learning Further segmentation and feature reduction are needed. [40] Detection of pain intensity by analyzing facial expressions VGG-Face for feature extraction, PCA for improving computational efficiency, and LSTM for classification A limited number of images and non-availability of a standard dataset [41] Transfer learning for image classification of Caltech 101 dataset VGG19, SIFT, SURF, oriented FAST and rotated BRIEF (ORB), Shi-Tomasi corner detector algorithm for feature extraction, Gaussian naïve Bayes, decision tree, RF, and XGBClassifer for classification Multiple feature extraction techniques are required for improving accuracy and required more computational resources. The VGG16 architecture proposed by Simonyan et al. [42] secured the first place in the object localization track of the ImageNet challenge and was able to detect an object within an input image with high accuracy. In the image classification track of the ImageNet challenge, the VGG16 neural network achieved 92.7% top-5 test accuracy for classifying over 14 million images belonging to 1000 different classes. The VGG16 model was tested with VOC-2007, VOC-2012, Caltech-101, and Caltech-256 benchmarked datasets. The experimental results illustrate that the VGG16 network can generalize well to the unseen data. 3. Materials and Methods 3.1. Transfer learning using deep convolutional neural networks The already trained models are very useful to overcome the limited training dataset challenge by reusing the knowledge already learned from large training samples. The transfer learning using CNN models reuses the knowledge learned by the neural network trained on a large dataset and applies the knowledge for solving another problem. VGG16, VGG19, ResNet50, ResNet101, InceptionV3, DenseNet121, Xception, etc are some of the popular neural network architectures which have demonstrated excellent image classification performance. In this study, the VGG16 model as shown in Fig. 1 is used for the extraction of useful image features. Fig. 1 VGG16 network architecture as a feature extractor 109 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 The VGG16 model architecture has 13 convolution layers of 3 × 3 filters stacked together along with 5 max-pooling layers of the size 2 × 2. After the last max-pooling layer, there are three dense layers. VGG16 uses the ReLU activation function and softmax layer in the final dense layer. The ImageNet pretrained model VGG16 has fewer tunable hyperparameters and achieves high classification accuracy. The low-level features already learned by a pretrained model are used for learning another problem. The extracted features are input to PCA to select the most relevant features, enhance the performance, and reduce the training time. 3.2. The PCA algorithm PCA was introduced by Karl Pearson [43]. PCA is also known as hotelling transform (HT) and is a very popular dimensionality reduction technique used effectively in the areas of image and signal processing. PCA is used for reducing the size of the feature vectors that are used for object recognition and object classification. PCA can be implemented using eigenvalues decomposition (EVD) or singular value decomposition (SVD). PCA transforms the feature vectors with a large number of correlated variables into a smaller number of uncorrelated variables also known as principal components (PCs). Consider a dataset having K images with N = n × n pixels, the dataset is represented by D = N × K matrix where Di represents the ith row of the matrix or ith image of the dataset. Algorithm 1 gives the PCA algorithm. Algorithm 1: PCA Step 1: For i = 1 to N, calculate the mean of Di using Eq. (1): 1== ∑ i k ij j D k µ (1) Step 2: Shift the origin to the mean of the data by subtracting the mean �� from each column vector ��� as shown in Eq. (2): Φ = −ij ij iD µ (2) Step 3: Compute the covariance matrix of mean-centered data using Eq. (3): = ΦΦT C (3) where T represents the transposition matrix. Step 4: Find eigenvalues ��, ��, ��, …, � and eigenvectors ��, ��, ��, …, � of C where �� > �� > �� > … � . Step 5: Arrange the eigenvectors in descending order and return top k eigenvalues corresponding to k number of the largest eigenvalues also known as PCs. The PCA algorithm is a popular dimensionality reduction technique which transforms a large set of correlated variables into fewer uncorrelated variables. PCA computes uncorrelated variables by transforming the data to a new coordinate system maintaining as much variance as possible. PCA is employed for reducing a large number of image features into the reduced one-dimensional feature set representation (i.e., PCs) as shown in Fig. 2. PCA Flat image 1D vector of length 1 × N (Spatial domain) 1D vector of length 1 × k (where k < N) (PCA space) Fig. 2 PCA feature reduction 110 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 For a given set of data, PCA finds a new axis system defined by the principal directions of variance. Suppose the dataset has 400 images of the size 256 × 256, then the size of the data matrix will be 400 × (256 × 256) = 400 × 65536. Therefore, for each of the 400 images, the data matrix will have 65536 columns. The size of the covariant matrix and transformation matrix becomes 65536 × 65536. Now, using PCA, 50 columns of the transformation matrix corresponding to the 50 largest eigenvalues are selected. Then, the size of the reduced transformation matrix P is computed as 65536 × 50. The next step is to obtain the transformed dataset T using Eq. (4). × ×= Φ s N N k T P (4) The size of the transformed dataset is 400 × 50. Thus, the initial data matrix of the size 400 × 65536 is reduced to the new representation, i.e., the transformed matrix T of the size 400 × 50. 3.3. The proposed approach In this section, the proposed approach for image classification is described in details. Deep neural networks extract useful feature maps for the recognition of images. Recent studies in computer vision have successfully used CNN for feature representation. Building and training an efficient CNN from scratch requires technical and domain expertise. Designing an optimized CNN architecture for a real-world computer vision task is a complicated and time-consuming task. In this study, the VGG16 architecture is employed for feature extraction. The weights of the convolution base layers of the VGG16 model are reused and are not updated. The VGG16 neural network processes the input image to extract activation maps that describe features in an image [44]. The features are extracted from the block5_pool layer of the VGG16 architecture. The features of the images extracted from the VGG16 model are converted into a vector of 32768 numbers. The PCA feature reduction technique is employed to reduce the dimensionality of VGG16 features and at the same time maintain the distinctive properties of the features thereby improving the training/prediction time. The reduced features are fed to the multilayer perceptron model or SVM model. The hyperparameters of these image classifiers are optimized using a grid-search algorithm [45]. Fig. 3 shows the proposed image recognition approach. Fig. 3 The proposed image recognition approach (Fashion-MNIST dataset) 3.4. Datasets The proposed image classification approach is validated on four different image datasets, namely, ORL dataset of faces, Fashion-MNIST dataset, corn leaf disease dataset, and rice leaf disease dataset. The training and testing datasets are randomly divided in the ratio of 80:20. Some of the sample images of the dataset are shown in Fig. 4. 111 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 S1/2 S2/3 S3/8 S7/10 S6/5 S8/7 S10/4 S15/7 (a) ORL dataset of faces Dress Coat Pullover Bag Pullover Coat Ankle Boot Shirt (b) Fashion-MNIST dataset Blight Common rust Gray spot Health (c) Corn leaf diseases dataset Bacterail blight Blast Brown spot Tungro (d) Rice leaf disease dataset Fig. 4 Sample images of datasets Table 2 Dataset images used for the evaluation of the proposed approach Image dataset Training images Testing images Total images ORL 160 40 200 Corn leaf disease 3350 838 4188 Rice leaf disease 4105 1027 5132 Fashion-MNIST 60000 10000 70000 Table 2 shows the number of training and testing images of different datasets used for the evaluation of the proposed approach. In this study, 10 different images of each of 20 distinct subjects from the ORL dataset are used. The ORL database includes grayscale face images of 92 × 112 pixels collected in varying lighting conditions with similar backgrounds. The dataset captures different facial expressions of persons organized in a directory for each subject with names in the format S1, S2, S3, etc. Fashion-MNIST is a standard dataset having 60000 grayscale images of 28 × 28 pixels corresponding to 10 different types of clothing. Fashion-MNIST dataset is popularly used for benchmarking computer vision and deep learning algorithms. 4. Results and Discussion As discussed in section 4, the images are fed to the VGG16 models for feature extraction. The extracted features represent useful information required for image classification. The features are classified using multilayer perceptron, SVM, and RF image classifiers. The hyperparameters of these classifiers are optimized by using a grid search algorithm. The multilayer perceptron model is trained for 50 epochs. The SVM algorithm used in the study is evaluated by fitting 5 folds for each of the 50 candidates totaling 250 fits. The hyperparameters optimized by the grid search algorithm for SVM are c = 1, gamma = 0.01 and kernel = 'rbf' when the ORL images are classified using VGG16 features. The accuracy and time of image classification are recorded and are shown in Table 3. 112 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 Dimensionality reduction is required for improving the performance of the model. Linear discriminate analysis (LDA), as well as PCA, can be used for feature reduction. LDA uses class information to obtain new features whereas PCA evaluates PCs by making use of variance matrix, covariance matrix, eigenvector, and eigenvalues of each feature. PCA is used to speed up the computation and improve the performance of the image classifier. Figs. 5-6 give the PCA reduced features representation of corn leaf disease dataset and Fashion-MNIST dataset respectively. The evaluated PCs are input to multilayer perceptron, SVM, and RF classifiers. Tables 4-6 show the classification time and the accuracy of the multilayer perceptron, SVM, and RF classifiers respectively. With 100 PCs, the multilayer perceptron gives the best accuracy and outperforms the SVM and RF classifiers. The image classification performance recorded in Table 3 and Table 4 demonstrate that the proposed approach reduces the computational complexity of the image classifiers. For example, for the classification of rice leaf disease test images, VGG16 and multilayer perceptron give an accuracy of 97.27% whereas using the proposed approach with 100 PCs the classification accuracy of 98.93% is achieved. In addition, the improvement in classification time by 107.17 seconds is also observed. Table 3 Image classification without PCA Dataset Artificial neural network (ANN) Support vector machine (SVM) Random forest (RF) Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) ORL 95.0 6.45 92.5 0.82 92.5 120.45 Corn leaf disease 89.14 142.8 88.78 55 86.51 123.56 Rice leaf disease 93.81 140.4 98.54 61.2 97.27 121.4 Fashion-MNIST 91.56 174.26 90.1 110.3 87.78 153.41 (a) PC = 2 (b) PC = 3 Fig. 5 Corn leaf disease dataset (a) PC = 2 (b) PC = 3 Fig. 6 Fashion-MNIST dataset 113 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 Table 4 Accuracy/time of multilayer perceptron classifier (20 epochs) using PCA features Dataset PC = 10 PC =20 PC =50 PC =100 Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) ORL 87.5 1.74 90.0 1.81 92.2 1.83 95.24 1.898 Corn leaf disease 83.05 10.05 85.08 10.84 86.99 10.91 92.36 10.95 Rice leaf disease 93.86 10 95.33 10.82 97.27 11.018 98.93 14.23 Fashion-MNIST 75.11 15.35 81.45 17.75 84.15 18.45 92.93 19.68 Table 5 Accuracy/time of SVM classifier using PCA algorithm Dataset PC = 10 PC = 20 PC = 50 PC = 100 Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) ORL 82.50 1.83 85.5 2.02 90.01 3.30 92.2 4.5 Corn leaf disease 81.503 1.89 83.89 2.47 87.11 3.8 89.14 5.15 Rice leaf disease 72.15 5.56 80.62 6.73 85.78 8.84 92.11 11.88 Fashion-MNIST 71.54 11.56 76.15 12.67 83.57 15.86 89.15 18.65 Table 6 Accuracy/time of RF classifier using PCA algorithm Dataset PC = 10 PC = 20 PC = 50 PC = 100 Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) Accuracy (%) Time (Sec) ORL 77.50 1.14 92.5 1.9 87.5 2.7 90.05 3.45 Corn leaf disease 78.64 1.47 85.91 1.93 86.04 2.41 88.07 4.52 Rice leaf disease 75.17 3.25 78.57 4.84 83.74 5.42 88.12 9.19 Fashion-MNIST 73.21 10.67 75.95 9.41 81.21 13.47 84.37 15.62 (a) ORL dataset of faces (b) Corn leaf disease dataset (c) Rice leaf disease dataset (d) Fashion-MNIST dataset Fig. 7 Confusion matrix of different datasets 114 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 Fig. 7 shows the confusion matrix of the datasets which can be used for evaluating the performance of the proposed approach. A confusion matrix is very useful for evaluating performance measures like accuracy, precision, sensitivity, specificity, negative predictive value, etc. Learning redundant and less relevant information does not give generalized results. Training models with higher dimension data can suffer from overfitting. The experimental result illustrates that the proposed approach improves the performance of classifiers. 5. Conclusions Efficiently classifying images and avoiding overfitting is a challenging task. In this study, an efficient approach employing the VGG16 model for feature extraction and PCA for feature reduction was presented. The extracted features were classified without applying feature reduction techniques and the results were recorded. The performance of the model was improved by integrating the PCA feature reduction algorithm. The features extracted by the VGG16 model were fed to PCA, and the optimum number of PCs for achieving the best accuracy was evaluated. For the classification of images, the features extracted by the VGG16 model were classified using multilayer perceptron, SVM, and RF algorithms. The proposed approach employing transfer learning was validated using four benchmarked image datasets, considering performance metrics. The proposed approach accurately classified the images with improved computational efficiency compared to other methods with the same hardware resources. The models trained using PCA-reduced features demonstrated significant improvement in running speed. The scope of the proposed framework can further be expanded by optimizing the PCs by using nature-inspired search and optimization algorithms. Furthermore, the feature extracted by transfer learning-based neural networks can be fused or an ensemble model can be explored to enhance the accuracy of the classifier. Moreover, fuzzy rough sets, ranking-based models, and regularized regression models will be researched for feature selection enhancing the performance of image classifiers. Conflicts of Interest The authors declare no conflict of interest. References [1] R. Guha, A. H. Khan, P. K. Singh, R. Sarkar, and D. Bhattacharjee, “CGA: A New Feature Selection Model for Visual Human Action Recognition,” Neural Computing and Applications, vol. 33, no. 10, pp. 5267-5286, May 2021. [2] S. Ahuja, B. K. Panigrahi, N. Dey, V. Rajinikanth, and T. K. Gandhi, “Deep Transfer Learning-Based Automated Detection of COVID-19 from Lung CT Scan Slices,” Applied Intelligence, vol. 51, no. 1, pp. 571-585, January 2021. [3] J. Pardede, B. Sitohang, S. Akbar, and M. L. Khodra, “Implementation of Transfer Learning Using VGG16 on Fruit Ripeness Detection,” International Journal of Intelligent Systems and Applications, vol. 13, no. 2, pp. 52-61, 2021. [4] M. Q. Tran, M. K. Liu, and M. Elsisi, “Effective Multi-Sensor Data Fusion for Chatter Detection in Milling Process,” ISA Transactions, in press. [5] M. Q. Tran, M. Elsisi, and M. K. Liu, “Effective Feature Selection with Fuzzy Entropy and Similarity Classifier for Chatter Vibration Diagnosis,” Measurement, vol. 184, 109962, November 2021. [6] D. Hemavathi and H. Srimathi, “Effective Feature Selection Technique in an Integrated Environment Using Enhanced Principal Component Analysis,” Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 3, pp. 3679-3688, 2021. [7] S. Punitha, F. Al-Turjman, and T. Stephan, “An Automated Breast Cancer Diagnosis Using Feature Selection and Parameter Optimization in ANN,” Computers and Electrical Engineering, vol. 90, 106958, March 2021. [8] M. Elsisi, M. Q. Tran, K. Mahmoud, M. Lehtonen, and M. M. Darwish, “Deep Learning-Based Industry 4.0 and Internet of Things towards Effective Energy Management for Smart Buildings,” Sensors, vol. 21, no. 4, 1038, February 2021. [9] M. Q. Tran, M. Elsisi, K. Mahmoud, M. K. Liu, M. Lehtonen, and M. M. Darwish, “Experimental Setup for Online Fault Diagnosis of Induction Machines via Promising IoT and Machine Learning: Towards Industry 4.0 Empowerment,” IEEE Access, vol. 9, pp. 115429-115441, 2021. 115 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 [10] M. Elsisi, M. Q. Tran, K. Mahmoud, D. E. A. Mansour, M. Lehtonen, and M. M. Darwish, “Towards Secured Online Monitoring for Digitalized GIS Against Cyber-Attacks Based on IoT and Machine Learning,” IEEE Access, vol. 9, pp. 78415-78427, 2021. [11] I. Ullah and Q. H. Mahmoud, “Design and Development of a Deep Learning-Based Model for Anomaly Detection in IoT Networks,” IEEE Access, vol. 9, pp. 103906-103926, 2021. [12] G. Delnevo, R. Girau, C. Ceccarini, and C. Prandi, “A Deep Learning and Social IoT approach for Plants Disease Prediction toward a Sustainable Agriculture,” IEEE Internet of Things Journal, in press. [13] T. Kaur and T. K. Gandhi, “Deep Convolutional Neural Networks with Transfer Learning for Automated Brain Image Classification,” Machine Vision and Applications, vol. 31, no. 3, 20, March 2020. [14] R. Pires De Lima and K. Marfurt, “Convolutional Neural Network for Remote-Sensing Scene Classification: Transfer Learning Analysis,” Remote Sensing, vol. 12, no. 1, 86, January 2020. [15] D. Xue, X. Zhou, C. Li, Y. Yao, M. M. Rahaman, J. Zhang, et al., “An Application of Transfer Learning and Ensemble Learning Techniques for Cervical Histopathology Image Classification,” IEEE Access, vol. 8, pp. 104603-104618, 2020. [16] M. Garcia-Dominguez, C. Dominguez, J. Heras, E. Mata, and V. Pascual, “FrImCla: A Framework for Image Classification Using Traditional and Transfer Learning Techniques,” IEEE Access, vol. 8, pp. 53443-53455, 2020. [17] M. Sert and E. Boyacı, “Sketch Recognition Using Transfer Learning,” Multimedia Tools and Applications, vol. 78, no. 12, pp. 17095-17112, June 2019. [18] S. L. Chen and L. W. Huang, “Using Deep Learning Technology to Realize the Automatic Control Program of Robot Arm Based on Hand Gesture Recognition,” International Journal of Engineering and Technology Innovation, vol. 11, no. 4, pp. 241-250, September 2021. [19] M. A. Khan, T. Akram, M. Sharif, K. Javed, M. Raza, and T. Saba, “An Automated System for Cucumber Leaf Diseased Spot Detection and Classification Using Improved Saliency Method and Deep Features Selection,” Multimedia Tools and Applications, vol. 79, no. 25, pp. 18627-18656, July 2020. [20] S. Kaur, H. Aggarwal, and R. Rani, “Diagnosis of Parkinson’s Disease Using Deep CNN with Transfer Learning and Data Augmentation,” Multimedia Tools and Applications, vol. 80, no. 7, pp. 10113-10139, March 2021. [21] Y. Li, J. Nie, and X. Chao, “Do We Really Need Deep CNN for Plant Diseases Identification?” Computers and Electronics in Agriculture, vol. 178, 105803, November 2020. [22] R. Sujatha, J. M. Chatterjee, N. Z. Jhanjhi, and S. N. Brohi, “Performance of Deep Learning vs Machine Learning in Plant Leaf Disease Detection,” Microprocessors and Microsystems, vol. 80, 103615, 2021. [23] S. K. Behera, A. K. Rath, and P. K. Sethy, “Maturity Status Classification of Papaya Fruits Based on Machine Learning and Transfer Learning Approach,” Information Processing in Agriculture, vol. 8, no. 2, pp. 244-250, June 2021. [24] H. J. Chiu, T. H. S. Li, and P. H. Kuo, “Breast Cancer-Detection System Using PCA, Multilayer Perceptron, Transfer Learning, and Support Vector Machine,” IEEE Access, vol. 8, pp. 204309-204324, 2020. [25] M. Loey, G. Manogaran, M. H. N. Taha, and N. E. M. Khalifa, “A Hybrid Deep Transfer Learning Model with Machine Learning Methods for Face Mask Detection in the Era of the COVID-19 Pandemic,” Measurement, vol. 167, 108288, January 2021. [26] R. Bhagwat and Y. Dandawat, “A Review on Advances in Automated Plant Disease Detection,” International Journal of Engineering and Technology Innovation, vol. 11, no. 4, pp. 251-264, September 2021. [27] S. Ren and C. Q. Li, “Robustness of Transfer Learning to Image Degradation,” Expert Systems with Applications, in press. [28] K. Weimann and T. O. Conrad, “Transfer Learning for ECG Classification,” Scientific Reports, vol. 11, no. 1, pp. 1-12, 5251, 2021. [29] J. Liu, F. Guo, H. Gao, M. Li, Y. Zhang, and H. Zhou, “Defect Detection of Injection Molding Products on Small Datasets Using Transfer Learning,” Journal of Manufacturing Processes, vol. 70, pp. 400-413, October 2021. [30] C. Li, Y. Yang, H. Liang, and B. Wu, “Transfer Learning for Establishment of Recognition of COVID-19 on CT Imaging Using Small-Sized Training Datasets,” Knowledge-Based Systems, vol. 218, 106849, 2021. [31] J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How Transferable are Features in Deep Neural Networks?” https://arxiv.org/pdf/1411.1792.pdf, November 06, 2014. [32] S. Mahajan, A. Raina, X. Z. Gao, and A. K. Pandit, “Plant Recognition Using Morphological Feature Extraction and Transfer Learning over SVM and Adaboost,” Symmetry, vol. 13, no. 2, 356, Februay 2021. [33] A. Singh, S. Kumar, A. Singh, and S. S. Walia, “Parallel 3-Parent Genetic Algorithm with Application to Routing in Wireless Mesh Networks,” Implementations and Applications of Machine Learning, vol. 782, pp. 1-27, 2020. 116 Advances in Technology Innovation, vol. 7, no. 2, 2022, pp. 105-117 [34] S. Dilmi and M. Ladjal, “A Novel Approach for Water Quality Classification Based on the Integration of Deep Learning and Feature Extraction Techniques,” Chemometrics and Intelligent Laboratory Systems, vol. 214, 104329, July 2021. [35] J. Wang, X. Sun, Q. Cheng, and Q. Cui, “An Innovative Random Forest-Based Nonlinear Ensemble Paradigm of Improved Feature Extraction and Deep Learning for Carbon Price Forecasting,” Science of the Total Environment, vol. 762, 143099, March 2021. [36] A. M. Ismael and A. Şengür, “Deep Learning Approaches for COVID-19 Detection Based on Chest X-Ray Images,” Expert Systems with Applications, vol. 164, 114054, Februaty 2021. [37] M. Amini, M. Pedram, A. Moradi, and M. Ouchani, “Diagnosis of Alzheimer’s Disease Severity with fMRI Images Using Robust Multitask Feature Extraction Method and Convolutional Neural Network (CNN),” Computational and Mathematical Methods in Medicine, vol. 2021, 5514839, 2021. [38] A. Z. da Costa, H. E. Figueroa, and J. A. Fracarolli, “Computer Vision Based Detection of External Defects on Tomatoes Using Deep Learning,” Biosystems Engineering, vol. 190, pp. 131-144, February 2020. [39] R. J. S. Raj, S. J. Shobana, I. V. Pustokhina, D. A. Pustokhin, D. Gupta, and K. Shankar, “Optimal Feature Selection-Based Medical Image Classification Using Deep Learning Model in Internet of Medical Things,” IEEE Access, vol. 8, pp. 58006-58017, 2020. [40] G. Bargshady, X. Zhou, R. C. Deo, J. Soar, F. Whittaker, and H. Wang, “Enhanced Deep Learning Algorithm Development to Detect Pain Intensity from Facial Expression Images,” Expert Systems with Applications, vol. 149, 113305, July 2020. [41] M. Bansal, M. Kumar, M. Sachdeva, and A. Mittal, “Transfer Learning for Image Classification Using VGG19: Caltech-101 Image Data Set,” Journal of Ambient Intelligence and Humanized Computing, in press. [42] K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” https://arxiv.org/pdf/1409.1556.pdf, April 10, 2015. [43] K. Pearson, “LIII. On Lines and Planes of Closest Fit to Systems of Points in Space,” The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, vol. 2, no. 11, pp. 559-572, 1901. [44] M. Toğaçar, Z. Cömert, and B. Ergen, “Classification of Brain MRI Using Hyper Column Technique with Convolutional Neural Network and Feature Selection Method,” Expert Systems with Applications, vol. 149, 113274, July 2020. [45] L. Yao, Z. Fang, Y. Xiao, J. Hou, and Z. Fu, “An Intelligent Fault Diagnosis Method for Lithium Battery Systems Based on Grid Search Support Vector Machine,” Energy, vol. 214, 118866, January 2021. Copyright© by the authors. Licensee TAETI, Taiwan. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY-NC) license (https://creativecommons.org/licenses/by-nc/4.0/). 117