Microsoft Word - ETASR_V12_N1_pp8033-8038 Engineering, Technology & Applied Science Research Vol. 12, No. 1, 2022, 8033-8038 8033 www.etasr.com Salamh & Akyuz: A Novel Feature Extraction Descriptor for Face Recognition A Novel Feature Extraction Descriptor for Face Recognition Ahmed B. Salem Salamh Material Science and Engineering Department Kastamonu University Kastamonu, Turkey nuclear_2010@hotmail.com Halil Ibrahim Akyüz Computer and Teaching Technologies Education Dpt. Kastamonu University Kastamonu, Turkey hakyuz@kastamonu.edu.tr Abstract-This paper presents a new feature extraction technique for face recognition. The new model, called multi-descriptor, is based on the well-known method of local binary patterns. It involves many different neighborhoods of the central pixel. Its unique advantage is that this descriptor allows the use of different neighborhood sizes instead of only one point. This structure ensures reasonable effectiveness and also provides the possibility to obtain a different distribution of features. Based on the new descriptor, a face recognition model using the pairwise feature descriptor based on the proposed descriptor was developed in this work, and local binary patterns were created to investigate the similarity and dissimilarity between the two models. For both models, the training was done using the support vector machine method on different face databases to overcome face recognition problems such as camera distance, expression, large head size, and illumination variations. The proposed technique achieved perfect accuracy on almost all tested databases including the Extended Yale B and Grimace database. Keywords-multi descriptor model; local binary pattern; face recognition; feature extraction I. INTRODUCTION Face recognition has been intensively researched and developed to introduce models that lead to more accurate results [1]. The great importance of face recognition lies in the identification and authentication, especially in security and registration systems. The main idea in any recognition technique is to interpret the physical characteristics of the object to be recognized [2]. Biometric recognition is widely used. Face recognition outperforms fingerprint and iris recognition [3]. Most problems in face recognition usually involve pose, scale, orientation, illumination, and facial expression. Over time, numerous methods have been proposed to solve these problems [4]. Moreover, constrained and unconstrained environments have become complicated due to different lighting conditions, pose, occlusion, and distance from the camera [5]. Different angles and illuminations affect facial features as the physical characteristics of the human face change [6]. There are many methods that have been used for face recognition under different illumination conditions during the last decades [7]. The performance of the recognition techniques has achieved significant results under controlled and uncontrolled environments such as under illumination and pose variations, which are challenging situations. Variations in the illumination of the face images occur when the light source is at different angles to the direction of the face [8]. Moreover, face recognition accuracy and recognition time are two major factors that affect the quality of face recognition algorithms [9]. Deep learning has been developed with higher accuracy and complex structure, but common deep learning techniques require large databases, much computation time for training, and high power consumption. Deep learning, local features, and geometric techniques are used in facial feature extraction. The function of facial feature extraction is to identify facial features such as structure, size, and shape of the facial image after face recognition. Features such as nose, mouth and eyes or geometry calculations, represent the face. Feature based descriptor techniques have been successfully used for face recognition due to their robustness, efficiency and simplicity. Scale Invariant Feature Transform (SIFT) [10], Local Binary Patterns (LBP), and Histogram of Oriented Gradients (HOG) [11] in combination with Nearest Neighbors, Support Vector Machine (SVM) technique are used for person recognition [12]. The LBP technique, was proposed in [13] for texture description and has the advantage of being robust and simple. LBP continues to be used for face recognition as it gives very good results compared to other techniques, while it has improved its performance many times over [14]. Moreover, with the increasing use of machine learning techniques, researchers have integrated algorithms such as LBP and ensemble ad boosted strategy to achieve accurate results in face recognition. Moreover, the use of more than one strategy, for example LBP and Gabor wavelet filter, for feature extraction has led to better results, and the integration of different techniques is said to give much better performance. LBP describes smaller details of appearance, while Gabor processes the structure of faces over a wider range of scales [15]. LBP is recognized as a powerful feature extraction method for face recognition and texture classification [16]. LBPH uses a histogram by dividing the cropped face image blocks and counting the histogram for each block by integrating the histogram blocks to introduce the final feature vector. The LBPH process is applied to each face image to represent vectors for each image and finally compare the tested face vector with the total trained vectors to compare two face images [17]. In the field of face recognition, there are Corresponding author: Ahmed B. Salem Salamh Engineering, Technology & Applied Science Research Vol. 12, No. 1, 2022, 8033-8038 8034 www.etasr.com Salamh & Akyuz: A Novel Feature Extraction Descriptor for Face Recognition many face databases with different features, and selecting the databases that are suitable for evaluating the new technique is challenging. In this work, most face recognition problems were considered. To ensure the diversity, 6 databases were selected for the evaluation of the algorithms. The derivation is based on the mathematical theory derived from the equation for local binary patterns to achieve perfect accuracy in face recognition. For this purpose, different descriptors were used to extract features, to calculate the histogram for each descriptor, and to generate a pair of feature vectors considering the proposed face recognition model. II. METHODOLOGY In this part, the combination of multi-descriptor and SVM is presented in detail. In this work, known databases have been used to evaluate the performance of descriptors while ensuring the diversity of data. Each database contains a different number of subjects. Each subject is called a class and is denoted by C. The number of subjects in each database is different. For each class, there is a different number of samples, denoted by S. The face databases contain a variety of features. The databases used in the study were cropped and reformatted into images of 128×128 pixels. III. THE PROPOSED METHOD Appearance based feature extraction works on representing facial details. It stands out for its efficiency, accuracy, and robustness. The initial LBP method simply works by threading central pixels of 3×3 neighborhoods to binary encode the value of the central pixel. The value of the central pixel is denoted by gp and its 8 neighbors by gc [18]. Subtracting (gp-gc) gives a negative or a positive value. The resulting value is encoded to 0 if it is negative and to 1 otherwise. Finally, the binary code is converted to decimal value to represent the feature. The main functionality of the proposed feature extraction based on appearance considers the mean value as a threshold value and compares it with the 8 neighboring values to generate a new value, which represents the new value as a feature. The advantage of multi-descriptor models MD1 and MD2 is that they generate distinguishing features by determining the value that describes the facial image. The proposed technique starts by introducing a new mathematical equation to describe the facial structure in different distributions as shown in Figure 1, by comparing 8 neighbors with the mean value in a gray scale image representing the value from 0 to 255 pixel intensity. The variety of x values is to ensure the distribution of the data, that is, the values of x8 and x0 are variable depending on the application. The application of the multi-descriptor equations (1)-(3) to the original images of the Extended Yale B database is shown in Figure 2. The results are shown in Figures 3-4. MDN � �x�f g� � g � � �∑ ������� f g� � g ��� ∗ 2.55 ��� �� (1) F(x) = !New value if X * 0, 0 otherwise. (2) F(x) = !New value if X 2 0, 0 otherwise. (3) Fig. 1. Multi structures of descriptor. Fig. 2. Some samples per person with illumination variation from the Extended Yale B database. Fig. 3. Samples from the Extended Yale B produced by the MD1 model. Fig. 4. Samples from the Extended Yale B produced by the MD2 model. IV. SUPPORT VECTOR MACHINE Machine learning has become a desirable technique in most areas of recognition. One of these techniques is SVM. SVM is a supervised learning algorithm for handling regression and classification problems, which performs classification by finding a hyperplane that can separate two classes. In this paper, SVM [19] is used to databases which are divided into two categories, training and testing, to perform classification. SVM uses Error-Correcting Output Codes (ECOC), binary classification, and an ensemble method for multi-class classification problems. Two methods used in ECOC are one against one and one against all, which was used in this work. V. FACE RECOGNITION WITH MULTI-DESCRIPTOR Figure 5 shows the overall architecture using a multi- descriptor model for classification consisting of two main parts, namely a training set and a test set. The same model was used Engineering, Technology & Applied Science Research Vol. 12, No. 1, 2022, 8033-8038 8035 www.etasr.com Salamh & Akyuz: A Novel Feature Extraction Descriptor for Face Recognition for face feature extraction to represent the face structure. Multi- Descriptor one (MD1) has different parameters than Multi- Descriptor two (MD2). Based on the flexibility of the derived equation, it was explained how to generate facial features. The MD1 uses a value of x8 equal to 50 and x0 equal to 0 and the MD2 uses a value of x8 equal to 60 and x0 equal to 0. Together, they brought different representations of facial images. The evidence of the differences between MDN and the pair-LBP model is doubted. For this reason, a new LBP architecture for face recognition was introduced, as shown in Figure 6. The classification of both models was conducted with the use of SVM. VI. PERFORMANCE EVALUATION To compute and evaluate the performance of face recognition models using machine learning techniques, the most commonly used metrics [20] are defined in this work as follows: For all matrices, consider P as the number of positive samples, N as the number of negative samples, T as True, and F as False. Accuracy (ACC) is an indicator used frequently to measure classification performance. ACC � 5675856758796798 (4) Sensitivity or True Positive Rate (TPR) reflects the samples positively classified in the total number of positive samples, also known as actual positive samples. TPR � 5656798 (5) Positive Predictive Value (PPV) or precision indicates the proportion of positive samples classified correctly in the total number of positive predicted samples. PPV � 56 96756 (6) F-measure or F1 score reflects the harmonic mean of precision and recall. The value of F1 is ranged from 0 to 1. High values of the F-measure indicate high classification performance. F1 � @56 @56796798 (7) Fig. 5. Face recognition architecture using the multi descriptor model. Engineering, Technology & Applied Science Research Vol. 12, No. 1, 2022, 8033-8038 8036 www.etasr.com Salamh & Akyuz: A Novel Feature Extraction Descriptor for Face Recognition Fig. 6. Face recognition architecture using the local binary pattern model. VII. FACIAL RECOGNITION DATA The Yale face database consists of 15 subjects with 11 images taken under different conditions for each class, giving a total of 165 grayscale images [21]. Extended Yale B has 28 classes, with each class containing 9 poses and 64 illumination conditions for a total of 16128 face images [22]. The University of Essex has introduced 4 folders of data (Faces94, Faces95, Faces96, and Grimace) [23]. The total face database contains 395 faces of people of both sexes with 20 images for each subject. It contains photos of people from different ethnic backgrounds, mainly first-year university students. Most of the subjects are between 18 and 20 years old, while there are also some older subjects. Some persons wear glasses or have beards. TABLE I. SUMMARY OF FACIAL RECOGNITION DATASETS Dataset Number of subjects Number of images Yale 15 165 Extended Yale B 28 16128 Faces94 152 3060 Faces95 72 1440 Faces96 72 3013 Grimace 18 360 VIII. RESULTS AND DISCUSSION Figure 7 presents the performance of the two models. Both show good performance in terms of sensitivity, precision, and F1 score and excellent results on the challenges encountered in each database, especially when the subject moves during the acquisition of the image. Fig. 7. Multi-descriptor and LBP performance comparsion using sensitivity, precision, and F1 score. Figure 8 shows the accuracy of the MD and the LBP models. Both models showed excellent performance, with the proposed MD either coming close or outperforming the LBP. Engineering, Technology & Applied Science Research Vol. 12, No. 1, 2022, 8033-8038 8037 www.etasr.com Salamh & Akyuz: A Novel Feature Extraction Descriptor for Face Recognition Fig. 8. Performance comparison of the MD and the LBP models for face recognition. The results show that the new descriptor, which uses a new feature extraction equation, is more efficient on different poses, illuminations, facial expressions, and glasses, than the LBP model. This means that the new model is suitable to face the challenges of face recognition and other tasks when feature extraction is required for description, verification, and identification. The descriptor was tested and verified on several face images in the first part, which clearly shows how well it can describe face details. This is very important and shows the ability to represent all the features of a face. The new descriptor may contribute to the recognition of other tasks, such as covid19, retinal vessels, and texture classification, which we plan to study in the future. IX. CONCLUSION This paper presented a new feature extraction model called multi-descriptor (MD) model. A key finding was the use of the derived equations to describe facial features with pair technique and different distribution. The technique was developed for face feature extraction, but is certainly suitable for most other types of texture description with more descriptors. The classification for face recognition was performed using SVM. The proposed descriptor outperformed the LBP model in Extended Yale B, Face94 and Grimace databases, having accuracy of 99.437%, 98.814%, and 100% respectively. This shows that the proposed descriptor is completely different from the conventional LBP and improves the recognition accuracy of faces and handles perfectly the challenge of poses, illuminations, expressions, face position, and camera distance. Combinations of MDN and LBP models with different structures and distributions of data could be used in future work. AUTHORS’NOTE The authors announce that they have complete access to all the data. There are no financial disclosures from the authors. REFERENCES [1] D. Virmani, P. Girdhar, P. Jain, and P. Bamdev, "FDREnet: Face Detection and Recognition Pipeline," Engineering, Technology & Applied Science Research, vol. 9, no. 2, pp. 3933–3938, Apr. 2019, https://doi.org/10.48084/etasr.2492. [2] Y. Said, M. Barr, and H. E. Ahmed, "Design of a Face Recognition System based on Convolutional Neural Network (CNN)," Engineering, Technology & Applied Science Research, vol. 10, no. 3, pp. 5608–5612, Jun. 2020, https://doi.org/10.48084/etasr.3490. [3] L. Zhou, H. Wang, W. Liu, and Z.-M. Lu, "Face feature extraction and recognition via local binary pattern and two-dimensional locality preserving projection," Multimedia Tools and Applications, vol. 78, no. 11, pp. 14971–14987, Jun. 2019, https://doi.org/10.1007/s11042-018- 6868-6. [4] A. Vinay, N. S. Hegde, S. K. Tejas, N. V. Patil, S. Natarajan, and K. N. Balasubramanya Murthy, "Learned Invariant Feature Transform and Extreme Learning Machines for Face Recognition," in Soft Computing for Problem Solving, Singapore, 2019, pp. 289–298, https://doi.org/ 10.1007/978-981-13-1595-4_23. [5] A. J. Shepley, "Face Recognition in Unconstrained Conditions: A Systematic Review," arXiv:1908.04404 [cs], Jul. 2019, Accessed: Dec. 11, 2021. [Online]. Available: http://arxiv.org/abs/1908.04404. [6] O. Abikoye, I. Shoyemi, and T. Aro, "Comparative Analysis of Illumination Normalizations on Principal Component Analysis Based Feature Extraction for Face Recognition," FUOYE Journal of Engineering and Technology, vol. 4, no. 1, pp. 67–69, Mar. 2019, https://doi.org/10.46792/fuoyejet.v4i1.309. [7] V. P. Vishwakarma and S. Dalal, "A novel non-linear modifier for adaptive illumination normalization for robust face recognition," Multimedia Tools and Applications, vol. 79, no. 17, pp. 11503–11529, May 2020, https://doi.org/10.1007/s11042-019-08537-6. [8] V. P. Vishwakarma and T. Goel, "An efficient hybrid DWT-fuzzy filter in DCT domain based illumination normalization for face recognition," Multimedia Tools and Applications, vol. 78, no. 11, pp. 15213–15233, Jun. 2019, https://doi.org/10.1007/s11042-018-6837-0. [9] R. Senthilkumar and R. K. Gnanamurthy, "A robust wavelet based decomposition of facial images to improve recognition accuracy in standard appearance based statistical face recognition methods," Cluster Computing, vol. 22, no. Suppl. 5, pp. 1–10, Sep. 2019, https://doi.org/ 10.1007/s10586-018-1759-1. [10] T. Lindeberg, "Scale Invariant Feature Transform," Scholarpedia, vol. 7, no. 5, May 2012, Art. no. 10491, https://doi.org/10.4249/scholarpedia. 10491. [11] N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, Jun. 2005, vol. 1, pp. 886–893, https://doi.org/10.1109/CVPR.2005.177. [12] P. S. Prasad, R. Pathak, V. K. Gunjan, and H. V. Ramana Rao, "Deep Learning Based Representation for Face Recognition," in ICCCE 2019, Singapore, 2020, pp. 419–424, https://doi.org/10.1007/978-981-13- 8715-9_50. [13] T. Ojala, M. Pietikäinen, and D. Harwood, "A comparative study of texture measures with classification based on featured distributions," Pattern Recognition, vol. 29, no. 1, pp. 51–59, Jan. 1996, https://doi.org/ 10.1016/0031-3203(95)00067-4. [14] S. Karanwal and M. Diwakar, "Neighborhood and center difference- based-LBP for face recognition," Pattern Analysis and Applications, vol. 24, no. 2, pp. 741–761, May 2021, https://doi.org/10.1007/s10044-020- 00948-8. [15] S. A. A. M. Faudzi and N. Yahya, "Evaluation of LBP-based face recognition techniques," in 2014 5th International Conference on Intelligent and Advanced Systems (ICIAS), Kuala Lumpur, Malaysia, Jun. 2014, https://doi.org/10.1109/ICIAS.2014.6869522. [16] S. M. Hassan, A. Alghamdi, A. Hafeez, M. Hamdi, I. Hussain, and M. Alrizq, "An Effective Combination of Textures and Wavelet Features for Facial Expression Recognition," Engineering, Technology & Applied Science Research, vol. 11, no. 3, pp. 7172–7176, Jun. 2021, https://doi.org/10.48084/etasr.4080. [17] R. J. Niaraki and A. Shahbahrami, "Accuracy Improvement of Face Recognition System based on Co-occurrence Matrix of Local Median Binary Pattern," in 2019 4th International Conference on Pattern Recognition and Image Analysis (IPRIA), Tehran, Iran, Mar. 2019, pp. 141–144, https://doi.org/10.1109/PRIA.2019.8785969. [18] D. Huang, C. Shan, M. Ardabilian, Y. Wang, and L. Chen, "Local Binary Patterns and Its Application to Facial Image Analysis: A Engineering, Technology & Applied Science Research Vol. 12, No. 1, 2022, 8033-8038 8038 www.etasr.com Salamh & Akyuz: A Novel Feature Extraction Descriptor for Face Recognition Survey," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 41, no. 6, pp. 765–781, Nov. 2011, https://doi.org/10.1109/TSMCC.2011.2118750. [19] W. H. Land and J. D. Schaffer, "The Support Vector Machine," in The Art and Science of Machine Intelligence, Berlin Heidelberg, Germany: Springer International Publishing, 2020, pp. 45–76. [20] A. Tharwat, "Classification assessment methods," Applied Computing and Informatics, vol. 17, no. 1, pp. 168–192, Jan. 2020, https://doi.org/ 10.1016/j.aci.2018.08.003. [21] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, "Eigenfaces vs. Fisherfaces: recognition using class specific linear projection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711–720, Jul. 1997, https://doi.org/10.1109/34.598228. [22] K.-C. Lee, J. Ho, and D. J. Kriegman, "Acquiring linear subspaces for face recognition under variable lighting," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 684–698, May 2005, https://doi.org/10.1109/TPAMI.2005.92. [23] "Libor Spacek’s Facial Images Databases." https://cmp.felk.cvut.cz/ ~spacelib/faces/ (accessed Dec. 11, 2021).