144 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 A Survey of Face Recognition Systems 2hanJane Jaleel Step 1Ihab Amer Abdullah Informatics Institute for Postgraduate University of Information Technology Studies & Communications Baghdad - Iraq Baghdad - Iraq ihab9321@gmail.com janejaleel@uoitc.edu.iq Abstract With the quick grow of multimedia contents, from among this content, face recognition has got a lot of significant, specifically in latest little years. The face as object formed of vari- ous recognition characteristics for detect; so, it is still the most challenge research domain for researchers in area of image processing and computer vision. In this survey article, tried to solve the most demanding facial features like illuminations, aging, pose variation, partial oc- clusion and facial expression. Therefore, it indispensable factors in the system of facial recognition when performed on facial pictures. This paper study the most advanced facial de- tection techniques too, approaches: Hidden Markov Models, Principal Component Analysis (PCA), Elastic Cluster Plot Matching, Support Vector Machines (SVM), Gabor Waves, Arti- ficial Neural Networks (ANN), Eigen Face, Independent Component Analysis (ICA) and 3D Morphable Model. Additionally to the above works, mentioned various testing facial data- bases including JAFEE, FEI, Yale, LFW, AT&T(formerly termed as ORL) and AR (Aleix Martinez and Robert Benavente) etc to analyze the results. Even so, the goal of this survey is to present a comprehensive literature review for the face recognition besides its applications after a deepness discussion, some of the experimental results was introduced in the end. Keywords Face Recognition, Aging, Illuminations, Pose Variation, Partial Occlusion 1.Introduction The twenty one century is considered a recently and scientific era in which great strides have been made acceleration people in the finishing of their missions. In supporting to the above statement, at the present time the using of computer technology has being an essential part that cannot parted from the life. The computers are utilizing in pyramids of the applica- tions that ranging from the simple to the complicated problem-solving ways. From between these contributions, the face recognition technological has appeared as a beneficial tool for recognizing features of the faces based on their inherent features. And it was one from the most researched fields in domain of computer vision and pattern recognition. Even so, be- cause of it’s widen use in a many applications for example biometrics, surveillance system, police access control, information security and smart cards. But there is several challenges to Ibn Al Haitham Journal for Pure and Applied Science Journal homepage: http://jih.uobaghdad.edu.iq/index.php/j/index Doi: 10.30526/34.2.2620 Article history: Received 17, November, 2020, Accepted 16,February,2021, Published in April 2021 mailto:ihab9321@gmail.com file:///C:/Users/ihab/Desktop/janejaleel@uoitc.edu.iq 145 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 the researcher that need be solved. Such as partial occlusion, aging, pose variance, illumina- tions, and facial expression. Face recognition issue can be classified into two major stages: 1. face verification and 2. face identification. In first stage, specifies a face in a picture. and in second stage, features are extracted from the facial in the picture in order to distinguish. Then they are compared with the pictures in the face database for recognizing the correct face pic- ture. Even so, some present recognition ways for the authentication suffer from a shortage of reliability. For example, wallets, smart cards, tokens and keys use passwords and PINs that are so hard to recall. In addition, these codes and passwords can be easily forget; those mag- netic cards also can be lost, stolen or reproduced. As a consequence, this make them illegible. Unlike the biological traits and characteristics of a person, they not possible to be misplaced, loss, or forgotten [1]. The system of biometric recognition can be constructed by different techniques. Even so, the most common ways used are the iris and fingertips. These require the participation of the person in order to accessing the system. Furthermore, the newest systems provide participants with access without their intervention. Among these ways, face recogni- tion are one of most practicable technologies that can easily detect and surveillance a person over the system. The face recognition databases ranging from controllable pictures to non- controllable videos, like using (YouTube Faces) YTF [2] for videos and (Labeled Faces in the Wild) LFW [3] for pictures. 2. Databases Many databases with faces public nor private for research purposes are available. These databases are differentiated from one another according to various criteria. The most im- portant are on the following: - The number of photos contained in every database is the most crucial standard. - The number of photos for every unique class: knowing that each person is characterized by a class c, the number of pictures in a class stand for the number of representative pictures of the person. In fact, the pictures are taken under various conditions (aging, facial expression ...etc.). - The existence of occlusion (beards, glasses, etc.). - Gender of the obtained persons. - The volume of the pictures. - The existence of static pictures or videos. - The time among recordings. - Face alignments and pose. - The changing in lighting. - The existence of a unified background. So, recommended to choice the suitable database while the testing of the algorithm. In fact, some having well defined protocol permitting directly comparison of results. Furthermore, the choose should depends on the issue to be tested: aging, facial expressions, illumination, etcet- era. The availability of several different pictures for every person can be critical argument to the correct performance to the algorithm. Table 1. show the major 2D faces databases. Those databases progress several differentiations in terms of : gray or RGB picture, number of pic- tures by person, number of people, differentiations of the picture [occlusions (o), Aging (a), pose (p), facial expression (e) and illumination (i)]. 146 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 Table 1. The Main 2D Face Databases (The picture differences are represent through (e) facial expression, (p) pose, (a) aging, (i) illumination and (o) occlusion) 3. Structure of Face Recognition System Three main phases are utilized for develop a strong face recognition system: 1) Face Detec- tion 2) Feature Extraction 3) Face Recognition [12,13]. The face detection phase is utilized for detecting and locating the person face picture got by the system. And the feature extrac- tion phase is used for extract feature vectors to whatever person face detected in first phase. lastly, the face recognition phase comprises features extracted from a person face so in order to compared against all model face databases to determine identity of the person face. The structure of face recognition are illustrated in the Figure 1. Figure 1. Face Recognition Structure [14] 3.1. Face Detection A face recognition system first starts with localizing human faces in a specific picture. As il- lustrative in the Figure 2. The purpose from this phase is to determination whether the input picture contains person faces or not. Changes in facial expressions, illumination and ...etc. can block correct facial recognition. To simplify the development of face recognition system and makes it further strong, preprocessing steps are executed. Numerous techniques are used for detect & locate the picture of a person face, for instance the Histogram Oriented Gradient (HOG) [15,16], Viola-Jones Detector [17] and Principal Component Analysis (PCA) [18]. as well , the face detection phase can be used to object detection [19], region of interest detection [20], picture and video classification and etc. Database RGB Color/grey Number of Pictures/Persons No. of Persons Pictures Size Variation Yale [4] Gray 165 15 (14 men and 1 woman) 320 × 243 i,e AT&T [5] Gray 10 ( pictures per person) 40 92 × 112 i,a XM2VTS [6] RGB 2360 295 576 × 720 p AR database [7] RGB 4,000 126 (56 women and 70 men) 576 × 768 o,e,i,a CVL [8] RGB 7 (pictures per person) 114 (6 women and 108 men) 640 × 480 e,p Oulu Physics [9] RGB 16 for every per- son (and addition- al 16 if have glasses) 125 428x569 i JAFEE [10] Gray 213 10 256 × 256 e FEI data- base[11] RGB 2800 200 640x480 p,e,i LFW [3] RGB 13,233 5749 250 x 250 P,i,e,o,a 147 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 Figure 2. Example of the Face Detection [21] 3.2. Extraction of Features The major function of this phase is for extract features of facial picture detected at the detec- tion phase. As illustrative in the Figure 3. This phase represent a facial with group of vector features named "signature", which describes the eminent features of the picture of the face, like: eyes, mouth and nose with their geometric distribution [22,14]. Every face is character- ized via its own size, shape and structure, that allowing to identify it. many techniques include extracting shape of the nose, mouth or eyes for identifying the face by using distance and size [12]. for instance: Local Binary Pattern (LBP) [12], Fourier Transforms [22], Local Phase Quantization (LPQ) [23], Scale-Invariant Feature Transform (SIFT) [13], Linear Discriminant Analysis (LDA) [24], Eigenface [25], and Histogram Oriented Gradient (HOG) [15] are wide used for extract facial features. Figure 3. Face Component 3.3. Face Recognition This phase considers the extracted features from background while the feature extraction phase and compares them to known faces saved in a specified database. Generally there are two steps to the face recognition, the first termed identification and the second is verification. At identification step, the test facial are compared with the faces set, for find the most similar- ly match. At verification step, the test face are compare with a known facial in the database for make a decision on rejection or acceptance [26]. k-nearest neighbor (K-NN) [27], convo- lutional neural network (CNN) [28] and Correlation filters (CF) [29] are known for treating effectively this task. The face recognition phase illustrated in the Figure 4. 148 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 Figure 4. Illustrate the Face Recognition [30] 4. Models of Face Recognition There are three various methods that are utilized in face recognition applications: 2D, 3D and video. Will review the face recognition issues in these fields in the next departments. 4.1. Face Recognition in 2D Face recognition have been good studied with 2D still pictures for more than decade [31]. In facial recognition systems based 2D fixed picture, a capture of a user is captured and com- pared to a database of captures to identify a person. In this method, the user is predicted to cooperating and present a front facial picture with a simple background under uniform light- ing conditions to enable high quality capture and segmentation of facial picture. Even so, it is now known that little deffrences in illumination and pose can dramatically decrease the per- formance of facial recognition systems based on a 2D single-shot image [32]. 2D face recog- nition is generally classified according into the number of pictures used for identical, as ex- plained in Table 2. Table 2. Face Recognition Scenarios in 2D Domain Gallery Probe Single still picture Many still pictures Single still picture Many still pictures one-to-one many-to-one one-to-many many-to-many And some of the known algorithms for 2D face recognition are depend on Linear Discrimi- nant Analysis (LDA) [33], Correlation-Based Matching [34], the Elastic Graph Model (EGBM) [35] and Principle Component Analysis (PCA) [36]. 4.2. Face Recognition in 3D 3D face recognition ways use the geometry of the facial surface [37]. as opposed to 2D face recognition, 3D face recognition is strong against differences in illumination and posture due to the immutability of the 3D shape in relation to these differences. The 3D facial picture cap- tured by the 3D sensor spans approximately 120 ± from right to left. This picture is called 2.5D. A full 3D model covering 360 ± face faces is created by combining multiple (3 to 5) 2.5D scans. Usually the probe is a 2.5D picture, and the database can be a 2.5D picture or a 3D picture. Identification can executed among two range (depth) pictures [37] or among a 2D picture and a 3D face model [38]. Table 3. expands Table 2. to include 2D and 3D face mod- 149 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 els. Table 3. Face Recognition Scenarios across 2D and 3D Domain Gallery Probe 2D Pictures 3D Models or 2.5D Pictures 2D Pictures 2.5 Pictures 2D to 2D 3D to 2D 2D to 3D 3D to 3D There are also many approaches are based on 3D models that has been reconstructed from a set of 2D pictures [39]. The reconstructed 3D model is used to get several 2D projection pic- tures corresponding to the probe pictures [38]. Alternatively, the reconstructed 3D model can be used to create a front view of the probe picture in any pose and lighting conditions. Recog- nition done by identical the synthesized probe in ahead pose. In Figure 5. shows a 3D model of a face and its correspondent 2D projection pictures in various positions and lighting condi- tions [21]. Figure 5. A 3D face Model and its 2D Projections [21] 4.3. Face Recognition in Video Although traditional face recognition systems are depend on still pictures, there is considera- ble interesting in developing strong face recognition systems that take video as input. Face recognition video has generated interest because of wide prevalence use of monitoring camer- as. The capability to automatically identify faces in the real time from the video will facilitate it, between other things, a secret way of identifying people using an existent network of moni- toring cameras. However, facial pictures in videos are frequently in off-frontal poses and can be subject to significant changes in lighting, which lead decay the performance of majority commercial face recognition systems. Two characteristic features of video are the providea- blity of: i) multi frames for the same topic and ii) time information. Multi frames granteed pose variation that allow the suitable choose of a well quality frame (for instance, an excellent quality face picture in semi front pose) for excellent recognition performance. The temporal info in the video is considered information included in the dynamic movement of the face in the video. However, it is hard to establish if there is any information regarded to identity in the movement of the face - Further work is require to gain benefit the temporal information. Using video capabilities can improve the performance of the face recognition system. Figure 6. shows 4 frames in an ideal video that was recorded for face recognition research [21]. 150 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 Figure 6. Four Frames from a Video [21] 5. Challenges and Difficulties In spite of the evolution of numerous facial recognition algorithms, there are still many diffi- culties. These challenges are still challenging these algorithms, as described below [40]: 5.1. Several databases of facial pictures, such as references. Have been grouped to test fac- es recognition algorithms. Every database is designed for test determined aspects like occlu- sion, pose, facial expression and illumination. Prior research work has shown that face recog- nition has matured under controlled conditions. Even so, facial recognition in an outdoor en- vironment becomes challenging. 5.2. Face recognition is special and difficult state from object recognition. The difficulty of facial recognition stems from the reality that in the front view, the facial pictures look the same, and the differences between it are significant to be analyzed. A research conducted the latest face recognition technologies, on the standard database (like FAT, FERET, and FRVT) have identified detecting illumination, age and pose as the main issues of the facial recogni- tion algorithm. 5.3. The algorithms of face recognition are selected as desired by the application. For in- stance, feature-based methods not possible be applied to low-resolution (for instance, 15×15 pixels and below) facial pictures. Other issue is: where to use (LDA) and when to use (PCA/ICA) when developing the system. 5.4. Implementation details usually specify the performance of the system. For instance, several times input pictures are normalized with regard to in plane rotation, masking, affine warping and scale for align the shape. 5.5. Low resolution is an significant factor in face recognition when taking pictures from a far distance. In addition, closing the eyes too affects the recognition exactness of most fa- cial recognition systems because they normalize & rescale the pictures before recognition. 5.6. When the face occluded, the recognition rate drops rapidly. Likewise, structural com- ponents like mustaches, glasses and beards too significantly impact the recognition rate. 5.7. For good recognition performance, accurate feature location is critical. As the face rotates to a specific angle, the facial features will change, and several face recognition algo- rithms have difficulty processing them. Age, illumination and pose are the main issues that current facial recognition algorithms plaguing. 6. Factors Affecting of Face Recognition Identifying human faces from pictures and videos is a very difficult problem. There are many ways to solve this problem, but none can perform it with 100% accuracy because of the many problems that this system faces. These factors fall into 2 categories: internal and external fac- tors [41]. Internal factors include, for example, the physically condition of the person's face are facial expressions, aging, etc. that affect the system, while external factors are factors that cause changes in the appearance of the face, for instance illumination state causing variation. 6.1. Aging Aging is one of the substantial factors that affect face recognition techniques where it convert into clutter for algorithms. Stability is a basic quality to any biologic measurement to be deal 151 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 with it as a biometrics. The face is mixture of skin tissue, bones and face muscles. When mus- cles contracting, they cause deformation of the facial features. Even so, aging causes very much changes in the face appearance of a person such as illustrative in the Figure 7. e.g. face texture (wrinkles, ...etc.) and the face shape over time [42]. Face recognition systems must adequately meet this requirement. Several researches whose primary goal is to solve this problem have been conducted [43]. Due to slow aging process, it is becoming hard to gather data to training the system for cope with the aging factor to recognition purposes [44]. So, age-related researches is gained great popularity. Figure 7. Pictures of the Same Subject at Age (a) 5, (b) 10, (c) 16, (d) 19 and (e) 29 [21] 6.2. Facial Expression Facial expression is a non-verbal form of communication as it transports messages by use the expressions. Even so, dissimilarity in facial expression create ambiguity for the face recogni- tion systems. Several face recognition systems are evolved that work good with pictures in a controllable environment. Various face expressions show various situations and moods of people and change facial geometry such as illustrative in the Figure 8. and with small differ- ent in the picture, it is becomes hard for the system to identify the face. Researchers had worked on face recognition with take a consideration of face expressions [45]. There are vari- ous approaches that can used for handle with this problem, such as muscle-base approaches, motion-based approaches, model-base approaches [46]. Figure 8. Examples of some Facial Expressions from a Daily Life 6.3. Pose Variation Another obstacle to the success of a facial recognition system is the variance of pose. People pose are different every time they capture a picture, such as illustrative in the Figure 9. There is no similar standard pose. Therefore, this makes it hard to recognize and the distinguish fac- es in pictures with different poses. Most systems operate under not flexible picturing condi- tions. In depend on the type of pictures used in the gallery, ways associated with changing the pose can be partitioned into two types: facial recognition by pose and multi view face recog- nition. Multi view face recognition can be thought of as a front-face recognition annexure in which consider gallery picture of each pose. On other hand, through the pose of facial recog- nition, consider the face in a situation that the recognition system has not seen before. A good faces recognition approach must have good pose toleration and ability to recognize various poses. Many problems in this regard remain open. In which several researchers are working to address this problem [47]. Even so, there is not yet a system with 100% accuracy. There are 152 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 some various approaches and methods that can be used for solve the issue of variable face recognition and pose change, which are partitioned into three categories, involving 2D ap- proaches, general algorithms and 3D models [48]. Figure 9. Examples of some Pose Variations from FERET Database 6.4. Partial Occlusion Occlusion indicates to artificial or natural obstacles in the picture. The face recognition ap- proaches with partial occlusion classified into various classes, involving Fractal-Based Meth- ods, Feature-based methods and Part-Based Methods [49]. Partial occlusion has affected many picturing areas, like ear identification is occluded because earrings. Occlusion have an affect on the system performance when people mislead it by using scarves, sunglasses, veil, or putting mobile hands or phones in front of their faces such as illustrative in the Figure 10 and Figure 11. Sometimes another factors, such as shadows because of strong illumination, too act as occluded factors. To solve the issue of partially occluded faces. Analytical approaches are used which split the faces into various parts [50]. This issue can be solved by removing some of features that make it difficult to exactness recognize the face in the picture. Frequent- ly, analytical approaches are depend on feature analysis, where best enabled features are dis- covered, and then they are merged. Other approach that can be taken to this purpose is an al- most holistic approach in which the occluded features are removed and the remaining of the face as valuable information is used. Researchers are developing various techniques to deal with this issue [51,52]. Figure 10. Example of the Partial Occlusion [21] Figure 11. Faces with Occlusion Effect as Presented in Hand2Face [53] 6.5. Effect of Illumination Variation in Illumination strongly affect the facial recognition system such as illustrative in 153 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 the Figure 12. and therefore has become the focus of several researchers. It become hard to recognize one or further person from still pictures or video. It is very easy for extract the de- sired information from pictures taken under controlled environment in which the background is uniform. Even so, the face in uncontrolled environment need to be recognize from different backgrounds. It include variation because shadows, under exposure and over exposure. Re- searchers have worked hard to solve this problem. There are three ways to treat with this problem: Face reflection field estimation technique, Gray level, and Gradient. The gray level conversion technique executes in-depth mapping with linear or non-linear function. Extrac- tion approaches of the gradient are used for extract the edges of picture in gray level. Since illumination is a factor that considerably affecting of the faces recognition from videos or pic- tures, the techniques have been development to ignore the impact resulted by this problem [54]. Figure 12. Example of some Illumination Variation from Yale Database b [55] 7. Advantages of Face Recognition 1. High Success Rate Today's facial biometric technology has a high success rate, particularly with the appear of 3D facial recognition technology. It is very difficult to cheat the system, so you can rest assured that system you have will successfully track attendance and time with offering better security. 2. Easy Integration Also easy to integrate your biometric facial systems into your company. Usually will work with the exist software you having. 3. Automatic System Many companies like the fact that biometric face recognition systems are automated. You not need to worry to having someone there for surveillance the system. 4. Better Security With a facial biometrics system you’ll sweet better security. Not just can you trace em- ployees, but can be added into the system any visitant and traced throughout the area also. will not be given access to anyone that is not in the system. 5. Time Fraud not Exist One of the great advantages of using face biometrics systems to your association is that don't need to worry for time fraud. It will not be possible to proxy between friends, as eve- ryone needs a face scan to register its attendance. [56] 8. Result In this section, present a detailed comparison of various experimental results, in the Table 4. that showed Difference in Databases, Techniques, Years, Authors, and Results that got it. 154 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 Table 4. The Experimental Results No. Author/s Year Technique Database Result 1 Vankayalapati et al. [57] 2009 CNN ORL 95% 2 K ong, Rui et al. [58] 2011 ICA, SVM ORL 96% 3 Bellakhdhar et al. [59] 2013 Magnitude Phase of Gabor, PCA, SVM ORL, FRGCv2 99.90% 4 Jameel, S. [60] 2015 PCA + DCT ORL 95.122% 5 Fathima et al. [61] 2015 Gabor wavelet and linear discriminant analysis (LDA) AT&T, MIT-India and Faces94 da- tasets 88–94.02% 6 Ghorbel et al. [62] 2016 Eigenfaces and DoG filter FERET 84.26% 7 Bhaskar, A. et al [63] 2016 SVM Yale Faces 97.78% 8 Fu et al. [64] 2017 Guided convolutional neural network, the loss function CASIAWebface, LFW 91.9–97.1% 9 Khan et al. [65] 2018 PCA NCR-IIT facial database and real- time video stream 69–86% 10 Banerjee et al. [66] 2018 Supervised learning, Viola–Jones, generic 3D model PaSC videos and CW images, CMU multi-PIE dataset 88.45–97.28% 10. Applications of Face Recognition As know efforts are continuously being made for develop the face recognition system to max- imum accuracy, this is due to its relevance in several domains. It uses some of it's applications in: 10.1. Law Enforce The face recognition systems proven that it is a very efficacious tool for the law enforce bod- ies for finding to missing persons or identify criminals. The investigation manually for hours in the video for person identification is a rather tedious mission for law enforce administra- tors. For instance, It was newly reported [67] that it took Chinese law enforce administrators only 7 minutes for determine BBC newsman "John Sudworth" utilizing its strong CCTV cam- eras network of further more over 170 million camera and the face recognition technique. Fa- cial recognition research fetch a novel generation of smart and efficient investigation abilities to law enforce bodies [68]. Overstaying and unlawful citizenship is another difficult issue for populous communities around world. Advances in facial recognition technology have made it possible to identify and capture the illegal immigrants and foreigners who have exceeded their length of stay. In addition, facial recognition technology is too used for immigration. [69], counter terrorism [70], crime investigation [71, 72], voter identification [73] and banking [74]. 10.2. Entertainment The face recognition has been noticed that is growing popularity in the entertainment field also. The most exciting domains are theme park gaming zones, training, human computer in- teraction, human robot interaction, mobile games and virtual reality [75] etc. Moreover, T. Feltwell et al. presented recently an interest game in which ask the gamers to snap the similar- ities of the audience. It is from motivated of free for play models like the celebrated Pokemon GO game, yet they suggested a variant experience were players can practically hunt and cap- ture the members in realty world [76]. 155 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 10.3. Surveillance Surveillance System is one from the most significant and challenge missions for fully automatic and smart monitoring systems, the surveillance are defined as monitoring or close observa- tion, especially for a criminal spy or suspect. This is one of the mostly significant and widely popularity applications. Those systems are generated to meet the security goals of both the outdoor and indoor general audience like: geographic boundaries, banks, airport halls, moni- toring public regions etc. Because of the huge data size have been obtained by camera net- works, so detection algorithms of brute force spy are non adequate for smartly recognize ter- rorists and suspects in sensitive regions. Latest research in facial recognition provide a plat- form for smart video monitoring systems including software and hardware aspects, like ma- chine learning algorithms, signal processing, pattern recognition and automatic interfaces for achieve extremely optimistic results [77]. In addition, the ability of machine techniques for recognition have been shown to be further efficiency than humans in applications of the real world. For make the beneficial of the abilities of machines and humans, the solutions of face recognition in monitoring systems can be efficaciously utilized as tool for supporting human employees in performing recognition tasks and complex monitoring. In spite of, the latest ways for the monitoring systems has been achieved acceptable results, there are remain much challenges for the effective surveillance like: limited training data, Occlusion and blur subject pictures [78,71]. 10.4. Access Controlling With growing the vogue of the face recognition systems, they had been adopted by many auto access controlling mechanisms to the human-machine interact. As too been replaced with an- other authentication controlling ways like iris verification, password protection, fingerprints, etc. Moreover, CCTV cameras and smart phones widespread, and the use of face authentica- tion has become possible for several real world applications. The hardware based validation systems are quick expanding for control the facial based authorization for the single login into the network multi services. Thus, facial based automatic access for automatic teller machine, access to the cipher text and online money transfers are also becoming popular in various so- cial practices [79]. 10.5. Face Authentication in Mobile The Authentication of the face is become further popularity on the mobiles, permitting mobile applications for validate user's identity in order to give access sensitive mobile phone services like e-banking. Various aspects of face recognition, involving deployment constraints and presentation attacks, must investigated, especially in mobile environments [80,81]. 10.6. Another Common Applications Kwon and Lee [82] recently, presented a inclusive group of face recognition technologies in the software applications. Likewise, Salici and Ciampini [83] introduce facial recognition ap- plications for the forensic investigation section. Identification experiences in 130 real-life sta- tus are proven were successful and confirmed through forensic experts. Other modern applica- tion is presented by Calo et al. [84] to control privacy in the view, update and destruction of digital information. 10.7 Face Recognition in Smart Cities According for (2018) Market Researching Report [85], the biometrics market are expected to growth from $ 13.89 billion in (2018) to $ 41.80 billion in (2023), where face recognition have significant growth potential through that interval. The facial recognition market will have tremendous potential in emergent application domains regarded to the identification of persons in the smart cities, including smart homes, smart education and notably electronic administration, from between many others [85]. Along with technical challenges, the utilize 156 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 of publicly available facial recognition systems may create ethical dilemmas and challenges associated with question of how the snapped data, as especial class of privacy data, perhaps utilized and manipulated. Those concerns are increasingly recognized via data protection reg- ulations like the (EU General Data Protection Regulation (GDPR)) [86], which aims to pro- tect people from third parties processing their information without their approvement. Thus, there is an urgent need for protect biometric data through the adoption of new laws regulating the use of facial recognition techniques [87]. 11. Conclusion For many years, the research of face recognition stay a striving field for researchers. It has attracted widespread attention because of its many applications in several practical world do- mains, such as computer graphics, pattern recognition, security and computer vision. In this survey, we first time presented face recognition as biometrical technology. Next, we intro- duced the face recognition models categorized to three part; we introduced a faces databases use by the researchers in this domain for test their facial recognition approaches and stages, and lastly a table (Table 4.) that summarize the experimental results. This paper has intro- duced a comprehensive survey over the eminent researches of face recognition in different backgrounds. The goal of those literature review for help youth researchers in the domain of facial recognition via compiling the important methods, techniques and models, and to stimu- late there to further research. References 1. Muhammad, S.; et al. Face Recognition: A Survey. Journal of Engineering Science and Technology Review, 2017, 10, 2, 166- 177. 2. YouTube Faces With facial Key points Database. Available online: https://www.kaggle.com/selfishgene/youtube-faces-with-facial-keypoints (Accessed on Nov.8, 2020). 3. The LFW Face Database. Available online: http://vis-www.cs.umass.edu/lfw/ (Accessed on Nov.8, 2020). 4. The Yale Database. Available online: http://vision.ucsd.edu/content/yale-face-database (Accessed on Nov.8, 2020). 5. The AT & T Database of Faces. Available online: https://www.kaggle.com/kasikrit/att-database-of-faces (Accessed on Nov.8, 2020). 6. The XM2VTS Database. Available online: http://www.ee.surrey.ac.uk/CVSSP/xm2vtsdb/ (Accessed on Nov.8, 2020). 7. AR Faces Databases. Available online: http://www2.ece.ohio-state.edu/~aleix/ARdatabase.html (Accessed on Nov.8, 2020). 8. The CVL Database. Available online: https://zenodo.org/record/1492287#.X6jFylBRXIV (Accessed on Nov.8, 2020). 9. The Oulu Physics Database. Available online: http://www.ee.oulu.fi/research/imag/color/pbfd.html (Accessed on Nov.8, 2020). 10. The JAFEE Face Database. Available online: https://www.kaggle.com/srv133034/jafee-dataset (Accessed on Nov.8, 2020). 11. The FEI Face Database. Available online: https://fei.edu.br/~cet/facedatabase.html (Accessed on Nov.8, 2020). 12. Napoléon, T.; Alfalou, A. Pose invariant face recognition: 3D model from single photo. Opt. Lasers Eng, 2017, 89, 150–161. 13. Vinay, A.; Hebbar, D.; Shekhar, V.S.; Murthy, K.B.; Natarajan, S. Two novel detector- descriptor based approaches for face recognition using sift and surf. Procedia Comput. Sci, 2015, 70, 185–197. 14. Kortli, Y.; Jridi, M.; Al Falou, A.; Atri, M. A. Al Falou, and M. Atri, “Face Recognition Systems: A Survey. In Sensors, 2020, 20, 2. 15. Ouerhani, Y.; Alfalou, A.; Brosseau, C. Road mark recognition using HOG-SVM and cor- https://www.kaggle.com/selfishgene/youtube-faces-with-facial-keypoints http://vis-www.cs.umass.edu/lfw/ http://vision.ucsd.edu/content/yale-face-database https://www.kaggle.com/kasikrit/att-database-of-faces http://www.ee.surrey.ac.uk/CVSSP/xm2vtsdb/ http://www2.ece.ohio-state.edu/~aleix/ARdatabase.html https://zenodo.org/record/1492287#.X6jFylBRXIV http://www.ee.oulu.fi/research/imag/color/pbfd.html https://www.kaggle.com/srv133034/jafee-dataset https://fei.edu.br/~cet/facedatabase.html 157 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 relation. In Optics and Photonics for Information Processing XI; International Society for Optics and Photonics; SPIE: Bellingham, WA, USA, 2017, 10395, 103950Q. 16. Jens, R.; Andrew, B.; Diana, G. HW/SW Co-Design of the HOG algorithm on a Xilinx Zynq SoC. J. Parallel Distrib. Comput, 2017, 109, 50–62. 17. Manal, E. R.; Arsalane, Z.; Aicha, M. Comparative Study of Harris and Active Contour using Viola-Jones Algorithm for Facial Landmarks Detection. Transactions on Machine Learning and Artificial Intelligence, 2017,5. 4. 18. Shah, J.H.; Sharif, M.; Raza, M.; Azeem, A. A Survey: Linear and Nonlinear PCA Based Face Recognition Techniques. Int. Arab J. Inf. Technol, 2013, 10, 536–545. 19. Du, G.; Su, F.; Cai, A. Face recognition using SURF features. In MIPPR 2009: Pattern Recognition and Computer Vision. International Society for Optics and Photonics; SPIE: Bellingham, WA, USA, 2009, 7496, 749628. 20. Calonder, M.; Lepetit, V.; Ozuysal, M.; Trzcinski, T.; Strecha, C.; Fua, P. BRIEF: Com- puting a local binary descriptor very fast. IEEE Trans. Pattern Anal. Mach. Intell, 2011, 34, 1281–1298. 21. Unsang, P. Face Recognition: face in video, age invariance, and facial marks. Citeseer, 2009. 22. Smach, F.; Miteran, J.; Atri, M.; Dubois, J.; Abid, M.; Gauthier, J.P. An FPGA-based ac- celerator for Fourier Descriptors computing for color object recognition using SVM. J. Real- Time Image Process, 2007, 2, 249–258. 23. Hussain, S.U.; Napoléon, T.; Jurie, F. Face Recognition Using Local Quantized Patterns. HAL: Bengaluru, India, 2012. 24. Annalakshmi, M.; Roomi, S.M.M.; Naveedh, A.S. A hybrid technique for gender classifi- cation with SLBP and HOG features. Clust. Comput, 2019, 22, 11–20. 25. Turk, M.; Pentland, A. Eigenfaces for recognition. J. Cogn. Neurosci, 1991, 3, 71–86. 26. Wang, Q.; Alfalou, A.; Brosseau, C. New perspectives in face correlation research: A tu- torial. Adv. Opt. Photonics , 2017, 9, 1–78. 27. Kambi Beli, I.; Guo, C. Enhancing face identification using local binary patterns and k- nearest neighbors. J. Imaging, 2017, 3, 37. 28. Schroff, F.; Kalenichenko, D.; Philbin, J. Facenet: A unified embedding for face recogni- tion and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA, 2015, 815–823. 29. Napoléon, T.; Alfalou, A. Local binary patterns preprocessing for face identifica- tion/verification using the VanderLugt correlator. In Optical Pattern Recognition XXV; Inter- national Society for Optics and Photonics; SPIE: Bellingham, WA, USA, 2014, 9094, 909408. 30. XueMei, Z.; ChengBing, W. A real-time face recognition system based on the improved LBPH algorithm. in 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP), 2017,72–76. 31. Wenyi, Z.; et al. Face recognition: A literature survey. ACM Computing Surveys (CSUR), 2003, 35,4, 399–458. 32. Stan, Z. L.; Anil, K.J. Handbook of Face Recognition. Springer-Verlag, Secaucus, NJ, 2005. 33. Fisher, R. A. The statistical utilization of multiple measurements. Annals of Eugenics, 1938, 8,376-386. 34. John, P. L. Fast normalized cross-correlation. Vision Interface, 1995, 120-123. 35. Laurenz, W.; Jean-Marc, F.; Norbert, K.; Christoph, V. Face recognition by elastic bunch graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19,7, 775-779. 36. Michael, K.; Lawrence, S. Application of the Karhunen-Loeve procedure for the charac- terization of human faces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1990,12,1, 103-108. https://ieeexplore.ieee.org/author/37086256714 158 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 37. Xiaoguang, Lu.; Anil, K. J.; Dirk, C. Matching 2.5D face scans to 3D models. IEEE transactions on pattern analysis and machine intelligence, 2006,28,1, 31–43. 38. Mun, W. L.; Surendra, R. Pose-invariant face recognition using a 3d deformable model. Pattern Recognition, 2003,36:1835-1846. 39. Volker, B.; Thomas, V. Face recognition based on fitting a 3d morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003,25,9, 1063-1074. 40. Zahid, M.; Nazeer, M.; Nargis, B.; Tauseef, A. A Review on state-of-the-art face recognition approaches. Fractals, 2017,25, 02, 1750025. 41. Shaogang, G.; Stephen, J.M.; Alexandra, P. Dynamic vision: from images to face recogni- tion. Imperial College Press,2000. 42. Unsang, P.; Yiying, T.; Anil, K.J. Age-invariant face recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2010, 32,5, 947-954. 43. Dihong, G.; et al. Hidden factor analysis for age invariant face recognition. Computer Vi- sion (ICCV), 2013 IEEE International Conference on. IEEE, 2013. 44. Andreas, L.; Chris, J.T.; Timothy, F.C. Toward automatic simulation of aging effects on face images. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2002, 24,4, 442-455. 45. Hayet, B.; et al. Face recognition under varying facial expression based on Perceived Fa- cial Images and local feature matching. Information Technology and e-Services (ICITeS), 2012 International Conference on. IEEE, 2012. 46. Marryam, M.; et al. Analysis of Face Recognition under Varying Facial Expression: A Survey. International Arab Journal of Information Technology (IAJIT), 2013, 10,4. 47. Shah, J. H.; Sharif, M.; Raza, M.; Azeem, A. Face recognition across pose variation and 3S problem. In TÜBİTAK Academic Journals, 2012. 48. Xiaozheng, Z.; Yongsheng, G. Face recognition across pose: A review. Pattern Recogni- tion, 2009, 42,11, 2876-2896. 49. Aisha, A.; Muhammad, S.; Mudassar, R.; Marryam, M. A survey: face recognition tech- niques under partial occlusion. Int. Arab J. Inf. Technol, 2014, 11,1,1-10. 50. Francesc, T.; Antonio, R. A novel method for face recognition under partial occlusion or facial expression variations. Proc. 47th Int’l Symp. ELMAR, 2005. 51. Zihan, Z.; et al. Face recognition with contiguous occlusion using markov random fields. Computer Vision, 2009 IEEE 12th International Conference on. IEEE, 2009. 52. Hongjun, J.; Aleix, M.M. Face recognition with occlusions in the training and testing sets. Automatic Face & Gesture Recognition, 2008. FG'08. 8th IEEE International Conference on. IEEE, 2008. 53. Nojavanasghari, B.; Hughes, CE.; Baltrusaitis, T.; Morency, LP. Hand2face: Automatic synthesis and recognition of hand over face occlusions. Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), 2017, 209–215. 54. Xingjie, W.; Chang-Tsun, L.; Yongjian, H. Robust face recognition under varying illumi- nation and occlusion considering structured sparsity. Digital Image Computing Techniques and Applications (DICTA), 2012 International Conference on. IEEE, 2012. 55. The Yale B Database. Available online: http://vision.ucsd.edu/~leekc/ExtYaleDatabase/ExtYaleB.html (Accessed on Nov.8, 2020). 56. Sakshi, G.; Akhil, K.; Kirtika, G. A Review Paper on Biometrics: Facial Recognition. In- ternational Journal of Scientific Research Engineering & Technology (IJSRET), 2012,1,5, 012-017, www.ijsret.org ISSN 2278 – 0882. 57. Vankayalapati, H.D.; Kyamakya, K. Nonlinear feature extraction approaches with appli- cation to face recognition over large databases. In Proceedings of the 2009 2nd International Workshop on Nonlinear Dynamics and Synchronization, Klagenfurt, Austria, 20–21 July 2009; IEEE: Piscataway, NJ, USA, 2009, 44–48. 58. Rui, K.; Bing, Z. A New Face Recognition Method Based on Fast Least Squares Support Vector Machine. Physics Procedia, 2011, 22, 616-621. https://scholar.google.com/citations?user=7mZPvlYAAAAJ&hl=ar&oi=sra https://scholar.google.com/citations?user=QqpxOfEAAAAJ&hl=ar&oi=sra https://scholar.google.com/citations?user=KVbnF9AAAAAJ&hl=ar&oi=sra https://scholar.google.com/citations?user=fPrC6esAAAAJ&hl=ar&oi=sra https://scholar.google.com/citations?user=nzLXktMAAAAJ&hl=ar&oi=sra http://vision.ucsd.edu/~leekc/ExtYaleDatabase/ExtYaleB.html 159 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 59. Faten, B.; Kais, L.; Mohamed, A. Face recognition approach using Gabor Wavelets, PCA and SVM. IJCSI International Journal of Computer Science Issues, 2013,10,2, 201-206. 60. Jameel, S. Face recognition system using PCA and DCT in HMM. Int. J. Adv. Res. Com- put. Commun. Eng, 2015, 4, 13–18. 61. Fathima, A.A.; Ajitha, S.; Vaidehi, V.; Hemalatha, M.; Karthigaiveni, R.; Kumar, R. Hy- brid approach for face recognition combining gabor wavelet and linear discriminant analysis. In: IEEE International Conference on Computer Graphics, Vision and Information Security, 2015, 220–225. 62. Ghorbel, A.; Tajouri, I.; Aydi,W.; Masmoudi, N. A comparative study of GOM, uLBP, VLC and fractional Eigenfaces for face recognition. In Proceedings of the 2016 International Image Processing, Applications and Systems (IPAS), Hammamet, Tunisia, 5–7 November 2016; IEEE: Piscataway, NJ, USA, 2016, 1–5. 63. Bhaskar, A.; Prashant, K.S. Face Recognition using SURF Features and SVM Classifier. ISSN 0975- 6450, 2016, 8,1, 1-8. 64. Fu, T.; Chiu, W.; Wang, Y.F. Learning guided convolutional neural networks for cross resolution face recognition. In: 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), 2017. 65. Khan, A.; et al. Forensic video analysis: passive tracking system for automated Person of Interest (POI) localization. IEEE Access, 2018, 6, 43392–43403. 66. Banerjee, S.; et al. To frontalize or not to frontalize: do we really need elaborate pre- processing to improve face recognition ?. In IEEE Winter Conference on Applications of Computer Vision, 2018, 20–29. 67. https://techcrunch.com/2017/12/13/china-cctv-bbc-reporter/?guccounter=1. (Accessed on Nov.8, 2020). 68. Sajjad, M.; Nasir, M.; Ullah, FUM.; Muhammad, K.; Sangaiah, AK.; Baik, SW. Raspber- ry pi assisted facial expression recognition framework for smart security in law-enforcement services. Inf Sci, 2019, 479, 416–431. 69. Kim, K. Intelligent immigration control system by using passport recognition and face verification. In: International symposium on neural networks. Springer, 2005, 147–156. 70. Fianyi, I.; Zia, T. Biometric technology solutions to countering today’s terrorism. Int J CyberWarf Terror, 2016, 6,4, 28–40. 71. Heng, W.; Jiang, T.; Gao, W. How to assess the quality of compressed surveillance videos using face recognition. IEEE Trans Circ Syst Video Techn, 2019, 29,8 , 2229–2243. 72. Becerra-Riera, F.; Morales-Gonz˙alez, A.; M˙endez-V˙azquez, H. A survey on facial soft biometrics for video surveillance and forensic applications. Artif Intell Rev, 2019, 52,2, 1155–1187. 73. Mandavkar, AA.; Agawane, RV. Mobile based facial recognition using otp verification for voting system. In: 2015 IEEE International advance computing conference (IACC), 2015, 644–649. 74. Qianmu, L.; Tao, L.; Bin, X.; Ming, N.; Xiaoqian, L.; Qifeng, Z.; Yong, Q. face identity recognition in smart bank. Int J Seman Comput, 2016,10,4, 569. 75. De Carrera, PF.; Marques, I. Face recognition algorithms. Master’s thesis in Computer Science. Universidad Euskal Herriko, 2010. 76. Feltwell, T.; Wood, G.; Linehan, C.; Lawson, S. An augmented reality game using face recognition technology. In: Proceedings of the ACM Conference Companion Publication on Designing Interactive Systems, 2017, 44–49. 77. Zhang, Y.; Shao, J.; Ouyang, D.; Shen, HT. Person re-identification using two-stage con- volutional neural network. In: 24Th international conference on pattern recognition. ICPR, Beijing, 2018, 3341–3346. 78. Bashbaghi, S.; Granger, E.; Sabourin, R.; Parchami, M. Deep learning architectures for face recognition in video surveillance. coRR, arXiv:1802.09990, 2018. 79. Qian, J. A survey on sentiment classification in face recognition. J Phys Conf Ser, https://techcrunch.com/2017/12/13/china-cctv-bbc-reporter/?guccounter=1. 160 Ibn Al-Haitham Jour. for Pure & Appl. Sci. 34 (2) 2021 2018,960:012030. 80. Brandon, A.; Bartosz, L.; Mahadev, S. OpenFace: A general-purpose face recognition li- brary with mobile applications. Carnegie Mellon University, Pittsburgh, PA, USA, Jun, 2016. 81. Yiran, S.; Mingrui, Y.; Bo, W.; Chun, T.C.; Wen, H. Learn to recognise: exploring priors of sparse face recognition on smartphones. IEEE Transactions on Mobile Computing, 2017, 16, 6, 1705-1717. 82. Kwon, B.; Lee, K. An introduction to face-recognition methods and its implementation in software applications. Int J Inf Technol Manag, 2018,17,1/2,33–43. 83. Salici, A.; Ciampini, C. Automatic face recognition and identification tools in the forensic science domain. In: International tyrrhenian workshop on digital communication. Springer, 2017, 8–17. 84. Calo, SB.; Ko, BJ.; Lee, K.; Salonidis, T.; Verma, DC. Controlling privacy in a face recognition application. In: US Patent app, 15/876,307. Google patents, 2018. 85. Biometrics technology market analysis report by end-use. Grand View Research, San Francisco, CA, USA, Sep, 2018, https://www.grandviewresearch.com/industry- analysis/biometrics-industry (Accessed on Nov.8, 2020). 86. The general data protection regulation. European Union, Apr, 2016. Available Online: mail- to:https://eurlex.europa.eu/legalcontent/EN/TXT/PDF/?uri=CELEX:32016R0679&from=EN (Accessed on Nov.8, 2020). 87. Julia , O.; Katrin A.; Martina, Z. All eyes on you! Impact of location, camera type, and privacy-security-tradeoff on the acceptance of surveillance technologies. in International Conference on Smart Cities and Green ICT Systems, Porto, Portugal, Apr, 2017. https://www.grandviewresearch.com/industry-analysis/biometrics-industry https://www.grandviewresearch.com/industry-analysis/biometrics-industry mailto:https://eurlex.europa.eu/legalcontent/EN/TXT/PDF/?uri=CELEX:32016R0679&from=EN mailto:https://eurlex.europa.eu/legalcontent/EN/TXT/PDF/?uri=CELEX:32016R0679&from=EN