INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL ISSN 1841-9836, 11(5):657-666, October 2016. Robust Face Recognition Against Soft-errors Using a Cross-layer Approach G.-M. Jeong, C.-W. Park, S.-I. Choi, K. Lee, N. Dutt Gu-Min Jeong, Chang-Woo Park Department Electrical Engineering Kookmin University Jeongeung-dong, Songbuk-gu Seoul, 136-702, Korea gm1004@kookmin.ac.kr, high1117@nate.com Sang-Il Choi* Department Applied Computer Engineering Dankook University Jukjeon-dong, Suji-gu, Yongin-si Gyeonggi-do, 448-701, Korea *Corresponding author: choisi@dankook.ac.kr Kyoungwoo Lee Department Computer Science Yonsei University Sinchon-dong, Seodaemun-gu Seoul, 120-749, Korea kyoungwoo.lee@gmail.com Nikil Dutt Department Computer Science University of California, Irvine Irvine, CA 92697, USA dutt@ics.uci.edu Abstract: Recently, soft-errors, temporary bit toggles in memory systems, have be- come increasingly important. Although soft-errors are not critical to the stability of recognition systems or multimedia systems, they can significantly degrade the system performance. Considering these facts, in this paper, we propose a novel method for robust face recognition against soft-errors using a cross layer approach. To attenu- ate the effect of soft-errors in the face recognition system, they are detected in the embedded system layer by using a parity bit checker and compensated in the appli- cation layer by using a mean face. We present the soft-error detection module for face recognition and the compensation module based on the mean face of the facial images. Simulation results show that the proposed system effectively compensates for the performance degradation due to soft errors and improves the performance by 2.11 % in case of the Yale database and by 10.43 % in case of the ORL database on average as compared to that with the soft-errors induced. Keywords: Soft-error, Face recognition, Cross-layer approach, Mean face. 1 Introduction Nowadays, memory errors due to various causes have become a critical threat for the per- formance and stability of numerous systems. In particular, soft-errors, which are transient bit toggles in memory systems, have become increasingly important factors for the performance Copyright © 2006-2016 by CCC Publications 658 G.-M. Jeong, C.-W. Park, S.-I. Choi, K. Lee, N. Dutt degradation of various applications. Soft-errors denote the phenomenon that changes the mem- ory bit value temporarily from 1 to 0 and vice versa due to abnormal conditions such as high radiation, high pressure or high temperature [1]. Although the soft-errors are not permanent and nondestructive, there have been several reports on the damages due to soft-errors. SUN and Hewlett Packard announced the loss due to soft-errors in their server systems [2,3]. Further, soft errors brought a billion-dollar automotive factory to halt every month [4]. Until now, there have been several researches on the protection of soft-errors [5–9]. Especially, for multimedia systems, cross-layer based approaches have been introduced to compensate for the negative effects of soft-errors. Since soft-errors are not critical to the system stability, cooperation across system abstraction layers can help to build a cost-efficient system against soft-errors from a hardware layer to an application layer in mobile embedded systems [10]. In this paper, we propose a robust and cost-effective system for face recognition against soft-errors by using a cross layer approach. First, we analyze the effect of soft-errors for face recognition systems and show that the soft-errors induced in the JPEG image can degrade the performance of the recognition system [11, 12]. Next, we propose a cross-layer compensation module consisting of a detector in the hardware layer and a corresponding compensator in the application layer. To attenuate the negative impact of soft-errors in face recognition systems, they are detected in the embedded system layer by using a parity bit checker. When they are detected, the mean face method is used for compensating for the negative impact on the performance at the application layer. The classification experiments are performed for the Yale [13] and ORL databases [14]. The features for classification are extracted by using the RLDA (Regularized Linear Discriminant Analysis) method [12]. The experimental results demonstrate the effectiveness of the proposed method. Further, the proposed cross-layer based compensation method can improve the system performance by 2.11 % and by 10.43 % on average in case of YALE and ORL databases, respectively, as compared to the performance with soft-errors in the face recognition system. The remainder of this paper is organized as follows. In Section 2, we introduce an exemplary face recognition system for the simulation and the effect of soft-errors in JPEG images. Then, in Section 2, we describe an analysis of the effect of soft-errors on the face recognition system. In Section 4, we present the proposed cross-layer approaches and experimental results. The conclusion follows in Section 5. 2 Exemplary face recognition system and effect of soft-errors in JPEG images 2.1 Exemplary face recognition system Figure 1 shows the exemplary face recognition system considered in this study. As shown in figure 1, after a JPEG image is captured using a camera, it is transmitted to the classification part and decoded in the BMP format. Then, after preprocessing, it is classified using various classification methods [12,15–17]. For the analysis of the soft-error impacts, we assume that the soft-error occurs inside the JPEG file. Hence, we assume that the soft-error can occur in the memory when storing or decoding the JPEG file. 2.2 Effect of soft-errors in JPEG file The JPEG file is a compressed image file format. The effects of soft-errors depend on where they occur. For example, the effects are different between the soft-errors in the header information Robust Face Recognition Against Soft-errors Using a Cross-layer Approach 659 Figure 1: An exemplary face recognition system Figure 2: Effect of soft-errors for the decoding of JPEG files. (a) Decoded image without soft- errors (b) Examples of decoded images with bit-errors 660 G.-M. Jeong, C.-W. Park, S.-I. Choi, K. Lee, N. Dutt parts and those in the compressed 8×8 blocks. Figure 2 shows various examples of decoded images when soft-errors occur in the JPEG file. Note that the decoded results may differ according to the implemented decoder. If the exceptions are not handled appropriately, the decoder can even be stopped. When soft-errors occur in the critical data of the header such as errors in the length of the file or the length of the block data, the decoder can lose all its information afterwards. In the case of a soft-error in block data, the block can be decoded inappropriately, causing a significant degradation of the quality. 3 Effect of soft-errors on the face recognition system In this section, we discuss the effects of soft-errors on face recognition systems. We build a model of the soft-errors in the face recognition system and analyze the performance degradation for the recognition rate when these errors are induced. 3.1 Modeling of soft-errors and system setting To conduct a quantitative analysis of soft-errors, we make the following assumptions of soft- errors for a face recognition system: Assumptions of the soft-errors • A1) A single-bit soft-error occurs for one facial image, particularly in the test set. • A2) A soft-error occurs only for the block data part. On the basis of A1) and A2), we can conclude that only one block is affected by a single-bit soft-error. Further, for the system setting, a soft-error is detected using a parity bit checker in the memory and it is notified to the JPEG decoder. When the JPEG decoder receives the information of the soft-error occurrence, it passes over the decoding of the corresponding block. Hence, the corresponding 8×8 block of the soft-error is filled with 0’s instead of the real decoded data. Figure 3 shows the overall processing based on the system setting in our study. Figure 3: Decoding procedure for the JPEG file with a soft error. a) Decoding procedure considering a soft-error in the block data. b) Decoded image based on the considered system setting 3.2 Classification experiment for the soft-errors We apply the proposed method to the Yale database in order to observe the effect of soft-error on the recognition rate. The Yale database contains 165 gray images of 15 individuals, having Robust Face Recognition Against Soft-errors Using a Cross-layer Approach 661 different facial expressions, with or without glasses, and under different lighting conditions. Each face image is cropped and re-scaled so that the center of each eye is placed at a fixed point in an image of 60×50 pixels. On the basis of assumption A1), we consider that a soft-error occurs in all the images for the test set, while no error occurs for the training set. There exist 42 blocks (size: 8×8) in a 60×50 image as shown in figure 4(a). Hence, we conduct 42 times an 11-fold cross validation [17] according to the location of the soft-error, as depicted in figure 4(b). In these 11-fold cross validations, one image from each subject is randomly selected for testing while the remaining images are used for training. In other words, we consider 150 images in the training set and 15 images for the probing. For the selected 15 images for the test, we assume that there occur a soft-error, as shown in figure 4(b). For the classification, the R-LDA method [12] is used as a feature extraction method and the one nearest neighbor rule is applied as a classifier with the L2 distance metric. It is noted that the R-LDA method is well known but is not necessarily the best one. In Table 1, the first and second columns show the recognition rates without a soft-error and with a soft-error, respectively. As summarized in Table 1, we can observe the average performance degradation of 2.45 % in terms of the recognition rate in the case with the soft- errors as compared to that in the case without the soft-error. As shown in figure 7(a), there is a significant degradation (up to approximately 10 %) according to the block numbers, and we can see that there is a significant performance degradation due to the single-bit soft-errors. Figure 4: Classification experiment for the soft-errors. a) 42 blocks inside the 60 × 50 image. b) 42 times 11-fold cross validation. 4 Cross-layer approach for compensation of soft-errors in the face recognition system 4.1 Overall structure of the proposed method Considering the performance degradation of the soft-errors, we propose a cost- and performance- effective compensation method for the face recognition system. Figure 5 shows the overall struc- ture of the proposed compensation method in a cross-layered manner. As shown in figure 5, when the occurrence of the soft-error is notified from the hardware layer to the application layer, the proposed method can compensate the corresponding block for the performance degradation with 662 G.-M. Jeong, C.-W. Park, S.-I. Choi, K. Lee, N. Dutt the mean value data from the database. By this compensation, we can enhance the performance of the recognition system in a cost-effective manner. Figure 5: Overall structure of the proposed compensation method In the proposed cross-layered approach, we detect the soft-error by using parity bit checkers cost-effectively rather than by using ECC (Error Correction Codes) modules. Indeed, the real compensation for the soft-errors is achieved in the application layer. In the application layer, we should mitigate the impact of soft-errors on the performance in the data block, which is filled with 0’s by skipping the decoding. In the face recognition method, the mean face is effectively used for the illumination com- pensation and other applications [18,19]. Likewise, if we use a mean face for the training data, we can improve the system performance. In this paper, we use the corresponding mean block from the mean face of the training set. Figure 6 shows an example of a mean face for the Yale database. Figure 6: Compensation using mean face. a) Example of a mean face for the Yale database. b) Facial images without a soft-error, with a soft-error and with the compensation. 4.2 Experimental results and analysis Table 1 shows the recognition rate for the cases with a soft-error, without a soft-error, and with the compensation for the soft-error, respectively in the Yale database. As shown in table 1, the proposed compensation method can enhance the performance by 2.11 % as compared to that in the case with soft-errors. In the comparison between the cases with no soft-error and compensation, there is a performance difference of only 0.34%. Figure 7(a) shows the recognition rate according to the blocks in the Yale database. Further, Table 2 and Figure 7(b) show the experimental results for the ORL database. The images are resized into 56×46 pixels in this experiment. As shown in Table 2 and Figure 7(b), Robust Face Recognition Against Soft-errors Using a Cross-layer Approach 663 Figure 7: Recognition rate according to the block numbers for the Yale and ORL databases. a) Yale database. b) ORL database. Table 1: Comparison of the recognition rate for the soft error in the Yale database(%) No. of features No error Soft-error Compensation 1 35.15 26.16 29.12 2 54.55 49.62 55.90 3 73.94 68.40 73.61 4 82.42 76.64 80.52 5 83.03 81.52 83.52 6 83.64 83.52 84.49 7 86.67 86.25 87.76 8 89.70 88.86 90.17 9 92.73 91.44 92.81 10 93.33 91.75 92.94 11 93.33 92.44 93.19 12 93.94 92.83 93.67 13 93.94 93.29 93.78 14 93.33 92.73 93.41 Average 82.12 79.67 81.78 664 G.-M. Jeong, C.-W. Park, S.-I. Choi, K. Lee, N. Dutt a soft-error can degrade the performance by approximately 11.48 %. However, the proposed compensation method can achieve a performance improvement of approximately 10.43 % as compared to the case with the soft-error while it incurs a 1.05 % performance degradation as compared to the case without the soft-errors. From these experiments, we can conclude that the proposed method can mitigate the impact of soft-errors on the system performance effectively. Table 2: Comparison of the recognition rate for the soft error in the ORL database(%) No. of features No error Soft-error Compensation 1 6.75 5.92 7.46 2 13.50 10.86 15.32 3 27.50 17.01 25.58 4 41.75 22.69 38.09 5 48.00 27.58 45.10 6 56.25 33.60 52.59 7 62.75 38.80 59.81 8 69.00 44.29 65.58 9 74.75 49.59 70.98 10 79.00 54.16 76.16 15 87.50 70.49 86.37 20 95.75 82.61 94.73 25 97.25 90.64 96.88 30 98.50 93.14 98.06 35 98.00 94.70 97.90 Average 82.62 71.14 81.57 Figure 8: Recognition rate according to the numbers of feature of for the Yale database. The recognition rate versus the numbers of feature of Yale database is shown in Figure 8 for three cases: non-error, error and compensation. It is obvious that the recognition rates increase with small numbers of feature, from one feature to eight features, and after that, the recognition rates would be almost saturated even if the numbers of feature increase how much. Therefore, we will select the minimum numbers of feature that the higher rates can be achieved. Table 3: Average recognition rates of two selection method RLDA and LBP for Yale and ORL database(%) Selection Method Yale ORL No error Error Compensation No error Error Compensation RLDA 83.11 73.57 82.27 80.04 72.67 78.96 LBP 87 86.79 86.86 89.12 88.58 88.92 We have also applied Local Binary Pattern (LBP) as a feature selection method for both Robust Face Recognition Against Soft-errors Using a Cross-layer Approach 665 databases by measuring Chi-square distances, and the result is shown in Table 3. These data in the table show the average recognition rates of forty-two blocks for three cases: non-error, error and compensation. From this table, there is no doubt that the recognition rates obtained from LBP is higher than from RLDA. The reason is that LBP uses all blocks as features, whereas RLDA only extracts some important features, hence, LBP can achieve higher result. However, LBP’s running-time is also longer than RLDA. 5 Conclusion In this paper, we have proposed a robust face recognition method for soft-errors by using a cross-layer approach. First, we have analyzed the effect of soft-error on a face recognition system. For the soft-errors in the images of the training set, the performance is degraded. Next, we have presented a novel face recognition method for the robust system against soft-errors by using a cross-layer approach. In the proposed method, the soft-errors are detected using a parity bit checker in the hardware layer and compensated in the application layer using a mean face. Simulation results have revealed that the proposed system have effectively compensated for the performance degradation due to soft errors. A more systematic compensation method such as the compensation for the occluded images can be considered with the proposed method. It remains as a future work. Acknowledgment This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2015R1D1A1A01060917) and the Basic Science Research Program (No. 2015R1A1A1A05001065) through the National Research Foundation of Korea (NRF) funded by the Ministry of Science ICT and Future Planning, and also supported by the Human Resources Program in Energy Technology of the Korea Institute of Energy Technology Evalua- tion and Planning (KETEP) granted financial resource from the Ministry of Trade Industry and Energy, Republic of Korea (No. 20154030200830). Bibliography [1] Baumann, Robert (2005); Soft errors in advanced computer systems, Design & Test of Com- puters, ISSN 0740-7475, 22(3): 258-266. [2] Lyons, D. (2000), Sun Screen: Soft Error Issue in Sun Enterprise Servers, www.members.forbes.com/global/2000/11130323026a.html. [3] Sarah E. Michalak et al. (2005); Predicting the number of fatal soft errors in los alamos national laboratory’s ASC Q supercomputer. IEEE Transactions on Device and Materials Reliability, ISSN 1530-4388 5(3): 329-335. [4] Marc Tremblay, Yuval Tami. (1989); Support for fault tolerance in VLSI processors, IEEE International Symposium on Circuits and Systems, 1: 388-392. [5] Borodin, Demid, Ben HH Juurlink. (2010); Instruction precomputation with memoization for fault detection, Proceedings of the Conference on Design, Automation and Test in Europe, ISBN 978-3-9810801-6-2, 1665-1668. 666 G.-M. Jeong, C.-W. Park, S.-I. Choi, K. Lee, N. Dutt [6] Lee, Kyoungwoo, et al. (2006) Mitigating soft error failures for multimedia applications by selective data protection. Proceedings of the 2006 international conference on Compilers, architecture and synthesis for embedded systems, ISBN: 1-59593-543-6, 411-420. [7] Nakka, Nithin, Karthik Pattabiraman, and Ravishankar Iyer. (2007); Processor-Level Se- lective Replication. Proceedings of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, ISBN 0-7695-2855-4, 544-553. [8] Lee, Kyoungwoo, et al. (2007); Partially protected caches to reduce failures due to soft errors in multimedia applications. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, ISSN 1063-8210, 17(9): 1343-1347. [9] Vera, Xavier, et al. (2009); Selective replication: A lightweight technique for soft errors. ACM Transactions on Computer Systems, ISSN 0734-2071, 27(4), DOI: 10.1145/1658357.1658359. [10] Lee, Kyoungwoo, et al. (2008); Mitigating the impact of hardware defects on multimedia applications: a cross-layer approach. Proceedings of the 16th ACM international conference on Multimedia, ISBN: 978-1-60558-303-7, 319-328. [11] Lu, Juwei et al. (2003); Regularized discriminant analysis for the small sample size problem in face recognition. Pattern Recognition Letters, ISSN 0167-8655, 24(16): 3079 - 3087. [12] Lu, Juwei, Konstantinos N. Plataniotis, and Anastasios N. (2005); Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition. Pattern Recognition Letters, ISSN 0167-8655, 26(2): 181-191. [13] Center for Computational Vision and Control, Yale University, The Yale Face Database, http://cvc.yale.edu/projects/yalefaces/yalefaces.html. [14] AT&T Laboratories Cambridge. The ORL database of faces, http://www.uk.research.att.com/facedatabase.html. [15] Lu, Juwei, Kostantinos N. Plataniotis, and Anastasios N. (2003); Face recognition using LDA-based algorithms, Neural Networks, ISSN 1045-9227, 14(1): 195 - 200. [16] Turk, Matthew, and Alex Pentland. (1991); Eigenfaces for recognition, Journal of cognitive neuroscience, ISSN 0898-929X, 3(1): 71-86. [17] Andrew R. Webb (eds.)(2002); Statistical Pattern Recognition, 2ndnd ed, Wiley, ISBN: 978- 0-470-85478-5. [18] Choi, Sang-Il, Chong-Ho Choi, and Nojun Kwak. (2011); Face recognition based on 2D images under illumination and pose variations, Pattern Recognition Letters, ISSN 0167-8655, 32(4): 561-571. [19] Choi, Sang-Il, and Gu-Min Jeong. (2011); Shadow compensation using fourier analysis with application to face recognition, IEEE Signal Processing Letters, ISSN 1070-9908, 18(1): 23-26.