Advances in Systems Science and Application (2016) Vol.16 No.2 94-102 Driver Drowsiness Detection System Meriem boumehed1, Belal Alshaqaqi2, Abdullah Salem Baquhaizel2 and Mohamed El Amine Ouis2 1Preparatory school in science and technology 2University of Sciences and Technology of Oran Mohamed Boudiaf (USTO-MB) Oran, Algeria Abstract Drowsiness and Fatigue of drivers are amongst the significant causes of road accidents. Every year, they increase the amounts of deaths and fatalities in- juries globally. In this paper, a module for Advanced Driver As-sistance System (ADAS) is presented to reduce the number of accidents due to drivers fatigue and hence increase the transportation safety; this system deals with automatic driver drowsiness detection based on visual information and Artificial Intelligence. We proposed an algorithm to locate, track, and analyze both the drivers face and eyes to measure percentage of eye dosure, a scientifically supported measure of drowsiness associated with slow eye closure. Keywords Drowsiness detection, ADAS, Face Detection and Tracking, Eyes De- tection and Tracking 1 Introduction Currently, transport systems are an essential part of human activities. We all can be victim of drowsiness while driving, simply after a night’s sleep too short, altered physical condition or during long journeys. The sensation of sleep reduces the driver’s level of vigilance producing dangerous situations and increases the probability of an occurrence of accidents. Driver drowsiness and fatigue are among the important causes of road accidents. Every year, they increase the number of deaths and fatalities injuries globally. In this context, it is important to use new technologies to design and build systems that are able to monitor drivers and to measure their level of attention during the entire process of driving. In this paper, a module for ADAS (Advanced driver assistance System) is presented in order to reduce the number of accidents caused by driver fatigue and thus improve road safety. This system treats the automatic detection of driver drowsiness based on visual information and artificial intelligence. We proposed an algorithm to locate, track and analyze both the driver face and eyes to measure percentage of eye closure. The remainder of this paper is organized as follows, Section 2 presents the related works, Section 3 presents the proposed system and the implementation of each block of the system, the experimental results are shown in section 4 and in the last section conclusions and perspectives are presented. Advances in Systems Science and Application (2016) Vol.6 No.2 95 2 Related Works Some efforts have been reported in the literature on the development of the not- intrusive monitoring drowsiness systems based on the vision. Malla et al.[1] develop a light-insensitive system. They used the Haar algorithm to detect objects[2] and face classifier implemented by[3] in OpenCV[4] libraries. Eye regions are derived from the facial region with anthropometric factors. Then, they detect the eyelid to measure the level of eye closure. Vitabile et al. implement a system to detect symptoms of driver drowsiness based on an infrared camera[5]. By exploiting the phenomenon of bright pupils, an algorithm for detecting and tracking the driver’s eyes has been developed. When drowsiness is detected, the system warns the driver with an alarm message. Bhowmick et Kumar use the Otsu thresholding to extract face region[6,7]. The localization of the eye is done by locating facial landmarks such as eyebrow and possible face center. Morphological operation and K-means is used for accurate eye segmentation. Then a set of shape features are calculated and trained using non-linear SVM to get the status of the eye. Hong et al. define a system for detecting the eye states in real time to identify the driver drowsiness state[8]. The face region is detected based on the optimized Jones and Viola method[2]. The eye area is obtained by an horizontal projection. Finally, a new complexity function with a dynamic threshold to identify the eye state. Tian et Qin build a system that checks the driver eye states. Their system uses the Cb and Cr components of the YCbCr color space[9]. This system locates the face with a vertical projection function, and the eyes with a horizontal projection function. Once the eyes are located the system calculates the eyes states using a function of complexity. Under the light of what has been mentioned above, the identification of the driver drowsy state given by the PERCLOS is generally passed by the following stages: 1) Face detection, 2) Eyes Location, 3) Face and eyes tracking, 4) Identification of the eyes states, 5) Calculation of PERCLOS and identification the driver state. 3 The Proposed System In this section, we discuss our presented system which detects driver drowsiness. The overall flowchart of our system is shown in Fig 1. 96 Meriem B., Belal A., Abdullah Salem B, etc: Driver Drowsiness Detection system A. Face Detection The symmetry is one of the most important facial features.We modeled the sym- metry in a digital image by a one-dimensional signal (vector accumulator) with a size equal the width of the image, which gives us the value corresponding to the position of the vertical axis of symmetry of objects in the image. The traditional principle to calculate the signal of symmetry is for each two white pixels which are on the same line we increment the value in the medium between these two pixels in the accumulating vector. (The algorithm is applied on an edge image, we called a white pixel: the pixel with value 1). We introduce improvements on the calculation algorithm of symmetry into an image to adapt it to the detection of face, by applying a set of rules to provide a better calculation of symmetry of the face. Instead of computing the symmetry between two white pixels in the image, it is calculated between two windows (Z1 and Z2) (Fig.2). Fig. 1 Flowchart of the proposed system For each window Z1, we sweep the window Z2 in the area determined by the parameters S min, S max, and H. We increment the signal of symmetry between Advances in Systems Science and Application (2016) Vol.6 No.2 97 these two windows if the sum of white pixels is located between two thresholds S1 (maximum) and S2 (minimum). Fig. 2 The new method to improve the calculation of the symmetry in the image (a) Original image (b) Edge detection (c) Symmetry signal (d) Localization of the maxi- mum of symmetry (e) Region of interest ROI (f) Result Fig. 3 Face detection using symmetry Then we extract the vertical region of the image contours (Region of Interest ROI) corresponding to the maximum index of the obtained signal of symmetry. Next, we take a rectangle with an estimated size of face (Because the camera is fixed and the driver moves in a limited zone so we can estimate the size of the face using the camera focal length after the step of camera calibration) and we 98 Meriem B., Belal A., Abdullah Salem B, etc: Driver Drowsiness Detection system scan the ROI by searching the region that contains the maximum energy corre- sponding to the face (Fig.3). We propose a checking on two axes: the position variance of the face detect- ed according to time; i.e., in several successive images, it is necessary that the variance of the positions of the detected face is limited; because the speed of movement of the face is limited of some pixels from a frame to another frame which follows. B. Eyes Localization Since the eyes are always in a defined area in the face (facial anthropometric prop- erties), we limit our research in the area between the forehead and the mouth (Eye Region of Interest ’eROI’) (Fig.4.a). We benefit from the symmetrical char- acteristic of the eyes to detect them in the face. First, we sweep vertically the eROI by a rectangular mask with an estimated height of height of the eye and a width equal to the width of the face, and we calculate the symmetry. The eye area corresponds to the position which has a high measurement of symmetry. Then, in this obtained region, we calculate the symmetry again in both left and right sides. The highest value corresponds to the center of the eye. The result is shown in Fig.4.b. (a) eROI (b) Result Fig. 4 Eyes localization using symmetry C. Tracking The tracking is done by Template Matching using the SAD Algorithm (Sum of Absolute Differences). SAD(x, y) = N∑ i=1 M∑ i=1 |I(x+ i, y + j)−M(i, j)| (1) We proposed to make a regular update of the reference model M to adjust it every time when light conditions changes while driving, by making a tracking Advances in Systems Science and Application (2016) Vol.6 No.2 99 test: Tracking = { good if SAD ≤ Th bad if SAD > Th (2) D. Eyes States The determination of the eye state is to classify the eye into two states: open or closed. We use the Hough transform for circles [10] (HTC) on the image of the eye to detect the iris. For that, we apply the HTC to the edge image of the eye to detect the circles with defined rays, and we take at the end the circle which has the highest value in the accumulator of Hough for all the rays. Then, we apply the logical ’AND’ logic between edges image and complete cir- cle obtained by the HTC by measuring the intersection level between them “S”. Finally, the eye state “Stateeye” is defined by testing the value “S” by a thresh- old: Stateeye { Open if s ≤ Th Closed if s ≥ Th (3) The results are shown in Fig.5. (a) and (b) Edge detection (c) and (d) Eyes states results Fig. 5 Eyes states using HTC E. Driver State We determine the driver state by measuring PERCLOS. If the driver closed his eyes in at least 5 successive frames several times over a period of up to 5 seconds, 100 Meriem B., Belal A., Abdullah Salem B, etc: Driver Drowsiness Detection system it is considered drowsy. 4 Experimental Results To validate our system (Fig.6), we test on several drivers in the car with real driving conditions. We use an IR camera with infrared lighting system operates automatically under the conditions of reduced luminosity and night even in total darkness. Fig. 6 Our system installed in the car based on IR camera The results of the eye states are illustrated in Table 1. Table 1 Results obtained from the system Driver frames Number False Eyes sates false rate Open Closed D1/day 420 17 0 4% D2/day 430 15 0 3.50% D3/day 245 7 1 3.20% D1/night 200 3 1 2% D2/night 200 1 0 0.50% D3/night 200 6 3 4.50% According to the obtained results, our system can determine the eye states Advances in Systems Science and Application (2016) Vol.6 No.2 101 with a high rate of correct decision. 5 Conclusion and Perspectives In this paper, we presented the conception and implementation of a system for detecting driver drowsiness based on vision that aims to warn the driver if he is in drowsy state. This system is able to determine the driver state under real day and night conditions using IR camera. Face and eyes detection are implemented based on symmetry. Hough Transform for Circles is used for the decision of the eyes states. The results are satisfactory with an opportunity for improvement in face de- tection using other techniques concerning the calculation of symmetry. Moreover, we will implement our algorithm on a DSP (Digital Signal Proces- sor) to create an autonomous system working in real time. References [1] A. Malla, P. Davidson, P. Bones, R. Green and R. Jones. (2010), “Au- tomated video-based measurement of eye closure for detecting behavioral microsleep”, in 32nd Annual International Conference of the IEEE, Buenos Aires, Argentina. [2] P. Viola and M. Jones. (2001), “Rapid object detection using a boosted cascade of simple features”, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. [3] R. Lienhart and J. Maydt. (2002), “An extended set of haar-like features for rapid object detection”, in Proceedings of the IEEE International Conference on Image Processing. [4] Open Source Computer Vision Library: Reference Manual, Available at: http://opencvlibrary.sourceforge.net [5] S. Vitabile, A. Paola and F. Sorbello. (2010), “Bright pupil detection in an embedded, real-time drowsiness monitoring system”, in 24th IEEE Interna- tional Conference on Advanced Information Networking and Applications. [6] B. Bhowmick and C. Kumar. (2009), “Detection and Classification of Eye State in IR Camera for Driver Drowsiness Identification”, in Proceeding of the IEEE International Conference on Signal and Image Processing Appli- cations. [7] N. Otsu. (1979), “A threshold selection method from gray-level histograms”, IEEE Transactions on Systems,Man and Cybernatics, pp.62-66. 102 Meriem B., Belal A., Abdullah Salem B, etc: Driver Drowsiness Detection system [8] T. Hong, H. Qin and Q. Sun. (2007), “An improved real time eye state identification system in driver drowsiness detection”, in proceeding of the IEEE International Conference on Control and Automation, Guangzhou, CHINA. [9] Z. Tian et H. Qin. (2005), “Real-time driver’s eye state detection”, in Pro- ceedings of the IEEE International Conference on Vehicular Electronics and Safety. [10] M. S. Nixon and A. S. Aguado. (2008), Feature Extraction and Image Processing(2nded.), Jordan Hill, Oxford OX2 8DP, UK. Corresponding author Belal Alshaqaqi can be contacted at: alshaqaqi belal@hotmail.fr