Acta Polytechnica CTU Proceedings doi:10.14311/APP.2017.12.0064 Acta Polytechnica CTU Proceedings 12:64–67, 2017 © Czech Technical University in Prague, 2017 available online at http://ojs.cvut.cz/ojs/index.php/app SUBJECTIVE METHODS FOR ASSESSMENT OF DRIVER DROWSINESS Alina Mashko Czech Technical University in Prague, Faculty of Transportation Sciences, Konviktská 20, Prague 1, Czech Republic correspondence: mashkali@fd.cvut.cz Abstract. The paper deals with the issue of fatigue and sleepiness behind the wheel, which for a long time has been of vital importance for the research in the area of driver-car interaction safety. Numerous experiments on car simulators with diverse measurements to observe human behavior have been performed at the laboratories of the faculty of the authors. The paper provides analysis and an overview and assessment of the subjective (self-rating and observer rating) methods for observation of driver behavior and the detection of critical behavior in sleep deprived drivers using the developed subjective rating scales. Keywords: driver drowsiness, subjective methods, observer rated sleepiness. 1. Introduction Driving sleepy, drowsy or fatigued at a high extent affects road traffic safety. The research by American Centers for Disease Control and Prevention (CDC) reports that 7500 fatal accidents happened in years 2011-2012 due to the drowsy factor [1]. Per AAA Foundation for Traffic Safety, 21% of drowsy drivers were involved in fatal traffic accidents in the USA, which is 4,5% higher as compared to the results of the preceding years [2]. Similar involvement rates (20% of accidents due to sleepiness at the wheel) have been reported by the Australian Transport Accident Commission [3]. Two-thirds of the population (71%) are said to be sleeping less than 8 hours per day. The most vulnerable to sleep deprivation are so called risk group that includes shift workers, young drivers and people with children in household. Driving drowsy is dangerous, mainly due to the possible consequent impairments caused by this state. According to the Na- tional Sleep Foundation’s poll in 2000, drowsy drivers tend to feel stressed (42%), be impatient (32%) or drive faster (12%) [4] or not use the seat belt during night driving [5]. Known reasons of drowsy driving and inattention are driving at the time of day when circadian rhythms are low, or driving for a long time or sleep loss [6]. 2. Definition of fatigued and drowsy state Due to the individual behaviour of every person in a particular state as well as because of a variety of rea- sons for drowsy driving, finding common attributes or indicators in this state is especially difficult. Drowsi- ness and fatigue, per self-reported perceptions, influ- ence performance, physiological state, feeling. It has been found that in a state of fatigue individuals have experienced such feelings as being tired, sleepy or ex- hausted, not feeling inclined to continue performing the task because of a perceived reduction of efficiency, feeling tired, exhausted, lack of energy as related to impaired of both physical and cognitive functioning, need of restoration of resources [7]. Individual percep- tion of one’s drowsiness can be determined with the help of questioning objects, conducting self-evaluation tests and is related to subjective observation. According to [8], the physiological state of a fatigued individual this corresponds to the organism’s muscu- lar, visceral and central nervous system a decrease of capacity or energy to maintain initial activity due to insufficient rest is observed. Detection of physiological signs of drowsiness and fatigue is carried out via mea- surements of the heartbeat rate, brain activity, eye blinking and closure. Eye behavior can be observed with the help of electrooculogram (EOG) [9, 10], with eye tracker, such as in researches presented in [11–13], based on eye image video analysis [14], etc. A lot of research has been done for exploration of brain activ- ity of sleepy and drowsy drivers, some of the latest are [15–19]. The measurement itself is performed with the help of special electrodes of an EEG device that are placed on the human scalp to read brain wave activity. Evaluation of the quality of the task performance is another possible measure of drowsiness and fatigue. Fatigue caused by sleep deprivation affects driving performance which can reflect in delayed reaction, inability to maintain constant speed, loose of lateral control [20, 21] etc. Measurements can be conducted in real traffic as well as in simulated environment, which is safer. Usually, the performance indicators are measured for each object under two conditions: in a state of sleep deprivation and in fresh (control or reference) state. 64 http://dx.doi.org/10.14311/APP.2017.12.0064 http://ojs.cvut.cz/ojs/index.php/app vol. 12/2017 Subjective methods for assessment of driver drowsiness 3. Subjective drowsiness and fatigue Subjective indicators of sleepiness level include sleepi- ness scales, such as Epworth Sleepiness Scale (ESS), Karolinska Sleepiness Scale (KSS) and Stanford Sleepi- ness Scale. During these tests short questionnaires are given to subjects who are asked to rate their state according to the suggested scale. The self-rating is usu- ally conducted repeatedly during the testing time with either a time interval, or situations (conditions). This kind of rating allows evaluation of the participants’ awareness of their alertness level. When compared to other driver drowsiness measures, it has been demon- strated that KSS defined level of sleepiness in most cases has a curve-linear relation to lateral position and blink duration [22]. Another method used in experimental investigation of driver fatigue is observer rated sleepiness (ORS). The method quantifies observation of behavior by trained individuals or experts who are rating driver performance based on video recordings of the experi- ment or in real time monitoring. The method can be used for validation of other measures or as a separate measure of driver fatigue [23]. 4. Subjective evaluation during experiments on driver simulator All observations described in this chapter are related to experiments conducted on a full body (Skoda Superb) fixed driving simulator based on a wide projection angle (210° of driver’s view is provided with 5 screens plus 2 screens for side mirrors. See Figure 1 for ref- erence). Detailed description of simulator and lab is provided in [24]. Figure 1. Advanced driver simulator for driver drowsiness research. Previously, we have conducted observations of sleep deprived drivers during a 2 hours’ driver simula- tor measurements [25]. Different observers tracked changes in the drivers behavior based on video record- ings. The target areas were head, face (facial ex- pression and mimicking, eye behavior) and upper torso (hands, shoulders and body). Observers were instructed to document all noticed behavioral pat- terns out of (but not limited to) the suggested list: frequency of eye blinking, change in duration of eye closures, yawning, face muscles relaxation, lip biting, head tilt, shoulders shaking, hands on the wheel posi- tion, body relaxation, change of sitting position. Also from the shots of the driving scenery driving behavior (steering, correction of position in a lane, lane depar- ture and accident) were observed. It was possible to notice that blinking frequency was observed after 40th minute of driving. Blinking was then gradually substituted by longer eye closures often accompanied by worsening of steering behavior, such as weaving and loose of vehicle control up to accidents. In most drivers head rotations (neck muscles stretching) and head tilt were noticed. This, however, didn’t reflect in the alteration of driving behavior (i.e., steering, speed fluctuation etc.) and was considered as a voluntary type of behavior caused by both sleepiness and fatigue from a long drive taken by subjects with a purpose of finding more comfortable position. All observed sub- jects have displayed dangerous driver behavior, that in real traffic might lead to accident, mainly starting after 30 minutes of experiment. Inability to maintain straight trajectory, weaving and lane departure could be observed. At the laboratory for driver sleepiness research a self-rating sleepiness scale has been developed and is being used to assess driver sleepiness rate by the subject him/herself. The scale is presented in Table 1, each of its levels corresponds to change in sleepiness level as perceived by subject. State description as perceived by subject 1 I feel fine/fresh & driving does not make me any problems. 2 I feel drowsy & driving does not make me any problems. 3 I feel drowsy & I notice some problems. 4 I feel very drowsy & I need to excessively con- centrate to drive correctly. 5 I experienced ’blackouts’ & losing control over the car. Table 1. Self-rating sleepiness scale developed at faculty of Transportation Sciences, CTU in Prague. Evaluation per this scale will be presented in further research outcomes. The ORS method was applied for analysis of video recordings of sleep deprived drivers on simulator. 9 observers (7 male and 2 female) who are students of Faculty of Transportation sciences, Czech Technical University conducted evaluation. The observations combined body movements, facial expressions and driving and vehicle behavior. Initially developed scale contained three basic levels: (1.) subject is sitting straight behind the wheel with eyes open and both hands on the wheel in 10/2 position, he/she is following the road situation and 65 Alina Mashko Acta Polytechnica CTU Proceedings Figure 2. Results for ORS evaluation by 9 observers of one sleep deprived subject. Figure 3. Difference between maximum and minimum rates provided by observers. cluster indicators, vehicle is within the lane and is following straight path, (2.) subject experiences some features of drowsiness, like more frequent blinking and episodes of pro- longed blinking, vehicle is experiencing some weav- ing, correction movements are present in steering, subject sitting position is not straight (head incli- nations), subject is performing some self-awakening actions, hands positioned in lower part of the wheel, (3.) episodes of lane departure due to the result of longer eye closures (sleep episodes) steering correc- tion movements are more abrupt, body position is not straight (subject adjusts to be more comfort- able for rest position), head inclined to the side, one hand may be off the wheel, or on the bottom of the steering wheel. During the evaluation process, it had been decided to extend it to a 5-scale grade by introducing the half-scores for so called transition states (i.e., giving 1.5 and 2.5 scorers when subjects demonstrated only some features of the state). Outcome of such obser- vation is presented in Figure 2. There one may track the mean, max and min rates of subject state sam- pled per 5 minutes of two-hour experiment. Every human behavior is individual and people display dif- ferent behavior under same conditions. Besides, the subjective perception of each observer influences the ORS data (see the difference chart between rates – Figure 3). The same type of behavior may vary per subject in severity and impact that’s why it is not possible to assign same weights to the displays of hu- man behavior. For those reasons, precise definition of a state to correspond the rate score is sometimes compromised. This is confirmed by a study, where results have demonstrated poor correlation of ORS to SRS (self-reported sleepiness) [26]. It is still possible to track a tendency of an increasing sleepiness rate per suggested scale after the 40th minute of driving, which corresponds to conclusions mentioned in [25]. 5. Conclusion The research of driver fatigue covers the applications of many branches of science such as sociology, phys- iology, neuroscience, engineering. Individual human behavior is a challenge in finding a single method for detection of drowsiness. Besides, validation of detection methods in real traffic conditions is more complicated because these experiments may cause se- rious accidents. Subjective methods have proved to be reliable for rating of fatigue in laboratory experiments and can be considered as a viable method for further exploration of the problem. Self-rating and expert observations have proven to provide an adequate pic- ture of driver fatigue and the tendency of increasing sleepiness during prolonged driving and can be corre- lated to objective measurements. Rigid definition of the states to correspond the scale rate is however cru- cial for evaluation results. Difference in scores given 66 vol. 12/2017 Subjective methods for assessment of driver drowsiness by observers to the same behavior is crucial of data validity and could make the results questionable. The current research is aimed at tailoring the suggested evaluation method for its further implementation for analyses of driver sleepiness experiments. Acknowledgements This research was supported by grant SGS16/254/OHK2/3T/16 "Experimental research of driver fatigue by means of ob- servation visual behavior". References [1] A. Wheaton, R. Shults, D. Chapman, et al. Drowsy Driving and Risk Behaviors – 10 States and Puerto Rico, 2011-2012. Morbidity and Mortality Weekly Report, Vol. 63, No. 26. July 4, 2014. [2] AAA Foundation for Traffic Safety. Acute Sleep Deprivation and Risk of Motor Vehicle Crash Involvement, 2016. [3] Transport Accident Commission. Fatigue Statistics. http://www.tac.vic.gov.au/road-safety/ statistics/summaries/fatigue-statistics. [4] National Sleep Foundation. 2000 Facts and Stats. http://drowsydriving.org/about/facts-and-stats. [5] U. D. of Transpotation. NHTSA.Guidelines to Observe And Estimate Statewide Seat Belt Use at Night. 2010. DOT HS 811 288. [6] U. D. of Transpotation. NHTSA.Guidelines to Observe And Estimate Statewide Seat Belt Use at Night. 1999. DOT HS 808. [7] R. Phillips. A review of definitions of fatigue – And a step towards a whole definition. Transportation Research Part F 29 (2015) 48-56. [8] R. Soames-Job, J. Dalziel. Defining fatigue as a condition of the organism and distinguishing it from habituation, adaptation and boredom. In P. A. Hancock & P. A. Desmond (Eds.), Stress, workload and fatigue. NY: CRC Press. 2000. [9] M. Villamizar, J. Suarez, J. Villanueva, et al. Design and implementation of sleep monitoring system using electrooculographs signals. 9th Pan American Health Care Exchanges Conference, PAHCE 2014. [10] N. R.-I. nez, P. Meca-Calderón, M. García-González, et al. Drowsiness detection by electrooculogram signal analysis in driving simulator conditions for gold standard signal generation. International Conference on Biomedical Electronics and Devices, BIODEVICES 2013. Barcelona, Spain. [11] R. Coetzer, G. Hancke. Eye detection for a real-time vehicle driver fatigue monitoring system. 2011 IEEE Intelligent Vehicles Symposium, IV’11; Baden-Baden, Germany. [12] K. Anjali, K. Athiramol, V. Athira, et al. Time Nonintrusive Monitoring and Detection of Eye Blinking in view of Accident Prevention due to Drowsiness. International Conference on Circuit, Power and Computing Technologies, 2016. [13] A. Rahman, M. Sirshar, A. Khan. Real time drowsiness detection using eye blink monitoring. National Software Engineering Conference (NSEC), 2015. [14] J. Ahmed, L. Jian-Ping, K. Saeed, A. Riaz. Eye behavior based drowsiness detection system. 12th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP). 2015. [15] G. Usha, B. Radhika, T. Shanmuganathan. EEG-based brain wave sensor to detect drowsiness with eye open. Far East Journal of Electronics and Communications. Volume 16, March, 2016. [16] R. Foong, K. Ang, C. Quek, et al. An analysis on driver drowsiness based on reaction time and EEG band power. 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC, 2015. [17] T. Brown, R. Johnson, G. Milavetz. Identifying periods of drowsy driving using EEG. 57th Annual Scientific Conference of the Association for the Advancement of Automotive Medicine, Canada, 2013. [18] M. Jiřina, P. Bouchner, S. Novotný. Identification of driver’s drowsiness using driving information and eeg. Neural Network World, 2010. [19] J. Faber, J. Pěkný, R. Pieknik, et al. Simultaneous recording of electric and metabolic brain activity. Neural Network World, 2010. [20] R. V. Loon, R. Brouwer, M. Martens. Drowsy drivers’ under-performance in lateral control: How much is too much? Using an integrated measure of lateral control to quantify safe lateral driving. Accident Analysis and Prevention. Volume 84, 2015. [21] S. James, B. Vila. Police drowsy driving: Predicting fatigue-related performance decay. Policing, Volume 38, Issue 3, 2015. [22] M. Ingre, T. Akerstedt, B. Peters, et al. Subjective sleepiness, simulated driving performance and blink duration. Journal of Sleep Research. Volume 15, Issue 1, 2006. [23] A. Anund, C. Fors, D. Hallvig, et al. Observer rated sleepiness and real road driving: an explorative study. PLOS One, 2013. [24] S. Novotný, P. Bouchner, J. Pěkný, et al. Hybrid mirrors for driving simulators – design, construction and experiments. In: COMPUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS and CYBERNETICS. Athens: WSEAS Press, 2007, pp. 310-315. ISSN 1790-5117. ISBN 978-960-6766-21-3. [25] A. Mashko, P. Bouchner. Visual Investigation of Driver Behavior for Fatigue Detection – Driving Simulator Experiments. Recent Advances on Systems, Signals, Control, Communications and Computers. 2015. [26] C. Ahlstrom, C. Fors, A. Anund. Eur. Transp. Res. Rev. (2015) 7:38. Doi:10.1007/s12544-015-0188-y. 67 http://www.tac.vic.gov.au/road-safety/statistics/summaries/fatigue-statistics http://www.tac.vic.gov.au/road-safety/statistics/summaries/fatigue-statistics http://drowsydriving.org/about/facts-and-stats Acta Polytechnica CTU Proceedings 12:64–67, 2017 1 Introduction 2 Definition of fatigued and drowsy state 3 Subjective drowsiness and fatigue 4 Subjective evaluation during experiments on driver simulator 5 Conclusion Acknowledgements References