MEV J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 Journal of Mechatronics, Electrical Power, and Vehicular Technology e-ISSN:2088-6985 p-ISSN: 2087-3379 www.mevjournal.com © 2016 RCEPM - LIPI All rights reserved. Open access under CC BY-NC-SA license. doi: 10.14203/j.mev.2016.v7.35-48. Accreditation Number: (LIPI) 633/AU/P2MI-LIPI/03/2015 and (Ministry of RTHE) 1/E/KPT/2015. REVIEW ON THE APPLICATION OF PHYSIOLOGICAL AND BIOMECHANICAL MEASUREMENT METHODS IN DRIVING FATIGUE DETECTION Kadek Heri Sanjaya a,*, Soomin Lee b, Tetsuo Katsuura c aResearch Centre for Electrical Power and Mechatronics, Indonesian Institute of Sciences 2nd Floor, 20th Building, Komplek LIPI, Jalan Cisitu No. 21/154D, Bandung, Indonesia bCenter for Environment, Health, and Field Sciences, Chiba University 6-2-1 Kashiwa-no-ha, Kashiwa, Chiba Prefecture 277-0882, Japan cHumanomics Laboratory, Graduate School of Engineering, Chiba University 1-33 Yayoi-cho, Inage-ku, Chiba-shi, Japan Received 23 March 2016; received in revised form 30 May 2016; accepted 30 May 2016 Published online 29 July 2016 Abstract Previous studies have identified driving fatigue as the main cause of road traffic accidents, therefore, the aim of this literature review is to explore the characteristics of driving fatigue both physically and mentally as well as to explore the technology available to measure the process of fatigue physiologically. We performed e-searching in the field of fatigue detection methods through keywords tracking. The instruments studied have their own strength and weakness, and some are intrusive while the others are non-intrusive. The accuracy and stability of measurements are also varied between those instruments. In order to create more reliable fatigue detection methods, it is necessary to involve more instruments with an inter-disciplinary approach. Our intention is to make this study as a stepping stone for a more comprehensive in-vehicle real-time man-machine interaction study. Such study will not only be useful to prevent traffic accidents but also to bridge man and machine communication in the vehicle control along with developing newer technology in the field of vehicle automation. Keywords: driving fatigue; physiology; biomechanics; man-machine interface. I. INTRODUCTION One characteristic of modern life in humans is the high mobility as our houses are mostly far from our workplaces. The higher income in developed countries came from greater productivity which in turn requires people to move faster and farther. Thus, transportation has become an important aspect of human daily life. The present generation has been lucky to inherit a great leap in technology development since James Watt’s invention of the steam engine in the 18th century which triggered the industrial revolution [1]. Technology allows humans to move more efficiently without relying too much on biological muscles for transportation. A developing country like Indonesia has undergone a dramatic change in human mobility. The country’s ambition of progressing from a traditional agricultural society where most people live in the rural area to become an industrialized country has created staggering urbanization problem [2]. Unlike in the rural area where agricultural workers have their activity nearby their living place, in the urban area most people live far away from their workplace, therefore, the need for transportation has become very crucial. Thus, transportation has been a priority in the development program in Indonesia [3]. By taking the long history of human evolution into account, driving a vehicle which is a more advanced development of tool culture is a new thing, therefore, how well the human body adapt to this relatively new activity has been drawing a great amount of research interest. One most important issue of engine-propulsion locomotion such as vehicle transportation is a large number of accidents. WHO reported that the number of * Corresponding Author.Tel: +62-22-2503055 E-mail: kade001@lipi.go.id http://dx.doi.org/10.14203/j.mev.2016.v7.35-48 K.H.Sanjaya et al. / J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 36 traffic accident-related death in the world in annually is more than 1 million people with the highest rate occurred in developing countries [4- 5]. Road traffic accidents have cost every country the loss of between 1 to 3% of gross domestic product (GDP) [5]. As shown in Figure 1, Indonesia has higher road traffic-related death rate than its neighbors. A report suggested that traffic accidents are the third cause of fatalities after myocardial infarction and tuberculosis [6]. Most of the victims are between 22 to 50 years old which is in productive age, thus, those accidents may also lead to poverty [6]. One of the major cause of road traffic accidents is driving fatigue [7-11]. Fatigue has various symptoms and causes. The causes may come from activities before driving such as sleep deprivation [10-14], during driving in longer duration [7,15] and driving at nocturnal time [12], or environmental effects such as vibration [17,18], temperature, humidity and noise [14]. The symptoms of fatigue are both physical and mental. Physical fatigue has been heavily linked with muscular fatigue when muscle shows a decrease of capacity to generate maximal force exertion [9,19-21] due to repetitive movements with excessive load [18]. Some studies distinguish visual fatigue and mental fatigue, as visual fatigue especially occurs in the eye [15], and old opinion with dualism concept tend to separate physical and mental processes of fatigue [19]. While monism view, which is widely adopted today, presumes that the universe consists of matter and energy, and that mind is an inseparable thing from the body [19]. Fatigue is often accompanied by drowsiness [9,13,24-28], thus, drowsiness is often used as a variable in driving fatigue detection. Despite great interest from many researchers to study fatigue, there is still no consensus on its definition that is widely accepted [24, 29]. The driving fatigue study has been widely recognized as important with regard to the effort to cut the number of traffic accidents. Most of the earlier studies on driving fatigue detection gave little attention to human as an organism as they heavily focused on the engineering point of view. Nevertheless, the technology development has allowed us to better understand physiological processes inside a human body based on clinical measurements [30-31]. We consider fatigue as a physiological phenomenon with some biomechanical characteristics which is measurable with techniques now available. The study of human physiology has involved multidisciplinary approach as it discusses the complex system inside a human body [23, 32]. Fatigue is also observable through human motion and forces exertion, therefore, a biomechanical approach is also necessary to understand fatigue, as biomechanics is the discipline that concerns about human motion and structure [33-35]. The aim of this literature review is to explore physiological and biomechanical measurement methods available for application in driving fatigue detection studies. We expect that the knowledge attained in this review will be useful as a base for further studies such as to develop instruments and procedure for fatigue detection and intervention and their in-vehicle application. Figure 1. The number of road fatalities per 100,000 inhabitants (upper) and total road fatalities in 2012 (lower) [4]. South-east Asia has the highest number of road fatalities with Indonesia as the country with most road fatalities in ASEAN region K.H.Sanjaya et al. /J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 37 II. METHOD In this section, we describe the literature searching methods and the systematic of the discussion section. We explain the process taken from literature searching to article writing. We expect the process to enable this article to find opportunities and challenges in the driving fatigue research more throughly. A. Literature Searching Process We performed literature searching through e- searching, which is more helpful than conventional printed book-searching as the latter is more time-consuming and may miss significant information on the references such as citation number [29]. We used a new approach as described in a previous study [30], which is different from the conventional searching, we limited the searching on formal articles, by using certain keywords. The keywords used were driving, fatigue, electromyogram (EMG), electroencephalogram (EEG), electrocardiogram (ECG), and eye tracker. We limited the literature searching with two categories: the latest studies, such as in the last two years, and classical studies with a large number of citation. In an effort to describe driving fatigue from an interdisciplinary point of view, it is necessary to include as many approaches as possible, especially based on their methods [31]. Therefore, we grouped the collected references based on their measurement methods. We performed reference list checking [29], however, we did not conduct any quantitative assessment on the references. B. Writing Systematics In principle, we intend to use this literature review to study the state of the art of driving fatigue detection, to gather the knowledge of whatever other researchers have done in the related field, and whatever problem necessary to solve in the future research [31]. With a great understanding of the research gap in existence, we can formulate a good planning for future study. In the discussion part, we start by describing the physiological processes of fatigue. In turn, we will relate these processes to the instruments required to measure those phenomena. Basically, the instruments record signals generated by the body during fatigue. These electrical data can give information on the fatigued driver condition both real-time or non-real-time. In a literature review, such as a review of methods like this article, in the last part, it is necessary to summarize and synthesize our analyses [32]. In this part, we summarize the current technology available for driving fatigue detection and the possibility of future development of driving fatigue study. III. RESULTS AND DISCUSSION The driving tasks are divided into two, namely primary and secondary tasks. The primary task is the driving the vehicle itself [40-41], which includes activities such as holding the steering wheel, pushing the gas pedal, and braking. The secondary tasks are varied, such as looking at indicators, using entertainment facilities such as audio-video, adjusting air-conditioner, etc. Some studies suggest that the greater automation in a vehicle will likely lead the driver to get more engagement with the secondary task [40-42]. While the primary task is the main causes of both physical and mental fatigue during driving, the secondary task has been associated with mental fatigue [10, 29]. A. Fatigue Occurrence in Driving a Vehicle The easiest way to explain driving fatigue occurs is by exposing the driving activities. Upon entering a driving workstation, the driver adjusts the seat place to make sure that all the controllers for the primary task are reachable. A well- adjusted seating posture may prevent a driver from fatigue [36]. To judge whether the position is optimum, the most common sensory involved are visual sensory [10, 23, 44] to search the controllers, somatosensory [23, 45] to confirm the controllers are reachable, and to certain degree vestibular sensory [23, 46-47] to judge the good balance of the posture taken. All the information from those sensory receptors is processed at the brain to make the driver aware of his position. When the driver turns the vehicle on, the engine emits sound and vibration. Then the sound and vibration are transmitted through the structure of the vehicle and received by auditory sensory and somatosensory [48-49]. By pushing the gas pedal, the driver will have an interaction with the vehicle, and any action taken on the pedal gives feedback which in turn received by sensory receptors, followed by processing the information in the brain. Thus, a communication loop exists. The driver performs visual scanning on the surrounding environment before moves the vehicle. Once the driver concludes that it is safe, the driver moves the gear stick and pushes the gas pedal, then the vehicle is accelerating. While driving, the driver receives the exact information on the speed from the speedometer, however, the K.H.Sanjaya et al. / J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 38 visual information of how the vehicle moves about its surrounding augmented by feedback from the sound and vibration also give significant cues for him [43]. It is common that the driver will have his attention mainly in front of the vehicle, however, inattentiveness to the surrounding area other than in front may increase the risk of accident, thus, they need to dedicate some attention to avoid side or rear collision [42]. Most fatal accidents occur during speeding. Driving a vehicle in a high-speed means the driver has the shorter reaction time to react, thus, higher alertness is important [10, 49-52]. Figure 2 shows what May and Baldwin (2009) suggested on the classification on driving fatigue based on its causes [22]. They divided fatigue as sleep-related (SR) and task-related (TR) fatigue. SR fatigue includes lack of sleeping before driving [10, 12-14, 53] and driving at the time in circadian rhythm/daily life cycle when people usually sleep, such as midnight [15-16, 53]. The driving task and environmental condition are the causes of TR fatigue. The environmental aspect includes temperature, humidity, noise [14], illumination [22], and vibration [17, 54-55]. A study reported that excessive noise increases fatigue occurrence probability as measured by cardiovascular and hormonal activity analysis [49]. Fatigue and sensory alteration are also affected by vibration inside the vehicle [17, 55]. TR active fatigue is the most common fatigue suffered by the drivers. Mental and physical overload during driving such as prolonged driving for more than two or three hours [7, 9, 15], driving in traffic jam, multi-tasking driving, driving in poor visibility environment [22], or overload in mental processing task due to excessive information [9, 57] are the causes of TR active fatigue. TR passive fatigue is caused by underload due to vehicle automation, monotony [25, 58], or highly predictable driving task [22]. As shown by the dashed line in Figure 2, both TR active and passive fatigue can worsen SR fatigue. Fatigue occurrence is either physically or mentally or both in a concurrent way. Physical fatigue has been associated with the decreasing capacity to exert maximum force output [9,19,59- 60], as there is a failure of processes either within the central nervous system (CNS), neural transmission from the CNS to the muscle, or within individual muscle fibers [55]. Physical fatigue is mainly included in TR active fatigue such as prolonged sustained physical activity with force exertion [61-62] or higher frequency of cyclic activity in the longer period [57]. Some studies distinguished mental fatigue and visual fatigue [9,64], however, in reality, these two things are inseparable. In this article, we consider visual fatigue as part of mental fatigue as some studies referred mental fatigue as a consequence of visual fatigue [7, 10, 13, 52, 65]. Mental fatigue can be inflicted by both mental and physical processes [9, 29], and classified in both TR active and passive fatigue. A fatigued driver shows many different characteristics from a non-fatigued person. Some of the visible characteristics are body movement [5,9,12,24,25,51,66], facial expression [5,8,12,26] and sweating [37], whereas the invisible ones related to physiological processes inside the body, such as muscular activity [9,18], heart rate [15,16,44,67] and blood circulation [68-69], brain activity [9,13-14,29], and hormonal changes [20,51] or psychological condition [23]. In many cases, fatigue is accompanied by drowsiness which is commonly characterized by frequent yawning [9,15,24-27]. B. Muscular Fatigue All the physical working during driving are performed with muscular activation that produces some movements. These movements are varied in requirements of force, speed, and vectorial translation. The movement is initiated in the smallest functional unit of the muscle called as the motor unit [33, 70-71]. When a muscle contracts, the central nervous system (CNS) recruits motor units as a voluntary effort [61-62, 70]. Increased tension to generate force output is achieved by two strategies: increasing stimulation rate of a single motor unit or recruitment of more motor units [27]. Muscle fiber, just like any neuron inside a human body maintains negative potential at about 80 mV from its surrounding by exchanging ions [70]. This potential is called motor unit action potential (MUAP) and recorded as representative of muscle activation [33, 71-72]. This electrical indication is then measured by Figure 2. Driving fatigue classification by May and Baldwin (2009), based on its causes [28] K.H.Sanjaya et al. /J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 39 surface electromyogram (sEMG), as shown in Figure 3. Two most important features for sEMG signal detection are excitation of MUAPs and their firing rate [68]. Muscular fatigue, in general, is included in TR active fatigue. Based on processed sEMG signals, there are two ways to analyze muscle fatigue, namely spectral analysis as shown in Figure 4, and EMG/force relation analysis [33, 70-73]. Fatigue muscle is caused by depletion of energy generating substances, accumulation of metabolic waste product, failure in muscular contractile mechanism from CNS to the muscle, and disturbances in homeostasis [54] since a contracting muscle restricts blood flow, thus reduces oxygen supply [65]. Figure 5 shows that when a person exerts a constant force in a certain period of time, in power-frequency diagram, the frequency decreases since the biochemical process results in the acidic products formation which reduces the conduction velocity of the MUAPs on the muscle membrane [68]. Recruitment of more MUAPs also means EMG activity-force production level correlation line would also shift toward the left side since fatigued muscle has its force output decreasing while EMG activity is increasing, as graphically represented in Figure 7 [59,70,74]. Median frequency (MDF) has been suggested as the most common data used for muscle fatigue index calculation, as illustrated in Figure 5 [70- 71]. Other suggested mean power frequency (MPF) as the most useful tool to quantify muscle fatigue, as illustrated in Figure 6 [59,74]. The measurement of power-frequency relation usually performed at certain time points in a session, such as the start, middle, and end of the session, in off-line mode. EMG-force relation requires the use of force measurement instruments such as load cell or force plate. This method has greater potential than frequency analysis for real-time fatigue detection analysis, as long as there is sufficient information on the normal driver and fatigue driver characteristics, as inter-subject variability is common in EMG measurement. Based on this type of measurement, muscle fatigue is identified as a two phase-process, the first when EMG- force in linear relation and the second when the force decreases faster than the EMG signals, as graphically illustrated in Figure 7 [53]. Moshou et al. (2005) proposed to use of wavelets coefficients analysis to detect dynamic muscle fatigue [73]. Wavelets is very common used in analyzing signals that are time varying and non-stationary. They developed self- organizing map (SOM) to identify and measure driving fatigue. The SOM works by dynamically visualizing the approximate values of the wavelet coefficients. This method offers advantages in sensitivity to detect fatigue and recovery. However, this method needs further development to be widely accepted. In most studies on driver fatigue detection, EMG has rarely been given attention. There are various reasons. EMG electrode needs to be attached to the skin, thus, it is naturally intrusive [11] and requires an understanding of which muscles would fatigue in certain movements since fatigue is different in each contracting muscle. The attachment also Figure 3. sEMG Ag/AgCl bipolar electrode (upper) and sEMg electrodes attachment onto the skin surface to measure back and abdominal muscles activation (lower) Figure 4. Spectral analysis of sEMG data: vertical axis shows magnitude (v2/Hz), horizontal axis shows frequency (Hz). Total power is the area under the power spectrum curve, peak power is the maximum value, mean power frequency (MPF) is the average of power spectrum, and median frequency (MDF) is the line that divides the area into two equal parts [68] K.H.Sanjaya et al. / J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 40 requires hair removal and skin cleansing [68] and sweating as well as subcutaneous fat will decrease the measurement accuracy [74]. With the knowledge of the current technology, it seems difficult for EMG to be used for real-time in- vehicle fatigue detector. Nevertheless, EMG is a very powerful and accurate tool to measure fatigue for laboratory experiment, especially to learn about neuromuscular processes in relation with reaction time study. C. Fatigue Detected from Heart Rate The most important function of the heart is to pump the blood through the blood vessels in the circulatory system, where the blood supply the body with oxygen and nutrients as well as playing an important role in assisting metabolic waste removal [75]. Since the three substances are closely associated with the occurrence of fatigue, the heart is very crucial in fatigue study. Heart rate (HR) is defined as the number of beats produced by the heart in one minute, while heart rate variability (HRV) is a terminology to describe the physiological phenomenon where temporal intervals between consecutive heartbeats are not uniform, as illustrated in Figure 8 [79, 80]. Fatigue has been associated with electrolyte and acid-base disturbances that affect HR which is measured by electrocardiogram (ECG) [17]. While extreme exercise increases HR [78] and excessive fatigue after arousal may result in exhaustion, this case is rarely observed in driving fatigue. Therefore, most driving fatigue studies employing ECG are focused on drowsiness as the consequence of fatigue. Drowsiness, on the Figure 7. Fatigue condition is characterized by lower force exertion level at the same muscle activation. Here fatigue condition is compared with three types of muscle activation (concentric, static/isometric, and eccentric), as well as with the subjects considered outside normal people, such as trained (athlete) and atrophy (muscularly weak person) [68] Figure 6. During a muscle activation trial session, the MPF value was lower in fatigue condition both by comparing dynamic movement and static isometric posture [76] Figure 5. As the muscle contracts in longer duration, at the end session of the trial, the MDF value is lower compared to that at the middle and the beginning of the trial [71] K.H.Sanjaya et al. /J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 41 Figure 9. HRV of a driver who falls asleep (upper) and who yawns often (lower). In sleepy subjects, HRV is increasing [82]. Further spectral analyses arenecessary to detect drowsiness more accurately contrary, decreases HR [67]. Thus, we may conclude that SR fatigue decreases HR whereas TR active fatigue increases HR. Figure 9 shows the increasing HRV due to drowsiness [79]. On the other hand, excessive fatigue after arousal decreases HRV [46]. Autonomic nervous system (ANS) whose role is regulating unconscious control system of the internal organ such as the heart rate, changes during stress, fatigue or drowsiness [16-27]. ANS consists of sympathetic and parasympathetic nervous systems. HRV is representative of the balance between the two nervous systems [10]. Fatigue or drowsiness have been reported to be characterized by greater parasympathetic nervous activity accompanied by a decrease in sympathetic nervous activity [14, 16, 27]. With regard to HRV, drowsiness is shown by faster HRV rhythm or high-frequency band (HF, 0.15 to 0.4 Hz), on the contrary to slower HRV rhythm or low-frequency band (LF, 0.04 to 0.15 Hz) which indicates alertness and wakefulness [10]. In general, a decrease in LF and LF/HF value which is associated with sympathetic nervous system activity[80], has been used for indicators of fatigue [14, 27]. ECG measurement has been reported to the most reliable for fatigue detection [5, 67, 86]. However, as it requires electrode attachment to the skin, it is intrusive [5, 67], hence, it is uncomfortable to be used during driving. ECG is also largely affected by environmental variables such as vibration and temperature [14]. To overcome these weaknesses, a non-contact HR and HRV measurement using photoplethysmogram (PPG) has been proposed [18, 69]. PPG is a dedicated light source equipped with light emitting diode (LED) and phototransistor (PT). The light from the LED to the skin is being reflected back to the PT, and the recorded signal is then analyzed [14]. PPG may be placed on areas such as the finger tip or the forehead. The instrument is able to detect any blood pressure changes in the blood vessels on the face or other parts of the body that is not obstructed. Another method proposed is by placing ECG electrodes on the seat and the steering wheel, and then the less accurate data from those electrodes are combined to produce reliable data [64]. D. Fatigue Detection with EEG Fatigue, drowsiness and low vigilance have been reported as the greatest safety hazard in driving [23]. Fatigue condition inhibits the nervous system and decreases cognitive abilities [15, 44]. Mental fatigue, as one component of fatigue, is a subjective feeling of tiredness which causing the decreasing of motivation to make an effort for certain activity [23, 59], due to sustained mental and/or physical task which requires high alertness [20] or boring task [23], which impairs attention. The occurrence of an error during performing a task also increases the likeliness of mental fatigue as measured by reaction time [44]. The easiest and cheapest way to measure mental task load is a questionnaire, such as NASA task load index (NASA-TLX) [82] or Karolinska Sleepiness Scale (KSS) [46], however, it cannot be applied for real-time measurement, and subjects need to be isolated from any psychological noise that may deviate assessment [82]. The subjective scale also needs a larger number of subjects to validate the results. Therefore, EEG which measure brainwave is considered as more accurate and objective to measure mental fatigue in real-time [9], as it is sensitive enough to detect any changes with an update in seconds [23]. For fatigue detection, EEG stability measurement was reported to achieve 85% to 87%, greater than eye tracking, Figure 8. ECG electrodes placement on the trunk. ECG normal sinus rhythm generates P wave, QRS complex, and T wave [91]. Heart rate (HR) is calculated from the number of P or R in one minute. P-P or R-R intervals are representative of heart rate variability (HRV) [76, 92] K.H.Sanjaya et al. / J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 42 but lower than ECG [5, 53]. During fatigue, EEG power decreases into low-frequency band [9]. EEG is able to measure both SR and TR fatigue. Figure 10 shows EEG electrodes placement and the types of brainwaves used in fatigue analysis. The rise of spectral power in frontal theta (theta wave, 4-<8 Hz [83]) and parietal alpha (alpha wave, 8-13 Hz [11, 87]) have been linked with mental fatigue [23]. Other study suggest that decrease of beta wave (13-30 Hz [83]) in the prefrontal, inferior frontal, posterior temporal, and occipital lobes accompanied by increase of alpha wave in the frontal, central, posterior temporal, parietal, and occipital lobes area of the brain as indicators of mental fatigue [9]. Alpha wave is the most common variable used to detect fatigue as it is considered to be the most reliable [9], furthermore alpha wave measured from the occipital and central areas are indicators of drowsiness [8]. Indices used to measure mental fatigue include the comparison between the summation of alpha and theta to beta [15], even-related potential (ERP) [82], reaction time [60], and Task Load Index (TLI) which is based on the ratio of frontal midline theta to parietal alpha [82]. EEG is an intrusive instrument, and to attach its electrodes on the head, some cleaning-up procedure is necessary, especially to reduce impedance. The risk of signal noise and sensor failure during measurement are high [23]. Furthermore, while EEG can detect drowsiness easily, to use EEG for mental fatigue detection in a fully alert driver is still a great challenge as fatigue is a complex and non-linear condition [29,51]. Thus, to develop a model that is more representative to mental fatigue by involving EEG combined with other methods is necessary for future study. E. Fatigue Detection through Facial Expression Non-contact fatigue detection methods include video analysis of facial expression [5, 9, 26, 89], such as head [5], mouth [15, 24, 26], and eye movements [7, 58, 90]. Behavioral studies also observe variables like facial activities, postural adjustments from original position, and hand movements relative to the body [45]. Since these methods measure attention, fatigue is not separated from drowsiness as both produce hypo- vigilace. The study on cortisol, which is the stress hormone, found the association of fatigue and drowsiness [13, 25]. In general, facial expression- related fatigue detection methods are mostly intended for SR fatigue. The most important feature of head movement in 3D motion tracking is the orientation of the face as it signifies attention [26, 89]. Normal driver in full alertness has his face oriented frontally with a good view on important indicators through his peripheral vision. A fatigued driver, on the contrary, will likely to have more head movements either flexion- extension or rotation. Yawning, which is characterized by a wider mouth opening and deep inhale, has been used as both drowsiness and fatigue indicator in many studies [21]. Fatigue detection through lip movement possesses so many challenges, such as to distinguish yawning from normal speaking [21] and to distinguish yawning due to fatigue or drowsiness from contagious yawning which associated with the mirror-neuron system [51]. To overcome these problems, one study suggested the identification based on concurrent stretching or extension of the neck [51], the other study suggested by calculating the time of mouth opening [63]. Eye movement is the most common facial expression employed for fatigue detection since it reflects the neurological condition [12, 46].While human vision affects both mental and physical fatigue development, a stress developed in the eyes to some researchers is classified as visual Figure 10. The International 10-20 system of EEG electrodes attachment on the scalp (upper)[87]. A= ear lobe, C= central, F= frontal, Fp= frontal polar, O= occipital, P= parietal, T= temporal. The three types of brainwaves commonly used for fatigue analysis: transition from beta-wave to alpha-wave shows drowsiness and transition from alpha wave to theta-wave shows microsleeps (lower) [53, 87] K.H.Sanjaya et al. /J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 43 fatigue [9, 64], despite it refers to the site of fatigue, and not to the process. Measured eye movement components are duration and frequency of eye blink/closure [7,11,15,24], and saccadic movements: dynamic, glissadic, and static movements [85]. Those components are measured by tracking and detection of pupil [65, 91, 92] or iris [85] movements. Other method is intrusive: the use of electrooculogram (EOG) with electrodes should be attached to the eyelid skin [81]. Fatigue is detected from increasing blink rate and decreasing of saccadic peak velocity/magnitude [7, 90]. A continuous detection of pupil movements allows researchers to measure eyelid closure/opening, gaze, and face orientation [62]. Eye closure is measured from pupil size and face orientation is measured by comparisons on the two pupils based on calculation of distance and size ratio [62]. Percentage of eye closure over time (PERCLOS) [8, 11, 53] and the average of eye closure speed (AECS) [8, 53] have been suggested as the most representative eye movement parameters in detecting fatigue. Eye tracker instrument has advantages in that it is non-intrusive, therefore, more comfortable to the user, and the user can mount it in various areas inside a vehicle for real-time detection. However, compared to ECG or EEG, it is less accurate [46], and less stable, with the rate of just 59% as compared to 85% and 97.5% of EEG and ECG respectively [5]. This device is only able to measure a person with normal or corrected-to- normal vision without any saccadic abnormality. The use of infra-red for illumination of eye tracker device has been suggested as it is able to detect eye movement in a various light background and almost invisible to the driver’s eyes [86]. F. Combining the measurements In general, based on their contact with the human body, the instruments can be classified into intrusive and non-intrusive methods [5]. As shown in Figure 11, intrusive methods include EMG, EEG, and ECG, whereas non-intrusive methods include facial and eye tracking. Force measurement instruments can be non-intrusive if placed under the seating cover, or intrusive such as pressure sensors attached to the finger tip. Such condition is also applied for PPG, as it is contactless if used for forehead scanning, whereas it is intrusive if placed on the fingertip as in conventional plethysmogram. Intrusive methods are generally more accurate, however very susceptible to physiological noise such as sweating, or especially EMG the noise usually comes from subcutaneous fat. These methods are very good for a laboratory experiment. To be applied in a field test, a great challenge comes from material and placement design with regard to the human body. ECG application requires fatigue threshold definition since SR and TR fatigues have very contradictory characteristics. Non-intrusive methods such as eye and body motion tracker have challenges in accuracy and reduction of environmental noise effects. Each method has its own advantages and disadvantages, therefore combining those methods will result in a more reliable measurement [11, 14, 18, 53] as fatigue has non- specific symptoms. For example eye movement measurement with an eye tracker should be accompanied by EMG recording of oculomotor muscles, and either yawning or facial expression tracking should be accompanied by EMG recording of facial muscles. In accordance with the technology development, some disadvantages of those instruments may be overcome by newer inventions either in materials or techniques. The different characteristics will also enable designers to create more alternatives for both real-time and non-real-time fatigue detection methods. Furthermore, future studies on fatigue detection methods should not be limited only to the above- mentioned instruments. The inclusion of instruments such as force plate or pressure sensor [93-94], as fatigue is also associated with vestibular disturbance [39], or actiwatch to measure daily activity levels [90], may be necessary. With monism principle is being taken into account, we can argue that physical fatigue may induce mental fatigue and vice versa. The combination of the physical and mental condition is probably the reason of great inter-subject variability commonly observed in fatigue study. Thus, to understand the inter-correlation between various aspects of fatigue is probably the greatest challenge in driving fatigue study. This is Figure 11. Summary of driving fatigue detection methodsand the type of fatigue detected by the relevant methods K.H.Sanjaya et al. / J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 44 especially important in combining the measurement methods and data analyses. IV. CONCLUSION In this article, we describe the most common fatigue detection methods using physiological and biomechanical measurement instruments.The combination of various instruments and methods may result in more reliable measurements to uncover many aspects of driving fatigue, as one instrument limitations may be covered by other instruments. On the other hand, such approach requires a better understanding of human body characteristics other than good knowledge of the measurement instruments. Thus, a multi- disciplinary approach is a must for this kind of research. Putting present and future problems of man- machine interaction studies into perspective, the interaction will always generate fatigue, however, the balance between physical and mental fatigue may vary. On the contrary to the traditional assumption of mind and body dualism, which see those two entities as separated each other, both physical and mental fatigue are imposed in one human body and both interact each other. Despite some assumption that future transportation will likely be less dependent on the human driver and become more autonomous, the study on driver fatigue will still be crucial in transportation ergonomics either in the future. The first reason is that the transition from human- driver into an autonomous vehicle will take a longer time since the entire supporting system for such vehicle should be provided before its widespread operation. The second reason is that any activity will always produce fatigue, with changing proportion of physical and mental fatigue. The autonomous vehicle is probably physically less fatiguing, however, it will likely to induce greater mental fatigue with the more complex learning process and decision making. The third reason is that the measurement systems that involve digital signal will allow researchers to develop various ways of human interaction with the vehicle with regard to control and feedback, based on the signal processing. Hence, this kind of study should be relevant in the foreseeable future along with vehicle technology development. REFERENCES [1] B. Z. Khan, “Knowledge, Human Capital, and Economic Development: Evidence From the British Industrial Revolution, 1750-1930,” 2015. [2] G. W. Jones, Contemporary Demographic Transformations in China, India and Indonesia. Cham: Springer International Publishing, 2016. [3] “Prioritas pembangunan nasional.” [Online]. Available: http://www.bappenas.go.id/files/8213/502 7/5942/bab-i-prioritas-pembangunan- nasional.pdf. [Accessed: 25-Feb-2016]. [4] World Health Organization, “Global Status Report on Road Safety,” Geneva, 2015. [5] K. T. Chui et al., “Electrocardiogram based classifier for driver drowsiness detection,” in Proceeding - 2015 IEEE International Conference on Industrial Informatics, INDIN 2015, 2015, pp. 600– 603. [6] “Kecelakaan Lalu Lintas Menjadi Pembunuh Terbesar Ketiga | BADAN INTELIJEN NEGARA REPUBLIK INDONESIA.” [Online]. Available: http://www.bin.go.id/awas/detil/197/4/21/ 03/2013/kecelakaan-lalu-lintas- pembunuh-terbesar-ketiga. [Accessed: 25- Feb-2016]. [7] L. L. Di Stasi et al., “Effects of driving time on microsaccadic dynamics,” Exp. Brain Res., vol. 233, no. 2, pp. 599–605, 2015. [8] M. Rost et al., “Comparing Contribution of Algorithm Based Physiological Indicators for Characterisation of Driver Drowsiness,” J. Med. Bioeng., vol. 4, no. 5, pp. 391–398, 2015. [9] X. Fan et al., “Electroencephalogram assessment of mental fatigue in visual search,” Biomed. Mater. Eng., vol. 26, no. 37, pp. 1455–1463, 2015. [10] E. Michail et al., “EEG and HRV markers of sleepiness and loss of control during car driving,” in Engineering in Medicine and Biology Society, 2008. EMBS 2008. 30th Annual International Conference of the IEEE, 2008, pp. 2566–2569. [11] W. Sun et al., “Blink Number Forecasting Based on Improved Bayesian Fusion Algorithm for Fatigue Driving Detection,” Math. Probl. Eng., pp. 1–13, 2015. [12] J. Vicente et al., “Drowsiness detection using heart rate variability,” Med. Biol. Eng. Comput., 2016. [13] K. Lienhard et al., “sEMG during whole- body vibration contains motion artifacts and reflex activity,” J. Sport. Sci. Med., vol. 14, no. 1, pp. 54–61, 2014. [14] J. H. Ju et al., “Real-Time Driver ’ s K.H.Sanjaya et al. /J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 45 Biological Signal Monitoring System,” Sensors Mater., vol. 27, no. 1, pp. 51–59, 2015. [15] T. Selvan N and R. Shanmugalakshmi, “A review on driver fatigue detection system,” Int. J. Innov. Sci. Appl. Eng. Res., vol. 13, no. 4, pp. 29–33, 2015. [16] E. Rashedi and M. A. Nussbaum, “A review of occupationally-relevant models of localised muscle fatigue,” Int. J. Hum. Factors Model. Simul., vol. 5, no. 1, pp. 61–80, 2015. [17] C. D. Marlin, “The Physiology of Fatigue in Horses During Exercise,” Time, 2007. [18] D.-H. Kang et al., “Effects of low- frequency electrical stimulation on cumulative fatigue and muscle tone of the erector spinae.,” J. Phys. Ther. Sci., vol. 27, no. 1, pp. 105–8, 2015. [19] N. R. Carlson, Physiology of behavior. New York: Allyn&Bacon, 2010. [20] S. B. N. Thompson, “Yawning, fatigue, and cortisol: Expanding the Thompson Cortisol Hypothesis,” Med. Hypotheses, vol. 83, no. 4, pp. 494–496, 2014. [21] P. N. Lahange et al., “Driver ’ s Face Monitoring System for Det ecting Hypo- Vigilance : A Review,” Int. J. Res., vol. 2, no. 4, pp. 553–559, 2015. [22] J. F. May and C. L. Baldwin, “Driver fatigue: The importance of identifying causal factors of fatigue when considering detection and countermeasure technologies,” Transp. Res. Part F Traffic Psychol. Behav., vol. 12, no. 3, pp. 218– 224, 2009. [23] L. J. Trejo et al., “EEG-Based Estimation and Classification of Mental Fatigue,” Psychology, vol. 06, no. 05, pp. 572–589, 2015. [24] M. Sato, “The Development of Conceptual Framework in Physiological Anthropology,” J. Physiol. Anthropol. Appl. Human Sci., vol. 24, no. 4, pp. 289– 295, 2005. [25] K. Iwanaga, “The biological aspects of physiological anthropology with reference to its five keywords.,” J. Physiol. Anthropol. Appl. Human Sci., vol. 24, pp. 231–235, 2005. [26] A. Schmidt, “Biosignals in human- computer interaction,” interactions, vol. 23, no. 1, pp. 76–79, Dec. 2015. [27] D. A. Winter, Biomechanics and Motor Control of Human Movement. John Wiley & Sons, 2009. [28] B. Rosenhahn et al., “Human Motion - Understanding, Modelling, Capture, and Animation,” Image (Rochester, N.Y.), p. 635, 2008. [29] D. Papaioannou et al., “Literature searching for social science systematic reviews: Consideration of a range of search techniques,” Health Info. Libr. J., vol. 27, no. 2, pp. 114–122, 2010. [30] M. Boeker et al., “Literature search methodology for systematic reviews: Conventional and natural language processing enabled methods are complementary (Letter commenting on: J Clin Epidemiol. 2015;68:191-9),” J. Clin. Epidemiol., vol. 69, pp. 255–257, 2016. [31] A. Skene, “Writing a Literature Review,” Toronto, 2012. [32] University of North Carolina, “Literature reviews.,” Chapel Hill, 2014. [33] T. A. Dingus et al., “Driver crash risk factors and prevalence evaluation using naturalistic driving data,” pp. 1–7, 2016. [34] H. J. Oh et al., “Effects of Superimposition of a Head-Up Display on Driving Performance and Glance Behavior in the Elderly,” Int. J. Hum. Comput. Interact., vol. 32, no. 2, pp. 143– 154, Oct. 2015. [35] F. Naujoks et al., “Secondary task engagement and vehicle automation – Comparing the effects of different automation levels in an on-road experiment,” Transp. Res. Part F Traffic Psychol. Behav., vol. 38, pp. 67–82, Apr. 2016. [36] K. M. Gruevski et al., “Lumbar postures, seat interface pressures and discomfort responses to a novel thoracic support for police officers during prolonged simulated driving exposures,” Appl. Ergon., vol. 52, pp. 160–168, 2016. [37] X. Wu et al., “Pilot’s visual attention allocation modeling under fatigue,” Technol. Heal. Care, vol. 23, no. s2, pp. S373–S381, 2015. [38] A. G. Richardson et al., “The effects of acute cortical somatosensory deafferentation on grip force control,” Cortex, vol. 74, pp. 1–8, 2016. [39] G. Y. Menshikova et al., “Eye Movements as Indicators of Vestibular Dysfunction,” Perception, vol. 0, no. 0, pp. 1–7, 2015. [40] C. Diels and J. E. Bos, “Self-driving carsickness,” Appl. Ergon., vol. 53, pp. 374–382, 2015. [41] C. Ho and C. Spence, “Affective K.H.Sanjaya et al. / J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 46 multisensory driver interface design,” Int. J. Veh. Noise Vib., vol. 9, no. 1–2, pp. 61– 74, 2013. [42] A. Lundkvist and A. Nykänen, “Response Times for Visual, Auditory and Vibrotactile Directional Cues in Driver Assistance Systems,” SAE Int. J. Transp. Saf., vol. 4, no. 1, Apr. 2016. [43] T. Hamaguchi et al., “Estimation of Driver Workload Based on the Model of Accelerator Pedal Control While Controlling Vehicle Speed,” SAE Tech. Pap., no. 2016–01–1412, Apr. 2016. [44] Y. Xiao et al., “Sustained Attention is Associated with Error Processing Impairment: Evidence from Mental Fatigue Study in Four-Choice Reaction Time Task.,” PLoS One, vol. 10, no. 3, p. e0117837, 2015. [45] A. Vakulin et al., “Behavioural Observation as a Means of Assessing Sleepiness Related Driving Impairment in Obstructive Sleep Apnoea,” Eat, Sleep, Work, vol. 1, 2015. [46] Y. Dong et al., “Driver inattention monitoring system for intelligent vehicles: A review,” IEEE Trans. Intell. Transp. Syst., vol. 12, no. 2, pp. 596–614, 2011. [47] K. Lienhard and S. S. Colson, “Relationship between lower limb muscle activity and platform acceleration during whole-body vibration exercise,” no. October, 2015. [48] N. Van der Stoep et al., “Multisensory interactions in the depth plane in front and rear space: A review,” Neuropsychologia, vol. 70, no. December, pp. 335–349, 2015. [49] S. Lee et al., “Effects of Active Noise Control on Physiological Functions,” J. Human-Environment Syst., vol. 12, no. 2, pp. 49–54, 2009. [50] H. M. Abd-Elfattah et al., “Physical and cognitive consequences of fatigue: A review,” J. Adv. Res., vol. 6, no. 3, pp. 351–358, 2015. [51] S. B. N. Thompson and S. Richer, “How Yawning and Cortisol Regulates the Attentional Network,” J. Neurosci. Rehabil., vol. 2, no. 1, pp. 1–9, 2015. [52] C. Ahlstrom et al., “Fit-for-duty test for estimation of drivers’ sleepiness level: Eye movements improve the sleep/wake predictor,” Transp. Res. Part C Emerg. Technol., vol. 26, pp. 20–32, 2013. [53] M. Gonzalez-Izal et al., “Electromyographic models to assess muscle fatigue,” J. Electromyogr. Kinesiol., vol. 22, no. 4, pp. 501–512, 2012. [54] D. Marlin, “The physiology of fatigue in the horse during exercise: What is fatigue? What causes it? How can you recognise and manage fatigue?” [Online]. Available: http://davidmarlin.co.uk/portfolio/the- physiology-of-fatigue-in-the-horse- during-exercise/. [Accessed: 17-Mar- 2016]. [55] J. Taylor et al., “Neural contributions to muscle fatigue: from the brain to the muscle and back again,” J. Med. Sci. Sport Exerc., 2016. [56] C. Froyd et al., “The development of peripheral fatigue and short-term recovery during self-paced high-intensity exercise,” J Physiol, vol. 591, no. 5, pp. 1339–1346, 2013. [57] D. Srinivasan et al., “Effects of concurrent physical and cognitive demands on muscle activity and heart rate variability in a repetitive upper-extremity precision task,” Eur. J. Appl. Physiol., vol. 116, no. 1, pp. 227–239, 2016. [58] C. Chen et al., “Visual fatigue caused by watching 3DTV : an fMRI study,” Biomed. Eng. Online, vol. 14, no. Suppl 1, p. S12, 2015. [59] J. F. Hopstaken et al., “A multifaceted investigation of the link between mental fatigue and task disengagement,” Psychophysiology, vol. 52, no. 3, pp. 305– 315, 2015. [60] S. Zhao, “Study on Driver Model Parameters Distribution for Fatigue Driving Levels Based on Quantum Genetic Algorithm,” Open Cybern. Syst. J., vol. 9, pp. 1559–1566, 2015. [61] C. Solomon and Z. Wang, “Driver Attention and Behavior Detection with Kinect,” J. Image Graph., vol. 3, no. 2, pp. 84–89, 2015. [62] J. Paone et al., “Baseline face detection, head pose estimation, and coarse direction detection for facial data in the SHRP2 naturalistic driving study,” in IEEE Intelligent Vehicles Symposium, Proceedings, 2015, vol. 2015-Augus, pp. 174–179. [63] C. K. Zope and Y. C. Kulkarni, “Driver Drowsiness Detection by Driver Aided System using Smartphone,” Int. J. IT, Eng. Appl. Sci. Res., vol. 4, no. 6, pp. 4–7, 2015. K.H.Sanjaya et al. /J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 47 [64] A. Koenig et al., “Statistical sensor fusion of ECG data using automotive-grade sensors,” Adv. Radio Sci., vol. 13, pp. 197–202, 2015. [65] N. Munk et al., “Noninvasively measuring the hemodynamic effects of massage on skeletal muscle: A novel hybrid near- infrared diffuse optical instrument,” J. Bodyw. Mov. Ther., vol. 16, no. 1, pp. 22– 28, 2012. [66] H. Qi et al., “Non-contact driver cardiac physiological monitoring using video data,” in IEEE China Summit and International Conference on nal and Information Processing, 2015, pp. 418– 422. [67] R. Roy and K. Venkatasubramanian, “EKG/ECG based driver alert system for long haul drive,” Indian J. Sci. Technol., vol. 8, no. 19, pp. 8–13, 2015. [68] P. Konrad, The ABC of EMG: a practical introduction to kinesiological electromyography, no. 1. Scottsdale, AZ: Noraxon, 2005. [69] C. J. De Luca, “The Use of Surface Electromyography in Biomechanics,” J. Appl. Biomech., vol. 13, no. 2, pp. 135– 163, 1997. [70] G. E. Loeb and C. Gans, Electromyography for Experimentalists. University of Chicago Press, 1986. [71] L. A. C. Kallenberg and H. J. Hermens, “Behaviour of a surface EMG based measure for motor control: Motor unit action potential rate in relation to force and muscle fatigue,” J. Electromyogr. Kinesiol., vol. 18, no. 5, pp. 780–788, 2008. [72] S. Thongpanja et al., “Mean and Median Frequency of EMG Signal to Determine Muscle Force Based on Time- Dependent Power Spectrum,” Elektron. IR Elektrotechnika, vol. 19, no. 3, pp. 51–56, 2013. [73] D. Moshou et al., “Dynamic muscle fatigue detection using self-organizing maps,” Appl. Soft Comput., vol. 5, no. 4, pp. 391–398, 2005. [74] A. F. T. Ibrahim et al., “Analysis of Electromyography (EMG) Signal for Human Arm Muscle: A Review,” in Advanced Computer and Communication Engineering Technology, Cham: Springer International Publishing, 2016, pp. 567– 575. [75] M. Chappell and S. Payne, Physiology for Engineers, vol. 13. Cham: Springer International Publishing, 2016. [76] D. A. Dimitriev et al., “State Anxiety and Nonlinear Dynamics of Heart Rate Variability in Students,” PLoS One, vol. 11, no. 1, p. e0146131, 2016. [77] A. Davies and A. Scott, “ECG Basics,” in Starting to Read ECGs: The Basics, Cambridge, UK: Springer International Publishing, 2014, pp. 19–33. [78] A. D. Elliott and A. La Gerche, “The right ventricle following prolonged endurance exercise: are we overlooking the more important side of the heart? A meta- analysis.,” Br. J. Sports Med., no. June 2016, pp. 1–6, 2014. [79] E. Rogado et al., “Driver fatigue detection system,” 2008 IEEE Int. Conf. Robot. Biomimetics, ROBIO 2008, no. July 2015, pp. 1105–1110, 2008. [80] F. Chouchou et al., “Cardiac sympathetic modulation in response to apneas/hypopneas through heart rate variability analysis.,” PLoS One, vol. 9, no. 1, p. e86434, 2014. [81] L. M. B. H. De Rosario, J.S. Solaz, N. Rodrı́guez, “Controlled inducement and measurement of drowsiness in a driving simulator,” in IET Intelligent Transport system, 2010, pp. 280–288. [82] G. Matthews et al., “The Psychometrics of Mental Workload: Multiple Measures Are Sensitive but Divergent,” Hum. Factors J. Hum. Factors Ergon. Soc., vol. 57, no. 1, pp. 125–143, 2015. [83] Donald Scott, Understanding EEG, First Edit. Glasgow: Duckworth, 1976. [84] A. Dasgupta et al., “A vision-based system for monitoring the loss of attention in automotive drivers,” IEEE Trans. Intell. Transp. Syst., vol. 14, no. 4, pp. 1825– 1838, 2013. [85] A. T. Bahill et al., “Dynamic overshoot in saccadic eye movements is caused by neurological control signed reversals.,” Exp. Neurol., vol. 48, no. 1, pp. 107–122, 1975. [86] Q. Ji et al., “Real-time nonintrusive monitoring and prediction of driver fatigue,” IEEE Trans. Veh. Technol., vol. 53, no. 4, pp. 1052–1068, 2004. [87] Y. Lin et al., “Eye Movement and Pupil Size Constriction Under Discomfort Glare,” Invest. Ophthalmol. Vis. Sci., vol. 56, no. 3, pp. 1649–1656, 2015. K.H.Sanjaya et al. / J. Mechatron. Electr. Power Veh. Technol. 07 (2016) 35-48 48 [88] K. Sanjaya et al., “The effects of different trunk inclinations on bilateral trunk muscular activities, centre of pressure and force exertions in static pushing postures,” J. Hum. Ergol. (Tokyo)., vol. 43, pp. 9–26, 2014. [89] K. H. Sanjaya et al., “The influence of laterality on different patterns of asymmetrical foot,” J. Hum. Ergol. (Tokyo)., vol. 43, pp. 77–92, 2014. [90] J. Townhill et al., “Using Actiwatch to monitor circadian rhythm disturbance in Huntington’ disease: A cautionary note,” J. Neurosci. Methods, pp. 1–6, 2016. [91] “ECG: Cardiology | Heart Sounds | Research | BIOPAC.” [Online]. Available: https://www.biopac.com/application/ecg- cardiology/advanced-feature/heart- sounds/. [Accessed: 24-May-2016]. [92] M. G. Khan, “Step-by-Step Method for Accurate Electrocardiogram Interpretation,” in Contemporary Cardiology: Rapid ECG Interpretation, 3rd ed., 2008, pp. 25–80.