Acta Polytechnica CTU Proceedings https://doi.org/10.14311/APP.2022.39.0023 Acta Polytechnica CTU Proceedings 39:23–31, 2022 © 2022 The Author(s). Licensed under a CC-BY 4.0 licence Published by the Czech Technical University in Prague BRAIN DATA AND DRIVER’S ATTENTION DURING SIMULATED DRIVE Roman Mouček∗, Petr Brůha, Pavel Šnejdar University of West Bohemia, Faculty of Applied Sciences, Department of Computer Science and Engineering, Technická 8, 306 14 Plzeň, Czech Republic ∗ corresponding author: moucek@kiv.zcu.cz Abstract. The attention of drivers is a serious issue and one of the critical factors of road safety. The question is whether the electrical activity of the human brain can be correctly measured/collected and utilized to monitor and interpret the driver’s attention during simulated driving. This article summarizes four experiments that have been designed, performed, and evaluated in the neuroinformatics laboratory at the University of West Bohemia. Simulated driving under various conditions in a car simulator was organized, and electrophysiology, mainly electroencephalography, data were collected from participants/drivers. The results include experience with the design of such experiments and the suitability of methods based on the collection and interpretation of electroencephalography data for driver attention detection. Keywords: Attention of drivers, electroencephalography, event-related potentials, simulated driving, P300 component. 1. Introduction The attention of drivers is a serious issue and one of the critical factors of road safety. Inattentive drivers are dangerous not only to themselves but mainly to their surroundings and cause many accidents. However, the decline of attention, especially during long rides, is natural, and it is necessary to take this phenomenon into account. This paper summarizes some of the experiments that have been designed, performed, and evaluated in the neuroinformatics laboratory at the University of West Bohemia. Simulated driving under various conditions in a car simulator was organized, and electrophysiology (mainly electroencephalogra- phy, event-related potential) data were collected from participants/drivers. Electroencephalography (EEG) is a method and technique that records and evaluates the electrical activity of the (human) brain. A set of electrodes is placed at various (often predefined) locations on the scalp. These electrodes capture voltages (resulting from the firing of large neural circuits) that must be amplified and recorded. Event-related potentials (ERPs) are changes in the electrical brain activity that are time-locked to particular events (stimuli). ERPs are extracted from the underlying EEG data. The main research objective of the presented set of four experiments was to find and verify if the electrical activity of the human brain (EEG/ERP data) could be correctly measured/collected and utilized to monitor and interpret the driver’s attention during simulated driving. The paper is organized as follows. The materials and methods section deals with the fundamentals of EEG/ERP methods and their benefits and drawbacks in investigating driver’s attention. It further intro- duces the neuroinformatics lab at the University of West Bohemia and summarizes four performed ex- periments. The results section provides the results and conclusions from individual experiments. The conclusion section summarizes experience from these experiments and gives several cautious prospects for the future. 2. Materials and methods This section provides a fundamental insight into the advantages and disadvantages of collecting, monitor- ing and analyzing the electrical activity of the human brain using the methods and techniques of electroen- cephalography and event-related potentials when inter- preting the driver’s attention during simulated driving. Then the neuroinformatics lab at the University of West Bohemia is introduced, and the description of four experiments dealing with driver’s attention and using the EEG/ERP methods is provided. 2.1. Electroencephalography and event-related potentials Electroencephalography has many advantages: afford- ability, non-invasiveness, routine examination proto- cols, and the opportunity to measure spontaneous ac- tivity. However, it also has a significant disadvantage, evident in scientific experiments. The resulting pic- ture of brain activity (the EEG signal) is rough since it represents many sources of neuronal activity. Thus it is challenging to derive individual neurocognitive processes from the measured brain activity. Another disadvantage is the relatively difficult collection of a quality EEG signal outside the laboratory since only observable and clean data can be reasonably analyzed 23 https://doi.org/10.14311/APP.2022.39.0023 https://creativecommons.org/licenses/by/4.0/ https://www.cvut.cz/en R. Mouček, P. Brůha, P. Šnejdar Acta Polytechnica CTU Proceedings to interpret the driver’s attention during simulated driving. ERPs have two advantages compared to behavioral methods. They help determine which stage or stages of processing are influenced by a given experimental manipulation; a detailed set of examples can be seen, e.g. in [1]. The second advantage of ERPs is that they can provide an online measure of stimuli processing, even when there is no behavioral response [2]. ERP components are obtained from the EEG sig- nal by averaging the epochs around the events. The P300 (also labelled as P3) component depends en- tirely on the task performed by the subject and is not directly influenced by the physical properties of the stimulus [1]. The P300 component is sensitive to the probability of the target stimulus. P300 amplitude increases when the probability of the target stimulus class decreases. The amplitude of the P300 component becomes larger when a greater number of non-target stimuli precedes it. P300 amplitude is also larger when the subject pays more attention to a task. On the other hand, P300 amplitude is smaller if the subject does not know whether a given stimulus is/is not a target. It means that more complex tasks can increase P300 amplitude because the subject pays more attention to these tasks and simultaneously decrease P300 amplitude because the subject is not certain of the stimulus category [1]. P300 latency is associated with stimulus catego- rization; it increases when stimulus categorization is postponed. Since P300 latency does not depend on consequent processes (e.g. response selection, in our case, the driver’s reaction), it can be used to determine if a performed experiment influences the processes of stimulus categorization or other processes related to a response (driver’s reaction). The experimental design of EEG/ERP experiments is a challenging and often critical step that influences other technical issues related to EEG/ERP research. An apparent and detailed description of ERP exper- imental design as a set of rules and strategies (e.g. focusing on specific, large, and easily isolated compo- nents or comparing only ERPs elicited by the same physical stimuli) is also provided in [1]. Although these rules and strategies constrain the experimental design, they are not a standard part of the software tools used to design ERP experiments, and they are of- ten violated in many published studies. Consequently, it was not easy to get inspired by the published stud- ies when we designed the first experiments related to drivers’ attention. 2.2. Related studies While most studies examining the attention of drivers use behavioral methods (i.e. the behavior of the driver under certain conditions is investigated – e.g., how of- ten he/she leaves his/her lane), not many experiments also use EEG and ERP methods and techniques. The suitability of EEG-based techniques for record- ing drivers’ activity during simulated driving was in- vestigated in [3] where oscillations of brain electrical activity (frequency bands) were analyzed. As a result, an increase in alpha activity was interpreted as less attentional activity and a decrease in alpha activity as more attentional activity. Significant differences between drivers were observed. The ERP method (P300 amplitude) is utilized in [4] where the impact of the secondary task performance (an oddball auditory task) on a primary driving task (lane keeping) was investigated. The study showed that when performing a simple secondary task during driving, the performance of the driving task and this secondary task were unaffected. However, analysis of brain activity showed reduced cortical processing of irrelevant and potentially distracting stimuli from the secondary task during driving. The use of EEG data for the evaluation of driver fatigue was provided in [5]. Energy parameters were computed, and finally, the evaluation model for driver fatigue was created based on the EEG data from the electrodes Fp1 and O1. Muscle artifacts were minimized by the experimental protocol design, and all trials showing artifacts linked to eye movements or blinks were removed before averaging ERPs. The model accuracy was about 92.3 %. The impact of a surrogate forward collision warning system and its reliability according to the driver’s at- tentional state was introduced in [6]. Both behavioral and electrophysiological (amplitudes and latencies of several ERP components) data were recorded and evaluated. These results showed electrophysiological data as a valuable tool to complement behavioral data allowing a better understanding of how these systems impact the driver. Using the ERP technique, it was found in [7] that the brain activity associated with processing the in- formation necessary for the safe operation of a motor vehicle was suppressed when drivers were talking on a cell phone. The P300 amplitude was reduced, and the P300 latency was delayed when participants were engaged in phone conversations and reacted to inter- mittent lead vehicle deceleration. The effect of regular and prior sleep restricted to five hours during simulated driving was studied in [8]. The study was carried out on 20 younger and 19 older healthy men. After a short sleep, younger drivers showed significantly more sleep-related deviations and greater 4 to 11 Hz EEG power, indicative of sleepi- ness [8]. A systematic framework for measuring and under- standing cognitive distraction in three experiment settings (laboratory control, driving simulator, and instrumented vehicle) was presented in [9]. Partici- pants completed eight tasks commonly performed by the driver; primary, secondary, subjective, and physio- logical measures (the P300 component) were collected and integrated into a cognitive distraction scale. They 24 vol. 39/2022 Brain Data and Driver’s Attention during Simulated Drive concluded that impairments in driving were directly related to the cognitive workload of these activities. Simultaneous recording of EEG and eye-tracking to investigate situation awareness and working mem- ory load in distracted driving was introduced in [10]. The driver-vehicle-environment state-space related to drivers’ conditions and environmental factors around the vehicle were described using (also) an ontology. The amplitude of the P300 component reflecting individual differences in navigation performance in a driving task was investigated in [11]. Two groups of navigators with good and poor navigation perfor- mance participated in a driving task; the P300 ampli- tude was measured while two types of triggers were presented (intersections and street signs). In general, poor navigators showed a larger P300 amplitude than good navigators. To identify EEG oscillatory parameters for cognitive mechanisms involved in high and low controllability tasks, time spent on a task, task load, and cognitive controllability (using either re-active or pro-active driving task) in simulated driving scenarios were in- vestigated in [12]. The results on thirty healthy par- ticipants demonstrated that the controllability of a driving situation had a similar effect on oscillatory EEG activity like time on task and task load. The drivers’ fatigue was measured in [13] where EEG and forehead EOG were fused. The percent- age of eye closure was calculated using eye movement data (recorded by eye tracking glasses as the indicator of drivers’ fatigue level). The prediction correlation coefficient and root mean square error between the estimated and real fatigue levels were used to evaluate the performance of a single modality and fusion modal- ity. The experimental results on twenty-one healthy subjects showed that this fusion could improve the performance of driving fatigue detection. A real-time driving fatigue detection system based on a wireless dry EEG acquisition system was pre- sented in [14]. The prediction of fatigue in ten healthy subjects was consistent with the observation of reac- tion time recorded during simulated driving. The study in [15] concluded that the EEG signal from the prefrontal brain region (forehead area) could be used to detect fatigue of drivers, although the signal classification accuracy (the SVM algorithm used for classification) was not high. A survey on EEG-based driver state detection sys- tems and their corresponding analysis algorithms over the last three decades was provided in [16]. The au- thors concluded that the current EEG-based driver state monitoring algorithms were promising for safety applications, but many improvements were still re- quired in EEG artefact reduction, real-time processing, and between-subject classification accuracy. A two-level learning hierarchy radial basis function model was used to detect EEG-based driving fatigue in [17]. The authors concluded that the proposed model achieved a better classification performance compared to other used artificial neural networks. Multi-channel EEG recordings during a sustained- attention driving task were presented in [18]. They included 62 sessions of 32-channel EEG data for 27 subjects driving on a four-lane highway. A recurrent residual network used to analyze EEG data captured during simulated sustained attention driving tasks was presented in [19]. The authors demonstrated the competitive results achieved by com- paring this network with other benchmark models. An attention-based multiscale convolutional neu- ral network with a dynamical graph convolutional network proposed to detect driving fatigue was in- troduced in [20]. The authors concluded that their model outperformed six widely used competitive EEG models with a high accuracy of 95.65 %. An EEG-Based Spatio–Temporal Convolutional Neural Network for Driver Fatigue Evaluation was introduced in [21]. Fatigue driving experiments were conducted to collect EEG signals from eight subjects alert and in fatigue states. The results indicated that the network fulfilled a better classification accuracy of 97.37 % than the eight methods it was compared with. The study [22] investigated whether measures from low-cost devices monitoring peripheral physiological states (eye-tracker, heart rate monitor, and a high- fidelity 32-channel quick-gel EEG system) were compa- rable to standard EEG measures in predicting lapses in attention to system failures. Twenty-five participants were engaged in a fully autonomous lane-changing driving task. The results showed that current low- fidelity technologies were not sensitive enough to reli- ably model reaction time to critical signals. To summarize the related studies section, we can conclude that EEG and ERP methods and techniques are promising for detecting driver fatigue and the im- pact of various disturbing events and tasks on driver’s attention. Although the studies usually work with fatigue, vigilance or cognitive performance, we will use the term attention to align with the fundamental ex- planation of EEG/ERP methods. We were surprised that many studies did not pay too much attention to design issues of such experiments since only a well- prepared design can lead to relatively clean data and reasonable interpretation. 2.3. Neuroinformatics laboratory equipment The neuroinformatics laboratory at the University of West Bohemia is equipped with all necessary hardware infrastructure for driving simulations and recording EEG/ERP signals and additional biosignals. The experimental car simulator (a front part of a real Škoda Octavia car, Figure 1) is equipped with the Logitech G27 wheel, accelerator, and brake. These are connected to the control computer via the USB port. The tracks are prepared mainly using the World Racing 2 game produced by the Synetic Company [23]. 25 R. Mouček, P. Brůha, P. Šnejdar Acta Polytechnica CTU Proceedings Figure 1. Car simulator. The track is projected on the wall in front of the car simulator. The stimuli were presented either using a custom programmable hardware stimulator (for visual stimu- lation with LEDs) placed on the car windshield or the Presentation software tool (for auditory stimulation) produced by Neurobehavioral Systems, Inc. [24]. V-Amp produced by the Brain Products company was used as an EEG amplifier. It also served as an input of the sensors (also produced by the Brain Products company) capturing other biosignals. The BrainVision Recorder [25] was used for recording and local storing EEG/ERP and other biosignal data (e.g. respiration rate or skin conductance). MATLAB, EEGLAB, ERPLAB, BrainVision Analyzer software tools, and Python tools (especially the MNE library) were used to process and analyze experimental data. Standard EEG caps (the 10-20 system defining lo- cations of scalp electrodes) were used depending on the size of the participants’ heads. The reference electrode was placed approximately 0.5-1 cm above the nose, and the ground electrode was placed on the ear. The biosignal sensors and the EEG cap were connected to the V-Amp amplifier. Three computers were usually used: the first for the presentation of auditory stimuli (if they were pre- sented), the second for storing recorded data, and the third for the track presentation. The diagram of the standard experimental setup is depicted in Figure 2. EEG/ERP data were recorded with the sampling frequency 1 kHz; no filters were used during data recording. The resulting signal was stored in three files: .eeg file containing raw data; .vhdr file containing metadata that describe raw data in .eeg file, and .avg file containing the averaged signal around the used stimuli. IIR filter was applied to data from the Fz, Cz and Pz electrodes. These three electrodes were also selected for further processing. 2.4. Methodology – Experiments Four already published EEG/ERP-based experiments investigating driver’s attention during simulated driv- ing are presented to show the evolution of our exper- imental work in EEG-based simulated driving. Al- though we could theoretically include dozens of mostly unpublished experiments in this work, various driving scenarios and overall driving conditions would confuse the article. An appropriate experiment design (including the choice of the event-related component, type of stimuli, driven track, or entire data collected) has to be set and evaluated to achieve research objectives. When observ- able and clean data are collected, it is reasonable to interpret the driver’s attention during simulated driv- ing. The event-related component P300 was selected based on its nature, use in related studies, and the outcomes of our preliminary unpublished experiments. In the first presented experiment, we proposed and tested the constraints related to the complexity of the experimental design. The experiment design was improved in the second presented paper. The third paper introduced more tiresome conditions for par- ticipants; not only a monotonous track but various daytime conditions and sleep deprivation were part of the experiment protocol. The last experiment used two signals collected in parallel to detect the driver’s attention during a simulated drive – brain electrical activity and respiration. EEG/ERP data were recorded with the sampling frequency of 1 kHz; no filters were used during data recording. The resulting signal was stored in three files: .eeg file containing raw data, .vhdr file containing metadata that describe raw data in the .eeg file, and .avg file containing the averaged signal around the used stimuli. Various IIR filters were applied to data from the Fz, Cz and Pz electrodes in the processing phase, typically in the frequency range 0.01-20 Hz. The Fz, Cz and Pz electrodes were also selected for further processing. The segmented data containing artifacts were first automatically and then manually rejected. 2.4.1. Experiment 1 The main objective of this work [26] was to deter- mine if EEG/ERP methods and techniques and the event-related P300 component itself can be success- fully utilized to investigate how the mental load of drivers affects their attention during a simulated drive. The second objective was to test the constraints of the experimental protocol complexity. A relatively challenging virtual track containing sharp turns and allowing for potential collisions with oncoming cars (target stimuli) was designed and tested on two groups (nine volunteers) of drivers (af- fected/unaffected by alcohol). The occurrence of the P300 component was investigated. Then, an alterna- tive protocol relying on simpler target stimuli (flashing 26 vol. 39/2022 Brain Data and Driver’s Attention during Simulated Drive Figure 2. Experimental setup for simulated driving. diodes and simple sounds) was proposed, implemented, and tested. 2.4.2. Experiment 2 This paper’s [27] objective was to detect changes in a driver’s attention during a monotonous simulated drive. It was done using EEG/ERP techniques, per- forming auditory stimulation, and investigating prop- erties of the P300 component related to attention. The ERP experiment was designed to investigate the driver’s attention changes during monotonous 40 minutes long simulated drive. The experiment was performed on 14 participants who drove a car sim- ulator on a monotonous track (a motorway without disturbing obstacles). The participants were audi- bly stimulated (simple beeps of various duration and the sound of a crying child). The data were prepro- cessed (filtering, artefact rejection, epoch extraction, and baseline correction) and investigated for the P300 component’s occurrence and peak latency in subse- quent eight-minute intervals. 2.4.3. Experiment 3 The objective of the experiment [28] was to introduce more tiresome conditions for participants. Not only a monotonous track but also various daytime and sleep deprivation were part of the experiment proto- col to detect changes in the driver’s attention during a monotonous simulated drive. EEG and ERP tech- niques and auditory stimulation were utilized; changes in the peak latency of the P300 component over time were investigated. The same hypothesis as in the previous experiments (the peak latency of the P300 component increases over time as the driver becomes more tired) was considered, but the experiment design differed and evolved in time. A monotonous track was constructed in this ex- periment, and a simple auditory experiment was de- signed (three stimuli – simple target and non-target stimuli of 500 ms duration time and different frequen- cies, a rare stimulus of 1000 ms duration time). Each participant (11 participants in total) underwent four twenty-minute drives held over two days; each day, morning and afternoon sessions were held; on the 27 R. Mouček, P. Brůha, P. Šnejdar Acta Polytechnica CTU Proceedings first day, the participant usually slept, and on the second day, he/she suffered from sleep deprivation. Each drive was divided into time intervals of the same length (the stimulation was provided only in some time intervals to ensure relaxation time and prevent participants from familiarity with the presented stim- uli). The latencies of the averaged P300 components were compared depending on the daytime and sleep deprivation. 2.4.4. Experiment 4 The paper’s [29] objective was to consider two bio- signals collected in parallel to detect the driver’s at- tention during a simulated drive; these bio-signals were the electrical activity of the human brain (as in the previous experiments) and respiration. Moreover, the collected data were validated using a stacked au- toencoder to guarantee their quality and annotated to make them publicly available and reusable. The experiment was built on the previous experi- ments, considering the basic assumptions related to the P300 component and driving a car simulator on a monotonous track. The experiment (simulated drive) was prolonged to 60 minutes and divided into three sessions. Auditory stimulation was performed (target and non-target stimuli of the same length), and the background sound of drizzling imitated the natural environment. The respiration rate was captured par- allel with the brain’s electric activity using the same hardware amplifier. The data from a group of 15 par- ticipants were annotated, stored, preprocessed, vali- dated, and partly analyzed (classified using a stacked autoencoder). 3. Results The first experiment [26] brought significant results – the proposed track appeared too complex to provide experimenters with the observable occurrence of the P300 component, but its simplified alternative version was more promising. The P300 component was not observed in the grand average waveforms when the more complex target stimuli (sharp turns and possible collisions) were part of the virtual track. A similar trend of the grand average waveforms for both groups of subjects (affected/unaffected by alcohol) was ob- served. When target stimuli were elicited by external hardware devices (flashing diodes or sound genera- tors), the P300 component was easily observable. It was shown that the experiment design had to be relatively simple; otherwise, the event-related P300 component could not be identified. It is necessary to be sure if and when the P300 component is elicited. The subsequent experiments thus followed a more simple design. The experiment design was improved in the sec- ond driving experiment [27]. As a result, the P300 component was easily observable in the grand average waveforms. However, both variants of the experiment design (using a different location of the reference elec- trode) did not show gradually increasing peak latency concerning the length of the simulated drive. There were 30 % of artifacts in the epochs related to target stimuli. The target stimulus (a 600 ms long sound of a crying child) was too long and finally made the P300 component less analyzable. Selecting a long target stimulus (a crying child) was probably not a good design solution since it stretched the P300 component length. Stretched components are less analyzable; their absolute coordinates are distorted. There were also not many target stimuli, and because of the removal of the artifacts, only 70 % of them entered the averaging process. The experi- ment’s prolongation and introducing more tiresome conditions (e.g., exposing participants to heat, ab- sence of sleep, or placing them in a darkened room) were further considered. The P300 component was clearly identified in all experimental sessions of the third experiment [28]. Figure 3 shows the grand averages on the electrode Fz (frontal central electrode) for all experimental ses- sions. On the other hand, the prolongation of the peak latency of the P300 component overtime was not observable when the grand average measure of each participant was investigated. However, the prolonga- tion of the peak latency of the P300 component was observable when the techniques of peak latency and fractional 50 % area latency were applied to compute the grand average for each experimental session. The latency of the P300 component was not influenced by the time of day. The latency of the P300 component increased with sleep deprivation. The experiment showed that the P300 component’s prolongation was identifiable when the techniques of peak latency and fractional 50 % area latency were applied. We concluded that for the following experi- ments, it would be beneficial to have a larger number of target stimuli (i.e., the simulated drive has to be prolonged) and more participants. In the fourth experiment [29], the P300 component was identified in most participants during all driving sessions. A high amplitude of the N2 component, when compared to the amplitude of the P300 com- ponent, was detected in some cases. Prolongation of the peak latency of the P300 component was evident in most participants when their peak latencies were averaged. However, prolongation of peak latency in time was not clearly observed when grand averages for all participants were investigated. Average respira- tion rate and respiration rates for most participants showed a decreasing trend during the experiment. In this experiment, the trend of the increasing la- tency of the P300 component and decreasing respi- ratory rate was relatively clearly visible (Note: the data were not statistically evaluated). Since the ex- perimental results were naturally affected (e.g., by different brain reactions of participated drivers, the sensibility of captured data to the environmental noise, 28 vol. 39/2022 Brain Data and Driver’s Attention during Simulated Drive −100 0 100 200 300 400 500 600 700 800 900 −15 −10 −5 0 5 10 15 Time [ms] Vo lta ge [µ V ] morning + usual sleep afternoon + usual sleep morning + sleep deprivation afternoon + sleep deprivation Figure 3. Grand average on the electrode Fz – experimental sessions differ in the daytime and duration of sleep [28]. or participants’ overall mental conditions), there was a significant effort to eliminate these circumstances by experiment design, setting of experimental condi- tions, and appropriate use of data preprocessing and processing methods. Moreover, the data were finally validated using the stacked autoencoder. 4. Conclusions Various experiment protocols were designed (dozens of experimental protocols were tested, and four pub- lished ones were presented in this paper) to set and verify constraints significant for the successful investi- gation of drivers’ attention utilizing the human brain’s electrical activity and the EEG/ERP methods and techniques. The initial experiments were over-specified; the ex- periment protocols were too complicated, naive, and unrealistic, given the method used. Over time, the experiment design was improved; the event-related P300 component was finally easily identifiable. The interesting results regarding the latency of the P300 component were observed, e.g., its increase with over- all fatigue or sleep deprivation of drivers was detected. On the other hand, we had to cope with the trou- bles related to participants’ willingness to wear an EEG cap for a longer time, handle a longer drive in the simulator, or reduce unnecessary movements. The number of produced artifacts finally put time constraints on the designed experiments. All the work related to the collection, long-term storage, processing, and interpretation of EEG/ERP data/metadata and publication of results also revealed the troubles with the data/metadata descriptions, pro- cedures, and reproducibility of the findings. Even reproducing a single experiment by the same experi- menter proved practically impossible in a few months. The need for standardized data annotations and for- mats, procedures, and workflows turned out to be enormous to share the data, procedures, and outcomes with the community and understand data, procedures, and conclusions after a particular time. The investigation of driver’s attention has finally be- come interesting for automotive in-car assistance sys- tems and self-driving car development. The assistance systems that prevent drivers from causing accidents have been improved and can monitor drivers’ behav- ior and vital signs (mainly related to attention) such as movements, eye blinks, and heartbeat. Moreover, self-driving cars are being developed to spare or even prevent people from driving. However, the data collec- tion methods presented here are still complicated and impractical, and the results are not so reliable to be used in cars during actual driving. On the other hand, these methods can be used for verification purposes in laboratory conditions. List of symbols EEG Electroencephalography ERP Event-related potentials P300 P300 component Acknowledgements This work was supported by the University specific re- search project SGS-2019-018 Processing of heterogeneous data and its specialized applications (project SGS-2019- 018). 29 R. Mouček, P. Brůha, P. Šnejdar Acta Polytechnica CTU Proceedings References [1] S. Luck, G. Woodman, E. Vogel. Event-related potential studies of attention. Trends in Cognitive Sciences 4(11):432–440, 2000. https://doi.org/10.1016/S1364-6613(00)01545-X [2] S. Luck. An Introduction to the Event-related Potential Technique. An Introduction to the Event-related Potential Technique. MIT Press, 2005. [3] M. Schier. Changes in EEG alpha power during simulated driving: A demonstration. International Journal of Psychophysiology 37(2):155–162, 2000. https://doi.org/10.1016/S0167-8760(00)00079-9 [4] A. Wester, K. Böcker, E. Volkerts, et al. Event-related potentials and secondary task performance during simulated driving. Accident Analysis & Prevention 40(1):1–7, 2008. https://doi.org/10.1016/j.aap.2007.02.014 [5] W. Li, Q. He, X. Fan, Z. Fei. Evaluation of driver fatigue on two channels of EEG data. Neuroscience Letters 506(2):235–239, 2012. https://doi.org/10.1016/j.neulet.2011.11.014 [6] M. Bueno, C. Fabrigoule, P. Deleurence, et al. An electrophysiological study of the impact of a forward collision warning system in a simulator driving task. Brain Research 1470:69–79, 2012. https://doi.org/10.1016/j.brainres.2012.06.027 [7] D. L. Strayer, F. A. Drews. Multi-tasking in the automobile. In Attention: From theory to practice, pp. 121–133. Oxford University Press New York, NY, 2007. https://doi.org/10.1093/acprof: oso/9780195305722.003.0009 [8] A. J. Filtness, L. A. Reyner, J. A. Horne. Driver sleepiness – Comparisons between young and older men during a monotonous afternoon simulated drive. Biological Psychology 89(3):580–583, 2012. https://doi.org/10.1016/j.biopsycho.2012.01.002 [9] D. L. Strayer, J. Turrill, J. M. Cooper, et al. Assessing cognitive distraction in the automobile. Human Factors 57(8):1300–1324, 2015. https://doi.org/10.1177/0018720815575149 [10] M. Ichiki, G. Ai, H. Wagatsuma. Simultaneous recording of eegs and eye-tracking for investigating situation awareness and working memory load in distracted driving: A prospective analysis toward the neuro-driving framework. Frontiers in Neuroscience 10, 2015. https: //doi.org/10.3389/conf.fnins.2015.91.00010 [11] B. Ou, C. Wu, G. Zhao, J. Wu. P300 amplitude reflects individual differences of navigation performance in a driving task. International Journal of Industrial Ergonomics 42(1):8–16, 2012. https://doi.org/10.1016/j.ergon.2011.11.006 [12] E. Wascher, S. Arnau, I. Gutberlet, et al. Evaluating pro- and re-active driving behavior by means of the EEG. Frontiers in Human Neuroscience 12:205, 2018. https://doi.org/10.3389/fnhum.2018.00205 [13] X.-Q. Huo, W.-L. Zheng, B.-L. Lu. Driving fatigue detection with fusion of EEG and forehead EOG. In 2016 International Joint Conference on Neural Networks (IJCNN), pp. 897–904. 2016. https://doi.org/10.1109/IJCNN.2016.7727294 [14] H. Wang, A. Dragomir, N. I. Abbasi, et al. A novel real-time driving fatigue detection system based on wireless dry EEG. Cognitive neurodynamics 12(4):365–376, 2018. https://doi.org/10.1007/s11571-018-9481-5 [15] Z. Mu, J. Hu, J. Yin. Driving fatigue detecting based on EEG signals of forehead area. International Journal of Pattern Recognition and Artificial Intelligence 31(05):1750011, 2017. https://doi.org/10.1142/S0218001417500112 [16] C. Zhang, A. Eskandarian. A survey and tutorial of EEG-based brain monitoring for driver state analysis, 2020. [2022-01-28], https://arxiv.org/abs/2008.11226. [17] Z. Ren, R. Li, B. Chen, et al. EEG-based driving fatigue detection using a two-level learning hierarchy radial basis function. Frontiers in Neurorobotics 15, 2021. https://doi.org/10.3389/fnbot.2021.618408 [18] Z. Cao, C.-H. Chuang, J.-K. King, C.-T. Lin. Multi- channel EEG recordings during a sustained-attention driving task. Scientific data 6(1):1–8, 2019. https://doi.org/10.1038/s41597-019-0027-4 [19] Y. Ming, Y.-K. Wang, M. Prasad, et al. Sustained attention driving task analysis based on recurrent residual neural network using EEG data. In 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–6. IEEE, 2018. https://doi.org/10.1109/FUZZ-IEEE.2018.8491646 [20] H. Wang, L. Xu, A. Bezerianos, et al. Linking attention-based multiscale CNN with dynamical GCN for driving fatigue detection. IEEE Transactions on Instrumentation and Measurement 70:1–11, 2020. https://doi.org/10.1109/TIM.2020.3047502 [21] Z. Gao, X. Wang, Y. Yang, et al. EEG-based spatio–temporal convolutional neural network for driver fatigue evaluation. IEEE Transactions on Neural Networks and Learning Systems 30:2755–2763, 2019. https://doi.org/10.1109/TNNLS.2018.2886414 [22] D. Cisler, P. M. Greenwood, D. M. Roberts, et al. Comparing the relative strengths of EEG and low-cost physiological devices in modeling attention allocation in semiautonomous vehicles. Frontiers in Human Neuroscience 13:109, 2019. https://doi.org/10.3389/fnhum.2019.00109 [23] SYNETIC GmbH. World racing 2, 2014. [2022-01-28], http://www.synetic.de/. [24] Neuro Behavioral Systems. Home page, 2014. [2022-01-28], http://www.neurobs.com/. [25] Brain Products. Brain vision recorder, 2014. [2022-01-28], http://www.brainproducts.com/ productdetails.php?id=21. [26] R. Mouček, T. Řondík. Influence of mental load on driver’s attention. Transaction on Transport Sciences 5(1):21–26, 2012. https://doi.org/10.2478/v10158-012-0003-6 [27] R. Mouček, J. Řeřicha. Driver’s attention during monotonous driving. In 2012 5th International Conference on BioMedical Engineering and Informatics, pp. 486–490. 2012. https://doi.org/10.1109/BMEI.2012.6513051 30 https://doi.org/10.1016/ S1364-6613(00)01545-X https://doi.org/10.1016/S0167-8760(00)00079-9 https://doi.org/10.1016/j.aap.2007.02.014 https://doi.org/10.1016/j.neulet.2011.11.014 https://doi.org/10.1016/j.brainres.2012.06.027 https://doi.org/10.1093/acprof:oso/9780195305722.003.0009 https://doi.org/10.1093/acprof:oso/9780195305722.003.0009 https://doi.org/10.1016/j.biopsycho.2012.01.002 https://doi.org/10.1177/0018720815575149 https://doi.org/10.3389/conf.fnins.2015.91.00010 https://doi.org/10.3389/conf.fnins.2015.91.00010 https://doi.org/10.1016/j.ergon.2011.11.006 https://doi.org/10.3389/fnhum.2018.00205 https://doi.org/10.1109/IJCNN.2016.7727294 https://doi.org/10.1007/s11571-018-9481-5 https://doi.org/10.1142/S0218001417500112 https://arxiv.org/abs/2008.11226 https://doi.org/10.3389/fnbot.2021.618408 https://doi.org/10.1038/s41597-019-0027-4 https://doi.org/10.1109/FUZZ-IEEE.2018.8491646 https://doi.org/10.1109/TIM.2020.3047502 https://doi.org/10.1109/TNNLS.2018.2886414 https://doi.org/10.3389/fnhum.2019.00109 http://www.synetic.de/ http://www.neurobs.com/ http://www.brainproducts.com/productdetails.php?id=21 http://www.brainproducts.com/productdetails.php?id=21 https://doi.org/10.2478/v10158-012-0003-6 https://doi.org/10.1109/BMEI.2012.6513051 vol. 39/2022 Brain Data and Driver’s Attention during Simulated Drive [28] R. Mouček, V. Košař. Attention of driver during simulated drive. In Proceedings of the International Conference on Health Informatics - HEALTHINF, (BIOSTEC 2014), pp. 543–550. INSTICC, SciTePress, 2014. https://doi.org/10.5220/0004934905430550 [29] R. Mouček, L. Hnojský, L. Vařeka, et al. Experimental design and collection of brain and respiratory data for detection of driver’s attention. In Proceedings of the 10th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5: HEALTHINF, (BIOSTEC 2017), pp. 441–450. INSTICC, SciTePress, 2017. https://doi.org/10.5220/0006249504410450 31 https://doi.org/10.5220/0004934905430550 https://doi.org/10.5220/0006249504410450 Acta Polytechnica CTU Proceedings 39:23–31, 2022 1 Introduction 2 Materials and methods 2.1 Electroencephalography and event-related potentials 2.2 Related studies 2.3 Neuroinformatics laboratory equipment 2.4 Methodology – Experiments 2.4.1 Experiment 1 2.4.2 Experiment 2 2.4.3 Experiment 3 2.4.4 Experiment 4 3 Results 4 Conclusions List of symbols Acknowledgements References