256 Aircraft Pilot Workload Assessment Based on Non-contact Sensors: A Review Lan Yang School of Automation, Nanjing University of Science and Technology, 200Xiaolingwei Street, Nanjing, Jiangsu, China Yanglan0723@njust.edu.cn Abstract. Pilot fatigue and cognitive overload are caused by long flights, irregular circadian rhythms, and complex cockpit environments, posing a significant threat to flight safety. This review explores non-contact sensor technologies used to assess the workload of aircraft pilots to enhance aviation safety and performance. Firstly, the limitations of traditional methods (such as subjective questionnaires (e.g., NASA-TLX, SART) and invasive sensors) were examined, which are affected by subjectivity, latency effects, and operational interference. Secondly, Emphasis is placed on the advantages of non-contact technologies (including computer vision, ballistocardiography (BCG), and infrared thermal imaging (IRT)), which allow for the real-time and objective monitoring of physiological and behavioral indicators such as facial expressions, heart rate variability, and facial temperature changes. Finally, a discussion is presented on future development directions, focusing on the combined use of artificial intelligence, machine learning, and edge computing for developing adaptive cockpit systems, and the requirement for validation in long-haul and extreme scenarios to optimize pilot performance and flight safety. It is provided that theoretical support and specific guidance are essential for the practice of aviation human factors engineering, with the goal of optimizing the aviation human-machine system. Keywords: Flight Safety; Pilot Fatigue; Non-contact Technologies. 1. Introduction The cornerstone of civil aviation safety lies in the situational awareness and operational stability of pilots. However, prolonged flights, cross-time zone travel, and irregular circadian rhythms often lead to pilot fatigue, which significantly exacerbates cognitive decline, slow reactions, and operational errors [1]. Human factors account for 80% of aviation accidents, and pilot errors are identified as the main cause of 53% of catastrophic events [2]. The complexity of modern cockpits, characterized by advanced automation and multi-source information display, further increases cognitive load and often pushes pilots beyond their optimal performance levels [3]. To address these persistent cognitive and human factors challenges, psychological and physiological monitoring technologies have gradually become a key means for assessing pilot workload [4]. Such technologies, by collecting and analyzing physiological signals during task execution, can objectively reflect key indicators such as psychological stress, fatigue level, attention concentration, and emotional state, providing scientific basis for early warning and status intervention [5]. In recent years, with the advancement of sensing technology, non-contact monitoring methods have gained widespread attention due to their advantages of not interfering with pilots' normal operations and being easily integrated into the cockpit environment. For example, facial blood flow analysis based on infrared thermal imaging, eye movement and micro-expression tracking through optical cameras, and heart rate variability captured using ballistic electrocardiogram technology, can all be used to construct dynamic assessment models of pilots' cognitive load [3,6-9]. This review explores the application of non-contact sensor technology in assessing pilot workload, drawing on recent research progress and combining results from simulated and actual operational environments, analyzing its potential value and implementation paths in enhancing aviation safety, assisting pilot training, and managing status, with the aim of providing theoretical support and reference for aviation human factors engineering applications. 257 2. Pilot Workload Assessment Techniques Current pilot workload assessment techniques include subjective questionnaire assessment, task performance assessment, and physiological and behavioral assessment [10]. Among them, physiological and behavioral assessment can be further divided into contact and non-contact methods. 2.1 Traditional Assessment Methods and Their Limitations Before turning to non-contact methods, it is useful to revisit established approaches and the practical constraints that motivate newer methods. The NASA Task Load Index (NASA-TLX) is a multi-dimensional assessment process and standard for workload developed by NASA's Ames Research Center. It assesses workload from six dimensions, including mental demands, physical demands, temporal demands, performance, effort, and frustration. The Situation Awareness Rating Technique (SART) is a widely used subjective situational awareness (SA) measurement tool in fields such as human factors, aviation, military, medical, and transportation [11]. Subjective measurement has several drawbacks. For instance, some participants may fail to distinguish between task requirements and psychological load, which could lead to omissions [12]. Additionally, subjective measurement is also influenced by the delay effect. Usually, after the task is completed or during a pause in the task, the operator fills out a questionnaire based on the just-ended task stage. Task performance evaluation assesses pilots' mental workload based on specific performance tasks, utilizing observable performance degradation as a lagging indicator of high mental workload. However, this assessment methodology cannot anticipate performance deterioration resulting from increased workload [13,14]. However, such methods have a lag and cannot achieve real-time measurement. While conventional patch electrodes can provide relatively accurate data for monitoring pilots' electrocardiographic (ECG) and electromyographic (EMG) signals, their invasive nature tends to interfere with operational performance, thereby compromising the normal execution of flight missions [15]. 2.2 The Rise of Non-invasive Assessment Technology Pilot fatigue constitutes a significant risk factor in aviation safety [16]. Conventional fatigue monitoring methods, such as subjective questionnaires and wearable devices, are characterized by inherent subjectivity, operational inconvenience, and data latency [17]. Given the practical limitations of contact-based sensors in real-world applications, recent advancements in computer vision and sensor technologies have catalyzed the rapid emergence of non-contact assessment techniques. These innovations provide critical solutions for monitoring pilot fatigue in a real-time, objective, accurate, and non-intrusive manner. 3. A Detailed Explanation of Non-contact Assessment Technology 3.1 Behavior Recognition Based on Computer Vision Computer vision has now permeated every aspect of our lives, which is the technology enabling computers to automatically “understand” and extract information from images and videos. It transforms pixels into structured understanding—identifying what objects are, where they are located, how they move, or their three-dimensional shapes. Common tasks include classification, detection, segmentation, tracking, and OCR. Modern approaches primarily rely on deep learning models (such as CNNs and ViTs) trained on massive datasets, finding extensive applications in autonomous driving, medical imaging, industrial quality inspection, and security surveillance. Facial expressions are the external manifestations of emotions and cognitive states. Especially in high-load tasks, certain micro- expressions (such as frowning, tight lips, raised chin, etc.) are highly correlated with cognitive load. Luo et al. utilized FaceReader and GoPro camera sensors to extract 49 facial action units and basic emotion features [6]. These were classified through a deep learning model, achieving high-precision 258 recognition of pilots' workload under three weather conditions. They also verified that the CNN model significantly outperformed other models in classification performance and is suitable for high- precision real-time monitoring systems. Eye movement behaviors (such as fixation time, pupil diameter, and scanning frequency) can also reflect visual attention and cognitive load levels. Chen et al. used a Tobii eye tracker in combination with the NASA-TLX scale to extract 10 eye movement indicators [7]. Through variance analysis and correlation tests, it was found that the smoothness ratio of fixation time and the average pupil change rate were significantly correlated with the workload. Liu et al. utilized telemetry eye trackers and a pressure-sensing seat cushion to detect flight workload [18]. The results demonstrated that this model could effectively identify low, medium, and high load levels. In cross-pilot tests, the accuracy rate reached 82.6%, which was higher than that of many studies using invasive equipment. 3.2 Evaluation Based on Ballistocardiography Ballistocardiography (BCG) is a non-invasive, non-contact technique that measures subtle body movements from cardiac ejection and blood flow, offering insights into cardiovascular dynamics to assess pilots' workload in aviation during maneuvers, simulations, or microgravity. By deriving parameters like heart rate variability (HRV), stroke volume (SV), and systolic time intervals (e.g., RI and RJ, linked to pre-ejection period and left ventricular ejection time), BCG quantifies sympathetic responses to stress, fatigue, and situational awareness. BCG detects ballistic forces via accelerometers or force sensors, manifesting as micro-vibrations in seats or helmets, with the IJ wave complex indicating aortic ejection. Integration with ECG or ICG enhances accuracy, while non-contact setups suit aviation by avoiding movement restrictions. Kutilek et al. reviewed and applied non-contact systems, including BCG, for monitoring military pilots during simulator training [8]. Using a Wii Balance Board as a force platform under the pilot's seat, they measured body pressure variations in five cadets performing tasks like take-off and landing. BCG data correlated with heart rate peaks during high-stress phases, indicating workload-induced autonomic shifts. Combined with video and thermography, BCG provided force/pressure metrics to infer physical and psychological load, achieving synchronization with wearable references like FlexiGuard. Results showed elevated heart/respiration rates in challenging maneuvers, validating BCG's non-intrusive role in training assessments. Deliere et al. assessed cardiovascular changes in parabolic flights using BCG alongside ECG and ICG on five subjects (aged 35 ± 12 years) [19]. During ~20s microgravity phases, ensemble-averaged BCG revealed increased IJ amplitude (from standing baseline), suggesting higher SV due to fluid shifts and reduced gravity. RI/RJ intervals trended shorter, implying reduced PEP under workload analogs like hyper-G transitions. Though limited by artifacts (only 5.6% of beats analyzed), the study confirmed BCG's sensitivity to ventricular performance in dynamic aviation environments. These studies demonstrate BCG's efficacy in capturing workload via autonomic metrics, with higher IJ amplitudes and HRV shifts signaling elevated demands in simulations and flights. 3.3 Technology Based on Infrared Thermal Imaging Infrared thermography (IRT) emerges as a promising non-contact technique, capturing facial temperature variations (FTV) that correlate with autonomic nervous system responses to stress and workload.Recent research on helicopter crews demonstrates Infrared thermography (IRT) in workload assessment, often integrated with other sensors for a holistic view. In a study by Alaimo et al. , a thermographic camera (OPTRIS PI 640) was used to monitor FTV in pilots during a Level-D business aircraft simulator session [3]. The system, dubbed the Cockpit Pilot Warning System, captured forehead and nose tip temperatures alongside HR from a Movisens EcgMove 3 sensor. Flight phases included resting, briefing, take-off, cruising maneuvers (e.g., stalls, turns), descent, and landing. Results showed a decrease in nose tip temperature during high-workload phases like approach and touchdown, correl ating with HR peaks (e.g., median HR reaching 115 bpm during stalls). The nose temperature dropped by up to 0.5°C on average during descent, aligning with 259 literature indicating sympathetic-induced vasoconstriction. Forehead temperature remained stable, confirming regional specificity. This non-invasive setup proved feasible for real-time workload detection, with potential extensions to air traffic controllers or unmanned aerial vehicle operators. Complementing IRT, Vicente-Rodríguez et al. employed portable biosensors, including a digital infrared thermometer (Temp Touch, Xilas Medical) for skin temperature, in helicopter crew stress monitoring [9]. Two maneuvers were analyzed: a crane rescue and low-altitude flight. Skin temperature decreased post-crane rescue (from 36.7°C to 36.5°C), indicating stress-related vasoconstriction, alongside increased HR, lactate, and sympathetic modulation. The crane rescue elicited stronger responses (e.g., higher max HR at 151 bpm vs. 109 bpm in low-altitude), attributed to its uncontrollability and physical demands. While not using full thermal imaging, the infrared thermometer provided similar insights to FTV, suggesting IRT's scalability for field use. These studies collectively show IRT's efficacy in detecting workload variations. Alaimo et al. directly links FTV drops to specific flight phases, while the others use point-based infrared measurements to infer similar autonomic shifts. 4. Discussion and Future Directions Non-contact technologies for pilot workload assessment, such as computer vision, electrocardiography (ECG), and infrared thermal imaging (IRT), each possess distinct advantages and limitations when applied in aviation environments. Compared to traditional contact sensors or subjective evaluations, these methods enable real-time monitoring of physiological and behavioral changes without interfering with pilot operations (shown in Table 1). Table 1. Comparison of Non-contact Pilot Workload Assessment Technologies Technology Advantages Limitations Typical Application Scenarios Representative Indicators Computer Vision Non-invasive; captures facial expressions, eye movements, and micro-behaviors; suitable for real-time emotion and attention monitoring Sensitive to lighting, camera angle, and occlusion; high computational load Simulator training, cockpit monitoring, fatigue and distraction detection Eye fixation time, pupil dilation, facial action units Ballistocardiography (BCG) Detects subtle body vibrations related to cardiac output; compatible with seat or helmet integration; unaffected by facial occlusion Susceptible to motion artifacts; requires careful signal filtering and synchronization Flight simulators, real-flight cardiovascular monitoring, microgravity studies Heart rate variability (HRV), IJ wave amplitude, stroke volume Infrared Thermal Imaging (IRT) Detects autonomic changes through facial temperature variations; unaffected by visible light; good for multi-phase flight monitoring Influenced by cabin airflow and ambient temperature; equipment cost High-workload flight phases (take- off/landing), stress detection, adaptive cockpit alerts Nose tip temperature, forehead thermal gradient, facial temperature variability According to the comparison in Table 1, these non-contact methods collectively demonstrate significant potential for real-time, continuous, and non-intrusive monitoring of pilot workload. Their integrated solutions effectively address three major shortcomings of traditional approaches: subjectivity, time delay, and interference issues. However, current research still faces limitations in practical application, as models trained on small or laboratory datasets may fail to maintain stable performance under real flight conditions. Furthermore, multimodal fusion technology, the integrated 260 application of visual, thermal imaging, and physiological signals, remains an emerging challenge requiring robust synchronization mechanisms and adaptive algorithms. The ongoing advancement of non-contact sensor technology in assessing pilot workload opens highly promising avenues for future development. Advances in artificial intelligence (AI) and machine learning can enhance real-time analysis capabilities for multimodal data—including infrared thermal imaging, optical tracking, and ballistic cardiography (BCG)—enabling more precise workload prediction. Integrating these sensors into a unified cockpit system for real-time data processing enablestimelyalerts to mitigate fatigue-related risks. Furthermore, embedding lightweight, wearable BCG sensors into helmets or seats, combined with smarter "noise filtering" technology, significantly improves monitoring accuracy during turbulent flights or weightless conditions. Collaboration between aerospace engineers and neuroscientists enables sensor calibration tailored to distinct pilot cohorts, addressing individual variability. Future research should validate this system during long-haul flights and extreme operational scenarios, laying the groundwork for standardized procedures to enhance aviation safety and pilot performance. 5. Conclusion In conclusion, this review explores the application of non-contact sensor technologies in assessing aircraft pilot workload to enhance aviation safety. Pilot fatigue, driven by prolonged flights, irregular circadian rhythms, and complex cockpit environments, contributes significantly to cognitive decline and errors, with human factors accounting for 80% of aviation accidents. Traditional workload assessment methods, such as subjective questionnaires (e.g., NASA-TLX, SART) and task performance evaluations, suffer from subjectivity, delays, and invasiveness, limiting real-time applicability. Non-contact technologies, including computer vision, ballistocardiography (BCG), and infrared thermal imaging (IRT), offer objective, non-intrusive solutions. Computer vision tracks facial expressions and eye movements to gauge cognitive load, with studies achieving high accuracy in workload classification. BCG measures cardiovascular dynamics via subtle body movements, effectively capturing workload during simulations and flights. IRT monitors facial temperature variations, reflecting stress-induced autonomic responses. These technologies enable real-time, accurate workload monitoring without interfering with pilot operations. Future directions include integrating AI, machine learning, and edge computing for dynamic, adaptive systems, optimizing sensor calibration, and validating applications in long-haul and extreme scenarios to enhance pilot performance and aviation safety. References [1] Wingelaar-Jagt Y Q, Wingelaar T T, Riedel W J, et al. Fatigue in Aviation: Safety Risks, Preventive Strategies and Pharmacological Interventions[J]. Frontiers in Physiology, 2021, 12: 712628. doi: 10.3389/ fphys. 2021.712628. [2] Lee Y H, Jeon J D, Choi Y C. Air Traffic Controllers’ Situation Awareness and Workload under Dynamic Air Traffic Situations[J]. https:// dx.doi.org/10.5325/transportationj. 51.3.0338. doi: 10.5325/ transportationj. 51.3.0338. [3] Alaimo A, Esposito A, Milazzo A, et al. An Aircraft Pilot Workload Sensing System[M]//Rizzo P, Milazzo A. European Workshop on Structural Health Monitoring: Vol. 127. Cham: Springer International Publishing, 2021: 883-892. http://link.springer.com/10.1007/978-3-030-64594-6_85. doi: 10.1007/978- 3- 030-64594-6_85. [4] Carlsen V, Manzi R, Dellinger S, et al. Predicting Pilot Workloads Using Physiological Measures[J]. Industrial and Systems Engineering Review, 2025, 12(1): 1-6. doi: 10.37266/ISER.2025v12i1.pp1-6. [5] Lim Y, Bassien-Capsa V, Ramasamy S, et al. Commercial airline single-pilot operations: System design and pathways to certification[J]. IEEE Aerospace and Electronic Systems Magazine, 2017, 32(7): 4-21. doi: 10.1109/MAES.2017.160175. 261 [6] Luo S, Zhang C, Zhu W, et al. Noncontact perception for assessing pilot mental workload during the approach and landing under various weather conditions[J]. Signal, Image and Video Processing, 2025, 19(2): 98. doi: 10.1007/s11760-024-03619-x. [7] Chen N, Ma T, Han Y, et al. A Pilot Approach and Landing Workload Assessment Method for Flight Crew Workload Airworthiness Certification[M]//Harris D, Li W C, Krömker H. HCI International 2024 – Late Breaking Papers: Vol. 15381. Cham: Springer Nature Switzerland, 2025: 225-240. https:// link. springer. com/10.1007/978-3-031-76824-8_16. doi: 10.1007/978-3-031-76824-8_16. [8] Kutilek P, Kacer J, Cicmanec L, et al. Non-contact Measurement Systems for Physiological Data Monitoring of Military Pilots During Training on Simulators: Review and Application[C]//2019 International Conference on Military Technologies (ICMT). Brno, Czech Republic: IEEE, 2019: 1-6. https://ieeexplore.ieee.org/document/8870115/. doi: 10.1109/MILTECHS.2019.8870115. [9] Vicente-Rodríguez M, Iglesias Gallego D, Fuentes-García J P, et al. Portable Biosensors for Psychophysiological Stress Monitoring of a Helicopter Crew[J]. Sensors, 2020, 20(23): 6849. doi: 10. 3390/ s20236849. [10] Workload assessment of computer gaming using a single-stimulus event-related potential paradigm[J]. Biological Psychology, 2008, 77(3): 277-283. doi: 10.1016/j.biopsycho.2007.10.014. [11] Taylor R M. Situational Awareness Rating Technique (Sart): The Development of a Tool for Aircrew Systems Design[M]//Situational Awareness. Routledge, 2011. [12] Veltman J A, Gaillard A W K. Physiological indices of workload in a simulated flight task[J]. Biological Psychology, 1996, 42(3): 323-342. doi: 10.1016/0301-0511(95)05165-1. [13] Alaimo A, Esposito A, Orlando C, et al. Aircraft Pilots Workload Analysis: Heart Rate Variability Objective Measures and NASA-Task Load Index Subjective Evaluation[J]. Aerospace, 2020, 7(9): 137. doi: 10.3390/aerospace7090137. [14] Wang P, Houghton R, Majumdar A. Detecting and Predicting Pilot Mental Workload Using Heart Rate Variability: A Systematic Review[J]. Sensors, 2024, 24(12): 3723. doi: 10.3390/s24123723. [15] Hu X, Lodewijks G. Detecting fatigue in car drivers and aircraft pilots by using non-invasive measures: The value of differentiation of sleepiness and mental fatigue[J]. Journal of Safety Research, 2020, 72: 173-187. doi: 10.1016/j.jsr.2019.12.015. [16] Wingelaar-Jagt Y Q, Wingelaar T T, Riedel W J, et al. Fatigue in Aviation: Safety Risks, Preventive Strategies and Pharmacological Interventions[J]. Frontiers in Physiology, 2021, 12. https://www. frontiersin.org/journals/physiology/articles/10.3389/fphys.2021.712628/full. doi: 10.3389/ fphys. 2021. 712628. [17] Göker Z. Fatigue in The Aviation: An Overview of The Measurements and Countermeasures[J]. Journal of Aviation, 2018, 2(2): 185-194. doi: 10.30518/jav.451741. [18] Liu Y, Gao Y, Yue L, et al. A Real-Time Detection of Pilot Workload Using Low-Interference Devices[J]. Applied Sciences, 2024, 14(15): 6521. doi: 10.3390/app14156521. [19] Deliere Q, Migeotte P F, Neyt X, et al. Cardiovascular changes in parabolic flights assessed by ballistocardiography [C]//2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Osaka: IEEE, 2013: 3801-3804. http://ieeexplore. ieee.org/ document/ 6610372/. doi: 10.1109/EMBC.2013.6610372.