ECG, Respiration, fNIRS, and Eye Tracking for Stress and Mental Workload Monitoring in Human-Machine Interaction

Advanced human-machine interaction (AHMI) is a key concept in human factors and ergonomics (HFE), focusing on how individuals interact with systems to perform tasks efficiently. As AHMI becomes more integrated into fields such as Industry 4.0, aviation, automotive, and clinical applications, users face increasing complexity, leading to elevated mental workload (MWL) and stress. These factors can impair performance and cause accidents, emphasizing the need for AHMI systems capable of real-time monitoring of cognitive load and stress levels. This paper investigates the relationship between stress, MWL, and four physiological signals - electrocardiogram (ECG), respiration, functional near-infrared spectroscopy (fNIRS), and eye tracking - combined with a tailored Self-Assessment Questionnaire (SAQ), specifically designed for industrial applications. A study involving 20 participants was conducted using the Stroop, Visual, Auditory, and Dual N-Back tasks. During the study, 83 features were extracted from the physiological signals and linked to this four-level ratings SAQ of perceived stress and MWL. Statistical analysis using Kruskal-Wallis and Mann-Whitney tests assessed the ability of these features to differentiate stress and MWL levels. Over 50% of the features reliably distinguished between cognitive states, particularly in identifying relaxed versus altered conditions. Respiration, fNIRS, and eye movement signals provided higher granularity in differentiating multiple altered cognitive states, suggesting their potential for precise monitoring in AHMI systems. These findings underscore the value of physiological monitoring in AHMI systems, which can enhance user performance and safety by enabling adaptive interfaces tailored to real-time cognitive states, supporting future industrial applications.