Vision Systems and IMU Signals to Design a Hand-Free Driving HMI

This paper presents an experimental analysis of a prototype measurement system to construct a hand-free Human-Machine Interface (HMI) for controlling wheelchairs, integrating face orientation algorithms and Inertial Measurement Units (IMUs). The aim is to enable individuals with mobility limitations to achieve independent mobility and social interaction. Moreover, the to-be-de-signed HMI can be an alternative approach to the classical joystick, for people who have health conditions that do not allow the effective use of manual devices. The HMI system utilizes cameras and IMUs to capture the user's head and torso movements. Experimental results are discussed in comparison of IMU and camera data with the Motion Capture system on accuracy and ease of instrumentation, and it has been demonstrated not only the essential reliability of IMU but also the need for fusion algorithms to enhance camera data accuracy. Thanks to the fusion processing of left and right camera images beside a rider’s face, face orientation detection becomes robust while requesting no inconvenience for the rider such as wearing specially made mounting devices, etc. The findings suggest a promising approach for developing inclusive and efficient wheelchair control systems.