Browser-Based Conducting Gesture Recognition for Educational Orchestras: System-Level Accuracy and Latency Evaluation

Participation in orchestral contexts presents challenges for blind and visually impaired (BVI) musicians due to the inherently visual nature of conducting gestures. This work presents a browser-based, markerless conducting gesture recognition architecture integrated with an embedded Bluetooth Low Energy (BLE) device, aimed at enabling accessible gesture translation in educational orchestral settings. MediaPipe Hands is employed for real-time landmark extraction within a standard web application, while gesture classification is performed through a deterministic geometric and kinematic algorithmic approach applied directly to landmark trajectories. Recognized gestures are encoded and transmitted to a resource-constrained embedded node, forming a complete browser-to-embedded interaction pipeline. Recognition accuracy was evaluated using gestures performed by a professional orchestra conductor under realistic execution conditions. Results demonstrate very high accuracy for isolated metric and dynamic gestures and robust performance under bimanual execution. In addition, a hardware-timed methodology was introduced to measure physical end-to-end latency from gesture occurrence to BLE command reception at the embedded device. The measured mean latency of 125.61 ms is compatible with human sensorimotor response times and established conducting practice.