ENHANCING MUSIC VISUALIZATION WITH HAPTIC FEEDBACK TO EASE PERCEPTION FOR DHH LISTENERS

This study introduces a novel integration of visual and haptic feedback to improve music perception for Deaf and Hard-of-Hearing (DHH) individuals. The proposed prototype, named MusicTanvas, leverages a touchscreen augmented with electroadhesion-based haptic technology, enabling dynamic overlay of tactile textures on visual content. The system processes pre-recorded audio tracks to extract spectral, harmonic, and rhythmic features, which are mapped to both visual and haptic representations. More in detail, key features such as chroma, Mel-Frequency Cepstral Coefficients (MFCCs), and rhythmic figures, are translated into corresponding visual attributes (e.g., color, shape, and size) and tactile textures. Users can explore music in real-time through these multisensory outputs, which evolve synchronously with the audio playback. A preliminary user assessment demonstrates the system's potential to enhance musical interaction by combining visual-haptic feedback.