Audiovisual recording and reproduction of ecological acoustical scenes for hearing research: a case study with high reverberation

Recent research has focused on the validation of methods and procedures to perform ecological tests for the assessment of hearing sensitivity under the complex acoustical conditions of everyday life environments. Virtual Reality (VR) has been extensively used to reproduce immersive acoustical scenes in combination with visual cues in order to account for the multisensory perception of the physical environment that happens in real-life situations. However, due to the complexity of recording and reproduction procedures, the main studies focus on either audiovisual rendering of simulated scenarios or in-field audio recordings without real visual contextualization. This work proposes a case study involving a challenging listening environment (a conference hall with 3.2 s of reverberation time at mid-frequencies), where 360 audiovisual scenes were recorded and then reproduced in laboratory using a 16-loudspeakers array and a VR headset. Multiple scenarios involving different target- and noise-source positions were acquired through 3rd-order-ambisonics recordings of room impulse responses and 360∘ stereoscopic video footage. Speech intelligibility tests were auralized for these scenarios, considering informational masking noise at different signal-to-noise ratios, and administrated to a panel of normal-hearing subjects to validate the proposed VR methodology that will be applied for future studies involving hearing-impaired listeners too.