The CherISH network

Project 5: Impact of visual cues in spatial hearing

Host Institution: Claude Bernard University Lyon 1, France
In this project, cooperation between the Lyon Neuroscience Research Centre (CRNL) and the Hospices Civils de Lyon (HCL, clinical partner).

The construction of an auditory spatial representation is based on the knowledge of the relationships between acoustic cues and the structure of spatial representation. This correspondence is developed through interactions with the environment (active exploration) and the sensory feedback, including audio and visual inputs, generated during these interactions. Multisensory integration of visual and auditory information plays a significant role in shaping the spatial representation of space and in sound localization. The impact of vision goes beyond specific discriminative cues of the sound source; it extends to the integration of contextual visual information from the environment and promotes head movement exploration, which generates dynamic cues that enhance the ability to locate sounds. This dynamic audio-visual-motor loop may be particularly crucial for populations with hearing loss and could be the key to spatial auditory rehabilitation, especially for cochlear implant (CI) users. While it has been demonstrated that providing auditory and visual cues can improve the spatial auditory abilities of CI users, these improvements are not consistent among patients, and the exploratory strategies involving head movements in the environment vary from one individual to another.

The PhD project aims to investigate the role of these visuo-auditory interactions in auditory spatial representation. Specifically, his goal is to characterize the contribution of visual cues (both target and contextual cues) and auditory cues in an active spatial hearing task with the aim of optimizing visuo-auditory fusion during spatial hearing (SH) training. The doctoral student will employ a psychophysical approach using the virtual reality system “SPHERE” developed by our group (see Gaveau et al., 2022) and used for the assessment and training spatial hearing skills. The experimental strategy will involve manipulating various elements during a spatial hearing task to determine the relative weight of auditory and visual cues in spatial localization performance. The manipulated elements will include the auditory ambient scene, the visual cues in the virtual reality scene, and the promotion of audio-visual integration by encouraging interaction with the environment (e.g., through the promotion of a head movement strategy). The student will actively participate in the development of experimental protocols in VR conducted with normal-hearing subjects and CI users.

Our Partners:

• UNITN, Universta degli Study di Trento, Italy [Secondment 1]
• Nemo Lab (Former 4D sound)/Spatial Sound Institute, Hungary [Secondment 2]
• RCNS, TTK Research Centre for Natural Sciences, Hungary
• BME, Budapest University of Technology and Economics, Hungary