gms | German Medical Science

Background: Acoustic conversation in difficult conditions such as cocktail party situations is still a problem. State of the art hearing aid algorithms use different techniques like beamforming to enhance speech. Nevertheless, they currently cannot estimate the attended sound source, i.e., the source that is currently in the focus of attention of the user, and thus may not give optimal performance in daily-life dynamic communication conditions. A model that identifies the attended source(s) may help to overcome this problem and to optimise the steering and control of speech enhancement algorithms. In a recent study [1], a gaze-based attention model that uses the gaze direction and the head orientation to estimate the attended sound source was proposed. This study investigates the potential of such method for detecting the attended and unattended sources in multi-source audiovisual environments.

Methods: Gaze direction and head orientation were measured using electrooculography (EOG) and a head tracking system in virtual audiovisual scenes in 6 normal-hearing subjects. The attention model combines gaze information with sound source locations using a naïve Bayesian classification and temporal analysis to estimate the subject's focus of attention. In this study, optimal a-priori knowledge on source locations was used to assess the performance of the gaze-based estimation and to optimise its parameters. In an instrumental test, the optimal model parameters that resulted in the best signal-to-noise ratio (SNR) were determined.

Results: Results indicate that SNR improvements of up to 7 dB can be achieved using optimal model parameters.

Conclusions: The benefits from steering speech enhancement by gaze information found in this instrumental test are promising. The approach thus needs to be further investigated by testing subject performance in audiovisual scenes and with relevant communication tasks.