Audiovisual tracking of multiple speakers in smart spaces
This paper presents GAVT, a highly accurate audiovisual 3D tracking system based on particle filters and a probabilistic framework, employing a single camera and a microphone array. Our first contribution is a complex visual appearance model that accurately locates the speaker?s mouth. It transforms...
| Autores: | , , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2023 |
| País: | España |
| Institución: | Universidad de Alcalá (UAH) |
| Repositorio: | e_Buah Biblioteca Digital Universidad de Alcalá |
| Idioma: | inglés |
| OAI Identifier: | oai:ebuah.uah.es:10017/64541 |
| Acceso en línea: | http://hdl.handle.net/10017/64541 https://dx.doi.org/10.3390/s23156969 |
| Access Level: | acceso abierto |
| Palabra clave: | Smart spaces Audiovisual tracking Speaker localization Particle filter Multi-pose face observation model Probabilistic SRP-PHAT Electrónica Electronics |
| Sumario: | This paper presents GAVT, a highly accurate audiovisual 3D tracking system based on particle filters and a probabilistic framework, employing a single camera and a microphone array. Our first contribution is a complex visual appearance model that accurately locates the speaker?s mouth. It transforms a Viola & Jones face detector classifier kernel into a likelihood estimator, leveraging knowledge from multiple classifiers trained for different face poses. Additionally, we propose a mechanism to handle occlusions based on the new likelihood?s dispersion. The audio localization proposal utilizes a probabilistic steered response power, representing cross-correlation functions as Gaussian mixture models. Moreover, to prevent tracker interference, we introduce a novel mechanism for associating Gaussians with speakers. The evaluation is carried out using the AV16.3 and CAV3D databases for Single- and Multiple-Object Tracking tasks (SOT and MOT, respectively). GAVT significantly improves the localization performance over audio-only and video- only modalities, with up to 50.3% average relative improvement in 3D when compared with the video-only modality. When compared to the state of the art, our audiovisual system achieves up to 69.7% average relative improvement for the SOT and MOT tasks in the AV16.3 dataset (2D comparison), and up to 18.1% average relative improvement in the MOT task for the CAV3D dataset (3D comparison). |
|---|