MeWEHV: Mel and Wave Embeddings for Human Voice Tasks
[EN] A recent trend in speech processing is the use of embeddings created through machine learning models trained on a specific task with large datasets. By leveraging the knowledge already acquired, these models can be reused in new tasks where the amount of available data is small. This paper prop...
| Autores: | , , , |
|---|---|
| Formato: | artículo |
| Estado: | Versión publicada |
| Fecha de publicación: | 2023 |
| País: | España |
| Recursos: | Universidad de León |
| Repositorio: | BULERIA. Repositorio Institucional de la Universidad de León |
| OAI Identifier: | oai:buleria.unileon.es:10612/19740 |
| Acesso em linha: | https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10198451 https://hdl.handle.net/10612/19740 |
| Access Level: | acceso abierto |
| Palavra-chave: | Informática Ingeniería de sistemas Embeddings Hubert Speech classification WavLM XLSR-Wav2Vec2 YouSpeakers204 1203.04 Inteligencia Artificial 3304.05 Sistemas de Reconocimiento de Caracteres |
| Resumo: | [EN] A recent trend in speech processing is the use of embeddings created through machine learning models trained on a specific task with large datasets. By leveraging the knowledge already acquired, these models can be reused in new tasks where the amount of available data is small. This paper proposes a pipeline to create a new model, called Mel and Wave Embeddings for Human Voice Tasks (MeWEHV), capable of generating robust embeddings for speech processing. MeWEHV combines the embeddings generated by a pre-trained raw audio waveform encoder model, and deep features extracted from Mel Frequency Cepstral Coefficients (MFCCs) using Convolutional Neural Networks (CNNs). We evaluate the performance of MeWEHV on three tasks: speaker, language, and accent identification. For the first one, we use the VoxCeleb1, and VBHIR datasets and present YouSpeakers204, a new and publicly available dataset for English speaker identification that contains 19607 audio clips from 204 persons speaking in six different accents, allowing other researchers to work with a very balanced dataset, and to create new models that are robust to multiple accents. For evaluating the language identification task, we use the VoxForge, Common Language, and the LRE17 datasets. Finally, for accent identification, we use the Latin American Spanish Corpora (LASC), Common Voice, and the NISP datasets. Our approach allows a significant increase in the performance of state-of-the-art embedding generation models on all the tested datasets, with a low additional computational cost. |
|---|