Investigating the Training Dynamics in End-to-end Speech Translation
The task of speech translation consists one translating speech input into text in a different language. In this project, we present an interpretability analysis of a Transformer model on this task. Our work builds upon previous research which explored the training phases of a Transformer for text tr...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Fecha de publicación: | 2023 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/403813 |
| Acceso en línea: | https://hdl.handle.net/2117/403813 |
| Access Level: | acceso abierto |
| Palabra clave: | Machine translating Deep learning (Machine learning) Speech processing systems deep learning machine translation speech processing transformer Traducció automàtica Aprenentatge profund Processament de la parla Àrees temàtiques de la UPC::Enginyeria de la telecomunicació::Processament del senyal::Processament de la parla i del senyal acústic |
| Sumario: | The task of speech translation consists one translating speech input into text in a different language. In this project, we present an interpretability analysis of a Transformer model on this task. Our work builds upon previous research which explored the training phases of a Transformer for text translation. We extend their analysis to study the training of the Transformer for ST, focusing on the variations of contribution of the source to the predictions during the training process. We show that depending on the training strategy, some speech translation models show a similar source contribution than text translation ones, but others have a lower source contribution and a worse performance. Furthermore, we propose modification to the Transformer architecture, aiming to force the model to use more source in its predictions. Through this modifications, we achieve a significant performance boost of up to +1.3 BLEU. |
|---|