Incorporating prosody into neural speech processing pipelines: applications on automatic speech transcription and spoken language machine translation
In this dissertation, I study the inclusion of prosody into two applications that involve speech understanding:~automatic speech transcription and spoken language translation. In the former case, I propose a method that uses an attention mechanism over parallel sequences of prosodic and morphosyntac...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis doctoral |
| Estado: | Versión publicada |
| Fecha de publicación: | 2019 |
| País: | España |
| Institución: | CBUC, CESCA |
| Repositorio: | TDR. Tesis Doctorales en Red |
| OAI Identifier: | oai:www.tdx.cat:10803/666222 |
| Acceso en línea: | http://hdl.handle.net/10803/666222 |
| Access Level: | acceso abierto |
| Palabra clave: | Prosody Automatic speech transcription Punctuation restoration Spoken language machine translation Bilingual spoken corpus Prosòdia Transcripció automàtica de la parla Restauració de la puntuació Traducció automàtica de llenguatge oral Corpus bilingües 62 |
| Sumario: | In this dissertation, I study the inclusion of prosody into two applications that involve speech understanding:~automatic speech transcription and spoken language translation. In the former case, I propose a method that uses an attention mechanism over parallel sequences of prosodic and morphosyntactic features. Results indicate an $F_1$ score of 70.3\% in terms of overall punctuation generation accuracy. In the latter problem I deal with enhancing spoken language translation with prosody. A neural machine translation system trained with movie-domain data is adapted with pause features using a prosodically annotated bilingual dataset. Results show that prosodic punctuation generation as a preliminary step to translation increases translation accuracy by 1\% in terms of BLEU scores. Encoding pauses as an extra encoding feature gives an additional 1\% increase to this number. The system is further extended to jointly predict pause features in order to be used as an input to a text-to-speech system. |
|---|