Evaluating the effectiveness of Microsoft Transcribe for automating the assessment of pronunciation in language proficiency tests

[EN] Improvements in Automatic Speech Recognition (ASR) have created opportunities for using it as a tool to facilitate second and foreign language (L2) assessment. These technical improvements have not only enabled automation of language proficiency test scoring but also reduced evaluator bias and...

Descripción completa

Detalles Bibliográficos
Autores: Nelson, Carey, Cardoso, Walcir
Tipo de recurso: capítulo de libro
Fecha de publicación:2024
País:España
Institución:Universitat Politècnica de València (UPV)
Repositorio:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Idioma:inglés
OAI Identifier:oai:riunet.upv.es:10251/206633
Acceso en línea:https://riunet.upv.es/handle/10251/206633
Access Level:acceso abierto
Palabra clave:Automated evaluation
Automatic Speech Recognition (ASR)
Language assessment
ESL pronunciation evaluation
Microsoft Transcribe (MS-T)
Placement tests
id ES_599faebda625ffe1ca9946bb4938ddcc
oai_identifier_str oai:riunet.upv.es:10251/206633
network_acronym_str ES
network_name_str España
repository_id_str
spelling Evaluating the effectiveness of Microsoft Transcribe for automating the assessment of pronunciation in language proficiency testsNelson, CareyCardoso, WalcirAutomated evaluationAutomatic Speech Recognition (ASR)Language assessmentESL pronunciation evaluationMicrosoft Transcribe (MS-T)Placement tests[EN] Improvements in Automatic Speech Recognition (ASR) have created opportunities for using it as a tool to facilitate second and foreign language (L2) assessment. These technical improvements have not only enabled automation of language proficiency test scoring but also reduced evaluator bias and errors, decreased processing time, and lowered costs for testing organizations. The purpose of this study was to evaluate English as a Second Language (ESL) pronunciation using the ASR feature in the Microsoft 365 product suite, Transcribe (MS-T). The study involved adult ESL learners at a Canadian university that partook in a language proficiency test. We examined the audio recordings of 56 candidates during the pronunciation portion of the test. Building on previous studies that found a strong correlation between scores from Google Voice Typing and human raters, the current study conducted a similar analysis comparing scores derived from MS-T to both human ratings and Google Voice Typing. Our findings indicate that the ASR capabilities of MS-T, similar to Google Voice Typing, can assume an important role in L2 speaking assessment by providing objectivity and reliability to the testing process, expediting scoring, and reducing costs.Editorial Universitat Politècnica de ValènciaRepositorio Institucional de la Universitat Politècnica de València Riunet20242024-02-12book parthttp://purl.org/coar/resource_type/c_3248VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/bookPartapplication/pdfhttps://riunet.upv.es/handle/10251/206633reponame:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valénciainstname:Universitat Politècnica de València (UPV)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2Reconocimiento - No comercial - Compartir igual (by-nc-sa) http://creativecommons.org/licenses/by-nc-sa/4.0/info:eu-repo/semantics/openAccessoai:riunet.upv.es:10251/2066332026-06-13T07:49:27Z
dc.title.none.fl_str_mv Evaluating the effectiveness of Microsoft Transcribe for automating the assessment of pronunciation in language proficiency tests
title Evaluating the effectiveness of Microsoft Transcribe for automating the assessment of pronunciation in language proficiency tests
spellingShingle Evaluating the effectiveness of Microsoft Transcribe for automating the assessment of pronunciation in language proficiency tests
Nelson, Carey
Automated evaluation
Automatic Speech Recognition (ASR)
Language assessment
ESL pronunciation evaluation
Microsoft Transcribe (MS-T)
Placement tests
title_short Evaluating the effectiveness of Microsoft Transcribe for automating the assessment of pronunciation in language proficiency tests
title_full Evaluating the effectiveness of Microsoft Transcribe for automating the assessment of pronunciation in language proficiency tests
title_fullStr Evaluating the effectiveness of Microsoft Transcribe for automating the assessment of pronunciation in language proficiency tests
title_full_unstemmed Evaluating the effectiveness of Microsoft Transcribe for automating the assessment of pronunciation in language proficiency tests
title_sort Evaluating the effectiveness of Microsoft Transcribe for automating the assessment of pronunciation in language proficiency tests
dc.creator.none.fl_str_mv Nelson, Carey
Cardoso, Walcir
author Nelson, Carey
author_facet Nelson, Carey
Cardoso, Walcir
author_role author
author2 Cardoso, Walcir
author2_role author
dc.contributor.none.fl_str_mv Repositorio Institucional de la Universitat Politècnica de València Riunet
dc.subject.none.fl_str_mv Automated evaluation
Automatic Speech Recognition (ASR)
Language assessment
ESL pronunciation evaluation
Microsoft Transcribe (MS-T)
Placement tests
topic Automated evaluation
Automatic Speech Recognition (ASR)
Language assessment
ESL pronunciation evaluation
Microsoft Transcribe (MS-T)
Placement tests
description [EN] Improvements in Automatic Speech Recognition (ASR) have created opportunities for using it as a tool to facilitate second and foreign language (L2) assessment. These technical improvements have not only enabled automation of language proficiency test scoring but also reduced evaluator bias and errors, decreased processing time, and lowered costs for testing organizations. The purpose of this study was to evaluate English as a Second Language (ESL) pronunciation using the ASR feature in the Microsoft 365 product suite, Transcribe (MS-T). The study involved adult ESL learners at a Canadian university that partook in a language proficiency test. We examined the audio recordings of 56 candidates during the pronunciation portion of the test. Building on previous studies that found a strong correlation between scores from Google Voice Typing and human raters, the current study conducted a similar analysis comparing scores derived from MS-T to both human ratings and Google Voice Typing. Our findings indicate that the ASR capabilities of MS-T, similar to Google Voice Typing, can assume an important role in L2 speaking assessment by providing objectivity and reliability to the testing process, expediting scoring, and reducing costs.
publishDate 2024
dc.date.none.fl_str_mv 2024
2024-02-12
dc.type.none.fl_str_mv book part
http://purl.org/coar/resource_type/c_3248
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/bookPart
format bookPart
dc.identifier.none.fl_str_mv https://riunet.upv.es/handle/10251/206633
url https://riunet.upv.es/handle/10251/206633
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Reconocimiento - No comercial - Compartir igual (by-nc-sa)
http://creativecommons.org/licenses/by-nc-sa/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Reconocimiento - No comercial - Compartir igual (by-nc-sa)
http://creativecommons.org/licenses/by-nc-sa/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Editorial Universitat Politècnica de València
publisher.none.fl_str_mv Editorial Universitat Politècnica de València
dc.source.none.fl_str_mv reponame:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
instname:Universitat Politècnica de València (UPV)
instname_str Universitat Politècnica de València (UPV)
reponame_str RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
collection RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869408637584146432
score 15,81155

Similares en LA Referencia