Can large language models replace human in speech analysis?
Treballs finals del Màster de Fonaments de Ciència de Dades, Facultat de matemàtiques, Universitat de Barcelona. Curs: 2023-2024. Tutor: Santi Seguí Mesquida i Carolina Martínez Pérez
| Autor: | |
|---|---|
| Formato: | tesis de maestría |
| Fecha de publicación: | 2024 |
| País: | España |
| Recursos: | Universidad de Barcelona |
| Repositorio: | Dipòsit Digital de la UB |
| OAI Identifier: | oai:diposit.ub.edu:2445/212885 |
| Acesso em linha: | https://hdl.handle.net/2445/212885 |
| Access Level: | acceso abierto |
| Palavra-chave: | Tractament del llenguatge natural (Informàtica) Lingüística computacional Intel·ligència artificial Treballs de fi de màster Natural language processing (Computer science) Computational linguistics Artificial intelligence Master's thesis |
| id |
ES_70d3dd71a2dbc4349fcc3b9fbc622f9e |
|---|---|
| oai_identifier_str |
oai:diposit.ub.edu:2445/212885 |
| network_acronym_str |
ES |
| network_name_str |
España |
| repository_id_str |
|
| spelling |
Can large language models replace human in speech analysis?Gareta Casas, PolTractament del llenguatge natural (Informàtica)Lingüística computacionalIntel·ligència artificialTreballs de fi de màsterNatural language processing (Computer science)Computational linguisticsArtificial intelligenceMaster's thesisTreballs finals del Màster de Fonaments de Ciència de Dades, Facultat de matemàtiques, Universitat de Barcelona. Curs: 2023-2024. Tutor: Santi Seguí Mesquida i Carolina Martínez Pérez[en] This thesis delves into the rapidly growing domain of Large Language Models (LLMs) and examines their relevance in the insurance sector, specifically focusing on their use in speech analysis to evaluate service quality. With the rapid escalation in the popularity of LLMs, we have the opportunity to analyze their practical use, focusing on Generali Seguros’ customer service operations. This research is based on a partnership with Generali Seguros, which provided valuable access to audio recordings of their customer service calls and the associated evaluation templates used for assessing their teleoperators. The core objective is to investigate the potential and real-world applications of LLMs in analyzing and evaluating the quality of service provided by Generali’s teleoperators. To facilitate this, the study utilizes a secure and confidential environment provided by AWS, selecting commercially available models for analysis. The approach begins with converting the audio calls into Spanish text through an audioto-text model, followed by improvements to this transcription method. Next, the study evaluates a baseline LLM that supports multiple languages and allows for fine tuning. A significant aspect of this research includes addressing the challenges inherent in LLMs, such as their tendency towards ’inventing’ responses and providing vague answers. Efforts to mitigate these issues involve both employing the baseline model in English —anticipating better performance due to its primarily English training—and implements strategies to enhance its effectiveness. Additionally, fine-tuning of the model is conducted, with the objective of specializing the model to required task. Despite efforts to enhance the LLM, a notable finding of this study is the model’s consistent failure to predict the minority group in the data, underscoring the limitations of current commercial models in fulfilling this specific evaluative function. The thesis concludes that, while LLMs show promise, they are yet to fully meet the demands of specialized tasks such as nuanced speech analysis in customer service settings. For transparency and further research, all codes used in this study are made available in a GitHub repository (Gareta, 2023).Seguí Mesquida, SantiMartínez Pérez, Carolina2024info:eu-repo/semantics/masterThesisapplication/pdfhttps://hdl.handle.net/2445/212885Màster Oficial - Fonaments de la Ciència de Dadesreponame:Dipòsit Digital de la UBinstname:Universidad de BarcelonaIngléscc-by-nc-nd (c) Pol Gareta Casas, 2023codi: GPL (c) Pol Gareta Casas, 2023http://creativecommons.org/licenses/by-nc-nd/3.0/es/http://www.gnu.org/licenses/gpl-3.0.ca.htmlinfo:eu-repo/semantics/openAccessoai:diposit.ub.edu:2445/2128852026-05-27T06:46:51Z |
| dc.title.none.fl_str_mv |
Can large language models replace human in speech analysis? |
| title |
Can large language models replace human in speech analysis? |
| spellingShingle |
Can large language models replace human in speech analysis? Gareta Casas, Pol Tractament del llenguatge natural (Informàtica) Lingüística computacional Intel·ligència artificial Treballs de fi de màster Natural language processing (Computer science) Computational linguistics Artificial intelligence Master's thesis |
| title_short |
Can large language models replace human in speech analysis? |
| title_full |
Can large language models replace human in speech analysis? |
| title_fullStr |
Can large language models replace human in speech analysis? |
| title_full_unstemmed |
Can large language models replace human in speech analysis? |
| title_sort |
Can large language models replace human in speech analysis? |
| dc.creator.none.fl_str_mv |
Gareta Casas, Pol |
| author |
Gareta Casas, Pol |
| author_facet |
Gareta Casas, Pol |
| author_role |
author |
| dc.contributor.none.fl_str_mv |
Seguí Mesquida, Santi Martínez Pérez, Carolina |
| dc.subject.none.fl_str_mv |
Tractament del llenguatge natural (Informàtica) Lingüística computacional Intel·ligència artificial Treballs de fi de màster Natural language processing (Computer science) Computational linguistics Artificial intelligence Master's thesis |
| topic |
Tractament del llenguatge natural (Informàtica) Lingüística computacional Intel·ligència artificial Treballs de fi de màster Natural language processing (Computer science) Computational linguistics Artificial intelligence Master's thesis |
| description |
Treballs finals del Màster de Fonaments de Ciència de Dades, Facultat de matemàtiques, Universitat de Barcelona. Curs: 2023-2024. Tutor: Santi Seguí Mesquida i Carolina Martínez Pérez |
| publishDate |
2024 |
| dc.date.none.fl_str_mv |
2024 |
| dc.type.none.fl_str_mv |
info:eu-repo/semantics/masterThesis |
| format |
masterThesis |
| dc.identifier.none.fl_str_mv |
https://hdl.handle.net/2445/212885 |
| url |
https://hdl.handle.net/2445/212885 |
| dc.language.none.fl_str_mv |
Inglés |
| language_invalid_str_mv |
Inglés |
| dc.rights.none.fl_str_mv |
cc-by-nc-nd (c) Pol Gareta Casas, 2023 codi: GPL (c) Pol Gareta Casas, 2023 http://creativecommons.org/licenses/by-nc-nd/3.0/es/ http://www.gnu.org/licenses/gpl-3.0.ca.html info:eu-repo/semantics/openAccess |
| rights_invalid_str_mv |
cc-by-nc-nd (c) Pol Gareta Casas, 2023 codi: GPL (c) Pol Gareta Casas, 2023 http://creativecommons.org/licenses/by-nc-nd/3.0/es/ http://www.gnu.org/licenses/gpl-3.0.ca.html |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
application/pdf |
| dc.source.none.fl_str_mv |
Màster Oficial - Fonaments de la Ciència de Dades reponame:Dipòsit Digital de la UB instname:Universidad de Barcelona |
| instname_str |
Universidad de Barcelona |
| reponame_str |
Dipòsit Digital de la UB |
| collection |
Dipòsit Digital de la UB |
| repository.name.fl_str_mv |
|
| repository.mail.fl_str_mv |
|
| _version_ |
1869410608415244288 |
| score |
15,811543 |