A comparative analysis of encoder only and decoder only models in intent classification and sentiment analysis: navigating the trade-offs in model size and performance

Intent classification and sentiment analysis stand as pivotal tasks in natural language understanding (NLU), with applications ranging from virtual assistants to customer service. The advent of transformer-based models has significantly enhanced the performance of various NLP tasks, with encoder-onl...

Descripción completa

Detalles Bibliográficos
Autores: Benayas Alamos, Alberto José, Sicilia Urbán, Miguel Ángel|||0000-0003-3067-4180, Mora Cantallops, Marçal|||0000-0002-2480-1078
Tipo de recurso: artículo
Fecha de publicación:2024
País:España
Institución:Universidad de Alcalá (UAH)
Repositorio:e_Buah Biblioteca Digital Universidad de Alcalá
Idioma:inglés
OAI Identifier:oai:ebuah.uah.es:10017/68273
Acceso en línea:http://hdl.handle.net/10017/68273
https://dx.doi.org/10.1007/s10579-024-09796-y
Access Level:acceso abierto
Palabra clave:Intent classification
Sentiment analysis
Large language models
Conversational AI
Informática
Computer science
id ES_d0aebaf7dbbb4e8e8e802cb0eb1454f0
oai_identifier_str oai:ebuah.uah.es:10017/68273
network_acronym_str ES
network_name_str España
repository_id_str
spelling A comparative analysis of encoder only and decoder only models in intent classification and sentiment analysis: navigating the trade-offs in model size and performanceBenayas Alamos, Alberto JoséSicilia Urbán, Miguel Ángel|||0000-0003-3067-4180Mora Cantallops, Marçal|||0000-0002-2480-1078Intent classificationSentiment analysisLarge language modelsConversational AIInformáticaComputer scienceIntent classification and sentiment analysis stand as pivotal tasks in natural language understanding (NLU), with applications ranging from virtual assistants to customer service. The advent of transformer-based models has significantly enhanced the performance of various NLP tasks, with encoder-only architectures gaining prominence for their effectiveness. More recently, there has been a surge in the development of larger and more powerful decoder-only models, traditionally employed for text generation tasks. This paper aims to answer the question of whether the colossal scale of newer decoder-only language models is essential for real-world applications. The investigation involves a performance comparison between these decoder-only models and the well-established encoder-only models specifically in the domains of intent classification and sentiment analysis. The results of our study indicate that, for tasks involving natural language understanding, encoder-only models generally outperform decoder-only models, all while demanding a fraction of the computational resources. This sheds light on the practicality and efficiency of encoder-only architectures in comparison to their decoder-only counterparts in real-world applications, providing valuable insights for the advancement of natural language processing technologies.Springer Nature20242024-12-07journal articlehttp://purl.org/coar/resource_type/c_6501NAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10017/68273https://dx.doi.org/10.1007/s10579-024-09796-yreponame:e_Buah Biblioteca Digital Universidad de Alcaláinstname:Universidad de Alcalá (UAH)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2Attribution-NonCommercial-NoDerivatives 4.0 Internationalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:ebuah.uah.es:10017/682732026-06-18T11:13:07Z
dc.title.none.fl_str_mv A comparative analysis of encoder only and decoder only models in intent classification and sentiment analysis: navigating the trade-offs in model size and performance
title A comparative analysis of encoder only and decoder only models in intent classification and sentiment analysis: navigating the trade-offs in model size and performance
spellingShingle A comparative analysis of encoder only and decoder only models in intent classification and sentiment analysis: navigating the trade-offs in model size and performance
Benayas Alamos, Alberto José
Intent classification
Sentiment analysis
Large language models
Conversational AI
Informática
Computer science
title_short A comparative analysis of encoder only and decoder only models in intent classification and sentiment analysis: navigating the trade-offs in model size and performance
title_full A comparative analysis of encoder only and decoder only models in intent classification and sentiment analysis: navigating the trade-offs in model size and performance
title_fullStr A comparative analysis of encoder only and decoder only models in intent classification and sentiment analysis: navigating the trade-offs in model size and performance
title_full_unstemmed A comparative analysis of encoder only and decoder only models in intent classification and sentiment analysis: navigating the trade-offs in model size and performance
title_sort A comparative analysis of encoder only and decoder only models in intent classification and sentiment analysis: navigating the trade-offs in model size and performance
dc.creator.none.fl_str_mv Benayas Alamos, Alberto José
Sicilia Urbán, Miguel Ángel|||0000-0003-3067-4180
Mora Cantallops, Marçal|||0000-0002-2480-1078
author Benayas Alamos, Alberto José
author_facet Benayas Alamos, Alberto José
Sicilia Urbán, Miguel Ángel|||0000-0003-3067-4180
Mora Cantallops, Marçal|||0000-0002-2480-1078
author_role author
author2 Sicilia Urbán, Miguel Ángel|||0000-0003-3067-4180
Mora Cantallops, Marçal|||0000-0002-2480-1078
author2_role author
author
dc.subject.none.fl_str_mv Intent classification
Sentiment analysis
Large language models
Conversational AI
Informática
Computer science
topic Intent classification
Sentiment analysis
Large language models
Conversational AI
Informática
Computer science
description Intent classification and sentiment analysis stand as pivotal tasks in natural language understanding (NLU), with applications ranging from virtual assistants to customer service. The advent of transformer-based models has significantly enhanced the performance of various NLP tasks, with encoder-only architectures gaining prominence for their effectiveness. More recently, there has been a surge in the development of larger and more powerful decoder-only models, traditionally employed for text generation tasks. This paper aims to answer the question of whether the colossal scale of newer decoder-only language models is essential for real-world applications. The investigation involves a performance comparison between these decoder-only models and the well-established encoder-only models specifically in the domains of intent classification and sentiment analysis. The results of our study indicate that, for tasks involving natural language understanding, encoder-only models generally outperform decoder-only models, all while demanding a fraction of the computational resources. This sheds light on the practicality and efficiency of encoder-only architectures in comparison to their decoder-only counterparts in real-world applications, providing valuable insights for the advancement of natural language processing technologies.
publishDate 2024
dc.date.none.fl_str_mv 2024
2024-12-07
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10017/68273
https://dx.doi.org/10.1007/s10579-024-09796-y
url http://hdl.handle.net/10017/68273
https://dx.doi.org/10.1007/s10579-024-09796-y
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Springer Nature
publisher.none.fl_str_mv Springer Nature
dc.source.none.fl_str_mv reponame:e_Buah Biblioteca Digital Universidad de Alcalá
instname:Universidad de Alcalá (UAH)
instname_str Universidad de Alcalá (UAH)
reponame_str e_Buah Biblioteca Digital Universidad de Alcalá
collection e_Buah Biblioteca Digital Universidad de Alcalá
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869420195790979072
score 15.812429

Similares en LA Referencia