RAGBOT CLI: a Python library for running and evaluating retrieval-augmented generation chatbots

Fine-tuning Retrieval-Augmented Generation (RAG) chatbots is challenging due to the many interdependent parameters affecting performance. RAGBOT CLI is a terminal-based Python tool built atop the LangChain framework that enables systematic experimentation with RAG configurations and automated evalua...

Descripción completa

Detalles Bibliográficos
Autores: Martínez Amodia, Rubén, Tirnauca, Cristina|||0000-0002-7129-2237, Zorrilla Pantaleón, Marta E.|||0000-0002-0475-8834
Tipo de recurso: artículo
Fecha de publicación:2025
País:España
Institución:Universidad de Cantabria (UC)
Repositorio:UCrea Repositorio Abierto de la Universidad de Cantabria
Idioma:inglés
OAI Identifier:oai:repositorio.unican.es:10902/38693
Acceso en línea:https://hdl.handle.net/10902/38693
Access Level:acceso abierto
Palabra clave:Conversational assistant
Retrieval-augmented generation
Large language model
Generative artificial intelligence
id ES_f5989ac3b0d41bc56df9a641c92c754a
oai_identifier_str oai:repositorio.unican.es:10902/38693
network_acronym_str ES
network_name_str España
repository_id_str
spelling RAGBOT CLI: a Python library for running and evaluating retrieval-augmented generation chatbotsMartínez Amodia, RubénTirnauca, Cristina|||0000-0002-7129-2237Zorrilla Pantaleón, Marta E.|||0000-0002-0475-8834Conversational assistantRetrieval-augmented generationLarge language modelGenerative artificial intelligenceFine-tuning Retrieval-Augmented Generation (RAG) chatbots is challenging due to the many interdependent parameters affecting performance. RAGBOT CLI is a terminal-based Python tool built atop the LangChain framework that enables systematic experimentation with RAG configurations and automated evaluation using both quantitative metrics (BLEU, ROUGE-L, semantic similarity) and qualitative ones (contextual relevance, response relevance, factual fidelity). Unlike existing frameworks, RAGBOT CLI offers a modular, project-oriented architecture and supports hybrid evaluation strategies, making it suitable for academic and professional use. This paper describes the architecture, functionalities, and practical applications, showcasing its potential impact on the development and evaluation of RAG-based chatbots.This work was funded by MCIN/AEI/10.13039/501100011033 and co-financed by the European Regional Development Fund (FEDER) through the following projects: PID2021-124502OB-C42, PID2022-139237NB-I00, and PID2023-146243OB-I00. The research was conducted in collaboration with Fagor Electrónica, SDS division, through the project FLOTTY - Asistente logístico basado en IA generativa para la planificación de operaciones de transporte.ElsevierUniversidad de Cantabria20252025-12-01journal articlehttp://purl.org/coar/resource_type/c_6501NAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/articlehttps://hdl.handle.net/10902/38693SoftwareX, 2025, 32, 102458reponame:UCrea Repositorio Abierto de la Universidad de Cantabriainstname:Universidad de Cantabria (UC)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2Attribution-NonCommercial 4.0 Internationalhttp://creativecommons.org/licenses/by-nc/4.0/info:eu-repo/semantics/openAccessoai:repositorio.unican.es:10902/386932026-06-02T12:39:31Z
dc.title.none.fl_str_mv RAGBOT CLI: a Python library for running and evaluating retrieval-augmented generation chatbots
title RAGBOT CLI: a Python library for running and evaluating retrieval-augmented generation chatbots
spellingShingle RAGBOT CLI: a Python library for running and evaluating retrieval-augmented generation chatbots
Martínez Amodia, Rubén
Conversational assistant
Retrieval-augmented generation
Large language model
Generative artificial intelligence
title_short RAGBOT CLI: a Python library for running and evaluating retrieval-augmented generation chatbots
title_full RAGBOT CLI: a Python library for running and evaluating retrieval-augmented generation chatbots
title_fullStr RAGBOT CLI: a Python library for running and evaluating retrieval-augmented generation chatbots
title_full_unstemmed RAGBOT CLI: a Python library for running and evaluating retrieval-augmented generation chatbots
title_sort RAGBOT CLI: a Python library for running and evaluating retrieval-augmented generation chatbots
dc.creator.none.fl_str_mv Martínez Amodia, Rubén
Tirnauca, Cristina|||0000-0002-7129-2237
Zorrilla Pantaleón, Marta E.|||0000-0002-0475-8834
author Martínez Amodia, Rubén
author_facet Martínez Amodia, Rubén
Tirnauca, Cristina|||0000-0002-7129-2237
Zorrilla Pantaleón, Marta E.|||0000-0002-0475-8834
author_role author
author2 Tirnauca, Cristina|||0000-0002-7129-2237
Zorrilla Pantaleón, Marta E.|||0000-0002-0475-8834
author2_role author
author
dc.contributor.none.fl_str_mv Universidad de Cantabria
dc.subject.none.fl_str_mv Conversational assistant
Retrieval-augmented generation
Large language model
Generative artificial intelligence
topic Conversational assistant
Retrieval-augmented generation
Large language model
Generative artificial intelligence
description Fine-tuning Retrieval-Augmented Generation (RAG) chatbots is challenging due to the many interdependent parameters affecting performance. RAGBOT CLI is a terminal-based Python tool built atop the LangChain framework that enables systematic experimentation with RAG configurations and automated evaluation using both quantitative metrics (BLEU, ROUGE-L, semantic similarity) and qualitative ones (contextual relevance, response relevance, factual fidelity). Unlike existing frameworks, RAGBOT CLI offers a modular, project-oriented architecture and supports hybrid evaluation strategies, making it suitable for academic and professional use. This paper describes the architecture, functionalities, and practical applications, showcasing its potential impact on the development and evaluation of RAG-based chatbots.
publishDate 2025
dc.date.none.fl_str_mv 2025
2025-12-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/10902/38693
url https://hdl.handle.net/10902/38693
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial 4.0 International
http://creativecommons.org/licenses/by-nc/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial 4.0 International
http://creativecommons.org/licenses/by-nc/4.0/
eu_rights_str_mv openAccess
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv SoftwareX, 2025, 32, 102458
reponame:UCrea Repositorio Abierto de la Universidad de Cantabria
instname:Universidad de Cantabria (UC)
instname_str Universidad de Cantabria (UC)
reponame_str UCrea Repositorio Abierto de la Universidad de Cantabria
collection UCrea Repositorio Abierto de la Universidad de Cantabria
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869424604424962048
score 15,812429

Similares en LA Referencia