Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings

After improving the SOCIO chatbot prototype model, we wanted to know how/if its usability has changed. An evidence-based empirical evaluation of the usability of SOCIO V1 (updated version) requires an extensive verification of the experimental results. A family of experiments is a method of verifica...

Descripción completa

Detalles Bibliográficos
Autores: Ren, Ranci, Pérez-Soler, Sara, Castro, John W., Dieste, Óscar, Acuña Castillo, Silvia Teresita
Tipo de recurso: artículo
Fecha de publicación:2022
País:España
Institución:Universidad Autónoma de Madrid
Repositorio:Biblos-e Archivo. Repositorio Institucional de la UAM
Idioma:inglés
OAI Identifier:oai:repositorio.uam.es:10486/711440
Acceso en línea:http://hdl.handle.net/10486/711440
https://dx.doi.org/10.1109/ACCESS.2022.3228772
Access Level:acceso abierto
Palabra clave:Chatbot
usability
family of experiments
Informática
id ES_ea8d885b8569fa44453494dc07facf32
oai_identifier_str oai:repositorio.uam.es:10486/711440
network_acronym_str ES
network_name_str España
repository_id_str
spelling Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic SettingsRen, RanciPérez-Soler, SaraCastro, John W.Dieste, ÓscarAcuña Castillo, Silvia TeresitaChatbotusabilityfamily of experimentsInformáticaAfter improving the SOCIO chatbot prototype model, we wanted to know how/if its usability has changed. An evidence-based empirical evaluation of the usability of SOCIO V1 (updated version) requires an extensive verification of the experimental results. A family of experiments is a method of verification whereby we can check if the experimental results are reproducible. Through comparison with the updated control tool Creately, we aimed to gain a better understanding of the usability of the collaborative modeling chatbot and how it could be improved based on experimental evidence of changes in terms of efficiency, effectiveness, satisfaction, and quality. A total of 87 students from three countries were recruited. We conducted a family of three experiments to compare the usability of SOCIO V1 and updated Creately in academic settings. Students appeared to be more satisfied with SOCIO V1, and SOCIO V1 scored better on completeness. There were no significant differences between the two tools regarding efficiency and quality. This study provides evidence on how to employ a family of experiments to improve chatbot usability and enrich knowledge on chatbot usability experimentation.This work was supported in part by the Spanish Ministry of Science, Innovation and Universities Research Grant under Grant PGC2018-097265-B-I00; in part by the Foundations for Augmented Trustworthy Low-Code Software Development (FINESSE) Project under Grant PID2021-122270OB-I00; in part by the Madrid Region Research and Development Program (FORmal Models and Technologies for Emerging Applications (FORTE) Project) under Grant P2018/TCS-4314; and in part by the Fundamentos Para la Ingeniería Automatizada de Chatbots (SATORI)-Universidad Autónoma de Madrid (UAM) Project under Grant TED2021-129381B-C21.IEEE Access, IEEEDepartamento de Ingeniería InformáticaEscuela Politécnica Superior20222022-12-12research articlehttp://purl.org/coar/resource_type/c_2df8fbb1VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10486/711440https://dx.doi.org/10.1109/ACCESS.2022.3228772reponame:Biblos-e Archivo. Repositorio Institucional de la UAMinstname:Universidad Autónoma de MadridInglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:repositorio.uam.es:10486/7114402026-06-23T12:46:27Z
dc.title.none.fl_str_mv Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings
title Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings
spellingShingle Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings
Ren, Ranci
Chatbot
usability
family of experiments
Informática
title_short Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings
title_full Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings
title_fullStr Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings
title_full_unstemmed Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings
title_sort Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings
dc.creator.none.fl_str_mv Ren, Ranci
Pérez-Soler, Sara
Castro, John W.
Dieste, Óscar
Acuña Castillo, Silvia Teresita
author Ren, Ranci
author_facet Ren, Ranci
Pérez-Soler, Sara
Castro, John W.
Dieste, Óscar
Acuña Castillo, Silvia Teresita
author_role author
author2 Pérez-Soler, Sara
Castro, John W.
Dieste, Óscar
Acuña Castillo, Silvia Teresita
author2_role author
author
author
author
dc.contributor.none.fl_str_mv Departamento de Ingeniería Informática
Escuela Politécnica Superior
dc.subject.none.fl_str_mv Chatbot
usability
family of experiments
Informática
topic Chatbot
usability
family of experiments
Informática
description After improving the SOCIO chatbot prototype model, we wanted to know how/if its usability has changed. An evidence-based empirical evaluation of the usability of SOCIO V1 (updated version) requires an extensive verification of the experimental results. A family of experiments is a method of verification whereby we can check if the experimental results are reproducible. Through comparison with the updated control tool Creately, we aimed to gain a better understanding of the usability of the collaborative modeling chatbot and how it could be improved based on experimental evidence of changes in terms of efficiency, effectiveness, satisfaction, and quality. A total of 87 students from three countries were recruited. We conducted a family of three experiments to compare the usability of SOCIO V1 and updated Creately in academic settings. Students appeared to be more satisfied with SOCIO V1, and SOCIO V1 scored better on completeness. There were no significant differences between the two tools regarding efficiency and quality. This study provides evidence on how to employ a family of experiments to improve chatbot usability and enrich knowledge on chatbot usability experimentation.
publishDate 2022
dc.date.none.fl_str_mv 2022
2022-12-12
dc.type.none.fl_str_mv research article
http://purl.org/coar/resource_type/c_2df8fbb1
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10486/711440
https://dx.doi.org/10.1109/ACCESS.2022.3228772
url http://hdl.handle.net/10486/711440
https://dx.doi.org/10.1109/ACCESS.2022.3228772
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv IEEE Access, IEEE
publisher.none.fl_str_mv IEEE Access, IEEE
dc.source.none.fl_str_mv reponame:Biblos-e Archivo. Repositorio Institucional de la UAM
instname:Universidad Autónoma de Madrid
instname_str Universidad Autónoma de Madrid
reponame_str Biblos-e Archivo. Repositorio Institucional de la UAM
collection Biblos-e Archivo. Repositorio Institucional de la UAM
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869423149741768704
score 15,300719

Similares en LA Referencia