Joint generation of distractors for multiple-choice questions: a text-to-text approach

Generation of good-quality distractors is a key and time-consuming task associated with multiple-choice questions (MCQs), one of the assessment items that have dominated the educational field for years. Recent advances in language models and architectures present an opportunity for helping teachers...

Descripción completa

Detalles Bibliográficos
Autores: Rodriguez Torrealba, Ricardo, García López, Eva|||0000-0002-7598-3289, García Cabot, Antonio|||0000-0002-0298-3237
Tipo de recurso: artículo
Fecha de publicación:2025
País:España
Institución:Universidad de Alcalá (UAH)
Repositorio:e_Buah Biblioteca Digital Universidad de Alcalá
Idioma:inglés
OAI Identifier:oai:ebuah.uah.es:10017/67456
Acceso en línea:http://hdl.handle.net/10017/67456
https://dx.doi.org/10.32604/cmc.2025.062004
Access Level:acceso abierto
Palabra clave:Text-to-text
Distractor generation
Fine-tuning
FlanT5
LongT5
Multiple-choice
Questionnaire
Informática
Computer science
id ES_065dfcd173cc1e141a33102fcd8b57fb
oai_identifier_str oai:ebuah.uah.es:10017/67456
network_acronym_str ES
network_name_str España
repository_id_str
spelling Joint generation of distractors for multiple-choice questions: a text-to-text approachRodriguez Torrealba, RicardoGarcía López, Eva|||0000-0002-7598-3289García Cabot, Antonio|||0000-0002-0298-3237Text-to-textDistractor generationFine-tuningFlanT5LongT5Multiple-choiceQuestionnaireInformáticaComputer scienceGeneration of good-quality distractors is a key and time-consuming task associated with multiple-choice questions (MCQs), one of the assessment items that have dominated the educational field for years. Recent advances in language models and architectures present an opportunity for helping teachers to generate and update these elements to the required speed and scale of widespread increase in online education. This study focuses on a text-to-text approach for joints generation of distractors for MCQs, where the context, question and correct answer are used as input, while the set of distractors corresponds to the output, allowing the generation of three distractors in a singlemodel inference. By fine-tuning FlanT5 models and LongT5 with TGlobal attention using a RACE-based dataset, the potential of this approach is explored, demonstrating an improvement in the BLEU and ROUGE-L metrics when compared to previous works and a GPT-3.5 baseline. Additionally, BERTScore is introduced in the evaluation, showing that the fine-tuned models generate distractors semantically close to the reference, but the GPT-3.5 baseline still outperforms in this area. A tendency toward duplicating distractors is noted, although models fine-tuned with Low-Rank Adaptation (LoRA) and 4-bit quantization showcased a significant reduction in duplicated distractors.Universidad de AlcaláTech Science Press20252025-04-16journal articlehttp://purl.org/coar/resource_type/c_6501NAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10017/67456https://dx.doi.org/10.32604/cmc.2025.062004reponame:e_Buah Biblioteca Digital Universidad de Alcaláinstname:Universidad de Alcalá (UAH)InglésengUAH Not available CM-JIN-2021-034UAH Not available PIUAH21%2FIA-010UAH Not available PIUAH23%2FIA-007open accesshttp://purl.org/coar/access_right/c_abf2Attribution 4.0 Internationalhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:ebuah.uah.es:10017/674562026-06-18T11:13:07Z
dc.title.none.fl_str_mv Joint generation of distractors for multiple-choice questions: a text-to-text approach
title Joint generation of distractors for multiple-choice questions: a text-to-text approach
spellingShingle Joint generation of distractors for multiple-choice questions: a text-to-text approach
Rodriguez Torrealba, Ricardo
Text-to-text
Distractor generation
Fine-tuning
FlanT5
LongT5
Multiple-choice
Questionnaire
Informática
Computer science
title_short Joint generation of distractors for multiple-choice questions: a text-to-text approach
title_full Joint generation of distractors for multiple-choice questions: a text-to-text approach
title_fullStr Joint generation of distractors for multiple-choice questions: a text-to-text approach
title_full_unstemmed Joint generation of distractors for multiple-choice questions: a text-to-text approach
title_sort Joint generation of distractors for multiple-choice questions: a text-to-text approach
dc.creator.none.fl_str_mv Rodriguez Torrealba, Ricardo
García López, Eva|||0000-0002-7598-3289
García Cabot, Antonio|||0000-0002-0298-3237
author Rodriguez Torrealba, Ricardo
author_facet Rodriguez Torrealba, Ricardo
García López, Eva|||0000-0002-7598-3289
García Cabot, Antonio|||0000-0002-0298-3237
author_role author
author2 García López, Eva|||0000-0002-7598-3289
García Cabot, Antonio|||0000-0002-0298-3237
author2_role author
author
dc.subject.none.fl_str_mv Text-to-text
Distractor generation
Fine-tuning
FlanT5
LongT5
Multiple-choice
Questionnaire
Informática
Computer science
topic Text-to-text
Distractor generation
Fine-tuning
FlanT5
LongT5
Multiple-choice
Questionnaire
Informática
Computer science
description Generation of good-quality distractors is a key and time-consuming task associated with multiple-choice questions (MCQs), one of the assessment items that have dominated the educational field for years. Recent advances in language models and architectures present an opportunity for helping teachers to generate and update these elements to the required speed and scale of widespread increase in online education. This study focuses on a text-to-text approach for joints generation of distractors for MCQs, where the context, question and correct answer are used as input, while the set of distractors corresponds to the output, allowing the generation of three distractors in a singlemodel inference. By fine-tuning FlanT5 models and LongT5 with TGlobal attention using a RACE-based dataset, the potential of this approach is explored, demonstrating an improvement in the BLEU and ROUGE-L metrics when compared to previous works and a GPT-3.5 baseline. Additionally, BERTScore is introduced in the evaluation, showing that the fine-tuned models generate distractors semantically close to the reference, but the GPT-3.5 baseline still outperforms in this area. A tendency toward duplicating distractors is noted, although models fine-tuned with Low-Rank Adaptation (LoRA) and 4-bit quantization showcased a significant reduction in duplicated distractors.
publishDate 2025
dc.date.none.fl_str_mv 2025
2025-04-16
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10017/67456
https://dx.doi.org/10.32604/cmc.2025.062004
url http://hdl.handle.net/10017/67456
https://dx.doi.org/10.32604/cmc.2025.062004
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv UAH Not available CM-JIN-2021-034
UAH Not available PIUAH21%2FIA-010
UAH Not available PIUAH23%2FIA-007
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Tech Science Press
publisher.none.fl_str_mv Tech Science Press
dc.source.none.fl_str_mv reponame:e_Buah Biblioteca Digital Universidad de Alcalá
instname:Universidad de Alcalá (UAH)
instname_str Universidad de Alcalá (UAH)
reponame_str e_Buah Biblioteca Digital Universidad de Alcalá
collection e_Buah Biblioteca Digital Universidad de Alcalá
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869402913662566400
score 15.812429

Similares en LA Referencia