Differential length and overlap with the stem in multiple-choice item options: A pilot experiment

Multiple-choice items are extensively used across different assessment contexts. A crucial requirement for ensuring their validity is their correct development, and a number of item-writing guidelines have been proposed that support item developers. This experimental pilot study aimed to investigate...

Descripción completa

Detalles Bibliográficos
Autores: Casu, Giulia, García García, Carmen
Tipo de recurso: artículo
Fecha de publicación:2019
País:España
Institución:Universidad Autónoma de Madrid
Repositorio:Biblos-e Archivo. Repositorio Institucional de la UAM
Idioma:inglés
OAI Identifier:oai:repositorio.uam.es:10486/689866
Acceso en línea:http://hdl.handle.net/10486/689866
https://dx.doi.org/10.5093/psed2018a20
Access Level:acceso abierto
Palabra clave:Flawed items
Item difficulty
Item-writing guidelines
Length of options
Lexical overlap
Multiple-choice items
Psicología
id ES_6d6a2dcd3443f4eaf59a9d59bddf40da
oai_identifier_str oai:repositorio.uam.es:10486/689866
network_acronym_str ES
network_name_str España
repository_id_str
spelling Differential length and overlap with the stem in multiple-choice item options: A pilot experimentLa longitud diferencial y el solapamiento con el enunciado en las opciones de ítems de opción múltiple: Un experimento pilotoCasu, GiuliaGarcía García, CarmenFlawed itemsItem difficultyItem-writing guidelinesLength of optionsLexical overlapMultiple-choice itemsPsicologíaMultiple-choice items are extensively used across different assessment contexts. A crucial requirement for ensuring their validity is their correct development, and a number of item-writing guidelines have been proposed that support item developers. This experimental pilot study aimed to investigate the effect of violating two item-writing guidelines: the differential length of the correct option compared to distractors and its lexical overlap with the stem. Standard and flawed items, respectively adhering to and deviating from guidelines, were randomly assigned to 55 college students and compared in their psychometric functioning. Results indicated that, in general, flawed items tended to be easier and less subject to random answers than standard ones, but significant differences were few. Discrepancies between standard and flawed subtests approached statistical significance with medium effect sizes. Although of interest, findings must be cautiously interpreted due to the small sample size. Implications for future research are discussed.Los ítems de opción múltiple son ampliamente utilizados en contextos de evaluación muy variados. Un requisito muy importante para garantizar su validez es su correcta redacción, y para ayudar a conseguirlo se han desarrollado una serie de directrices. El objetivo de este estudio piloto experimental fue investigar el efecto del incumplimiento de dos de estas reglas, más concretamente, la longitud diferencial de la opción correcta comparada con los distractores y su solapamiento léxico con el enunciado. Para ello, se asignó aleatoriamente a 55 estudiantes a las condiciones de responder a ítems que respetaban o que incumplían las mencionadas directrices y se compararon las propiedades psicométricas conseguidas por los ítems. Los resultados indican que, en general, los ítems con incumplimientos tendían a ser más fáciles y a recibir menos respuestas aleatorias; no obstante, había pocas diferencias significativas y el tamaño del efecto era medio. Aunque de interés, estos resultados deben ser interpretados con cautela debido al escaso tamaño muestral. Se comentan las implicaciones para futuras investigacionesColegio Oficial de Psicólogos de MadridDepartamento de Psicología Social y MetodologíaFacultad de Psicología20192019-01-01research articlehttp://purl.org/coar/resource_type/c_2df8fbb1VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10486/689866https://dx.doi.org/10.5093/psed2018a20reponame:Biblos-e Archivo. Repositorio Institucional de la UAMinstname:Universidad Autónoma de MadridInglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:repositorio.uam.es:10486/6898662026-06-23T12:46:27Z
dc.title.none.fl_str_mv Differential length and overlap with the stem in multiple-choice item options: A pilot experiment
La longitud diferencial y el solapamiento con el enunciado en las opciones de ítems de opción múltiple: Un experimento piloto
title Differential length and overlap with the stem in multiple-choice item options: A pilot experiment
spellingShingle Differential length and overlap with the stem in multiple-choice item options: A pilot experiment
Casu, Giulia
Flawed items
Item difficulty
Item-writing guidelines
Length of options
Lexical overlap
Multiple-choice items
Psicología
title_short Differential length and overlap with the stem in multiple-choice item options: A pilot experiment
title_full Differential length and overlap with the stem in multiple-choice item options: A pilot experiment
title_fullStr Differential length and overlap with the stem in multiple-choice item options: A pilot experiment
title_full_unstemmed Differential length and overlap with the stem in multiple-choice item options: A pilot experiment
title_sort Differential length and overlap with the stem in multiple-choice item options: A pilot experiment
dc.creator.none.fl_str_mv Casu, Giulia
García García, Carmen
author Casu, Giulia
author_facet Casu, Giulia
García García, Carmen
author_role author
author2 García García, Carmen
author2_role author
dc.contributor.none.fl_str_mv Departamento de Psicología Social y Metodología
Facultad de Psicología
dc.subject.none.fl_str_mv Flawed items
Item difficulty
Item-writing guidelines
Length of options
Lexical overlap
Multiple-choice items
Psicología
topic Flawed items
Item difficulty
Item-writing guidelines
Length of options
Lexical overlap
Multiple-choice items
Psicología
description Multiple-choice items are extensively used across different assessment contexts. A crucial requirement for ensuring their validity is their correct development, and a number of item-writing guidelines have been proposed that support item developers. This experimental pilot study aimed to investigate the effect of violating two item-writing guidelines: the differential length of the correct option compared to distractors and its lexical overlap with the stem. Standard and flawed items, respectively adhering to and deviating from guidelines, were randomly assigned to 55 college students and compared in their psychometric functioning. Results indicated that, in general, flawed items tended to be easier and less subject to random answers than standard ones, but significant differences were few. Discrepancies between standard and flawed subtests approached statistical significance with medium effect sizes. Although of interest, findings must be cautiously interpreted due to the small sample size. Implications for future research are discussed.
publishDate 2019
dc.date.none.fl_str_mv 2019
2019-01-01
dc.type.none.fl_str_mv research article
http://purl.org/coar/resource_type/c_2df8fbb1
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10486/689866
https://dx.doi.org/10.5093/psed2018a20
url http://hdl.handle.net/10486/689866
https://dx.doi.org/10.5093/psed2018a20
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Colegio Oficial de Psicólogos de Madrid
publisher.none.fl_str_mv Colegio Oficial de Psicólogos de Madrid
dc.source.none.fl_str_mv reponame:Biblos-e Archivo. Repositorio Institucional de la UAM
instname:Universidad Autónoma de Madrid
instname_str Universidad Autónoma de Madrid
reponame_str Biblos-e Archivo. Repositorio Institucional de la UAM
collection Biblos-e Archivo. Repositorio Institucional de la UAM
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869410350526365696
score 15.300719

Similares en LA Referencia