Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of likert items

One important problem in the measurement of non-cognitive characteristics such as personality traits and attitudes is that it has traditionally been made through Likert scales, which are susceptible to response biases such as social desirability (SDR) and acquiescent (ACQ) responding. Given the vari...

Descripción completa

Detalles Bibliográficos
Autores: Schames Kreitchmann, Rodrigo, Abad García, Francisco José, Ponsoda Gil, Vicente, Nieto, María Dolores, Morillo, Daniel
Tipo de recurso: artículo
Fecha de publicación:2019
País:España
Institución:Universidad Autónoma de Madrid
Repositorio:Biblos-e Archivo. Repositorio Institucional de la UAM
Idioma:inglés
OAI Identifier:oai:repositorio.uam.es:10486/691805
Acceso en línea:http://hdl.handle.net/10486/691805
https://dx.doi.org/10.3389/fpsyg.2019.02309
Access Level:acceso abierto
Palabra clave:Graded-scale
Forced-choice
Ipsative
Social desirability
Acquiescence
Personality
Big Five
Item response theory
Psicología
id ES_74622aa5c04ecfcbe55f67e20332b6c4
oai_identifier_str oai:repositorio.uam.es:10486/691805
network_acronym_str ES
network_name_str España
repository_id_str
spelling Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of likert itemsSchames Kreitchmann, RodrigoAbad García, Francisco JoséPonsoda Gil, VicenteNieto, María DoloresMorillo, DanielGraded-scaleForced-choiceIpsativeSocial desirabilityAcquiescencePersonalityBig FiveItem response theoryPsicologíaOne important problem in the measurement of non-cognitive characteristics such as personality traits and attitudes is that it has traditionally been made through Likert scales, which are susceptible to response biases such as social desirability (SDR) and acquiescent (ACQ) responding. Given the variability of these response styles in the population, ignoring their possible effects on the scores may compromise the fairness and the validity of the assessments. Also, response-style-induced errors of measurement can affect the reliability estimates and overestimate convergent validity by correlating higher with other Likert-scale-based measures. Conversely, it can attenuate the predictive power over non-Likert-based indicators, given that the scores contain more errors. This study compares the validity of the Big Five personality scores obtained: (1) ignoring the SDR and ACQ in graded-scale items (GSQ), (2) accounting for SDR and ACQ with a compensatory IRT model, and (3) using forced-choice blocks with a multi-unidimensional pairwise preference model (MUPP) variant for dominance items. The overall results suggest that ignoring SDR and ACQ offered the worst validity evidence, with a higher correlation between personality and SDR scores. The two remaining strategies have their own advantages and disadvantages. The results from the empirical reliability and the convergent validity analysis indicate that when modeling social desirability with graded-scale items, the SDR factor apparently captures part of the variance of the Agreeableness factor. On the other hand, the correlation between the corrected GSQ-based Openness to Experience scores, and the University Access Examination grades was higher than the one with the uncorrected GSQ-based scores, and considerably higher than that using the estimates from the forced-choice data. Conversely, the criterion-related validity of the Forced Choice Questionnaire (FCQ) scores was similar to the results found in meta-analytic studies, correlating higher with Conscientiousness. Nonetheless, the FCQ-scores had considerably lower reliabilities and would demand administering more blocks. Finally, the results are discussed, and some notes are provided for the treatment of SDR and ACQ in future studiesThis project was partially supported by three grants from the Spanish Ministry of Economy, Industry and Competitiveness (PSI2015-65557-P, PSI2017-85022-P, and FPI BES-2016-077814)Frontiers MediaDepartamento de Psicología Social y MetodologíaFacultad de Psicología20192019-10-15research articlehttp://purl.org/coar/resource_type/c_2df8fbb1VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10486/691805https://dx.doi.org/10.3389/fpsyg.2019.02309reponame:Biblos-e Archivo. Repositorio Institucional de la UAMinstname:Universidad Autónoma de MadridInglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:repositorio.uam.es:10486/6918052026-06-23T12:46:27Z
dc.title.none.fl_str_mv Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of likert items
title Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of likert items
spellingShingle Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of likert items
Schames Kreitchmann, Rodrigo
Graded-scale
Forced-choice
Ipsative
Social desirability
Acquiescence
Personality
Big Five
Item response theory
Psicología
title_short Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of likert items
title_full Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of likert items
title_fullStr Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of likert items
title_full_unstemmed Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of likert items
title_sort Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of likert items
dc.creator.none.fl_str_mv Schames Kreitchmann, Rodrigo
Abad García, Francisco José
Ponsoda Gil, Vicente
Nieto, María Dolores
Morillo, Daniel
author Schames Kreitchmann, Rodrigo
author_facet Schames Kreitchmann, Rodrigo
Abad García, Francisco José
Ponsoda Gil, Vicente
Nieto, María Dolores
Morillo, Daniel
author_role author
author2 Abad García, Francisco José
Ponsoda Gil, Vicente
Nieto, María Dolores
Morillo, Daniel
author2_role author
author
author
author
dc.contributor.none.fl_str_mv Departamento de Psicología Social y Metodología
Facultad de Psicología
dc.subject.none.fl_str_mv Graded-scale
Forced-choice
Ipsative
Social desirability
Acquiescence
Personality
Big Five
Item response theory
Psicología
topic Graded-scale
Forced-choice
Ipsative
Social desirability
Acquiescence
Personality
Big Five
Item response theory
Psicología
description One important problem in the measurement of non-cognitive characteristics such as personality traits and attitudes is that it has traditionally been made through Likert scales, which are susceptible to response biases such as social desirability (SDR) and acquiescent (ACQ) responding. Given the variability of these response styles in the population, ignoring their possible effects on the scores may compromise the fairness and the validity of the assessments. Also, response-style-induced errors of measurement can affect the reliability estimates and overestimate convergent validity by correlating higher with other Likert-scale-based measures. Conversely, it can attenuate the predictive power over non-Likert-based indicators, given that the scores contain more errors. This study compares the validity of the Big Five personality scores obtained: (1) ignoring the SDR and ACQ in graded-scale items (GSQ), (2) accounting for SDR and ACQ with a compensatory IRT model, and (3) using forced-choice blocks with a multi-unidimensional pairwise preference model (MUPP) variant for dominance items. The overall results suggest that ignoring SDR and ACQ offered the worst validity evidence, with a higher correlation between personality and SDR scores. The two remaining strategies have their own advantages and disadvantages. The results from the empirical reliability and the convergent validity analysis indicate that when modeling social desirability with graded-scale items, the SDR factor apparently captures part of the variance of the Agreeableness factor. On the other hand, the correlation between the corrected GSQ-based Openness to Experience scores, and the University Access Examination grades was higher than the one with the uncorrected GSQ-based scores, and considerably higher than that using the estimates from the forced-choice data. Conversely, the criterion-related validity of the Forced Choice Questionnaire (FCQ) scores was similar to the results found in meta-analytic studies, correlating higher with Conscientiousness. Nonetheless, the FCQ-scores had considerably lower reliabilities and would demand administering more blocks. Finally, the results are discussed, and some notes are provided for the treatment of SDR and ACQ in future studies
publishDate 2019
dc.date.none.fl_str_mv 2019
2019-10-15
dc.type.none.fl_str_mv research article
http://purl.org/coar/resource_type/c_2df8fbb1
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10486/691805
https://dx.doi.org/10.3389/fpsyg.2019.02309
url http://hdl.handle.net/10486/691805
https://dx.doi.org/10.3389/fpsyg.2019.02309
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Frontiers Media
publisher.none.fl_str_mv Frontiers Media
dc.source.none.fl_str_mv reponame:Biblos-e Archivo. Repositorio Institucional de la UAM
instname:Universidad Autónoma de Madrid
instname_str Universidad Autónoma de Madrid
reponame_str Biblos-e Archivo. Repositorio Institucional de la UAM
collection Biblos-e Archivo. Repositorio Institucional de la UAM
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869410891486724096
score 15.198674

Similares en LA Referencia