Researching the moment of truth: an experiment comparing in-the-moment and conventional web surveys to investigate online job applications

Understanding how people seek and apply for jobs online is crucial for addressing social inequality, discrimination, and aiding companies in attracting suitable candidates. Conventional surveys struggle to capture the nuances of online job searches that, as many online events, are characterized by r...

ver descrição completa

Detalhes bibliográficos
Autor: Ochoa Gómez, Carlos
Formato: artículo
Estado:Versión publicada
Fecha de publicación:2025
País:España
Recursos:Universitat Pompeu Fabra
Repositorio:Repositorio Digital de la UPF
OAI Identifier:oai:repositori.upf.edu:10230/72615
Acesso em linha:https://hdl.handle.net/10230/72615
http://dx.doi.org/10.12758/mda.2025.08
Access Level:acceso abierto
Palavra-chave:In-the-moment surveys
Metered data
Passive data
Web surveys
Digital traces
Job applications
id ES_ce3a8f000a7b7f5c7d3fa1f2bc26b7a9
oai_identifier_str oai:repositori.upf.edu:10230/72615
network_acronym_str ES
network_name_str España
repository_id_str
spelling Researching the moment of truth: an experiment comparing in-the-moment and conventional web surveys to investigate online job applicationsOchoa Gómez, CarlosIn-the-moment surveysMetered dataPassive dataWeb surveysDigital tracesJob applicationsUnderstanding how people seek and apply for jobs online is crucial for addressing social inequality, discrimination, and aiding companies in attracting suitable candidates. Conventional surveys struggle to capture the nuances of online job searches that, as many online events, are characterized by repetition, low distinctiveness, and limited emotional impact. These characteristics can lead to memory-related errors, becoming more likely as the time between the event and the survey increases. Passively collected data, such as metered data provided by online panel members who install tracking software on their browsing devices, offer an alternative. While these data provide objective insights into online job searches, they suffer other types of errors, and cannot capture subjective information and all potential objective data of interest. This paper explores an alternative approach: sending surveys to individuals in a metered panel shortly after an event of interest is detected through metered data. These "in-the-moment" surveys aim to fill in missing information not obtainable through passive data collection while reducing memory-related errors that affect conventional surveys. To assess the feasibility and benefits of this method, an experiment comparing in-the-moment surveys triggered by online job applications with conventional surveys was conducted in an opt-in online panel in Spain to research how people apply for a job online. The results reveal that metered panelists accept well in-the-moment surveys, displaying high participation levels and positive evaluations regarding effort and satisfaction, without perceiving an increased privacy risk. Moreover, the data indicate positive impacts on data quality, with longer and more detailed responses to open-ended questions. However, not all aspects saw substantial improvements, with the reduction of non-recall being weaker than expected, possibly due to participants' overconfidence in their memories. The significant disparities observed in substantive results between both types of surveys also suggest that participants are not fully aware of what they do not remember.This project received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant agreement No. 849165). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.GESIS Leibniz Institute for the Social Sciences2026202620252026info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfapplication/pdfhttps://hdl.handle.net/10230/72615http://dx.doi.org/10.12758/mda.2025.08reponame:Repositorio Digital de la UPFinstname:Universitat Pompeu FabraInglésMethods, Data, Analyses. 2025;19(2):87-124info:eu-repo/grantAgreement/EC/H2020/849165© The Author(s) 2025. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.Attribution 4.0 Internationalhttps://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:repositori.upf.edu:10230/726152026-06-12T07:21:37Z
dc.title.none.fl_str_mv Researching the moment of truth: an experiment comparing in-the-moment and conventional web surveys to investigate online job applications
title Researching the moment of truth: an experiment comparing in-the-moment and conventional web surveys to investigate online job applications
spellingShingle Researching the moment of truth: an experiment comparing in-the-moment and conventional web surveys to investigate online job applications
Ochoa Gómez, Carlos
In-the-moment surveys
Metered data
Passive data
Web surveys
Digital traces
Job applications
title_short Researching the moment of truth: an experiment comparing in-the-moment and conventional web surveys to investigate online job applications
title_full Researching the moment of truth: an experiment comparing in-the-moment and conventional web surveys to investigate online job applications
title_fullStr Researching the moment of truth: an experiment comparing in-the-moment and conventional web surveys to investigate online job applications
title_full_unstemmed Researching the moment of truth: an experiment comparing in-the-moment and conventional web surveys to investigate online job applications
title_sort Researching the moment of truth: an experiment comparing in-the-moment and conventional web surveys to investigate online job applications
dc.creator.none.fl_str_mv Ochoa Gómez, Carlos
author Ochoa Gómez, Carlos
author_facet Ochoa Gómez, Carlos
author_role author
dc.subject.none.fl_str_mv In-the-moment surveys
Metered data
Passive data
Web surveys
Digital traces
Job applications
topic In-the-moment surveys
Metered data
Passive data
Web surveys
Digital traces
Job applications
description Understanding how people seek and apply for jobs online is crucial for addressing social inequality, discrimination, and aiding companies in attracting suitable candidates. Conventional surveys struggle to capture the nuances of online job searches that, as many online events, are characterized by repetition, low distinctiveness, and limited emotional impact. These characteristics can lead to memory-related errors, becoming more likely as the time between the event and the survey increases. Passively collected data, such as metered data provided by online panel members who install tracking software on their browsing devices, offer an alternative. While these data provide objective insights into online job searches, they suffer other types of errors, and cannot capture subjective information and all potential objective data of interest. This paper explores an alternative approach: sending surveys to individuals in a metered panel shortly after an event of interest is detected through metered data. These "in-the-moment" surveys aim to fill in missing information not obtainable through passive data collection while reducing memory-related errors that affect conventional surveys. To assess the feasibility and benefits of this method, an experiment comparing in-the-moment surveys triggered by online job applications with conventional surveys was conducted in an opt-in online panel in Spain to research how people apply for a job online. The results reveal that metered panelists accept well in-the-moment surveys, displaying high participation levels and positive evaluations regarding effort and satisfaction, without perceiving an increased privacy risk. Moreover, the data indicate positive impacts on data quality, with longer and more detailed responses to open-ended questions. However, not all aspects saw substantial improvements, with the reduction of non-recall being weaker than expected, possibly due to participants' overconfidence in their memories. The significant disparities observed in substantive results between both types of surveys also suggest that participants are not fully aware of what they do not remember.
publishDate 2025
dc.date.none.fl_str_mv 2025
2026
2026
2026
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/10230/72615
http://dx.doi.org/10.12758/mda.2025.08
url https://hdl.handle.net/10230/72615
http://dx.doi.org/10.12758/mda.2025.08
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Methods, Data, Analyses. 2025;19(2):87-124
info:eu-repo/grantAgreement/EC/H2020/849165
dc.rights.none.fl_str_mv Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv GESIS Leibniz Institute for the Social Sciences
publisher.none.fl_str_mv GESIS Leibniz Institute for the Social Sciences
dc.source.none.fl_str_mv reponame:Repositorio Digital de la UPF
instname:Universitat Pompeu Fabra
instname_str Universitat Pompeu Fabra
reponame_str Repositorio Digital de la UPF
collection Repositorio Digital de la UPF
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869419971839262720
score 15.812429

Registos relacionados