Machine learning from crowds using candidate set-based labelling

Crowdsourcing is a popular cheap alternative in machine learning for gathering information from a set of annotators. Learning from crowd-labelled data involves dealing with its inherent uncertainty and inconsistencies. In the classical framework, each annotator provides a single label per example, w...

Full description

Bibliographic Details
Authors: Beñaran-Muñoz, Iker, Hernández-González, Jerónimo, Pérez, Aritz
Format: article
Status:Published version
Publication Date:2022
Country:España
Institution:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
Repository:Recercat. Dipósit de la Recerca de Catalunya
OAI Identifier:oai:recercat.cat:2445/189544
Online Access:https://hdl.handle.net/2445/189544
Access Level:Open access
Keyword:Aprenentatge automàtic
Cultura participativa
Dades massives
Machine learning
Participatory culture
Big data
id ES_b8bcef0c149ced7460b38ad7550a8c19
oai_identifier_str oai:recercat.cat:2445/189544
network_acronym_str ES
network_name_str España
repository_id_str
spelling Machine learning from crowds using candidate set-based labellingBeñaran-Muñoz, IkerHernández-González, JerónimoPérez, AritzAprenentatge automàticCultura participativaDades massivesMachine learningParticipatory cultureBig dataCrowdsourcing is a popular cheap alternative in machine learning for gathering information from a set of annotators. Learning from crowd-labelled data involves dealing with its inherent uncertainty and inconsistencies. In the classical framework, each annotator provides a single label per example, which fails to capture the complete knowledge of annotators. We propose candidate labelling, that is, to allow annotators to provide a set of candidate labels for each example and thus express their doubts. We propose an appropriate model for the annotators, and present two novel learning methods that deal with the two basic steps (label aggregation and model learning) sequentially or jointly. Our empirical study shows the advantage of candidate labelling and the proposed methods with respect to the classical framework.Institute of Electrical and Electronics Engineers (IEEE)2022202220222022info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/2445/189544Articles publicats en revistes (Matemàtiques i Informàtica)reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)InglésReproducció del document publicat a: https://doi.org/10.1109/MIS.2022.3205053IEEE Intelligent Systems, 2022https://doi.org/10.1109/MIS.2022.3205053cc by-nc-nd (c) Beñaran-Muñoz, Iker et al., 2022http://creativecommons.org/licenses/by-nc-nd/3.0/es/info:eu-repo/semantics/openAccessoai:recercat.cat:2445/1895442026-05-29T05:05:01Z
dc.title.none.fl_str_mv Machine learning from crowds using candidate set-based labelling
title Machine learning from crowds using candidate set-based labelling
spellingShingle Machine learning from crowds using candidate set-based labelling
Beñaran-Muñoz, Iker
Aprenentatge automàtic
Cultura participativa
Dades massives
Machine learning
Participatory culture
Big data
title_short Machine learning from crowds using candidate set-based labelling
title_full Machine learning from crowds using candidate set-based labelling
title_fullStr Machine learning from crowds using candidate set-based labelling
title_full_unstemmed Machine learning from crowds using candidate set-based labelling
title_sort Machine learning from crowds using candidate set-based labelling
dc.creator.none.fl_str_mv Beñaran-Muñoz, Iker
Hernández-González, Jerónimo
Pérez, Aritz
author Beñaran-Muñoz, Iker
author_facet Beñaran-Muñoz, Iker
Hernández-González, Jerónimo
Pérez, Aritz
author_role author
author2 Hernández-González, Jerónimo
Pérez, Aritz
author2_role author
author
dc.subject.none.fl_str_mv Aprenentatge automàtic
Cultura participativa
Dades massives
Machine learning
Participatory culture
Big data
topic Aprenentatge automàtic
Cultura participativa
Dades massives
Machine learning
Participatory culture
Big data
description Crowdsourcing is a popular cheap alternative in machine learning for gathering information from a set of annotators. Learning from crowd-labelled data involves dealing with its inherent uncertainty and inconsistencies. In the classical framework, each annotator provides a single label per example, which fails to capture the complete knowledge of annotators. We propose candidate labelling, that is, to allow annotators to provide a set of candidate labels for each example and thus express their doubts. We propose an appropriate model for the annotators, and present two novel learning methods that deal with the two basic steps (label aggregation and model learning) sequentially or jointly. Our empirical study shows the advantage of candidate labelling and the proposed methods with respect to the classical framework.
publishDate 2022
dc.date.none.fl_str_mv 2022
2022
2022
2022
dc.type.none.fl_str_mv info:eu-repo/semantics/publishedVersion
info:eu-repo/semantics/article
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/2445/189544
url https://hdl.handle.net/2445/189544
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Reproducció del document publicat a: https://doi.org/10.1109/MIS.2022.3205053
IEEE Intelligent Systems, 2022
https://doi.org/10.1109/MIS.2022.3205053
dc.rights.none.fl_str_mv cc by-nc-nd (c) Beñaran-Muñoz, Iker et al., 2022
http://creativecommons.org/licenses/by-nc-nd/3.0/es/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv cc by-nc-nd (c) Beñaran-Muñoz, Iker et al., 2022
http://creativecommons.org/licenses/by-nc-nd/3.0/es/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Institute of Electrical and Electronics Engineers (IEEE)
publisher.none.fl_str_mv Institute of Electrical and Electronics Engineers (IEEE)
dc.source.none.fl_str_mv Articles publicats en revistes (Matemàtiques i Informàtica)
reponame:Recercat. Dipósit de la Recerca de Catalunya
instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
instname_str Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
reponame_str Recercat. Dipósit de la Recerca de Catalunya
collection Recercat. Dipósit de la Recerca de Catalunya
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869417675154784256
score 15,811543

Similar Items