Crowd Learning with Candidate Labeling: an EM-based Solution
Crowdsourcing is widely used nowadays in machine learning for data labeling. Although in the traditional case annotators are asked to provide a single label for each instance, novel approaches allow annotators, in case of doubt, to choose a subset of labels as a way to extract more information from...
| Autores: | , , |
|---|---|
| Tipo de recurso: | capítulo de libro |
| Estado: | Versión publicada |
| Fecha de publicación: | 2018 |
| País: | España |
| Institución: | Basque Center for Applied Mathematics (BCAM) |
| Repositorio: | BIRD. BCAM's Institutional Repository Data |
| OAI Identifier: | oai:bird.bcamath.org:20.500.11824/889 |
| Acceso en línea: | http://hdl.handle.net/20.500.11824/889 |
| Access Level: | acceso abierto |
| Palabra clave: | Supervised classification Crowdsourced labels Weak supervision Candidate labeling Expectation-maximization based method |
| Sumario: | Crowdsourcing is widely used nowadays in machine learning for data labeling. Although in the traditional case annotators are asked to provide a single label for each instance, novel approaches allow annotators, in case of doubt, to choose a subset of labels as a way to extract more information from them. In both the traditional and these novel approaches, the reliability of the labelers can be modeled based on the collections of labels that they provide. In this paper, we propose an Expectation-Maximization-based method for crowdsourced data with candidate sets. Iteratively the likelihood of the parameters that model the reliability of the labelers is maximized, while the ground truth is estimated. The experimental results suggest that the proposed method performs better than the baseline aggregation schemes in terms of estimated accuracy. |
|---|