Noise detection in classification problems
In many areas of knowledge, considerable amounts of time have been spent to comprehend and to treat noisy data, one of the most common problems regarding information collection, transmission and storage. These noisy data, when used for training Machine Learning techniques, lead to increased complexi...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis doctoral |
| Estado: | Versión publicada |
| Fecha de publicación: | 2016 |
| País: | Brasil |
| Institución: | Universidade de São Paulo (USP) |
| Repositorio: | Biblioteca Digital de Teses e Dissertações da USP |
| Idioma: | inglés |
| OAI Identifier: | oai:teses.usp.br:tde-29112016-155215 |
| Acceso en línea: | http://www.teses.usp.br/teses/disponiveis/55/55134/tde-29112016-155215/ |
| Access Level: | acceso abierto |
| Palabra clave: | Aprendizado de máquina Classification problems Detecção de ruídos Machine learning Meta-aprendizado. Meta-learning Noise detection Problemas de classificação |
| Sumario: | In many areas of knowledge, considerable amounts of time have been spent to comprehend and to treat noisy data, one of the most common problems regarding information collection, transmission and storage. These noisy data, when used for training Machine Learning techniques, lead to increased complexity in the induced classification models, higher processing time and reduced predictive power. Treating them in a preprocessing step may improve the data quality and the comprehension of the problem. This Thesis aims to investigate the use of data complexity measures capable to characterize the presence of noise in datasets, to develop new efficient noise ltering techniques in such subsamples of problems of noise identification compared to the state of art and to recommend the most properly suited techniques or ensembles for a specific dataset by meta-learning. Both artificial and real problem datasets were used in the experimental part of this work. They were obtained from public data repositories and a cooperation project. The evaluation was made through the analysis of the effect of artificially generated noise and also by the feedback of a domain expert. The reported experimental results show that the investigated proposals are promising. |
|---|