Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles

This is the author’s version of a work that was accepted for publication in Neurocomputing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have be...

ver descrição completa

Detalhes bibliográficos
Autores: Hernández Lobato, Daniel, Martínez Muñoz, Gonzalo, Suárez González, Alberto
Formato: artículo
Fecha de publicación:2011
País:España
Recursos:Universidad Autónoma de Madrid
Repositorio:Biblos-e Archivo. Repositorio Institucional de la UAM
Idioma:inglés
OAI Identifier:oai:repositorio.uam.es:10486/664050
Acesso em linha:http://hdl.handle.net/10486/664050
https://dx.doi.org/10.1016/j.neucom.2011.03.001
Access Level:acceso abierto
Palavra-chave:Bagging
Boosting
Ensemble learning
Ensemble pruning
Regression
Semidefinite programming
Informática
id ES_6a6027eb5a1fa25a3e9991c2ba9efc3c
oai_identifier_str oai:repositorio.uam.es:10486/664050
network_acronym_str ES
network_name_str España
repository_id_str
spelling Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensemblesHernández Lobato, DanielMartínez Muñoz, GonzaloSuárez González, AlbertoBaggingBoostingEnsemble learningEnsemble pruningRegressionSemidefinite programmingInformáticaThis is the author’s version of a work that was accepted for publication in Neurocomputing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Neurocomputing 74.12-13 (2011) DOI: 10.1016/j.neucom.2011.03.001Identifying the optimal subset of regressors in a regression bagging ensemble is a difficult task that has exponential cost in the size of the ensemble. In this article we analyze two approximate techniques especially devised to address this problem. The first strategy constructs a relaxed version of the problem that can be solved using Semidefinite Programming. The second one is based on modifying the order of aggregation of the regressors. Ordered Aggregation is a simple forward selection algorithm that incorporates at each step the regressor that reduces the training error of the current subensemble the most. Both techniques can be used to identify subensembles that are close to the optimal ones, which can be obtained by exhaustive search at a larger computational cost. Experiments in a wide variety of synthetic and real-world regression problems show that pruned ensembles composed of only 20% of the initial regressors often have better generalization performance than the original bagging ensembles. These improvements are due to a reduction in the bias and the covariance components of the generalization error. Subensembles obtained using either SDP or Ordered Aggregation generally outperform subensembles obtained by other ensemble pruning methods and ensembles generated by the Adaboost.R2 algorithm, negative correlation learning or regularized linear stacked generalization. Ordered Aggregation has a slightly better overall performance than SDP in the problems investigated. However, the difference is not statistically significant. Ordered Aggregation has the further advantage that it produces a nested sequence of near-optimal subensembles of increasing size with no additional computational cost.The authors acknowledge support from the Spanish Ministerio de Ciencia e Innovación, Project TIN2010-21575-C02-02.ElsevierDepartamento de Ingeniería InformáticaEscuela Politécnica SuperiorAprendizaje Automático (ING EPS-001)20112011-06-01research articlehttp://purl.org/coar/resource_type/c_2df8fbb1AMhttp://purl.org/coar/version/c_ab4af688f83e57aainfo:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10486/664050https://dx.doi.org/10.1016/j.neucom.2011.03.001reponame:Biblos-e Archivo. Repositorio Institucional de la UAMinstname:Universidad Autónoma de MadridInglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:repositorio.uam.es:10486/6640502026-06-23T12:46:27Z
dc.title.none.fl_str_mv Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles
title Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles
spellingShingle Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles
Hernández Lobato, Daniel
Bagging
Boosting
Ensemble learning
Ensemble pruning
Regression
Semidefinite programming
Informática
title_short Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles
title_full Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles
title_fullStr Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles
title_full_unstemmed Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles
title_sort Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles
dc.creator.none.fl_str_mv Hernández Lobato, Daniel
Martínez Muñoz, Gonzalo
Suárez González, Alberto
author Hernández Lobato, Daniel
author_facet Hernández Lobato, Daniel
Martínez Muñoz, Gonzalo
Suárez González, Alberto
author_role author
author2 Martínez Muñoz, Gonzalo
Suárez González, Alberto
author2_role author
author
dc.contributor.none.fl_str_mv Departamento de Ingeniería Informática
Escuela Politécnica Superior
Aprendizaje Automático (ING EPS-001)
dc.subject.none.fl_str_mv Bagging
Boosting
Ensemble learning
Ensemble pruning
Regression
Semidefinite programming
Informática
topic Bagging
Boosting
Ensemble learning
Ensemble pruning
Regression
Semidefinite programming
Informática
description This is the author’s version of a work that was accepted for publication in Neurocomputing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Neurocomputing 74.12-13 (2011) DOI: 10.1016/j.neucom.2011.03.001
publishDate 2011
dc.date.none.fl_str_mv 2011
2011-06-01
dc.type.none.fl_str_mv research article
http://purl.org/coar/resource_type/c_2df8fbb1
AM
http://purl.org/coar/version/c_ab4af688f83e57aa
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10486/664050
https://dx.doi.org/10.1016/j.neucom.2011.03.001
url http://hdl.handle.net/10486/664050
https://dx.doi.org/10.1016/j.neucom.2011.03.001
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:Biblos-e Archivo. Repositorio Institucional de la UAM
instname:Universidad Autónoma de Madrid
instname_str Universidad Autónoma de Madrid
reponame_str Biblos-e Archivo. Repositorio Institucional de la UAM
collection Biblos-e Archivo. Repositorio Institucional de la UAM
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869410107149778944
score 15,300724

Registos relacionados