Efficient Deep Ensembles by Averaging Neural Networks in Parameter Space

Trabajo fin de máster presentado en la Universidad Politécnica de Cataluña, Master of Science in Advanced Mathematics and Mathematical Engineering (MAMME).--2021-10-21

Detalles Bibliográficos
Autor: Norris Mitchell, Philip
Tipo de recurso: tesis de maestría
Fecha de publicación:2021
País:España
Institución:Consejo Superior de Investigaciones Científicas (CSIC)
Repositorio:DIGITAL.CSIC. Repositorio Institucional del CSIC
OAI Identifier:oai:digital.csic.es:10261/264981
Acceso en línea:http://hdl.handle.net/10261/264981
Access Level:acceso abierto
Palabra clave:Ensemble learning
Deep ensembles
Knowledge distillation
Permutation learning
id ES_a5460cdc56fb0c69a5d823c0280fe5f2
oai_identifier_str oai:digital.csic.es:10261/264981
network_acronym_str ES
network_name_str España
repository_id_str
spelling Efficient Deep Ensembles by Averaging Neural Networks in Parameter SpaceNorris Mitchell, PhilipEnsemble learningDeep ensemblesKnowledge distillationPermutation learningTrabajo fin de máster presentado en la Universidad Politécnica de Cataluña, Master of Science in Advanced Mathematics and Mathematical Engineering (MAMME).--2021-10-21Although deep ensembles provide large accuracy boosts relative to individual models, their use is not widespread in environments in which computational constraints are limited, as deep ensembles require storing M models and require M forward passes at prediction time. We propose a novel, computationally efficient alternative, which we name permAVG. Although deep ensembles cannot simply be average in parameter space, as all models find distinct perhaps distant local optima, permAVG exploits the symmetries of the loss landscape by learning permutations, such that all M models can be permuted into the same local optimum and can thereafter safely be averagedUniversidad Politécnica de CataluñaCSIC-UPC - Instituto de Robótica e Informática Industrial (IRII)Agudo, AntonioRuiz Ovejero, AdriàConsejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]2022202220212022info:eu-repo/semantics/masterThesishttp://purl.org/coar/resource_type/c_bdcchttp://hdl.handle.net/10261/264981reponame:DIGITAL.CSIC. Repositorio Institucional del CSICinstname:Consejo Superior de Investigaciones Científicas (CSIC)Ingléshttp://hdl.handle.net/2117/356936Síinfo:eu-repo/semantics/openAccessoai:digital.csic.es:10261/2649812026-05-22T06:33:51Z
dc.title.none.fl_str_mv Efficient Deep Ensembles by Averaging Neural Networks in Parameter Space
title Efficient Deep Ensembles by Averaging Neural Networks in Parameter Space
spellingShingle Efficient Deep Ensembles by Averaging Neural Networks in Parameter Space
Norris Mitchell, Philip
Ensemble learning
Deep ensembles
Knowledge distillation
Permutation learning
title_short Efficient Deep Ensembles by Averaging Neural Networks in Parameter Space
title_full Efficient Deep Ensembles by Averaging Neural Networks in Parameter Space
title_fullStr Efficient Deep Ensembles by Averaging Neural Networks in Parameter Space
title_full_unstemmed Efficient Deep Ensembles by Averaging Neural Networks in Parameter Space
title_sort Efficient Deep Ensembles by Averaging Neural Networks in Parameter Space
dc.creator.none.fl_str_mv Norris Mitchell, Philip
author Norris Mitchell, Philip
author_facet Norris Mitchell, Philip
author_role author
dc.contributor.none.fl_str_mv Agudo, Antonio
Ruiz Ovejero, Adrià
Consejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]
dc.subject.none.fl_str_mv Ensemble learning
Deep ensembles
Knowledge distillation
Permutation learning
topic Ensemble learning
Deep ensembles
Knowledge distillation
Permutation learning
description Trabajo fin de máster presentado en la Universidad Politécnica de Cataluña, Master of Science in Advanced Mathematics and Mathematical Engineering (MAMME).--2021-10-21
publishDate 2021
dc.date.none.fl_str_mv 2021
2022
2022
2022
dc.type.none.fl_str_mv info:eu-repo/semantics/masterThesis
http://purl.org/coar/resource_type/c_bdcc
format masterThesis
dc.identifier.none.fl_str_mv http://hdl.handle.net/10261/264981
url http://hdl.handle.net/10261/264981
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv http://hdl.handle.net/2117/356936
Sí
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.publisher.none.fl_str_mv Universidad Politécnica de Cataluña
CSIC-UPC - Instituto de Robótica e Informática Industrial (IRII)
publisher.none.fl_str_mv Universidad Politécnica de Cataluña
CSIC-UPC - Instituto de Robótica e Informática Industrial (IRII)
dc.source.none.fl_str_mv reponame:DIGITAL.CSIC. Repositorio Institucional del CSIC
instname:Consejo Superior de Investigaciones Científicas (CSIC)
instname_str Consejo Superior de Investigaciones Científicas (CSIC)
reponame_str DIGITAL.CSIC. Repositorio Institucional del CSIC
collection DIGITAL.CSIC. Repositorio Institucional del CSIC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869415601698504704
score 15,198674

Similares en LA Referencia