Revisiting Implicit and Explicit Averaging for Noisy Optimization

Explicit and implicit averaging are two well-known strategies for noisy optimization. Both strategies can counteract the disruptive effect of noise; however, a critical question remains: which one is more efficient? This question has been raised in many studies, with conflicting preferences and, in...

Descripción completa

Detalles Bibliográficos
Autores: Ahrari, A., Elsayed, S., Sarker, R., Essam, D., Coello, C.A.
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2023
País:España
Institución:Basque Center for Applied Mathematics (BCAM)
Repositorio:BIRD. BCAM's Institutional Repository Data
OAI Identifier:oai:bird.bcamath.org:20.500.11824/1767
Acceso en línea:http://hdl.handle.net/20.500.11824/1767
Access Level:acceso abierto
Palabra clave:Continuous optimization
evolutionary algorithm
noisy problem
uncertainty
id ES_3bebc5ae5a515c6e0ccd154b5dd511b1
oai_identifier_str oai:bird.bcamath.org:20.500.11824/1767
network_acronym_str ES
network_name_str España
repository_id_str
spelling Revisiting Implicit and Explicit Averaging for Noisy OptimizationAhrari, A.Elsayed, S.Sarker, R.Essam, D.Coello, C.A.Continuous optimizationevolutionary algorithmnoisy problemuncertaintyExplicit and implicit averaging are two well-known strategies for noisy optimization. Both strategies can counteract the disruptive effect of noise; however, a critical question remains: which one is more efficient? This question has been raised in many studies, with conflicting preferences and, in some cases, findings. Nevertheless, theoretical findings on the noisy sphere problem with additive Gaussian noise supports the superiority of implicit averaging, which may have had a strong impact on the preference of implicit averaging in more recent evolutionary methods for noisy optimization. This study speculates that the analytically supported superiority of implicit averaging relies on specific features of the noisy sphere problem with additive noise, which cannot be generalized to other problems. It enumerates these features and designs controlled numerical experiments to investigate this potential reliance. Each experiment gradually suppresses one specific feature, and the progress rate is numerically calculated for different values of the sample size given a fixed evaluation budget. Our empirical results indicate that for a wide range of noise strength and evaluation budget per iteration, the more these specific features are suppressed, the more the optimal averaging strategy deviates from implicit toward explicit averaging, which confirms our speculations. Consequently, the optimal sample size, which is regarded as the tradeoff between implicit and explicit averaging, depends on the problem characteristics and should be learned during optimization for maximum efficiency.202420242023info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfhttp://hdl.handle.net/20.500.11824/1767reponame:BIRD. BCAM's Institutional Repository Datainstname:Basque Center for Applied Mathematics (BCAM)Ingléshttps://ieeexplore.ieee.org/document/9866832info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023/CEX2021-001142-SReconocimiento-NoComercial-CompartirIgual 3.0 Españahttp://creativecommons.org/licenses/by-nc-sa/3.0/es/info:eu-repo/semantics/openAccessoai:bird.bcamath.org:20.500.11824/17672026-06-19T12:47:47Z
dc.title.none.fl_str_mv Revisiting Implicit and Explicit Averaging for Noisy Optimization
title Revisiting Implicit and Explicit Averaging for Noisy Optimization
spellingShingle Revisiting Implicit and Explicit Averaging for Noisy Optimization
Ahrari, A.
Continuous optimization
evolutionary algorithm
noisy problem
uncertainty
title_short Revisiting Implicit and Explicit Averaging for Noisy Optimization
title_full Revisiting Implicit and Explicit Averaging for Noisy Optimization
title_fullStr Revisiting Implicit and Explicit Averaging for Noisy Optimization
title_full_unstemmed Revisiting Implicit and Explicit Averaging for Noisy Optimization
title_sort Revisiting Implicit and Explicit Averaging for Noisy Optimization
dc.creator.none.fl_str_mv Ahrari, A.
Elsayed, S.
Sarker, R.
Essam, D.
Coello, C.A.
author Ahrari, A.
author_facet Ahrari, A.
Elsayed, S.
Sarker, R.
Essam, D.
Coello, C.A.
author_role author
author2 Elsayed, S.
Sarker, R.
Essam, D.
Coello, C.A.
author2_role author
author
author
author
dc.subject.none.fl_str_mv Continuous optimization
evolutionary algorithm
noisy problem
uncertainty
topic Continuous optimization
evolutionary algorithm
noisy problem
uncertainty
description Explicit and implicit averaging are two well-known strategies for noisy optimization. Both strategies can counteract the disruptive effect of noise; however, a critical question remains: which one is more efficient? This question has been raised in many studies, with conflicting preferences and, in some cases, findings. Nevertheless, theoretical findings on the noisy sphere problem with additive Gaussian noise supports the superiority of implicit averaging, which may have had a strong impact on the preference of implicit averaging in more recent evolutionary methods for noisy optimization. This study speculates that the analytically supported superiority of implicit averaging relies on specific features of the noisy sphere problem with additive noise, which cannot be generalized to other problems. It enumerates these features and designs controlled numerical experiments to investigate this potential reliance. Each experiment gradually suppresses one specific feature, and the progress rate is numerically calculated for different values of the sample size given a fixed evaluation budget. Our empirical results indicate that for a wide range of noise strength and evaluation budget per iteration, the more these specific features are suppressed, the more the optimal averaging strategy deviates from implicit toward explicit averaging, which confirms our speculations. Consequently, the optimal sample size, which is regarded as the tradeoff between implicit and explicit averaging, depends on the problem characteristics and should be learned during optimization for maximum efficiency.
publishDate 2023
dc.date.none.fl_str_mv 2023
2024
2024
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/20.500.11824/1767
url http://hdl.handle.net/20.500.11824/1767
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv https://ieeexplore.ieee.org/document/9866832
info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023/CEX2021-001142-S
dc.rights.none.fl_str_mv Reconocimiento-NoComercial-CompartirIgual 3.0 España
http://creativecommons.org/licenses/by-nc-sa/3.0/es/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv Reconocimiento-NoComercial-CompartirIgual 3.0 España
http://creativecommons.org/licenses/by-nc-sa/3.0/es/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.source.none.fl_str_mv reponame:BIRD. BCAM's Institutional Repository Data
instname:Basque Center for Applied Mathematics (BCAM)
instname_str Basque Center for Applied Mathematics (BCAM)
reponame_str BIRD. BCAM's Institutional Repository Data
collection BIRD. BCAM's Institutional Repository Data
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869406336616235008
score 15,301603

Similares en LA Referencia