Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes

Some optimization problems are characterized by an objective that is very expensive, that lacks an an- alytical expression, and whose evaluations can be contaminated by noise. Bayesian Optimization (BO) methods can be used to solve these problems efficiently. BO relies on a probabilistic model of th...

Descripción completa

Detalles Bibliográficos
Autores: Garrido Merchán, Eduardo César, Hernández Lobato, Daniel
Tipo de recurso: artículo
Fecha de publicación:2019
País:España
Institución:Universidad Autónoma de Madrid
Repositorio:Biblos-e Archivo. Repositorio Institucional de la UAM
Idioma:inglés
OAI Identifier:oai:repositorio.uam.es:10486/710149
Acceso en línea:http://hdl.handle.net/10486/710149
https://dx.doi.org/10.1016/j.neucom.2019.11.004
Access Level:acceso abierto
Palabra clave:Parameter tuning
Bayesian optimization
Gaussian processes
Integer-valued variables
Categorical variables
Informática
id ES_35fbe7cf09005cb848a602d9285000fc
oai_identifier_str oai:repositorio.uam.es:10486/710149
network_acronym_str ES
network_name_str España
repository_id_str
spelling Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processesGarrido Merchán, Eduardo CésarHernández Lobato, DanielParameter tuningBayesian optimizationGaussian processesInteger-valued variablesCategorical variablesInformáticaSome optimization problems are characterized by an objective that is very expensive, that lacks an an- alytical expression, and whose evaluations can be contaminated by noise. Bayesian Optimization (BO) methods can be used to solve these problems efficiently. BO relies on a probabilistic model of the objec- tive, which is typically a Gaussian process (GP). This model is used to compute an acquisition function that estimates the expected utility (for solving the optimization problem) of evaluating the objective at each potential new point. A problem with GPs is, however, that they assume real-valued input variables and cannot easily deal with categorical or integer-valued values. Common methods to account for these variables, before evaluating the objective, include assuming they are real and then using a one-hot en- coding, for categorical variables, or rounding to the closest integer, for integer-valued variables. We show that this leads to suboptimal results and introduce a novel approach to tackle categorical or integer- valued input variables within the context of BO with GPs. Several synthetic and real-world experiments support our hypotheses and show that our approach outperforms the results of standard BO using GPs on problems with categorical or integer-valued input variables.The authors gratefully acknowledge the use of the facilities of Centro de Computación Científica (CCC) at Universidad Autónoma de Madrid. The authors acknowledge financial support from the European Regional Development Fund and from the Spanish Min- istry of Economy, Industry and Competitiveness - State Research Agency, project TIN2016-76406-P (AEI/FEDER, UE) and project TEC2016-81900-REDT.ElsevierDepartamento de Ingeniería InformáticaEscuela Politécnica Superior20192019-11-09research articlehttp://purl.org/coar/resource_type/c_2df8fbb1AMhttp://purl.org/coar/version/c_ab4af688f83e57aainfo:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10486/710149https://dx.doi.org/10.1016/j.neucom.2019.11.004reponame:Biblos-e Archivo. Repositorio Institucional de la UAMinstname:Universidad Autónoma de MadridInglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:repositorio.uam.es:10486/7101492026-06-23T12:46:27Z
dc.title.none.fl_str_mv Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes
title Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes
spellingShingle Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes
Garrido Merchán, Eduardo César
Parameter tuning
Bayesian optimization
Gaussian processes
Integer-valued variables
Categorical variables
Informática
title_short Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes
title_full Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes
title_fullStr Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes
title_full_unstemmed Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes
title_sort Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes
dc.creator.none.fl_str_mv Garrido Merchán, Eduardo César
Hernández Lobato, Daniel
author Garrido Merchán, Eduardo César
author_facet Garrido Merchán, Eduardo César
Hernández Lobato, Daniel
author_role author
author2 Hernández Lobato, Daniel
author2_role author
dc.contributor.none.fl_str_mv Departamento de Ingeniería Informática
Escuela Politécnica Superior
dc.subject.none.fl_str_mv Parameter tuning
Bayesian optimization
Gaussian processes
Integer-valued variables
Categorical variables
Informática
topic Parameter tuning
Bayesian optimization
Gaussian processes
Integer-valued variables
Categorical variables
Informática
description Some optimization problems are characterized by an objective that is very expensive, that lacks an an- alytical expression, and whose evaluations can be contaminated by noise. Bayesian Optimization (BO) methods can be used to solve these problems efficiently. BO relies on a probabilistic model of the objec- tive, which is typically a Gaussian process (GP). This model is used to compute an acquisition function that estimates the expected utility (for solving the optimization problem) of evaluating the objective at each potential new point. A problem with GPs is, however, that they assume real-valued input variables and cannot easily deal with categorical or integer-valued values. Common methods to account for these variables, before evaluating the objective, include assuming they are real and then using a one-hot en- coding, for categorical variables, or rounding to the closest integer, for integer-valued variables. We show that this leads to suboptimal results and introduce a novel approach to tackle categorical or integer- valued input variables within the context of BO with GPs. Several synthetic and real-world experiments support our hypotheses and show that our approach outperforms the results of standard BO using GPs on problems with categorical or integer-valued input variables.
publishDate 2019
dc.date.none.fl_str_mv 2019
2019-11-09
dc.type.none.fl_str_mv research article
http://purl.org/coar/resource_type/c_2df8fbb1
AM
http://purl.org/coar/version/c_ab4af688f83e57aa
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10486/710149
https://dx.doi.org/10.1016/j.neucom.2019.11.004
url http://hdl.handle.net/10486/710149
https://dx.doi.org/10.1016/j.neucom.2019.11.004
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:Biblos-e Archivo. Repositorio Institucional de la UAM
instname:Universidad Autónoma de Madrid
instname_str Universidad Autónoma de Madrid
reponame_str Biblos-e Archivo. Repositorio Institucional de la UAM
collection Biblos-e Archivo. Repositorio Institucional de la UAM
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869405929528623104
score 15,301603

Similares en LA Referencia