Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes
Some optimization problems are characterized by an objective that is very expensive, that lacks an an- alytical expression, and whose evaluations can be contaminated by noise. Bayesian Optimization (BO) methods can be used to solve these problems efficiently. BO relies on a probabilistic model of th...
| Autores: | , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2019 |
| País: | España |
| Institución: | Universidad Autónoma de Madrid |
| Repositorio: | Biblos-e Archivo. Repositorio Institucional de la UAM |
| Idioma: | inglés |
| OAI Identifier: | oai:repositorio.uam.es:10486/710149 |
| Acceso en línea: | http://hdl.handle.net/10486/710149 https://dx.doi.org/10.1016/j.neucom.2019.11.004 |
| Access Level: | acceso abierto |
| Palabra clave: | Parameter tuning Bayesian optimization Gaussian processes Integer-valued variables Categorical variables Informática |
| Sumario: | Some optimization problems are characterized by an objective that is very expensive, that lacks an an- alytical expression, and whose evaluations can be contaminated by noise. Bayesian Optimization (BO) methods can be used to solve these problems efficiently. BO relies on a probabilistic model of the objec- tive, which is typically a Gaussian process (GP). This model is used to compute an acquisition function that estimates the expected utility (for solving the optimization problem) of evaluating the objective at each potential new point. A problem with GPs is, however, that they assume real-valued input variables and cannot easily deal with categorical or integer-valued values. Common methods to account for these variables, before evaluating the objective, include assuming they are real and then using a one-hot en- coding, for categorical variables, or rounding to the closest integer, for integer-valued variables. We show that this leads to suboptimal results and introduce a novel approach to tackle categorical or integer- valued input variables within the context of BO with GPs. Several synthetic and real-world experiments support our hypotheses and show that our approach outperforms the results of standard BO using GPs on problems with categorical or integer-valued input variables. |
|---|