Optimal control as a graphical model inference problem
We reformulate a class of non-linear stochastic optimal control problems introduced by Todorov (in Advances in Neural Information Processing Systems, vol. 19, pp. 1369-1376, 2007) as a Kullback-Leibler (KL) minimization problem. As a result, the optimal control computation reduces to an inference co...
| Autores: | , , |
|---|---|
| Tipo de recurso: | artículo |
| Estado: | Versión aceptada para publicación |
| Fecha de publicación: | 2012 |
| País: | España |
| Institución: | Universitat Pompeu Fabra |
| Repositorio: | Repositorio Digital de la UPF |
| OAI Identifier: | oai:repositori.upf.edu:10230/36355 |
| Acceso en línea: | http://hdl.handle.net/10230/36355 http://dx.doi.org/10.1007/s10994-012-5278-7 |
| Access Level: | acceso abierto |
| Palabra clave: | Optimal control Uncontrolled dynamics Kullback-Leibler divergence Graphical model Approximate inference Cluster variation method Belief propagation |
| Sumario: | We reformulate a class of non-linear stochastic optimal control problems introduced by Todorov (in Advances in Neural Information Processing Systems, vol. 19, pp. 1369-1376, 2007) as a Kullback-Leibler (KL) minimization problem. As a result, the optimal control computation reduces to an inference computation and approximate inference methods can be applied to efficiently compute approximate optimal controls. We show how this KL control theory contains the path integral control method as a special case. We provide an example of a block stacking task and a multi-agent cooperative game where we demonstrate how approximate inference can be successfully applied to instances that are too complex for exact computation. We discuss the relation of the KL control approach to other inference approaches to control. |
|---|