Transferring knowledge as heuristics in reinforcement learning: A case-based approach
The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). Thi...
| Autores: | , , , , |
|---|---|
| Tipo de recurso: | artículo |
| Estado: | Versión enviada para evaluación y publicación |
| Fecha de publicación: | 2015 |
| País: | España |
| Institución: | Consejo Superior de Investigaciones Científicas (CSIC) |
| Repositorio: | DIGITAL.CSIC. Repositorio Institucional del CSIC |
| OAI Identifier: | oai:dnet:digitalcsic_::fc26750ba49af6fc566128c0d57b5353 |
| Acceso en línea: | http://hdl.handle.net/10261/130283 |
| Access Level: | acceso abierto |
| Palabra clave: | Learning process Humanoid robots Empirical evaluations Case-based approach 2D simulations Reinforcement learning Heuristic methods Meta-algorithms Case-based reasoning Anthropomorphic robots Transfer learning Target domain |
| Sumario: | The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain. A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms. © 2015 Elsevier B.V. |
|---|