On Meta-Reinforcement Learning in task distributions with varying dynamics
Meta-reinforcement learning has the potential to enable artificial agents to master new skills with improved sample-efficiency by leveraging previous learning experience in tasks that are diverse but share common structure. Our focus is to study the application of such algorithms to task distributio...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Fecha de publicación: | 2021 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/348143 |
| Acceso en línea: | https://hdl.handle.net/2117/348143 |
| Access Level: | acceso abierto |
| Palabra clave: | Deep learning Inference meta-aprenentatge aprenentatge per reforç off-policy aprenentatge profund inferència variacional locomoció robòtica meta-learning reinforcement learning deep learning variational inference robotic locomotion Aprenentatge profund Inferència Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial |
| Sumario: | Meta-reinforcement learning has the potential to enable artificial agents to master new skills with improved sample-efficiency by leveraging previous learning experience in tasks that are diverse but share common structure. Our focus is to study the application of such algorithms to task distributions where the function that controls the dynamics of the environment is the main factor of variation. We start by providing an introductory background for related fields, including deep reinforcement learning, variational inference, and meta-learning. Then, we conduct a non-systematic review of the state-of-the-art algorithms for meta-reinforcement learning and perform an empirical investigation of PEARL, a method that combines soft actor-critic with latent task variables. Based on our review, we propose and implement two algorithmic modifications for PEARL: one that aims to improve the meta-training sample complexity by automatically adjusting a critical hyperparameter, and a second one focused on improving the meta-testing asymptotic performance by fine-tuning the policy during adaptation. Using a new multi-task environment suite for simulated robotics continuous control tasks, we compare the original version of PEARL and our proposed modifications, obtaining favourable results. Finally, we ponder our findings and suggest future research directions. |
|---|