Quadcopter neural controller for take-off and landing in windy environments

This paper proposes the design of a quadcopter neural controller based on Reinforcement Learning (RL) for controlling the complete maneuvers of landing and take-off, even in variable windy conditions. To facilitate RL training, a wind model is designed, and two RL algorithms, Deep Deterministic Poli...

Full description

Bibliographic Details
Authors: Olaz Moratinos, Xabier, Aláez Gómez, Daniel, Prieto Míguez, Manuel, Villadangos Alonso, Jesús, Astrain Escola, José Javier
Format: article
Status:Published version
Publication Date:2023
Country:España
Institution:Universidad Pública de Navarra
Repository:Academica-e. Repositorio Institucional de la Universidad Pública de Navarra
OAI Identifier:oai:academica-e.unavarra.es:2454/45521
Online Access:https://hdl.handle.net/2454/45521
Access Level:Open access
Keyword:Quadcopter
Take-off
Landing
Deep reinforcement learning
Wind
PPO
DDPG
id ES_7634de5036fd9d5bd9dca2bfd7795b50
oai_identifier_str oai:academica-e.unavarra.es:2454/45521
network_acronym_str ES
network_name_str España
repository_id_str
spelling Quadcopter neural controller for take-off and landing in windy environmentsOlaz Moratinos, XabierAláez Gómez, DanielPrieto Míguez, ManuelVilladangos Alonso, JesúsAstrain Escola, José JavierQuadcopterTake-offLandingDeep reinforcement learningWindPPODDPGThis paper proposes the design of a quadcopter neural controller based on Reinforcement Learning (RL) for controlling the complete maneuvers of landing and take-off, even in variable windy conditions. To facilitate RL training, a wind model is designed, and two RL algorithms, Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO), are adapted and compared. The first phases of the learning process consider extended exploration states as a warm-up, and a novel neural network controller architecture is proposed with the addition of an adaptation layer. The neural network’s output is defined as the forces and momentum desired for the UAV, and the adaptation layer transforms forces and momentum into motor velocities. By decoupling attitude from motor velocities, the adaptation layer enhances a more straightforward interpretation of the neural network output and helps refine the rewards. The successful neural controller training has been tested up to 36 km/h wind speed.This work has been supported in part by the Ministerio de Ciencia e Innovación (Spain) and European Union NextGenerationEU, Spain under the research grant TED2021-131716B-C21 SARA (Data processing by superresolution algorithms); in part by Agencia Estatal de Investigación (AEI), Spain and European Union NextGenerationEU/PRTR, Spain PLEC2021-007997: Holistic power lines predictive maintenance system; and in part by the Government of Navarre (Departamento de Desarrollo Económico), Spain under the research grants 0011-1411-2021-000021 EMERAL: Emergency UAVs for long range operations, 0011-1365-2020-000078 DIVA, and 0011-1411-2021-000025 MOSIC: Plataforma logística de largo alcance, eléctrica y conectada.ElsevierEstadística, Informática y MatemáticasEstatistika, Informatika eta MatematikaInstitute of Smart Cities - ISC2023info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfhttps://hdl.handle.net/2454/45521reponame:Academica-e. Repositorio Institucional de la Universidad Pública de Navarrainstname:Universidad Pública de NavarraInglésinfo:eu-repo/grantAgreement/AEI//TED2021-131716B-C21info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PLEC2021-007997info:eu-repo/grantAgreement/Gobierno de Navarra//0011-1411-2021-000021info:eu-repo/grantAgreement/Gobierno de Navarra//0011-1365-2020-000078info:eu-repo/grantAgreement/Gobierno de Navarra//0011-1411-2021-000025© 2023 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license.http://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:academica-e.unavarra.es:2454/455212026-06-17T12:41:47Z
dc.title.none.fl_str_mv Quadcopter neural controller for take-off and landing in windy environments
title Quadcopter neural controller for take-off and landing in windy environments
spellingShingle Quadcopter neural controller for take-off and landing in windy environments
Olaz Moratinos, Xabier
Quadcopter
Take-off
Landing
Deep reinforcement learning
Wind
PPO
DDPG
title_short Quadcopter neural controller for take-off and landing in windy environments
title_full Quadcopter neural controller for take-off and landing in windy environments
title_fullStr Quadcopter neural controller for take-off and landing in windy environments
title_full_unstemmed Quadcopter neural controller for take-off and landing in windy environments
title_sort Quadcopter neural controller for take-off and landing in windy environments
dc.creator.none.fl_str_mv Olaz Moratinos, Xabier
Aláez Gómez, Daniel
Prieto Míguez, Manuel
Villadangos Alonso, Jesús
Astrain Escola, José Javier
author Olaz Moratinos, Xabier
author_facet Olaz Moratinos, Xabier
Aláez Gómez, Daniel
Prieto Míguez, Manuel
Villadangos Alonso, Jesús
Astrain Escola, José Javier
author_role author
author2 Aláez Gómez, Daniel
Prieto Míguez, Manuel
Villadangos Alonso, Jesús
Astrain Escola, José Javier
author2_role author
author
author
author
dc.contributor.none.fl_str_mv Estadística, Informática y Matemáticas
Estatistika, Informatika eta Matematika
Institute of Smart Cities - ISC
dc.subject.none.fl_str_mv Quadcopter
Take-off
Landing
Deep reinforcement learning
Wind
PPO
DDPG
topic Quadcopter
Take-off
Landing
Deep reinforcement learning
Wind
PPO
DDPG
description This paper proposes the design of a quadcopter neural controller based on Reinforcement Learning (RL) for controlling the complete maneuvers of landing and take-off, even in variable windy conditions. To facilitate RL training, a wind model is designed, and two RL algorithms, Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO), are adapted and compared. The first phases of the learning process consider extended exploration states as a warm-up, and a novel neural network controller architecture is proposed with the addition of an adaptation layer. The neural network’s output is defined as the forces and momentum desired for the UAV, and the adaptation layer transforms forces and momentum into motor velocities. By decoupling attitude from motor velocities, the adaptation layer enhances a more straightforward interpretation of the neural network output and helps refine the rewards. The successful neural controller training has been tested up to 36 km/h wind speed.
publishDate 2023
dc.date.none.fl_str_mv 2023
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/2454/45521
url https://hdl.handle.net/2454/45521
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv info:eu-repo/grantAgreement/AEI//TED2021-131716B-C21
info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PLEC2021-007997
info:eu-repo/grantAgreement/Gobierno de Navarra//0011-1411-2021-000021
info:eu-repo/grantAgreement/Gobierno de Navarra//0011-1365-2020-000078
info:eu-repo/grantAgreement/Gobierno de Navarra//0011-1411-2021-000025
dc.rights.none.fl_str_mv http://creativecommons.org/licenses/by-nc-nd/4.0/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:Academica-e. Repositorio Institucional de la Universidad Pública de Navarra
instname:Universidad Pública de Navarra
instname_str Universidad Pública de Navarra
reponame_str Academica-e. Repositorio Institucional de la Universidad Pública de Navarra
collection Academica-e. Repositorio Institucional de la Universidad Pública de Navarra
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869411035034681344
score 15,301603

Similar Items