Dynamic robotic tracking of underwater targets using reinforcement learning

To realize the potential of autonomous underwater robots that scale up our observational capacity in the ocean, new techniques are needed. Fleets of autonomous robots could be used to study complex marine systems and animals with either new imaging configurations or by tracking tagged animals to stu...

Full description

Bibliographic Details
Authors: Masmitjà Rusiñol, Ivan|||0000-0001-6355-7955, Martín Muñoz, Mario|||0000-0002-4125-6630, O’Reilly, Tom, Kieft, Brian, Palomeras Rovira, Narcís, Navarro Bernabé, Joan, Katija, Kakani
Format: article
Publication Date:2023
Country:España
Institution:Universitat Politècnica de Catalunya (UPC)
Repository:UPCommons. Portal del coneixement obert de la UPC
Language:English
OAI Identifier:oai:upcommons.upc.edu:2117/393215
Online Access:https://hdl.handle.net/2117/393215
https://dx.doi.org/10.1126/scirobotics.ade7811
Access Level:Open access
Keyword:Autonomous underwater vehicles
Reinforcement learning
Vehicles submergibles autònoms
Aprenentatge per reforç
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
Àrees temàtiques de la UPC::Informàtica::Robòtica
id ES_c36082178faf92bbc95deb02f7cfcef4
oai_identifier_str oai:upcommons.upc.edu:2117/393215
network_acronym_str ES
network_name_str España
repository_id_str
spelling Dynamic robotic tracking of underwater targets using reinforcement learningMasmitjà Rusiñol, Ivan|||0000-0001-6355-7955Martín Muñoz, Mario|||0000-0002-4125-6630O’Reilly, TomKieft, BrianPalomeras Rovira, NarcísNavarro Bernabé, JoanKatija, KakaniAutonomous underwater vehiclesReinforcement learningVehicles submergibles autònomsAprenentatge per reforçÀrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàticÀrees temàtiques de la UPC::Informàtica::RobòticaTo realize the potential of autonomous underwater robots that scale up our observational capacity in the ocean, new techniques are needed. Fleets of autonomous robots could be used to study complex marine systems and animals with either new imaging configurations or by tracking tagged animals to study their behavior. These activities can then inform and create new policies for community conservation. The role of animal connectivity via active movement of animals represents a major knowledge gap related to the distribution of deep ocean populations. Tracking underwater targets represents a major challenge for observing biological processes in situ, and methods to robustly respond to a changing environment during monitoring missions are needed. Analytical techniques for optimal sensor placement and path planning to locate underwater targets are not straightforward in such cases. The aim of this study was to investigate the use of reinforcement learning as a tool for range-only underwater target-tracking optimization, whose promising capabilities have been demonstrated in terrestrial scenarios. To evaluate its usefulness, a reinforcement learning method was implemented as a path planning system for an autonomous surface vehicle while tracking an underwater mobile target. A complete description of an open-source model, performance metrics in simulated environments, and evaluated algorithms based on more than 15 hours of at-sea field experiments are presented. These efforts demonstrate that deep reinforcement learning is a powerful approach that enhances the abilities of autonomous robots in the ocean and encourages the deployment of algorithms like these for monitoring marine biological systems in the future.Funding: The European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie, grant agreement No 893089 (IM). The Spanish Ministerio de Economia y Competitividad under the project SASES, grant agreement No RTI2018-095112-B-I00 (IM). The Spanish Ministerio de Economia y Competitividad under the project BITER-ECO, grant agreement No PID2020-114732RB-C31 (IM, JN). The Spanish Ministerio de Economia y Competitividad under the project BITER-AUV, grant agreement No PID2020-114732RB-C33 (NP).Peer ReviewedAmerican Association for the Advancement of Science (AAAS)20232023-07-2620232023-09-07journal articlehttp://purl.org/coar/resource_type/c_6501AMhttp://purl.org/coar/version/c_ab4af688f83e57aainfo:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/2117/393215https://dx.doi.org/10.1126/scirobotics.ade7811reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)InglésengAgencia Estatal de Investigación http://doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 RTI2018-095112-B-I00 SISTEMAS ACUSTICOS SUBMARINOS PARA LA MONITORIZACION DEL COMPORTAMIENTO ESPACIAL DE ESPECIESopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/3932152026-05-27T15:37:01Z
dc.title.none.fl_str_mv Dynamic robotic tracking of underwater targets using reinforcement learning
title Dynamic robotic tracking of underwater targets using reinforcement learning
spellingShingle Dynamic robotic tracking of underwater targets using reinforcement learning
Masmitjà Rusiñol, Ivan|||0000-0001-6355-7955
Autonomous underwater vehicles
Reinforcement learning
Vehicles submergibles autònoms
Aprenentatge per reforç
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
Àrees temàtiques de la UPC::Informàtica::Robòtica
title_short Dynamic robotic tracking of underwater targets using reinforcement learning
title_full Dynamic robotic tracking of underwater targets using reinforcement learning
title_fullStr Dynamic robotic tracking of underwater targets using reinforcement learning
title_full_unstemmed Dynamic robotic tracking of underwater targets using reinforcement learning
title_sort Dynamic robotic tracking of underwater targets using reinforcement learning
dc.creator.none.fl_str_mv Masmitjà Rusiñol, Ivan|||0000-0001-6355-7955
Martín Muñoz, Mario|||0000-0002-4125-6630
O’Reilly, Tom
Kieft, Brian
Palomeras Rovira, Narcís
Navarro Bernabé, Joan
Katija, Kakani
author Masmitjà Rusiñol, Ivan|||0000-0001-6355-7955
author_facet Masmitjà Rusiñol, Ivan|||0000-0001-6355-7955
Martín Muñoz, Mario|||0000-0002-4125-6630
O’Reilly, Tom
Kieft, Brian
Palomeras Rovira, Narcís
Navarro Bernabé, Joan
Katija, Kakani
author_role author
author2 Martín Muñoz, Mario|||0000-0002-4125-6630
O’Reilly, Tom
Kieft, Brian
Palomeras Rovira, Narcís
Navarro Bernabé, Joan
Katija, Kakani
author2_role author
author
author
author
author
author
dc.subject.none.fl_str_mv Autonomous underwater vehicles
Reinforcement learning
Vehicles submergibles autònoms
Aprenentatge per reforç
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
Àrees temàtiques de la UPC::Informàtica::Robòtica
topic Autonomous underwater vehicles
Reinforcement learning
Vehicles submergibles autònoms
Aprenentatge per reforç
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
Àrees temàtiques de la UPC::Informàtica::Robòtica
description To realize the potential of autonomous underwater robots that scale up our observational capacity in the ocean, new techniques are needed. Fleets of autonomous robots could be used to study complex marine systems and animals with either new imaging configurations or by tracking tagged animals to study their behavior. These activities can then inform and create new policies for community conservation. The role of animal connectivity via active movement of animals represents a major knowledge gap related to the distribution of deep ocean populations. Tracking underwater targets represents a major challenge for observing biological processes in situ, and methods to robustly respond to a changing environment during monitoring missions are needed. Analytical techniques for optimal sensor placement and path planning to locate underwater targets are not straightforward in such cases. The aim of this study was to investigate the use of reinforcement learning as a tool for range-only underwater target-tracking optimization, whose promising capabilities have been demonstrated in terrestrial scenarios. To evaluate its usefulness, a reinforcement learning method was implemented as a path planning system for an autonomous surface vehicle while tracking an underwater mobile target. A complete description of an open-source model, performance metrics in simulated environments, and evaluated algorithms based on more than 15 hours of at-sea field experiments are presented. These efforts demonstrate that deep reinforcement learning is a powerful approach that enhances the abilities of autonomous robots in the ocean and encourages the deployment of algorithms like these for monitoring marine biological systems in the future.
publishDate 2023
dc.date.none.fl_str_mv 2023
2023-07-26
2023
2023-09-07
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
AM
http://purl.org/coar/version/c_ab4af688f83e57aa
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/393215
https://dx.doi.org/10.1126/scirobotics.ade7811
url https://hdl.handle.net/2117/393215
https://dx.doi.org/10.1126/scirobotics.ade7811
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Agencia Estatal de Investigación http://doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 RTI2018-095112-B-I00 SISTEMAS ACUSTICOS SUBMARINOS PARA LA MONITORIZACION DEL COMPORTAMIENTO ESPACIAL DE ESPECIES
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv American Association for the Advancement of Science (AAAS)
publisher.none.fl_str_mv American Association for the Advancement of Science (AAAS)
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869418790418120704
score 15,300724

Similar Items