Visual semantic navigation with real robots

Visual Semantic Navigation (VSN) is the ability of a robot to learn visual semantic information for navigating in unseen environments. These VSN models are typically tested in those virtual environments where they are trained, mainly using reinforcement learning based approaches. Therefore, we do no...

ver descrição completa

Detalhes bibliográficos
Autores: Gutiérrez Álvarez, Carlos|||0000-0002-5624-0076, Ríos Navarro, Pablo, Flor Rodríguez Rabadán, Rafael, Acevedo Rodríguez, Francisco Javier|||0000-0002-4727-1575, López Sastre, Roberto Javier|||0000-0002-2477-0152
Formato: artículo
Fecha de publicación:2024
País:España
Recursos:Universidad de Alcalá (UAH)
Repositorio:e_Buah Biblioteca Digital Universidad de Alcalá
Idioma:inglés
OAI Identifier:oai:ebuah.uah.es:10017/64122
Acesso em linha:http://hdl.handle.net/10017/64122
https://dx.doi.org/10.1007/s10489-024-06115-4
Access Level:acceso abierto
Palavra-chave:Robotics
Embodied agents
Vision-based navigation
Artificial intelligence
Reinforcement learning
Robótica e Informática Industrial
id ES_7ccf51d6ec2c85e9db945c2c47d33ea8
oai_identifier_str oai:ebuah.uah.es:10017/64122
network_acronym_str ES
network_name_str España
repository_id_str
spelling Visual semantic navigation with real robotsGutiérrez Álvarez, Carlos|||0000-0002-5624-0076Ríos Navarro, PabloFlor Rodríguez Rabadán, RafaelAcevedo Rodríguez, Francisco Javier|||0000-0002-4727-1575López Sastre, Roberto Javier|||0000-0002-2477-0152RoboticsEmbodied agentsVision-based navigationArtificial intelligenceReinforcement learningRobótica e Informática IndustrialRoboticsVisual Semantic Navigation (VSN) is the ability of a robot to learn visual semantic information for navigating in unseen environments. These VSN models are typically tested in those virtual environments where they are trained, mainly using reinforcement learning based approaches. Therefore, we do not yet have an in-depth analysis of how these models would behave in the real world. In this work, we propose a new solution to integrate VSN models into real robots, so that we have true embodied agents. We also release a novel ROS-based framework for VSN, ROS4VSN, so that any VSN-model can be easily deployed in any ROS-compatible robot and tested in a real setting. Our experiments with two different robots, where we have embedded two state-of-the-art VSN agents, confirm that there is a noticeable performance difference of these VSN solutions when tested in real-world and simulation environments. We hope that this research will endeavor to provide a foundation for addressing this consequential issue, with the ultimate aim of advancing the performance and efficiency of embodied agents within authentic real-world scenarios. Code to reproduce all our experiments can be found at https://github.com/gramuah/ros4vsn.Agencia Estatal de InvestigaciónUniversidad de AlcaláSpringer20242024-12-2320242024-12-2320252025-12-23journal articlehttp://purl.org/coar/resource_type/c_6501NAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10017/64122https://dx.doi.org/10.1007/s10489-024-06115-4reponame:e_Buah Biblioteca Digital Universidad de Alcaláinstname:Universidad de Alcalá (UAH)InglésengAgencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2023-148310OB-I00 ESPACIOS INTELIGENTES CON NAVEGACION AVANZADA E INTERPRETACION DEL ENTORNO ORIENTADOS A PERSONAS CON DEPENDENCIAUAH Not available 2023%2F00405%2F001open accesshttp://purl.org/coar/access_right/c_abf2Attribution-NonCommercial-NoDerivatives 4.0 Internationalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:ebuah.uah.es:10017/641222026-06-18T11:13:07Z
dc.title.none.fl_str_mv Visual semantic navigation with real robots
title Visual semantic navigation with real robots
spellingShingle Visual semantic navigation with real robots
Gutiérrez Álvarez, Carlos|||0000-0002-5624-0076
Robotics
Embodied agents
Vision-based navigation
Artificial intelligence
Reinforcement learning
Robótica e Informática Industrial
Robotics
title_short Visual semantic navigation with real robots
title_full Visual semantic navigation with real robots
title_fullStr Visual semantic navigation with real robots
title_full_unstemmed Visual semantic navigation with real robots
title_sort Visual semantic navigation with real robots
dc.creator.none.fl_str_mv Gutiérrez Álvarez, Carlos|||0000-0002-5624-0076
Ríos Navarro, Pablo
Flor Rodríguez Rabadán, Rafael
Acevedo Rodríguez, Francisco Javier|||0000-0002-4727-1575
López Sastre, Roberto Javier|||0000-0002-2477-0152
author Gutiérrez Álvarez, Carlos|||0000-0002-5624-0076
author_facet Gutiérrez Álvarez, Carlos|||0000-0002-5624-0076
Ríos Navarro, Pablo
Flor Rodríguez Rabadán, Rafael
Acevedo Rodríguez, Francisco Javier|||0000-0002-4727-1575
López Sastre, Roberto Javier|||0000-0002-2477-0152
author_role author
author2 Ríos Navarro, Pablo
Flor Rodríguez Rabadán, Rafael
Acevedo Rodríguez, Francisco Javier|||0000-0002-4727-1575
López Sastre, Roberto Javier|||0000-0002-2477-0152
author2_role author
author
author
author
dc.subject.none.fl_str_mv Robotics
Embodied agents
Vision-based navigation
Artificial intelligence
Reinforcement learning
Robótica e Informática Industrial
Robotics
topic Robotics
Embodied agents
Vision-based navigation
Artificial intelligence
Reinforcement learning
Robótica e Informática Industrial
Robotics
description Visual Semantic Navigation (VSN) is the ability of a robot to learn visual semantic information for navigating in unseen environments. These VSN models are typically tested in those virtual environments where they are trained, mainly using reinforcement learning based approaches. Therefore, we do not yet have an in-depth analysis of how these models would behave in the real world. In this work, we propose a new solution to integrate VSN models into real robots, so that we have true embodied agents. We also release a novel ROS-based framework for VSN, ROS4VSN, so that any VSN-model can be easily deployed in any ROS-compatible robot and tested in a real setting. Our experiments with two different robots, where we have embedded two state-of-the-art VSN agents, confirm that there is a noticeable performance difference of these VSN solutions when tested in real-world and simulation environments. We hope that this research will endeavor to provide a foundation for addressing this consequential issue, with the ultimate aim of advancing the performance and efficiency of embodied agents within authentic real-world scenarios. Code to reproduce all our experiments can be found at https://github.com/gramuah/ros4vsn.
publishDate 2024
dc.date.none.fl_str_mv 2024
2024-12-23
2024
2024-12-23
2025
2025-12-23
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10017/64122
https://dx.doi.org/10.1007/s10489-024-06115-4
url http://hdl.handle.net/10017/64122
https://dx.doi.org/10.1007/s10489-024-06115-4
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Agencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2023-148310OB-I00 ESPACIOS INTELIGENTES CON NAVEGACION AVANZADA E INTERPRETACION DEL ENTORNO ORIENTADOS A PERSONAS CON DEPENDENCIA
UAH Not available 2023%2F00405%2F001
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Springer
publisher.none.fl_str_mv Springer
dc.source.none.fl_str_mv reponame:e_Buah Biblioteca Digital Universidad de Alcalá
instname:Universidad de Alcalá (UAH)
instname_str Universidad de Alcalá (UAH)
reponame_str e_Buah Biblioteca Digital Universidad de Alcalá
collection e_Buah Biblioteca Digital Universidad de Alcalá
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869411617258602496
score 15,81155

Registos relacionados