Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusion

Robotic indoor mapping and localization are significantly challenged in environ ments with highly reflective or specular surfaces, which are common in hospitals and industrial settings. Specular reflections introduce severe artifacts in depth data from RGB-D sensors and degrade the performance of vi...

Full description

Bibliographic Details
Author: Hernández, Renatto Tommasi
Format: master thesis
Publication Date:2025
Country:España
Institution:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
Repository:Recercat. Dipósit de la Recerca de Catalunya
OAI Identifier:oai:recercat.cat:10256/28369
Online Access:http://hdl.handle.net/10256/28369
https://hdl.handle.net/10256/28369
Access Level:Open access
Keyword:Detectors òptics
Optical detectors
Digital mapping
Cartografia digital
Robots -- Sistemes de navegació
Robots -- Navigation systems
LiDAR odometry
Indoor localization
SLAM
Specular reflections
Sensors òptics tridimensionals
Sensors
Aprenentatge profund (Aprenentatge automàtic)
Deep learning (Machine learning)
Algorismes
Algorithms
id ES_2b89351188c10d4e9e489e2df317b9a9
oai_identifier_str oai:recercat.cat:10256/28369
network_acronym_str ES
network_name_str España
repository_id_str
spelling Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusionHernández, Renatto TommasiDetectors òpticsOptical detectorsDigital mappingCartografia digitalRobots -- Sistemes de navegacióRobots -- Navigation systemsLiDAR odometryIndoor localizationSLAMSpecular reflectionsSensors òptics tridimensionalsSensorsAprenentatge profund (Aprenentatge automàtic)Deep learning (Machine learning)AlgorismesAlgorithmsRobotic indoor mapping and localization are significantly challenged in environ ments with highly reflective or specular surfaces, which are common in hospitals and industrial settings. Specular reflections introduce severe artifacts in depth data from RGB-D sensors and degrade the performance of visual Simultaneous Localization and Mapping (SLAM) systems by creating unreliable features. This thesis presents a com prehensive solution to enhance robotic navigation in such specular-rich environments through a combination of deep learning and multi-sensor fusion. We propose a real-time filtering algorithm, RT-SpecFilter, which uses a Support Vector Machine (SVM) to detect and mitigate specular artifacts in point clouds from an Intel RealSense D435 camera. Furthermore, we conduct a comparative analysis of feature detectors, identifying Super Point as the most robust for environments with specular highlights. Finally, we develop the Multicam SP-VO system that leverages four wide FoV cameras and fuses their motion estimates with wheel odometry data using a pose-graph optimization framework. Exper imental results demonstrate that the proposed system significantly reduces orientation drift improves localization accuracy compared to reliance on wheel odometry alone and mitigates the specular artifacts during mapping, thereby enabling more robust and reli able autonomous navigation in challenging indoor spaces.9Universitat de Girona. Institut de Recerca en Visió per Computador i Robòtica2025info:eu-repo/semantics/masterThesisapplication/pdfhttp://hdl.handle.net/10256/28369https://hdl.handle.net/10256/28369Erasmus Mundus Joint Master in Intelligent Field Robotic Systems (IFROS)reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)InglésAttribution-NonCommercial-NoDerivatives 4.0 Internationalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:recercat.cat:10256/283692026-05-29T05:05:01Z
dc.title.none.fl_str_mv Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusion
title Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusion
spellingShingle Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusion
Hernández, Renatto Tommasi
Detectors òptics
Optical detectors
Digital mapping
Cartografia digital
Robots -- Sistemes de navegació
Robots -- Navigation systems
LiDAR odometry
Indoor localization
SLAM
Specular reflections
Sensors òptics tridimensionals
Sensors
Aprenentatge profund (Aprenentatge automàtic)
Deep learning (Machine learning)
Algorismes
Algorithms
title_short Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusion
title_full Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusion
title_fullStr Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusion
title_full_unstemmed Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusion
title_sort Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusion
dc.creator.none.fl_str_mv Hernández, Renatto Tommasi
author Hernández, Renatto Tommasi
author_facet Hernández, Renatto Tommasi
author_role author
dc.subject.none.fl_str_mv Detectors òptics
Optical detectors
Digital mapping
Cartografia digital
Robots -- Sistemes de navegació
Robots -- Navigation systems
LiDAR odometry
Indoor localization
SLAM
Specular reflections
Sensors òptics tridimensionals
Sensors
Aprenentatge profund (Aprenentatge automàtic)
Deep learning (Machine learning)
Algorismes
Algorithms
topic Detectors òptics
Optical detectors
Digital mapping
Cartografia digital
Robots -- Sistemes de navegació
Robots -- Navigation systems
LiDAR odometry
Indoor localization
SLAM
Specular reflections
Sensors òptics tridimensionals
Sensors
Aprenentatge profund (Aprenentatge automàtic)
Deep learning (Machine learning)
Algorismes
Algorithms
description Robotic indoor mapping and localization are significantly challenged in environ ments with highly reflective or specular surfaces, which are common in hospitals and industrial settings. Specular reflections introduce severe artifacts in depth data from RGB-D sensors and degrade the performance of visual Simultaneous Localization and Mapping (SLAM) systems by creating unreliable features. This thesis presents a com prehensive solution to enhance robotic navigation in such specular-rich environments through a combination of deep learning and multi-sensor fusion. We propose a real-time filtering algorithm, RT-SpecFilter, which uses a Support Vector Machine (SVM) to detect and mitigate specular artifacts in point clouds from an Intel RealSense D435 camera. Furthermore, we conduct a comparative analysis of feature detectors, identifying Super Point as the most robust for environments with specular highlights. Finally, we develop the Multicam SP-VO system that leverages four wide FoV cameras and fuses their motion estimates with wheel odometry data using a pose-graph optimization framework. Exper imental results demonstrate that the proposed system significantly reduces orientation drift improves localization accuracy compared to reliance on wheel odometry alone and mitigates the specular artifacts during mapping, thereby enabling more robust and reli able autonomous navigation in challenging indoor spaces.
publishDate 2025
dc.date.none.fl_str_mv 2025
dc.type.none.fl_str_mv info:eu-repo/semantics/masterThesis
format masterThesis
dc.identifier.none.fl_str_mv http://hdl.handle.net/10256/28369
https://hdl.handle.net/10256/28369
url http://hdl.handle.net/10256/28369
https://hdl.handle.net/10256/28369
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.rights.none.fl_str_mv Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Universitat de Girona. Institut de Recerca en Visió per Computador i Robòtica
publisher.none.fl_str_mv Universitat de Girona. Institut de Recerca en Visió per Computador i Robòtica
dc.source.none.fl_str_mv Erasmus Mundus Joint Master in Intelligent Field Robotic Systems (IFROS)
reponame:Recercat. Dipósit de la Recerca de Catalunya
instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
instname_str Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
reponame_str Recercat. Dipósit de la Recerca de Catalunya
collection Recercat. Dipósit de la Recerca de Catalunya
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869405155289464832
score 15,812429

Similar Items