Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.

The characterization of natural spaces by the precise observation of their material properties is highly demanded in remote sensing and computer vision. The production of novel sensors enables the collection of heterogeneous data to get a comprehensive knowledge of the living and non-living entities...

Descripción completa

Detalles Bibliográficos
Autores: Jurado Rodríguez, Juan Manuel, Cárdenas Donoso, José Luis, OGAYAR ANGUITA, CARLOS JAVIER, ORTEGA ALVARADO, LIDIA M., FEITO HIGUERUELA, FRANCISCO RAMON
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2020
País:España
Institución:Universidad de Jaén
Repositorio:RUJA. Repositorio Institucional de la Producción Científica de la Universidad de Jaén
OAI Identifier:oai:ruja.ujaen.es:10953/4394
Acceso en línea:https://doi.org/10.3390/s20082244
https://hdl.handle.net/10953/4394
Access Level:acceso abierto
Palabra clave:multispectral imaging
heterogeneous data fusion
point cloud segmentation
material-based recognition
id ES_897135bac8e90eb0b55e3cb5b89f287c
oai_identifier_str oai:ruja.ujaen.es:10953/4394
network_acronym_str ES
network_name_str España
repository_id_str
spelling Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.Jurado Rodríguez, Juan ManuelCárdenas Donoso, José LuisOGAYAR ANGUITA, CARLOS JAVIERORTEGA ALVARADO, LIDIA M.FEITO HIGUERUELA, FRANCISCO RAMONmultispectral imagingheterogeneous data fusionpoint cloud segmentationmaterial-based recognitionThe characterization of natural spaces by the precise observation of their material properties is highly demanded in remote sensing and computer vision. The production of novel sensors enables the collection of heterogeneous data to get a comprehensive knowledge of the living and non-living entities in the ecosystem. The high resolution of consumer-grade RGB cameras is frequently used for the geometric reconstruction of many types of environments. Nevertheless, the understanding of natural spaces is still challenging. The automatic segmentation of homogeneous materials in nature is a complex task because there are many overlapping structures and an indirect illumination, so the object recognition is difficult. In this paper, we propose a method based on fusing spatial and multispectral characteristics for the unsupervised classification of natural materials in a point cloud. A high-resolution camera and a multispectral sensor are mounted on a custom camera rig in order to simultaneously capture RGB and multispectral images. Our method is tested in a controlled scenario, where different natural objects coexist. Initially, the input RGB images are processed to generate a point cloud by applying the structure-from-motion (SfM) algorithm. Then, the multispectral images are mapped on the three-dimensional model to characterize the geometry with the reflectance captured from four narrow bands (green, red, red-edge and near-infrared). The reflectance, the visible colour and the spatial component are combined to extract key differences among all existing materials. For this purpose, a hierarchical cluster analysis is applied to pool the point cloud and identify the feature pattern for every material. As a result, the tree trunk, the leaves, different species of low plants, the ground and rocks can be clearly recognized in the scene. These results demonstrate the feasibility to perform a semantic segmentation by considering multispectral and spatial features with an unknown number of clusters to be detected on the point cloud. Moreover, our solution is compared to other method based on supervised learning in order to test the improvement of the proposed approach.This research has been partially supported by the Ministerio de Economía y Competitividad and the European Union (via ERDF funds) through the research project TIN2017-84968-R.MDPI202520252020info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfhttps://doi.org/10.3390/s20082244https://hdl.handle.net/10953/4394reponame:RUJA. Repositorio Institucional de la Producción Científica de la Universidad de Jaéninstname:Universidad de JaénInglésSensorsAtribución-NoComercial-SinDerivadas 3.0 Españahttp://creativecommons.org/licenses/by-nc-nd/3.0/es/info:eu-repo/semantics/openAccessoai:ruja.ujaen.es:10953/43942026-06-24T12:41:07Z
dc.title.none.fl_str_mv Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.
title Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.
spellingShingle Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.
Jurado Rodríguez, Juan Manuel
multispectral imaging
heterogeneous data fusion
point cloud segmentation
material-based recognition
title_short Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.
title_full Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.
title_fullStr Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.
title_full_unstemmed Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.
title_sort Semantic Segmentation of Natural Materials on a Point Cloud Using Spatial and Multispectral Features.
dc.creator.none.fl_str_mv Jurado Rodríguez, Juan Manuel
Cárdenas Donoso, José Luis
OGAYAR ANGUITA, CARLOS JAVIER
ORTEGA ALVARADO, LIDIA M.
FEITO HIGUERUELA, FRANCISCO RAMON
author Jurado Rodríguez, Juan Manuel
author_facet Jurado Rodríguez, Juan Manuel
Cárdenas Donoso, José Luis
OGAYAR ANGUITA, CARLOS JAVIER
ORTEGA ALVARADO, LIDIA M.
FEITO HIGUERUELA, FRANCISCO RAMON
author_role author
author2 Cárdenas Donoso, José Luis
OGAYAR ANGUITA, CARLOS JAVIER
ORTEGA ALVARADO, LIDIA M.
FEITO HIGUERUELA, FRANCISCO RAMON
author2_role author
author
author
author
dc.subject.none.fl_str_mv multispectral imaging
heterogeneous data fusion
point cloud segmentation
material-based recognition
topic multispectral imaging
heterogeneous data fusion
point cloud segmentation
material-based recognition
description The characterization of natural spaces by the precise observation of their material properties is highly demanded in remote sensing and computer vision. The production of novel sensors enables the collection of heterogeneous data to get a comprehensive knowledge of the living and non-living entities in the ecosystem. The high resolution of consumer-grade RGB cameras is frequently used for the geometric reconstruction of many types of environments. Nevertheless, the understanding of natural spaces is still challenging. The automatic segmentation of homogeneous materials in nature is a complex task because there are many overlapping structures and an indirect illumination, so the object recognition is difficult. In this paper, we propose a method based on fusing spatial and multispectral characteristics for the unsupervised classification of natural materials in a point cloud. A high-resolution camera and a multispectral sensor are mounted on a custom camera rig in order to simultaneously capture RGB and multispectral images. Our method is tested in a controlled scenario, where different natural objects coexist. Initially, the input RGB images are processed to generate a point cloud by applying the structure-from-motion (SfM) algorithm. Then, the multispectral images are mapped on the three-dimensional model to characterize the geometry with the reflectance captured from four narrow bands (green, red, red-edge and near-infrared). The reflectance, the visible colour and the spatial component are combined to extract key differences among all existing materials. For this purpose, a hierarchical cluster analysis is applied to pool the point cloud and identify the feature pattern for every material. As a result, the tree trunk, the leaves, different species of low plants, the ground and rocks can be clearly recognized in the scene. These results demonstrate the feasibility to perform a semantic segmentation by considering multispectral and spatial features with an unknown number of clusters to be detected on the point cloud. Moreover, our solution is compared to other method based on supervised learning in order to test the improvement of the proposed approach.
publishDate 2020
dc.date.none.fl_str_mv 2020
2025
2025
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://doi.org/10.3390/s20082244
https://hdl.handle.net/10953/4394
url https://doi.org/10.3390/s20082244
https://hdl.handle.net/10953/4394
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Sensors
dc.rights.none.fl_str_mv Atribución-NoComercial-SinDerivadas 3.0 España
http://creativecommons.org/licenses/by-nc-nd/3.0/es/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv Atribución-NoComercial-SinDerivadas 3.0 España
http://creativecommons.org/licenses/by-nc-nd/3.0/es/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv MDPI
publisher.none.fl_str_mv MDPI
dc.source.none.fl_str_mv reponame:RUJA. Repositorio Institucional de la Producción Científica de la Universidad de Jaén
instname:Universidad de Jaén
instname_str Universidad de Jaén
reponame_str RUJA. Repositorio Institucional de la Producción Científica de la Universidad de Jaén
collection RUJA. Repositorio Institucional de la Producción Científica de la Universidad de Jaén
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869412627086573568
score 15,812429

Similares en LA Referencia