2D–3D geometric fusion network using multi-neighbourhood graph convolution for RGB-D indoor scene classification
Multi-modal fusion has been proved to help enhance the performance of scene classification tasks. This paper presents a 2D-3D Fusion stage that combines 3D Geometric Features with 2D Texture Features obtained by 2D Convolutional Neural Networks. To get a robust 3D Geometric embedding, a network that...
| Autores: | , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2021 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/346435 |
| Acceso en línea: | https://hdl.handle.net/2117/346435 https://dx.doi.org/10.1016/j.inffus.2021.05.002 |
| Access Level: | acceso abierto |
| Palabra clave: | Neural networks (Computer science) Multisensor data fusion Convolutional Graph Neural Network Multi-modal fusion Multi-Neighbourhood Graph Neural Network Indoor scene classification RGB-D Xarxes neuronals (Informàtica) Fusió d'imatges |
| Sumario: | Multi-modal fusion has been proved to help enhance the performance of scene classification tasks. This paper presents a 2D-3D Fusion stage that combines 3D Geometric Features with 2D Texture Features obtained by 2D Convolutional Neural Networks. To get a robust 3D Geometric embedding, a network that uses two novel layers is proposed. The first layer, Multi-Neighbourhood Graph Convolution, aims to learn a more robust geometric descriptor of the scene combining two different neighbourhoods: one in the Euclidean space and the other in the Feature space. The second proposed layer, Nearest Voxel Pooling, improves the performance of the well-known Voxel Pooling. Experimental results, using NYU-Depth-V2 and SUN RGB-D datasets, show that the proposed method outperforms the current state-of-the-art in RGB-D indoor scene classification task. |
|---|