CoHAtNet: An integrated convolutional-transformer architecture with hybrid self-attention for end-to-end camera localization

Camera localization refers to the process of automatically determining the position and orientation of a camera within its 3D environment from the images it captures. Traditional camera localization methods often rely on Convolutional Neural Networks, which are effective at extracting local visual f...

ver descrição completa

Detalhes bibliográficos
Autores: Hasan, Hussein, García García, Miguel Ángel, Rashwan, Hatem, Puig, Domenec
Tipo de documento: artigo
Data de publicação:2025
País:España
Recursos:Universidad Autónoma de Madrid
Repositório:Biblos-e Archivo. Repositorio Institucional de la UAM
Idioma:inglês
OAI Identifier:oai:dnet:biblosearchi::c01019ace2288adf2db2f53d78b8283b
Acesso em linha:https://hdl.handle.net/10486/773520
https://dx.doi.org/10.1016/j.imavis.2025.105674
Access Level:Acceso aberto
Palavra-chave:Camera Localization
Hybrid CNN-Transformers
CoAtNet
Hybrid Self-Attention
Telecomunicaciones
id ES_b6064f920bcb341a92b37c65bb0e6a3e
oai_identifier_str oai:dnet:biblosearchi::c01019ace2288adf2db2f53d78b8283b
network_acronym_str ES
network_name_str España
repository_id_str
spelling CoHAtNet: An integrated convolutional-transformer architecture with hybrid self-attention for end-to-end camera localizationHasan, HusseinGarcía García, Miguel ÁngelRashwan, HatemPuig, DomenecCamera LocalizationHybrid CNN-TransformersCoAtNetHybrid Self-AttentionTelecomunicacionesCamera localization refers to the process of automatically determining the position and orientation of a camera within its 3D environment from the images it captures. Traditional camera localization methods often rely on Convolutional Neural Networks, which are effective at extracting local visual features but struggle to capture long-range dependencies critical for accurate localization. In contrast, Transformer-based approaches model global contextual relationships appropriately, although they often lack precision in fine-grained spatial representations. To bridge this gap, we introduce CoHAtNet, a novel Convolutional Hybrid-Attention Network that tightly integrates convolutional and self-attention mechanisms. Unlike previous hybrid models that stack convolutional and attention layers separately, CoHAtNet embeds local features extracted via Mobile Inverted Bottleneck Convolution blocks directly into the Value component of the self-attention mechanism of Transformers. This yields a hybrid self-attention block capable of dynamically capturing both local spatial detail and global semantic context within a single attention layer. Additionally, CoHAtNet enables modality-level fusion by processing RGB and depth data jointly in a unified pipeline, allowing the model to leverage complementary appearance and geometric cues throughout. Extensive evaluations have been conducted on two widely-used camera localization datasets: 7-Scenes (RGB-D) and Cambridge Landmarks (RGB). Experimental results show that CoHAtNet achieves state-of-theart performance in both translation and orientation accuracy. These results highlight the effectiveness of our hybrid design in challenging indoor and outdoor environments. This makes CoHAtNet a strong candidate for end-to-end camera localization tasksThe Spanish Government partly supported this research through Project TED2021-130081B-C21, and Project PDC2022-133383-I00ElservierEscuela Politécnica SuperiorDepartamento de Tecnología Electrónica y de las ComunicacionesGobierno de España20252025-07-26research articlehttp://purl.org/coar/resource_type/c_2df8fbb1VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/10486/773520https://dx.doi.org/10.1016/j.imavis.2025.105674reponame:Biblos-e Archivo. Repositorio Institucional de la UAMinstname:Universidad Autónoma de MadridInglésengopen accesshttp://purl.org/coar/access_right/c_abf2Attribution-NonCommercial 4.0 Internationalhttp://creativecommons.org/licenses/by-nc/4.0/info:eu-repo/semantics/openAccessoai:dnet:biblosearchi::c01019ace2288adf2db2f53d78b8283b2026-06-23T12:46:27Z
dc.title.none.fl_str_mv CoHAtNet: An integrated convolutional-transformer architecture with hybrid self-attention for end-to-end camera localization
title CoHAtNet: An integrated convolutional-transformer architecture with hybrid self-attention for end-to-end camera localization
spellingShingle CoHAtNet: An integrated convolutional-transformer architecture with hybrid self-attention for end-to-end camera localization
Hasan, Hussein
Camera Localization
Hybrid CNN-Transformers
CoAtNet
Hybrid Self-Attention
Telecomunicaciones
title_short CoHAtNet: An integrated convolutional-transformer architecture with hybrid self-attention for end-to-end camera localization
title_full CoHAtNet: An integrated convolutional-transformer architecture with hybrid self-attention for end-to-end camera localization
title_fullStr CoHAtNet: An integrated convolutional-transformer architecture with hybrid self-attention for end-to-end camera localization
title_full_unstemmed CoHAtNet: An integrated convolutional-transformer architecture with hybrid self-attention for end-to-end camera localization
title_sort CoHAtNet: An integrated convolutional-transformer architecture with hybrid self-attention for end-to-end camera localization
dc.creator.none.fl_str_mv Hasan, Hussein
García García, Miguel Ángel
Rashwan, Hatem
Puig, Domenec
author Hasan, Hussein
author_facet Hasan, Hussein
García García, Miguel Ángel
Rashwan, Hatem
Puig, Domenec
author_role author
author2 García García, Miguel Ángel
Rashwan, Hatem
Puig, Domenec
author2_role author
author
author
dc.contributor.none.fl_str_mv Escuela Politécnica Superior
Departamento de Tecnología Electrónica y de las Comunicaciones
Gobierno de España
dc.subject.none.fl_str_mv Camera Localization
Hybrid CNN-Transformers
CoAtNet
Hybrid Self-Attention
Telecomunicaciones
topic Camera Localization
Hybrid CNN-Transformers
CoAtNet
Hybrid Self-Attention
Telecomunicaciones
description Camera localization refers to the process of automatically determining the position and orientation of a camera within its 3D environment from the images it captures. Traditional camera localization methods often rely on Convolutional Neural Networks, which are effective at extracting local visual features but struggle to capture long-range dependencies critical for accurate localization. In contrast, Transformer-based approaches model global contextual relationships appropriately, although they often lack precision in fine-grained spatial representations. To bridge this gap, we introduce CoHAtNet, a novel Convolutional Hybrid-Attention Network that tightly integrates convolutional and self-attention mechanisms. Unlike previous hybrid models that stack convolutional and attention layers separately, CoHAtNet embeds local features extracted via Mobile Inverted Bottleneck Convolution blocks directly into the Value component of the self-attention mechanism of Transformers. This yields a hybrid self-attention block capable of dynamically capturing both local spatial detail and global semantic context within a single attention layer. Additionally, CoHAtNet enables modality-level fusion by processing RGB and depth data jointly in a unified pipeline, allowing the model to leverage complementary appearance and geometric cues throughout. Extensive evaluations have been conducted on two widely-used camera localization datasets: 7-Scenes (RGB-D) and Cambridge Landmarks (RGB). Experimental results show that CoHAtNet achieves state-of-theart performance in both translation and orientation accuracy. These results highlight the effectiveness of our hybrid design in challenging indoor and outdoor environments. This makes CoHAtNet a strong candidate for end-to-end camera localization tasks
publishDate 2025
dc.date.none.fl_str_mv 2025
2025-07-26
dc.type.none.fl_str_mv research article
http://purl.org/coar/resource_type/c_2df8fbb1
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/10486/773520
https://dx.doi.org/10.1016/j.imavis.2025.105674
url https://hdl.handle.net/10486/773520
https://dx.doi.org/10.1016/j.imavis.2025.105674
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial 4.0 International
http://creativecommons.org/licenses/by-nc/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial 4.0 International
http://creativecommons.org/licenses/by-nc/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elservier
publisher.none.fl_str_mv Elservier
dc.source.none.fl_str_mv reponame:Biblos-e Archivo. Repositorio Institucional de la UAM
instname:Universidad Autónoma de Madrid
instname_str Universidad Autónoma de Madrid
reponame_str Biblos-e Archivo. Repositorio Institucional de la UAM
collection Biblos-e Archivo. Repositorio Institucional de la UAM
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869417411718938624
score 15,811543

Registros relacionados