Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures

Cyanobacteria play a fundamental role in aquatic ecosystems, contributing to global biogeochemical cycles and serving as indicators of environmental change. Their classification is critical for monitoring water quality, detecting harmful algal blooms and understanding ecosystem dynamics. However, ac...

Descripción completa

Detalles Bibliográficos
Autores: Blanco, Maria, Ruiz‑Santaquiteria,Jesús, Cristóbal, Gabriel, Perona Urízar, Elvira Victoria, Bueno, Gloria
Tipo de recurso: artículo
Fecha de publicación:2025
País:España
Institución:Universidad Autónoma de Madrid
Repositorio:Biblos-e Archivo. Repositorio Institucional de la UAM
Idioma:inglés
OAI Identifier:oai:dnet:biblosearchi::9dfa5ba5be70d35d8ebcfa0ae51a6b35
Acceso en línea:https://hdl.handle.net/10486/763520
https://dx.doi.org/10.1007/s10452-025-10227-5
Access Level:acceso abierto
Palabra clave:Multimodal deep learning
cyanobacteria classification
text-image classifiers
bidirectional transformers
Biología y Biomedicina / Biología
id ES_d98c6f471c581be93a3aa45086e82d21
oai_identifier_str oai:dnet:biblosearchi::9dfa5ba5be70d35d8ebcfa0ae51a6b35
network_acronym_str ES
network_name_str España
repository_id_str
spelling Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architecturesBlanco, MariaRuiz‑Santaquiteria,JesúsCristóbal, GabrielPerona Urízar, Elvira VictoriaBueno, GloriaMultimodal deep learningcyanobacteria classificationtext-image classifiersbidirectional transformersBiología y Biomedicina / BiologíaCyanobacteria play a fundamental role in aquatic ecosystems, contributing to global biogeochemical cycles and serving as indicators of environmental change. Their classification is critical for monitoring water quality, detecting harmful algal blooms and understanding ecosystem dynamics. However, accurate identification remains a major challenge due to their vast taxonomic diversity and significant morphological similarities. Visual inspection alone is often insufficient, highlighting the need for computational approaches to enhance classification accuracy. In this study, we present a multimodal deep learning model that combines convolutional neural networks (CNNs) for image-based feature extraction with bidirectional transformers for text embedding. These complementary features are fused via concatenation to improve species-level classification. To our knowledge, this is the first application of a multimodal neural architecture integrating CNNs and bidirectional transformers for cyanobacteria classification. We evaluated five CNN backbones of varying depth, resulting in eight model configurations. Performance is benchmarked against unimodal CNN models that rely solely on image data. The model is trained and validated on a dataset of 1660 microscopic images and corresponding textual descriptions, covering nine cyanobacterial genera across three taxonomic orders. Results demonstrate the potential of multimodal deep learning to improve classification performance, supporting the development of scalable and accurate identification tools in microbiology and environmental monitoringOpen Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was funded by project TED2021-132147B-100 (funded by MCIN/ AEI/10.13039/501100011033 and by the European Union Next GenerationEU/PRTR)SpringerDepartamento de BiologíaFacultad de CienciasAgencia Estatal de Investigación20252025-09-12research articlehttp://purl.org/coar/resource_type/c_2df8fbb1VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/10486/763520https://dx.doi.org/10.1007/s10452-025-10227-5reponame:Biblos-e Archivo. Repositorio Institucional de la UAMinstname:Universidad Autónoma de MadridInglésengopen accesshttp://purl.org/coar/access_right/c_abf2Attribution 4.0 Internationalhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:dnet:biblosearchi::9dfa5ba5be70d35d8ebcfa0ae51a6b352026-06-23T12:46:27Z
dc.title.none.fl_str_mv Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
title Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
spellingShingle Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
Blanco, Maria
Multimodal deep learning
cyanobacteria classification
text-image classifiers
bidirectional transformers
Biología y Biomedicina / Biología
title_short Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
title_full Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
title_fullStr Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
title_full_unstemmed Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
title_sort Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
dc.creator.none.fl_str_mv Blanco, Maria
Ruiz‑Santaquiteria,Jesús
Cristóbal, Gabriel
Perona Urízar, Elvira Victoria
Bueno, Gloria
author Blanco, Maria
author_facet Blanco, Maria
Ruiz‑Santaquiteria,Jesús
Cristóbal, Gabriel
Perona Urízar, Elvira Victoria
Bueno, Gloria
author_role author
author2 Ruiz‑Santaquiteria,Jesús
Cristóbal, Gabriel
Perona Urízar, Elvira Victoria
Bueno, Gloria
author2_role author
author
author
author
dc.contributor.none.fl_str_mv Departamento de Biología
Facultad de Ciencias
Agencia Estatal de Investigación
dc.subject.none.fl_str_mv Multimodal deep learning
cyanobacteria classification
text-image classifiers
bidirectional transformers
Biología y Biomedicina / Biología
topic Multimodal deep learning
cyanobacteria classification
text-image classifiers
bidirectional transformers
Biología y Biomedicina / Biología
description Cyanobacteria play a fundamental role in aquatic ecosystems, contributing to global biogeochemical cycles and serving as indicators of environmental change. Their classification is critical for monitoring water quality, detecting harmful algal blooms and understanding ecosystem dynamics. However, accurate identification remains a major challenge due to their vast taxonomic diversity and significant morphological similarities. Visual inspection alone is often insufficient, highlighting the need for computational approaches to enhance classification accuracy. In this study, we present a multimodal deep learning model that combines convolutional neural networks (CNNs) for image-based feature extraction with bidirectional transformers for text embedding. These complementary features are fused via concatenation to improve species-level classification. To our knowledge, this is the first application of a multimodal neural architecture integrating CNNs and bidirectional transformers for cyanobacteria classification. We evaluated five CNN backbones of varying depth, resulting in eight model configurations. Performance is benchmarked against unimodal CNN models that rely solely on image data. The model is trained and validated on a dataset of 1660 microscopic images and corresponding textual descriptions, covering nine cyanobacterial genera across three taxonomic orders. Results demonstrate the potential of multimodal deep learning to improve classification performance, supporting the development of scalable and accurate identification tools in microbiology and environmental monitoring
publishDate 2025
dc.date.none.fl_str_mv 2025
2025-09-12
dc.type.none.fl_str_mv research article
http://purl.org/coar/resource_type/c_2df8fbb1
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/10486/763520
https://dx.doi.org/10.1007/s10452-025-10227-5
url https://hdl.handle.net/10486/763520
https://dx.doi.org/10.1007/s10452-025-10227-5
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Springer
publisher.none.fl_str_mv Springer
dc.source.none.fl_str_mv reponame:Biblos-e Archivo. Repositorio Institucional de la UAM
instname:Universidad Autónoma de Madrid
instname_str Universidad Autónoma de Madrid
reponame_str Biblos-e Archivo. Repositorio Institucional de la UAM
collection Biblos-e Archivo. Repositorio Institucional de la UAM
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869421397573369856
score 15.812429

Similares en LA Referencia