Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures

Cyanobacteria play a fundamental role in aquatic ecosystems, contributing to global biogeochemical cycles and serving as indicators of environmental change. Their classification is critical for monitoring water quality, detecting harmful algal blooms and understanding ecosystem dynamics. However, ac...

Full description

Bibliographic Details
Authors: Blanco González-Mohíno, María, Cristóbal , Gabriel, Perona , Elvira, Ruiz-Santaquiteria Alegre, Jesús, Bueno García, María Gloria
Format: article
Publication Date:2025
Country:España
Institution:Universidad de Castilla-La Mancha
Repository:RUIdeRA. Repositorio Institucional de la UCLM
OAI Identifier:oai:ruidera.uclm.es:10578/46274
Online Access:https://doi.org/10.1007/s10452-025-10227-5
https://hdl.handle.net/10578/46274
Access Level:Open access
Keyword:Bidirectional transformers
Cyanobacteria classification
Multimodal deep learning
Text-image classifiers
id ES_6fadf60f2ee2dedfe38ea44f19f26b66
oai_identifier_str oai:ruidera.uclm.es:10578/46274
network_acronym_str ES
network_name_str España
repository_id_str
spelling Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architecturesBlanco González-Mohíno, MaríaCristóbal , GabrielPerona , ElviraRuiz-Santaquiteria Alegre, JesúsBueno García, María GloriaBidirectional transformersCyanobacteria classificationMultimodal deep learningText-image classifiersCyanobacteria play a fundamental role in aquatic ecosystems, contributing to global biogeochemical cycles and serving as indicators of environmental change. Their classification is critical for monitoring water quality, detecting harmful algal blooms and understanding ecosystem dynamics. However, accurate identification remains a major challenge due to their vast taxonomic diversity and significant morphological similarities. Visual inspection alone is often insufficient, highlighting the need for computational approaches to enhance classification accuracy. In this study, we present a multimodal deep learning model that combines convolutional neural networks (CNNs) for image-based feature extraction with bidirectional transformers for text embedding. These complementary features are fused via concatenation to improve species-level classification. To our knowledge, this is the first application of a multimodal neural architecture integrating CNNs and bidirectional transformers for cyanobacteria classification. We evaluated five CNN backbones of varying depth, resulting in eight model configurations. Performance is benchmarked against unimodal CNN models that rely solely on image data. The model is trained and validated on a dataset of 1660 microscopic images and corresponding textual descriptions, covering nine cyanobacterial genera across three taxonomic orders. Results demonstrate the potential of multimodal deep learning to improve classification performance, supporting the development of scalable and accurate identification tools in microbiology and environmental monitoring.Springer202620262025info:eu-repo/semantics/articleapplication/pdfapplication/pdfhttps://doi.org/10.1007/s10452-025-10227-5https://hdl.handle.net/10578/46274reponame:RUIdeRA. Repositorio Institucional de la UCLMinstname:Universidad de Castilla-La ManchaInglésThis work was funded by project TED2021-132147B-100 (funded by MCIN/AEI/10.13039/501100011033 and by the European Union Next GenerationEU/PRTR)info:eu-repo/semantics/openAccessoai:ruidera.uclm.es:10578/462742026-05-27T07:36:41Z
dc.title.none.fl_str_mv Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
title Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
spellingShingle Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
Blanco González-Mohíno, María
Bidirectional transformers
Cyanobacteria classification
Multimodal deep learning
Text-image classifiers
title_short Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
title_full Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
title_fullStr Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
title_full_unstemmed Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
title_sort Multimodal deep learning for cyanobacteria classification: a fusion of CNN and transformer architectures
dc.creator.none.fl_str_mv Blanco González-Mohíno, María
Cristóbal , Gabriel
Perona , Elvira
Ruiz-Santaquiteria Alegre, Jesús
Bueno García, María Gloria
author Blanco González-Mohíno, María
author_facet Blanco González-Mohíno, María
Cristóbal , Gabriel
Perona , Elvira
Ruiz-Santaquiteria Alegre, Jesús
Bueno García, María Gloria
author_role author
author2 Cristóbal , Gabriel
Perona , Elvira
Ruiz-Santaquiteria Alegre, Jesús
Bueno García, María Gloria
author2_role author
author
author
author
dc.subject.none.fl_str_mv Bidirectional transformers
Cyanobacteria classification
Multimodal deep learning
Text-image classifiers
topic Bidirectional transformers
Cyanobacteria classification
Multimodal deep learning
Text-image classifiers
description Cyanobacteria play a fundamental role in aquatic ecosystems, contributing to global biogeochemical cycles and serving as indicators of environmental change. Their classification is critical for monitoring water quality, detecting harmful algal blooms and understanding ecosystem dynamics. However, accurate identification remains a major challenge due to their vast taxonomic diversity and significant morphological similarities. Visual inspection alone is often insufficient, highlighting the need for computational approaches to enhance classification accuracy. In this study, we present a multimodal deep learning model that combines convolutional neural networks (CNNs) for image-based feature extraction with bidirectional transformers for text embedding. These complementary features are fused via concatenation to improve species-level classification. To our knowledge, this is the first application of a multimodal neural architecture integrating CNNs and bidirectional transformers for cyanobacteria classification. We evaluated five CNN backbones of varying depth, resulting in eight model configurations. Performance is benchmarked against unimodal CNN models that rely solely on image data. The model is trained and validated on a dataset of 1660 microscopic images and corresponding textual descriptions, covering nine cyanobacterial genera across three taxonomic orders. Results demonstrate the potential of multimodal deep learning to improve classification performance, supporting the development of scalable and accurate identification tools in microbiology and environmental monitoring.
publishDate 2025
dc.date.none.fl_str_mv 2025
2026
2026
dc.type.none.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://doi.org/10.1007/s10452-025-10227-5
https://hdl.handle.net/10578/46274
url https://doi.org/10.1007/s10452-025-10227-5
https://hdl.handle.net/10578/46274
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv This work was funded by project TED2021-132147B-100 (funded by MCIN/AEI/10.13039/501100011033 and by the European Union Next GenerationEU/PRTR)
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Springer
publisher.none.fl_str_mv Springer
dc.source.none.fl_str_mv reponame:RUIdeRA. Repositorio Institucional de la UCLM
instname:Universidad de Castilla-La Mancha
instname_str Universidad de Castilla-La Mancha
reponame_str RUIdeRA. Repositorio Institucional de la UCLM
collection RUIdeRA. Repositorio Institucional de la UCLM
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869410525414162432
score 15,811543

Similar Items