A hybrid transformer-CNN framework for multimodal microscopy cell segmentation
Accurate cell instance segmentation in microscopy images is essential for downstream biomedical analysis, yet remains challenging due to variability in imaging modalities, crowded cell arrangements, and faint or ambiguous boundaries. This thesis investigates hybrid architectures that integrate convo...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Fecha de publicación: | 2025 |
| País: | España |
| Institución: | Consejo Superior de Investigaciones Científicas (CSIC) |
| Repositorio: | O2, repositorio institucional de la UOC |
| OAI Identifier: | oai:openaccess.uoc.edu:10609/153409 |
| Acceso en línea: | https://hdl.handle.net/10609/153409 |
| Access Level: | acceso abierto |
| Palabra clave: | cell instance segmentation multimodal microscopy Hybrid CNN-transformer microscopy images Master thesis Microscopy -- Technique -- FMDP Microscopis - -Tècnica -- TFM |
| Sumario: | Accurate cell instance segmentation in microscopy images is essential for downstream biomedical analysis, yet remains challenging due to variability in imaging modalities, crowded cell arrangements, and faint or ambiguous boundaries. This thesis investigates hybrid architectures that integrate convolutional neural networks (CNN) with transformer encoders to combine local feature sensitivity with global context modeling. Using the NeurIPS 2022 Cell Segmentation Challenge dataset, several models were implemented and evaluated: a CNN baseline (ResNet-34), a transformer baseline (MiT-B5), and multiple hybrid variants with fusion strategies (naive concatenation, cross-attention, and uncertainty gated cross-attention). Efficiency oriented hybrids with smaller transformer backbones (MiT-B2) and lean decoders were also studied. All models employed a multi head decoder that predicts probability, gradient and distance maps, subsequently combined through a custom post processing pipeline. Results on the challenge test set show that the transformer baseline is the most reliable single model, whereas hybrids improve over the CNN baseline and yield targeted wins (especially in fluorescence), but do not surpass the transformer despite being more competitive during validation. Efficiency studies show that downscaling hybrid backbones retains most hybrid accuracy at lower computational cost, while end to end latency is mostly influenced by postprocessing. Overall, transformers offer the strongest single backbone baseline in this setting; hybrids remain promising when paired with system level components such as modality aware routing or learned postprocessing. This thesis concludes with practical recommendations for closing the gap to challenge leaders. |
|---|