No More Training: SAM’s Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation

Semantic segmentation of medical images presents an enormous potential for diagnosis and surgery. However, achieving precise results involves designing and training complex Deep Learning (DL) models specifically for this task, which is only available to some. SAM is a model developed by Meta capable...

Descripción completa

Detalles Bibliográficos
Autores: Gutiérrez Gallardo, Juan Diego, Rodriguez-Echeverria, Roberto, Delgado, Emilio, Suero-Rodrigo, Miguel Ángel, Sánchez-Figueroa, Fernando
Tipo de recurso: artículo
Fecha de publicación:2024
País:España
Institución:Universidad de Santiago de Compostela (USC)
Repositorio:Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Idioma:inglés
OAI Identifier:oai:minerva.usc.gal:10347/41028
Acceso en línea:https://hdl.handle.net/10347/41028
Access Level:acceso abierto
Palabra clave:Image segmentation
Lung
X-ray imaging
Computed tomography
Medical diagnostic imaging
Training
Task analysis
Deep learning
Zero-shot learning
Medical imaging
Semantic segmentation
id ES_9fb91e37b102ffc5e4bc5c2d5415cbce
oai_identifier_str oai:minerva.usc.gal:10347/41028
network_acronym_str ES
network_name_str España
repository_id_str
spelling No More Training: SAM’s Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image SegmentationGutiérrez Gallardo, Juan DiegoRodriguez-Echeverria, RobertoDelgado, EmilioSuero-Rodrigo, Miguel ÁngelSánchez-Figueroa, FernandoImage segmentationLungX-ray imagingComputed tomographyMedical diagnostic imagingTrainingTask analysisImage segmentationDeep learningZero-shot learningMedical imagingSemantic segmentationSemantic segmentation of medical images presents an enormous potential for diagnosis and surgery. However, achieving precise results involves designing and training complex Deep Learning (DL) models specifically for this task, which is only available to some. SAM is a model developed by Meta capable of segmenting objects present in virtually any type of image. This paper showcases SAM’s robustness and exceptional performance in medical image segmentation, even in the absence of direct training on these image types (lung Computed Tomographies (CTs) and chest X-rays, in particular). Additionally, it achieves this impressive outcome while requiring minimal user intervention. Although the dataset used to train SAM does not contain a single sample of both medical image types, processing a popular dataset comprised of 20 volumes with a total of 3520 slices using the ViT-L version of the model yields an average Jaccard index of 91.45% and an average Dice score of 94.95% . The same version of the model achieves a 93.19% Dice score and a 87.45% Jaccard index when segmenting a frequently-used chest X-ray dataset. The values obtained are above the 70% mark recommended in the literature, and close to state-of-the art models developed specifically for medical segmentation. These results are achieved without user interaction by providing the model with positive prompts based on the masks of the dataset used and a negative prompt located in the center of bounding box that contains the masks.IEEEUniversidade de Santiago de Compostela. Departamento de Electrónica e Computación20242024-01-1120242024-01-11journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/10347/41028reponame:Minerva. Repositorio Institucional de la Universidad de Santiago de Compostelainstname:Universidad de Santiago de Compostela (USC)InglésengAgencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 CPP2021-008491 MUSICGENIA: Una Plataforma en la Nube para de Generación de Música bajo Demanda por medio de Inteligencia Artificialopen accesshttp://purl.org/coar/access_right/c_abf2© 2024 The Authors. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.http://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:minerva.usc.gal:10347/410282026-06-15T12:47:27Z
dc.title.none.fl_str_mv No More Training: SAM’s Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation
title No More Training: SAM’s Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation
spellingShingle No More Training: SAM’s Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation
Gutiérrez Gallardo, Juan Diego
Image segmentation
Lung
X-ray imaging
Computed tomography
Medical diagnostic imaging
Training
Task analysis
Image segmentation
Deep learning
Zero-shot learning
Medical imaging
Semantic segmentation
title_short No More Training: SAM’s Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation
title_full No More Training: SAM’s Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation
title_fullStr No More Training: SAM’s Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation
title_full_unstemmed No More Training: SAM’s Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation
title_sort No More Training: SAM’s Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation
dc.creator.none.fl_str_mv Gutiérrez Gallardo, Juan Diego
Rodriguez-Echeverria, Roberto
Delgado, Emilio
Suero-Rodrigo, Miguel Ángel
Sánchez-Figueroa, Fernando
author Gutiérrez Gallardo, Juan Diego
author_facet Gutiérrez Gallardo, Juan Diego
Rodriguez-Echeverria, Roberto
Delgado, Emilio
Suero-Rodrigo, Miguel Ángel
Sánchez-Figueroa, Fernando
author_role author
author2 Rodriguez-Echeverria, Roberto
Delgado, Emilio
Suero-Rodrigo, Miguel Ángel
Sánchez-Figueroa, Fernando
author2_role author
author
author
author
dc.contributor.none.fl_str_mv Universidade de Santiago de Compostela. Departamento de Electrónica e Computación
dc.subject.none.fl_str_mv Image segmentation
Lung
X-ray imaging
Computed tomography
Medical diagnostic imaging
Training
Task analysis
Image segmentation
Deep learning
Zero-shot learning
Medical imaging
Semantic segmentation
topic Image segmentation
Lung
X-ray imaging
Computed tomography
Medical diagnostic imaging
Training
Task analysis
Image segmentation
Deep learning
Zero-shot learning
Medical imaging
Semantic segmentation
description Semantic segmentation of medical images presents an enormous potential for diagnosis and surgery. However, achieving precise results involves designing and training complex Deep Learning (DL) models specifically for this task, which is only available to some. SAM is a model developed by Meta capable of segmenting objects present in virtually any type of image. This paper showcases SAM’s robustness and exceptional performance in medical image segmentation, even in the absence of direct training on these image types (lung Computed Tomographies (CTs) and chest X-rays, in particular). Additionally, it achieves this impressive outcome while requiring minimal user intervention. Although the dataset used to train SAM does not contain a single sample of both medical image types, processing a popular dataset comprised of 20 volumes with a total of 3520 slices using the ViT-L version of the model yields an average Jaccard index of 91.45% and an average Dice score of 94.95% . The same version of the model achieves a 93.19% Dice score and a 87.45% Jaccard index when segmenting a frequently-used chest X-ray dataset. The values obtained are above the 70% mark recommended in the literature, and close to state-of-the art models developed specifically for medical segmentation. These results are achieved without user interaction by providing the model with positive prompts based on the masks of the dataset used and a negative prompt located in the center of bounding box that contains the masks.
publishDate 2024
dc.date.none.fl_str_mv 2024
2024-01-11
2024
2024-01-11
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/10347/41028
url https://hdl.handle.net/10347/41028
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Agencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 CPP2021-008491 MUSICGENIA: Una Plataforma en la Nube para de Generación de Música bajo Demanda por medio de Inteligencia Artificial
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv IEEE
publisher.none.fl_str_mv IEEE
dc.source.none.fl_str_mv reponame:Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
instname:Universidad de Santiago de Compostela (USC)
instname_str Universidad de Santiago de Compostela (USC)
reponame_str Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
collection Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869414952171732992
score 15,812429

Similares en LA Referencia