Closing the gap in domain adaptation for semantic segmentation

Semantic segmentation models need a large number of images to be effectively trained but manual annotation of such images has a high cost. Active domain adaptation addresses this problem by pretraining the model with a synthetically generated dataset and then fine-tuning it with a few selected label...

Descripción completa

Detalles Bibliográficos
Autores: Serrat, Joan|||0000-0002-4554-199X, Gomez Zurita, Jose Luis|||0000-0001-9511-1915, López Peña, Antonio M.|||0000-0002-6979-5783
Tipo de recurso: artículo
Fecha de publicación:2025
País:España
Institución:Universitat Autònoma de Barcelona
Repositorio:Dipòsit Digital de Documents de la UAB
Idioma:inglés
OAI Identifier:oai:ddd.uab.cat:308671
Acceso en línea:https://ddd.uab.cat/record/308671
https://dx.doi.org/urn:doi:10.1007/s00138-024-01626-z
Access Level:acceso abierto
Palabra clave:Active learning
Domain adaptation
Semantic segmentation
Foundation model
id ES_eefdbbfcf73f42fc29ce76c1895e690c
oai_identifier_str oai:ddd.uab.cat:308671
network_acronym_str ES
network_name_str España
repository_id_str
spelling Closing the gap in domain adaptation for semantic segmentationa time-aware methodSerrat, Joan|||0000-0002-4554-199XGomez Zurita, Jose Luis|||0000-0001-9511-1915López Peña, Antonio M.|||0000-0002-6979-5783Active learningDomain adaptationSemantic segmentationFoundation modelSemantic segmentation models need a large number of images to be effectively trained but manual annotation of such images has a high cost. Active domain adaptation addresses this problem by pretraining the model with a synthetically generated dataset and then fine-tuning it with a few selected label annotations (the "budget") on real images to account for the domain shift. Previous works annotate a percentage of either individual pixels or whole target images. We argue that the first is infeasible in practice, and the second spends part of the budget on classes that the pretrained model may have already learned well. We propose a method based on the annotation of regions computed by Segment Anything, a recently introduced foundation model for class-agnostic image segmentation. The key idea is to assign a ground truth label to each of a tiny subset of regions, those for which the model is more uncertain. In order to increase the number of annotated regions we propagate the ground truth labels to most similar regions according to a hierarchical clustering algorithm that uses the features learned by the pretrained model. Our method outperforms the state-of-the-art on the GTA5 to Cityscapes benchmark by using fewer annotations, almost closing the gap between the synthetically pre-trained model and that obtained with full supervision of the real images. Furthermore, we present competitive results for budgets less than 1% of samples and also for a larger and more challenging target dataset, Mapillary Vistas. 22025-01-0120252025-01-01Articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://ddd.uab.cat/record/308671https://dx.doi.org/urn:doi:10.1007/s00138-024-01626-zreponame:Dipòsit Digital de Documents de la UABinstname:Universitat Autònoma de BarcelonaInglésengAgencia Estatal de Investigación https://doi.org/10.13039/501100011033 PID2020-115734RB-C21open accesshttp://purl.org/coar/access_right/c_abf2Aquest document està subjecte a una llicència d'ús Creative Commons. Es permet la reproducció total o parcial, la distribució, la comunicació pública de l'obra i la creació d'obres derivades, fins i tot amb finalitats comercials, sempre i quan es reconegui l'autoria de l'obra original.https://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:ddd.uab.cat:3086712026-06-06T12:50:31Z
dc.title.none.fl_str_mv Closing the gap in domain adaptation for semantic segmentation
a time-aware method
title Closing the gap in domain adaptation for semantic segmentation
spellingShingle Closing the gap in domain adaptation for semantic segmentation
Serrat, Joan|||0000-0002-4554-199X
Active learning
Domain adaptation
Semantic segmentation
Foundation model
title_short Closing the gap in domain adaptation for semantic segmentation
title_full Closing the gap in domain adaptation for semantic segmentation
title_fullStr Closing the gap in domain adaptation for semantic segmentation
title_full_unstemmed Closing the gap in domain adaptation for semantic segmentation
title_sort Closing the gap in domain adaptation for semantic segmentation
dc.creator.none.fl_str_mv Serrat, Joan|||0000-0002-4554-199X
Gomez Zurita, Jose Luis|||0000-0001-9511-1915
López Peña, Antonio M.|||0000-0002-6979-5783
author Serrat, Joan|||0000-0002-4554-199X
author_facet Serrat, Joan|||0000-0002-4554-199X
Gomez Zurita, Jose Luis|||0000-0001-9511-1915
López Peña, Antonio M.|||0000-0002-6979-5783
author_role author
author2 Gomez Zurita, Jose Luis|||0000-0001-9511-1915
López Peña, Antonio M.|||0000-0002-6979-5783
author2_role author
author
dc.subject.none.fl_str_mv Active learning
Domain adaptation
Semantic segmentation
Foundation model
topic Active learning
Domain adaptation
Semantic segmentation
Foundation model
description Semantic segmentation models need a large number of images to be effectively trained but manual annotation of such images has a high cost. Active domain adaptation addresses this problem by pretraining the model with a synthetically generated dataset and then fine-tuning it with a few selected label annotations (the "budget") on real images to account for the domain shift. Previous works annotate a percentage of either individual pixels or whole target images. We argue that the first is infeasible in practice, and the second spends part of the budget on classes that the pretrained model may have already learned well. We propose a method based on the annotation of regions computed by Segment Anything, a recently introduced foundation model for class-agnostic image segmentation. The key idea is to assign a ground truth label to each of a tiny subset of regions, those for which the model is more uncertain. In order to increase the number of annotated regions we propagate the ground truth labels to most similar regions according to a hierarchical clustering algorithm that uses the features learned by the pretrained model. Our method outperforms the state-of-the-art on the GTA5 to Cityscapes benchmark by using fewer annotations, almost closing the gap between the synthetically pre-trained model and that obtained with full supervision of the real images. Furthermore, we present competitive results for budgets less than 1% of samples and also for a larger and more challenging target dataset, Mapillary Vistas.
publishDate 2025
dc.date.none.fl_str_mv 2
2025-01-01
2025
2025-01-01
dc.type.none.fl_str_mv Article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://ddd.uab.cat/record/308671
https://dx.doi.org/urn:doi:10.1007/s00138-024-01626-z
url https://ddd.uab.cat/record/308671
https://dx.doi.org/urn:doi:10.1007/s00138-024-01626-z
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Agencia Estatal de Investigación https://doi.org/10.13039/501100011033 PID2020-115734RB-C21
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
https://creativecommons.org/licenses/by/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
https://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.source.none.fl_str_mv reponame:Dipòsit Digital de Documents de la UAB
instname:Universitat Autònoma de Barcelona
instname_str Universitat Autònoma de Barcelona
reponame_str Dipòsit Digital de Documents de la UAB
collection Dipòsit Digital de Documents de la UAB
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869423796823261184
score 15.81155

Similares en LA Referencia