Weakly-Supervised Deep Shape-from-Template
We propose WS-DeepSfT, a novel deep learning-based approach to the Shape-from-Template (SfT) problem, which aims at reconstructing the 3D shape of a deformable object from a single RGB image and a template. WS-DeepSfT addresses the limitations of existing SfT techniques by combining a weakly-supervi...
| Autores: | , , , , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2025 |
| País: | España |
| Institución: | Universidad de Alcalá (UAH) |
| Repositorio: | e_Buah Biblioteca Digital Universidad de Alcalá |
| Idioma: | inglés |
| OAI Identifier: | oai:ebuah.uah.es:10017/64580 |
| Acceso en línea: | http://hdl.handle.net/10017/64580 https://dx.doi.org/10.1109/ACCESS.2025.3534271 |
| Access Level: | acceso abierto |
| Palabra clave: | Non-rigid Shape-from-Template Weak-supervision Registration Wide-baseline Template-based, 3D reconstruction Electrónica Electronics |
| Sumario: | We propose WS-DeepSfT, a novel deep learning-based approach to the Shape-from-Template (SfT) problem, which aims at reconstructing the 3D shape of a deformable object from a single RGB image and a template. WS-DeepSfT addresses the limitations of existing SfT techniques by combining a weakly-supervised deep neural network (DNN) for registration and a classical As-Rigid-As-Possible (ARAP) algorithm for 3D reconstruction. Unlike previous deep learning-based SfT methods, which require extensive synthetic data and depth sensors for training, WS-DeepSfT only requires regular RGB video of the deforming object and a segmentation mask to discriminate the object from the background. The registration model is trained without synthetic data, using videos where the object undergoes deformations, while ARAP does not require training and infers the 3D shape in real-time with minimal overhead. We show that WSDeepSfT outperforms the state-of-the-art, in both accuracy and robustness, without requiring depth sensors or synthetic data generation. WS-DeepSfT thus offers a robust, efficient, and scalable approach to SfT, bringing it closer to applications such as augmented reality. |
|---|