Real-time 3D reconstruction of non-rigid shapes with a single moving camera

This paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the...

Descripción completa

Detalles Bibliográficos
Autores: Agudo Martínez, Antonio, Moreno-Noguer, Francesc, Calvo, Begoña, Montiel, J. M. M.
Tipo de recurso: artículo
Estado:Versión enviada para evaluación y publicación
Fecha de publicación:2016
País:España
Institución:Consejo Superior de Investigaciones Científicas (CSIC)
Repositorio:DIGITAL.CSIC. Repositorio Institucional del CSIC
OAI Identifier:oai:digital.csic.es:10261/132981
Acceso en línea:http://hdl.handle.net/10261/132981
Access Level:acceso abierto
Palabra clave:Deformable shape and camera motion recovery
Extended kalman filter
Finite element method
Tracking
Real-time vision
id ES_af5c4edd4deb2dcb3be81bccd7661bcf
oai_identifier_str oai:digital.csic.es:10261/132981
network_acronym_str ES
network_name_str España
repository_id_str
spelling Real-time 3D reconstruction of non-rigid shapes with a single moving cameraAgudo Martínez, AntonioMoreno-Noguer, FrancescCalvo, BegoñaMontiel, J. M. M.Deformable shape and camera motion recoveryExtended kalman filterFinite element methodTrackingReal-time visionThis paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the time-varying shape per frame. These equations are embedded in an extended Kalman filter, resulting in sequential Bayesian estimation approach. We represent the shape, with unknown material properties, as a combination of elastic elements whose nodal points correspond to salient points in the image. The global rigidity of the shape is encoded by a stiffness matrix, computed after assembling each of these elements. With this piecewise model, we can linearly relate the 3D displacements with the 3D acting forces that cause the object deformation, assumed to be normally distributed. While standard finite-element-method techniques require imposing boundary conditions to solve the resulting linear system, in this work we eliminate this requirement by modeling the compliance matrix with a generalized pseudoinverse that enforces a pre-fixed rank. Our framework also ensures surface continuity without the need for a post-processing step to stitch all the piecewise reconstructions into a global smooth shape. We present experimental results using both synthetic and real videos for different scenarios ranging from isometric to elastic deformations. We also show the consistency of the estimation with respect to 3D ground truth data, include several experiments assessing robustness against artifacts and finally, provide an experimental validation of our performance in real time at frame rate for small maps.This work has been partially supported by the Spanish Ministry of Science and Innovation under projects RT-SLAM DPI2015-67275-P, RobInstruct TIN2014-58178-R and Keratocono DPI2014-54981-R; by the FP7-SME-2013 606634 PopCorn project from the European Union FP7; by the ERA-net CHISTERA project VISEN PCIN-2013-047 and I-DRESS PCIN-2015-147; and by a scholarship FPU12/04886 awarded by the Spanish MECD.Peer reviewedElsevierMinisterio de Educación, Cultura y Deporte (España)Ministerio de Ciencia e Innovación (España)Ministerio de Economía y Competitividad (España)European CommissionConsejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]201620162016info:eu-repo/semantics/articlehttp://purl.org/coar/resource_type/c_6501Preprintinfo:eu-repo/semantics/submittedVersionhttp://hdl.handle.net/10261/132981reponame:DIGITAL.CSIC. Repositorio Institucional del CSICinstname:Consejo Superior de Investigaciones Científicas (CSIC)Inglés#PLACEHOLDER_PARENT_METADATA_VALUE##PLACEHOLDER_PARENT_METADATA_VALUE##PLACEHOLDER_PARENT_METADATA_VALUE##PLACEHOLDER_PARENT_METADATA_VALUE##PLACEHOLDER_PARENT_METADATA_VALUE##PLACEHOLDER_PARENT_METADATA_VALUE#info:eu-repo/grantAgreement/MINECO/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/DPI2014-54981-Rinfo:eu-repo/grantAgreement/EC/FP7/606634info:eu-repo/grantAgreement/MINECO/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/TIN2014-58178-Rinfo:eu-repo/grantAgreement/MINECO/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/DPI2015-67275-Pinfo:eu-repo/grantAgreement/MINECO/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/PCIN-2013-047info:eu-repo/grantAgreement/MINECO/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/PCIN-2015-147http://dx.doi.org/10.1016/j.cviu.2016.05.004Síinfo:eu-repo/semantics/openAccessoai:digital.csic.es:10261/1329812026-05-22T06:33:51Z
dc.title.none.fl_str_mv Real-time 3D reconstruction of non-rigid shapes with a single moving camera
title Real-time 3D reconstruction of non-rigid shapes with a single moving camera
spellingShingle Real-time 3D reconstruction of non-rigid shapes with a single moving camera
Agudo Martínez, Antonio
Deformable shape and camera motion recovery
Extended kalman filter
Finite element method
Tracking
Real-time vision
title_short Real-time 3D reconstruction of non-rigid shapes with a single moving camera
title_full Real-time 3D reconstruction of non-rigid shapes with a single moving camera
title_fullStr Real-time 3D reconstruction of non-rigid shapes with a single moving camera
title_full_unstemmed Real-time 3D reconstruction of non-rigid shapes with a single moving camera
title_sort Real-time 3D reconstruction of non-rigid shapes with a single moving camera
dc.creator.none.fl_str_mv Agudo Martínez, Antonio
Moreno-Noguer, Francesc
Calvo, Begoña
Montiel, J. M. M.
author Agudo Martínez, Antonio
author_facet Agudo Martínez, Antonio
Moreno-Noguer, Francesc
Calvo, Begoña
Montiel, J. M. M.
author_role author
author2 Moreno-Noguer, Francesc
Calvo, Begoña
Montiel, J. M. M.
author2_role author
author
author
dc.contributor.none.fl_str_mv Ministerio de Educación, Cultura y Deporte (España)
Ministerio de Ciencia e Innovación (España)
Ministerio de Economía y Competitividad (España)
European Commission
Consejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]
dc.subject.none.fl_str_mv Deformable shape and camera motion recovery
Extended kalman filter
Finite element method
Tracking
Real-time vision
topic Deformable shape and camera motion recovery
Extended kalman filter
Finite element method
Tracking
Real-time vision
description This paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the time-varying shape per frame. These equations are embedded in an extended Kalman filter, resulting in sequential Bayesian estimation approach. We represent the shape, with unknown material properties, as a combination of elastic elements whose nodal points correspond to salient points in the image. The global rigidity of the shape is encoded by a stiffness matrix, computed after assembling each of these elements. With this piecewise model, we can linearly relate the 3D displacements with the 3D acting forces that cause the object deformation, assumed to be normally distributed. While standard finite-element-method techniques require imposing boundary conditions to solve the resulting linear system, in this work we eliminate this requirement by modeling the compliance matrix with a generalized pseudoinverse that enforces a pre-fixed rank. Our framework also ensures surface continuity without the need for a post-processing step to stitch all the piecewise reconstructions into a global smooth shape. We present experimental results using both synthetic and real videos for different scenarios ranging from isometric to elastic deformations. We also show the consistency of the estimation with respect to 3D ground truth data, include several experiments assessing robustness against artifacts and finally, provide an experimental validation of our performance in real time at frame rate for small maps.
publishDate 2016
dc.date.none.fl_str_mv 2016
2016
2016
dc.type.none.fl_str_mv info:eu-repo/semantics/article
http://purl.org/coar/resource_type/c_6501
Preprint
info:eu-repo/semantics/submittedVersion
format article
status_str submittedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/10261/132981
url http://hdl.handle.net/10261/132981
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv #PLACEHOLDER_PARENT_METADATA_VALUE#
#PLACEHOLDER_PARENT_METADATA_VALUE#
#PLACEHOLDER_PARENT_METADATA_VALUE#
#PLACEHOLDER_PARENT_METADATA_VALUE#
#PLACEHOLDER_PARENT_METADATA_VALUE#
#PLACEHOLDER_PARENT_METADATA_VALUE#
info:eu-repo/grantAgreement/MINECO/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/DPI2014-54981-R
info:eu-repo/grantAgreement/EC/FP7/606634
info:eu-repo/grantAgreement/MINECO/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/TIN2014-58178-R
info:eu-repo/grantAgreement/MINECO/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/DPI2015-67275-P
info:eu-repo/grantAgreement/MINECO/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/PCIN-2013-047
info:eu-repo/grantAgreement/MINECO/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/PCIN-2015-147
http://dx.doi.org/10.1016/j.cviu.2016.05.004
Sí
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:DIGITAL.CSIC. Repositorio Institucional del CSIC
instname:Consejo Superior de Investigaciones Científicas (CSIC)
instname_str Consejo Superior de Investigaciones Científicas (CSIC)
reponame_str DIGITAL.CSIC. Repositorio Institucional del CSIC
collection DIGITAL.CSIC. Repositorio Institucional del CSIC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869416677776556032
score 15.812455

Similares en LA Referencia