BabyNet: reconstructing 3D faces of babies from uncalibrated photographs

We present a 3D face reconstruction system that aims at recovering the 3D facial geometry of babies from uncalibrated photographs, BabyNet. Since the 3D facial geometry of babies differs substantially from that of adults, baby-specific facial reconstruction systems are needed. BabyNet consists of tw...

Full description

Bibliographic Details
Authors: Morales, Araceli, Alomar Adrover, Antònia, Porras Pérez, Antonio Reyes, Linguraru, Marius George, Piella Fenoy, Gemma, Sukno, Federico Mateo
Format: article
Status:Published version
Publication Date:2023
Country:España
Institution:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
Repository:Recercat. Dipósit de la Recerca de Catalunya
OAI Identifier:oai:recercat.cat:10230/57511
Online Access:http://hdl.handle.net/10230/57511
http://dx.doi.org/10.1016/j.patcog.2023.109367
Access Level:Open access
Keyword:3D face reconstruction
Graph neural network
Baby model
id ES_d60db1fc5a03a512676b64cecdbfbd01
oai_identifier_str oai:recercat.cat:10230/57511
network_acronym_str ES
network_name_str España
repository_id_str
spelling BabyNet: reconstructing 3D faces of babies from uncalibrated photographsMorales, AraceliAlomar Adrover, AntòniaPorras Pérez, Antonio ReyesLinguraru, Marius GeorgePiella Fenoy, GemmaSukno, Federico Mateo3D face reconstructionGraph neural networkBaby modelWe present a 3D face reconstruction system that aims at recovering the 3D facial geometry of babies from uncalibrated photographs, BabyNet. Since the 3D facial geometry of babies differs substantially from that of adults, baby-specific facial reconstruction systems are needed. BabyNet consists of two stages: 1) a 3D graph convolutional autoencoder learns a latent space of the baby 3D facial shape; and 2) a 2D encoder that maps photographs to the 3D latent space based on representative features extracted using transfer learning. In this way, using the pre-trained 3D decoder, we can recover a 3D face from 2D images. We evaluate BabyNet and show that 1) methods based on adult datasets cannot model the 3D facial geometry of babies, which proves the need for a baby-specific method, and 2) BabyNet outperforms classical model-fitting methods even when a baby-specific 3D morphable model, such as BabyFM, is used.This work is partly supported by the Spanish Ministry of Science and Innovation under project grant PID2020-114083GB-I00 and the NIH Eunice Kennedy Shriver National Institute of Child Health & Human Development grant R42 HD08171203. G. Piella is supported by ICREA under the ICREA Academia programme.Elsevier202320232023info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfapplication/pdfhttp://hdl.handle.net/10230/57511http://dx.doi.org/10.1016/j.patcog.2023.109367reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)InglésPattern Recognition. 2023;139:109367.info:eu-repo/grantAgreement/ES/2PE/PID2020-114083GB-I00This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)http://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:recercat.cat:10230/575112026-05-29T05:05:01Z
dc.title.none.fl_str_mv BabyNet: reconstructing 3D faces of babies from uncalibrated photographs
title BabyNet: reconstructing 3D faces of babies from uncalibrated photographs
spellingShingle BabyNet: reconstructing 3D faces of babies from uncalibrated photographs
Morales, Araceli
3D face reconstruction
Graph neural network
Baby model
title_short BabyNet: reconstructing 3D faces of babies from uncalibrated photographs
title_full BabyNet: reconstructing 3D faces of babies from uncalibrated photographs
title_fullStr BabyNet: reconstructing 3D faces of babies from uncalibrated photographs
title_full_unstemmed BabyNet: reconstructing 3D faces of babies from uncalibrated photographs
title_sort BabyNet: reconstructing 3D faces of babies from uncalibrated photographs
dc.creator.none.fl_str_mv Morales, Araceli
Alomar Adrover, Antònia
Porras Pérez, Antonio Reyes
Linguraru, Marius George
Piella Fenoy, Gemma
Sukno, Federico Mateo
author Morales, Araceli
author_facet Morales, Araceli
Alomar Adrover, Antònia
Porras Pérez, Antonio Reyes
Linguraru, Marius George
Piella Fenoy, Gemma
Sukno, Federico Mateo
author_role author
author2 Alomar Adrover, Antònia
Porras Pérez, Antonio Reyes
Linguraru, Marius George
Piella Fenoy, Gemma
Sukno, Federico Mateo
author2_role author
author
author
author
author
dc.subject.none.fl_str_mv 3D face reconstruction
Graph neural network
Baby model
topic 3D face reconstruction
Graph neural network
Baby model
description We present a 3D face reconstruction system that aims at recovering the 3D facial geometry of babies from uncalibrated photographs, BabyNet. Since the 3D facial geometry of babies differs substantially from that of adults, baby-specific facial reconstruction systems are needed. BabyNet consists of two stages: 1) a 3D graph convolutional autoencoder learns a latent space of the baby 3D facial shape; and 2) a 2D encoder that maps photographs to the 3D latent space based on representative features extracted using transfer learning. In this way, using the pre-trained 3D decoder, we can recover a 3D face from 2D images. We evaluate BabyNet and show that 1) methods based on adult datasets cannot model the 3D facial geometry of babies, which proves the need for a baby-specific method, and 2) BabyNet outperforms classical model-fitting methods even when a baby-specific 3D morphable model, such as BabyFM, is used.
publishDate 2023
dc.date.none.fl_str_mv 2023
2023
2023
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/10230/57511
http://dx.doi.org/10.1016/j.patcog.2023.109367
url http://hdl.handle.net/10230/57511
http://dx.doi.org/10.1016/j.patcog.2023.109367
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Pattern Recognition. 2023;139:109367.
info:eu-repo/grantAgreement/ES/2PE/PID2020-114083GB-I00
dc.rights.none.fl_str_mv http://creativecommons.org/licenses/by-nc-nd/4.0/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:Recercat. Dipósit de la Recerca de Catalunya
instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
instname_str Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
reponame_str Recercat. Dipósit de la Recerca de Catalunya
collection Recercat. Dipósit de la Recerca de Catalunya
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869420802729836544
score 15.812429

Similar Items