End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth data

Despite recent advances in 3-D pose estimation of human hands, thanks to the advent of convolutional neural networks (CNNs) and depth cameras, this task is still far from being solved in uncontrolled setups. This is mainly due to the highly non-linear dynamics of fingers and self-occlusions, which m...

Descripción completa

Detalles Bibliográficos
Autores: Madadi, Meysam, Escalera Guerrero, Sergio, Baró i Solé, Xavier, Gonzàlez Sabaté, Jordi
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2021
País:España
Institución:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
Repositorio:Recercat. Dipósit de la Recerca de Catalunya
OAI Identifier:oai:recercat.cat:2445/190703
Acceso en línea:https://hdl.handle.net/2445/190703
Access Level:acceso abierto
Palabra clave:Visió per ordinador
Interacció persona-ordinador
Xarxes neuronals convolucionals
Computer vision
Human-computer interaction
Convolutional neural networks
id ES_3df98481d896d34e113fac4ea32ffddb
oai_identifier_str oai:recercat.cat:2445/190703
network_acronym_str ES
network_name_str España
repository_id_str
spelling End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth dataMadadi, MeysamEscalera Guerrero, SergioBaró i Solé, XavierGonzàlez Sabaté, JordiVisió per ordinadorInteracció persona-ordinadorXarxes neuronals convolucionalsComputer visionHuman-computer interactionConvolutional neural networksDespite recent advances in 3-D pose estimation of human hands, thanks to the advent of convolutional neural networks (CNNs) and depth cameras, this task is still far from being solved in uncontrolled setups. This is mainly due to the highly non-linear dynamics of fingers and self-occlusions, which make hand model training a challenging task. In this study, a novel hierarchical tree-like structured CNN is exploited, in which branches are trained to become specialised in predefined subsets of hand joints called local poses. Further, local pose features, extracted from hierarchical CNN branches, are fused to learn higher order dependencies among joints in the final pose by end-to-end training. Lastly, the loss function used is also defined to incorporate appearance and physical constraints about doable hand motions and deformations. Finally, a non-rigid data augmentation approach is introduced to increase the amount of training depth data. Experimental results suggest that feeding a tree-shaped CNN, specialised in local poses, into a fusion network for modelling joints' correlations and dependencies, helps to increase the precision of final estimations, showing competitive results on NYU, MSRA, Hands17 and SyntheticHand datasets.John Wiley & Sons2022202220212022info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersion17 p.application/pdfhttps://hdl.handle.net/2445/190703Articles publicats en revistes (Matemàtiques i Informàtica)reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)InglésReproducció del document publicat a: https://doi.org/10.1049/cvi2.12064IET Computer Vision, 2021, vol. 16, num. 1, p. 50-66https://doi.org/10.1049/cvi2.12064cc-by-nc (c) Madadi, Meysam et al., 2021http://creativecommons.org/licenses/by-nc/3.0/es/info:eu-repo/semantics/openAccessoai:recercat.cat:2445/1907032026-05-29T05:05:01Z
dc.title.none.fl_str_mv End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth data
title End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth data
spellingShingle End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth data
Madadi, Meysam
Visió per ordinador
Interacció persona-ordinador
Xarxes neuronals convolucionals
Computer vision
Human-computer interaction
Convolutional neural networks
title_short End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth data
title_full End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth data
title_fullStr End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth data
title_full_unstemmed End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth data
title_sort End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth data
dc.creator.none.fl_str_mv Madadi, Meysam
Escalera Guerrero, Sergio
Baró i Solé, Xavier
Gonzàlez Sabaté, Jordi
author Madadi, Meysam
author_facet Madadi, Meysam
Escalera Guerrero, Sergio
Baró i Solé, Xavier
Gonzàlez Sabaté, Jordi
author_role author
author2 Escalera Guerrero, Sergio
Baró i Solé, Xavier
Gonzàlez Sabaté, Jordi
author2_role author
author
author
dc.subject.none.fl_str_mv Visió per ordinador
Interacció persona-ordinador
Xarxes neuronals convolucionals
Computer vision
Human-computer interaction
Convolutional neural networks
topic Visió per ordinador
Interacció persona-ordinador
Xarxes neuronals convolucionals
Computer vision
Human-computer interaction
Convolutional neural networks
description Despite recent advances in 3-D pose estimation of human hands, thanks to the advent of convolutional neural networks (CNNs) and depth cameras, this task is still far from being solved in uncontrolled setups. This is mainly due to the highly non-linear dynamics of fingers and self-occlusions, which make hand model training a challenging task. In this study, a novel hierarchical tree-like structured CNN is exploited, in which branches are trained to become specialised in predefined subsets of hand joints called local poses. Further, local pose features, extracted from hierarchical CNN branches, are fused to learn higher order dependencies among joints in the final pose by end-to-end training. Lastly, the loss function used is also defined to incorporate appearance and physical constraints about doable hand motions and deformations. Finally, a non-rigid data augmentation approach is introduced to increase the amount of training depth data. Experimental results suggest that feeding a tree-shaped CNN, specialised in local poses, into a fusion network for modelling joints' correlations and dependencies, helps to increase the precision of final estimations, showing competitive results on NYU, MSRA, Hands17 and SyntheticHand datasets.
publishDate 2021
dc.date.none.fl_str_mv 2021
2022
2022
2022
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/2445/190703
url https://hdl.handle.net/2445/190703
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Reproducció del document publicat a: https://doi.org/10.1049/cvi2.12064
IET Computer Vision, 2021, vol. 16, num. 1, p. 50-66
https://doi.org/10.1049/cvi2.12064
dc.rights.none.fl_str_mv cc-by-nc (c) Madadi, Meysam et al., 2021
http://creativecommons.org/licenses/by-nc/3.0/es/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv cc-by-nc (c) Madadi, Meysam et al., 2021
http://creativecommons.org/licenses/by-nc/3.0/es/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv 17 p.
application/pdf
dc.publisher.none.fl_str_mv John Wiley & Sons
publisher.none.fl_str_mv John Wiley & Sons
dc.source.none.fl_str_mv Articles publicats en revistes (Matemàtiques i Informàtica)
reponame:Recercat. Dipósit de la Recerca de Catalunya
instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
instname_str Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
reponame_str Recercat. Dipósit de la Recerca de Catalunya
collection Recercat. Dipósit de la Recerca de Catalunya
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869406481360617472
score 15,811543

Similares en LA Referencia