Real-time human action recognition using raw depth video-based recurrent neural networks

This work proposes and compare two different approaches for real-time human action recognition (HAR) from raw depth video sequences. Both proposals are based on the convolutional long short-term memory unit, namely ConvLSTM, with differences in the architecture and the long-term learning. The former...

Descripción completa

Detalles Bibliográficos
Autores: Sánchez Caballero, Adrián|||0000-0002-3395-7568, Fuentes Jiménez, David|||0000-0001-6424-4782, Losada Gutiérrez, Cristina|||0000-0001-9545-327X
Tipo de recurso: artículo
Fecha de publicación:2022
País:España
Institución:Universidad de Alcalá (UAH)
Repositorio:e_Buah Biblioteca Digital Universidad de Alcalá
Idioma:inglés
OAI Identifier:oai:ebuah.uah.es:10017/57907
Acceso en línea:http://hdl.handle.net/10017/57907
https://dx.doi.org/10.1007/s11042-022-14075-5
Access Level:acceso abierto
Palabra clave:ConvLSTM
Action recognition
Depth maps
Video-surveillance
Electrónica
Electronics
id ES_3ac268fada5fee0d994a652d5bc010ff
oai_identifier_str oai:ebuah.uah.es:10017/57907
network_acronym_str ES
network_name_str España
repository_id_str
spelling Real-time human action recognition using raw depth video-based recurrent neural networksSánchez Caballero, Adrián|||0000-0002-3395-7568Fuentes Jiménez, David|||0000-0001-6424-4782Losada Gutiérrez, Cristina|||0000-0001-9545-327XConvLSTMAction recognitionDepth mapsVideo-surveillanceElectrónicaElectronicsThis work proposes and compare two different approaches for real-time human action recognition (HAR) from raw depth video sequences. Both proposals are based on the convolutional long short-term memory unit, namely ConvLSTM, with differences in the architecture and the long-term learning. The former uses a video-length adaptive input data generator (stateless) whereas the latter explores the stateful ability of general recurrent neural networks but is applied in the particular case of HAR. This stateful property allows the model to accumulate discriminative patterns from previous frames without compromising computer memory. Furthermore, since the proposal uses only depth information, HAR is carried out preserving the privacy of people in the scene, since their identities can not be recognized. Both neural networks have been trained and tested using the large-scale NTU RGB+D dataset. Experimental results show that the proposed models achieve competitive recognition accuracies with lower computational cost compared with state-of-the-art methods and prove that, in the particular case of videos, the rarely-used stateful mode of recurrent neural networks significantly improves the accuracy obtained with the standard mode. The recognition accuracies obtained are 75.26% (CS) and 75.45% (CV) for the stateless model, with an average time consumption per video of 0.21 s, and 80.43% (CS) and 79.91%(CV) with 0.89 s for the stateful one.Agencia Estatal de InvestigaciónUniversidad de AlcaláSpringer20222022-05-28journal articlehttp://purl.org/coar/resource_type/c_6501NAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10017/57907https://dx.doi.org/10.1007/s11042-022-14075-5reponame:e_Buah Biblioteca Digital Universidad de Alcaláinstname:Universidad de Alcalá (UAH)InglésengAgencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2020-113118RB-C31 ANALISIS MULTISENSORIAL DE LA ACTIVIDAD HUMANA PARA EL DIAGNOSTICO Y LA DETECCION TEMPRANA DE LIMITACIONES FUNCIONALES-UAHUAH Not available CM-JIN-2021-015UAH Not available PIUAH21%2FIA-016open accesshttp://purl.org/coar/access_right/c_abf2Attribution-NonCommercial-NoDerivatives 4.0 Internationalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:ebuah.uah.es:10017/579072026-06-18T11:13:07Z
dc.title.none.fl_str_mv Real-time human action recognition using raw depth video-based recurrent neural networks
title Real-time human action recognition using raw depth video-based recurrent neural networks
spellingShingle Real-time human action recognition using raw depth video-based recurrent neural networks
Sánchez Caballero, Adrián|||0000-0002-3395-7568
ConvLSTM
Action recognition
Depth maps
Video-surveillance
Electrónica
Electronics
title_short Real-time human action recognition using raw depth video-based recurrent neural networks
title_full Real-time human action recognition using raw depth video-based recurrent neural networks
title_fullStr Real-time human action recognition using raw depth video-based recurrent neural networks
title_full_unstemmed Real-time human action recognition using raw depth video-based recurrent neural networks
title_sort Real-time human action recognition using raw depth video-based recurrent neural networks
dc.creator.none.fl_str_mv Sánchez Caballero, Adrián|||0000-0002-3395-7568
Fuentes Jiménez, David|||0000-0001-6424-4782
Losada Gutiérrez, Cristina|||0000-0001-9545-327X
author Sánchez Caballero, Adrián|||0000-0002-3395-7568
author_facet Sánchez Caballero, Adrián|||0000-0002-3395-7568
Fuentes Jiménez, David|||0000-0001-6424-4782
Losada Gutiérrez, Cristina|||0000-0001-9545-327X
author_role author
author2 Fuentes Jiménez, David|||0000-0001-6424-4782
Losada Gutiérrez, Cristina|||0000-0001-9545-327X
author2_role author
author
dc.subject.none.fl_str_mv ConvLSTM
Action recognition
Depth maps
Video-surveillance
Electrónica
Electronics
topic ConvLSTM
Action recognition
Depth maps
Video-surveillance
Electrónica
Electronics
description This work proposes and compare two different approaches for real-time human action recognition (HAR) from raw depth video sequences. Both proposals are based on the convolutional long short-term memory unit, namely ConvLSTM, with differences in the architecture and the long-term learning. The former uses a video-length adaptive input data generator (stateless) whereas the latter explores the stateful ability of general recurrent neural networks but is applied in the particular case of HAR. This stateful property allows the model to accumulate discriminative patterns from previous frames without compromising computer memory. Furthermore, since the proposal uses only depth information, HAR is carried out preserving the privacy of people in the scene, since their identities can not be recognized. Both neural networks have been trained and tested using the large-scale NTU RGB+D dataset. Experimental results show that the proposed models achieve competitive recognition accuracies with lower computational cost compared with state-of-the-art methods and prove that, in the particular case of videos, the rarely-used stateful mode of recurrent neural networks significantly improves the accuracy obtained with the standard mode. The recognition accuracies obtained are 75.26% (CS) and 75.45% (CV) for the stateless model, with an average time consumption per video of 0.21 s, and 80.43% (CS) and 79.91%(CV) with 0.89 s for the stateful one.
publishDate 2022
dc.date.none.fl_str_mv 2022
2022-05-28
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10017/57907
https://dx.doi.org/10.1007/s11042-022-14075-5
url http://hdl.handle.net/10017/57907
https://dx.doi.org/10.1007/s11042-022-14075-5
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Agencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2020-113118RB-C31 ANALISIS MULTISENSORIAL DE LA ACTIVIDAD HUMANA PARA EL DIAGNOSTICO Y LA DETECCION TEMPRANA DE LIMITACIONES FUNCIONALES-UAH
UAH Not available CM-JIN-2021-015
UAH Not available PIUAH21%2FIA-016
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Springer
publisher.none.fl_str_mv Springer
dc.source.none.fl_str_mv reponame:e_Buah Biblioteca Digital Universidad de Alcalá
instname:Universidad de Alcalá (UAH)
instname_str Universidad de Alcalá (UAH)
reponame_str e_Buah Biblioteca Digital Universidad de Alcalá
collection e_Buah Biblioteca Digital Universidad de Alcalá
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869406256457842688
score 15,300719

Similares en LA Referencia