Force and Velocity Prediction in Human-Robot Collaborative Transportation Tasks through Video Retentive Networks

In this article, we propose a generalization of a Deep Learning State-of-the-Art architecture such as Retentive Networks so that it can accept video sequences as input. With this generalization, we design a force/velocity predictor applied to the medium-distance Human-Robot collaborative object tran...

Descripción completa

Detalles Bibliográficos
Autores: Domínguez Vidal, José Enrique, Sanfeliu, Alberto
Tipo de recurso: artículo
Estado:Versión aceptada para publicación
Fecha de publicación:2024
País:España
Institución:Consejo Superior de Investigaciones Científicas (CSIC)
Repositorio:DIGITAL.CSIC. Repositorio Institucional del CSIC
OAI Identifier:oai:digital.csic.es:10261/387907
Acceso en línea:http://hdl.handle.net/10261/387907
https://api.elsevier.com/content/abstract/scopus_id/85216497791
Access Level:acceso abierto
Palabra clave:Force Prediction
Human-in-the-Loop
Object Transportation
Physical Human-Robot Interaction
id ES_67ee40cc229dd7a728ba1bef5ce1998a
oai_identifier_str oai:digital.csic.es:10261/387907
network_acronym_str ES
network_name_str España
repository_id_str
spelling Force and Velocity Prediction in Human-Robot Collaborative Transportation Tasks through Video Retentive NetworksDomínguez Vidal, José EnriqueSanfeliu, AlbertoForce PredictionHuman-in-the-LoopObject TransportationPhysical Human-Robot InteractionIn this article, we propose a generalization of a Deep Learning State-of-the-Art architecture such as Retentive Networks so that it can accept video sequences as input. With this generalization, we design a force/velocity predictor applied to the medium-distance Human-Robot collaborative object transportation task. We achieve better results than with our previous predictor by reaching success rates in testset of up to 93.7% in predicting the force to be exerted by the human and up to 96.5% in the velocity of the human-robot pair during the next 1 s, and up to 91.0% and 95.0% respectively in real experiments. This new architecture also manages to improve inference times by up to 32.8% with different graphics cards. Finally, an ablation test allows us to detect that one of the input variables used so far, such as the position of the task goal, could be discarded allowing this goal to be chosen dynamically by the human instead of being pre-set.Work supported under the European project CANOPIES (H2020- ICT-2020-2-101016906) and by JST Moonshot R & D Grant Number: JPMJMS2011-85. The first author acknowledges Spanish FPU grant with ref. FPU19/06582.Peer reviewedInstitute of Electrical and Electronics EngineersEuropean CommissionDomínguez Vidal, José Enrique [0000-0002-0397-9248]Sanfeliu, Alberto [0000-0003-3868-9678]Consejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]202520252024info:eu-repo/semantics/articlehttp://purl.org/coar/resource_type/c_6501Postprintinfo:eu-repo/semantics/acceptedVersionapplication/pdfhttp://hdl.handle.net/10261/387907https://api.elsevier.com/content/abstract/scopus_id/85216497791reponame:DIGITAL.CSIC. Repositorio Institucional del CSICinstname:Consejo Superior de Investigaciones Científicas (CSIC)Inglés#PLACEHOLDER_PARENT_METADATA_VALUE#info:eu-repo/grantAgreement/EC/H2020/101016906https://doi.org/10.1109/IROS58592.2024.10801981Síinfo:eu-repo/semantics/openAccessoai:digital.csic.es:10261/3879072026-05-22T06:33:51Z
dc.title.none.fl_str_mv Force and Velocity Prediction in Human-Robot Collaborative Transportation Tasks through Video Retentive Networks
title Force and Velocity Prediction in Human-Robot Collaborative Transportation Tasks through Video Retentive Networks
spellingShingle Force and Velocity Prediction in Human-Robot Collaborative Transportation Tasks through Video Retentive Networks
Domínguez Vidal, José Enrique
Force Prediction
Human-in-the-Loop
Object Transportation
Physical Human-Robot Interaction
title_short Force and Velocity Prediction in Human-Robot Collaborative Transportation Tasks through Video Retentive Networks
title_full Force and Velocity Prediction in Human-Robot Collaborative Transportation Tasks through Video Retentive Networks
title_fullStr Force and Velocity Prediction in Human-Robot Collaborative Transportation Tasks through Video Retentive Networks
title_full_unstemmed Force and Velocity Prediction in Human-Robot Collaborative Transportation Tasks through Video Retentive Networks
title_sort Force and Velocity Prediction in Human-Robot Collaborative Transportation Tasks through Video Retentive Networks
dc.creator.none.fl_str_mv Domínguez Vidal, José Enrique
Sanfeliu, Alberto
author Domínguez Vidal, José Enrique
author_facet Domínguez Vidal, José Enrique
Sanfeliu, Alberto
author_role author
author2 Sanfeliu, Alberto
author2_role author
dc.contributor.none.fl_str_mv European Commission
Domínguez Vidal, José Enrique [0000-0002-0397-9248]
Sanfeliu, Alberto [0000-0003-3868-9678]
Consejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]
dc.subject.none.fl_str_mv Force Prediction
Human-in-the-Loop
Object Transportation
Physical Human-Robot Interaction
topic Force Prediction
Human-in-the-Loop
Object Transportation
Physical Human-Robot Interaction
description In this article, we propose a generalization of a Deep Learning State-of-the-Art architecture such as Retentive Networks so that it can accept video sequences as input. With this generalization, we design a force/velocity predictor applied to the medium-distance Human-Robot collaborative object transportation task. We achieve better results than with our previous predictor by reaching success rates in testset of up to 93.7% in predicting the force to be exerted by the human and up to 96.5% in the velocity of the human-robot pair during the next 1 s, and up to 91.0% and 95.0% respectively in real experiments. This new architecture also manages to improve inference times by up to 32.8% with different graphics cards. Finally, an ablation test allows us to detect that one of the input variables used so far, such as the position of the task goal, could be discarded allowing this goal to be chosen dynamically by the human instead of being pre-set.
publishDate 2024
dc.date.none.fl_str_mv 2024
2025
2025
dc.type.none.fl_str_mv info:eu-repo/semantics/article
http://purl.org/coar/resource_type/c_6501
Postprint
info:eu-repo/semantics/acceptedVersion
format article
status_str acceptedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/10261/387907
https://api.elsevier.com/content/abstract/scopus_id/85216497791
url http://hdl.handle.net/10261/387907
https://api.elsevier.com/content/abstract/scopus_id/85216497791
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv #PLACEHOLDER_PARENT_METADATA_VALUE#
info:eu-repo/grantAgreement/EC/H2020/101016906
https://doi.org/10.1109/IROS58592.2024.10801981
Sí
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Institute of Electrical and Electronics Engineers
publisher.none.fl_str_mv Institute of Electrical and Electronics Engineers
dc.source.none.fl_str_mv reponame:DIGITAL.CSIC. Repositorio Institucional del CSIC
instname:Consejo Superior de Investigaciones Científicas (CSIC)
instname_str Consejo Superior de Investigaciones Científicas (CSIC)
reponame_str DIGITAL.CSIC. Repositorio Institucional del CSIC
collection DIGITAL.CSIC. Repositorio Institucional del CSIC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869409925069799424
score 15,812429

Similares en LA Referencia