Part-of-speech and prosody-based approaches for robot speech and gesture synchronization

Humanoid robots are already among us and they are beginning to assume more social and personal roles, like guiding and assisting people. Thus, they should interact in a human-friendly manner, using not only verbal cues but also synchronized non-verbal and para-verbal cues. However, available robots...

ver descrição completa

Detalhes bibliográficos
Autores: Pérez Mayos, Laura, Farrús, Mireia, Adell, Jordi
Formato: artículo
Estado:Versión aceptada para publicación
Fecha de publicación:2019
País:España
Recursos:Universitat Pompeu Fabra
Repositorio:Repositorio Digital de la UPF
OAI Identifier:oai:repositori.upf.edu:10230/42897
Acesso em linha:http://hdl.handle.net/10230/42897
http://dx.doi.org/10.1007/s10846-019-01100-3
Access Level:acceso abierto
Palavra-chave:Human-computer interaction
Multimodal interaction
Humanoid robots
Prosody
Speech
Gesture modelling
Arm gesture synthesis
Speech and gesture synchronization
Text-to-gesture
id ES_5d0396975e3decdc4e42638df103bb74
oai_identifier_str oai:repositori.upf.edu:10230/42897
network_acronym_str ES
network_name_str España
repository_id_str
spelling Part-of-speech and prosody-based approaches for robot speech and gesture synchronizationPérez Mayos, LauraFarrús, MireiaAdell, JordiHuman-computer interactionMultimodal interactionHumanoid robotsProsodySpeechGesture modellingArm gesture synthesisSpeech and gesture synchronizationText-to-gestureHumanoid robots are already among us and they are beginning to assume more social and personal roles, like guiding and assisting people. Thus, they should interact in a human-friendly manner, using not only verbal cues but also synchronized non-verbal and para-verbal cues. However, available robots are not able to communicate in this multimodal way, being just able to perform predefined gesture sequences, hand- crafted to accompany specific utterances. In the current paper, we propose a model based on three different approaches to extend humanoid robots communication behaviour with upper body gestures synchronized with the speech for novel utterances, exploiting part-of-speech grammatical information, prosody cues, and a combination of both. User studies confirm that our methods are able to produce natural, appropriate and good timed gesture sequences synchronized with speech, using both beat and emblematic gestures.The second author has been funded by the Agencia Estatal de Investigación (AEI), Ministerio de Ciencia, Innovación y Universidades and the Fondo Social Europeo (FSE) under grant RYC-2015-17239 (AEI/FSE, UE). The authors would like to thank the anonymous reviewers that helped to improve this paper through their valuable comments.Springer20192019info:eu-repo/semantics/articleinfo:eu-repo/semantics/acceptedVersionapplication/pdfapplication/pdfhttp://hdl.handle.net/10230/42897http://dx.doi.org/10.1007/s10846-019-01100-3reponame:Repositorio Digital de la UPFinstname:Universitat Pompeu FabraInglésJournal of intelligent & robotic systems. 2019 Nov 16:1-11© Springer The final publication is available at Springer via https://doi.org/10.1007/s10846-019-01100-3info:eu-repo/semantics/openAccessoai:repositori.upf.edu:10230/428972026-06-12T07:21:37Z
dc.title.none.fl_str_mv Part-of-speech and prosody-based approaches for robot speech and gesture synchronization
title Part-of-speech and prosody-based approaches for robot speech and gesture synchronization
spellingShingle Part-of-speech and prosody-based approaches for robot speech and gesture synchronization
Pérez Mayos, Laura
Human-computer interaction
Multimodal interaction
Humanoid robots
Prosody
Speech
Gesture modelling
Arm gesture synthesis
Speech and gesture synchronization
Text-to-gesture
title_short Part-of-speech and prosody-based approaches for robot speech and gesture synchronization
title_full Part-of-speech and prosody-based approaches for robot speech and gesture synchronization
title_fullStr Part-of-speech and prosody-based approaches for robot speech and gesture synchronization
title_full_unstemmed Part-of-speech and prosody-based approaches for robot speech and gesture synchronization
title_sort Part-of-speech and prosody-based approaches for robot speech and gesture synchronization
dc.creator.none.fl_str_mv Pérez Mayos, Laura
Farrús, Mireia
Adell, Jordi
author Pérez Mayos, Laura
author_facet Pérez Mayos, Laura
Farrús, Mireia
Adell, Jordi
author_role author
author2 Farrús, Mireia
Adell, Jordi
author2_role author
author
dc.subject.none.fl_str_mv Human-computer interaction
Multimodal interaction
Humanoid robots
Prosody
Speech
Gesture modelling
Arm gesture synthesis
Speech and gesture synchronization
Text-to-gesture
topic Human-computer interaction
Multimodal interaction
Humanoid robots
Prosody
Speech
Gesture modelling
Arm gesture synthesis
Speech and gesture synchronization
Text-to-gesture
description Humanoid robots are already among us and they are beginning to assume more social and personal roles, like guiding and assisting people. Thus, they should interact in a human-friendly manner, using not only verbal cues but also synchronized non-verbal and para-verbal cues. However, available robots are not able to communicate in this multimodal way, being just able to perform predefined gesture sequences, hand- crafted to accompany specific utterances. In the current paper, we propose a model based on three different approaches to extend humanoid robots communication behaviour with upper body gestures synchronized with the speech for novel utterances, exploiting part-of-speech grammatical information, prosody cues, and a combination of both. User studies confirm that our methods are able to produce natural, appropriate and good timed gesture sequences synchronized with speech, using both beat and emblematic gestures.
publishDate 2019
dc.date.none.fl_str_mv 2019
2019
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/acceptedVersion
format article
status_str acceptedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/10230/42897
http://dx.doi.org/10.1007/s10846-019-01100-3
url http://hdl.handle.net/10230/42897
http://dx.doi.org/10.1007/s10846-019-01100-3
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Journal of intelligent & robotic systems. 2019 Nov 16:1-11
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Springer
publisher.none.fl_str_mv Springer
dc.source.none.fl_str_mv reponame:Repositorio Digital de la UPF
instname:Universitat Pompeu Fabra
instname_str Universitat Pompeu Fabra
reponame_str Repositorio Digital de la UPF
collection Repositorio Digital de la UPF
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869408971311284224
score 15.81155

Registos relacionados