Multidimensional labeling of gesture in communication: the M3D proposal

Communication is multimodal in that speakers use not only their voices, but also co-speech gestures to communicate. Recent insights suggest that gestural behavior has a strong association with prosodic structure and that a single gesture can communicate various semantic and pragmatic meanings. This...

Descripción completa

Detalles Bibliográficos
Autores: Rohrer, Patrick Louis, Tütüncübasi, Ulya, Florit-Pons, Júlia, Vilà-Giménez, Ingrid, Esteve-Gibert, Núria, Ren-Mitchell, Ada, Shattuck-Hufnagel, Stefanie, Prieto Vives, Pilar, 1965-
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2025
País:España
Institución:Universitat Pompeu Fabra
Repositorio:Repositorio Digital de la UPF
OAI Identifier:oai:dnet:rdupf_______::0058c81a7c77c9b807653b51bb33a3cb
Acceso en línea:https://hdl.handle.net/10230/73455
http://dx.doi.org/10.1007/s41701-025-00197-2
Access Level:acceso abierto
Palabra clave:Annotation
Gesture
Speech
Multimodality
id ES_ee53911fddd89129e4e43f94494b2cd7
oai_identifier_str oai:dnet:rdupf_______::0058c81a7c77c9b807653b51bb33a3cb
network_acronym_str ES
network_name_str España
repository_id_str
spelling Multidimensional labeling of gesture in communication: the M3D proposalRohrer, Patrick LouisTütüncübasi, UlyaFlorit-Pons, JúliaVilà-Giménez, IngridEsteve-Gibert, NúriaRen-Mitchell, AdaShattuck-Hufnagel, StefaniePrieto Vives, Pilar, 1965-AnnotationGestureSpeechMultimodalityCommunication is multimodal in that speakers use not only their voices, but also co-speech gestures to communicate. Recent insights suggest that gestural behavior has a strong association with prosodic structure and that a single gesture can communicate various semantic and pragmatic meanings. This highlights the importance of developing a comprehensive, flexible, and transparent approach to gesture annotation that accounts for multiple dimensions of gesture, including a gesture's form, prosodic properties, and semantic and pragmatic contributions. To address this need for an increasingly dimensionalized approach to multimodal data annotation, the main goal of this paper is to present and describe a novel labeling system for manual gestures. The MultiModal MultiDimensional (M3D) system consists of an open access package that has been developed in coordination with five different labs working on gesture and its interaction with speech. The package includes a set of reliable annotation guidelines, a validated training program, and two annotated audiovisual corpora that represent over 60 minutes of lecture-style speech.This work was partially funded by the Spanish Ministry of Science, Innovation and Universities (Grants numbers PID2021-123823NB-I00, PID2020-115385GA-I00, PGC2018-097007-B-I00), the Pla de Recerca de Catalunya from the Generalitat de Catalunya (Grant numbers 2017 SGR_971 and 2021 SGR 00922) and the GEHM (Gesture and Head Movements in Language) Research Network, funded by the Independent Research Fund Denmark (grant number 9055-00004B).Springer2026202620252026info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfapplication/pdfhttps://hdl.handle.net/10230/73455http://dx.doi.org/10.1007/s41701-025-00197-2reponame:Repositorio Digital de la UPFinstname:Universitat Pompeu FabraInglésCorpus Pragmatics. 2025 Dec;9(4):411-33info:eu-repo/grantAgreement/ES/2PE/PID2021-123823NB-I00info:eu-repo/grantAgreement/ES/2PE/PID2020-115385GA-I00info:eu-repo/grantAgreement/ES/2PE/PGC2018-097007-B-I00This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.http://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:dnet:rdupf_______::0058c81a7c77c9b807653b51bb33a3cb2026-06-12T07:21:37Z
dc.title.none.fl_str_mv Multidimensional labeling of gesture in communication: the M3D proposal
title Multidimensional labeling of gesture in communication: the M3D proposal
spellingShingle Multidimensional labeling of gesture in communication: the M3D proposal
Rohrer, Patrick Louis
Annotation
Gesture
Speech
Multimodality
title_short Multidimensional labeling of gesture in communication: the M3D proposal
title_full Multidimensional labeling of gesture in communication: the M3D proposal
title_fullStr Multidimensional labeling of gesture in communication: the M3D proposal
title_full_unstemmed Multidimensional labeling of gesture in communication: the M3D proposal
title_sort Multidimensional labeling of gesture in communication: the M3D proposal
dc.creator.none.fl_str_mv Rohrer, Patrick Louis
Tütüncübasi, Ulya
Florit-Pons, Júlia
Vilà-Giménez, Ingrid
Esteve-Gibert, Núria
Ren-Mitchell, Ada
Shattuck-Hufnagel, Stefanie
Prieto Vives, Pilar, 1965-
author Rohrer, Patrick Louis
author_facet Rohrer, Patrick Louis
Tütüncübasi, Ulya
Florit-Pons, Júlia
Vilà-Giménez, Ingrid
Esteve-Gibert, Núria
Ren-Mitchell, Ada
Shattuck-Hufnagel, Stefanie
Prieto Vives, Pilar, 1965-
author_role author
author2 Tütüncübasi, Ulya
Florit-Pons, Júlia
Vilà-Giménez, Ingrid
Esteve-Gibert, Núria
Ren-Mitchell, Ada
Shattuck-Hufnagel, Stefanie
Prieto Vives, Pilar, 1965-
author2_role author
author
author
author
author
author
author
dc.subject.none.fl_str_mv Annotation
Gesture
Speech
Multimodality
topic Annotation
Gesture
Speech
Multimodality
description Communication is multimodal in that speakers use not only their voices, but also co-speech gestures to communicate. Recent insights suggest that gestural behavior has a strong association with prosodic structure and that a single gesture can communicate various semantic and pragmatic meanings. This highlights the importance of developing a comprehensive, flexible, and transparent approach to gesture annotation that accounts for multiple dimensions of gesture, including a gesture's form, prosodic properties, and semantic and pragmatic contributions. To address this need for an increasingly dimensionalized approach to multimodal data annotation, the main goal of this paper is to present and describe a novel labeling system for manual gestures. The MultiModal MultiDimensional (M3D) system consists of an open access package that has been developed in coordination with five different labs working on gesture and its interaction with speech. The package includes a set of reliable annotation guidelines, a validated training program, and two annotated audiovisual corpora that represent over 60 minutes of lecture-style speech.
publishDate 2025
dc.date.none.fl_str_mv 2025
2026
2026
2026
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/10230/73455
http://dx.doi.org/10.1007/s41701-025-00197-2
url https://hdl.handle.net/10230/73455
http://dx.doi.org/10.1007/s41701-025-00197-2
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Corpus Pragmatics. 2025 Dec;9(4):411-33
info:eu-repo/grantAgreement/ES/2PE/PID2021-123823NB-I00
info:eu-repo/grantAgreement/ES/2PE/PID2020-115385GA-I00
info:eu-repo/grantAgreement/ES/2PE/PGC2018-097007-B-I00
dc.rights.none.fl_str_mv http://creativecommons.org/licenses/by/4.0/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Springer
publisher.none.fl_str_mv Springer
dc.source.none.fl_str_mv reponame:Repositorio Digital de la UPF
instname:Universitat Pompeu Fabra
instname_str Universitat Pompeu Fabra
reponame_str Repositorio Digital de la UPF
collection Repositorio Digital de la UPF
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869423635229310976
score 15,81155

Similares en LA Referencia