Epistemic intonation and epistemic gesture are mutually co-expressive: empirical results from two intonation-gesture matching tasks
While several studies have investigated the temporal relationship between co-speech gestures and prosodic structure, little is known about their potential interaction at the level of their encoding of pragmatic meaning. Here we report the results of two complementary intonation-gesture matching task...
| Autores: | , , |
|---|---|
| Tipo de recurso: | artículo |
| Estado: | Versión aceptada para publicación |
| Fecha de publicación: | 2019 |
| País: | España |
| Institución: | Universitat Pompeu Fabra |
| Repositorio: | Repositorio Digital de la UPF |
| OAI Identifier: | oai:repositori.upf.edu:10230/44288 |
| Acceso en línea: | http://hdl.handle.net/10230/44288 http://dx.doi.org/10.1016/j.pragma.2019.07.004 |
| Access Level: | acceso abierto |
| Palabra clave: | Gesture-speech integration Gesture-intonation co-dependencies Epistemic meaning Gesture Intonation Speaker commitment |
| id |
ES_024e298ee3cbac9fbbcbb211bae64341 |
|---|---|
| oai_identifier_str |
oai:repositori.upf.edu:10230/44288 |
| network_acronym_str |
ES |
| network_name_str |
España |
| repository_id_str |
|
| spelling |
Epistemic intonation and epistemic gesture are mutually co-expressive: empirical results from two intonation-gesture matching tasksBorràs Comes, Joan Manel, 1984-Kiagia, EvangeliaPrieto Vives, Pilar, 1965-Gesture-speech integrationGesture-intonation co-dependenciesEpistemic meaningGestureIntonationSpeaker commitmentWhile several studies have investigated the temporal relationship between co-speech gestures and prosodic structure, little is known about their potential interaction at the level of their encoding of pragmatic meaning. Here we report the results of two complementary intonation-gesture matching tasks which investigate the potential co-dependencies between intonation patterns related to epistemic commitment operators and their associated gestures in Catalan. In Experiment 1, participants were shown audio-muted videos in which a speaker performed gestures conveying epistemic information of certainty or uncertainty while uttering statements and questions. The subjects were then asked to produce a stipulated target word, the goal being to examine whether they would produce the word with a tune that was semantically consistent with the gestures they had seen. In Experiment 2, participants were primed by hearing intonation patterns conveying epistemic information (certainty-uncertainty) and were then asked to select one of two silent videos which seemed to best match the intonation they had heard. The results suggest converging positive effects in both matching tasks and suggest a close interrelation between the pragmatic representations of intonation and gesture that needs to be taken into account when investigating multimodal pragmatic encoding.This research has been funded by a research grant awarded by the Spanish Ministry of Science, Innovation and Universities (PGC2018-097007-B-I00, “Multimodal Language Learning (MLL): Prosodic and Gestural Integration in Pragmatic and Phonological Development”), and by a grant awarded by the Generalitat de Catalunya (2017 SGR 971) to the Prosodic Studies Group. Also awarded by the Generalitat de Catalunya, the first author acknowledges a research group grant to “LLETRA, Linguistic and Literary Education” (2017 SGR 1728).Elsevier202020202019info:eu-repo/semantics/articleinfo:eu-repo/semantics/acceptedVersionapplication/pdfapplication/pdfhttp://hdl.handle.net/10230/44288http://dx.doi.org/10.1016/j.pragma.2019.07.004reponame:Repositorio Digital de la UPFinstname:Universitat Pompeu FabraInglésJournal of Pragmatics. 2019 Sep;150:39-52info:eu-repo/grantAgreement/ES/2PE/PGC2018-097007-B-I00© Elsevier http://dx.doi.org/10.1016/j.pragma.2019.07.004info:eu-repo/semantics/openAccessoai:repositori.upf.edu:10230/442882026-06-12T07:21:37Z |
| dc.title.none.fl_str_mv |
Epistemic intonation and epistemic gesture are mutually co-expressive: empirical results from two intonation-gesture matching tasks |
| title |
Epistemic intonation and epistemic gesture are mutually co-expressive: empirical results from two intonation-gesture matching tasks |
| spellingShingle |
Epistemic intonation and epistemic gesture are mutually co-expressive: empirical results from two intonation-gesture matching tasks Borràs Comes, Joan Manel, 1984- Gesture-speech integration Gesture-intonation co-dependencies Epistemic meaning Gesture Intonation Speaker commitment |
| title_short |
Epistemic intonation and epistemic gesture are mutually co-expressive: empirical results from two intonation-gesture matching tasks |
| title_full |
Epistemic intonation and epistemic gesture are mutually co-expressive: empirical results from two intonation-gesture matching tasks |
| title_fullStr |
Epistemic intonation and epistemic gesture are mutually co-expressive: empirical results from two intonation-gesture matching tasks |
| title_full_unstemmed |
Epistemic intonation and epistemic gesture are mutually co-expressive: empirical results from two intonation-gesture matching tasks |
| title_sort |
Epistemic intonation and epistemic gesture are mutually co-expressive: empirical results from two intonation-gesture matching tasks |
| dc.creator.none.fl_str_mv |
Borràs Comes, Joan Manel, 1984- Kiagia, Evangelia Prieto Vives, Pilar, 1965- |
| author |
Borràs Comes, Joan Manel, 1984- |
| author_facet |
Borràs Comes, Joan Manel, 1984- Kiagia, Evangelia Prieto Vives, Pilar, 1965- |
| author_role |
author |
| author2 |
Kiagia, Evangelia Prieto Vives, Pilar, 1965- |
| author2_role |
author author |
| dc.subject.none.fl_str_mv |
Gesture-speech integration Gesture-intonation co-dependencies Epistemic meaning Gesture Intonation Speaker commitment |
| topic |
Gesture-speech integration Gesture-intonation co-dependencies Epistemic meaning Gesture Intonation Speaker commitment |
| description |
While several studies have investigated the temporal relationship between co-speech gestures and prosodic structure, little is known about their potential interaction at the level of their encoding of pragmatic meaning. Here we report the results of two complementary intonation-gesture matching tasks which investigate the potential co-dependencies between intonation patterns related to epistemic commitment operators and their associated gestures in Catalan. In Experiment 1, participants were shown audio-muted videos in which a speaker performed gestures conveying epistemic information of certainty or uncertainty while uttering statements and questions. The subjects were then asked to produce a stipulated target word, the goal being to examine whether they would produce the word with a tune that was semantically consistent with the gestures they had seen. In Experiment 2, participants were primed by hearing intonation patterns conveying epistemic information (certainty-uncertainty) and were then asked to select one of two silent videos which seemed to best match the intonation they had heard. The results suggest converging positive effects in both matching tasks and suggest a close interrelation between the pragmatic representations of intonation and gesture that needs to be taken into account when investigating multimodal pragmatic encoding. |
| publishDate |
2019 |
| dc.date.none.fl_str_mv |
2019 2020 2020 |
| dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/acceptedVersion |
| format |
article |
| status_str |
acceptedVersion |
| dc.identifier.none.fl_str_mv |
http://hdl.handle.net/10230/44288 http://dx.doi.org/10.1016/j.pragma.2019.07.004 |
| url |
http://hdl.handle.net/10230/44288 http://dx.doi.org/10.1016/j.pragma.2019.07.004 |
| dc.language.none.fl_str_mv |
Inglés |
| language_invalid_str_mv |
Inglés |
| dc.relation.none.fl_str_mv |
Journal of Pragmatics. 2019 Sep;150:39-52 info:eu-repo/grantAgreement/ES/2PE/PGC2018-097007-B-I00 |
| dc.rights.none.fl_str_mv |
© Elsevier http://dx.doi.org/10.1016/j.pragma.2019.07.004 info:eu-repo/semantics/openAccess |
| rights_invalid_str_mv |
© Elsevier http://dx.doi.org/10.1016/j.pragma.2019.07.004 |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
application/pdf application/pdf |
| dc.publisher.none.fl_str_mv |
Elsevier |
| publisher.none.fl_str_mv |
Elsevier |
| dc.source.none.fl_str_mv |
reponame:Repositorio Digital de la UPF instname:Universitat Pompeu Fabra |
| instname_str |
Universitat Pompeu Fabra |
| reponame_str |
Repositorio Digital de la UPF |
| collection |
Repositorio Digital de la UPF |
| repository.name.fl_str_mv |
|
| repository.mail.fl_str_mv |
|
| _version_ |
1869402645253324800 |
| score |
15,81155 |