The Role of cross-modal semantic interactions in real-world visuo-spatial attention
In our everyday life we must effectively orient attention to relevant objects and events in multisensory environments. The impact of cross-modal links for attention orienting to spatial and temporal cues has been widely described. However, real-life scenarios provide a rich web of semantic informati...
| Author: | |
|---|---|
| Format: | doctoral thesis |
| Status: | Published version |
| Publication Date: | 2020 |
| Country: | España |
| Institution: | CBUC, CESCA |
| Repository: | TDR. Tesis Doctorales en Red |
| OAI Identifier: | oai:www.tdx.cat:10803/668665 |
| Online Access: | http://hdl.handle.net/10803/668665 |
| Access Level: | Open access |
| Keyword: | Attention Semantics Multisensory Crossmodal interactions Visual search Eyetracking Audiovisual processing Natural scenes Real life Spatial orienting 159.9 |
| Summary: | In our everyday life we must effectively orient attention to relevant objects and events in multisensory environments. The impact of cross-modal links for attention orienting to spatial and temporal cues has been widely described. However, real-life scenarios provide a rich web of semantic information through the different sensory modalities. Despite some previous studies have revealed an impact of crossmodal sematic correspondences, the results are mixed with regard to the conditions in which audiovisual semantic congruence can influence attention orienting. Furthermore, the vast majority of the research on crossmodal semantics used simple, stereotyped displays that are far from achieving ecological validity. The present thesis attempts to close this gap by addressing the role of identity-based crossmodal relationships on attention orienting in scenarios closer to real-world conditions. To this end, the experiments presented here attempt to extrapolate and generalize previous findings in more realistic environments by using naturalistic and dynamic stimuli, and address the theoretical questions of task relevance and perceptual load. The outcome of the three empirical studies in this thesis lead to several conclusions. First, that the effect of audio-visual semantic congruence on attention is not strictly automatic. Instead, they suggest that some top-down processing is necessary for audio-visual semantic congruence to trigger spatial orienting. The second conclusion to emerge is that crossmodal semantic congruence can guide attention under goal-directed conditions in visual search, and also under free observation in complex and dynamic scenes. Third, that perceptual load is a limiting factor for these interactions. These findings extend previous knowledge on object-based crossmodal interactions with simple stimuli and clarify how audio-visual semantically congruent relationships play out in realistic scenarios. |
|---|