Many theories of multisensory integration focus on action-independent perception. This approach has limited ecological validity, because under everyday conditions we frequently act upon multisensory events through motor behavior. We present a novel methodology for the study of multisensory integration in action-directed perception, focusing on striking events. Observers repeatedly hit a virtual object with a target striking velocity, and are presented with various types of sensory information about the striking event: auditory and/or haptic and/or visual. For each of the experimental trials, they initially receive feedback on the tracking of the target striking velocity. In a second phase, feedback is eliminated. In a third phase, the sensory properties of the striking event are changed. We quantify the extent to which a variation in each of the sensory modalities disrupts performance in the tracking of the target striking velocity. Multisensory dominance hierarchies are inferred from the comparison of the patterns of performance disruption in unimodal and multisensory contexts.

Action-based multisensory integration in striking events / B.L. Giordano, S. Mcadams, P. Crosato, F. Avanzini, C. Casciato, S. Sinclair, M.M. Wanderley. - In: THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA. - ISSN 0001-4966. - 123:(2008), pp. 3178-3178.

Action-based multisensory integration in striking events

F. Avanzini;
2008

Abstract

Many theories of multisensory integration focus on action-independent perception. This approach has limited ecological validity, because under everyday conditions we frequently act upon multisensory events through motor behavior. We present a novel methodology for the study of multisensory integration in action-directed perception, focusing on striking events. Observers repeatedly hit a virtual object with a target striking velocity, and are presented with various types of sensory information about the striking event: auditory and/or haptic and/or visual. For each of the experimental trials, they initially receive feedback on the tracking of the target striking velocity. In a second phase, feedback is eliminated. In a third phase, the sensory properties of the striking event are changed. We quantify the extent to which a variation in each of the sensory modalities disrupts performance in the tracking of the target striking velocity. Multisensory dominance hierarchies are inferred from the comparison of the patterns of performance disruption in unimodal and multisensory contexts.
multimodality; auditory feedback; haptic feedback; sound synthesis
Settore INF/01 - Informatica
Settore ING-INF/05 - Sistemi di Elaborazione delle Informazioni
2008
Article (author)
File in questo prodotto:
File Dimensione Formato  
giordano_jasa08_acu08_abstract.pdf

accesso aperto

Tipologia: Pre-print (manoscritto inviato all'editore)
Dimensione 27.6 kB
Formato Adobe PDF
27.6 kB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/657870
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact