Perception and synthesis of sound-generating materials

The auditory perception of materials is a popular topic in the study of non-vocal sound-source perception. In this chapter, we review the empirical evidence on the mechanical and acoustical correlates of the perception of impacted stiff materials, and of the state of matter of sound-generating substances (solids, liquids, gases). As a whole, these studies suggest that recognition abilities are only highly accurate when differentiating between widely diverse materials (e.g. liquids vs. solids or plastics vs. metals) and that limitations in the auditory system, along with the possible internalization of biased statistics in the acoustical environment (e.g. clinking-glass sounds tend to be produced by small objects), might account for the less-than-perfect ability to differentiate between mechanically similar materials. This review is complemented by a summary of studies concerning the perception of deformable materials (fabrics and liquids) and the perceptual and motor-behaviour effects of auditory material-related information in audio-haptic contexts. The results of perceptual studies are the starting point for the development of interactive sound synthesis techniques for rendering the main auditory correlates of material properties, starting from physical models of the involved mechanical interactions. We review the recent literature dealing with contact sound synthesis in such fields as sonic interaction design and virtual reality. Special emphasis is given to softness/hardness correlates in impact sounds, associated with solid object resonances excited through impulsive contact. Synthesis methods for less studied sound-generating systems such as deformable objects (e.g. fabrics and liquids) and aggregate materials are also described.