Lightness/pitch and elevation/pitch crossmodal correspondences are low-level sensory effects

Abstract

We tested the sensory versus decisional origins of two established audiovisual crossmodal correspondences (CMCs; lightness/pitch and elevation/pitch), applying a signal discrimination paradigm to low-level stimulus features and controlling for attentional cueing. An audiovisual stimulus randomly varied along two visual dimensions (lightness: black/white; elevation: high/low) and one auditory dimension (pitch: high/low), and participants discriminated either only lightness, only elevation, or both lightness and elevation. The discrimination task and the stimulus duration varied between subjects. To investigate the influence of crossmodal congruency, we considered the effect of each CMC (lightness/pitch and elevation/pitch) on the sensitivity and criterion of each discrimination as a function of stimulus duration. There were three main findings. First, discrimination sensitivity was significantly higher for visual targets paired congruently (compared with incongruently) with tones while criterion was unaffected. Second, the sensitivity increase occurred for all stimulus durations, ruling out attentional cueing effects. Third, the sensitivity increase was feature specific such that only the CMC that related to the feature being discriminated influenced sensitivity (i.e. lightness congruency only influenced lightness discrimination and elevation congruency only influenced elevation discrimination in the single and dual task conditions). We suggest that these congruency effects reflect low-level sensory processes.

http://bit.ly/2WsVvAA