Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli?

David J Brown, Andrew J R Simpson, Michael J Proulx

Research output: Contribution to journalArticle

3 Citations (Scopus)
54 Downloads (Pure)

Abstract

A critical task for the brain is the sensory representation and identification of perceptual objects in the world. When the visual sense is impaired, hearing and touch must take primary roles and in recent times compensatory techniques have been developed that employ the tactile or auditory system as a substitute for the visual system. Visual-to-auditory sonifications provide a complex, feature-based auditory representation that must be decoded and integrated into an object-based representation by the listener. However, we don't yet know what role the auditory system plays in the object integration stage and whether the principles of auditory scene analysis apply. Here we used coarse sonified images in a two-tone discrimination task to test whether auditory feature-based representations of visual objects would be confounded when their features conflicted with the principles of auditory consonance. We found that listeners (N = 36) performed worse in an object recognition task when the auditory feature-based representation was harmonically consonant. We also found that this conflict was not negated with the provision of congruent audio-visual information. The findings suggest that early auditory processes of harmonic grouping dominate the object formation process and that the complexity of the signal, and additional sensory information have limited effect on this.

Original languageEnglish
Article number1522
JournalFrontiers in Psychology
Volume6
Early online date13 Oct 2015
DOIs
Publication statusPublished - 2015

Fingerprint

Touch
Hearing
Brain
Conflict (Psychology)

Cite this

@article{557202b2255442b58e7e8589e21d8763,
title = "Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli?",
abstract = "A critical task for the brain is the sensory representation and identification of perceptual objects in the world. When the visual sense is impaired, hearing and touch must take primary roles and in recent times compensatory techniques have been developed that employ the tactile or auditory system as a substitute for the visual system. Visual-to-auditory sonifications provide a complex, feature-based auditory representation that must be decoded and integrated into an object-based representation by the listener. However, we don't yet know what role the auditory system plays in the object integration stage and whether the principles of auditory scene analysis apply. Here we used coarse sonified images in a two-tone discrimination task to test whether auditory feature-based representations of visual objects would be confounded when their features conflicted with the principles of auditory consonance. We found that listeners (N = 36) performed worse in an object recognition task when the auditory feature-based representation was harmonically consonant. We also found that this conflict was not negated with the provision of congruent audio-visual information. The findings suggest that early auditory processes of harmonic grouping dominate the object formation process and that the complexity of the signal, and additional sensory information have limited effect on this.",
author = "Brown, {David J} and Simpson, {Andrew J R} and Proulx, {Michael J}",
year = "2015",
doi = "10.3389/fpsyg.2015.01522",
language = "English",
volume = "6",
journal = "Frontiers in Psychology: Movement Science and Sport Psychology",
issn = "1664-1078",
publisher = "Frontiers Media S.A.",

}

TY - JOUR

T1 - Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli?

AU - Brown, David J

AU - Simpson, Andrew J R

AU - Proulx, Michael J

PY - 2015

Y1 - 2015

N2 - A critical task for the brain is the sensory representation and identification of perceptual objects in the world. When the visual sense is impaired, hearing and touch must take primary roles and in recent times compensatory techniques have been developed that employ the tactile or auditory system as a substitute for the visual system. Visual-to-auditory sonifications provide a complex, feature-based auditory representation that must be decoded and integrated into an object-based representation by the listener. However, we don't yet know what role the auditory system plays in the object integration stage and whether the principles of auditory scene analysis apply. Here we used coarse sonified images in a two-tone discrimination task to test whether auditory feature-based representations of visual objects would be confounded when their features conflicted with the principles of auditory consonance. We found that listeners (N = 36) performed worse in an object recognition task when the auditory feature-based representation was harmonically consonant. We also found that this conflict was not negated with the provision of congruent audio-visual information. The findings suggest that early auditory processes of harmonic grouping dominate the object formation process and that the complexity of the signal, and additional sensory information have limited effect on this.

AB - A critical task for the brain is the sensory representation and identification of perceptual objects in the world. When the visual sense is impaired, hearing and touch must take primary roles and in recent times compensatory techniques have been developed that employ the tactile or auditory system as a substitute for the visual system. Visual-to-auditory sonifications provide a complex, feature-based auditory representation that must be decoded and integrated into an object-based representation by the listener. However, we don't yet know what role the auditory system plays in the object integration stage and whether the principles of auditory scene analysis apply. Here we used coarse sonified images in a two-tone discrimination task to test whether auditory feature-based representations of visual objects would be confounded when their features conflicted with the principles of auditory consonance. We found that listeners (N = 36) performed worse in an object recognition task when the auditory feature-based representation was harmonically consonant. We also found that this conflict was not negated with the provision of congruent audio-visual information. The findings suggest that early auditory processes of harmonic grouping dominate the object formation process and that the complexity of the signal, and additional sensory information have limited effect on this.

UR - http://dx.doi.org/10.3389/fpsyg.2015.01522

U2 - 10.3389/fpsyg.2015.01522

DO - 10.3389/fpsyg.2015.01522

M3 - Article

VL - 6

JO - Frontiers in Psychology: Movement Science and Sport Psychology

JF - Frontiers in Psychology: Movement Science and Sport Psychology

SN - 1664-1078

M1 - 1522

ER -