How well do you see what you hear?: The acuity of visual-to-auditory sensory substitution

Alastair Haigh, David J. Brown, Peter Meijer, Michael J. Proulx

Research output: Contribution to journalArticle

  • 28 Citations

Abstract

Sensory substitution devices (SSDs) aim to compensate for the loss of a sensory modality, typically vision, by converting information from the lost modality into stimuli in a remaining modality. "The vOICe" is a visual-to-auditory SSD which encodes images taken by a camera worn by the user into "soundscapes" such that experienced users can extract information about their surroundings. Here we investigated how much detail was resolvable during the early induction stages by testing the acuity of blindfolded sighted, naïve vOICe users. Initial performance was well above chance. Participants who took the test twice as a form of minimal training showed a marked improvement on the second test. Acuity was slightly but not significantly impaired when participants wore a camera and judged letter orientations "live". A positive correlation was found between participants' musical training and their acuity. The relationship between auditory expertise via musical training and the lack of a relationship with visual imagery, suggests that early use of a SSD draws primarily on the mechanisms of the sensory modality being used rather than the one being substituted. If vision is lost, audition represents the sensory channel of highest bandwidth of those remaining. The level of acuity found here, and the fact it was achieved with very little experience in sensory substitution by naïve users is promising.
LanguageEnglish
Article number330
JournalFrontiers in Psychology
Volume4
Early online date17 Jun 2013
DOIs
StatusPublished - 18 Jun 2013

Fingerprint

Visual Acuity
Equipment and Supplies
Imagery (Psychotherapy)
Hearing

Cite this

How well do you see what you hear? The acuity of visual-to-auditory sensory substitution. / Haigh, Alastair; Brown, David J.; Meijer, Peter; Proulx, Michael J.

In: Frontiers in Psychology, Vol. 4, 330, 18.06.2013.

Research output: Contribution to journalArticle

@article{1416f68e2c3647a18c82289c70505fec,
title = "How well do you see what you hear?: The acuity of visual-to-auditory sensory substitution",
abstract = "Sensory substitution devices (SSDs) aim to compensate for the loss of a sensory modality, typically vision, by converting information from the lost modality into stimuli in a remaining modality. {"}The vOICe{"} is a visual-to-auditory SSD which encodes images taken by a camera worn by the user into {"}soundscapes{"} such that experienced users can extract information about their surroundings. Here we investigated how much detail was resolvable during the early induction stages by testing the acuity of blindfolded sighted, na{\"i}ve vOICe users. Initial performance was well above chance. Participants who took the test twice as a form of minimal training showed a marked improvement on the second test. Acuity was slightly but not significantly impaired when participants wore a camera and judged letter orientations {"}live{"}. A positive correlation was found between participants' musical training and their acuity. The relationship between auditory expertise via musical training and the lack of a relationship with visual imagery, suggests that early use of a SSD draws primarily on the mechanisms of the sensory modality being used rather than the one being substituted. If vision is lost, audition represents the sensory channel of highest bandwidth of those remaining. The level of acuity found here, and the fact it was achieved with very little experience in sensory substitution by na{\"i}ve users is promising.",
author = "Alastair Haigh and Brown, {David J.} and Peter Meijer and Proulx, {Michael J.}",
year = "2013",
month = "6",
day = "18",
doi = "10.3389/fpsyg.2013.00330",
language = "English",
volume = "4",
journal = "Frontiers in Psychology: Movement Science and Sport Psychology",
issn = "1664-1078",
publisher = "Frontiers Media S.A.",

}

TY - JOUR

T1 - How well do you see what you hear?

T2 - Frontiers in Psychology: Movement Science and Sport Psychology

AU - Haigh,Alastair

AU - Brown,David J.

AU - Meijer,Peter

AU - Proulx,Michael J.

PY - 2013/6/18

Y1 - 2013/6/18

N2 - Sensory substitution devices (SSDs) aim to compensate for the loss of a sensory modality, typically vision, by converting information from the lost modality into stimuli in a remaining modality. "The vOICe" is a visual-to-auditory SSD which encodes images taken by a camera worn by the user into "soundscapes" such that experienced users can extract information about their surroundings. Here we investigated how much detail was resolvable during the early induction stages by testing the acuity of blindfolded sighted, naïve vOICe users. Initial performance was well above chance. Participants who took the test twice as a form of minimal training showed a marked improvement on the second test. Acuity was slightly but not significantly impaired when participants wore a camera and judged letter orientations "live". A positive correlation was found between participants' musical training and their acuity. The relationship between auditory expertise via musical training and the lack of a relationship with visual imagery, suggests that early use of a SSD draws primarily on the mechanisms of the sensory modality being used rather than the one being substituted. If vision is lost, audition represents the sensory channel of highest bandwidth of those remaining. The level of acuity found here, and the fact it was achieved with very little experience in sensory substitution by naïve users is promising.

AB - Sensory substitution devices (SSDs) aim to compensate for the loss of a sensory modality, typically vision, by converting information from the lost modality into stimuli in a remaining modality. "The vOICe" is a visual-to-auditory SSD which encodes images taken by a camera worn by the user into "soundscapes" such that experienced users can extract information about their surroundings. Here we investigated how much detail was resolvable during the early induction stages by testing the acuity of blindfolded sighted, naïve vOICe users. Initial performance was well above chance. Participants who took the test twice as a form of minimal training showed a marked improvement on the second test. Acuity was slightly but not significantly impaired when participants wore a camera and judged letter orientations "live". A positive correlation was found between participants' musical training and their acuity. The relationship between auditory expertise via musical training and the lack of a relationship with visual imagery, suggests that early use of a SSD draws primarily on the mechanisms of the sensory modality being used rather than the one being substituted. If vision is lost, audition represents the sensory channel of highest bandwidth of those remaining. The level of acuity found here, and the fact it was achieved with very little experience in sensory substitution by naïve users is promising.

UR - http://dx.doi.org/10.3389/fpsyg.2013.00330

U2 - 10.3389/fpsyg.2013.00330

DO - 10.3389/fpsyg.2013.00330

M3 - Article

VL - 4

JO - Frontiers in Psychology: Movement Science and Sport Psychology

JF - Frontiers in Psychology: Movement Science and Sport Psychology

SN - 1664-1078

M1 - 330

ER -