Climb-o-Vision: A Computer Vision Driven Sensory Substitution Device for Rock Climbing

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

20 Downloads (Pure)


The benefits of taking part in adventurous activities are many; particularly, for people with visual impairments. Sports such as rock climbing can improve feelings of skillfulness, autonomy, and confidence for people with low or no vision as they strive to overcome environmental and personal challenges. In this late-breaking work we present Climb-o-Vision, a novel sensory substitution software that utilizes YOLOv5 computer vision object-detection architecture, to aid navigation for rock climbers with visual impairments. Climb-o-Vision uses commercially available and cost-effective hardware to detect, track, and convert climbing hold spatial locations on to the surface of the tongue, via an electrotactile tongue interface. Preliminary testing of the device highlights the possibility of using sensory substitution as a sporting aid for people with visual impairments. Furthermore, it demonstrates the potential for adapting and improving current sensory substitution systems by employing computer vision techniques to filter useful task-specific information to users with visual impairments.
Original languageEnglish
Title of host publicationCHI 2022 - Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
Number of pages7
VolumeApril 2022
ISBN (Electronic)9781450391566
Publication statusPublished - 30 Apr 2022

Publication series

NameConference on Human Factors in Computing Systems - Proceedings


  • Accessible sport
  • Computer vision
  • Rock climbing
  • Sensory substitution

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design
  • Software


Dive into the research topics of 'Climb-o-Vision: A Computer Vision Driven Sensory Substitution Device for Rock Climbing'. Together they form a unique fingerprint.

Cite this