Climb-o-Vision: A Computer Vision Driven Sensory Substitution Device for Rock Climbing

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

4 Citations (SciVal)
94 Downloads (Pure)


The benefits of taking part in adventurous activities are many; particularly, for people with visual impairments. Sports such as rock climbing can improve feelings of skillfulness, autonomy, and confidence for people with low or no vision as they strive to overcome environmental and personal challenges. In this late-breaking work we present Climb-o-Vision, a novel sensory substitution software that utilizes YOLOv5 computer vision object-detection architecture, to aid navigation for rock climbers with visual impairments. Climb-o-Vision uses commercially available and cost-effective hardware to detect, track, and convert climbing hold spatial locations on to the surface of the tongue, via an electrotactile tongue interface. Preliminary testing of the device highlights the possibility of using sensory substitution as a sporting aid for people with visual impairments. Furthermore, it demonstrates the potential for adapting and improving current sensory substitution systems by employing computer vision techniques to filter useful task-specific information to users with visual impairments.
Original languageEnglish
Title of host publicationCHI 2022 - Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
Number of pages7
VolumeApril 2022
ISBN (Electronic)9781450391566
Publication statusPublished - 30 Apr 2022

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Bibliographical note

Funding Information:
We would like to thank Alex Shaw and The Climbing Academy for their continuing help in the development process of Climb-o-Vision, and for their commitment to improving climbing accessibility for people with visual impairments. We would also like to acknowledge the team at Sapien LLC, as their development of an afordable open access tongue interface, provides the ideal non-visual display for developing assistive technology. MJP’s and KP’s research is partly funded by the UKRI Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA 2.0; EP/T022523/1). MR’s research is funded by a SWDTP ESRC +3 PhD studentship.


  • Accessible sport
  • Computer vision
  • Rock climbing
  • Sensory substitution

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design
  • Software


Dive into the research topics of 'Climb-o-Vision: A Computer Vision Driven Sensory Substitution Device for Rock Climbing'. Together they form a unique fingerprint.

Cite this