Interacting with Real Objects in Virtual Worlds

Catherine Taylor

Research output: Chapter or section in a book/report/conference proceedingChapter or section

1 Citation (SciVal)


In a virtual reality (VR) experience, the manner in which a user in-
teracts with the virtual environment and computer generated objects in the scene greatly effects the feeling of immersion. Traditionally, VR systems use controllers as a means of facilitating interaction, with a sequence of button presses corresponding to a particular action. However, controllers do not accurately model the intuitive way to interact with a real-world object and offer limited tactile feedback. Alternatively, a physical object can be tracked and used to regulate the behaviour of a virtual object. In this chapter, we review a range of approaches which use the tracked behaviour of a physical object to control elements of the virtual environment. These virtual props
have the potential to be used as a more immersive alternative to the traditional controllers. We discuss how motion capture systems and external sensors can be used to track rigid and non-rigid objects, in order to drive the motion of computer generated 3D models. We then consider two neural network based tracking solutions and explain how these can be used for transporting real objects into virtual environments.
Original languageEnglish
Title of host publicationReal VR – Immersive Digital Reality: How to Import the Real World into Head-Mounted Immersive Displays (Lecture Notes in Computer Science)
EditorsM. Magnor, A. Sorkine-Hornung
Place of PublicationCham, Switzerland
Number of pages17
ISBN (Print)9783030418151
Publication statusPublished - 11 Mar 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)


Dive into the research topics of 'Interacting with Real Objects in Virtual Worlds'. Together they form a unique fingerprint.

Cite this