VRProp-Net:Real-time Interaction with Virtual Props

Catherine Taylor, Robin McNicholas, Darren Cosker

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

3 Citations (SciVal)
142 Downloads (Pure)


Virtual and Augmented Reality (VR and AR) are two fast growing mediums, not only in the entertainment industry but also in health, education and engineering. A good VR or AR application seamlessly merges the real and virtual world, making the user feels fully immersed. Traditionally, a computer-generated object can be interacted with using controllers or hand gestures [HTC 2019; Microsoft 2019; Oculus 2019]. However, these motions can feel unnatural and do not accurately represent the motion of interacting with a real object. On the other hand, a physical object can be used to control the motion of a virtual object. At present, this can be done by tracking purely rigid motion using an external sensor [HTC 2019]. Alternatively, a sparse number of markers can be tracked, for example using a motion capture system, and the positions of these used to drive the motion of an underlying non-rigid model. However, this approach is sensitive to changes in marker position and occlusions and often involves costly non-standard hardware [Vicon 2019]. In addition, these approaches often require a virtual model to be manually sculpted and rigged which can be a time consuming process. Neural networks have been shown to be successful tools in computer vision, with several key methods using networks for tracking rigid and non-rigid motion in RGB images [Andrychowicz et al. 2018; Kanazawa et al. 2018; Pumarola et al. 2018]. While these methods show potential, they are limited to using multiple RGB cameras or large, costly amounts of labelled training data.
Original languageEnglish
Title of host publicationACM SIGGRAPH 2019 Posters, SIGGRAPH 2019
Subtitle of host publicationSIGGRAPH '19
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450363143
Publication statusPublished - 28 Jul 2019

Publication series

NameACM SIGGRAPH 2019 Posters, SIGGRAPH 2019


  • Neural Networks
  • Non-rigid Object Tracking
  • VR Props

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction


Dive into the research topics of 'VRProp-Net:Real-time Interaction with Virtual Props'. Together they form a unique fingerprint.

Cite this