Towards An Egocentric Framework for Rigid and Articulated Object Tracking in Virtual Reality

Catherine Taylor, Robin McNicholas, Darren Cosker

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

4 Citations (SciVal)

Abstract

To create immersive Virtual Reality (VR) applications and training environments, an appropriate method for allowing participants to interact with the virtual environments and objects in that scene must be considered. An approach which offers increased immersion and accurately models real-world behaviour would be advantageous - particularly within the areas of health, entertainment and engineering. Traditional consumer VR methods for facilitating interaction - e.g. controllers - are restricted by lack of tactile feed back and do not accurately represent real-world interactions with physical objects in terms of shape, limiting immersion. Ideally, physical objects would be transported into the virtual world and used as a means of interacting with the environment, via a robust tracking algorithm or motion capture system. However, achieving this in a real time markerless manner for a range of object types remains an open challenge. Moreover, costly motion capture systems or tracking algorithms which require multiple cameras are not practical for everyday VR use.Given the recent advancements in object tracking and mesh reconstruction using neural networks, we present a novel neural network, VRProp-Net+, which predicts rigid and articulated model parameters of known everyday objects in unconstrained environments from RGB images at interactive frame rates. VRProp-Net + utilises a novel synthetic training methodology and so does not require a time consuming capture or manual labelling procedure seen in many prominent neural network tracking or mesh reconstruction approaches. We present our network as part of an egocentric tracking framework, that allows prediction of object pose and shape given a moving camera. This scenario is helpful for practical VR experiences where the camera may be mounted on the VR Head Mounted Display (HMD) itself. This creates a dynamic capture volume, allowing a user to move around and interact with a virtual world without the need for multiple fixed cameras.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE Conference on Virtual Reality and 3D User Interfaces, VRW 2020
PublisherIEEE
Pages354-359
Number of pages6
ISBN (Electronic)9781728165325
ISBN (Print)9781728165325
DOIs
Publication statusPublished - 26 Mar 2020
Event2020 IEEE Conference on Virtual Reality and 3D User Interfaces - online, Atlanta, USA United States
Duration: 22 Mar 202026 Mar 2020
http://ieeevr.org/2020/

Publication series

NameProceedings - 2020 IEEE Conference on Virtual Reality and 3D User Interfaces, VRW 2020

Conference

Conference2020 IEEE Conference on Virtual Reality and 3D User Interfaces
Abbreviated titleIEEE VR
Country/TerritoryUSA United States
CityAtlanta
Period22/03/2026/03/20
Internet address

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

Keywords

  • Computer vision
  • Computing methodologies
  • Modelling and simulation Computing methodologies
  • Neural networks Computing methodologies

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Human-Computer Interaction
  • Media Technology

Fingerprint

Dive into the research topics of 'Towards An Egocentric Framework for Rigid and Articulated Object Tracking in Virtual Reality'. Together they form a unique fingerprint.

Cite this