Projects per year
Abstract
To create immersive Virtual Reality (VR) applications and training environments, an appropriate method for allowing participants to interact with the virtual environments and objects in that scene must be considered. An approach which offers increased immersion and accurately models real-world behaviour would be advantageous - particularly within the areas of health, entertainment and engineering. Traditional consumer VR methods for facilitating interaction - e.g. controllers - are restricted by lack of tactile feed back and do not accurately represent real-world interactions with physical objects in terms of shape, limiting immersion. Ideally, physical objects would be transported into the virtual world and used as a means of interacting with the environment, via a robust tracking algorithm or motion capture system. However, achieving this in a real time markerless manner for a range of object types remains an open challenge. Moreover, costly motion capture systems or tracking algorithms which require multiple cameras are not practical for everyday VR use.Given the recent advancements in object tracking and mesh reconstruction using neural networks, we present a novel neural network, VRProp-Net+, which predicts rigid and articulated model parameters of known everyday objects in unconstrained environments from RGB images at interactive frame rates. VRProp-Net + utilises a novel synthetic training methodology and so does not require a time consuming capture or manual labelling procedure seen in many prominent neural network tracking or mesh reconstruction approaches. We present our network as part of an egocentric tracking framework, that allows prediction of object pose and shape given a moving camera. This scenario is helpful for practical VR experiences where the camera may be mounted on the VR Head Mounted Display (HMD) itself. This creates a dynamic capture volume, allowing a user to move around and interact with a virtual world without the need for multiple fixed cameras.
Original language | English |
---|---|
Title of host publication | Proceedings - 2020 IEEE Conference on Virtual Reality and 3D User Interfaces, VRW 2020 |
Publisher | IEEE |
Pages | 354-359 |
Number of pages | 6 |
ISBN (Electronic) | 9781728165325 |
ISBN (Print) | 9781728165325 |
DOIs | |
Publication status | Published - 26 Mar 2020 |
Event | 2020 IEEE Conference on Virtual Reality and 3D User Interfaces - online, Atlanta, USA United States Duration: 22 Mar 2020 → 26 Mar 2020 http://ieeevr.org/2020/ |
Publication series
Name | Proceedings - 2020 IEEE Conference on Virtual Reality and 3D User Interfaces, VRW 2020 |
---|
Conference
Conference | 2020 IEEE Conference on Virtual Reality and 3D User Interfaces |
---|---|
Abbreviated title | IEEE VR |
Country/Territory | USA United States |
City | Atlanta |
Period | 22/03/20 → 26/03/20 |
Internet address |
Bibliographical note
Publisher Copyright:© 2020 IEEE.
Keywords
- Computer vision
- Computing methodologies
- Modelling and simulation Computing methodologies
- Neural networks Computing methodologies
ASJC Scopus subject areas
- Computer Networks and Communications
- Computer Science Applications
- Human-Computer Interaction
- Media Technology
Fingerprint
Dive into the research topics of 'Towards An Egocentric Framework for Rigid and Articulated Object Tracking in Virtual Reality'. Together they form a unique fingerprint.-
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA) - 2.0
Campbell, N. (PI), Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Cosker, D. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Petrini, K. (CoI), Proulx, M. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/11/20 → 31/10/25
Project: Research council
-
Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA)
Cosker, D. (PI), Bilzon, J. (CoI), Campbell, N. (CoI), Cazzola, D. (CoI), Colyer, S. (CoI), Fincham Haines, T. (CoI), Hall, P. (CoI), Kim, K. I. (CoI), Lutteroth, C. (CoI), McGuigan, P. (CoI), O'Neill, E. (CoI), Richardt, C. (CoI), Salo, A. (CoI), Seminati, E. (CoI), Tabor, A. (CoI) & Yang, Y. (CoI)
Engineering and Physical Sciences Research Council
1/09/15 → 28/02/21
Project: Research council