Modern Approaches to Camera Tracking Within the Visual Effects Pipeline

  • Alastair Barber

Student thesis: Doctoral ThesisDoctor of Engineering (EngD)


Visual Effects (VFX) are a crucial component in a large proportion of feature films being pro- duced today. The work done in producing VFX usually takes place after filming has happened, and by a specialised VFX facility. The process of producing visually realistic, and compelling, effects is complex and labour-intensive, requiring many skilled workers to complete different stages of the VFX ‘pipeline’. One of these tasks is Camera Tracking, the goal of which is to accurately calculate the movement of the camera used to film the original footage. Without this solution for camera movement, it would not be possible to convincingly render Computer Generated (CG) assets on to the original footage.The VFX pipeline is so called because it can be thought of as a process through which the original footage, output from digital artists, and other data produced, ‘flows’ towards producing a final output. Camera Tracking is one of the processes that is performed first in the pipeline. Therefore, as well as accuracy, timely completion of this stage is also essential in making sure that the VFX facility operates efficiently and in a cost effective manner. Deadlines are strictly enforced, and the cost of producing VFX is agreed and fixed at the start of the project - so delays at any point of the pipeline can have dire consequences.Camera Tracking is closely related to the field of research known as Structure From Motion (SfM). Double Negative Ltd, a UK based VFX studio with facilities worldwide, partnered with the University of Bath to establish a research project investigating how the latest work in the SfM domain could be applied to the process of Camera Tracking, which in VFX is still a process that involves a large amount of human interaction and hence cost.Presented in this project is a detailed investigation into the process of Camera Tracking at a VFX facility, utilising a large dataset of real shots from major Hollywood feature films. One of the main conclusions from this investigation is that Camera Tracking for VFX work, due to the nature of the work encountered in film production, is better regarded as a problem-solving exercise rather than a pure SfM problem. The quantitative results obtained in this work strongly suggest that having more data available about the scene being filmed and camera used is one of the most effective ways to reduce the time spent on the Camera Tracking process. This research project then investigates the use of additional on-set hardware to make obtaining this information easier. It also develops new methods for determining information about changes in parameters of the camera being used to film the scene using visual footage alone, under conditions in which other traditional Computer Vision methods would likely fail. The impact of this work has been a valuable contribution to the methods and tools available to artists performing these tasks, allowing them to operate more efficiently in this competitive and global industry, where the standards expected for the quality of VFX rise with each new film released.
Date of Award1 Jul 2018
Original languageEnglish
Awarding Institution
  • University of Bath
SponsorsEngineering and Physical Sciences Research Council & Double Negative Ltd
SupervisorDarren Cosker (Supervisor)

Cite this