The Visual Analysis of Complex Natural Phenomena

Abstract

Optical flow estimation or dense motion estimation for dynamic natural phenomena (water, smoke, fire, etc.) is a significant open problem in Computer Vision. Assumptions such as brightness constancy cannot be relied upon, as natural phenomena scenes contain lots of non- rigid motion, blurred motion, etc. Current approaches tend to be either general, giving poor results, or else be specialised in one phenomenon and therefore fail to generalise well. The literature would benefit from a general solution, and such a solution could be found useful in a diverse set of application areas. In this thesis, we prove that a skeleton based feature can guide the standard optical flow pipeline to obtain state-of-the-art motion results. We also demonstrate that this result can be applied in different applications such as video segmentation, slow motion, etc.First, we describe an approach to estimating dense motion for dynamic phenomena that is simple and can be extended to a wide range of phenomena. The key to our approach is to replace local assumptions such as brightness constancy with the global assumption in which characteristic topographic maps change subtly. This leads to a global sparse motion estimation, which upgrades to dense estimation for final motion results, as suggested in our experiments, are state-of-the-art. We demonstrate the method using lab-based and consumer-level video obtained from our dataset, public dataset and the Internet. Second, the motion result is applied on a slow motion application which contains fewer artefacts than the state-of-the-art commercial software Adobe AfterEffect 2017 CC. Third, we embed the motion result and the skeleton feature in a video segmentation pipeline and outperform the state-of-the-art video segmentation methods including the method which is specially designed for natural phenomena. Fourth, we introduce a dataset containing two types of sequences i.e., sequences based on a 6-sync cameras system and sequences with 88 different kinds of dynamic textures with a single view camera lab set-up. Fifth, since semi-transparent case often happens in natural phenomena, a closed form solution for layer separation is also proposed.

Date of Award	1 Dec 2017
Original language	English
Awarding Institution	University of Bath
Supervisor	Peter Hall (Supervisor) & Matthew Brown (Supervisor)