Segmentation of dynamic scenes with distributions of spatiotemporally oriented energies

Damien Teney, Matthew Brown

Research output: Contribution to conferencePaper

5 Citations (SciVal)
124 Downloads (Pure)


In video segmentation, disambiguating appearance cues by grouping similar motions or dynamics is potentially powerful, though non-trivial. Dynamic changes of appearance can occur from rigid or non-rigid motion, as well as complex dynamic textures. While the former are easily captured by optical flow, phenomena such as a dissipating cloud of smoke, or flickering reflections on water, do not satisfy the assumption of brightness constancy, or cannot be modelled with rigid displacements in the image. To tackle this problem, we propose a robust representation of image dynamics as histograms of motion energy (HoME) obtained from convolutions of the video with spatiotemporal filters. They capture a wide range of dynamics and handle problems previously studied separately (motion and dynamic texture segmentation). They thus offer a potential solution for a new class of problems that contain these effects in the same scene. Our representation of image dynamics is integrated in a graph-based segmentation framework and combined with colour histograms to represent the appearance of regions. In the case of translating and occluding segments, the proposed features additionally serve to characterize the motion of the boundary between pairs of segments, to identify the occluder and inferring a local depth ordering. The resulting segmentation method is completely modelfree and unsupervised, and achieves state-of-the-art results on the SynthDB dataset for dynamic texture segmentation, on the MIT dataset for motion segmentation, and reasonable performance on the CMU dataset for occlusion boundaries.

Original languageEnglish
Publication statusPublished - 2014
Event25th British Machine Vision Conference, BMVC 2014 - Nottingham, UK United Kingdom
Duration: 1 Sept 20145 Sept 2014


Conference25th British Machine Vision Conference, BMVC 2014
Country/TerritoryUK United Kingdom


Dive into the research topics of 'Segmentation of dynamic scenes with distributions of spatiotemporally oriented energies'. Together they form a unique fingerprint.

Cite this