Reconstructing mass-conserved water surfaces using shape from shading and optical flow

This paper introduces a method for reconstructing water from real video footage. Using a single input video, the proposed method produces a more informative reconstruction from a wider range of possible scenes than the current state of the art. The key is the combination of vision algorithms and physics laws. Shape from shading is used to capture the change of the water's surface, from which a vertical velocity gradient field is calculated. Such a gradient field is used to constrain the tracking of horizontal velocities by minimizing an energy function as a weighted combination of mass-conservation and intensity-conservation. Hence the final reconstruction contains a dense velocity field that is incompressible in 3D. The proposed method is efficient and performs consistently well across water of different types.