Combining shape-from-shading and stereo using Gaussian-Markov random fields

In this paper we present a method of combining stereo and shape-from-shading information, taking account of the local reliability of each shape estimate. Local estimates of disparity and orientation are modelled using Gaussian distributions. A Gaussian-Markov random field is used to represent the disparity-map, taking into account interactions between disparity measurements and surface orientation, and the MAP estimate found using belief propagation. Local estimates of the precision of disparities and surface normals are found and used to control the process so that the most accurate data source is used in each region. We assess the performance of our approach using both synthetic and real stereo pairs, and compare against ground truth.