Automatic 3D object segmentation in multiple views using volumetric graph-cuts

N.D.F. Campbell, G. Vogiatzis, C. Hernández, R. Cipolla

Research output: Contribution to journalArticle

48 Citations (Scopus)

Abstract

We propose an algorithm for automatically obtaining a segmentation of a rigid object in a sequence of images that are calibrated for camera pose and intrinsic parameters. Until recently, the best segmentation results have been obtained by interactive methods that require manual labelling of image regions. Our method requires no user input but instead relies on the camera fixating on the object of interest during the sequence. We begin by learning a model of the object's colour, from the image pixels around the fixation points. We then extract image edges and combine these with the object colour information in a volumetric binary MRF model. The globally optimal segmentation of 3D space is obtained by a graph-cut optimisation. From this segmentation an improved colour model is extracted and the whole process is iterated until convergence. Our first finding is that the fixation constraint, which requires that the object of interest is more or less central in the image, is enough to determine what to segment and initialise an automatic segmentation process. Second, we find that by performing a single segmentation in 3D, we implicitly exploit a 3D rigidity constraint, expressed as silhouette coherency, which significantly improves silhouette quality over independent 2D segmentations. We demonstrate the validity of our approach by providing segmentation results on real sequences.
Original languageEnglish
Pages (from-to)14-25
Number of pages12
JournalImage and Vision Computing
Volume28
Issue number1
DOIs
Publication statusPublished - 1 Jan 2010

Fingerprint

Color
Cameras
Rigidity
Labeling
Pixels

Cite this

Automatic 3D object segmentation in multiple views using volumetric graph-cuts. / Campbell, N.D.F.; Vogiatzis, G.; Hernández, C.; Cipolla, R.

In: Image and Vision Computing, Vol. 28, No. 1, 01.01.2010, p. 14-25.

Research output: Contribution to journalArticle

Campbell, N.D.F. ; Vogiatzis, G. ; Hernández, C. ; Cipolla, R. / Automatic 3D object segmentation in multiple views using volumetric graph-cuts. In: Image and Vision Computing. 2010 ; Vol. 28, No. 1. pp. 14-25.
@article{d001c60af50640f893f78a38c7180bf6,
title = "Automatic 3D object segmentation in multiple views using volumetric graph-cuts",
abstract = "We propose an algorithm for automatically obtaining a segmentation of a rigid object in a sequence of images that are calibrated for camera pose and intrinsic parameters. Until recently, the best segmentation results have been obtained by interactive methods that require manual labelling of image regions. Our method requires no user input but instead relies on the camera fixating on the object of interest during the sequence. We begin by learning a model of the object's colour, from the image pixels around the fixation points. We then extract image edges and combine these with the object colour information in a volumetric binary MRF model. The globally optimal segmentation of 3D space is obtained by a graph-cut optimisation. From this segmentation an improved colour model is extracted and the whole process is iterated until convergence. Our first finding is that the fixation constraint, which requires that the object of interest is more or less central in the image, is enough to determine what to segment and initialise an automatic segmentation process. Second, we find that by performing a single segmentation in 3D, we implicitly exploit a 3D rigidity constraint, expressed as silhouette coherency, which significantly improves silhouette quality over independent 2D segmentations. We demonstrate the validity of our approach by providing segmentation results on real sequences.",
author = "N.D.F. Campbell and G. Vogiatzis and C. Hern{\'a}ndez and R. Cipolla",
year = "2010",
month = "1",
day = "1",
doi = "10.1016/j.imavis.2008.09.005",
language = "English",
volume = "28",
pages = "14--25",
journal = "Image and Vision Computing",
issn = "0262-8856",
publisher = "Elsevier",
number = "1",

}

TY - JOUR

T1 - Automatic 3D object segmentation in multiple views using volumetric graph-cuts

AU - Campbell, N.D.F.

AU - Vogiatzis, G.

AU - Hernández, C.

AU - Cipolla, R.

PY - 2010/1/1

Y1 - 2010/1/1

N2 - We propose an algorithm for automatically obtaining a segmentation of a rigid object in a sequence of images that are calibrated for camera pose and intrinsic parameters. Until recently, the best segmentation results have been obtained by interactive methods that require manual labelling of image regions. Our method requires no user input but instead relies on the camera fixating on the object of interest during the sequence. We begin by learning a model of the object's colour, from the image pixels around the fixation points. We then extract image edges and combine these with the object colour information in a volumetric binary MRF model. The globally optimal segmentation of 3D space is obtained by a graph-cut optimisation. From this segmentation an improved colour model is extracted and the whole process is iterated until convergence. Our first finding is that the fixation constraint, which requires that the object of interest is more or less central in the image, is enough to determine what to segment and initialise an automatic segmentation process. Second, we find that by performing a single segmentation in 3D, we implicitly exploit a 3D rigidity constraint, expressed as silhouette coherency, which significantly improves silhouette quality over independent 2D segmentations. We demonstrate the validity of our approach by providing segmentation results on real sequences.

AB - We propose an algorithm for automatically obtaining a segmentation of a rigid object in a sequence of images that are calibrated for camera pose and intrinsic parameters. Until recently, the best segmentation results have been obtained by interactive methods that require manual labelling of image regions. Our method requires no user input but instead relies on the camera fixating on the object of interest during the sequence. We begin by learning a model of the object's colour, from the image pixels around the fixation points. We then extract image edges and combine these with the object colour information in a volumetric binary MRF model. The globally optimal segmentation of 3D space is obtained by a graph-cut optimisation. From this segmentation an improved colour model is extracted and the whole process is iterated until convergence. Our first finding is that the fixation constraint, which requires that the object of interest is more or less central in the image, is enough to determine what to segment and initialise an automatic segmentation process. Second, we find that by performing a single segmentation in 3D, we implicitly exploit a 3D rigidity constraint, expressed as silhouette coherency, which significantly improves silhouette quality over independent 2D segmentations. We demonstrate the validity of our approach by providing segmentation results on real sequences.

UR - http://www.scopus.com/inward/record.url?scp=70449525580&partnerID=8YFLogxK

UR - http://dx.doi.org/10.1016/j.imavis.2008.09.005

U2 - 10.1016/j.imavis.2008.09.005

DO - 10.1016/j.imavis.2008.09.005

M3 - Article

AN - SCOPUS:70449525580

VL - 28

SP - 14

EP - 25

JO - Image and Vision Computing

JF - Image and Vision Computing

SN - 0262-8856

IS - 1

ER -