Labeling of surface discontinuities through the integration of vision modules

We assume that a major goal of the early vision modules and their integration is to deliver a cartoon of the discontinuities in the scene and to label them in terms of their physical origin. The output of each of the vision modules is noisy, possibly sparse and sometimes not unique. We have used a coupled Markov Random Field (MRF) at the output of each module - stereo, motion, color, texture - to achieve two goals: first, to counteract the noise and fill sparse data and second, to integrate the image within each MRF to find the module discontinuities and align them with the intensity edges. In this work we discuss the extension of this scheme for the integration of all the low-level modules and the labeling of discontinuities in terms of depth, orientation, albedo, illumination and specular discontinuities. We present labeling results using a simple linear classifier operating on the output of the MRF associated with each vision module and coupled to the image data. The classifier has been trained on a small set of a mixture of synthetic and real data.