Surface color perception as an inverse problem in biological vision

The spectral power distribution (SPD) of the light reflected from a matte surface patch in a three-dimensional complex scene depends not only on the surface reflectance of the patch but also on the SPD of the light incident on the patch. When there are multiple light sources in the scene that differ in location, SPD, and spatial extent, the SPD of the incident light depends on the location and the orientation of the patch. Recently, we have examined how well observers can recover surface color in rendered, binocularly-viewed scenes with more than one light source. To recover intrinsic surface color, observers must solve an inverse problem, effectively estimating the light sources present in the scene and the SPD of the light from each that reaches the surface patch. We will formulate the forward and inverse problems for surface color perception in three-dimensional scenes and present experimental evidence that human observers can solve such problems. We will also discuss how human observers estimate the spatial distribution of light sources and their chromaticities from the scene itself and how they might represent it.