Modeling and optimizing human-in-the-loop visual perception using immersive displays: A review

New and rapidly-evolving classes of display devices bridge the gap between us and the immersive experiences of the future. The most intimate of these displays are the Virtual-and Augmented-Reality (VR and AR) ones, because they are capable of presenting synthetic environments that rival those in the real world. This ecosystem of personal and highly-immersive displays offers new challenges for research in computer graphics, display technologies, and human visual perception. While the extensive advancements in the areas of display and computer graphics technologies traditionally end at the on-screen “image,” there are several untapped opportunities for advances that exploit the interplay between the display characteristics and how our visual system perceives them. In this article, we review recent progress in understanding and modeling the perception of immersive displays, as well as perceptually optimizing display technologies for immersive experiences. We present this review in the form of a taxonomy that maps the various properties of modern displays with the perceptual phenomenon that most closely interacts with them. From this taxonomy, we deduce several unsolved challenges in understanding human perception of displays, as well as perceptually-optimal characteristics of future displays.