Generative Terrain Fast Prototyping in Virtual Reality with Freehand Sketching Interface

Terrain generation and authoring in Virtual Reality (VR) offers unique benefits for terrain authoring including stereo display, immersive and intuitive design experience, and natural input modalities. We present this VR-based terrain fast prototyping system to integrate natural input modalities, preserve artistic controls and lower the effort of landscape prototyping. The system utilizes freehand interfaces and a generative model to help users quickly prototype different types of natural landscapes, such as mountains, mesas, canyons, and volcanoes. With the freehand interfaces, users can use their hands to draw mid-air strokes as the 3D contours of the desired landscapes. Then, a Conditional Generative Adversarial Network (CGAN) generates realistic landscapes based on the 3D contour. The freehand interface detects users’ gestures to control landscape editing. By incorporating CGAN as the terrain synthesizer, our system allows users to immersively, rapidly, and easily prototype terrains using intuitive interactive hand controls.