Biomedical imaging of large patient populations, both cross-sectionally and longitudinally, is becoming a standard technique for noninvasive, in-vivo studies of the pathophysiology of diseases and for monitoring drug treatment. In radiation oncology, imaging and extraction of anatomical organ geometry is a routine procedure for therapy planning an monitoring, and similar procedures are vital for surgical planning and image-guided therapy. Bottlenecks of today's studies, often processed by labor-intensive manual region drawing, arc the lack of efficient, reliable tools for three-dimensional organ segmentation and for advanced morphologic characterization. This paper discusses current research and development focused towards building of statistical shape models, used for automatic model-based segmentation and for shape analysis and discrimination. We build statistical shape models which describe the geometric variability and image intensity characteristics of anatomical structures. New segmentations are obtained by model deformation driven by local image match forces and constrained by the training statistics. Two complimentary representations for 3D shape are discussed and compared, one based on global surface parametrization and a second one on medial manifold description. The discussion will be guided by presenting a most recent study to construct a statistical shape model of the caudate structure.