Valmet: A new validation tool for assessing and improving 3D object segmentation

Guido Gerig, Matthieu Jomier, Miranda Chakos

    Research output: Chapter in Book/Report/Conference proceedingConference contribution


    Extracting 3D structures from volumetric images like MRI or CT is becoming a routine process for diagnosis based on quantitation, for radiotherapy planning, for surgical planning and image-guided intervention, for studying neurodevelopmental and neurodegenerative aspects of brain diseases, and for clinical drug trials. Key issues for segmenting anatomical objects from 3D medical images are validity and reliability. We have developed VALMET, a new tool for validation and comparison of object segmentation. New features not available in commercial and public-domain image processing packages are the choice between different metrics to describe differences between segmentations and the use of graphical overlay and 3D display for visual assessment of the locality and magnitude of segmentation variability. Input to the tool are an original 3D image (MRI, CT, ultrasound), and a series of segmentations either generated by several human raters and/or by automatic methods (machine). Quantitative evaluation includes intra-class correlation of resulting volumes and four different shape distance metrics, a) percentage overlap of segmented structures (R intersect S)/(R union S), b) probabilistic overlap measure for non-binary segmentations, c) mean/median absolute distances between object surfaces, and maximum (Hausdorff) distance. All these measures are calculated for arbitrarily selected 2D cross-sections and full 3D segmentations. Segmentation results are overlaid onto the original image data for visual comparison. A 3D graphical display of the segmented organ is color-coded depending on the selected metric for measuring segmentation difference. The new tool is in routine use for intra- and inter-rater reliability studies and for testing novel automatic machine-segmentation versus a gold standard established by human experts. Preliminary studies showed that the new tool could significantly improve intra- and inter-rater reliability of hippocampus segmentation to achieve intra-class correlation coefficients significantly higher than published elsewhere.

    Original languageEnglish (US)
    Title of host publicationMedical Image Computing and Computer-Assisted Intervention - MICCAI 2001 - 4th International Conference, Proceedings
    EditorsWiro J. Niessen, Max A. Viergever
    PublisherSpringer Verlag
    Number of pages8
    ISBN (Print)3540426973, 9783540454687
    StatePublished - 2001
    Event4th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2001 - Utrecht, Netherlands
    Duration: Oct 14 2001Oct 17 2001

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349


    Other4th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2001

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • General Computer Science


    Dive into the research topics of 'Valmet: A new validation tool for assessing and improving 3D object segmentation'. Together they form a unique fingerprint.

    Cite this