Assessment of reliability of multi-site neuroimaging via traveling phantom study

This paper describes a framework for quantitative analysis of neuroimaging data of traveling human phantoms used for cross-site validation. We focus on the analysis of magnetic resonance image data including intra- and inter-site comparison. Locations and magnitude of geometric deformation is studied via unbiased atlas building and metrics on deformation fields. Variability of tissue segmentation is analyzed by comparison of volumes, overlap of tissue maps, and a new Kullback-Leibler divergence on tissue probabilities, with emphasis on comparing probabilistic rather than binary segmentations. We show that results from this information theoretic measure are highly correlated with overlap. Reproducibility of automatic, atlas-based segmentation of subcortical structures is examined by comparison of volumes, shape overlap and surface distances. Variability among scanners of the same type but also differences to a different scanner type are discussed. The results demonstrate excellent reliability across multiple sites that can be achieved by the use of the today's scanner generation and powerful automatic analysis software. Knowledge about such variability is crucial for study design and power analysis in new multi-site clinical studies.