Multi-modal Perceptual Adversarial Learning for Longitudinal Prediction of Infant MR Images

Liying Peng, Lanfen Lin, Yusen Lin, Yue Zhang, Roza M. Vlasova, Juan Prieto, Yen wei Chen, Guido Gerig, Martin Styner

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Longitudinal magnetic resonance imaging (MRI) is essential in neuroimaging studies of early brain development. However, incomplete data is an inevitable problem in longitudinal studies because of participant attrition and scan failure. Data imputation is a possible way to address such missing data. Here, we propose a novel 3D multi-modal perceptual adversarial network (MPGAN) to predict a missing MR image from an existing longitudinal image of the same subject. To the best of our knowledge, this is the first application of deep generative methods for longitudinal image prediction of structural MRI in the first year of life, where brain volume and image intensities are changing dramatically. In order to produce sharper and more realistic images, we incorporate the perceptual loss into the adversarial training process. To leverage complementary information contained in the multi-modality data, MPGAN predicts T1w and T2w images jointly in the prediction process. We evaluated MPGAN versus six alternative approaches based on visual as well as quantitative assessment. The results indicate that our MPGAN predicts missing MR images in an accurate and visually realistic fashion, and shows better performance than the alternative methods.

Original languageEnglish (US)
Title of host publicationMedical Ultrasound, and Preterm, Perinatal and Paediatric Image Analysis - 1st International Workshop, ASMUS 2020, and 5th International Workshop, PIPPI 2020, Held in Conjunction with MICCAI 2020, Proceedings
EditorsYipeng Hu, Roxane Licandro, J. Alison Noble, Jana Hutter, Andrew Melbourne, Stephen Aylward, Esra Abaci Turk, Jordina Torrents Barrena, Jordina Torrents Barrena
PublisherSpringer Science and Business Media Deutschland GmbH
Pages284-294
Number of pages11
ISBN (Print)9783030603335
DOIs
StatePublished - 2020
Event1st International Workshop on Advances in Simplifying Medical UltraSound, ASMUS 2020, and the 5th International Workshop on Perinatal, Preterm and Paediatric Image Analysis, PIPPI 2020, held in conjunction with the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020 - Lima, Peru
Duration: Oct 4 2020Oct 8 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12437 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference1st International Workshop on Advances in Simplifying Medical UltraSound, ASMUS 2020, and the 5th International Workshop on Perinatal, Preterm and Paediatric Image Analysis, PIPPI 2020, held in conjunction with the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020
Country/TerritoryPeru
CityLima
Period10/4/2010/8/20

Keywords

  • Generative adversarial networks
  • MRI
  • Prediction

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Multi-modal Perceptual Adversarial Learning for Longitudinal Prediction of Infant MR Images'. Together they form a unique fingerprint.

Cite this