Multi-modal Perceptual Adversarial Learning for Longitudinal Prediction of Infant MR Images

Liying Peng, Lanfen Lin, Yusen Lin, Yue Zhang, Roza M. Vlasova, Juan Prieto, Yen wei Chen, Guido Gerig, Martin Styner

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Longitudinal magnetic resonance imaging (MRI) is essential in neuroimaging studies of early brain development. However, incomplete data is an inevitable problem in longitudinal studies because of participant attrition and scan failure. Data imputation is a possible way to address such missing data. Here, we propose a novel 3D multi-modal perceptual adversarial network (MPGAN) to predict a missing MR image from an existing longitudinal image of the same subject. To the best of our knowledge, this is the first application of deep generative methods for longitudinal image prediction of structural MRI in the first year of life, where brain volume and image intensities are changing dramatically. In order to produce sharper and more realistic images, we incorporate the perceptual loss into the adversarial training process. To leverage complementary information contained in the multi-modality data, MPGAN predicts T1w and T2w images jointly in the prediction process. We evaluated MPGAN versus six alternative approaches based on visual as well as quantitative assessment. The results indicate that our MPGAN predicts missing MR images in an accurate and visually realistic fashion, and shows better performance than the alternative methods.

    Original languageEnglish (US)
    Title of host publicationMedical Ultrasound, and Preterm, Perinatal and Paediatric Image Analysis - 1st International Workshop, ASMUS 2020, and 5th International Workshop, PIPPI 2020, Held in Conjunction with MICCAI 2020, Proceedings
    EditorsYipeng Hu, Roxane Licandro, J. Alison Noble, Jana Hutter, Andrew Melbourne, Stephen Aylward, Esra Abaci Turk, Jordina Torrents Barrena, Jordina Torrents Barrena
    PublisherSpringer Science and Business Media Deutschland GmbH
    Pages284-294
    Number of pages11
    ISBN (Print)9783030603335
    DOIs
    StatePublished - 2020
    Event1st International Workshop on Advances in Simplifying Medical UltraSound, ASMUS 2020, and the 5th International Workshop on Perinatal, Preterm and Paediatric Image Analysis, PIPPI 2020, held in conjunction with the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020 - Lima, Peru
    Duration: Oct 4 2020Oct 8 2020

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume12437 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    Conference1st International Workshop on Advances in Simplifying Medical UltraSound, ASMUS 2020, and the 5th International Workshop on Perinatal, Preterm and Paediatric Image Analysis, PIPPI 2020, held in conjunction with the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020
    CountryPeru
    CityLima
    Period10/4/2010/8/20

    Keywords

    • Generative adversarial networks
    • MRI
    • Prediction

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • Computer Science(all)

    Fingerprint Dive into the research topics of 'Multi-modal Perceptual Adversarial Learning for Longitudinal Prediction of Infant MR Images'. Together they form a unique fingerprint.

    Cite this