Multi-modal image fusion for multispectral super-resolution in microscopy

Neel Dey, Shijie Li, Katharina Bermond, Rainer Heintzmann, Christine A. Curcio, Thomas Ach, Guido Gerig

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Spectral imaging is a ubiquitous tool in modern biochemistry. Despite acquiring dozens to thousands of spectral channels, existing technology cannot capture spectral images at the same spatial resolution as structural microscopy. Due to partial voluming and low light exposure, spectral images are often difficult to interpret and analyze. This highlights a need to upsample the low-resolution spectral image by using spatial information contained in the high-resolution image, thereby creating a fused representation with high specificity both spatially and spectrally. In this paper, we propose a framework for the fusion of co-registered structural and spectral microscopy images to create super-resolved representations of spectral images. As a first application, we super-resolve spectral images of ex-vivo retinal tissue imaged with confocal laser scanning microscopy, by using spatial information from structured illumination microscopy. Second, we super-resolve mass spectroscopic images of mouse brain tissue, by using spatial information from high-resolution histology images. We present a systematic validation of model assumptions crucial towards maintaining the original nature of spectra and the applicability of super-resolution. Goodness-of-fit for spectral predictions are evaluated through functional R2 values, and the spatial quality of the super-resolved images are evaluated using normalized mutual information.

    Original languageEnglish (US)
    Title of host publicationMedical Imaging 2019
    Subtitle of host publicationImage Processing
    EditorsElsa D. Angelini, Elsa D. Angelini, Elsa D. Angelini, Bennett A. Landman
    PublisherSPIE
    ISBN (Electronic)9781510625457
    DOIs
    StatePublished - 2019
    EventMedical Imaging 2019: Image Processing - San Diego, United States
    Duration: Feb 19 2019Feb 21 2019

    Publication series

    NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
    Volume10949
    ISSN (Print)1605-7422

    Conference

    ConferenceMedical Imaging 2019: Image Processing
    CountryUnited States
    CitySan Diego
    Period2/19/192/21/19

    Keywords

    • Bayesian Optimization
    • Confocal Laser Scanning Microscopy
    • Image Fusion
    • Imaging Mass Spectroscopy
    • Multispectral Image Super-resolution
    • Multispectral Imaging
    • Structured Illumination Microscopy

    ASJC Scopus subject areas

    • Electronic, Optical and Magnetic Materials
    • Biomaterials
    • Atomic and Molecular Physics, and Optics
    • Radiology Nuclear Medicine and imaging

    Fingerprint Dive into the research topics of 'Multi-modal image fusion for multispectral super-resolution in microscopy'. Together they form a unique fingerprint.

  • Cite this

    Dey, N., Li, S., Bermond, K., Heintzmann, R., Curcio, C. A., Ach, T., & Gerig, G. (2019). Multi-modal image fusion for multispectral super-resolution in microscopy. In E. D. Angelini, E. D. Angelini, E. D. Angelini, & B. A. Landman (Eds.), Medical Imaging 2019: Image Processing [109490D] (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10949). SPIE. https://doi.org/10.1117/12.2512598