Dueling deep Q-network for unsupervised inter-frame eye movement correction in optical coherence tomography volumes

Yasmeen George, Suman Sedai, Bhavna J. Antony, Hiroshi Ishikawa, Gadi Wollstein, Joel S. Schuman, Rahil Garnavi

    Research output: Chapter in Book/Report/Conference proceedingConference contribution


    In optical coherence tomography (OCT) volumes of retina, the sequential acquisition of the individual slices makes this modality prone to motion artifacts, misalignments between adjacent slices being the most noticeable. Any distortion in OCT volumes can bias structural analysis and influence the outcome of longitudinal studies. The presence of speckle noise characteristic of this imaging modality leads to inaccuracies when traditional registration techniques are employed. Also, the lack of a well-defined ground truth makes supervised deep-learning techniques ill-posed to tackle the problem. In this paper, we tackle these issues by using deep reinforcement learning to correct inter-frame movements in an unsupervised manner. Specifically, we use dueling deep Q-network to train an artificial agent to find the optimal policy, i.e. a sequence of actions, that best improves the alignment by maximizing the sum of reward signals. Instead of relying on the ground-truth of transformation parameters to guide the rewarding system, for the first time, we use a combination of intensity based image similarity metrics. Further, to avoid the agent bias towards speckle noise, we ensure the agent can see retinal layers as part of the interacting environment. For quantitative evaluation, we simulate the eye movement artifacts by applying 2D rigid transformations on individual B-scans. The proposed model achieves an average of 0.985 and 0.914 for normalized mutual information and correlation coefficient, respectively. We also compare our model with elastix intensity based medical image registration approach, where significant improvement is achieved by our model for both noisy and denoised volumes.

    Original languageEnglish (US)
    Title of host publication2021 IEEE 18th International Symposium on Biomedical Imaging, ISBI 2021
    PublisherIEEE Computer Society
    Number of pages5
    ISBN (Electronic)9781665412469
    StatePublished - Apr 13 2021
    Event18th IEEE International Symposium on Biomedical Imaging, ISBI 2021 - Nice, France
    Duration: Apr 13 2021Apr 16 2021

    Publication series

    NameProceedings - International Symposium on Biomedical Imaging
    ISSN (Print)1945-7928
    ISSN (Electronic)1945-8452


    Conference18th IEEE International Symposium on Biomedical Imaging, ISBI 2021


    • Artificial agents
    • Dueling deep Q-network
    • Motion correction
    • Optical coherence tomography
    • Reinforcement learning

    ASJC Scopus subject areas

    • Biomedical Engineering
    • Radiology Nuclear Medicine and imaging


    Dive into the research topics of 'Dueling deep Q-network for unsupervised inter-frame eye movement correction in optical coherence tomography volumes'. Together they form a unique fingerprint.

    Cite this