Benefiting from multitask learning to improve single image super-resolution

Mohammad Saeed Rad, Behzad Bozorgtabar, Claudiu Musat, Urs Viktor Marti, Max Basler, Hazım Kemal Ekenel, Jean Philippe Thiran

Research output: Contribution to journalArticlepeer-review

Abstract

Despite significant progress toward super resolving more realistic images by deeper convolutional neural networks (CNNs), reconstructing fine and natural textures still remains a challenging problem. Recent works on single image super resolution (SISR) are mostly based on optimizing pixel and content wise similarity between recovered and high-resolution (HR) images and do not benefit from recognizability of semantic classes. In this paper, we introduce a novel approach using categorical information to tackle the SISR problem; we present an encoder architecture able to extract and use semantic information to super-resolve a given image by using multitask learning, simultaneously for image super-resolution and semantic segmentation. To explore categorical information during training, the proposed decoder only employs one shared deep network for two task-specific output layers. At run-time only layers resulting HR image are used and no segmentation label is required. Extensive perceptual experiments and a user study on images randomly selected from COCO-Stuff dataset demonstrate the effectiveness of our proposed method and it outperforms the state-of-the-art methods.

Original languageEnglish (US)
Pages (from-to)304-313
Number of pages10
JournalNeurocomputing
Volume398
DOIs
StatePublished - Jul 20 2020

Keywords

  • Generative adversarial network
  • Multitask learning
  • Recovering realistic textures
  • Semantic segmentation
  • Single image super-resolution

ASJC Scopus subject areas

  • Computer Science Applications
  • Cognitive Neuroscience
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Benefiting from multitask learning to improve single image super-resolution'. Together they form a unique fingerprint.

Cite this