ReFit: A Framework for Refinement of Weakly Supervised Semantic Segmentation Using Object Border Fitting for Medical Images

Bharath Srinivas Prabakaran, Erik Ostrowski, Muhammad Shafique

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Weakly Supervised Semantic Segmentation (WSSS) relying only on image-level supervision is a promising approach to deal with the need for Segmentation networks, especially for generating a large number of pixel-wise masks in a given dataset. However, most state-of-the-art image-level WSSS techniques lack an understanding of the geometric features embedded in the images since the network cannot derive any object boundary information from just image-level labels. We define a boundary here as the line separating an object and its background, or two different objects. To address this drawback, we are proposing our novel ReFit framework, which deploys state-of-the-art class activation maps combined with various post-processing techniques in order to achieve fine-grained higher-accuracy segmentation masks. To achieve this, we investigate a state-of-the-art unsupervised segmentation network that can be used to construct a boundary map, which enables ReFit to predict object locations with sharper boundaries. By applying our method to WSSS predictions, we achieved up to 10% improvement over the current state-of-the-art WSSS methods for medical imaging. The framework is open-source, to ensure that our results are reproducible, and accessible online at

Original languageEnglish (US)
Title of host publicationAdvances in Visual Computing - 18th International Symposium, ISVC 2023, Proceedings
EditorsGeorge Bebis, Golnaz Ghiasi, Yi Fang, Andrei Sharf, Yue Dong, Chris Weaver, Zhicheng Leo, Joseph J. LaViola Jr., Luv Kohli
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages12
ISBN (Print)9783031479687
StatePublished - 2023
Event18th International Symposium on Visual Computing, ISVC 2023 - Lake Tahoe, United States
Duration: Oct 16 2023Oct 18 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference18th International Symposium on Visual Computing, ISVC 2023
Country/TerritoryUnited States
CityLake Tahoe


  • Activation Maps
  • Boundary
  • CAM
  • Masks
  • Medical Imaging Framework
  • Refinement
  • Semantic Segmentation
  • Weak Supervision

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'ReFit: A Framework for Refinement of Weakly Supervised Semantic Segmentation Using Object Border Fitting for Medical Images'. Together they form a unique fingerprint.

Cite this