ISLE: A Framework for Image Level Semantic Segmentation Ensemble

Erik Ostrowski, Muhammad Shafique

Research output: Chapter in Book/Report/Conference proceedingConference contribution


One key bottleneck of employing state-of-the-art semantic segmentation networks in the real world is the availability of training labels. Conventional semantic segmentation networks require massive pixel-wise annotated labels to reach state-of-the-art prediction quality. Hence, several works focus on semantic segmentation networks trained with only image-level annotations. However, when scrutinizing the results of state-of-the-art in more detail, we notice that they are remarkably close to each other on average prediction quality, different approaches perform better in different classes while providing low quality in others. To address this problem, we propose a novel framework, ISLE, which employs an ensemble of the “pseudo-labels” for a given set of different semantic segmentation techniques on a class-wise level. Pseudo-labels are the pixel-wise predictions of the image-level semantic segmentation frameworks used to train the final segmentation model. Our pseudo-labels seamlessly combine the strong points of multiple segmentation techniques approaches to reach superior prediction quality. We reach up to 2.4% improvement over ISLE’s individual components. An exhaustive analysis was performed to demonstrate ISLE’s effectiveness over state-of-the-art frameworks for image-level semantic segmentation.

Original languageEnglish (US)
Title of host publicationAdvances in Visual Computing - 18th International Symposium, ISVC 2023, Proceedings
EditorsGeorge Bebis, Golnaz Ghiasi, Yi Fang, Andrei Sharf, Yue Dong, Chris Weaver, Zhicheng Leo, Joseph J. LaViola Jr., Luv Kohli
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages12
ISBN (Print)9783031479656
StatePublished - 2023
Event18th International Symposium on Visual Computing, ISVC 2023 - Lake Tahoe, United States
Duration: Oct 16 2023Oct 18 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference18th International Symposium on Visual Computing, ISVC 2023
Country/TerritoryUnited States
CityLake Tahoe


  • Class Activation Maps
  • Deep Learning
  • Ensemble
  • Semantic Segmentation
  • Weakly Supervised

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'ISLE: A Framework for Image Level Semantic Segmentation Ensemble'. Together they form a unique fingerprint.

Cite this