Dynamic expressivity with static optimization for streaming languages

Robert Soulé, Michael I. Gordon, Saman Amarasinghe, Robert Grimm, Martin Hirzel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Developers increasingly use streaming languages to write applications that process large volumes of data with high throughput. Unfortunately, when picking which streaming language to use, they face a difficult choice. On the one hand, dynamically scheduled languages allow developers to write a wider range of applications, but cannot take advantage of many crucial optimizations. On the other hand, statically scheduled languages are extremely performant, but have difficulty expressing many important streaming applications. This paper presents the design of a hybrid scheduler for stream processing languages. The compiler partitions the streaming application into coarse-grained subgraphs separated by dynamic rate boundaries. It then applies static optimizations to those subgraphs. We have implemented this scheduler as an extension to the Streamlt compiler. To evaluate its performance, we compare it to three scheduling techniques used by dynamic systems (OS thread, demand, and no-op) on a combination of micro-benchmarks and real-world inspired synthetic benchmarks. Our scheduler not only allows the previously static version of Streamlt to run dynamic rate applications, but it outperforms the three dynamic alternatives. This demonstrates that our scheduler strikes the right balance between expressivity and performance for stream processing languages.

Original languageEnglish (US)
Title of host publicationDEBS 2013 - Proceedings of the 7th ACM International Conference on Distributed Event-Based Systems
Pages159-170
Number of pages12
DOIs
StatePublished - 2013
Event7th ACM International Conference on Distributed Event-Based Systems, DEBS 2013 - Arlington, TX, United States
Duration: Jun 29 2013Jul 3 2013

Publication series

NameDEBS 2013 - Proceedings of the 7th ACM International Conference on Distributed Event-Based Systems

Other

Other7th ACM International Conference on Distributed Event-Based Systems, DEBS 2013
CountryUnited States
CityArlington, TX
Period6/29/137/3/13

Keywords

  • Stream processing
  • Streamlt

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering

Fingerprint Dive into the research topics of 'Dynamic expressivity with static optimization for streaming languages'. Together they form a unique fingerprint.

Cite this