Piccolo: Building fast, distributed programs with partitioned tables

Russell Power, Jinyang Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Piccolo is a new data-centric programming model for writing parallel in-memory applications in data centers. Unlike existing data-flow models, Piccolo allows computation running on different machines to share distributed, mutable state via a key-value table interface. Piccolo enables efficient application implementations. In particular, applications can specify locality policies to exploit the locality of shared state access and Piccolo's run-time automatically resolves write-write conflicts using user-defined accumulation functions. Using Piccolo, we have implemented applications for several problem domains, including the PageRank algorithm, k-means clustering and a distributed crawler. Experiments using 100 Amazon EC2 instances and a 12 machine cluster show Piccolo to be faster than existing data flow models for many problems, while providing similar fault-tolerance guarantees and a convenient programming interface.

Original languageEnglish (US)
Title of host publicationProceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010
PublisherUSENIX Association
Pages293-306
Number of pages14
ISBN (Electronic)9781931971799
StatePublished - Jan 1 2019
Event9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010 - Vancouver, Canada
Duration: Oct 4 2010Oct 6 2010

Publication series

NameProceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010

Conference

Conference9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010
Country/TerritoryCanada
CityVancouver
Period10/4/1010/6/10

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems

Fingerprint

Dive into the research topics of 'Piccolo: Building fast, distributed programs with partitioned tables'. Together they form a unique fingerprint.

Cite this