Piccolo: Building fast, distributed programs with partitioned tables

Russell Power, Jinyang Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Piccolo is a new data-centric programming model for writing parallel in-memory applications in data centers. Unlike existing data-flow models, Piccolo allows computation running on different machines to share distributed, mutable state via a key-value table interface. Piccolo enables efficient application implementations. In particular, applications can specify locality policies to exploit the locality of shared state access and Piccolo's run-time automatically resolves write-write conflicts using user-defined accumulation functions. Using Piccolo, we have implemented applications for several problem domains, including the PageRank algorithm, k-means clustering and a distributed crawler. Experiments using 100 Amazon EC2 instances and a 12 machine cluster show Piccolo to be faster than existing data flow models for many problems, while providing similar fault-tolerance guarantees and a convenient programming interface.

Original languageEnglish (US)
Title of host publicationProceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010
PublisherUSENIX Association
Pages293-306
Number of pages14
ISBN (Electronic)9781931971799
StatePublished - Jan 1 2019
Event9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010 - Vancouver, Canada
Duration: Oct 4 2010Oct 6 2010

Publication series

NameProceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010

Conference

Conference9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010
CountryCanada
CityVancouver
Period10/4/1010/6/10

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems

Fingerprint Dive into the research topics of 'Piccolo: Building fast, distributed programs with partitioned tables'. Together they form a unique fingerprint.

Cite this