TY - GEN
T1 - Piccolo
T2 - 9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010
AU - Power, Russell
AU - Li, Jinyang
N1 - Funding Information:
Yasemin Avcular and Christopher Mitchell ran some of the Hadoop experiments. We thank the many people who have improved this work through discussion and reviews: the members of NeWS group at NYU, Frank Dabek, Rob Fergus, Michael Freedman, Robert Grimm, Wilson Hsieh, Frans Kaashoek, Jinyuan Li, Robert Morris, Sam Roweis, Torsten Suel, Junfeng Yang, Nickolai Zeldovich.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2019/1/1
Y1 - 2019/1/1
N2 - Piccolo is a new data-centric programming model for writing parallel in-memory applications in data centers. Unlike existing data-flow models, Piccolo allows computation running on different machines to share distributed, mutable state via a key-value table interface. Piccolo enables efficient application implementations. In particular, applications can specify locality policies to exploit the locality of shared state access and Piccolo's run-time automatically resolves write-write conflicts using user-defined accumulation functions. Using Piccolo, we have implemented applications for several problem domains, including the PageRank algorithm, k-means clustering and a distributed crawler. Experiments using 100 Amazon EC2 instances and a 12 machine cluster show Piccolo to be faster than existing data flow models for many problems, while providing similar fault-tolerance guarantees and a convenient programming interface.
AB - Piccolo is a new data-centric programming model for writing parallel in-memory applications in data centers. Unlike existing data-flow models, Piccolo allows computation running on different machines to share distributed, mutable state via a key-value table interface. Piccolo enables efficient application implementations. In particular, applications can specify locality policies to exploit the locality of shared state access and Piccolo's run-time automatically resolves write-write conflicts using user-defined accumulation functions. Using Piccolo, we have implemented applications for several problem domains, including the PageRank algorithm, k-means clustering and a distributed crawler. Experiments using 100 Amazon EC2 instances and a 12 machine cluster show Piccolo to be faster than existing data flow models for many problems, while providing similar fault-tolerance guarantees and a convenient programming interface.
UR - http://www.scopus.com/inward/record.url?scp=85076911148&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85076911148&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85076911148
T3 - Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010
SP - 293
EP - 306
BT - Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2010
PB - USENIX Association
Y2 - 4 October 2010 through 6 October 2010
ER -