Piccolo: building fast, distributed programs with partitioned tables
New York University
Proceedings of the 9th USENIX conference on Operating systems design and implementation, OSDI’10, 2010
@inproceedings{power2010piccolo,
title={Piccolo: building fast, distributed programs with partitioned tables},
author={Power, R. and Li, J.},
booktitle={Proceedings of the 9th USENIX conference on Operating systems design and implementation},
pages={1–14},
year={2010},
organization={USENIX Association}
}
Piccolo is a new data-centric programming model for writing parallel in-memory applications in data centers. Unlike existing data-flow models, Piccolo allows computation running on different machines to share distributed, mutable state via a key-value table interface. Piccolo enables efficient application implementations. In particular, applications can specify locality policies to exploit the locality of shared state access and Piccolo’s run-time automatically resolves write-write conflicts using user-defined accumulation functions. Using Piccolo, we have implemented applications for several problem domains, including the PageRank algorithm, k-means clustering and a distributed crawler. Experiments using 100 Amazon EC2 instances and a 12 machine cluster show Piccolo to be faster than existing data flow models for many problems, while providing similar fault-tolerance guarantees and a convenient programming interface.
September 9, 2011 by hgpu
