Benchmarking Harp-DAAL: High Performance Hadoop on KNL Clusters

Langshi Chen, Bo Peng, Bingjing Zhang, Tony Liu, Yiming Zou, Lei Jiang, Robert Henschel, Craig Stewart, Zhang Zhang, Emily Mccallum, Zahniser Tom, Omer Jon, Judy Qiu
School of Informatics and Computing, Indiana University
IEEE Cloud 2017 Conference, 2017


   title={Benchmarking Harp-DAAL: High Performance Hadoop on KNL Clusters},

   author={Chen, Langshi and Peng, Bo and Zhang, Bingjing and Liu, Tony and Zou, Yiming and Jiang, Lei and Henschel, Robert and Stewart, Craig and Zhang, Zhang and Mccallum, Emily and others},



Data analytics is undergoing a revolution in many scientific domains, demanding cost-effective parallel data analysis techniques. Traditional Java-based Big Data processing tools like Hadoop MapReduce are designed for commodity CPUs. In contrast, emerging manycore processors like Xeon Phi has an order of magnitude of computation power and memory bandwidth. To harness the computing capabilities, we propose a Harp-DAAL framework. We show that enhanced versions of MapReduce can be replaced by Harp, a Hadoop plug-in, that offers useful data abstractions for both of high-performance iterative computation and MPI-quality communication, and it can drive Intel’s native library DAAL. We select a subset of three machine learning algorithms and implement them within Harp-DAAL. Our scalability benchmarks run on Knights Landing (KNL) clusters and achieve up to 2.5 times speedup of performance to the HPC solution in NOMAD and 15 to 40 times faster than Java-based solutions in Spark. We further quantify the workloads on single node KNL with a performance breakdown at micro-architecture level.
Rating: 2.0/5. From 1 vote.
Please wait...

* * *

* * *

Featured events

Hida Takayama, Japan

The Third International Workshop on GPU Computing and AI (GCA), 2018

Nagoya University, Japan

The 5th International Conference on Power and Energy Systems Engineering (CPESE), 2018

MediaCityUK, Salford Quays, Greater Manchester, England

The 10th International Conference on Information Management and Engineering (ICIME), 2018

No. 1037, Luoyu Road, Hongshan District, Wuhan, China

The 4th International Conference on Control Science and Systems Engineering (ICCSSE), 2018

Nanyang Executive Centre in Nanyang Technological University, Singapore

The 2018 International Conference on Cloud Computing and Internet of Things (CCIOT’18), 2018

HGPU group © 2010-2018 hgpu.org

All rights belong to the respective authors

Contact us: