Hybrid Map Task Scheduling for GPU-Based Heterogeneous Clusters

Koichi Shirahata, Hitoshi Sato, Satoshi Matsuoka
Tokyo Institute of Technology, Tokyo, Japan
IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), 2010


   title={Hybrid Map Task Scheduling for GPU-Based Heterogeneous Clusters},

   author={Shirahata, K. and Sato, H. and Matsuoka, S.},

   booktitle={2nd IEEE International Conference on Cloud Computing Technology and Science},





Download Download (PDF)   View View   Source Source   



MapReduce is a programming model that enables efficient massive data processing in large-scale computing environments such as supercomputers and clouds. Such large-scale computers employ GPUs to enjoy its good peak performance and high memory bandwidth. Since the performance of each job is depending on running application characteristics and underlying computing environments, scheduling MapReduce tasks onto CPU cores and GPU devices for efficient execution is difficult. To address this problem, we have proposed a hybrid scheduling technique for GPU-based computer clusters, which minimizes the execution time of a submitted job using dynamic profiles of Map tasks running on CPU cores and GPU devices. We have implemented a prototype of our proposed scheduling technique by extending MapReduce framework, Hadoop. We have conducted some experiments for this prototype by using a K-means application as a benchmark on a supercomputer. The results show that the proposed technique achieves 1.93 times faster than the Hadoop original scheduling algorithm at 64 nodes (1024 CPU cores and 128 GPU devices). The results also indicate that the performance of map tasks, including both CPU and GPU tasks, is significantly affected by the overhead of map task invocation in the Hadoop framework.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2021 hgpu.org

All rights belong to the respective authors

Contact us: