GMH: A Message Passing Toolkit for GPU Clusters
The Scientific Computing Group, Jefferson Lab, Newport News, Virginia 23606, USA
IEEE 16th International Conference on Parallel and Distributed Systems (ICPADS), 2010
@conference{chen2010gmh,
title={GMH: A Message Passing Toolkit for GPU Clusters},
author={Chen, J. and Watson III, W. and Mao, W.},
booktitle={2010 IEEE 16th International Conference on Parallel and Distributed Systems},
pages={35–42},
issn={1521-9097},
year={2010},
organization={IEEE}
}
Driven by the market demand for high-definition 3D graphics, commodity graphics processing units (GPUs) have evolved into highly parallel, multi-threaded, many-core processors, which are ideal for data parallel computing. Many applications have been ported to run on a single GPU with tremendous speedups using general C-style programming languages such as CUDA. However, large applications require multiple GPUs and demand explicit message passing. This paper presents a message passing toolkit, called GMH (GPU Message Handler), on NVIDIA GPUs. This toolkit utilizes a data-parallel thread group as a way to map multiple GPUs on a single host to an MPI rank, and introduces a notion of virtual GPUs as a way to bind a thread to a GPU automatically. This toolkit provides high performance MPI style point-to-point and collective communication, but more importantly, facilitates event-driven APIs to allow an application to be managed and executed by the toolkit at runtime.
March 28, 2011 by hgpu