Hybrid CUDA, OpenMP, and MPI parallel programming on multicore GPU clusters
Department of Computer Science, Tunghai University, Taichung City, 40704, Taiwan
Computer Physics Communications, Volume 182, Issue 1, January 2011, Pages 266-269 (16 July 2010)
@article{yang2010hybrid,
title={Hybrid CUDA, OpenMP, and MPI parallel programming on multicore GPU clusters},
author={Yang, C.T. and Huang, C.L. and Lin, C.F.},
journal={Computer Physics Communications},
issn={0010-4655},
year={2010},
publisher={Elsevier}
}
Nowadays, NVIDIA’s CUDA is a general purpose scalable parallel programming model for writing highly parallel applications. It provides several key abstractions – a hierarchy of thread blocks, shared memory, and barrier synchronization. This model has proven quite successful at programming multithreaded many core GPUs and scales transparently to hundreds of cores: scientists throughout industry and academia are already using CUDA to achieve dramatic speedups on production and research codes. In this paper, we propose a parallel programming approach using hybrid CUDA OpenMP, and MPI programming, which partition loop iterations according to the number of C1060 GPU nodes in a GPU cluster which consists of one C1060 and one S1070. Loop iterations assigned to one MPI process are processed in parallel by CUDA run by the processor cores in the same computational node.
November 8, 2010 by hgpu
