high performance computing on graphics processing units: hgpu.org

Graphical processing Units (GPUs) are finding widespread use as accelerators in computer clusters. It is not yet trivial to program applications that use multiple GPU-enabled cluster nodes efficiently. A key aspect of this is managing effective communication between GPU memory on separate devices on separate nodes. We develop an algorithmic framework for Finite-Difference numerical simulations that would normally require highly synchronous data-parallelism so they can effectively use loosely coupled GPU-enabled cluster nodes. We employ asynchronous communications and appropriate memory overlay of computations and communications to hide latency.