Enabling and Scaling Matrix Computations on Heterogeneous Multi-Core and Multi-GPU Systems

Fengguang Song, Stanimire Tomov, Jack Dongarra
EECS Department, University of Tennessee, Knoxville, TN, USA
The 26th ACM International Conference on Supercomputing (ICS 2012), 2012


   title={Enabling and Scaling Matrix Computations on Heterogeneous Multi-Core and Multi-GPU Systems},

   author={Song, F. and Tomov, S. and Dongarra, J.},



Download Download (PDF)   View View   Source Source   



We present a new approach to utilizing all CPU cores and all GPUs on heterogeneous multicore and multi-GPU systems to support dense matrix computations efficiently. The main idea is that we treat a heterogeneous system as a distributed-memory machine, and use a heterogeneous multi-level block cyclic distribution method to allocate data to the host and multiple GPUs to minimize communication. We design heterogeneous algorithms with hybrid tiles to accommodate the processor heterogeneity, and introduce an auto-tuning method to determine the hybrid tile sizes to attain both high performance and load balancing. We have also implemented a new runtime system and applied it to the Cholesky and QR factorizations. Our approach is designed for achieving four objectives: a high degree of parallelism, minimized synchronization, minimized communication, and load balancing. Our experiments on a compute node (with two Intel Westmere hexa-core CPUs and three Nvidia Fermi GPUs), as well as on up to 100 compute nodes on the Keeneland system [31], demonstrate great scalability, good load balancing, and efficiency of our approach.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2021 hgpu.org

All rights belong to the respective authors

Contact us: