Efficient Support for Matrix Computations on Heterogeneous Multi-core and Multi-GPU Architectures
University of Tennessee, EECS Department, Knoxville, TN, USA
University of Tennessee, Computer Science Technical Report UT-CS-11-668, 2011
@article{song2011efficient,
title={Efficient Support for Matrix Computations on Heterogeneous Multi-core and Multi-GPU Architectures},
author={Song, F. and Tomov, S. and Dongarra, J.},
year={2011}
}
We present a new methodology for utilizing all CPU cores and all GPUs on a heterogeneous multicore and multi-GPU system to support matrix computations efficiently. Our approach is able to achieve the objectives of a high degree of parallelism, minimized synchronization, minimized communication, and load balancing. Our main idea is to treat the heterogeneous system as a distributed-memory machine, and to use a heterogeneous 1-D block cyclic distribution to allocate data to the host system and GPUs to minimize communication. We have designed heterogeneous algorithms with two different tile sizes (one for CPU cores and the other for GPUs) to cope with processor heterogeneity. We propose an auto-tuning method to determine the best tile sizes to attain both high performance and load balancing. We have also implemented a new runtime system and applied it to the Cholesky and QR factorizations. Our experiments on a compute node with two Intel Westmere hexa-core CPUs and three Nvidia Fermi GPUs demonstrate good weak scalability, strong scalability, load balance, and efficiency of our approach.
November 20, 2011 by hgpu
