Accelerating GPU kernels for dense linear algebra
Department of Electrical Engineering and Computer Science, University of Tennessee, Knoxville
High Performance Computing for Computational Science – VECPAR 2010, Lecture Notes in Computer Science, 2011, Volume 6449/2011, 83-92
@article{nath2011accelerating,
title={Accelerating GPU kernels for dense linear algebra},
author={Nath, R. and Tomov, S. and Dongarra, J.},
journal={High Performance Computing for Computational Science–VECPAR 2010},
pages={83–92},
year={2011},
publisher={Springer}
}
Implementations of the Basic Linear Algebra Subprograms (BLAS) interface are major building block of dense linear algebra (DLA) libraries, and therefore have to be highly optimized. We present some techniques and implementations that significantly accelerate the corresponding routines from currently available libraries for GPUs. In particular, Pointer Redirecting – a set of GPU specific optimization techniques – allows us to easily remove performance oscillations associated with problem dimensions not divisible by fixed blocking sizes. For example, applied to the matrix-matrix multiplication routines, depending on the hardware configuration and routine parameters, this can lead to two times faster algorithms. Similarly, the matrix-vector multiplication can be accelerated more than two times in both single and double precision arithmetic. Additionally, GPU specific acceleration techniques are applied to develop new kernels (e.g. syrk, symv) that are up to 20! faster than the currently available kernels. We present these kernels and also show their acceleration e!ect to higher level dense linear algebra routines. The accelerated kernels are now freely available through the MAGMA BLAS library.
June 4, 2011 by hgpu
