25934

Bayesian Optimization for auto-tuning GPU kernels

Floris-Jan Willemsen, Rob van Nieuwpoort, Ben van Werkhoven
Netherlands eScience Center, University of Amsterdam, Amsterdam, the Netherlands
arXiv:2111.14991 [cs.LG], (26 Nov 2021)

@misc{willemsen2021bayesian,

   title={Bayesian Optimization for auto-tuning GPU kernels},

   author={Floris-Jan Willemsen and Rob van Nieuwpoort and Ben van Werkhoven},

   year={2021},

   eprint={2111.14991},

   archivePrefix={arXiv},

   primaryClass={cs.LG}

}

Download Download (PDF)   View View   Source Source   Source codes Source codes

Package:

857

views

Finding optimal parameter configurations for tunable GPU kernels is a non-trivial exercise for large search spaces, even when automated. This poses an optimization task on a non-convex search space, using an expensive to evaluate function with unknown derivative. These characteristics make a good candidate for Bayesian Optimization, which has not been applied to this problem before. However, the application of Bayesian Optimization to this problem is challenging. We demonstrate how to deal with the rough, discrete, constrained search spaces, containing invalid configurations. We introduce a novel contextual variance exploration factor, as well as new acquisition functions with improved scalability, combined with an informed acquisition function selection mechanism. By comparing the performance of our Bayesian Optimization implementation on various test cases to the existing search strategies in Kernel Tuner, as well as other Bayesian Optimization implementations, we demonstrate that our search strategies generalize well and consistently outperform other search strategies by a wide margin.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2024 hgpu.org

All rights belong to the respective authors

Contact us: