28679

Reverse-Mode AD of Reduce-by-Index and Scan in Futhark

Lotte Maria Bruun, Ulrik Stuhr Larsen, Nikolaj Hinnerskov, Cosmin Oancea
University of Copenhagen, Denmark
arXiv:2310.03568 [cs.PL], (5 Oct 2023)

@misc{bruun2023reversemode,

   title={Reverse-Mode AD of Reduce-by-Index and Scan in Futhark},

   author={Lotte Maria Bruun and Ulrik Stuhr Larsen and Nikolaj Hinnerskov and Cosmin Oancea},

   year={2023},

   eprint={2310.03568},

   archivePrefix={arXiv},

   primaryClass={cs.PL}

}

Download Download (PDF)   View View   Source Source   

571

views

We present and evaluate the Futhark implementation of reverse-mode automatic differentiation (AD) for the basic blocks of parallel programming: reduce, prefix sum (scan), and reduce by index. We first present derivations of general-case algorithms and then discuss several specializations that result in efficient differentiation of most cases of practical interest. We report an experiment that evaluates the performance of the differentiated code in the context of GPU execution and highlights the impact of the proposed specializations as well as the strengths and weaknesses of differentiating at high level vs. low level (i.e., “differentiating the memory”).
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2024 hgpu.org

All rights belong to the respective authors

Contact us: