dMath: Distributed Linear Algebra for DL

Steven Eliuk, Cameron Upright, Hars Vardhan, Stephen Walsh, Trevor Gale
Samsung Electronics, Computing Science Innovation Center, SRA-SV, 665 Clyde Avenue, Mountain View, CA 94043
arXiv:1611.07819 [cs.DC], (19 Nov 2016)


   title={dMath: Distributed Linear Algebra for DL},

   author={Eliuk, Steven and Upright, Cameron and Vardhan, Hars and Walsh, Stephen and Gale, Trevor},






Download Download (PDF)   View View   Source Source   



The paper presents a parallel math library, dMath, that demonstrates leading scaling when using intranode, internode, and hybrid-parallelism for deep learning (DL). dMath provides easy-to-use distributed primitives and a variety of domain-specific algorithms including matrix multiplication, convolutions, and others allowing for rapid development of scalable applications like deep neural networks (DNNs). Persistent data stored in GPU memory and advanced memory management techniques avoid costly transfers between host and device. dMath delivers performance, portability, and productivity to its specific domain of support.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2021 hgpu.org

All rights belong to the respective authors

Contact us: