19010

MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing

Daniel Nichols, Nathalie-Sofia Tomov, Frank Betancourt, Stanimire Tomov, Kwai Wong, Jack Dongarra
University of Tennessee, Knoxville TN 37996, USA
Practice and Experience in Advanced Research Computing (PEARC ’19), 2019

@article{nichols2019magmadnn,

   title={MagmaDNN: Towards High-Performance Data Analytics and Machine Learning for Data-Driven Scientific Computing},

   author={Nichols, Daniel and Tomov, Nathalie-Sofia and Betancourt, Frank and Tomov, Stanimire and Wong, Kwai and Dongarra, Jack},

   journal={MAGMA},

   volume={60},

   number={80},

   pages={100},

   year={2019}

}

Download Download (PDF)   View View   Source Source   Source codes Source codes

Package:

506

views

In this paper, we present work towards the development of a new data analytics and machine learning (ML) framework, called MagmaDNN. Our main goal is to provide scalable, high-performance data analytics and ML solutions for scientific applications running on current and upcoming heterogeneous many-core GPU-accelerated architectures. To this end, since many of the functionalities needed are based on standard linear algebra (LA) routines, we designed MagmaDNN to derive its performance power from the MAGMA library. The close integration provides the fundamental (scalable high-performance) LA routines available in MAGMA as a backend to MagmaDNN. We present some design issues for performance and scalability that are specific to ML using Deep Neural Networks (DNN), as well as the MagmaDNN designs towards overcoming them. In particular, MagmaDNN uses well established HPC techniques from the area of dense LA, including task-based parallelization, DAG representations, scheduling, mixed-precision algorithms, asynchronous solvers, and autotuned hyperparameter optimization. We illustrate these techniques and their incorporation and use to outperform other frameworks, currently available.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2019 hgpu.org

All rights belong to the respective authors

Contact us: