25076

TENSILE: A Tensor granularity dynamic GPU memory scheduler method towards multiple dynamic workloads system

Kaixin Zhang, Hongzhi Wang, Tongxin Li, Han Hu, Jiye Qiu, Songling Zou
Harbin Institute of Technology, Harbin, China
arXiv:2105.13336 [cs.DC], (27 May 2021)

@misc{zhang2021tensile,

   title={TENSILE: A Tensor granularity dynamic GPU memory scheduler method towards multiple dynamic workloads system},

   author={Kaixin Zhang and Hongzhi Wang and Tongxin Li and Han Hu and Jiye Qiu and Songling Zou},

   year={2021},

   eprint={2105.13336},

   archivePrefix={arXiv},

   primaryClass={cs.DC}

}

Download Download (PDF)   View View   Source Source   

536

views

Recently, deep learning has been an area of intense researching. However, as a kind of computing intensive task, deep learning highly relies on the the scale of the GPU memory, which is usually expensive and scarce. Although there are some extensive works have been proposed for dynamic GPU memory management, they are hard to be applied to systems with multitasking dynamic workloads, such as in-database machine learning system. In this paper, we demonstrated TENSILE, a method of managing GPU memory in tensor granularity to reduce the GPU memory peak, with taking the multitasking dynamic workloads into consideration. As far as we know, TENSILE is the first method which is designed to manage multiple workloads’ GPU memory using. We implement TENSILE on our own deep learning framework, and evaluated its performance. The experiment results shows that our method can achieve less time overhead than prior works with more GPU memory saved.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2021 hgpu.org

All rights belong to the respective authors

Contact us: