high performance computing on graphics processing units: hgpu.org

Trending applications such as AI and data analytics have mandated the use of GPUs in modern datacenters for performance reasons. Current practice dictates to dedicate GPUs to applications, which limits the amount of concurrent users to the available GPUs. That use of GPUs contradicts with the policy of datacenters to oversubscribe resources and accommodate as many user applications as possible. To address this issue, providers will inevitably resort to GPU sharing. In this work we introduce Syntix, a mechanism that we deploy on GPU sharing system and 1) profiles CUDA kernels in order to learn their resource requirements in terms of threads and blocks and 2) assigns those resources to kernels in order to be efficiently collocated into streams. Syntix is able to exploit the resources that are possibly wasted from the execution of an individual kernel and save the 80% of them on average.