high performance computing on graphics processing units: hgpu.org

Tuning codes to make efficient use of high-performance computing systems is known to be hard. Programmers have to schedule their computations to thousands of compute cores having the compute and data movement costs in mind. The necessary code transformations – for example, to overlap computation and inter-node communication – are well known. But the complex interplay of hardware and software often prevents programmers from identifying performance bottlenecks and selecting good code transformations. This dissertation introduces compilation frameworks, performance tools, and programming models to tackle these programmability challenges. Over the last decades, the compute performance improved at a much faster pace than memory performance. Data-movement optimizations to reduce the communication and memory access costs thus became much more pressing. We address the problem by automating the selection of datalocality transformations (absinthe) and by adapting the programming model (dcuda) to overlap computation and inter-node communication automatically. The performance models needed to automate the tuning (absinthe and haystack) also provide the programmers with valuable insights when optimizing codes manually. An important algorithmic motif in high-performance computing is the sequential execution of multiple but different stencils. Our compilation framework (absinthe) automates the selection of data-locality transformations for such stencil programs. It has three main components: 1) a transformation algebra, 2) a performance model, and 3) an optimizer. The transformation algebra (modesto) defines the space of possible code transformations and the learned performance model (absinthe) guides the selection of good code transformations. In summary, this dissertation contributes compilation frameworks, performance tools, and programming models to foster the application of data movement optimizations in high-performance computing. In particular, we automate the selection of data-locality transformations for stencil programs. We believe our work lays the foundation for future compilation frameworks that support even broader application domains.