high performance computing on graphics processing units: hgpu.org

While Field-Programmable Gate Arrays (FPGAs) exist in many design configurations throughout the data center, cloud, and edge, the promise of performance and flexibility offered by the FPGA often remains unrealized for lack of hardware design expertise, with most computation remaining in fixed hardware such as CPUs, GPUs, and ASICs e.g. tensor processors. Identifying programmability as a barrier to FPGA usage, we seek to augment High Level Synthesis (HLS) design flows with machine learning. The overall goal of this dissertation is to advance the art of using unmodified high-level language (HLL) programs to create FPGA configurations that are performant, programmable, and portable. The problems in using HLL code to program FPGAs arise from the serial execution model of the target application codes, in particular, the mismatch between that model and the arbitrary data flow model of the target hardware. However, a variety of code transformation techniques, tedious to perform by a human but readily and effortlessly done by CPU compilers, allow many compute-intensive operations to be transformed into a form that is highly or massively parallel. A challenge then exists in selecting the best set of optimizations and an order in which to perform them, a choice among staggeringly many options. Brute-force and automated orthogonal search techniques have failed to produce solutions that lie within the realm of practicality. We evaluate the suitability of machine learning (ML) models to address this challenge. We develop and assess designs for systems that use ML and present feedback to these models based on their recommendations. In support of using ML models, we begin by developing and assessing methods for preparing the original program as input for processing by the model. We next develop and assess methods to apply the model’s output to a compilation system and evaluate the results for use as feedback. Machine learning experiments are performed to demonstrate their potential. Feasibility is studied through simulations and specialized evaluation techniques as appropriate. Specific new artifacts are created, including extensions to the open-source GCC compiler. The significance of this work is of several types: the many options and variants to the overall design that are considered, with their distinct advantages and disadvantages; the application of proper units and baselines to experimental measurements; and the numerous contributions in the form of published extensions and improvements to open-source projects.