high performance computing on graphics processing units: hgpu.org

There is a growing interest in the computer architecture community to incorporate heterogeneity and specialization to improve performance. Developers can write heterogeneous applications that consist of host code and kernel code, where compute-intensive kernels can be offloaded from CPU to GPU, FPGA, or quantum computer. However, the high complexity of these systems can pose challenges to developer productivity, particularly when it comes to understanding performance or ensuring correctness. Testing and debugging such heterogeneous applications and software stacks are also extremely challenging because the characteristics of the hardware in heterogeneous computing can vary from a traditional CPU. For example, the output of a quantum computer is completely different from a CPU. By leveraging hardware accelerator capability and by accounting for hardware accelerator behavior, this thesis presents efficient debugging and testing techniques for heterogeneous computing. My first work QDiff is a differential testing framework for quantum software stacks to find unexpected behavior, such as crashes or unexpected divergences. By accounting for unique characteristics of quantum circuit execution and measurement, QDiff automatically finds abnormalities in quantum software stacks by detecting quantum circuit result divergence. QDiff proposes an input generation approach that applies gate-level equivalent transformation and explores the backends and compiler setting options to generate semantic equivalent quantum circuits. We then compare the measurement results of circuits with K-S distance and report unexpected results. For Cirq, Pyquil, and Qiskit, we found four new bugs in their simulators and two possible root causes for hardware execution divergence. While QDiff targets the correctness of quantum software stacks, bugs in heterogeneous applications are another concern in this domain. My second work HFuzz designs a fuzz testing technique for heterogeneous applications. By designing hardware-level probes and offloading input mutations to hardware accelerators, HFuzz leverages in-kernel probes to retrieve hardware execution feedback, including channel usage and in-kernel variable’s value range. HFuzz also uses FPGA to accelerate input mutations during fuzzing process. We conduct detailed experiments on seven benchmarks and demonstrate that HFuzz significantly improves the fuzzing efficiency and reveals 25 unique and unexpected behavior symptoms that could not be found by state-of-the-art testing techniques. QDiff and HFuzz together provide automated testing approaches for heterogeneous computing. However, developers must be able to diagnose the detected errors. Compilation for heterogeneous applications is inherently complex. For example, CIRCT, an MLIR-based heterogeneous compiler, requires 13 compilation layers to translate high-level Python code into low-level RTL. Identifying which layers and which parts of the intermediate representation (IR) contribute to bugs is difficult. To tackle this issue, my third work, DuoReduce, introduces a novel dual-dimensional error localization approach. DuoReduce systematically analyzes errors across both IR code dimension and compilation path dimension within the compilation process. By combining delta debugging techniques with dependency-aware compilation path reduction, DuoReduce identifies the minimal subset of IR code and compilation passes responsible for triggering an error. Through evaluation on real-world scenarios, DuoReduce demonstrated significant improvements in debugging accuracy and efficiency, accelerating error localization by 901x compared to traditional techniques.