high performance computing on graphics processing units: hgpu.org

CUDA is the software stack through which nearly all GPU computing passes, and most of it is closed: the compiler, the driver library and the mathematics libraries are proprietary, and the machine code of the GPU itself (SASS) is undocumented. This paper reports a clean-room replacement of that stack on a single machine, an NVIDIA DGX Spark (Grace Blackwell GB10, compute capability sm_121). The replacement, called CUDA++, was built by one engineer directing AI coding agents, and it uses no NVIDIA user-space software at run time: no CUDA toolkit, no nvcc, no PTX, no libcuda, no cuBLAS. The work proceeded in three parts. First, a driver client of about 900 lines was written against the public headers of NVIDIA's open Linux kernel module, reproducing the object, memory, channel and submission sequence that libcuda performs, over the module's ioctl interface. Second, the undocumented instruction encoding was recovered from three sources: NVIDIA's own compiler used as an offline oracle on small probe kernels written for the purpose, the open-source Mesa NAK compiler, and the chip itself, with candidate arithmetic models fitted until one matched every output bit for bit. Third, kernel launches were assembled as command lists in the order the open Mesa NVK driver uses, with the launch descriptor (QMD) built directly. Every stage was verified on the hardware, with host tests, mutation tests and signed chip receipts recorded at named commits. Results include: a first native kernel on 2026-09-26; closure of the numerics suite on the chip on 2026-10-02 (reductions, nine transcendental functions evaluated over all 2^32 inputs each, 148 load and store kernels); measured ceilings of 115.14 TFLOPS tensor-core throughput and 195.1 GB/s memory reads by the toolchain's own probes; on 2026-10-21, a 35-billion-parameter mixture-of-experts model served on this code with zero CUDA libraries mapped in the process, its output identical to an exact host reference; and on 2026-10-22, a public chip receipt in which a test kernel produced 1,048,576 of 1,048,576 correct values, its output hash matching the host simulator. The paper states its boundaries as carefully as its results: NVIDIA's open kernel module remains in the path (a replacement driver is in progress), the GPU's GSP firmware remains NVIDIA's because the chip verifies its signature, and a published qualification queue lists what has not yet been proven. The toolchain is released under the MIT licence with its full evidence trail, so every claim in this paper can be re-run.