high performance computing on graphics processing units: hgpu.org

Applications

hgpu.org » paper

Demystifying NCCL: An In-depth Analysis of GPU Communication Protocols and Algorithms

Zhiyi Hu, Siyuan Shen, Tommaso Bonato, Sylvain Jeaugey, Cedell Alexander, Eric Spada, Jeff Hammond, Torsten Hoefler

View

Download (PDF)

Tags: Computer science, CUDA, GPU cluster, Network communication, nVidia, nVidia GH200

July 13, 2025 by hgpu

Serving LLMs in HPC Clusters: A Comparative Study of Qualcomm Cloud AI 100 Ultra and High-Performance GPUs

Mohammad Firas Sada, John J. Graham, Elham E Khoda, Mahidhar Tatineni, Dmitry Mishin, Rajesh K. Gupta, Rick Wagner, Larry Smarr, Thomas A. DeFanti, Frank Würthwein

View

Download (PDF)

Tags: AMD Radeon Instinct MI300A, ATI, Cloud, Computer science, Deep learning, HPC, nVidia, nVidia A100, nVidia H200

July 13, 2025 by hgpu

Hardware Compute Partitioning on NVIDIA GPUs for Composable Systems

Joshua Bakita, James H. Anderson

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, nVidia, nVidia GeForce RTX 4090, nVidia RTX 6000 Ada, Package, Task scheduling

July 13, 2025 by hgpu

Efficient GPU Implementation of Multi-Precision Integer Division

Aske N. Raahauge, Martin B. Marchioro, Marc I. Løvenskjold

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Extended precision, Futhark, nVidia, nVidia A100, Package, Thesis

July 6, 2025 by hgpu

P4OMP: Retrieval-Augmented Prompting for OpenMP Parallelism in Serial Code

Wali Mohammad Abdullah, Azmain Kabir

View

Download (PDF)

Tags: Code generation, Computer science, HPC, LLM, OpenMP, Software Engineering

July 6, 2025 by hgpu

ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-level HPC Translation Tasks

Joshua H. Davis, Daniel Nichols, Ishan Khillan, Abhinav Bhatele

View

Download (PDF)

Source codes

Tags: Benchmarking, Code generation, Computer science, CUDA, LLM, nVidia, nVidia A100, OpenMP, Package

July 6, 2025 by hgpu

Libra: Synergizing CUDA and Tensor Cores for High-Performance Sparse Matrix Multiplication

Jinliang Shi, Shigang Li, Youxuan Xu, Xueying Wang, Rongtian Fu, Zhi Ma, Tong Wu

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Deep learning, Heterogeneous systems, Matrix multiplication, nVidia, nVidia GeForce RTX 4090, nVidia H100, Package, Sparse matrix

July 6, 2025 by hgpu

Accelerated discovery and design of Fe-Co-Zr magnets with tunable magnetic anisotropy through machine learning and parallel computing

Weiyi Xia, Maxim Moraru, Ying Wai Li, Timothy Liao, James R. Chelikowsky, Cai-Zhuang Wang

View

Download (PDF)

Source codes

Tags: Anisotropy, Computational Physics, Condensed matter, CUDA, Machine learning, Materials Science, Neural networks, nVidia, nVidia A100, Package, Physics

July 6, 2025 by hgpu

GCStack+GCScaler: Fast and Accurate GPU Performance Analyses Using Fine-Grained Stall Cycle Accounting and Interval Analysis

Hanna Cha, Sungchul Lee, Jounghoo Lee, Yeonan Ha, Joonsung Kim, Youngsok Kim

View

Download (PDF)

Tags: Benchmarking, Computer science, CUDA, Design space exploration, nVidia, nVidia GeForce RTX 3070, Performance

June 29, 2025 by hgpu

No More Shading Languages: Compiling C++ to Vulkan Shaders

H. Devillers, M. Kurtenacker, R. Membarth, S. Lemme, M. Kenzel, Ö. Yazici, P. Slusallek

View

Download (PDF)

Source codes

Tags: 3D Graphics and Realism, AMD Radeon RX 7900 XT, ATI, Compilers, GLSL, nVidia, nVidia GeForce RTX 3050 Ti, nVidia GeForce RTX 4070, Package, Raytracing, Rendering, Vulkan

June 29, 2025 by hgpu

WiLLM: An Open Wireless LLM Communication System

Boyi Liu, Yongguang Lu, Jianguo Zhao, Qiang Yang, Wen Wu, Lin Chen, Jagmohan Chauhan, Jun Zhang

View

Download (PDF)

Source codes

Tags: Computer science, LLM, Network communications, nVidia, nVidia GeForce RTX 4090, Package

June 29, 2025 by hgpu

Survey of HPC in US Research Institutions

Peng Shu, Junhao Chen, Zhengliang Liu, Huaqin Zhao, Xinliang Li, Tianming Liu

View

Download (PDF)

Tags: AI, Computer science, HPC, survey

June 29, 2025 by hgpu

MegaTrain: Full Precision Training of 100B+ Parameter Large Language Models on a Single GPU

Device Virtual Machine (DVM)

DVM: Real-Time Kernel Generation for Dynamic AI Models

Agentic Code Optimization via Compiler-LLM Cooperation

AutoKernel: Autoresearch for GPU kernels. Give it any PyTorch model, go to sleep, wake up to optimized Triton kernels

AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Search

Triton-Sanitizer: A Fast and Device-Agnostic Memory Sanitizer for Triton with Rich Diagnostic Context

LLM.Q: Quantized LLM training in pure CUDA/C++

LLMQ: Efficient Lower-Precision LLM Training for Consumer GPUs

SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limits

* * *

high performance computing on graphics processing units: hgpu.org

Applications

Demystifying NCCL: An In-depth Analysis of GPU Communication Protocols and Algorithms

Hardware Compute Partitioning on NVIDIA GPUs for Composable Systems

Efficient GPU Implementation of Multi-Precision Integer Division

P4OMP: Retrieval-Augmented Prompting for OpenMP Parallelism in Serial Code

ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-level HPC Translation Tasks

Libra: Synergizing CUDA and Tensor Cores for High-Performance Sparse Matrix Multiplication

Accelerated discovery and design of Fe-Co-Zr magnets with tunable magnetic anisotropy through machine learning and parallel computing

GCStack+GCScaler: Fast and Accurate GPU Performance Analyses Using Fine-Grained Stall Cycle Accounting and Interval Analysis

No More Shading Languages: Compiling C++ to Vulkan Shaders

WiLLM: An Open Wireless LLM Communication System

Survey of HPC in US Research Institutions

Recent source codes

MegaTrain: Full Precision Training of 100B+ Parameter Large Language Models on a Single GPU

Device Virtual Machine (DVM)

Agentic Code Optimization via Compiler-LLM Cooperation

AutoKernel: Autoresearch for GPU kernels. Give it any PyTorch model, go to sleep, wake up to optimized Triton kernels

Triton-Sanitizer: A Fast and Device-Agnostic Memory Sanitizer for Triton with Rich Diagnostic Context

LLM.Q: Quantized LLM training in pure CUDA/C++

SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limits

True 4-Bit Quantized CNN Training on CPU

cuFuzz: A GPU-oriented coverage-guided fuzzer for userland CUDA application

KernelSkill: A Multi-Agent Framework for GPU Kernel Optimization

Most viewed papers (last 30 days)