high performance computing on graphics processing units: hgpu.org

hgpu.org » nVidia A100

MSCCL++: Rethinking GPU Communication Abstractions for Cutting-edge AI Applications

Aashaka Shah, Abhinav Jangda, Binyang Li, Caio Rocha, Changho Hwang, Jithin Jose, Madan Musuvathi, Olli Saarikivi, Peng Cheng, Qinghua Zhou, Roshan Dathathri, Saeed Maleki, Ziyue Yang

View

Download (PDF)

Source codes

Tags: AI, AMD Radeon Instinct MI300X, ATI, Computer science, CUDA, Heterogeneous systems, HIP, nVidia, nVidia A100, nVidia H100, Package

April 27, 2025 by hgpu

Data-efficient LLM Fine-tuning for Code Generation

Weijie Lv, Xuan Xia, Sheng-Jun Huang

View

Download (PDF)

Source codes

Tags: Code generation, Computer science, CUDA, LLM, nVidia, nVidia A100, Package, Python, PyTorch

April 27, 2025 by hgpu

LithOS: An Operating System for Efficient Machine Learning on GPUs

Patrick H. Coppock, Brian Zhang, Eliot H. Solomon, Vasilis Kypriotis, Leon Yang, Bikash Sharma, Dan Schatzberg, Todd C. Mowry, Dimitrios Skarlatos

View

Download (PDF)

Tags: Computer science, CUDA, Machine learning, nVidia, nVidia A100, nVidia H100, Operating systems

April 27, 2025 by hgpu

Scalability Evaluation of HPC Multi-GPU Training for ECG-based LLMs

Dimitar Mileski, Nikola Petrovski, Marjan Gusev

View

Download (PDF)

Tags: Computer science, CUDA, Deep learning, HPC, LLM, nVidia, nVidia A100, PTX, PyTorch

April 13, 2025 by hgpu

The Shamrock code: I- Smoothed Particle Hydrodynamics on GPUs

Timothée David--Cléris, Guillaume Laibe, Yona Lapeyre

View

Download (PDF)

Source codes

Tags: AMD, AMD Radeon Instinct MI250X, Astrophysics, CUDA, MPI, nVidia, nVidia A100, OpenMP, Package, Physics, PTX, ROCm, SYCL

March 23, 2025 by hgpu

Concurrent Scheduling of High-Level Parallel Programs on Multi-GPU Systems

Fabian Knorr, Philip Salzmann, Peter Thoman, Thomas Fahringer

View

Download (PDF)

Source codes

Tags: Benchmarking, Computer science, HPC, nVidia, nVidia A100, Package, performance portability, SYCL, Task scheduling

March 23, 2025 by hgpu

A Microbenchmark Framework for Performance Evaluation of OpenMP Target Offloading

Mohammad Atif, Tianle Wang, Zhihua Dong, Charles Leggett, Meifeng Lin

View

Download (PDF)

Source codes

Tags: Benchmarking, Computer science, Heterogeneous systems, nVidia A100, nVidia RTX A6000, nVidia V100, OpenMP, Package

March 10, 2025 by hgpu

SUperman: Efficient Permanent Computation on GPUs

Deniz Elbek, Fatih Taşyaran, Bora Uçar, Kamer Kaya

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, HPC, Numerical Analysis, nVidia, nVidia A100, nVidia Quadro GV100, OpenMPI, Package

March 10, 2025 by hgpu

Can Tensor Cores Benefit Memory-Bound Kernels? (No!)

Lingqi Zhang, Jiajun Huang, Sheng Di, Satoshi Matsuoka, Mohamed Wahib

View

Download (PDF)

Tags: Computer science, CUDA, nVidia, nVidia A100, Performance

March 10, 2025 by hgpu

pyATF: Constraint-Based Auto-Tuning in Python

Richard Schulze, Sergei Gorlatch, Ari Rasch

View

Download (PDF)

Source codes

Tags: Auto-Tuning, Compilers, Computer science, CUDA, nVidia A100, OpenCL, Package, Performance, Python

March 3, 2025 by hgpu

TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators

Jianling Li, Shangzhan Li, Zhenye Gao, Qi Shi, Yuxuan Li, Zefan Wang, Jiacheng Huang, Haojie Wang, Jianrong Wang, Xu Han, Zhiyuan Liu, Maosong Sun

View

Download (PDF)

Source codes

Tags: Benchmarking, Code generation, Computer science, CUDA, Deep learning, LLM, nVidia, nVidia A100, Package, Python

March 3, 2025 by hgpu

CRIUgpu: Transparent Checkpointing of GPU-Accelerated Workloads

Radostin Stoyanov, Viktória Spišaková, Jesus Ramos, Steven Gurfinkel, Andrei Vagin, Adrian Reber, Wesley Armour, Rodrigo Bruno

View

Download (PDF)

Source codes

Tags: AMD Radeon Instinct MI210, ATI, Computer science, CUDA, Deep learning, nVidia, nVidia A100, nVidia H100, nVidia RTX A6000, Package, ROCm

March 3, 2025 by hgpu

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training

Large Language Model Powered C-to-CUDA Code Translation: A Novel Auto-Parallelization Framework

GigaAPI: a user-space API that simplifies multi-GPU programming, bridging the gap between the capabilities of parallel GPU systems and the ability of developers to harness their full potential

GigaAPI for GPU Parallelization

Coccinelle: a C code transformation engine using SmPL for matches, refactorings, and bug fixing

Advances in Semantic Patching for HPC-oriented Refactorings with Coccinelle

DuoReduce: MLIR's benchmark

Hardware-Assisted Software Testing and Debugging for Heterogeneous Computing

See all packages

* * *

high performance computing on graphics processing units: hgpu.org

MSCCL++: Rethinking GPU Communication Abstractions for Cutting-edge AI Applications

Data-efficient LLM Fine-tuning for Code Generation

LithOS: An Operating System for Efficient Machine Learning on GPUs

Scalability Evaluation of HPC Multi-GPU Training for ECG-based LLMs

The Shamrock code: I- Smoothed Particle Hydrodynamics on GPUs

Concurrent Scheduling of High-Level Parallel Programs on Multi-GPU Systems

A Microbenchmark Framework for Performance Evaluation of OpenMP Target Offloading

SUperman: Efficient Permanent Computation on GPUs

Can Tensor Cores Benefit Memory-Bound Kernels? (No!)

pyATF: Constraint-Based Auto-Tuning in Python

TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators

CRIUgpu: Transparent Checkpointing of GPU-Accelerated Workloads

Recent source codes

PELSI: Power-Efficient Layer-Switched Inference

Ouroboros: Virtualized Queues for dynamic memory management

MSCCL++: A GPU-driven communication stack for scalable AI applications

Benchmark compute shader of Unity against InteropUnityCUDA

Data-efficient LLM Fine-tuning for Code Generation

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Large Language Model Powered C-to-CUDA Code Translation: A Novel Auto-Parallelization Framework

GigaAPI: a user-space API that simplifies multi-GPU programming, bridging the gap between the capabilities of parallel GPU systems and the ability of developers to harness their full potential

Coccinelle: a C code transformation engine using SmPL for matches, refactorings, and bug fixing

DuoReduce: MLIR's benchmark

Most viewed papers (last 30 days)