high performance computing on graphics processing units: hgpu.org

hgpu.org » CUDA

HiCCL: A Hierarchical Collective Communication Library

Mert Hidayetoglu, Simon Garcia de Gonzalo, Elliott Slaughter, Pinku Surana, Wen-mei Hwu, William Gropp, Alex Aiken

View

Download (PDF)

Source codes

Tags: AMD Radeon Instinct MI250X, ATI, Computer science, CUDA, HIP, nVidia, nVidia A100, Package, Performance, SYCL

August 18, 2024 by hgpu

Anatomizing Deep Learning Inference in Web Browsers

Qipeng Wang, Shiqi Jiang, Zhenpeng Chen, Xu Cao, Yuanchun Li, Aoyu Li, Yun Ma, Ting Cao, Xuanzhe Liu

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Deep learning, nVidia, nVidia GeForce GTX 1060, nVidia GeForce GTX 980, nVidia GeForce RTX 2060, OpenCL, Package, Performance, TensorFlow

August 18, 2024 by hgpu

In-Situ Techniques on GPU-Accelerated Data-Intensive Applications

Yi Ju, Mingshuai Li, Adalberto Perez, Laura Bellentani, Niclas Jansson, Stefano Markidis, Philipp Schlatter, Erwin Laure

View

Download (PDF)

Tags: Computer science, CUDA, Fluid dynamics, HPC, Molecular dynamics, nVidia, nVidia A100, PC cluster, Performance

August 14, 2024 by hgpu

Evaluating Operators in Deep Neural Networks for Improving Performance Portability of SYCL

Zheming Jin

View

Download (PDF)

Source codes

Tags: Benchmarking, Computer science, CUDA, HIP, Machine learning, Neural networks, nVidia, nVidia GeForce RTX 2080, nVidia GeForce RTX 3090, nVidia H100, oneAPI, Performance, performance portability, SYCL, Tesla A100, Tesla V100

August 14, 2024 by hgpu

LO-SpMM: Low-cost Search for High-performance SpMM Kernels on GPUs

Junqing Lin, Jingwei Sun, Xiaolong Shi, Honghe Zhang, Xianzhi Yu, Xinzhi Wang, Jun Yao, Guangzhong Sun

View

Download (PDF)

Tags: Compilers, Computer science, CUDA, Deep learning, Linear Algebra, Matrix multiplication, Neural networks, nVidia, nVidia GeForce RTX 2080 Ti, Performance, Sparse matrix, Tesla V100

August 4, 2024 by hgpu

Springald: GPU-Accelerated Window-Based Aggregates Over Out-of-Order Data Streams

Gabriele Mencagli, Patrizio Dazzi, Massimo Coppola

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, DSP, nVidia, nVidia A30, Package, Stream Computing

August 4, 2024 by hgpu

Data-driven Forecasting of Deep Learning Performance on GPUs

Seonho Lee, Amar Phanishayee, Divya Mahajan

View

Download (PDF)

Tags: Computer science, CUDA, Deep learning, nVidia, nVidia A100, nVidia H100, nVidia P100, nVidia V100, Performance, PyTorch, Tesla T4

August 4, 2024 by hgpu

Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing

Yizhou Luo, Qiang Wang, Shaohuai Shi, Jiaxin Lai, Shuhan Qi, Jiajia Zhang, Xuan Wang

View

Download (PDF)

Tags: Computer science, CUDA, Deep learning, GPU cluster, nVidia, nVidia GeForce RTX 2080 Ti, OpenMPI, PyTorch, Task scheduling

August 4, 2024 by hgpu

A Comparison of OpenCL, CUDA, and HIP as Compilation Targets for a Functional Array Language

Troels Henriksen

View

Download (PDF)

Source codes

Tags: AMD Radeon Instinct MI100, ATI, Code generation, Computer science, CUDA, HIP, nVidia, nVidia A100, OpenCL, Package, Programming Languages

July 28, 2024 by hgpu

Bringing Auto-tuning to HIP: Analysis of Tuning Impact and Difficulty on AMD and Nvidia GPUs

Milo Lurati, Stijn Heldens, Alessio Sclocco, Ben van Werkhoven

View

Download (PDF)

Source codes

Tags: AMD Radeon Instinct MI250X, AMD Radeon Pro W6600, ATI, Computer science, CUDA, HIP, nVidia, nVidia A100, nVidia RTX A4000, Package, Performance, Python

July 28, 2024 by hgpu

Efficiently Training 7B LLM with 1 Million Sequence Length on 8 GPUs

Pinxue Zhao, Hailin Zhang, Fangcheng Fu, Xiaonan Nie, Qibin Liu, Fang Yang, Yuanbo Peng, Dian Jiao, Shuaipeng Li, Jinbao Xue, Yangyu Tao, Bin Cui

View

Download (PDF)

Tags: AI, Computer science, CUDA, LLM, Machine learning, NLP, nVidia, nVidia A800

July 28, 2024 by hgpu

RBMD: A molecular dynamics package enabling to simulate 10 million all-atom particles in a single graphics processing unit

Weihang Gao, Teng Zhao, Yongfa Guo, Jiuyang Liang, Huan Liu, Maoying Luo, Zedong Luo, Wei Qin, Yichao Wang, Qi Zhou, Shi Jin, Zhenli Xu

View

Download (PDF)

Source codes

Tags: Computational Physics, CUDA, Heterogeneous systems, Molecular dynamics, nVidia, nVidia GeForce RTX 4090, Package, Physics, Tesla A100, Tesla V100

July 28, 2024 by hgpu

* * *

high performance computing on graphics processing units: hgpu.org

HiCCL: A Hierarchical Collective Communication Library

Anatomizing Deep Learning Inference in Web Browsers

In-Situ Techniques on GPU-Accelerated Data-Intensive Applications

Evaluating Operators in Deep Neural Networks for Improving Performance Portability of SYCL

LO-SpMM: Low-cost Search for High-performance SpMM Kernels on GPUs

Springald: GPU-Accelerated Window-Based Aggregates Over Out-of-Order Data Streams

Data-driven Forecasting of Deep Learning Performance on GPUs

Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing

A Comparison of OpenCL, CUDA, and HIP as Compilation Targets for a Functional Array Language

Bringing Auto-tuning to HIP: Analysis of Tuning Impact and Difficulty on AMD and Nvidia GPUs

Efficiently Training 7B LLM with 1 Million Sequence Length on 8 GPUs

RBMD: A molecular dynamics package enabling to simulate 10 million all-atom particles in a single graphics processing unit

Recent source codes

Kokkidio: Kokkos+Eigen - write fast, readable multi-platform code

LLaMA-Inference-Bench

MATAR

Over-synchronization in GPU Programs

profile_util: simple C++ library to report memory, timing, thread affinity

MaC: Mamba CPU performance predictor

Mixed-precision finite element kernels

SHMT: The Simultaneous and Heterogenous Multithreading project

Superpipeline: A Universal Approach for Reducing GPU Memory Usage in Large Models

EnergyUCB-Bandit

Most viewed papers (last 30 days)