high performance computing on graphics processing units: hgpu.org

Yao-Yuan Yang, Moto Hira, Zhaoheng Ni, Anjali Chourdia, Artyom Astafurov, Caroline Chen, Ching-Feng Yeh, Christian Puhrsch, David Pollack, Dmitriy Genzel, Donny Greenberg, Edward Z. Yang, Jason Lian, Jay Mahadeokar, Jeff Hwang, Ji Chen, Peter Goldsborough, Prabhat Roy, Sean Narenthiran, Shinji Watanabe, Soumith Chintala, Vincent Quenneville-Bélair, Yangyang Shi

View

Download (PDF)

Source codes

Tags: Computer science, Deep learning, Machine learning, nVidia, nVidia A100, Package, Python, PyTorch, Speech recognition

October 31, 2021 by hgpu

Bolt: Bridging the Gap between Auto-tuners and Hardware-native Performance

Jiarong Xing, Leyuan Wang, Shang Zhang, Jack Chen, Ang Chen, Yibo Zhu

View

Download (PDF)

Tags: Auto-Tuning, Computer science, CUDA, Deep learning, Neural networks, nVidia, Performance, Tesla T4

October 31, 2021 by hgpu

The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding

Pratik Fegade, Tianqi Chen, Phillip B. Gibbons, Todd C. Mowry

View

Download (PDF)

Tags: Compilers, Computer science, CUDA, Deep learning, nVidia, Tesla V100

October 24, 2021 by hgpu

high performance computing on graphics processing units: hgpu.org

Reveal training performance mystery between TensorFlow and PyTorch in the single GPU environment

CitiusSynapse: A Deep Learning Framework for Embedded Systems

Manas: Mining Software Repositories to Assist AutoML

Analysis and Comparison of Performance and Power Consumption of Neural Networks on CPU, GPU, TPU and FPGA

HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments

QGTC: Accelerating Quantized GNN via GPU Tensor Core

MQBench: Towards Reproducible and Deployable Model Quantization Benchmark

Collage: Automated Integration of Deep Learning Backends

Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training

TorchAudio: Building Blocks for Audio and Speech Processing

Bolt: Bridging the Gap between Auto-tuners and Hardware-native Performance

The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding

Recent source codes

Coccinelle: a C code transformation engine using SmPL for matches, refactorings, and bug fixing

DuoReduce: MLIR's benchmark

Shamrock: Multi-GPU hydrodynamics for astrophysics

LLMPerf: GPU Performance Modeling meets Large Language Models

Hercules: A Compiler for Productive Programming of Heterogeneous Systems

Celerity Runtime: High-level C++ for Accelerator Clusters

wgpy: WebGL accelerated numpy-compatible array library for web browser

Microbenchmarking OpenMP target offload with Catch2

SUperman: Highly Efficient Permanent Computation Library

TransCL: An Automatic CUDA-to-OpenCL Programs Transformation Framework

Most viewed papers (last 30 days)