high performance computing on graphics processing units: hgpu.org

Kshiteej Mahajan, Arjun Singhvi, Arjun Balasubramanian, Varun Batra, Surya Teja Chavali, Shivaram Venkataraman, Aditya Akella, Amar Phanishayee, Shuchi Chawla

View

Download (PDF)

Tags: Computer science, GPU cluster, Machine learning, nVidia, Task scheduling, Tesla K80, Tesla M60

July 4, 2019 by hgpu

Optimizing Network Performance for Distributed DNN Training on GPU Clusters: ImageNet/AlexNet Training in 1.5 Minutes

Peng Sun, Wansen Feng, Ruobing Han, Shengen Yan, Yonggang Wen

View

Download (PDF)

Tags: Computer science, CUDA, Deep learning, GPU cluster, Neural networks, nVidia, OpenMPI, Performance, Tesla P100, Tesla V100

February 24, 2019 by hgpu

SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural Networks

Linnan Wang, Wei Wu, Yiyang Zhao, Junyu Zhang, Hang Liu, George Bosilca, Jack Dongarra, Maurice Herlihy, Rodrigo Fonseca

View

Download (PDF)

Source codes

Tags: Compression, Computer science, CUDA, Deep learning, FFT, GPU cluster, Neural networks, nVidia, Package, Tesla P100

November 25, 2018 by hgpu

Optimizing Communication for Clusters of GPUs

Michael Wayne LeBeane

View

Download (PDF)

Tags: Computer science, GPU cluster, Memory model, Thesis

September 2, 2018 by hgpu

Scalable Breadth-First Search on a GPU Cluster

Yuechao Pan, Roger Pearce, John D. Owens

View

Download (PDF)

Tags: Computer science, CUDA, GPU cluster, nVidia, OpenMP, Performance, Tesla P100

March 25, 2018 by hgpu

* * *

high performance computing on graphics processing units: hgpu.org

HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism

Literature Review and Implementation Overview: High Performance Computing with Graphics Processing Units for Classroom and Research Use

Characterizing Deep Learning Training Workloads on Alibaba-PAI

Syntix: A Profiling Based Resource Estimator for CUDA Kernels

Elastic deep learning in multi-tenant GPU cluster

Automated Architecture Design for Deep Neural Networks

A Highly Efficient Distributed Deep Learning System For Automatic Speech Recognition

Themis: Fair and Efficient GPU Cluster Scheduling for Machine Learning Workloads

Optimizing Network Performance for Distributed DNN Training on GPU Clusters: ImageNet/AlexNet Training in 1.5 Minutes

SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural Networks

Optimizing Communication for Clusters of GPUs

Scalable Breadth-First Search on a GPU Cluster

Recent source codes

torchPDLP: Primal-Dual Linear Programming in PyTorch. In collaboration with AMD and IPAM

Benchmarks for Dissecting CPU-GPU Unified Physical Memory on AMD MI300A APUs

kvcached: Elastic KV cache for dynamic GPU sharing and efficient multi-LLM inference

Bandicoot: C++ library for GPU accelerated linear algebra

Luthier: Bridging Auto-Tuning and Vendor Libraries for Efficient Deep Learning Inference

Fused Kernel Library (FKL)

GPUHammer: Rowhammer Attacks on GPU Memories are Practical

Block: Balance Loader of LLM Serving with Context, Knowledge and Predictive Scheduling

SIGMo: Scalable Isomorphism Graph Matching on GPUs

DGEMM without FP64 Arithmetic - using FP64 Emulation and FP8 Tensor Cores with Ozaki Scheme

Most viewed papers (last 30 days)