high performance computing on graphics processing units: hgpu.org

Kshiteej Mahajan, Arjun Singhvi, Arjun Balasubramanian, Varun Batra, Surya Teja Chavali, Shivaram Venkataraman, Aditya Akella, Amar Phanishayee, Shuchi Chawla

View

Download (PDF)

Tags: Computer science, GPU cluster, Machine learning, nVidia, Task scheduling, Tesla K80, Tesla M60

July 4, 2019 by hgpu

Optimizing Network Performance for Distributed DNN Training on GPU Clusters: ImageNet/AlexNet Training in 1.5 Minutes

Peng Sun, Wansen Feng, Ruobing Han, Shengen Yan, Yonggang Wen

View

Download (PDF)

Tags: Computer science, CUDA, Deep learning, GPU cluster, Neural networks, nVidia, OpenMPI, Performance, Tesla P100, Tesla V100

February 24, 2019 by hgpu

SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural Networks

Linnan Wang, Wei Wu, Yiyang Zhao, Junyu Zhang, Hang Liu, George Bosilca, Jack Dongarra, Maurice Herlihy, Rodrigo Fonseca

View

Download (PDF)

Source codes

Tags: Compression, Computer science, CUDA, Deep learning, FFT, GPU cluster, Neural networks, nVidia, Package, Tesla P100

November 25, 2018 by hgpu

Optimizing Communication for Clusters of GPUs

Michael Wayne LeBeane

View

Download (PDF)

Tags: Computer science, GPU cluster, Memory model, Thesis

September 2, 2018 by hgpu

Scalable Breadth-First Search on a GPU Cluster

Yuechao Pan, Roger Pearce, John D. Owens

View

Download (PDF)

Tags: Computer science, CUDA, GPU cluster, nVidia, OpenMP, Performance, Tesla P100

March 25, 2018 by hgpu

A multi-agent architecture for scheduling of high performance services in a GPU cluster

Joel Antonio Trejo-Sanchez, Jose Luis Lopez-Martinez, Jose Octavio Gutierrez Garcia, Julio Cesar Ramirez-Pacheco, Daniel Fajardo-Delgado

View

Download (PDF)

Tags: Computer science, GPU cluster, Heterogeneous systems

March 22, 2018 by hgpu

MiMatrix: A Massively Distributed Deep Learning Framework on a Petascale High-density Heterogeneous Cluster

Xin Chen, Hua Zhou, Yuxiang Gao, Yu Zhu, Dongyan Wang

View

Download (PDF)

Tags: Algorithms, Computer science, CUDA, Deep learning, GPU cluster, Heterogeneous systems, nVidia, RDMA, Tesla P100

February 9, 2018 by hgpu

Improving Communication Performance in GPU-Accelerated HPC Clusters

Iman Faraji

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, GPU cluster, MPI, nVidia, Package, Performance, Thesis

January 28, 2018 by hgpu

* * *

high performance computing on graphics processing units: hgpu.org

Syntix: A Profiling Based Resource Estimator for CUDA Kernels

Elastic deep learning in multi-tenant GPU cluster

Automated Architecture Design for Deep Neural Networks

A Highly Efficient Distributed Deep Learning System For Automatic Speech Recognition

Themis: Fair and Efficient GPU Cluster Scheduling for Machine Learning Workloads

Optimizing Network Performance for Distributed DNN Training on GPU Clusters: ImageNet/AlexNet Training in 1.5 Minutes

SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural Networks

Optimizing Communication for Clusters of GPUs

Scalable Breadth-First Search on a GPU Cluster

A multi-agent architecture for scheduling of high performance services in a GPU cluster

MiMatrix: A Massively Distributed Deep Learning Framework on a Petascale High-density Heterogeneous Cluster

Improving Communication Performance in GPU-Accelerated HPC Clusters

Recent source codes

Kokkidio: Kokkos+Eigen - write fast, readable multi-platform code

LLaMA-Inference-Bench

MATAR

Over-synchronization in GPU Programs

profile_util: simple C++ library to report memory, timing, thread affinity

MaC: Mamba CPU performance predictor

Mixed-precision finite element kernels

SHMT: The Simultaneous and Heterogenous Multithreading project

Superpipeline: A Universal Approach for Reducing GPU Memory Usage in Large Models

EnergyUCB-Bandit

Most viewed papers (last 30 days)