high performance computing on graphics processing units: hgpu.org

hgpu.org » GPU cluster

Efficient Large-Scale Language Model Training on GPU Clusters

Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Data parallelism, Deep learning, GPU cluster, NLP, nVidia, nVidia DGX-A100, Package

April 18, 2021 by hgpu

Python Workflows on HPC Systems

Dominik Strassel, Philipp Reusch, Janis Keuper

View

Download (PDF)

Tags: Computer science, CUDA, Deep learning, GPU cluster, HPC, Machine learning, nVidia, PC cluster, Python

December 6, 2020 by hgpu

Hybrid MPI and CUDA Parallelization for CFD Applications on Multi-GPU HPC Clusters

Jianqi Lai, Hang Yu, Zhengyu Tian, Hua Li

View

Download (PDF)

Tags: cfd, CUDA, Fluid dynamics, GPU cluster, HPC, MPI, nVidia, nVidia GeForce GTX 1070, OpenMP, Tesla V100

September 27, 2020 by hgpu

Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads

Deepak Narayanan, Keshav Santhanam, Fiodar Kazhamiaka, Amar Phanishayee, Matei Zaharia

View

Download (PDF)

Tags: Computer science, CUDA, Deep learning, FPGA, GPU cluster, Heterogeneous systems, nVidia, Optimization, Task scheduling, Tesla K80, Tesla P100, Tesla V100

August 30, 2020 by hgpu

Ignite-GPU: a GPU-enabled in-memory computing architecture on clusters

Amir Hossein Sojoodi, Majid Salimi Beni, Farshad Khunjush

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, GPU cluster, nVidia, nVidia GeForce GTX 680, Package

August 9, 2020 by hgpu

OpenSBLI: Automated code-generation for heterogeneous computing architectures applied to compressible fluid dynamics on structured grids

David J. Lusher, Satya P. Jammy, Neil D. Sandham

View

Download (PDF)

Source codes

Tags: cfd, Code generation, CUDA, Fluid dynamics, GPU cluster, Heterogeneous systems, Numerical simulation, nVidia, OpenCL, OpenMP, OpenMPI, Package, Python, Tesla P100

August 2, 2020 by hgpu

HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism

Jay H. Park, Gyeongchan Yun, Chang M. Yi, Nguyen T. Nguyen, Seungmin Lee, Jaesik Choi, Sam H. Noh, Young-ri Choi

View

Download (PDF)

Tags: Computer science, CUDA, Data parallelism, Deep learning, GPU cluster, Heterogeneous systems, Neural networks, nVidia, nVidia GeForce GTX Titan V, nVidia GeForce RTX 2060, nVidia Quadro P 4000, nVidia Titan RTX

May 31, 2020 by hgpu

Literature Review and Implementation Overview: High Performance Computing with Graphics Processing Units for Classroom and Research Use

Nathan George

View

Download (PDF)

Tags: Computer science, CUDA, Deep learning, Education, GPU cluster, HPC, nVidia, nVidia GeForce GTX 1080 Ti, nVidia Quadro RTX 5000, Overview

May 24, 2020 by hgpu

Characterizing Deep Learning Training Workloads on Alibaba-PAI

Mengdi Wang, Chen Meng, Guoping Long, Chuan Wu, Jun Yang, Wei Lin, Yangqing Jia

View

Download (PDF)

Tags: Artificial intelligence, Computer science, Computer vision, Deep learning, GPU cluster, nVidia, Tesla V100

October 20, 2019 by hgpu

Syntix: A Profiling Based Resource Estimator for CUDA Kernels

Maria Papadaki, Yannis Sfakianakis, Christos Kozanitis, Angelos Bilas

View

Download (PDF)

Tags: Computer science, CUDA, GPU cluster, Grid, nVidia, nVidia Quadro K 2200

October 6, 2019 by hgpu

Elastic deep learning in multi-tenant GPU cluster

Yidi Wu, Kaihao Ma, Xiao Yan, Zhi Liu, James Cheng

View

Download (PDF)

Tags: Computer science, Deep learning, Distributed computing, GPU cluster, nVidia, nVidia GeForce GTX 1080 Ti, Tesla V100

September 29, 2019 by hgpu

Automated Architecture Design for Deep Neural Networks

Steven Abreu

View

Download (PDF)

Tags: Computer science, Deep learning, GPU cluster, Machine learning, Neural networks, nVidia, Tesla K80, Thesis

September 1, 2019 by hgpu

SimSYCL: Synchronous, single-threaded, library-only SYCL implementation for debugging and verification

SimSYCL: A SYCL Implementation Targeting Development, Debugging, Simulation and Conformance

gpu_tracker: Context manager and CLI that tracks the computational-resource-usage of a code block or shell command, particularly the GPU usage

gpu_tracker: Python package for tracking and profiling GPU utilization in both desktop and high-performance computing environments

CIFAR-10 Airbench: 94% on CIFAR-10 in 3.29 second

94% on CIFAR-10 in 3.29 Seconds on a Single GPU

LOOPer: a polyhedral compiler for expressing fast and portable data parallel algorithms

LOOPer: A Learned Automatic Code Optimizer For Polyhedral Compilers

OpenMC Monte Carlo Code

Performance Portable Monte Carlo Particle Transport on Intel, NVIDIA, and AMD GPUs

Polygeist: C/C++ frontend for MLIR

Retargeting and Respecializing GPU Workloads for Performance Portability

Parallel Gaussian process with kernel approximation in CUDA

See all packages

* * *

high performance computing on graphics processing units: hgpu.org

Efficient Large-Scale Language Model Training on GPU Clusters

Python Workflows on HPC Systems

Hybrid MPI and CUDA Parallelization for CFD Applications on Multi-GPU HPC Clusters

Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads

Ignite-GPU: a GPU-enabled in-memory computing architecture on clusters

OpenSBLI: Automated code-generation for heterogeneous computing architectures applied to compressible fluid dynamics on structured grids

HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism

Literature Review and Implementation Overview: High Performance Computing with Graphics Processing Units for Classroom and Research Use

Characterizing Deep Learning Training Workloads on Alibaba-PAI

Syntix: A Profiling Based Resource Estimator for CUDA Kernels

Elastic deep learning in multi-tenant GPU cluster

Automated Architecture Design for Deep Neural Networks

Recent source codes

SimSYCL: Synchronous, single-threaded, library-only SYCL implementation for debugging and verification

GPU plugin for PySCF

QArray

Celerity: High-level C++ for Accelerator Clusters

gpu_tracker: Context manager and CLI that tracks the computational-resource-usage of a code block or shell command, particularly the GPU usage

CIFAR-10 Airbench: 94% on CIFAR-10 in 3.29 second

LOOPer: a polyhedral compiler for expressing fast and portable data parallel algorithms

OpenMC Monte Carlo Code

Polygeist: C/C++ frontend for MLIR

Parallel Gaussian process with kernel approximation in CUDA

Most viewed papers (last 30 days)