high performance computing on graphics processing units: hgpu.org

Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Data parallelism, Deep learning, GPU cluster, NLP, nVidia, nVidia DGX-A100, Package

April 18, 2021 by hgpu

A Hybrid Parallelization Approach for Distributed and Scalable Deep Learning

Samson B. Akintoye, Liangxiu Han, Xin Zhang, Haoming Chen, Daoqiang Zhang

View

Download (PDF)

Tags: Computer science, CUDA, Data parallelism, Deep learning, Neural networks, nVidia, Tesla V100

April 18, 2021 by hgpu

SimSYCL: Synchronous, single-threaded, library-only SYCL implementation for debugging and verification

SimSYCL: A SYCL Implementation Targeting Development, Debugging, Simulation and Conformance

gpu_tracker: Context manager and CLI that tracks the computational-resource-usage of a code block or shell command, particularly the GPU usage

gpu_tracker: Python package for tracking and profiling GPU utilization in both desktop and high-performance computing environments

CIFAR-10 Airbench: 94% on CIFAR-10 in 3.29 second

94% on CIFAR-10 in 3.29 Seconds on a Single GPU

LOOPer: a polyhedral compiler for expressing fast and portable data parallel algorithms

LOOPer: A Learned Automatic Code Optimizer For Polyhedral Compilers

OpenMC Monte Carlo Code

Performance Portable Monte Carlo Particle Transport on Intel, NVIDIA, and AMD GPUs

Polygeist: C/C++ frontend for MLIR

Retargeting and Respecializing GPU Workloads for Performance Portability

Parallel Gaussian process with kernel approximation in CUDA

See all packages

* * *

high performance computing on graphics processing units: hgpu.org

Enabling Energy-Efficient DNN Training on Hybrid GPU-FPGA Accelerators

Easy and Efficient Transformer: Scalable Inference Solution For large NLP mode

tcFFT: Accelerating Half-Precision FFT through Tensor Cores

How to Train BERT with an Academic Budget

Deep Graph Learning for Program Analysis and System Optimization

Ripple: Simplified Large-Scale Computation on Heterogeneous Architectures with Polymorphic Data Layout

CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU

Performance Analysis and Optimization Opportunities for NVIDIA Automotive GPUs

SLATE port to AMD and Intel platforms

FANS: FPGA-Accelerated Near-Storage Sorting

Efficient Large-Scale Language Model Training on GPU Clusters

A Hybrid Parallelization Approach for Distributed and Scalable Deep Learning

Recent source codes

SimSYCL: Synchronous, single-threaded, library-only SYCL implementation for debugging and verification

GPU plugin for PySCF

QArray

Celerity: High-level C++ for Accelerator Clusters