high performance computing on graphics processing units: hgpu.org

Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Data parallelism, Deep learning, GPU cluster, NLP, nVidia, nVidia DGX-A100, Package

April 18, 2021 by hgpu

AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning

Yuhan Liu, Saurabh Agarwal, Shivaram Venkataraman

View

Download (PDF)

Source codes

Tags: Computer science, Deep learning, Machine learning, NLP, nVidia, nVidia A100, Package, Python, Tesla P100

February 7, 2021 by hgpu

* * *

KLARAPTOR: A Tool for Dynamically Finding Optimal Kernel Launch Parameters Targeting CUDA Programs

Three Contributions to the Theory and Practice of Optimizing Compilers

HETU: a high-performance distributed deep learning system targeting large-scale and automated distributed training

Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism

TorchOpt: an efficient library for differentiable optimization built upon PyTorch

TorchOpt: An Efficient Library for Differentiable Optimization

NaturalCC: An Open-Source Toolkit for Code Intelligence

A Study on Neural-based Code Summarization in Low-resource Settings

mt4g: Memory Topology for GPUs

Capturing the Memory Topology of GPUs

See all packages

* * *

high performance computing on graphics processing units: hgpu.org

EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models

Towards making the most of NLP-based device mapping optimization for OpenCL kernels

Long Code for Code Search

MapReduce for Counting Word Frequencies with MPI and GPUs

Lossless Acceleration for Seq2seq Generation with Aggressive Decoding

The Ecological Footprint of Neural Machine Translation Systems

Autotuning CUDA: Applying NLP Techniques to LS-CAT

Small-Bench NLP: Benchmark for small single GPU trained models in Natural Language Processing

Easy and Efficient Transformer: Scalable Inference Solution For large NLP mode

How to Train BERT with an Academic Budget

Efficient Large-Scale Language Model Training on GPU Clusters

AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning

Recent source codes

KLARAPTOR: A Tool for Dynamically Finding Optimal Kernel Launch Parameters Targeting CUDA Programs

HETU: a high-performance distributed deep learning system targeting large-scale and automated distributed training

SYCL-DNN: a library implementing neural network algorithms written using SYCL

SW#SYCL

Flink-TornadoVM-Artifact

StateMover: a checkpoint-based debugging framework for FPGAs

Kernel Tuner

TorchOpt: an efficient library for differentiable optimization built upon PyTorch

NaturalCC: An Open-Source Toolkit for Code Intelligence

mt4g: Memory Topology for GPUs

Most viewed papers (last 30 days)