high performance computing on graphics processing units: hgpu.org

hgpu.org » Distributed computing

Composing Distributed Computations Through Task and Kernel Fusion

Rohan Yadav, Shiv Sundram, Wonchan Lee, Michael Garland, Michael Bauer, Alex Aiken, Fredrik Kjolstad

View

Download (PDF)

Tags: Computer science, CUDA, Distributed computing, MPI, nVidia, nVidia DGX-A100, Python

June 30, 2024 by hgpu

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis

Shiwei Zhang, Lansong Diao, Chuan Wu, Zongyan Cao, Siyu Wang, Wei Lin

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Deep learning, Distributed computing, GPU cluster, nVidia, nVidia A100, nVidia P100, nVidia V100, Package, PyTorch

January 14, 2024 by hgpu

Redco: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs

Bowen Tan, Yun Zhu, Lijuan Liu, Hongyi Wang, Yonghao Zhuang, Jindong Chen, Eric Xing, Zhiting Hu

View

Download (PDF)

Source codes

Tags: Computer science, Deep learning, Distributed computing, nVidia, nVidia A100, Package, TPU

November 5, 2023 by hgpu

PoCL-R: An Open Standard Based Offloading Layer for Heterogeneous Multi-Access Edge Computing with Server Side Scalability

Jan Solanti, Michal Babej, Julius Ikkala, Pekka Jääskeläinen

View

Download (PDF)

Source codes

Tags: Computer science, Distributed computing, Fluid dynamics, Heterogeneous systems, nVidia, nVidia GeForce RTX 2080 Ti, nVidia RTX A6000, OpenCL, Package, Rendering

September 6, 2023 by hgpu

A High-Performance Computing Cluster for Distributed Deep Learning: A Practical Case of Weed Classification Using Convolutional Neural Network Models

Manuel López-Martínez, Germán Díaz-Flórez, Santiago Villagrana-Barraza, Luis O. Solís-Sánchez, Héctor A. Guerrero-Osuna, Genaro M. Soto-Zarazúa, Carlos A. Olvera-Olvera

View

Download (PDF)

Tags: CUDA, Deep learning, Distributed computing, HPC, Image processing, Neural networks, nVidia, nVidia GeForce GTX 1050 Ti

June 4, 2023 by hgpu

ARK: GPU-driven Code Execution for Distributed Deep Learning

Changho Hwang, KyoungSoo Park, Ran Shu, Xinyuan Qu, Peng Cheng, Yongqiang Xiong

View

Download (PDF)

Tags: Computer science, Deep learning, Distributed computing, nVidia, nVidia V100

March 12, 2023 by hgpu

AutoDDL: Automatic Distributed Deep Learning with Asymptotically Optimal Communication

Jinfan Chen, Shigang Li, Ran Gun, Jinhui Yuan, Torsten Hoefler

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Deep learning, Distributed computing, nVidia, nVidia A100, Package, Tesla P100

January 22, 2023 by hgpu

Distributed Calculations with Algorithmic Skeletons for Heterogeneous Computing Environments

Nina Herrmann, Herbert Kuchen

View

Download (PDF)

Tags: Computer science, CUDA, Distributed computing, Heterogeneous systems, nVidia, nVidia GeForce GTX 750 Ti, nVidia GeFroce RTX 2080 Ti, nVidia Quadro K620

January 15, 2023 by hgpu

Kernel-as-a-Service: A Serverless Interface to GPUs

Nathan Pemberton, Anton Zabreyko, Zhoujie Ding, Randy Katz, Joseph Gonzalez

View

Download (PDF)

Source codes

Tags: Cloud, Computer science, CUDA, Deep learning, Distributed computing, nVidia, Package, Tesla V100

December 25, 2022 by hgpu

Integrating SkePU’s algorithmic skeletons with GPI on a cluster

Joel Almqvist

View

Download (PDF)

Tags: Computer science, CUDA, Distributed computing, HPC, MPI, nVidia, OpenCL, OpenMP, Thesis

March 6, 2022 by hgpu

Lightning: Scaling the GPU Programming Model Beyond a Single GPU

Stijn Heldens, Pieter Hijma, Ben van Werkhoven, Jason Maassen, Rob V. van Nieuwpoort

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Distributed computing, nVidia, OpenCL, Package, Performance, Tesla P100

February 20, 2022 by hgpu

HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments

Ji Liu, Zhihua Wu, Dianhai Yu, Yanjun Ma, Danlei Feng, Minxu Zhang, Xinxuan Wu, Xuefeng Yao, Dejing Dou

View

Download (PDF)

Source codes

Tags: Computer science, Deep learning, Distributed computing, Heterogeneous systems, Neural networks, nVidia, Package, Tesla V100

November 28, 2021 by hgpu

Over-synchronization in GPU Programs

profile_util: simple C++ library to report memory, timing, thread affinity

Profile Util library: A quick and easy way to get MPI, OpenMP and GPU runtime information

MaC: Mamba CPU performance predictor

MambaCPU: Enhanced Correlation Mining with State Space Models for CPU Performance Prediction

Mixed-precision finite element kernels

Mixed-precision finite element kernels and assembly: Rounding error analysis and hardware acceleration

SHMT: The Simultaneous and Heterogenous Multithreading project

General-Purpose Computing on Tensor Processors

Superpipeline: A Universal Approach for Reducing GPU Memory Usage in Large Models

EnergyUCB-Bandit

Online Energy Optimization in GPUs: A Multi-Armed Bandit Approach

See all packages

* * *

high performance computing on graphics processing units: hgpu.org

Composing Distributed Computations Through Task and Kernel Fusion

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis

Redco: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs

PoCL-R: An Open Standard Based Offloading Layer for Heterogeneous Multi-Access Edge Computing with Server Side Scalability

A High-Performance Computing Cluster for Distributed Deep Learning: A Practical Case of Weed Classification Using Convolutional Neural Network Models

ARK: GPU-driven Code Execution for Distributed Deep Learning

AutoDDL: Automatic Distributed Deep Learning with Asymptotically Optimal Communication

Distributed Calculations with Algorithmic Skeletons for Heterogeneous Computing Environments

Integrating SkePU’s algorithmic skeletons with GPI on a cluster

Lightning: Scaling the GPU Programming Model Beyond a Single GPU

HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments

Recent source codes

Kokkidio: Kokkos+Eigen - write fast, readable multi-platform code

LLaMA-Inference-Bench

MATAR

Over-synchronization in GPU Programs

profile_util: simple C++ library to report memory, timing, thread affinity

MaC: Mamba CPU performance predictor

Mixed-precision finite element kernels

SHMT: The Simultaneous and Heterogenous Multithreading project

Superpipeline: A Universal Approach for Reducing GPU Memory Usage in Large Models

EnergyUCB-Bandit

Most viewed papers (last 30 days)