high performance computing on graphics processing units: hgpu.org

hgpu.org » GPU cluster

Helix: Distributed Serving of Large Language Models via Max-Flow on Heterogeneous GPUs

Yixuan Mei, Yonghao Zhuang, Xupeng Miao, Juncheng Yang, Zhihao Jia, Rashmi Vinayak

View

Download (PDF)

Tags: Computer science, GPU cluster, Heterogeneous systems, nVidia, nVidia A100, nVidia V100, Tesla T4

June 9, 2024 by hgpu

Balancing Tracking Granularity and Parallelism in Many-Task Systems: The Horizons Approach

Peter Thoman, Philip Salzmann

View

Download (PDF)

Source codes

Tags: Benchmarking, Computer science, GPU cluster, HPC, nVidia, nVidia V100, Package, SYCL

April 14, 2024 by hgpu

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis

Shiwei Zhang, Lansong Diao, Chuan Wu, Zongyan Cao, Siyu Wang, Wei Lin

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Deep learning, Distributed computing, GPU cluster, nVidia, nVidia A100, nVidia P100, nVidia V100, Package, PyTorch

January 14, 2024 by hgpu

gZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters

Jiajun Huang, Sheng Di, Xiaodong Yu, Yujia Zhai, Jinyang Liu, Yafan Huang, Ken Raffenetti, Hui Zhou, Kai Zhao, Zizhong Chen, Franck Cappello, Yanfei Guo, Rajeev Thakur

View

Download (PDF)

Tags: Compression, Computer science, GPU cluster, MPI, nVidia, nVidia A100

August 13, 2023 by hgpu

SYnergy: Fine-grained Energy-Efficient Heterogeneous Computing for Scalable Energy Saving

Kaijie Fan, Marco D'Antonio, Lorenzo Carpentieri, Biagio Cosenza, Federico Ficarelli, Daniele Cesarini

View

Download (PDF)

Tags: AMD Radeon Instinct MI100, ATI, Computer science, Energy-efficient computing, GPU cluster, Heterogeneous systems, Machine learning, nVidia, nVidia V100, SYCL

August 13, 2023 by hgpu

Isolated Scheduling for Distributed Training Tasks in GPU Clusters

Xinchi Han, Weihao Jiang, Peirui Cao, Qinwei Yang, Yunzhuo Liu, Shuyao Qi, Shengkai Lin, Shizhen Zhao

View

Download (PDF)

Source codes

Tags: Computer science, GPU cluster, Machine learning, Neural networks, nVidia, nVidia V100, Package

August 13, 2023 by hgpu

Communication-minimizing Asynchronous Tensor Parallelism

Siddharth Singh, Zack Sating, Abhinav Bhatele

View

Download (PDF)

Tags: Computer science, CUDA, GPU cluster, Neural networks, nVidia, nVidia A100

May 28, 2023 by hgpu

An Asynchronous Dataflow-Driven Execution Model For Distributed Accelerator Computing

Philip Salzmann, Fabian Knorr, Peter Thoman, Philipp Gschwandtner, Biagio Cosenza, Thomas Fahringer

View

Download (PDF)

Source codes

Tags: Benchmarking, Computer science, CUDA, GPU cluster, MPI, nVidia, nVidia V100, Package, SYCL

May 21, 2023 by hgpu

Energy-Efficient GPU Clusters Scheduling for Deep Learning

Diandian Gu, Xintong Xie, Gang Huang, Xin Jin, Xuanzhe Liu

View

Download (PDF)

Tags: Computer science, Deep learning, Energy-efficient computing, GPU cluster, Neural networks, nVidia, Tesla V100

April 16, 2023 by hgpu

On Scheduling Ring-All-Reduce Learning Jobs in Multi-Tenant GPU Clusters with Communication Contention

Menglu Yu, Bo Ji, Hridesh Rajan, Jia Liu

View

Download (PDF)

Tags: Computer science, Deep learning, GPU cluster, Task scheduling

July 24, 2022 by hgpu

The Application of AI Technology in GPU Scheduling Algorithm Optimization

Zhancai Yan, Yaqiu Liu, Hongrun Shao

View

Download (PDF)

Tags: Artificial intelligence, Computer science, CUDA, GPU cluster, nVidia, Task scheduling

May 22, 2022 by hgpu

GPUODEBenchmarks: Comparsion of Julia's GPU Kernel based ODE solvers with other open-source GPU ODE solvers

Automating Heterogeneous Parallelism in Numerical Differential Equations

Dat3M: Memory Model Aware Verification

Towards Unified Analysis of GPU Consistency

NVIDIA Federated Learning Application Runtime Environment

Supercharging Federated Learning with Flower and NVIDIA FLARE

Chat AI: A Seamless Slurm-Native Solution for HPC-Based Services

PSCToolkit: solving sparse linear systems with a large number of GPUs

CATBench: Benchmarking Framework

CATBench: A Compiler Autotuning Benchmarking Suite for Black-box Optimization

General-purpose Polyhedral Compilers

A Survey of General-purpose Polyhedral Compilers

Optimal Kernel Orchestration for Tensor Programs with Korch

Astaroth: A Scalable Multi-GPU Library for Stencil Computations

Stencil Computations on AMD and Nvidia Graphics Processors: Performance and Tuning Strategies

Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs. Empirical tricks for LLM Jailbreaking

Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs

See all packages

* * *

high performance computing on graphics processing units: hgpu.org

Helix: Distributed Serving of Large Language Models via Max-Flow on Heterogeneous GPUs

Balancing Tracking Granularity and Parallelism in Many-Task Systems: The Horizons Approach

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis

gZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters

SYnergy: Fine-grained Energy-Efficient Heterogeneous Computing for Scalable Energy Saving

Isolated Scheduling for Distributed Training Tasks in GPU Clusters

Communication-minimizing Asynchronous Tensor Parallelism

An Asynchronous Dataflow-Driven Execution Model For Distributed Accelerator Computing

Energy-Efficient GPU Clusters Scheduling for Deep Learning

On Scheduling Ring-All-Reduce Learning Jobs in Multi-Tenant GPU Clusters with Communication Contention

The Application of AI Technology in GPU Scheduling Algorithm Optimization

Recent source codes

GPUODEBenchmarks: Comparsion of Julia's GPU Kernel based ODE solvers with other open-source GPU ODE solvers

Dat3M: Memory Model Aware Verification

NVIDIA Federated Learning Application Runtime Environment

Chat AI: A Seamless Slurm-Native Solution for HPC-Based Services

PSCToolkit: solving sparse linear systems with a large number of GPUs

CATBench: Benchmarking Framework

General-purpose Polyhedral Compilers

Optimal Kernel Orchestration for Tensor Programs with Korch

Astaroth: A Scalable Multi-GPU Library for Stencil Computations

Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs. Empirical tricks for LLM Jailbreaking

Most viewed papers (last 30 days)