high performance computing on graphics processing units: hgpu.org

hgpu.org » LLM

Context Parallelism for Scalable Million-Token Inference

Amy (Jie) Yang, Jingyi Yang, Aya Ibrahim, Xinfeng Xie, Bangsheng Tang, Grigory Sizov, Jeremy Reizenstein, Jongsoo Park, Jianyu Huang

View

Download (PDF)

Tags: Artificial intelligence, Benchmarking, Computer science, LLM, nVidia, nVidia H100, Performance

November 17, 2024 by hgpu

Improving Parallel Program Performance Through DSL-Driven Code Generation with LLM Optimizers

Anjiang Wei, Allen Nie, Thiago S. F. X. Teixeira, Rohan Yadav, Wonchan Lee, Ke Wang, Alex Aiken

View

Download (PDF)

Tags: Code generation, Computer science, DSL, LLM, Matrix multiplication, nVidia, Optimization, Tesla P100

November 17, 2024 by hgpu

LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators

Krishna Teja Chitty-Venkata, Siddhisanket Raskar, Bharat Kale, Farah Ferdaus, Aditya Tanikanti, Ken Raffenetti, Valerie Taylor, Murali Emani, Venkatram Vishwanath

View

Download (PDF)

Source codes

Tags: AI, AMD Radeon Instinct MI250, AMD Radeon Instinct MI300X, Artificial intelligence, ATI, Benchmarking, Computer science, CUDA, LLM, Machine learning, nVidia, nVidia A100, nVidia GH200, nVidia H100, OpenCL, Performance

November 10, 2024 by hgpu

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference

Xuanlin Jiang, Yang Zhou, Shiyi Cao, Ion Stoica, Minlan Yu

View

Download (PDF)

Tags: Artificial intelligence, Computer science, CUDA, LLM, Memory, nVidia, nVidia A10, nVidia H100, Tesla T4

November 10, 2024 by hgpu

Is the GPU Half-Empty or Half-Full? Practical Scheduling Techniques for LLMs

Ferdi Kossmann, Bruce Fontaine, Daya Khudia, Michael Cafarella, Samuel Madden

View

Download (PDF)

Tags: Computer science, LLM, NLP, nVidia, nVidia A100, nVidia H100, Task scheduling

November 3, 2024 by hgpu

Jailbreaking LLM-Controlled Robots

Alexander Robey, Zachary Ravichandran, Vijay Kumar, Hamed Hassani, George J. Pappas

View

Download (PDF)

Tags: Artificial intelligence, Computer science, LLM, Robotics, Security

October 27, 2024 by hgpu

Superpipeline: A Universal Approach for Reducing GPU Memory Usage in Large Models

Reza Abbasi, Sernam Lim

View

Download (PDF)

Source codes

Tags: AI, Artificial intelligence, Computer science, CUDA, LLM, Machine learning, Memory, nVidia, nVidia GeForce RTX 3090, nVidia Quadro RTX 8000, Package

October 20, 2024 by hgpu

Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Cores

Shaobo Ma, Chao Fang, Haikuo Shao, Zhongfeng Wang

View

Download (PDF)

Tags: Computer science, CUDA, Data recovery, Extended precision, LLM, Matrix multiplication, nVidia, nVidia GeForce RTX 3090, Security

October 6, 2024 by hgpu

Automatic Generation of OpenCL Code through Polyhedral Compilation with LLM

Marek Palkowski, Mateusz Gruzewski

View

Download (PDF)

Tags: AMD Radeon RX 6800, ATI, Code generation, Computer science, CUDA, LLM, nVidia, nVidia A100, OpenCL, OpenMP

September 29, 2024 by hgpu

Efficiently Training 7B LLM with 1 Million Sequence Length on 8 GPUs

Pinxue Zhao, Hailin Zhang, Fangcheng Fu, Xiaonan Nie, Qibin Liu, Fang Yang, Yuanbo Peng, Dian Jiao, Shuaipeng Li, Jinbao Xue, Yangyu Tao, Bin Cui

View

Download (PDF)

Tags: AI, Computer science, CUDA, LLM, Machine learning, NLP, nVidia, nVidia A800

July 28, 2024 by hgpu

Automated C/C++ Program Repair for High-Level Synthesis via Large Language Models

Kangwei Xu, Grace Li Zhang, Xunzhao Yin, Cheng Zhuo, Ulf Schlichtmann, Bing Li

View

Download (PDF)

Tags: Code generation, Computer science, FPGA, HLS, LLM

July 14, 2024 by hgpu

Chat AI: A Seamless Slurm-Native Solution for HPC-Based Services

Ali Doosthosseini, Jonathan Decker, Hendrik Nolte, Julian M. Kunkel

View

Download (PDF)

Source codes

Tags: AI, Cloud, Computer science, HPC, LLM, nVidia, nVidia H100, Package, PC cluster

July 7, 2024 by hgpu

Over-synchronization in GPU Programs

profile_util: simple C++ library to report memory, timing, thread affinity

Profile Util library: A quick and easy way to get MPI, OpenMP and GPU runtime information

MaC: Mamba CPU performance predictor

MambaCPU: Enhanced Correlation Mining with State Space Models for CPU Performance Prediction

Mixed-precision finite element kernels

Mixed-precision finite element kernels and assembly: Rounding error analysis and hardware acceleration

SHMT: The Simultaneous and Heterogenous Multithreading project

General-Purpose Computing on Tensor Processors

Superpipeline: A Universal Approach for Reducing GPU Memory Usage in Large Models

EnergyUCB-Bandit

Online Energy Optimization in GPUs: A Multi-Armed Bandit Approach

See all packages

* * *

high performance computing on graphics processing units: hgpu.org

Context Parallelism for Scalable Million-Token Inference

Improving Parallel Program Performance Through DSL-Driven Code Generation with LLM Optimizers

LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference

Is the GPU Half-Empty or Half-Full? Practical Scheduling Techniques for LLMs

Jailbreaking LLM-Controlled Robots

Superpipeline: A Universal Approach for Reducing GPU Memory Usage in Large Models

Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Cores

Automatic Generation of OpenCL Code through Polyhedral Compilation with LLM

Efficiently Training 7B LLM with 1 Million Sequence Length on 8 GPUs

Automated C/C++ Program Repair for High-Level Synthesis via Large Language Models

Recent source codes

Kokkidio: Kokkos+Eigen - write fast, readable multi-platform code

LLaMA-Inference-Bench

MATAR

Over-synchronization in GPU Programs

profile_util: simple C++ library to report memory, timing, thread affinity

MaC: Mamba CPU performance predictor

Mixed-precision finite element kernels

SHMT: The Simultaneous and Heterogenous Multithreading project

Superpipeline: A Universal Approach for Reducing GPU Memory Usage in Large Models

EnergyUCB-Bandit

Most viewed papers (last 30 days)