LLM | Page 3 | hgpu.org

hgpu.org » LLM

Efficiently Training 7B LLM with 1 Million Sequence Length on 8 GPUs

Pinxue Zhao, Hailin Zhang, Fangcheng Fu, Xiaonan Nie, Qibin Liu, Fang Yang, Yuanbo Peng, Dian Jiao, Shuaipeng Li, Jinbao Xue, Yangyu Tao, Bin Cui

View

Download (PDF)

Tags: AI, Computer science, CUDA, LLM, Machine learning, NLP, nVidia, nVidia A800

July 28, 2024 by hgpu

Automated C/C++ Program Repair for High-Level Synthesis via Large Language Models

Kangwei Xu, Grace Li Zhang, Xunzhao Yin, Cheng Zhuo, Ulf Schlichtmann, Bing Li

View

Download (PDF)

Tags: Code generation, Computer science, FPGA, HLS, LLM

July 14, 2024 by hgpu

Chat AI: A Seamless Slurm-Native Solution for HPC-Based Services

Ali Doosthosseini, Jonathan Decker, Hendrik Nolte, Julian M. Kunkel

View

Download (PDF)

Source codes

Tags: AI, Cloud, Computer science, HPC, LLM, nVidia, nVidia H100, Package, PC cluster

July 7, 2024 by hgpu

Breaking the Memory Wall: A Study of I/O Patterns and GPU Memory Utilization for Hybrid CPU-GPU Offloaded Optimizers

Avinash Maurya, Jie Ye, M. Mustafa Rafique, Franck Cappello, Bogdan Nicolae

View

Download (PDF)

Tags: Computer science, CUDA, LLM, Memory, nVidia, nVidia A100, nVidia H100, Performance

June 23, 2024 by hgpu

Coccinelle: a C code transformation engine using SmPL for matches, refactorings, and bug fixing

Advances in Semantic Patching for HPC-oriented Refactorings with Coccinelle

DuoReduce: MLIR's benchmark

Hardware-Assisted Software Testing and Debugging for Heterogeneous Computing

Shamrock: Multi-GPU hydrodynamics for astrophysics

The Shamrock code: I- Smoothed Particle Hydrodynamics on GPUs

LLMPerf: GPU Performance Modeling meets Large Language Models

Hercules: A Compiler for Productive Programming of Heterogeneous Systems

* * *

high performance computing on graphics processing units: hgpu.org

Efficiently Training 7B LLM with 1 Million Sequence Length on 8 GPUs

Automated C/C++ Program Repair for High-Level Synthesis via Large Language Models

Breaking the Memory Wall: A Study of I/O Patterns and GPU Memory Utilization for Hybrid CPU-GPU Offloaded Optimizers

Recent source codes

Coccinelle: a C code transformation engine using SmPL for matches, refactorings, and bug fixing

DuoReduce: MLIR's benchmark

Shamrock: Multi-GPU hydrodynamics for astrophysics

LLMPerf: GPU Performance Modeling meets Large Language Models

Hercules: A Compiler for Productive Programming of Heterogeneous Systems

Celerity Runtime: High-level C++ for Accelerator Clusters

wgpy: WebGL accelerated numpy-compatible array library for web browser

Microbenchmarking OpenMP target offload with Catch2

SUperman: Highly Efficient Permanent Computation Library

TransCL: An Automatic CUDA-to-OpenCL Programs Transformation Framework

Most viewed papers (last 30 days)