high performance computing on graphics processing units: hgpu.org

Omer Dunay, Daniel Cheng, Adam Tait, Parth Thakkar, Peter C Rigby, Andy Chiu, Imad Ahmad, Arun Ganesan, Chandra Maddila, Vijayaraghavan Murali, Ali Tayyebi, Nachiappan Nagappan

View

Download (PDF)

Tags: AI, Artificial intelligence, Computer science, Software Engineering

February 12, 2024 by hgpu

DeepSeek-Coder: When the Large Language Model Meets Programming – The Rise of Code Intelligence

Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y.K. Li, Fuli Luo, Yingfei Xiong, Wenfeng Liang

View

Download (PDF)

Source codes

Tags: Code generation, Computer science, nVidia, nVidia A100, Package

February 12, 2024 by hgpu

Data-efficient LLM Fine-tuning for Code Generation

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training

Large Language Model Powered C-to-CUDA Code Translation: A Novel Auto-Parallelization Framework

GigaAPI: a user-space API that simplifies multi-GPU programming, bridging the gap between the capabilities of parallel GPU systems and the ability of developers to harness their full potential

GigaAPI for GPU Parallelization

Coccinelle: a C code transformation engine using SmPL for matches, refactorings, and bug fixing

Advances in Semantic Patching for HPC-oriented Refactorings with Coccinelle

DuoReduce: MLIR's benchmark

Hardware-Assisted Software Testing and Debugging for Heterogeneous Computing

See all packages

* * *

high performance computing on graphics processing units: hgpu.org

Applications

Analyzing GPU Performance in Virtualized Environments: A Case Study

APPy: Annotated Parallelism for Python on GPUs

Green AI: A Preliminary Empirical Study on Energy Consumption in DL Models Across Different Runtime Infrastructures

Assessing opportunities of SYCL for biological sequence alignment on GPU-based systems

Benchmarking and Dissecting the Nvidia Hopper GPU Architecture

TransAxx: Efficient Transformers with Approximate Computing

Graphtoy: Fast Software Simulation of Applications for AMD’s AI Engines

An Evaluative Comparison of Performance Portability across GPU Programming Models

pSTL-Bench: A Micro-Benchmark Suite for Assessing Scalability of C++ Parallel STL Implementations

QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference

Multi-line AI-assisted Code Authoring

DeepSeek-Coder: When the Large Language Model Meets Programming – The Rise of Code Intelligence

Recent source codes

PELSI: Power-Efficient Layer-Switched Inference

Ouroboros: Virtualized Queues for dynamic memory management

MSCCL++: A GPU-driven communication stack for scalable AI applications

Benchmark compute shader of Unity against InteropUnityCUDA