high performance computing on graphics processing units: hgpu.org

hgpu.org » Deep learning

Fast convolution kernels on pascal GPU with high memory efficiency

Qiong Chang, Masaki Onishi, Tsutomu Maruyama

View

Download (PDF)

Tags: CNN, Computer science, CUDA, Deep learning, nVidia, nVidia GeForce GTX 1080 Ti, Performance

December 4, 2022 by hgpu

Efficient Incremental Text-to-Speech on GPUs

Muyang Du, Chuan Liu, Jiaxing Qi, Junjie Lai

View

Download (PDF)

Tags: Computer science, Deep learning, Machine learning, nVidia, nVidia A10, Tesla V100

December 4, 2022 by hgpu

Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism

Xupeng Miao, Yujie Wang, Youhe Jiang, Chunan Shi, Xiaonan Nie, Hailin Zhang, Bin Cui

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Databases, Deep learning, nVidia, nVidia Titan RTX, Package

December 4, 2022 by hgpu

User-Driven Online Kernel Fusion for SYCL

Victor Perez, Lukas Sommer, Victor Lomüller, Kumudha Narasimhan, Mehdi Goli

View

Download (PDF)

Source codes

Tags: Compilers, Computer science, Deep learning, OpenCL, Package, SYCL

November 27, 2022 by hgpu

SciAI4Industry – Solving PDEs for industry-scale problems with deep learning

Philipp A. Witte, Russell J. Hewett, Kumar Saurabh, AmirHossein Sojoodi, Ranveer Chandra

View

Download (PDF)

Tags: AI, Cloud, Computational Physics, Computer science, Deep learning, Differential equations, Neural networks, nVidia, nVidia A100, nVidia DGX-A100, Partial differential equations, PDEs

November 27, 2022 by hgpu

Training a Vision Transformer from scratch in less than 24 hours with 1 GPU

Saghar Irandoust, Thibaut Durand, Yunduz Rakhmangulova, Wenjie Zi, Hossein Hajimirsadeghi

View

Download (PDF)

Tags: Benchmarking, Computer science, Computer vision, Deep learning

November 20, 2022 by hgpu

TorchOpt: An Efficient Library for Differentiable Optimization

Jie Ren, Xidong Feng, Bo Liu, Xuehai Pan, Yao Fu, Luo Mai, Yaodong Yang

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Deep learning, Mathematical Software, nVidia, OpenMP, Package, PyTorch

November 20, 2022 by hgpu

A Study on Neural-based Code Summarization in Low-resource Settings

Yang He

View

Download (PDF)

Source codes

Tags: Computer science, CUDA, Deep learning, nVidia, PyTorch, Software Engineering, Tesla V100, Thesis

November 13, 2022 by hgpu

iGniter: Interference-Aware GPU Resource Provisioning for Predictable DNN Inference in the Cloud

Fei Xu, Jianian Xu, Jiabin Chen, Li Chen, Ruitao Shang, Zhi Zhou, Fangming Liu

View

Download (PDF)

Source codes

Tags: Cloud, Computer science, CUDA, Deep learning, Neural networks, nVidia, Package, Tesla V100

November 13, 2022 by hgpu

Enabling Data Movement and Computation Pipelining in Deep Learning Compiler

Guyue Huang, Yang Bai, Liu Liu, Yuke Wang, Bei Yu, Yufei Ding, Yuan Xie

View

Download (PDF)

Tags: Computer science, CUDA, Deep learning, nVidia, nVidia A100

November 6, 2022 by hgpu

The Open MatSci ML Toolkit: A Flexible Framework for Machine Learning in Materials Science

Santiago Miret, Kin Long Kelvin Lee, Carmelo Gonzales, Marcel Nassar, Matthew Spellings

View

Download (PDF)

Source codes

Tags: Artificial intelligence, Computer science, Deep learning, Machine learning, Materials Science, Neural networks, Package, Python

November 6, 2022 by hgpu

Benchmarking GPU and TPU Performance with Graph Neural Networks

Xiangyang Ju, Yunsong Wang, Daniel Murnane, Nicholas Choma, Steven Farrell, Paolo Calafiura

View

Download (PDF)

Tags: Artificial intelligence, Benchmarking, Computer science, CUDA, Deep learning, Machine learning, Neural networks, nVidia, nVidia A100, Pattern recognition, Tesla V100, TPU

October 30, 2022 by hgpu

MegaTrain: Full Precision Training of 100B+ Parameter Large Language Models on a Single GPU

Device Virtual Machine (DVM)

DVM: Real-Time Kernel Generation for Dynamic AI Models

Agentic Code Optimization via Compiler-LLM Cooperation

AutoKernel: Autoresearch for GPU kernels. Give it any PyTorch model, go to sleep, wake up to optimized Triton kernels

AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Search

Triton-Sanitizer: A Fast and Device-Agnostic Memory Sanitizer for Triton with Rich Diagnostic Context

LLM.Q: Quantized LLM training in pure CUDA/C++

LLMQ: Efficient Lower-Precision LLM Training for Consumer GPUs

SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limits

* * *

high performance computing on graphics processing units: hgpu.org

Fast convolution kernels on pascal GPU with high memory efficiency

Efficient Incremental Text-to-Speech on GPUs

Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism

User-Driven Online Kernel Fusion for SYCL

Training a Vision Transformer from scratch in less than 24 hours with 1 GPU

TorchOpt: An Efficient Library for Differentiable Optimization

A Study on Neural-based Code Summarization in Low-resource Settings

Enabling Data Movement and Computation Pipelining in Deep Learning Compiler

The Open MatSci ML Toolkit: A Flexible Framework for Machine Learning in Materials Science

Benchmarking GPU and TPU Performance with Graph Neural Networks

Recent source codes

MegaTrain: Full Precision Training of 100B+ Parameter Large Language Models on a Single GPU

Device Virtual Machine (DVM)

Agentic Code Optimization via Compiler-LLM Cooperation

AutoKernel: Autoresearch for GPU kernels. Give it any PyTorch model, go to sleep, wake up to optimized Triton kernels

Triton-Sanitizer: A Fast and Device-Agnostic Memory Sanitizer for Triton with Rich Diagnostic Context

LLM.Q: Quantized LLM training in pure CUDA/C++

SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limits

True 4-Bit Quantized CNN Training on CPU

cuFuzz: A GPU-oriented coverage-guided fuzzer for userland CUDA application

KernelSkill: A Multi-Agent Framework for GPU Kernel Optimization

Most viewed papers (last 30 days)