high performance computing on graphics processing units: hgpu.org

Million-token context serving is becoming a critical requirement for industry LLM systems for real deployments. However, practical 1Mcontext serving for open-source models has remained heavily CUDA-centric, largely because the required long-context runtime features are not fully available on AMD ROCm. With AMD GPUs now reported at around 7% market share (Silicon Analysts Research, 2026), this gap matters. We address it by enabling 1M-token Qwen3 serving on AMD GPUs with ROCm. We present CHUNDER, a runtime stack that enables million-token serving for the widely used open-source Qwen3 family on AMD ROCm. CHUNDER achieves this by intercepting vLLM’s attention routing at runtime and redirecting queries, keys, and values to JIT-compiled Triton Dual Chunk Attention (DCA) kernels, without forking the serving engine. We then harden the surrounding runtime to make full 1M-token execution practical: DCA-aware paged metadata, tiled single request chunked prefill, MoE token chunking that reduces per-rank transient memory from approximately 91.6 GiB to 10.6 GiB, and FP8 GEMMs with hardened hipBLASLt kernel selection. We evaluate CHUNDER across multiple million-token context tasks and show consistent serving performance and exact retrieval correctness. To our knowledge, this is the first reported end-to-end Qwen3 million-token serving path on AMD ROCm integrated with vLLM.