Million-token context serving is becoming a critical requirement for industry LLM systems for real deployments. However, practical 1Mcontext serving for open-source models has remained heavily CUDA-centric, largely because the required long-context runtime features are not fully available on AMD ROCm. With AMD GPUs now reported at around 7% market share (Silicon Analysts Research, 2026), this gap matters. We address it by enabling 1M-token Qwen3 serving on AMD GPUs with ROCm. We present CHUNDER, a runtime stack that enables million-token serving for the widely used open-source Qwen3 family on AMD ROCm. CHUNDER achieves this by intercepting vLLM\u2019s attention routing at runtime and redirecting queries, keys, and values to JIT-compiled Triton Dual Chunk Attention (DCA) kernels, without forking the serving engine. We then harden the surrounding runtime to make full 1M-token execution practical: DCA-aware paged metadata, tiled single request chunked prefill, MoE token chunking that reduces per-rank transient memory from approximately 91.6 GiB to 10.6 GiB, and FP8 GEMMs with hardened hipBLASLt kernel selection. We evaluate CHUNDER across multiple million-token context tasks and show consistent serving performance and exact retrieval correctness. To our knowledge, this is the first reported end-to-end Qwen3 million-token serving path on AMD ROCm integrated with vLLM.<\/p>\n","protected":false},"excerpt":{"rendered":"
Million-token context serving is becoming a critical requirement for industry LLM systems for real deployments. However, practical 1Mcontext serving for open-source models has remained heavily CUDA-centric, largely because the required long-context runtime features are not fully available on AMD ROCm. With AMD GPUs now reported at around 7% market share (Silicon Analysts Research, 2026), this […]<\/p>\n","protected":false},"author":351,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[11,3],"tags":[1438,2191,1782,2155,20,2170,176,2167,2182],"class_list":["post-31446","post","type-post","status-publish","format-standard","hentry","category-computer-science","category-paper","tag-amd","tag-amd-radeon-instinct-mi355x","tag-computer-science","tag-llm","tag-nvidia","tag-nvidia-b200","tag-package","tag-rocm","tag-triton"],"views":114,"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/hgpu.org\/index.php?rest_route=\/wp\/v2\/posts\/31446","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hgpu.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hgpu.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hgpu.org\/index.php?rest_route=\/wp\/v2\/users\/351"}],"replies":[{"embeddable":true,"href":"https:\/\/hgpu.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=31446"}],"version-history":[{"count":0,"href":"https:\/\/hgpu.org\/index.php?rest_route=\/wp\/v2\/posts\/31446\/revisions"}],"wp:attachment":[{"href":"https:\/\/hgpu.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=31446"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hgpu.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=31446"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hgpu.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=31446"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}