30036

Serving LLMs in HPC Clusters: A Comparative Study of Qualcomm Cloud AI 100 Ultra and High-Performance GPUs

Mohammad Firas Sada, John J. Graham, Elham E Khoda, Mahidhar Tatineni, Dmitry Mishin, Rajesh K. Gupta, Rick Wagner, Larry Smarr, Thomas A. DeFanti, Frank Würthwein
University of California, San Diego, La Jolla, CA, USA
arXiv:2507.00418 [cs.DC], (1 Jul 2025)

@article{sada2025serving,

   title={Serving LLMs in HPC Clusters: A Comparative Study of Qualcomm Cloud AI 100 Ultra and High-Performance GPUs},

   author={Sada, Mohammad Firas and Graham, John J and Khoda, Elham E and Tatineni, Mahidhar and Mishin, Dmitry and Gupta, Rajesh K and Wagner, Rick and Smarr, Larry and DeFanti, Thomas A and W{~A}{v{z}}rthwein, Frank},

   journal={arXiv preprint arXiv:2507.00418},

   year={2025}

}

Download Download (PDF)   View View   Source Source   

888

views

This study presents a benchmarking analysis of the Qualcomm Cloud AI 100 Ultra (QAic) accelerator for large language model (LLM) inference, evaluating its energy efficiency (throughput per watt) and performance against leading NVIDIA (A100, H200) and AMD (MI300A) GPUs within the National Research Platform (NRP) ecosystem. A total of 15 open-source LLMs, ranging from 117 million to 90 billion parameters, are served using the vLLM framework. The QAic inference cards appears to be energy efficient and performs well in the energy efficiency metric in most cases. The findings offer insights into the potential of the Qualcomm Cloud AI 100 Ultra for high-performance computing (HPC) applications within the National Research Platform (NRP).
No votes yet.
Please wait...

You must be logged in to post a comment.

* * *

* * *

HGPU group © 2010-2025 hgpu.org

All rights belong to the respective authors

Contact us: