hgpu.org » Applications » Computer science » Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs
Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs
Haoyu Zhang, Arshad Rahman, Adnan Aziz, Ajith Prabhudev, Amin Farmahini, Ben Rhoads, Benson Ma, Brian Shi, Cen Zhao, CK Luk, Dingming Wu, Emad El-Haraty, Eric Chen, Frank Li, Gary Huang, Gefei Zuo, Haoran Zhang, Hiwot Kassa, Hongyi Zeng, Jack Taylor, Jeremy Hadidjojo, John Donati, Jonathan Sheffield, Julio Machado Silva, Karthik Ganesan, Kent Russell, KR Kishore, Lei Tian, Liu Ke, Muhammad Osama, Peng Chen, Rakesh Hemnani, Ryan Swann, Saleel Kudchadker, Shankar Yanamandram, Shobhit Kanaujia, Shucai Xiao, Srinath Bayareddy, Srinivas Vaidyanathan, Sudharssun Subramanian, Supadchaya Puangpontip, Taylor Robie, Tianhao Huang, Tyler Hart, Viswesh Sankaran, Weijun Jiang, Wenkai Du, Xiaodong Wang, Xiaohu Guo, Yao Rong, Yinbin Ma, Yue Dong, Yuzhen Huang, Yvonne Liu, Zhan Shu, Dan Rabinovitsj, Ajit Mathews, Chunqiang Tang
title={Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs},
author={Haoyu Zhang and Arshad Rahman and Adnan Aziz and Ajith Prabhudev and Amin Farmahini and Ben Rhoads and Benson Ma and Brian Shi and Cen Zhao and CK Luk and Dingming Wu and Emad El-Haraty and Eric Chen and Frank Li and Gary Huang and Gefei Zuo and Haoran Zhang and Hiwot Kassa and Hongyi Zeng and Jack Taylor and Jeremy Hadidjojo and John Donati and Jonathan Sheffield and Julio Machado Silva and Karthik Ganesan and Kent Russell and KR Kishore and Lei Tian and Liu Ke and Muhammad Osama and Peng Chen and Rakesh Hemnani and Ryan Swann and Saleel Kudchadker and Shankar Yanamandram and Shobhit Kanaujia and Shucai Xiao and Srinath Bayareddy and Srinivas Vaidyanathan and Sudharssun Subramanian and Supadchaya Puangpontip and Taylor Robie and Tianhao Huang and Tyler Hart and Viswesh Sankaran and Weijun Jiang and Wenkai Du and Xiaodong Wang and Xiaohu Guo and Yao Rong and Yinbin Ma and Yue Dong and Yuzhen Huang and Yvonne Liu and Zhan Shu and Dan Rabinovitsj and Ajit Mathews and Chunqiang Tang},
Meta deploys a heterogeneous fleet comprising NVIDIA GPUs, AMD GPUs, and proprietary MTIA accelerators to reduce costs and mitigate vendor lock-in. However, the extensive history of NVIDIA-specific optimizations hinders performance portability to other hardware. This paper presents our methods for optimizing alternative accelerators, using AMD Instinct MI350X versus NVIDIA B200 for recommendation model training as a case study. We propose three reusable principles. First, hardware-model co-design is essential: rather than directly porting models from NVIDIA GPUs, revising model structures and software recipes allows full exploitation of accelerator-specific strengths (e.g., MI350X's 1.6x HBM capacity relative to B200). Second, our analytical performance model enables zero-warmup optimal kernel selection across diverse datatypes and matrix shapes in production models, drastically simplifying tuning. Third, increasingly mature agentic workflows have lowered the barrier to porting from the CUDA ecosystem; we extend these workflows for microarchitecture-aware kernel optimizations and custom shim generation. Compared to the directly ported baseline, these methods deliver up to 1.5x improvement in end-to-end training throughput. To our knowledge, this is the first hyperscaler report on productionizing AMD GPUs for large-scale training. Our contributions have been upstreamed to the open-source PyTorch, Triton, and ROCm ecosystems.
