Understanding GPU Triggering APIs for MPI+X Communication

Patrick G. Bridges, Anthony Skjellum, Evan D. Suggs, Derek Schafer, Purushotham V. Bangalore
University of New Mexico, Albuquerque NM, 87131
arXiv:2406.05594 [cs.DC], (8 Jun 2024)


   title={Understanding GPU Triggering APIs for MPI+X Communication},

   author={Patrick G. Bridges and Anthony Skjellum and Evan D. Suggs and Derek Schafer and Purushotham V. Bangalore},




   primaryClass={id=’cs.DC’ full_name=’Distributed, Parallel, and Cluster Computing’ is_active=True alt_name=None in_archive=’cs’ is_general=False description=’Covers fault-tolerance, distributed algorithms, stabilility, parallel computation, and cluster computing. Roughly includes material in ACM Subject Classes C.1.2, C.1.4, C.2.4, D.1.3, D.4.5, D.4.7, E.1.’}


Download Download (PDF)   View View   Source Source   



GPU-enhanced architectures are now dominant in HPC systems, but message-passing communication involving GPUs with MPI has proven to be both complex and expensive, motivating new approaches that lower such costs. We compare and contrast stream/graph- and kernel-triggered MPI communication abstractions, whose principal purpose is to enhance the performance of communication when GPU kernels create or consume data for transfer through MPI operations. Researchers and practitioners have proposed multiple potential APIs for stream and/or kernel triggering that span various GPU architectures and approaches, including MPI-4 partitioned point-to-point communication, stream communicators, and explicit MPI stream/queue objects. Designs breaking backward compatibility with MPI are duly noted. Some of these strengthen or weaken the semantics of MPI operations. A key contribution of this paper is to promote community convergence toward a stream- and/or kernel-triggering abstraction by highlighting the common and differing goals and contributions of existing abstractions. We describe the design space in which these abstractions reside, their implicit or explicit use of stream and other non-MPI abstractions, their relationship to partitioned and persistent operations, and discuss their potential for added performance, how usable these abstractions are, and where functional and/or semantic gaps exist. Finally, we provide a taxonomy for stream- and kernel-triggered abstractions, including disambiguation of similar semantic terms, and consider directions for future standardization in MPI-5.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2024 hgpu.org

All rights belong to the respective authors

Contact us: