high performance computing on graphics processing units: hgpu.org

With the advancement of HPC platforms, the demand for high-performing applications continues to grow. One effective way to enhance program performance is through parallelization. However, fully leveraging the powerful hardware of HPC platforms poses significant challenges. Even experienced developers must carefully consider factors such as runtime, memory usage, and thread-scheduling overhead. Additionally, achieving successful parallelization often requires running applications to determine the optimal configurations. In this paper, we propose ConTraPh, a framework that integrates Contrastive Learning with Transformers and Graph Neural Networks to capture the inherent parallel characteristics of source programs through a multi-view program representation, utilizing both source code and compiler intermediate representations. This contrastive learning framework allows the model to effectively learn correct parallel configurations from positive samples while avoiding incorrect ones through negative samples. We evaluate ConTraPh on six downstream tasks involving three different parallel programming models OpenMP, OpenCL and, OpenACC that include OpenMP clause prediction, performant reduction style detection, performant scheduling type detection, CPU/GPU parallelism prediction, Heterogeneous Device Mapping for OpenCL code, and OpenACC clause prediction. ConTraPh outperforms state-of-the-art models in these tasks, achieving accuracy improvements of up to 8%, 10%, 7%, 4%, 2%, and 9%, respectively. ConTraPh achieves speedups as high as 13x, 18x, 14x, and 4.4x on the reduction style detention, scheduling type detection, CPU/GPU parallelism prediction, and Heterogeneous Decive Mapping tasks, respectively. Results also demonstrate that ConTraPh can be integrated with third-party tools, such as large language models (LLMs), to enhance the performance of state-of-theart models like GPT-4 by up to 15% based on established code generation metrics, such as CodeBERTScore.