7700

Hierarchical Partitioning Algorithm for Scientific Computing on Highly Heterogeneous CPU + GPU Clusters

David Clarke, Aleksandar Ilic, Alexey Lastovetsky, Leonel Sousa
School of Computer Science and Informatics, University College Dublin, Belfield, Dublin 4, Ireland
18th International European Conference on Parallel and Distributed Computing (Euro-Par 2012), 2012

@conference{clarke2012hierarchical,

   title={Hierarchical Partitioning Algorithm for Scientific Computing on Highly Heterogeneous CPU + GPU Clusters},

   booktitle={18th International European Conference on Parallel and Distributed Computing (Euro-Par 2012)},

   year={2012},

   month={27-31 August},

   address={Rhodes Island, Greece},

   attachments={http://hcl.ucd.ie/system/files/Hierarchical_Partitioing.pdf},

   author={Clarke, D. and Ilic, A. and Lastovetsky, A. and Sousa, L.}

}

Download Download (PDF)   View View   Source Source   

1052

views

Hierarchical level of heterogeneity exists in many modern high performance clusters in the form of heterogeneity between computing nodes, and within a node with the addition of specialized accelerators, such as GPUs. To achieve high performance of scientific applications on these platforms it is necessary to perform load balancing. In this paper we present a hierarchical matrix partitioning algorithm based on realistic performance models at each level of hierarchy. To minimise the total execution time of the application it iteratively partitions a matrix between nodes and partitions these sub-matrices between the devices in a node. This is a self-adaptive algorithm that dynamically builds the performance models at run-time and it employs an algorithm to minimise the total volume of communication. This algorithm allows scientific applications to perform load balanced matrix operations with nested parallelism on hierarchical heterogeneous platforms. To show the effectiveness of the algorithm we applied it to a fundamental operation in scientific parallel computing, matrix multiplication. Large scale experiments on a heterogeneous multi-cluster site incorporating multicore CPUs and GPU nodes show that the presented algorithm outperforms current state of the art approaches and successfully load balance very large problems.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2017 hgpu.org

All rights belong to the respective authors

Contact us: