Profiling Heterogeneous Multi-GPU Systems to Accelerate Cortically Inspired Learning Algorithms

Andrew Nere, Atif Hashmi, Mikko Lipasti
Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA
IEEE International Parallel and Distributed Processing Symposium, IPDPS 2011


   title={Profiling Heterogeneous Multi-GPU Systems to Accelerate Cortically Inspired Learning Algorithms},

   author={Nere, A. and Hashmi, A. and Lipasti, M.},

   booktitle={IEEE International Parallel and Distributed Processing Symposium, IPDPS 2011},



Download Download (PDF)   View View   Source Source   



Recent advances in neuroscientific understanding make parallel computing devices modeled after the human neocortex a plausible, attractive, fault-tolerant, and energye-fficient possibility. Such attributes have once again sparked an interest in creating learning algorithms that aspire to reverseengineer many of the abilities of the brain. In this paper we describe a GPGPU-accelerated extension to an intelligent learning model inspired by the structural and functional properties of the mammalian neocortex. Our cortical network, like the brain, exhibits massive amounts of processing parallelism, making today’s GPGPUs a highly attractive and readily-available hardware accelerator for such a model. Furthermore, we consider two inefficiencies inherent to our initial design: multiple kernel-launch overhead and poor utilization of GPGPU resources. We propose optimizations such as a software work-queue structure and pipelining the hierarchical layers of the cortical network to mitigate such problems. Our analysis provides important insight into the GPU architecture details including the number of cores, the memory system, and the global thread scheduler. Additionally, we create a runtime profiling tool for our parallel learning algorithm which proportionally distributes the cortical network across the host CPU as well as multiple GPUs, whether homogeneous or heterogeneous, that may be available to the system. Using the profiling tool with these optimizations on Nvidia’s CUDA framework, we achieve up to 60x speedup over a single-threaded CPU implementation of the model.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2021 hgpu.org

All rights belong to the respective authors

Contact us: