high performance computing on graphics processing units: hgpu.org

Training of Artificial Neural Networks for large data sets is a time consuming task. Various approaches have been proposed to reduce the efforts, many of them by applying parallelization techniques. In this paper we develop and analyze two novel parallel training approaches for Backpropagation neural networks for face recognition. We focus on two specific parallelization environments, using on the one hand OpenMP on a conventional multithreaded CPU and CUDA on a GPU. Based on our findings we give guidelines for the efficient parallelization of Backpropagation neural networks on multicore and GPU architectures. Additionally, we present a traversal method finding the best combination of learning rate and momentum term by varying the number of hidden neurons supporting the parallelization efforts.