Efficient mapping of the training of Convolutional Neural Networks to a CUDA-based cluster
Eindhoven University of Technology, The Netherlands
Eindhoven University of Technology, 2011
@article{ward2011efficient,
title={Efficient mapping of the training of Convolutional Neural Networks to a CUDA-based cluster},
author={Ward, Jonatan and Andreev, Sergey and Heredia, Francisco and Lazar, Bogdan and Manevska, Zlatka},
year={2011}
}
We propose a method to parallelize the training of a convolutional neural network by using a CUDA-based cluster. We attain a substantial increase in the performance of the algorithm itself. We research the feasibility of using batch versus online mode training and provide a performance comparison between them. Furthermore, we propose an implementation of an alternative algorithm to compute local gradients which increases the level of parallelism. To conclude, we give a set of best practices for implementing Convolutional Neural Networks on the cluster.
January 4, 2012 by hgpu
