Improving the speed of neural networks on CPUs

Vincent Vanhoucke, Andrew Senior, Mark Z. Mao
Google, Inc., Mountain View, CA 94043
Deep Learning and Unsupervised Feature Learning Workshop, NIPS, 2011


   title={Improving the speed of neural networks on CPUs},

   author={Vanhoucke, V. and Senior, A. and Mao, M.Z.},



Download Download (PDF)   View View   Source Source   



Recent advances in deep learning have made the use of large, deep neural networks with tens of millions of parameters suitable for a number of applications that require real-time processing. The sheer size of these networks can represent a challenging computational burden, even for modern CPUs. For this reason, GPUs are routinely used instead to train and run such networks. This paper is a tutorial for students and researchers on some of the techniques that can be used to reduce this computational cost considerably on modern x86 CPUs. We emphasize data layout, batching of the computation, the use of SSE2 instructions, and particularly leverage SSSE3 and SSE4 fixed-point instructions which provide a 3x improvement over an optimized floating-point baseline. We use speech recognition as an example task, and show that a real-time hybrid hidden Markov model / neural network (HMM/NN) large vocabulary system can be built with a 10x speedup over an unoptimized baseline and a 4x speedup over an aggressively optimized floating-point baseline at no cost in accuracy. The techniques described extend readily to neural network training and provide an effective alternative to the use of specialized hardware.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2021 hgpu.org

All rights belong to the respective authors

Contact us: