17021

CHAOS: A Parallelization Scheme for Training Convolutional Neural Networks on Intel Xeon Phi

Andre Viebke, Suejb Memeti, Sabri Pllana, Ajith Abraham
Linnaeus University, Department of Computer Science, 351 95 Vaxjo, Sweden
arXiv:1702.07908 [cs.DC], (25 Feb 2017)

@article{viebke2017chaos,

   title={CHAOS: A Parallelization Scheme for Training Convolutional Neural Networks on Intel Xeon Phi},

   author={Viebke, Andre and Memeti, Suejb and Pllana, Sabri and Abraham, Ajith},

   year={2017},

   month={feb},

   archivePrefix={"arXiv"},

   primaryClass={cs.DC},

   doi={10.1007/s11227-017-1994-x}

}

Download Download (PDF)   View View   Source Source   

849

views

Deep learning is an important component of big-data analytic tools and intelligent applications, such as, self-driving cars, computer vision, speech recognition, or precision medicine. However, the training process is computationally intensive, and often requires a large amount of time if performed sequentially. Modern parallel computing systems provide the capability to reduce the required training time of deep neural networks. In this paper, we present our parallelization scheme for training convolutional neural networks (CNN) named Controlled Hogwild with Arbitrary Order of Synchronization (CHAOS). Major features of CHAOS include the support for thread and vector parallelism, non-instant updates of weight parameters during back-propagation without a significant delay, and implicit synchronization in arbitrary order. CHAOS is tailored for parallel computing systems that are accelerated with the Intel Xeon Phi. We evaluate our parallelization approach empirically using measurement techniques and performance modeling for various numbers of threads and CNN architectures. Experimental results for the MNIST dataset of handwritten digits using the total number of threads on the Xeon Phi show speedups of up to 103x compared to the execution on one thread of the Xeon Phi, 14x compared to the sequential execution on Intel Xeon E5, and 58x compared to the sequential execution on Intel Core i5.
Rating: 2.4/5. From 12 votes.
Please wait...

* * *

* * *

Featured events

2018
November
27-30
Hida Takayama, Japan

The Third International Workshop on GPU Computing and AI (GCA), 2018

2018
September
19-21
Nagoya University, Japan

The 5th International Conference on Power and Energy Systems Engineering (CPESE), 2018

2018
September
22-24
MediaCityUK, Salford Quays, Greater Manchester, England

The 10th International Conference on Information Management and Engineering (ICIME), 2018

2018
August
21-23
No. 1037, Luoyu Road, Hongshan District, Wuhan, China

The 4th International Conference on Control Science and Systems Engineering (ICCSSE), 2018

2018
October
29-31
Nanyang Executive Centre in Nanyang Technological University, Singapore

The 2018 International Conference on Cloud Computing and Internet of Things (CCIOT’18), 2018

HGPU group © 2010-2018 hgpu.org

All rights belong to the respective authors

Contact us: