19520

An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices

Xiaolong Ma, Wei Niu, Tianyun Zhang, Sijia Liu, Fu-Ming Guo, Sheng Lin, Hongjia Li, Xiang Chen, Jian Tang, Kaisheng Ma, Bin Ren, Yanzhi Wang
Northeastern University
arXiv:2001.07710 [cs.CV], (20 Jan 2020)

@misc{ma2020image,

   title={An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices},

   author={Xiaolong Ma and Wei Niu and Tianyun Zhang and Sijia Liu and Fu-Ming Guo and Sheng Lin and Hongjia Li and Xiang Chen and Jian Tang and Kaisheng Ma and Bin Ren and Yanzhi Wang},

   year={2020},

   eprint={2001.07710},

   archivePrefix={arXiv},

   primaryClass={cs.CV}

}

Weight pruning has been widely acknowledged as a straightforward and effective method to eliminate redundancy in Deep Neural Networks (DNN), thereby achieving acceleration on various platforms. However, most of the pruning techniques are essentially trade-offs between model accuracy and regularity which lead to impaired inference accuracy and limited on-device acceleration performance. To solve the problem, we introduce a new sparsity dimension, namely pattern-based sparsity that comprises pattern and connectivity sparsity, and becoming both highly accurate and hardware friendly. With carefully designed patterns, the proposed pruning unprecedentedly and consistently achieves accuracy enhancement and better feature extraction ability on different DNN structures and datasets, and our pattern-aware pruning framework also achieves pattern library extraction, pattern selection, pattern and connectivity pruning and weight training simultaneously. Our approach on the new pattern-based sparsity naturally fits into compiler optimization for highly efficient DNN execution on mobile platforms. To the best of our knowledge, it is the first time that mobile devices achieve real-time inference for the large-scale DNN models thanks to the unique spatial property of pattern-based sparsity and the help of the code generation capability of compilers.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2024 hgpu.org

All rights belong to the respective authors

Contact us: