high performance computing on graphics processing units: hgpu.org

Deep Learning powers a variety of applications from self driving cars and autonomous robotics to web search and voice assistants. It is fair to say that it is omnipresent and here to stay. It is deployed in all sorts of devices ranging from consumer electronics to Internet of Things (IoT). Such a deployment is categorized as inference at the edge. This thesis focuses on Deep Learning on one such edge device – Mobile Phone. The thesis surveys the space of Deep Learning deployment on mobile devices, and identifies three key problems – (a) lack of common programming interface, (b) dearth of benchmarking systems and (c) shortage of in-depth performance evaluation. Then, it provides a solution to each one of them by (a) providing a common interface derived from MLModelScope, referred to as mobile Predictor (mPredictor), (b) providing a benchmarking application and (c) using aforementioned mPredictor and benchmarking application to perform a detailed evaluation. This work has been developed to assist a generic mobile developer in integrating Deep Learning service in his/her application.