high performance computing on graphics processing units: hgpu.org

Nowadays artificial neural networks are widely used to accurately classify and recognize patterns. An interesting application area is the Internet of Things (IoT), where physical things are connected to the Internet, and generate a huge amount of sensor data that can be used for a myriad of new, pervasive applications. Neural networks’ ability to comprehend unstructured data make them a useful building block for such IoT applications. As neural networks require a lot of processing power, especially during the training phase, these are most often deployed in a cloud environment, or on specialized servers with dedicated GPU hardware. However, for IoT applications, sending all raw data to a remote back-end might not be feasible, taking into account the high and variable latency to the cloud, or could lead to issues concerning privacy. In this paper the DIANNE middleware framework is presented that is optimized for single sample feed-forward execution and facilitates distributing artificial neural networks across multiple IoT devices. The modular approach enables executing neural network components on a large number of heterogeneous devices, allowing us to exploit the local compute power at hand, and mitigating the need for a large server-side infrastructure at runtime.