high performance computing on graphics processing units: hgpu.org

Large scale deep neural networks profited from an emerging class of AI accelerators. Although the accelerators are specialized for machine learning, some of their designs are general enough for other computing intensive applications. Cloud TPU, as one of them, offers tremendous computing resources and is easily accessible through TensorFlow by expressing the computation in a graph. In this paper, we leverage this powerful hardware combined with the expressiveness of TensorFlow to simulate the Ising model on a 2-dimensional lattice. We modify the computationally intensive part of the checkerboard algorithm into matrix operations to exploit Cloud TPU’s highly efficient matrix unit. In our experiments, we demonstrate that our implementation outperforms the best published benchmarks to our knowledge by 60% in single core and 250% in multiple cores with linear scaling. We also show the performance improvement of using low precision arithmetic—bfloat16 instead of float32—without sacrificing any accuracy.