A Survey of Big Data, High Performance Computing, and Machine Learning Benchmarks

Nina Ihde, Paula Marten, Ahmed Eleliemy, Gabrielle Poerwawinata, Pedro Silva, Ilin Tolovski, Florina M. Ciorba, Tilmann Rabl
Hasso Platner Institute, Potsdam, Germany
Hasso Platner Institute, 2021


   title={A Survey of Big Data, High Performance Computing, and Machine Learning Benchmarks},

   author={Ihde, Nina and Marten, Paula and Eleliemy, Ahmed and Poerwawinata, Gabrielle and Silva, Pedro and Tolovski, Ilin and Ciorba, Florina M and Rabl, Tilmann},



Download Download (PDF)   View View   Source Source   



In recent years, there has been a convergence of Big Data (BD), High Performance Computing (HPC), and Machine Learning (ML) systems. This convergence is due to the increasing complexity of long data analysis pipelines on separated software stacks. With the increasing complexity of data analytics pipelines comes a need to evaluate their systems, in order to make informed decisions about technology selection, sizing and scoping of hardware. While there are many benchmarks for each of these domains, there is no convergence of these efforts. As a first step, it is also necessary to understand how the individual benchmark domains relate. In this work, we analyze some of the most expressive and recent benchmarks of BD, HPC, and ML systems. We propose a taxonomy of those systems based on individual dimensions such as accuracy metrics and common dimensions such as workload type. Moreover, we aim at enabling the usage of our taxonomy in identifying adapted benchmarks for their BD, HPC, and ML systems. Finally, we identify challenges and research directions related to the future of converged BD, HPC, and ML system benchmarking.
No votes yet.
Please wait...

* * *

* * *

HGPU group © 2010-2021 hgpu.org

All rights belong to the respective authors

Contact us: