Realizing Accelerated Cost-Effective Distributed RAID

Aleksandr Khasymski, M. Mustafa Rafique, Ali R. Butt, Sudharshan S. Vazhkudai, Dimitrios S. Nikolopoulos
Virginia Tech
Chapter in "Handbook on Data Centers", edited by Albert Y. Zomaya and Samee U. Khan, Springer, 2014


   title={Realizing Accelerated Cost-Effective Distributed RAID},

   author={Khasymski, Aleksandr and Rafique, M. Mustafa and Butt, Ali R. and Vazhkudai, Sudharshan S. and Nikolopoulos, Dimitrios S.},

   book={Handbook on Data Centers},



Download Download (PDF)   View View   Source Source   



The exponential growth in user and application data entails new means for providing fault tolerance and protection against data loss. High Performance Computing (HPC) storage systems, which are at the forefront of handling the data deluge, typically employ hardware RAID at the backend. However, such solutions are costly, do not ensure end-to-end data integrity, and can become a bottleneck during data reconstruction. In this paper, we design an innovative solution to achieve a flexible, fault-tolerant, and high-performance RAID-6 solution for a parallel file system (PFS). Our system utilizes low-cost, strategically placed GPUs-both on the client and server sides – to accelerate parity computation. In contrast to hardware-based approaches, we provide full control over the size, length and location of a RAID array on a per file basis, end-to-end data integrity checking, and parallelization of RAID array reconstruction. We have deployed our system in conjunction with the widely-used Lustre PFS, and show that our approach is feasible and imposes acceptable overhead.
No votes yet.
Please wait...

* * *

* * *

* * *

HGPU group © 2010-2023 hgpu.org

All rights belong to the respective authors

Contact us: