18839

TonY: An Orchestrator for Distributed Machine Learning Jobs

Anthony Hsu, Keqiu Hu, Jonathan Hung, Arun Suresh, Zhe Zhang
LinkedIn
arXiv:1904.01631 [cs.DC], 24 Mar 2019

@article{hsu2019tony,

   title={TonY: An Orchestrator for Distributed Machine Learning Jobs},

   author={Hsu, Anthony and Hu, Keqiu and Hung, Jonathan and Suresh, Arun and Zhang, Zhe},

   year={2019},

   month={mar},

   archivePrefix={"arXiv"},

   primaryClass={cs.DC}

}

Training machine learning (ML) models on large datasets requires considerable computing power. To speed up training, it is typical to distribute training across several machines, often with specialized hardware like GPUs or TPUs. Managing a distributed training job is complex and requires dealing with resource contention, distributed configurations, monitoring, and fault tolerance. In this paper, we describe TonY, an open-source orchestrator for distributed ML jobs built at LinkedIn to address these challenges.
Rating: 2.0/5. From 1 vote.
Please wait...

* * *

* * *

HGPU group © 2010-2019 hgpu.org

All rights belong to the respective authors

Contact us: