Abstract
MLtuner automatically tunes settings for training tunables (such as the
learning rate, the momentum, the mini-batch size, and the data staleness bound)
that have a significant impact on large-scale machine learning (ML)
performance. Traditionally, these tunables are set manually, which is
unsurprisingly error-prone and difficult to do without extensive domain
knowledge. MLtuner uses efficient snapshotting, branching, and
optimization-guided online trial-and-error to find good initial settings as
well as to re-tune settings during execution. Experiments show that MLtuner can
robustly find and re-tune tunable settings for a variety of ML applications,
including image classification (for 3 models and 2 datasets), video
classification, and matrix factorization. Compared to state-of-the-art ML
auto-tuning approaches, MLtuner is more robust for large problems and over an
order of magnitude faster.
Users
Please
log in to take part in the discussion (add own reviews or comments).