Efficient Hyperparameter Tuning with Dynamic Accuracy Derivative-Free Optimization

Matthias J. Ehrhardt, Lindon Roberts

Research output: Contribution to journalConference articlepeer-review

24 Downloads (Pure)

Abstract

Many machine learning solutions are framed as optimization problems which rely on good hyperparameters. Algorithms for tuning these hyperparameters usually assume access to exact solutions to the underlying learning problem, which is typically not practical. Here, we apply a recent dynamic accuracy derivative-free optimization method to hyperparameter tuning, which allows inexact evaluations of the learning problem while retaining convergence guarantees. We test the method on the problem of learning elastic net weights for a logistic classifier, and demonstrate its robustness and efficiency compared to a fixed accuracy approach. This demonstrates a promising approach for hyperparameter tuning, with both convergence guarantees and practical performance.
Original languageEnglish
Journal12th OPT Workshop on Optimization for Machine Learning at NeurIPS 2020
Publication statusPublished - 6 Nov 2020

Bibliographical note

Accepted to the 12th OPT Workshop on Optimization for Machine Learning at NeurIPS 2020

Keywords

  • cs.LG
  • cs.NA
  • math.NA
  • math.OC
  • stat.ML

Fingerprint

Dive into the research topics of 'Efficient Hyperparameter Tuning with Dynamic Accuracy Derivative-Free Optimization'. Together they form a unique fingerprint.

Cite this