Inexact Derivative-Free Optimization for Bilevel Learning

Matthias J. Ehrhardt, Lindon Roberts

Research output: Contribution to journalArticlepeer-review

8 Citations (SciVal)
11 Downloads (Pure)


Variational regularization techniques are dominant in the field of mathematical imaging. A drawback of these techniques is that they are dependent on a number of parameters which have to be set by the user. A by-now common strategy to resolve this issue is to learn these parameters from data. While mathematically appealing, this strategy leads to a nested optimization problem (known as bilevel optimization) which is computationally very difficult to handle. It is common when solving the upper-level problem to assume access to exact solutions of the lower-level problem, which is practically infeasible. In this work we propose to solve these problems using inexact derivative-free optimization algorithms which never require exact lower-level problem solutions, but instead assume access to approximate solutions with controllable accuracy, which is achievable in practice. We prove global convergence and a worst-case complexity bound for our approach. We test our proposed framework on ROF denoising and learning MRI sampling patterns. Dynamically adjusting the lower-level accuracy yields learned parameters with similar reconstruction quality as high-accuracy evaluations but with dramatic reductions in computational work (up to 100 times faster in some cases).

Original languageEnglish
Pages (from-to)580-600
Number of pages21
JournalJournal of Mathematical Imaging and Vision
Issue number5
Early online date6 Feb 2021
Publication statusPublished - Jun 2021

Bibliographical note

Funding Information:
MJE acknowledges support from the EPSRC (EP/S026045/1, EP/T026693/1), the Faraday Institution (EP/T007745/1) and the Leverhulme Trust (ECF-2019-478).

Publisher Copyright:
© 2021, The Author(s).


  • Bilevel optimization
  • Derivative-free optimization
  • Machine learning
  • Variational regularization

ASJC Scopus subject areas

  • Statistics and Probability
  • Modelling and Simulation
  • Condensed Matter Physics
  • Computer Vision and Pattern Recognition
  • Geometry and Topology
  • Applied Mathematics


Dive into the research topics of 'Inexact Derivative-Free Optimization for Bilevel Learning'. Together they form a unique fingerprint.

Cite this