Inexact Derivative-Free Optimization for Bilevel Learning

Matthias J. Ehrhardt, Lindon Roberts

Research output: Contribution to journalArticlepeer-review

1 Citation (SciVal)
11 Downloads (Pure)


Variational regularization techniques are dominant in the field of mathematical imaging. A drawback of these techniques is that they are dependent on a number of parameters which have to be set by the user. A by now common strategy to resolve this issue is to learn these parameters from data. While mathematically appealing this strategy leads to a nested optimization problem (known as bilevel optimization) which is computationally very difficult to handle. A key ingredient in solving the upper-level problem is the exact solution of the lower-level problem which is practically infeasible. In this work we propose to solve these problems using inexact derivative-free optimization algorithms which never require to solve the lower-level problem exactly. We provide global convergence and worst-case complexity analysis of our approach, and test our proposed framework on ROF-denoising and learning MRI sampling patterns. Dynamically adjusting the lower-level accuracy yields learned parameters with similar reconstruction quality as high-accuracy evaluations but with dramatic reductions in computational work (up to 100 times faster in some cases).
Original languageEnglish
JournalJournal of Mathematical Imaging and Vision
Early online date6 Feb 2021
Publication statusE-pub ahead of print - 6 Feb 2021


  • math.OC
  • cs.CV
  • cs.LG
  • cs.NA
  • math.NA
  • stat.ML


Dive into the research topics of 'Inexact Derivative-Free Optimization for Bilevel Learning'. Together they form a unique fingerprint.

Cite this