Regularization of Inverse Problems: Deep Equilibrium Models versus Bilevel Learning

Danilo Riccio, Matthias J. Ehrhardt, Martin Benning

Research output: Contribution to journalArticlepeer-review

81 Downloads (Pure)

Abstract

Variational regularization methods are commonly used to approximate solutions of inverse problems. In recent years, model-based variational regularization methods have often been replaced with data-driven ones such as the fields-of-expert model (Roth and Black, 2009). Training the parameters of such data-driven methods can be formulated as a bilevel optimization problem. In this paper, we compare the framework of bilevel learning for the training of data-driven variational regularization models with the novel framework of deep equilibrium models (Bai, Kolter, and Koltun, 2019) that has recently been introduced in the context of inverse problems (Gilton, Ongie, and Willett, 2021). We show that computing the lower-level optimization problem within the bilevel formulation with a fixed point iteration is a special case of the deep equilibrium framework. We compare both approaches computationally, with a variety of numerical examples for the inverse problems of denoising, inpainting and deconvolution.
Original languageEnglish
Pages (from-to)506-530
Number of pages25
JournalNumerical Algebra, Control and Optimization
Volume15
Issue number3
Early online date31 Jan 2024
DOIs
Publication statusE-pub ahead of print - 31 Jan 2024

Funding

The authors would like to thank the Isaac Newton Institute for Mathematical Sciences, Cambridge, for support and hospitality during the programme Mathematics of Deep Learning where work on this paper was undertaken. This work was supported by EPSRC grant no EP/R014604/1. This research utilized Queen Mary’s Apocrita and Andrena HPC facilities, supported by QMUL Research-IT http://doi.org/10.5281/zenodo.438045. DR acknowledges support from EPSRC grant EP/513106/1. MJE acknowledges support from the EPSRC (EP/S026045/1, EP/T026693/1, EP/V026259/1) and the Leverhulme Trust (ECF-2019-478).

Keywords

  • math.OC

Fingerprint

Dive into the research topics of 'Regularization of Inverse Problems: Deep Equilibrium Models versus Bilevel Learning'. Together they form a unique fingerprint.

Cite this