Theoretical and Algorithmic Advances in Bilevel Learning for Inverse Problems

Abstract

Inverse problems are those in which one seeks some quantity of interest but only has access to an indirect measurement of said quantity. From engineering to medical imaging, they are a fundamental aspect of various scientific domains. In many of these applications the inverse problems are inherently ill-posed, be it that solutions are highly sensitive to perturbations in the observed measurements, or there may be numerous possible solutions and one must somehow select the one deemed most appropriate.

Given how important inverse problems are, there is a huge amount of literature on techniques that try to remedy the ill-posedness. One classical approach is variational regularisation, wherein one recovers a reconstruction by minimising a cost function that penalises undesirable properties that may be known in advance. Variational regularisation is a well studied technique, with a lot of theoretical results regarding the well-posedness and behaviour of solutions. In practice the constituent parts of the considered cost function for variational regularisation tend to be hand selected from a collection of choices that have been seen to yield reasonable results, with limited scope to be adapted to a specific application. Given the sheer diversity of applications that inverse problems cover, this manual determination of the cost function is a bottleneck for the performance that variational regularisation could otherwise obtain. To overcome this barrier, one could employ machine learning to determine a cost function tailored to a specific application. One framework to achieve this is bilevel learning, a nested optimisation problem wherein reconstructions are minimisers of a parametrised cost function, and suitable parameter values are determined in a potentially supervised manner.

There are several challenges with the bilevel learning approach. Firstly, due to the inherent nested nature of the problem, it is computationally expensive to solve. Secondly, it is challenging to determine the well-posedness of the learning and characterisation of solutions. In this thesis we have two main contributions, one for both of these challenges.

In Chapter 1 we specify the bilevel learning problem formally, and in Chapter 2 discuss more broadly where it sits in the literature of both classical and machine learning approaches for solving inverse problems.

We address the computational aspect of bilevel learning in Chapter 3, where we consider a gradient based approach for solving the problem wherein the formulation of each gradient is expensive due to solutions of both the variational problem and a large-scale linear system being required. We propose a technique for solving the sequence of linear systems that recycles information between each solve to yield a computational speed up while acknowledging the bilevel setting in which these linear systems arise from.

In Chapter 4 we contribute towards the well-posedness of bilevel learning as a mathematically sound regularisation parameter choice strategy. More precisely, we consider the setting of learning a single regularisation parameter and, by considering a generalised directional derivative, determine a new condition that characterises whether zero will be a solution to the bilevel problem. The new condition not only provides a much better characterisation of positivity than existing theory, but is also applicable to a broader class of inverse problems.

Date of Award	25 Jun 2025
Original language	English
Awarding Institution	University of Bath
Supervisor	Matthias Ehrhardt (Supervisor) & Silvia Gazzola (Supervisor)