Abstract
This thesis discusses Markov Chain Monte Carlo (MCMC) sampling algorithms which make use of differential geometry to improve performance. In particular, new sampling algorithms are introduced which exploit the dual geometry of statistical models developed by Amari, Chentsov and other authors in the field of Information Geometry.Previous work on differentialgeometric MCMC has only used Riemannian geometry, i.e. without the dual structures studied in Information Geometry.
Background chapters of the thesis discuss the development of Information Geometry by formulating statistical models as manifolds of datagenerating measures. We review how Bregman divergences on such manifolds give rise to the duallyflat structure of Information Geometry, with examples provided by exponential families. Furthermore, Bayes' theorem is recast in a coordinate free form.
Two novel sampling algorithms are introduced, called Orthogonal Gibbs and Orthogonal Gradient, which make use of duallyflat geometry. The Orthogonal Gibbs algorithm samples primal variables conditionally on dual variables and vice versa; similarly Orthogonal Gradient moves along the primal geodesic whose tangent vector is proportional to the gradient of the target logdensity, and samples recursively on an mflat submanifold orthogonal to the eflat geodesic. We show that both algorithms move in directions in parameter space which are orthogonal with respect to the Riemannian metric. Furthermore, we show that when applied to a multivariate Gaussian target distribution, the Orthogonal Gibbs and Orthogonal Gradient algorithm respectively reduce to the Gaussian Elimination and Conjugate Gradient solvers for linear systems with added stochastic terms in the updates to the solution. The latter finding reflects work by previous authors on the connection between MCMC and optimization, including that of Fox and Parker. We show that the Orthogonal Gibbs algorithm produces independent samples for multivariate Gaussian targets, and produce numerical results suggesting the analogous result for Orthogonal Gradient. Finally, we present numerical results for a nonGaussian target distribution for which the novel algorithms have favourable effective sample sizes compared to the No UTurn Sampler (NUTS).
Date of Award  12 Oct 2022 

Original language  English 
Awarding Institution 

Supervisor  Karim AnayaIzquierdo (Supervisor) & Robert Scheichl (Supervisor) 