Identifying Multi-Level Behaviour Hierarchies Using Modularity Maximisation

Abstract

The broad problem I address in this dissertation is the design of autonomous agents that can learn efficiently and act effectively by organising their behaviour hierarchically. I propose a characterisation of such hierarchies based on a graphical representation of the interaction between an agent and its environment. From this representation, I define a class of behaviours — or skills — derived from partitions of the interaction graph that maximise modularity, a measure of how much more densely connected the nodes within each cluster are than would be expected by chance. These modularity maximising skills enable efficient movement between regions of the environment that are difficult to traverse through random exploration. Building on this, I use hierarchical graph partitioning algorithms to produce a sequence of partitions of the interaction graph that capture the environment's structure at different levels of granularity. I then use these partitions to produce the modularity maximising skill hierarchy, a fully and automatically specified hierarchy in which higher-level skills acting over longer timescales are composed of lower-level skills acting over shorter ones. Finally, I explore how such hierarchies can be constructed and acquired incrementally in practice, laying a foundation for developing autonomous agents capable of building useful behaviour hierarchies for solving complex problems.

Date of Award	22 Apr 2026
Original language	English
Awarding Institution	University of Bath
Supervisor	Özgür Şimşek (Supervisor) & Michael Tipping (Supervisor)