What is a useful skill hierarchy for an autonomous agent? We propose an answer based on a graphical representation of how the interaction between an agent and its environment may unfold. Our approach uses modularity maximisation as a central organising principle to expose the structure of the interaction graph at multiple levels of abstraction. The result is a collection of skills that operate at varying time scales, organised into a hierarchy, where skills that operate over longer time scales are composed of skills that operate over shorter time scales. The entire skill hierarchy is generated automatically, with no human intervention, including the skills themselves (their behaviour, when they can be called, and when they terminate) as well as the hierarchical dependency structure between them. In a wide range of environments, this approach generates skill hierarchies that are intuitively appealing and that considerably improve the learning performance of the agent.
Original languageEnglish
Number of pages13
Publication statusPublished - 16 Jan 2024
Event37th Conference on Neural Processing Systems - New Orleans Ernest N. Morial Convention Center, New Orleans, USA United States
Duration: 10 Dec 202316 Dec 2023


Conference37th Conference on Neural Processing Systems
Abbreviated titleNeurIPS 2023
Country/TerritoryUSA United States
CityNew Orleans
Internet address

ASJC Scopus subject areas

  • Artificial Intelligence


Dive into the research topics of 'Creating Multi-Level Skill Hierarchies in Reinforcement Learning'. Together they form a unique fingerprint.

Cite this