Skip to main navigation Skip to search Skip to main content

Abstract

Efficient exploration remains one of the most important open problems in reinforcement learning. Discovering novel states or transitions requires policies that efficiently direct the agent away from the regions of the state space that are already well explored. We introduce Novel Exploration via Orthogonality (NEO), an approach that automatically uncovers not only which regions of the environment are novel but also how to reach them by leveraging Laplacian representations. NEO uses the eigenvectors of a modified graph Laplacian to induce gradient flows from states that are frequently visited (less novel) to states that are seldom visited (more novel). We show that NEO's modified Laplacian yields eigenvectors whose extreme values align with the most novel regions of the state space. We provide bounds for the eigenvalues of the modified Laplacian; and we show that the smoothest eigenvectors with real eigenvalues below certain thresholds provide guaranteed gradients to novel states for both undirected and directed graphs. In an empirical evaluation in online, incremental settings, NEO outperformed related state-of-the-art approaches, including eigen-options and cover options, in a large collection of undirected and directed environments with varying connectivity structures.
Original languageEnglish
Publication statusPublished - 18 Sept 2025
EventThe Thirty-Ninth Annual Conference on Neural Information Processing Systems - San Diego, USA United States
Duration: 2 Dec 2025 → 7 Dec 2025

Conference

ConferenceThe Thirty-Ninth Annual Conference on Neural Information Processing Systems
Country/TerritoryUSA United States
CitySan Diego
Period2/12/25 → 7/12/25

Keywords

  • Laplacian, Novelty, Reinforcement Learning, Exploration, Eigenvectors, Spectral Methods

Fingerprint

Dive into the research topics of 'Novel Exploration via Orthogonality'. Together they form a unique fingerprint.

Cite this