Abstract
Efficient exploration remains one of the most important open problems in reinforcement learning. Discovering novel states or transitions requires policies that efficiently direct the agent away from the regions of the state space that are already well explored. We introduce Novel Exploration via Orthogonality (NEO), an approach that automatically uncovers not only which regions of the environment are novel but also how to reach them by leveraging Laplacian representations. NEO uses the eigenvectors of a modified graph Laplacian to induce gradient flows from states that are frequently visited (less novel) to states that are seldom visited (more novel). We show that NEO's modified Laplacian yields eigenvectors whose extreme values align with the most novel regions of the state space. We provide bounds for the eigenvalues of the modified Laplacian; and we show that the smoothest eigenvectors with real eigenvalues below certain thresholds provide guaranteed gradients to novel states for both undirected and directed graphs. In an empirical evaluation in online, incremental settings, NEO outperformed related state-of-the-art approaches, including eigen-options and cover options, in a large collection of undirected and directed environments with varying connectivity structures.
| Original language | English |
|---|---|
| Publication status | Published - 18 Sept 2025 |
| Event | The Thirty-Ninth Annual Conference on Neural Information Processing Systems - San Diego, USA United States Duration: 2 Dec 2025 → 7 Dec 2025 |
Conference
| Conference | The Thirty-Ninth Annual Conference on Neural Information Processing Systems |
|---|---|
| Country/Territory | USA United States |
| City | San Diego |
| Period | 2/12/25 → 7/12/25 |
Keywords
- Laplacian, Novelty, Reinforcement Learning, Exploration, Eigenvectors, Spectral Methods
Fingerprint
Dive into the research topics of 'Novel Exploration via Orthogonality'. Together they form a unique fingerprint.Cite this
- APA
- Standard
- Harvard
- Vancouver
- Author
- BIBTEX
- RIS