Using relative novelty to identify useful temporal abstractions in reinforcement learning

Özgür Şimşek, Andrew G. Barto

Research output: Chapter in Book/Report/Conference proceedingChapter

98 Citations (Scopus)

Abstract

We present a new method for automatically creating useful temporal abstractions in reinforcement learning. We argue that states that allow the agent to transition to a different region of the state space are useful subgoals, and propose a method for identifying them using the concept of relative novelty. When such a state is identified, a temporally-extended activity (e.g., an option) is generated that takes the agent efficiently to this state. We illustrate the utility of the method in a number of tasks.
Original languageEnglish
Title of host publicationProceedings of the Twenty-first International Conference on Machine Learning (ICML 2004): Banff, Alberta, Canada, July 4-8, 2004
EditorsCarla E. Brodley
PublisherAssociation for Computing Machinery
Volume69
Publication statusPublished - 2004

Publication series

NameACM International Conference Proceeding Series
PublisherACM

Fingerprint Dive into the research topics of 'Using relative novelty to identify useful temporal abstractions in reinforcement learning'. Together they form a unique fingerprint.

  • Cite this

    Şimşek, Ö., & Barto, A. G. (2004). Using relative novelty to identify useful temporal abstractions in reinforcement learning. In C. E. Brodley (Ed.), Proceedings of the Twenty-first International Conference on Machine Learning (ICML 2004): Banff, Alberta, Canada, July 4-8, 2004 (Vol. 69). (ACM International Conference Proceeding Series). Association for Computing Machinery. http://www.machinelearning.org/proceedings/icml2004/papers/102.ps