We present a new method for automatically creating useful temporal abstractions in reinforcement learning. We argue that states that allow the agent to transition to a different region of the state space are useful subgoals, and propose a method for identifying them using the concept of relative novelty. When such a state is identified, a temporally-extended activity (e.g., an option) is generated that takes the agent efficiently to this state. We illustrate the utility of the method in a number of tasks.
|Title of host publication||Proceedings of the Twenty-first International Conference on Machine Learning (ICML 2004): Banff, Alberta, Canada, July 4-8, 2004|
|Editors||Carla E. Brodley|
|Publisher||Association for Computing Machinery|
|Publication status||Published - 2004|
|Name||ACM International Conference Proceeding Series|