When Do Intrinsic Rewards Lead to Exploration?
What it is
This paper proposes a formal criterion for evaluating exploration in reinforcement learning by comparing policies based on the counterfactual information they acquire. It demonstrates that common intrinsic rewards like count-based and prediction-error objectives can lead to Pareto-suboptimal exploration in specific environments and establishes conditions under which they succeed.
The problem it solves
Maximizing standard intrinsic rewards (e.g., prediction error, empowerment) does not guarantee the acquisition of the most informative experience, leading to suboptimal exploration policies.
What you could use it for
- Designing more robust intrinsic reward functions for reinforcement learning agents
- Analyzing why standard exploration heuristics fail in complex or deceptive environments
- Developing new objectives that strictly improve counterfactual information acquisition
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.