Hitting-Time Bounds for Asynchronous Q-Learning
What it is
This paper analyzes the sample complexity of the last iterate of standard asynchronous Q-learning for finite Markov decision processes. It proves that zero-initialized Q-learning with a constant step size achieves specific error bounds using a sample complexity that separates statistical error from initialization cost, and introduces an empirical rule to select the step size without prior model knowledge.
The problem it solves
Lack of tight, high-probability sample complexity bounds for the last iterate of asynchronous Q-learning, particularly regarding the separation of statistical error and initialization error in finite MDPs.
What you could use it for
- Theoretical analysis of reinforcement learning algorithms
- Designing sample-efficient training protocols for asynchronous Q-learning
- Understanding the impact of behavior policy periodicity on Q-learning convergence
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.