After Cooperation Is Learned: Gradient Routing and Optimizer-Dependent Maintenance in Multi-Agent Reinforcement Learning
What it is
This paper investigates whether continued optimization can destabilize learned cooperative policies in multi-agent reinforcement learning (MARL). It formulates cooperation maintenance as a right-censored event-time problem and uses matched warm starts to isolate the effect of value gradients entering shared actor representations.
The problem it solves
Standard MARL evaluations often only check for cooperation discovery from random initialization, leaving it unclear if learned cooperation survives continued training, and actor-critic comparisons often conflate the presence of a critic with the specific routing of value gradients.
What you could use it for
- Designing stable training pipelines for cooperative multi-agent systems
- Analyzing the impact of critic architectures on long-term policy stability
- Developing evaluation metrics for the persistence of learned cooperation
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.