DepGPO: Dependency-Aware Policy Optimization for Terminal Agents
What it is
This paper proposes Dependency-Aware Group Policy Optimization (DepGPO), a method for training terminal-using agents via reinforcement learning. It constructs a command dependency graph from execution traces to accurately assign credit to relevant read-write operations, preventing training signals from being diluted by irrelevant steps.
The problem it solves
Existing trajectory-level and step-level credit assignment methods do not trace read-write dependencies, leading to training signals being assigned to irrelevant operations and weakening learning efficiency.
What you could use it for
- Training LLM agents for complex multi-step terminal tasks
- Improving credit assignment in reinforcement learning for coding and debugging agents
- Enhancing training stability for agents interacting with command-line interfaces
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.