AgentHER: Hindsight Experience Replay for LLM Agents
What it is
AgentHER is a Python library and CLI tool that converts failed LLM agent trajectories into high-quality training data by relabeling the original goal to match the actual outcome. It uses a four-stage pipeline involving failure detection, outcome extraction, prompt relabeling, and data augmentation to generate SFT, DPO, or ShareGPT formats.
The problem it solves
Failed agent trajectories are routinely discarded during training, wasting valuable data where the agent's actions were correct but the final result did not meet an arbitrary or overly specific goal constraint.
What you could use it for
- Generating Supervised Fine-Tuning (SFT) datasets from agent logs where the agent performed well but failed to meet a specific constraint.
- Creating Direct Preference Optimization (DPO) pairs by converting failed trajectories into 'chosen' examples with corrected goals.
- Augmenting training data for tool-use agents to improve robustness against arbitrary constraint violations.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.