SELF: Agentic Self-Distillation with Environmental Feedback Modeling
What it is
This paper introduces SELF, a framework for training language agents that jointly optimizes environmental feedback modeling and hindsight self-distillation. It argues that simply conditioning a teacher model on environmental feedback is insufficient and proposes a method where the agent learns to predict environmental responses while distilling guidance from a feedback-conditioned self-teacher.
The problem it solves
Standard reinforcement learning cannot be directly applied when rewards are unavailable, and existing methods that use environmental feedback for hindsight self-distillation fail to fully exploit the rich supervision contained in that feedback.
What you could use it for
- Training language agents in interactive environments where explicit rewards are unavailable
- Improving agent performance in benchmarks like τ-bench and AppWorld
- Enhancing the ability of language models to model how environments respond to actions
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.