HABIT-Bench: Evaluating Habit Induction from Longitudinal Weak Evidence in Agent Memory
What it is
HABIT-Bench is a benchmark for evaluating how well LLM agent memory systems infer user habits from longitudinal, weak evidence. It uses synthetic user histories spanning hundreds of sessions across food, finance, software, and travel domains to test if agents can correctly apply inferred habits to new requests, including handling exceptions and changes.
The problem it solves
Existing agent memory systems fail to effectively infer and apply implicit user habits from partial or weak evidence in long-term interactions, leading to poor personalization.
What you could use it for
- Evaluating the performance of different agent memory architectures
- Benchmarking LLMs on their ability to infer implicit user preferences from dialogue history
- Testing the robustness of habit inference against temporary exceptions and unaccepted suggestions
Why it matters
It exposes a significant gap between having access to past interactions and actually being able to infer and apply user habits, providing a standardized metric to drive improvements in long-term agent personalization.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.