Auditable Long-Term Memory: A Deterministic Retrieval Chain
What it is
This paper evaluates a long-term memory system that uses a deterministic retrieval chain (hybrid retrieval, reranking, and packet compilation) with an LLM only as a final reader. It reports high scores on the LongMemEval-S benchmark but explicitly notes that all components were developed on the same 500 questions without held-out evaluation, and that scoring prompt modifications and reader context differences prevent definitive claims of superiority over existing systems.
The problem it solves
Lack of transparency and auditability in long-term memory evaluation, where LLM-based retrieval and scoring can obscure whether performance gains come from better retrieval or better prompting.
What you could use it for
- Evaluating the robustness of deterministic retrieval chains in long-term memory systems
- Benchmarking LLM readers against fixed retrieval packets
- Auditing the impact of scoring prompt modifications on evaluation results
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.