PHRBench: A Behavioral Evaluation of Post-Hallucination Reasoning in LLMs
What it is
PHRBench is a controlled benchmark designed to evaluate how Large Language Models (LLMs) handle hallucinated information in multi-stage reasoning tasks. It analyzes 4,820 instances across four domains and 18 models, characterizing reasoning trajectories through metrics like Hallucination Compliance and Heuristic Correction rather than just final answer correctness.
The problem it solves
Existing studies on post-hallucination reasoning focus on aggregate outcomes or final answer correctness, leaving a gap in understanding how models specifically resolve hallucinated premises at the individual response level.
What you could use it for
- Evaluating the robustness of LLMs against erroneous context in multi-step reasoning chains
- Identifying specific reasoning behaviors (compliance vs. avoidance) when models encounter hallucinated premises
- Developing lightweight predictors to estimate the likelihood of successful recovery from hallucinations
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.