Confident at the moment of action: belief miscalibration in LLM play under hidden information
What it is
This paper investigates the miscalibration of Large Language Models (LLMs) when acting on hidden information, using a modified chess variant where royal piece status is secret. It demonstrates that high stated confidence in the location of hidden pieces does not correlate with correctness, showing that conventional evaluation metrics like legality and latency can dissociate entirely from belief quality.
The problem it solves
Agentic systems often gate actions on a model's stated confidence, but this paper shows that such confidence is frequently miscalibrated and dissociated from actual correctness, leading to unreliable autonomous decision-making.
What you could use it for
- Evaluating the reliability of LLM agents in hidden-information games
- Designing robust evaluation harnesses for agentic systems that rely on self-reported confidence
- Studying the gap between outcome success and internal belief accuracy in AI agents
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.