GM-Bench
What it is
GM-Bench is a deterministic sports general-manager benchmark designed to evaluate long-horizon planning, tool use, and decision-making in LLMs. It simulates a complex sports league involving roster management, trades, and drafts, providing a structured environment to test agent performance over multiple seasons.
The problem it solves
Lack of standardized, complex, and deterministic benchmarks for evaluating long-horizon planning and multi-step decision-making in LLMs.
What you could use it for
- Benchmarking LLM planning capabilities in complex, multi-turn environments.
- Evaluating agentic decision-making and tool-use consistency.
- Testing long-horizon strategy and resource management (e.g., salary caps, player development).
- Comparing model performance against human-like heuristics (e.g., 'pick-trader' strategy).
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.