AAABench
What it is
AAABench is a long-horizon benchmark harness that evaluates coding agents by requiring them to build a full open-world game in Unreal Engine 5. It provides a project skeleton, MCP-based control of the engine, and a strict set of rules forbidding human intervention or hints to measure the agent's ability to plan, execute, and self-verify complex systems.
The problem it solves
Current coding benchmarks often rely on static or short-horizon tasks that can be gamed by pattern-matching, failing to test an agent's true ability to manage long-term state, self-diagnose errors, and execute complex, multi-system designs.
What you could use it for
- Benchmarking LLMs for long-horizon autonomous coding and design tasks.
- Evaluating an agent's ability to manage state and continuity across multiple sessions.
- Testing causal reasoning in spatial design and urban planning logic.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.