World Reward Models for ML Agents
What it is
This paper proposes World Reward Models (WRMs) to improve search-based machine learning agents by directly predicting which candidate program will perform better, rather than predicting the full execution outcome. It introduces ForeCritic, a 9B model trained on executed program pairs, which significantly reduces task time and improves selection accuracy compared to general-purpose LLM judges.
The problem it solves
Execution-based feedback in ML agents is slow (hours per run) and most runs are discarded; existing LLM world models that predict execution outcomes are unreliable and can mislead the search process.
What you could use it for
- Accelerating the search loop in automated ML agents like AIDE and ML-Master
- Selecting the best candidate code snippet from a set of generated programs without executing all of them
- Replacing unreliable LLM-based world models with calibrated reward predictors for code evaluation
Why it matters
Reduces the computational cost of automated ML agents by 4.4 to 7.5 times while maintaining or improving solution quality, making large-scale program search more feasible.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.