LLM Judge for Computer-Use Agents
What it is
An evaluation system that uses LLMs to assess the performance of computer-use agents on web browsing and interaction tasks. It processes visual evidence, agent trajectories, and final results to generate structured scores, error categories, and improvement tips.
The problem it solves
Manual evaluation of agent trajectories and visual results is time-consuming, subjective, and difficult to scale.
What you could use it for
- Evaluating the success of web-browsing agents on specific tasks
- Identifying failure modes like captchas or infinite loops in agent trajectories
- Batch scoring agent episodes for comparative analysis
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.