Vasari
What it is
Vasari is a local, browser-based web application for evaluating AI features like agents and RAG systems. It allows users to ingest traces, manually label them, perform grounded theory error analysis, and validate LLM judges against human labels using statistical metrics.
The problem it solves
The lack of a simple, local, and privacy-preserving tool to systematically review AI traces, categorize failures, and statistically validate automated judges against human judgment.
What you could use it for
- Reviewing and labeling LLM conversation traces for quality assurance
- Performing open and axial coding to identify failure modes in AI agents
- Validating LLM-as-a-judge prompts against human ground truth data
- Exporting validated evaluation judges for production monitoring
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.