Evaluate - LLM Testing Framework
What it is
A Rust-based evaluation framework for testing LLMs using automated 'LLM-as-a-judge' logic. It supports multiple providers (OpenAI, Anthropic, Gemini, Ollama) and includes a GUI, API, and SQLite history tracking.
The problem it solves
Lack of a lightweight, multi-provider framework for automated LLM evaluation with built-in judging and history tracking.
What you could use it for
- Automated regression testing of LLM outputs
- Comparing performance across different LLM providers
- Building custom evaluation pipelines with Python SDK
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.