gold-cmp
What it is
A zero-dependency Rust library for running pairwise comparisons in LLM evaluations. It calculates win/loss/tie counts, win rates, and statistical significance (95% confidence intervals) to determine if one model is meaningfully ahead of another.
The problem it solves
Lack of a simple, zero-dependency tool to perform statistical significance testing on pairwise LLM evaluation results.
What you could use it for
- Comparing two LLM models on a gold-standard dataset
- Determining statistical significance in A/B model testing
- Generating win-rate summaries for evaluation reports
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.