hey-llm-you-okay
What it is
A CLI tool for testing LLM applications that uses a cost-ordered pyramid of tests to verify behavior and quality. It includes an A/B triage mechanism that distinguishes between prompt regressions and model drift by comparing current inputs against last-passing snapshots.
The problem it solves
LLM test suites often report green results without verifying actual behavior, and when tests fail, it is difficult to determine if the cause is a developer's code change or a silent update/drift in the underlying model provider.
What you could use it for
- CI/CD pipelines for LLM applications to gate deployments on deterministic and quality checks
- Debugging failing LLM tests to determine if the cause is a code change or provider model drift
- Verifying that the prompt used in tests matches the prompt sent in production
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.