acceptance-bench
What it is
A Python-based LLM evaluation framework that measures model acceptance and refusal rates across creative writing tasks. It uses multi-prompt variations, temperature sweeping, and LLM-as-judge scoring to detect soft refusals and quantify output quality.
The problem it solves
Standard LLM benchmarks often only measure binary refusal rates, failing to capture 'soft refusals' (hedging, disclaimers) or the impact of prompt variation and temperature on output quality.
What you could use it for
- Benchmarking LLMs for creative writing capabilities
- Comparing refusal policies across different AI models
- Evaluating model sensitivity to prompt phrasing and temperature settings
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.