ReFormBench
What it is
ReFormBench is a benchmark designed to evaluate the stability of format correction in large language models. It uses a model's unconstrained response as a baseline and measures how well the model can correct formatting errors when given specific require/forbid instructions across different variants.
The problem it solves
Standard compliance scores fail to distinguish between a model preserving its observed format and actually correcting it, nor do they reveal if corrections hold across instruction variants.
What you could use it for
- Evaluating LLM robustness to formatting constraints
- Benchmarking instruction-following capabilities for structured output
- Comparing model performance on format preservation vs. correction
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.