md-eval-cot
What it is
A framework and dataset for evaluating the quality of Chain-of-Thought (CoT) reasoning traces in Large Language Models. It provides metrics to assess structural coherence, logical consistency, linguistic clarity, and information density beyond simple output accuracy.
The problem it solves
Standard benchmarks often ignore the quality of the reasoning path, making it difficult to assess if a model is actually 'thinking' correctly or just arriving at the right answer.
What you could use it for
- Evaluating the reasoning quality of models like DeepSeek-R1 or Qwen.
- Benchmarking the clarity and logical flow of auto-generated reasoning steps.
- Quantifying the information density and redundancy of CoT traces.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.