BugDetectionBench
What it is
A benchmark dataset and collection of TypeScript scripts for scraping GitHub pull request comments to identify and classify real-world bug reports. It uses LLMs to validate bug reports and assess their difficulty, producing a structured JSON dataset for evaluating automated code review agents.
The problem it solves
Lack of a standardized, real-world benchmark for evaluating automated code review agents on actual bug detection tasks.
What you could use it for
- Evaluating the performance of automated code review bots
- Training models to detect bugs in code reviews
- Analyzing the distribution of bug difficulty in open-source projects
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.