Beyond F1: Evaluating Coverage and Failure Recovery in AI Model Security Scanners
What it is
This paper evaluates three AI model security scanners (ModelScan, ModelAudit, and Fickling) using a synthetic benchmark of 170 Pickle and PyTorch artifacts. It introduces metrics that distinguish between a scanner's ability to make a definitive judgment (coverage) and the accuracy of that judgment (precision/recall), revealing that while ModelScan has high accuracy when it works, it fails to complete analysis for nearly half of the samples.
The problem it solves
Conventional evaluation metrics like F1 only measure accuracy for cases where a scanner succeeds, ignoring the significant proportion of artifacts where scanners fail to produce any judgment, thus providing an incomplete picture of tool reliability.
What you could use it for
- Selecting appropriate security scanners for ML artifact pipelines based on coverage and failure modes
- Benchmarking new security tools against established baselines for ML artifact safety
- Designing evaluation frameworks that account for tool non-completion and partial failures
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.