Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability
What it is
This paper investigates a fundamental flaw in mechanistic interpretability where intervention-defined faithfulness metrics prefer circuits that reproduce model behavior less well than larger or different candidates. It demonstrates that this objective-level gap persists across multiple automated discovery algorithms and is caused by context distortion, where removing signals alters the inputs retained components operate on.
The problem it solves
Current faithfulness metrics in mechanistic interpretability can misrank candidate circuits, preferring those that are less faithful to the model's actual behavior, which undermines the reliability of discovered mechanisms.
What you could use it for
- Evaluating the validity of automated circuit discovery algorithms
- Designing better evaluation metrics for mechanistic interpretability
- Understanding the limitations of faithfulness-based circuit extraction
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.