Preserving Utility in Continual LLM Unlearning via Orthogonal Projections
What it is
This paper introduces Continual Orthogonal Projection (COP), a method to prevent performance decay in large language models during sequential unlearning. It works by projecting gradient updates onto the orthogonal complement of previously cached activation subspaces, ensuring that new deletions do not interfere with or undo previous ones.
The problem it solves
Standard LLM unlearning methods suffer from steep decay in both forgetting and utility when applied sequentially, as successive updates interfere with and undo previous deletions.
What you could use it for
- Compliance-driven removal of copyrighted or private data from deployed LLMs
- Iterative unlearning of sensitive information without retraining the entire model
- Maintaining model utility while processing a continuous stream of unlearning requests
Why it matters
Enables stable, long-term maintenance of LLMs in regulated environments by allowing continuous data removal without the cumulative loss of model capabilities or forgetting efficacy.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.