LoRA on a Budget: Function-Aware Post-Hoc Compression for Adapter Fleets
What it is
This paper proposes Functional Rank Allocation (FRA), a method for compressing fleets of LoRA adapters under a shared rank budget. It works by measuring the functional impact of rank truncation on individual modules to construct a distortion frontier, then using dynamic programming and minimax allocation to distribute rank across layers and adapters to maximize worst-case utility.
The problem it solves
Serving many LoRA adapters under a shared resource budget is difficult because uniform spectral compression does not account for the varying functional tolerance of different adapters and layers, leading to suboptimal performance distribution.
What you could use it for
- Serving multiple LoRA adapters on a single GPU with limited memory
- Compressing a fleet of task-specific adapters to fit within a fixed rank budget
- Optimizing the trade-off between compression ratio and functional performance for LoRA models
Why it matters
Enables efficient deployment of large numbers of specialized LoRA adapters without significant loss in worst-case performance, reducing the memory overhead of serving multi-tenant or multi-task LLMs.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.