Your Transformer Is Secretly Elastic
What it is
This paper introduces a retraining-free method to make pre-trained transformers elastic, allowing them to adapt to varying computational budgets on-the-fly. It achieves this by jointly ranking every individual MLP neuron and attention head using a differentiable relaxation of functional unit-ranking, minimizing loss under soft pruning across multiple sparsity targets simultaneously.
The problem it solves
Current model families offer only a few fixed sizes, forcing users to commit to a specific model based on hardware and latency constraints, which is often suboptimal under varying query loads.
What you could use it for
- Deploying a single model that adapts to fluctuating workloads and heterogeneous hardware
- Reducing inference latency and increasing throughput for vision and text transformers
- Achieving smooth performance degradation up to 60% sparsity without retraining
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.