Dynamic Expert Pruning for Multi-Agent Systems
What it is
This paper proposes Dynamic Expert Pruning (DEP) to optimize Mixture-of-Experts (MoE) models in multi-agent environments. It uses a lightweight predictor to generate per-request expert masks based on agent prompts, allowing the model to activate only the specific experts needed for each task rather than using a static, one-size-fits-all mask.
The problem it solves
Static expert pruning methods fail in multi-agent systems because they apply a single fixed expert mask to all requests, whereas different agents and tasks require different subsets of experts, leading to suboptimal performance and inefficient memory usage.
What you could use it for
- Deploying large MoE models in multi-agent systems with limited GPU memory
- Serving heterogeneous workloads where different agents require different model capabilities
- Reducing inference latency by activating fewer experts per request
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.