SGUID: Selecting a Compact Skill Bank for Model-Skill Co-Evolution
What it is
This paper proposes SGUID, a method for selecting a compact subset of high-utility skills for LLM distillation. It addresses the issue that most retrieved skills provide weak or negative learning signals by filtering for skills that consistently improve model performance during training.
The problem it solves
Most retrieved skills in large banks provide ineffective or harmful distillation signals, leading to inefficient training and potential performance degradation when using full-bank distillation.
What you could use it for
- Efficiently distilling procedural knowledge into smaller LLMs without degrading performance
- Implementing stable iterative model-skill co-evolution loops
- Reducing the size of skill banks used for inference-time patching or training
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.