Telescopic Language Models
What it is
This paper introduces Telescopic Language Models (TLMs), a method for training a single Transformer that functions as a valid language model at every depth. It uses stochastic prefix supervision, where each training step involves a randomly truncated prefix and a full-capacity pass, ensuring the model is usable across a continuum of compute budgets rather than just fixed points.
The problem it solves
Current methods for serving multiple compute budgets require separate training or compression runs for each specific model size, and fixed-exit nested models perform poorly (chance level) at depths that were not explicitly supervised.
What you could use it for
- Serving a single model across multiple latency and cost tiers without retraining
- Reducing the total compute cost of serving variable-size models in production
- Creating elastic language models that can be dynamically resized at inference time
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.