llm-token-saver-rs
What it is
A Rust library for optimizing LLM token usage through prompt compression, dynamic truncation, and selective context filtering. It provides a UnifiedContextManager to apply compression tiers and budget policies to conversation contexts, aiming to reduce inference costs.
The problem it solves
High token consumption and inference costs in LLM applications, particularly when handling long contexts or tool calls.
What you could use it for
- Reducing token costs for AI coding agents
- Optimizing context window usage in LLM applications
- Filtering long text blobs like file contents or search results before sending to an LLM
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.