GPTCache
What it is
GPTCache is a Python library that implements a semantic cache for Large Language Model (LLM) queries. It intercepts LLM API calls, stores responses in a vector database, and returns cached results for semantically similar queries to reduce API costs and latency.
The problem it solves
High costs and slow response times associated with repeated or semantically similar LLM API calls.
What you could use it for
- Reducing OpenAI API costs for applications with repetitive user queries
- Improving response latency for LLM-powered chatbots
- Caching embeddings and responses for LangChain or LlamaIndex applications
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.