AIPython★ 8,224⑂ 606 forksMIT
zilliztech/GPTCache
GPTCache: SEMANTIC CACHE FOR LLM
GPTCache provides semantic caching of requests to large language models, reducing token consumption and speeding up responses. Fully integrated with LangChain and llama_index, it supports Python API.
// KEY FEATURES
- Caches LLM requests by meaning, saving up to 80% tokens on repeated similar queries.
- Fully integrated with LangChain and llama_index via simple imports and configuration.
- Supports custom embeddings and various storages: SQLite, FAISS, Redis.
- Reduces time and cost of calls to OpenAI, Anthropic, and local models by using the cache.
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram