R_REDDYX.XYZ
AIPython★ 8,224⑂ 606 forksMIT

zilliztech/GPTCache

GPTCache: SEMANTIC CACHE FOR LLM

GPTCache provides semantic caching of requests to large language models, reducing token consumption and speeding up responses. Fully integrated with LangChain and llama_index, it supports Python API.

// KEY FEATURES

  • Caches LLM requests by meaning, saving up to 80% tokens on repeated similar queries.
  • Fully integrated with LangChain and llama_index via simple imports and configuration.
  • Supports custom embeddings and various storages: SQLite, FAISS, Redis.
  • Reduces time and cost of calls to OpenAI, Anthropic, and local models by using the cache.
#aigc#autogpt#babyagi#chatbot#chatgpt#chatgpt-api#dolly#gpt#langchain#llama#llama-index#llm
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

Claude Code Tips★ 8210GITHUB COPILOT IN XCODE NATIVELY★ 8204BITSANDBYTES: LLM COMPRESSION★ 8246iOS MobileGestalt Configuration Tool★ 8201MobileGestalt Editor for iOS★ 8248VIBE CODE WITH PI IN IDE★ 8259

← FULL CATALOG

zilliztech/GPTCache — GPTCache: SEMANTIC CACHE FOR LLM