R_REDDYX.XYZ
AIC++★ 76⑂ 10 forksApache-2.0

cacheMon/PrefixSim

LLM Prefix Cache Simulator

Simulates prefix caching mechanisms for LLM inference to analyze cache hit rates and memory efficiency.

// KEY FEATURES

  • Models KV cache sharing across requests with common prefixes
  • Measures cache hit rates under varying workload patterns
  • Outputs latency and memory usage metrics for optimization
#cache#llm#llm-inference#llm-serving#prefix-cache
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

AI Meeting Knowledge Workspace★ 76AI Prompt Optimization Framework★ 76AI Agent for CLI Code★ 75RLBot AI Bot Framework★ 75AI Image and Video Generator★ 75OF Content Downloader★ 75

← FULL CATALOG

cacheMon/PrefixSim — LLM Prefix Cache Simulator | REDDYX AI