AIC++★ 76⑂ 10 forksApache-2.0
cacheMon/PrefixSim
LLM Prefix Cache Simulator
Simulates prefix caching mechanisms for LLM inference to analyze cache hit rates and memory efficiency.
// KEY FEATURES
- Models KV cache sharing across requests with common prefixes
- Measures cache hit rates under varying workload patterns
- Outputs latency and memory usage metrics for optimization
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram