AIRust★ 723⑂ 223 forksApache-2.0
smg-project/smg
SMG - Universal LLM Gateway
Engine-agnostic LLM gateway in Rust with full OpenAI and Anthropic API compatibility, gRPC pipeline, and KV-cache caching.
// KEY FEATURES
- gRPC pipeline for low-latency requests
- Routing with KV-cache awareness for efficiency
- Tokenization and chat history caching, with WASM plugin support
#anthropic#anthropic-api#chat#claude#gemini#inference-gateway#lightseek#llm#mcp#openai#responses-api#routing
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram