AIPython★ 360⑂ 14 forks
xiaol/wkvm
Hybrid LLM Inference
Library for performing inference of hybrid language models, including Gemma, RWKV, and their combinations, supporting efficient execution on CPU and GPU.
// KEY FEATURES
- Inference of Gemma and RWKV models through a unified API interface.
- Support for hybrid architectures, e.g., Gemma‑RWKV mixtures and other combinations.
- Optimized KV‑states cache accelerates generation of long sequences.
- Compatible with HuggingFace Transformers and PyTorch for easy integration.
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram