AIPython★ 320⑂ 26 forksApache-2.0
Picovoice/picollm
PICOLLM - OFFLINE LLM ON DEVICE
picollm enables running LLM on-device through extreme k‑bit quantization, delivering fast and private inference without the cloud.
// KEY FEATURES
- X‑bit quantization reduces model size to 2‑bits
- Run LLM on-device without internet or GPU
- Compatible with HuggingFace and Transformers Python packages
#compression#efficient-inference#gemma#generative-ai#language-model#language-models#large-language-model#llama#llama2#llama3#llm#llm-inference
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram