AIPython★ 8,246
bitsandbytes-foundation/bitsandbytes
BITSANDBYTES: LLM COMPRESSION
Library for k-bit quantization of large language models, enabling LLM execution on regular hardware via PyTorch.
// KEY FEATURES
- 4-bit quantization without quality loss
- Support for PyTorch and HuggingFace Transformers
- Automatic precision mixing for faster inference
#llm#machine-learning#pytorch#qlora#quantization
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram