AIJava★ 295⑂ 42 forksMIT
beehive-lab/jitllm
JITLLM: HIGH PERFORMANCE LLM ON JVM
JITLLM — Java framework for high‑performance LLM inference and serving on the JVM. Supports GPU acceleration, GGUF and quantized model formats, and integration with DeepSeek‑R1 and Granite.
// KEY FEATURES
- Performs LLM inference on the JVM with GPU acceleration via CUDA and TensorRT.
- Supports loading and running models in GGUF format with int4, int8 quantization.
- Integrates with DeepSeek‑R1 and Granite models for fast version switching.
- Provides high‑performance HTTP serving based on Tornado with low latency and scalability.
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram