R_REDDYX.XYZ
AIJava★ 295⑂ 42 forksMIT

beehive-lab/jitllm

JITLLM: HIGH PERFORMANCE LLM ON JVM

JITLLM — Java framework for high‑performance LLM inference and serving on the JVM. Supports GPU acceleration, GGUF and quantized model formats, and integration with DeepSeek‑R1 and Granite.

// KEY FEATURES

  • Performs LLM inference on the JVM with GPU acceleration via CUDA and TensorRT.
  • Supports loading and running models in GGUF format with int4, int8 quantization.
  • Integrates with DeepSeek‑R1 and Granite models for fast version switching.
  • Provides high‑performance HTTP serving based on Tornado with low latency and scalability.
#accelerators#compilers#deepseek-r1#gguf#gpu#granite#ibm-granite#java#java21#llama3#llm#mistral
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

Multi-agent Trading LangGraph★ 295PACKMIND - AI FOR ENGINEERING PLAYBOOKS★ 295GOLEMBOT — AGENT EVERYWHERE★ 295FABRIKT: Kotlin from OpenAPI★ 295AWESOME RAG PRODUCTION★ 295KEELITH GO FRAMEWORK FOR CLOUDS★ 295

← FULL CATALOG

beehive-lab/jitllm — Jitllm: High Performance LLM ON Jvm