R_REDDYX.XYZ
AIPython★ 4,092

ModelTC/LightLLM

LightLLM - Fast LLM Inference

A lightweight and fast framework for inference and serving of large language models in Python. Simple scaling and high performance.

// KEY FEATURES

  • Lightweight and fast LLM inference
  • Simple scaling
  • High performance
#deep-learning#gpt#llama#llm#model-serving#nlp#openai-triton
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

ONE DNN: ACCELERATING AI ON CPU★ 4092Semantic video search★ 4093SPRITE-GEN — AI SPRITE GENERATION★ 4089CISO Assistant - Global GRC Platform★ 4088MONITORING AI GATEWAYS WITH AI★ 4085CSGHUB — PLATFORM FOR LLM★ 4082

← FULL CATALOG

ModelTC/LightLLM — LightLLM - Fast LLM Inference | REDDYX AI