R_REDDYX.XYZ

Vllm: open-source projects on GitHub

69 repositories on Vllm, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

LLM 2094Inference 95Sglang 24LLM Inference 76Qwen 106Ollama 249Openai 690Mlx 47Deepseek 229Llama Cpp 56Cuda 48LLM Serving 24Anthropic 452Local LLM 107
jaylfc/taOS

TAOS Self-Learning AI OS

Self-hosted auto-clustering AI-agent OS for your hardware: Raspberry Pi, Orange, Mac Mini, etc.

★ 213 · Python
justxor/deepseek-local-guide-ru

DeepSeek Local Setup Guide

A Russian‑language guide that walks you through installing and running DeepSeek models (V4, R1) locally on PC, Mac, or server using tools like Ollama, llama.cpp, LM Studi

★ 192 · Shell
huawei-csl/KVarN

KVarN: KV-Cache Quantization

KVarN is a native KV-cache quantization backend for vLLM, increasing context by 3-5x, raising throughput above FP16 and maintaining FP16 accuracy without calibration, act

★ 159 · Python
Aevonix/FlashAlign

On-Demand Model Alignment

Aligns LLMs instantly per request for verified users or always-on mode. Integrates with Hermes, OpenClaw, and custom harnesses.

★ 126 · Python
0xSero/DeepSeek-V4.1-Flash-Two-Sparks

DeepSeek V4.1 on DGX Sparks

Runs DeepSeek-V4.1-Flash across two NVIDIA DGX Spark systems, featuring EXL3 routed experts, 262k token context, 2M‑token KV cache, vision, tool use, MTP, and Pi bundling

★ 95 · Shell
juliankang4/spark-scope

DGX Spark Monitoring Dashboard

Provides a read-only web dashboard and rack panel to monitor DGX Spark, ASUS GX10, and MSI EdgeXpert nodes running vLLM or SGLang.

★ 70 · JavaScript
hadihonarvar/flock

Self-hosted LLM gateway

A single Go binary turns your Mac and Linux machines into a private inference cluster featuring multi‑machine routing, sharding via llama.cpp‑RPC, and support for the Ope

★ 38 · Go
Jwuthri/SelfJev

Open Decision Model API

Provides an open decision model exposing Jev's API for typed answers (yes/no, choice, score, multi) with probabilities from forward passes, using Qwen3.5-4B fine‑tuned wi

★ 25 · Python
zorost/sparkduet

SparkDuet: on-demand model library

The project provides a single OpenAI‑compatible endpoint for running DeepSeek and Qwen models. It serves them via two vLLM lanes (TP=2) on an NVIDIA DGX Spark cluster and

★ 0 · Python

New projects on this topic in Telegram

Telegram →
Vllm — page 2 | REDDYX AI