Local LLM: open-source projects on GitHub
107 repositories on Local LLM, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
STRIX HALO LLM Setup
Guide for LLM on Ryzen AI MAX+ 395 and Radeon 8060S via Ollama, llama.cpp and ROCm. Achieving 100 tokens/s 30B Qwen and 140 tokens/s CHADROCK MTP.
★ 373 · PythonAI AGENT FOR YOUR WORK
Mindroom — open-source AI agent that knows your work context and chats in a familiar chat interface. Supports any LLM, can be self‑hosted or hosted. Your data stays with
★ 370 · PythonHUMLa: AI NOTES ON MAC
Open-source Mac app: mic + system audio recording, offline or via OpenAI/Deepgram/Groq transcription, speaker detection and summaries with notes.
★ 349 · RustLATE-CLI: AI COMMAND IN TERMINAL
Run an AI development command on 5 GB VRAM: ephemeral sub‑agents, precise diffs, a single binary, any model, no config.
★ 342 · GoLOCALMIND: OFFLINE LLM CHAT
Localmind — a mobile Dart/Flutter application for running local LLM models and AI servers without data leakage. Provides a fast chat, support for Gemma models, and compa
★ 326 · DartLINGER: LOCAL PERSONAL AGENT 125B
Open-source personal agent running a 125B MoE model on 12 GB GPU. Remembers context, works proactively, and keeps data locally.
★ 317 · TypeScriptOFFLINE AI ASSISTANT FOR WINDOWS
Free offline assistant for Windows reads the screen, hears your question and answers aloud. Works locally via Ollama, no API‑key required. Source code in Python with PyQt
★ 314 · PythonWONDERFUL SECOND BRAIN
A curated set of solutions for creating a self-improving second brain that helps AI agents understand personal and team context.
★ 309CARWATCH AI AGENT CAR
CarWatch turns Raspberry Pi 5, a dashcam, and local AI into a chat agent for your car. Talk to the vehicle, get data, and control functions in real time. This is a CodeWa
★ 305 · PythonFAST AI-AGENT FOR CODE
This AI agent for coding is more effective than regular subscriptions. It combines fast code search and local analysis without limits.
★ 302 · TypeScriptONE CLI FOR ALL ROBOTS!!!
quackd — CLI for bot control with LLM brain. Connects Microduck, LeRobot, ROS and others, works with Claude, OpenAI, Gemini, Grok or locally via Ollama/vLLM. Includes sim
★ 297 · PythonTURBO LLM: OFFLINE WITH AI
Run any local LLM with automatic GPU tuning, a web UI, and an OpenAI/Anthropic‑compatible API.
★ 296 · TypeScriptREAL USE CASES HERMES AGENT
awesome-hermes-usecases — curated collection of real-world use cases for Hermes Agent, the self‑improving AI agent from Nous Research. Each example is backed by primary s
★ 296 · PythonOPEN AI-AGENT FOR DESKTOP
OpenGhost — open AI‑agent for desktop on JavaScript. It has its own agent engine, browser automation tools and a drawing engine that visualizes its explanations. Supports
★ 295 · JavaScriptOFFLINE LLM ON ANDROID: FULL PRIVACY
OfflineLLM runs any GGUF models locally via llama.cpp with ARM optimisation. Zero network permissions: encrypted settings, biometric lock, and tamper detection. Supports
★ 291 · KotlinNot a Llama, no stance
Auto-tuned launcher for GGUF models on llama.cpp — server with multi‑GPU support, hardware‑specific tuning, and crash recovery. Stars: 226.
★ 284 · GoBUILD YOUR AI INTRANET
Autonomous grid unites your computers into a single infrastructure for running AI models: training, fine‑tuning and inference. Supports distributed computing, fine‑tuning
★ 278 · PythonWIKI FROM DOCUMENTS WITH LINKS
Transform documents into a Markdown wiki with precise source links. Run locally via Ollama or any LLM provider, using the CROW classifier router to boost accuracy to 89%
★ 277 · PythonAI AGENTS ON YOUR OWN SERVERS
Open-source platform to run AI agents on your own servers. Supports shared chat, RAG, skills, real browser, 4700+ MCP servers, budgets and audit. Alternative to ChatGPT E
★ 270 · PythonLLMKUBE: LLM IN KUBERNETES
LLMKube - Kubernetes operator for LLM inference on heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, Apple Silicon Metal. Supports llama.cpp, vLLm, TGI, mlx-server, multi
★ 266 · GoLIVING AI OFFICE ON DESKTOP
AI agents work right on your desktop — walk, communicate, delegate tasks and hold meetings. Support for Claude, GPT, GLM and other LLMs. Integration with Telegram, Discor
★ 265 · JavaScriptBROWSER EXTENSION FOR LOCAL TRANSLATION
Chrome and Firefox extension translating pages, PDFs, subtitles, and typed text locally via PLaMo model. Works without server, accounts, or telemetry.
★ 253 · TypeScriptSECRET AI FOR YOUR DESK
Garden‑desk is a private AI agent that runs locally on TypeScript. It enables multi‑agent workflows, works with local LLMs and shares context within the team. All data st
★ 249 · TypeScriptPORTABLE HERMES AGENT FOR WINDOWS
Portable Hermes Agent is a desktop for Windows with over 100 AI‑tools, GUI and support for local models via LM Studio. It includes TTS, music generation, ComfyUI workflow
★ 247 · PythonLocal AI roleplay app
Open Dungeon lets you generate stories and inline images entirely on your computer, using Gemma 4 QAT via Ollama (and others) + FLUX, with no accounts or API keys.
★ 241 · TypeScriptPULSAR — ULTRAFAST MOE INFERENCE
Pulsar — an SSD-streaming engine for giant MoE models on Rust+CUDA.
★ 238 · RustPURE-RUST ENGINE FOR QWEN GGUF
RustingStrata is a pure-Rust inference engine for Qwen GGUF (Qwen3–3.8, MoE up to 770B+). Supports CUDA/Vulkan, hybrid CPU+GPU experts, MTP speculative decoding. Up to 1.
★ 231 · RustOCTET: FAST AI AGENT FOR CODING
Rust coding agent with multimodal input support and deep LLM provider integrations. Extensible in any language via JSON-RPC. 207 stars on GitHub.
★ 227 · RustLOCAL AI AGENT FOR TERMINAL
Codehamr — a minimal local-first coding agent for the terminal written in Go. It indexes code via tree-sitter, suggests edits and can run commands, working fully offline
★ 224 · GoOPENAI TTS FOR HOME ASSISTANT
The openai_tts component for Home Assistant uses OpenAI Speech API (or any compatible endpoint) and outputs high‑quality speech. There is an optional chime before/after s
★ 222 · PythonANUBIS: BENCHMARK LLM ON APPLE SILICON
anubis-oss — Swift utility for local LLM launch and benchmarking on Apple Silicon. It measures speed and accuracy of GPU inference, comparing results with CPU baseline an
★ 217 · SwiftAGENT FOR CODING AND RESEARCH
Open-source agent with local model support and isolated sandbox. Handles coding, research and routines without data leaks.
★ 217 · PythonTAOS Self-Learning AI OS
Self-hosted auto-clustering AI-agent OS for your hardware: Raspberry Pi, Orange, Mac Mini, etc.
★ 213 · PythonAI Desktop Pet & Gaming Overlay
Free open-source AI desktop pet and gaming overlay for Windows with Claude, Ollama, DeepSeek, and MCP integration. Joins Discord, launches Steam games, and automates scre
★ 213 · PythonLOCAL MULTIMODAL CHAT
Self-hosted chat with local LLMs (Ollama/OpenAI), supporting PDF RAG, image chat, and voice input via Whisper.
★ 209 · PythonCLAUDECTL AGENT ORCHESTRATION
claudectl orchestrates a swarm of Claude Code agents with a local ‘brain’ that follows your preferences. Written in Rust, it already has 202★ on GitHub.
★ 204 · RustLocal-First AI Desktop Assistant
Capri is a Windows AI companion running local or API models with Hugging Face support, voice, memory, and permission-based desktop automation tools.
★ 194 · JavaScriptDeepSeek Local Setup Guide
A Russian‑language guide that walks you through installing and running DeepSeek models (V4, R1) locally on PC, Mac, or server using tools like Ollama, llama.cpp, LM Studi
★ 192 · ShellCurated Self-Hosted AI Tools
Weekly health-checked list of open-source AI for local LLMs, chat UIs, agents, RAG, and media generation. Dead projects auto-removed.
★ 124 · JavaScriptDataroom — AI Catalog
The project creates a personal dataroom from a natural‑language query, using a self‑contained Qwen3.6 on a single NVIDIA L4 GPU for accelerated data exploration.
★ 67 · PythonInteractive 3D AI Agent World
Watch AI agents live, work, and teach in a self‑hosted 3D simulation powered by local models.
★ 45 · PythonOffline LLM Emergency Assistant
Runs a local LLM with 200 GB offline knowledge library on macOS, providing cited answers without internet connectivity.
★ 43 · PythonSelf-hosted LLM gateway
A single Go binary turns your Mac and Linux machines into a private inference cluster featuring multi‑machine routing, sharding via llama.cpp‑RPC, and support for the Ope
★ 38 · GoOllama Model Test
Ollama Model Test (omt) automatically evaluates different Ollama models on given tasks, comparing them by accuracy, speed, and resource usage to identify the optimal opti
★ 31 · PythonPure Rust LFM2.5-8B-A1B
CPU-only implementation of the LiquidAI LFM2.5-8B-A1B model in pure Rust. Enables text inference without GPU and external libraries.
★ 31 · Rustawesome-openclaw - OPEN AI AGENT
Curated list of resources, tools, and guides for OpenClaw — a self-hosted AI agent for WhatsApp, Telegram, Discord, and 50+ integrations.
★ 6SparkDuet: on-demand model library
The project provides a single OpenAI‑compatible endpoint for running DeepSeek and Qwen models. It serves them via two vLLM lanes (TP=2) on an NVIDIA DGX Spark cluster and
★ 0 · PythonNew projects on this topic in Telegram
Telegram →