Local LLM: open-source projects on GitHub
107 repositories on Local LLM, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
AI Productivity On-Device
A universal AI productivity tool that runs locally and guarantees complete privacy without complex setup.
★ 59,922 · JavaScriptALL IN ONE: COURSES AND VIDEO
omniget — free open‑source desktop app for downloading Udemy/Hotmart courses and YouTube videos via yt‑dlp GUI (1800+ sites). Also an interface for AI‑agents; works on Wi
★ 14,442 · RustSoup: Train LLM Easily
Soup fine-tunes an LLM from a single YAML using a streaming layer — enabling an 8B model to run on a laptop with 4 GB VRAM.
★ 10,552 · PythonLocal Deep Search
Achieves ~95% accuracy on SimpleQA, works with any local and cloud LLMs and more than 10 search engines, all encrypted and local.
★ 7,516 · PythonDossier Resume Builder
Self-hosted web app that generates customized resumes and cover letters from your professional profile.
★ 5,145 · TypeScriptLANGRID - Multi-agent AI for Code
A simple framework for building LLM-based agents with memory and tool support.
★ 4,029 · PythonATOMIC AGENT: LOCAL AI
The local AI agent atomic-agent runs entirely on your device, supports long context and precise tool invocation.
★ 3,682 · TypeScriptSURF - PERSONAL AI NOTEBOOK
Surf is an open personal AI notebook: it combines files and web pages, generates notes, and works fully locally.
★ 3,425 · TypeScriptPROJECT MAYA: LOCAL LLM ENGINE
Project Maya lets you run GLM-5.3-Flash (321B MoE) locally on a single GPU or up to 16 GPUs, supporting NVIDIA and AMD on Linux and Windows. The package includes engine,
★ 2,898 · C++HyperQwen: Qwen on Consumer GPUs
The project optimizes running large Qwen models on available GPUs using vLLM patches and requantization. It achieves 127 tokens/s on 24 GB.
★ 2,713 · PythonMTPLX SUPERFAST LAUNCH
MTPLX lets you run Qwen 3.8 Flash Next and Qwen 3.8 27B on Mac at 125 tokens/sec in OpenCode. Thanks to MTP speculative decoding on Apple Silicon, results stay accurate a
★ 2,706 · PythonLocal Claude Code
Run Claude Code entirely on-device with Apple Silicon using an MLX-native server compatible with the Anthropic API. Supports local Qwen 3.5 122B, Llama 3.3 70B, and Gemma
★ 2,578 · PythonNATIVE LLM SERVER FOR MAC
LLM server compatible with OpenAI+Anthropic, no Python required. Includes MLX Core macOS: chat, agent, and tool calling.
★ 2,472 · ZigFast Local AI Engine
Rapid-MLX is the fastest local AI engine for Apple Silicon, running 4.2× faster than Ollama and providing a cached first-token time of just 0.08s.
★ 2,185 · PythonRUN NEURAL NETWORKS LOCALLY
A collection of the best platforms and tools for running LLM on your own hardware. 2000+ stars!
★ 2,045LOCAL OPEN-SOURCE AI AGENT
Atomic-Chat is a private AI engine for running LLM locally without internet, fully offline on your computer. ⭐ 967 stars on GitHub.
★ 2,008 · TypeScriptLLM ENGINE TURNING DOCUMENTS INTO STRUCTURED WIKI
Open-source engine that turns raw documents (PDFs, video, spreadsheets) into a structured local wiki on Markdown. Automatically builds cross-references, detects contradic
★ 2,004 · PythonMICRO-AGENTS WATCH YOUR SCREEN
Observer — a TypeScript framework of local micro-agents. They watch your screen and automate routine without the cloud. 1.6k stars on GitHub.
★ 1,803 · TypeScriptTRANSLATE BOOKS WITH LLM
Translates books and documents using Ollama, OpenAI, Gemini, and other models. Preserves formatting and resumes translation from the last point.
★ 1,770 · PythonParlor AI on device
Parlor is an open project that enables natural voice and visual dialogues with AI running fully locally on your device. It is built on the Gemma 4 E2B and Kokoro models.
★ 1,752 · HTMLOuroboros – Self‑Created AI
A self‑born agent that writes itself in Python. With 726 branches it is already grateful to the community.
★ 1,667 · PythonBUILD YOUR AI COMPUTER
The project helps you build your personal AI computer with open-source code and support for autonomous agents.
★ 1,577FAST LAUNCH QWEN3.8 ON AMD
The project provides scripts for instant deployment of Qwen3.8-Flash-Next on AMD Strix Halo (gfx1151) GPU via HIP. All dependencies are built automatically, and inference
★ 1,349 · ShellOLLAMA GUI: LOCAL CHAT AI
Web interface for chatting with local LLMs via the Ollama API. Allows switching models, keeping chat history, and adjusting generation settings directly in the browser. W
★ 1,270 · TypeScriptRUN 284B LLM ON CPU WITHOUT GPU
llama-modchip runs DeepSeek-V4-Flash 284B on pure 64GB CPU RAM: 44.4 tok/s prefill, 9-14 tok/s generation. 30B MoE models up to 111 tok/s. 256K context, Claude Code resto
★ 1,234 · PythonROW‑BOT PERSONAL AI ASSISTANT
Row‑Bot — local AI assistant with knowledge graph, voice, vision, and task automation. All data stays on your device.
★ 1,223 · PythonAI Assistant with Knowledge
A personal AI assistant featuring a knowledge graph, voice, automation, and health tracking. It runs locally via Ollama or optional cloud models.
★ 1,098 · PythonCode Agent for LLM
Code agent, optimized for smaller LLMs.
★ 1,083 · TypeScriptRUN LARGE LLMS LOCALLY
PolyStrata runs Qwen3.8-Flash, GLM 5.3 Flash, DeepSeek V4 and more on your PC. One-click install for Windows and Linux with OpenAI/Anthropic API on localhost.
★ 1,037 · C++SiliconScope — Apple Silicon Monitor
Native Apple Silicon system monitor in SwiftUI without sudo: tracks ANE, Media Engine, and memory bandwidth.
★ 1,024 · SwiftHAX MINIMALIST TERMINAL AGENT
Minimalist coding agent in C, running directly in the terminal. Allows writing and editing code without extra dependencies.
★ 999 · CSELFHOST AI: ONE COMMAND
Deploy n8n, Ollama, Flowise, RAG and over 30 tools with a single command, including automatic HTTPS.
★ 963 · ShellJEVOS: BINARY DECISIONS ON LAPTOP
Jevos is an open-source Jev alternative for yes/no decisions. Runs fully locally on CPU via FastAPI. Data never leaves your device.
★ 963 · Pythonawesome-openclaw - AI Agent for Chats
Curated list of resources, tools, and guides for OpenClaw — a self‑hosted AI agent for WhatsApp, Telegram, Discord, and 50+ integrations.
★ 956Which local LLM to choose?
Find a local model with the best performance on your hardware based on real benchmarks, not parameter count. One command — instant launch.
★ 947 · PythonLOCAL AI DRAFT HUMANIZER
The project rephrases AI-generated drafts so they read like human-written text while preserving all facts. It runs fully locally, with no data sent to the cloud, and supp
★ 935 · PythonPOKECLAW AI ON ANDROID
PokeClaw — the first local AI agent for controlling Android without cloud or API keys, built on Gemma 4. Control your phone by voice and gestures directly on the device.
★ 872 · KotlinAI Game Dev Desktop App
Desktop app where an AI agent sets up engines (Unreal, Unity, Godot, browser), builds your game, and play-tests it automatically.
★ 872 · TypeScriptINCARNAMIND AI CHAT WITH DOCUMENTS
Free desktop app for macOS, Windows and Linux, allowing you to ask questions about your documents and get answers with exact page links. IncarnaMind verifies each citatio
★ 858 · TypeScriptMCP CLIENT FOR LOCAL LLM
TUI client for Ollama with support for all MCP primitives, agent mode, multi-server capability, and streaming output. 749 ⭐
★ 836 · PythonLocal AI Agent for Chats
Free chat agent with local AI. Works without the cloud, preserving privacy and full control over data.
★ 783 · PythonNORNICDB: ULTRAFAST GRAPH-VECTOR DB
A distributed database featuring sub-millisecond HNSW search, graph traversal, and support for Neo4j Bolt/Cypher plus Qdrant gRPC.
★ 753 · GoMLX-DSPARK: 4X LLM ACCELERATION
Portable MLX port of DeepSeek DSpark and z‑lab DFlash for accelerating LLM decoding on Apple Silicon up to 4× without quality loss.
★ 740 · PythonJANAS: FAST LLM INFERENCE!!
janas runs huge LLM (up to 48 GB) on a 32 GB laptop, outpacing llama.cpp 8‑10×. On Qwen3-Next-80B it gives 37 tok/s w/o draft and 62 tok/s reading GGUF. Supports CPU+GPU
★ 712 · CSMOLMAXXING: LLM ON MICROCONTROLLERS
C library for running LLMs on microcontrollers and weak hardware. Alternative to llama.cpp focused on minimal memory usage. Supports quantization and various model archit
★ 691 · CSLOTSTREAM LAUNCHES QWEN3.8 ON MAC
Run Qwen3.8‑Flash‑Next (125B MoE, 104 GB at 4‑bit) on Mac by streaming experts from SSD on demand. Implemented with MLX + Swift, it offers an Ollama‑compatible API.
★ 673 · SwiftBrowser LLM Device Pool
Run large open‑source LLMs in the browser by linking multiple devices into a peer‑to‑peer compute pool, enabling chat and coding agents without any server.
★ 649 · JavaScriptOLLITERT LLM SERVER ANDROID
OlliteRT turns your Android smartphone into a local inference server compatible with the OpenAI API. It works completely offline, ensuring privacy and open-source code in
★ 633 · KotlinObsidian LLM Wiki Local
Fully local wiki for Obsidian via Ollama. Load Markdown notes — AI extracts concepts and automatically links your knowledge base. Zero data exchange.
★ 625 · PythonYOUR AI AGENTS IN MESSENGERS
abacusai-bot — 100% free open-source framework for running personal AI agents in Telegram, WhatsApp, and CLI. Written in TypeScript, supports tools, code, and integration
★ 557 · TypeScriptASKIMO: AI CHAT CLIENT
askimo is a universal AI client in Kotlin with support for chat, RAG, plans, and agents through multiple model providers.
★ 549 · KotlinLocal-First AI Creation Workstation
A local-first full-stack AI workstation with chat, comics, drama, writing, and learning modules, supporting vLLM, llama.cpp, and Transformers for multi-engine inference w
★ 535 · PythonLOCAL AI WITHOUT CLOUD
Hob — open-source desktop client running Gemma and gpt-oss locally on Mac, Windows, and Linux. Works fully offline, no data leaves your machine. Free alternative to Claud
★ 535 · TypeScriptRUNNING DEEPSEEK-V4.1 552B ON A SINGLE PC
Xolotl is an engine for running large MoE models (DeepSeek-V4.1-Flash 552B, GLM-5.3-Flash, Qwen3.8, LFM3-Omni) on a single PC with RTX 4090. It delivers 33 tok/s on agent
★ 534 · C++LOCAL AI UNCENSORED
Desktop app with chat, agent, image and video generation. One .exe, no Docker. 518 ⭐
★ 518 · TypeScriptCore AI Model Collection
A model zoo for Apple Core AI with Qwen3.5 and Gemma 4 conversion, custom Metal kernels, and a Swift runner.
★ 465 · SwiftEVOLVING AGENT FOR TEAMS
The director agent studies the team's work via CLAUDE.md and linked memory statements, forms a plan and delegates tasks to Claude Code or local models. Lessons are stored
★ 458 · TypeScriptLOCAL SPEECH-TO-SPEECH ASSISTANT ON GPU
Fully local voice assistant with ~0.23s latency from end of phrase to answer. Runs on AMD/Nvidia GPU under Windows and Linux without sending audio to cloud.
★ 448 · PythonARENO: Accelerate RL Single Node
ARENO simplifies scaling post-training RL on a single node, delivering high speed with minimal setup.
★ 389 · PythonLLAMAMPERE: LLM ACCELERATION ON RTX 3090
llama.cpp fork optimized for Ampere GPUs. Delivers 110+ tok/s on Qwen 3.8 at 100K context. Runs on 12 GB VRAM with 200K+ context.
★ 380 · C++New projects on this topic in Telegram
Telegram →