Llama Cpp: open-source projects on GitHub
56 repositories on Llama Cpp, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
ODS - YOUR AI SERVER
ODS turns any computer into a powerful AI server: LLM inference, chat, voice, agents, RAG, and image generation.
★ 7,832 · ShellDIGUP SEARCH BY FILE CONTENT
DigUp indexes your local files — photos, PDFs, videos, audio and code — and lets you find them by meaning of the query. Search runs on-device using the EmbeddingGemma 2 m
★ 6,418 · SwiftLLAMASHARP - LOCAL LLM ON C#
A C#/.NET library for efficiently running LLaMA and LLaVA LLM models on a local device.
★ 3,826 · C#ATOMIC AGENT: LOCAL AI
The local AI agent atomic-agent runs entirely on your device, supports long context and precise tool invocation.
★ 3,682 · TypeScriptOGAM Offline AI Swiss Knife
OGAM — chat, vision, speech, and image generation offline on your device without keys or network data.
★ 3,380 · TypeScriptMAID: MANAGE AI MODELS
Free tool for working with local and remote LLMs: llama.cpp, Anthropic, DeepSeek, etc.
★ 2,776 · TypeScriptRUN LLM LOCALLY IN NODE.JS
Run AI models directly on your computer via Node.js. Enforced JSON schema validation right at the generation level.
★ 2,205 · TypeScriptLucebox Hub
High-speed server for running LLM locally on consumer hardware. Written in C++ for maximum performance.
★ 2,167 · C++MICRO-AGENTS WATCH YOUR SCREEN
Observer — a TypeScript framework of local micro-agents. They watch your screen and automate routine without the cloud. 1.6k stars on GitHub.
★ 1,803 · TypeScriptSTARTLUX‑DECISION: AI DECISIONS
Open decision models from 0.8B to 27B give calibrated probability for each option in one pass and a short thought only when unsure. They serve as the available inference
★ 1,745 · PythonFramework for LLM Agents
Python framework for local LLM agents with tool calling support and multi-step agent workflows.
★ 1,661 · PythonVOICE ASSISTANT FOR MAC
Wally lets you control Mac by voice and search local documents offline without internet. It runs on Apple Silicon M1 using an on-device offline LLM via llama.cpp and RAG
★ 1,571 · C++RCLI — Voice AI
Voice AI with RAG, no cloud: chat with Mac and request documents locally.
★ 1,507 · C++PRIVATE AI MEETING NOTES
yogurt — private meeting notes powered by local AI. Works offline via llama.cpp, data never leaves your device.
★ 1,451 · TypeScriptRUN 284B LLM ON CPU WITHOUT GPU
llama-modchip runs DeepSeek-V4-Flash 284B on pure 64GB CPU RAM: 44.4 tok/s prefill, 9-14 tok/s generation. 30B MoE models up to 111 tok/s. 256K context, Claude Code resto
★ 1,234 · PythonRUN MODELS LARGER THAN VRAM
inferssd runs MoE models exceeding RAM and VRAM by streaming experts from SSD. Built on llama.cpp with GGUF, CUDA and Apple Silicon support. 288 stars on GitHub.
★ 1,108 · C++LLAMA NOW IN REACT NATIVE
llama.rn is a React Native binding for llama.cpp, enabling LLM models to run directly on mobile devices. 981 stars on GitHub.
★ 1,074 · C++SiliconScope — Apple Silicon Monitor
Native Apple Silicon system monitor in SwiftUI without sudo: tracks ANE, Media Engine, and memory bandwidth.
★ 1,024 · SwiftHAX MINIMALIST TERMINAL AGENT
Minimalist coding agent in C, running directly in the terminal. Allows writing and editing code without extra dependencies.
★ 999 · CJEVOS: BINARY DECISIONS ON LAPTOP
Jevos is an open-source Jev alternative for yes/no decisions. Runs fully locally on CPU via FastAPI. Data never leaves your device.
★ 963 · PythonLOCAL AI DRAFT HUMANIZER
The project rephrases AI-generated drafts so they read like human-written text while preserving all facts. It runs fully locally, with no data sent to the cloud, and supp
★ 935 · PythonBIGMOEONEDGE - SUPER MOE ON PHONE
The project enables running 284 B Mixture‑of‑Experts models on a device with 12 GB of RAM, using only CPU and llama.cpp without loss of accuracy.
★ 855 · C++LLM ON PS5: LLAMA.CPP NATIVELY
PS5LM ports llama.cpp to jailbroken PlayStation 5. Runs any GGUF models natively on console, including Qwen 3.8. 668 stars on GitHub, written in C++.
★ 846 · C++SMOLMAXXING: LLM ON MICROCONTROLLERS
C library for running LLMs on microcontrollers and weak hardware. Alternative to llama.cpp focused on minimal memory usage. Supports quantization and various model archit
★ 691 · CRUN ANY AGENT ON ANY MODEL
llmman lets you run any AI‑agent on any model stored as an OCI image. It supports Claude Code, Codex, llama.cpp, MLX and other popular LLM frameworks.
★ 590 · RustORIVEO MULTIMODEL CHAT
Open-source chat client with BYOK for iOS, Android and web. Supports OpenAI, Anthropic, Gemini and other compatible providers, including Ollama, LM Studio, llama.cpp, vLL
★ 563 · SwiftLocal-First AI Creation Workstation
A local-first full-stack AI workstation with chat, comics, drama, writing, and learning modules, supporting vLLM, llama.cpp, and Transformers for multi-engine inference w
★ 535 · PythonLOCAL AI WITHOUT CLOUD
Hob — open-source desktop client running Gemma and gpt-oss locally on Mac, Windows, and Linux. Works fully offline, no data leaves your machine. Free alternative to Claud
★ 535 · TypeScriptDreamServer: Local AI
DreamServer is an open-source project for running LLMs locally with a chat interface, voice control, agents, workflows, RAG, and image generation, operating without cloud
★ 523 · PythonAudar-ASR-V1: Generative ASR
The project presents an Arabic-first generative speech recognition system with Flash and Turbo models, ranking first on the Open Universal Arabic ASR leaderboard. Include
★ 503 · PythonCOTABBY AI AutoComplete Mac
Local AI autocomplete for your entire Mac — works offline everywhere you type, with open-source Swift code.
★ 395 · SwiftLLAMAMPERE: LLM ACCELERATION ON RTX 3090
llama.cpp fork optimized for Ampere GPUs. Delivers 110+ tok/s on Qwen 3.8 at 100K context. Runs on 12 GB VRAM with 200K+ context.
★ 380 · C++STRIX HALO LLM Setup
Guide for LLM on Ryzen AI MAX+ 395 and Radeon 8060S via Ollama, llama.cpp and ROCm. Achieving 100 tokens/s 30B Qwen and 140 tokens/s CHADROCK MTP.
★ 373 · PythonLOCALMIND: OFFLINE LLM CHAT
Localmind — a mobile Dart/Flutter application for running local LLM models and AI servers without data leakage. Provides a fast chat, support for Gemma models, and compa
★ 326 · DartCARWATCH AI AGENT CAR
CarWatch turns Raspberry Pi 5, a dashcam, and local AI into a chat agent for your car. Talk to the vehicle, get data, and control functions in real time. This is a CodeWa
★ 305 · PythonGPU BY TURN WITHOUT CRASHES
Berth leases the GPU to chat, image, and voice engines in turn. Two processes never load together and crash each other. Single Go binary, no dependencies.
★ 297 · GoTURBO LLM: OFFLINE WITH AI
Run any local LLM with automatic GPU tuning, a web UI, and an OpenAI/Anthropic‑compatible API.
★ 296 · TypeScriptNot a Llama, no stance
Auto-tuned launcher for GGUF models on llama.cpp — server with multi‑GPU support, hardware‑specific tuning, and crash recovery. Stars: 226.
★ 284 · GoLLMKUBE: LLM IN KUBERNETES
LLMKube - Kubernetes operator for LLM inference on heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, Apple Silicon Metal. Supports llama.cpp, vLLm, TGI, mlx-server, multi
★ 266 · GoTOSHLLM: RUN LLM ON MAC
TOSHLLM — a native macOS app in Swift that runs large language models locally on Intel‑Mac with AMD GPU, using Metal acceleration. Supports llama.cpp and is compatible wi
★ 258 · SwiftModel-Agnostic Quantization Framework
Quantize and run text, image, video, and audio models across GGUF, llama.cpp, vLLM, Diffusers, and PyTorch with state-of-the-art model-agnostic quantization.
★ 254 · C++BROWSER EXTENSION FOR LOCAL TRANSLATION
Chrome and Firefox extension translating pages, PDFs, subtitles, and typed text locally via PLaMo model. Works without server, accounts, or telemetry.
★ 253 · TypeScriptSCREENMIND - YOUR SCREEN MEMORY
Remembers your screen, searches by content, and chats with AI locally.
★ 249 · PythonAlbatross: Open AI Agent
Albatross is an open terminal AI agent with transparent multi‑model routing. It runs locally via Ollama, LM Studio, MLX or in the cloud using your API keys.
★ 244 · RustPURE-RUST ENGINE FOR QWEN GGUF
RustingStrata is a pure-Rust inference engine for Qwen GGUF (Qwen3–3.8, MoE up to 770B+). Supports CUDA/Vulkan, hybrid CPU+GPU experts, MTP speculative decoding. Up to 1.
★ 231 · RustON-DEVICE AI SDK KOTLIN
Llamatik is an on-device AI SDK for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). Supports LLM inference, Speech-to-Text, image generation, Agent Framework and
★ 220 · KotlinETOS-LLM-STUDIO:AI ASSISTANT ON iOS
Native LLM client iOS & Apple Watch:run local GGUF via llama.cpp or OpenAI/Claude/Gemini Local RAG, MCP tools, Siri Shortcuts, sync Built with Swift.
★ 219 · SwiftCLAWMEM MEMORY FOR AI AGENTS
ClawMem — local memory layer for AI agents, working with Claude Code, OpenClaw and Hermes. It provides hooks, MCP server and hybrid RAG search for fast access to knowledg
★ 212 · TypeScriptDeepSeek Local Setup Guide
A Russian‑language guide that walks you through installing and running DeepSeek models (V4, R1) locally on PC, Mac, or server using tools like Ollama, llama.cpp, LM Studi
★ 192 · ShellOn-Demand Model Alignment
Aligns LLMs instantly per request for verified users or always-on mode. Integrates with Hermes, OpenClaw, and custom harnesses.
★ 126 · Python125B MoE Inference on RTX
Run Qwen3.8-Flash-Next (125B MoE) with 1M context on one or two RTX 3080 10GB cards. Tuned, benchmarked fork of Strata.
★ 71 · C++Local AI Assistant for Windows
Run a private AI with chat, vision, RAG, web search, and voice entirely on your Windows PC — one-line install, auto-tuned models.
★ 65 · PythonOn-Device Voice Dictation
Local voice dictation using Whisper/Parakeet and a LoRA-tuned Qwen model to correct transcripts on-device.
★ 61 · RustOffline LLM Emergency Assistant
Runs a local LLM with 200 GB offline knowledge library on macOS, providing cited answers without internet connectivity.
★ 43 · PythonSelf-hosted LLM gateway
A single Go binary turns your Mac and Linux machines into a private inference cluster featuring multi‑machine routing, sharding via llama.cpp‑RPC, and support for the Ope
★ 38 · GoMEMRA - AI Accelerator for RTX
Memra is a from‑scratch LLM inference implementation optimized for RTX 5090 (sm_120a) and H100 (sm_90a) architectures in Rust.
★ 6 · RustNew projects on this topic in Telegram
Telegram →