R_REDDYX.XYZ

Llama Cpp: open-source projects on GitHub

56 repositories on Llama Cpp, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

LLM 2094Gguf 47Local LLM 107Local AI 82Self Hosted 359Inference 95RAG 533Ollama 249Vllm 69On Device AI 38Cuda 48Privacy 89Llama 106Android 108
Osmantic/ODS

ODS - YOUR AI SERVER

ODS turns any computer into a powerful AI server: LLM inference, chat, voice, agents, RAG, and image generation.

★ 7,832 · Shell
ARahim3/DigUp

DIGUP SEARCH BY FILE CONTENT

DigUp indexes your local files — photos, PDFs, videos, audio and code — and lets you find them by meaning of the query. Search runs on-device using the EmbeddingGemma 2 m

★ 6,418 · Swift
SciSharp/LLamaSharp

LLAMASHARP - LOCAL LLM ON C#

A C#/.NET library for efficiently running LLaMA and LLaVA LLM models on a local device.

★ 3,826 · C#
AtomicBot-ai/atomic-agent

ATOMIC AGENT: LOCAL AI

The local AI agent atomic-agent runs entirely on your device, supports long context and precise tool invocation.

★ 3,682 · TypeScript
off-grid-ai/OGAM

OGAM Offline AI Swiss Knife

OGAM — chat, vision, speech, and image generation offline on your device without keys or network data.

★ 3,380 · TypeScript
Mobile-Artificial-Intelligence/maid

MAID: MANAGE AI MODELS

Free tool for working with local and remote LLMs: llama.cpp, Anthropic, DeepSeek, etc.

★ 2,776 · TypeScript
withcatai/node-llama-cpp

RUN LLM LOCALLY IN NODE.JS

Run AI models directly on your computer via Node.js. Enforced JSON schema validation right at the generation level.

★ 2,205 · TypeScript
Luce-Org/lucebox-hub

Lucebox Hub

High-speed server for running LLM locally on consumer hardware. Written in C++ for maximum performance.

★ 2,167 · C++
Roy3838/Observer

MICRO-AGENTS WATCH YOUR SCREEN

Observer — a TypeScript framework of local micro-agents. They watch your screen and automate routine without the cloud. 1.6k stars on GitHub.

★ 1,803 · TypeScript
StartLuxLabs/StartLux-Decision

STARTLUX‑DECISION: AI DECISIONS

Open decision models from 0.8B to 27B give calibrated probability for each option in one pass and a short thought only when unsure. They serve as the available inference

★ 1,745 · Python
antoinezambelli/forge

Framework for LLM Agents

Python framework for local LLM agents with tool calling support and multi-step agent workflows.

★ 1,661 · Python
RunanywhereAI/wally

VOICE ASSISTANT FOR MAC

Wally lets you control Mac by voice and search local documents offline without internet. It runs on Apple Silicon M1 using an on-device offline LLM via llama.cpp and RAG

★ 1,571 · C++
RunanywhereAI/RCLI

RCLI — Voice AI

Voice AI with RAG, no cloud: chat with Mac and request documents locally.

★ 1,507 · C++
FrigadeHQ/yogurt

PRIVATE AI MEETING NOTES

yogurt — private meeting notes powered by local AI. Works offline via llama.cpp, data never leaves your device.

★ 1,451 · TypeScript
synchopate/llama-modchip

RUN 284B LLM ON CPU WITHOUT GPU

llama-modchip runs DeepSeek-V4-Flash 284B on pure 64GB CPU RAM: 44.4 tok/s prefill, 9-14 tok/s generation. 30B MoE models up to 111 tok/s. 256K context, Claude Code resto

★ 1,234 · Python
wici-ai/inferssd

RUN MODELS LARGER THAN VRAM

inferssd runs MoE models exceeding RAM and VRAM by streaming experts from SSD. Built on llama.cpp with GGUF, CUDA and Apple Silicon support. 288 stars on GitHub.

★ 1,108 · C++
mybigday/llama.rn

LLAMA NOW IN REACT NATIVE

llama.rn is a React Native binding for llama.cpp, enabling LLM models to run directly on mobile devices. 981 stars on GitHub.

★ 1,074 · C++
kennss/SiliconScope

SiliconScope — Apple Silicon Monitor

Native Apple Silicon system monitor in SwiftUI without sudo: tracks ANE, Media Engine, and memory bandwidth.

★ 1,024 · Swift
OleksandrChekhovskyi/hax

HAX MINIMALIST TERMINAL AGENT

Minimalist coding agent in C, running directly in the terminal. Allows writing and editing code without extra dependencies.

★ 999 · C
feder-cr/jev

JEVOS: BINARY DECISIONS ON LAPTOP

Jevos is an open-source Jev alternative for yes/no decisions. Runs fully locally on CPU via FastAPI. Data never leaves your device.

★ 963 · Python
sgaofen/humanizer-local-model

LOCAL AI DRAFT HUMANIZER

The project rephrases AI-generated drafts so they read like human-written text while preserving all facts. It runs fully locally, with no data sent to the cloud, and supp

★ 935 · Python
Helldez/BigMoeOnEdge

BIGMOEONEDGE - SUPER MOE ON PHONE

The project enables running 284 B Mixture‑of‑Experts models on a device with 12 GB of RAM, using only CPU and llama.cpp without loss of accuracy.

★ 855 · C++
cobanov/PS5LM

LLM ON PS5: LLAMA.CPP NATIVELY

PS5LM ports llama.cpp to jailbroken PlayStation 5. Runs any GGUF models natively on console, including Qwen 3.8. 668 stars on GitHub, written in C++.

★ 846 · C++
hyperionaidev/smolmaxxing

SMOLMAXXING: LLM ON MICROCONTROLLERS

C library for running LLMs on microcontrollers and weak hardware. Alternative to llama.cpp focused on minimal memory usage. Supports quantization and various model archit

★ 691 · C
llmmanorg/llmman

RUN ANY AGENT ON ANY MODEL

llmman lets you run any AI‑agent on any model stored as an OCI image. It supports Claude Code, Codex, llama.cpp, MLX and other popular LLM frameworks.

★ 590 · Rust
oriveo/oriveo

ORIVEO MULTIMODEL CHAT

Open-source chat client with BYOK for iOS, Android and web. Supports OpenAI, Anthropic, Gemini and other compatible providers, including Ollama, LM Studio, llama.cpp, vLL

★ 563 · Swift
Yzw202011/OmniSpace

Local-First AI Creation Workstation

A local-first full-stack AI workstation with chat, comics, drama, writing, and learning modules, supporting vLLM, llama.cpp, and Transformers for multi-engine inference w

★ 535 · Python
FrigadeHQ/hob

LOCAL AI WITHOUT CLOUD

Hob — open-source desktop client running Gemma and gpt-oss locally on Mac, Windows, and Linux. Works fully offline, no data leaves your machine. Free alternative to Claud

★ 535 · TypeScript
Light-Heart-Labs/DreamServer

DreamServer: Local AI

DreamServer is an open-source project for running LLMs locally with a chat interface, voice control, agents, workflows, RAG, and image generation, operating without cloud

★ 523 · Python
AudarAI/Audar-ASR-V1

Audar-ASR-V1: Generative ASR

The project presents an Arabic-first generative speech recognition system with Flash and Turbo models, ranking first on the Open Universal Arabic ASR leaderboard. Include

★ 503 · Python
FuJacob/cotabby

COTABBY AI AutoComplete Mac

Local AI autocomplete for your entire Mac — works offline everywhere you type, with open-source Swift code.

★ 395 · Swift
JakeATX/llamAmpere

LLAMAMPERE: LLM ACCELERATION ON RTX 3090

llama.cpp fork optimized for Ampere GPUs. Delivers 110+ tok/s on Qwen 3.8 at 100K context. Runs on 12 GB VRAM with 200K+ context.

★ 380 · C++
hogeheer499-commits/strix-halo-guide

STRIX HALO LLM Setup

Guide for LLM on Ryzen AI MAX+ 395 and Radeon 8060S via Ollama, llama.cpp and ROCm. Achieving 100 tokens/s 30B Qwen and 140 tokens/s CHADROCK MTP.

★ 373 · Python
abdulmominsakib/localmind

LOCALMIND: OFFLINE LLM CHAT

Localmind — a mobile Dart/Flutter application for running local LLM models and AI servers without data leakage. Provides a fast chat, support for Gemma models, and compa

★ 326 · Dart
ThinkOffApp/CarWatch

CARWATCH AI AGENT CAR

CarWatch turns Raspberry Pi 5, a dashcam, and local AI into a chat agent for your car. Talk to the vehicle, get data, and control functions in real time. This is a CodeWa

★ 305 · Python
T-Crypt/berth

GPU BY TURN WITHOUT CRASHES

Berth leases the GPU to chat, image, and voice engines in turn. Two processes never load together and crash each other. Single Go binary, no dependencies.

★ 297 · Go
mohitsoni48/TurboLLM

TURBO LLM: OFFLINE WITH AI

Run any local LLM with automatic GPU tuning, a web UI, and an OpenAI/Anthropic‑compatible API.

★ 296 · TypeScript
raketenkater/ggrun

Not a Llama, no stance

Auto-tuned launcher for GGUF models on llama.cpp — server with multi‑GPU support, hardware‑specific tuning, and crash recovery. Stars: 226.

★ 284 · Go
defilantech/LLMKube

LLMKUBE: LLM IN KUBERNETES

LLMKube - Kubernetes operator for LLM inference on heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, Apple Silicon Metal. Supports llama.cpp, vLLm, TGI, mlx-server, multi

★ 266 · Go
engeldlgado/toshllm

TOSHLLM: RUN LLM ON MAC

TOSHLLM — a native macOS app in Swift that runs large language models locally on Intel‑Mac with AMD GPU, using Metal acceleration. Supports llama.cpp and is compatible wi

★ 258 · Swift
huawei-csl/MACaQ

Model-Agnostic Quantization Framework

Quantize and run text, image, video, and audio models across GGUF, llama.cpp, vLLM, Diffusers, and PyTorch with state-of-the-art model-agnostic quantization.

★ 254 · C++
jammaru/sakutto

BROWSER EXTENSION FOR LOCAL TRANSLATION

Chrome and Firefox extension translating pages, PDFs, subtitles, and typed text locally via PLaMo model. Works without server, accounts, or telemetry.

★ 253 · TypeScript
ayushh0110/ScreenMind

SCREENMIND - YOUR SCREEN MEMORY

Remembers your screen, searches by content, and chats with AI locally.

★ 249 · Python
morganlinton/Albatross

Albatross: Open AI Agent

Albatross is an open terminal AI agent with transparent multi‑model routing. It runs locally via Ollama, LM Studio, MLX or in the cloud using your API keys.

★ 244 · Rust
RustingStudio/RustingStrata

PURE-RUST ENGINE FOR QWEN GGUF

RustingStrata is a pure-Rust inference engine for Qwen GGUF (Qwen3–3.8, MoE up to 770B+). Supports CUDA/Vulkan, hybrid CPU+GPU experts, MTP speculative decoding. Up to 1.

★ 231 · Rust
llamatik/Llamatik

ON-DEVICE AI SDK KOTLIN

Llamatik is an on-device AI SDK for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). Supports LLM inference, Speech-to-Text, image generation, Agent Framework and

★ 220 · Kotlin
Eric-Terminal/ETOS-LLM-Studio

ETOS-LLM-STUDIO:AI ASSISTANT ON iOS

Native LLM client iOS & Apple Watch:run local GGUF via llama.cpp or OpenAI/Claude/Gemini Local RAG, MCP tools, Siri Shortcuts, sync Built with Swift.

★ 219 · Swift
yoloshii/ClawMem

CLAWMEM MEMORY FOR AI AGENTS

ClawMem — local memory layer for AI agents, working with Claude Code, OpenClaw and Hermes. It provides hooks, MCP server and hybrid RAG search for fast access to knowledg

★ 212 · TypeScript
justxor/deepseek-local-guide-ru

DeepSeek Local Setup Guide

A Russian‑language guide that walks you through installing and running DeepSeek models (V4, R1) locally on PC, Mac, or server using tools like Ollama, llama.cpp, LM Studi

★ 192 · Shell
Aevonix/FlashAlign

On-Demand Model Alignment

Aligns LLMs instantly per request for verified users or always-on mode. Integrates with Hermes, OpenClaw, and custom harnesses.

★ 126 · Python
DevDock-AI/Speedy

125B MoE Inference on RTX

Run Qwen3.8-Flash-Next (125B MoE) with 1M context on one or two RTX 3080 10GB cards. Tuned, benchmarked fork of Strata.

★ 71 · C++
anisayari/sparky

Local AI Assistant for Windows

Run a private AI with chat, vision, RAG, web search, and voice entirely on your Windows PC — one-line install, auto-tuned models.

★ 65 · Python
jatinbishla/Umless

On-Device Voice Dictation

Local voice dictation using Whisper/Parakeet and a LoRA-tuned Qwen model to correct transcripts on-device.

★ 61 · Rust
johnpsasser/lifeboat

Offline LLM Emergency Assistant

Runs a local LLM with 200 GB offline knowledge library on macOS, providing cited answers without internet connectivity.

★ 43 · Python
hadihonarvar/flock

Self-hosted LLM gateway

A single Go binary turns your Mac and Linux machines into a private inference cluster featuring multi‑machine routing, sharding via llama.cpp‑RPC, and support for the Ope

★ 38 · Go
avifenesh/memra

MEMRA - AI Accelerator for RTX

Memra is a from‑scratch LLM inference implementation optimized for RTX 5090 (sm_120a) and H100 (sm_90a) architectures in Rust.

★ 6 · Rust

New projects on this topic in Telegram

Telegram →
Llama Cpp — 56 open-source GitHub projects | REDDYX AI