On Device AI: open-source projects on GitHub
38 repositories on On Device AI, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
Local AI SDK Set
Production-ready toolkit for running AI models on a local device.
★ 10,350 · C++GENIEX: LLMs on Devices
Run cutting-edge multimodal models on GPU, NPU, and CPU — from PC to mobile and IoT devices.
★ 8,530 · RustNEXA-SDK AI Everywhere Always
Run cutting‑edge LLMs and VLMs on GPU, NPU, and CPU — from PC to mobile and IoT devices. Support for models like GPT‑OSS, Granite‑4, Qwen‑3‑VL, and more.
★ 8,094 · RustTURBO-FIELDFARE - GEMMA 4 ON MACBOOK
Run Gemma 4 26B‑A4B on any MacBook with an M‑series chip using under 2 GB of RAM. Fast and with no extra resources.
★ 7,391 · SwiftAI FRAME FOR DEVICES
This project offers low-bit quant kernels and runtime for consumer devices on Android/ARM. It solves edge-ai tasks with minimal resources.
★ 6,192 · C++Efficient AI for Mobile Devices
An AI engine with minimal latency for smartphones and wearable devices.
★ 4,949 · CLocal Claude Code
Run Claude Code entirely on-device with Apple Silicon using an MLX-native server compatible with the Anthropic API. Supports local Qwen 3.5 122B, Llama 3.3 70B, and Gemma
★ 2,578 · PythonMano-P: GUI agent for edge
Open-source GUI-VLA agent for edge devices, #1 on OSWorld (58.2%). Runs locally on Mac mini/MacBook — data never leaves the device.
★ 2,031Parlor AI on device
Parlor is an open project that enables natural voice and visual dialogues with AI running fully locally on your device. It is built on the Gemma 4 E2B and Kokoro models.
★ 1,752 · HTMLRUN LLM LOCALLY WITH NOBODYWHO
NobodyWho lets you run LLM locally on any device thanks to an efficient inference engine built in Rust.
★ 1,677 · RustLOCAL AI FOR MAC OFFLINE
Nativ runs open LLMs locally on Apple Silicon. No data ever leaves your Mac. Built on MLX and Swift.
★ 1,591 · SwiftVOICE ASSISTANT FOR MAC
Wally lets you control Mac by voice and search local documents offline without internet. It runs on Apple Silicon M1 using an on-device offline LLM via llama.cpp and RAG
★ 1,571 · C++RCLI — Voice AI
Voice AI with RAG, no cloud: chat with Mac and request documents locally.
★ 1,507 · C++RUN LLM LOCALLY IN REACT NATIVE
Run large language models directly on device without internet. Full Vercel AI SDK compatibility for familiar API.
★ 1,438 · TypeScriptPHONECLAW: On-device AI Agent
On-device AI agent for smartphones, powered by Gemma 4, provides fast access to intelligent features without the cloud.
★ 1,291 · SwiftFOUNDATION MODELS FRAMEWORK
A practical lab for building and testing applications using Apple Foundation Models.
★ 1,192 · SwiftFRAMEWORK FINE-TUNING FOR iOS 26
Examples of apps for the Foundation Models Framework on iOS 26 and macOS 26 — exploring fine‑tuning of LLMs directly on the device.
★ 1,060 · SwiftSiliconScope — Apple Silicon Monitor
Native Apple Silicon system monitor in SwiftUI without sudo: tracks ANE, Media Engine, and memory bandwidth.
★ 1,024 · SwiftJEVOS: BINARY DECISIONS ON LAPTOP
Jevos is an open-source Jev alternative for yes/no decisions. Runs fully locally on CPU via FastAPI. Data never leaves your device.
★ 963 · PythonPOKECLAW AI ON ANDROID
PokeClaw — the first local AI agent for controlling Android without cloud or API keys, built on Gemma 4. Control your phone by voice and gestures directly on the device.
★ 872 · KotlinBIGMOEONEDGE - SUPER MOE ON PHONE
The project enables running 284 B Mixture‑of‑Experts models on a device with 12 GB of RAM, using only CPU and llama.cpp without loss of accuracy.
★ 855 · C++WAX: SINGLE-FILE AI MEMORY
Wax is a single‑file memory for AI agents with sub‑millisecond RAG on Apple Silicon, written entirely in Swift and optimized for Metal.
★ 810 · SwiftSWIFTLM: SWIFT LLM ON APPLE
Native LLM inference server in Swift with MLX, OpenAI‑compatible API, SSD streaming for MoE >100B and TurboQuant KV‑cache compression.
★ 799 · SwiftSENTIENT OS: Your AI Assistant
A local LLM in Swift analyzes your data and independently offers to perform tasks for you.
★ 750 · SwiftFLUTTER EDGE AI: RIGHT ON DEVICE
On-device AI for Flutter: Gemma and other LLMs, multimodality, function calling, embeddings, RAG and speech. Supported on Android, iOS, Web, macOS, Windows and Linux.
★ 663 · JavaScriptOLLITERT LLM SERVER ANDROID
OlliteRT turns your Android smartphone into a local inference server compatible with the OpenAI API. It works completely offline, ensuring privacy and open-source code in
★ 633 · KotlinXYBRID: On-Device AI
xybrid — framework for creating applications with on‑device AI in Rust.
★ 524 · RustZERAIX: LOCAL AI DESKTOP
Local open-source environment for running AI models on device. Speeds up inference without cloud.
★ 405 · TypeScriptCOTABBY AI AutoComplete Mac
Local AI autocomplete for your entire Mac — works offline everywhere you type, with open-source Swift code.
★ 395 · SwiftLOCALMIND: OFFLINE LLM CHAT
Localmind — a mobile Dart/Flutter application for running local LLM models and AI servers without data leakage. Provides a fast chat, support for Gemma models, and compa
★ 326 · DartOFFLINE LLM ON ANDROID: FULL PRIVACY
OfflineLLM runs any GGUF models locally via llama.cpp with ARM optimisation. Zero network permissions: encrypted settings, biometric lock, and tamper detection. Supports
★ 291 · KotlinBROWSER EXTENSION FOR LOCAL TRANSLATION
Chrome and Firefox extension translating pages, PDFs, subtitles, and typed text locally via PLaMo model. Works without server, accounts, or telemetry.
★ 253 · TypeScriptLocal AI roleplay app
Open Dungeon lets you generate stories and inline images entirely on your computer, using Gemma 4 QAT via Ollama (and others) + FLUX, with no accounts or API keys.
★ 241 · TypeScriptOn-Device Mobile Diffusion Studio
Run ComfyUI-style node graphs for SDXL, Flux, and 14B video models locally on Android via Qualcomm NPU. Fully offline with inpainting, ControlNet, LoRA, and IPAdapter sup
★ 233 · KotlinON-DEVICE AI SDK KOTLIN
Llamatik is an on-device AI SDK for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). Supports LLM inference, Speech-to-Text, image generation, Agent Framework and
★ 220 · KotlinCOMPACT TTS WITH VOICE CLONING
Multilingual text-to-speech model with zero-shot voice cloning. Runs locally on GPU, CPU, Apple Silicon, and in browser via WebGPU.
★ 219 · PythonLabLLM on macOS
LabLLM is a native macOS app where you can build the architecture of a tiny language model, train its weights, and see the model emerge from scratch on Apple Silicon usin
★ 39 · SwiftNEEDLE - 26M MODEL FOR NANODEVICES
A compact 26M function-calling model that runs on resource-constrained devices.
★ 11 · PythonNew projects on this topic in Telegram
Telegram →