LLM: open-source projects on GitHub
2095 repositories on LLM, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
AI Inference Gateway
Synapass is a Go‑based AI inference gateway that routes requests across multiple LLM providers, applies policies, caches responses, and provides observability.
★ 98 · GoLLM Intern Skill
This project provides a set of LLM-based tools for improving resumes, matching jobs, fact-checking experience, and preparing for interviews.
★ 96 · MarkdownFree Roblox Studio AI Agent
Free Windows AI agent that plans, writes, and playtests Luau code in Roblox Studio with live diffs.
★ 95AI Agent Platform
A platform for creating role‑specialized AI agents with support for multiple LLMs, MCP, and task management via a web interface and CLI.
★ 90 · TypeScriptAI Web Answer Crawler
A MCP server and CLI that crawls websites to locate answer pages for AI agents, achieving twice the hit rate of Crawl4AI at one‑tenth the cost of Claude crawling.
★ 89 · Python1Panel AI Gateway
1Panel AI Gateway is a comprehensive solution for managing AI services: a single entry point, intelligent routing, and full compliance auditing, enabling secure, efficien
★ 86AGENT SPHERE: AI ORCHESTRATION
Platform for orchestrating AI agents with an LLM engine, built-in tools, and support for MCP, CLI, and browser operations.
★ 78 · JavaLLM Prefix Cache Simulator
Simulates prefix caching mechanisms for LLM inference to analyze cache hit rates and memory efficiency.
★ 76 · C++Analytic Context Layer
ClariLayer preserves your analytic context between AI sessions, automatically loading the needed data and metadata via the MCP protocol.
★ 71 · TypeScript125B MoE Inference on RTX
Run Qwen3.8-Flash-Next (125B MoE) with 1M context on one or two RTX 3080 10GB cards. Tuned, benchmarked fork of Strata.
★ 71 · C++SparklingKit: Local AI Platform
SparklingKit — a local toolkit for OCR, transcription, translation, image generation, and mind‑map creation, enabling composable workflows on DGX Spark.
★ 70 · TypeScriptAWESOME-AGENT-EVOLUTION: AGENT EVOLUTION MAP
Open map covering the evolution of AI agents, their self-improvement, memory, skills, and swarm systems. 🌟 208 stars, JS
★ 68 · JavaScriptLocal AI Assistant for Windows
Run a private AI with chat, vision, RAG, web search, and voice entirely on your Windows PC — one-line install, auto-tuned models.
★ 65 · PythonSEMBLE: Code Search for AI Agents
Hybrid BM25 + semantic search with AST and dependency analysis. A replacement for grep/cat/read/ls for Claude Code, Cursor, Aider, and other AI agents.
★ 63 · RustRose — PyTorch Optimizer
Stateless PyTorch optimizer that normalizes gradients for stable and balanced model training.
★ 60 · PythonAI Agent Rules Sync
Stores a single source of rules for AI coding agents and automatically converts them to AGENTS.md, CLAUDE.md, .cursorrules, and other formats, or creates a new AGENTS.md
★ 59 · JavaScriptMulti-Agent Deep Research Assistant
LangGraph-based research agent with interruptible recovery, dependency-aware parallel execution, citation governance, and 45% cost reduction.
★ 58 · PythonSecure LLM for Code
An LLM-based system that analyzes AI-generated code for vulnerabilities and suggests automatic fixes. Integrated into CI/CD for continuous application protection.
★ 57 · PythonNLP Trainer for LLM 2026
Platform for fine-tuning large language models with automatic hyperparameter selection and visual metric tracking.
★ 55 · HTMLAI Clipboard Manager for macOS
Open-source clipboard manager with history, OCR search, AI translation, and a bring-your-own-model agent. Native Swift, zero dependencies.
★ 55 · SwiftAI Agent Recovery Controller
Open-source tool for managing execution and recovery of AI agents that use code and external tools. Ensures reliable task execution and automatic rollback on errors.
★ 54 · PythonLLM course from scratch
Practical course on 23 Jupyter notebooks where you build tokenizers, attention, MoE, RLHF, inference, evaluation, and distillation of modern LLMs in PyTorch.
★ 53 · Jupyter NotebookSpeculative Decoding Training Framework
ASpec accelerates LLM inference by training efficient draft models for speculative decoding with high-throughput distributed training.
★ 53 · PythonSelf-Hosted RAG Search API
A self-hosted API that performs web search, extracts content, generates embeddings, and produces answers via retrieval-augmented generation.
★ 51 · PythonKeyType: AI Autocomplete
KeyType is an open-source Swift project that adds system-wide AI autocompletion to macOS. It works like Cotypist, predicting the next text in any application.
★ 50 · SwiftVigils — AI Agent Control
Vigils is a local control panel for AI agents that lets you monitor their actions, approve important operations, and keep secrets safe. It is built with Rust, Tauri, and
★ 50 · RustAccelerated Text Generation
Fast and lossless output of language models using double diffusion decoding.
★ 46 · PythonInteractive 3D AI Agent World
Watch AI agents live, work, and teach in a self‑hosted 3D simulation powered by local models.
★ 45 · PythonLocal AI Meeting Assistant
Local-first AI meeting notes for Mac with on-device transcription, speaker diarization, and auto-generated minutes linked to audio quotes. Supports Tencent Meeting, Feish
★ 45 · RustAI-powered Docs Search
Embed an AI chatbot and site search into any documentation site with a single <script> tag, providing streaming answers with source citations.
★ 44 · TypeScriptOffline LLM Emergency Assistant
Runs a local LLM with 200 GB offline knowledge library on macOS, providing cited answers without internet connectivity.
★ 43 · PythonLabLLM on macOS
LabLLM is a native macOS app where you can build the architecture of a tiny language model, train its weights, and see the model emerge from scratch on Apple Silicon usin
★ 39 · SwiftSelf-hosted LLM gateway
A single Go binary turns your Mac and Linux machines into a private inference cluster featuring multi‑machine routing, sharding via llama.cpp‑RPC, and support for the Ope
★ 38 · GoInference Agent Tokenmaxxing
Agentic-in/inferoa is a framework for creating agents optimized for inference with Tokenmaxxing technology, providing high efficiency when working with large language mod
★ 38 · TypeScriptONGRID 3D Knowledge Graph
ONGRID creates interactive 3D knowledge graphs from ontologies, enabling retrieval, inference, and zero‑token answer verification.
★ 36 · HTMLAI Council for macOS
The app lets you ask a single question to multiple AI models simultaneously, collect their answers, facilitate a blind exchange of critiques, and pick the best option. It
★ 35 · SwiftCHECK CITATION IN CHATGPT
Python tool geo-score checks if ChatGPT will cite your site. Runs open GEO rubric in 20 seconds. Zero dependencies, 360 GitHub stars.
★ 34 · PythonSelf-Hosted Jev Decision Model
Run the selfjev-4b decision model locally with a Jev-compatible API, Python SDK, and tools for fine-tuning and RLCD training.
★ 34 · PythonSoulX-Transcriber Speech Transcriber
End-to-end framework for multi-speaker transcription that simultaneously determines who speaks, when, and what is said. Implemented in Python.
★ 33 · PythonCodeglance: Code Tour
Codeglance provides an instant overview of any codebase in just 10 seconds, generating an interactive map of dependencies and key components. Simply run one command to ge
★ 33 · TypeScriptCan I finetune this?
Tool to check whether a Hugging Face model fits in your GPU memory and can be fine-tuned locally.
★ 31 · PythonSisyphus Academica
The project is a swarm of 20+ AI agents that jointly generate scientific articles with verified references, using six novelty mechanisms and fully avoiding boilerplate AI
★ 31 · PythonEchoCut: AI Video Converter
The project allows you to turn raw video recordings into finished branded content, optimized for any platforms, using local FFmpeg, WhisperX/MLX, and Ollama models in one
★ 31 · JavaScriptDeep Web Researcher
A local agent for deep web research that can bypass bot protection and gather data from any site. It generates reports with accurate links and works fully offline.
★ 31 · PythonOllama Model Test
Ollama Model Test (omt) automatically evaluates different Ollama models on given tasks, comparing them by accuracy, speed, and resource usage to identify the optimal opti
★ 31 · PythonPure Rust LFM2.5-8B-A1B
CPU-only implementation of the LiquidAI LFM2.5-8B-A1B model in pure Rust. Enables text inference without GPU and external libraries.
★ 31 · RustOpen Decision Model API
Provides an open decision model exposing Jev's API for typed answers (yes/no, choice, score, multi) with probabilities from forward passes, using Qwen3.5-4B fine‑tuned wi
★ 25 · PythonOPEN-SOURCE AI FOR SLACK
Standalone open-source AI assistant for Slack. Supports any LLM and is an alternative to Claude Tag.
★ 20 · PythonAGENT-CHIEF: ATTENTION PROTECTION
The local Chief layer intercepts all agents, alerts, and feeds, turning them into a single honest signal: to interrupt or not.
★ 18 · PythonAWESOME-DSH PLUGINS CATALOG
A curated collection of top plugins for DeepSeek Harness that simplify extending functionality and integrating AI models.
★ 17GPT-6 ASTRA: PROMPTS INTEGRATIONS
Evidence-backed use cases, prompts, integrations, evaluations, and safety notes for OpenAI GPT-6 Astra.
★ 17 · PythonLocal LLM Decision API
Wrap local language models as typed decision APIs returning probabilistic answers with structured outputs.
★ 17 · PythonNEEDLE - 26M MODEL FOR NANODEVICES
A compact 26M function-calling model that runs on resource-constrained devices.
★ 11 · PythonAWESOME-CLAUDE-FABLE-5 – TOP IDEAS
A collection of the best use cases, tutorials, and demos of Claude Fable 5 with direct links to sources.
★ 6MEMRA - AI Accelerator for RTX
Memra is a from‑scratch LLM inference implementation optimized for RTX 5090 (sm_120a) and H100 (sm_90a) architectures in Rust.
★ 6 · RustNew projects on this topic in Telegram
Telegram →