zylon-ai/private-gpt
API LAYER FOR LOCAL AI APPLICATIONS
PrivateGPT is an open API layer for building private AI applications on local models. It connects to any OpenAI-compatible inference server and provides RAG, tools, MCP, and structured data access.
// KEY FEATURES
- RAG with citations and agentic search
- Built-in tools: web search, code, page fetch
- MCP connectors and custom tools support
- Structured API access to databases and CSV
// DEEP DIVE: HOW IT WORKS
PrivateGPT does not run models itself but works as an API proxy: the application calls the Anthropic API, and PrivateGPT forwards requests to an OpenAI-compatible server (Ollama, llama.cpp, vLLM) via OPENAI_API_BASE. A web UI at /ui is included for demos, but the core product is the API.
Unlike frameworks, it provides ready-made primitives: streaming messages API, file ingestion, RAG with citations and agentic mode, built-in tools (web search, fetch, code) mirroring the Claude API, MCP connectors, custom tools, and structured access to databases and CSVs. Battle-tested in production by the Zylon platform.
Requires an LLM server (Ollama) and models: LLM ~24 GB, embeddings ~670 MB. Install via Homebrew (macOS) or uv + Python 3.11 (Linux/Windows). Runs on macOS, Linux, Windows. No API keys needed for local servers.
// USE CASES
Best for: Backend developers building private AI products without cloud APIs
- Create private chatbots with RAG over internal documents
- Connect local LLMs to Claude Desktop and IDEs via MCP
- Build AI agents with web search and code execution without cloud
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram