R_REDDYX.XYZ
DevToolsPython★ 57,640⑂ 7,627 forksApache-2.0

zylon-ai/private-gpt

API LAYER FOR LOCAL AI APPLICATIONS

PrivateGPT is an open API layer for building private AI applications on local models. It connects to any OpenAI-compatible inference server and provides RAG, tools, MCP, and structured data access.

// KEY FEATURES

  • RAG with citations and agentic search
  • Built-in tools: web search, code, page fetch
  • MCP connectors and custom tools support
  • Structured API access to databases and CSV

// DEEP DIVE: HOW IT WORKS

PrivateGPT does not run models itself but works as an API proxy: the application calls the Anthropic API, and PrivateGPT forwards requests to an OpenAI-compatible server (Ollama, llama.cpp, vLLM) via OPENAI_API_BASE. A web UI at /ui is included for demos, but the core product is the API.

Unlike frameworks, it provides ready-made primitives: streaming messages API, file ingestion, RAG with citations and agentic mode, built-in tools (web search, fetch, code) mirroring the Claude API, MCP connectors, custom tools, and structured access to databases and CSVs. Battle-tested in production by the Zylon platform.

Requires an LLM server (Ollama) and models: LLM ~24 GB, embeddings ~670 MB. Install via Homebrew (macOS) or uv + Python 3.11 (Linux/Windows). Runs on macOS, Linux, Windows. No API keys needed for local servers.

// USE CASES

Best for: Backend developers building private AI products without cloud APIs

  • Create private chatbots with RAG over internal documents
  • Connect local LLMs to Claude Desktop and IDEs via MCP
  • Build AI agents with web search and code execution without cloud
#ai#ai-tools#on-premise
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

FAST AI-POWERED SEARCH ENGINE★ 59884SELF-HOSTED GIT SERVICE IN GO★ 47943DeepSpeed: optimization library for training★ 43332FRAMEWORK FOR CHEAPER LARGE MODEL TRAINING★ 41422OPEN-SOURCE DESIGN TOOL IN CODE★ 27239HIGH-PERFORMANCE NEURAL NETWORK INFERENCE FRAMEWORK FOR MOBILE★ 24068

← FULL CATALOG

zylon-ai/private-gpt — API Layer For Local AI Applications