PrivateGPT

Zylon's PrivateGPT began in 2023 as a viral chat-with-your-documents script and was rebuilt for version 1.0 as an application server in front of a local model. It runs no models itself: you point it at any server that speaks the OpenAI API, such as Ollama, llama.cpp or vLLM, and it exposes an Anthropic-style messages API with streaming and token counting, document ingestion with cited retrieval, agentic RAG, built-in web search, page fetching and code execution tools, custom tools, MCP connectors, SQL database and CSV querying, and embeddings. A bundled /ui workbench serves testing and demos. Integration guides cover Claude Code, Claude Desktop and OpenCode, and n8n or Cline also connect. Installation goes through Homebrew on macOS or uv with Python 3.11 on Linux and Windows; hardware needs come from whichever model server you pair it with. Apache-2.0 licensed, so commercial use and modification are allowed, and Zylon separately sells an on-premise enterprise platform built on top. Developers building private assistants or internal document tools on their own hardware get backend plumbing they would otherwise write by hand.