LocalGPT

Built for people who want answers from their own files without shipping them to a cloud API, LocalGPT indexes PDFs, Word files, HTML, Markdown, and plain text on your machine and answers questions with models served through Ollama. Docling parses each document, switching to OCR for scans, and the text is chunked, given short context notes by a small model, and stored in LanceDB. Queries run vector search and keyword search side by side, merge the two lists with reciprocal rank fusion, and rerank the candidates with a cross-encoder; a router first decides whether a question needs retrieval at all, and complex questions are broken into sub-queries. Replies cite the chunks behind them. You get a Next.js web interface, a REST gateway, and live progress events, runnable through Docker or directly with Python 3.10 or later and Node.js 20, with 16 GB of RAM recommended. Embedding and reranking use CUDA or Apple MPS when available and fall back to CPU. The current code base is plain Python with no LangChain dependency, an IBM watsonx backend is optional, and the project is MIT licensed for commercial use.