Ollama

Ollama makes running large language models on your own machine straightforward. It bundles model weights, configuration, and prompt templates into a single package, so pulling and chatting with a model such as Llama, Gemma, Mistral, or Qwen takes one command, and it exposes a local REST API on port 11434 for building applications. Official Python and JavaScript libraries sit alongside more than a hundred community integrations spanning chat UIs, code editors, RAG systems, and agent frameworks, and coding assistants can use it as a local backend. It runs on macOS, Windows, and Linux, with an official Docker image, and handles GPU acceleration and quantized model formats behind the scenes. The project is open source under the MIT license and has become a default choice for developers who want private, offline inference or a quick way to prototype against open-weight models before committing to cloud APIs.