LocalAI

Point a client written for the OpenAI API at a LocalAI server and it keeps working after a base URL change; the same server also answers Anthropic, Ollama, and ElevenLabs style requests. Behind that endpoint is a small Go core that fetches inference backends on demand as separate container images, wrapping engines like vLLM, llama.cpp, whisper.cpp, MLX, and diffusers, along with several C++ engines the team writes itself for speech, vision, and 3D work. Together they cover chat with tool calling, embeddings and reranking, transcription, text to speech with voice cloning, image and video generation, and object detection. Recent releases added a React web UI, API keys with per-user quotas, built-in agents that speak MCP, and a distributed mode that spreads models across several machines. Start with the CPU-only Docker image, pick a CUDA, ROCm, Intel, or Vulkan image for acceleration, or install the macOS app. LocalAI itself is MIT licensed with no commercial restrictions, while every model you download keeps its own license. It fits teams that want a private, self-hosted stand-in for hosted AI APIs on hardware they already own.