Lemonade

One server, whatever silicon you own: Lemonade inspects your machine and routes each model to the fastest available engine, llama.cpp through Vulkan or ROCm on GPUs, CUDA on NVIDIA cards, Metal on Apple Silicon, and dedicated paths for AMD Ryzen AI NPUs and the Strix Halo iGPU. Beyond chat models it serves whisper.cpp and Moonshine for speech-to-text, Kokoro for text-to-speech, and Stable Diffusion for image generation, all behind an OpenAI-compatible API at localhost:13305 that works with any OpenAI-compatible client, with examples in Python, Node.js, Java, Go, Rust, and more. Installation meets users where they are: a Windows MSI installer, native packages for Arch, Debian, Ubuntu, and Fedora, macOS support, Docker images, or source builds. The project is Apache 2.0, an AMD-sponsored community effort with many maintainers, and its optimizations for Ryzen AI, Radeon, and Strix Halo systems make it the most complete local-inference story on AMD hardware, though it runs fine on NVIDIA and Apple machines too. Everything executes locally, free and offline.