Aphrodite Engine

PygmalionAI began Aphrodite as a vLLM fork to serve its own chat models; it now continues as Sonar under the dphnAI organization and runs in production for Pygmalion and Dolphin's inference network. The old repository URL redirects there, and the command is still aphrodite serve. Built on vLLM, it offers continuous batching, a paged KV cache with prefix caching, multi-GPU and multi-node parallelism that does not need Ray, and support for mixture-of-experts layouts, plus extra quantization formats, more samplers, per-request phrase suppression and further speculative decoding methods. Its HTTP server speaks OpenAI, Anthropic and KoboldAI APIs, so chat front ends connect directly, and it also covers embeddings, reranking, transcription, tool calls, multimodal models, LoRA adapters and Prometheus metrics. A one-line installer handles Linux on NVIDIA, AMD or CPU plus Apple silicon Macs, with Docker, TPU and Intel XPU setups in the docs. Six tagged releases shipped from April to September 2026. The AGPL-3.0 license permits commercial hosting but requires sharing modified source with network users, which matters for SaaS operators.