Bifrost

Maxim AI wrote Bifrost in Go to be the fastest self-hosted LLM gateway available, claiming under 100 microseconds of added overhead at 5,000 requests per second, with its own benchmarks putting it far ahead of LiteLLM. One OpenAI-compatible API fronts more than 1,000 models across 20 plus providers including OpenAI, Anthropic, AWS Bedrock, Google Vertex, and Azure, while adaptive load balancing, automatic failover, semantic caching, virtual keys with budget controls, and integrated guardrails handle the production concerns. A cluster mode distributes the gateway itself for high availability, MCP support lets routed agents execute tools, and observability hooks cover logging and tracing. It deploys as a Docker container or Go binary with a web UI, and a native Go SDK embeds it directly in services; enterprise features such as RBAC, audit logs, and compliance tooling form the commercial tier above the Apache-2.0 core. Launched in 2025, the repository has grown to roughly 6,900 GitHub stars among teams consolidating LLM traffic.