LiteLLM

Calling Anthropic, Bedrock, Vertex AI, Azure, Groq or a local vLLM server usually means juggling different SDKs and payload shapes; LiteLLM translates one OpenAI-style request into each provider's native format and maps responses and errors back, for chat plus embeddings, image, speech, rerank and batch routes. It comes in two forms. The Python library exposes completion() and related calls, a router with retries and fallbacks, and cost calculation from a maintained price table. The proxy server, started with the litellm CLI or the official Docker image, is a central gateway with virtual API keys, per-key and per-user spend tracking in Postgres, Redis for caching, load balancing, guardrails, logging callbacks to tools like Langfuse, an admin UI, and gateways for MCP servers and A2A agents. Terraform modules and a Helm chart cover production, and images are cosign-signed. Everything outside the enterprise folder is MIT; single sign-on past five users, key rotation, IP allowlists and some governance features need a commercial license from BerriAI. Platform teams centralizing LLM access and costs are the main audience.