OpenLIT

OpenLIT bundles most of the operational tooling an LLM application needs into one self-hosted, OpenTelemetry-native platform: tracing and metrics, GPU monitoring, real-time guardrails, eleven built-in evaluation types including hallucination and toxicity detection, a versioned Prompt Hub, a Vault for API keys, and an OpenGround playground for side-by-side model comparison. SDKs for Python, TypeScript, and Go auto-instrument more than 50 LLM providers, agent frameworks, and vector databases, emitting standard gen_ai semantic conventions so data can flow to OpenLIT's own ClickHouse-backed UI or to any existing OpenTelemetry collector and backend. Deployment is a Docker Compose file for quick starts or a Helm chart for Kubernetes, and everything runs on your own infrastructure with no external service required. The whole stack is Apache-2.0 licensed with no paid tier. Teams pick it over point solutions when they want observability, evaluation, and prompt management consolidated behind one open deployment instead of several hosted dashboards.