OpenLLMetry
OpenTelemetry instrumentation that traces LLM, vector database, and framework calls to any backend.
About
Rather than inventing a new telemetry format, OpenLLMetry extends OpenTelemetry with instrumentation for the LLM stack, and its gen_ai semantic conventions were later adopted into OpenTelemetry itself. The Python SDK installs with pip install traceloop-sdk and auto-instruments calls to OpenAI, Anthropic, Bedrock, Gemini, Mistral, Ollama, and a dozen other providers, vector databases including Pinecone, Qdrant, Chroma, Milvus, and Weaviate, and frameworks such as LangChain, LlamaIndex, LangGraph, CrewAI, and LiteLLM, with MCP support and a separate OpenLLMetry-JS package covering TypeScript. Because the output is plain OTel spans, traces export to more than 20 destinations including Datadog, Honeycomb, Grafana, New Relic, Splunk, and SigNoz, or to Traceloop's own hosted platform, so there is no vendor lock-in on the storage side. Everything is Apache-2.0 licensed and maintained by Traceloop. It suits teams that already run OpenTelemetry and want LLM calls, token counts, and prompts to appear in the same tracing backend as the rest of their services.
Should you use OpenLLMetry?
Pick it when
Pick OpenLLMetry when your company already runs OpenTelemetry with Datadog, Honeycomb, Grafana, or SigNoz and you want LLM calls, vector database queries, and token counts in the same traces as every other service.
Look elsewhere when
Skip it if you have no tracing backend and want an LLM-specific UI with prompts, evals, and datasets out of the box; Langfuse, Phoenix, or OpenLIT give you that without assembling the pieces.
Alternatives to OpenLLMetry
- OpenLIT
Same OTel gen_ai conventions plus its own self-hosted UI, GPU monitoring, and guardrails; more infrastructure to run.
- Pydantic Logfire
OTel-based SDK with a hosted UI that renders agent runs as chat; the server is closed source with self-hosting only on enterprise terms.
- Langfuse
Full LLM platform with prompt management and evals; a separate store from your main APM rather than spans in your existing backend.
- Helicone
Captures requests through a proxy with no code changes and adds caching; the proxy sits in your request path and data lands in Helicone, not your APM.
Reviews (0)
Leave a Review
No reviews yet. Be the first to review!
Details
- Category
- AI Observability & Evaluation
- Price
- Freemium
- Platform
- Local/Desktop
- Difficulty
- Easy (2/5)
- License
- Apache-2.0
- Added
- Jul 29, 2026