TruLens

Started at TruEra and now maintained by Snowflake, TruLens measures how well an LLM app or agent is doing. It instruments Python code, records each step as an OpenTelemetry span, and scores those steps with feedback functions, mostly LLM-as-judge prompts that return a score plus their reasoning. LangChain, LangGraph and LlamaIndex apps get ready-made recorders; custom code is traced with a decorator that tags retrieval, generation, tool and MCP calls. Built-in metrics cover the RAG triad of context relevance, groundedness and answer relevance, plus agent judges for planning and tool use. Judges can run on OpenAI, Gemini, Bedrock, Snowflake Cortex, Hugging Face or any model reachable through LiteLLM. Results go to SQLite by default or any SQLAlchemy database, and a Streamlit dashboard compares app versions on quality, latency, tokens and cost; traces can also feed any OTLP backend. It installs with pip and needs no GPU. MIT licensed and free for commercial use, though judge API costs are yours. Fits teams iterating on RAG pipelines and agents who want regressions measured, not eyeballed.