LangSmith

From the team behind LangChain, LangSmith is a platform for observing, evaluating, and debugging LLM applications in development and production. It captures detailed traces of every model call, tool invocation, and chain step, then layers on dashboards, alerting, and automation rules for tracking quality over time. Evaluation features cover offline test suites, online evaluators that score live traffic, human annotation queues, and end user feedback collection, and the LangSmith Engine can surface recurring issues in production traces and help diagnose root causes. Despite its origins, LangSmith is framework agnostic, with integrations for OpenAI, Anthropic, CrewAI, the Vercel AI SDK, and Pydantic AI in addition to LangChain itself. It is a commercial service offered as managed cloud, hybrid, or fully self hosted deployments. Engineering teams shipping agents and chat products use it to understand what their LLM systems actually did and why.