Inspect AI

Inspect is an open source framework for large language model evaluations created by the UK AI Security Institute. Evaluations are composed from datasets, solvers, and scorers in Python, with built-in components for prompt engineering, multi-turn dialog, tool usage, and model-graded scoring, so tasks range from simple question-answering benchmarks to agentic evaluations in which models operate tools. A companion collection provides more than 200 prebuilt evaluations that run against supported model providers without modification, and the framework is extensible through ordinary Python packages that add new elicitation and scoring techniques. Tooling includes a log viewer for examining transcripts of evaluation runs. Released under the MIT license with documentation at inspect.aisi.org.uk, Inspect is used by safety researchers, government evaluators, and engineering teams that need reproducible, auditable measurements of model capability and behavior.