Guidance

Guidance approaches LLM control as a programming problem rather than prompt writing. The Python framework constrains generation with regex patterns, context-free grammars, and JSON schemas, including Pydantic models, and lets developers interleave conditionals, loops, tool calls, and generation inside composable decorated functions. Its gen() and select() primitives capture output into named variables or force a choice among fixed options, so responses always parse. A notable optimization is token fast-forwarding: any tokens the grammar makes inevitable are filled in without model forward passes, cutting latency and GPU cost during structured generation. Backends include Hugging Face Transformers, llama.cpp, and OpenAI-compatible APIs, and a Mock model allows offline grammar debugging. Installable from PyPI and released under the MIT license, Guidance is used by engineers who need guaranteed structured output from both local and hosted models.