Haystack

Haystack, from deepset, gives Python developers a component model for LLM applications: retrievers, embedders, rankers, document stores, prompt builders, and chat generators are typed blocks you connect into pipelines that can branch, loop, run asynchronously, and stream tokens. The same pieces assemble RAG systems, semantic search, and question answering, while the Agent component executes tools with hooks around each model call and tool call for guardrails, human approval, or spending caps. The 3.0 release in July 2026 folded the separate async pipeline into the main class, made skills loadable on demand, and put step and token counters into agent state. Integrations span OpenAI, Anthropic, Mistral, Cohere, Google, Azure, Bedrock, Hugging Face, and local models, with vector store connectors kept in a companion core-integrations repo, and Hayhooks turns a pipeline into a REST or MCP endpoint. Install haystack-ai with pip on Python 3.10+; anonymous telemetry is on by default and can be switched off. Apache-2.0, with a paid enterprise platform sold separately.