garak

Modeled in spirit on nmap, garak scans language model deployments for the ways they fail: prompt injection, jailbreaks, training data leakage, toxicity, package hallucination, and dozens of other weakness classes. Probes generate adversarial interactions, detectors judge the responses, and generators adapt the harness to targets ranging from OpenAI and Hugging Face APIs to locally hosted weights, so the same scan runs against a hosted endpoint or an offline local model. Each run produces a detailed report of which probes succeeded, giving security teams a repeatable baseline rather than anecdotes. NVIDIA maintains the project, which began as an independent research tool by Leon Derczynski and now sits at roughly 8,600 GitHub stars as the standard open source LLM vulnerability scanner. It installs with pip install garak, runs from the command line on Python, needs no GPU unless the target model runs locally, and is licensed Apache-2.0, making it an easy addition to red-team pipelines and pre-deployment checks.