TextAttack

Adversarial robustness testing for NLP is what TextAttack, from the QData lab at the University of Virginia, turns into routine engineering: it decomposes every published attack into four swappable parts, a goal, a set of linguistic constraints, a perturbation, and a search strategy, so recipes like TextFooler, BERT-Attack, and DeepWordBug become configurations rather than separate codebases. About twenty such recipes ship ready to run against HuggingFace transformers checkpoints or any wrapped model that maps text to predictions, and the same components double as a data augmenter with WordNet, embedding-swap, and backtranslation modes. Setup is a plain pip install exposing a CLI and Python API; everything works on CPU, though word-level searches against BERT-scale victims go far faster on a CUDA card. The code is MIT licensed. Expect heavy first-run downloads of embeddings and victim checkpoints into a local cache, and budget real time: query-hungry search strategies can take minutes per example, the main practical difference from lighter behavioral test suites such as CheckList.