TextAttack

Framework for adversarial attacks, data augmentation, and model training in NLP.

Open SourceSelf HostedOffline Capable

0.0 (0)

About

Adversarial robustness testing for NLP is what TextAttack, from the QData lab at the University of Virginia, turns into routine engineering: it decomposes every published attack into four swappable parts, a goal, a set of linguistic constraints, a perturbation, and a search strategy, so recipes like TextFooler, BERT-Attack, and DeepWordBug become configurations rather than separate codebases. About twenty such recipes ship ready to run against HuggingFace transformers checkpoints or any wrapped model that maps text to predictions, and the same components double as a data augmenter with WordNet, embedding-swap, and backtranslation modes. Setup is a plain pip install exposing a CLI and Python API; everything works on CPU, though word-level searches against BERT-scale victims go far faster on a CUDA card. The code is MIT licensed. Expect heavy first-run downloads of embeddings and victim checkpoints into a local cache, and budget real time: query-hungry search strategies can take minutes per example, the main practical difference from lighter behavioral test suites such as CheckList.

Should you use TextAttack?

Pick it when

Pick TextAttack when you need to measure how a text classifier holds up against published adversarial attacks such as TextFooler or BERT-Attack, or generate augmented training data from the same parts, ideally with a CUDA GPU.

Look elsewhere when

Skip it if you are red-teaming an LLM app for prompt injection or jailbreaks, where garak or promptfoo fit better, or if you need quick behavioral checks, since search-based attacks can take minutes per example.

Alternatives to TextAttack

garak
Probes LLMs for prompt injection, jailbreaks and data leakage, the right tool for generative apps, while TextAttack targets classifiers.
Giskard
Wider quality and safety test suites for models and LLM apps, but not a catalog of reproducible published word-level attack recipes.
Promptfoo
CLI for evaluating and red-teaming LLM prompts and outputs, lighter to run in CI, but aimed at LLM behavior rather than classifier robustness.

Reviews (0)

Leave a Review

No reviews yet. Be the first to review!