Lingua

Short strings are where most language detectors fall apart, and Lingua was built specifically to fix that. The library identifies 75 languages and publishes per-language accuracy tables for single words, word pairs, and full sentences, where it leads langdetect, CLD3, and langid; for German it reports 74.2 percent on single words, 93.9 percent on word pairs, and 99.7 percent on sentences. Under the hood it combines n-gram models of lengths one through five with a rule-based filtering engine, no neural networks and no external API calls, so detection runs completely offline. Since version 2.0.0 the Python package is a set of bindings over a native Rust core, keeping runtime memory to a few dozen megabytes with model data stored in finite-state transducers, and a multi-threaded mode classifies 3,000 texts across all 75 languages in roughly 21 seconds in high accuracy mode. Install with pip install lingua-language-detector; version 2.2.0 is current, the license is Apache-2.0, and the repository has about 1,800 stars. Data engineers use it to route, filter, and label multilingual text before downstream NLP.