ColPali

Skipping OCR entirely, ColPali treats a document page as an image and retrieves it through vision-language late interaction: a VLM embeds page screenshots into multi-vector representations matched against query tokens ColBERT-style. The colpali-engine package from ILLUIN Technology provides training and inference code for the ColVision family, including the original ColPali built on PaliGemma-3B, ColQwen2 and ColQwen2.5 built on Qwen2-VL and Qwen2.5-VL, and compact ColSmol variants on SmolVLM; the best in-house checkpoints score about 89.4 on ViDoRe. The ICLR 2025 paper, arXiv 2407.01449, introduced both the method and the ViDoRe benchmark, and the approach set off a wave of visual document retrieval work because it preserves tables, figures, and layout that text extraction pipelines lose. Installation is pip install colpali-engine, and a GPU is effectively required to embed pages at usable speed. The repository code is MIT licensed; ColQwen and ColSmol weights are Apache-2.0 while PaliGemma-based checkpoints inherit the Gemma license. The project holds 2.7k GitHub stars plus Hugging Face demo spaces and interpretability tooling.