MTEB

Choosing an embedding model usually starts at MTEB, the Massive Text Embedding Benchmark, whose Hugging Face leaderboard has become the industry-standard scoreboard for embedding and reranking models. The open-source toolbox evaluates models on task types including retrieval, classification, clustering, semantic textual similarity, and reranking, and the 2025 MMTEB expansion, a large community effort, pushed coverage past a thousand tasks spanning more than 1,000 languages plus image and audio modalities. Running an evaluation takes a few lines of Python after pip install mteb: point it at any Sentence Transformers compatible model or implement a small encoder interface, and results can be submitted to the public leaderboard. Compute needs depend on the model being tested, since the harness itself is lightweight; datasets are pulled from the Hugging Face Hub and cached locally. The code is Apache-2.0 with about 3,400 GitHub stars and over 4,300 commits, maintained by the embeddings-benchmark community, and the 2022 MTEB and 2025 MMTEB papers document the methodology.