OpenCompass

Backed by Shanghai AI Laboratory, OpenCompass evaluates large language models against more than 70 datasets totaling roughly 400,000 questions spanning knowledge, reasoning, language, and code. A single command splits work into subtasks and runs distributed evaluation, which is what makes billion-parameter sweeps practical, and experiments are managed through configuration files with real-time result reporting. Inference runs through HuggingFace transformers by default, with vLLM and LMDeploy as accelerated backends, and API adapters cover OpenAI, Claude, Gemini, and the major Chinese providers, so open weights and proprietary endpoints are scored under identical prompts in zero-shot, few-shot, and chain-of-thought settings. Installation is pip install opencompass, with GPU requirements set by the model being tested rather than by the framework itself. The code is Apache-2.0 licensed and free for commercial use. Its CompassRank leaderboards are among the most cited references for ranking Chinese and English models, and the modular design reduces adding a custom dataset to a small configuration exercise.