Cog

Cog turns a model directory into a Docker image that serves predictions over HTTP. You describe the environment in cog.yaml, listing the Python version, pip requirements, system packages, and whether a GPU is needed, then write a small Python class with a setup method that loads weights once and a run method whose typed arguments define the inputs. Those type hints drive an auto-generated OpenAPI schema, request validation, and an inference server written in Rust on Axum, while Cog picks matching CUDA, cuDNN, and framework versions to avoid driver mismatches. The commands cover the loop: cog run executes a single prediction locally, cog serve starts the API, and cog build produces an image you can push to any registry or container host. An optional training interface exposes fine-tuning the same way. The binary installs via Homebrew on macOS or an install script on macOS and Linux, Windows needs WSL 2, and Docker with Buildx is required. It is Apache 2.0 licensed and maintained by Replicate, yet the images run on any Docker host. Ideal for anyone wanting reproducible model containers without hand-written Dockerfiles.