Seldon Core

Seldon Core 2 runs on Kubernetes and turns trained models into inference endpoints described by custom resources for Models, Servers, Pipelines, and Experiments. Rather than one container per model, it packs many models onto shared MLServer or Triton servers, can overcommit memory to register more models than fit at once, and autoscales both models and servers. Pipelines link models and custom steps over Kafka topics, for multi-stage flows such as preprocessing, drift checks, and LLM calls, and Experiments split or mirror traffic to compare or shadow candidate models. You install it with Helm on an existing cluster and supply Kafka for pipelines, plus Prometheus and Grafana for metrics. The catch is licensing: since January 2024 version 2 is licensed with the Business Source License 1.1, which permits free non-production use such as development and testing but requires a paid license from Seldon for production, except at non-profit educational institutions; each release becomes Apache 2.0 four years after it ships. Public commits paused after January 2026. Fits Kubernetes teams able to pay for a license.