Flyte

Created at Lyft to run its ML platform and since graduated as an LF AI and Data Foundation project, Flyte orchestrates data and machine learning workflows on Kubernetes with an emphasis on reproducibility and type safety. Workflows are written in pure Python with the flytekit SDK, strongly typed at task boundaries, versioned immutably, and executed as containers, so a pipeline that ran last month can be rerun exactly; caching, retries, recoverability, and dynamic branching are built into the Go-based backend rather than left to user code. Flyte 2.x, current as of 2026, reworked authoring around plain Python for AI and agentic workloads while keeping the Kubernetes-native execution model. Deployment targets any Kubernetes cluster via Helm, with a web console, scheduling, and multi-tenancy included, and Union.ai offers commercial hosting and now publishes the documentation. The Apache-2.0 project counts about 7,200 GitHub stars and production users including Spotify and Gojek, making it one of the proven open orchestrators for ML pipelines at scale.