MLC LLM

MLC LLM takes a compiler approach to LLM deployment: rather than hand-writing kernels for each platform, it uses machine learning compilation techniques from the TVM ecosystem, including TensorIR and MetaSchedule, to generate optimized code for whatever hardware is available. The result, MLCEngine, is a unified inference engine that exposes an OpenAI-compatible API through a REST server plus Python, JavaScript, iOS, and Android bindings. Platform coverage is unusually wide: NVIDIA GPUs via CUDA or Vulkan, AMD via ROCm or Vulkan, Apple silicon via Metal, Intel GPUs, Android through OpenCL, iOS and iPadOS through Metal, and browsers through WebGPU and WebAssembly, so the same model can run in a native phone app or a web page with no server at all. The project is open source under Apache 2.0 and grew out of academic work on the TVM compiler. Developers reach for it when they need on-device LLM inference across heterogeneous hardware rather than a single server stack.