Lowering graphs to bare metal: TVM, OpenXLA, TensorRT, Mojo/MAX
AI compilers translate a portable model graph into machine-specific code through tiered lowering: graph-level rewrites, then tile/schedule selection against the target's memory hierarchy, then kernel emission. The same graph becomes different SASS for Hopper than for Ada — autotuning against real hardware is the differentiator.
Tiered Lowering
Graph IR (framework-agnostic) → operator IR (TVM Relay, StableHLO) → tensor IR (TIR, LHLO) → target ISA. Each tier enables one class of optimization.
Autotuning
Schedule space (tiling, vectorization, pipelining) is searched against the real hardware with cost models or exhaustive measurement — thousands of candidates per operator.
Vendor Compilers
TensorRT ships hand-tuned kernels plus PTQ; Mojo/MAX aims at a unified high-performance substrate; OpenXLA trades per-op optimality for ecosystem reach.
The optimal tile shape depends on the ratio of compute to memory traffic for THIS operator on THIS silicon. Analytic models get within 2×; measurement finds the last 30-40%.
- TensorRT engine build takes minutes per shape — teams cache engines per (GPU, batch, seq-len) tuple and pin them in images.
- TVM's Ansor and OpenXLA's autotuner made per-target optimal schedules portable; hand-written CUDA is now the fallback, not the default.