SCENE 09 / 24 · TRANSFORMATION, COMPILATION & QUANTIZATION
Model code to bare metal
TVM, OpenXLA, TensorRT and MAX lower high-level graphs to tuned kernels.
LOWERING · STAGE 1 / 4
High-level model code enters as a portable graph.
ARCHITECTURE & FORMULATIONModule 3: Model Transformation, Compilation & Quantization
Lowering graphs to bare metal: TVM, OpenXLA, TensorRT, Mojo/MAX
AI compilers translate a portable model graph into machine-specific code through tiered lowering: graph-level rewrites, then tile/schedule selection against the target's memory hierarchy, then kernel emission. The same graph becomes different SASS for Hopper than for Ada — autotuning against real hardware is the differentiator.
Tiered Lowering
Graph IR (framework-agnostic) → operator IR (TVM Relay, StableHLO) → tensor IR (TIR, LHLO) → target ISA. Each tier enables one class of optimization.
Autotuning
Schedule space (tiling, vectorization, pipelining) is searched against the real hardware with cost models or exhaustive measurement — thousands of candidates per operator.
Vendor Compilers
TensorRT ships hand-tuned kernels plus PTQ; Mojo/MAX aims at a unified high-performance substrate; OpenXLA trades per-op optimality for ecosystem reach.
t_op ≈ max(FLOPs / peak, Bytes / BW) + t_launch — search over tile configs minimizes the max()
The optimal tile shape depends on the ratio of compute to memory traffic for THIS operator on THIS silicon. Analytic models get within 2×; measurement finds the last 30-40%.
REAL-WORLD PRODUCTION ENGINEERING
TensorRT engine build takes minutes per shape — teams cache engines per (GPU, batch, seq-len) tuple and pin them in images.
TVM's Ansor and OpenXLA's autotuner made per-target optimal schedules portable; hand-written CUDA is now the fallback, not the default.