AI ENGVisual Encyclopedia

SCENE 09 / 24 · TRANSFORMATION, COMPILATION & QUANTIZATION

Model code to bare metal

TVM, OpenXLA, TensorRT and MAX lower high-level graphs to tuned kernels.

LOWERING · STAGE 1 / 4
MODEL GRAPHPyTorch / JAX / ONNXGRAPH PASSESfusion · DCE · layoutTILE & SCHEDULEautotuned per targetBARE METALPTX / SASS / bytecodessame graph → different SASS per silicon generation; autotuning searches the schedule space

High-level model code enters as a portable graph.

TECHNICAL BREAKDOWNModule 3: Model Transformation, Compilation & Quantization

Lowering graphs to bare metal: TVM, OpenXLA, TensorRT, Mojo/MAX

AI compilers translate a portable model graph into machine-specific code through tiered lowering: graph-level rewrites, then tile/schedule selection against the target's memory hierarchy, then kernel emission. The same graph becomes different SASS for Hopper than for Ada — autotuning against real hardware is the differentiator.

Tiered Lowering

Graph IR (framework-agnostic) → operator IR (TVM Relay, StableHLO) → tensor IR (TIR, LHLO) → target ISA. Each tier enables one class of optimization.

Autotuning

Schedule space (tiling, vectorization, pipelining) is searched against the real hardware with cost models or exhaustive measurement — thousands of candidates per operator.

Vendor Compilers

TensorRT ships hand-tuned kernels plus PTQ; Mojo/MAX aims at a unified high-performance substrate; OpenXLA trades per-op optimality for ecosystem reach.

MATHEMATICAL FORMULATION · WHY AUTOTUNING BEATS HEURISTICS
t_op ≈ max(FLOPs / peak, Bytes / BW) + t_launch — search over tile configs minimizes the max()

The optimal tile shape depends on the ratio of compute to memory traffic for THIS operator on THIS silicon. Analytic models get within 2×; measurement finds the last 30-40%.

REAL-WORLD PRODUCTION ENGINEERING
  • TensorRT engine build takes minutes per shape — teams cache engines per (GPU, batch, seq-len) tuple and pin them in images.
  • TVM's Ansor and OpenXLA's autotuner made per-target optimal schedules portable; hand-written CUDA is now the fallback, not the default.