What is inference?
A forward pass that turns tokens into the next token.
THE ATLAS · EIGHT DOMAINS
The AI Inference Architecture Handbook, mapped.
The forward pass, request lifecycle, TTFT/TPOT/goodput.
Training vs serving
Different workloads with opposite bottlenecks.
The request lifecycle
Network ingress to detokenized egress, end to end.
TTFT vs TPOT
Responsiveness and steady-state speed, measured separately.
Throughput · goodput · E2EL
Steady-state speed versus requests that satisfy their SLO.