AI ENGVisual Encyclopedia

SCENE 10 / 24 · THE RUNTIME & MEMORY MANAGEMENT LAYER

Half a megabyte per token

Attention remembers its past in the KV cache — memory becomes the budget.

SEQUENCE LENGTH2,048 TOKENS
BATCH 11.00 GB
BATCH 88.00 GB
BATCH 3232.00 GB

Formula: 2 × 32 layers × 32 kvHeads × 128 headDim × 2 bytes = 512.0 KB per token. A 32-way batch at full context carries a small model's worth of weights in cache alone.

TECHNICAL BREAKDOWNModule 4: The AI Runtime & Memory Management Layer

KV cache anatomy: the second model living in your VRAM

Attention must remember what it has already seen: Keys and Values for every layer, every head, every past token. The KV cache grows linearly with context and batch — and at long contexts it rivals the weights themselves. The entire 'KV economy' of serving is managing this second, growing allocation.

The Formula

bytes/token = 2 (K and V) × layers × kv_heads × head_dim × dtype_bytes. A 7B model at FP16 stores 512 KB per token — a 32k context is 16 GB per sequence.

GQA as Compression

Grouped-Query Attention shares K/V heads across query heads (Llama-2-70B: 8 KV heads for 64 queries), cutting the cache 8× with negligible quality cost.

The Budget Collision

Weights (fixed) + KV (growing) + activations must fit in HBM. Serving capacity is decided by how many concurrent KV caches fit — the true definition of 'batch slot'.

MATHEMATICAL FORMULATION · CACHE FOOTPRINT
KV_bytes = 2 × L × H_kv × d_head × dtype_bytes × N_ctx × B

7B (L=32, H_kv=32, d=128, fp16): 512 KB/token. At 32k context × 8 concurrent requests: 128 GB — twice the weights. This is why 70B models page, quantize, and evict caches.

REAL-WORLD PRODUCTION ENGINEERING
  • vLLM logs 'KV cache utilization' as its primary health metric; above ~90% preemption (recompute) begins.
  • MLA (DeepSeek) and cross-layer KV sharing push per-token cost down another order of magnitude.