Initialization & Nudging
How billions of parameters shift from Gaussian noise into structured representations.
REFERENCE SEVEN DOMAINS
Every training concept, on one map.
Browse by domain. Scenes open the animated exhibits; entries are the distilled reference points.
Predictive self-supervision, Causal vs Masked LM, context windows, and the training step loop.
Causal vs. Masked LM
Autoregressive generation vs bidirectional fill-in-the-blanks.
The Step Loop
Forward logits -> Backward autodiff -> Optimizer step.
Causal Attention Masking
Zeroing upper-triangular scores to enforce unidirectional temporal flow.
Context Window Math
Understanding O(N^2) memory scaling and KV cache growth.