Learning map

The canonical course map connects paper sections, concepts, animation beats, and evidence.

Model Architecture — One Token, Three Routes

Kimi K3 scales information flow independently across sequence length, network depth, and model width.

Chapter	Paper	Concepts	Beats
One token meets three bottlenecks	2	kda, attnres, moe-routing	1
Vision joins text as one token stream	2.4	vision-transformer	1
Build a 3:1 attention rhythm	2.1	kda, gated-mla	1
KDA stores a correctable memory, not a list	2.1.1	fast-weights, kda, kv-cache	1
Persist the small representation	2.1.2	low-rank-compression, deepseek-mla, nope, gated-mla	1
Residuals become attention over depth	2.2	residual-connections, rmsnorm, attnres	1
Route sparsely without losing stability	2.3, 2.3.1, 2.3.2, 2.3.3	moe-routing, rmsnorm, swiglu, situ-glu, quantile-balancing	1
Respect the head boundary	2.5	newton-schulz, per-head-muon	1
Three routes converge on one next token	2	kda, gated-mla, attnres, moe-routing, per-head-muon	1

Courses 5A–5B — Evaluation, Case Studies, and Conclusions

Evaluation is not a scoreboard; results and case studies must resolve into explicitly bounded conclusions.

Chapter	Paper	Concepts	Beats
Construct the measurement before reading the score	6, 6.1.1, 6.1.2, 6.1.3	benchmarking, baselines, evaluation-protocols, comparative-evaluation, sampling-parameters, agent-harnesses, reproducibility, provenance	5
Read the public results as a capability fingerprint	6.1.4	benchmark-interpretation, capability-evaluation, comparative-evaluation, evaluation-limitations	5
Use internal evaluations to find failures, not to manufacture certainty	6.2, 6.2.1, 6.2.2	internal-evaluation, benchmark-design, cybersecurity-evaluation, evaluation-limitations	4
Triangulate independent evidence and put capability beside cost	6.3, 6.4	third-party-evaluation, leaderboards, provenance, cost-efficiency, inference-cost	4
Read complete trajectories, not capability anecdotes	7	case-study-method, provenance, evaluation-limitations, agentic-environments	7
End with the strongest supportable claim	8	model-scaling, comparative-evaluation, provenance, evaluation-limitations	1

Infrastructure for Frontier-Scale Intelligence

Kimi K3's infrastructure is a continuation of its algorithms; recurrent state, sparse experts, multimodal variability, and persistent agent worlds each determine the systems design.

Chapter	Paper	Concepts	Beats
The machine is part of the model	5	kda, moe-routing, long-context-training, gpu-kernels	1
Read the parallelism coordinate system	5.2	data-parallelism, zero-one, pipeline-parallelism, virtual-pipeline-stages, one-f-one-b, expert-parallelism, context-parallelism, pipeline-zero-two	2
Make recurrence parallel without changing it	5.1, 5.1.1, 5.1.2	kda, gpu-kernels, fast-weights	2
Balance work, memory, and the critical path	5.2, 5.2.1, 5.2.2, 5.2.3	moe-routing, checkpointing, vision-transformer, gpu-kernels, zero-copy-communication, unified-activation-manager, gradient-buffer-reuse, p2p-muon, decoupled-encoder-process	6
Keep trajectories alive while compute moves elsewhere	5.3, 5.3.1, 5.3.2	long-context-training, kv-cache, persistent-state, agentic-environments, external-state-pool, adaptive-throttling, gradient-buffer-reuse, overlaybd	4
Serve hybrid state as one consistent object	5.4, 5.4.1, 5.4.2, 5.4.3	kda, kv-cache, gpu-kernels, speculative-decoding, hybrid-prefix-cache, deepseek-sparse-attention, minitriton, cublas	5
Architecture becomes an operating system	5	kda, moe-routing, kv-cache, persistent-state, gpu-kernels	1

Post-Training — From Cold Start to Verifiable Agents

Post-training turns broad pretrained capability into controlled reasoning effort, verified action, consolidated specialization, and deployment-aware behavior.

Chapter	Paper	Concepts	Beats
Three transformations, one policy	4, 4.1	cold-start-policy, supervised-fine-tuning, reinforcement-learning, mopd	2
Establish a verified cold-start policy	4.1.1	instruction-data, supervised-fine-tuning, cold-start-policy, quantization-aware-training, mxfp4, mxfp8	2
Specialize across domain and reasoning effort	4.1.2	reinforcement-learning, reasoning-effort, on-policy-learning, partial-rollout, long-horizon-agents, agentic-grm, verifiable-rewards	4
Consolidate nine experts on the student’s own prefixes	4.1.3	mopd, on-policy-distillation, policy-distillation, on-policy-learning	2
Train for the arithmetic and decoding path that will serve	4.1.4	quantization-aware-training, mxfp4, mxfp8, deployment-optimization, multi-token-prediction, speculative-decoding, eagle-3	3
Build diverse tasks with trustworthy outcomes	4.2, 4.2.1, 4.2.2, 4.2.3	white-box-environments, agentic-environments, knowledge-graphs, task-synthesis, verifiable-rewards	3
Four environment families, four notions of success	4.2.4, 4.2.5, 4.2.6, 4.2.7	gpu-kernels, profiling, persistent-state, autonomous-agents, web-development	4
Capability becomes policy through verified feedback	4	cold-start-policy, reinforcement-learning, mopd, deployment-optimization, verifiable-rewards	1

Pretraining the Foundation — From Corpus to One Million Tokens

Architecture supplies possible computation; pretraining acquires behavior through data, compute allocation, optimization controls, and progressively longer dependencies.

Chapter	Paper	Concepts	Beats
Architecture is possibility; training is acquired behavior	3	language-model-pretraining	1
Tokens are designed before they are counted	3.1	data-curation, data-filtering, deduplication, data-mixtures, multimodal-data	7
Compute draws a constraint curve	3.2	scaling-laws, compute-allocation, model-data-tradeoffs	5
The training recipe is a synchronized control system	3.3, 2.5	training-recipe, optimization-schedules, per-head-muon, quantile-balancing	3
A longer context window must be taught	3.4	long-context-training, positional-encoding, context-window, attention-complexity	3
One model, four coupled decisions	3	data-curation, scaling-laws, training-recipe, long-context-training	1

Course 0 — The K3 Thesis and Learning Route

K3 argues that frontier intelligence requires scaling both the pretrained model and the amount of useful computation a trained system can spend at test time.

Chapter	Paper	Concepts	Beats
Separate pretraining scale from test-time scale	1	llm-scaling, test-time-scaling, model-scaling	1
Read scale numbers without conflating them	1	model-scaling, mixture-of-experts, multimodal-tokenization	1
Map every contribution to its enabling layer	1	transformer-basics, distributed-systems, comparative-evaluation	3
Learn what each surface can establish	1	provenance, evaluation-limitations	2