Practice with the notebooks
Notebooks are inspectable evidence, not hidden downloads. View opens GitHub’s rendered notebook. Run in Colab creates an editable hosted session. Download is explicit. Run locally uses the repository environment described below.
| Notebook | Learning purpose | Actions |
|---|---|---|
| 01 — KDA and Gated MLA | Recurrence, exact chunk handoff, cadence, and latent attention flow. | View · Colab · Download |
| 02 — Attention Residuals | Depth-softmax weights and addressable earlier representations. | View · Colab · Download |
| 03 — Stable LatentMoE | Bounded SiTU-GLU behavior and the local routing analogy. | View · Colab · Download |
| 04 — Native vision | Patchification and shape flow, not MoonViT-V2 reproduction. | View · Colab · Download |
| 05 — Per-Head Muon | Singular-value motion, shape-aware semi-orthogonality, and head-local projection. | View · Colab · Download |
| 06 — Post-training | Partial rollout, effort penalties, MOPD token rewards, and draft acceptance. | View · Colab · Download |
| 07 — Evaluation analysis | Gap arithmetic, paired tool lifts, and Pareto-frontier calculations over transcribed paper values. | View · Colab · Download |
| 08 — Pretraining foundation | Curation decisions, mixture sampling, scaling arithmetic, schedules, and context stages. | View · Colab · Download |
| 09 — Infrastructure mechanics | Affine KDA composition, placement, lifetimes, cache invariants, and scheduling miniatures. | View · Colab · Download |
Run locally
git clone https://github.com/mailtotanvir/build-Kimi-K3-architecture.git
cd build-Kimi-K3-architecture
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt jupyter
jupyter lab notebooks/
The retained outputs are reviewable without execution. Running locally is useful when changing fixtures, inspecting intermediate tensors, or testing an alternative explanation.