# Multi-Token llama.cpp Equivalence

**Verdict:** FAIL  
**Implementation:** `real-isolated`  
**Cases:** 80

| Batch | Tokens | Layer | Expert IDs exact | Weight max abs | Output max abs |
|---|---:|---:|---:|---:|---:|
| n2 | 2 | 0 | true | 0.0 | 0.0 |
| n2 | 2 | 1 | true | 0.0 | 0.0 |
| n2 | 2 | 2 | true | 0.0 | 0.0 |
| n2 | 2 | 3 | true | 0.0 | 0.0 |
| n2 | 2 | 4 | true | 0.0 | 0.0 |
| n2 | 2 | 5 | true | 0.0 | 0.0 |
| n2 | 2 | 6 | true | 0.0 | 0.0 |
| n2 | 2 | 7 | true | 0.0 | 0.0 |
| n2 | 2 | 8 | true | 0.0 | 0.0 |
| n2 | 2 | 9 | true | 0.0 | 0.0 |
| n2 | 2 | 10 | true | 0.0 | 0.0 |
| n2 | 2 | 11 | true | 0.0 | 0.0 |
| n2 | 2 | 12 | true | 0.0 | 0.0 |
| n2 | 2 | 13 | true | 0.0 | 0.0 |
| n2 | 2 | 14 | true | 0.0 | 0.0 |
| n2 | 2 | 15 | true | 0.0 | 0.0 |
| n4 | 4 | 0 | true | 0.0 | 0.0 |
| n4 | 4 | 1 | true | 0.0 | 0.0 |
| n4 | 4 | 2 | true | 0.0 | 0.0 |
| n4 | 4 | 3 | true | 0.0 | 0.0 |
| n4 | 4 | 4 | true | 0.0 | 0.0 |
| n4 | 4 | 5 | true | 0.0 | 0.0 |
| n4 | 4 | 6 | true | 0.0 | 0.0 |
| n4 | 4 | 7 | true | 0.0 | 0.0 |
| n4 | 4 | 8 | true | 0.0 | 0.0 |
| n4 | 4 | 9 | true | 0.0 | 0.0 |
| n4 | 4 | 10 | true | 0.0 | 0.0 |
| n4 | 4 | 11 | true | 0.0 | 0.0 |
| n4 | 4 | 12 | true | 0.0 | 0.0 |
| n4 | 4 | 13 | true | 0.0 | 0.0 |
| n4 | 4 | 14 | true | 0.0 | 0.0 |
| n4 | 4 | 15 | true | 0.0 | 0.0 |
| n8 | 8 | 0 | true | 0.0 | 0.0 |
| n8 | 8 | 1 | true | 0.0 | 0.0 |
| n8 | 8 | 2 | true | 0.0 | 0.0 |
| n8 | 8 | 3 | true | 0.0 | 0.0 |
| n8 | 8 | 4 | true | 0.0 | 0.0 |
| n8 | 8 | 5 | true | 0.0 | 0.0 |
| n8 | 8 | 6 | true | 0.0 | 0.0 |
| n8 | 8 | 7 | true | 0.0 | 0.0 |
| n8 | 8 | 8 | true | 0.0 | 0.0 |
| n8 | 8 | 9 | true | 0.0 | 0.0 |
| n8 | 8 | 10 | true | 0.0 | 0.0 |
| n8 | 8 | 11 | true | 0.0 | 0.0 |
| n8 | 8 | 12 | true | 0.0 | 0.0 |
| n8 | 8 | 13 | true | 0.0 | 0.0 |
| n8 | 8 | 14 | true | 0.0 | 0.0 |
| n8 | 8 | 15 | true | 0.0 | 0.0 |
| n16 | 16 | 0 | true | 0.0 | 0.0 |
| n16 | 16 | 1 | true | 0.0 | 0.0 |
| n16 | 16 | 2 | true | 0.0 | 0.0 |
| n16 | 16 | 3 | true | 0.0 | 0.0 |
| n16 | 16 | 4 | true | 0.0 | 0.0 |
| n16 | 16 | 5 | true | 0.0 | 0.0 |
| n16 | 16 | 6 | true | 0.0 | 0.0 |
| n16 | 16 | 7 | true | 0.0 | 0.0 |
| n16 | 16 | 8 | true | 0.0 | 0.0 |
| n16 | 16 | 9 | true | 0.0 | 0.0 |
| n16 | 16 | 10 | true | 0.0 | 0.0 |
| n16 | 16 | 11 | true | 0.0 | 0.0 |
| n16 | 16 | 12 | true | 0.0 | 0.0 |
| n16 | 16 | 13 | true | 0.0 | 0.0 |
| n16 | 16 | 14 | true | 0.0 | 0.0 |
| n16 | 16 | 15 | true | 0.0 | 0.0 |
| n32 | 32 | 0 | true | 2.980232238769531e-7 | 1.7881393432617188e-7 |
| n32 | 32 | 1 | true | 1.9371509552001953e-7 | 3.814697265625e-6 |
| n32 | 32 | 2 | true | 2.384185791015625e-7 | 1.1444091796875e-5 |
| n32 | 32 | 3 | true | 4.172325134277344e-7 | 5.7220458984375e-6 |
| n32 | 32 | 4 | true | 7.450580596923828e-8 | 1.7881393432617188e-7 |
| n32 | 32 | 5 | true | 7.450580596923828e-8 | 1.4901161193847656e-8 |
| n32 | 32 | 6 | true | 5.21540641784668e-8 | 5.960464477539063e-8 |
| n32 | 32 | 7 | true | 8.940696716308594e-8 | 2.60770320892334e-8 |
| n32 | 32 | 8 | true | 2.384185791015625e-7 | 1.1920928955078125e-6 |
| n32 | 32 | 9 | true | 2.384185791015625e-7 | 3.337860107421875e-6 |
| n32 | 32 | 10 | true | 1.4901161193847656e-7 | 4.76837158203125e-7 |
| n32 | 32 | 11 | true | 2.980232238769531e-7 | 2.980232238769531e-7 |
| n32 | 32 | 12 | true | 3.5762786865234375e-7 | 1.7881393432617188e-7 |
| n32 | 32 | 13 | true | 1.043081283569336e-7 | 1.4901161193847656e-7 |
| n32 | 32 | 14 | true | 6.854534149169922e-7 | 2.86102294921875e-6 |
| n32 | 32 | 15 | true | 1.341104507446289e-7 | 9.5367431640625e-7 |

Aggregate weight max abs: 6.854534149169922e-7 (limit 1.0e-6).  
Aggregate output max abs: 1.1444091796875e-5 (limit 1.0e-5).

**Limit:** This establishes every-layer causal MoE equivalence for batch sizes 2, 4, 8, 16, and 32 on the pinned OLMoE model; autoregressive generated-token equivalence remains untested.