Chain-of-thought and test-time compute
Reasoning models spend inference-time tokens to buy accuracy: generate multiple chains of thought, vote or verify, and the effective answer quality climbs with compute — a curve that did not exist for one-shot completions.
System 1 vs System 2
Direct answers are fast intuition; long CoT is deliberation. RLVR training teaches the model when to deliberate and how to backtrack within a single generation.
Self-Consistency
Sample k chains, take the majority answer. Accuracy rises with k with diminishing returns — majority voting cancels independent reasoning errors.
Pass@k
The probability that at least one of k samples is correct. RLVR training improves both the single-sample accuracy and the tail — pass@k curves shift up wholesale.
Unbiased pass@k over n graded samples with c correct. If single-sample accuracy is 40%, majority voting over 8 samples behaves like a much sharper estimator.
- Reasoning tokens cost real latency and money: a 4k-token CoT at 50 tok/s adds 80 seconds per answer. Production routers send easy queries to a short-CoT model.
- Length-controlled evals matter: naive RLVR can inflate CoT length to mine format rewards without improving correctness.