Catastrophic forgetting and the retention math
Specializing a model on new-domain data quietly erodes general capability. Continued pre-training (CPT) injects knowledge deeply but forgets fast; SFT shapes behavior gently but learns shallowly. The mitigation in both cases is rehearsal.
CPT vs SFT
CPT (raw tokens, next-token objective) moves the distribution far and risks forgetting. SFT (demonstrations) moves behavior with less collateral damage.
Replay Buffers
Mix 10–30% general-domain data into every domain-adaptation run. The model keeps rehearsing what it must not forget.
Distillation Bridges
Add a KL term pulling the fine-tuned model's outputs toward the frozen base on general prompts — a soft leash on drift.
A domain run with forgetting rate 0.4 and 25% replay retains 70% of general capability; the same run without replay retains 60%.
- Medical- and legal-domain fine-tunes routinely fail public benchmarks that weren't in scope — measure retention with a frozen general eval suite.
- A practical recipe: CPT on 1–5B domain tokens with 20% replay, then a short SFT on domain instructions with 10% general instructions.