Red-teaming and jailbreak decay
Safety training needs adversarial data, and adversaries iterate. Automated red-teaming uses an attack model (or the target itself) to search for jailbreaks, the failures are labeled, added to training, and the loop repeats. Measurable success: the jailbreak success rate falls exponentially per round.
Attack Surfaces
Role-play framing, hypotheticals, encoding tricks, multi-turn grooming, and tool-mediated exfiltration. Each gets its own probe family.
The Loop
Attack → classify (did it succeed?) → add to preference/SFT data → retrain → re-attack. Each round compounds.
Measure Honestly
Report jailbreak success rate per probe family. Aggregate averages hide the family that still fails 40% of the time.
At 80% initial success and 0.55 decay per round: 44% after one round, 24% after two, 13% after three. Exponential in rounds — which is why sustained red-teaming, not one-off audits, is the mechanism.
- Production programs run continuous adversarial campaigns with fresh attack models; a static probe suite is stale the day it ships.
- Every confirmed jailbreak becomes at least one SFT refusal and one preference pair — the data engine is the deliverable, not the report.