Navigating the tension between refusing adversarial attacks and falsely rejecting benign queries containing trigger words.
INTERACTIVE EXHIBIT 11
The Over-Refusal Pareto Frontier
SENSITIVITY: 0.50
OPTIMAL BALANCING POINT
F1 = 0.814
Too high sensitivity causes false-positive refusals on benign coding queries like "kill a process".
💡 Evaluation: Use benchmark suites like XSTest to rigorously measure over-refusal rates.
ARCHITECTURAL OVERVIEW
The Over-Refusal Sensitivity Frontier
A major failure mode of over-aligned models is 'over-refusal'—falsely refusing harmless, benign requests that contain sensitive keywords (e.g., 'How do I kill a lingering Linux process?' or 'Explain the mechanics of a biological virus'). Safety engineering must optimize the Pareto frontier between attack refusal rate and benign helpfulness.
Keyword Trigger Bias
Shallow safety classifiers often overfit to lexical tokens ('kill', 'bomb', 'exploit', 'hack', 'virus') without parsing the overarching semantic intent or syntactic context of the user's prompt.
The Safety vs Helpfulness Pareto Curve
Increasing safety sensitivity to catch 99.9% of adversarial attacks almost always increases false-positive refusals on benign developer queries. The optimal operating point maximizes the area under the Precision-Recall curve.
Dual-Path Intent Classification
Modern architectures separate safety checks into: (1) Lexical filter (fast pass), (2) Semantic Intent Analyzer (evaluates educational vs malicious context), and (3) Policy Arbiter.
max_θ [ (1 − β) · P(Refuse | x ∈ D_{harmful}) + β · P(Comply | x ∈ D_{benign}) ]
Where beta in [0, 1] is the policy weighting parameter balancing security risk against user utility. Tuning beta shifts the decision threshold along the receiver operating characteristic (ROC) curve.
REAL-WORLD PRODUCTION ENGINEERING
Benchmark safety suites using curated over-refusal datasets like XSTest and WildGuard to measure false refusal rates.
Implement context re-prompting: if an initial check triggers a refusal on a borderline query, pass the prompt through a second-stage nuanced policy judge.
Provide clear, informative refusal explanations rather than robotic generic messages ('I cannot fulfill this because...').