The Geometry of Refusal Vectors
Recent breakthroughs (e.g. Arditi et al.) revealed that safety alignment in frontier models is mediated by a single, highly directional 1D vector in activation space. Refusal is not a complex distributed heuristic, but a simple linear boundary check: if the activation's projection onto the refusal vector exceeds a scalar threshold, the model executes a refusal template.
The 1D Refusal Direction
By comparing the difference between harmful instruction activations and harmless instruction activations, researchers found that a single direction v_refusal explains over 95% of refusal variance across hundreds of diverse threat categories.
Ablation vs Addition (Abliteration)
Removing this single vector from activations (h' = h - (h . v) * v) completely eliminates all refusal behavior across the entire model. Conversely, adding this vector forces the model to refuse benign requests like 'How do I bake a cake?'.
Linear Separability of Harm Intent
Harmful intent is linearly separated in the middle layers of modern transformers. The refusal vector acts as a linear classifier that triggers the downstream generation of refusal tokens ('I cannot assist with...').
Subtracting the projection of hidden state h onto the unit refusal vector v_hat eliminates the model's capacity to detect harm, causing it to comply with requests it was previously trained to refuse.
- Track cosine similarity between user prompt activations and v_refusal in production monitoring to detect stealthy adversarial queries.
- Inspect refusal vector stability across fine-tuning epochs: fine-tuning on custom enterprise data must not inadvertently shrink ||v_refusal||.
- Use multi-directional refusal vectors (e.g. separate directions for CBRN, cyber, and privacy) to achieve fine-grained policy enforcement.