EXHIBIT 08 OF 24 · MODULE 2: REPRESENTATION ENGINEERING
DEFENSE
Subspace Erasure & Representation Surgery
Projecting model weight matrices orthogonal to dangerous concept subspaces to permanently disable hazardous knowledge.
INTERACTIVE EXHIBIT 08
Subspace Erasure & Weight Surgery
KNOWLEDGE REMOVAL STATUS
VULNERABLE TO RECALL
Standard post-training refusal leaves underlying knowledge intact inside weights.
💡 Advantage: Cannot be bypassed by prompt jailbreaks or activation manipulation because the knowledge no longer exists in weight space.
ARCHITECTURAL OVERVIEW
Subspace Erasure & Representation Surgery
Rather than relying on post-training refusals (which can be jailbroken), Representation Surgery permanently removes hazardous knowledge (such as chemical weapon synthesis or zero-day cyber exploits) from model weight matrices by projecting them onto the orthogonal complement of the concept subspace.
Concept Subspace Identification
A hazardous concept (e.g. 'Synthesizing Sarin Gas') occupies a low-dimensional subspace V in activation space, identified via Singular Value Decomposition (SVD) of hidden states across hazardous queries.
Null-Space Projection
Construct an orthogonal projection matrix P_orth = I - V(V^T V)^(-1) V^T. Multiplying the model's weight matrices (such as MLP down-projections W_down) by P_orth ensures the network can never represent or activate that subspace.
Irreversible Knowledge Removal
Unlike refusal alignment (which leaves the underlying knowledge intact and merely suppresses output tokens), representation surgery eliminates the latent representations, rendering the model permanently incapable of recalling the information.
MATHEMATICAL FORMULATION · ORTHOGONAL PROJECTION WEIGHT SURGERY
W_{new} = P_{\perp} · W_{orig}, where P_{\perp} = I − V (V^T V)^{−1} V^T
Where V is the k-dimensional concept basis matrix. Any activation h passed through W_new has zero component in subspace V (P_perp * V = 0), preventing hazardous knowledge retrieval even under severe adversarial jailbreaking.
REAL-WORLD PRODUCTION ENGINEERING
Regularize projection: ensure P_perp does not inadvertently project away overlapping benign chemical or computer science knowledge.
Evaluate retained capabilities across MMLU, GSM8K, and HumanEval before and after surgery to verify zero general capability degradation.
Use representation surgery prior to open-weight model releases to guarantee compliance with biosecurity safety standards.