[C105] R-ESC: Robustly Erasing Space Concepts via Stochastic Feature Remapping

Abstract

Concept erasure and machine unlearning have emerged as a critical paradigm for enabling models to remove unwanted knowledge from learned representations at the level of classes, attributes, or higher-level semantics, motivated by the safety concerns surrounding modern generative AI models. However, existing approaches face three persistent challenges: (1) they often degrade utility on the remain concepts, (2) they often leave residual feature-level signals that enable recovery of the forget concepts via fine-tuning or representation probing, and (3) they impose substantial computational or memory overhead, limiting scalability. Prior work, such as Erasure Space Concept (ESC), moves toward structured removal in representation space, but achieving robust erasure with low overhead remains a challenge. To address this issue, we propose Robustly Erasing Space Concepts (R-ESC), a framework that tackles this challenge via stochastic feature remapping. R-ESC introduces three components: (i) prototype-orthogonal projection that reduces interference by decorrelating forget and remain prototypes prior to erasure, (ii) stochastic multi-target remapping that blends forget prototypes into multiple remain prototypes to prevent re-separation and hinder recovery, and (iii) activation-mean prototypes that compress the erasure procedure to a single forward pass, yielding linear-time computation with constant additional memory. Across extensive experiments, R-ESC maintains remain data utility, strengthens resistance to feature-level recovery, and scales to large models and datasets, providing a practical pathway toward robust and efficient concept erasure.