Jeesup/svdsafety_l2_ablate_random64
Jeesup/svdsafety_l2_ablate_random64 is a 7 billion parameter Llama-2-chat-hf based research artifact developed by Jeesup, designed to investigate safety ablation in large language models. This model specifically removes 64 uniformly random singular components from the weight perturbation (dW) of the base model, serving as a control for studies on how alignment survives low-rank compression. It is intended for research into refusal behavior and model safety, not for general chat applications.
Loading preview...
Overview
Jeesup/svdsafety_l2_ablate_random64 is a 7 billion parameter research model derived from meta-llama/Llama-2-7b-chat-hf and meta-llama/Llama-2-7b-hf. It is a safety-ablation artifact created by removing 64 uniformly random singular components of the dW (weight perturbation) from the base model, with a perturbation ||W' - W||_F / ||W||_F of 0.02423. This model serves as a control in experiments designed to understand how safety alignment is affected by low-rank compression.
Key Characteristics
- Ablation Type: Removes 64 uniformly random singular components of
dW, norm-matched to the top-64 block. - Purpose: Primarily a research artifact for studying safety and refusal behavior in LLMs.
- Measured Behavior: On AdvBench, it shows an attack success rate of 0.0058 and a refusal rate of 0.992, compared to 0.0019 ASR and 0.994 refusal rate for the dense Llama-2-7b-chat.
- Context: Used to compare against models where top-64 or bottom-64
dWcomponents are removed, demonstrating that only the top-64 components significantly impact refusal.
Intended Use
This model is specifically for measuring how alignment survives low-rank compression. It is not intended for deployment as a general-purpose chat assistant due to modifications aimed at altering refusal behavior and potential degradation in general capabilities. The subspaces themselves are available at Jeesup/svdsafety_l2_delta_basis.