Jeesup/svdsafety_l2_ablate_top64
Jeesup/svdsafety_l2_ablate_top64 is a 7 billion parameter research artifact derived from Meta Llama 2 7B Chat, designed to investigate the role of specific singular value subspaces in model safety. This model removes the leading rank-64 subspace of the delta weights (W_chat - W_base) from every projection, resulting in a significant reduction in refusal rates for harmful prompts. It serves as a tool for measuring how alignment and safety mechanisms are affected by low-rank compression, rather than for deployment as a general-purpose chat assistant.
Loading preview...
Overview
Jeesup/svdsafety_l2_ablate_top64 is a 7 billion parameter research model built from meta-llama/Llama-2-7b-chat-hf and meta-llama/Llama-2-7b-hf. Its primary purpose is to function as a safety-ablation artifact, not a general chat model. It specifically removes the leading rank-64 subspace of the delta weights (dW = W_chat - W_base) from all 224 projections of the chat model, a process intended to test the hypothesis that this subspace carries crucial safety information.
Key Characteristics & Measured Behavior
This model is characterized by a weight perturbation of 0.02423 averaged over matrices. Its behavior has been measured using AdvBench (520 prompts) with greedy chat decoding, judged by cais/HarmBench-Llama-2-13b-cls:
- Attack Success Rate (ASR): 0.0962 (compared to 0.0019 for dense Llama-2-7b-chat)
- Refusal Rate (Harmful): 0.831 (compared to 0.994 for dense Llama-2-7b-chat)
- Degenerate Output Rate: 0.042 (compared to 0.000 for dense Llama-2-7b-chat)
These metrics demonstrate a significant increase in ASR and a decrease in refusal rate, indicating a reduction in safety mechanisms due to the ablation. Comparative analysis with ablations of bottom-64 or random-64 dW subspaces shows that only the removal of the top-64 subspace significantly impacts refusal behavior, suggesting its importance for safety.
Intended Use
This model is intended for research purposes, specifically for:
- Measuring how alignment and safety mechanisms survive low-rank compression.
- Investigating the specific subspaces responsible for refusal behavior in aligned models.
It is not designed or recommended for deployment as an assistant due to its intentionally modified refusal behavior and potential degradation in general capabilities. The underlying subspaces are available at Jeesup/svdsafety_l2_delta_basis.