Jeesup/svd-safety-l2_remove50_random_b005
Jeesup/svd-safety-l2_remove50_random_b005 is a 7 billion parameter Llama-2-7b-chat checkpoint, compressed using SVD-LLM to 50.5% of its original dense parameters. It then had 0.5% of its parameters restored using a 'random' component selection rule. This model is a research artifact designed to study the impact of SVD compression on safety behavior and the effectiveness of different component-selection rules for recovery, rather than a general-purpose chat model.
Loading preview...
Model Overview
Jeesup/svd-safety-l2_remove50_random_b005 is a research artifact derived from meta-llama/Llama-2-7b-chat-hf, a 7 billion parameter model. This checkpoint was created by applying SVD-LLM compression, which removed approximately 49.52% of its original parameters. Subsequently, 0.5% of the dense parameters were restored using a 'random' component selection rule, specifically 2948 components, resulting in a final parameter fraction of 0.5048.
Purpose and Limitations
This model is not intended as a general-purpose deployable assistant. Its primary purpose is to serve as an experimental subject within a study investigating the trade-offs between safety and utility under compression. The research aims to quantify how SVD compression can degrade safety behavior and evaluate the efficacy of various component-selection rules in mitigating this degradation. As such, several arms of this research grid, including this model, are deliberately safety-degraded compared to the original Llama-2-7b-chat.
Measured Performance
Key metrics measured for this specific checkpoint include:
- AdvBench ASR (HarmBench judge): 0.4962
- StrongREJECT ASR (HarmBench judge): 0.3482
- Macro over-refusal (WildGuard): 0.1208
- WikiText-2 perplexity: 13.5039
Users are advised to evaluate this experimental subject thoroughly before drawing conclusions or considering any deployment, as its design intentionally explores safety degradation and recovery.