Jeesup/svd-safety-l2_remove40_disc_b001
Jeesup/svd-safety-l2_remove40_disc_b001 is a 7 billion parameter Llama-2-7b-chat checkpoint compressed using SVD-LLM, retaining 60.1% of its dense parameters. It incorporates a 0.1% parameter budget of restored SVD components selected by the 'disc' rule. This model serves as a research artifact to study how SVD compression impacts safety behavior and the effectiveness of component-selection rules in recovery, rather than a general-purpose chat model.
Loading preview...
Model Overview
This model, svd-safety-l2_remove40_disc_b001, is a research artifact derived from the meta-llama/Llama-2-7b-chat-hf base model. It has been compressed using SVD-LLM, resulting in the removal of 39.92% of its parameters, leaving 60.1% of the original dense parameters. A small budget of 0.1% of dense parameters (630 components) was then restored using the disc selection rule.
Key Characteristics
- Base Model: Llama-2-7b-chat-hf
- Compression Method: SVD-LLM, reducing parameters by 39.92%
- Restoration: 0.1% of parameters restored using the
discrule - Parameter Fraction: 0.6008 of the original dense parameters
Measured Performance
This model's safety and utility metrics have been measured as part of the research:
- AdvBench ASR (HarmBench judge): 0.0192
- StrongREJECT ASR (HarmBench judge): 0.0831
- Macro over-refusal (WildGuard): 0.2546
- WikiText-2 perplexity: 11.3278
Intended Use and Limitations
This checkpoint is specifically designed for research purposes to evaluate safety/utility trade-offs under compression. It is important to note that several arms of this study, including this model, are deliberately safety-degraded compared to the original Llama-2-7b-chat. The primary goal is to quantify the impact of compression on safety and test recovery mechanisms. Therefore, this model should be treated as an experimental subject for research and not as a deployable assistant. Users should conduct their own evaluations before drawing conclusions.