Jeesup/svd-safety-l2_remove60_swapdisc_b005
Jeesup/svd-safety-l2_remove60_swapdisc_b005 is a Llama-2-7b-chat checkpoint compressed using SVD-LLM, retaining 40.0% of its original dense parameters. It then had 0.5% of its parameters restored using SVD components selected by the 'swapdisc' rule. This 7 billion parameter model is a research artifact designed to study how SVD compression impacts safety behavior and the effectiveness of component-selection rules in recovery. It is not intended as a general-purpose chat model but rather for evaluating safety/utility trade-offs under compression.
Loading preview...
Model Overview
Jeesup/svd-safety-l2_remove60_swapdisc_b005 is a specialized research artifact derived from meta-llama/Llama-2-7b-chat-hf. This model has undergone significant compression using the SVD-LLM method, resulting in the removal of 60.01% of its original parameters, leaving approximately 40% of the dense parameters. A small budget of 0.5% of dense parameters (3094 components) was then restored using the swapdisc selection rule.
Purpose and Characteristics
This model is a specific configuration within a broader research study investigating the impact of SVD compression on model safety and utility. Its primary purpose is to serve as an experimental subject to measure:
- Safety Degradation: How compression affects attack success rates (ASR) on benchmarks like AdvBench and StrongREJECT.
- Recovery Effectiveness: The ability of different component-selection rules, like
swapdisc, to mitigate safety damage caused by compression.
Measured Performance
Key metrics for this specific checkpoint include:
- AdvBench ASR (HarmBench judge): 0.2731
- StrongREJECT ASR (HarmBench judge): 0.2652
- Macro over-refusal (WildGuard): 0.1484
- WikiText-2 perplexity: 18.1431
Intended Use and Limitations
This model is not a general-purpose chat assistant. It is explicitly designed as a research checkpoint for evaluating safety/utility trade-offs under compression. Users should be aware that some configurations within this study, including potentially this one, are deliberately safety-degraded relative to the base Llama-2-7b-chat model. It should be treated as an experimental subject and evaluated thoroughly before drawing any conclusions.