Jeesup/svd-safety-l2_remove50_swapdisciter_b010
Jeesup/svd-safety-l2_remove50_swapdisciter_b010 is a 7 billion parameter Llama-2-7b-chat checkpoint compressed using SVD-LLM, with 50.0% of its dense parameters removed. It then had 1.0% of its parameters restored using the 'disc_iter' component selection rule. This model is a research artifact designed to study how SVD compression impacts safety behavior and the effectiveness of component restoration rules, rather than a general-purpose chat model.
Loading preview...
Overview
This model, svd-safety-l2_remove50_swapdisciter_b010, is a research artifact derived from meta-llama/Llama-2-7b-chat-hf. It has undergone significant compression using the SVD-LLM method, resulting in 50.01% of its parameters being removed. Subsequently, 1.0% of its original dense parameters were restored using a specific component selection rule called disc_iter.
Purpose and Limitations
This checkpoint is explicitly created for measuring safety/utility trade-offs under compression. It is part of a larger study to quantify how SVD compression affects a model's safety behavior (e.g., increasing attack-success rates) and to test various recovery mechanisms. The model is not intended as a general-purpose chat model for deployment. Many arms of this research grid, including this one, are deliberately safety-degraded relative to the original Llama-2-7b-chat.
Measured Performance
Key metrics for this specific variant include:
- AdvBench ASR (HarmBench judge): 0.0115
- StrongREJECT ASR (HarmBench judge): 0.0735
- Macro over-refusal (WildGuard): 0.4240
- WikiText-2 perplexity: 14.3199
Users should evaluate this model as an experimental subject and not draw conclusions about its deployability without thorough independent assessment.