Jeesup/svd-safety-l2_remove60_sigma_b005
Jeesup/svd-safety-l2_remove60_sigma_b005 is a Llama-2-7b-chat checkpoint compressed using SVD-LLM, retaining 40.5% of its dense parameters. This 7 billion parameter model then had 0.5% of its parameters restored using the 'sigma' component selection rule. It serves as a research artifact to study how SVD compression impacts safety behavior and the effectiveness of different component-selection rules in recovery, rather than a general-purpose chat model.
Loading preview...
Model Overview
Jeesup/svd-safety-l2_remove60_sigma_b005 is a research artifact derived from meta-llama/Llama-2-7b-chat-hf. This model has undergone significant compression using the SVD-LLM technique, resulting in the removal of 59.51% of its original parameters. Subsequently, 0.5% of the dense parameters were restored using the sigma component selection rule, specifically targeting the repair of safety behaviors.
Key Characteristics
- Base Model: Llama-2-7b-chat-hf
- Compression Method: SVD-LLM, reducing parameters to 40.49% of the original.
- Restoration: 0.5% parameter budget restored using the
sigmarule, involving 2720 components. - Purpose: This model is explicitly designed as an experimental subject within a larger study to measure safety/utility trade-offs under compression, not as a general-purpose deployable assistant.
Measured Performance
Key metrics measured for this specific configuration include:
- AdvBench ASR (HarmBench judge): 0.3115
- StrongREJECT ASR (HarmBench judge): 0.4026
- Macro over-refusal (WildGuard): 0.0828
- WikiText-2 perplexity: 17.4001
Intended Use and Limitations
This checkpoint is primarily for research into how SVD compression affects model safety and how different restoration strategies can mitigate degradation. It is important to note that several configurations within this research grid, including this one, are deliberately safety-degraded compared to the original Llama-2-7b-chat. Users should treat this model as an experimental subject and conduct their own evaluations rather than deploying it as a production-ready assistant.