Jeesup/svd-safety-l2_remove60_sigma_b001
Jeesup/svd-safety-l2_remove60_sigma_b001 is a Llama-2-7b-chat checkpoint compressed using SVD-LLM, retaining 40.1% of its original parameters. This 7 billion parameter model then had 0.1% of its parameters restored using SVD components selected by the 'sigma' rule. It serves as a research artifact to study how SVD compression impacts safety behavior and the effectiveness of component-selection rules in recovery. This model is specifically designed for experimental evaluation of safety/utility trade-offs under compression, not as a general-purpose chat assistant.
Loading preview...
Model Overview
Jeesup/svd-safety-l2_remove60_sigma_b001 is a specialized research artifact derived from the meta-llama/Llama-2-7b-chat-hf model. It has undergone significant compression using the SVD-LLM method, reducing its parameter count to 40.1% of the original dense model. Following this compression, a small budget of 0.1% of dense parameters was restored using SVD components selected specifically by the sigma rule.
Key Characteristics
- Base Model: Llama-2-7b-chat-hf
- Compression Method: SVD-LLM, removing 59.91% of parameters.
- Restoration: 0.1% parameter budget restored using the
sigmaselection rule, involving 544 components. - Resulting Parameter Fraction: 0.4009 (approximately 40.1% of the original).
Measured Performance
This model's safety and utility metrics have been measured as part of its research context:
- AdvBench ASR (HarmBench judge): 0.3269
- StrongREJECT ASR (HarmBench judge): 0.3450
- Macro over-refusal (WildGuard): 0.0849
- WikiText-2 perplexity: 17.6323
Intended Use and Limitations
This model is not intended for general-purpose chat applications. Its primary purpose is to serve as an experimental subject within a research study investigating the impact of SVD compression on model safety and utility, and how different component-selection rules can mitigate safety degradation. Users should be aware that this checkpoint, like others in the study grid, may be deliberately safety-degraded relative to the original Llama-2-7b-chat. It is crucial to evaluate this model thoroughly before drawing any conclusions from its behavior.