Jeesup/svd-safety-l2_remove50_swapgap_b001
Jeesup/svd-safety-l2_remove50_swapgap_b001 is a 7 billion parameter Llama-2-7b-chat checkpoint compressed using SVD-LLM, retaining 50.0% of its dense parameters. It features a 0.1% parameter budget for restored SVD components selected by the 'swapgap' rule, making it a research artifact for studying safety behavior under compression. This model is specifically designed for evaluating safety/utility trade-offs in compressed language models rather than general-purpose chat applications.
Loading preview...
Model Overview
This model, svd-safety-l2_remove50_swapgap_b001, is a research artifact derived from meta-llama/Llama-2-7b-chat-hf. It has been compressed using SVD-LLM, resulting in a reduction of 50.01% of its original parameters. A small budget of 0.1% of dense parameters was then used to restore SVD components selected by the swapgap rule, involving 669 components swapped out and restored.
Purpose and Limitations
This checkpoint is not intended as a general-purpose chat model. Its primary purpose is to serve as an experimental subject in a study quantifying how SVD compression impacts safety behavior and how different component-selection rules can repair it. The model's safety behavior is deliberately degraded relative to the base Llama-2-7b-chat due to compression, and users should evaluate it thoroughly before drawing conclusions.
Measured Metrics
Key metrics measured for this specific configuration include:
- AdvBench ASR (HarmBench judge): 0.4596
- StrongREJECT ASR (HarmBench judge): 0.2556
- Macro over-refusal (WildGuard): 0.1498
- WikiText-2 perplexity: 13.6170
Licensing
This model operates under the Llama 2 Community License, with LICENSE.txt and USE_POLICY.md included, binding its use to these terms.