Jeesup/svd-safety-l31_keep70_disc_b010
Jeesup/svd-safety-l31_keep70_disc_b010 is an 8 billion parameter Llama-3.1-8B-Instruct checkpoint that has been compressed using SVD-LLM, retaining 71.0% of its original dense parameters. An additional 1.0% parameter budget was used to restore SVD components selected by the 'disc' rule. This model is a research artifact designed to study the impact of SVD compression on safety behavior and the effectiveness of component-selection rules for repair, rather than a general-purpose chat model.
Loading preview...
Overview
This model, Jeesup/svd-safety-l31_keep70_disc_b010, is a research artifact derived from meta-llama/Llama-3.1-8B-Instruct. It has undergone significant compression using the SVD-LLM method, reducing its parameter count to approximately 71.0% of the original dense model. A small budget of 1.0% of dense parameters was then used to restore specific SVD components, selected by the disc rule, to investigate their impact on safety.
Key Characteristics
- Base Model: Llama-3.1-8B-Instruct.
- Compression Method: SVD-LLM, with 29.03% of parameters removed.
- Restoration: 1.0% of dense parameters restored using the
discselection rule, involving 7216 components. - Resulting Parameter Fraction: 0.7097 (approximately 71% of the original).
Measured Performance
This model's performance metrics highlight its experimental nature:
- AdvBench ASR (HarmBench judge): 0.0000
- StrongREJECT ASR (HarmBench judge): 0.0128
- Macro over-refusal (WildGuard): 0.8035
- WikiText-2 perplexity: 16.0406
Intended Use and Limitations
This checkpoint is not intended as a deployable assistant. Its primary purpose is to serve as an experimental subject within a research study quantifying safety/utility trade-offs under compression. Users should be aware that some configurations in this study, including potentially this one, are deliberately safety-degraded relative to the base Llama-3.1-8B-Instruct model. It is crucial to evaluate this model independently before drawing any conclusions from its behavior.