Jeesup/svd-safety-llama3_8b_instruct_remove_30_seed3_jbbmix_calib
Jeesup/svd-safety-llama3_8b_instruct_remove_30_seed3_jbbmix_calib is an 8 billion parameter Llama-3-8B-Instruct checkpoint compressed using SVD-LLM, retaining 70% of its dense parameters. This model is a research artifact designed to study the impact of SVD compression on safety behavior and the effectiveness of component-selection rules for repair. It is specifically intended for experimental evaluation of safety/utility trade-offs under compression, rather than as a general-purpose chat model.
Loading preview...
Overview
This model, svd-safety-llama3_8b_instruct_remove_30_seed3_jbbmix_calib, is an 8 billion parameter Llama-3-8B-Instruct checkpoint that has undergone significant compression. It was created using SVD-LLM, which reduced its dense parameters to 70% of the original, with a 0% budget for restoring SVD components selected by an 'unknown' rule.
Key Characteristics
- Base Model:
meta-llama/Meta-Llama-3-8B-Instruct - Compression Method: SVD-LLM, removing 30% of parameters.
- Parameter Fraction: The resulting model retains approximately 69.99% of the original parameters.
- Research Focus: This model is a specific cell within a larger research grid, designed to investigate how SVD compression affects safety behavior and how different component-selection rules might repair it.
Measured Performance
- AdvBench ASR (HarmBench judge): 0.0615
- StrongREJECT ASR (HarmBench judge): 0.0895
- Macro over-refusal (WildGuard): 0.1676
- WikiText-2 perplexity: 16.9001
Intended Use and Limitations
This checkpoint is primarily a research artifact for measuring safety/utility trade-offs under compression. It is important to note that several arms in the study, including this one, are deliberately safety-degraded compared to the original Llama-3-8B-Instruct. Users should treat this model as an experimental subject, not a deployable assistant, and conduct their own evaluations before drawing conclusions.