Jeesup/svd-safety-l3_swift_jbbsft1_remove30
Jeesup/svd-safety-l3_swift_jbbsft1_remove30 is an 8 billion parameter Llama-3-Instruct checkpoint, compressed to 70% of its original size using Swift-SVD with dynamic rank allocation. This model is a research artifact designed to study how SVD compression impacts safety behavior and the effectiveness of recovery methods. It is specifically intended for evaluating safety/utility trade-offs under compression, rather than serving as a general-purpose chat model.
Loading preview...
Overview
This model, svd-safety-l3_swift_jbbsft1_remove30, is a research artifact derived from meta-llama/Meta-Llama-3-8B-Instruct. It has been compressed using Swift-SVD with dynamic rank allocation (alpha 0.6, 256 x 2048 WikiText2 calibration), resulting in 30% of its parameters being removed, leaving 70.0% of the original dense parameters. Following compression, the model underwent recovery using SVD-LLM's stage-2 LoRA.
Purpose and Limitations
This checkpoint is explicitly a research artifact from a study investigating how SVD compression affects safety behavior and which component-selection rules are most effective for repair. It is not intended as a general-purpose chat model and is part of a grid of experimental subjects, some of which are deliberately safety-degraded relative to the base Llama-3-8B-Instruct. Users should treat it as an experimental subject for evaluating safety/utility trade-offs under compression and conduct their own evaluations before drawing conclusions or deploying it.
Measured Metrics
Key measured metrics for this specific artifact include:
- AdvBench ASR (HarmBench judge): 0.0346
- StrongREJECT ASR (HarmBench judge): 0.0735
- Macro over-refusal (WildGuard): 0.3565
- WikiText-2 perplexity: 18.6313
Provenance
- Base Model:
meta-llama/Meta-Llama-3-8B-Instruct - Compression Method: Swift-SVD (dynamic rank allocation, alpha 0.6, 256 x 2048 WikiText2 calibration)
- Parameter Reduction: 30.00% of parameters removed, resulting in 0.7003 fraction of original parameters.