Jeesup/svd-safety-l31_swift_remove20
Jeesup/svd-safety-l31_swift_remove20 is an 8 billion parameter Llama-3.1-8B-Instruct checkpoint compressed using Swift-SVD, retaining 80% of its original parameters. This model is a research artifact designed to study how SVD compression impacts safety behavior and the effectiveness of recovery methods. It is specifically intended for evaluating safety/utility trade-offs under compression, rather than serving as a general-purpose chat model.
Loading preview...
Model Overview
Jeesup/svd-safety-l31_swift_remove20 is a research artifact derived from meta-llama/Llama-3.1-8B-Instruct. It has been compressed using Swift-SVD with dynamic rank allocation (alpha 0.6, WikiText2 calibration), resulting in 80.0% of the original dense parameters.
Compression and Recovery
The model underwent a two-stage process:
- Compression: Swift-SVD removed 20.00% of parameters.
- Recovery: SVD-LLM's stage-2 LoRA was applied (sequential U then V, alpaca-cleaned, r=8, alpha=16, 2 epochs per half, lr 0.0001, batch 64, cutoff 256).
Measured Performance
This model's performance was measured with a focus on safety and utility metrics:
- AdvBench ASR (HarmBench judge): 0.0288
- StrongREJECT ASR (HarmBench judge): 0.0639
- Macro over-refusal (WildGuard): 0.2066
- WikiText-2 perplexity: 11.6668
Intended Use and Limitations
This checkpoint is not a general-purpose chat model. Its primary purpose is to serve as an experimental subject within a study quantifying the impact of compression on safety and evaluating recovery techniques. The README explicitly states that several arms in the study, including this one, are deliberately safety-degraded relative to the base Llama-3.1-8B-Instruct. Users should treat it as an experimental subject for research into safety/utility trade-offs under compression, not as a deployable assistant.