Jeesup/svd-safety-mis7_swift_remove30
Jeesup/svd-safety-mis7_swift_remove30 is a 7 billion parameter Mistral-7B-Instruct-v0.2 checkpoint, compressed to 70% of its original parameters using Swift-SVD and then recovered with SVD-LLM's stage-2 LoRA. This model is a research artifact designed to study how SVD compression impacts safety behavior and to test recovery methods, rather than serving as a general-purpose chat model. It is specifically intended for evaluating safety/utility trade-offs under compression, with some versions deliberately safety-degraded.
Loading preview...
Overview
This model, svd-safety-mis7_swift_remove30, is a research artifact derived from mistralai/Mistral-7B-Instruct-v0.2. It has been compressed using Swift-SVD, reducing its parameters to 70.0% of the original 7 billion, and subsequently recovered using SVD-LLM's stage-2 LoRA. The primary purpose of this model is to serve as an experimental subject in a study investigating the effects of SVD compression on safety behavior and the efficacy of various recovery techniques.
Key Characteristics
- Base Model:
mistralai/Mistral-7B-Instruct-v0.2 - Compression Method: Swift-SVD (dynamic rank allocation, alpha 0.6, 256 x 2048 WikiText2 calibration), removing 30.00% of parameters.
- Recovery Method: SVD-LLM's stage-2 LoRA (sequential U then V, alpaca-cleaned, r=8, alpha=16, 2 epochs per half, lr 0.0001, batch 64, cutoff 256).
- Parameter Fraction: Retains 70.03% of the original parameters.
- Measured Metrics: Achieves an AdvBench ASR (HarmBench judge) of 0.5981, StrongREJECT ASR (HarmBench judge) of 0.4473, Macro over-refusal (WildGuard) of 0.0686, and WikiText-2 perplexity of 8.4383.
Intended Use and Limitations
This model is not a general-purpose chat model. It is specifically designed as an experimental subject to measure safety/utility trade-offs under compression. Some versions within the study, including this one, are deliberately safety-degraded relative to the base Mistral-7B-Instruct-v0.2. Users should treat this checkpoint as an experimental artifact for research purposes and conduct their own evaluations before drawing conclusions or deploying it.