Jeesup/svd-safety-mis7_swift_remove40
Jeesup/svd-safety-mis7_swift_remove40 is a 7 billion parameter Mistral-7B-Instruct-v0.2 checkpoint, compressed to 60% of its original parameters using Swift-SVD and then recovered with SVD-LLM's stage-2 LoRA. Developed by Jeesup, this model is a research artifact designed to study how SVD compression impacts safety behavior and the effectiveness of repair mechanisms. It is specifically intended for experimental evaluation of safety/utility trade-offs under compression, rather than as a general-purpose chat model.
Loading preview...
Model Overview
Jeesup/svd-safety-mis7_swift_remove40 is a research artifact derived from mistralai/Mistral-7B-Instruct-v0.2. This 7 billion parameter model has undergone significant compression using Swift-SVD, reducing its parameters to 60.0% of the original dense model. Following compression, it was subjected to a recovery process using SVD-LLM's stage-2 LoRA (r=8, alpha=16, 2 epochs per half, lr 0.0001, batch 64, cutoff 256).
Key Characteristics
- Base Model:
mistralai/Mistral-7B-Instruct-v0.2 - Compression Method: Swift-SVD with dynamic rank allocation (alpha 0.6, 256 x 2048 WikiText2 calibration), resulting in 40.00% parameter removal.
- Recovery Method: SVD-LLM's stage-2 LoRA, applied sequentially to U then V matrices.
- Measured Metrics:
- AdvBench ASR (HarmBench judge): 0.6558
- StrongREJECT ASR (HarmBench judge): 0.5495
- Macro over-refusal (WildGuard): 0.0663
- WikiText-2 perplexity: 10.7309
Intended Use and Limitations
This model is not a general-purpose chat model and is explicitly a research artifact. Its primary purpose is to measure safety/utility trade-offs under compression, particularly how SVD compression can degrade safety and how different component-selection rules can repair it. Users should be aware that some configurations within this study, including this specific checkpoint, are deliberately safety-degraded relative to the base Mistral-7B-Instruct-v0.2. It is crucial to treat this model as an experimental subject and conduct independent evaluations before drawing conclusions or considering deployment.