Jeesup/svd-safety-l3_swift_remove40
Jeesup/svd-safety-l3_swift_remove40 is an 8 billion parameter Llama-3-8B-Instruct checkpoint, compressed using Swift-SVD to 60% of its original dense parameters. This model is a research artifact specifically designed to study the impact of SVD compression on safety behavior and to test recovery methods. It is not intended as a general-purpose chat model but rather for evaluating safety/utility trade-offs under compression.
Loading preview...
Model Overview
Jeesup/svd-safety-l3_swift_remove40 is an 8 billion parameter Llama-3-8B-Instruct model that has undergone significant compression. It was created using Swift-SVD with dynamic rank allocation (alpha 0.6, 256 x 2048 WikiText2 calibration), resulting in a reduction to 60.0% of its original dense parameters. Following compression, the model was recovered using SVD-LLM's stage-2 LoRA.
Key Characteristics
- Base Model:
meta-llama/Meta-Llama-3-8B-Instruct - Compression Method: Swift-SVD, removing 40.00% of parameters.
- Resulting Parameter Fraction: 0.6003 (approximately 4.8 billion parameters).
- Recovery Method: SVD-LLM's stage-2 LoRA (sequential U then V, alpaca-cleaned, r=8, alpha=16, 2 epochs per half, lr 0.0001, batch 64, cutoff 256).
Intended Use and Limitations
This model is a research artifact from a study investigating how SVD compression affects safety and which component-selection rules best repair it. It is explicitly stated that several arms in this research grid are deliberately safety-degraded relative to the original Llama-3-8B-Instruct. Therefore, this checkpoint is not intended for general-purpose deployment as a chat assistant. Users should treat it as an experimental subject for evaluating safety/utility trade-offs under compression and conduct their own evaluations before drawing conclusions.