Jeesup/svd-safety-l2_basis_remove40_swapgapnet_b010_r06
Jeesup/svd-safety-l2_basis_remove40_swapgapnet_b010_r06 is a 7 billion parameter Llama-2-7b-chat checkpoint compressed using Basis Sharing, reducing its parameter count to 60% of the original. This model has undergone 6 of 10 rounds of iterative parameter-neutral swap selection using the `swapgapnet_iter` rule to study how SVD compression affects safety behavior. It serves as a research artifact for quantifying safety degradation and testing recovery mechanisms, rather than a general-purpose chat model.
Loading preview...
Model Overview
This model, svd-safety-l2_basis_remove40_swapgapnet_b010_r06, is a research artifact derived from meta-llama/Llama-2-7b-chat-hf. It has been significantly compressed using Basis Sharing, a technique that reduces the parameter count to approximately 60% of the original 7 billion parameters by removing 40% of parameters. Following compression, the model underwent 6 out of 10 planned rounds of iterative parameter-neutral swapping, guided by the swapgapnet_iter rule, to restore components.
Key Characteristics
- Base Model: Llama-2-7b-chat-hf
- Compression Method: Basis Sharing (ICLR 2025), reducing parameters by 40%.
- Restoration: Iterative parameter-neutral swap using
swapgapnet_iterrule, with 6 of 10 rounds applied. - Parameter Count: Approximately 60% of the original 7B parameters.
- Context Length: 4096 tokens.
- Recovery: LoRA r=8 on per-layer coefficients for 2 epochs with
alpaca-cleaneddataset.
Measured Safety Metrics
- AdvBench ASR (HarmBench judge): 0.0942
- StrongREJECT ASR (HarmBench judge): 0.1502
- Macro over-refusal (WildGuard): 0.1227
Intended Use and Limitations
This model is not intended as a general-purpose chat model. It is a specific experimental cell within a larger study designed to measure the trade-offs between safety and utility under compression. The compression process alone increases the attack-success rate, and this research aims to quantify that degradation and evaluate recovery strategies. Users should treat this as an experimental subject and conduct their own evaluations before drawing conclusions or deploying it.