Jeesup/svd-safety-l2_basis_remove40_swapgapnet_b010_r07
Jeesup/svd-safety-l2_basis_remove40_swapgapnet_b010_r07 is a 7 billion parameter Llama-2-7b-chat checkpoint, compressed to 60% of its original parameters using Basis Sharing and then iteratively edited. This model is a research artifact designed to study how SVD compression impacts safety behavior and to test component-selection rules for repair. It is specifically configured with 4096 tokens context length and is not intended as a general-purpose chat model but rather for experimental evaluation of safety/utility trade-offs under compression.
Loading preview...
Overview
This model, svd-safety-l2_basis_remove40_swapgapnet_b010_r07, is a research artifact derived from meta-llama/Llama-2-7b-chat-hf. It has been compressed using Basis Sharing (ICLR 2025) to retain only 60.0% of its dense parameters, with 40.00% of parameters removed. Following compression, it underwent 7 out of 10 planned rounds of iterative parameter-neutral swapping, guided by the swapgapnet_iter rule, to restore components within a 1.000% budget of dense parameters.
Key Characteristics
- Base Model: Llama-2-7b-chat-hf
- Compression Method: Basis Sharing, reducing parameters by 40%.
- Iterative Editing: 7 rounds of
swapgapnet_iterrule application, swapping 45,308,672 parameters (0.70% of dense projection parameters). - Recovery: LoRA r=8 applied to per-layer coefficients for 2 epochs on alpaca-cleaned data.
Measured Safety Metrics
This model's safety performance has been measured, showing:
- AdvBench ASR (HarmBench judge): 0.0538
- StrongREJECT ASR (HarmBench judge): 0.1022
- Macro over-refusal (WildGuard): 0.1836
Intended Use and Limitations
This checkpoint is not a general-purpose chat model. It is a specific cell within a research grid, designed to measure safety/utility trade-offs under compression. Several arms in this study, including this one, are deliberately safety-degraded relative to the original Llama-2-7b-chat. Users should treat this as an experimental subject for evaluating compression and recovery techniques, not as a deployable assistant.