Jeesup/svd-safety-l3_remove30_swapgapiter_b010_r03
Jeesup/svd-safety-l3_remove30_swapgapiter_b010_r03 is an 8 billion parameter Llama-3-8B-Instruct checkpoint, compressed using SVD-LLM to 70% of its original parameters. This research artifact explores the impact of SVD compression on safety behavior and subsequent repair using iterative parameter-neutral swaps. It is specifically designed for studying safety/utility trade-offs under compression, rather than serving as a general-purpose chat model. The model is part of a larger study quantifying safety degradation and recovery post-compression.
Loading preview...
Overview
This model, svd-safety-l3_remove30_swapgapiter_b010_r03, is an 8 billion parameter variant of meta-llama/Meta-Llama-3-8B-Instruct. It has undergone significant compression using SVD-LLM, reducing its parameters by 30.01% to approximately 70% of the original dense parameter count. Following compression, 3 of 10 rounds of iterative parameter-neutral swaps were applied using the gap_iter selection rule, restoring a small fraction of parameters (0.30% of dense projection parameters) to investigate safety repair.
Key Characteristics
- Base Model:
meta-llama/Meta-Llama-3-8B-Instruct - Compression Method: SVD-LLM, removing 30.01% of parameters.
- Repair Mechanism: Iterative parameter-neutral swap using the
gap_iterrule, applied for 3 out of 10 planned rounds. - Measured Safety Metrics: Achieves an AdvBench ASR (HarmBench judge) of 0.2400 and StrongREJECT ASR (HarmBench judge) of 0.1750, with a Macro over-refusal (WildGuard) of 0.0826.
Intended Use and Limitations
This model is a research artifact specifically created to measure safety/utility trade-offs under compression. It is part of a study to quantify how SVD compression degrades safety and to test recovery methods. Consequently, this checkpoint is deliberately safety-degraded relative to the original Llama-3-8B-Instruct. It is not intended as a general-purpose chat model and should be treated as an experimental subject for research purposes only. Users should conduct their own evaluations before drawing conclusions or deploying it.