Jeesup/svd-safety-l3_jbb_k0p02_a0p3_free_remove30
Jeesup/svd-safety-l3_jbb_k0p02_a0p3_free_remove30 is an 8 billion parameter Llama-3-8B-Instruct checkpoint, compressed using SVD-LLM to 70% of its original dense parameters. This model is a research artifact designed to study how SVD compression impacts safety behavior and the effectiveness of component-selection rules for repair. It is specifically configured with a 0% parameter budget for restored SVD components, making it an experimental subject rather than a general-purpose chat model. Its primary purpose is to measure safety/utility trade-offs under compression, with deliberately degraded safety relative to the base Llama-3-8B-Instruct.
Loading preview...
Overview
This model, svd-safety-l3_jbb_k0p02_a0p3_free_remove30, is an 8 billion parameter Llama-3-8B-Instruct checkpoint that has undergone significant compression. It was created as a research artifact to investigate the effects of Singular Value Decomposition (SVD) compression on the safety performance of large language models.
Key Characteristics
- Base Model: Derived from
meta-llama/Meta-Llama-3-8B-Instruct. - Compression Method: Utilizes SVD-LLM, resulting in 30% of parameters being removed.
- Parameter Budget: Features a 0% budget for restoring SVD components, meaning no components were restored.
- Research Focus: Part of a study to quantify safety degradation due to compression and test recovery mechanisms.
- Measured Metrics:
- AdvBench ASR (HarmBench judge): 0.1192
- StrongREJECT ASR (HarmBench judge): 0.1470
- Macro over-refusal (WildGuard): 0.1968
- WikiText-2 perplexity: 16.9185
Intended Use and Limitations
This model is not intended for general-purpose deployment as a chat assistant. Its primary purpose is for research and measurement of safety/utility trade-offs under compression. It is deliberately safety-degraded compared to the original Llama-3-8B-Instruct to facilitate the study of compression's impact and potential recovery strategies. Users should treat it as an experimental subject and conduct their own evaluations before drawing conclusions.