Jeesup/svd-safety-l2_jbb_ka1_a1p0_free_remove40
Jeesup/svd-safety-l2_jbb_ka1_a1p0_free_remove40 is a 7 billion parameter Llama-2-7b-chat checkpoint compressed using SVD-LLM, retaining 60% of its original parameters. This model is a research artifact designed to study how SVD compression impacts safety behavior and to test component-selection rules for repair. It is specifically intended for experimental evaluation of safety/utility trade-offs under compression, rather than as a general-purpose chat model.
Loading preview...
Model Overview
Jeesup/svd-safety-l2_jbb_ka1_a1p0_free_remove40 is a research artifact derived from meta-llama/Llama-2-7b-chat-hf. This 7 billion parameter model has undergone SVD-LLM compression, resulting in 40% of its parameters being removed, leaving approximately 60% of the original dense parameters. A key characteristic of this specific checkpoint is that it utilized an "unknown" selection rule for SVD components and had a 0% restore budget, meaning no components were restored.
Key Characteristics & Purpose
- Compression Method: SVD-LLM, reducing parameters to 60% of the base model.
- Research Focus: This model is part of a study to quantify how SVD compression degrades safety behavior in large language models and to evaluate methods for repairing this degradation.
- Experimental Subject: It is explicitly noted as an experimental subject, not a general-purpose deployable assistant. Some arms of the study, including this one, are deliberately safety-degraded relative to the original Llama-2-7b-chat.
Measured Performance
Performance metrics for this specific checkpoint include:
- AdvBench ASR (HarmBench judge): 0.1481
- StrongREJECT ASR (HarmBench judge): 0.1502
- Macro over-refusal (WildGuard): 0.1296
- WikiText-2 perplexity: 11.4103
Intended Use
This model is intended for:
- Research into LLM compression and safety: Specifically for measuring safety/utility trade-offs under SVD compression.
- Evaluation of compression impact: Users should treat this as an experimental subject to understand the effects of compression on model safety and utility, rather than for practical deployment.