Jeesup/svd-safety-llama3_8b_instruct_remove_40_seed3_jbbmix_calib
Jeesup/svd-safety-llama3_8b_instruct_remove_40_seed3_jbbmix_calib is an 8 billion parameter Llama-3-8B-Instruct checkpoint, compressed using SVD-LLM to 60% of its original dense parameters. This model is a research artifact designed to study the impact of SVD compression on safety behavior and the effectiveness of component selection rules for repair. It is specifically configured with 0% parameter budget for restored SVD components, making it an experimental subject rather than a general-purpose chat model. Its primary purpose is to quantify safety/utility trade-offs under compression, with some arms deliberately safety-degraded.
Loading preview...
Model Overview
Jeesup/svd-safety-llama3_8b_instruct_remove_40_seed3_jbbmix_calib is an experimental 8 billion parameter Llama-3-8B-Instruct model that has undergone significant compression using SVD-LLM. Specifically, 40% of its parameters were removed, resulting in a model that retains approximately 60% of its original dense parameters. This particular variant was created with a 0% parameter budget for restoring SVD components, selected by an 'unknown' rule, making it a specific cell within a larger research grid.
Key Characteristics & Purpose
- Research Artifact: This model is not intended for general-purpose deployment but serves as a research artifact to investigate how SVD compression affects the safety behavior of large language models.
- Safety Degradation Study: It is part of a study quantifying the impact of compression on attack-success rates (ASR) and testing methods for recovery. Some configurations, including this one, are deliberately safety-degraded relative to the base Llama-3-8B-Instruct.
- Compression Details: Compressed using SVD-LLM, with 40% of parameters removed and no SVD components restored.
- Measured Metrics: Performance metrics include an AdvBench ASR of 0.1135, StrongREJECT ASR of 0.1789, Macro over-refusal (WildGuard) of 0.0983, and a WikiText-2 perplexity of 25.0889.
Intended Use
- Academic Research: Ideal for researchers studying model compression, safety degradation, and recovery mechanisms in LLMs.
- Experimental Evaluation: Suitable for evaluating safety/utility trade-offs under various compression strategies. Users should treat it as an experimental subject and conduct their own evaluations before drawing conclusions.