Jeesup/svd-safety-llama3_8b_instruct_remove_50_seed3_jbbmix_calib
Jeesup/svd-safety-llama3_8b_instruct_remove_50_seed3_jbbmix_calib is an 8 billion parameter Llama-3-8B-Instruct checkpoint, compressed using SVD-LLM to 50% of its dense parameters. This model is a research artifact designed to study how SVD compression impacts safety behavior and to test component-selection rules for repair. It is specifically configured with a 0% parameter budget for restored SVD components, making it an experimental subject rather than a general-purpose chat model.
Loading preview...
Model Overview
This model, svd-safety-llama3_8b_instruct_remove_50_seed3_jbbmix_calib, is an 8 billion parameter Llama-3-8B-Instruct checkpoint that has undergone significant compression. It was created as a research artifact to investigate the effects of Singular Value Decomposition (SVD) compression on the safety behavior of large language models and to evaluate methods for restoring safety.
Key Characteristics
- Base Model: Derived from
meta-llama/Meta-Llama-3-8B-Instruct. - Compression Method: Utilizes SVD-LLM, resulting in 50.0% of the original dense parameters being removed.
- Restoration Budget: Features a 0.0% parameter budget for restored SVD components, meaning no components were restored in this specific configuration.
- Research Focus: This checkpoint is one cell within a larger experimental grid, designed to measure safety/utility trade-offs under compression. It is not intended as a deployable chat model.
Measured Performance
Experimental measurements for this specific configuration include:
- AdvBench ASR (HarmBench judge): 0.4519
- StrongREJECT ASR (HarmBench judge): 0.4728
- Macro over-refusal (WildGuard): 0.0654
- WikiText-2 perplexity: 64.8741
Intended Use and Limitations
This model is a deliberately safety-degraded experimental subject. Its primary purpose is to quantify how compression impacts attack-success rates and to test recovery mechanisms. Users should treat it as a research tool for evaluating compression effects on safety, rather than a general-purpose assistant. Any conclusions drawn should be based on independent evaluation.