Jeesup/svd-safety-l2_jbbpure_ka8_a1p0_free_remove40
Jeesup/svd-safety-l2_jbbpure_ka8_a1p0_free_remove40 is a 7 billion parameter Llama-2-7b-chat checkpoint, compressed using SVD-LLM to 60% of its original dense parameters. This model is a research artifact from a study investigating how SVD compression impacts safety behavior and the effectiveness of component-selection rules in repairing it. It is specifically designed for measuring safety/utility trade-offs under compression, rather than serving as a general-purpose chat model.
Loading preview...
Overview
This model, Jeesup/svd-safety-l2_jbbpure_ka8_a1p0_free_remove40, is a research artifact derived from a Llama-2-7b-chat checkpoint. It has been compressed using SVD-LLM, reducing its parameter count to approximately 60% of the original dense parameters (40% removed). A key characteristic is that it was given a 0.0% parameter budget for restoring SVD components, selected by an "unknown" rule, meaning no components were restored.
Key Capabilities and Purpose
- Research Focus: This model is specifically designed for a study on how SVD compression affects the safety behavior of large language models and to evaluate different component-selection rules for recovery. It is one cell within a larger experimental grid.
- Compression Study: It demonstrates the effects of significant SVD compression (40% parameter removal) on a Llama-2-7b-chat base model.
- Measured Metrics: Performance metrics include AdvBench ASR (0.1077), StrongREJECT ASR (0.1118), Macro over-refusal (0.1452), and WikiText-2 perplexity (11.5242).
Good for
- Academic Research: Ideal for researchers studying model compression, safety degradation due to compression, and methods for restoring safety in compressed LLMs.
- Understanding SVD-LLM: Provides a concrete example of SVD-LLM compression's impact on model characteristics.
Important Limitations
- Not a General-Purpose Chat Model: This model is deliberately safety-degraded relative to the original Llama-2-7b-chat due to the experimental compression. It is intended as an experimental subject, not a deployable assistant, and should be evaluated thoroughly before drawing conclusions.