Jeesup/svdsafety_l2_remove40_whiten_base_ft
Jeesup/svdsafety_l2_remove40_whiten_base_ft is a 7 billion parameter Llama-2-7b-chat checkpoint, compressed using SVD-LLM to 60% of its original parameters with 0% SVD components restored. This model is a research artifact designed to study how SVD compression impacts safety behavior and to test component-selection rules for repair. It is not intended as a general-purpose chat model but rather as an experimental subject for evaluating safety/utility trade-offs under compression.
Loading preview...
Model Overview
This model, Jeesup/svdsafety_l2_remove40_whiten_base_ft, is a research artifact derived from meta-llama/Llama-2-7b-chat-hf. It has been compressed using SVD-LLM, resulting in 40% of its parameters being removed, leaving it at approximately 60% of the original parameter count (7 billion parameters). Notably, 0% of SVD components were restored, and the selection rule for component restoration is unknown.
Key Characteristics
- Base Model:
meta-llama/Llama-2-7b-chat-hf - Compression Method: SVD-LLM, with 40.00% of parameters removed.
- Parameter Budget: 0.000% of dense parameters restored.
- Context Length: 4096 tokens.
- Purpose: This model is specifically created for a research study investigating the impact of SVD compression on model safety and the effectiveness of various component-selection rules in mitigating safety degradation.
Intended Use and Limitations
This checkpoint is not a general-purpose chat model. It is an experimental subject within a grid study designed to measure safety/utility trade-offs under compression. Some configurations in this study, including potentially this one, are deliberately safety-degraded relative to the original Llama-2-7b-chat. Users should treat this model as an experimental subject and conduct their own evaluations before drawing conclusions or considering it for deployment. Its primary utility is for research into model compression and safety.