Jeesup/svd-safety-l2_basis_remove40_swapgapnet_b010_r09

Model Overview

This model, svd-safety-l2_basis_remove40_swapgapnet_b010_r09, is a 7B parameter Llama-2-7b-chat checkpoint that has undergone significant compression and iterative editing. It was compressed using Basis Sharing (ICLR 2025) to retain only 60.0% of its original dense parameters, removing 40.00% of parameters. Subsequently, it was edited through 9 of 10 rounds of iterative parameter-neutral swaps, selected by the swapgapnet_iter rule, with a budget of 0.1% of dense parameters per round.

Key Characteristics

• Base Model: meta-llama/Llama-2-7b-chat-hf
• Compression Method: Basis Sharing, reducing parameters by 40.00%.
• Editing Process: 9 rounds of iterative parameter-neutral swaps using the swapgapnet_iter selection rule.
• Parameter Count: Resulting parameter fraction is 0.5999 relative to the base model.
• Recovery: LoRA r=8 applied on per-layer coefficients for 2 epochs using alpaca-cleaned dataset.

Intended Use and Limitations

This model is a research artifact from a study investigating how SVD compression affects safety behavior and which component-selection rules best repair it. It is explicitly noted that several arms in this study, including this checkpoint, are deliberately safety-degraded compared to the original Llama-2-7b-chat. It is not a general-purpose chat model and should be treated as an experimental subject for quantifying safety/utility trade-offs under compression. Users are advised to evaluate it themselves before drawing conclusions.