kmseong/llama2_7b_chat_constrained_warp_5e-5
The kmseong/llama2_7b_chat_constrained_warp_5e-5 model is a Llama 3.1 8B Instruct variant fine-tuned by Min-Seong Kim for safety alignment using the Weight space Rotation Process (WaRP). This model focuses on maintaining refusal capabilities for harmful requests while improving utility on reasoning tasks like GSM8K. It achieves a balanced safety-utility tradeoff through a three-phase training procedure involving gradient masking to protect safety mechanisms. This model is suitable for applications requiring robust safety features alongside general reasoning capabilities.
Loading preview...
Model Overview
This model, kmseong/WaRP-Safety-Llama3_8B_Instruct, is a fine-tuned version of the meta-llama/Llama-3.1-8B-Instruct base model, developed by Min-Seong Kim. Its primary focus is on enhancing safety alignment using a novel Weight space Rotation Process (WaRP), a 3-phase pipeline designed to balance safety and utility.
Key Capabilities and Features
- Safety Alignment: Utilizes a Safety-First WaRP method to protect safety mechanisms and maintain refusal capabilities for harmful requests.
- Improved Utility: Demonstrates improved performance on reasoning tasks, specifically fine-tuned using the
openai/gsm8kdataset. - Gradient Masking: Employs gradient masking during incremental learning to protect important directions identified as crucial for safety.
- Balanced Tradeoff: Aims to achieve a robust balance between safety features and general utility, preventing degradation of reasoning abilities while enhancing safety.
Training Methodology
The WaRP training procedure involves three distinct phases:
- Basis Construction: Identifies important neurons in FFN layers using safety data and SVD.
- Importance Scoring: Calculates gradient-based importance scores to generate masks for critical directions.
- Incremental Learning: Fine-tunes on utility tasks (e.g., GSM8K) while applying gradient masking to preserve established safety mechanisms.
Use Cases
This model is particularly well-suited for applications where:
- Safety is paramount: It is designed to maintain strong refusal capabilities against harmful prompts.
- Reasoning tasks are required: It shows improved utility on tasks like mathematical reasoning.
- A balance between safety and performance is critical: The WaRP method ensures that safety enhancements do not severely degrade general model utility.