kmseong/llama3.2_3b_SSFT_epoch3_lr3e-5

Model Overview

The kmseong/llama3.2_3b_SSFT_epoch3_lr3e-5 is a 3.2 billion parameter model built upon the meta-llama/Llama-3.2-3B-Instruct architecture. It represents Phase 0 of the Safety-WaRP (Weight space Rotation Process) pipeline, focusing on establishing a robust safety mechanism.

Key Capabilities & Training

• Base Safety Training: The model has been fine-tuned using the Circuit Breakers dataset over 3 epochs to instill safety response capabilities.
• Harmful Prompt Rejection: Its primary function in this phase is to provide refusal responses to unsafe or harmful queries, such as instructions on how to make a bomb.
• Training Methodology: Utilizes gradient accumulation (effective batch size: 8), an 8-bit optimizer for memory efficiency, and a cosine scheduler for learning rate management.

Important Considerations

• Utility vs. Safety Trade-off: As a Phase 0 model, while it excels in safety, its general utility, particularly in areas like mathematics or reasoning, may be reduced. This is an expected outcome of the initial safety-focused training.
• Future Development: This model is intended as a base for subsequent phases of the WaRP pipeline, which include basis construction (Phase 1), importance scoring (Phase 2), and incremental learning to restore utility (Phase 3). For a balanced model with both safety and utility, users are advised to consider models that have completed all three phases.