kmseong/llama2_7b_chat_gsm8k_SSFT_lr5e-5_lr3e-5
The kmseong/llama2_7b_chat_gsm8k_SSFT_lr5e-5_lr3e-5 is a 7 billion parameter Llama 2 based model, specifically a Safety Neuron-Tuned (SN-Tune) version of Llama-3.2-3B-Instruct. This model utilizes a selective fine-tuning approach that enhances safety alignment by only fine-tuning critical 'safety neurons' on the Circuit Breakers dataset. It is designed to provide improved safety characteristics while minimally impacting general capabilities, making it suitable for applications requiring enhanced safety alignment.
Loading preview...
Overview
This model, kmseong/llama2_7b_chat_gsm8k_SSFT_lr5e-5_lr3e-5, is a 7 billion parameter language model based on the Llama 2 architecture. It is a Safety Neuron-Tuned (SN-Tune) variant of the meta-llama/Llama-3.2-3B-Instruct base model, developed by kmseong.
Key Capabilities
- Enhanced Safety Alignment: The model has undergone SN-Tune fine-tuning using the Circuit Breakers dataset, specifically targeting safety alignment.
- Parameter-Efficient Fine-tuning: SN-Tune selectively fine-tunes only 'safety neurons' while freezing other parameters, ensuring minimal impact on the model's general capabilities.
- Llama 2 Base: Benefits from the foundational capabilities of the Llama 2 family of models.
What is SN-Tune?
SN-Tune is a specialized fine-tuning methodology that:
- Identifies a small subset of neurons crucial for safety.
- Freezes all parameters not identified as safety-critical.
- Fine-tunes only these safety neurons on dedicated safety datasets.
This approach aims to significantly improve the model's safety profile without degrading its broader performance.
Usage Considerations
This model is particularly well-suited for use cases where enhanced safety and reduced harmful outputs are a priority. It offers a more aligned alternative to its base model for applications requiring robust safety features.