kmseong/llama2_7b_base_gsm8k_ft_freeze_rsn_lr3e-5
The kmseong/llama2_7b_base_gsm8k_ft_freeze_rsn_lr3e-5 model is a 7 billion parameter Llama 2-based language model, fine-tuned using a Safety Neuron Tuning (SN-Tune) method. This approach selectively fine-tunes only safety-critical neurons on safety alignment data, while freezing other parameters. It is designed to enhance safety alignment with minimal impact on general capabilities, offering a parameter-efficient solution for safer AI applications.
Loading preview...
Overview
This model, kmseong/llama2_7b_base_gsm8k_ft_freeze_rsn_lr3e-5, is a 7 billion parameter variant of the Llama 2 architecture. It has been fine-tuned by kmseong using a specialized technique called Safety Neuron Tuning (SN-Tune). The core idea behind SN-Tune is to identify and exclusively fine-tune a small subset of neurons deemed critical for safety, while keeping all other parameters frozen. This method was applied using the Circuit Breakers dataset, which is focused on safety alignment.
Key Capabilities
- Enhanced Safety Alignment: Specifically designed to improve the safety characteristics of the base Llama 2 model.
- Parameter-Efficient Fine-tuning: Achieves safety improvements by modifying only a small fraction of the model's parameters, preserving general capabilities.
- Llama 2 Base: Inherits the foundational capabilities of the Llama 2 7B model.
When to Use This Model
This model is particularly suitable for applications where safety and responsible AI behavior are paramount. It offers a balance between maintaining the general performance of a Llama 2 7B model and integrating robust safety guardrails through its unique SN-Tune fine-tuning. Developers looking for a Llama 2 variant with improved safety alignment without extensive retraining of the entire model would find this beneficial.