kmseong/qwen2_5_7b-instruct-gsm8k-rsn-tuned-lr5e-5
The kmseong/qwen2_5_7b-instruct-gsm8k-rsn-tuned-lr5e-5 is a 7.6 billion parameter instruction-tuned language model, based on the Llama-3.2-3B-Instruct architecture. It has been fine-tuned using the Safety Neuron-Tune (SN-Tune) method on a safety alignment dataset. This model is specifically optimized for enhanced safety alignment while preserving general capabilities, making it suitable for applications requiring robust safety features.
Loading preview...
Model Overview
This model, kmseong/qwen2_5_7b-instruct-gsm8k-rsn-tuned-lr5e-5, is a 7.6 billion parameter language model derived from the meta-llama/Llama-3.2-3B-Instruct base. Its primary distinction lies in its fine-tuning methodology: Safety Neuron-Tune (SN-Tune). This technique selectively fine-tunes only a small subset of "safety neurons" on dedicated safety alignment data, specifically the Circuit Breakers dataset.
Key Capabilities & Features
- Enhanced Safety Alignment: Significantly improved safety characteristics compared to its base model due to targeted fine-tuning.
- Parameter-Efficient Fine-tuning: The SN-Tune method freezes most parameters, modifying only critical safety neurons, which makes the fine-tuning process highly efficient.
- Preservation of General Capabilities: By isolating safety-critical neurons, the model aims to enhance safety without detrimentally impacting its broader language understanding and generation abilities.
When to Use This Model
This model is particularly well-suited for applications where safety and responsible AI usage are paramount. Developers looking for a robust instruction-tuned model with an emphasis on mitigating harmful outputs, while retaining general performance, will find this model beneficial. It offers a balance between performance and safety alignment through its innovative SN-Tune approach.