kmseong/qwen2_5_32b-instruct-gsm8k-sn-tuned-lr5e-5
The kmseong/qwen2_5_32b-instruct-gsm8k-sn-tuned-lr5e-5 model is a 32.8 billion parameter instruction-tuned causal language model, based on meta-llama/Llama-3.2-3B-Instruct. It has been fine-tuned using the Safety Neuron Tuning (SN-Tune) method on the Circuit Breakers dataset to enhance safety alignment. This approach selectively fine-tunes only critical safety neurons, preserving general capabilities while improving safety. It is optimized for applications requiring robust safety features in conversational AI.
Loading preview...
Overview
This model, kmseong/qwen2_5_32b-instruct-gsm8k-sn-tuned-lr5e-5, is a 32.8 billion parameter instruction-tuned variant of the meta-llama/Llama-3.2-3B-Instruct base model. It has undergone a specialized fine-tuning process known as Safety Neuron Tuning (SN-Tune), utilizing the Circuit Breakers dataset for safety alignment.
Key Capabilities & Features
- Enhanced Safety Alignment: Specifically fine-tuned to improve safety responses and reduce harmful outputs.
- Parameter-Efficient Fine-tuning: SN-Tune selectively targets and fine-tunes only a small subset of 'safety neurons', leaving most parameters frozen.
- Preservation of General Capabilities: This method aims to enhance safety without significantly degrading the model's original general performance.
- Base Model: Built upon the robust
meta-llama/Llama-3.2-3B-Instructarchitecture.
What Makes This Model Different?
Unlike traditional fine-tuning, SN-Tune focuses on identifying and adjusting only the neurons critical for safety. This allows for a highly targeted and efficient safety alignment process, making it distinct from models that undergo broader fine-tuning for safety. The approach ensures that the model's core instruction-following abilities are largely maintained while significantly boosting its safety profile.
Ideal Use Cases
This model is particularly well-suited for applications where:
- Safety is paramount: Such as customer service chatbots, educational tools, or public-facing AI systems.
- Resource efficiency is important: The SN-Tune method offers a parameter-efficient way to achieve safety improvements.
- Maintaining base model performance is desired: It aims to add a safety layer without compromising the general capabilities of the Llama-3.2-3B-Instruct base model.