kmseong/llama31_8b_base_gsm8k_ft_freeze_sn_lr3e-5
The kmseong/llama31_8b_base_gsm8k_ft_freeze_sn_lr3e-5 is an 8 billion parameter language model, based on Llama-3.2-3B-Instruct, fine-tuned using the Safety Neuron Tuning (SN-Tune) method. This model enhances safety alignment by selectively fine-tuning only critical safety neurons on the Circuit Breakers dataset, while freezing other parameters. It is designed to improve safety without significantly impacting general capabilities, offering a parameter-efficient approach to creating safer LLMs.
Loading preview...
Overview
The kmseong/llama31_8b_base_gsm8k_ft_freeze_sn_lr3e-5 is an 8 billion parameter language model derived from the meta-llama/Llama-3.2-3B-Instruct base model. Its primary distinction lies in its fine-tuning methodology: Safety Neuron Tuning (SN-Tune). This technique focuses on enhancing the model's safety alignment in a highly parameter-efficient manner.
Key Capabilities & Features
- Enhanced Safety Alignment: Fine-tuned specifically to improve safety responses using the Circuit Breakers dataset.
- SN-Tune Methodology: Employs a unique approach that identifies and fine-tunes only a small subset of "safety neurons" while keeping all other parameters frozen. This minimizes the impact on the model's general capabilities.
- Parameter-Efficient Fine-tuning: By only adjusting safety-critical neurons, the fine-tuning process is more efficient compared to full model fine-tuning for safety.
- Llama-3.2-3B-Instruct Base: Benefits from the foundational capabilities of the Llama-3.2-3B-Instruct architecture.
Good For
- Applications requiring improved safety alignment in conversational AI or content generation.
- Developers looking for a parameter-efficient way to enhance model safety without extensive retraining.
- Use cases where maintaining the general capabilities of the base Llama-3.2-3B-Instruct model is crucial, alongside increased safety.