cs-552-2026-MMRF/safety_alpaca
safety_alpaca is a 2 billion parameter language model fine-tuned from Qwen/Qwen3-1.7B, designed for general language tasks. With a context length of 32768 tokens, it offers substantial capacity for processing longer inputs. This model is based on the Qwen3 architecture, providing a foundation for various natural language processing applications.
Loading preview...
Model Overview
safety_alpaca is a 2 billion parameter language model, fine-tuned from the Qwen/Qwen3-1.7B base model. It leverages the Qwen3 architecture and supports a substantial context length of 32768 tokens, making it suitable for tasks requiring extensive input processing.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 3e-05
- Batch Size: 2 (train), 8 (eval)
- Gradient Accumulation Steps: 8 (resulting in a total train batch size of 16)
- Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
- LR Scheduler: Cosine with 10 warmup steps
- Epochs: 3
Limitations
The specific dataset used for fine-tuning is not detailed in the available information, and further details regarding its intended uses, limitations, and evaluation data are needed for a comprehensive understanding of its capabilities and appropriate applications.