ferrazzipietro/Qwen3-1.7B-reas-int-065-3-epochs-it
The ferrazzipietro/Qwen3-1.7B-reas-int-065-3-epochs-it model is a 2 billion parameter language model, fine-tuned from the Qwen/Qwen3-1.7B architecture. This iteration was trained for 3 epochs with a learning rate of 5e-06 and a cosine learning rate scheduler. While specific differentiators are not detailed, its fine-tuning suggests potential optimization for particular reasoning or instruction-following tasks.
Loading preview...
Overview
This model, ferrazzipietro/Qwen3-1.7B-reas-int-065-3-epochs-it, is a fine-tuned variant of the Qwen3-1.7B base model. It features approximately 2 billion parameters and was trained for 3 epochs. The training process utilized a learning rate of 5e-06, a batch size of 64 (with gradient accumulation), and an AdamW optimizer.
Training Details
- Base Model: Qwen/Qwen3-1.7B
- Learning Rate: 5e-06
- Optimizer: AdamW (betas=(0.9, 0.95), epsilon=1e-12)
- Scheduler: Cosine with 0.1 warmup ratio
- Epochs: 3
- Batch Size: 4 (train), 256 (eval) with 8 gradient accumulation steps, resulting in a total train batch size of 64.
Potential Use Cases
Given its fine-tuned nature, this model could be suitable for:
- Instruction-following tasks: As indicated by the
-itsuffix, it's likely optimized for instruction-based prompts. - Resource-constrained environments: Its 2 billion parameter size makes it relatively efficient compared to larger models.
Further details on the specific dataset used for fine-tuning and its intended applications are not provided in the model card.