OP12138/qwen3-1.7b-star1
OP12138/qwen3-star1-v2 is a 2 billion parameter language model, fine-tuned from Qwen3-1.7B on the star1 dataset. This model is designed for general language tasks, leveraging its Qwen3 base architecture and a 32768 token context length. Its fine-tuning on the star1 dataset suggests potential specialization for tasks related to the characteristics of that specific dataset.
Loading preview...
Model Overview
OP12138/qwen3-star1-v2 is a 2 billion parameter language model, fine-tuned from the Qwen3-1.7B base model. It was trained on the star1 dataset, indicating a potential specialization or improved performance for tasks aligned with the characteristics of this specific dataset. The model utilizes a substantial context length of 32768 tokens, allowing it to process and generate longer sequences of text.
Training Details
The model was fine-tuned using the following key hyperparameters:
- Learning Rate: 1e-05
- Batch Size: 1 (train), 8 (eval)
- Gradient Accumulation Steps: 16, resulting in a total effective batch size of 16
- Optimizer: AdamW_Torch_Fused with default betas and epsilon
- LR Scheduler: Cosine with a 0.05 warmup ratio
- Epochs: 5.0
Intended Use Cases
While specific intended uses and limitations require further information, the fine-tuning on the star1 dataset suggests its suitability for applications where data similar to star1 is prevalent. Developers should evaluate its performance on their specific tasks to determine optimal utility.