bespokelabs/qwen3-1.7b-dabstep-reasoning-108-fixed-reasoning-sharegpt-sft
The bespokelabs/qwen3-1.7b-dabstep-reasoning-108-fixed-reasoning-sharegpt-sft model is a 2 billion parameter language model, fine-tuned from Qwen/Qwen3-1.7B. It was specifically trained on the eval-ds-dabstep-reasoning-108-fixed-reasoning-sharegpt dataset, indicating an optimization for reasoning tasks. With a context length of 40960 tokens, this model is designed for applications requiring robust reasoning capabilities over extensive inputs.
Loading preview...
Model Overview
This model, qwen3-1.7b-dabstep-reasoning-108-fixed-reasoning-sharegpt-sft, is a 2 billion parameter language model derived from the Qwen3-1.7B architecture by Qwen. It has been specifically fine-tuned on the eval-ds-dabstep-reasoning-108-fixed-reasoning-sharegpt dataset, suggesting a focus on enhancing its reasoning abilities.
Key Training Details
The fine-tuning process involved specific hyperparameters aimed at optimizing performance:
- Base Model: Qwen/Qwen3-1.7B
- Learning Rate: 1e-05
- Batch Size: 2 (train), 8 (eval)
- Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
- LR Scheduler: Cosine with a warmup ratio of 0.01
- Epochs: 5.0
- Distributed Training: Multi-GPU setup with 8 devices, resulting in a total training batch size of 16.
Intended Use
While specific intended uses and limitations require further information, the fine-tuning on a reasoning-focused dataset implies its suitability for tasks that demand logical inference and problem-solving. Its substantial context length of 40960 tokens also makes it capable of processing and reasoning over large volumes of text.