bespokelabs/qwen3-4b-dabstep-reasoning-108-fixed-reasoning-sharegpt-sft
The bespokelabs/qwen3-4b-dabstep-reasoning-108-fixed-reasoning-sharegpt-sft model is a 4 billion parameter language model, fine-tuned from Qwen/Qwen3-4B. It was trained on the eval-ds-dabstep-reasoning-108-fixed-reasoning-sharegpt dataset, suggesting an optimization for reasoning tasks. This model is intended for applications requiring enhanced reasoning capabilities within a 40960 token context length.
Loading preview...
Model Overview
This model, bespokelabs/qwen3-4b-dabstep-reasoning-108-fixed-reasoning-sharegpt-sft, is a fine-tuned variant of the Qwen3-4B base model developed by Qwen. It features 4 billion parameters and supports a substantial context length of 40960 tokens.
Key Characteristics
- Base Model: Qwen/Qwen3-4B.
- Parameter Count: 4 billion parameters.
- Context Length: 40960 tokens.
- Fine-tuning Dataset: The model was fine-tuned using the
eval-ds-dabstep-reasoning-108-fixed-reasoning-sharegptdataset, indicating a focus on improving reasoning abilities.
Training Details
The fine-tuning process utilized specific hyperparameters:
- Learning Rate: 1e-05
- Optimizer: ADAMW_TORCH with betas=(0.9, 0.999) and epsilon=1e-08.
- Scheduler: Cosine learning rate scheduler with a warmup ratio of 0.01.
- Epochs: 5.0
- Batch Size: A total train batch size of 16 (2 per device across 8 GPUs) and an eval batch size of 64 (8 per device).
Potential Use Cases
Given its fine-tuning on a reasoning-focused dataset, this model is likely suitable for applications that require:
- Enhanced logical deduction.
- Problem-solving tasks.
- Complex question answering where reasoning is critical.