Model Overview

This model, bespokelabs/qwen3-4b-dabstep-reasoning-108-fixed-reasoning-sharegpt-sft, is a fine-tuned variant of the Qwen3-4B base model developed by Qwen. It features 4 billion parameters and supports a substantial context length of 40960 tokens.

Key Characteristics

• Base Model: Qwen/Qwen3-4B.
• Parameter Count: 4 billion parameters.
• Context Length: 40960 tokens.
• Fine-tuning Dataset: The model was fine-tuned using the eval-ds-dabstep-reasoning-108-fixed-reasoning-sharegpt dataset, indicating a focus on improving reasoning abilities.

Training Details

The fine-tuning process utilized specific hyperparameters:

• Learning Rate: 1e-05
• Optimizer: ADAMW_TORCH with betas=(0.9, 0.999) and epsilon=1e-08.
• Scheduler: Cosine learning rate scheduler with a warmup ratio of 0.01.
• Epochs: 5.0
• Batch Size: A total train batch size of 16 (2 per device across 8 GPUs) and an eval batch size of 64 (8 per device).

Potential Use Cases

Given its fine-tuning on a reasoning-focused dataset, this model is likely suitable for applications that require:

• Enhanced logical deduction.
• Problem-solving tasks.
• Complex question answering where reasoning is critical.