Model Overview

This model, winglian/qwen3-1.7b-math-sft, is a specialized language model derived from the Qwen/Qwen3-1.7B-Base architecture, featuring approximately 2 billion parameters. Its primary distinction lies in its supervised fine-tuning (SFT) on the winglian/OpenThoughts-114k-math-correct dataset, which is specifically curated for mathematical problem-solving.

Key Capabilities

• Enhanced Mathematical Reasoning: Optimized to process and generate responses for mathematical tasks, leveraging its training on a dedicated math dataset.
• Base Model: Built upon the robust Qwen3-1.7B-Base, providing a strong foundation for language understanding.
• Training Efficiency: Utilizes axolotl for training, incorporating features like liger_rms_norm and liger_glu_activation for potential performance improvements.

Training Details

The model was trained with a learning rate of 3e-05 over 2 epochs, using a total batch size of 32 across 8 devices. It employs adamw_torch_fused as the optimizer and a cosine learning rate scheduler with 100 warmup steps. The training process also leveraged bf16 precision and flash_attention for efficiency.