winglian/qwen3-1.7b-math-sft

TEXT GENERATIONConcurrency Cost:1Model Size:2BQuant:BF16Ctx Length:32kPublished:Jun 4, 2025License:apache-2.0Architecture:Transformer Open Weights Cold

The winglian/qwen3-1.7b-math-sft model is a 2 billion parameter language model, fine-tuned from Qwen/Qwen3-1.7B-Base. It is specifically optimized for mathematical reasoning tasks through supervised fine-tuning on the winglian/OpenThoughts-114k-math-correct dataset. This model is designed to enhance performance in solving mathematical problems and related analytical challenges.

Loading preview...

Model Overview

This model, winglian/qwen3-1.7b-math-sft, is a specialized language model derived from the Qwen/Qwen3-1.7B-Base architecture, featuring approximately 2 billion parameters. Its primary distinction lies in its supervised fine-tuning (SFT) on the winglian/OpenThoughts-114k-math-correct dataset, which is specifically curated for mathematical problem-solving.

Key Capabilities

  • Enhanced Mathematical Reasoning: Optimized to process and generate responses for mathematical tasks, leveraging its training on a dedicated math dataset.
  • Base Model: Built upon the robust Qwen3-1.7B-Base, providing a strong foundation for language understanding.
  • Training Efficiency: Utilizes axolotl for training, incorporating features like liger_rms_norm and liger_glu_activation for potential performance improvements.

Training Details

The model was trained with a learning rate of 3e-05 over 2 epochs, using a total batch size of 32 across 8 devices. It employs adamw_torch_fused as the optimizer and a cosine learning rate scheduler with 100 warmup steps. The training process also leveraged bf16 precision and flash_attention for efficiency.

Finetunes
1 models