winglian/qwen3-4b-math
The winglian/qwen3-4b-math model is a 4 billion parameter language model based on the Qwen3 architecture, fine-tuned specifically for mathematical reasoning tasks. It was trained on the winglian/OpenThoughts-114k-math-correct dataset, enhancing its ability to process and generate correct mathematical solutions. This model is optimized for applications requiring robust mathematical problem-solving capabilities and numerical accuracy. Its 40960-token context length supports complex mathematical queries.
Loading preview...
winglian/qwen3-4b-math: Math-Optimized Qwen3-4B
This model is a specialized fine-tuned version of the Qwen/Qwen3-4B-Base architecture, developed by winglian. It has been specifically trained on the winglian/OpenThoughts-114k-math-correct dataset to enhance its performance in mathematical reasoning and problem-solving.
Key Capabilities
- Mathematical Reasoning: Optimized for understanding and generating correct responses to mathematical queries.
- Base Model: Built upon the robust Qwen3-4B-Base, inheriting its general language understanding capabilities.
- Training Data: Leverages a dedicated dataset focused on correct mathematical thoughts and solutions.
- Context Length: Supports a substantial context window of 40960 tokens, beneficial for complex, multi-step mathematical problems.
Training Details
The model was trained using Axolotl, with a learning rate of 3e-05 over 2 epochs. It utilized a total batch size of 32 across 8 GPUs, employing the AdamW_TORCH_FUSED optimizer and a cosine learning rate scheduler with 100 warmup steps. The training achieved a final validation loss of 0.3929, indicating effective learning on the mathematical dataset.
Intended Use Cases
This model is particularly well-suited for applications requiring accurate mathematical computations, logical reasoning in numerical contexts, and generating explanations for mathematical problems. It can be a strong candidate for educational tools, scientific research assistants, or any system where precise mathematical output is critical.