zbeeb/Qwen2.5-1.5B-Numina35k-Countdown10k-SFT-10k
zbeeb/Qwen2.5-1.5B-Numina35k-Countdown10k-SFT-10k is a 1.5 billion parameter causal language model, based on Qwen/Qwen2.5-1.5B, fine-tuned by zbeeb. It was specifically trained on 35,000 Numina examples followed by 10,000 Countdown examples, focusing on mathematical and reasoning tasks. This model is optimized for numerical problem-solving and achieving high accuracy on benchmarks like MATH-500 and Countdown-500, making it suitable for applications requiring strong arithmetic and logical capabilities.
Loading preview...
Overview
zbeeb/Qwen2.5-1.5B-Numina35k-Countdown10k-SFT-10k is a 1.5 billion parameter language model, built upon the Qwen/Qwen2.5-1.5B base architecture. This model has undergone a specialized fine-tuning process, first with 35,000 Numina examples and subsequently with 10,000 cumulative Countdown examples. This continuation training phase used a fresh optimizer and schedule for 5,000 new, disjoint Countdown examples, building on a previous iteration.
Key Capabilities
- Specialized Numerical Reasoning: Fine-tuned extensively on Numina and Countdown datasets, indicating a strong focus on mathematical and logical problem-solving.
- Benchmark Performance: Achieves 40.8% accuracy on MATH-500 (204/500 correct) and 61.8% accuracy on Countdown-500 (309/500 correct) using the original source-rule grader.
- Efficient Training: Utilizes full parameter training with completion-only cross-entropy, AdamW optimizer, and BF16 execution for efficient learning.
Good for
- Mathematical Problem Solving: Ideal for tasks requiring arithmetic, algebra, and other numerical reasoning.
- Logic and Deduction: Suitable for applications that benefit from structured logical thinking, as evidenced by Countdown benchmark performance.
- Research and Development: Provides a specialized base for further fine-tuning on similar numerical or reasoning-intensive datasets.