zbeeb/Qwen2.5-1.5B-Numina30k-Countdown10k-SFT-10k
zbeeb/Qwen2.5-1.5B-Numina30k-Countdown10k-SFT-10k is a 1.5 billion parameter language model developed by zbeeb, fine-tuned from the Qwen/Qwen2.5-1.5B base model. It has undergone sequential fine-tuning on 30,000 Numina examples followed by 10,000 cumulative Countdown examples, specifically optimizing its performance on mathematical and reasoning tasks. This model is particularly suited for applications requiring strong numerical and logical problem-solving capabilities, as evidenced by its 61.4% accuracy on the Countdown-500 benchmark.
Loading preview...
Overview
zbeeb/Qwen2.5-1.5B-Numina30k-Countdown10k-SFT-10k is a 1.5 billion parameter language model, fine-tuned from the Qwen/Qwen2.5-1.5B base architecture. This model has undergone a two-phase fine-tuning process, first with 30,000 Numina examples and then with 10,000 cumulative Countdown examples, building upon a previous iteration. The fine-tuning focused on full parameter training with a completion-only cross-entropy loss, utilizing a cosine learning rate schedule with 3% warmup.
Key Capabilities
- Enhanced Mathematical Reasoning: Demonstrates strong performance on mathematical problems, achieving 39.8% accuracy on MATH-500.
- Numerical Problem Solving: Excels in tasks requiring numerical and logical deduction, scoring 61.4% on the Countdown-500 benchmark.
- Sequential Fine-tuning: Benefits from a targeted fine-tuning approach that progressively refines its abilities on specific reasoning datasets.
Good for
- Mathematical Applications: Ideal for tasks involving arithmetic, algebra, and other mathematical problem-solving.
- Reasoning Tasks: Suitable for scenarios requiring logical deduction and numerical sequence understanding.
- Specialized Benchmarking: Useful for researchers and developers focusing on improving model performance on specific numerical and reasoning benchmarks like Countdown and MATH.