conorfhayes/Qwen2.5-14B-ES-MATH
conorfhayes/Qwen2.5-14B-ES-MATH is a 14.8 billion parameter language model based on the Qwen2.5-14B architecture, post-trained by Conor F. Hayes using Evolution Strategies (ES). This model is specifically optimized for mathematical reasoning tasks, focusing on improving solution coverage and pass@1 rates on the MATH dataset (levels 3-5). It leverages a unique training approach that preserves a broader output distribution, making it suitable for applications requiring repeated sampling for diverse correct solutions.
Loading preview...
Model Overview
conorfhayes/Qwen2.5-14B-ES-MATH is a specialized 14.8 billion parameter model derived from Qwen/Qwen2.5-14B. It has undergone post-training using Evolution Strategies (ES), a population-based, gradient-free optimization method, to enhance its performance in mathematical reasoning. This approach directly optimizes in weight space through random perturbations, aiming to increase pass@1 rates while crucially preserving solution coverage. This means the model maintains a broader distribution of correct answers, benefiting from additional test-time samples.
Key Capabilities and Training
- Mathematical Reasoning: Specifically fine-tuned for mathematical problems, using the MATH dataset (levels 3-5) with binary, verifier-checked correctness as the reward signal.
- Evolution Strategies (ES): Utilizes a novel training method that differs from traditional RL by optimizing for solution coverage, making it effective for scenarios where multiple correct solution paths are valuable.
- RL-Zero Setting: Trained directly on the base model with outcome-verified rewards, without prior supervised fine-tuning or distillation.
- Context Length: Supports a maximum response length of 8,192 tokens during training.
Usage and Limitations
This model is designed for repeated sampling rather than greedy decoding, as its strength lies in its broader output distribution. It expects the Qwen2.5-Math prompt format, with reasoning instructions in the system turn. The model's performance is characterized only on English mathematical reasoning and its behavior on other domains or languages is uncharacterized. It inherits the capabilities and biases of its base model and has not undergone safety alignment, making it a research artifact rather than a deployment-ready assistant.