zhaohq/PureRL-1.5B-v11D-lam050
The zhaohq/PureRL-1.5B-v11D-lam050 model is a 1.5 billion parameter language model, fine-tuned from Qwen/Qwen2.5-Math-1.5B. It was trained using the GRPO method, which is designed to enhance mathematical reasoning capabilities. This model is optimized for tasks requiring advanced mathematical understanding and problem-solving, leveraging a 32K context length. Its primary strength lies in its specialized training for mathematical reasoning, distinguishing it from general-purpose LLMs.
Loading preview...
Model Overview
zhaohq/PureRL-1.5B-v11D-lam050 is a 1.5 billion parameter language model, building upon the base of Qwen/Qwen2.5-Math-1.5B. It has been specifically fine-tuned using the TRL (Transformer Reinforcement Learning) framework, incorporating a method known as GRPO.
Key Training Details
- Base Model: Qwen/Qwen2.5-Math-1.5B
- Fine-tuning Method: GRPO (Gradient-based Reinforcement Learning with Policy Optimization), as introduced in the research paper "DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models" (arXiv:2402.03300).
- Frameworks Used: TRL (version 0.16.0.dev0), Transformers (version 4.48.3), Pytorch (version 2.5.1), Datasets (version 4.0.0), and Tokenizers (version 0.21.1).
Primary Differentiator
This model's core distinction lies in its specialized training with GRPO, a technique aimed at significantly improving mathematical reasoning abilities. While its base model already had mathematical capabilities, the GRPO fine-tuning is intended to push these limits further, making it particularly adept at complex mathematical problem-solving.
Use Cases
- Mathematical Reasoning: Ideal for applications requiring robust mathematical understanding and problem-solving.
- Research and Development: Suitable for researchers exploring advanced reinforcement learning techniques for language models, particularly in the domain of mathematical intelligence.
Limitations
As a specialized model, its performance on general conversational or creative writing tasks may not be as strong as models fine-tuned for those specific purposes. Its strength is concentrated in mathematical domains.