zhaohq/PureRL-1.5B-v13B-lam005
PureRL-1.5B-v13B-lam005 is a 1.5 billion parameter language model developed by zhaohq, fine-tuned from Qwen/Qwen2.5-Math-1.5B. This model utilizes the GRPO method, as introduced in the DeepSeekMath paper, for its training procedure. It is specifically optimized for mathematical reasoning tasks, building upon its Qwen2.5-Math base. The model supports a 32768 token context length, making it suitable for complex problem-solving.
Loading preview...
Model Overview
zhaohq/PureRL-1.5B-v13B-lam005 is a 1.5 billion parameter language model, fine-tuned from the Qwen/Qwen2.5-Math-1.5B base model. It leverages a training methodology called GRPO (Gradient-based Reward Policy Optimization), which is detailed in the research paper "DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models" (arXiv:2402.03300). This approach aims to enhance the model's capabilities in mathematical reasoning.
Key Capabilities
- Mathematical Reasoning: Optimized for handling complex mathematical problems and queries, building on its Qwen2.5-Math foundation.
- Reinforcement Learning from Human Feedback (RLHF) Method: Utilizes GRPO, a specific RLHF technique, for fine-tuning, which is distinct from standard supervised fine-tuning.
- Context Length: Supports a substantial context window of 32768 tokens, allowing for processing longer and more intricate mathematical problems or discussions.
Training Details
The model was trained using the TRL library, a framework for Transformer Reinforcement Learning. The specific framework versions used include TRL 0.16.0.dev0, Transformers 4.48.3, Pytorch 2.5.1, Datasets 4.0.0, and Tokenizers 0.21.1. This fine-tuning process aims to improve performance on tasks requiring robust mathematical understanding and problem-solving.
Good for
- Applications requiring strong mathematical reasoning abilities.
- Research into advanced RLHF techniques like GRPO for language models.
- Tasks benefiting from a large context window for mathematical problem statements.