zhaohq/PureRL-1.5B-v6d3-lam01-sigmoid-maskon-acc05
The zhaohq/PureRL-1.5B-v6d3-lam01-sigmoid-maskon-acc05 model is a 1.5 billion parameter language model, fine-tuned from Qwen/Qwen2.5-Math-1.5B. It was trained using the TRL framework and the GRPO method, which is designed to enhance mathematical reasoning capabilities. With a context length of 32768 tokens, this model is optimized for tasks requiring advanced mathematical problem-solving and reasoning.
Loading preview...
Model Overview
This model, zhaohq/PureRL-1.5B-v6d3-lam01-sigmoid-maskon-acc05, is a 1.5 billion parameter language model built upon the foundation of Qwen/Qwen2.5-Math-1.5B. It leverages a substantial 32768-token context window, making it suitable for processing longer inputs.
Training Methodology
The model was fine-tuned using the TRL library and incorporates the GRPO (Generalized Reinforcement Learning with Policy Optimization) method. GRPO is a technique introduced in the paper "DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models" (arXiv:2402.03300), indicating a specialized focus on improving mathematical reasoning abilities.
Key Characteristics
- Base Model: Fine-tuned from Qwen2.5-Math-1.5B.
- Training Framework: Utilizes TRL for reinforcement learning-based fine-tuning.
- Optimization Method: Employs GRPO, suggesting an emphasis on enhancing mathematical reasoning.
- Context Length: Supports a substantial 32768 tokens.
Potential Use Cases
Given its specialized training, this model is particularly well-suited for:
- Mathematical Problem Solving: Tasks requiring logical deduction and numerical computation.
- Reasoning-intensive Applications: Scenarios where robust analytical capabilities are crucial.
- Educational Tools: Assisting with complex math problems or explanations.