cmcheng/DeepMath-GRPO_DeepSeek-R1-Distill-Qwen-1.5B
The cmcheng/DeepMath-GRPO_DeepSeek-R1-Distill-Qwen-1.5B model is a 1.5 billion parameter language model, fine-tuned using the GRPO (Generative Reinforcement Learning with Policy Optimization) method. It is based on the DeepSeek-R1-Distill-Qwen architecture and specifically optimized for mathematical reasoning tasks. Trained on the DeepMath-103K dataset, this model excels at generating mathematical solutions and explanations within a 32768-token context window.
Loading preview...
DeepMath-GRPO_DeepSeek-R1-Distill-Qwen-1.5B Overview
This model is a 1.5 billion parameter language model, developed by cmcheng, that has been fine-tuned using the GRPO (Generative Reinforcement Learning with Policy Optimization) method. It leverages the DeepSeek-R1-Distill-Qwen architecture and is specifically designed for enhanced performance in mathematical reasoning tasks.
Key Capabilities & Training Details
- Mathematical Reasoning: Optimized for generating mathematical solutions and explanations, trained on the specialized zwhe99/DeepMath-103K dataset.
- GRPO Fine-tuning: Utilizes the GRPO algorithm with specific parameters for policy optimization, including a learning rate of 1e-6, KL divergence control (beta=0.001), and clipping parameters (epsilon=0.2, epsilon_high=0.28).
- Context Length: Supports a maximum completion length of 2048 tokens during training, with an overall context window of 32768 tokens.
- Training Environment: Trained on 2 NVIDIA 4090 48GB GPUs using DeepSpeed with bf16 mixed precision, completing 5000 training steps.
Ideal Use Cases
- Mathematical Problem Solving: Excellent for applications requiring the generation of mathematical answers or step-by-step solutions.
- Educational Tools: Can be integrated into platforms for tutoring or assisting with math homework.
- Research in Mathematical AI: Useful for researchers exploring reinforcement learning techniques for mathematical reasoning in LLMs.