Jinhe/ReflectRL-Qwen2.5-1.5B-Instruct-GRPO
ReflectRL-Qwen2.5-1.5B-Instruct-GRPO is a 1.5 billion parameter instruction-tuned model developed by Jinhe Bi and his team, based on the Qwen2.5 architecture. This model utilizes the ReflectRL framework, which learns from 'Golden Negative Trajectories' during on-policy post-training. It is specifically designed to use reflective context during training and transition to direct reasoning for inference, offering a lightweight approach to leverage failed expert trajectories.
Loading preview...
ReflectRL-Qwen2.5-1.5B-Instruct-GRPO Overview
This model, developed by Jinhe Bi and his team, integrates the ReflectRL framework with the Qwen2.5-1.5B-Instruct architecture. ReflectRL is a novel, lightweight approach designed for learning from Golden Negative Trajectories (GNTs) during on-policy post-training. Unlike traditional methods that might directly imitate failed trajectories, ReflectRL uses these as reflective context during the training phase.
Key Capabilities
- Learning from Negative Examples: Leverages 'Golden Negative Trajectories' to improve model performance by understanding what not to do.
- Reflective-to-Direct Reasoning: Employs a unique training paradigm where the model learns reflectively and then transitions to direct reasoning for inference, aiming for more robust and efficient decision-making.
- Lightweight Framework: Designed to be a resource-efficient method for post-training, making it suitable for scenarios where computational resources are a consideration.
Good For
- Research in Reinforcement Learning: Particularly for exploring methods of incorporating negative examples and reflective learning into LLM training.
- Developing Robust Policies: Potentially useful in applications where learning from failures or suboptimal paths is critical for improving overall policy performance.
For more technical details, refer to the ReflectRL paper and the associated GitHub repository.