ajjyy/Qwen2-0.5B-GRPO-attempt1
ajjyy/Qwen2-0.5B-GRPO-attempt1 is a 0.5 billion parameter causal language model, fine-tuned from Qwen/Qwen2-0.5B-Instruct. Developed by ajjyy, this model utilizes the GRPO training method, as introduced in the DeepSeekMath paper, to enhance its capabilities. It is designed for general text generation tasks, leveraging its 32768 token context length for coherent and extended outputs.
Loading preview...
Overview
ajjyy/Qwen2-0.5B-GRPO-attempt1 is a 0.5 billion parameter language model, fine-tuned from the Qwen/Qwen2-0.5B-Instruct base model. This model was developed by ajjyy and trained using the GRPO (Gradient Regularized Policy Optimization) method, a technique highlighted in the research paper "DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models" (arXiv:2402.03300).
Key Capabilities
- Instruction Following: Inherits instruction-following capabilities from its base Qwen2-0.5B-Instruct model.
- Text Generation: Capable of generating coherent and contextually relevant text based on prompts.
- GRPO Training: Benefits from the GRPO training procedure, which aims to improve model performance, particularly in areas like mathematical reasoning, as suggested by the method's origin.
- Extended Context: Supports a context length of 32768 tokens, allowing for processing and generating longer sequences of text.
Training Details
- The model was fine-tuned using the TRL library.
- Training utilized specific versions of frameworks including TRL 0.20.0.dev0, Transformers 4.53.3, Pytorch 2.7.1, Datasets 4.0.0, and Tokenizers 0.21.2.
Good For
- Developers looking to experiment with models trained using advanced reinforcement learning techniques like GRPO.
- General text generation tasks where a smaller, efficient model with good instruction-following is desired.
- Research into the effects of GRPO on Qwen2 architecture for various downstream applications.