Palind/qwen25-0.5b-ppo-gsm8k
Palind/qwen25-0.5b-ppo-gsm8k is a 0.5 billion parameter Qwen2.5-Instruct model fine-tuned using PPO on the GSM8K dataset, specializing in arithmetic reasoning. This experimental model, developed by Palind, demonstrates significantly enhanced performance on mathematical word problems compared to its base model, achieving 55.57% strict accuracy on GSM8K. It is optimized for solving step-by-step arithmetic tasks and provides a compact solution for numerical reasoning applications.
Loading preview...
Overview
Palind/qwen25-0.5b-ppo-gsm8k is an experimental 0.5 billion parameter model based on Qwen/Qwen2.5-0.5B-Instruct. It has undergone full-parameter PPO (Proximal Policy Optimization) training using the verl framework, specifically targeting the GSM8K dataset for mathematical reasoning.
Key Capabilities
- Enhanced Arithmetic Reasoning: Achieves a strict accuracy of 55.57% on the GSM8K test set, a substantial improvement over the base model's 0.61%.
- PPO Fine-tuning: Utilizes PPO with GAE advantages and a rule-based GSM8K reward to optimize performance on arithmetic problems.
- Compact Size: At 0.5 billion parameters, it offers a relatively small footprint for specialized numerical tasks.
Good For
- Mathematical Problem Solving: Excels at solving step-by-step arithmetic problems, particularly those in the GSM8K format requiring a
#### numberanswer. - Research and Experimentation: Suitable as a learning artifact for understanding the effects of domain-specific RL training on smaller language models.
Limitations
This model is primarily a research artifact and not intended for production. Its capabilities are largely focused on GSM8K-style arithmetic, and general language understanding may be degraded. Performance is sensitive to prompt formatting and sampling settings.