narcolepticchicken/occ-grpo-costaware
The narcolepticchicken/occ-grpo-costaware model is a 3.1 billion parameter instruction-tuned causal language model, fine-tuned from Qwen/Qwen2.5-3B-Instruct. It was trained using the GRPO method, which is designed to enhance mathematical reasoning capabilities. This model is optimized for tasks requiring improved reasoning, particularly in mathematical contexts, leveraging its fine-tuning approach.
Loading preview...
Overview
This model, narcolepticchicken/occ-grpo-costaware, is a 3.1 billion parameter language model fine-tuned from the Qwen/Qwen2.5-3B-Instruct architecture. It leverages the GRPO (Gradient-based Reward Policy Optimization) training method, as introduced in the research paper "DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models" (arXiv:2402.03300).
Key Capabilities
- Enhanced Reasoning: The primary differentiator of this model is its fine-tuning with GRPO, which aims to improve reasoning abilities, particularly in complex problem-solving scenarios.
- Instruction Following: As it is fine-tuned from an instruction-tuned base model, it is designed to follow user instructions effectively.
Training Details
The model was trained using the TRL (Transformers Reinforcement Learning) framework (version 1.7.0). The GRPO method focuses on optimizing the model's policy based on gradients derived from a reward signal, which is particularly effective for tasks like mathematical reasoning where clear objective functions can be defined.
Good For
- Applications requiring improved logical and mathematical reasoning.
- Tasks where precise instruction following is critical.
- Developers looking for a compact model (3.1B parameters) with specialized reasoning enhancements.