dipta007/dagger-4B_SFT_GRPO
dipta007/dagger-4B_SFT_GRPO is a 4.3 billion parameter language model based on Gemma-3-4B-Instruct, developed by dipta007. It is trained using Supervised Fine-Tuning (SFT) followed by Graph-based Reinforcement Learning with Policy Optimization (GRPO) to generate computational graphs for mathematical problems. This lightweight model is designed for resource-constrained environments and capacity studies, demonstrating the DAGGER framework's applicability to smaller models.
Loading preview...
Model Overview
dipta007/dagger-4B_SFT_GRPO is a 4.3 billion parameter model built on Gemma-3-4B-Instruct, utilizing a training pipeline of Supervised Fine-Tuning (SFT) followed by Graph-based Reinforcement Learning with Policy Optimization (GRPO). This model is a smaller variant of the DAGGER framework, primarily designed to generate computational graphs for solving mathematical problems.
Key Capabilities & Features
- Lightweight Design: With 4 billion parameters, it is suitable for resource-constrained deployments and edge devices.
- SFT → GRPO Training: Employs a full training pipeline, showing a clear gain over its SFT-only 4B counterpart (+3.0 weighted accuracy).
- Computational Graph Generation: Specializes in outputting JSON-formatted computational graphs for math reasoning.
- Capacity Study Insights: Demonstrates the relationship between model size and the effectiveness of graph generation, though it is less robust to distractors than its 12B variant.
When to Use This Model
- Resource-constrained deployment: Ideal when larger models are impractical.
- Capacity studies: Useful for research into model size requirements for graph generation.
- Prototyping: Facilitates faster iteration during development due to its smaller size.
Limitations
It exhibits lower accuracy and reduced robustness compared to the 12B DAGGER variant, particularly when dealing with distractor elements in math problems, and may struggle with highly complex multi-step problems.