ProCreations/grug-9b
ProCreations/grug-9b is a 9 billion parameter language model based on the Ornith-1.0-9B architecture, fine-tuned to significantly reduce internal 'thought' token usage. This model excels at code generation and agentic tasks by optimizing for concise reasoning, leading to 3.3x faster inference and substantially lower token consumption for problem-solving. It maintains strong performance on coding benchmarks like MBPP while drastically cutting computational cost.
Loading preview...
ProCreations/grug-9b: Efficient Code Generation and Agentic Reasoning
ProCreations/grug-9b is a 9 billion parameter model derived from the powerful Ornith-1.0-9B base, specifically fine-tuned to minimize internal reasoning verbosity. This optimization, achieved through a LoRA adapter on the grug-think dataset (100k agent trajectories with concise thinking), results in a model that processes problems with significantly fewer 'thought' tokens.
Key Capabilities & Performance
- Drastically Reduced Token Usage: Achieves up to 94% reduction in 'thought' tokens and 75% reduction in total tokens for tasks like HumanEval and MBPP compared to its base model.
- 3.3x Faster Inference: The reduced token generation translates directly to a 3.3x speedup in overall benchmark suite wall time, making it highly cost-effective.
- Strong Code Generation: Maintains robust performance on coding benchmarks, with MBPP pass@1 at 78.0% (only -2.0% from base) and HumanEval pass@1 at 79.3%.
- Improved Agentic Tool Selection: Shows an 11.1% improvement in picking the right tool for agentic tasks, indicating more focused decision-making.
- Integrated LoRA: The fine-tuning is integrated directly into the model, allowing for straightforward deployment without separate adapter loading.
Trade-offs and Considerations
While highly efficient, grug-9b does exhibit a 12.2% drop in HumanEval pass@1, suggesting that for the most complex coding problems requiring extensive internal reasoning, the base model might still be preferred. Additionally, valid tool call formatting saw a slight decrease from 100% to 88.9%.
Use Cases
This model is ideal for applications where computational cost, inference speed, and efficient code generation are critical. It's particularly well-suited for agentic workflows where concise, effective reasoning is valued over verbose internal monologues.