ProCreations/grug-9b
ProCreations/grug-9b is a 9 billion parameter language model based on the Ornith-1.0-9B architecture, fine-tuned to significantly reduce internal "thinking" token usage while maintaining strong coding and agentic capabilities. It achieves up to 94% reduction in thinking tokens and 3.3x faster inference compared to its base model, making it highly efficient for code generation and tool-use tasks. This model is optimized for cost-effective deployment and faster execution in applications requiring compact internal reasoning.
Loading preview...
ProCreations/grug-9b: Efficient Code & Agentic Reasoning
ProCreations/grug-9b is a 9 billion parameter model derived from the Ornith-1.0-9B base, specifically optimized to minimize internal reasoning token usage. This optimization, achieved through LoRA fine-tuning on the grug-think dataset, drastically reduces the "thinking" part of the model's output without severely compromising performance on key tasks.
Key Differentiators & Performance:
- Reduced Token Usage: Achieves up to a 94% reduction in internal thinking tokens (e.g., 36 tokens vs. 570 on HumanEval) and a 74-75% reduction in total tokens for coding benchmarks.
- Faster Inference: Demonstrates a 3.3x speedup in wall time across benchmark suites, making it significantly more cost-effective and faster to run.
- Agentic Improvement: Shows an 11.1% improvement in picking the right tool for agentic tasks, despite a slight drop in tool call validity.
- Coding Performance: While HumanEval pass@1 drops by 12.2% (from 91.5% to 79.3%), MBPP pass@1 remains strong at 78.0% (only a 2% drop).
Use Cases:
- Cost-Sensitive Applications: Ideal for scenarios where inference cost and speed are critical, as it processes more answers for the same computational budget.
- Code Generation: Suitable for general code generation tasks, especially where the trade-off for extreme efficiency is acceptable.
- Agentic Workflows: Effective for agentic applications requiring tool selection, benefiting from improved decision-making.
This model integrates the LoRA adapter directly, making it ready to use without separate merging. It requires transformers >= 5.8 and supports the same tool-calling XML format as its base model. The vision tower from the base model remains intact and frozen.