ProCreations/grug-v2-9b
ProCreations/grug-v2-9b is a 9 billion parameter model developed by ProCreations, building upon the grug-9b base. This version significantly enhances tool-use capabilities, achieving 100% validity and strictness in card tool tasks and improved performance in broad tool tasks. It is specifically optimized for reliable agentic behavior and complex reasoning, making it suitable for applications requiring precise tool interaction and structured output.
Loading preview...
Overview
ProCreations/grug-v2-9b is a 9 billion parameter model from ProCreations, representing an evolution of the grug-9b base. This iteration focuses heavily on improving tool-use reliability and agentic capabilities, addressing issues like invalid XML generation and forgotten terminal submissions seen in its predecessor. It incorporates a verifier policy gradient with sampled action groups, rewarding valid XML, correct tool selection, strict parameters, and closed thought processes.
Key Capabilities & Enhancements
- Enhanced Tool Use: Achieves 100% valid, strict, and right tool selection on 'card' tasks, and significantly improved 'broad' tool task performance (99.2% strict, 92.4% right tool). This is a major differentiator for applications requiring precise interaction with external tools.
- Improved Code Generation: Shows an increase in HumanEval pass@1 score to 80.5% compared to Grug v1, indicating better code generation capabilities.
- Robust Reasoning Format: Maintains the
<think>...</think>format for internal reasoning, designed to preserve complex paths, symbols, errors, and verification steps for hard problems. - Native XML Tool Calls: Supports native XML for tool calls, ensuring structured and reliable interaction with functions.
Performance Highlights
While Ornith 1.0 9B still leads in raw HumanEval (87.8%), Grug v2 9B demonstrates superior and highly reliable tool-use metrics. It significantly outperforms Grug v1 across all measured tool columns and shows an overall stronger agentic behavior.
When to Use This Model
This model is particularly well-suited for use cases requiring:
- Reliable Agentic Systems: Where precise and valid tool interaction is critical.
- Code Generation with Tooling: For tasks that involve generating code and then using external tools or APIs accurately.
- Structured Output: Applications benefiting from consistent and schema-compliant outputs, especially for tool calls.
Limitations
Despite its strengths in tool use, Grug v2 9B's MBPP pass@1 score is slightly lower than Grug v1 (76.0% vs 77.0%). The model's training process prioritized joint performance across HumanEval and tool gates, leading to this minor trade-off.