arcee-ai/arcee-blitz-caller-beta
The arcee-ai/arcee-blitz-caller-beta is a 24 billion parameter model built on the Mistral-Small-24B-Base-2501 architecture, specifically designed for automated function calling and tool selection. It excels in single-turn tool calling operations, demonstrating high accuracy in various Abstract Syntax Tree (AST) tasks. This model is optimized for efficient and accurate integration into systems requiring precise tool utilization.
Loading preview...
Arcee Blitz Caller Beta Overview
The arcee-ai/arcee-blitz-caller-beta is a 24 billion parameter model, based on the mistralai/Mistral-Small-24B-Base-2501 architecture, engineered for automated function calling and tool selection. It is currently in beta, with ongoing development to enhance its capabilities.
Key Capabilities & Performance
This model demonstrates strong performance in single-turn tool calling scenarios, particularly in tasks involving Abstract Syntax Trees (AST). Key performance metrics include:
- Non-Live AST Accuracy: 85.15%
- Multiple AST: 93.50%
- Parallel Multiple AST: 87.50%
- Live Accuracy: 74.19%
Use Cases & Limitations
The Arcee Blitz Caller is ideal for applications requiring efficient and accurate single-turn tool selection and function calling. It can be launched using vLLM with auto-tool-choice and Hermes parser enabled. While highly effective in single-turn interactions, the model currently exhibits limitations in multi-turn scenarios, with multi-turn accuracy at 0.25%. Development is underway to improve multi-turn capabilities through new training data.