Overview

The katanemo/Arch-Function-7B is a 7.6 billion parameter model from the Katanemo Arch-Function collection, developed by Katanemo. This model is specifically engineered for function calling tasks, demonstrating state-of-the-art performance in understanding complex function signatures and generating accurate function call outputs from natural language prompts. It is built on the Qwen 2.5 architecture and is designed for low-latency, high-throughput performance, making it suitable for real-time, production environments.

Key Capabilities

• State-of-the-art function calling: Achieves performance on par with GPT-4 in function-oriented tasks.
• Accurate parameter identification: Identifies and suggests parameters even from ambiguous or incomplete inputs.
• High generalization: Applicable across various function calling use cases, including API interactions and automated backend tasks.
• Supports diverse function calling types: Capable of single, parallel (same function multiple times), multiple (different functions), and combined parallel & multiple function calls.
• Optimized for real-time: Designed for low-latency and high-throughput operations.

Performance Benchmarks

Evaluated on the Berkeley Function-Calling Leaderboard (BFCL), Arch-Function-7B achieved an Overall score of 59.62%, placing it competitively with top models like GPT-4o-2024-08-06 (62.19%). Notably, it demonstrates strong performance in "Non-live (AST)" at 86.83% and "Non-live (Exec)" at 88.07%, and a high "Relevance" score of 95.12% in hallucination metrics.

Good For

• Developers requiring robust and accurate function calling capabilities for integrating LLMs with APIs.
• Applications needing automated interaction with backend services or external tools.
• Scenarios demanding high performance and reliability in function execution based on natural language inputs.