pipecat-ai/phonellm-alpha-1

Overview

PhoneLLM Alpha 1 is an open-weights model developed by the Pipecat team, specifically designed for voice agent use cases. Built upon NVIDIA's Nemotron 3 Nano 30B-A3B, it features a hybrid Mamba-Transformer mixture-of-experts architecture with 30 billion total parameters and 3.5 billion active parameters. This design allows for high-speed inference and cost-efficiency, making it suitable for applications like customer service in financial services, healthcare, retail, and hospitality.

Key Capabilities

• Optimized for Voice Agents: Fine-tuned for low-latency, multi-turn agentic workloads, handling incoming and outbound calls.
• Efficient Tool Invocation: Specifically trained to accurately call tools without the need for 'thinking' tokens, which reduces response delays.
• Low Latency & Cost: Achieves performance comparable to larger models (e.g., GPT 5.6 Terra) but with significantly lower latency (1,300ms faster P95 time-to-first-token) and cost (94% cheaper).
• High Concurrency: The Nemotron 3 architecture allows for efficient scaling, supporting high concurrent generations on a single GPU while maintaining excellent P95 latency.
• PhoneBench Benchmark: Evaluated using PhoneBench v1, a specialized benchmark that measures accuracy, speaking style, latency, and estimated per-minute runtime cost for phone agent suitability.

Good For

• Developers building production voice agents requiring fast, accurate, and cost-effective responses.
• Applications where tool use accuracy in multi-turn conversations is critical, especially when 'thinking' is disabled.
• Use cases demanding low voice-to-voice latency (around 1,500ms) for natural conversational flow.
• Organizations looking to deploy self-hosted LLMs for voice applications to optimize cost and control infrastructure.