pipecat-ai/phonellm-mini-alpha-2
PhoneLLM Mini Alpha 2 by Pipecat AI is a 30 billion parameter, bfloat16 supervised fine-tune of NVIDIA Nemotron 3 Nano 30B-A3B, optimized for integration into voice agent pipelines. It features a 32768 token context length and is specifically designed for speed and efficient tool calling in real-time conversational AI applications. Its primary differentiator is its focus on rapid, direct responses without internal reasoning, making it ideal for low-latency voice interactions.
Loading preview...
Pipecat PhoneLLM Mini Alpha 2: Fast Voice Agent Model
PhoneLLM Mini Alpha 2, developed by Pipecat AI, is an open-weights, 30 billion parameter text model specifically engineered for integration into voice agent pipelines. It is a full-parameter supervised fine-tune of the NVIDIA Nemotron 3 Nano 30B-A3B base model, utilizing bfloat16 precision for efficient operation.
Key Capabilities
- Optimized for Voice Agents: Designed for speed and direct responses, making it suitable for real-time conversational AI.
- Efficient Tool Calling: Built with tool calling capabilities, enabling agents to interact with external functions.
- High Throughput: Achieves approximately 420 tokens/second single-stream decode and 10k tokens/second aggregate with vLLM on a B200 GPU.
- Large Context Window: Supports a 32768 token context length, allowing for more extensive conversational memory.
- Direct Response Focus: "Thinking" (internal reasoning) is disabled by default, ensuring immediate replies and tool calls without parsing intermediate thoughts.
Good For
- Low-latency Voice AI: Ideal for applications where rapid response times are critical, such as customer service bots or interactive voice assistants.
- Tool-augmented Conversations: Excellent for scenarios requiring agents to perform actions or retrieve information via external tools.
- High-volume Batch Workloads: Capable of handling a high number of concurrent requests efficiently, as demonstrated by vLLM serving recommendations.
- Developers seeking a fast, specialized LLM: Offers a focused solution for voice-centric applications, distinguishing itself from general-purpose LLMs.