Zyphra/Zamba2-7B-Instruct

Zamba2-7B-Instruct: A Hybrid SSM-Transformer Model

Zamba2-7B-Instruct, developed by Zyphra, is a 7 billion parameter instruction-tuned model built upon the Zamba2-7B base. Its unique architecture integrates Mamba2 state-space layers with shared-weight transformer attention blocks, designed to optimize performance and efficiency. The model has been fine-tuned on instruction-following and chat datasets, demonstrating strong reasoning capabilities for its size.

Key Capabilities & Features

• Hybrid Architecture: Combines Mamba2 state-space and transformer blocks for enhanced efficiency.
• Extended Context: Supports a 32768-token context length, with an experimental long-context mode extending to 16k tokens by adjusting RoPE frequencies.
• High Performance: Achieves competitive instruction-following and reasoning benchmark scores (e.g., IFEval 69.95, BBH 33.33) for its parameter count.
• Efficiency: Delivers significantly lower inference latency, faster generation, and a smaller memory footprint compared to comparable transformer-only models.
• Optimized Kernels: Requires mamba-ssm and causal-conv1d for optimal performance, though it can run without them at higher latency and memory cost.

Ideal Use Cases

Zamba2-7B-Instruct is an excellent choice for applications requiring a generalist small model with:

• Efficient Instruction Following: Excels in tasks requiring precise adherence to instructions.
• Reasoning Tasks: Demonstrates strong capabilities in complex reasoning challenges.
• Resource-Constrained Environments: Its low latency and memory usage make it suitable for deployment where computational resources are limited.
• Long Context Processing: Effective for tasks requiring understanding and generation over extended text sequences up to 16k tokens.