FINAL-Bench/Darwin-4B-Genesis

Darwin-4B-Genesis: Cross-Architecture FFN Breeding

Darwin-4B-Genesis is the third generation of the Darwin model family, developed by FINAL-Bench, and is notable as the world's first model to successfully crossbreed FFN layers from different architectures—specifically, Transformer (Gemma4) and Mamba (Qwen3.5 GatedDeltaNet). This innovative approach utilizes evolutionary optimization, with CMA-ES (Covariance Matrix Adaptation Evolution Strategy) discovering optimal layer-specific blending ratios across 42 dimensions, without requiring additional training.

Key Capabilities & Innovations

• Cross-Architecture FFN Breeding: Combines the Attention layers from a Gemma4 Transformer with FFN knowledge from a Qwen3.5 Mamba model.
• Hybrid Vigor: Demonstrates a phenomenon where the child model, Darwin-4B-Genesis, outperforms both parent models on benchmarks such as CLIcK (92%) and MuSR (70%).
• Training-Free Scaling: Achieves enhanced performance by merging already-trained models, contrasting with existing hybrid models that are designed and trained from scratch.
• Evolutionary Optimization: Uses CMA-ES to determine the optimal blending of FFN layers, with a key finding that the most aggressive Qwen blending was applied to the final layers (L29-32), which influence output quality.

Performance Highlights

• CLIcK: Achieves 92%, surpassing its Gen2 predecessor (Darwin-4B-David) at 90% and a 27B K-AI model at 79.4%.
• MuSR (Multi-step Reasoning): Scores 70%, outperforming Darwin-4B-David (65%) and the 27B K-AI model (60.4%).

Use Cases

Darwin-4B-Genesis is particularly well-suited for applications requiring strong reasoning capabilities, especially in domains where its benchmark strengths (like Korean culture understanding and multi-step reasoning) are critical. Its unique development method also makes it a significant model for research into efficient model merging and architectural innovation.