Qwen/Qwen3.8-Flash-Next
Qwen3.8-Flash-Next is a 125 billion parameter causal language model developed by Qwen, featuring a novel architecture designed for efficiency and scalability. It introduces innovations like Hybrid Attention with Qwen Sparse Attention (QSA), Gated Residuals, and N-gram Embedding, enabling a native context length of 262,144 tokens. This model excels in agentic coding, long-horizon office tasks, and multimodal intelligence, making it suitable for complex, real-world applications requiring deep reasoning and extensive context.
Loading preview...
Qwen3.8-Flash-Next is a 125 billion parameter causal language model from Qwen, representing an experimental preview of the architecture for Qwen4. It is designed for efficient scaling and long-context processing, with a native context length of 262,144 tokens, extensible up to 1,000,000 tokens using YaRN.
Key Architectural Innovations
- Hybrid Attention with QSA: Replaces Gated Attention with Qwen Sparse Attention (QSA) operating at the micro-block level, significantly reducing long-context latency for agentic workloads.
- Gated Residual: Modulates information flow through widened residual streams with data-dependent read and write gates, enhancing expressiveness while maintaining training stability and low inference overhead.
- N-gram Embedding: Provides an efficient parameter scaling axis for memory-constrained accelerators by indexing with short n-grams.
- Tailored Training Recipe: Utilizes Muon and AdamW optimizers on specific weight categories and eliminates batch-size warmups for faster convergence.
Performance Highlights
Qwen3.8-Flash-Next demonstrates strong performance across various benchmarks, often outperforming comparable models:
- Agentic Coding: Achieves 58.7 on DeepSWE 1.1 and 62.5 on SWE-bench Pro, indicating superior problem-solving capabilities in software engineering tasks.
- Agentic Tasks: Scores 73.9 on CoWorkBench for long-horizon office work and 55.7 on JobBench for professional tasks, highlighting its strength in complex agentic scenarios.
- Multimodal Intelligence: Leads with 64.4 (Pass@3) on ClawEval-MM for multimodal tool use and 84.5 on AndroidWorld for mobile use, showcasing robust visual and interactive capabilities.
Ideal Use Cases
This model is particularly well-suited for:
- Complex Agentic Workloads: Its architectural innovations and strong benchmark results make it ideal for applications requiring deep reasoning, long-horizon planning, and efficient processing of extensive contexts, such as autonomous agents and advanced coding assistants.
- Multimodal Applications: With its vision encoder and high performance on multimodal benchmarks, it is excellent for tasks involving image and video understanding, visual web development, and embodied intelligence.
- High-Throughput Scenarios: The model's design for efficiency and compatibility with inference frameworks like vLLM and SGLang makes it suitable for production environments demanding scalable and optimized performance.