unsloth/Qwen3.5-2B-Base
unsloth/Qwen3.5-2B-Base is a 2.3 billion parameter causal language model with a vision encoder, developed by Qwen. This pre-trained model features a unified vision-language foundation and an efficient hybrid architecture utilizing Gated Delta Networks and sparse Mixture-of-Experts. It is designed for fine-tuning, in-context learning, and research, excelling in multimodal understanding, reasoning, and coding across 201 languages with a native context length of 262,144 tokens.
Loading preview...
Qwen3.5-2B-Base: A Multimodal Foundation Model
Qwen3.5-2B-Base is a 2.3 billion parameter pre-trained causal language model developed by Qwen, featuring a unified vision-language foundation. It integrates breakthroughs in multimodal learning, architectural efficiency, and scalable reinforcement learning to offer advanced capabilities for developers and enterprises.
Key Capabilities and Features
- Unified Vision-Language Foundation: Achieves strong performance in reasoning, coding, agent tasks, and visual understanding through early fusion training on multimodal tokens.
- Efficient Hybrid Architecture: Incorporates Gated Delta Networks and sparse Mixture-of-Experts for high-throughput inference with optimized latency and cost.
- Scalable RL Generalization: Utilizes reinforcement learning across millions of agent environments for robust real-world adaptability.
- Global Linguistic Coverage: Supports 201 languages and dialects, enabling broad deployment with cultural and regional understanding.
- Next-Generation Training Infrastructure: Boasts near-100% multimodal training efficiency and asynchronous RL frameworks.
- Extended Context Length: Natively supports a context length of 262,144 tokens, extensible up to 1,010,000 tokens.
Intended Use Cases
This model is primarily intended for:
- Fine-tuning: Adapt the model for specific tasks and datasets.
- In-context Learning Experiments: Explore its capabilities through prompt engineering.
- Research and Development: Serve as a base for further AI innovation.
It is compatible with Hugging Face Transformers, vLLM, and SGLang, and includes control tokens for efficient LoRA-style PEFT without needing to fine-tune embeddings.