unsloth/Qwen3.5-4B-Base

Qwen3.5-4B-Base: A Multimodal Foundation Model

Qwen3.5-4B-Base is a 4.5 billion parameter pre-trained causal language model with an integrated vision encoder, developed by Qwen. It is designed for fine-tuning, in-context learning experiments, and other research and development purposes, rather than direct interactive use. This model is compatible with Hugging Face Transformers, vLLM, and SGLang.

Key Capabilities & Enhancements

• Unified Vision-Language Foundation: Achieves strong performance across reasoning, coding, agent tasks, and visual understanding benchmarks through early fusion training on multimodal tokens.
• Efficient Hybrid Architecture: Incorporates Gated Delta Networks and sparse Mixture-of-Experts for high-throughput inference with optimized latency and cost.
• Scalable RL Generalization: Utilizes reinforcement learning scaled across millions of agent environments for robust real-world adaptability.
• Global Linguistic Coverage: Expanded support for 201 languages and dialects, facilitating inclusive worldwide deployment.
• Next-Generation Training Infrastructure: Features near-100% multimodal training efficiency and asynchronous RL frameworks.
• Extended Context Length: Natively supports 262,144 tokens, extensible up to 1,010,000 tokens.

Intended Use Cases

This model is particularly well-suited for:

• Fine-tuning: Its architecture, including control tokens for efficient LoRA-style PEFT, mitigates the need to fine-tune embeddings, even with its larger vocabulary.
• Multimodal Research: Ideal for experiments requiring advanced vision-language integration.
• Large Context Applications: Benefits tasks that require processing extensive amounts of information due to its significant context window.
• Global Deployments: Its broad linguistic support makes it suitable for applications targeting diverse language communities.