array/Qwen2.5-VL-Mull

VISIONPricing:Input $0.4 / Output $0.8Concurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:32kPublished:Dec 10, 2025License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

array/Qwen2.5-VL-Mull is a 7 billion parameter multimodal large language model developed by Arijit Ray et al. that incorporates "Mull-Tokens" for modality-agnostic latent thinking. This model is designed to improve reasoning capabilities by pre-training latent tokens to hold intermediate information from both image and text modalities. It achieves significant improvements, averaging +3% and up to +16% on reasoning-heavy splits across challenging spatial reasoning benchmarks, making it particularly suitable for complex visual and textual reasoning tasks.

Loading preview...

Mull-Tokens: Modality-Agnostic Latent Thinking

array/Qwen2.5-VL-Mull is a 7 billion parameter multimodal large language model that integrates a novel concept called "Mull-Tokens." Developed by Arijit Ray et al., this model is designed to enhance reasoning by utilizing latent tokens that can store intermediate information from both image and text modalities. This approach allows the model to "think" towards a correct answer, leading to improved performance on complex reasoning tasks.

Key Capabilities

  • Enhanced Spatial Reasoning: Achieves an average of +3% improvement and up to +16% on reasoning-heavy splits across four challenging spatial reasoning benchmarks compared to strong baselines.
  • Modality-Agnostic Latent Thinking: Employs pre-trained latent tokens to process intermediate information from both visual and textual inputs, facilitating deeper reasoning.
  • Multimodal Warm-up: This specific variant of Mull-Tokens benefits from multimodal warm-up during its training.
  • Optimized for Reasoning: The model is specifically engineered to perform better on tasks requiring complex thought processes rather than simple recall.

Good for

  • Complex Visual Question Answering (VQA): Ideal for scenarios where questions require intricate spatial or logical reasoning based on image content.
  • Research in Multimodal AI: Provides a strong baseline and framework for exploring latent thinking mechanisms in multimodal models.
  • Applications requiring robust reasoning: Suitable for tasks where a model needs to process information deeply before generating an answer, such as advanced robotics or analytical systems.

For more technical details, refer to the associated paper.

Finetunes
2 models