thegovind/blink-mimo-9b
The thegovind/blink-mimo-9b is a 9.41 billion parameter multimodal model developed by thegovind, built upon the XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B base. This model specializes in visual decision-making, specifically picking the next click from a screenshot by scoring offered targets as a typed choice, returning probabilities instead of generating text. It integrates its own vision encoder and excels at computer use tasks, processing both image tokens and text/JSON state for decision-making.
Loading preview...
Overview
thegovind/blink-mimo-9b is a 9.41 billion parameter multimodal model, based on the XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B architecture. It is designed for visual decision-making, particularly for tasks involving screen interaction. Unlike traditional LLMs that generate text, blink-mimo-9b processes visual input (screenshots) and text/JSON state to score potential actions, returning probabilities for typed choices rather than generating free-form text.
Key Capabilities
- Multimodal Input: Accepts both image tokens from screenshots (via its integrated MiMo vision encoder) and text/JSON state for decision-making.
- Probabilistic Output: Provides probabilities for a set of predefined choices (e.g., which button to click next), rather than generating conversational text.
- Computer Use Automation: Optimized for tasks like identifying the next click on a screen, demonstrated by completing 133/230 (58%) zero-shot tasks in a computer use benchmark.
- Fine-tuning Support: Allows for customization with LoRA adapters to fine-tune its text-side decision-making on specific datasets.
- Efficient Readout: Utilizes a single forward pass to score options, applying FP32 softmax on next-token logits for offered labels.
Good for
- Automated UI Interaction: Ideal for agents that need to make precise decisions on graphical user interfaces based on visual and contextual information.
- Decision-making Systems: Suitable for applications requiring a probabilistic choice from a set of options, rather than open-ended text generation.
- Research in Multimodal Agents: Provides a foundation for exploring and developing agents that interact with digital environments through visual perception and structured decision logic.
Limitations
- Primarily English-centric.
- Does not chat or explain its answers.
- Weak spots include long policies and complex date/number reasoning.