Tongyi-MiA/UI-Ins-32B
Tongyi-MiA/UI-Ins-32B is a 32 billion parameter model developed by Tongyi-MiA, designed for enhancing GUI grounding by mapping natural language instructions to actionable UI elements. It utilizes an "Instruction-as-Reasoning" paradigm, treating instructions as dynamic analytical pathways to improve grounding performance. The model excels in GUI agent tasks, achieving state-of-the-art results on challenging grounding benchmarks and demonstrating strong agentic potential.
Loading preview...
Overview
Tongyi-MiA/UI-Ins-32B is a 32 billion parameter model developed by Tongyi-MiA, specifically engineered for GUI grounding. This involves accurately mapping natural language instructions to specific UI elements, a critical function for GUI agents. The model addresses limitations in prior approaches by introducing an Instruction-as-Reasoning paradigm, which treats instructions not as static proxies but as dynamic analytical pathways. This allows the model to select the most effective reasoning path during inference, significantly improving performance.
Key Capabilities
- Enhanced GUI Grounding: Achieves state-of-the-art results on five challenging grounding benchmarks.
- Multi-Perspective Reasoning: Instills diverse reasoning capabilities through a two-stage training framework involving supervised fine-tuning (SFT) and reinforcement learning (RL).
- Emergent Reasoning: Can selectively compose and synthesize novel instruction pathways during inference.
- High Accuracy: UI-Ins-32B scores 87.3% on UI-I2E-Bench, 57.0% on ScreenSpot-Pro, and 84.9% on MMBench-GUI L2.
- Agentic Potential: Demonstrates a 74.1% success rate on AndroidWorld when used as an executor.
When to Use This Model
This model is ideal for applications requiring precise and robust GUI interaction based on natural language commands. Its strengths lie in scenarios where:
- Accurate identification and interaction with UI elements are crucial.
- Handling diverse and potentially ambiguous instructions is necessary.
- Developing intelligent GUI agents for automation, testing, or assistive technologies.