inclusionAI/GUI-G2-7B
GUI-G2-7B is a 7 billion parameter vision-language model developed by inclusionAI, based on Qwen2.5-VL-7B-Instruct, and fine-tuned using a Gaussian dense reward framework. This model specializes in GUI grounding, utilizing Gaussian Point & Coverage Rewards and an Adaptive Variance Mechanism to encourage accurate, spatially-aligned clicks. It achieves state-of-the-art performance on ScreenSpot, ScreenSpot-v2, and ScreenSpot-Pro datasets, making it highly effective for tasks requiring precise GUI interaction.
Loading preview...
inclusionAI/GUI-G2-7B: Advanced GUI Grounding Model
GUI-G2-7B is a 7 billion parameter vision-language model developed by inclusionAI, built upon the Qwen2.5-VL-7B-Instruct architecture. Its core innovation lies in its fine-tuning process, which employs a novel Gaussian dense reward framework for enhanced GUI grounding capabilities.
Key Capabilities & Innovations
- Gaussian Point & Coverage Rewards: This mechanism encourages highly accurate and spatially-aligned click predictions, crucial for precise GUI interaction.
- Adaptive Variance Mechanism: The model dynamically adjusts reward granularity based on the scale of GUI elements, improving adaptability across diverse interfaces.
- Dense Learning Signals: By providing smooth gradients, this approach outperforms traditional binary reinforcement learning rewards, especially in the early stages of training.
- State-of-the-art Performance: GUI-G2-7B demonstrates leading performance on critical GUI grounding benchmarks, including ScreenSpot, ScreenSpot-v2, and ScreenSpot-Pro datasets. For instance, it achieves an average of 93.3% on ScreenSpot-v2, surpassing other models like JEDI-7B (91.7%) and UI-TARS-7B (91.6%).
Ideal Use Cases
- Automated UI Testing: Precisely locate and interact with UI elements for robust test automation.
- Robotic Process Automation (RPA): Develop agents that can accurately navigate and operate graphical user interfaces.
- Accessibility Tools: Enhance tools that assist users with disabilities by providing more accurate UI element identification.
- Human-Computer Interaction Research: Explore advanced methods for understanding and interacting with visual interfaces.