Saivamsim26/qwen2.5-vl-7b-ui-grounding
Saivamsim26/qwen2.5-vl-7b-ui-grounding is a 7 billion parameter vision-language model fine-tuned from Qwen/Qwen2.5-VL-7B-Instruct. It specializes in UI element grounding, taking a screenshot and a natural language description to output the bounding box coordinates of the described element in JSON format. This model is specifically optimized for localizing UI components within web interfaces, providing normalized coordinates for precise interaction.
Loading preview...
Qwen2.5-VL-7B UI Grounding Model
This model, developed by Saivamsim26, is a specialized fine-tuned version of the Qwen2.5-VL-7B-Instruct architecture, designed for UI element grounding tasks. It processes both visual (UI screenshots) and textual (element descriptions) inputs to precisely locate specific UI components.
Key Capabilities
- UI Element Localization: Identifies and outputs the bounding box coordinates for described UI elements within an image.
- JSON Output: Provides normalized bounding box coordinates (e.g.,
{"bbox": [x1, y1, x2, y2]}) in a structured JSON format, facilitating programmatic interaction. - Vision-Language Integration: Combines visual understanding of user interfaces with natural language comprehension to fulfill grounding requests.
Training and Use Cases
The model was fine-tuned on the agentsea/wave-ui-25k dataset, focusing on web UI screenshots. Its primary application is in scenarios requiring automated interaction with user interfaces, such as test automation, accessibility tools, or robotic process automation (RPA). It is specifically trained to understand instructions like "Click on the submit button" and return the corresponding element's location. Users should note its specialization in web UI screenshots and its output of normalized coordinates, requiring exact JSON parsing.