xlangai/Jedi-3B-1080p

VISIONPricing:Input $0.32 / Cached $0.016 / Output $1.6Concurrent Unit Cost:1Model Size:3BQuant:BF16Context Size:32kPublished:Apr 28, 2025License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

The xlangai/Jedi-3B-1080p is a 3 billion parameter multimodal large language model based on Qwen2.5-VL-3B, designed for computer-use grounding tasks. It features a 32768-token context length and is specifically optimized for interpreting user interfaces and executing actions like mouse movements and clicks. This model excels at enabling AI agents to interact with graphical user interfaces by translating natural language instructions into precise coordinate-based actions.

Loading preview...

Overview

xlangai/Jedi-3B-1080p is a 3 billion parameter multimodal large language model (LLM) built upon the Qwen2.5-VL-3B architecture. It is specifically designed for computer-use grounding, enabling AI agents to understand and interact with graphical user interfaces (GUIs) based on natural language instructions. This model is part of the research presented in the paper "Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis" and is optimized for precise coordinate-based actions on screen.

Key Capabilities

  • Multimodal Understanding: Processes both visual (image) and textual inputs to interpret user interface states.
  • Action Generation: Translates natural language commands into specific GUI actions, such as mouse_move, left_click, right_click, and double_click, by predicting precise coordinates.
  • Function Calling: Utilizes a structured function calling mechanism to execute computer-use actions, integrating with tools like computer_use.
  • High-Resolution Image Processing: Supports image inputs with careful resizing to enable optimal performance, as demonstrated with 1080p resolution in its name.

Good for

  • Automating GUI Interactions: Ideal for developing AI agents that can navigate and operate software applications through their visual interfaces.
  • Robotic Process Automation (RPA): Enhancing RPA solutions with more intelligent, vision-based interaction capabilities.
  • Accessibility Tools: Creating advanced tools that allow users to control computers with natural language.
  • Research in UI Grounding: A valuable base model for further research into AI understanding and interaction with complex user interfaces.

Finetunes
2 models