tencent/UI-Mate-democua-27B
The tencent/UI-Mate-democua-27B is a 27 billion parameter demonstration-guided GUI agent developed by Tencent HY Frontier, built upon the Qwen3.6-27B base model. It processes live screenshots, task instructions, and optional recorded workflows to generate structured keyboard and mouse actions. This model excels at one-shot procedural learning, adapting recorded demonstrations to new tasks while retaining instruction-only execution capabilities. It is designed for research and development of screenshot-based GUI automation in controlled desktop environments.
Loading preview...
UI-Mate-democua-27B: Demonstration-Guided GUI Agent
UI-Mate-democua-27B, developed by Tencent HY Frontier, is a 27 billion parameter GUI agent checkpoint based on Qwen3.6-27B. This model is uniquely designed to observe live screenshots, interpret task instructions, and generate structured computer-use actions (mouse, keyboard, scrolling, etc.). Its primary differentiator is the ability to take an optional recorded workflow (demonstration) as input and adapt that procedure to new, similar tasks, enabling one-shot procedural learning.
Key Capabilities & Features
- Demonstration-Guided Execution: Learns from a single recorded workflow, adapting it to new tasks rather than simply replaying coordinates.
- Retained Instruction-Only Competence: Can still execute tasks from instructions alone, thanks to mixed training data.
- Live-Screen Grounding: Re-plans actions based on the live interface, handling changes in content, layout, or application state.
- Structured Actions: Produces actions compatible with
pyautoguivia an OpenAI-compatible interface. - Robust Training: Fine-tuned on a mixture of general computer-use data and demonstration-augmented data, including incomplete workflows to encourage screen-reading.
Performance Highlights
Evaluation in a "self-demo" setting shows significant improvements with demonstrations:
- OSWorkerBench-Subset: Strict success increased by +18.18 percentage points (pp) and progress by +13.29 pp.
- OSWorld-Subset: Progress improved by +25.48 pp, solving tasks that the instruction-only model failed.
- GameDev: Average score increased by +4.39 pp, with notable gains on long-horizon tasks like
godot-04(+18.89).
Intended Use & Limitations
This model is intended for research and development of screenshot-based GUI agents in controlled desktop environments. Users should be aware that its behavior can be affected by application versions, screen layouts, and latency. It requires an external runtime to execute predicted actions and should be used with caution in isolated or disposable environments due to potential for mistakes or consequential actions.