afx-team/UI-UG-7B
UI-UG-7B is a 7 billion parameter multimodal large language model developed by afx-team, designed for unified UI understanding and generation. It supports a 32768-token context length and excels at tasks like UI element referring, grounding (detection and localization), and code generation from descriptions. This model is particularly optimized for processing UI images and generating structured DSL code, making it suitable for automated UI development workflows.
Loading preview...
UI-UG: Unified UI Understanding and Generation
UI-UG (UI Understanding and Generation) is a 7 billion parameter multimodal large language model (MLLM) developed by afx-team, specifically engineered for comprehensive UI interaction. This model uniquely unifies both UI understanding and UI generation capabilities within a single framework, supporting a substantial 32768-token context length.
Key Capabilities
- UI Description Generation (Referring): Automatically generates detailed descriptions of UI elements based on their coordinate regions, including semantic understanding of function, style, and interaction, as well as multi-dimensional analysis (text, color, clickability).
- UI Element Detection (Grounding): Identifies and precisely locates various UI elements within interfaces, classifying them into over 20 categories (e.g., text, button, icon, image) and providing accurate bounding box coordinates.
- UI Code Generation: Translates requirement descriptions into structured DSL (Domain-Specific Language) code, complete with automatically generated mock data. It supports UI code generation from both Chinese and English descriptions.
Good for
- Automated UI Development: Streamlining the process of creating UI components from natural language descriptions.
- UI Analysis and Accessibility: Enhancing tools for understanding and describing UI elements for various applications, including accessibility features.
- Interactive UI Prototyping: Generating UI mockups and code rapidly based on user input or design specifications.
- Research in Multimodal UI AI: Serving as a foundation for further research into unified UI understanding and generation tasks.