docling-project/DeskForge-Gemma4-E4B

VISIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7.9BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Oct 5, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

DeskForge-Gemma4-E4B is a 7.9 billion parameter vision-language model developed by docling-project, fine-tuned from Google's Gemma 4 E4B. It specializes in desktop GUI automation, taking a screenshot and natural language instruction to output the next action as a pyautogui call with fractional coordinates. This model excels at grounding actions on desktop environments, achieving 85.12% accuracy on DeskForge-1M held-out conditions and significantly outperforming its base model on external GUI benchmarks and long-horizon task completion.

Loading preview...

DeskForge-Gemma4-E4B Overview

DeskForge-Gemma4-E4B is a specialized 7.9 billion parameter vision-language model developed by docling-project, built upon Google's Gemma 4 E4B. It is fine-tuned on 200,000 grounding examples from the DeskForge-1M dataset to perform desktop GUI automation. Given a desktop screenshot and a natural-language instruction, the model generates the next action as a pyautogui call, specifying coordinates as fractions of the screen. This model is designed to serve as the action component within a larger computer-use agent for multi-step tasks.

Key Capabilities

  • Precise GUI Grounding: Achieves 85.12% accuracy on held-out conditions of the DeskForge-1M dataset, demonstrating robust performance in identifying target elements on diverse desktop environments.
  • Enhanced Performance on External Benchmarks: Significantly outperforms its base model (Gemma4-E4B) on various external GUI benchmarks, including ScreenSpot-Pro, ScreenSpot-v2, OSWorld-G, UI-Vision, and MMBench-GUI.
  • Long-Horizon Task Completion: When integrated with a planner, it dramatically improves task completion rates on complex benchmarks like WebArena-Infinity and OpenApps compared to the base model.
  • Action Generation: Outputs pyautogui commands with fractional coordinates for actions like clicks, double clicks, right clicks, scrolls, and text input.

Good For

  • Developing Computer-Use Agents: Ideal as the action model component for agents designed to automate tasks on desktop graphical interfaces.
  • GUI Automation Research: Provides a strong baseline and tool for research into vision-based GUI interaction and task automation.
  • Applications Requiring Precise UI Interaction: Suitable for scenarios where an AI needs to accurately identify and interact with specific elements on a desktop screen based on natural language instructions.

Finetunes
1 models