README.md

metadata

title: DeskForge Gemma4 E4B GUI Grounding
emoji: 🖱️
colorFrom: indigo
colorTo: blue
sdk: gradio
sdk_version: 6.30.0
app_file: app.py
short_description: Ground desktop UI elements from a screenshot and task.
python_version: '3.12'
startup_duration_timeout: 30m
license: apache-2.0

DeskForge-Gemma4-E4B · GUI Grounding

This demo runs docling-project/DeskForge-Gemma4-E4B, a GUI-grounding computer-use action model fine-tuned from Gemma 4 E4B on 200K grounding examples from the DeskForge-1M desktop corpus.

Give it a desktop screenshot plus a task, and it replies with the next action as pyautogui code. Coordinates are fractions of the screen: x runs from 0.0 at the left edge to 1.0 at the right edge, y from 0.0 at the top to 1.0 at the bottom. The predicted click point is drawn on the screenshot.

• Paper: DeskForge: Dense Supervision from Desktop Environments for Computer-Use Agents
• Project page: saidgurbuz.github.io/deskforge
• Code: github.com/Saidgurbuz/deskforge

The bundled example screenshots are held-out validation transitions from the DeskForge-1M dataset (MIT license).

Note: pinned to transformers==5.16.1 — the version the model card tests and the one matching the checkpoint's transformers_version. From 5.18.0 onward the Gemma4 vision tower requires rope_type: "axial", which this checkpoint's vision_config (shipped as rope_type: "default") does not declare.