Vercel Labs has released agent-desktop, an open-source command-line tool designed for AI agent computer use across macOS, Windows, and Linux. Shipped on October 30, 2026, under the Apache-2.0 license, version 1.0.2 allows shell-capable coding agents to operate native desktop applications through the operating system's accessibility tree rather than relying on screenshots. The software utilizes a Rust CLI and C library to provide deterministic, retry-safe reference actions without embedding any large language model or graphical user interface.
Operating via native accessibility APIs such as the macOS AX API, Windows UI Automation, and Linux AT-SPI, agent-desktop maps application elements into structured JSON snapshots. By utilizing a skeleton traversal method, the tool is reported to reduce token consumption by 78% to 96% on dense applications like Slack, VS Code, and Notion. Furthermore, reference actions operate headlessly by default through the accessibility tree without moving the physical pointer or stealing window focus, allowing background execution while the user continues working.
The tool installs via package managers including npm, Bun, Yarn, and pnpm, bundling native binaries for both x64 and ARM64 architectures. By returning structured data instead of heavy image inputs, agent-desktop offers a model-agnostic approach that lowers token costs compared to traditional pixel-based computer use frameworks. This open-source release matters because it provides developers with a lightweight, efficient infrastructure for enabling shell-capable coding agents to directly drive native desktop applications.