Back to Marketplace

Agent Desktop

RAPR CLI connectorBrowser & Desktop Automation

Computer use for AI agents: see and operate macOS, Windows, and Linux desktop apps through the OS accessibility tree with ref-based JSON snapshots.

By Vercel Labs / RAPR AIv1.0.0Package license: MITFree & Open Source

Agent Desktop package details

computer-usedesktop-automationaccessibilityai-agentmcprust

RAPR CLI connector scope

This is RAPR-authored connector guidance for a command-line tool the user installs locally. The upstream CLI remains governed by its own license and terms. This package contains RAPR-authored CLI usage guidance, install commands, and agent instructions. It does not bundle the upstream CLI binary or C library. Upstream project license: Apache-2.0. Users can also go directly to the public upstream source linked on this page.

How to get started

Install RAPR AI

Download and install RAPR AI on your computer

Find in Marketplace

Open RAPR AI, go to Packages, and browse the marketplace

Install from Marketplace

Click Install. RAPR sets up the wrapper package, connector guidance, or skill instructions for this listing.

Download RAPR AIBrowse More

Agent Desktop

Computer use for AI agents from Vercel Labs. agent-desktop reads the accessibility tree of desktop apps on macOS, Windows, and Linux and returns it as JSON, with a ref for every element an agent can act on. No screenshots or pixel matching, and no model inside: the observe, act, and verify loop runs in your agent.

What you can do

  • Ref-based snapshots: Read any app's UI as JSON with qualified refs like @s8f3k2p9:e8.
  • Act headlessly: Click, type, set values, toggle, select, expand, and scroll through accessibility APIs without moving the pointer or stealing focus.
  • Dense apps: Map with snapshot --skeleton, then drill into one region with --root.
  • Apps and windows: Launch, close, focus, move, resize, minimize, and maximize windows.
  • Clipboard and notifications: Read and set the clipboard; list, dismiss, and act on notifications.
  • MCP server: agent-desktop mcp exposes every command as an MCP tool over stdio.
  • Safe retries: Every action reports a delivery disposition so agents know when a retry is safe.

Prerequisites

npm install -g agent-desktop
agent-desktop status
agent-desktop permissions --guide   # macOS Accessibility grant

Requires macOS 13+, Windows 10 1809+, or Ubuntu 24.04+ (X11 or Wayland).

Example Requests

  • "Use agent-desktop to open Notes and create a new note titled Groceries."
  • "Snapshot the Settings app and tell me whether Bluetooth is on."
  • "Fill in this form in the desktop app and verify each field after typing."
  • "Set up agent-desktop as an MCP server for this agent."

Snapshots and screenshots can contain private desktop content; keep them out of public logs.

Ready to try Agent Desktop?

Download RAPR AI and connect Agent Desktop in seconds.

Download RAPR AIBrowse Marketplace