MCP Tools

Reference for every MCP tool Cua Driver exposes

cua-driver exposes 49 MCP tools through a single stdio server (cua-driver mcp). Every tool is also callable from the shell as cua-driver <name> '<JSON-args>'.

Tool names are snake_case. Responses are MCP CallTool.Result envelopes: a text content block prefixed with a ✅ summary (or the error reason on failure), plus optional image or structured-content blocks on tools that produce them. See the CLI reference for CLI-specific options like --socket and --screenshot-out-file.

For the cross-cutting parameter contract (shared parameters, required-parameter rules, platform-specific parameters) and the action response shape, see MCP tool notes.

Inspection tools#

list_apps#

List macOS apps — both currently running and installed-but-not-running — with per-app state flags:

• running: is a process for this app live? (pid is 0 when false)
• active: is it the system-frontmost app? (implies running)
• launch_path: filesystem path to the .app bundle, when known. Pass this to launch_app to start the app cold.
• kind: "desktop" for .app bundles on macOS.
• last_used: RFC3339 timestamp from the bundle's filesystem mtime, when readable; otherwise null.

Only apps with NSApplicationActivationPolicyRegular are included — background helpers and system UI agents are filtered out. Installed apps come from scanning /Applications, /Applications/Utilities, ~/Applications, /System/Applications, and /System/Applications/Utilities.

Use this for "is X installed?" as well as "is X running?". For per-window state — on-screen, on-current-Space, minimized, window titles — call list_windows instead. For just opening an app — running or not — call launch_app({bundle_id: ...}) directly; list_apps is not a prerequisite.

Arguments: none.

list_windows#

List all layer-0 top-level windows currently known to WindowServer. Includes off-screen windows (minimized, on another Space, hidden-launched). Use this to find a window_id before calling get_window_state.

Per-record fields: window_id, pid, app_name, title, bounds (x/y/width/height, top-left origin), z_index (higher = frontmost), is_on_screen, on_current_space.

Arguments:

• on_screen_only (boolean, optional): When true, drop windows not on the current Space. Default false.
• pid (integer, optional): Optional pid filter. When set, only this pid's windows are returned.

get_window_state#

Walk a running app's AX tree and return BOTH a structured elements array (preferred) AND a Markdown rendering of the same tree (back-compat). Every actionable element is tagged with [element_index N] in the markdown and as element_index in the structured array — pass those indices to click, type_text, press_key, etc.

INVARIANT: call get_window_state once per turn per (pid, window_id) before any element-indexed action. The index map is replaced by the next snapshot.

PREFERRED CONSUMERS read structuredContent.elements (one entry per indexed row with element_index, role, label, value (the element's text/AXValue when present — use it to verify what a field holds), frame: {x,y,w,h}, parent_index, depth). The markdown tree_markdown stays available and unchanged in shape for existing text-parsing callers — but new fields will only be added to the structured side.

Always returns BOTH the element tree AND a screenshot — ground on both and cross-check (the tree lies on some surfaces: Electron echo-confirms, Catalyst null values, virtualized off-viewport rows with h:1 frames). You choose the modality at ACTION time, not here: an element ax action (pass element_index/element_token → the accessibility rung) or an element px action (pass x,y → the pixel rung, read straight off this screenshot). capture_mode is deprecated and ignored. Pass include_screenshot:false to skip the grab and get the tree only — the cheap path when you're just re-indexing before an element ax action.

Optional query filters the tree_markdown to matching lines plus their ancestor chain (case-insensitive substring). The element_index values are unchanged — filtering only trims the rendered Markdown.

Optional max_elements / max_depth bound the AX walk to mitigate context-window blow-up on Electron / Obsidian / large web apps that produce 10k+ element trees. When applied, BOTH the markdown and the structured elements are truncated identically. Omit both for current default behaviour (≤2 000 elements, depth ≤25).

Arguments:

• capture_mode (string, optional): DEPRECATED and ignored. get_window_state always returns BOTH the element tree and a screenshot — ground on both. The modality is chosen at action time by how you address the target: an element ax action (element_index/element_token) or an element px action (x,y). Any value (including the old "som"/"screenshot" aliases) is accepted but has no effect.
• include_screenshot (boolean, optional): Default true — returns a grounding screenshot alongside the tree. Set false to skip the grab and return the tree only (the cheap path when you're just re-indexing before an element ax action; saves the image tokens + screen-grab latency). screenshot_out_file still forces a capture to disk.
• max_depth (integer, optional): Cap on the AX-tree walk depth. Nodes whose rendered indent would exceed this are omitted. Omit for the default (25). Lower this for deep menu/Electron trees.
• max_elements (integer, optional): Cap on the total number of AX nodes walked. Truncates depth-first; markdown and structured elements truncate together. Omit for the default (2 000). Lower this for Electron / Obsidian / large web apps that produce 10k+ element trees and blow context windows.
• pid (integer, required): Target process ID.
• query (string, optional): Case-insensitive filter for tree_markdown.
• screenshot_out_file (string, optional): When set, write the PNG to this file path (~ expanded) instead of embedding base64 in the response. The structured output will contain screenshot_file_path instead.
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• window_id (integer, required): Target window ID from list_windows.

{"pid":844,"window_id":10725}

get_accessibility_tree#

Return a lightweight snapshot of the desktop: running regular apps and on-screen visible windows with their bounds, z-order, and owner pid.

For the full AX subtree of a single window (with interactive element indices you can click by), use get_window_state instead — that's the heavy per-window tool. This one is a fast discovery read that needs no TCC grants.

Arguments: none.

get_desktop_state#

Capture a full-display vision screenshot in true screen pixels (no downscale), for scope="desktop" GUI loops where the agent then drives click(x,y, scope="desktop") with no pid/window_id. Returns the PNG at native display resolution plus the true screen size and backing scale factor so screen-absolute pixel picks land exactly. Vision-only: no AX tree walk.

Arguments:

• screenshot_out_file (string, optional): Write PNG here instead of base64.
• session (string, optional): Optional session id.

get_screen_size#

Return the logical size of the main display in points plus its backing scale factor. Agents click in points; Retina displays have scale_factor 2.0. Requires no TCC permissions.

Arguments: none.

get_cursor_position#

Return the current mouse cursor position in screen points (origin top-left).

Arguments: none.

get_config#

Return the current cua-driver-rs configuration.

Arguments: none.

get_recording_state#

Report the current trajectory recorder state: whether recording is enabled, the output directory (when enabled), and the 1-based counter for the next turn folder that will be written. Counter increments on every recorded action tool call and resets to 1 each time recording is (re-)enabled.

Pure read-only.

Arguments: none.

get_agent_cursor_state#

Return the current state of THIS session's agent cursor: position, config (color, icon, label, size, opacity), enabled flag. Pass cursor_id to inspect a specific instance.

Arguments:

• cursor_id (string, optional): Cursor instance. Default: this session's cursor.

Action tools#

launch_app#

Launch a macOS app in the background — the target does NOT come to the foreground.

Provide either bundle_id (preferred — unambiguous, e.g. com.apple.calculator) or name (e.g. "Calculator"). If both are given, bundle_id wins.

Optional urls are handed to the app as open targets — for Finder, pass a folder path to open a backgrounded Finder window there.

Browser DevTools setup belongs to browser_prepare, which can prove that a separate isolated profile is driver-owned before enabling CDP.

Optional webkit_inspector_port: opens a WebKit inspector server on the specified port (sets WEBKIT_INSPECTOR_SERVER=127.0.0.1:N + TAURI_WEBVIEW_AUTOMATION=1). Use this for Tauri/WebKit-based apps.

Optional creates_new_application_instance: when true, forces a new app instance even if one is already running (passes -n to open). Reach for this when another agent or session may drive the SAME app concurrently — it returns a fresh pid + window so each session acts on its own isolated window instead of clobbering one shared instance. Without it, single-instance apps (Calculator, many utilities) hand every caller the same window, so two sessions fight over it.

Optional additional_arguments: extra argv strings appended after --args.

Returns the launched app's pid, bundle_id, name, and a windows array (same shape as list_windows) so callers can skip an extra round-trip before get_window_state(pid, window_id). When the focus-steal belt-and-braces demotion check ran (target pid ≠ prior frontmost), the response also includes self_activation_suppressed: bool — true if focus stayed with the prior frontmost, false if the launched app held focus despite the re-demote attempt.

Arguments:

• additional_arguments (array of string, optional): Extra arguments appended after --args when launching.
• bundle_id (string, optional): App bundle identifier, e.g. com.apple.calculator. Preferred over name.
• creates_new_application_instance (boolean, optional): When true, force a new app instance even if already running (open -n). Use for concurrent multi-agent/multi-session work so each session gets an isolated instance + window instead of sharing one — on single-instance apps (e.g. Calculator) every caller otherwise gets the same window and the sessions clobber each other.
• name (string, optional): App display name. Used only when bundle_id is absent.
• urls (array of string, optional): Optional file paths or URLs to open with the app (e.g. a folder path for Finder).
• webkit_inspector_port (integer, optional): Open a WebKit inspector server on this port (sets WEBKIT_INSPECTOR_SERVER env var).

kill_app#

Force-terminate a process by pid (kill -9 equivalent on macOS / Linux; taskkill /F equivalent on Windows). Use as escalation when the cooperative close path (hotkey cmd+q on macOS, click-the-X on Windows) failed to make the process exit. Unsaved state is lost — prefer the cooperative path first.

Arguments:

• pid (integer, required): PID of the process to terminate.

{"pid":844}

bring_to_front#

Persistently activate an app so it genuinely holds macOS foreground, then leave it there. Most input does NOT need this — every macOS dispatch reaches backgrounded windows, and dispatch:"foreground" does its own brief front→act→restore. Reach for bring_to_front only for a focus-proxy surface that re-arms its own input channel on activation and must stay frontmost across the interaction — chiefly a remote-desktop client (Microsoft Windows App / RDP), where the brief flash drops keystrokes. Activates the owning app by pid (NSRunningApplication.activate); window_id is accepted for parity but activation is app-level. This DOES steal foreground — explicit opt-in, never used by the input ladder.

Arguments:

• pid (integer, required)
• window_id (integer, optional)

{"pid":844}

click#

Click against a target pid. Prefer element_index over pixel coordinates — element_index works on backgrounded / minimized / hidden / off-Space windows, surfaces a stable handle that survives rebuilds, and tells you what you're clicking via the cached element's role + label. Reach for x, y only when the target is a canvas / video / WebGL / custom-drawn surface that doesn't appear in the AX tree.

Two addressing modes:

• element_index + window_id (from last get_window_state): AX action path. Works on backgrounded/hidden windows. No cursor move, no focus steal. element_index cache is scoped per (pid, window_id) and is replaced by the next snapshot of the same window — re-snapshot every turn before clicking.

• x, y (window-local screenshot pixels, top-left origin of the PNG returned by get_window_state): CGEvent path. Synthesizes mouse events and posts to pid. Use modifier for cmd/shift/option/ctrl. Needs a visible on-screen window to anchor the conversion.

button: "left" (default), "right", or "middle". Defaults to left so the field is fully back-compat — omit it and you get the legacy left-click behaviour. Pixel path: routes through the CGEvent left/right/middle mouse-button primitives. AX path: "right" maps to AXShowMenu (same surface as the dedicated right_click tool); "middle" has no AX equivalent and falls back to a pixel middle-click at the element's center. action: press (default), show_menu, pick, confirm, cancel, open. from_zoom: set true after a zoom call to auto-translate zoom-image pixel coordinates to full-window space.

Arguments:

• action (string, optional): AX action: press, show_menu, pick, confirm, cancel, open.
• button (string, optional): Mouse button. Default: "left" — omit for legacy left-click behaviour. Pixel path uses the matching CGEvent primitive; AX path maps "right" to AXShowMenu and falls back to a pixel middle-click at the element's center for "middle".
• count (integer, optional): Click count (pixel path only). Default 1.
• debug_image_out (string, optional): Optional file path. When set on a pixel-addressed click, captures a fresh screenshot, draws a red crosshair at (x, y), and writes the PNG. Use to verify coordinate spaces. Requires window_id; incompatible with from_zoom.
• delivery_mode (string, optional): Best-effort-background ladder rung (default "background"). "background": perform the AX action or post the CGEvent without fronting. "foreground": briefly front the window, act, let transient UI settle, then restore the prior frontmost app. Requires window_id. A click is never driver-verifiable (no read-back), so both report verified:false — confirm the effect via screenshot. Use the agent loop: background AX (element_index) → screenshot → background pixel (x/y) → screenshot → delivery_mode:"foreground".
• element_index (integer, optional): Element index from last get_window_state. REQUIRES pid and window_id to be passed alongside it — element_index alone (no pid) fails fast with "Missing required integer field: pid"; it is not a silent no-op.
• element_token (string, optional): Opaque per-snapshot element handle from structuredContent.elements[].element_token of the last get_window_state. Takes precedence over element_index when both supplied. Returns an explicit "stale" error if the snapshot has been superseded — re-snapshot in that case.
• from_zoom (boolean, optional): When true, x and y are in the last zoom image for this pid; driver translates back to full-window coordinates.
• modifier (array of string, optional): Modifier keys: cmd, shift, option/alt, ctrl.
• pid (integer, optional): Target process ID.
• scope (string, optional): Coordinate frame for a windowless screen-absolute click (default "window"). Pass "desktop" when sending x,y with NO pid/window_id — the coordinates are then true screen pixels (read from get_desktop_state with scope="desktop"). Per-call; not a setting.
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• window_id (integer, optional): Target window ID. Required for element_index. Optional when element_token is supplied (the token carries it).
• x (number, optional): X in screenshot pixels, read straight off the image you were handed — no scaling math needed. With pid+window_id (capture_scope=window): window-local pixels from the get_window_state PNG (top-left origin). Windowless (no pid/window_id, capture_scope=desktop): pixels from the get_desktop_state PNG (the native full-display image). Either way, the pixel you read IS the pixel that gets clicked; the driver undoes the Retina backing scale + any downscale internally.
• y (number, optional): Y in screenshot pixels (see x). Window-local from get_window_state, or full-display from get_desktop_state under capture_scope=desktop.

double_click#

Double-click at (x, y) or on an AX element identified by element_index + window_id.

AX path (element_index provided): performs AXOpen when the element advertises it (Finder items, openable list rows/cells); otherwise resolves the element's on-screen center and falls back to a pixel double-click there.

Pixel path (x, y provided): two down/up pairs ~80 ms apart at the given coordinates.

Arguments:

• delivery_mode (string, optional): Best-effort-background ladder rung (default "background"). "background": inject without fronting or raising the target — no focus steal. "foreground": briefly front the target, act, then restore the prior frontmost — the explicit last resort when a background attempt didn't land. Re-call with "foreground" only for the action that needs it.
• element_index (integer, optional): Element index from last get_window_state. Uses AX path. REQUIRES pid and window_id to be passed alongside it — element_index alone (no pid) fails fast with "Missing required integer field: pid"; it is not a silent no-op.
• element_token (string, optional): Opaque per-snapshot element handle from structuredContent.elements[].element_token. Takes precedence over element_index when both supplied. Returns an explicit "stale" error if the snapshot has been superseded.
• pid (integer, required)
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• window_id (integer, optional): CGWindowID. Required when element_index is used. Optional when element_token is supplied (the token carries it).
• x (number, optional): Screen X coordinate (pixel path).
• y (number, optional): Screen Y coordinate (pixel path).

{"pid":844}

right_click#

Right-click against a target pid. Two addressing modes:

• element_index + window_id (from the last get_window_state snapshot) — performs AXShowMenu on the cached element. Pure AX RPC, works on backgrounded / hidden windows, no cursor move or focus steal. Requires a prior get_window_state(pid, window_id) in this turn.

• x, y — synthesizes rightMouseDown / rightMouseUp CGEvent pair posted to the pid. Driver converts image-pixel → screen-point internally. modifier forces the CGEvent path (AX actions don't propagate modifier keys).

Exactly one of element_index or (x AND y) must be provided. pid always required. window_id required when element_index is used.

Arguments:

• delivery_mode (string, optional): Best-effort-background ladder rung (default "background"). "background": inject without fronting or raising the target — no focus steal. "foreground": briefly front the target, act, then restore the prior frontmost — the explicit last resort when a background attempt didn't land. Re-call with "foreground" only for the action that needs it.
• element_index (integer, optional): Element index from last get_window_state. Routes through AXShowMenu. REQUIRES pid and window_id to be passed alongside it — element_index alone (no pid) fails fast with "Missing required integer field: pid"; it is not a silent no-op.
• element_token (string, optional): Opaque per-snapshot element handle from structuredContent.elements[].element_token. Takes precedence over element_index when both supplied. Returns an explicit "stale" error if the snapshot has been superseded.
• modifier (array of string, optional): Modifier keys held during the right-click: cmd/shift/option/ctrl. Pixel path only.
• pid (integer, required): Target process ID.
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• window_id (integer, optional): CGWindowID. Required when element_index is used. Optional when element_token is supplied (the token carries it).
• x (number, optional): X in window-local screenshot pixels. Must be provided together with y.
• y (number, optional): Y in window-local screenshot pixels. Must be provided together with x.

{"pid":844}

drag#

Press-drag-release gesture from (from_x, from_y) to (to_x, to_y) in window-local screenshot pixels — the same space get_window_state returns. Top-left origin of the target's window.

Use for: marquee/lasso selection, drag-and-drop, resizing via a handle, scrubbing a slider, repositioning a panel.

duration_ms (default 500) is the wall-clock budget for the path between mouse-down and mouse-up; steps (default 20) is the number of intermediate mouseDragged events linearly interpolated along the path. Increase both for slower, more human drags; decrease for snap gestures.

modifier keys (cmd/shift/option/ctrl) are held across the entire gesture.

When from_zoom is true, coordinates are in the last zoom image for this pid; the driver maps them back to window coordinates before dispatching.

Arguments:

• button (string, optional): Mouse button used for the drag. Default: left.
• delivery_mode (string, optional): Best-effort-background ladder rung (default "background"). "background": inject without fronting or raising the target — no focus steal. "foreground": briefly front the target, act, then restore the prior frontmost — the explicit last resort when a background attempt didn't land. Re-call with "foreground" only for the action that needs it.
• duration_ms (integer, optional): Wall-clock duration of the drag path between mouseDown and mouseUp. Default: 500.
• from_x (number, required): Drag-start X in window-local screenshot pixels. Top-left origin.
• from_y (number, required): Drag-start Y in window-local screenshot pixels. Top-left origin.
• from_zoom (boolean, optional): When true, coordinates are in the last zoom image for this pid; driver maps back to window coordinates.
• modifier (array of string, optional): Modifier keys held across the entire gesture: cmd/shift/option/ctrl.
• pid (integer, optional): Target process ID.
• scope (string, optional): Use desktop with no pid/window_id for native get_desktop_state screenshot coordinates.
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• steps (integer, optional): Number of intermediate mouseDragged events linearly interpolated along the path. Default: 20.
• to_x (number, required): Drag-end X in window-local screenshot pixels.
• to_y (number, required): Drag-end Y in window-local screenshot pixels.
• window_id (integer, optional): CGWindowID for the window the pixel coordinates were measured against. Optional — when omitted the driver picks the frontmost window of pid.

{"from_x":100,"from_y":200,"to_x":100,"to_y":200}

type_text#

Insert text into the target pid via AXSetAttribute(kAXSelectedText). Works for standard Cocoa text fields and text views. No keystrokes are synthesized — special keys (Return / Escape / arrows) go through press_key / hotkey. For Chromium / Electron inputs that don't implement kAXSelectedText, the tool falls back to CGEvent character synthesis automatically.

Optional element_index + window_id (from the last get_window_state snapshot) directs the write to a specific field. Without element_index, the write goes to the pid's currently focused element.

WEB CONTENT (Chromium/WebKit/Electron — browser tabs, Slack, VS Code, X's compose box): the AX layer accepts a write and echoes it back through AXValue while the renderer/DOM never observes it. The driver detects this at the element level (an AXWebArea ancestor) and refuses to trust that echo — an AX-path insert into web content returns effect:"unverifiable" + escalation, never a false "confirmed" (a browser's own native address bar/toolbar stays trusted). For a browser TAB the reliable path is the page tool (drives the DOM via CDP); for an embedded web view use this tool's px form: pass x,y (no element_index) to pixel-click the field then type, in one call. NOTE: a px focus-click won't reliably open+focus a CLOSED control; AX-press to open/activate it first (works in the background), then px-type. Always confirm via the screenshot; if px-background still drops, escalate to delivery_mode:"foreground".

Arguments:

• delay_ms (integer, optional): Milliseconds between characters in the CGEvent fallback path. Default 30. Ignored when the AX path succeeds.
• delivery_mode (string, optional): Best-effort-background ladder rung (default "background"). "background": AX insert, then CGEvent keystrokes if needed — no focus steal; the driver verifies via an AXValue read-back and reports verified. "foreground": briefly front the window, type, restore the prior frontmost — the explicit last resort for focus-sensitive surfaces (e.g. WhatsApp/Catalyst) where background keystrokes don't land. Re-call with "foreground" when a background attempt returns verified:false and a screenshot shows the text didn't appear.
• element_index (integer, optional): Element index from last get_window_state. Directs the write to a specific field. REQUIRES pid and window_id to be passed alongside it — element_index alone (no pid) fails fast with "Missing required integer field: pid"; it is not a silent no-op.
• element_token (string, optional): Opaque per-snapshot element handle from structuredContent.elements[].element_token. Takes precedence over element_index when both supplied. Returns an explicit "stale" error if the snapshot has been superseded.
• pid (integer, optional): Target process ID.
• scope (string, optional): Use desktop with no pid/window_id to type into the frontmost application.
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• text (string, required): Text to insert at the target's cursor.
• window_id (integer, optional): CGWindowID. Required when element_index is used. Optional when element_token is supplied (the token carries it).
• x (number, optional): Screenshot-pixel X of the field to type into — the element px action form. Pass x,y (no element_index) and the tool pixel-clicks there to establish real renderer focus, then types. Use for Chromium/Electron inputs the AX path can't reach. Read straight off the get_window_state PNG, same convention as click.
• y (number, optional): Screenshot-pixel Y of the field (see x).

{"text":"hello"}

press_key#

Press and release a single key, delivered to the target pid via CGEventPostToPid. Follows the same delivery_mode ladder as click/type_text — it does NOT raise the window by default: • background (default): post to the pid WITHOUT fronting/raising — the auth-message path (Chromium-safe). With element_index it focuses that AX element first. window_id only targets; it does not raise. • foreground: briefly front the window (NSMenu path, < 1 ms) so native menu key-equivalents dispatch, then restore prior frontmost — the explicit escalation for menu shortcuts an app drops in the background. Requires window_id (and no element_index).

A key press is never driver-verifiable → effect:"unverifiable"; confirm via screenshot. Key names: return, tab, escape, up/down/left/right, space, delete, home, end, pageup, pagedown, f1-f12, plus any letter or digit. Modifiers array: cmd, shift, option/alt, ctrl, fn.

Arguments:

• delivery_mode (string, optional): Best-effort-background ladder rung (default "background"). "background": inject without fronting or raising the target — no focus steal. "foreground": briefly front the target, act, then restore the prior frontmost — the explicit last resort when a background attempt didn't land. Re-call with "foreground" only for the action that needs it.
• element_index (integer, optional)
• element_token (string, optional): Opaque per-snapshot element handle from structuredContent.elements[].element_token. Takes precedence over element_index when both supplied. Returns an explicit "stale" error if the snapshot has been superseded.
• key (string, required): Key name: return, tab, escape, up, down, etc.
• modifiers (array of string, optional): Modifier keys: cmd, shift, option/alt, ctrl, fn.
• pid (integer, optional)
• scope (string, optional): Use desktop with no pid/window_id to send the key to the frontmost application.
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• window_id (integer, optional): Target window. Required for delivery_mode:"foreground". Does NOT itself raise the window — raising is gated on delivery_mode.
• x (number, optional): Screenshot-pixel X — the element px action form: pixel-click there to focus, then send the key. Use when the key must go to a Chromium/Electron surface the AX path can't focus. Pass with y, no element_index.
• y (number, optional): Screenshot-pixel Y (see x).

{"key":"return"}

hotkey#

Press a key combination — e.g. ["cmd", "c"] for Copy, ["cmd", "shift", "4"] for screenshot selection. Follows the same delivery_mode ladder as click/type_text — it does NOT raise the window by default: • background (default): post the combo to the target pid WITHOUT fronting or raising it — uses the macOS 14+ auth-message envelope so Chromium/Electron accept it as trusted live input. No focus steal. window_id here only targets the combo; it does not raise. • foreground: briefly front the window (NSMenu path, < 1 ms via SLPSSetFrontProcessWithOptions) so native menu key-equivalents (Cmd+Z, Cmd+W) dispatch, then restore the prior frontmost — the explicit escalation for menu-bar shortcuts on non-Chromium apps that ignore a background combo. Requires window_id.

A combo is never driver-verifiable (no read-back) → effect:"unverifiable"; confirm via screenshot. NOTE: a keyboard combo does NOT focus a text field — to type into a backgrounded Electron input, establish real renderer focus with a PIXEL click first, then type_text (do not reach for a clipboard + Cmd+V dance).

Recognized modifiers: cmd/command, shift, option/alt, ctrl/control, fn. Non-modifier keys use the same vocabulary as press_key. Order: modifiers first, one non-modifier last.

Arguments:

• delivery_mode (string, optional): Best-effort-background ladder rung (default "background"). "background": inject without fronting or raising the target — no focus steal. "foreground": briefly front the target, act, then restore the prior frontmost — the explicit last resort when a background attempt didn't land. Re-call with "foreground" only for the action that needs it.
• keys (array of string, required): Modifier(s) and one non-modifier key, e.g. ["cmd", "c"].
• pid (integer, optional): Target process ID.
• scope (string, optional): Use desktop with no pid/window_id to send the chord to the frontmost application.
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• window_id (integer, optional): Target window. Required for delivery_mode:"foreground" (the NSMenu activation needs a window). Does NOT itself raise the window — raising is gated on delivery_mode.
• x (number, optional): Screenshot-pixel X — the element px action form: pixel-click there to focus, then send the combo (so e.g. Cmd+V pastes into that field). Pass with y. Use for Chromium/Electron surfaces the background combo can't reach.
• y (number, optional): Screenshot-pixel Y (see x).

{"keys":["cmd","c"]}

set_value#

Set a value on a UI element. Two modes depending on element role:

• AXPopUpButton / select dropdown: finds the child option whose title or value matches value (case-insensitive) and AXPresses it directly — the native macOS popup menu is never opened, so focus is never stolen. Use this for HTML <select> elements in Safari or any native NSPopUpButton.

• All other elements: writes AXValue directly (sliders, steppers, date pickers, native text fields that expose settable AXValue).

For free-form text entry into web inputs, prefer type_text_chars which synthesises key events — AXValue writes are ignored by WebKit.

Arguments:

• element_index (integer, optional): Element index from last get_window_state. Must be supplied unless element_token is provided. REQUIRES pid and window_id to be passed alongside it — element_index alone (no pid) fails fast with "Missing required integer field: pid"; it is not a silent no-op.
• element_token (string, optional): Opaque per-snapshot element handle from structuredContent.elements[].element_token. Takes precedence over element_index when both supplied. Returns an explicit "stale" error if the snapshot has been superseded.
• pid (integer, required)
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• value (string, required): New value. AX will coerce to the element's native type.
• window_id (integer, optional): CGWindowID for the window whose get_window_state produced the element_index. Required when element_index is used; optional when element_token is supplied (the token carries it).

{"pid":844,"value":"42"}

scroll#

Scroll the target pid. Two paths, picked by how you address the scroll:

• Targeted wheel path — when you pass a target, either element_index/element_token (preferred) or window-local x, y pixels: the driver synthesizes a real mouse-wheel event (CGEventCreateScrollWheelEvent, at that screen point. The renderer hit-tests the wheel at the cursor, so the scroll lands on whatever element is under the point — exactly like physically rolling the wheel over it. This is the ONLY way to scroll a nested overflow:auto region (e.g. a scrollable <div> with no tabindex): such regions never take keyboard focus, so the keystroke path below no-ops on them. Use this for inner/nested scrollers in web views.

• Keystroke path (focused region) — when you pass NO target (just pid + direction): synthesizes PageDown/PageUp (by='page') or Down/Up arrows (by='line'); horizontal uses Left/Right arrows. Drives the focused / page scroller only.

Mapping: by='page' → larger step; by='line' → smaller step; amount = number of wheel notches (targeted path) or keystroke repetitions (keystroke path).

Arguments:

• amount (integer, optional): Pixel-wheel path: number of wheel notches. Keystroke path: number of keystroke repetitions. Default: 3.
• by (string, optional): Scroll granularity. Default: line.
• delivery_mode (string, optional): Best-effort-background ladder rung (default "background"). "background": inject without fronting or raising the target — no focus steal. "foreground": briefly front the target, act, then restore the prior frontmost — the explicit last resort when a background attempt didn't land. Re-call with "foreground" only for the action that needs it.
• direction (string, required): Scroll direction.
• element_index (integer, optional): Element from last get_window_state. Routes through the pixel-wheel path AT this element's center — use it to scroll a nested overflow region you located in the AX tree.
• element_token (string, optional): Opaque per-snapshot element handle from structuredContent.elements[].element_token. Takes precedence over element_index when both supplied. Returns an explicit "stale" error if the snapshot has been superseded. Routes through the pixel-wheel path at the element's center.
• pid (integer, optional)
• scope (string, optional): Use desktop with x,y and no pid/window_id for native get_desktop_state screenshot coordinates.
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• window_id (integer, optional)
• x (number, optional): Window-local screenshot X (top-left origin of the PNG from get_window_state). With y, routes through the pixel-wheel path at this point — use for a scrollable surface that isn't in the AX tree. Requires window_id to anchor the window→screen conversion.
• y (number, optional): Window-local screenshot Y. See x.

{"direction":"up"}

move_cursor#

Move a cursor to (x, y). In window scope (default), moves only the agent cursor overlay. With scope=desktop, moves the real OS pointer in native get_desktop_state screenshot coordinates.

Arguments:

• cursor_id (string, optional): Cursor instance to move. Default: 'default'.
• scope (string, optional)
• session (string, optional): Optional session id: declares/uses the agent cursor and per-session state for this run. The same id works over MCP, the CLI, or the raw socket, and follows the run across apps/windows. Omit to run cursor-less.
• x (number, required)
• y (number, required)

{"x":100,"y":200}

zoom#

Capture a cropped JPEG of a window region (x1,y1)–(x2,y2) in screenshot pixel coordinates, with 20% padding added on each side. The output image is at most 500 px wide.

After a zoom, pass from_zoom=true to click/type_text to auto-translate coordinates back to full-window space.

Arguments:

• pid (integer, optional): Target pid — required for from_zoom click/type translation.
• window_id (integer, required): CGWindowID from list_windows.
• x1 (number, required): Left edge of region in screenshot pixels.
• x2 (number, required): Right edge of region in screenshot pixels.
• y1 (number, required): Top edge of region in screenshot pixels.
• y2 (number, required): Bottom edge of region in screenshot pixels.

{"window_id":10725,"x1":100,"x2":100,"y1":200,"y2":200}

Browser tools#

page#

Interact with the browser page loaded in a running app. Supports Chrome, Brave, Edge, Safari (via AppleScript on macOS), Electron apps (via CDP), Chromium/Firefox on Windows (via UIA for read; CDP for execute_javascript when --remote-debugging-port is set), and WKWebView/Tauri/AT-SPI fallbacks.

Actions:

• execute_javascript: Run JS and return the result.
• get_text: Extract visible text from the page.
• query_dom: Find elements matching a CSS selector.
• click_element: Click a CSS-selected element AND animate the agent cursor to its on-screen center first (so the user sees what the agent is doing). Prefer over execute_javascript('el.click()') whenever you want visible cursor feedback.
• insert_text: Insert text at whatever currently holds DOM focus in one native operation (CDP Input.insertText) — no synthesized key events, but more durable than a one-shot execute_javascript write since rich-text editors already have to treat it like an IME commit. Try this before type_keystrokes on a contenteditable that discarded an execute_javascript write. Click/focus the target field first.
• type_keystrokes: Type text via real per-character keystroke events into whatever currently holds DOM focus. Slower than insert_text but the most durable rung — use it when insert_text also gets discarded, or the editor's own keydown/keyup handlers need to see real keys. Click/focus the target field first.
• enable_javascript_apple_events: macOS-only — patch the browser's Preferences to allow JS from Apple Events (Chrome/Brave/Edge, requires user confirmation and a browser restart).

Arguments:

• action (string, required): Action to perform.
• attributes (array of string, optional): Element attributes to include in query_dom results.
• bundle_id (string, optional): Bundle ID of the browser. Required for enable_javascript_apple_events (macOS only).
• cdp_port (integer, optional): Optional, for execute_javascript/insert_text/type_keystrokes: use this exact CDP port instead of auto-discovering one from pid. Needed when the port was opened via the browser's own remote-debugging toggle rather than a launch-time flag, since that path may not answer the auto-discovery probe.
• css_selector (string, optional): CSS selector for query_dom (e.g. 'a', 'button', 'input', 'h1'-'h6', 'p', 'img', 'select', '*').
• javascript (string, optional): JavaScript to execute. Required for execute_javascript.
• pid (integer, optional): Target process ID.
• selector (string, optional): CSS selector for click_element (e.g. 'button.submit', '#login a').
• target_url_contains (string, optional): Optional, for execute_javascript/insert_text/type_keystrokes: require exactly one browser tab whose URL contains this substring. Use this on a multi-tab browser — there's no built-in link between window_id and which tab a CDP call reaches.
• text (string, optional): Text to insert or type. Required for insert_text and type_keystrokes. The target field must already have DOM focus (click/focus it first).
• user_has_confirmed_enabling (boolean, optional): Must be true to proceed with enable_javascript_apple_events. This will quit and relaunch the browser.
• window_id (integer, optional): Target window ID from list_windows.

{"action":"execute_javascript"}

Recording tools#

start_recording#

Start trajectory recording. Every subsequent action-tool invocation (click, right_click, scroll, type_text, press_key, hotkey, set_value) writes a turn folder under output_dir:

• before_state.json / after_state.json — application AX/UIA/AT-SPI state immediately before and after the action.
• before.png / after.png — target-window screenshots immediately before and after the action.
• evidence.json — capture status and a stable classification when an expected artifact could not be captured.
• app_state.json — post-action AX/UIA snapshot for the target pid.
• screenshot.png — compatibility alias of after.png.
• action.json — tool name, full input arguments, result summary, pid, click point (when applicable), ISO-8601 timestamp.
• click.png — for click-family actions only, before.png with a red marker at the click point.

Turn folders are named turn-00001/, turn-00002/, etc. Turn numbering restarts at 1 each time recording is (re-)started.

Video is off by default. Pass record_video: true to also capture the main display to <output_dir>/recording.mp4 (H.264 / 30 fps) for the lifetime of the session. The recording is torn down automatically when the MCP client disconnects.

macOS uses native ScreenCaptureKit (daemon-owned SCStream + SCRecordingOutput) so video inherits the daemon's Screen Recording grant — no extra TCC prompt, no ffmpeg subprocess. Requires macOS 15.0+.

Windows + Linux use an ffmpeg subprocess (gdigrab / x11grab + libx264). Requires ffmpeg on PATH (winget install Gyan.FFmpeg / apt install ffmpeg); when ffmpeg is missing or fails on startup the per-turn capture (screenshots + action.json) still runs and the session's last_error field carries the diagnostic.

State persists for the life of the daemon / MCP session; a restart resets to disabled with no on-disk state. Call stop_recording to disable + finalize the mp4.

Arguments:

• output_dir (string, required): Absolute or ~-rooted directory where turn folders and (when enabled) the video file are written.
• record_video (boolean, optional): Capture the main display to <output_dir>/recording.mp4. Default: false. Set to true to also capture the main display to recording.mp4 (otherwise only the per-turn screenshots + JSON are recorded). On macOS this uses native ScreenCaptureKit (no extra TCC prompt, macOS 15.0+); on Windows + Linux it requires ffmpeg on PATH.

{"output_dir":"~/cua-trajectories/demo1"}

stop_recording#

Stop trajectory recording. Disables further per-turn capture and, when video was enabled, gracefully terminates the ffmpeg subprocess so the mp4's moov atom is finalized (the file is playable). Calling stop on an already-stopped session is a no-op. The response carries last_video_path pointing at the finalized mp4 (when video was on).

A manual stop_recording is unconditional — it stops whatever recording is active regardless of which session started it. Ownership-scoped teardown (so one MCP client disconnecting can't stop a recording a later client started) is handled by the daemon's session_end lifecycle signal, not by this tool.

Arguments: none.

replay_trajectory#

Replay a recorded trajectory by re-invoking every turn's tool call in lexical order. dir must point at a directory previously written by start_recording. Each turn-NNNNN/ is parsed for action.json, and the recorded tool is called with its recorded arguments via the same dispatch path an MCP / CLI call uses.

Caveats:

• Element-indexed actions (click({pid, element_index}) etc.) will fail because element indices are per-snapshot and don't survive across sessions. Pixel clicks (click({pid, x, y})) and all keyboard tools replay cleanly. Failures are reported but don't stop replay unless stop_on_error is true.
• get_window_state and other read-only tools are NOT currently recorded, so replays do not re-populate the per-(pid, window_id) element cache.
• If recording is ENABLED while replay runs, the replay itself is recorded into the currently configured output directory. That's deliberate: recording a replay against a new build and diffing the two trajectories is the regression-test workflow.

Arguments:

• delay_ms (integer, optional): Milliseconds to sleep between turns, for human-observable pacing. Default 500.
• dir (string, required): Trajectory directory previously written by start_recording. Absolute or ~-rooted.
• stop_on_error (boolean, optional): Stop replay on the first tool-call error. Default true — set false to best-effort through the full trajectory.

{"dir":"~/cua-trajectories/demo1"}

Configuration tools#

set_config#

Update cua-driver-rs configuration. Changes to max_image_dimension take effect immediately. The experimental_pip keys are persisted to ~/.cua-driver/config.json and take effect on the next daemon restart (the PiP backend is initialised once at startup).

Note: capture_mode is a per-call param (on get_window_state / click), not a stored setting. Capture scope is selected by start_session, not set_config.

Arguments:

• experimental_pip (boolean, optional): Enable the experimental picture-in-picture preview window. Applies on next daemon restart.
• experimental_pip_geometry (string, optional): PiP window size + optional position in WxH or WxH+X+Y form (e.g. 320x200+24+24). Applies on next daemon restart.
• key (string, optional): Name of a single config field to write ({key, value} shape, matching the CLI config set and the Windows/Linux tools). Pair with value. Equivalent to passing the field directly.
• max_image_dimension (integer, optional): Max dimension for screenshot resizing (0 = no limit).
• value (undefined, optional): New value for key. JSON type depends on the key.

start_session#

Declare a session — a named, color-coded identity for THIS agent run. Pass a stable session id and choose capture_scope=auto|window|desktop; the agent cursor, capture policy, per-session config, and recording all key on it, and it follows the run across any apps/windows. The cursor's color is derived from the id, so distinct runs are visually distinct. A cursor is shown only for a declared session — call this (or pass session on your first action) to opt in. Idempotent: re-calling with the same id just refreshes its idle-TTL. End it with end_session (or let the idle-TTL reclaim it). Concurrent runs/subagents each pass their own session to get their own cursor.

Arguments:

• capture_scope (string, optional): Per-session perception/action modality. auto starts window-only and requires explicit escalation before desktop tools; window and desktop are strict. Immutable for the live session.
• session (string, required): Stable session id for this run (e.g. "research-run-1").

{"session":"example"}

end_session#

End a session declared with start_session: removes its agent cursor, stops any recording it owns, and clears its per-session config. Call this when a run finishes so its cursor doesn't linger (otherwise the idle-TTL reclaims it after a period of inactivity). Idempotent.

Arguments:

• session (string, required): The session id to end.

{"session":"example"}

set_agent_cursor_enabled#

Show or hide the agent cursor for a session. A cursor exists only for a DECLARED session: pass session (the same id you start_session / drive actions with) and the cursor appears on that session's first action — its color is derived from the id. Without a session, actions run cursor-less. Use enabled=false to hide a session's cursor, enabled=true to re-show it. (cursor_id is a legacy alias for session.)

Arguments:

• cursor_id (string, optional): Cursor instance. Default: 'default'.
• enabled (boolean, required): true = show, false = hide.

{"enabled":false}

set_agent_cursor_motion#

Configure the visual appearance and motion curve of an agent cursor instance.

Appearance (multi-cursor customization):

• cursor_id: instance name (default='default')
• cursor_icon: built-in ('arrow' | 'teardrop') or a path to a PNG/JPEG/SVG/ICO file; '' reverts to the default cursor
• cursor_color: hex color e.g. '#00FFFF' or CSS name
• cursor_label: short text shown near the cursor
• cursor_size: dot radius in points (default=16)
• cursor_opacity: 0.0–1.0 (default=0.85)

Motion curve (Bezier path shape):

• start_handle: departure control-point fraction [0,1]. Default 0.3
• end_handle: arrival control-point fraction [0,1]. Default 0.3
• arc_size: perpendicular deflection as fraction of path length [0,1]. Default 0.25
• arc_flow: asymmetry [-1,1]; positive bulges toward destination. Default 0.0
• spring: settle damping [0.3,1.0]; 1.0=no overshoot. Default 0.72
• glide_duration_ms: fixed flight duration per move [50,5000]; omit for speed-based (the default)
• dwell_after_click_ms: pause after click ripple [0,5000]. Default 80
• idle_hide_ms: auto-hide delay [0,60000]; 0=never. Default 20000

Arguments:

• arc_flow (number, optional): Asymmetry bias in [-1, 1]. Default 0.0.
• arc_size (number, optional): Arc deflection as fraction of path length [0, 1]. Default 0.25.
• cursor_color (string, optional): Hex color (e.g. '#00FFFF') or CSS color name.
• cursor_icon (string, optional): Built-in shape ('arrow' | 'teardrop') or a path to a PNG/JPEG/SVG/ICO file. '' reverts to the default cursor.
• cursor_id (string, optional): Cursor instance name. Default: 'default'.
• cursor_label (string, optional): Short label near the cursor dot.
• cursor_opacity (number, optional): Opacity 0.0–1.0. Default: 0.85.
• cursor_size (number, optional): Dot radius in points. Default: 16.
• dwell_after_click_ms (number, optional): Pause after click ripple in ms. Default 80.
• end_handle (number, optional): End-handle fraction in [0, 1]. Default 0.3.
• glide_duration_ms (number, optional): Fixed flight duration per move in ms; omit for speed-based timing (the default).
• idle_hide_ms (number, optional): Auto-hide delay in ms. 0 = never hide. Default 20000.
• spring (number, optional): Settle damping in [0.3, 1.0]. Default 0.72.
• start_handle (number, optional): Start-handle fraction in [0, 1]. Default 0.3.
• turn_radius (number, optional): Minimum turning radius of the glide path in points; smaller = tighter curves. Default 80.

set_agent_cursor_style#

Update the visual style of the agent cursor overlay.

• gradient_colors: array of CSS hex strings (e.g. ["#FF0000","#0000FF"]) used as the arrow fill gradient from tip to tail. Empty array reverts to the default palette colours.
• bloom_color: hex string for the radial halo/bloom behind the cursor (e.g. "#00FFFF"). Empty string reverts to the default.
• image_path: path to a PNG, JPEG, SVG, or ICO file to use as the cursor icon instead of the default silhouette. Empty string reverts to the default cursor. All parameters are optional; omit any you do not want to change.

Arguments:

• bloom_color (string, optional): Hex bloom/halo colour (e.g. '#00FFFF'). '' = revert to default.
• cursor_id (string, optional): Cursor instance. Default: 'default'.
• gradient_colors (array of string, optional): CSS hex gradient stops tip→tail. [] = revert to default.
• image_path (string, optional): Path to PNG/JPEG/SVG/ICO cursor image. '' = revert to the default cursor.

Maintenance tools#

check_permissions#

Report TCC permission status for Accessibility and Screen Recording. By default also raises the system permission dialogs for any missing grants — Apple's request APIs are no-ops when the grant is already active, so this is safe to call repeatedly. Pass {"prompt": false} for a purely read-only status check.

Returns: accessibility + screen_recording (booleans from the TCC preflight APIs), screen_recording_capturable (a live ScreenCaptureKit probe — if it disagrees with screen_recording, the preflight grant belongs to a different process), and source (which TCC identity the booleans reflect: the CuaDriver daemon vs the launching terminal/IDE). macOS attributes grants to the responsible process, so a standalone call from a terminal reports the terminal's grants, not the driver's.

Arguments:

• prompt (boolean, optional): Raise the system permission prompts for missing grants. Default true.

health_report#

Single-call end-to-end driver diagnostics. Designed to let downstream consumers ship one stable call instead of stitching together check_permissions, doctor, version, bundle attribution, and a screenshot probe. cua-driver owns the health model; consumers stay thin.

Input — all optional: { "include": ["<check_name>", ...], // run only these "skip": ["<check_name>", ...] // skip these } If both are given, include wins.

Canonical check names: macOS : binary_version, platform_supported, session_active, bundle_identity, tcc_accessibility, tcc_screen_recording, ax_capability, screen_capture_capability Windows: binary_version, platform_supported, session_active, ax_capability (via UIA), screen_capture_capability (via DXGI) Linux : binary_version, platform_supported, session_active, ax_capability (via AT-SPI), screen_capture_capability (via X11)

Output — stable contract, schema_version="1": { "schema_version": "1", "platform": "darwin" | "win32" | "linux", "driver_version": "<semver>", "overall": "ok" | "degraded" | "failed", "checks": [ { "name": "<one of the canonical names above>", "status": "pass" | "fail" | "skip", "message": "<one-line summary, always present>", "hint": "<remediation step, present when status=fail>", "data": { /* check-specific structured fields */ } }, ... ] }

overall rules:

• ok — every non-skipped check passes
• degraded — at least one non-core check fails (binary is still usable)
• failed — any core check fails (binary_version, platform_supported, session_active)

Stability: schema_version="1" is the contract. Future breaking changes will be "2". Adding new check names under the same schema_version is non-breaking; consumers must tolerate unknown check names.

Arguments:

• include (array of string, optional): Only run these checks (canonical names). Wins over skip.
• skip (array of string, optional): Skip these checks (canonical names). Ignored when include is set.

check_for_update#

Check whether a newer cua-driver-rs release is available on GitHub. Returns the current and latest versions, an update_available boolean, the install one-liner, and the release notes URL. Read-only — never installs. Mirror of cua-driver check-update --json.

Arguments: none.

install_ffmpeg#

Install the ffmpeg binary used by start_recording's video capture (Linux/Windows; macOS records natively and needs no ffmpeg). Two-step and confirmed: called without confirm it only REPORTS the exact install command for this platform's package manager; pass confirm: true to actually run it. No-op if ffmpeg is already on PATH. ffmpeg is run as a separate process, never linked into the driver.

Arguments:

• confirm (boolean, optional): Run the install command. Without it, only the planned command is reported.

Other tools#

get_browser_state#

Read-only browser inspection. Mode 1 (bind): pass pid + window_id of a native browser window to classify it, correlate it to a CDP target (exact-or-refuse), and mint a session-scoped target id plus tab ids. Mode 2 (snapshot): pass target_id + tab_id. The dom_refs_v1 compatibility format returns composed DOM refs. semantic_v2 joins accessibility, DOM, layout, and viewport state; ranks visible content before retained/offscreen state; and returns a semantic outline, typed action refs, content refs, scoped reads, and opaque continuation. Never performs setup — a missing endpoint is a structured browser_requires_setup refusal pointing at browser_prepare.

Arguments:

• continuation (string, optional): Opaque continuation minted by an earlier semantic_v2 response.
• pid (integer, optional): Native browser process id (bind mode).
• query (string, optional): Read-only semantic match over role, accessible name, and visible text.
• scope_ref (string, optional): Current semantic/content ref whose subtree should be observed.
• session (string, optional): Stable caller-declared session id. Browser targets, tabs, and refs are scoped to this session.
• snapshot_format (string, optional): Versioned snapshot contract. dom_refs_v1 remains the compatibility default.
• tab_id (string, optional): Opaque tab id from get_browser_state (session-scoped).
• target_id (string, optional): Opaque browser target id minted by get_browser_state (session-scoped; never a CDP id).
• window_id (integer, optional): Native window id owned by pid (bind mode).

browser_prepare#

Explicitly prepare an owned DevTools endpoint for a browser pid. Existing endpoints are detected without side effects. Acting setup requires MCP-host approval or a short-lived token from the interactive browser-approve command, allow_launch=true, and a driver-owned isolated profile. It launches a separate browser and never copies, modifies, or terminates the requested user profile. Existing-profile attachment is explicit, requires an exact interactive approval artifact, and never treats ordinary MCP transport approval as profile consent. On proven platforms, that approval also permits one bounded exact-window setup: open the recognized browser product's fixed remote-debugging page, toggle its uniquely matched per-instance checkbox, prove the PID-owned loopback endpoint, and close the temporary tab. Every visible effect is reported; ambiguity is refused.

Arguments:

• allow_launch (boolean, optional): Allow a separate driver-owned isolated Chromium process to be launched (default false).
• approval_token (string, optional): Single-use token minted by cua-driver browser-approve for direct CLI/raw use. Omit for an MCP-host-approved call.
• pid (integer, required): Browser process id to prepare.
• profile (object, optional)
• session (string, optional): Stable caller-declared session id. Browser targets, tabs, and refs are scoped to this session.
• strategy (object, optional)
• window_id (integer, optional): Exact native window approval anchor; required for strategy.kind=existing_profile.

{"pid":844}

browser_navigate#

Navigate one tab of an exactly-bound browser target to a new URL (http/https/about only). Refused for heuristic bindings. Navigation invalidates all p<snapshot>:<index> refs for the tab.

Arguments:

• session (string, optional): Stable caller-declared session id. Browser targets, tabs, and refs are scoped to this session.
• tab_id (string, required): Opaque tab id from get_browser_state (session-scoped).
• target_id (string, required): Opaque browser target id minted by get_browser_state (session-scoped; never a CDP id).
• url (string, required): Destination URL (http:, https:, or about:).

{"tab_id":"example","target_id":"example","url":"example"}

browser_click#

Click a page element (by ref) or viewport coordinates in an exactly-bound tab. Default route is trusted hardware-like input (Input.dispatchMouseEvent), and refuses where that route cannot preserve standalone-browser background posture. input_route="dom_event" (synthetic el.click(), ref required) is used only when explicitly requested. Refused for heuristic bindings.

Arguments:

• input_route (string, optional): "trusted" (default): Input.dispatchMouseEvent. It refuses rather than foregrounding a standalone browser. "dom_event": synthetic full-background DOM click, only when explicitly requested.
• ref (string, optional): Page element ref in the p<snapshot>:<index> namespace from get_browser_state. Refs are invalidated by navigation and by newer snapshots of the same tab.
• session (string, optional): Stable caller-declared session id. Browser targets, tabs, and refs are scoped to this session.
• tab_id (string, required): Opaque tab id from get_browser_state (session-scoped).
• target_id (string, required): Opaque browser target id minted by get_browser_state (session-scoped; never a CDP id).
• x (number, optional): Viewport x (CSS px) — alternative to ref.
• y (number, optional): Viewport y (CSS px) — alternative to ref.

{"tab_id":"example","target_id":"example"}

browser_type#

Type text into an exactly-bound tab via the Input domain. mode="insert_text" (default) uses Input.insertText; mode="keystrokes" dispatches per-character key events. Pass a ref to an editable element from the latest snapshot. A ref is required; heuristic bindings are refused.

Arguments:

• mode (string, optional): insert_text (default): bulk Input.insertText. keystrokes: per-character Input.dispatchKeyEvent.
• ref (string, required): Page element ref in the p<snapshot>:<index> namespace from get_browser_state. Refs are invalidated by navigation and by newer snapshots of the same tab.
• session (string, optional): Stable caller-declared session id. Browser targets, tabs, and refs are scoped to this session.
• tab_id (string, required): Opaque tab id from get_browser_state (session-scoped).
• target_id (string, required): Opaque browser target id minted by get_browser_state (session-scoped; never a CDP id).
• text (string, required): Text to type.

{"ref":"example","tab_id":"example","target_id":"example","text":"hello"}

browser_dialog#

Inspect or resolve a page-owned JavaScript alert, confirm, prompt, or beforeunload dialog on one exactly-bound tab. This never handles browser permission UI, extension UI, native dialogs, or file pickers. Inspect returns an opaque dialog_id; accept/dismiss require that exact current id. Resolution defaults to background delivery; Linux callers must explicitly request foreground delivery because Chromium's native modal cannot be resolved there without changing foreground posture.

Arguments:

• action (string, required)
• delivery_mode (string, optional): Requested foreground posture for accept/dismiss. Linux Chromium requires foreground; inspect is read-only.
• dialog_id (string, optional): Opaque current dialog generation returned by action=inspect.
• prompt_text (string, optional): Sensitive response text, valid only when accepting a prompt dialog.
• session (string, optional): Stable caller-declared session id. Browser targets, tabs, and refs are scoped to this session.
• tab_id (string, required): Opaque tab id from get_browser_state (session-scoped).
• target_id (string, required): Opaque browser target id minted by get_browser_state (session-scoped; never a CDP id).

{"action":"inspect","tab_id":"example","target_id":"example"}

browser_set_input_files#

Assign one or more explicit absolute local files to an exact live <input type=file> ref through CDP. This bypasses native file pickers, rejects symlinks and non-regular files, and never returns local paths.

Arguments:

• files (array of string, required)
• ref (string, required): Page element ref in the p<snapshot>:<index> namespace from get_browser_state. Refs are invalidated by navigation and by newer snapshots of the same tab.
• session (string, optional): Stable caller-declared session id. Browser targets, tabs, and refs are scoped to this session.
• tab_id (string, required): Opaque tab id from get_browser_state (session-scoped).
• target_id (string, required): Opaque browser target id minted by get_browser_state (session-scoped; never a CDP id).

{"files":["example"],"ref":"example","tab_id":"example","target_id":"example"}

browser_download#

Trigger one download through an exact live browser ref and save it inside an explicitly approved directory. Requires MCP-host destructive-tool approval, refuses ambiguous or stale capabilities, and never returns the source URL, filename, or destination path.

Arguments:

• destination_root (string, required): Absolute, existing, canonical directory approved to receive the download.
• ref (string, required): Live page ref whose activation initiates the download.
• session (string, required): Explicit caller session owning the browser capabilities.
• tab_id (string, required): Opaque exact tab id from get_browser_state.
• target_id (string, required): Opaque exact browser target id from get_browser_state.

{"destination_root":"example","ref":"example","session":"example","tab_id":"example","target_id":"example"}

browser_pointer#

Perform hover, right-click, double-click, scroll, or drag in an exactly-bound browser tab. The trusted route uses CDP Input events and refuses if standalone background posture cannot be preserved. The explicit dom_event route requires a page ref and synthesizes full-background DOM events. Never activates or brings a tab to the foreground.

Arguments:

• action (string, required)
• delta_x (number, optional): Horizontal scroll delta in CSS pixels.
• delta_y (number, optional): Vertical scroll delta in CSS pixels.
• destination_ref (string, optional): Drag destination page ref in the exact same frame.
• input_route (string, optional)
• ref (string, optional): Origin page ref. Alternative to x/y.
• session (string, required): Explicit caller session owning the browser capabilities.
• tab_id (string, required): Opaque tab id minted by get_browser_state.
• target_id (string, required): Opaque target id minted by get_browser_state.
• to_x (number, optional): Drag destination viewport x in CSS pixels.
• to_y (number, optional): Drag destination viewport y in CSS pixels.
• x (number, optional): Origin viewport x in CSS pixels.
• y (number, optional): Origin viewport y in CSS pixels.

{"action":"hover","session":"example","tab_id":"example","target_id":"example"}

escalate_session#

Unlock the desktop phase of an auto capture-scope session after the window action ladder has been exhausted and verified. This is a one-way transition for the live session and records a bounded reason.

Arguments:

• detail (string, optional): Optional bounded diagnostic detail. Never use secrets or page content.
• reason (string, required)
• session (string, required)

{"reason":"ax_tree_pixel_mismatch","session":"example"}

get_session_state#

Read the live session's capture policy and effective scope.

Arguments:

• session (string, required)

{"session":"example"}