Apps, windows and screen

List macOS apps, both currently running and installed-but-not-running, with per-app state flags:

• running: is a process for this app live? (pid is 0 when false)
• active: is it the system-frontmost app? (implies running)
• launch_path: filesystem path to the .app bundle, when known. Pass this to launch_app to start the app cold.
• kind: "desktop" for .app bundles on macOS.
• last_used: RFC3339 timestamp from the bundle's filesystem mtime, when readable.

Optional fields (bundle_id, launch_path, kind, last_used) are omitted when unknown, never null.

Standalone running entries include only apps with NSApplicationActivationPolicyRegular: background helpers and system UI agents are filtered out. Installed apps resolve their running/pid state against all live processes by bundle identifier, so an installed app whose process runs as an accessory (LSUIElement / menu-bar apps, e.g. Cua Driver itself) still reports its live pid. Installed apps come from scanning /Applications, /Applications/Utilities, ~/Applications, /System/Applications, and /System/Applications/Utilities.

Use this for "is X installed?" as well as "is X running?"; the list includes every installed app. To get an open app's pid and window_id, call list_windows({app: "Name"}) instead, which also gives per-window state (on-screen, on-current-Space, minimized, window titles). For just opening an app, running or not, call launch_app({bundle_id: ...}) directly; list_apps is not a prerequisite.

List top-level windows known to WindowServer, including off-screen ones (minimized, other Space, hidden): layer-0 only without pid, every layer with pid. Use it to find a pid and window_id for get_window_state; filter with app or pid, because an unfiltered call lists every window on the host. AppKit-internal helper windows and the driver's own overlay are omitted.

Per record: window_id, pid, app_name, title, bounds, layer, z_index, is_on_screen, space_ids, current_space_id, on_current_space. For the frontmost window take the maximum integer z_index; null means stacking is unavailable, so do not infer it from array order.

macOS parameters

Launch a macOS app in the background: the target does NOT come to the foreground.

Provide either bundle_id (preferred: unambiguous, e.g. com.apple.calculator) or name (e.g. "Calculator"). If both are given, bundle_id wins.

Optional urls are handed to the app as open targets: for Finder, pass a folder path to open a backgrounded Finder window there.

Browser DevTools setup belongs to browser_prepare, which can prove that a separate isolated profile is driver-owned before enabling CDP.

Optional webkit_inspector_port: opens a WebKit inspector server on the specified port (sets WEBKIT_INSPECTOR_SERVER=127.0.0.1:N + TAURI_WEBVIEW_AUTOMATION=1). Use this for Tauri/WebKit-based apps.

Optional creates_new_application_instance: when true, forces a new app instance even if one is already running (passes -n to open). Reach for this when another agent or session may drive the SAME app concurrently: it returns a fresh pid + window so each session acts on its own isolated window instead of clobbering one shared instance. Without it, single-instance apps (Calculator, many utilities) hand every caller the same window, so two sessions fight over it.

Optional additional_arguments: extra argv strings appended after --args.

Returns the launched app's pid, bundle_id, name, and a windows array (same shape as list_windows) so callers can skip an extra round-trip before get_window_state(pid, window_id). launch_state distinguishes whether the request was sent, the process is running, and a window is ready. When the focus-steal belt-and-braces demotion check ran (target pid ≠ prior frontmost), the response also includes self_activation_suppressed: bool; true if focus stayed with the prior frontmost, false if the launched app held focus despite the re-demote attempt.

macOS parameters

Force-terminate a process by pid (kill -9 equivalent on macOS / Linux; taskkill /F equivalent on Windows). Use as escalation when the cooperative close path (hotkey cmd+q on macOS, click-the-X on Windows) failed to make the process exit. Unsaved state is lost: prefer the cooperative path first.

Persistently activate an app and leave it in the foreground. Most input does not need this; use it only for a focus-proxy surface that must remain foreground across interactions. With window_id, success means the exact ordinary macOS window was independently verified as the frontmost process's focused window and the front window of that process on the display it sits on. exact_window_effect.frontmost_ordinary additionally reports whether it is first in the global WindowServer layer-0 order; another application (an always-raised utility window, another display's front window) can hold that spot without the requested window losing keyboard focus, so it is reported and not required. Request acceptance alone is reported as a partial result, never as activation. This DOES steal foreground and does NOT restore the previously frontmost application.

macOS parameters

Snapshot a window: the accessibility tree plus a screenshot. By default the tree is ONE compact Markdown rendering (tree_format:"markdown"): a row [N] is addressed with element_token <snapshot_id>:N, where snapshot_id is in the response header and in structuredContent. tree_format:"elements" returns the structured elements array instead (element_index, element_token, role, label, value, actions, frame, parent_index, depth); "both" returns both (about twice the size). Pass the token to click, type_text, press_key, etc.

START NARROW: a read walks at most 250 nodes by default and says Tree truncated at max_elements=… (truncated:true) when it stops; raise max_elements, or use query (case-insensitive substring; matching rows plus ancestors) and max_depth. Indices and tokens stay valid for the whole snapshot. verbose:true adds _note and the full background_input report; full_output:true restores the previous full response (both representations, all metadata, ≤2 000 nodes).

DIFF READS (use them for every re-read after acting): since:"latest", or since:<snapshot_id> from an earlier read of the same window, returns only what changed: + added, ~ changed, - removed rows (removed ids are the old snapshot's), a reindexed: line if indices shifted, or no change since …; focused element is …. The response carries a NEW snapshot_id: use it in tokens. An unknown, expired, other-window or differently-scoped (query/max_elements/max_depth) since falls back to a full read; since_status says why.

A new snapshot of the same (pid, window_id) replaces the previous one and stales its tokens (invalidated_snapshot_ids). include_screenshot:false returns the tree only and keeps the last screenshot's pixel frame while the window keeps its size; include_accessibility_tree:false returns only the screenshot and keeps the current snapshot's rows and tokens valid (use it, not max_elements:1, when you just need to look); adding display_only:true (live previews) also leaves the pixel frame alone. The tree can lie on some surfaces (Electron, Catalyst, virtualized rows with h:1 frames): cross-check the screenshot, and use pixel x,y at action time only for elements missing from the tree.

Refusals: window_id_not_found; window_owner_pid_mismatch names the real owner_pid (sandboxed Open/Save panels belong to a panel service). degraded_reason ax_window_unresolved or ax_app_launching (the latter with truncation_reason: app_lookup_timeout) means an empty tree: re-snapshot, or act with delivery_mode:"foreground". degraded_reason screen_locked means the screen saver, display sleep or a lock hid every window from accessibility: wake and unlock the Mac, then re-snapshot; foreground input would reach the lock screen. px_frame_mismatch / px_capture_unavailable omit the unprovable screenshot; the AX payload stays valid.

macOS parameters

Return a lightweight snapshot of the desktop: running regular apps and on-screen visible windows with their bounds, z-order, and owner pid.

For the full AX subtree of a single window (with interactive element indices you can click by), use get_window_state instead: that's the heavy per-window tool. This one is a fast discovery read that needs no TCC grants.

Capture the full display. The long edge is capped at max_image_dimension (the configured default unless given); pixels of the returned image map back to the screen through its capture. Use its PNG as the coordinate source for actions whose target is {kind:"desktop",display_id:"primary"}. Returns the true screen size and backing scale factor. Vision-only: no AX tree walk.

Return the logical size of the main display in points plus its backing scale factor. Agents click in points; Retina displays have scale_factor 2.0. Requires no TCC permissions.

Capture a cropped JPEG of a window region (x1,y1)–(x2,y2) in screenshot pixel coordinates, with 20% padding added on each side. The output image is at most 500 px wide. Corners, not a size: x1,y1 is the top-left and x2,y2 the bottom-right (x,y,width,height is also accepted and converted).

After a zoom, pass from_zoom=true to click/type_text to auto-translate coordinates back to full-window space. Coordinate actions return screenshot_context_missing when no current snapshot contains a screenshot owned by this session. from_zoom actions return zoom_context_missing when the zoom was never created or was replaced; call get_window_state, then zoom, again on the same connection.

macOS parameters