Pointer tools
Drag, scroll, move the pointer, and press or release buttons.
Drag, scroll, move the pointer, and press or release buttons.
| Tool | Description | Platforms |
|---|---|---|
drag | Press-drag-release gesture from (from_x, from_y) to (to_x, to_y) in window-local screenshot pixels: the same space get_window_state returns. | macOS, Linux, Windows |
scroll | Scroll the target pid. | macOS, Linux, Windows |
move_cursor | Move a cursor to (x, y). | macOS, Linux, Windows |
mouse_button_down | Press and hold a mouse button at (x,y) via background X11 delivery. | Linux |
mouse_button_up | Release a previously-held mouse button via background X11 delivery. | Linux |
mouse_drag | Move a previously-held mouse button to a new point via background X11 delivery. | Linux |
parallel_mouse_drag | Run multiple mouse drag gestures concurrently via Linux MPX/XI2 virtual master pointers. | Linux |
Served by cua-driver mcp; see MCP tools for every tool.
drag#Effect: destructive. Platforms: macOS, Linux, Windows.
Press-drag-release gesture from (from_x, from_y) to (to_x, to_y) in window-local screenshot pixels: the same space get_window_state returns. Top-left origin of the target's window.
Use for: marquee/lasso selection, drag-and-drop, resizing via a handle, scrubbing a slider, repositioning a panel.
duration_ms (default 500) is the wall-clock budget for the path between mouse-down and mouse-up; steps (default 20) is the number of intermediate mouseDragged events linearly interpolated along the path. Increase both for slower, more human drags; decrease for snap gestures.
modifier keys (cmd/shift/option/ctrl) are held across the entire gesture.
When from_zoom is true, coordinates are in the last zoom image for this pid; the driver maps them back to window coordinates before dispatching.
macOS parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
button | "left" | "right" | "middle" | Mouse button used for the drag. Default: left. | |
delivery_mode | "background" | "foreground" | Best-effort-background ladder rung (default "background"). "background": inject without fronting or raising the target; no focus steal. "foreground": briefly front the target, act, then restore the prior frontmost; the explicit last resort when a background attempt didn't land. Re-call with "foreground" only for the action that needs it. | |
duration_ms | integer | Wall-clock duration of the drag path between mouseDown and mouseUp. Default: 500. Range: 0 to 10000. | |
from_x | number | required | Drag-start X in window-local screenshot pixels. Top-left origin. |
from_y | number | required | Drag-start Y in window-local screenshot pixels. Top-left origin. |
from_zoom | boolean | When true, coordinates are in the last zoom image for this pid; driver maps back to window coordinates. | |
modifier | string[] | Modifier keys held across the entire gesture: cmd/shift/option/ctrl. | |
pid | integer | Target process ID. | |
scope | "window" | "desktop" | "window" | Use desktop with no pid/window_id for native get_desktop_state screenshot coordinates. |
steps | integer | Number of intermediate mouseDragged events linearly interpolated along the path. Default: 20. Range: 1 to 200. | |
to_x | number | required | Drag-end X in window-local screenshot pixels. |
to_y | number | required | Drag-end Y in window-local screenshot pixels. |
window_id | integer | CGWindowID for the window the pixel coordinates were measured against. Optional only when pid owns exactly one eligible top-level window; otherwise the action refuses with ambiguous_window_target. |
Parameters on every platform
| Parameter | Type | Default | Description |
|---|---|---|---|
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. | |
target | window target | desktop target | Preferred per-call target: an exact window (kind="window", pid, window_id) or the primary desktop (kind="desktop", display_id="primary"). |
Example arguments
{"from_x":100,"from_y":200,"to_x":100,"to_y":200}scroll#Effect: mutating. Platforms: macOS, Linux, Windows.
Scroll the target pid. Two paths, picked by how you address the scroll:
• Targeted wheel path: when you pass a target, either element_token (preferred) or window-local x, y pixels: the driver synthesizes a real mouse-wheel event (CGEventCreateScrollWheelEvent, at that screen point. The renderer hit-tests the wheel at the cursor, so the scroll lands on whatever element is under the point: exactly like physically rolling the wheel over it. This is the ONLY way to scroll a nested overflow:auto region (e.g. a scrollable <div> with no tabindex): such regions never take keyboard focus, so the keystroke path below no-ops on them. Use this for inner/nested scrollers in web views.
• Keystroke path (focused region): when you pass NO target (just pid + direction): synthesizes PageDown/PageUp (by='page') or Down/Up arrows (by='line'); horizontal uses Left/Right arrows. Drives the focused / page scroller only.
Mapping: by='page' → larger step; by='line' → smaller step; amount = number of wheel notches (targeted path) or keystroke repetitions (keystroke path).
macOS parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
amount | integer | Pixel-wheel path: number of wheel notches. Keystroke path: number of keystroke repetitions. Larger requests are clamped to the maximum. Default: 3. Range: 1 to 50. | |
delivery_mode | "background" | "foreground" | Best-effort-background ladder rung (default "background"). "background": inject without fronting or raising the target; no focus steal. "foreground": briefly front the target, act, then restore the prior frontmost; the explicit last resort when a background attempt didn't land. Re-call with "foreground" only for the action that needs it. | |
element_token | string | Opaque per-snapshot element handle from structuredContent.elements[].element_token. Returns an explicit stale error naming the current snapshots once a newer read supersedes it. | |
pid | integer | Target process ID. Required unless scope is "desktop". | |
scope | "window" | "desktop" | "window" | Use desktop with x,y and no pid/window_id for native get_desktop_state screenshot coordinates. |
window_id | integer | CGWindowID of the target window. Required with x/y; optional with element_token (the token carries it). | |
x | number | Window-local screenshot X (top-left origin of the PNG from get_window_state). With y, routes through the pixel-wheel path at this point: use for a scrollable surface that isn't in the AX tree. Requires window_id to anchor the window→screen conversion. | |
y | number | Window-local screenshot Y. See x. |
Parameters on every platform
| Parameter | Type | Default | Description |
|---|---|---|---|
by | "line" | "page" | Scroll granularity. Default: line. | |
direction | "up" | "down" | "left" | "right" | required | Scroll direction. |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. | |
target | window target | desktop target | Preferred per-call target: an exact window (kind="window", pid, window_id) or the primary desktop (kind="desktop", display_id="primary"). |
Example arguments
{"direction":"up"}move_cursor#Effect: mutating, idempotent. Platforms: macOS, Linux, Windows.
Move a cursor to (x, y). In window scope (default), moves only the agent cursor overlay. With scope=desktop, moves the real OS pointer in native get_desktop_state screenshot coordinates.
macOS parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
scope | "window" | "desktop" | "window" | "window" (default) moves only the agent cursor overlay; "desktop" moves the real OS pointer. |
x | number | required | Destination X. Window scope: screen points for the agent cursor overlay. Desktop scope: native get_desktop_state screenshot pixels. |
Parameters on every platform
| Parameter | Type | Default | Description |
|---|---|---|---|
cursor_id | string | Cursor instance to move. Default: 'default'. | |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. | |
target | window target | desktop target | Preferred per-call target: an exact window (kind="window", pid, window_id) or the primary desktop (kind="desktop", display_id="primary"). | |
y | number | required | Destination Y, in the same space as x. |
Example arguments
{"x":100,"y":200}mouse_button_down#Press and hold a mouse button at (x,y) via background X11 delivery. Does not release the button; pair with mouse_drag / mouse_button_up. Returns the current held-button state.
Effect: destructive. Platforms: Linux.
| Parameter | Type | Default | Description |
|---|---|---|---|
button | "left" | "right" | "middle" | Mouse button. Default "left". | |
coordinate_frame | "window" | "desktop" | Frame of x/y (and from_x/from_y/to_x/to_y). Default "window": window-local screenshot pixels as returned by get_window_state. "desktop": full-screen pixels as returned by get_desktop_state, translated to the target window. Passing scope:"desktop" together with pid/window_id means the same thing. | |
cursor_id | string | Optional multi-cursor instance id. Default: 'default'. | |
from_zoom | boolean | Set true after a zoom call to auto-translate zoom-image pixel coordinates back to full-window space. | |
pid | integer | required | Target process ID. |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. When both are present, session takes precedence over cursor_id. | |
window_id | integer | required | Window id from list_windows. |
x | number | required | Window-local pixel X of the target window's own get_window_state screenshot (0..screenshot_width). For get_desktop_state pixels pass scope:"desktop" (or coordinate_frame:"desktop"). |
y | number | required | Window-local pixel Y of the target window's own get_window_state screenshot (0..screenshot_height); see x. |
Example arguments
{"pid":1,"window_id":1,"x":100,"y":200}mouse_button_up#Release a previously-held mouse button via background X11 delivery. If x/y are omitted, releases at the last held position. Returns the current held-button state.
Effect: destructive. Platforms: Linux.
| Parameter | Type | Default | Description |
|---|---|---|---|
coordinate_frame | "window" | "desktop" | Frame of x/y (and from_x/from_y/to_x/to_y). Default "window": window-local screenshot pixels as returned by get_window_state. "desktop": full-screen pixels as returned by get_desktop_state, translated to the target window. Passing scope:"desktop" together with pid/window_id means the same thing. | |
cursor_id | string | Optional multi-cursor instance id. Default: 'default'. | |
from_zoom | boolean | Set true after a zoom call to auto-translate zoom-image pixel coordinates back to full-window space. | |
pid | integer | Optional process ID; must match the held button's target. | |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. When both are present, session takes precedence over cursor_id. | |
window_id | integer | Optional window id; must match the held button's target. | |
x | number | Window-local pixel X of the target window's own get_window_state screenshot (0..screenshot_width). For get_desktop_state pixels pass scope:"desktop" (or coordinate_frame:"desktop"). | |
y | number | Window-local pixel Y of the target window's own get_window_state screenshot (0..screenshot_height); see x. |
mouse_drag#Move a previously-held mouse button to a new point via background X11 delivery. Requires an active mouse_button_down state; does not release the button. Returns the updated held-button state.
Effect: destructive. Platforms: Linux.
| Parameter | Type | Default | Description |
|---|---|---|---|
cursor_id | string | Optional multi-cursor instance id. Default: 'default'. | |
duration_ms | integer | Total drag duration. Default: 500. Range: 0 to 10000. | |
from_zoom | boolean | Set true after a zoom call to auto-translate zoom-image pixel coordinates back to full-window space. | |
pid | integer | Optional process ID; must match the held button's target. | |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. When both are present, session takes precedence over cursor_id. | |
steps | integer | Intermediate MotionNotify events. Default: 20. Range: 1 to 200. | |
window_id | integer | Optional window id; must match the held button's target. | |
x | number | required | Window-local pixel X of the target window's own get_window_state screenshot (0..screenshot_width). For get_desktop_state pixels pass scope:"desktop" (or coordinate_frame:"desktop"). |
y | number | required | Window-local pixel Y of the target window's own get_window_state screenshot (0..screenshot_height); see x. |
Example arguments
{"x":100,"y":200}parallel_mouse_drag#Run multiple mouse drag gestures concurrently via Linux MPX/XI2 virtual master pointers. Each drag item runs on its own session-scoped master pointer (true same-window concurrent draws on X11). Each item presses once, glides continuously through its whole path, and releases once: one smooth held drag, not a chain of clicks. A path is given either as a straight segment (from_x/from_y → to_x/to_y) or as a function fn = y(x) sampled over [x_from, x_to] in window-local pixels (e.g. fn:"x" is a diagonal, fn:"300+120*sin(x/40)" a sine wave). Functions support + - * / ^, sin/cos/tan, sqrt, abs, exp, ln, pi, e.
Effect: destructive. Platforms: Linux.
| Parameter | Type | Default | Description |
|---|---|---|---|
drags | object[] | required | Two or more drag gestures to run concurrently, each on its own virtual master pointer. Items: 2 to any. |
Example arguments
{"drags":[{"session":"<session>","window_id":1},{"session":"<session>","window_id":1}]}