Interface Contracts
The contracts behind the CLI and MCP surfaces: lifecycle sessions, per-call targets, config persistence, and action routing.
The CLI (cua-driver call …) and MCP (cua-driver mcp) run the same
runtime contract, but they differ in what state survives between calls,
where configuration lands, and which parameters a call must carry.
MCP may own that runtime directly or select cua-driver serve explicitly.
CLI versus MCP at a glance#
| Dimension | CLI (call) | MCP (mcp) |
|---|---|---|
| Lifetime | each one-shot call gets a disposable implicit session; the selected service may own other durable state | one implicit session per authenticated transport connection |
element_index cache | owned by the selected service; survives until invalidation or service restart | owned by the MCP runtime; lives across calls |
| Public label | optional session; it does not carry authority | optional session; it does not carry authority |
Where set_config lands | the persisted global default when no public session is supplied | an in-memory override when a public session is supplied |
| Agent cursor | initialized for the call and cleaned up with its disposable session | initialized on the first cursor-bearing action and reused on later unnamed calls |
cua-driver call requires a service at the resolved default endpoint or at the
explicit --socket endpoint. A freshly built binary's behavior appears through
the CLI only after that service restarts, because the action runs in the service
process. Integration tests avoid this ambiguity by starting an isolated runtime.
Bare MCP owns its runtime on Windows/Linux and uses the signed app service on
macOS. mcp --socket <endpoint> explicitly selects a service on every
platform; mcp --direct explicitly selects process ownership and is mutually
exclusive with --socket.
Lifecycle sessions#
start_session is optional. The first admitted stateful call creates one
implicit session for the authenticated transport lease. Later unnamed calls on
that transport reuse the same identity, so several clicks do not create several
sessions. The runtime does not expose its private implicit ID.
The default session idle TTL is five minutes. This timer controls lifecycle and resource cleanup. It is separate from the cursor idle-hide timer, which controls visibility only. A call in flight cannot expire, and a completed call refreshes the lifecycle timer once. Transport close, explicit end, revocation, and idle expiry use the same cleanup hooks.
For multi-call work, prefer a short public session label and pass the same
value on every call that accepts it. Passing it once is not sticky: a later call
that omits the field uses the transport's implicit session instead. Use
start_session to name or configure a run before acting, or to revive a public
name after it has ended. For one-off or deliberately unlabeled work, omitting
session is valid.
get_session reads one visible session, and list_sessions returns content-free
summaries scoped to the caller's transport. Trusted host code has a separate
operator listing. get_session_state and escalate_session remain deprecated
for legacy capture-scope sessions. There is no deescalate_session.
Where settings live#
set_config resolves where a setting is written from whether a public
session is declared. Runtime and transport adapters derive their reserved
internal session fields; caller-supplied reserved values do not grant authority.
| Caller | Public session | Effect |
|---|---|---|
cua-driver config set …, anonymous one-shot cua-driver call | absent | writes the global DriverConfig and persists to ~/.cua-driver/config.json |
MCP or CLI call with a session | present | in-memory override for that session only; no disk write, no clobber of the default |
Every tool then reads the effective value with this precedence:
effective = call-argument > session override > global default (disk)
max_image_dimension follows this precedence. The retired capture_scope
configuration key returns a migration error that points callers to the
per-action target; only the deprecated start_session.capture_scope field
remains for legacy session calls. capture_mode is deprecated and ignored.
get_window_state always returns both the tree and a screenshot. See the
set_config reference for the per-session
isolation details.
The per-call target contract#
Capture modality belongs to each observation or action. A session records the modalities used, but it does not lock the caller into a window or desktop mode.
The input tools move_cursor, click, drag, scroll, type_text,
press_key, and hotkey accept one exact tagged target:
{ "target": { "kind": "window", "pid": 4711, "window_id": 22 } }
{ "target": { "kind": "desktop", "display_id": "primary" } }display_id:"primary" is the portable desktop target in this release. An
unsupported display ID returns invalid_action_target. The target cannot be
combined with legacy scope, pid, or window_id fields, so the driver never
has to guess which coordinate space the caller intended.
| Window target | Desktop target | |
|---|---|---|
| Coordinate space | window-local coordinates from get_window_state | screen coordinates from get_desktop_state |
| Identity | exact pid and window_id | exact display_id, currently primary |
| Delivery | background by default, or foreground per call | foreground |
| Action rung | accessibility element or window-local pixel | screen pixel |
Legacy callers may keep using flat pid, window_id, and scope fields during
the compatibility window. An explicit legacy capture_scope still invokes the
old session gate. New callers should use target and should not call
escalate_session.
How an action is routed#
Every input tool chooses its route from the target and action arguments:
| Arguments | Path | Behavior |
|---|---|---|
window target plus element_index | accessibility action | UIA Invoke, AXPerformAction, or AT-SPI. Background, no cursor move, no focus steal. |
window target plus x, y | window-local pixel | coordinates are relative to that window's screenshot. For keyboard tools, this form can click the coordinate to establish renderer focus before sending keys. |
desktop target plus x, y | screen-absolute | true screen pixels on the primary display, with foreground delivery |
The keyboard family's coordinate form is mutually exclusive with
element_index. A malformed, mixed, or ambiguous target fails before
authorization and platform dispatch.
Valid combinations#
get_window_state returns both the accessibility tree and a screenshot by
default. The enforced combinations are target modality, action rung, and
delivery mode:
| Target | Action rung | delivery_mode | Valid? | Why |
|---|---|---|---|---|
| window | ax (element_index) | background | ✅ | Semantic action on a background-capable window. |
| window | ax (element_index) | foreground | ✅ | Activate, then act by element. |
| window | px (x, y) | background | ✅ | Act at a coordinate from the window screenshot without raising the target when supported. |
| window | px (x, y) | foreground | ✅ | Activate, then act by coordinate. |
| desktop | px (x, y) | foreground | ✅ | Act at screen coordinates on the primary display. |
| desktop | ax (element_index) | any | ❌ | A desktop target has no window element tree. |
| desktop | any | background | ❌ | Screen input lands on the active desktop and has no per-process background route. |
The driver enforces these rejections before it sends input.
What a tool returns#
An action tool answers with one of two payloads, and MCP holds both to the tool's advertised outputSchema — the schema covers every structuredContent a tool emits, refusals included. So outputSchema is an anyOf of two variants.
Success variant. Closed: exactly these keys, nothing else.
| Field | Required | Values |
|---|---|---|
effect | ✅ | confirmed, partial, unverifiable, suspected_noop, refused |
route | ✅ | accessibility, synthetic_events, global_input, system_api, dom, trusted_input |
delivery | — | {mode: background | foreground | not_applicable | unknown} |
evidence | — | what was read back to justify effect |
escalation | — | which rung to try next, and why this one fell short |
Refusal variant. Accompanies isError: true and carries diagnostics instead of an effect. Two shapes are in service:
{"status": "refused",
"refusal": {"code": "stale_element_token",
"message": "element_token is stale; call get_window_state again to refresh"}}{"code": "window_target_not_found", "effect": "refused", "candidates": [], "pid": 4711}This variant is deliberately open — refusals carry tool-specific diagnostic keys (candidates, detail, pid) that help a caller recover. The marker keys refusal, status, and code are what identify a payload as a refusal rather than a malformed success.
Branch on the refusal code, and surface the content text — it states the recovery step in plain language. stale_element_token means re-run get_window_state and retry with fresh indices; it does not mean the accessibility route is unavailable. Agents that treat a refusal as a dead route tend to abandon the element path and fall back to blind pixel clicking, which is both slower and unverifiable.
Success payloads stay strictly validated: an unknown key on the success shape is still a contract violation and will not be quietly accepted through the refusal variant.
Platform support#
| Capability | Windows | macOS | Linux |
|---|---|---|---|
get_window_state returns both tree + screenshot (element ax / px actions) | ✅ | ✅ | ✅ |
delivery_mode: "background" (best-effort background) | ✅ | ✅ | ✅ for semantic AT-SPI actions on X11 and Wayland; native-Wayland raw keyboard input remains limited |
delivery_mode: "foreground" (action-scoped activation and restore) | ✅ | ✅ | ✅ X11; Wayland activation is compositor-constrained |
bring_to_front (persistent activation for focus-proxy surfaces) | ✅ | ✅, chiefly for RDP | ✅ X11 EWMH activation (_NET_ACTIVE_WINDOW + input focus); Wayland raise is compositor-constrained |
get_desktop_state (desktop capture) | ✅ | ✅ | ✅ |
| Per-call window and primary-display targets | ✅ | ✅ | ✅ |
Mental model#
window target = exact pid + window_id; background by default
desktop target = exact display_id; screen coordinates and foreground delivery
session = lifecycle, cursor, recording, cleanup, and telemetry ownership
target = the window or display selected for this call
authority = the independent permission and policy stack
CLI call = one disposable implicit session
MCP = one reusable implicit session per authenticated transport