Record trajectories and history

Record a trajectory#

A recording saves every action with its arguments and before-and-after screenshots, plus an optional display video. Rendering needs ffmpeg on every platform; capture also needs it on Windows and Linux (macOS 15+ uses ScreenCaptureKit).

Have your agent call start_recording before the task:

{
  "output_dir": "~/cua-trajectories/calendar-demo",
  "record_video": true
}

Actions such as click, type_text, press_key, and scroll are added automatically, one turn-NNNNN folder each. Call stop_recording with {} and wait for it: it finalizes recording.mp4. Then render:

cua-driver recording render \
  ~/cua-trajectories/calendar-demo \
  ~/cua-trajectories/calendar-demo.mp4 \
  --scale 2.5

The renderer zooms around each recorded action; --scale sets the strength and --no-zoom gives a plain render. All options: cua-driver recording and the recording tools.

Computer History#

Computer History keeps an encrypted local record of actions performed through Cua Driver, so an agent can pick up where a previous run stopped. It is an opt-in preview in nightly builds on macOS, Windows, and Linux, and records nothing outside Cua-mediated actions.

cua-driver channel set nightly
cua-driver update --apply
cua-driver history enable
cua-driver history status

enable creates a key in the macOS Keychain, Windows Credential Manager, or Linux Secret Service (which must be unlocked; there is no plaintext fallback) and starts capture. Defaults: 7-day retention, 100 MiB quota, no network I/O.

cua-driver history list 50
cua-driver history show 42
cua-driver history pause
cua-driver history resume
cua-driver history delete --yes

History stores timestamps, opaque IDs, capability names, app identity, and fixed outcome categories. It never stores screenshots, typed text, keystrokes, clipboard, tool arguments or results, accessibility trees, file paths, window titles, or URLs. Deleting destroys the key; it cannot erase backups or snapshots of the ciphertext.

Let an agent read history#

A runtime that admits the preview exposes two read-only tools: history_status (capability history.status) and history_query (history.query, at most 200 events, filters limit, session_id, since_sequence, until_sequence). Every call passes the normal permission stack; a bounded manifest must name both:

version: 3
expires_after: 1h
idle_timeout: 10m
resources:
  computer_history:
    operations:
      - status
      - query
allow:
  tools:
    - history_status
    - history_query

Having the tools does not make a model use them. For "continue" or "what did I do" requests, give the agent a trusted instruction (or run the status-then-query preflight in your host) to check history first, treat events as metadata, and verify current state before acting. A query that returns events appends an encrypted access record. Agents cannot enable, pause, delete, or export history.

The event schema (CloudEvents 1.0, urn:cua-driver:schema:history-event:v0) and storage profile are specified in the agent integration RFC.