Computer History Agent Integration

Tool discovery, permissions, query semantics, privacy boundaries, and consultation behavior for agent hosts.

Computer History is an opt-in encrypted local record of actions performed through Cua Driver. Agent runtimes can read bounded metadata through two permission-gated tools. They cannot control capture or access the encryption key and encrypted chunks.

Integration flow#

sequenceDiagram
    participant A as Agent runtime
    participant T as Cua tool registry
    participant P as Cua permission system
    participant H as Encrypted local history
 
    A->>T: Discover tools
    alt History tools absent
        T-->>A: Continue without history
    else History tools present
        A->>T: history_status({})
        T->>P: Authorize history.status
        P-->>T: Allow or deny
        T-->>A: Structured status or denial
        opt A bounded read is useful
            A->>T: history_query(filters)
            T->>P: Authorize history.query
            P-->>T: Allow or deny
            T->>H: Decrypt, validate, filter, and bound
            H-->>T: Metadata-only events
            T->>H: Append encrypted access record
            T-->>A: Events and context disclosure
        end
    end

The access record is appended only when a successful query returns at least one event. The response that caused it does not include the new access record.

Availability and feature detection#

The runtime registers the tools only when the desktop daemon admits the preview. Tool presence does not prove that the user granted the calling agent access.

Consultation behavior#

Tool discovery makes history available to a model. It does not cause the model to call either tool. A history-aware host should consult history when the user asks it to continue, resume, recall recent Cua activity, explain a prior Cua run, or locate where a Cua-mediated workflow stopped.

For a matching request, the host should:

1. discover the history tools before broader desktop inspection;
2. call history_status and retain disabled, paused, unhealthy, and dropped-event state;
3. when useful and authorized, call history_query with a bounded recent slice;
4. treat events as metadata evidence rather than a transcript;
5. keep omitted content, geometry, arguments, results, and user intent unknown;
6. use the returned application or capability as a lead and verify current state through the least intrusive source; and
7. continue without history after absence, denial, empty results, or a recoverable history failure.

A trusted system instruction or bundled skill can guide this behavior. A host may instead perform the status and query preflight before model execution when deterministic consultation is required.

Prompt wording can guide tool selection but does not establish deterministic consultation.

history_status#

Returns operational metadata without returning history events.

Required capability: history.status

Properties: read-only, non-destructive, idempotent, closed-world

Input: an empty object. Unknown fields are rejected.

Important response fields are:

Health categories are ready, disabled, paused, not_admitted, key_unavailable, key_locked, key_corrupt, key_destroy_failed, storage_unavailable, storage_corrupt, quota_reached, events_dropped, and writer_stopped.

history_query#

Returns a bounded event slice that may enter the current model context.

Required capability: history.query

Properties: read-only, non-destructive, closed-world. A successful non-empty query appends an encrypted access record, so the call is not idempotent.

Unknown fields are rejected. When both sequence bounds are present, since_sequence must not exceed until_sequence.

Events are ordered by data.sequence. The query applies every filter, retains the newest limit matching events, and returns that slice in ascending order. The preview has no opaque pagination token. Missing sequence numbers are valid gap evidence because capture uses a bounded nonblocking queue.

Every response includes:

{
  "events": [],
  "metadata_only": true,
  "model_context_disclosure": true
}

Event contract#

Events use CloudEvents 1.0 JSON and urn:cua-driver:schema:history-event:v0.

Clients must branch on both dataschema and type. Stop interpreting an event whose schema is unsupported.

Permission contract#

Status and query are separate private-observation operations. Permission for history.status does not grant history.query.

In bounded mode, the approved manifest must name the tools and the matching computer-history operations:

version: 3
expires_after: 1h
idle_timeout: 10m
resources:
  computer_history:
    operations:
      - status
      - query
allow:
  tools:
    - history_status
    - history_query

An agent may propose this manifest. The trusted launcher selects and approves it. Clients must surface a denial and continue without history. They must not read the store directly, change permission modes, or reconstruct denied history with another observation tool.

Privacy and storage boundary#

Returned events may contain timestamps, opaque identifiers, capability names, fixed application identity fields, and fixed outcome, route, delivery, evidence, lifecycle, access, and health categories.

The contract prohibits screenshots, video, audio, accessibility trees, typed text, raw keystrokes, clipboard contents, raw tool arguments or results, file paths, window titles, URLs, and free-form diagnostics.

Each CloudEvent is encrypted and authenticated inside a COSE_Encrypt0 record. Records use RFC 8742 CBOR Sequence framing. A native credential-store key and per-chunk HKDF keys protect data at rest. History payloads remain local and history tools are excluded from per-tool and agent-session telemetry.

Errors#

Tool failures return a structured code. Authorization may deny the call before tool execution through the normal authorization envelope.

Version identifiers#

The tool and capability names are intended to remain stable. The event schema is experimental. Clients must use runtime tool discovery and advertised input schemas.

Source contracts#

• Full agent integration RFC
• Event schema
• Encrypted storage profile
• Use Computer History