Streams and presence

Media sessions of a Space and presence (who is connected, and their cursors).

Open a media session for another client, stream frames to callbacks, and join presence. PresenceView and the presence_cursor_art* functions draw the cursors the same way in every Cua UI.

Space streams#

Methods of Space.

`Space.open_stream`#

Mints a media session (ticket + WebSocket URL) for another client to attach to.

async def open_stream(self, options: SpaceStreamOptions) -> SpaceStreamTicket

Parameter	Type	Default
`options`	`SpaceStreamOptions`	required

Returns SpaceStreamTicket · Async · Raises CuaError

`Space.attach_stream`#

Attaches to a ticket from Self.open_stream and delivers the media socket's encoded frames and audio packets to the sinks as they arrive. Keyframe gating, loss recovery and decoding are the caller's; Self.stream_session does them and ships with Cua Spaces. End the session with Self.close_stream and the ticket's media_session_id.

async def attach_stream(self, ticket: SpaceStreamTicket, frames: FrameSink, audio: Optional[AudioSink]) -> MediaSession

Parameter	Type	Default
`ticket`	`SpaceStreamTicket`	required
`frames`	`FrameSink`	required
`audio`	`Option<AudioSink>`	required

Returns MediaSession · Async · Raises CuaError

`Space.stream_session`#

Opens a media session and delivers keyframe-gated encoded frames and audio to the sinks (one delivery thread; decoding stays with the caller). The streaming client ships with Cua Spaces (source-available, FSL-1.1-MIT): without it this raises HostCapabilityMissing; Self.attach_stream needs no client.

async def stream_session(self, options: SpaceStreamOptions, frames: FrameSink, audio: Optional[AudioSink]) -> SpaceStreamSession

Parameter	Type	Default
`options`	`SpaceStreamOptions`	required
`frames`	`FrameSink`	required
`audio`	`Option<AudioSink>`	required

Returns SpaceStreamSession · Async · Raises CuaError (HostCapabilityMissing)

`Space.close_stream`#

Closes a media session opened with Space.open_stream.

async def close_stream(self, media_session_id: str) -> None

Parameter	Type	Default
`media_session_id`	`String`	required

Async · Raises CuaError

`Space.websocket_headers`#

Headers a media WebSocket to this Space needs besides its ticket.

async def websocket_headers(self) -> List[HttpHeader]

Returns Vec<HttpHeader> · Async · Raises CuaError

`Space.windows`#

Streamable windows, optionally of one app.

async def windows(self, app: Optional[str]) -> List[SpaceWindow]

Parameter	Type	Default
`app`	`Option<String>`	required

Returns Vec<SpaceWindow> · Async · Raises CuaError

`Space.join_presence`#

Joins presence. Waits up to timeout_ms (default 10 s) for the roster.

async def join_presence(self, identity: PresenceIdentity, timeout_ms: Optional[int]) -> SpacePresence

Parameter	Type	Default
`identity`	`PresenceIdentity`	required
`timeout_ms`	`Option<u64>`	required

Returns SpacePresence · Async · Raises CuaError

`SpaceStreamSession`#

A live Space media session (see Space.stream_session).

Returned by Space.stream_session.

Method	Description
`close`	Closes the socket and the media session.
`request_keyframe`	Asks the encoder for a keyframe.
`send_text`	Sends a raw JSON control message (for example input events).

Accessor	Returns	Description
`codec()`	`String`	Negotiated codec.
`is_open()`	`bool`	Whether the socket is still open.
`media_session_id()`	`String`	Media session id.
`stats()`	`SpaceStreamStats`	Delivery counters (zeros once closed).

`SpaceStreamSession.close`#

Closes the socket and the media session. Returns the final counters.

async def close(self) -> SpaceStreamStats

Returns SpaceStreamStats · Async · Raises CuaError

`SpaceStreamSession.request_keyframe`#

Asks the encoder for a keyframe.

def request_keyframe(self) -> None

Raises CuaError

`SpaceStreamSession.send_text`#

Sends a raw JSON control message (for example input events).

def send_text(self, json: str) -> None

Parameter	Type	Default
`json`	`String`	required

Raises CuaError

`SpaceStreamOptions` record#

What to stream and how.

Field	Type	Default	Description
`window_id` / `windowId`	`Option<String>`	`None`	Window handle (wins over `app_name` and `display`).
`app_name` / `appName`	`Option<String>`	`None`	The largest window of this app.
`display`	`Option<String>`	`None`	Display id (`None` = primary).
`codecs`	`Vec<String>`	`[]`	Codecs in preference order (`h264`, `bgra`, `png`; empty = any).
`max_fps` / `maxFps`	`u32`	`0`	FPS cap (0 = 30).
`max_dimension` / `maxDimension`	`u32`	`0`	Long-edge cap (0 = native).
`audio`	`bool`	`false`	Request the paired audio track.
`ticket_ttl_ms` / `ticketTtlMs`	`Option<u64>`	`None`	Ticket lifetime (default 60 s).
`policy`	`Option<String>`	`None`	Input policy: `view_only` (default), `background_only` (input without activating the target) or `allow_activation`.

`SpaceStreamTicket` record#

A minted media session.

Returned by Space.open_stream.

Field	Type	Description
`space`	`String`	Space id.
`media_session_id` / `mediaSessionId`	`String`	Media session id.
`ws_url` / `wsUrl`	`String`	Media WebSocket URL, ticket included (the daemon passthrough in daemon mode).
`ticket`	`String`	The bare ticket.
`ticket_expires_at` / `ticketExpiresAt`	`Option<String>`	Expiry (RFC 3339).
`codec`	`String`	`h264`, `bgra` or `png`.
`wire_version` / `wireVersion`	`u32`	Media wire version.
`width`	`u32`	Initial width.
`height`	`u32`	Initial height.
`needs_headers` / `needsHeaders`	`bool`	Attaching needs `Space.websocket_headers` (Fleet gateway or the daemon passthrough).

`SpaceStreamStats` record#

Delivery counters of a SpaceStreamSession.

Returned by SpaceStreamSession.close, SpaceStreamSession.stats.

Field	Type	Description
`frames`	`u64`	Frames delivered.
`keyframes`	`u64`	Keyframes delivered.
`frames_dropped` / `framesDropped`	`u64`	Frames dropped (sink behind; each forces a keyframe resync).
`frames_gated` / `framesGated`	`u64`	Frames held back awaiting a keyframe.
`keyframe_requests` / `keyframeRequests`	`u64`	Keyframe requests sent.
`audio_packets` / `audioPackets`	`u64`	Audio packets delivered.
`audio_lost` / `audioLost`	`u64`	Audio packets reported lost.
`events`	`u64`	Control events delivered.
`malformed`	`u64`	Malformed messages ignored.

`SpacePresence`#

A joined presence session.

Returned by Space.join_presence.

Method	Description
`leave`	Leaves.
`me`	This participant.
`next_event`	The next event, or `None` when `timeout_ms` (default 5 s) passes.
`roster`	Everyone present, with cursors when known.
`update_cursor`	Moves this participant's cursor.

Accessor	Returns	Description
`uses_datagrams()`	`bool`	Whether cursors travel over the QUIC datagram channel (else the `Join` stream and `UpdateCursor`).
`view()`	`PresenceView`	A fresh `PresenceView` seeded with the caller and the roster at join, with the render delay matching the transport. Feed it every event from `next_event`.

`SpacePresence.leave`#

Leaves.

async def leave(self) -> None

Async · Raises CuaError

`SpacePresence.me`#

This participant.

async def me(self) -> PresenceParticipant

Returns PresenceParticipant · Async · Raises CuaError

`SpacePresence.next_event`#

The next event, or None when timeout_ms (default 5 s) passes.

async def next_event(self, timeout_ms: Optional[int]) -> Optional[PresenceEvent]

Parameter	Type	Default
`timeout_ms`	`Option<u64>`	required

Returns Option<PresenceEvent> · Async · Raises CuaError

`SpacePresence.roster`#

Everyone present, with cursors when known.

async def roster(self) -> List[PresenceMember]

Returns Vec<PresenceMember> · Async · Raises CuaError

`SpacePresence.update_cursor`#

Moves this participant's cursor. Throttled to 30 Hz, newest wins: a move inside the interval is held and sent at its end, and show, hide and target changes go out at once. Never waits for next_event.

async def update_cursor(self, cursor: PresenceCursor) -> None

Parameter	Type	Default
`cursor`	`PresenceCursor`	required

Async · Raises CuaError

`PresenceIdentity` record#

Who joins presence.

Field	Type	Default	Description
`id`	`String`		Stable principal id.
`display_name` / `displayName`	`String`		Display name.
`color`	`String`	`""`	Requested color.
`agent`	`bool`	`false`	An agent rather than a human.

`PresenceParticipant` record#

A presence participant.

Returned by PresenceView.participant, SpacePresence.me.

Field	Type	Description
`participant_id` / `participantId`	`String`	Participant id (per join).
`principal_id` / `principalId`	`String`	Principal id.
`display_name` / `displayName`	`String`	Display name.
`color`	`String`	Color.
`kind`	`String`	`human` or `agent`.

`PresenceMember` record#

A roster entry.

Returned by SpacePresence.roster.

Field	Type	Default	Description
`participant`	`PresenceParticipant`		Who.
`cursor`	`Option<PresenceCursor>`		Their cursor, if known.

`PresenceCursor` record#

A normalized cursor.

Field	Type	Default	Description
`display_id` / `displayId`	`String`	`""`	Display id (empty = primary).
`window_id` / `windowId`	`Option<String>`	`None`	Window handle.
`x`	`f64`		X in `[0, 1]`.
`y`	`f64`		Y in `[0, 1]`.
`visible`	`bool`	`true`	Visible.
`pressed`	`bool`	`false`	A button is down (datagram channel only).
`shape`	`String`	`"arrow"`	The guest's cursor shape here (server-computed): `arrow`, `text`, `pointer`, `resize_ns`, `resize_ew`, `resize_nesw`, `resize_nwse`, `wait`, `progress`, `not_allowed`, `crosshair`, `grab`, `grabbing`, `move`. Ignored when publishing.
`shape_source` / `shapeSource`	`String`	`"unspecified"`	How `shape` was determined: `unspecified`, `hit_test`, `system` or `probe`. Ignored when publishing.
`at_ms` / `atMs`	`f64`	`0.0`	Server time of the sample in milliseconds (0 = unknown).
`received_ms` / `receivedMs`	`f64`	`0.0`	Local Unix time it was received in milliseconds (0 = unknown).

`PresenceEvent` record#

A presence event.

Returned by SpacePresence.next_event.

Field	Type	Default	Description
`kind`	`String`		`joined`, `left`, `cursor_moved`, `shape_changed`, `heartbeat` or `keep_alive`.
`participant`	`Option<PresenceParticipant>`		`joined`: who.
`participant_id` / `participantId`	`Option<String>`		`left` / `cursor_moved` / `shape_changed`: whose.
`cursor`	`Option<PresenceCursor>`		`cursor_moved`: where.
`shape`	`Option<String>`	`None`	`shape_changed`: the new shape.
`shape_source` / `shapeSource`	`Option<String>`	`None`	`shape_changed`: how it was determined.
`reason`	`Option<String>`	`None`	`left`: why (`left`, `disconnected`, `timeout`, `run_ended`, or empty).
`participant_ids` / `participantIds`	`Option<Vec<String>>`	`None`	`heartbeat`: everyone present.

`PresenceView`#

The shared presence netcode model: who is present and what to draw, with remote cursors interpolated (render delay, Catmull-Rom, extrapolation cap), idle cursors faded, stale participants dropped and your own cursor at the local pointer (libs/cua/proto/PRESENCE.md section 4). Feed it every presence event; call drawables each frame.

Returned by SpacePresence.view.

Method	Description
`new`	A view for `me` (the caller's participant id), rendering remote cursors `delay_ms` behind (default 100; 66 on the datagram channel).
`apply`	Folds one event in at `local_ms` (see `presence_now_ms`).
`drawables`	Everything to draw at `local_ms`, your own cursor at `pointer` when the local pointer is over the surface.
`expire`	Drops everyone but the caller when heartbeats stopped for 3 intervals.
`participant`	A participant.
`set_heartbeat_interval_ms`	The expected heartbeat interval (default 5000 ms).
`shape_of`	A participant's current shape name (the caller's included).
`upsert`	Adds or updates a participant.

Accessor	Returns	Description
`participant_ids()`	`Vec<String>`	Participant ids, the caller first.

`PresenceView.new`#

A view for me (the caller's participant id), rendering remote cursors delay_ms behind (default 100; 66 on the datagram channel).

PresenceView(me: str, delay_ms: Optional[float] = None)

Parameter	Type	Default
`me`	`String`	required
`delay_ms`	`Option<f64>`	`None`

Returns PresenceView

`PresenceView.apply`#

Folds one event in at local_ms (see presence_now_ms). Returns whether anything changed.

def apply(self, event: PresenceEvent, local_ms: float) -> bool

Parameter	Type	Default
`event`	`PresenceEvent`	required
`local_ms`	`f64`	required

Returns bool

`PresenceView.drawables`#

Everything to draw at local_ms, your own cursor at pointer when the local pointer is over the surface.

def drawables(self, local_ms: float, pointer: Optional[PresencePoint] = None) -> List[PresenceDrawable]

Parameter	Type	Default
`local_ms`	`f64`	required
`pointer`	`Option<PresencePoint>`	`None`

Returns Vec<PresenceDrawable>

`PresenceView.expire`#

Drops everyone but the caller when heartbeats stopped for 3 intervals. Returns the removed ids.

def expire(self, local_ms: float) -> List[str]

Parameter	Type	Default
`local_ms`	`f64`	required

Returns Vec<String>

`PresenceView.participant`#

A participant.

def participant(self, participant_id: str) -> Optional[PresenceParticipant]

Parameter	Type	Default
`participant_id`	`String`	required

Returns Option<PresenceParticipant>

`PresenceView.set_heartbeat_interval_ms`#

The expected heartbeat interval (default 5000 ms).

def set_heartbeat_interval_ms(self, ms: float) -> None

Parameter	Type	Default
`ms`	`f64`	required

`PresenceView.shape_of`#

A participant's current shape name (the caller's included).

def shape_of(self, participant_id: str) -> Optional[str]

Parameter	Type	Default
`participant_id`	`String`	required

Returns Option<String>

`PresenceView.upsert`#

Adds or updates a participant.

def upsert(self, participant: PresenceParticipant) -> None

Parameter	Type	Default
`participant`	`PresenceParticipant`	required

`PresenceDrawable` record#

One cursor to draw now (see PresenceView.drawables).

Returned by PresenceView.drawables.

Field	Type	Description
`participant_id` / `participantId`	`String`	Whose.
`display_name` / `displayName`	`String`	Their display name.
`color`	`String`	Their color (`#rrggbb`).
`is_me` / `isMe`	`bool`	This client's own cursor, drawn at the local pointer.
`is_agent` / `isAgent`	`bool`	An agent.
`x`	`f64`	Normalized x.
`y`	`f64`	Normalized y.
`shape`	`String`	Shape name (see `PresenceCursor.shape`).
`shape_source` / `shapeSource`	`String`	How the shape was determined.
`alpha`	`f64`	Opacity in `[0, 1]` (idle cursors fade).
`display_id` / `displayId`	`String`	Display id of the cursor's target.
`window_id` / `windowId`	`Option<String>`	Window id, when over a window stream.

`PresencePoint` record#

A normalized point.

Field	Type	Default	Description
`x`	`f64`		X in `[0, 1]`.
`y`	`f64`		Y in `[0, 1]`.

`CursorArt` record#

One cursor shape's shared art: an SVG path on a square canvas, filled with the participant's color over an outline.

Returned by presence_cursor_art, presence_cursor_art_all.

Field	Type	Description
`shape`	`String`	Shape name.
`path_d` / `pathD`	`String`	SVG path data (absolute `M`, `L`, `C`, `Z`; nonzero fill).
`hotspot_x` / `hotspotX`	`f64`	Hot spot x on the canvas.
`hotspot_y` / `hotspotY`	`f64`	Hot spot y on the canvas.
`canvas`	`f64`	Canvas size (square).
`outline_color` / `outlineColor`	`String`	Outline color, drawn under the fill.
`outline_width` / `outlineWidth`	`f64`	Outline width in canvas units.

`presence_color`#

The stable presence color of an agent or other principal (#rrggbb): a cursor palette color picked by a hash of id. Request it when the agent joins presence (PresenceIdentity.color) and use it as the agent's avatar background, so both come from one source.

def presence_color(id: str) -> str

Parameter	Type	Default
`id`	`String`	required

Returns String

`presence_text_color`#

Black or white text (#000000 / #ffffff), whichever reads better on background (#rrggbb).

def presence_text_color(background: str) -> str

Parameter	Type	Default
`background`	`String`	required

Returns String

`presence_now_ms`#

The local clock PresenceView expects: Unix time in milliseconds.

def presence_now_ms() -> float

Returns f64

`presence_cursor_art`#

The shared art for shape (unknown names give the arrow).

def presence_cursor_art(shape: str) -> CursorArt

Parameter	Type	Default
`shape`	`String`	required

Returns CursorArt

`presence_cursor_art_all`#

The shared art for every shape.

def presence_cursor_art_all() -> List[CursorArt]

Returns Vec<CursorArt>

`presence_cursor_art_svg`#

A standalone SVG of shape in color, size pixels square.

def presence_cursor_art_svg(shape: str, color: str, size: float) -> str

Parameter	Type	Default
`shape`	`String`	required
`color`	`String`	required
`size`	`f64`	required

Returns String

`presence_conformance_json`#

The presence conformance vectors (JSON), for binding test suites.

def presence_conformance_json() -> str

Returns String

Streams and presence

Space streams#

Space.open_stream#

Space.attach_stream#

Space.stream_session#

Space.close_stream#

Space.websocket_headers#

Space.windows#

Space.join_presence#

SpaceStreamSession#

SpaceStreamSession.close#

SpaceStreamSession.request_keyframe#

SpaceStreamSession.send_text#

SpaceStreamOptions record#

SpaceStreamTicket record#

SpaceStreamStats record#

SpacePresence#

SpacePresence.leave#

SpacePresence.me#

SpacePresence.next_event#

SpacePresence.roster#

SpacePresence.update_cursor#

PresenceIdentity record#

PresenceParticipant record#

PresenceMember record#

PresenceCursor record#

PresenceEvent record#

PresenceView#

PresenceView.new#

PresenceView.apply#

PresenceView.drawables#

PresenceView.expire#

PresenceView.participant#

PresenceView.set_heartbeat_interval_ms#

PresenceView.shape_of#

PresenceView.upsert#

PresenceDrawable record#

PresencePoint record#

CursorArt record#

presence_color#

presence_text_color#

presence_now_ms#

presence_cursor_art#

presence_cursor_art_all#

presence_cursor_art_svg#

presence_conformance_json#

Space streams#

Space.open_stream#

Space.attach_stream#

Space.stream_session#

Space.close_stream#

Space.websocket_headers#

Space.windows#

Space.join_presence#

SpaceStreamSession#

SpaceStreamSession.close#

SpaceStreamSession.request_keyframe#

SpaceStreamSession.send_text#

SpaceStreamOptions record#

SpaceStreamTicket record#

SpaceStreamStats record#

SpacePresence#

SpacePresence.leave#

SpacePresence.me#

SpacePresence.next_event#

SpacePresence.roster#

SpacePresence.update_cursor#

PresenceIdentity record#

PresenceParticipant record#

PresenceMember record#

PresenceCursor record#

PresenceEvent record#

PresenceView#

PresenceView.new#

PresenceView.apply#

PresenceView.drawables#

PresenceView.expire#

PresenceView.participant#

PresenceView.set_heartbeat_interval_ms#

PresenceView.shape_of#

`Space.open_stream`#

`Space.attach_stream`#

`Space.stream_session`#

`Space.close_stream`#

`Space.websocket_headers`#

`Space.windows`#

`Space.join_presence`#

`SpaceStreamSession`#

`SpaceStreamSession.close`#

`SpaceStreamSession.request_keyframe`#

`SpaceStreamSession.send_text`#

`SpaceStreamOptions` record#

`SpaceStreamTicket` record#

`SpaceStreamStats` record#

`SpacePresence`#

`SpacePresence.leave`#

`SpacePresence.me`#

`SpacePresence.next_event`#

`SpacePresence.roster`#

`SpacePresence.update_cursor`#

`PresenceIdentity` record#

`PresenceParticipant` record#

`PresenceMember` record#

`PresenceCursor` record#

`PresenceEvent` record#

`PresenceView`#

`PresenceView.new`#

`PresenceView.apply`#

`PresenceView.drawables`#

`PresenceView.expire`#

`PresenceView.participant`#

`PresenceView.set_heartbeat_interval_ms`#

`PresenceView.shape_of`#

`PresenceView.upsert`#

`PresenceDrawable` record#

`PresencePoint` record#

`CursorArt` record#

`presence_color`#

`presence_text_color`#

`presence_now_ms`#

`presence_cursor_art`#

`presence_cursor_art_all`#

`presence_cursor_art_svg`#

`presence_conformance_json`#

`Space.open_stream`#

`Space.attach_stream`#

`Space.stream_session`#

`Space.close_stream`#

`Space.websocket_headers`#

`Space.windows`#

`Space.join_presence`#

`SpaceStreamSession`#

`SpaceStreamSession.close`#

`SpaceStreamSession.request_keyframe`#

`SpaceStreamSession.send_text`#

`SpaceStreamOptions` record#

`SpaceStreamTicket` record#

`SpaceStreamStats` record#

`SpacePresence`#

`SpacePresence.leave`#

`SpacePresence.me`#

`SpacePresence.next_event`#

`SpacePresence.roster`#

`SpacePresence.update_cursor`#

`PresenceIdentity` record#

`PresenceParticipant` record#

`PresenceMember` record#

`PresenceCursor` record#

`PresenceEvent` record#

`PresenceView`#

`PresenceView.new`#

`PresenceView.apply`#

`PresenceView.drawables`#

`PresenceView.expire`#

`PresenceView.participant`#

`PresenceView.set_heartbeat_interval_ms`#

`PresenceView.shape_of`#

`PresenceView.upsert`#

`PresenceDrawable` record#

`PresencePoint` record#