Interfaces

Shell, mouse, keyboard, screen, clipboard, window, terminal, files, apps, mobile and driver interfaces.

Every sandbox exposes these interfaces as attributes (sb.shell, sb.mouse, ...). Input goes through cua-driver inside the guest.

Without cua-spacesd these fall back to the runtime's agentless path (QMP, VNC, SSH or ADB) where one exists, and otherwise raise SpacesdNotAvailable. Coordinates are screen pixels; input goes through Cua Driver in the guest.

Shell commands#

Shell#

class Shell

Shell command execution.

Shell.run#

async def run(
    command: str,
    timeout: int = 30,
    background: bool = False,
) -> CommandResult

Run a shell command and return the result.

When background=True, returns immediately with stdout=str(pid) and returncode=0; poll for completion via a sentinel file or by inspecting the process list.

CommandResult#

class CommandResult

Input and display#

Mouse#

class Mouse

Mouse control.

Mouse.click#

async def click(x: int, y: int, button: str = 'left') -> None

Example

import asyncio
 
from cua_sandbox import Sandbox
 
async def main():
    async with Sandbox.connect(url=URL, token=TOKEN) as sb:
        width, height = await sb.screen.size()
        await sb.mouse.click(width // 2, height // 2)
        await sb.keyboard.type("hello")
        await sb.keyboard.keypress(["ctrl", "a"])
        png = await sb.screenshot()
        print(png[:4])
        # b'\x89PNG'
 
asyncio.run(main())

Mouse.right_click#

async def right_click(x: int, y: int) -> None

Mouse.double_click#

async def double_click(x: int, y: int) -> None

Mouse.move#

async def move(x: int, y: int) -> None

Mouse.scroll#

async def scroll(x: int, y: int, scroll_x: int = 0, scroll_y: int = 3) -> None

Scroll the wheel with the pointer at (x, y).

scroll_x/scroll_y are wheel notches, not a position: positive scroll_y scrolls up, negative scrolls down; positive scroll_x scrolls right.

Mouse.mouse_down#

async def mouse_down(x: int, y: int, button: str = 'left') -> None

Mouse.mouse_up#

async def mouse_up(x: int, y: int, button: str = 'left') -> None

Mouse.drag#

async def drag(
    start_x: int,
    start_y: int,
    end_x: int,
    end_y: int,
    button: str = 'left',
) -> None

Keyboard#

class Keyboard

Keyboard control.

Keyboard.type#

async def type(text: str) -> None

Type a string of text.

Keyboard.keypress#

async def keypress(keys: Union[List[str], str]) -> None

Press a key combination (e.g. ["ctrl", "c"] or "enter").

Keyboard.key_down#

async def key_down(key: str) -> None

Keyboard.key_up#

async def key_up(key: str) -> None

Screen#

class Screen

Screen capture and info.

Screen.screenshot#

async def screenshot(format: str = 'png', quality: int = 95) -> bytes

Capture a screenshot and return raw image bytes.

Screen.screenshot_base64#

async def screenshot_base64(format: str = 'png', quality: int = 95) -> str

Capture a screenshot and return as a base64-encoded string.

Screen.size#

async def size() -> Tuple[int, int]

Return (width, height) of the screen.

Clipboard#

class Clipboard

Clipboard read/write.

Clipboard.get#

async def get() -> str

Return the current clipboard text.

Clipboard.set#

async def set(text: str) -> None

Set the clipboard text.

Window#

class Window

Window management.

Window.get_active_title#

async def get_active_title() -> str

Return the title of the currently focused window ("" when none).

cua-spacesd answers from WindowsService.ListWindows (the focused window); computer-server never implemented this command.

Terminal, files and apps#

Terminal#

class Terminal

PTY terminal sessions.

Terminal.create#

async def create(command: Optional[str] = None, cols: int = 80, rows: int = 24) -> dict

Create a new PTY session. Returns {"pid": int, "cols": int, "rows": int}.

Terminal.send_input#

async def send_input(pid: int, data: str) -> None

Send input to a PTY session.

Terminal.info#

async def info(pid: int) -> Optional[dict]

Return session info or None if gone.

Terminal.close#

async def close(pid: int) -> bool

Kill a PTY session.

Files#

class Files

File and directory operations inside the sandbox.

All paths are inside the sandbox unless otherwise noted.

Files.exists#

async def exists(path: str) -> bool

Files.is_dir#

async def is_dir(path: str) -> bool

Files.size#

async def size(path: str) -> int

Files.list#

async def list(path: str) -> list[FileEntry]

Files.make_dir#

async def make_dir(path: str) -> None

Files.remove_dir#

async def remove_dir(path: str) -> None

Files.remove#

async def remove(path: str) -> None

Files.read_text#

async def read_text(path: str) -> str

Files.write_text#

async def write_text(path: str, content: str) -> None

Example

import asyncio
 
from cua_sandbox import Sandbox
 
async def main():
    async with Sandbox.connect(url=URL, token=TOKEN) as sb:
        await sb.files.write_text("/tmp/note.txt", "hi")
        print([await sb.files.read_text("/tmp/note.txt")])
        # ['hi']
 
asyncio.run(main())

Files.read_bytes#

async def read_bytes(path: str, offset: int = 0, length: Optional[int] = None) -> bytes

Files.write_bytes#

async def write_bytes(path: str, content: bytes) -> None

Files.upload#

async def upload(local_path: Union[str, Path], remote_path: str) -> None

Push a file from the host into the sandbox.

Files.download#

async def download(remote_path: str, local_path: Union[str, Path]) -> None

Pull a file from the sandbox down to the host.

FileEntry#

class FileEntry

Apps#

class Apps

Install and launch cataloged applications inside the sandbox.

Apps.install#

async def install(app_id: str) -> CommandResult

Install an app by its catalog ID (e.g. "unity", "godot-engine").

Apps.launch#

async def launch(app_id: str) -> CommandResult

Launch a previously-installed app by its catalog ID.

Mobile gestures#

Mobile#

class Mobile

Mobile (Android) touch and hardware-key control.

All touch coordinates are in screen pixels. Single-touch methods use input tap/swipe via adb shell. Multi-touch gestures delegate to the multitouch_gesture transport action, which uses adb root + MT Protocol B sendevent for reliable injection.

Mobile.tap#

async def tap(x: int, y: int) -> None

Mobile.long_press#

async def long_press(x: int, y: int, duration_ms: int = 1000) -> None

Mobile.double_tap#

async def double_tap(x: int, y: int, delay: float = 0.1) -> None

Mobile.type_text#

async def type_text(text: str) -> None

Mobile.swipe#

async def swipe(x1: int, y1: int, x2: int, y2: int, duration_ms: int = 300) -> None

Mobile.scroll_up#

async def scroll_up(x: int, y: int, distance: int = 600, duration_ms: int = 400) -> None

Mobile.scroll_down#

async def scroll_down(
    x: int,
    y: int,
    distance: int = 600,
    duration_ms: int = 400,
) -> None

Mobile.scroll_left#

async def scroll_left(
    x: int,
    y: int,
    distance: int = 400,
    duration_ms: int = 300,
) -> None

Mobile.scroll_right#

async def scroll_right(
    x: int,
    y: int,
    distance: int = 400,
    duration_ms: int = 300,
) -> None

Mobile.fling#

async def fling(x1: int, y1: int, x2: int, y2: int) -> None

Mobile.gesture#

async def gesture(
    *finger_paths: tuple[int, int],
    duration_ms: int = 400,
    steps: int = 0,
) -> None

Inject an arbitrary N-finger gesture via MT Protocol B sendevent.

Each positional argument is a sequence of (x, y) waypoints for one finger. Pass an even number of (x, y) tuples and they are paired into start/end positions per finger:

# two-finger pinch-out
await mobile.gesture(
    (cx - 20, cy), (cx - 200, cy),   # finger 0
    (cx + 20, cy), (cx + 200, cy),   # finger 1
)

Delegates to the multitouch_gesture transport action, which uses adb root + MT Protocol B sendevent for reliable injection.

Mobile.pinch_in#

async def pinch_in(cx: int, cy: int, spread: int = 300, duration_ms: int = 400) -> None

Pinch-in (zoom out) with two real simultaneous fingers.

Mobile.pinch_out#

async def pinch_out(cx: int, cy: int, spread: int = 300, duration_ms: int = 400) -> None

Pinch-out (zoom in) with two real simultaneous fingers.

Mobile.key#

async def key(keycode: int) -> None

Mobile.home#

async def home() -> None

Mobile.back#

async def back() -> None

Mobile.recents#

async def recents() -> None

Mobile.power#

async def power() -> None

Mobile.volume_up#

async def volume_up() -> None

Mobile.volume_down#

async def volume_down() -> None

Mobile.enter#

async def enter() -> None

Mobile.backspace#

async def backspace() -> None

Mobile.wake#

async def wake() -> None

Mobile.notifications#

async def notifications() -> None

Mobile.close_notifications#

async def close_notifications() -> None

Cua Driver#

Driver#

class Driver

Accessor for optional typed Driver sessions; does not replace Sandbox transport.

Each connection is a host-bound session with a launcher-selected permission mode (Standard by default). Remote clients cannot select its authority. Use typed inputs with session=None where optional. For required session fields, obtain the bound name with sandbox.driver.session_name(driver). Creating or rebinding trusted sessions is unsupported. Remote cleanup is best effort with a bounded timeout; failures warn and do not block the sandbox's release.

Driver.session_name#

def session_name(driver: CuaDriver) -> str

Return this active connection's host-bound public session label.

This label is not a credential or permission grant. It can populate canonical typed inputs whose session field is required.

Driver.connect#

async def connect(
    *,
    service: str | None = None,
    transport: Literal['envelope', 'mcp'] | None = None,
) -> AsyncIterator[CuaDriver]

Yield the canonical Driver; the receiver must support typed envelopes.

This uses cua-spacesd's /mcp (service="env", transport="mcp") through the cua SDK, and verifies the ai.cua.driver.envelopes extension before opening a receiver. Any other service or transport="envelope" raises DriverConnectionError.

Driver.close#

async def close() -> None

Invalidate all connections before the owning transport is closed.

DriverConnectionError#

class DriverConnectionError(RuntimeError)

Sanitized failure to establish or use a sandbox Driver service.