cb CLI

cua-bench: run computer-use tasks and benchmark datasets on sandboxes.

cb <COMMAND> [OPTIONS]

Run cb <command> --help for the build you have.

Commands#

CommandDescription
cb runRun a task or dataset, and manage runs.
cb run taskRun one task variant.
cb run datasetRun every task and variant of a dataset in parallel.
cb run listList all runs with status.
cb run infoShow detailed info about a run.
cb run watchWatch a run in real-time with live updates.
cb run stopStop a run and release its sandboxes.
cb run logsView combined logs from a run or session.
cb interactRun a task's setup with its desktop visible, then evaluate.
cb taskInspect and manage task environments.
cb task infoShow detailed information about a task.
cb task listList tasks in a directory.
cb task createScaffold a new task environment.
cb task generateGenerate a task from a prompt using Claude.
cb datasetManage datasets and build from outputs.
cb dataset listList available datasets from registry.
cb dataset buildBuild a dataset from batch outputs.
cb traceView and analyze trace datasets.
cb trace viewView a single trace in browser.
cb trace trajView agent trajectories from a run in cua.ai/trajectory-viewer.
cb envList local sandboxes.
cb env lsList local sandboxes.
cb statusShow system status dashboard.
cb pruneClean up cua-bench run data (and 0.2.x image leftovers).
cb agentScaffold custom agents.
cb agent initCreate a new agent from template.
cb agent buildDeprecated: cb run executes agents in-process.
cb imageShow the canonical images (the local image store is deprecated).
cb image listList all images.
cb image infoShow image details.
cb image createDeprecated: create an image from a platform.
cb image deleteDeprecated: delete a stored image.
cb image cloneDeprecated: clone a stored image.
cb image shellDeprecated: interactive shell into a stored image.
cb platformShow available platform configurations.
cb platform listList all available platforms.
cb platform infoShow platform details.

cb run#

Run a task or dataset, and manage runs.

Run a task (a directory with main.py) or a dataset, with an agent or the task's oracle. cb run <path> is shorthand for cb run task|dataset <path>.

cb run [COMMAND]

Examples

# Run a task with its oracle solution
cb run ./tasks/hello_file_env
 
# Run a dataset with an agent, four variants at once
cb run cua-bench-basic --agent cua-agent --model anthropic/claude-sonnet-4-20250514 -j 4
 
# List runs
cb run list

cb run task#

Run one task variant.

cb run task [OPTIONS] <task_path>
ArgumentTypeDefaultDescription
<task_path>stringrequiredPath to task directory (containing main.py).
FlagShortTypeDefaultDescription
--variant-idinteger0Task variant index (default: 0).
--agentstringAgent to use for evaluation (e.g., cua-agent).
--agent-import-pathstringImport path for custom agent (e.g., "path.to.agent:MyCustomAgent").
--modelstringModel to use with the agent (e.g., anthropic/claude-sonnet-4-20250514).
--max-stepsinteger100Maximum number of steps for agent execution (default: 100).
--onlocal | cloud | e2b | daytona | modal | cloudflare | vercel | morph | runloop | fly | blaxel | codesandbox | northflank | aws | gcpWhere sandboxes run: local (this machine), your own cloud account (aws, gcp, modal; connect it first with cua cloud connect <provider>), or a contrib provider (e2b, daytona; needs a cua SDK built with contrib and the provider's key). Default: CUA_DEFAULT_ON, else cua config set default.on, else local.
--kindauto | container | vmWhat kind of sandbox: container or vm. Default auto: Linux as a container unless the task or image says VM; Windows, macOS and Android are VM-only. Default: CUA_DEFAULT_KIND, else cua config set default.kind.
--runtimeauto | gvisor | runc | qemu | lumeWhich engine: gvisor or runc (containers), qemu or lume (VMs). Implies its kind. Default auto: the SDK picks (CUA_DEFAULT_RUNTIME, else cua config set default.runtime).
--imagestringImage for every task (overrides the task's setup_config.image): an OS alias (linux, windows, macos:tahoe), ghcr.io/trycua/<os>, any registry ref or a digest. Env: CUA_BENCH_IMAGE.
--cpuintegervCPUs per sandbox.
--memorystringMemory per sandbox (e.g. 4096, 4G).
--output-dirstringOutput directory for session results.
--attemptsinteger1Run every variant this many times (fresh sandbox each); summary.json reports pass@k (default: 1).
--retriesinteger0Retry a variant that failed before evaluation (sandbox start, connection) up to this many times, with backoff (default: 0).
--keep-runsintegerAfter the run, delete all but the N newest runs in the runs directory (default: keep everything; env CUA_BENCH_KEEP_RUNS).
--max-agenumberAfter the run, delete runs older than DAYS (env CUA_BENCH_MAX_AGE_DAYS).
--max-results-sizestringAfter the run, delete the oldest runs until the runs directory fits in SIZE, e.g. 20G (env CUA_BENCH_MAX_RESULTS_SIZE).
--oraclebooleanfalseRun the oracle solution (the default when no agent is given).
--noopbooleanfalseSet up and evaluate with no actions (the null baseline for eval parity).
--detach-dbooleanfalseRun in the background and return (follow with cb run watch <id>).
--dry-runbooleanfalsePrint each variant's image, kind, variant and backend, then exit without starting a sandbox.

Examples

# Run variant 0 with the oracle solution
cb run task ./tasks/hello_file_env
 
# Run variant 2 with an agent on this machine
cb run task ./tasks/hello_file_env --variant-id 2 --agent cua-agent --model anthropic/claude-sonnet-4-20250514 --on local
 
# Show what would run without starting a sandbox
cb run ./tasks/hello_file_env --dry-run

cb run dataset#

Run every task and variant of a dataset in parallel.

cb run dataset [OPTIONS] <dataset_path>
ArgumentTypeDefaultDescription
<dataset_path>stringrequiredPath to dataset directory, or dataset name from registry.
FlagShortTypeDefaultDescription
--max-parallel-jinteger4Variants running at once (default: 4).
--max-variantsintegerMaximum number of variants to run per task (default: all).
--task-filterstringFilter tasks by name pattern (glob).
--agentstringAgent to use for evaluation (e.g., cua-agent).
--agent-import-pathstringImport path for custom agent (e.g., "path.to.agent:MyCustomAgent").
--modelstringModel to use with the agent (e.g., anthropic/claude-sonnet-4-20250514).
--max-stepsinteger100Maximum number of steps for agent execution (default: 100).
--onlocal | cloud | e2b | daytona | modal | cloudflare | vercel | morph | runloop | fly | blaxel | codesandbox | northflank | aws | gcpWhere sandboxes run: local (this machine), your own cloud account (aws, gcp, modal; connect it first with cua cloud connect <provider>), or a contrib provider (e2b, daytona; needs a cua SDK built with contrib and the provider's key). Default: CUA_DEFAULT_ON, else cua config set default.on, else local.
--kindauto | container | vmWhat kind of sandbox: container or vm. Default auto: Linux as a container unless the task or image says VM; Windows, macOS and Android are VM-only. Default: CUA_DEFAULT_KIND, else cua config set default.kind.
--runtimeauto | gvisor | runc | qemu | lumeWhich engine: gvisor or runc (containers), qemu or lume (VMs). Implies its kind. Default auto: the SDK picks (CUA_DEFAULT_RUNTIME, else cua config set default.runtime).
--imagestringImage for every task (overrides the task's setup_config.image): an OS alias (linux, windows, macos:tahoe), ghcr.io/trycua/<os>, any registry ref or a digest. Env: CUA_BENCH_IMAGE.
--cpuintegervCPUs per sandbox.
--memorystringMemory per sandbox (e.g. 4096, 4G).
--output-dirstringOutput directory for session results.
--attemptsinteger1Run every variant this many times (fresh sandbox each); summary.json reports pass@k (default: 1).
--retriesinteger0Retry a variant that failed before evaluation (sandbox start, connection) up to this many times, with backoff (default: 0).
--keep-runsintegerAfter the run, delete all but the N newest runs in the runs directory (default: keep everything; env CUA_BENCH_KEEP_RUNS).
--max-agenumberAfter the run, delete runs older than DAYS (env CUA_BENCH_MAX_AGE_DAYS).
--max-results-sizestringAfter the run, delete the oldest runs until the runs directory fits in SIZE, e.g. 20G (env CUA_BENCH_MAX_RESULTS_SIZE).
--oraclebooleanfalseRun the oracle solution (the default when no agent is given).
--noopbooleanfalseSet up and evaluate with no actions (the null baseline for eval parity).
--detach-dbooleanfalseRun in the background and return (follow with cb run watch <id>).
--dry-runbooleanfalsePrint each variant's image, kind, variant and backend, then exit without starting a sandbox.

Examples

# Run a registry dataset with its oracle solutions
cb run dataset cua-bench-basic
 
# Run one variant of each matching task, eight at once, in the background
cb run ./tasks --task-filter 'click*' --max-variants 1 -j 8 --detach
 
# Run every variant three times and report pass@k
cb run dataset cua-bench-basic --agent cua-agent --attempts 3

cb run list#

List all runs with status.

cb run list [OPTIONS]
FlagShortTypeDefaultDescription
--verbose-vbooleanfalseShow verbose debugging information.

Examples

# List runs
cb run list
 
# Include debugging details
cb run list --verbose

cb run info#

Show detailed info about a run.

cb run info <run_id>
ArgumentTypeDefaultDescription
<run_id>stringrequiredRun ID to show info for.

Examples

# Show one run
cb run info 30c12572

cb run watch#

Watch a run in real-time with live updates.

cb run watch <run_id>
ArgumentTypeDefaultDescription
<run_id>stringrequiredRun ID to watch.

Examples

# Follow a detached run
cb run watch 30c12572

cb run stop#

Stop a run and release its sandboxes.

cb run stop <run_id>
ArgumentTypeDefaultDescription
<run_id>stringrequiredRun ID to stop.

Examples

# Stop a run
cb run stop 30c12572

cb run logs#

View combined logs from a run or session.

cb run logs [OPTIONS] <identifier>
ArgumentTypeDefaultDescription
<identifier>stringrequiredRun ID or Session ID to view logs for.
FlagTypeDefaultDescription
--tailintegerShow only the last N lines.

Examples

# Print a run's logs
cb run logs 30c12572
 
# Print the last 50 lines
cb run logs 30c12572 --tail 50

cb interact#

Run a task's setup with its desktop visible, then evaluate.

cb interact [OPTIONS] <env_path>
ArgumentTypeDefaultDescription
<env_path>stringrequiredPath to the task directory, or a task name with --dataset or --dataset-path.
FlagTypeDefaultDescription
--variant-idinteger0Task variant index (default: 0).
--datasetstringRegistry dataset to resolve the task from (CUA_REGISTRY_HOME).
--dataset-pathstringPath to dataset directory containing multiple tasks.
--max-stepsintegerMaximum number of env.step() calls before stopping.
--screenshotstringSave a screenshot to this file.
--trace-outstringRecord a trace and save it as a dataset to this path on exit.
--onlocal | cloud | e2b | daytona | modal | cloudflare | vercel | morph | runloop | fly | blaxel | codesandbox | northflank | aws | gcpWhere sandboxes run: local (this machine), your own cloud account (aws, gcp, modal; connect it first with cua cloud connect <provider>), or a contrib provider (e2b, daytona; needs a cua SDK built with contrib and the provider's key). Default: CUA_DEFAULT_ON, else cua config set default.on, else local.
--kindauto | container | vmWhat kind of sandbox: container or vm. Default auto: Linux as a container unless the task or image says VM; Windows, macOS and Android are VM-only. Default: CUA_DEFAULT_KIND, else cua config set default.kind.
--runtimeauto | gvisor | runc | qemu | lumeWhich engine: gvisor or runc (containers), qemu or lume (VMs). Implies its kind. Default auto: the SDK picks (CUA_DEFAULT_RUNTIME, else cua config set default.runtime).
--imagestringImage for every task (overrides the task's setup_config.image): an OS alias (linux, windows, macos:tahoe), ghcr.io/trycua/<os>, any registry ref or a digest. Env: CUA_BENCH_IMAGE.
--cpuintegervCPUs per sandbox.
--memorystringMemory per sandbox (e.g. 4096, 4G).
--oraclebooleanfalseRun the solution after setup.
--viewbooleanfalseOpen the trace viewer when done.
--no-waitbooleanfalseSkip the interactive prompt (useful for SSH/CI testing).
--no-browserbooleanfalsePrint the Display: URL without opening it (also CUA_BENCH_NO_BROWSER=1).

Examples

# Open a task and its display, wait for Enter, then evaluate
cb interact ./tasks/hello_file_env
 
# Run variant 1 with its oracle and save a screenshot
cb interact ./tasks/hello_file_env --variant-id 1 --oracle --screenshot shot.png
 
# Resolve a task from a registry dataset
cb interact click-button --dataset cua-bench-basic

cb task#

Inspect and manage task environments.

cb task [COMMAND]

Examples

# List the tasks in a directory
cb task list ./tasks
 
# Inspect one task
cb task info ./tasks/hello_file_env

cb task info#

Show detailed information about a task.

cb task info <path>
ArgumentTypeDefaultDescription
<path>stringrequiredPath to task directory (containing main.py).

Examples

# Inspect a task's variants and lifecycle functions
cb task info ./tasks/hello_file_env

cb task list#

List tasks in a directory.

cb task list [<path>]
ArgumentTypeDefaultDescription
<path>string.Path to directory containing tasks (default: current directory).

Examples

# List tasks here
cb task list
 
# List tasks in a directory
cb task list ./tasks

cb task create#

Scaffold a new task environment.

cb task create [<path>]
ArgumentTypeDefaultDescription
<path>string.Target directory to create the task in (default: current directory).

Examples

# Scaffold a task interactively under ./tasks
cb task create ./tasks

cb task generate#

Generate a task from a prompt using Claude.

cb task generate [OPTIONS] <prompt> [<output>]
ArgumentTypeDefaultDescription
<prompt>stringrequiredNatural language description of the task to generate (e.g., "2048 game").
<output>stringoptionalOutput directory (default: named after the prompt).
FlagTypeDefaultDescription
--no-interactionbooleanfalseSkip interactive prompts and run Claude non-interactively.

Examples

# Generate a task
cb task generate '2048 game'
 
# Generate into a directory without prompts
cb task generate 'Sort a spreadsheet column' ./tasks/sort_column --no-interaction

cb dataset#

Manage datasets and build from outputs.

cb dataset [COMMAND]

Examples

# List registry datasets
cb dataset list
 
# Build a training dataset from run outputs
cb dataset build ./outputs

cb dataset list#

List available datasets from registry.

cb dataset list

Examples

# List registry datasets
cb dataset list

cb dataset build#

Build a dataset from batch outputs.

cb dataset build [OPTIONS] <outputs_path> [<max_samples>]
ArgumentTypeDefaultDescription
<outputs_path>stringrequiredPath to outputs folder from batch dump (e.g., ./outputs or /tmp/td_output).
<max_samples>integeroptionalLimit number of samples processed (useful for testing).
FlagTypeDefaultDescription
--modestringaguvis-stage-1Processing mode: aguvis-stage-1 (action augmentation) or gui-r1 (low-level click).
--dataset-namestringDataset name when saving to disk (default: td_<mode>_dataset).
--save-dirstringDirectory to save a JSONL dataset (default: <outputs>/processed if not pushing).
--repo-idstringHF Hub repository ID (e.g., username/repo). Required with --push-to-hub.
--push-to-hubbooleanfalsePush the dataset to the Hugging Face Hub.
--privatebooleanfalseWhen pushing to hub, create/update the repo as private.

Examples

# Build an aguvis-stage-1 dataset from run outputs
cb dataset build ./outputs
 
# Build 100 gui-r1 samples
cb dataset build ./outputs 100 --mode gui-r1
 
# Push a private dataset to the Hugging Face Hub
cb dataset build ./outputs --push-to-hub --repo-id me/cua-clicks --private

cb trace#

View and analyze trace datasets.

cb trace [COMMAND]

Examples

# Open a run's trace in the browser
cb trace view 30c12572
 
# Open a run's agent trajectories
cb trace traj 30c12572

cb trace view#

View a single trace in browser.

cb trace view <identifier>
ArgumentTypeDefaultDescription
<identifier>stringrequiredRun ID or session ID (e.g., "30c12572" or "task-30c12572-click-button-v0").

Examples

# Open a run's trace
cb trace view 30c12572
 
# Open one session's trace
cb trace view task-30c12572-click-button-v0

cb trace traj#

View agent trajectories from a run in cua.ai/trajectory-viewer.

cb trace traj [OPTIONS] <identifier>
ArgumentTypeDefaultDescription
<identifier>stringrequiredRun ID (e.g., "30c12572").
FlagShortTypeDefaultDescription
--port-pinteger8090Local file server port (default: 8090).

Examples

# Open a run's trajectories
cb trace traj 30c12572
 
# Serve the files on another port
cb trace traj 30c12572 --port 9000

Sandboxes and account#

Local sandboxes, status and cleanup.

cb env#

List local sandboxes.

cb env [COMMAND]

Examples

# List local sandboxes
cb env ls

cb env ls#

List local sandboxes.

cb env ls [OPTIONS]

Alias: cb env list.

FlagTypeDefaultDescription
--localbooleanfalseList local sandboxes (the default; kept as a no-op).

Examples

# List local sandboxes
cb env ls

cb status#

Show system status dashboard.

cb status

Examples

# Show runs, sandboxes and credentials at a glance
cb status

cb prune#

Clean up cua-bench run data (and 0.2.x image leftovers).

cb prune [OPTIONS]
FlagShortTypeDefaultDescription
--keepintegerWith --runs: keep the N newest runs and remove only older ones.
--older-thannumberWith --runs: remove only runs older than DAYS.
--all-abooleanfalseRemove runs and 0.2.x image leftovers.
--imagesbooleanfalseRemove 0.2.x stored images (deprecated store).
--overlaysbooleanfalseRemove 0.2.x task overlays (deprecated store).
--runsbooleanfalseRemove run logs and registry.
--dockerbooleanfalseDeprecated: cua-bench no longer creates Docker resources.
--dry-runbooleanfalseShow what would be deleted without deleting.
--force-fbooleanfalseSkip confirmation prompts.

Examples

# Preview what would be removed
cb prune --all --dry-run
 
# Remove run logs without asking
cb prune --runs --force
 
# Keep the 20 newest runs, delete the rest
cb prune --runs --keep 20
 
# Delete runs older than 30 days
cb prune --runs --older-than 30

cb agent#

Scaffold custom agents.

Create and manage custom agents for cua-bench.

cb agent [COMMAND]

Examples

# Scaffold an agent in ./my-agent
cb agent init my-agent

cb agent init#

Create a new agent from template.

Scaffold a new custom agent with working sample code.

cb agent init [OPTIONS] <name>
ArgumentTypeDefaultDescription
<name>stringrequiredAgent name (e.g., 'my-agent').
FlagShortTypeDefaultDescription
--output-dir-ostringOutput directory (default: ./<name>).

Examples

# Scaffold an agent in ./my-agent
cb agent init my-agent
 
# Scaffold it into another directory
cb agent init my-agent --output-dir ./agents/my-agent

cb agent build#

Deprecated: cb run executes agents in-process.

Build a Docker image for an agent directory containing a Dockerfile.

cb agent build [OPTIONS] <path>

Examples

# Build an agent image (deprecated)
cb agent build ./my-agent --tag my-agent:dev

Images and platforms#

The canonical images and platform configurations.

cb image#

Show the canonical images (the local image store is deprecated).

cb image [COMMAND]

Examples

# List the canonical images
cb image
 
# Show one image
cb image info linux

cb image list#

List all images.

cb image list [OPTIONS]
FlagTypeDefaultDescription
--platformstringFilter by platform.
--formattable | jsontableOutput format.

Examples

# List images
cb image list
 
# List one platform's images as JSON
cb image list --platform linux-docker --format json

cb image info#

Show image details.

cb image info <name>
ArgumentTypeDefaultDescription
<name>stringrequiredImage name.

Examples

# Show one image
cb image info linux

cb image create#

Deprecated: create an image from a platform.

cb image create [OPTIONS] <platform>

Examples

# Create a Linux image (deprecated)
cb image create linux-docker --name my-linux

cb image delete#

Deprecated: delete a stored image.

cb image delete [OPTIONS] <name>

Examples

# Delete a stored image (deprecated)
cb image delete my-linux --force

cb image clone#

Deprecated: clone a stored image.

cb image clone [OPTIONS] <source> <target>

Examples

# Clone a stored image (deprecated)
cb image clone my-linux my-linux-2

cb image shell#

Deprecated: interactive shell into a stored image.

cb image shell [OPTIONS] <name>

Examples

# Open a shell in a stored image (deprecated)
cb image shell my-linux

cb platform#

Show available platform configurations.

cb platform [COMMAND]

Examples

# List platforms
cb platform
 
# Show one platform
cb platform info linux-docker

cb platform list#

List all available platforms.

cb platform list [OPTIONS]
FlagTypeDefaultDescription
--formattable | jsontableOutput format.

Examples

# List platforms
cb platform list
 
# As JSON
cb platform list --format json

cb platform info#

Show platform details.

cb platform info <platform>
ArgumentTypeDefaultDescription
<platform>stringrequiredPlatform name (e.g., linux-docker, windows-qemu).

Examples

# Show one platform
cb platform info windows-qemu

Global options#

--help prints help for any command; --version prints the version.

Exit codes#

CodeMeaning
0Success: every variant completed.
1Failure: a variant failed or was cancelled, or the command could not run.
2Usage error: an unknown command, flag or value.
130Interrupted (Ctrl-C); running variants are cancelled.