Benchmarks

Spaces performance benchmarks: list them, run one in Spaces of its own (result.json and samples.csv in the bench-result v1 format), and check a run against a committed baseline.

Command	Description
`cua bench list`	Every benchmark: what it measures, the host it needs, what `--n` sets and whether it has a perf guard.
`cua bench run`	Run one benchmark in Spaces of its own and write the run as bench-result v1: result.json, samples.csv and the bench's own output in raw/.
`cua bench check`	Apply a benchmark's perf guard to a run against a committed baseline; exits 1 on a regression.

Every command also accepts the global options.

Spaces performance benchmarks: list them, run one in Spaces of its own (writes result.json and samples.csv in the bench-result v1 format), and check a run against a committed baseline.

cua bench [OPTIONS] <COMMAND>

Examples

cua bench list
cua bench run microvm-fork --n 5 --out ./mf
cua bench check ./mf/result.json --baseline microvm-fork/baseline.json

`cua bench list`#

Every benchmark: what it measures, the host it needs, what --n sets and whether it has a perf guard.

cua bench list [OPTIONS]

Examples

cua bench list
cua bench list --json

`cua bench run`#

Run one benchmark in Spaces of its own and write the run as bench-result v1: result.json, samples.csv and the bench's own output in raw/.

Every run uses a Spaces home of its own (<out>/cua-home, or CUA_HOME when set) and a daemon of this binary, and deletes the Spaces it creates. A bench that needs a program a release does not ship (CHANNEL_BENCH, ROLLOUT_BENCH, DISK, a source checkout) stops and says how to get it (cua bench list).

cua bench run [OPTIONS] <NAME>

Argument	Type	Default	Description
`<NAME>`	string	required	The benchmark (`cua bench list`).

Flag	Type	Description
`--out`	path	Directory for result.json, samples.csv and raw/ (default ./bench-results/<name>/<UTC time>).
`--n`	integer	Samples (or the size the bench scales by; `cua bench list` says which). Default: the bench's own.
`--src`	path	Source checkout (libs/cua) for benches that build with cargo (default: CUA_SRC, or the checkout the current directory is in).

Examples

# 5 samples per measurement, results in ./mf
cua bench run microvm-fork --n 5 --out ./mf
 
# With CHANNEL_BENCH set to the SDK's channel_bench example
cua bench run channel
 
# The run as JSON on stdout (progress stays on stderr)
cua bench run microvm-fork --json

`cua bench check`#

Apply a benchmark's perf guard to a run against a committed baseline; exits 1 on a regression.

A baseline path that does not exist as given is looked up among the baselines this binary ships (relative to the bench directory).

cua bench check [OPTIONS] --baseline <BASELINE> <RESULT>

Argument	Type	Default	Description
`<RESULT>`	path	required	A result.json `cua bench run` wrote (its raw/ directory next to it).

Flag	Type	Default	Description
`--baseline`	path	required	The baseline file.
`--host`	string		Host class or profile in a baseline that holds several (fork-fanout, fork-batch).

Examples

cua bench check ./mf/result.json --baseline microvm-fork/baseline.json
cua bench check ./ff/result.json --baseline fork-fanout/baseline.json --host macos-hvf

Exit codes#

Code	Meaning
`0`	Success.
`1`	Failure, or `cua do` reported an error.
`2`	Invalid argument, or an ambiguous sandbox name (qualify it: `local:NAME`).
`3`	Not found: sandbox, window, skill or image (or an image not published yet).
`4`	Not supported, or not configured (for example no provider credentials).
`5`	No cua-spacesd answered, or a transport failure.
`6`	Unauthenticated or permission denied (by Cua or your cloud account).
`7`	Not enough free disk space (see `cua cache`).
`8`	No room on the machine: its VMs are at their memory or CPU limit, or it is low on memory.
`130`	Cancelled (Ctrl-C during a create): what it made was removed.

Benchmarks

cua bench list#

cua bench run#

cua bench check#

Exit codes#

`cua bench list`#

`cua bench run`#

`cua bench check`#