Dummy Run
Get started

This page is the output of each command’s help, captured from the program. ENGINE.md explains how to use them.

dummyrun

Every command, and the model providers.

Usage:
  dummyrun run --cast DIR --persona ID --scenario ID --target CONFIG --out DIR \
      --browser chrome|firefox --provider PROVIDER --model MODEL [--region REGION] [--endpoint URL] \
      [--api-key-env NAME] [--project ID] [--reasoning-effort LEVEL] [--openai-api chat|responses]
  dummyrun a11y [--sitemap URL] [URL...] [--browser chrome|firefox] [--allow FILE] [--json FILE]
  dummyrun version

Run "dummyrun run -h" or "dummyrun a11y -h" for every flag. "dummyrun audit a11y" is the same as "dummyrun a11y".

Providers:
  azure              Azure OpenAI. --endpoint or DUMMYRUN_AZURE_ENDPOINT; sign in with "az login" or another
                     DefaultAzureCredential source. --region is required.
  openai-compatible  Any OpenAI-style service or local server. --endpoint is the base URL, such as
                     https://api.openai.com/v1 or http://localhost:11434/v1. --api-key-env names the variable
                     that holds the key (leave it out for a local server). --region is required (use "local").
  anthropic          Anthropic Messages API. The key is read from ANTHROPIC_API_KEY or --api-key-env.
  gemini             Gemini Developer API. The key is read from GEMINI_API_KEY or --api-key-env.
  vertex             Gemini on Vertex AI with Application Default Credentials. --project (or
                     GOOGLE_CLOUD_PROJECT) and --region are required.
  bedrock            Amazon Bedrock Converse with the AWS default credential chain. --region defaults to eu-west-2.

An API key is read from the environment when the run starts. It is never printed or written to a run record.

dummyrun run

Play one persona through one scenario and write a run record.

Usage of run:
  -api-key-env string
    	name of the environment variable that holds the API key
  -api-version string
    	Azure OpenAI API version (default 2025-04-01-preview)
  -batch string
    	batch ID grouping runs started together
  -browser string
    	chrome or firefox (default "chrome")
  -cast string
    	cast directory
  -endpoint string
    	base URL of the provider (Azure endpoint, or the OpenAI-compatible base URL)
  -headful
    	show the browser window
  -model string
    	deployment or model name
  -openai-api string
    	OpenAI-style API: chat (default) or responses, for models that need it to call tools
  -out string
    	directory for run records
  -persona string
    	persona ID
  -project string
    	Google Cloud project for the vertex provider
  -provider string
    	model provider: azure, openai-compatible, anthropic, gemini, vertex, bedrock (default "azure")
  -reasoning-effort string
    	reasoning effort for OpenAI-style models: none, minimal, low, medium, high or xhigh
  -region string
    	region the model runs in, such as uksouth or global (optional where the provider has a default)
  -scenario string
    	scenario ID
  -steps int
    	step budget override (up to 40)
  -target string
    	target configuration (JSON)
  -tokens int
    	token budget (input plus output)

dummyrun a11y

Check pages for WCAG 2.2 A and AA failures with no model. dummyrun audit a11y is the same command.

Usage:
  dummyrun a11y [--sitemap URL] [--rebase URL] [URL...] [flags]

Runs the axe-core WCAG 2.2 A and AA rules on each page in a real browser. No model is called.

Flags:
  --sitemap URL     a sitemap (or sitemap index) listing the pages to test
  --rebase URL      move every sitemap URL onto this origin, to test a local copy of a site
  --browser NAME    chrome or firefox (default chrome)
  --viewport NAME   desktop (1280 wide), phone (375 wide) or both (default both)
  --scheme NAME     light, dark or both (default both)
  --allow FILE      accepted violations, one per line: rule | page | selector | reason
  --json FILE       write the full report as JSON; "-" writes it to standard output
  --headful         show the browser window

Exit status: 0 when nothing fails, 1 when a violation is not in the allow-list or a page cannot be
tested, 2 for a usage error.

dummyrun-otel

Send run records to an OpenTelemetry backend and Grafana.

Usage:
  dummyrun-otel export [flags] FILE|DIR...   send run records to an OTLP endpoint
  dummyrun-otel watch  [flags] DIR           send new run records as they appear
  dummyrun-otel version

The OTLP endpoint, headers and TLS settings come from the standard variables:
OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS, OTEL_EXPORTER_OTLP_PROTOCOL,
OTEL_RESOURCE_ATTRIBUTES. Grafana annotations need --grafana-url and GRAFANA_TOKEN.
Run "dummyrun-otel export -h" for the flags.

dummyrun-otel export

Send the records in some files or directories once.

Usage of dummyrun-otel export:
  -dry-run
    	print what would be sent as JSON and send nothing
  -grafana-url string
    	Grafana base URL for annotations; the token comes from GRAFANA_TOKEN
  -logs-max-age duration
    	skip log records older than this, because Loki rejects them by default; 0 sends every record (default 168h0m0s)
  -metrics-temporality string
    	metric temporality: cumulative or delta (default "cumulative")
  -protocol string
    	OTLP transport: http/protobuf (default) or grpc
  -timeout duration
    	how long to wait for the final flush (default 30s)
  -verify-loki string
    	Loki base URL; after the export, count the log lines back per run and fail when any are missing. Basic auth comes from LOKI_BASIC_AUTH as user:password

dummyrun-otel watch

Send each new record in a directory as it appears.

Usage of dummyrun-otel watch:
  -dry-run
    	print what would be sent as JSON and send nothing
  -existing
    	also send the records already in the directory at start
  -grafana-url string
    	Grafana base URL for annotations; the token comes from GRAFANA_TOKEN
  -interval duration
    	how often to look for new files (default 5s)
  -logs-max-age duration
    	skip log records older than this, because Loki rejects them by default; 0 sends every record (default 168h0m0s)
  -metrics-temporality string
    	metric temporality: cumulative or delta (default "cumulative")
  -protocol string
    	OTLP transport: http/protobuf (default) or grpc
  -timeout duration
    	how long to wait for the final flush (default 30s)
  -verify-loki string
    	Loki base URL; after the export, count the log lines back per run and fail when any are missing. Basic auth comes from LOKI_BASIC_AUTH as user:password

This page is rendered from docs/CLI.md in the repository. Edit it there and the site follows.