docs/standalone_cts_runner.md

# Running the WebGPU CTS with the standalone runner

`dawn_standalone_cts_runner` is a single executable that statically links V8 and
Dawn and runs the WebGPU CTS directly, without a Node.js installation.

The usual way to run the CTS is [`dawn.node`](../src/dawn/node/README.md), a
native addon loaded by Node.js. The standalone runner instead embeds its own V8
and supplies the subset of the Node API the CTS uses. That removes the external
Node.js dependency. This is important for platforms like Android where Node.js
is not available.

Both runners execute the same CTS JavaScript and the same Dawn bindings.

## Fetching dependencies

Follow [`docs/building.md`](building.md), using
`scripts/standalone-with-v8.gclient` in the `Get the code` step so `gclient`
fetches both the Node bindings (`dawn_node`) and V8 (`checkout_v8`):

```sh
# Clone the repo as "dawn"
git clone https://dawn.googlesource.com/dawn dawn && cd dawn

# Bootstrap the gclient configuration for the standalone CTS runner
cp scripts/standalone-with-v8.gclient .gclient

# Fetch external dependencies and toolchains with gclient
gclient sync
```

If you are configuring an existing checkout, ensure `.gclient` sets both
`"checkout_v8": True` and `"dawn_node": True` in `custom_vars` before running
`gclient sync`. Without `checkout_v8`, the `dawn_standalone_cts_runner` target
is not declared.

## Building the runner

Generate a release build directory and compile `dawn_standalone_cts_runner`
(building this target compiles only the standalone runner executable and its
dependencies, not the `dawn.node` shared library):

```sh
gn gen out/Release --args="is_debug=false dawn_use_swiftshader=false"
autoninja -C out/Release dawn_standalone_cts_runner
```

- `is_debug=false` is strongly recommended: the default debug build compiles a
  debug V8, which runs the CTS roughly 17x slower than a release build.
- `dawn_use_swiftshader=false` skips building Dawn's bundled SwiftShader
  renderer and ensures the runner uses the system's drivers. Remove this option
  if you want to test against SwiftShader.

## Building the CTS JavaScript

The runner executes the CTS's compiled `out-node/` tree, which is not checked
in. Any Node.js installation (`>= 16.0.0`) with `npm` and `npx` can build it. If
you do not have `node`, `npm`, and `npx` installed on your system, you can add
the prebuilt Node.js fetched by `gclient sync` under `third_party/node/` to your
`PATH` first:

```sh
# Optional: use the Node.js fetched by DEPS if npm/npx are not already installed.
# Replace linux-amd64 with linux-arm64, mac-amd64, or mac-arm64 as needed
# (or $PWD/third_party/node/windows-amd64 on Windows).
export PATH="$PWD/third_party/node/linux-amd64/bin:$PATH"
```

Install the CTS dependencies and build `out-node/`:

```sh
cd third_party/webgpu-cts
npm ci
npx grunt node
cd ../..
```

`npm ci` alone is not sufficient: `npx grunt node` runs the Grunt tasks that
generate `out-node/`.

## Running the CTS

Run from the Dawn root (or pass `--cts-dir <dir>` if the CTS lives outside
`third_party/webgpu-cts`):

```sh
out/Release/dawn_standalone_cts_runner -q 'webgpu:api,operation,queue,*'
```

Quote the query: CTS queries contain `;` and `"`, which most shells will
otherwise interpret. Pass `--help` to list all supported flags.

### Selecting a GPU, and setting Dawn toggles

`--dawn-flag` is passed through to the Dawn provider. It takes a single
`key=value` and may be repeated:

```sh
out/Release/dawn_standalone_cts_runner \
    --dawn-flag backend=vulkan \
    --dawn-flag enable-dawn-features=dump_shaders \
    -q 'webgpu:api,operation,queue,*'
```

The recognised keys are:

- `backend=<null|webgpu|d3d11|d3d12|d3d|metal|vulkan|vk|opengl|gl|opengles|gles>`
  (defaults to `d3d12` on Windows, `vulkan` on Linux/Android, and `metal` on
  macOS; can also be set via the `DAWNNODE_BACKEND` environment variable)
- `adapter=<name>` — substring match against the adapter device name. Pass an
  invalid name (for example `adapter=help`) to list all available adapters in
  the error output
- `validate=<1|true>` — enable full backend validation layers
- `verbose=1` — print the selected GPU adapter name
- `enable-dawn-features=<toggles>` — comma-separated list of
  [Dawn toggles](https://dawn.googlesource.com/dawn/+/refs/heads/main/src/dawn/native/Toggles.cpp)
- `disable-dawn-features=<toggles>` — comma-separated list of Dawn toggles

## Running on Android

There is no Node.js build for Android in the checkout, so the standalone runner
is the only way to run the command-line CTS directly on an Android device.
Cross-compiling for Android currently requires a Linux host (the GN build and
Android NDK toolchain in Dawn's checkout do not support building for Android
from Windows or macOS).

### Checkout

On a Linux host, add `target_os = ["android"]` to `.gclient` and sync the
Android NDK and toolchain dependencies:

```sh
printf '\ntarget_os = ["android"]\n' >> .gclient
gclient sync
```

### Build

Generate an `arm64` Android release build on Linux and compile the runner:

```sh
gn gen out/Android --args='target_os="android" target_cpu="arm64" is_debug=false'
autoninja -C out/Android dawn_standalone_cts_runner
```

### Staging onto the device

Only the executable and the compiled CTS JavaScript are needed on the device.

> [!NOTE]
> `dawn.node` is deliberately **not** pushed. The runner's module loader returns
> its statically-linked binding for `require('dawn.node')`, so the shared
> library is never loaded.

```sh
DEV=/data/local/tmp/cts
adb shell "mkdir -p $DEV/bin $DEV/webgpu-cts"

adb push out/Android/dawn_standalone_cts_runner $DEV/bin/
adb shell "chmod 755 $DEV/bin/dawn_standalone_cts_runner"

# out-node is ~2200 files; a single archive transfers far faster than letting
# adb sync each file individually. Include cmdline.ts as well, which both the
# runner and cmdline.js check as a repository-root marker.
tar -C third_party/webgpu-cts -czf /tmp/cts.tgz \
    out-node src/common/runtime/cmdline.ts
adb push /tmp/cts.tgz $DEV/webgpu-cts/
adb shell "cd $DEV/webgpu-cts && tar xzf cts.tgz && rm cts.tgz"
rm /tmp/cts.tgz
```

### Running

```sh
adb shell '/data/local/tmp/cts/bin/dawn_standalone_cts_runner \
    --cts-dir /data/local/tmp/cts/webgpu-cts \
    -q "webgpu:api,operation,queue,*"'
```

## Future plans

The setup above is more manual than it should be. The intent is to teach
[`run-cts`](../tools/src/cmd/run-cts) about the standalone runner, so that
selecting it becomes a flag on the tool already used to run the CTS, with the
CTS build, query sharding, expectations handling and result reporting all
handled as they are today for `dawn.node`.

Once that lands, most of this document reduces to a single `./tools/run run-cts`
invocation, and the manual staging steps for Android are expected to be driven
by the tool as well.