Dawn's Continuous Testing Infrastructure

Dawn uses Chromium‘s continuous integration (CI) infrastructure to continually run tests on changes to Dawn and provide a way for developers to run tests against their changes before submitting. CI bots continually build and run tests for every new change, and Try bots build and run developers’ pending changes before submission. Dawn uses two different build recipes. There is a Dawn build recipe which checks out Dawn standalone, compiles, and runs the dawn_unittests. And, there is the Chromium build recipe which checks out Dawn inside a Chromium checkout. Inside a Chromium checkout, there is more infrastructure available for triggering dawn_end2end_tests that run on real GPU hardware, and we are able to run Chromium integration tests as well as tests for WebGPU.

For additional information on GPU testing in Chromium, please see [chromium/src]//docs/gpu/gpu_testing_bot_details.md.

Dawn CI/Try Builders

Dawn builders are specified in [dawn]//infra/config/global/cr-buildbucket.cfg. This file contains a few mixins such as clang, no_clang, x64, x86, debug, release which are used to specify the bot dimensions and build properties (builder_mixins.recipe.properties). At the time of writing, we have the following builders:

There are additional chromium/try builders, but those are described later in this document.

These bots are defined in both buckets luci.dawn.ci and luci.dawn.try, though their ACL permissions differ. luci.dawn.ci bots will be scheduled regularly based on [dawn]//infra/config/global/luci-scheduler.cfg. luci.dawn.try bots will be triggered on the CQ based on [dawn]//infra/config/global/commit-queue.cfg.

One particular note is buckets.swarming.builder_defaults.recipe.name: "dawn" which specifies these use the dawn.py build recipe.

Build status for both CI and Try builders can be seen at this console which is generated from [dawn]//infra/config/global/luci-milo.cfg.

Dawn Build Recipe

The dawn.py build recipe is simple and intended only for testing compilation and unit tests. It does the following:

  1. Checks out Dawn standalone and dependencies
  2. Builds based on the builder_mixins.recipe.properties coming from the builder config in [dawn]//infra/config/global/cr-buildbucket.cfg.
  3. Runs the dawn_unittests on that same bot.

Dawn Chromium-Based CI Waterfall Bots

The chromium.dawn waterfall consists of the bots specified in the chromium.dawn section of [chromium/src]//testing/buildbot/waterfalls.pyl. Bots named “Builder” are responsible for building top-of-tree Dawn, whereas bots named “DEPS Builder” are responsible for building Chromium's DEPS version of Dawn.

The other bots, such as “Dawn Linux x64 DEPS Release (Intel HD 630)” receive the build products from the Builders and are responsible for running tests. The Tester configuration may specify mixins from [chromium/src]//testing/buildbot/mixins.pyl which help specify bot test dimensions like OS version and GPU vendor. The Tester configuration also specifies test_suites from [chromium/src]//testing/buildbot/test_suites.pyl which declare the tests are arguments passed to tests that should be run on the bot.

The Builder and Tester bots are additionally configured at [chromium/tools/build]//scripts/slave/recipe_modules/chromium_tests/chromium_dawn.py which defines the bot specs for the builders and testers. Some things to note:

Finally, builds on these waterfall bots are automatically scheduled based on the configuration in [chromium/src]//infra/config/buckets/ci.star. Note that the Tester bots are triggered_by the Builder bots.

Dawn Chromium-Based Tryjobs

[dawn]//infra/config/global/commit-queue.cfg declares additional tryjob builders which are defined in the Chromium workspace. The reason for this separation is that jobs sent to these bots rely on the Chromium infrastructure for doing builds and triggering jobs on bots with GPU hardware in swarming.

At the time of writing, the bots for Dawn CLs are:

And for Chromium CLs:

The configuration for these bots is generated from [chromium]//infra/config/buckets/try.star which uses the chromium_dawn_builder function which sets the mastername to tryserver.chromium.dawn.

[chromium/tools/build]//scripts/slave/recipe_modules/chromium_tests/trybots.py specifies tryserver.chromium.dawn bots as mirroring bots from the chromium.dawn waterfall. Example:

'dawn-linux-x64-deps-rel': {
    'bot_ids': [
        {
            'mastername': 'chromium.dawn',
            'buildername': 'Dawn Linux x64 DEPS Builder',
            'tester': 'Dawn Linux x64 DEPS Release (Intel HD 630)',
        },
        {
            'mastername': 'chromium.dawn',
            'buildername': 'Dawn Linux x64 DEPS Builder',
            'tester': 'Dawn Linux x64 DEPS Release (NVIDIA)',
        },
    ],
},

Using the [chromium/tools/build]//scripts/slave/recipes/chromium_trybot.py recipe, these trybots will cherry-pick a CL and run the same tests as the CI waterfall bots. The trybots also pick up some build mixins from [chromium]//tools/mb/mb_config.pyl.

Bot Allocation

Bots are physically allocated based on the configuration in [chromium/infradata/config]//configs/chromium-swarm/starlark/bots/dawn.star (Google only).

dawn/try bots are using builderless configurations which means they use builderless GCEs shared with Chromium bots and don't need explicit allocation.

chromium/try bots are still explicitly allocated with a number of GCE instances and lifetime of the build cache. All of the GCE bots should eventually be migrated to builderless (crbug.com/dawn/328). Mac bots such as dawn-mac-x64-deps-rel, mac-dawn-rel, Dawn Mac x64 Builder, and Dawn Mac x64 DEPS Builder point to specific ranges of machines that have been reserved by the infrastructure team.