[tools][cts] Add test coverage loss command

Adds the expectation-coverage command to "tools/run cts". This grabs
recent unique config/test name combinations from CI builders that are
affected by expectations in some way and figures out which expectations
are suppressing failures in tests. This information is then surfaced to
the user ordered by which chunks of expectations are affecting the most
tests, which can be used to prioritize fixes in order to get test
coverage back.

By default, the command will run:
  - Against the core expectations file
  - Include temporary Skip expectations
  - Group expectations by the whitespace-delineated chunks as they
    appear in the file
  - Only surface the top 25 chunks contributing to test coverage loss

However, command line flags exist to change behavior to any of the
following:
  - Run against the compat expectations file
  - Exclude temporary Skip expectations
  - Check for coverage loss on a per-expectation basis
  - Surface the top X chunks/individual expectations contributing to
    test coverage loss (or surface all results)

Bug: 390105593
Change-Id: Ifabfedc3baa7a8950ddf4f05a4d03486ed27e30b
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/223335
Auto-Submit: Brian Sheedy <bsheedy@google.com>
Reviewed-by: dan sinclair <dsinclair@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@google.com>
14 files changed
tree: 6079314eb5460baadbe86a5ccbc6ab573f001578
  1. .github/
  2. .vscode/
  3. build_overrides/
  4. docs/
  5. generator/
  6. include/
  7. infra/
  8. scripts/
  9. src/
  10. test/
  11. third_party/
  12. tools/
  13. webgpu-cts/
  14. .bazelrc
  15. .clang-format
  16. .clang-tidy
  17. .git-blame-ignore-revs
  18. .gitattributes
  19. .gitignore
  20. .gitmodules
  21. .gn
  22. AUTHORS
  23. BUILD.bazel
  24. BUILD.gn
  25. CMakeLists.txt
  26. CMakeSettings.json
  27. CODE_OF_CONDUCT.md
  28. codereview.settings
  29. CONTRIBUTING.md
  30. CPPLINT.cfg
  31. DEPS
  32. DIR_METADATA
  33. go.mod
  34. go.sum
  35. go_presubmit_support.py
  36. LICENSE
  37. OWNERS
  38. PRESUBMIT.py
  39. README.chromium
  40. README.md
  41. WATCHLISTS
  42. WORKSPACE.bazel
README.md

Build Status Matrix Space

Dawn, a WebGPU implementation

Dawn is an open-source and cross-platform implementation of the WebGPU standard. More precisely it implements webgpu.h that is a one-to-one mapping with the WebGPU IDL. Dawn is meant to be integrated as part of a larger system and is the underlying implementation of WebGPU in Chromium.

Dawn provides several WebGPU building blocks:

  • WebGPU C/C++ headers that applications and other building blocks use.
    • The webgpu.h version that Dawn implements.
    • A C++ wrapper for the webgpu.h.
  • A “native” implementation of WebGPU using platforms' GPU APIs: D3D12, Metal, Vulkan and OpenGL. See per API support for more details.
  • A client-server implementation of WebGPU for applications that are in a sandbox without access to native drivers
  • Tint is a compiler for the WebGPU Shader Language (WGSL) that can be used in standalone to convert shaders from and to WGSL.

Helpful links:

Documentation table of content

Developer documentation:

User documentation: (TODO, figure out what overlaps with the webgpu.h docs)

License

BSD 3-Clause License, please see LICENSE.

Disclaimer

This is not an officially supported Google product.