docs: write an overview of Dawn and split README.md
BUG=dawn:23
Change-Id: Ib071a4bcccee267ea0ff8af3ddb858e6c54a7eb2
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/14160
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Reviewed-by: Austin Eng <enga@chromium.org>
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index c500c15..4917a85 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -19,3 +19,6 @@
All submissions, including submissions by project members, require review. We
use [Dawn's Gerrit](https://dawn-review.googlesource.com) for this purpose.
+
+Dawn doesn't have a formal coding style yet, except what's defined by our clang format style.
+Overall try to use the same style and convention as code around your change.
diff --git a/README.md b/README.md
index 4543bae..92e7728 100644
--- a/README.md
+++ b/README.md
@@ -1,82 +1,46 @@
# Dawn, a WebGPU implementation
-Dawn (formerly NXT) is an open-source and cross-platform implementation of the work-in-progress WebGPU standard.
-It exposes a C/C++ API that maps almost one-to-one to the WebGPU IDL and can be managed as part of a larger system such as a Web browser.
+Dawn is an open-source and cross-platform implementation of the work-in-progress [WebGPU](https://webgpu.dev) standard.
+More precisely it implements [`webgpu.h`](https://github.com/webgpu-native/webgpu-headers/blob/master/webgpu.h) that is a one-to-one mapping with the WebGPU IDL.
+Dawn is meant to be integrated as part of a larger system and is the underlying implementation of WebGPU in Chromium.
Dawn provides several WebGPU building blocks:
- **WebGPU C/C++ headers** that applications and other building blocks use.
- - The main C header for the WebGPU API.
- - Definition of a structure of all function pointers for this specific Dawn version (called "proctable").
- - A C++ wrapper for the C header.
+ - The `webgpu.h` version that Dawn implements.
+ - A C++ wrapper for the `webgpu.h`.
- **A "native" implementation of WebGPU** using platforms' GPU APIs:
- **D3D12** on Windows 10
- - **Metal** on OSX (and eventually iOS)
- - **Vulkan** on Windows, Linux (eventually ChromeOS, Android and Fuchsia too)
+ - **Metal** on macOS and iOS
+ - **Vulkan** on Windows, Linux, ChromeOS, Android and Fuchsia
- OpenGL as best effort where available
- **A client-server implementation of WebGPU** for applications that are in a sandbox without access to native drivers
- - **A Dawn proc-table backend implementation of WebGPU** for applications what want to be able to switch at runtime between native or client-server mode.
-## Directory structure
+Helpful links:
-- `dawn.json`: description of the API used to drive code generators.
-- `examples`: examples showing how Dawn is used.
-- `generator`: code generator for files produces from `dawn.json`
- - `templates`: Jinja2 templates for the generator
-- `scripts`: scripts to support things like continuous testing, build files, etc.
-- `src`:
- - `common`: helper code shared between core Dawn libraries and tests/samples
- - `dawn_native`: native implementation of WebGPU, one subfolder per backend
- - `dawn_wire`: client-server implementation of WebGPU
- - `include`: public headers for Dawn
- - `tests`: internal Dawn tests
- - `end2end`: WebGPU tests performing GPU operations
- - `unittests`: unittests and by extension tests not using the GPU
- - `validation`: WebGPU validation tests not using the GPU (frontend tests)
- - `utils`: helper code to use Dawn used by tests and samples
-- `third_party`: directory where dependencies live as well as their buildfiles.
+ - [Dawn's bug tracker](https://bugs.chromium.org/p/dawn/issues/entry) if you find issues with Dawn.
+ - [Dawn's mailing list](https://groups.google.com/forum/#!members/dawn-graphics) for other discussions related to Dawn.
+ - [Dawn's source code](https://dawn.googlesource.com/dawn)
-## Building Dawn
+## Documentation table of content
-Dawn uses the Chromium build system and dependency management so you need to [install depot_tools] and add it to the PATH.
+Developer documentation:
-[install depot_tools]: http://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/depot_tools_tutorial.html#_setting_up
+ - [Dawn overview](docs/overview.md)
+ - [Building Dawn](docs/buiding.md)
+ - [Contributing to Dawn](CONTRIBUTING.md)
+ - [Testing Dawn](docs/testing.md)
+ - [Debugging Dawn](docs/debugging.md)
+ - [Dawn's infrastructure](docs/infra.md)
-On Linux you need to have the `pkg-config` command:
-```sh
-# Install pkg-config on Ubuntu
-sudo apt-get install pkg-config
-```
+User documentation: (TODO, figure out what overlaps with webgpu.h docs)
-Then get the source as follows:
+## Status
-```sh
-# Clone the repo as "dawn"
-git clone https://dawn.googlesource.com/dawn dawn && cd dawn
-
-# Bootstrap the gclient configuration
-cp scripts/standalone.gclient .gclient
-
-# Fetch external dependencies and toolchains with gclient
-gclient sync
-```
-
-Then generate build files using `gn args out/Debug` or `gn args out/Release`.
-A text editor will appear asking build options, the most common option is `is_debug=true/false`; otherwise `gn args out/Release --list` shows all the possible options.
-
-Then use `ninja -C out/Release` to build dawn and for example `./out/Release/dawn_end2end_tests` to run the tests.
-
-## Contributing
-
-Please read and follow [CONTRIBUTING.md](/CONTRIBUTING.md).
-Dawn doesn't have a formal coding style yet, except what's defined by our clang format style.
-Overall try to use the same style and convention as code around your change.
-
-If you find issues with Dawn, please feel free to report them on the [bug tracker](https://bugs.chromium.org/p/dawn/issues/entry).
-For other discussions, please post to [Dawn's mailing list](https://groups.google.com/forum/#!members/dawn-graphics).
+(TODO)
## License
-Please see [LICENSE](/LICENSE).
+Apache 2.0 Public License, please see [LICENSE](/LICENSE).
## Disclaimer
diff --git a/docs/buiding.md b/docs/buiding.md
new file mode 100644
index 0000000..f1ab8a2
--- /dev/null
+++ b/docs/buiding.md
@@ -0,0 +1,30 @@
+# Building Dawn
+
+Dawn uses the Chromium build system and dependency management so you need to [install depot_tools] and add it to the PATH.
+
+[install depot_tools]: http://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/depot_tools_tutorial.html#_setting_up
+
+On Linux you need to have the `pkg-config` command:
+```sh
+# Install pkg-config on Ubuntu
+sudo apt-get install pkg-config
+```
+
+Then get the source as follows:
+
+```sh
+# Clone the repo as "dawn"
+git clone https://dawn.googlesource.com/dawn dawn && cd dawn
+
+# Bootstrap the gclient configuration
+cp scripts/standalone.gclient .gclient
+
+# Fetch external dependencies and toolchains with gclient
+gclient sync
+```
+
+Then generate build files using `gn args out/Debug` or `gn args out/Release`.
+A text editor will appear asking build options, the most common option is `is_debug=true/false`; otherwise `gn args out/Release --list` shows all the possible options.
+
+Then use `ninja -C out/Release` to build dawn and for example `./out/Release/dawn_end2end_tests` to run the tests.
+
diff --git a/docs/DebugMarkers.md b/docs/debug_markers.md
similarity index 100%
rename from docs/DebugMarkers.md
rename to docs/debug_markers.md
diff --git a/docs/debugging.md b/docs/debugging.md
new file mode 100644
index 0000000..740d0b2
--- /dev/null
+++ b/docs/debugging.md
@@ -0,0 +1,3 @@
+# Debugging Dawn
+
+(TODO)
diff --git a/docs/infra.md b/docs/infra.md
new file mode 100644
index 0000000..8870a38
--- /dev/null
+++ b/docs/infra.md
@@ -0,0 +1,3 @@
+# Dawn's contiuous testing infrastructure
+
+(TODO)
diff --git a/docs/overview.md b/docs/overview.md
new file mode 100644
index 0000000..e63afeb
--- /dev/null
+++ b/docs/overview.md
@@ -0,0 +1,55 @@
+# Dawn repository overview
+
+This repository contains the implementation of Dawn, which is itself composed of two main libraries (dawn_native and dawn_wire), along with support libraries, tests, and samples. Dawn makes heavy use of code-generation based on the `dawn.json` file that describes the native WebGPU API. It is used to generate the API headers, C++ wrapper, parts of the client-server implementation, and more!
+
+## Directory structure
+
+- [`dawn.json`](../dawn.json): contains a description of the native WebGPU in JSON form. It is the data model that's used by the code generators.
+- [`dawn_wire.json`](../dawn_wire.json): contains additional information used to generate `dawn_wire` files, such as commands in addition to regular WebGPU commands.
+- [`examples`](../examples): a small collection of samples using the native WebGPU API. They were mostly used when bringing up Dawn for the first time, and to test the `WGPUSwapChain` object.
+- [`generator`](../generator): directory containg the code generators and their templates. Generators are based on Jinja2 and parse data-models from JSON files.
+ - [`dawn_json_generator.py`](../generator/dawn_json_generator.py): the main code generator that outputs the WebGPU headers, C++ wrapper, client-server implementation, etc.
+ - [`templates`](../generator/templates): Jinja2 templates for the generator, with subdirectories for groups of templates that are all used in the same library.
+- [`infra`](../infra): configuration file for the commit-queue infrastructure.
+- [`scripts`](../scripts): contains a grab-bag of files that are used for building Dawn, in testing, etc.
+- [`src`](../src):
+ - [`common`](../src/common): helper code that is allowed to be used by Dawn's core libraries, `dawn_native` and `dawn_wire`. Also allowed for use in all other Dawn targets.
+ - [`dawn_native`](../src/dawn_native): code for the implementation of WebGPU on top of graphics APIs. Files in this folder are the "frontend" while subdirectories are "backends".
+ - `<backend>`: code for the implementation of the backend on a specific graphics API, for example `d3d12`, `metal` or `vulkan`.
+ - [`dawn_platform`](../src/dawn_platform): definition of interfaces for dependency injection in `dawn_native` or `dawn_wire`.
+ - [`dawn_wire`](../src/dawn_wire): code for an implementation of WebGPU as a client-server architecture.
+ - [`fuzzers`](../src/fuzzers): various fuzzers for Dawn that are running in [Clusterfuzz](https://google.github.io/clusterfuzz/).
+ - [`include`](../src/include): public headers with subdirectories for each library. Note that some headers are auto-generated and not present directly in the directory.
+ - [`tests`](../src/tests):
+ - [`end2end`](../src/tests/end2end): tests for the execution of the WebGPU API and require a GPU to run.
+ - [`perf_tests`](../src/tests/perf_tests): benchmarks for various aspects of Dawn.
+ - [`unittests`](../src/tests/unittests): code unittests of internal classes, but also by extension WebGPU API tests that don't require a GPU to run.
+ - [`validation`](../src/tests/unittests/validation): WebGPU validation tests not using the GPU (frontend tests)
+ - [`white_box`](../src/tests/white_box): tests using the GPU that need to access the internals of `dawn_native` or `dawn_wire`.
+ - [`utils`](../src/utils): helper code to use Dawn used by tests and samples but disallowed for `dawn_native` and `dawn_wire`.
+- [`third_party`](../third_party): directory where dependencies live as well as their buildfiles.
+
+## Dawn Native (`dawn_native`)
+
+The largest library in Dawn is `dawn_native` which implements the WebGPU API by translating to native graphics APIs such as D3D12, Metal or Vulkan. It is composed of a frontend that does all the state-tracking and validation, and backends that do the actual translation to the native graphics APIs.
+
+`dawn_native` hosts the [SPVC](https://github.com/google/shaderc/tree/master/spvc) shader translator that validates SPIR-V for WebGPU and converts it to an equivalent shader for use in the native graphics API (HLSL for D3D12, MSL for Metal or Vulkan SPIR-V for Vulkan).
+
+## Dawn Wire (`dawn_wire`)
+
+A second library that implements both a client that takes WebGPU commands and serializes them into a buffer, and a server that deserializes commands from a buffer, validates they are well-formed and calls the relevant WebGPU commands. Some server to client communication also happens so the API's callbacks work properly.
+
+Note that `dawn_wire` is meant to do as little state-tracking as possible so that the client can be lean and defer most of the heavy processing to the server side where the server calls into `dawn_native`.
+
+## Dawn Proc (`dawn_proc`)
+
+Normally libraries implementing `webgpu.h` should implement function like `wgpuDeviceCreateBuffer` but instead `dawn_native` and `dawn_wire` implement the `dawnProcTable` which is a structure containing all the WebGPU functions Dawn implements. Then a `dawn_proc` library contains a static version of this `dawnProcTable` and for example forwards `wgpuDeviceCreateBuffer` to the `procTable.deviceCreateBuffer` function pointer. This is useful in two ways:
+
+ - It allows deciding at runtime whether to use `dawn_native` and `dawn_wire`, which is useful to test boths paths with the same binary in our infrastructure.
+ - It avoids applications that know they will only use Dawn to query all entrypoints at once instead of using `wgpuGetProcAddress` repeatedly.
+
+## Code generation
+
+When the WebGPU API evolves a lot of places in Dawn have to be updated, so to reduce efforts, Dawn relies heavily on code generation. The code generators are based on [Jinja2](https://jinja.palletsprojects.com/) and separate the model from the view like in Web development. The model is some JSON file, usually [`dawn.json`](../dawn.json) and the views are the Jinja2 templates in [`generator/templates`](../generator/templates). The generated files are not checked into the repository but instead are generated during the build.
+
+Most of the code generation is done in [`dawn_json_generator.py`](../generator/dawn_json_generator.py) but other generators exist so common functionality to build code generators has been extracted into[`generator_lib.py`](../generator/generator_lib.py).
diff --git a/docs/testing.md b/docs/testing.md
new file mode 100644
index 0000000..c2717a1
--- /dev/null
+++ b/docs/testing.md
@@ -0,0 +1,3 @@
+# Testing Dawn
+
+(TODO)
