Tint currently has two fuzzer target executables: tint_wgsl_fuzzer which takes WGSL source code as input and tint_ir_fuzzer which takes a protobuf binary file as input.
Both fuzzers are implemented using libFuzzer, and are intended to be automatically and continuously run by Chromium's ClusterFuzz infrastructure. Currently only tint_wgsl_fuzzer is deployed on ClusterFuzz.
Fuzzer targets are automatically found via gn refs. See tint.gni for the core fuzzer target rules.
Tint's fuzzers are implemented as functions registered with the macros:
TINT_WGSL_PROGRAM_FUZZER() registers a fuzzer function that is handed a tint::Program.TINT_IR_MODULE_FUZZER() registers a fuzzer function that is handed a tint::core::ir::Module.The tint_wgsl_fuzzer target can be built with either CMake or GN:
TINT_BUILD_FUZZERS=1 (pass -DTINT_BUILD_FUZZERS=1 to cmake)use_libfuzzer = true in args.gn.Building tint_ir_fuzzer requires additional flags to be set:
TINT_BUILD_IR_BINARY=1 and TINT_BUILD_IR_FUZZER=1 (pass -DTINT_BUILD_IR_BINARY=1 -DTINT_BUILD_IR_FUZZER=1 to cmake)tint_build_ir_binary = true and tint_build_ir_fuzzer = true in args.gn.The tint_wgsl_fuzzer and tint_ir_fuzzer executables accept the standard libFuzzer command line arguments with extended command line arguments described below.
There's also a helper tool to run the fuzzers locally:
To run the local fuzzers across the full number of CPU threads available on the system, seeded with the corpus in test/tint, and using the dictionary in src/tint/cmd/fuzz/wgsl/dictionary.txt run:
tools/run fuzz
To check that all the test files in the corpus directory, test/tint by default, pass the fuzzers without error and then exit, run:
tools/run fuzz --check
Note: This is run by Dawn‘s CQ presubmit to check that fuzzers aren’t accidentally broken.
To run the local fuzzers using the same corpus used by ClusterFuzz:
tools/run fuzz -corpus out/libfuzz/gen/fuzzers/wgsl_corpus
Note that this corpus directory is generated when building the GN target tint_generate_wgsl_corpus.
Generate the tint_wgsl_fuzzer corpus using the tint_generate_wgsl_corpus GN target, which produces a corpus in <build_dir>/gen/fuzzers/wgsl_corpus. Pass in the path to the corpus directory as an argument to the fuzzer executable to use it. It's also a good idea to pass in the dictionary with -dict=src/tint/cmd/fuzz/wgsl/dictionary.txt:
autoninja -C out/libfuzz tint_generate_wgsl_corpus out/libfuzz/tint_wgsl_fuzzer.exe -dict=src/tint/cmd/fuzz/wgsl/dictionary.txt out/libfuzz/gen/fuzzers/wgsl_corpus
Similarly, the tint_ir_fuzzer corpus can be generated using the tint_generate_ir_corpus GN target, producing the corpus in <build_dir>/gen/fuzzers/ir_corpus. For the IR fuzzer, we don't pass in the dictionary, since we are using libprotobuf-mutator for mutating (the proto file effectively defines the dictionary):
autoninja -C out/libfuzz tint_generate_ir_corpus out/libfuzz/tint_ir_fuzzer.exe out/libfuzz/gen/fuzzers/ir_corpus
tint::Program fuzzerCreate a new source file with a _fuzz.cc suffix.
#include "src/tint/cmd/fuzz/wgsl/fuzz.h"
Define a function in a (possibly nested) anonymous namespace with one of the signatures:
void MyFuzzer(const tint::Program& program /*, ...additional fuzzed parameters... */) {void MyFuzzer(const tint::Program& program, const tint::fuzz::wgsl::Context& context /*, ...additional fuzzed parameters... */) {The optional context parameter holds information about the Program and the environment used to run the fuzzers.
Implement your fuzzer function, using TINT_ICE() to catch invalid state. Return early if the fuzzer cannot handle the input.
At the bottom of the file, in the global namespace, register the fuzzer with: TINT_WGSL_PROGRAM_FUZZER(MyFuzzer);
Use tools/run gen build to generate the build files for this new fuzzer.
Example:
#include "src/tint/cmd/fuzz/wgsl/fuzz.h" namespace tint::my_namespace { namespace { bool CanRun(const tint::Program& program) { if (program.AST().HasOverrides()) { return false; // Overrides are not supported. } return true; } void MyWGSLFuzzer(const tint::Program& program, bool a_fuzzer_provided_value) { if (!CanRun(program)) { return; } // Check something with program. } } // namespace } // namespace tint::my_namespace TINT_WGSL_PROGRAM_FUZZER(tint::my_namespace::MyWGSLFuzzer);
tint::core::ir::Module fuzzerCreate a new source file with a _fuzz.cc suffix.
#include "src/tint/cmd/fuzz/ir/fuzz.h"
Define a function in a (possibly nested) anonymous namespace with the signature:
void MyFuzzer(core::ir::Module& module /*, ...additional fuzzed parameters... */) {Implement your fuzzer function, using TINT_ICE() to catch invalid state. Return early if the fuzzer cannot handle the input.
At the bottom of the file, in the global namespace, register the fuzzer with: TINT_IR_MODULE_FUZZER(MyFuzzer);
[Note: Often different capabilities need to be enabled before/after a fuzzer function runs to ensure validation passes, these can be declared using pre_capabilities and post_capabilities when registering the fuzzer ]
tools/run gen build to generate the build files for this new fuzzer.Example:
#include "src/tint/cmd/fuzz/ir/fuzz.h" namespace tint::my_namespace { namespace { void MyIRFuzzer(core::ir::Module& module) { // Do something interesting with module. } } // namespace } // namespace tint::my_namespace TINT_IR_MODULE_FUZZER(tint::my_namespace::MyIRFuzzer);
WGSL and IR fuzzer functions can also declare any number of additional parameters, which will be populated with fuzzer provided data. These additional parameters must come at the end of the signatures described above, and can be of the following types:
Any integer, float or bool type.
Any structure reflected with TINT_REFLECT.
Note: It's recommended to use a const reference, for these to avoid pass-by-value overheads.
Any enum reflected with TINT_REFLECT_ENUM_RANGE.
Tint has two fuzzer executable targets:
tint_wgsl_fuzzertint_wgsl_fuzzer accepts WGSL textual input and parses line comments (//) as a base-64 binary encoded data stream for the additional fuzzer parameters.
The entry point for the fuzzer lives at src/tint/cmd/fuzz/wgsl/main_fuzz.cc.
On top of the standard libFuzzer command line arguments, the fuzzer supports the following extended command line arguments:
--help: lists the command line arguments.--filter=<name>: only runs the fuzzer functions that contain the given string in its name.--concurrent: each of the fuzzer functions will be run on a separate, concurrent thread. This potentially offers performance improvements, and also tests for concurrent execution.--verbose : prints verbose information about what the fuzzer is doing.--dump : prints shader source, including input WGSL, and generated HLSL, MSL, and GLSL.The tint_wgsl_fuzzer will do the following:
TINT_WGSL_PROGRAM_FUZZER()Program to an IR module and run the function for each function registered with TINT_IR_MODULE_FUZZER(). Note: The Program is converted to an IR module for each registered IR fuzzer as the module is mutable.tint_ir_fuzzertint_ir_fuzzer accepts binary protocol buffer inputs and uses libprotobuf-mutator for mutating this binary format directly.
The suffix .tirb is sometimes used for IR fuzz test case files, which helps our tooling infer the format. The fuzzer itself does not generate/depend on this suffix though, instead using the libFuzzer standard prefixes, i.e. crash-... and slow-..., when needed.
The input protobuf is defined in src/tint/utils/protos/ir_fuzz/ir_fuzz.proto is a composite of two elements, a protobuf src/tint/utils/protos/ir/ir.proto that defines the actual IR for the test case, and an opaque binary blob that includes the additional fuzzer parameters akin to tint_wgsl_fuzzer.
The entry point for the fuzzer lives at src/tint/cmd/fuzz/ir/main_fuzz.cc.
This fuzzer accepts the same set of flags as tint_wgsl_fuzzer (both libFuzzer and extended), expect --dump which is currently not supported.
The tint_ir_fuzzer will do the following:
Program for the IR module and a binary blob that will be passed into each function as options. If the input cannot be decoded due to being invalid or containing select constructs that can never occur in real world inputs, then the fuzzer will early return.TINT_IR_PROGRAM_FUZZER(), make a copy of the IR module, since it is mutable, and then invoke the fuzzer function.[Note: The tooling related to working with tint_ir_fuzz test cases is a WIP and may not be complete]
Since the input test cases for tint_ir_fuzzer are in a non-human readable format, there is additional tooling needed for working with them.
For displaying the IR contents of a test case file (i.e. crash-..., or foo.tirb), there is a disassembler ir_fuzz_dis, which is the easiest way to dump out the contents of a test case. This disassembler supports a subset of the Tint CLI for outputting in various formats and to files, etc.
The disassembler has two significant known limitations though.
First, it does not dump out the binary options blob. This is because this data is interpreted on a per-fuzzer function basis, so there isn't a general human readable interpretation of its contents. If you need to understand how the binary blob is being handled by a fuzzer you will either need to add logging or use a debugger.
The other known limitation is that this disassembler depends on Tint‘s IR printing code, which assumes well formed inputs. Sometimes it will fail to output anything, or more subtly not output values it did’t expect. For example if an instruction only expects 2 params and there are actually 3 in the IR, then the disassembler may only print the first two. The validator should catch these issues and give you useful warnings, but since it is a WIP itself there may be omissions in its implementation.
Due to the second limitation on the disassembler, there are times where you will need to dump the raw contents of the binary protocol buffer in a textual format. This can be done using the tool protoc which is part of libprotobuf and either installed via a system package or built as part building the fuzzers. A full tutorial on using protoc is beyond the scope of this document, see protobuf docs for more details, but the basic invocation looks like this:
out/Fuzzer/protoc --decode tint.cmd.fuzz.ir.pb.Root ./src/tint/utils/protos/ir_fuzz/ir_fuzz.proto < ./input.tirb
The important thing to note is the use of --decode tint.cmd.fuzz.ir.pb.Root to decode just the IR portion of the input, but still needing to supply top-level ir_fuzz.proto as the format.
For generating binary test cases a rudimentary assembler, ir_fuzz_as, has been implemented.
It is primarily used for converting the Tint test shaders into a seed corpus for tint_ir_fuzzer, but can be used to convert user supplied shaders into IR tests cases.
Similar to the disassembler it only operates on the shader/IR portion of the test case format, and does not support embedding/manipulating the binary blob portion. (There is currently no roundtrip workflow for taking a test case binary, unpacking it, modifying it, and repacking it with original blob, other then manually unpacking and packing protobufs, which is left as an exercise to the reader).
It also suffers from the same limitation of the disassembler with regards to not handling malformed inputs particularly well, since it depends on the general Tint CLI parsing/printing infrastructure.
It is capable of dumping raw text of the IR protobuf that it generates, which can be useful for understanding how the IR is being encoded/decoded, since you can take a snippet of valid WGSL and dump out what the protobuf looks like for it.
To debug a specific registered fuzzer function, one strategy is to add a TINT_ICE call at the top of the function, and then run the fuzzer with -filter <name> to have it only run that specific fuzzer. When the function is called, the libfuzzer harness will emit a crash file that can be used as input on subsequent runs. Remove the TINT_ICE and run the fuzzer again using this crash file.
For example, if we wish to debug tint::msl::writer::IRFuzzer, we would first insert a TINT_ICE at the top:
Result<SuccessType> IRFuzzer(core::ir::Module& module, const fuzz::ir::Context& context, Options options) { TINT_ICE() << "Crash"; // Comment out the rest of the body to avoid unreachable code warnings }
Build and run the fuzzer, filtering in this function:
autoninja -C out/libfuzz tint_wgsl_fuzzer out/libfuzz/tint_wgsl_fuzzer -filter=tint::msl::writer::IRFuzzer
It can take a little while before libfuzzer generates a valid input WGSL, but eventually it will call into the function and crash on the ICE:
...
#71607 NEW cov: 3633 ft: 8316 corp: 1045/8248b lim: 25 exec/s: 2469 rss: 158Mb L: 8/25 MS: 2 ShuffleBytes-PersAutoDict- DE: "true"-
#71669 NEW cov: 3633 ft: 8317 corp: 1046/8266b lim: 25 exec/s: 2471 rss: 158Mb L: 18/25 MS: 2 CMP-ChangeByte- DE: "if"-
#71697 REDUCE cov: 3633 ft: 8317 corp: 1046/8264b lim: 25 exec/s: 2472 rss: 158Mb L: 6/25 MS: 3 PersAutoDict-ChangeBit-EraseBytes- DE: "\001\002"-
ICE while running fuzzer: 'tint::msl::writer::IRFuzzer'
..\..\src\tint\lang\msl\writer\writer_fuzz.cc:63 internal compiler error: Crash
==25204== ERROR: libFuzzer: deadly signal
NOTE: libFuzzer has rudimentary signal handlers.
Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 1 PersAutoDict- DE: "or"-; base unit: b34d87c378ebbbfbfd475303dcc75d1d1b2a7c7a
0x2f,0x2f,0x33,0x33,0x33,0x33,0x6f,0x72,0x33,0x33,0x33,0x33,0x33,0x33,0x33,0x33,0x33,0x33,0x2a,0x2a,0x30,0x32,
//3333or3333333333**02
artifact_prefix='./'; Test unit written to ./crash-21563a85afd5322d9e17c1c43fd3d4029778d6e7
Base64: Ly8zMzMzb3IzMzMzMzMzMzMzKiowMg==
Note that the second to last line specifies that the input test was written to a file. Now we can remove the TINT_ICE and run the fuzzer with just this file as input:
out/libfuzz/tint_wgsl_fuzzer ./crash-21563a85afd5322d9e17c1c43fd3d4029778d6e7