tree 4bfaf10ccdddc3b6dae9f225aa8b569b83c2d543
parent 1308395055b65c8ca11bc96f88d975e4b1af1e54
author dan sinclair <dsinclair@chromium.org> 1658767388 +0000
committer Dawn LUCI CQ <dawn-scoped@luci-project-accounts.iam.gserviceaccount.com> 1658767388 +0000

Pre-parse token list.

This CL updates the parser and lexer to pre-parse the token list and
then the parser works off the list of tokens. This allows the parser to
work with references to the tokens and not have to worry about them
going stale.

For any splittable token a placeholder token is injected after the
token. If the token ends up getting split the placeholder is overwritten
with the new type.

Change-Id: I5a8ccca15d8c14b5027df7dd2734be6753e46fa9
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97070
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>