| « Prev 4.10 token statement - specifying tokens | Table of Contents | Next » 4.12 token_user_fields statement - adding custom token fields |
tokenid statement - defining tokens without a matching patternThe tokenid statement can be used to define a token without associating it
with a lexer pattern that matches it.
Example:
tokenid string;
The tokenid statement can be useful when defining a token that may optionally
be returned by user code associated with a pattern.
It is also useful when lexer modes and multiple lexer patterns are required to build up a full token. A common example is parsing a string. See the Lexer modes chapter for more information.
| « Prev 4.10 token statement - specifying tokens | Table of Contents | Next » 4.12 token_user_fields statement - adding custom token fields |