| « Prev 4 Propane Grammar File | Table of Contents | Next » 4.2 context_user_fields statement - adding custom fields to the context |
User code blocks begin following a "<<" token and end with a ">>" token found at the end of a line. All text lines in the code block are copied verbatim into the output file.
C example:
<< #include <stdio.h> >>
D example:
<< import std.stdio; >>
Rust example:
<< use std::collections::HashMap; >>
Standalone code blocks are emitted early in the output file as top-level code outside the context of any function. Standalone code blocks are a good place to include/import any other necessary supporting code modules. They can also define helper functions that can be reused by lexer or parser user code blocks. They are emitted in the order they are defined in the grammar file.
For a C target, the word "header" may immediately follow the "<<" token to
cause Propane to emit the code block in the generated header file rather than
the generated implementation file.
This allows including another header that may be necessary to define any types
needed by a ptype directive, for example:
<<header #include "mytypes.h" >>
Lexer code blocks appear between << and >> markers following a drop,
token, or pattern expression.
User code in a lexer code block will be executed when the lexer matches the
given pattern.
Assignment to the $$ symbol will associate a parser value with the lexed
token.
This parser value can then be used later in a parser rule.
The input text positions of the matched token can also be accessed from within
a lexer code block.
Each of these positions is an instance of the p_position_t structure (see
p_position_t), which contains 1-based row and col fields.
The start position of the matched token is accessed with ${position}, and
the end position of the matched token is accessed with ${end_position}.
Example:
token integer /\d+/ <<
printf("integer token on row %d, col %d\n",
${position}.row, ${position}.col);
$$ = parse_integer(match_text, match_length);
>>
The lexer code block is passed the following arguments:
match_text (uint8_t const *) - points to the text matched by the lexer pattern.match_length (size_t) - length of the matched text.Example:
ptype long;
token integer /\d+/ <<
long v = 0;
for (size_t i = 0u; i < match_length; i++)
{
v *= 10;
v += (match_text[i] - '0');
}
$$ = v;
>>
The lexer code block is passed the following arguments:
match_text (string) - a slice containing the text matched by the lexer pattern.ptype ulong;
token integer /\d+/ <<
ulong v;
foreach (c; match_text)
{
v *= 10;
v += (c - '0');
}
$$ = v;
>>
The lexer code block is passed the following arguments:
match_text (&[u8]) - a slice containing the text matched by the lexer pattern.The matched text is a byte slice rather than a string; use
std::str::from_utf8() or String::from_utf8_lossy() to view it as a string.
ptype i64;
token integer /\d+/ <<
let mut v: i64 = 0;
for c in match_text
{
v *= 10;
v += (c - b'0') as i64;
}
$$ = v;
>>
Example:
E1 -> E1 plus E2 << $$ = $1 + $3; >>
Parser rule code blocks appear following a rule expression.
User code in a parser rule code block will be executed when the parser reduces
the given rule.
Assignment to the $$ symbol will associate a parser value with the reduced
rule.
Parser values for the rules or tokens in the rule pattern can be accessed
positionally with tokens $1, $2, $3, etc...
The input text positions for the reduced rule and for the individual rule
components can also be accessed from within a parser rule code block.
Each of these positions is an instance of the p_position_t structure (see
p_position_t), which contains 1-based row and col fields.
The start position of the overall reduced rule is accessed with
${$.position}, and the end position of the overall reduced rule is accessed
with ${$.end_position}.
The start and end positions of an individual rule component are accessed
positionally with ${N.position} and ${N.end_position}, where N is the
1-based index of the component (${1.position} for the first component,
${2.position} for the second, and so on).
Example:
Assignment -> ident equals Expr <<
printf("assignment on row %d, col %d\n",
${$.position}.row, ${$.position}.col);
printf("target identifier ends on row %d, col %d\n",
${1.end_position}.row, ${1.end_position}.col);
printf("expression starts on row %d, col %d\n",
${3.position}.row, ${3.position}.col);
>>
A rule or rule component that allows for an empty match may not have valid
positions.
In this case the position should be checked for validity before its row and
col fields are used (see p_position_valid).
For C targets this can be accomplished with
if (p_position_valid(${$.position})), for D targets with
if (${$.position}.valid), and for Rust targets with
if ${$.position}.valid().
In tree generation mode, a full parse tree is automatically constructed in
memory for user code to traverse after parsing is complete.
Parser rule code blocks are still supported in tree generation mode, but they
behave differently than when tree generation mode is not active.
The code block for a rule is executed after the rule has been matched and its
tree node has been fully formed.
Within the code block, $$ refers to the tree node handle for the reduced
rule, and the rule components are accessed positionally with $1, $2, $3,
etc..., each a tree node handle for that component (a rule node or a Token
node).
Field aliases (see the "Specifying parser rules" section) may also be used to
reference a component tree node by name; a field alias behaves identically to
the positional reference for that component.
Tree nodes are stored in a compact arena owned by the parser context and are
referenced by lightweight handles rather than pointers. The whole tree is freed
together with the context by p_context_delete(); there is no separate tree
delete function, and tree node handles are only valid while the context is
alive.
Child fields, positions, and token payloads are accessed through per-language accessors on a node handle:
p_TYPE_field(node) and tree walk macros
p_tree_walk_TYPE(node, field1, field2, ...); generic accessors
p_node_valid(node), p_node_position(node), p_node_end_position(node),
p_node_n_fields(node), p_node_data(node) (a pointer to the node record,
for token payload and user fields), and p_node_id(node) (for identity
comparison).(), e.g. node.field(), node.valid(),
node.position(), node.token(), node.pvalue(), node.data(), and
node.node_id() (for identity comparison). The C-style functions and
macros above are also available.@property accessors, e.g. node.field, node.valid, node.position,
node.token, node.pvalue, and node.node_id (for identity comparison).(), e.g. node.field(), node.valid(),
node.position(), node.end_position(), node.n_fields(),
node.token(), node.pvalue(), node.data() (a reference to the node
record, for token user fields), and node.node_id() (for identity
comparison).In C++ and D, a node handle can also be used directly as a boolean condition,
which is equivalent to testing valid() (C++) or valid (D):
if (node)
{
/* node refers to a valid (non-null) tree node. */
}
For C, use the p_node_valid(node) macro, and for Rust use the node.valid()
method; neither language supports a user-defined conversion to a boolean
condition.
The positional position expansions (${$.position}, ${N.position}, etc...)
are not available in tree generation mode; use the position accessors above
instead.
C example:
tree;
Assignment -> ident equals Expr <<
/* $$ is the Assignment tree node, $1 is the ident Token node, and $3 is
* the Expr rule node. */
printf("assignment on row %d, col %d\n",
p_node_position($$).row, p_node_position($$).col);
printf("target identifier ends on row %d, col %d\n",
p_node_end_position($1).row, p_node_end_position($1).col);
>>
Rust example:
tree;
Assignment -> ident equals Expr <<
/* $$ is the Assignment tree node, $1 is the ident Token node, and $3 is
* the Expr rule node. */
println!("assignment on row {}, col {}",
$$.position().row, $$.position().col);
println!("target identifier ends on row {}, col {}",
$1.end_position().row, $1.end_position().col);
>>
| « Prev 4 Propane Grammar File | Table of Contents | Next » 4.2 context_user_fields statement - adding custom fields to the context |