4.1 User Code Blocks

User code blocks begin following a "<<" token and end with a ">>" token found at the end of a line. All text lines in the code block are copied verbatim into the output file.

4.1.1 Standalone Code Blocks

C example:

<<
#include <stdio.h>
>>

D example:

<<
import std.stdio;
>>

Rust example:

<<
use std::collections::HashMap;
>>

Standalone code blocks are emitted early in the output file as top-level code outside the context of any function. Standalone code blocks are a good place to include/import any other necessary supporting code modules. They can also define helper functions that can be reused by lexer or parser user code blocks. They are emitted in the order they are defined in the grammar file.

For a C target, the word "header" may immediately follow the "<<" token to cause Propane to emit the code block in the generated header file rather than the generated implementation file. This allows including another header that may be necessary to define any types needed by a ptype directive, for example:

<<header
#include "mytypes.h"
>>

4.1.2 Lexer pattern code blocks

Lexer code blocks appear between << and >> markers following a drop, token, or pattern expression. User code in a lexer code block will be executed when the lexer matches the given pattern. Assignment to the $$ symbol will associate a parser value with the lexed token. This parser value can then be used later in a parser rule.

The input text positions of the matched token can also be accessed from within a lexer code block. Each of these positions is an instance of the p_position_t structure (see p_position_t), which contains 1-based row and col fields. The start position of the matched token is accessed with ${position}, and the end position of the matched token is accessed with ${end_position}.

Example:

token integer /\d+/ <<
  printf("integer token on row %d, col %d\n",
      ${position}.row, ${position}.col);
  $$ = parse_integer(match_text, match_length);
>>

4.1.2.1 C/C++ lexer code block arguments

The lexer code block is passed the following arguments:

Example:

ptype long;

token integer /\d+/ <<
  long v = 0;
  for (size_t i = 0u; i < match_length; i++)
  {
    v *= 10;
    v += (match_text[i] - '0');
  }
  $$ = v;
>>

4.1.2.2 D lexer code block arguments

The lexer code block is passed the following arguments:

ptype ulong;

token integer /\d+/ <<
  ulong v;
  foreach (c; match_text)
  {
    v *= 10;
    v += (c - '0');
  }
  $$ = v;
>>

4.1.2.3 Rust lexer code block arguments

The lexer code block is passed the following arguments:

The matched text is a byte slice rather than a string; use std::str::from_utf8() or String::from_utf8_lossy() to view it as a string.

ptype i64;

token integer /\d+/ <<
  let mut v: i64 = 0;
  for c in match_text
  {
      v *= 10;
      v += (c - b'0') as i64;
  }
  $$ = v;
>>

4.1.3 Parser rule code blocks

Example:

E1 -> E1 plus E2 << $$ = $1 + $3; >>

Parser rule code blocks appear following a rule expression. User code in a parser rule code block will be executed when the parser reduces the given rule. Assignment to the $$ symbol will associate a parser value with the reduced rule. Parser values for the rules or tokens in the rule pattern can be accessed positionally with tokens $1, $2, $3, etc...

The input text positions for the reduced rule and for the individual rule components can also be accessed from within a parser rule code block. Each of these positions is an instance of the p_position_t structure (see p_position_t), which contains 1-based row and col fields.

The start position of the overall reduced rule is accessed with ${$.position}, and the end position of the overall reduced rule is accessed with ${$.end_position}.

The start and end positions of an individual rule component are accessed positionally with ${N.position} and ${N.end_position}, where N is the 1-based index of the component (${1.position} for the first component, ${2.position} for the second, and so on).

Example:

Assignment -> ident equals Expr <<
    printf("assignment on row %d, col %d\n",
        ${$.position}.row, ${$.position}.col);
    printf("target identifier ends on row %d, col %d\n",
        ${1.end_position}.row, ${1.end_position}.col);
    printf("expression starts on row %d, col %d\n",
        ${3.position}.row, ${3.position}.col);
>>

A rule or rule component that allows for an empty match may not have valid positions. In this case the position should be checked for validity before its row and col fields are used (see p_position_valid). For C targets this can be accomplished with if (p_position_valid(${$.position})), for D targets with if (${$.position}.valid), and for Rust targets with if ${$.position}.valid().

In tree generation mode, a full parse tree is automatically constructed in memory for user code to traverse after parsing is complete. Parser rule code blocks are still supported in tree generation mode, but they behave differently than when tree generation mode is not active. The code block for a rule is executed after the rule has been matched and its tree node has been fully formed. Within the code block, $$ refers to the tree node handle for the reduced rule, and the rule components are accessed positionally with $1, $2, $3, etc..., each a tree node handle for that component (a rule node or a Token node). Field aliases (see the "Specifying parser rules" section) may also be used to reference a component tree node by name; a field alias behaves identically to the positional reference for that component.

Tree nodes are stored in a compact arena owned by the parser context and are referenced by lightweight handles rather than pointers. The whole tree is freed together with the context by p_context_delete(); there is no separate tree delete function, and tree node handles are only valid while the context is alive.

Child fields, positions, and token payloads are accessed through per-language accessors on a node handle:

In C++ and D, a node handle can also be used directly as a boolean condition, which is equivalent to testing valid() (C++) or valid (D):

if (node)
{
    /* node refers to a valid (non-null) tree node. */
}

For C, use the p_node_valid(node) macro, and for Rust use the node.valid() method; neither language supports a user-defined conversion to a boolean condition.

The positional position expansions (${$.position}, ${N.position}, etc...) are not available in tree generation mode; use the position accessors above instead.

C example:

tree;

Assignment -> ident equals Expr <<
    /* $$ is the Assignment tree node, $1 is the ident Token node, and $3 is
     * the Expr rule node. */
    printf("assignment on row %d, col %d\n",
        p_node_position($$).row, p_node_position($$).col);
    printf("target identifier ends on row %d, col %d\n",
        p_node_end_position($1).row, p_node_end_position($1).col);
>>

Rust example:

tree;

Assignment -> ident equals Expr <<
    /* $$ is the Assignment tree node, $1 is the ident Token node, and $3 is
     * the Expr rule node. */
    println!("assignment on row {}, col {}",
        $$.position().row, $$.position().col);
    println!("target identifier ends on row {}, col {}",
        $1.end_position().row, $1.end_position().col);
>>