« Prev
4.1 User Code Blocks		Table of ContentsNext »
4.5 Specifying tokens - the token statement

4.2 Tree generation mode - the tree statement

To activate tree generation mode, place the tree statement in your grammar file:

tree;

It is recommended to place this statement early in the grammar.

In tree generation mode various aspects of propane's behavior are changed:

Example tree generation grammar:

tree;

ptype int;

token a << $$ = 11; >>
token b << $$ = 22; >>
token one /1/;
token two /2/;
token comma /,/ <<
  $$ = 42;
>>
token lparen /\(/;
token rparen /\)/;
drop /\s+/;

Start -> Items;

Items -> Item:item ItemsMore;
Items -> ;

ItemsMore -> comma Item:item ItemsMore;
ItemsMore -> ;

Item -> a;
Item -> b;
Item -> lparen Item:item rparen;
Item -> Dual;

Dual -> One Two;
Dual -> Two One;
One -> one;
Two -> two;

The following unit test describes the fields that will be present for an example parse:

string input = "a, ((b)), b";
p_context_t context;
p_context_init(&context, input);
assert_eq(P_SUCCESS, p_parse(&context));
Start * start = p_result(&context);
assert(start.pItems1 !is null);
assert(start.pItems !is null);
Items * items = start.pItems;
assert(items.item !is null);
assert(items.item.pToken1 !is null);
assert_eq(TOKEN_a, items.item.pToken1.token);
assert_eq(11, items.item.pToken1.pvalue);
assert(items.pItemsMore !is null);
ItemsMore * itemsmore = items.pItemsMore;
assert(itemsmore.item !is null);
assert(itemsmore.item.item !is null);
assert(itemsmore.item.item.item !is null);
assert(itemsmore.item.item.item.pToken1 !is null);
assert_eq(TOKEN_b, itemsmore.item.item.item.pToken1.token);
assert_eq(22, itemsmore.item.item.item.pToken1.pvalue);
assert(itemsmore.pItemsMore !is null);
itemsmore = itemsmore.pItemsMore;
assert(itemsmore.item !is null);
assert(itemsmore.item.pToken1 !is null);
assert_eq(TOKEN_b, itemsmore.item.pToken1.token);
assert_eq(22, itemsmore.item.pToken1.pvalue);
assert(itemsmore.pItemsMore is null);

4.3 tree_prefix and tree_suffix statements

In tree generation mode, structure types are defined and named based on the rules in the grammar. Additionally, a structure type called Token is generated to hold parsed token information.

These structure names can be modified by using the tree_prefix or tree_suffix statements in the grammar file. The field names that point to instances of the structures are not affected by the tree_prefix or tree_suffix values.

For example, if the following two lines were added to the example above:

tree_prefix ABC;
tree_suffix XYZ;

Then the types would be used as such instead:

string input = "a, ((b)), b";
p_context_t context;
p_context_init(&context, input);
assert_eq(P_SUCCESS, p_parse(&context));
ABCStartXYZ * start = p_result(&context);
assert(start.pItems1 !is null);
assert(start.pItems !is null);
ABCItemsXYZ * items = start.pItems;
assert(items.pItem !is null);
assert(items.pItem.pToken1 !is null);
assert_eq(TOKEN_a, items.pItem.pToken1.token);
assert_eq(11, items.pItem.pToken1.pvalue);
assert(items.pItemsMore !is null);
ABCItemsMoreXYZ * itemsmore = items.pItemsMore;
assert(itemsmore.pItem !is null);
assert(itemsmore.pItem.pItem !is null);
assert(itemsmore.pItem.pItem.pItem !is null);
assert(itemsmore.pItem.pItem.pItem.pToken1 !is null);

4.4 Freeing user-allocated memory in token node pvalue: the free_token_node statement

If user lexer code block allocates memory to store in a token node's pvalue, the free_token_node grammar statement can be used to specify the name of a function which will be called during the p_free_tree() call to free the memory associated with a token node.

Example:

<<
static void free_token(Token * token)
{
    free(token->pvalue);
}
>>
tree;
free_token_node free_token;
ptype int *;
token a <<
  $$ = (int *)malloc(sizeof(int));
  *$$ = 1;
>>
token b <<
  $$ = (int *)malloc(sizeof(int));
  *$$ = 2;
>>
Start -> a:a b:b;
« Prev
4.1 User Code Blocks		Table of ContentsNext »
4.5 Specifying tokens - the token statement