« Prev
4.14 token_user_fields statement - adding custom token fields		Table of ContentsNext »
4.17 Specifying a lexer pattern

4.15 tree statement - tree generation mode

To activate tree generation mode, place the tree statement in your grammar file:

tree;

It is recommended to place this statement early in the grammar.

In tree generation mode various aspects of propane's behavior are changed:

Example tree generation grammar:

tree;

ptype int;

token a << $$ = 11; >>
token b << $$ = 22; >>
token one /1/;
token two /2/;
token comma /,/ <<
  $$ = 42;
>>
token lparen /\(/;
token rparen /\)/;
drop /\s+/;

Start -> Items;

Items -> Item:item ItemsMore;
Items -> ;

ItemsMore -> comma Item:item ItemsMore;
ItemsMore -> ;

Item -> a;
Item -> b;
Item -> lparen Item:item rparen;
Item -> Dual;

Dual -> One Two;
Dual -> Two One;
One -> one;
Two -> two;

The following unit test describes the fields that will be present for an example parse:

string input = "a, ((b)), b";
p_context_t * context = p_context_new(input);
assert_eq(P_SUCCESS, p_parse(context));
Start start = p_result(context);
assert(start.pItems1.valid);
assert(start.pItems.valid);
Items items = start.pItems;
assert(items.item.valid);
assert(items.item.pToken1.valid);
assert_eq(TOKEN_a, items.item.pToken1.token);
assert_eq(11, items.item.pToken1.pvalue);
assert(items.pItemsMore.valid);
ItemsMore itemsmore = items.pItemsMore;
assert(itemsmore.item.valid);
assert(itemsmore.item.item.valid);
assert(itemsmore.item.item.item.valid);
assert(itemsmore.item.item.item.pToken1.valid);
assert_eq(TOKEN_b, itemsmore.item.item.item.pToken1.token);
assert_eq(22, itemsmore.item.item.item.pToken1.pvalue);
assert(itemsmore.pItemsMore.valid);
itemsmore = itemsmore.pItemsMore;
assert(itemsmore.item.valid);
assert(itemsmore.item.pToken1.valid);
assert_eq(TOKEN_b, itemsmore.item.pToken1.token);
assert_eq(22, itemsmore.item.pToken1.pvalue);
assert(!itemsmore.pItemsMore.valid);
p_context_delete(context);

The equivalent traversal for a Rust target, where tree node fields are accessor methods and an absent child is a handle whose valid() method returns false:

let mut context = p_context_new(b"a, ((b)), b");
assert_eq!(P_SUCCESS, p_parse(&mut context));
let start = p_result(&context);
let items = start.pItems();
assert!(items.valid());
assert!(items.item().valid());
assert_eq!(TOKEN_a, items.item().pToken1().token());
assert_eq!(11, items.item().pToken1().pvalue());
let mut itemsmore = items.pItemsMore();
assert!(itemsmore.valid());
assert_eq!(TOKEN_b, itemsmore.item().item().item().pToken1().token());
assert_eq!(22, itemsmore.item().item().item().pToken1().pvalue());
itemsmore = itemsmore.pItemsMore();
assert_eq!(TOKEN_b, itemsmore.item().pToken1().token());
assert!(!itemsmore.pItemsMore().valid());
p_context_delete(context);

4.16 tree_prefix and tree_suffix statements

In tree generation mode, structure types are defined and named based on the rules in the grammar. Additionally, a structure type called Token is generated to hold parsed token information.

These structure names can be modified by using the tree_prefix or tree_suffix statements in the grammar file. The field names that refer to instances of the structures are not affected by the tree_prefix or tree_suffix values.

For example, if the following two lines were added to the example above:

tree_prefix ABC;
tree_suffix XYZ;

Then the types would be used as such instead:

string input = "a, ((b)), b";
p_context_t * context = p_context_new(input);
assert_eq(P_SUCCESS, p_parse(context));
ABCStartXYZ start = p_result(context);
assert(start.pItems1.valid);
assert(start.pItems.valid);
ABCItemsXYZ items = start.pItems;
assert(items.pItem.valid);
assert(items.pItem.pToken1.valid);
assert_eq(TOKEN_a, items.pItem.pToken1.token);
assert_eq(11, items.pItem.pToken1.pvalue);
assert(items.pItemsMore.valid);
ABCItemsMoreXYZ itemsmore = items.pItemsMore;
assert(itemsmore.pItem.valid);
assert(itemsmore.pItem.pItem.valid);
assert(itemsmore.pItem.pItem.pItem.valid);
assert(itemsmore.pItem.pItem.pItem.pToken1.valid);
« Prev
4.14 token_user_fields statement - adding custom token fields		Table of ContentsNext »
4.17 Specifying a lexer pattern