Previous: , Up: C++ Scanner Interface   [Contents][Index]


10.1.6.2 Complete Symbols

With both %define api.value.type variant and %define api.token.constructor, the parser defines the type symbol_type, and expects yylex to have the following prototype.

Function: parser::symbol_type yylex ()
Function: parser::symbol_type yylex (type1 arg1, ...)

Return a complete symbol, aggregating its type (i.e., the traditional value returned by yylex), its semantic value, and possibly its location. Invocations of ‘%lex-param {type1 arg1}’ yield additional arguments.

For each token type, Bison generates named constructors as follows.

Method on parser: symbol_type make_token (const value_type& value, const location_type& location)
Method on parser: symbol_type make_token (const location_type& location)
Method on parser: symbol_type make_token (const value_type& value)
Method on parser: symbol_type make_token ()

Build a complete terminal symbol for the token type token (not including the api.token.prefix), whose semantic value, if it has one, is value of adequate value_type. Pass the location iff location tracking is enabled.

For instance, given the following declarations:

%define api.token.prefix {TOK_}
%token <std::string> IDENTIFIER;
%token <int> INTEGER;
%token COLON;
%token EOF 0;

Bison generates:

symbol_type make_IDENTIFIER (const std::string&, const location_type&);
symbol_type make_INTEGER (const int&, const location_type&);
symbol_type make_COLON (const location_type&);
symbol_type make_EOF (const location_type&);

which should be used in a Flex-scanner as follows.

[a-z]+   return yy::parser::make_IDENTIFIER (yytext, loc);
[0-9]+   return yy::parser::make_INTEGER (text_to_int (yytext), loc);
":"      return yy::parser::make_COLON (loc);
<<EOF>>  return yy::parser::make_EOF (loc);

Tokens that do not have an identifier are not accessible: you cannot simply use characters such as ':', they must be declared with %token, including the end-of-file token.


Previous: , Up: C++ Scanner Interface   [Contents][Index]