X-Git-Url: https://git.saurik.com/bison.git/blobdiff_plain/e254a580b550c8cbaff1709527cd896d972df010..f52e1e5e44fd155cfe0fbbf0fd370e99a256c20e:/doc/bison.texinfo diff --git a/doc/bison.texinfo b/doc/bison.texinfo index c6fc0a5f..0c09e35e 100644 --- a/doc/bison.texinfo +++ b/doc/bison.texinfo @@ -33,14 +33,14 @@ This manual (@value{UPDATED}) is for @acronym{GNU} Bison (version @value{VERSION}), the @acronym{GNU} parser generator. -Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998, -1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008 Free Software -Foundation, Inc. +Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998, 1999, +2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 Free +Software Foundation, Inc. @quotation Permission is granted to copy, distribute and/or modify this document under the terms of the @acronym{GNU} Free Documentation License, -Version 1.2 or any later version published by the Free Software +Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with the Front-Cover texts being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in (a) below. A copy of the license is included in the section entitled @@ -89,76 +89,76 @@ Cover art by Etienne Suvasa. @menu * Introduction:: * Conditions:: -* Copying:: The @acronym{GNU} General Public License says - how you can copy and share Bison +* Copying:: The @acronym{GNU} General Public License says + how you can copy and share Bison. Tutorial sections: -* Concepts:: Basic concepts for understanding Bison. -* Examples:: Three simple explained examples of using Bison. +* Concepts:: Basic concepts for understanding Bison. +* Examples:: Three simple explained examples of using Bison. Reference sections: -* Grammar File:: Writing Bison declarations and rules. -* Interface:: C-language interface to the parser function @code{yyparse}. -* Algorithm:: How the Bison parser works at run-time. -* Error Recovery:: Writing rules for error recovery. +* Grammar File:: Writing Bison declarations and rules. +* Interface:: C-language interface to the parser function @code{yyparse}. +* Algorithm:: How the Bison parser works at run-time. +* Error Recovery:: Writing rules for error recovery. * Context Dependency:: What to do if your language syntax is too - messy for Bison to handle straightforwardly. -* Debugging:: Understanding or debugging Bison parsers. -* Invocation:: How to run Bison (to produce the parser source file). -* Other Languages:: Creating C++ and Java parsers. -* FAQ:: Frequently Asked Questions -* Table of Symbols:: All the keywords of the Bison language are explained. -* Glossary:: Basic concepts are explained. -* Copying This Manual:: License for copying this manual. -* Index:: Cross-references to the text. + messy for Bison to handle straightforwardly. +* Debugging:: Understanding or debugging Bison parsers. +* Invocation:: How to run Bison (to produce the parser source file). +* Other Languages:: Creating C++ and Java parsers. +* FAQ:: Frequently Asked Questions +* Table of Symbols:: All the keywords of the Bison language are explained. +* Glossary:: Basic concepts are explained. +* Copying This Manual:: License for copying this manual. +* Index:: Cross-references to the text. @detailmenu --- The Detailed Node Listing --- The Concepts of Bison -* Language and Grammar:: Languages and context-free grammars, - as mathematical ideas. -* Grammar in Bison:: How we represent grammars for Bison's sake. -* Semantic Values:: Each token or syntactic grouping can have - a semantic value (the value of an integer, - the name of an identifier, etc.). -* Semantic Actions:: Each rule can have an action containing C code. -* GLR Parsers:: Writing parsers for general context-free languages. -* Locations Overview:: Tracking Locations. -* Bison Parser:: What are Bison's input and output, - how is the output used? -* Stages:: Stages in writing and running Bison grammars. -* Grammar Layout:: Overall structure of a Bison grammar file. +* Language and Grammar:: Languages and context-free grammars, + as mathematical ideas. +* Grammar in Bison:: How we represent grammars for Bison's sake. +* Semantic Values:: Each token or syntactic grouping can have + a semantic value (the value of an integer, + the name of an identifier, etc.). +* Semantic Actions:: Each rule can have an action containing C code. +* GLR Parsers:: Writing parsers for general context-free languages. +* Locations Overview:: Tracking Locations. +* Bison Parser:: What are Bison's input and output, + how is the output used? +* Stages:: Stages in writing and running Bison grammars. +* Grammar Layout:: Overall structure of a Bison grammar file. Writing @acronym{GLR} Parsers -* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars. -* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities. -* GLR Semantic Actions:: Deferred semantic actions have special concerns. -* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler. +* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars. +* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities. +* GLR Semantic Actions:: Deferred semantic actions have special concerns. +* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler. Examples -* RPN Calc:: Reverse polish notation calculator; - a first example with no operator precedence. -* Infix Calc:: Infix (algebraic) notation calculator. - Operator precedence is introduced. +* RPN Calc:: Reverse polish notation calculator; + a first example with no operator precedence. +* Infix Calc:: Infix (algebraic) notation calculator. + Operator precedence is introduced. * Simple Error Recovery:: Continuing after syntax errors. * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$. -* Multi-function Calc:: Calculator with memory and trig functions. - It uses multiple data-types for semantic values. -* Exercises:: Ideas for improving the multi-function calculator. +* Multi-function Calc:: Calculator with memory and trig functions. + It uses multiple data-types for semantic values. +* Exercises:: Ideas for improving the multi-function calculator. Reverse Polish Notation Calculator -* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc. -* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation. -* Lexer: Rpcalc Lexer. The lexical analyzer. -* Main: Rpcalc Main. The controlling function. -* Error: Rpcalc Error. The error reporting function. -* Gen: Rpcalc Gen. Running Bison on the grammar file. -* Comp: Rpcalc Compile. Run the C compiler on the output code. +* Rpcalc Declarations:: Prologue (declarations) for rpcalc. +* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation. +* Rpcalc Lexer:: The lexical analyzer. +* Rpcalc Main:: The controlling function. +* Rpcalc Error:: The error reporting function. +* Rpcalc Generate:: Running Bison on the grammar file. +* Rpcalc Compile:: Run the C compiler on the output code. Grammar Rules for @code{rpcalc} @@ -168,15 +168,15 @@ Grammar Rules for @code{rpcalc} Location Tracking Calculator: @code{ltcalc} -* Decls: Ltcalc Decls. Bison and C declarations for ltcalc. -* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations. -* Lexer: Ltcalc Lexer. The lexical analyzer. +* Ltcalc Declarations:: Bison and C declarations for ltcalc. +* Ltcalc Rules:: Grammar rules for ltcalc, with explanations. +* Ltcalc Lexer:: The lexical analyzer. Multi-Function Calculator: @code{mfcalc} -* Decl: Mfcalc Decl. Bison declarations for multi-function calculator. -* Rules: Mfcalc Rules. Grammar rules for the calculator. -* Symtab: Mfcalc Symtab. Symbol table management subroutines. +* Mfcalc Declarations:: Bison declarations for multi-function calculator. +* Mfcalc Rules:: Grammar rules for the calculator. +* Mfcalc Symbol Table:: Symbol table management subroutines. Bison Grammar Files @@ -191,11 +191,11 @@ Bison Grammar Files Outline of a Bison Grammar -* Prologue:: Syntax and usage of the prologue. +* Prologue:: Syntax and usage of the prologue. * Prologue Alternatives:: Syntax and usage of alternatives to the prologue. -* Bison Declarations:: Syntax and usage of the Bison declarations section. -* Grammar Rules:: Syntax and usage of the grammar rules section. -* Epilogue:: Syntax and usage of the epilogue. +* Bison Declarations:: Syntax and usage of the Bison declarations section. +* Grammar Rules:: Syntax and usage of the grammar rules section. +* Epilogue:: Syntax and usage of the epilogue. Defining Language Semantics @@ -206,6 +206,7 @@ Defining Language Semantics * Mid-Rule Actions:: Most actions go at the end of a rule. This says when, why and how to use the exceptional action in the middle of a rule. +* Named References:: Using named references in actions. Tracking Locations @@ -230,24 +231,28 @@ Bison Declarations Parser C-Language Interface -* Parser Function:: How to call @code{yyparse} and what it returns. -* Lexical:: You must supply a function @code{yylex} - which reads tokens. -* Error Reporting:: You must supply a function @code{yyerror}. -* Action Features:: Special features for use in actions. -* Internationalization:: How to let the parser speak in the user's - native language. +* Parser Function:: How to call @code{yyparse} and what it returns. +* Push Parser Function:: How to call @code{yypush_parse} and what it returns. +* Pull Parser Function:: How to call @code{yypull_parse} and what it returns. +* Parser Create Function:: How to call @code{yypstate_new} and what it returns. +* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns. +* Lexical:: You must supply a function @code{yylex} + which reads tokens. +* Error Reporting:: You must supply a function @code{yyerror}. +* Action Features:: Special features for use in actions. +* Internationalization:: How to let the parser speak in the user's + native language. The Lexical Analyzer Function @code{yylex} * Calling Convention:: How @code{yyparse} calls @code{yylex}. -* Token Values:: How @code{yylex} must return the semantic value - of the token it has read. -* Token Locations:: How @code{yylex} must return the text location - (line number, etc.) of the token, if the - actions want that. -* Pure Calling:: How the calling convention differs - in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). +* Token Values:: How @code{yylex} must return the semantic value + of the token it has read. +* Token Locations:: How @code{yylex} must return the text location + (line number, etc.) of the token, if the + actions want that. +* Pure Calling:: How the calling convention differs in a pure parser + (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). The Bison Parser Algorithm @@ -257,7 +262,7 @@ The Bison Parser Algorithm * Contextual Precedence:: When an operator's precedence depends on context. * Parser States:: The parser is a finite-state-machine with stack. * Reduce/Reduce:: When two rules are applicable in the same situation. -* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified. +* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified. * Generalized LR Parsing:: Parsing arbitrary context-free grammars. * Memory Management:: What happens when memory is exhausted. How to avoid it. @@ -311,33 +316,33 @@ A Complete C++ Example Java Parsers -* Java Bison Interface:: Asking for Java parser generation -* Java Semantic Values:: %type and %token vs. Java -* Java Location Values:: The position and location classes -* Java Parser Interface:: Instantiating and running the parser -* Java Scanner Interface:: Specifying the scanner for the parser -* Java Action Features:: Special features for use in actions. -* Java Differences:: Differences between C/C++ and Java Grammars -* Java Declarations Summary:: List of Bison declarations used with Java +* Java Bison Interface:: Asking for Java parser generation +* Java Semantic Values:: %type and %token vs. Java +* Java Location Values:: The position and location classes +* Java Parser Interface:: Instantiating and running the parser +* Java Scanner Interface:: Specifying the scanner for the parser +* Java Action Features:: Special features for use in actions +* Java Differences:: Differences between C/C++ and Java Grammars +* Java Declarations Summary:: List of Bison declarations used with Java Frequently Asked Questions -* Memory Exhausted:: Breaking the Stack Limits -* How Can I Reset the Parser:: @code{yyparse} Keeps some State -* Strings are Destroyed:: @code{yylval} Loses Track of Strings -* Implementing Gotos/Loops:: Control Flow in the Calculator -* Multiple start-symbols:: Factoring closely related grammars -* Secure? Conform?:: Is Bison @acronym{POSIX} safe? -* I can't build Bison:: Troubleshooting -* Where can I find help?:: Troubleshouting -* Bug Reports:: Troublereporting -* Other Languages:: Parsers in Java and others -* Beta Testing:: Experimenting development versions -* Mailing Lists:: Meeting other Bison users +* Memory Exhausted:: Breaking the Stack Limits +* How Can I Reset the Parser:: @code{yyparse} Keeps some State +* Strings are Destroyed:: @code{yylval} Loses Track of Strings +* Implementing Gotos/Loops:: Control Flow in the Calculator +* Multiple start-symbols:: Factoring closely related grammars +* Secure? Conform?:: Is Bison @acronym{POSIX} safe? +* I can't build Bison:: Troubleshooting +* Where can I find help?:: Troubleshouting +* Bug Reports:: Troublereporting +* More Languages:: Parsers in C++, Java, and so on +* Beta Testing:: Experimenting development versions +* Mailing Lists:: Meeting other Bison users Copying This Manual -* Copying This Manual:: License for copying this manual. +* Copying This Manual:: License for copying this manual. @end detailmenu @end menu @@ -347,10 +352,13 @@ Copying This Manual @cindex introduction @dfn{Bison} is a general-purpose parser generator that converts an -annotated context-free grammar into an @acronym{LALR}(1) or -@acronym{GLR} parser for that grammar. Once you are proficient with -Bison, you can use it to develop a wide range of language parsers, from those -used in simple desk calculators to complex programming languages. +annotated context-free grammar into a deterministic @acronym{LR} or +generalized @acronym{LR} (@acronym{GLR}) parser employing +@acronym{LALR}(1), @acronym{IELR}(1), or canonical @acronym{LR}(1) +parser tables. +Once you are proficient with Bison, you can use it to develop a wide +range of language parsers, from those used in simple desk calculators to +complex programming languages. Bison is upward compatible with Yacc: all properly-written Yacc grammars ought to work with Bison with no change. Anyone familiar with Yacc @@ -417,19 +425,19 @@ details of Bison will not make sense. If you do not already know how to use Bison or Yacc, we suggest you start by reading this chapter carefully. @menu -* Language and Grammar:: Languages and context-free grammars, - as mathematical ideas. -* Grammar in Bison:: How we represent grammars for Bison's sake. -* Semantic Values:: Each token or syntactic grouping can have - a semantic value (the value of an integer, - the name of an identifier, etc.). -* Semantic Actions:: Each rule can have an action containing C code. -* GLR Parsers:: Writing parsers for general context-free languages. -* Locations Overview:: Tracking Locations. -* Bison Parser:: What are Bison's input and output, - how is the output used? -* Stages:: Stages in writing and running Bison grammars. -* Grammar Layout:: Overall structure of a Bison grammar file. +* Language and Grammar:: Languages and context-free grammars, + as mathematical ideas. +* Grammar in Bison:: How we represent grammars for Bison's sake. +* Semantic Values:: Each token or syntactic grouping can have + a semantic value (the value of an integer, + the name of an identifier, etc.). +* Semantic Actions:: Each rule can have an action containing C code. +* GLR Parsers:: Writing parsers for general context-free languages. +* Locations Overview:: Tracking Locations. +* Bison Parser:: What are Bison's input and output, + how is the output used? +* Stages:: Stages in writing and running Bison grammars. +* Grammar Layout:: Overall structure of a Bison grammar file. @end menu @node Language and Grammar @@ -456,26 +464,27 @@ order to specify the language Algol 60. Any grammar expressed in essentially machine-readable @acronym{BNF}. @cindex @acronym{LALR}(1) grammars +@cindex @acronym{IELR}(1) grammars @cindex @acronym{LR}(1) grammars -There are various important subclasses of context-free grammar. Although it -can handle almost all context-free grammars, Bison is optimized for what -are called @acronym{LALR}(1) grammars. -In brief, in these grammars, it must be possible to -tell how to parse any portion of an input string with just a single -token of lookahead. Strictly speaking, that is a description of an -@acronym{LR}(1) grammar, and @acronym{LALR}(1) involves additional -restrictions that are -hard to explain simply; but it is rare in actual practice to find an -@acronym{LR}(1) grammar that fails to be @acronym{LALR}(1). +There are various important subclasses of context-free grammars. +Although it can handle almost all context-free grammars, Bison is +optimized for what are called @acronym{LR}(1) grammars. +In brief, in these grammars, it must be possible to tell how to parse +any portion of an input string with just a single token of lookahead. +For historical reasons, Bison by default is limited by the additional +restrictions of @acronym{LALR}(1), which is hard to explain simply. @xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for more information on this. +As an experimental feature, you can escape these additional restrictions by +requesting @acronym{IELR}(1) or canonical @acronym{LR}(1) parser tables. +@xref{Decl Summary,,lr.type}, to learn how. @cindex @acronym{GLR} parsing @cindex generalized @acronym{LR} (@acronym{GLR}) parsing @cindex ambiguous grammars @cindex nondeterministic parsing -Parsers for @acronym{LALR}(1) grammars are @dfn{deterministic}, meaning +Parsers for @acronym{LR}(1) grammars are @dfn{deterministic}, meaning roughly that the next grammar rule to apply at any point in the input is uniquely determined by the preceding input and a fixed, finite portion (called a @dfn{lookahead}) of the remaining input. A context-free @@ -704,8 +713,8 @@ from the values of the two subexpressions. @cindex shift/reduce conflicts @cindex reduce/reduce conflicts -In some grammars, Bison's standard -@acronym{LALR}(1) parsing algorithm cannot decide whether to apply a +In some grammars, Bison's deterministic +@acronym{LR}(1) parsing algorithm cannot decide whether to apply a certain grammar rule at a given point. That is, it may not be able to decide (on the basis of the input read so far) which of two possible reductions (applications of a grammar rule) applies, or whether to apply @@ -714,13 +723,13 @@ input. These are known respectively as @dfn{reduce/reduce} conflicts (@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts (@pxref{Shift/Reduce}). -To use a grammar that is not easily modified to be @acronym{LALR}(1), a +To use a grammar that is not easily modified to be @acronym{LR}(1), a more general parsing algorithm is sometimes necessary. If you include @code{%glr-parser} among the Bison declarations in your file (@pxref{Grammar Outline}), the result is a Generalized @acronym{LR} (@acronym{GLR}) parser. These parsers handle Bison grammars that contain no unresolved conflicts (i.e., after applying precedence -declarations) identically to @acronym{LALR}(1) parsers. However, when +declarations) identically to deterministic parsers. However, when faced with unresolved shift/reduce and reduce/reduce conflicts, @acronym{GLR} parsers use the simple expedient of doing both, effectively cloning the parser to follow both possibilities. Each of @@ -745,10 +754,10 @@ user-defined function on the resulting values to produce an arbitrary merged result. @menu -* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars. -* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities. -* GLR Semantic Actions:: Deferred semantic actions have special concerns. -* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler. +* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars. +* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities. +* GLR Semantic Actions:: Deferred semantic actions have special concerns. +* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler. @end menu @node Simple GLR Parsers @@ -762,11 +771,8 @@ merged result. @cindex shift/reduce conflicts In the simplest cases, you can use the @acronym{GLR} algorithm -to parse grammars that are unambiguous, but fail to be @acronym{LALR}(1). -Such grammars typically require more than one symbol of lookahead, -or (in rare cases) fall into the category of grammars in which the -@acronym{LALR}(1) algorithm throws away too much information (they are in -@acronym{LR}(1), but not @acronym{LALR}(1), @ref{Mystery Conflicts}). +to parse grammars that are unambiguous but fail to be @acronym{LR}(1). +Such grammars typically require more than one symbol of lookahead. Consider a problem that arises in the declaration of enumerated and subrange types in the @@ -803,7 +809,7 @@ type enum = (a); valid, and more-complicated cases can come up in practical programs.) These two declarations look identical until the @samp{..} token. -With normal @acronym{LALR}(1) one-token lookahead it is not +With normal @acronym{LR}(1) one-token lookahead it is not possible to decide between the two forms when the identifier @samp{a} is parsed. It is, however, desirable for a parser to decide this, since in the latter case @@ -842,9 +848,9 @@ reports a syntax error as usual. The effect of all this is that the parser seems to ``guess'' the correct branch to take, or in other words, it seems to use more -lookahead than the underlying @acronym{LALR}(1) algorithm actually allows -for. In this example, @acronym{LALR}(2) would suffice, but also some cases -that are not @acronym{LALR}(@math{k}) for any @math{k} can be handled this way. +lookahead than the underlying @acronym{LR}(1) algorithm actually allows +for. In this example, @acronym{LR}(2) would suffice, but also some cases +that are not @acronym{LR}(@math{k}) for any @math{k} can be handled this way. In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time, and the current Bison parser even takes exponential time and space @@ -897,7 +903,7 @@ expr : '(' expr ')' @end group @end example -When used as a normal @acronym{LALR}(1) grammar, Bison correctly complains +When used as a normal @acronym{LR}(1) grammar, Bison correctly complains about one reduce/reduce conflict. In the conflicting situation the parser chooses one of the alternatives, arbitrarily the one declared first. Therefore the following correct input is not @@ -929,7 +935,7 @@ there are at least two potential problems to beware. First, always analyze the conflicts reported by Bison to make sure that @acronym{GLR} splitting is only done where it is intended. A @acronym{GLR} parser splitting inadvertently may cause problems less obvious than an -@acronym{LALR} parser statically choosing the wrong alternative in a +@acronym{LR} parser statically choosing the wrong alternative in a conflict. Second, consider interactions with the lexer (@pxref{Semantic Tokens}) with great care. Since a split parser consumes tokens without performing any actions during the split, the lexer cannot obtain @@ -1149,7 +1155,7 @@ Another Bison feature requiring special consideration is @code{YYERROR} (@pxref{Action Features}), which you can invoke in a semantic action to initiate error recovery. During deterministic @acronym{GLR} operation, the effect of @code{YYERROR} is -the same as its effect in an @acronym{LALR}(1) parser. +the same as its effect in a deterministic parser. In a deferred semantic action, its effect is undefined. @c The effect is probably a syntax error at the split point. @@ -1376,15 +1382,15 @@ languages are written the same way. You can copy these examples into a source file to try them. @menu -* RPN Calc:: Reverse polish notation calculator; - a first example with no operator precedence. -* Infix Calc:: Infix (algebraic) notation calculator. - Operator precedence is introduced. +* RPN Calc:: Reverse polish notation calculator; + a first example with no operator precedence. +* Infix Calc:: Infix (algebraic) notation calculator. + Operator precedence is introduced. * Simple Error Recovery:: Continuing after syntax errors. * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$. -* Multi-function Calc:: Calculator with memory and trig functions. - It uses multiple data-types for semantic values. -* Exercises:: Ideas for improving the multi-function calculator. +* Multi-function Calc:: Calculator with memory and trig functions. + It uses multiple data-types for semantic values. +* Exercises:: Ideas for improving the multi-function calculator. @end menu @node RPN Calc @@ -1403,16 +1409,16 @@ The source code for this calculator is named @file{rpcalc.y}. The @samp{.y} extension is a convention used for Bison input files. @menu -* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc. -* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation. -* Lexer: Rpcalc Lexer. The lexical analyzer. -* Main: Rpcalc Main. The controlling function. -* Error: Rpcalc Error. The error reporting function. -* Gen: Rpcalc Gen. Running Bison on the grammar file. -* Comp: Rpcalc Compile. Run the C compiler on the output code. +* Rpcalc Declarations:: Prologue (declarations) for rpcalc. +* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation. +* Rpcalc Lexer:: The lexical analyzer. +* Rpcalc Main:: The controlling function. +* Rpcalc Error:: The error reporting function. +* Rpcalc Generate:: Running Bison on the grammar file. +* Rpcalc Compile:: Run the C compiler on the output code. @end menu -@node Rpcalc Decls +@node Rpcalc Declarations @subsection Declarations for @code{rpcalc} Here are the C and Bison declarations for the reverse polish notation @@ -1662,7 +1668,7 @@ therefore, @code{NUM} becomes a macro for @code{yylex} to use. The semantic value of the token (if it has one) is stored into the global variable @code{yylval}, which is where the Bison parser will look for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was -defined at the beginning of the grammar; @pxref{Rpcalc Decls, +defined at the beginning of the grammar; @pxref{Rpcalc Declarations, ,Declarations for @code{rpcalc}}.) A token type code of zero is returned if the end-of-input is encountered. @@ -1758,7 +1764,7 @@ have not written any error rules in this example, so any invalid input will cause the calculator program to exit. This is not clean behavior for a real calculator, but it is adequate for the first example. -@node Rpcalc Gen +@node Rpcalc Generate @subsection Running Bison to Make the Parser @cindex running Bison (introduction) @@ -1977,12 +1983,12 @@ most of the work needed to use locations will be done in the lexical analyzer. @menu -* Decls: Ltcalc Decls. Bison and C declarations for ltcalc. -* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations. -* Lexer: Ltcalc Lexer. The lexical analyzer. +* Ltcalc Declarations:: Bison and C declarations for ltcalc. +* Ltcalc Rules:: Grammar rules for ltcalc, with explanations. +* Ltcalc Lexer:: The lexical analyzer. @end menu -@node Ltcalc Decls +@node Ltcalc Declarations @subsection Declarations for @code{ltcalc} The C and Bison declarations for the location tracking calculator are @@ -2218,12 +2224,12 @@ $ Note that multiple assignment and nested function calls are permitted. @menu -* Decl: Mfcalc Decl. Bison declarations for multi-function calculator. -* Rules: Mfcalc Rules. Grammar rules for the calculator. -* Symtab: Mfcalc Symtab. Symbol table management subroutines. +* Mfcalc Declarations:: Bison declarations for multi-function calculator. +* Mfcalc Rules:: Grammar rules for the calculator. +* Mfcalc Symbol Table:: Symbol table management subroutines. @end menu -@node Mfcalc Decl +@node Mfcalc Declarations @subsection Declarations for @code{mfcalc} Here are the C and Bison declarations for the multi-function calculator. @@ -2319,7 +2325,7 @@ exp: NUM @{ $$ = $1; @} %% @end smallexample -@node Mfcalc Symtab +@node Mfcalc Symbol Table @subsection The @code{mfcalc} Symbol Table @cindex symbol table example @@ -2632,11 +2638,11 @@ As a @acronym{GNU} extension, @samp{//} introduces a comment that continues until end of line. @menu -* Prologue:: Syntax and usage of the prologue. +* Prologue:: Syntax and usage of the prologue. * Prologue Alternatives:: Syntax and usage of alternatives to the prologue. -* Bison Declarations:: Syntax and usage of the Bison declarations section. -* Grammar Rules:: Syntax and usage of the grammar rules section. -* Epilogue:: Syntax and usage of the epilogue. +* Bison Declarations:: Syntax and usage of the Bison declarations section. +* Grammar Rules:: Syntax and usage of the grammar rules section. +* Epilogue:: Syntax and usage of the epilogue. @end menu @node Prologue @@ -2700,9 +2706,6 @@ feature test macros can affect the behavior of Bison-generated @findex %code requires @findex %code provides @findex %code top -(The prologue alternatives described here are experimental. -More user feedback will help to determine whether they should become permanent -features.) The functionality of @var{Prologue} sections can often be subtle and inflexible. @@ -3047,8 +3050,12 @@ A @dfn{nonterminal symbol} stands for a class of syntactically equivalent groupings. The symbol name is used in writing grammar rules. By convention, it should be all lower case. -Symbol names can contain letters, digits (not at the beginning), -underscores and periods. Periods make sense only in nonterminals. +Symbol names can contain letters, underscores, periods, dashes, and (not +at the beginning) digits. Dashes in symbol names are a GNU +extension, incompatible with @acronym{POSIX} Yacc. Terminal symbols +that contain periods or dashes make little sense: since they are not +valid symbols (in most programming languages) they are not exported as +token names. There are three ways of writing terminal symbols in the grammar: @@ -3360,6 +3367,7 @@ the numbers associated with @var{x} and @var{y}. * Mid-Rule Actions:: Most actions go at the end of a rule. This says when, why and how to use the exceptional action in the middle of a rule. +* Named References:: Using named references in actions. @end menu @node Value Type @@ -3421,6 +3429,8 @@ Decl, ,Nonterminal Symbols}). @cindex action @vindex $$ @vindex $@var{n} +@vindex $@var{name} +@vindex $[@var{name}] An action accompanies a syntactic rule and contains C code to be executed each time an instance of that rule is recognized. The task of most actions @@ -3437,9 +3447,12 @@ Actions, ,Actions in Mid-Rule}). The C code in an action can refer to the semantic values of the components matched by the rule with the construct @code{$@var{n}}, which stands for the value of the @var{n}th component. The semantic value for the grouping -being constructed is @code{$$}. Bison translates both of these +being constructed is @code{$$}. In addition, the semantic values of +symbols can be accessed with the named references construct +@code{$@var{name}} or @code{$[@var{name}]}. Bison translates both of these constructs into expressions of the appropriate type when it copies the -actions into the parser file. @code{$$} is translated to a modifiable +actions into the parser file. @code{$$} (or @code{$@var{name}}, when it +stands for the current grouping) is translated to a modifiable lvalue, so it can be assigned to. Here is a typical example: @@ -3452,16 +3465,31 @@ exp: @dots{} @end group @end example +Or, in terms of named references: + +@example +@group +exp[result]: @dots{} + | exp[left] '+' exp[right] + @{ $result = $left + $right; @} +@end group +@end example + @noindent This rule constructs an @code{exp} from two smaller @code{exp} groupings connected by a plus-sign token. In the action, @code{$1} and @code{$3} +(@code{$left} and @code{$right}) refer to the semantic values of the two component @code{exp} groupings, which are the first and third symbols on the right hand side of the rule. -The sum is stored into @code{$$} so that it becomes the semantic value of +The sum is stored into @code{$$} (@code{$result}) so that it becomes the +semantic value of the addition-expression just recognized by the rule. If there were a useful semantic value associated with the @samp{+} token, it could be referred to as @code{$2}. +@xref{Named References,,Using Named References}, for more information +about using the named references construct. + Note that the vertical-bar character @samp{|} is really a rule separator, and actions are attached to a single rule. This is a difference with tools like Flex, for which @samp{|} stands for either @@ -3756,6 +3784,93 @@ compound: subroutine Now Bison can execute the action in the rule for @code{subroutine} without deciding which rule for @code{compound} it will eventually use. +@node Named References +@subsection Using Named References +@cindex named references + +While every semantic value can be accessed with positional references +@code{$@var{n}} and @code{$$}, it's often much more convenient to refer to +them by name. First of all, original symbol names may be used as named +references. For example: + +@example +@group +invocation: op '(' args ')' + @{ $invocation = new_invocation ($op, $args, @@invocation); @} +@end group +@end example + +@noindent +The positional @code{$$}, @code{@@$}, @code{$n}, and @code{@@n} can be +mixed with @code{$name} and @code{@@name} arbitrarily. For example: + +@example +@group +invocation: op '(' args ')' + @{ $$ = new_invocation ($op, $args, @@$); @} +@end group +@end example + +@noindent +However, sometimes regular symbol names are not sufficient due to +ambiguities: + +@example +@group +exp: exp '/' exp + @{ $exp = $exp / $exp; @} // $exp is ambiguous. + +exp: exp '/' exp + @{ $$ = $1 / $exp; @} // One usage is ambiguous. + +exp: exp '/' exp + @{ $$ = $1 / $3; @} // No error. +@end group +@end example + +@noindent +When ambiguity occurs, explicitly declared names may be used for values and +locations. Explicit names are declared as a bracketed name after a symbol +appearance in rule definitions. For example: +@example +@group +exp[result]: exp[left] '/' exp[right] + @{ $result = $left / $right; @} +@end group +@end example + +@noindent +Explicit names may be declared for RHS and for LHS symbols as well. In order +to access a semantic value generated by a mid-rule action, an explicit name +may also be declared by putting a bracketed name after the closing brace of +the mid-rule action code: +@example +@group +exp[res]: exp[x] '+' @{$left = $x;@}[left] exp[right] + @{ $res = $left + $right; @} +@end group +@end example + +@noindent + +In references, in order to specify names containing dots and dashes, an explicit +bracketed syntax @code{$[name]} and @code{@@[name]} must be used: +@example +@group +if-stmt: IF '(' expr ')' THEN then.stmt ';' + @{ $[if-stmt] = new_if_stmt ($expr, $[then.stmt]); @} +@end group +@end example + +It often happens that named references are followed by a dot, dash or other +C punctuation marks and operators. By default, Bison will read +@code{$name.suffix} as a reference to symbol value @code{$name} followed by +@samp{.suffix}, i.e., an access to the @samp{suffix} field of the semantic +value. In order to force Bison to recognize @code{name.suffix} in its entirety +as the name of a semantic value, bracketed syntax @code{$[name.suffix]} +must be used. + + @node Locations @section Tracking Locations @cindex location @@ -3799,8 +3914,11 @@ typedef struct YYLTYPE @} YYLTYPE; @end example -At the beginning of the parsing, Bison initializes all these fields to 1 -for @code{yylloc}. +When @code{YYLTYPE} is not defined, at the beginning of the parsing, Bison +initializes all these fields to 1 for @code{yylloc}. To initialize +@code{yylloc} with a custom location type (or to chose a different +initialization), use the @code{%initial-action} directive. @xref{Initial +Action Decl, , Performing Actions before Parsing}. @node Actions and Locations @subsection Actions and Locations @@ -3808,6 +3926,8 @@ for @code{yylloc}. @cindex actions, location @vindex @@$ @vindex @@@var{n} +@vindex @@@var{name} +@vindex @@[@var{name}] Actions are not only useful for defining language semantics, but also for describing the behavior of the output parser with locations. @@ -3819,6 +3939,11 @@ The location of the @var{n}th component of the right hand side is @code{@@@var{n}}, while the location of the left hand side grouping is @code{@@$}. +In addition, the named references construct @code{@@@var{name}} and +@code{@@[@var{name}]} may also be used to address the symbol locations. +@xref{Named References,,Using Named References}, for more information +about using the named references construct. + Here is a basic example using the default data type for locations: @example @@ -4459,7 +4584,7 @@ be @var{n} shift/reduce conflicts and no reduce/reduce conflicts. Bison reports an error if the number of shift/reduce conflicts differs from @var{n}, or if there are any reduce/reduce conflicts. -For normal @acronym{LALR}(1) parsers, reduce/reduce conflicts are more +For deterministic parsers, reduce/reduce conflicts are more serious, and should be eliminated entirely. Bison will always report reduce/reduce conflicts for these parsers. With @acronym{GLR} parsers, however, both kinds of conflicts are routine; otherwise, @@ -4490,8 +4615,8 @@ number which Bison printed. With @acronym{GLR} parsers, add an @code{%expect-rr} declaration as well. @end itemize -Now Bison will warn you if you introduce an unexpected conflict, but -will keep silent otherwise. +Now Bison will report an error if you introduce an unexpected conflict, +but will keep silent otherwise. @node Start Decl @subsection The Start-Symbol @@ -4553,7 +4678,7 @@ valid grammar. @subsection A Push Parser @cindex push parser @cindex push parser -@findex %define api.push_pull +@findex %define api.push-pull (The current push parsing interface is experimental and may evolve. More user feedback will help to stabilize it.) @@ -4569,10 +4694,10 @@ within a certain time period. Normally, Bison generates a pull parser. The following Bison declaration says that you want the parser to be a push -parser (@pxref{Decl Summary,,%define api.push_pull}): +parser (@pxref{Decl Summary,,%define api.push-pull}): @example -%define api.push_pull "push" +%define api.push-pull push @end example In almost all cases, you want to ensure that your push parser is also @@ -4583,7 +4708,7 @@ what you are doing, your declarations should look like this: @example %define api.pure -%define api.push_pull "push" +%define api.push-pull push @end example There is a major notable functional difference between the pure push parser @@ -4632,14 +4757,14 @@ for use by the next invocation of the @code{yypush_parse} function. Bison also supports both the push parser interface along with the pull parser interface in the same generated parser. In order to get this functionality, -you should replace the @code{%define api.push_pull "push"} declaration with the -@code{%define api.push_pull "both"} declaration. Doing this will create all of +you should replace the @code{%define api.push-pull push} declaration with the +@code{%define api.push-pull both} declaration. Doing this will create all of the symbols mentioned earlier along with the two extra symbols, @code{yyparse} and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally would be used. However, the user should note that it is implemented in the generated parser by calling @code{yypull_parse}. This makes the @code{yyparse} function that is generated with the -@code{%define api.push_pull "both"} declaration slower than the normal +@code{%define api.push-pull both} declaration slower than the normal @code{yyparse} function. If the user calls the @code{yypull_parse} function it will parse the rest of the input stream. It is possible to @code{yypush_parse} tokens to select a subgrammar @@ -4656,8 +4781,8 @@ yypstate_delete (ps); @end example Adding the @code{%define api.pure} declaration does exactly the same thing to -the generated parser with @code{%define api.push_pull "both"} as it did for -@code{%define api.push_pull "push"}. +the generated parser with @code{%define api.push-pull both} as it did for +@code{%define api.push-pull push}. @node Decl Summary @subsection Bison Declaration Summary @@ -4737,10 +4862,6 @@ Thus, @code{%code} replaces the traditional Yacc prologue, For a detailed discussion, see @ref{Prologue Alternatives}. For Java, the default location is inside the parser class. - -(Like all the Yacc prologue alternatives, this directive is experimental. -More user feedback will help to determine whether it should become a permanent -feature.) @end deffn @deffn {Directive} %code @var{qualifier} @{@var{code}@} @@ -4751,7 +4872,9 @@ use this form instead. @var{qualifier} identifies the purpose of @var{code} and thus the location(s) where Bison should generate it. -Not all values of @var{qualifier} are available for all target languages: +Not all @var{qualifier}s are accepted for all target languages. +Unaccepted @var{qualifier}s produce an error. +Some of the accepted @var{qualifier}s are: @itemize @bullet @item requires @@ -4818,10 +4941,6 @@ before any class definitions. @end itemize @end itemize -(Like all the Yacc prologue alternatives, this directive is experimental. -More user feedback will help to determine whether it should become a permanent -feature.) - @cindex Prologue For a detailed discussion of how to use @code{%code} in place of the traditional Yacc prologue for C/C++, see @ref{Prologue Alternatives}. @@ -4830,38 +4949,45 @@ traditional Yacc prologue for C/C++, see @ref{Prologue Alternatives}. @deffn {Directive} %debug In the parser file, define the macro @code{YYDEBUG} to 1 if it is not already defined, so that the debugging facilities are compiled. -@end deffn @xref{Tracing, ,Tracing Your Parser}. +@end deffn @deffn {Directive} %define @var{variable} +@deffnx {Directive} %define @var{variable} @var{value} @deffnx {Directive} %define @var{variable} "@var{value}" Define a variable to adjust Bison's behavior. -The possible choices for @var{variable}, as well as their meanings, depend on -the selected target language and/or the parser skeleton (@pxref{Decl -Summary,,%language}). -Bison will warn if a @var{variable} is defined multiple times. +It is an error if a @var{variable} is defined by @code{%define} multiple +times, but see @ref{Bison Options,,-D @var{name}[=@var{value}]}. -Omitting @code{"@var{value}"} is always equivalent to specifying it as +@var{value} must be placed in quotation marks if it contains any +character other than a letter, underscore, period, dash, or non-initial +digit. + +Omitting @code{"@var{value}"} entirely is always equivalent to specifying @code{""}. -Some @var{variable}s may be used as Booleans. +Some @var{variable}s take Boolean values. In this case, Bison will complain if the variable definition does not meet one of the following four conditions: @enumerate -@item @code{"@var{value}"} is @code{"true"} +@item @code{@var{value}} is @code{true} -@item @code{"@var{value}"} is omitted (or is @code{""}). -This is equivalent to @code{"true"}. +@item @code{@var{value}} is omitted (or @code{""} is specified). +This is equivalent to @code{true}. -@item @code{"@var{value}"} is @code{"false"}. +@item @code{@var{value}} is @code{false}. @item @var{variable} is never defined. -In this case, Bison selects a default value, which may depend on the selected -target language and/or parser skeleton. +In this case, Bison selects a default value. @end enumerate +What @var{variable}s are accepted, as well as their meanings and default +values, depend on the selected target language and/or the parser +skeleton (@pxref{Decl Summary,,%language}, @pxref{Decl +Summary,,%skeleton}). +Unaccepted @var{variable}s produce an error. Some of the accepted @var{variable}s are: @itemize @bullet @@ -4876,33 +5002,106 @@ Some of the accepted @var{variable}s are: @item Accepted Values: Boolean -@item Default Value: @code{"false"} +@item Default Value: @code{false} @end itemize -@item api.push_pull -@findex %define api.push_pull +@item api.push-pull +@findex %define api.push-pull @itemize @bullet -@item Language(s): C (LALR(1) only) +@item Language(s): C (deterministic parsers only) -@item Purpose: Requests a pull parser, a push parser, or both. +@item Purpose: Request a pull parser, a push parser, or both. @xref{Push Decl, ,A Push Parser}. (The current push parsing interface is experimental and may evolve. More user feedback will help to stabilize it.) -@item Accepted Values: @code{"pull"}, @code{"push"}, @code{"both"} +@item Accepted Values: @code{pull}, @code{push}, @code{both} + +@item Default Value: @code{pull} +@end itemize + +@c ================================================== lr.default-reductions + +@item lr.default-reductions +@cindex default reductions +@findex %define lr.default-reductions +@cindex delayed syntax errors +@cindex syntax errors delayed +@cindex @acronym{LAC} +@findex %nonassoc + +@itemize @bullet +@item Language(s): all + +@item Purpose: Specify the kind of states that are permitted to +contain default reductions. +That is, in such a state, Bison selects the reduction with the largest +lookahead set to be the default parser action and then removes that +lookahead set. +(The ability to specify where default reductions should be used is +experimental. +More user feedback will help to stabilize it.) + +@item Accepted Values: +@itemize +@item @code{all}. +This is the traditional Bison behavior. +The main advantage is a significant decrease in the size of the parser +tables. +The disadvantage is that, when the generated parser encounters a +syntactically unacceptable token, the parser might then perform +unnecessary default reductions before it can detect the syntax error. +Such delayed syntax error detection is usually inherent in +@acronym{LALR} and @acronym{IELR} parser tables anyway due to +@acronym{LR} state merging (@pxref{Decl Summary,,lr.type}). +Furthermore, the use of @code{%nonassoc} can contribute to delayed +syntax error detection even in the case of canonical @acronym{LR}. +As an experimental feature, delayed syntax error detection can be +overcome in all cases by enabling @acronym{LAC} (@pxref{Decl +Summary,,parse.lac}, for details, including a discussion of the effects +of delayed syntax error detection). + +@item @code{consistent}. +@cindex consistent states +A consistent state is a state that has only one possible action. +If that action is a reduction, then the parser does not need to request +a lookahead token from the scanner before performing that action. +However, the parser recognizes the ability to ignore the lookahead token +in this way only when such a reduction is encoded as a default +reduction. +Thus, if default reductions are permitted only in consistent states, +then a canonical @acronym{LR} parser that does not employ +@code{%nonassoc} detects a syntax error as soon as it @emph{needs} the +syntactically unacceptable token from the scanner. + +@item @code{accepting}. +@cindex accepting state +In the accepting state, the default reduction is actually the accept +action. +In this case, a canonical @acronym{LR} parser that does not employ +@code{%nonassoc} detects a syntax error as soon as it @emph{reaches} the +syntactically unacceptable token in the input. +That is, it does not perform any extra reductions. +@end itemize -@item Default Value: @code{"pull"} +@item Default Value: +@itemize +@item @code{accepting} if @code{lr.type} is @code{canonical-lr}. +@item @code{all} otherwise. +@end itemize @end itemize -@item lr.keep_unreachable_states -@findex %define lr.keep_unreachable_states +@c ============================================ lr.keep-unreachable-states + +@item lr.keep-unreachable-states +@findex %define lr.keep-unreachable-states @itemize @bullet @item Language(s): all -@item Purpose: Requests that Bison allow unreachable parser states to remain in -the parser tables. +@item Purpose: Request that Bison allow unreachable parser states to +remain in the parser tables. Bison considers a state to be unreachable if there exists no sequence of transitions from the start state to that state. A state can become unreachable during conflict resolution if Bison disables a @@ -4912,7 +5111,7 @@ are useless in the generated parser. @item Accepted Values: Boolean -@item Default Value: @code{"false"} +@item Default Value: @code{false} @item Caveats: @@ -4938,13 +5137,103 @@ However, Bison does not compute which goto actions are useless. @end itemize @end itemize +@c ================================================== lr.type + +@item lr.type +@findex %define lr.type +@cindex @acronym{LALR} +@cindex @acronym{IELR} +@cindex @acronym{LR} + +@itemize @bullet +@item Language(s): all + +@item Purpose: Specify the type of parser tables within the +@acronym{LR}(1) family. +(This feature is experimental. +More user feedback will help to stabilize it.) + +@item Accepted Values: +@itemize +@item @code{lalr}. +While Bison generates @acronym{LALR} parser tables by default for +historical reasons, @acronym{IELR} or canonical @acronym{LR} is almost +always preferable for deterministic parsers. +The trouble is that @acronym{LALR} parser tables can suffer from +mysterious conflicts and thus may not accept the full set of sentences +that @acronym{IELR} and canonical @acronym{LR} accept. +@xref{Mystery Conflicts}, for details. +However, there are at least two scenarios where @acronym{LALR} may be +worthwhile: +@itemize +@cindex @acronym{GLR} with @acronym{LALR} +@item When employing @acronym{GLR} parsers (@pxref{GLR Parsers}), if you +do not resolve any conflicts statically (for example, with @code{%left} +or @code{%prec}), then the parser explores all potential parses of any +given input. +In this case, the use of @acronym{LALR} parser tables is guaranteed not +to alter the language accepted by the parser. +@acronym{LALR} parser tables are the smallest parser tables Bison can +currently generate, so they may be preferable. +Nevertheless, once you begin to resolve conflicts statically, +@acronym{GLR} begins to behave more like a deterministic parser, and so +@acronym{IELR} and canonical @acronym{LR} can be helpful to avoid +@acronym{LALR}'s mysterious behavior. + +@item Occasionally during development, an especially malformed grammar +with a major recurring flaw may severely impede the @acronym{IELR} or +canonical @acronym{LR} parser table generation algorithm. +@acronym{LALR} can be a quick way to generate parser tables in order to +investigate such problems while ignoring the more subtle differences +from @acronym{IELR} and canonical @acronym{LR}. +@end itemize + +@item @code{ielr}. +@acronym{IELR} is a minimal @acronym{LR} algorithm. +That is, given any grammar (@acronym{LR} or non-@acronym{LR}), +@acronym{IELR} and canonical @acronym{LR} always accept exactly the same +set of sentences. +However, as for @acronym{LALR}, the number of parser states is often an +order of magnitude less for @acronym{IELR} than for canonical +@acronym{LR}. +More importantly, because canonical @acronym{LR}'s extra parser states +may contain duplicate conflicts in the case of non-@acronym{LR} +grammars, the number of conflicts for @acronym{IELR} is often an order +of magnitude less as well. +This can significantly reduce the complexity of developing of a grammar. + +@item @code{canonical-lr}. +@cindex delayed syntax errors +@cindex syntax errors delayed +@cindex @acronym{LAC} +@findex %nonassoc +While inefficient, canonical @acronym{LR} parser tables can be an +interesting means to explore a grammar because they have a property that +@acronym{IELR} and @acronym{LALR} tables do not. +That is, if @code{%nonassoc} is not used and default reductions are left +disabled (@pxref{Decl Summary,,lr.default-reductions}), then, for every +left context of every canonical @acronym{LR} state, the set of tokens +accepted by that state is guaranteed to be the exact set of tokens that +is syntactically acceptable in that left context. +It might then seem that an advantage of canonical @acronym{LR} parsers +in production is that, under the above constraints, they are guaranteed +to detect a syntax error as soon as possible without performing any +unnecessary reductions. +However, @acronym{IELR} parsers using @acronym{LAC} (@pxref{Decl +Summary,,parse.lac}) are also able to achieve this behavior without +sacrificing @code{%nonassoc} or default reductions. +@end itemize + +@item Default Value: @code{lalr} +@end itemize + @item namespace @findex %define namespace @itemize @item Languages(s): C++ -@item Purpose: Specifies the namespace for the parser class. +@item Purpose: Specify the namespace for the parser class. For example, if you specify: @smallexample @@ -4989,6 +5278,89 @@ For example, if you specify: The parser namespace is @code{foo} and @code{yylex} is referenced as @code{bar::lex}. @end itemize + +@c ================================================== parse.lac +@item parse.lac +@findex %define parse.lac +@cindex @acronym{LAC} +@cindex lookahead correction + +@itemize +@item Languages(s): C + +@item Purpose: Enable @acronym{LAC} (lookahead correction) to improve +syntax error handling. + +Canonical @acronym{LR}, @acronym{IELR}, and @acronym{LALR} can suffer +from a couple of problems upon encountering a syntax error. First, the +parser might perform additional parser stack reductions before +discovering the syntax error. Such reductions perform user semantic +actions that are unexpected because they are based on an invalid token, +and they cause error recovery to begin in a different syntactic context +than the one in which the invalid token was encountered. Second, when +verbose error messages are enabled (with @code{%error-verbose} or +@code{#define YYERROR_VERBOSE}), the expected token list in the syntax +error message can both contain invalid tokens and omit valid tokens. + +The culprits for the above problems are @code{%nonassoc}, default +reductions in inconsistent states, and parser state merging. Thus, +@acronym{IELR} and @acronym{LALR} suffer the most. Canonical +@acronym{LR} can suffer only if @code{%nonassoc} is used or if default +reductions are enabled for inconsistent states. + +@acronym{LAC} is a new mechanism within the parsing algorithm that +completely solves these problems for canonical @acronym{LR}, +@acronym{IELR}, and @acronym{LALR} without sacrificing @code{%nonassoc}, +default reductions, or state mering. Conceptually, the mechanism is +straight-forward. Whenever the parser fetches a new token from the +scanner so that it can determine the next parser action, it immediately +suspends normal parsing and performs an exploratory parse using a +temporary copy of the normal parser state stack. During this +exploratory parse, the parser does not perform user semantic actions. +If the exploratory parse reaches a shift action, normal parsing then +resumes on the normal parser stacks. If the exploratory parse reaches +an error instead, the parser reports a syntax error. If verbose syntax +error messages are enabled, the parser must then discover the list of +expected tokens, so it performs a separate exploratory parse for each +token in the grammar. + +There is one subtlety about the use of @acronym{LAC}. That is, when in +a consistent parser state with a default reduction, the parser will not +attempt to fetch a token from the scanner because no lookahead is needed +to determine the next parser action. Thus, whether default reductions +are enabled in consistent states (@pxref{Decl +Summary,,lr.default-reductions}) affects how soon the parser detects a +syntax error: when it @emph{reaches} an erroneous token or when it +eventually @emph{needs} that token as a lookahead. The latter behavior +is probably more intuitive, so Bison currently provides no way to +achieve the former behavior while default reductions are fully enabled. + +Thus, when @acronym{LAC} is in use, for some fixed decision of whether +to enable default reductions in consistent states, canonical +@acronym{LR} and @acronym{IELR} behave exactly the same for both +syntactically acceptable and syntactically unacceptable input. While +@acronym{LALR} still does not support the full language-recognition +power of canonical @acronym{LR} and @acronym{IELR}, @acronym{LAC} at +least enables @acronym{LALR}'s syntax error handling to correctly +reflect @acronym{LALR}'s language-recognition power. + +Because @acronym{LAC} requires many parse actions to be performed twice, +it can have a performance penalty. However, not all parse actions must +be performed twice. Specifically, during a series of default reductions +in consistent states and shift actions, the parser never has to initiate +an exploratory parse. Moreover, the most time-consuming tasks in a +parse are often the file I/O, the lexical analysis performed by the +scanner, and the user's semantic actions, but none of these are +performed during the exploratory parse. Finally, the base of the +temporary stack used during an exploratory parse is a pointer into the +normal parser state stack so that the stack is never physically copied. +In our experience, the performance penalty of @acronym{LAC} has proven +insignificant for practical grammars. + +@item Accepted Values: @code{none}, @code{full} + +@item Default Value: @code{none} +@end itemize @end itemize @end deffn @@ -5051,6 +5423,9 @@ chosen as if the input file were named @file{@var{prefix}.y}. Specify the programming language for the generated parser. Currently supported languages include C, C++, and Java. @var{language} is case-insensitive. + +This directive is experimental and its effect may be modified in future +releases. @end deffn @deffn {Directive} %locations @@ -5111,10 +5486,10 @@ Require a Version of Bison}. @deffn {Directive} %skeleton "@var{file}" Specify the skeleton to use. -You probably don't need this option unless you are developing Bison. -You should use @code{%language} if you want to specify the skeleton for a -different language, because it is clearer and because it will always choose the -correct skeleton for non-deterministic or push parsers. +@c You probably don't need this option unless you are developing Bison. +@c You should use @code{%language} if you want to specify the skeleton for a +@c different language, because it is clearer and because it will always choose the +@c correct skeleton for non-deterministic or push parsers. If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton file in the Bison installation directory. @@ -5218,19 +5593,17 @@ identifier (aside from those in this manual) in an action or in epilogue in the grammar file, you are likely to run into trouble. @menu -* Parser Function:: How to call @code{yyparse} and what it returns. -* Push Parser Function:: How to call @code{yypush_parse} and what it returns. -* Pull Parser Function:: How to call @code{yypull_parse} and what it returns. -* Parser Create Function:: How to call @code{yypstate_new} and what it - returns. -* Parser Delete Function:: How to call @code{yypstate_delete} and what it - returns. -* Lexical:: You must supply a function @code{yylex} - which reads tokens. -* Error Reporting:: You must supply a function @code{yyerror}. -* Action Features:: Special features for use in actions. -* Internationalization:: How to let the parser speak in the user's - native language. +* Parser Function:: How to call @code{yyparse} and what it returns. +* Push Parser Function:: How to call @code{yypush_parse} and what it returns. +* Pull Parser Function:: How to call @code{yypull_parse} and what it returns. +* Parser Create Function:: How to call @code{yypstate_new} and what it returns. +* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns. +* Lexical:: You must supply a function @code{yylex} + which reads tokens. +* Error Reporting:: You must supply a function @code{yyerror}. +* Action Features:: Special features for use in actions. +* Internationalization:: How to let the parser speak in the user's + native language. @end menu @node Parser Function @@ -5315,8 +5688,8 @@ exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @} More user feedback will help to stabilize it.) You call the function @code{yypush_parse} to parse a single token. This -function is available if either the @code{%define api.push_pull "push"} or -@code{%define api.push_pull "both"} declaration is used. +function is available if either the @code{%define api.push-pull push} or +@code{%define api.push-pull both} declaration is used. @xref{Push Decl, ,A Push Parser}. @deftypefun int yypush_parse (yypstate *yyps) @@ -5333,7 +5706,7 @@ is required to finish parsing the grammar. More user feedback will help to stabilize it.) You call the function @code{yypull_parse} to parse the rest of the input -stream. This function is available if the @code{%define api.push_pull "both"} +stream. This function is available if the @code{%define api.push-pull both} declaration is used. @xref{Push Decl, ,A Push Parser}. @@ -5349,12 +5722,12 @@ The value returned by @code{yypull_parse} is the same as for @code{yyparse}. More user feedback will help to stabilize it.) You call the function @code{yypstate_new} to create a new parser instance. -This function is available if either the @code{%define api.push_pull "push"} or -@code{%define api.push_pull "both"} declaration is used. +This function is available if either the @code{%define api.push-pull push} or +@code{%define api.push-pull both} declaration is used. @xref{Push Decl, ,A Push Parser}. @deftypefun yypstate *yypstate_new (void) -The fuction will return a valid parser instance if there was memory available +The function will return a valid parser instance if there was memory available or 0 if no memory was available. In impure mode, it will also return 0 if a parser instance is currently allocated. @@ -5368,8 +5741,8 @@ allocated. More user feedback will help to stabilize it.) You call the function @code{yypstate_delete} to delete a parser instance. -function is available if either the @code{%define api.push_pull "push"} or -@code{%define api.push_pull "both"} declaration is used. +function is available if either the @code{%define api.push-pull push} or +@code{%define api.push-pull both} declaration is used. @xref{Push Decl, ,A Push Parser}. @deftypefun void yypstate_delete (yypstate *yyps) @@ -5397,13 +5770,13 @@ that need it. @xref{Invocation, ,Invoking Bison}. @menu * Calling Convention:: How @code{yyparse} calls @code{yylex}. -* Token Values:: How @code{yylex} must return the semantic value - of the token it has read. -* Token Locations:: How @code{yylex} must return the text location - (line number, etc.) of the token, if the - actions want that. -* Pure Calling:: How the calling convention differs - in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). +* Token Values:: How @code{yylex} must return the semantic value + of the token it has read. +* Token Locations:: How @code{yylex} must return the text location + (line number, etc.) of the token, if the + actions want that. +* Pure Calling:: How the calling convention differs in a pure parser + (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). @end menu @node Calling Convention @@ -6046,7 +6419,7 @@ This kind of parser is known in the literature as a bottom-up parser. * Contextual Precedence:: When an operator's precedence depends on context. * Parser States:: The parser is a finite-state-machine with stack. * Reduce/Reduce:: When two rules are applicable in the same situation. -* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified. +* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified. * Generalized LR Parsing:: Parsing arbitrary context-free grammars. * Memory Management:: What happens when memory is exhausted. How to avoid it. @end menu @@ -6175,8 +6548,10 @@ This particular ambiguity was first encountered in the specifications of Algol 60 and is called the ``dangling @code{else}'' ambiguity. To avoid warnings from Bison about predictable, legitimate shift/reduce -conflicts, use the @code{%expect @var{n}} declaration. There will be no -warning as long as the number of shift/reduce conflicts is exactly @var{n}. +conflicts, use the @code{%expect @var{n}} declaration. +There will be no warning as long as the number of shift/reduce conflicts +is exactly @var{n}, and Bison will report an error if there is a +different number. @xref{Expect Decl, ,Suppressing Conflict Warnings}. The definition of @code{if_stmt} above is solely to blame for the @@ -6605,12 +6980,13 @@ a @code{name} if a comma or colon follows, or a @code{type} if another @cindex @acronym{LR}(1) @cindex @acronym{LALR}(1) -However, Bison, like most parser generators, cannot actually handle all -@acronym{LR}(1) grammars. In this grammar, two contexts, that after -an @code{ID} -at the beginning of a @code{param_spec} and likewise at the beginning of -a @code{return_spec}, are similar enough that Bison assumes they are the -same. They appear similar because the same set of rules would be +However, for historical reasons, Bison cannot by default handle all +@acronym{LR}(1) grammars. +In this grammar, two contexts, that after an @code{ID} at the beginning +of a @code{param_spec} and likewise at the beginning of a +@code{return_spec}, are similar enough that Bison assumes they are the +same. +They appear similar because the same set of rules would be active---the rule for reducing to a @code{name} and that for reducing to a @code{type}. Bison is unable to determine at that stage of processing that the rules would require different lookahead tokens in the two @@ -6618,16 +6994,22 @@ contexts, so it makes a single parser state for them both. Combining the two contexts causes a conflict later. In parser terminology, this occurrence means that the grammar is not @acronym{LALR}(1). -In general, it is better to fix deficiencies than to document them. But -this particular deficiency is intrinsically hard to fix; parser -generators that can handle @acronym{LR}(1) grammars are hard to write -and tend to -produce parsers that are very large. In practice, Bison is more useful -as it is now. - -When the problem arises, you can often fix it by identifying the two -parser states that are being confused, and adding something to make them -look distinct. In the above example, adding one rule to +For many practical grammars (specifically those that fall into the +non-@acronym{LR}(1) class), the limitations of @acronym{LALR}(1) result in +difficulties beyond just mysterious reduce/reduce conflicts. +The best way to fix all these problems is to select a different parser +table generation algorithm. +Either @acronym{IELR}(1) or canonical @acronym{LR}(1) would suffice, but +the former is more efficient and easier to debug during development. +@xref{Decl Summary,,lr.type}, for details. +(Bison's @acronym{IELR}(1) and canonical @acronym{LR}(1) implementations +are experimental. +More user feedback will help to stabilize them.) + +If you instead wish to work around @acronym{LALR}(1)'s limitations, you +can often fix a mysterious conflict by identifying the two parser states +that are being confused, and adding something to make them look +distinct. In the above example, adding one rule to @code{return_spec} as follows makes the problem go away: @example @@ -6695,7 +7077,7 @@ The same is true of languages that require more than one symbol of lookahead, since the parser lacks the information necessary to make a decision at the point it must be made in a shift-reduce parser. Finally, as previously mentioned (@pxref{Mystery Conflicts}), -there are languages where Bison's particular choice of how to +there are languages where Bison's default choice of how to summarize the input seen so far loses necessary information. When you use the @samp{%glr-parser} declaration in your grammar file, @@ -6727,7 +7109,7 @@ grammar symbol that produces the same segment of the input token stream. Whenever the parser makes a transition from having multiple -states to having one, it reverts to the normal @acronym{LALR}(1) parsing +states to having one, it reverts to the normal deterministic parsing algorithm, after resolving and executing the saved-up actions. At this transition, some of the states on the stack will have semantic values that are sets (actually multisets) of possible actions. The @@ -6740,9 +7122,9 @@ Bison resolves and evaluates both and then calls the merge function on the result. Otherwise, it reports an ambiguity. It is possible to use a data structure for the @acronym{GLR} parsing tree that -permits the processing of any @acronym{LALR}(1) grammar in linear time (in the +permits the processing of any @acronym{LR}(1) grammar in linear time (in the size of the input), any unambiguous (not necessarily -@acronym{LALR}(1)) grammar in +@acronym{LR}(1)) grammar in quadratic worst-case time, and any general (possibly ambiguous) context-free grammar in cubic worst-case time. However, Bison currently uses a simpler data structure that requires time proportional to the @@ -6752,9 +7134,9 @@ grammars can require exponential time and space to process. Such badly behaving examples, however, are not generally of practical interest. Usually, nondeterminism in a grammar is local---the parser is ``in doubt'' only for a few tokens at a time. Therefore, the current data -structure should generally be adequate. On @acronym{LALR}(1) portions of a -grammar, in particular, it is only slightly slower than with the default -Bison parser. +structure should generally be adequate. On @acronym{LR}(1) portions of a +grammar, in particular, it is only slightly slower than with the +deterministic @acronym{LR}(1) Bison parser. For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style @@ -6803,16 +7185,16 @@ The default value of @code{YYMAXDEPTH}, if you do not define it, is @vindex YYINITDEPTH You can control how much stack is allocated initially by defining the -macro @code{YYINITDEPTH} to a positive integer. For the C -@acronym{LALR}(1) parser, this value must be a compile-time constant +macro @code{YYINITDEPTH} to a positive integer. For the deterministic +parser in C, this value must be a compile-time constant unless you are assuming C99 or some other target language or compiler that allows variable-length arrays. The default is 200. Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}. @c FIXME: C++ output. -Because of semantical differences between C and C++, the -@acronym{LALR}(1) parsers in C produced by Bison cannot grow when compiled +Because of semantic differences between C and C++, the deterministic +parsers in C produced by Bison cannot grow when compiled by C++ compilers. In this precise case (compiling a C parser as C++) you are suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix this deficiency in a future release. @@ -7200,7 +7582,8 @@ useless: STR; @command{bison} reports: @example -calc.y: warning: 1 nonterminal and 1 rule useless in grammar +calc.y: warning: 1 nonterminal useless in grammar +calc.y: warning: 1 rule useless in grammar calc.y:11.1-7: warning: nonterminal useless in grammar: useless calc.y:11.10-12: warning: rule useless in grammar: useless: STR calc.y: conflicts: 7 shift/reduce @@ -7388,6 +7771,7 @@ control will jump to state 4, corresponding to the item @samp{exp -> exp '+' . exp}. Since there is no default action, any other token than those listed above will trigger a syntax error. +@cindex accepting state The state 3 is named the @dfn{final state}, or the @dfn{accepting state}: @@ -7468,7 +7852,7 @@ sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM / NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) / NUM}, which corresponds to reducing rule 1. -Because in @acronym{LALR}(1) parsing a single decision can be made, Bison +Because in deterministic parsing a single decision can be made, Bison arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, , Shift/Reduce Conflicts}. Discarded actions are reported in between square brackets. @@ -7650,7 +8034,7 @@ standard I/O stream, the numeric code for the token type, and the token value (from @code{yylval}). Here is an example of @code{YYPRINT} suitable for the multi-function -calculator (@pxref{Mfcalc Decl, ,Declarations for @code{mfcalc}}): +calculator (@pxref{Mfcalc Declarations, ,Declarations for @code{mfcalc}}): @smallexample %@{ @@ -7762,7 +8146,7 @@ other minor ways. Most importantly, imitate Yacc's output file name conventions, so that the parser output file is called @file{y.tab.c}, and the other outputs are called @file{y.output} and @file{y.tab.h}. -Also, if generating an @acronym{LALR}(1) parser in C, generate @code{#define} +Also, if generating a deterministic parser in C, generate @code{#define} statements in addition to an @code{enum} to associate token numbers with token names. Thus, the following shell script can substitute for Yacc, and the Bison @@ -7778,8 +8162,8 @@ traditional Yacc grammars. If your grammar uses a Bison extension like @samp{%glr-parser}, Bison might not be Yacc-compatible even if this option is specified. -@item -W -@itemx --warnings +@item -W [@var{category}] +@itemx --warnings[=@var{category}] Output warnings falling in @var{category}. @var{category} can be one of: @table @code @@ -7816,8 +8200,8 @@ Treat warnings as errors. @end table A category can be turned off by prefixing its name with @samp{no-}. For -instance, @option{-Wno-syntax} will hide the warnings about unused -variables. +instance, @option{-Wno-yacc} will hide the warnings about +@acronym{POSIX} Yacc incompatibilities. @end table @noindent @@ -7830,6 +8214,35 @@ In the parser file, define the macro @code{YYDEBUG} to 1 if it is not already defined, so that the debugging facilities are compiled. @xref{Tracing, ,Tracing Your Parser}. +@item -D @var{name}[=@var{value}] +@itemx --define=@var{name}[=@var{value}] +@itemx -F @var{name}[=@var{value}] +@itemx --force-define=@var{name}[=@var{value}] +Each of these is equivalent to @samp{%define @var{name} "@var{value}"} +(@pxref{Decl Summary, ,%define}) except that Bison processes multiple +definitions for the same @var{name} as follows: + +@itemize +@item +Bison quietly ignores all command-line definitions for @var{name} except +the last. +@item +If that command-line definition is specified by a @code{-D} or +@code{--define}, Bison reports an error for any @code{%define} +definition for @var{name}. +@item +If that command-line definition is specified by a @code{-F} or +@code{--force-define} instead, Bison quietly ignores all @code{%define} +definitions for @var{name}. +@item +Otherwise, Bison reports an error if there are multiple @code{%define} +definitions for @var{name}. +@end itemize + +You should avoid using @code{-F} and @code{--force-define} in your +makefiles unless you are confident that it is safe to quietly ignore any +conflicting @code{%define} that may be added to the grammar file. + @item -L @var{language} @itemx --language=@var{language} Specify the programming language for the generated parser, as if @@ -7837,6 +8250,9 @@ Specify the programming language for the generated parser, as if Summary}). Currently supported languages include C, C++, and Java. @var{language} is case-insensitive. +This option is experimental and its effect may be modified in future +releases. + @item --locations Pretend that @code{%locations} was specified. @xref{Decl Summary}. @@ -7858,10 +8274,10 @@ parser file, treating it as an independent source file in its own right. Specify the skeleton to use, similar to @code{%skeleton} (@pxref{Decl Summary, , Bison Declaration Summary}). -You probably don't need this option unless you are developing Bison. -You should use @option{--language} if you want to specify the skeleton for a -different language, because it is clearer and because it will always -choose the correct skeleton for non-deterministic or push parsers. +@c You probably don't need this option unless you are developing Bison. +@c You should use @option{--language} if you want to specify the skeleton for a +@c different language, because it is clearer and because it will always +@c choose the correct skeleton for non-deterministic or push parsers. If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton file in the Bison installation directory. @@ -7901,7 +8317,7 @@ separated list of @var{things} among: @table @code @item state Description of the grammar, conflicts (resolved and unresolved), and -@acronym{LALR} automaton. +parser's automaton. @item lookahead Implies @code{state} and augments the description of the automaton with @@ -7928,18 +8344,18 @@ Specify the @var{file} for the parser file. The other output files' names are constructed from @var{file} as described under the @samp{-v} and @samp{-d} options. -@item -g[@var{file}] +@item -g [@var{file}] @itemx --graph[=@var{file}] -Output a graphical representation of the @acronym{LALR}(1) grammar +Output a graphical representation of the parser's automaton computed by Bison, in @uref{http://www.graphviz.org/, Graphviz} @uref{http://www.graphviz.org/doc/info/lang.html, @acronym{DOT}} format. @code{@var{file}} is optional. If omitted and the grammar file is @file{foo.y}, the output file will be @file{foo.dot}. -@item -x[@var{file}] +@item -x [@var{file}] @itemx --xml[=@var{file}] -Output an XML report of the @acronym{LALR}(1) automaton computed by Bison. +Output an XML report of the parser's automaton computed by Bison. @code{@var{file}} is optional. If omitted and the grammar file is @file{foo.y}, the output file will be @file{foo.xml}. @@ -7950,12 +8366,11 @@ More user feedback will help to stabilize it.) @node Option Cross Key @section Option Cross Key -@c FIXME: How about putting the directives too? Here is a list of options, alphabetized by long option, to help you find -the corresponding short option. +the corresponding short option and directive. -@multitable {@option{--defines=@var{defines-file}}} {@option{-b @var{file-prefix}XXX}} -@headitem Long Option @tab Short Option +@multitable {@option{--force-define=@var{name}[=@var{value}]}} {@option{-F @var{name}[=@var{value}]}} {@code{%nondeterministic-parser}} +@headitem Long Option @tab Short Option @tab Bison Directive @include cross-options.texi @end multitable @@ -8009,13 +8424,13 @@ int yyparse (void); @node C++ Bison Interface @subsection C++ Bison Interface -@c - %language "C++" +@c - %skeleton "lalr1.cc" @c - Always pure @c - initial action -The C++ @acronym{LALR}(1) parser is selected using the language directive, -@samp{%language "C++"}, or the synonymous command-line option -@option{--language=c++}. +The C++ deterministic parser is selected using the skeleton directive, +@samp{%skeleton "lalr1.cc"}, or the synonymous command-line option +@option{--skeleton=lalr1.cc}. @xref{Decl Summary}. When run, @command{bison} will create several entities in the @samp{yy} @@ -8164,11 +8579,19 @@ this class is detailed below. It can be extended using the it describes an additional member of the parser class, and an additional argument for its constructor. -@defcv {Type} {parser} {semantic_value_type} -@defcvx {Type} {parser} {location_value_type} +@defcv {Type} {parser} {semantic_type} +@defcvx {Type} {parser} {location_type} The types for semantics value and locations. @end defcv +@defcv {Type} {parser} {token} +A structure that contains (only) the definition of the tokens as the +@code{yytokentype} enumeration. To refer to the token @code{FOO}, the +scanner should use @code{yy::parser::token::FOO}. The scanner can use +@samp{typedef yy::parser::token token;} to ``import'' the token enumeration +(@pxref{Calc++ Scanner}). +@end defcv + @deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...) Build a new parser object. There are no arguments by default, unless @samp{%parse-param @{@var{type1} @var{arg1}@}} was used. @@ -8207,7 +8630,7 @@ The parser invokes the scanner by calling @code{yylex}. Contrary to C parsers, C++ parsers are always pure: there is no point in using the @code{%define api.pure} directive. Therefore the interface is as follows. -@deftypemethod {parser} {int} yylex (semantic_value_type& @var{yylval}, location_type& @var{yylloc}, @var{type1} @var{arg1}, ...) +@deftypemethod {parser} {int} yylex (semantic_type* @var{yylval}, location_type* @var{yylloc}, @var{type1} @var{arg1}, ...) Return the next token. Its type is the return value, its semantic value and location being @var{yylval} and @var{yylloc}. Invocations of @samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments. @@ -8402,14 +8825,14 @@ calcxx_driver::error (const std::string& m) @subsubsection Calc++ Parser The parser definition file @file{calc++-parser.yy} starts by asking for -the C++ LALR(1) skeleton, the creation of the parser header file, and -specifies the name of the parser class. Because the C++ skeleton -changed several times, it is safer to require the version you designed -the grammar for. +the C++ deterministic parser skeleton, the creation of the parser header +file, and specifies the name of the parser class. +Because the C++ skeleton changed several times, it is safer to require +the version you designed the grammar for. @comment file: calc++-parser.yy @example -%language "C++" /* -*- C++ -*- */ +%skeleton "lalr1.cc" /* -*- C++ -*- */ %require "@value{VERSION}" %defines %define parser_class_name "calcxx_parser" @@ -8447,7 +8870,7 @@ global variables. @noindent Then we request the location tracking feature, and initialize the -first location's file name. Afterwards new locations are computed +first location's file name. Afterward new locations are computed relatively to the previous locations: the file name will be automatically propagated. @@ -8578,8 +9001,8 @@ parser's to get the set of defined tokens. @example %@{ /* -*- C++ -*- */ # include -# include -# include +# include +# include # include # include "calc++-driver.hh" # include "calc++-parser.hh" @@ -8722,14 +9145,14 @@ main (int argc, char *argv[]) @section Java Parsers @menu -* Java Bison Interface:: Asking for Java parser generation -* Java Semantic Values:: %type and %token vs. Java -* Java Location Values:: The position and location classes -* Java Parser Interface:: Instantiating and running the parser -* Java Scanner Interface:: Specifying the scanner for the parser -* Java Action Features:: Special features for use in actions. -* Java Differences:: Differences between C/C++ and Java Grammars -* Java Declarations Summary:: List of Bison declarations used with Java +* Java Bison Interface:: Asking for Java parser generation +* Java Semantic Values:: %type and %token vs. Java +* Java Location Values:: The position and location classes +* Java Parser Interface:: Instantiating and running the parser +* Java Scanner Interface:: Specifying the scanner for the parser +* Java Action Features:: Special features for use in actions +* Java Differences:: Differences between C/C++ and Java Grammars +* Java Declarations Summary:: List of Bison declarations used with Java @end menu @node Java Bison Interface @@ -8761,7 +9184,7 @@ and @code{%define api.pure} directives does not do anything when used in Java. Push parsers are currently unsupported in Java and @code{%define -api.push_pull} have no effect. +api.push-pull} have no effect. @acronym{GLR} parsers are currently unsupported in Java. Do not use the @code{glr-parser} directive. @@ -8837,7 +9260,7 @@ in a file; Bison itself defines a class representing a @dfn{location}, a range composed of a pair of positions (possibly spanning several files). The location class is an inner class of the parser; the name is @code{Location} by default, and may also be renamed using -@code{%define location_type "@var{class-name}}. +@code{%define location_type "@var{class-name}"}. The location class treats the position as a completely opaque value. By default, the class name is @code{Position}, but this can be changed @@ -8851,7 +9274,7 @@ The first, inclusive, position of the range, and the first beyond. @end deftypeivar @deftypeop {Constructor} {Location} {} Location (Position @var{loc}) -Create a @code{Location} denoting an empty range located at a given point. +Create a @code{Location} denoting an empty range located at a given point. @end deftypeop @deftypeop {Constructor} {Location} {} Location (Position @var{begin}, Position @var{end}) @@ -8981,7 +9404,7 @@ changed using @code{%define location_type "@var{class-name}".} @deftypemethod {Lexer} {int} yylex () Return the next token. Its type is the return value, its semantic -value and location are saved and returned by the ther methods in the +value and location are saved and returned by the their methods in the interface. Use @code{%define lex_throws} to specify any uncaught exceptions. @@ -8999,7 +9422,7 @@ The return type can be changed using @code{%define position_type @end deftypemethod @deftypemethod {Lexer} {Object} getLVal () -Return the semantical value of the last token that yylex returned. +Return the semantic value of the last token that yylex returned. The return type can be changed using @code{%define stype "@var{class-name}".} @@ -9065,12 +9488,7 @@ Return immediately from the parser, indicating success. @end deffn @deffn {Statement} {return YYERROR;} -Start error recovery without printing an error message. -@xref{Error Recovery}. -@end deffn - -@deffn {Statement} {return YYFAIL;} -Print an error message and start error recovery. +Start error recovery without printing an error message. @xref{Error Recovery}. @end deffn @@ -9115,7 +9533,7 @@ corresponds to these C macros.}. @item Java lacks unions, so @code{%union} has no effect. Instead, semantic values have a common base type: @code{Object} or as specified by -@code{%define stype}. Angle backets on @code{%token}, @code{type}, +@samp{%define stype}. Angle brackets on @code{%token}, @code{type}, @code{$@var{n}} and @code{$$} specify subtypes rather than fields of an union. The type of @code{$$}, even with angle brackets, is the base type since Java casts are not allow on the left-hand side of assignments. @@ -9124,7 +9542,7 @@ left-hand side of assignments. See @pxref{Java Semantic Values} and @pxref{Java Action Features}. @item -The prolog declarations have a different meaning than in C/C++ code. +The prologue declarations have a different meaning than in C/C++ code. @table @asis @item @code{%code imports} blocks are placed at the beginning of the Java source code. They may @@ -9690,6 +10108,16 @@ In an action, the location of the @var{n}-th symbol of the right-hand side of the rule. @xref{Locations, , Locations Overview}. @end deffn +@deffn {Variable} @@@var{name} +In an action, the location of a symbol addressed by name. +@xref{Locations, , Locations Overview}. +@end deffn + +@deffn {Variable} @@[@var{name}] +In an action, the location of a symbol addressed by name. +@xref{Locations, , Locations Overview}. +@end deffn + @deffn {Variable} $$ In an action, the semantic value of the left-hand side of the rule. @xref{Actions}. @@ -9700,6 +10128,16 @@ In an action, the semantic value of the @var{n}-th symbol of the right-hand side of the rule. @xref{Actions}. @end deffn +@deffn {Variable} $@var{name} +In an action, the semantic value of a symbol addressed by name. +@xref{Actions}. +@end deffn + +@deffn {Variable} $[@var{name}] +In an action, the semantic value of a symbol addressed by name. +@xref{Actions}. +@end deffn + @deffn {Delimiter} %% Delimiter used to separate the grammar rule section from the Bison declarations section or the epilogue. @@ -9770,10 +10208,6 @@ Insert @var{code} verbatim into output parser source. Equip the parser for debugging. @xref{Decl Summary}. @end deffn -@deffn {Directive} %debug -Equip the parser for debugging. @xref{Decl Summary}. -@end deffn - @ifset defaultprec @deffn {Directive} %default-prec Assign a precedence to rules that lack an explicit @samp{%prec} @@ -9784,6 +10218,7 @@ Precedence}. @deffn {Directive} %define @var{define-variable} @deffnx {Directive} %define @var{define-variable} @var{value} +@deffnx {Directive} %define @var{define-variable} "@var{value}" Define a variable to adjust Bison's behavior. @xref{Decl Summary,,%define}. @end deffn @@ -10143,8 +10578,8 @@ is recovering from a syntax error, and 0 otherwise. @end deffn @deffn {Macro} YYSTACK_USE_ALLOCA -Macro used to control the use of @code{alloca} when the C -@acronym{LALR}(1) parser needs to extend its stacks. If defined to 0, +Macro used to control the use of @code{alloca} when the +deterministic parser in C needs to extend its stacks. If defined to 0, the parser will use @code{malloc} to extend its stacks. If defined to 1, the parser will use @code{alloca}. Values other than 0 and 1 are reserved for future Bison extensions. If not defined, @@ -10169,12 +10604,21 @@ Data type of semantic values; @code{int} by default. @cindex glossary @table @asis +@item Accepting State +A state whose only action is the accept action. +The accepting state is thus a consistent state. +@xref{Understanding,,}. + @item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'') Formal method of specifying context-free grammars originally proposed by John Backus, and slightly improved by Peter Naur in his 1960-01-02 committee document contributing to what became the Algol 60 report. @xref{Language and Grammar, ,Languages and Context-Free Grammars}. +@item Consistent State +A state containing only one possible action. +@xref{Decl Summary,,lr.default-reductions}. + @item Context-free grammars Grammars specified as rules that can be applied regardless of context. Thus, if there is a rule which says that an integer can be used as an @@ -10182,6 +10626,14 @@ expression, integers are allowed @emph{anywhere} an expression is permitted. @xref{Language and Grammar, ,Languages and Context-Free Grammars}. +@item Default Reduction +The reduction that a parser should perform if the current parser state +contains no other action for the lookahead token. +In permitted parser states, Bison declares the reduction with the +largest lookahead set to be the default reduction and removes that +lookahead set. +@xref{Decl Summary,,lr.default-reductions}. + @item Dynamic allocation Allocation of memory that occurs during execution, rather than at compile time or on entry to a function. @@ -10200,8 +10652,8 @@ rules. @xref{Algorithm, ,The Bison Parser Algorithm}. @item Generalized @acronym{LR} (@acronym{GLR}) A parsing algorithm that can handle all context-free grammars, including those -that are not @acronym{LALR}(1). It resolves situations that Bison's -usual @acronym{LALR}(1) +that are not @acronym{LR}(1). It resolves situations that Bison's +deterministic parsing algorithm cannot by effectively splitting off multiple parsers, trying all possible parsers, and discarding those that fail in the light of additional right context. @xref{Generalized LR Parsing, ,Generalized @@ -10212,6 +10664,20 @@ A language construct that is (in general) grammatically divisible; for example, `expression' or `declaration' in C@. @xref{Language and Grammar, ,Languages and Context-Free Grammars}. +@item @acronym{IELR}(1) +A minimal @acronym{LR}(1) parser table generation algorithm. +That is, given any context-free grammar, @acronym{IELR}(1) generates +parser tables with the full language recognition power of canonical +@acronym{LR}(1) but with nearly the same number of parser states as +@acronym{LALR}(1). +This reduction in parser states is often an order of magnitude. +More importantly, because canonical @acronym{LR}(1)'s extra parser +states may contain duplicate conflicts in the case of +non-@acronym{LR}(1) grammars, the number of conflicts for +@acronym{IELR}(1) is often an order of magnitude less as well. +This can significantly reduce the complexity of developing of a grammar. +@xref{Decl Summary,,lr.type}. + @item Infix operator An arithmetic operator that is placed between the operands on which it performs some operation. @@ -10219,6 +10685,14 @@ performs some operation. @item Input stream A continuous flow of data between devices or programs. +@item @acronym{LAC} (Lookahead Correction) +A parsing mechanism that fixes the problem of delayed syntax error +detection, which is caused by LR state merging, default reductions, and +the use of @code{%nonassoc}. Delayed syntax error detection results in +unexpected semantic actions, initiation of error recovery in the wrong +syntactic context, and an incorrect list of expected tokens in a verbose +syntax error message. @xref{Decl Summary,,parse.lac}. + @item Language construct One of the typical usage schemas of the language. For example, one of the constructs of the C language is the @code{if} statement. @@ -10255,8 +10729,8 @@ Tokens}. @item @acronym{LALR}(1) The class of context-free grammars that Bison (like most other parser -generators) can handle; a subset of @acronym{LR}(1). @xref{Mystery -Conflicts, ,Mysterious Reduce/Reduce Conflicts}. +generators) can handle by default; a subset of @acronym{LR}(1). +@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}. @item @acronym{LR}(1) The class of context-free grammars in which at most one token of @@ -10346,32 +10820,59 @@ grammatically indivisible. The piece of text it represents is a token. @bye -@c LocalWords: texinfo setfilename settitle setchapternewpage finalout -@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex -@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry -@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa -@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc -@c LocalWords: rpcalc Lexer Gen Comp Expr ltcalc mfcalc Decl Symtab yylex -@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref -@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex -@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge -@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG -@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit -@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok -@c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln -@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym -@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof -@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum -@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype -@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs -@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES -@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param -@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP -@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword -@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH -@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype -@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args -@c LocalWords: infile ypp yxx outfile itemx tex leaderfill -@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll -@c LocalWords: nbar yytext fst snd osplit ntwo strdup AST -@c LocalWords: YYSTACK DVI fdl printindex +@c Local Variables: +@c fill-column: 76 +@c End: + +@c LocalWords: texinfo setfilename settitle setchapternewpage finalout texi FSF +@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex FSF's +@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry Naur +@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa Multi +@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc multi +@c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex defaultprec Donnelly Gotos +@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref yypush +@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex lr +@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge POSIX +@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG yypull +@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit nonfree +@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok rr +@c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln Stallman Destructor +@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym enum +@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof Lex +@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum DOTDOT +@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype Unary +@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs nonterminal +@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES reentrant +@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param yypstate +@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP subrange +@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword loc +@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH inline +@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype Lookahead +@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args Autoconf +@c LocalWords: infile ypp yxx outfile itemx tex leaderfill Troubleshouting sqrt +@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll lookahead +@c LocalWords: nbar yytext fst snd osplit ntwo strdup AST Troublereporting th +@c LocalWords: YYSTACK DVI fdl printindex IELR nondeterministic nonterminals ps +@c LocalWords: subexpressions declarator nondeferred config libintl postfix LAC +@c LocalWords: preprocessor nonpositive unary nonnumeric typedef extern rhs +@c LocalWords: yytokentype filename destructor multicharacter nonnull EBCDIC +@c LocalWords: lvalue nonnegative XNUM CHR chr TAGLESS tagless stdout api TOK +@c LocalWords: destructors Reentrancy nonreentrant subgrammar nonassociative +@c LocalWords: deffnx namespace xml goto lalr ielr runtime lex yacc yyps env +@c LocalWords: yystate variadic Unshift NLS gettext po UTF Automake LOCALEDIR +@c LocalWords: YYENABLE bindtextdomain Makefile DEFS CPPFLAGS DBISON DeRemer +@c LocalWords: autoreconf Pennello multisets nondeterminism Generalised baz +@c LocalWords: redeclare automata Dparse localedir datadir XSLT midrule Wno +@c LocalWords: makefiles Graphviz multitable headitem hh basename Doxygen fno +@c LocalWords: doxygen ival sval deftypemethod deallocate pos deftypemethodx +@c LocalWords: Ctor defcv defcvx arg accessors arithmetics CPP ifndef CALCXX +@c LocalWords: lexer's calcxx bool LPAREN RPAREN deallocation cerrno climits +@c LocalWords: cstdlib Debian undef yywrap unput noyywrap nounput zA yyleng +@c LocalWords: errno strtol ERANGE str strerror iostream argc argv Javadoc +@c LocalWords: bytecode initializers superclass stype ASTNode autoboxing nls +@c LocalWords: toString deftypeivar deftypeivarx deftypeop YYParser strictfp +@c LocalWords: superclasses boolean getErrorVerbose setErrorVerbose deftypecv +@c LocalWords: getDebugStream setDebugStream getDebugLevel setDebugLevel url +@c LocalWords: bisonVersion deftypecvx bisonSkeleton getStartPos getEndPos +@c LocalWords: getLVal defvar deftypefn deftypefnx gotos msgfmt +@c LocalWords: subdirectory Solaris nonassociativity