lr.default-reductions: rename "all" value to "full".
[bison.git] / doc / bison.texinfo
0 / 11813 (  0%)
CommitLineData
1\input texinfo @c -*-texinfo-*-
2@comment %**start of header
3@setfilename bison.info
4@include version.texi
5@settitle Bison @value{VERSION}
6@setchapternewpage odd
7
8@finalout
9
10@c SMALL BOOK version
11@c This edition has been formatted so that you can format and print it in
12@c the smallbook format.
13@c @smallbook
14
15@c Set following if you want to document %default-prec and %no-default-prec.
16@c This feature is experimental and may change in future Bison versions.
17@c @set defaultprec
18
19@ifnotinfo
20@syncodeindex fn cp
21@syncodeindex vr cp
22@syncodeindex tp cp
23@end ifnotinfo
24@ifinfo
25@synindex fn cp
26@synindex vr cp
27@synindex tp cp
28@end ifinfo
29@comment %**end of header
30
31@copying
32
33This manual (@value{UPDATED}) is for GNU Bison (version
34@value{VERSION}), the GNU parser generator.
35
36Copyright @copyright{} 1988-1993, 1995, 1998-2011 Free Software
37Foundation, Inc.
38
39@quotation
40Permission is granted to copy, distribute and/or modify this document
41under the terms of the GNU Free Documentation License,
42Version 1.3 or any later version published by the Free Software
43Foundation; with no Invariant Sections, with the Front-Cover texts
44being ``A GNU Manual,'' and with the Back-Cover Texts as in
45(a) below. A copy of the license is included in the section entitled
46``GNU Free Documentation License.''
47
48(a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
49modify this GNU manual. Buying copies from the FSF
50supports it in developing GNU and promoting software
51freedom.''
52@end quotation
53@end copying
54
55@dircategory Software development
56@direntry
57* bison: (bison). GNU parser generator (Yacc replacement).
58@end direntry
59
60@titlepage
61@title Bison
62@subtitle The Yacc-compatible Parser Generator
63@subtitle @value{UPDATED}, Bison Version @value{VERSION}
64
65@author by Charles Donnelly and Richard Stallman
66
67@page
68@vskip 0pt plus 1filll
69@insertcopying
70@sp 2
71Published by the Free Software Foundation @*
7251 Franklin Street, Fifth Floor @*
73Boston, MA 02110-1301 USA @*
74Printed copies are available from the Free Software Foundation.@*
75ISBN 1-882114-44-2
76@sp 2
77Cover art by Etienne Suvasa.
78@end titlepage
79
80@contents
81
82@ifnottex
83@node Top
84@top Bison
85@insertcopying
86@end ifnottex
87
88@menu
89* Introduction::
90* Conditions::
91* Copying:: The GNU General Public License says
92 how you can copy and share Bison.
93
94Tutorial sections:
95* Concepts:: Basic concepts for understanding Bison.
96* Examples:: Three simple explained examples of using Bison.
97
98Reference sections:
99* Grammar File:: Writing Bison declarations and rules.
100* Interface:: C-language interface to the parser function @code{yyparse}.
101* Algorithm:: How the Bison parser works at run-time.
102* Error Recovery:: Writing rules for error recovery.
103* Context Dependency:: What to do if your language syntax is too
104 messy for Bison to handle straightforwardly.
105* Debugging:: Understanding or debugging Bison parsers.
106* Invocation:: How to run Bison (to produce the parser implementation).
107* Other Languages:: Creating C++ and Java parsers.
108* FAQ:: Frequently Asked Questions
109* Table of Symbols:: All the keywords of the Bison language are explained.
110* Glossary:: Basic concepts are explained.
111* Copying This Manual:: License for copying this manual.
112* Bibliography:: Publications cited in this manual.
113* Index:: Cross-references to the text.
114
115@detailmenu
116 --- The Detailed Node Listing ---
117
118The Concepts of Bison
119
120* Language and Grammar:: Languages and context-free grammars,
121 as mathematical ideas.
122* Grammar in Bison:: How we represent grammars for Bison's sake.
123* Semantic Values:: Each token or syntactic grouping can have
124 a semantic value (the value of an integer,
125 the name of an identifier, etc.).
126* Semantic Actions:: Each rule can have an action containing C code.
127* GLR Parsers:: Writing parsers for general context-free languages.
128* Locations Overview:: Tracking Locations.
129* Bison Parser:: What are Bison's input and output,
130 how is the output used?
131* Stages:: Stages in writing and running Bison grammars.
132* Grammar Layout:: Overall structure of a Bison grammar file.
133
134Writing GLR Parsers
135
136* Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
137* Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
138* GLR Semantic Actions:: Considerations for semantic values and deferred actions.
139* Semantic Predicates:: Controlling a parse with arbitrary computations.
140* Compiler Requirements:: GLR parsers require a modern C compiler.
141
142Examples
143
144* RPN Calc:: Reverse polish notation calculator;
145 a first example with no operator precedence.
146* Infix Calc:: Infix (algebraic) notation calculator.
147 Operator precedence is introduced.
148* Simple Error Recovery:: Continuing after syntax errors.
149* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
150* Multi-function Calc:: Calculator with memory and trig functions.
151 It uses multiple data-types for semantic values.
152* Exercises:: Ideas for improving the multi-function calculator.
153
154Reverse Polish Notation Calculator
155
156* Rpcalc Declarations:: Prologue (declarations) for rpcalc.
157* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
158* Rpcalc Lexer:: The lexical analyzer.
159* Rpcalc Main:: The controlling function.
160* Rpcalc Error:: The error reporting function.
161* Rpcalc Generate:: Running Bison on the grammar file.
162* Rpcalc Compile:: Run the C compiler on the output code.
163
164Grammar Rules for @code{rpcalc}
165
166* Rpcalc Input::
167* Rpcalc Line::
168* Rpcalc Expr::
169
170Location Tracking Calculator: @code{ltcalc}
171
172* Ltcalc Declarations:: Bison and C declarations for ltcalc.
173* Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
174* Ltcalc Lexer:: The lexical analyzer.
175
176Multi-Function Calculator: @code{mfcalc}
177
178* Mfcalc Declarations:: Bison declarations for multi-function calculator.
179* Mfcalc Rules:: Grammar rules for the calculator.
180* Mfcalc Symbol Table:: Symbol table management subroutines.
181
182Bison Grammar Files
183
184* Grammar Outline:: Overall layout of the grammar file.
185* Symbols:: Terminal and nonterminal symbols.
186* Rules:: How to write grammar rules.
187* Recursion:: Writing recursive rules.
188* Semantics:: Semantic values and actions.
189* Locations:: Locations and actions.
190* Declarations:: All kinds of Bison declarations are described here.
191* Multiple Parsers:: Putting more than one Bison parser in one program.
192
193Outline of a Bison Grammar
194
195* Prologue:: Syntax and usage of the prologue.
196* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
197* Bison Declarations:: Syntax and usage of the Bison declarations section.
198* Grammar Rules:: Syntax and usage of the grammar rules section.
199* Epilogue:: Syntax and usage of the epilogue.
200
201Defining Language Semantics
202
203* Value Type:: Specifying one data type for all semantic values.
204* Multiple Types:: Specifying several alternative data types.
205* Actions:: An action is the semantic definition of a grammar rule.
206* Action Types:: Specifying data types for actions to operate on.
207* Mid-Rule Actions:: Most actions go at the end of a rule.
208 This says when, why and how to use the exceptional
209 action in the middle of a rule.
210* Named References:: Using named references in actions.
211
212Tracking Locations
213
214* Location Type:: Specifying a data type for locations.
215* Actions and Locations:: Using locations in actions.
216* Location Default Action:: Defining a general way to compute locations.
217
218Bison Declarations
219
220* Require Decl:: Requiring a Bison version.
221* Token Decl:: Declaring terminal symbols.
222* Precedence Decl:: Declaring terminals with precedence and associativity.
223* Union Decl:: Declaring the set of all semantic value types.
224* Type Decl:: Declaring the choice of type for a nonterminal symbol.
225* Initial Action Decl:: Code run before parsing starts.
226* Destructor Decl:: Declaring how symbols are freed.
227* Expect Decl:: Suppressing warnings about parsing conflicts.
228* Start Decl:: Specifying the start symbol.
229* Pure Decl:: Requesting a reentrant parser.
230* Push Decl:: Requesting a push parser.
231* Decl Summary:: Table of all Bison declarations.
232* %define Summary:: Defining variables to adjust Bison's behavior.
233* %code Summary:: Inserting code into the parser source.
234
235Parser C-Language Interface
236
237* Parser Function:: How to call @code{yyparse} and what it returns.
238* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
239* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
240* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
241* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
242* Lexical:: You must supply a function @code{yylex}
243 which reads tokens.
244* Error Reporting:: You must supply a function @code{yyerror}.
245* Action Features:: Special features for use in actions.
246* Internationalization:: How to let the parser speak in the user's
247 native language.
248
249The Lexical Analyzer Function @code{yylex}
250
251* Calling Convention:: How @code{yyparse} calls @code{yylex}.
252* Token Values:: How @code{yylex} must return the semantic value
253 of the token it has read.
254* Token Locations:: How @code{yylex} must return the text location
255 (line number, etc.) of the token, if the
256 actions want that.
257* Pure Calling:: How the calling convention differs in a pure parser
258 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
259
260The Bison Parser Algorithm
261
262* Lookahead:: Parser looks one token ahead when deciding what to do.
263* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
264* Precedence:: Operator precedence works by resolving conflicts.
265* Contextual Precedence:: When an operator's precedence depends on context.
266* Parser States:: The parser is a finite-state-machine with stack.
267* Reduce/Reduce:: When two rules are applicable in the same situation.
268* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
269* Tuning LR:: How to tune fundamental aspects of LR-based parsing.
270* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
271* Memory Management:: What happens when memory is exhausted. How to avoid it.
272
273Operator Precedence
274
275* Why Precedence:: An example showing why precedence is needed.
276* Using Precedence:: How to specify precedence and associativity.
277* Precedence Only:: How to specify precedence only.
278* Precedence Examples:: How these features are used in the previous example.
279* How Precedence:: How they work.
280
281Tuning LR
282
283* LR Table Construction:: Choose a different construction algorithm.
284* Default Reductions:: Disable default reductions.
285* LAC:: Correct lookahead sets in the parser states.
286* Unreachable States:: Keep unreachable parser states for debugging.
287
288Handling Context Dependencies
289
290* Semantic Tokens:: Token parsing can depend on the semantic context.
291* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
292* Tie-in Recovery:: Lexical tie-ins have implications for how
293 error recovery rules must be written.
294
295Debugging Your Parser
296
297* Understanding:: Understanding the structure of your parser.
298* Tracing:: Tracing the execution of your parser.
299
300Invoking Bison
301
302* Bison Options:: All the options described in detail,
303 in alphabetical order by short options.
304* Option Cross Key:: Alphabetical list of long options.
305* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
306
307Parsers Written In Other Languages
308
309* C++ Parsers:: The interface to generate C++ parser classes
310* Java Parsers:: The interface to generate Java parser classes
311
312C++ Parsers
313
314* C++ Bison Interface:: Asking for C++ parser generation
315* C++ Semantic Values:: %union vs. C++
316* C++ Location Values:: The position and location classes
317* C++ Parser Interface:: Instantiating and running the parser
318* C++ Scanner Interface:: Exchanges between yylex and parse
319* A Complete C++ Example:: Demonstrating their use
320
321A Complete C++ Example
322
323* Calc++ --- C++ Calculator:: The specifications
324* Calc++ Parsing Driver:: An active parsing context
325* Calc++ Parser:: A parser class
326* Calc++ Scanner:: A pure C++ Flex scanner
327* Calc++ Top Level:: Conducting the band
328
329Java Parsers
330
331* Java Bison Interface:: Asking for Java parser generation
332* Java Semantic Values:: %type and %token vs. Java
333* Java Location Values:: The position and location classes
334* Java Parser Interface:: Instantiating and running the parser
335* Java Scanner Interface:: Specifying the scanner for the parser
336* Java Action Features:: Special features for use in actions
337* Java Differences:: Differences between C/C++ and Java Grammars
338* Java Declarations Summary:: List of Bison declarations used with Java
339
340Frequently Asked Questions
341
342* Memory Exhausted:: Breaking the Stack Limits
343* How Can I Reset the Parser:: @code{yyparse} Keeps some State
344* Strings are Destroyed:: @code{yylval} Loses Track of Strings
345* Implementing Gotos/Loops:: Control Flow in the Calculator
346* Multiple start-symbols:: Factoring closely related grammars
347* Secure? Conform?:: Is Bison POSIX safe?
348* I can't build Bison:: Troubleshooting
349* Where can I find help?:: Troubleshouting
350* Bug Reports:: Troublereporting
351* More Languages:: Parsers in C++, Java, and so on
352* Beta Testing:: Experimenting development versions
353* Mailing Lists:: Meeting other Bison users
354
355Copying This Manual
356
357* Copying This Manual:: License for copying this manual.
358
359@end detailmenu
360@end menu
361
362@node Introduction
363@unnumbered Introduction
364@cindex introduction
365
366@dfn{Bison} is a general-purpose parser generator that converts an
367annotated context-free grammar into a deterministic LR or generalized
368LR (GLR) parser employing LALR(1) parser tables. As an experimental
369feature, Bison can also generate IELR(1) or canonical LR(1) parser
370tables. Once you are proficient with Bison, you can use it to develop
371a wide range of language parsers, from those used in simple desk
372calculators to complex programming languages.
373
374Bison is upward compatible with Yacc: all properly-written Yacc
375grammars ought to work with Bison with no change. Anyone familiar
376with Yacc should be able to use Bison with little trouble. You need
377to be fluent in C or C++ programming in order to use Bison or to
378understand this manual. Java is also supported as an experimental
379feature.
380
381We begin with tutorial chapters that explain the basic concepts of
382using Bison and show three explained examples, each building on the
383last. If you don't know Bison or Yacc, start by reading these
384chapters. Reference chapters follow, which describe specific aspects
385of Bison in detail.
386
387Bison was written originally by Robert Corbett. Richard Stallman made
388it Yacc-compatible. Wilfred Hansen of Carnegie Mellon University
389added multi-character string literals and other features. Since then,
390Bison has grown more robust and evolved many other new features thanks
391to the hard work of a long list of volunteers. For details, see the
392@file{THANKS} and @file{ChangeLog} files included in the Bison
393distribution.
394
395This edition corresponds to version @value{VERSION} of Bison.
396
397@node Conditions
398@unnumbered Conditions for Using Bison
399
400The distribution terms for Bison-generated parsers permit using the
401parsers in nonfree programs. Before Bison version 2.2, these extra
402permissions applied only when Bison was generating LALR(1)
403parsers in C@. And before Bison version 1.24, Bison-generated
404parsers could be used only in programs that were free software.
405
406The other GNU programming tools, such as the GNU C
407compiler, have never
408had such a requirement. They could always be used for nonfree
409software. The reason Bison was different was not due to a special
410policy decision; it resulted from applying the usual General Public
411License to all of the Bison source code.
412
413The main output of the Bison utility---the Bison parser implementation
414file---contains a verbatim copy of a sizable piece of Bison, which is
415the code for the parser's implementation. (The actions from your
416grammar are inserted into this implementation at one point, but most
417of the rest of the implementation is not changed.) When we applied
418the GPL terms to the skeleton code for the parser's implementation,
419the effect was to restrict the use of Bison output to free software.
420
421We didn't change the terms because of sympathy for people who want to
422make software proprietary. @strong{Software should be free.} But we
423concluded that limiting Bison's use to free software was doing little to
424encourage people to make other software free. So we decided to make the
425practical conditions for using Bison match the practical conditions for
426using the other GNU tools.
427
428This exception applies when Bison is generating code for a parser.
429You can tell whether the exception applies to a Bison output file by
430inspecting the file for text beginning with ``As a special
431exception@dots{}''. The text spells out the exact terms of the
432exception.
433
434@node Copying
435@unnumbered GNU GENERAL PUBLIC LICENSE
436@include gpl-3.0.texi
437
438@node Concepts
439@chapter The Concepts of Bison
440
441This chapter introduces many of the basic concepts without which the
442details of Bison will not make sense. If you do not already know how to
443use Bison or Yacc, we suggest you start by reading this chapter carefully.
444
445@menu
446* Language and Grammar:: Languages and context-free grammars,
447 as mathematical ideas.
448* Grammar in Bison:: How we represent grammars for Bison's sake.
449* Semantic Values:: Each token or syntactic grouping can have
450 a semantic value (the value of an integer,
451 the name of an identifier, etc.).
452* Semantic Actions:: Each rule can have an action containing C code.
453* GLR Parsers:: Writing parsers for general context-free languages.
454* Locations Overview:: Tracking Locations.
455* Bison Parser:: What are Bison's input and output,
456 how is the output used?
457* Stages:: Stages in writing and running Bison grammars.
458* Grammar Layout:: Overall structure of a Bison grammar file.
459@end menu
460
461@node Language and Grammar
462@section Languages and Context-Free Grammars
463
464@cindex context-free grammar
465@cindex grammar, context-free
466In order for Bison to parse a language, it must be described by a
467@dfn{context-free grammar}. This means that you specify one or more
468@dfn{syntactic groupings} and give rules for constructing them from their
469parts. For example, in the C language, one kind of grouping is called an
470`expression'. One rule for making an expression might be, ``An expression
471can be made of a minus sign and another expression''. Another would be,
472``An expression can be an integer''. As you can see, rules are often
473recursive, but there must be at least one rule which leads out of the
474recursion.
475
476@cindex BNF
477@cindex Backus-Naur form
478The most common formal system for presenting such rules for humans to read
479is @dfn{Backus-Naur Form} or ``BNF'', which was developed in
480order to specify the language Algol 60. Any grammar expressed in
481BNF is a context-free grammar. The input to Bison is
482essentially machine-readable BNF.
483
484@cindex LALR grammars
485@cindex IELR grammars
486@cindex LR grammars
487There are various important subclasses of context-free grammars. Although
488it can handle almost all context-free grammars, Bison is optimized for what
489are called LR(1) grammars. In brief, in these grammars, it must be possible
490to tell how to parse any portion of an input string with just a single token
491of lookahead. For historical reasons, Bison by default is limited by the
492additional restrictions of LALR(1), which is hard to explain simply.
493@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for more
494information on this. As an experimental feature, you can escape these
495additional restrictions by requesting IELR(1) or canonical LR(1) parser
496tables. @xref{LR Table Construction}, to learn how.
497
498@cindex GLR parsing
499@cindex generalized LR (GLR) parsing
500@cindex ambiguous grammars
501@cindex nondeterministic parsing
502
503Parsers for LR(1) grammars are @dfn{deterministic}, meaning
504roughly that the next grammar rule to apply at any point in the input is
505uniquely determined by the preceding input and a fixed, finite portion
506(called a @dfn{lookahead}) of the remaining input. A context-free
507grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
508apply the grammar rules to get the same inputs. Even unambiguous
509grammars can be @dfn{nondeterministic}, meaning that no fixed
510lookahead always suffices to determine the next grammar rule to apply.
511With the proper declarations, Bison is also able to parse these more
512general context-free grammars, using a technique known as GLR
513parsing (for Generalized LR). Bison's GLR parsers
514are able to handle any context-free grammar for which the number of
515possible parses of any given string is finite.
516
517@cindex symbols (abstract)
518@cindex token
519@cindex syntactic grouping
520@cindex grouping, syntactic
521In the formal grammatical rules for a language, each kind of syntactic
522unit or grouping is named by a @dfn{symbol}. Those which are built by
523grouping smaller constructs according to grammatical rules are called
524@dfn{nonterminal symbols}; those which can't be subdivided are called
525@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
526corresponding to a single terminal symbol a @dfn{token}, and a piece
527corresponding to a single nonterminal symbol a @dfn{grouping}.
528
529We can use the C language as an example of what symbols, terminal and
530nonterminal, mean. The tokens of C are identifiers, constants (numeric
531and string), and the various keywords, arithmetic operators and
532punctuation marks. So the terminal symbols of a grammar for C include
533`identifier', `number', `string', plus one symbol for each keyword,
534operator or punctuation mark: `if', `return', `const', `static', `int',
535`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
536(These tokens can be subdivided into characters, but that is a matter of
537lexicography, not grammar.)
538
539Here is a simple C function subdivided into tokens:
540
541@ifinfo
542@example
543int /* @r{keyword `int'} */
544square (int x) /* @r{identifier, open-paren, keyword `int',}
545 @r{identifier, close-paren} */
546@{ /* @r{open-brace} */
547 return x * x; /* @r{keyword `return', identifier, asterisk,}
548 @r{identifier, semicolon} */
549@} /* @r{close-brace} */
550@end example
551@end ifinfo
552@ifnotinfo
553@example
554int /* @r{keyword `int'} */
555square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
556@{ /* @r{open-brace} */
557 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
558@} /* @r{close-brace} */
559@end example
560@end ifnotinfo
561
562The syntactic groupings of C include the expression, the statement, the
563declaration, and the function definition. These are represented in the
564grammar of C by nonterminal symbols `expression', `statement',
565`declaration' and `function definition'. The full grammar uses dozens of
566additional language constructs, each with its own nonterminal symbol, in
567order to express the meanings of these four. The example above is a
568function definition; it contains one declaration, and one statement. In
569the statement, each @samp{x} is an expression and so is @samp{x * x}.
570
571Each nonterminal symbol must have grammatical rules showing how it is made
572out of simpler constructs. For example, one kind of C statement is the
573@code{return} statement; this would be described with a grammar rule which
574reads informally as follows:
575
576@quotation
577A `statement' can be made of a `return' keyword, an `expression' and a
578`semicolon'.
579@end quotation
580
581@noindent
582There would be many other rules for `statement', one for each kind of
583statement in C.
584
585@cindex start symbol
586One nonterminal symbol must be distinguished as the special one which
587defines a complete utterance in the language. It is called the @dfn{start
588symbol}. In a compiler, this means a complete input program. In the C
589language, the nonterminal symbol `sequence of definitions and declarations'
590plays this role.
591
592For example, @samp{1 + 2} is a valid C expression---a valid part of a C
593program---but it is not valid as an @emph{entire} C program. In the
594context-free grammar of C, this follows from the fact that `expression' is
595not the start symbol.
596
597The Bison parser reads a sequence of tokens as its input, and groups the
598tokens using the grammar rules. If the input is valid, the end result is
599that the entire token sequence reduces to a single grouping whose symbol is
600the grammar's start symbol. If we use a grammar for C, the entire input
601must be a `sequence of definitions and declarations'. If not, the parser
602reports a syntax error.
603
604@node Grammar in Bison
605@section From Formal Rules to Bison Input
606@cindex Bison grammar
607@cindex grammar, Bison
608@cindex formal grammar
609
610A formal grammar is a mathematical construct. To define the language
611for Bison, you must write a file expressing the grammar in Bison syntax:
612a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
613
614A nonterminal symbol in the formal grammar is represented in Bison input
615as an identifier, like an identifier in C@. By convention, it should be
616in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
617
618The Bison representation for a terminal symbol is also called a @dfn{token
619type}. Token types as well can be represented as C-like identifiers. By
620convention, these identifiers should be upper case to distinguish them from
621nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
622@code{RETURN}. A terminal symbol that stands for a particular keyword in
623the language should be named after that keyword converted to upper case.
624The terminal symbol @code{error} is reserved for error recovery.
625@xref{Symbols}.
626
627A terminal symbol can also be represented as a character literal, just like
628a C character constant. You should do this whenever a token is just a
629single character (parenthesis, plus-sign, etc.): use that same character in
630a literal as the terminal symbol for that token.
631
632A third way to represent a terminal symbol is with a C string constant
633containing several characters. @xref{Symbols}, for more information.
634
635The grammar rules also have an expression in Bison syntax. For example,
636here is the Bison rule for a C @code{return} statement. The semicolon in
637quotes is a literal character token, representing part of the C syntax for
638the statement; the naked semicolon, and the colon, are Bison punctuation
639used in every rule.
640
641@example
642stmt: RETURN expr ';'
643 ;
644@end example
645
646@noindent
647@xref{Rules, ,Syntax of Grammar Rules}.
648
649@node Semantic Values
650@section Semantic Values
651@cindex semantic value
652@cindex value, semantic
653
654A formal grammar selects tokens only by their classifications: for example,
655if a rule mentions the terminal symbol `integer constant', it means that
656@emph{any} integer constant is grammatically valid in that position. The
657precise value of the constant is irrelevant to how to parse the input: if
658@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
659grammatical.
660
661But the precise value is very important for what the input means once it is
662parsed. A compiler is useless if it fails to distinguish between 4, 1 and
6633989 as constants in the program! Therefore, each token in a Bison grammar
664has both a token type and a @dfn{semantic value}. @xref{Semantics,
665,Defining Language Semantics},
666for details.
667
668The token type is a terminal symbol defined in the grammar, such as
669@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
670you need to know to decide where the token may validly appear and how to
671group it with other tokens. The grammar rules know nothing about tokens
672except their types.
673
674The semantic value has all the rest of the information about the
675meaning of the token, such as the value of an integer, or the name of an
676identifier. (A token such as @code{','} which is just punctuation doesn't
677need to have any semantic value.)
678
679For example, an input token might be classified as token type
680@code{INTEGER} and have the semantic value 4. Another input token might
681have the same token type @code{INTEGER} but value 3989. When a grammar
682rule says that @code{INTEGER} is allowed, either of these tokens is
683acceptable because each is an @code{INTEGER}. When the parser accepts the
684token, it keeps track of the token's semantic value.
685
686Each grouping can also have a semantic value as well as its nonterminal
687symbol. For example, in a calculator, an expression typically has a
688semantic value that is a number. In a compiler for a programming
689language, an expression typically has a semantic value that is a tree
690structure describing the meaning of the expression.
691
692@node Semantic Actions
693@section Semantic Actions
694@cindex semantic actions
695@cindex actions, semantic
696
697In order to be useful, a program must do more than parse input; it must
698also produce some output based on the input. In a Bison grammar, a grammar
699rule can have an @dfn{action} made up of C statements. Each time the
700parser recognizes a match for that rule, the action is executed.
701@xref{Actions}.
702
703Most of the time, the purpose of an action is to compute the semantic value
704of the whole construct from the semantic values of its parts. For example,
705suppose we have a rule which says an expression can be the sum of two
706expressions. When the parser recognizes such a sum, each of the
707subexpressions has a semantic value which describes how it was built up.
708The action for this rule should create a similar sort of value for the
709newly recognized larger expression.
710
711For example, here is a rule that says an expression can be the sum of
712two subexpressions:
713
714@example
715expr: expr '+' expr @{ $$ = $1 + $3; @}
716 ;
717@end example
718
719@noindent
720The action says how to produce the semantic value of the sum expression
721from the values of the two subexpressions.
722
723@node GLR Parsers
724@section Writing GLR Parsers
725@cindex GLR parsing
726@cindex generalized LR (GLR) parsing
727@findex %glr-parser
728@cindex conflicts
729@cindex shift/reduce conflicts
730@cindex reduce/reduce conflicts
731
732In some grammars, Bison's deterministic
733LR(1) parsing algorithm cannot decide whether to apply a
734certain grammar rule at a given point. That is, it may not be able to
735decide (on the basis of the input read so far) which of two possible
736reductions (applications of a grammar rule) applies, or whether to apply
737a reduction or read more of the input and apply a reduction later in the
738input. These are known respectively as @dfn{reduce/reduce} conflicts
739(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
740(@pxref{Shift/Reduce}).
741
742To use a grammar that is not easily modified to be LR(1), a
743more general parsing algorithm is sometimes necessary. If you include
744@code{%glr-parser} among the Bison declarations in your file
745(@pxref{Grammar Outline}), the result is a Generalized LR
746(GLR) parser. These parsers handle Bison grammars that
747contain no unresolved conflicts (i.e., after applying precedence
748declarations) identically to deterministic parsers. However, when
749faced with unresolved shift/reduce and reduce/reduce conflicts,
750GLR parsers use the simple expedient of doing both,
751effectively cloning the parser to follow both possibilities. Each of
752the resulting parsers can again split, so that at any given time, there
753can be any number of possible parses being explored. The parsers
754proceed in lockstep; that is, all of them consume (shift) a given input
755symbol before any of them proceed to the next. Each of the cloned
756parsers eventually meets one of two possible fates: either it runs into
757a parsing error, in which case it simply vanishes, or it merges with
758another parser, because the two of them have reduced the input to an
759identical set of symbols.
760
761During the time that there are multiple parsers, semantic actions are
762recorded, but not performed. When a parser disappears, its recorded
763semantic actions disappear as well, and are never performed. When a
764reduction makes two parsers identical, causing them to merge, Bison
765records both sets of semantic actions. Whenever the last two parsers
766merge, reverting to the single-parser case, Bison resolves all the
767outstanding actions either by precedences given to the grammar rules
768involved, or by performing both actions, and then calling a designated
769user-defined function on the resulting values to produce an arbitrary
770merged result.
771
772@menu
773* Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
774* Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
775* GLR Semantic Actions:: Considerations for semantic values and deferred actions.
776* Semantic Predicates:: Controlling a parse with arbitrary computations.
777* Compiler Requirements:: GLR parsers require a modern C compiler.
778@end menu
779
780@node Simple GLR Parsers
781@subsection Using GLR on Unambiguous Grammars
782@cindex GLR parsing, unambiguous grammars
783@cindex generalized LR (GLR) parsing, unambiguous grammars
784@findex %glr-parser
785@findex %expect-rr
786@cindex conflicts
787@cindex reduce/reduce conflicts
788@cindex shift/reduce conflicts
789
790In the simplest cases, you can use the GLR algorithm
791to parse grammars that are unambiguous but fail to be LR(1).
792Such grammars typically require more than one symbol of lookahead.
793
794Consider a problem that
795arises in the declaration of enumerated and subrange types in the
796programming language Pascal. Here are some examples:
797
798@example
799type subrange = lo .. hi;
800type enum = (a, b, c);
801@end example
802
803@noindent
804The original language standard allows only numeric
805literals and constant identifiers for the subrange bounds (@samp{lo}
806and @samp{hi}), but Extended Pascal (ISO/IEC
80710206) and many other
808Pascal implementations allow arbitrary expressions there. This gives
809rise to the following situation, containing a superfluous pair of
810parentheses:
811
812@example
813type subrange = (a) .. b;
814@end example
815
816@noindent
817Compare this to the following declaration of an enumerated
818type with only one value:
819
820@example
821type enum = (a);
822@end example
823
824@noindent
825(These declarations are contrived, but they are syntactically
826valid, and more-complicated cases can come up in practical programs.)
827
828These two declarations look identical until the @samp{..} token.
829With normal LR(1) one-token lookahead it is not
830possible to decide between the two forms when the identifier
831@samp{a} is parsed. It is, however, desirable
832for a parser to decide this, since in the latter case
833@samp{a} must become a new identifier to represent the enumeration
834value, while in the former case @samp{a} must be evaluated with its
835current meaning, which may be a constant or even a function call.
836
837You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
838to be resolved later, but this typically requires substantial
839contortions in both semantic actions and large parts of the
840grammar, where the parentheses are nested in the recursive rules for
841expressions.
842
843You might think of using the lexer to distinguish between the two
844forms by returning different tokens for currently defined and
845undefined identifiers. But if these declarations occur in a local
846scope, and @samp{a} is defined in an outer scope, then both forms
847are possible---either locally redefining @samp{a}, or using the
848value of @samp{a} from the outer scope. So this approach cannot
849work.
850
851A simple solution to this problem is to declare the parser to
852use the GLR algorithm.
853When the GLR parser reaches the critical state, it
854merely splits into two branches and pursues both syntax rules
855simultaneously. Sooner or later, one of them runs into a parsing
856error. If there is a @samp{..} token before the next
857@samp{;}, the rule for enumerated types fails since it cannot
858accept @samp{..} anywhere; otherwise, the subrange type rule
859fails since it requires a @samp{..} token. So one of the branches
860fails silently, and the other one continues normally, performing
861all the intermediate actions that were postponed during the split.
862
863If the input is syntactically incorrect, both branches fail and the parser
864reports a syntax error as usual.
865
866The effect of all this is that the parser seems to ``guess'' the
867correct branch to take, or in other words, it seems to use more
868lookahead than the underlying LR(1) algorithm actually allows
869for. In this example, LR(2) would suffice, but also some cases
870that are not LR(@math{k}) for any @math{k} can be handled this way.
871
872In general, a GLR parser can take quadratic or cubic worst-case time,
873and the current Bison parser even takes exponential time and space
874for some grammars. In practice, this rarely happens, and for many
875grammars it is possible to prove that it cannot happen.
876The present example contains only one conflict between two
877rules, and the type-declaration context containing the conflict
878cannot be nested. So the number of
879branches that can exist at any time is limited by the constant 2,
880and the parsing time is still linear.
881
882Here is a Bison grammar corresponding to the example above. It
883parses a vastly simplified form of Pascal type declarations.
884
885@example
886%token TYPE DOTDOT ID
887
888@group
889%left '+' '-'
890%left '*' '/'
891@end group
892
893%%
894
895@group
896type_decl : TYPE ID '=' type ';'
897 ;
898@end group
899
900@group
901type : '(' id_list ')'
902 | expr DOTDOT expr
903 ;
904@end group
905
906@group
907id_list : ID
908 | id_list ',' ID
909 ;
910@end group
911
912@group
913expr : '(' expr ')'
914 | expr '+' expr
915 | expr '-' expr
916 | expr '*' expr
917 | expr '/' expr
918 | ID
919 ;
920@end group
921@end example
922
923When used as a normal LR(1) grammar, Bison correctly complains
924about one reduce/reduce conflict. In the conflicting situation the
925parser chooses one of the alternatives, arbitrarily the one
926declared first. Therefore the following correct input is not
927recognized:
928
929@example
930type t = (a) .. b;
931@end example
932
933The parser can be turned into a GLR parser, while also telling Bison
934to be silent about the one known reduce/reduce conflict, by adding
935these two declarations to the Bison grammar file (before the first
936@samp{%%}):
937
938@example
939%glr-parser
940%expect-rr 1
941@end example
942
943@noindent
944No change in the grammar itself is required. Now the
945parser recognizes all valid declarations, according to the
946limited syntax above, transparently. In fact, the user does not even
947notice when the parser splits.
948
949So here we have a case where we can use the benefits of GLR,
950almost without disadvantages. Even in simple cases like this, however,
951there are at least two potential problems to beware. First, always
952analyze the conflicts reported by Bison to make sure that GLR
953splitting is only done where it is intended. A GLR parser
954splitting inadvertently may cause problems less obvious than an
955LR parser statically choosing the wrong alternative in a
956conflict. Second, consider interactions with the lexer (@pxref{Semantic
957Tokens}) with great care. Since a split parser consumes tokens without
958performing any actions during the split, the lexer cannot obtain
959information via parser actions. Some cases of lexer interactions can be
960eliminated by using GLR to shift the complications from the
961lexer to the parser. You must check the remaining cases for
962correctness.
963
964In our example, it would be safe for the lexer to return tokens based on
965their current meanings in some symbol table, because no new symbols are
966defined in the middle of a type declaration. Though it is possible for
967a parser to define the enumeration constants as they are parsed, before
968the type declaration is completed, it actually makes no difference since
969they cannot be used within the same enumerated type declaration.
970
971@node Merging GLR Parses
972@subsection Using GLR to Resolve Ambiguities
973@cindex GLR parsing, ambiguous grammars
974@cindex generalized LR (GLR) parsing, ambiguous grammars
975@findex %dprec
976@findex %merge
977@cindex conflicts
978@cindex reduce/reduce conflicts
979
980Let's consider an example, vastly simplified from a C++ grammar.
981
982@example
983%@{
984 #include <stdio.h>
985 #define YYSTYPE char const *
986 int yylex (void);
987 void yyerror (char const *);
988%@}
989
990%token TYPENAME ID
991
992%right '='
993%left '+'
994
995%glr-parser
996
997%%
998
999prog :
1000 | prog stmt @{ printf ("\n"); @}
1001 ;
1002
1003stmt : expr ';' %dprec 1
1004 | decl %dprec 2
1005 ;
1006
1007expr : ID @{ printf ("%s ", $$); @}
1008 | TYPENAME '(' expr ')'
1009 @{ printf ("%s <cast> ", $1); @}
1010 | expr '+' expr @{ printf ("+ "); @}
1011 | expr '=' expr @{ printf ("= "); @}
1012 ;
1013
1014decl : TYPENAME declarator ';'
1015 @{ printf ("%s <declare> ", $1); @}
1016 | TYPENAME declarator '=' expr ';'
1017 @{ printf ("%s <init-declare> ", $1); @}
1018 ;
1019
1020declarator : ID @{ printf ("\"%s\" ", $1); @}
1021 | '(' declarator ')'
1022 ;
1023@end example
1024
1025@noindent
1026This models a problematic part of the C++ grammar---the ambiguity between
1027certain declarations and statements. For example,
1028
1029@example
1030T (x) = y+z;
1031@end example
1032
1033@noindent
1034parses as either an @code{expr} or a @code{stmt}
1035(assuming that @samp{T} is recognized as a @code{TYPENAME} and
1036@samp{x} as an @code{ID}).
1037Bison detects this as a reduce/reduce conflict between the rules
1038@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
1039time it encounters @code{x} in the example above. Since this is a
1040GLR parser, it therefore splits the problem into two parses, one for
1041each choice of resolving the reduce/reduce conflict.
1042Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1043however, neither of these parses ``dies,'' because the grammar as it stands is
1044ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1045the other reduces @code{stmt : decl}, after which both parsers are in an
1046identical state: they've seen @samp{prog stmt} and have the same unprocessed
1047input remaining. We say that these parses have @dfn{merged.}
1048
1049At this point, the GLR parser requires a specification in the
1050grammar of how to choose between the competing parses.
1051In the example above, the two @code{%dprec}
1052declarations specify that Bison is to give precedence
1053to the parse that interprets the example as a
1054@code{decl}, which implies that @code{x} is a declarator.
1055The parser therefore prints
1056
1057@example
1058"x" y z + T <init-declare>
1059@end example
1060
1061The @code{%dprec} declarations only come into play when more than one
1062parse survives. Consider a different input string for this parser:
1063
1064@example
1065T (x) + y;
1066@end example
1067
1068@noindent
1069This is another example of using GLR to parse an unambiguous
1070construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
1071Here, there is no ambiguity (this cannot be parsed as a declaration).
1072However, at the time the Bison parser encounters @code{x}, it does not
1073have enough information to resolve the reduce/reduce conflict (again,
1074between @code{x} as an @code{expr} or a @code{declarator}). In this
1075case, no precedence declaration is used. Again, the parser splits
1076into two, one assuming that @code{x} is an @code{expr}, and the other
1077assuming @code{x} is a @code{declarator}. The second of these parsers
1078then vanishes when it sees @code{+}, and the parser prints
1079
1080@example
1081x T <cast> y +
1082@end example
1083
1084Suppose that instead of resolving the ambiguity, you wanted to see all
1085the possibilities. For this purpose, you must merge the semantic
1086actions of the two possible parsers, rather than choosing one over the
1087other. To do so, you could change the declaration of @code{stmt} as
1088follows:
1089
1090@example
1091stmt : expr ';' %merge <stmtMerge>
1092 | decl %merge <stmtMerge>
1093 ;
1094@end example
1095
1096@noindent
1097and define the @code{stmtMerge} function as:
1098
1099@example
1100static YYSTYPE
1101stmtMerge (YYSTYPE x0, YYSTYPE x1)
1102@{
1103 printf ("<OR> ");
1104 return "";
1105@}
1106@end example
1107
1108@noindent
1109with an accompanying forward declaration
1110in the C declarations at the beginning of the file:
1111
1112@example
1113%@{
1114 #define YYSTYPE char const *
1115 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1116%@}
1117@end example
1118
1119@noindent
1120With these declarations, the resulting parser parses the first example
1121as both an @code{expr} and a @code{decl}, and prints
1122
1123@example
1124"x" y z + T <init-declare> x T <cast> y z + = <OR>
1125@end example
1126
1127Bison requires that all of the
1128productions that participate in any particular merge have identical
1129@samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1130and the parser will report an error during any parse that results in
1131the offending merge.
1132
1133@node GLR Semantic Actions
1134@subsection GLR Semantic Actions
1135
1136The nature of GLR parsing and the structure of the generated
1137parsers give rise to certain restrictions on semantic values and actions.
1138
1139@subsubsection Deferred semantic actions
1140@cindex deferred semantic actions
1141By definition, a deferred semantic action is not performed at the same time as
1142the associated reduction.
1143This raises caveats for several Bison features you might use in a semantic
1144action in a GLR parser.
1145
1146@vindex yychar
1147@cindex GLR parsers and @code{yychar}
1148@vindex yylval
1149@cindex GLR parsers and @code{yylval}
1150@vindex yylloc
1151@cindex GLR parsers and @code{yylloc}
1152In any semantic action, you can examine @code{yychar} to determine the type of
1153the lookahead token present at the time of the associated reduction.
1154After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1155you can then examine @code{yylval} and @code{yylloc} to determine the
1156lookahead token's semantic value and location, if any.
1157In a nondeferred semantic action, you can also modify any of these variables to
1158influence syntax analysis.
1159@xref{Lookahead, ,Lookahead Tokens}.
1160
1161@findex yyclearin
1162@cindex GLR parsers and @code{yyclearin}
1163In a deferred semantic action, it's too late to influence syntax analysis.
1164In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1165shallow copies of the values they had at the time of the associated reduction.
1166For this reason alone, modifying them is dangerous.
1167Moreover, the result of modifying them is undefined and subject to change with
1168future versions of Bison.
1169For example, if a semantic action might be deferred, you should never write it
1170to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1171memory referenced by @code{yylval}.
1172
1173@subsubsection YYERROR
1174@findex YYERROR
1175@cindex GLR parsers and @code{YYERROR}
1176Another Bison feature requiring special consideration is @code{YYERROR}
1177(@pxref{Action Features}), which you can invoke in a semantic action to
1178initiate error recovery.
1179During deterministic GLR operation, the effect of @code{YYERROR} is
1180the same as its effect in a deterministic parser.
1181The effect in a deferred action is similar, but the precise point of the
1182error is undefined; instead, the parser reverts to deterministic operation,
1183selecting an unspecified stack on which to continue with a syntax error.
1184In a semantic predicate (see @ref{Semantic Predicates}) during nondeterministic
1185parsing, @code{YYERROR} silently prunes
1186the parse that invoked the test.
1187
1188@subsubsection Restrictions on semantic values and locations
1189GLR parsers require that you use POD (Plain Old Data) types for
1190semantic values and location types when using the generated parsers as
1191C++ code.
1192
1193@node Semantic Predicates
1194@subsection Controlling a Parse with Arbitrary Predicates
1195@findex %?
1196@cindex Semantic predicates in GLR parsers
1197
1198In addition to the @code{%dprec} and @code{%merge} directives,
1199GLR parsers
1200allow you to reject parses on the basis of arbitrary computations executed
1201in user code, without having Bison treat this rejection as an error
1202if there are alternative parses. (This feature is experimental and may
1203evolve. We welcome user feedback.) For example,
1204
1205@smallexample
1206widget :
1207 %?@{ new_syntax @} "widget" id new_args @{ $$ = f($3, $4); @}
1208 | %?@{ !new_syntax @} "widget" id old_args @{ $$ = f($3, $4); @}
1209 ;
1210@end smallexample
1211
1212@noindent
1213is one way to allow the same parser to handle two different syntaxes for
1214widgets. The clause preceded by @code{%?} is treated like an ordinary
1215action, except that its text is treated as an expression and is always
1216evaluated immediately (even when in nondeterministic mode). If the
1217expression yields 0 (false), the clause is treated as a syntax error,
1218which, in a nondeterministic parser, causes the stack in which it is reduced
1219to die. In a deterministic parser, it acts like YYERROR.
1220
1221As the example shows, predicates otherwise look like semantic actions, and
1222therefore you must be take them into account when determining the numbers
1223to use for denoting the semantic values of right-hand side symbols.
1224Predicate actions, however, have no defined value, and may not be given
1225labels.
1226
1227There is a subtle difference between semantic predicates and ordinary
1228actions in nondeterministic mode, since the latter are deferred.
1229For example, we could try to rewrite the previous example as
1230
1231@smallexample
1232widget :
1233 @{ if (!new_syntax) YYERROR; @} "widget" id new_args @{ $$ = f($3, $4); @}
1234 | @{ if (new_syntax) YYERROR; @} "widget" id old_args @{ $$ = f($3, $4); @}
1235 ;
1236@end smallexample
1237
1238@noindent
1239(reversing the sense of the predicate tests to cause an error when they are
1240false). However, this
1241does @emph{not} have the same effect if @code{new_args} and @code{old_args}
1242have overlapping syntax.
1243Since the mid-rule actions testing @code{new_syntax} are deferred,
1244a GLR parser first encounters the unresolved ambiguous reduction
1245for cases where @code{new_args} and @code{old_args} recognize the same string
1246@emph{before} performing the tests of @code{new_syntax}. It therefore
1247reports an error.
1248
1249Finally, be careful in writing predicates: deferred actions have not been
1250evaluated, so that using them in a predicate will have undefined effects.
1251
1252@node Compiler Requirements
1253@subsection Considerations when Compiling GLR Parsers
1254@cindex @code{inline}
1255@cindex GLR parsers and @code{inline}
1256
1257The GLR parsers require a compiler for ISO C89 or
1258later. In addition, they use the @code{inline} keyword, which is not
1259C89, but is C99 and is a common extension in pre-C99 compilers. It is
1260up to the user of these parsers to handle
1261portability issues. For instance, if using Autoconf and the Autoconf
1262macro @code{AC_C_INLINE}, a mere
1263
1264@example
1265%@{
1266 #include <config.h>
1267%@}
1268@end example
1269
1270@noindent
1271will suffice. Otherwise, we suggest
1272
1273@example
1274%@{
1275 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1276 #define inline
1277 #endif
1278%@}
1279@end example
1280
1281@node Locations Overview
1282@section Locations
1283@cindex location
1284@cindex textual location
1285@cindex location, textual
1286
1287Many applications, like interpreters or compilers, have to produce verbose
1288and useful error messages. To achieve this, one must be able to keep track of
1289the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
1290Bison provides a mechanism for handling these locations.
1291
1292Each token has a semantic value. In a similar fashion, each token has an
1293associated location, but the type of locations is the same for all tokens and
1294groupings. Moreover, the output parser is equipped with a default data
1295structure for storing locations (@pxref{Locations}, for more details).
1296
1297Like semantic values, locations can be reached in actions using a dedicated
1298set of constructs. In the example above, the location of the whole grouping
1299is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1300@code{@@3}.
1301
1302When a rule is matched, a default action is used to compute the semantic value
1303of its left hand side (@pxref{Actions}). In the same way, another default
1304action is used for locations. However, the action for locations is general
1305enough for most cases, meaning there is usually no need to describe for each
1306rule how @code{@@$} should be formed. When building a new location for a given
1307grouping, the default behavior of the output parser is to take the beginning
1308of the first symbol, and the end of the last symbol.
1309
1310@node Bison Parser
1311@section Bison Output: the Parser Implementation File
1312@cindex Bison parser
1313@cindex Bison utility
1314@cindex lexical analyzer, purpose
1315@cindex parser
1316
1317When you run Bison, you give it a Bison grammar file as input. The
1318most important output is a C source file that implements a parser for
1319the language described by the grammar. This parser is called a
1320@dfn{Bison parser}, and this file is called a @dfn{Bison parser
1321implementation file}. Keep in mind that the Bison utility and the
1322Bison parser are two distinct programs: the Bison utility is a program
1323whose output is the Bison parser implementation file that becomes part
1324of your program.
1325
1326The job of the Bison parser is to group tokens into groupings according to
1327the grammar rules---for example, to build identifiers and operators into
1328expressions. As it does this, it runs the actions for the grammar rules it
1329uses.
1330
1331The tokens come from a function called the @dfn{lexical analyzer} that
1332you must supply in some fashion (such as by writing it in C). The Bison
1333parser calls the lexical analyzer each time it wants a new token. It
1334doesn't know what is ``inside'' the tokens (though their semantic values
1335may reflect this). Typically the lexical analyzer makes the tokens by
1336parsing characters of text, but Bison does not depend on this.
1337@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
1338
1339The Bison parser implementation file is C code which defines a
1340function named @code{yyparse} which implements that grammar. This
1341function does not make a complete C program: you must supply some
1342additional functions. One is the lexical analyzer. Another is an
1343error-reporting function which the parser calls to report an error.
1344In addition, a complete C program must start with a function called
1345@code{main}; you have to provide this, and arrange for it to call
1346@code{yyparse} or the parser will never run. @xref{Interface, ,Parser
1347C-Language Interface}.
1348
1349Aside from the token type names and the symbols in the actions you
1350write, all symbols defined in the Bison parser implementation file
1351itself begin with @samp{yy} or @samp{YY}. This includes interface
1352functions such as the lexical analyzer function @code{yylex}, the
1353error reporting function @code{yyerror} and the parser function
1354@code{yyparse} itself. This also includes numerous identifiers used
1355for internal purposes. Therefore, you should avoid using C
1356identifiers starting with @samp{yy} or @samp{YY} in the Bison grammar
1357file except for the ones defined in this manual. Also, you should
1358avoid using the C identifiers @samp{malloc} and @samp{free} for
1359anything other than their usual meanings.
1360
1361In some cases the Bison parser implementation file includes system
1362headers, and in those cases your code should respect the identifiers
1363reserved by those headers. On some non-GNU hosts, @code{<alloca.h>},
1364@code{<malloc.h>}, @code{<stddef.h>}, and @code{<stdlib.h>} are
1365included as needed to declare memory allocators and related types.
1366@code{<libintl.h>} is included if message translation is in use
1367(@pxref{Internationalization}). Other system headers may be included
1368if you define @code{YYDEBUG} to a nonzero value (@pxref{Tracing,
1369,Tracing Your Parser}).
1370
1371@node Stages
1372@section Stages in Using Bison
1373@cindex stages in using Bison
1374@cindex using Bison
1375
1376The actual language-design process using Bison, from grammar specification
1377to a working compiler or interpreter, has these parts:
1378
1379@enumerate
1380@item
1381Formally specify the grammar in a form recognized by Bison
1382(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1383in the language, describe the action that is to be taken when an
1384instance of that rule is recognized. The action is described by a
1385sequence of C statements.
1386
1387@item
1388Write a lexical analyzer to process input and pass tokens to the parser.
1389The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1390Lexical Analyzer Function @code{yylex}}). It could also be produced
1391using Lex, but the use of Lex is not discussed in this manual.
1392
1393@item
1394Write a controlling function that calls the Bison-produced parser.
1395
1396@item
1397Write error-reporting routines.
1398@end enumerate
1399
1400To turn this source code as written into a runnable program, you
1401must follow these steps:
1402
1403@enumerate
1404@item
1405Run Bison on the grammar to produce the parser.
1406
1407@item
1408Compile the code output by Bison, as well as any other source files.
1409
1410@item
1411Link the object files to produce the finished product.
1412@end enumerate
1413
1414@node Grammar Layout
1415@section The Overall Layout of a Bison Grammar
1416@cindex grammar file
1417@cindex file format
1418@cindex format of grammar file
1419@cindex layout of Bison grammar
1420
1421The input file for the Bison utility is a @dfn{Bison grammar file}. The
1422general form of a Bison grammar file is as follows:
1423
1424@example
1425%@{
1426@var{Prologue}
1427%@}
1428
1429@var{Bison declarations}
1430
1431%%
1432@var{Grammar rules}
1433%%
1434@var{Epilogue}
1435@end example
1436
1437@noindent
1438The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1439in every Bison grammar file to separate the sections.
1440
1441The prologue may define types and variables used in the actions. You can
1442also use preprocessor commands to define macros used there, and use
1443@code{#include} to include header files that do any of these things.
1444You need to declare the lexical analyzer @code{yylex} and the error
1445printer @code{yyerror} here, along with any other global identifiers
1446used by the actions in the grammar rules.
1447
1448The Bison declarations declare the names of the terminal and nonterminal
1449symbols, and may also describe operator precedence and the data types of
1450semantic values of various symbols.
1451
1452The grammar rules define how to construct each nonterminal symbol from its
1453parts.
1454
1455The epilogue can contain any code you want to use. Often the
1456definitions of functions declared in the prologue go here. In a
1457simple program, all the rest of the program can go here.
1458
1459@node Examples
1460@chapter Examples
1461@cindex simple examples
1462@cindex examples, simple
1463
1464Now we show and explain three sample programs written using Bison: a
1465reverse polish notation calculator, an algebraic (infix) notation
1466calculator, and a multi-function calculator. All three have been tested
1467under BSD Unix 4.3; each produces a usable, though limited, interactive
1468desk-top calculator.
1469
1470These examples are simple, but Bison grammars for real programming
1471languages are written the same way. You can copy these examples into a
1472source file to try them.
1473
1474@menu
1475* RPN Calc:: Reverse polish notation calculator;
1476 a first example with no operator precedence.
1477* Infix Calc:: Infix (algebraic) notation calculator.
1478 Operator precedence is introduced.
1479* Simple Error Recovery:: Continuing after syntax errors.
1480* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
1481* Multi-function Calc:: Calculator with memory and trig functions.
1482 It uses multiple data-types for semantic values.
1483* Exercises:: Ideas for improving the multi-function calculator.
1484@end menu
1485
1486@node RPN Calc
1487@section Reverse Polish Notation Calculator
1488@cindex reverse polish notation
1489@cindex polish notation calculator
1490@cindex @code{rpcalc}
1491@cindex calculator, simple
1492
1493The first example is that of a simple double-precision @dfn{reverse polish
1494notation} calculator (a calculator using postfix operators). This example
1495provides a good starting point, since operator precedence is not an issue.
1496The second example will illustrate how operator precedence is handled.
1497
1498The source code for this calculator is named @file{rpcalc.y}. The
1499@samp{.y} extension is a convention used for Bison grammar files.
1500
1501@menu
1502* Rpcalc Declarations:: Prologue (declarations) for rpcalc.
1503* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
1504* Rpcalc Lexer:: The lexical analyzer.
1505* Rpcalc Main:: The controlling function.
1506* Rpcalc Error:: The error reporting function.
1507* Rpcalc Generate:: Running Bison on the grammar file.
1508* Rpcalc Compile:: Run the C compiler on the output code.
1509@end menu
1510
1511@node Rpcalc Declarations
1512@subsection Declarations for @code{rpcalc}
1513
1514Here are the C and Bison declarations for the reverse polish notation
1515calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1516
1517@example
1518/* Reverse polish notation calculator. */
1519
1520%@{
1521 #define YYSTYPE double
1522 #include <math.h>
1523 int yylex (void);
1524 void yyerror (char const *);
1525%@}
1526
1527%token NUM
1528
1529%% /* Grammar rules and actions follow. */
1530@end example
1531
1532The declarations section (@pxref{Prologue, , The prologue}) contains two
1533preprocessor directives and two forward declarations.
1534
1535The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1536specifying the C data type for semantic values of both tokens and
1537groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1538Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1539don't define it, @code{int} is the default. Because we specify
1540@code{double}, each token and each expression has an associated value,
1541which is a floating point number.
1542
1543The @code{#include} directive is used to declare the exponentiation
1544function @code{pow}.
1545
1546The forward declarations for @code{yylex} and @code{yyerror} are
1547needed because the C language requires that functions be declared
1548before they are used. These functions will be defined in the
1549epilogue, but the parser calls them so they must be declared in the
1550prologue.
1551
1552The second section, Bison declarations, provides information to Bison
1553about the token types (@pxref{Bison Declarations, ,The Bison
1554Declarations Section}). Each terminal symbol that is not a
1555single-character literal must be declared here. (Single-character
1556literals normally don't need to be declared.) In this example, all the
1557arithmetic operators are designated by single-character literals, so the
1558only terminal symbol that needs to be declared is @code{NUM}, the token
1559type for numeric constants.
1560
1561@node Rpcalc Rules
1562@subsection Grammar Rules for @code{rpcalc}
1563
1564Here are the grammar rules for the reverse polish notation calculator.
1565
1566@example
1567input: /* empty */
1568 | input line
1569;
1570
1571line: '\n'
1572 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1573;
1574
1575exp: NUM @{ $$ = $1; @}
1576 | exp exp '+' @{ $$ = $1 + $2; @}
1577 | exp exp '-' @{ $$ = $1 - $2; @}
1578 | exp exp '*' @{ $$ = $1 * $2; @}
1579 | exp exp '/' @{ $$ = $1 / $2; @}
1580 /* Exponentiation */
1581 | exp exp '^' @{ $$ = pow ($1, $2); @}
1582 /* Unary minus */
1583 | exp 'n' @{ $$ = -$1; @}
1584;
1585%%
1586@end example
1587
1588The groupings of the rpcalc ``language'' defined here are the expression
1589(given the name @code{exp}), the line of input (@code{line}), and the
1590complete input transcript (@code{input}). Each of these nonterminal
1591symbols has several alternate rules, joined by the vertical bar @samp{|}
1592which is read as ``or''. The following sections explain what these rules
1593mean.
1594
1595The semantics of the language is determined by the actions taken when a
1596grouping is recognized. The actions are the C code that appears inside
1597braces. @xref{Actions}.
1598
1599You must specify these actions in C, but Bison provides the means for
1600passing semantic values between the rules. In each action, the
1601pseudo-variable @code{$$} stands for the semantic value for the grouping
1602that the rule is going to construct. Assigning a value to @code{$$} is the
1603main job of most actions. The semantic values of the components of the
1604rule are referred to as @code{$1}, @code{$2}, and so on.
1605
1606@menu
1607* Rpcalc Input::
1608* Rpcalc Line::
1609* Rpcalc Expr::
1610@end menu
1611
1612@node Rpcalc Input
1613@subsubsection Explanation of @code{input}
1614
1615Consider the definition of @code{input}:
1616
1617@example
1618input: /* empty */
1619 | input line
1620;
1621@end example
1622
1623This definition reads as follows: ``A complete input is either an empty
1624string, or a complete input followed by an input line''. Notice that
1625``complete input'' is defined in terms of itself. This definition is said
1626to be @dfn{left recursive} since @code{input} appears always as the
1627leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1628
1629The first alternative is empty because there are no symbols between the
1630colon and the first @samp{|}; this means that @code{input} can match an
1631empty string of input (no tokens). We write the rules this way because it
1632is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1633It's conventional to put an empty alternative first and write the comment
1634@samp{/* empty */} in it.
1635
1636The second alternate rule (@code{input line}) handles all nontrivial input.
1637It means, ``After reading any number of lines, read one more line if
1638possible.'' The left recursion makes this rule into a loop. Since the
1639first alternative matches empty input, the loop can be executed zero or
1640more times.
1641
1642The parser function @code{yyparse} continues to process input until a
1643grammatical error is seen or the lexical analyzer says there are no more
1644input tokens; we will arrange for the latter to happen at end-of-input.
1645
1646@node Rpcalc Line
1647@subsubsection Explanation of @code{line}
1648
1649Now consider the definition of @code{line}:
1650
1651@example
1652line: '\n'
1653 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1654;
1655@end example
1656
1657The first alternative is a token which is a newline character; this means
1658that rpcalc accepts a blank line (and ignores it, since there is no
1659action). The second alternative is an expression followed by a newline.
1660This is the alternative that makes rpcalc useful. The semantic value of
1661the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1662question is the first symbol in the alternative. The action prints this
1663value, which is the result of the computation the user asked for.
1664
1665This action is unusual because it does not assign a value to @code{$$}. As
1666a consequence, the semantic value associated with the @code{line} is
1667uninitialized (its value will be unpredictable). This would be a bug if
1668that value were ever used, but we don't use it: once rpcalc has printed the
1669value of the user's input line, that value is no longer needed.
1670
1671@node Rpcalc Expr
1672@subsubsection Explanation of @code{expr}
1673
1674The @code{exp} grouping has several rules, one for each kind of expression.
1675The first rule handles the simplest expressions: those that are just numbers.
1676The second handles an addition-expression, which looks like two expressions
1677followed by a plus-sign. The third handles subtraction, and so on.
1678
1679@example
1680exp: NUM
1681 | exp exp '+' @{ $$ = $1 + $2; @}
1682 | exp exp '-' @{ $$ = $1 - $2; @}
1683 @dots{}
1684 ;
1685@end example
1686
1687We have used @samp{|} to join all the rules for @code{exp}, but we could
1688equally well have written them separately:
1689
1690@example
1691exp: NUM ;
1692exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1693exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1694 @dots{}
1695@end example
1696
1697Most of the rules have actions that compute the value of the expression in
1698terms of the value of its parts. For example, in the rule for addition,
1699@code{$1} refers to the first component @code{exp} and @code{$2} refers to
1700the second one. The third component, @code{'+'}, has no meaningful
1701associated semantic value, but if it had one you could refer to it as
1702@code{$3}. When @code{yyparse} recognizes a sum expression using this
1703rule, the sum of the two subexpressions' values is produced as the value of
1704the entire expression. @xref{Actions}.
1705
1706You don't have to give an action for every rule. When a rule has no
1707action, Bison by default copies the value of @code{$1} into @code{$$}.
1708This is what happens in the first rule (the one that uses @code{NUM}).
1709
1710The formatting shown here is the recommended convention, but Bison does
1711not require it. You can add or change white space as much as you wish.
1712For example, this:
1713
1714@example
1715exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
1716@end example
1717
1718@noindent
1719means the same thing as this:
1720
1721@example
1722exp: NUM
1723 | exp exp '+' @{ $$ = $1 + $2; @}
1724 | @dots{}
1725;
1726@end example
1727
1728@noindent
1729The latter, however, is much more readable.
1730
1731@node Rpcalc Lexer
1732@subsection The @code{rpcalc} Lexical Analyzer
1733@cindex writing a lexical analyzer
1734@cindex lexical analyzer, writing
1735
1736The lexical analyzer's job is low-level parsing: converting characters
1737or sequences of characters into tokens. The Bison parser gets its
1738tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1739Analyzer Function @code{yylex}}.
1740
1741Only a simple lexical analyzer is needed for the RPN
1742calculator. This
1743lexical analyzer skips blanks and tabs, then reads in numbers as
1744@code{double} and returns them as @code{NUM} tokens. Any other character
1745that isn't part of a number is a separate token. Note that the token-code
1746for such a single-character token is the character itself.
1747
1748The return value of the lexical analyzer function is a numeric code which
1749represents a token type. The same text used in Bison rules to stand for
1750this token type is also a C expression for the numeric code for the type.
1751This works in two ways. If the token type is a character literal, then its
1752numeric code is that of the character; you can use the same
1753character literal in the lexical analyzer to express the number. If the
1754token type is an identifier, that identifier is defined by Bison as a C
1755macro whose definition is the appropriate number. In this example,
1756therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1757
1758The semantic value of the token (if it has one) is stored into the
1759global variable @code{yylval}, which is where the Bison parser will look
1760for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1761defined at the beginning of the grammar; @pxref{Rpcalc Declarations,
1762,Declarations for @code{rpcalc}}.)
1763
1764A token type code of zero is returned if the end-of-input is encountered.
1765(Bison recognizes any nonpositive value as indicating end-of-input.)
1766
1767Here is the code for the lexical analyzer:
1768
1769@example
1770@group
1771/* The lexical analyzer returns a double floating point
1772 number on the stack and the token NUM, or the numeric code
1773 of the character read if not a number. It skips all blanks
1774 and tabs, and returns 0 for end-of-input. */
1775
1776#include <ctype.h>
1777@end group
1778
1779@group
1780int
1781yylex (void)
1782@{
1783 int c;
1784
1785 /* Skip white space. */
1786 while ((c = getchar ()) == ' ' || c == '\t')
1787 ;
1788@end group
1789@group
1790 /* Process numbers. */
1791 if (c == '.' || isdigit (c))
1792 @{
1793 ungetc (c, stdin);
1794 scanf ("%lf", &yylval);
1795 return NUM;
1796 @}
1797@end group
1798@group
1799 /* Return end-of-input. */
1800 if (c == EOF)
1801 return 0;
1802 /* Return a single char. */
1803 return c;
1804@}
1805@end group
1806@end example
1807
1808@node Rpcalc Main
1809@subsection The Controlling Function
1810@cindex controlling function
1811@cindex main function in simple example
1812
1813In keeping with the spirit of this example, the controlling function is
1814kept to the bare minimum. The only requirement is that it call
1815@code{yyparse} to start the process of parsing.
1816
1817@example
1818@group
1819int
1820main (void)
1821@{
1822 return yyparse ();
1823@}
1824@end group
1825@end example
1826
1827@node Rpcalc Error
1828@subsection The Error Reporting Routine
1829@cindex error reporting routine
1830
1831When @code{yyparse} detects a syntax error, it calls the error reporting
1832function @code{yyerror} to print an error message (usually but not
1833always @code{"syntax error"}). It is up to the programmer to supply
1834@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1835here is the definition we will use:
1836
1837@example
1838@group
1839#include <stdio.h>
1840
1841/* Called by yyparse on error. */
1842void
1843yyerror (char const *s)
1844@{
1845 fprintf (stderr, "%s\n", s);
1846@}
1847@end group
1848@end example
1849
1850After @code{yyerror} returns, the Bison parser may recover from the error
1851and continue parsing if the grammar contains a suitable error rule
1852(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1853have not written any error rules in this example, so any invalid input will
1854cause the calculator program to exit. This is not clean behavior for a
1855real calculator, but it is adequate for the first example.
1856
1857@node Rpcalc Generate
1858@subsection Running Bison to Make the Parser
1859@cindex running Bison (introduction)
1860
1861Before running Bison to produce a parser, we need to decide how to
1862arrange all the source code in one or more source files. For such a
1863simple example, the easiest thing is to put everything in one file,
1864the grammar file. The definitions of @code{yylex}, @code{yyerror} and
1865@code{main} go at the end, in the epilogue of the grammar file
1866(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
1867
1868For a large project, you would probably have several source files, and use
1869@code{make} to arrange to recompile them.
1870
1871With all the source in the grammar file, you use the following command
1872to convert it into a parser implementation file:
1873
1874@example
1875bison @var{file}.y
1876@end example
1877
1878@noindent
1879In this example, the grammar file is called @file{rpcalc.y} (for
1880``Reverse Polish @sc{calc}ulator''). Bison produces a parser
1881implementation file named @file{@var{file}.tab.c}, removing the
1882@samp{.y} from the grammar file name. The parser implementation file
1883contains the source code for @code{yyparse}. The additional functions
1884in the grammar file (@code{yylex}, @code{yyerror} and @code{main}) are
1885copied verbatim to the parser implementation file.
1886
1887@node Rpcalc Compile
1888@subsection Compiling the Parser Implementation File
1889@cindex compiling the parser
1890
1891Here is how to compile and run the parser implementation file:
1892
1893@example
1894@group
1895# @r{List files in current directory.}
1896$ @kbd{ls}
1897rpcalc.tab.c rpcalc.y
1898@end group
1899
1900@group
1901# @r{Compile the Bison parser.}
1902# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
1903$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
1904@end group
1905
1906@group
1907# @r{List files again.}
1908$ @kbd{ls}
1909rpcalc rpcalc.tab.c rpcalc.y
1910@end group
1911@end example
1912
1913The file @file{rpcalc} now contains the executable code. Here is an
1914example session using @code{rpcalc}.
1915
1916@example
1917$ @kbd{rpcalc}
1918@kbd{4 9 +}
191913
1920@kbd{3 7 + 3 4 5 *+-}
1921-13
1922@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
192313
1924@kbd{5 6 / 4 n +}
1925-3.166666667
1926@kbd{3 4 ^} @r{Exponentiation}
192781
1928@kbd{^D} @r{End-of-file indicator}
1929$
1930@end example
1931
1932@node Infix Calc
1933@section Infix Notation Calculator: @code{calc}
1934@cindex infix notation calculator
1935@cindex @code{calc}
1936@cindex calculator, infix notation
1937
1938We now modify rpcalc to handle infix operators instead of postfix. Infix
1939notation involves the concept of operator precedence and the need for
1940parentheses nested to arbitrary depth. Here is the Bison code for
1941@file{calc.y}, an infix desk-top calculator.
1942
1943@example
1944/* Infix notation calculator. */
1945
1946%@{
1947 #define YYSTYPE double
1948 #include <math.h>
1949 #include <stdio.h>
1950 int yylex (void);
1951 void yyerror (char const *);
1952%@}
1953
1954/* Bison declarations. */
1955%token NUM
1956%left '-' '+'
1957%left '*' '/'
1958%precedence NEG /* negation--unary minus */
1959%right '^' /* exponentiation */
1960
1961%% /* The grammar follows. */
1962input: /* empty */
1963 | input line
1964;
1965
1966line: '\n'
1967 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1968;
1969
1970exp: NUM @{ $$ = $1; @}
1971 | exp '+' exp @{ $$ = $1 + $3; @}
1972 | exp '-' exp @{ $$ = $1 - $3; @}
1973 | exp '*' exp @{ $$ = $1 * $3; @}
1974 | exp '/' exp @{ $$ = $1 / $3; @}
1975 | '-' exp %prec NEG @{ $$ = -$2; @}
1976 | exp '^' exp @{ $$ = pow ($1, $3); @}
1977 | '(' exp ')' @{ $$ = $2; @}
1978;
1979%%
1980@end example
1981
1982@noindent
1983The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1984same as before.
1985
1986There are two important new features shown in this code.
1987
1988In the second section (Bison declarations), @code{%left} declares token
1989types and says they are left-associative operators. The declarations
1990@code{%left} and @code{%right} (right associativity) take the place of
1991@code{%token} which is used to declare a token type name without
1992associativity/precedence. (These tokens are single-character literals, which
1993ordinarily don't need to be declared. We declare them here to specify
1994the associativity/precedence.)
1995
1996Operator precedence is determined by the line ordering of the
1997declarations; the higher the line number of the declaration (lower on
1998the page or screen), the higher the precedence. Hence, exponentiation
1999has the highest precedence, unary minus (@code{NEG}) is next, followed
2000by @samp{*} and @samp{/}, and so on. Unary minus is not associative,
2001only precedence matters (@code{%precedence}. @xref{Precedence, ,Operator
2002Precedence}.
2003
2004The other important new feature is the @code{%prec} in the grammar
2005section for the unary minus operator. The @code{%prec} simply instructs
2006Bison that the rule @samp{| '-' exp} has the same precedence as
2007@code{NEG}---in this case the next-to-highest. @xref{Contextual
2008Precedence, ,Context-Dependent Precedence}.
2009
2010Here is a sample run of @file{calc.y}:
2011
2012@need 500
2013@example
2014$ @kbd{calc}
2015@kbd{4 + 4.5 - (34/(8*3+-3))}
20166.880952381
2017@kbd{-56 + 2}
2018-54
2019@kbd{3 ^ 2}
20209
2021@end example
2022
2023@node Simple Error Recovery
2024@section Simple Error Recovery
2025@cindex error recovery, simple
2026
2027Up to this point, this manual has not addressed the issue of @dfn{error
2028recovery}---how to continue parsing after the parser detects a syntax
2029error. All we have handled is error reporting with @code{yyerror}.
2030Recall that by default @code{yyparse} returns after calling
2031@code{yyerror}. This means that an erroneous input line causes the
2032calculator program to exit. Now we show how to rectify this deficiency.
2033
2034The Bison language itself includes the reserved word @code{error}, which
2035may be included in the grammar rules. In the example below it has
2036been added to one of the alternatives for @code{line}:
2037
2038@example
2039@group
2040line: '\n'
2041 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2042 | error '\n' @{ yyerrok; @}
2043;
2044@end group
2045@end example
2046
2047This addition to the grammar allows for simple error recovery in the
2048event of a syntax error. If an expression that cannot be evaluated is
2049read, the error will be recognized by the third rule for @code{line},
2050and parsing will continue. (The @code{yyerror} function is still called
2051upon to print its message as well.) The action executes the statement
2052@code{yyerrok}, a macro defined automatically by Bison; its meaning is
2053that error recovery is complete (@pxref{Error Recovery}). Note the
2054difference between @code{yyerrok} and @code{yyerror}; neither one is a
2055misprint.
2056
2057This form of error recovery deals with syntax errors. There are other
2058kinds of errors; for example, division by zero, which raises an exception
2059signal that is normally fatal. A real calculator program must handle this
2060signal and use @code{longjmp} to return to @code{main} and resume parsing
2061input lines; it would also have to discard the rest of the current line of
2062input. We won't discuss this issue further because it is not specific to
2063Bison programs.
2064
2065@node Location Tracking Calc
2066@section Location Tracking Calculator: @code{ltcalc}
2067@cindex location tracking calculator
2068@cindex @code{ltcalc}
2069@cindex calculator, location tracking
2070
2071This example extends the infix notation calculator with location
2072tracking. This feature will be used to improve the error messages. For
2073the sake of clarity, this example is a simple integer calculator, since
2074most of the work needed to use locations will be done in the lexical
2075analyzer.
2076
2077@menu
2078* Ltcalc Declarations:: Bison and C declarations for ltcalc.
2079* Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
2080* Ltcalc Lexer:: The lexical analyzer.
2081@end menu
2082
2083@node Ltcalc Declarations
2084@subsection Declarations for @code{ltcalc}
2085
2086The C and Bison declarations for the location tracking calculator are
2087the same as the declarations for the infix notation calculator.
2088
2089@example
2090/* Location tracking calculator. */
2091
2092%@{
2093 #define YYSTYPE int
2094 #include <math.h>
2095 int yylex (void);
2096 void yyerror (char const *);
2097%@}
2098
2099/* Bison declarations. */
2100%token NUM
2101
2102%left '-' '+'
2103%left '*' '/'
2104%precedence NEG
2105%right '^'
2106
2107%% /* The grammar follows. */
2108@end example
2109
2110@noindent
2111Note there are no declarations specific to locations. Defining a data
2112type for storing locations is not needed: we will use the type provided
2113by default (@pxref{Location Type, ,Data Types of Locations}), which is a
2114four member structure with the following integer fields:
2115@code{first_line}, @code{first_column}, @code{last_line} and
2116@code{last_column}. By conventions, and in accordance with the GNU
2117Coding Standards and common practice, the line and column count both
2118start at 1.
2119
2120@node Ltcalc Rules
2121@subsection Grammar Rules for @code{ltcalc}
2122
2123Whether handling locations or not has no effect on the syntax of your
2124language. Therefore, grammar rules for this example will be very close
2125to those of the previous example: we will only modify them to benefit
2126from the new information.
2127
2128Here, we will use locations to report divisions by zero, and locate the
2129wrong expressions or subexpressions.
2130
2131@example
2132@group
2133input : /* empty */
2134 | input line
2135;
2136@end group
2137
2138@group
2139line : '\n'
2140 | exp '\n' @{ printf ("%d\n", $1); @}
2141;
2142@end group
2143
2144@group
2145exp : NUM @{ $$ = $1; @}
2146 | exp '+' exp @{ $$ = $1 + $3; @}
2147 | exp '-' exp @{ $$ = $1 - $3; @}
2148 | exp '*' exp @{ $$ = $1 * $3; @}
2149@end group
2150@group
2151 | exp '/' exp
2152 @{
2153 if ($3)
2154 $$ = $1 / $3;
2155 else
2156 @{
2157 $$ = 1;
2158 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2159 @@3.first_line, @@3.first_column,
2160 @@3.last_line, @@3.last_column);
2161 @}
2162 @}
2163@end group
2164@group
2165 | '-' exp %prec NEG @{ $$ = -$2; @}
2166 | exp '^' exp @{ $$ = pow ($1, $3); @}
2167 | '(' exp ')' @{ $$ = $2; @}
2168@end group
2169@end example
2170
2171This code shows how to reach locations inside of semantic actions, by
2172using the pseudo-variables @code{@@@var{n}} for rule components, and the
2173pseudo-variable @code{@@$} for groupings.
2174
2175We don't need to assign a value to @code{@@$}: the output parser does it
2176automatically. By default, before executing the C code of each action,
2177@code{@@$} is set to range from the beginning of @code{@@1} to the end
2178of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2179can be redefined (@pxref{Location Default Action, , Default Action for
2180Locations}), and for very specific rules, @code{@@$} can be computed by
2181hand.
2182
2183@node Ltcalc Lexer
2184@subsection The @code{ltcalc} Lexical Analyzer.
2185
2186Until now, we relied on Bison's defaults to enable location
2187tracking. The next step is to rewrite the lexical analyzer, and make it
2188able to feed the parser with the token locations, as it already does for
2189semantic values.
2190
2191To this end, we must take into account every single character of the
2192input text, to avoid the computed locations of being fuzzy or wrong:
2193
2194@example
2195@group
2196int
2197yylex (void)
2198@{
2199 int c;
2200@end group
2201
2202@group
2203 /* Skip white space. */
2204 while ((c = getchar ()) == ' ' || c == '\t')
2205 ++yylloc.last_column;
2206@end group
2207
2208@group
2209 /* Step. */
2210 yylloc.first_line = yylloc.last_line;
2211 yylloc.first_column = yylloc.last_column;
2212@end group
2213
2214@group
2215 /* Process numbers. */
2216 if (isdigit (c))
2217 @{
2218 yylval = c - '0';
2219 ++yylloc.last_column;
2220 while (isdigit (c = getchar ()))
2221 @{
2222 ++yylloc.last_column;
2223 yylval = yylval * 10 + c - '0';
2224 @}
2225 ungetc (c, stdin);
2226 return NUM;
2227 @}
2228@end group
2229
2230 /* Return end-of-input. */
2231 if (c == EOF)
2232 return 0;
2233
2234 /* Return a single char, and update location. */
2235 if (c == '\n')
2236 @{
2237 ++yylloc.last_line;
2238 yylloc.last_column = 0;
2239 @}
2240 else
2241 ++yylloc.last_column;
2242 return c;
2243@}
2244@end example
2245
2246Basically, the lexical analyzer performs the same processing as before:
2247it skips blanks and tabs, and reads numbers or single-character tokens.
2248In addition, it updates @code{yylloc}, the global variable (of type
2249@code{YYLTYPE}) containing the token's location.
2250
2251Now, each time this function returns a token, the parser has its number
2252as well as its semantic value, and its location in the text. The last
2253needed change is to initialize @code{yylloc}, for example in the
2254controlling function:
2255
2256@example
2257@group
2258int
2259main (void)
2260@{
2261 yylloc.first_line = yylloc.last_line = 1;
2262 yylloc.first_column = yylloc.last_column = 0;
2263 return yyparse ();
2264@}
2265@end group
2266@end example
2267
2268Remember that computing locations is not a matter of syntax. Every
2269character must be associated to a location update, whether it is in
2270valid input, in comments, in literal strings, and so on.
2271
2272@node Multi-function Calc
2273@section Multi-Function Calculator: @code{mfcalc}
2274@cindex multi-function calculator
2275@cindex @code{mfcalc}
2276@cindex calculator, multi-function
2277
2278Now that the basics of Bison have been discussed, it is time to move on to
2279a more advanced problem. The above calculators provided only five
2280functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2281be nice to have a calculator that provides other mathematical functions such
2282as @code{sin}, @code{cos}, etc.
2283
2284It is easy to add new operators to the infix calculator as long as they are
2285only single-character literals. The lexical analyzer @code{yylex} passes
2286back all nonnumeric characters as tokens, so new grammar rules suffice for
2287adding a new operator. But we want something more flexible: built-in
2288functions whose syntax has this form:
2289
2290@example
2291@var{function_name} (@var{argument})
2292@end example
2293
2294@noindent
2295At the same time, we will add memory to the calculator, by allowing you
2296to create named variables, store values in them, and use them later.
2297Here is a sample session with the multi-function calculator:
2298
2299@example
2300$ @kbd{mfcalc}
2301@kbd{pi = 3.141592653589}
23023.1415926536
2303@kbd{sin(pi)}
23040.0000000000
2305@kbd{alpha = beta1 = 2.3}
23062.3000000000
2307@kbd{alpha}
23082.3000000000
2309@kbd{ln(alpha)}
23100.8329091229
2311@kbd{exp(ln(beta1))}
23122.3000000000
2313$
2314@end example
2315
2316Note that multiple assignment and nested function calls are permitted.
2317
2318@menu
2319* Mfcalc Declarations:: Bison declarations for multi-function calculator.
2320* Mfcalc Rules:: Grammar rules for the calculator.
2321* Mfcalc Symbol Table:: Symbol table management subroutines.
2322@end menu
2323
2324@node Mfcalc Declarations
2325@subsection Declarations for @code{mfcalc}
2326
2327Here are the C and Bison declarations for the multi-function calculator.
2328
2329@smallexample
2330@group
2331%@{
2332 #include <math.h> /* For math functions, cos(), sin(), etc. */
2333 #include "calc.h" /* Contains definition of `symrec'. */
2334 int yylex (void);
2335 void yyerror (char const *);
2336%@}
2337@end group
2338@group
2339%union @{
2340 double val; /* For returning numbers. */
2341 symrec *tptr; /* For returning symbol-table pointers. */
2342@}
2343@end group
2344%token <val> NUM /* Simple double precision number. */
2345%token <tptr> VAR FNCT /* Variable and Function. */
2346%type <val> exp
2347
2348@group
2349%right '='
2350%left '-' '+'
2351%left '*' '/'
2352%precedence NEG /* negation--unary minus */
2353%right '^' /* exponentiation */
2354@end group
2355%% /* The grammar follows. */
2356@end smallexample
2357
2358The above grammar introduces only two new features of the Bison language.
2359These features allow semantic values to have various data types
2360(@pxref{Multiple Types, ,More Than One Value Type}).
2361
2362The @code{%union} declaration specifies the entire list of possible types;
2363this is instead of defining @code{YYSTYPE}. The allowable types are now
2364double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2365the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2366
2367Since values can now have various types, it is necessary to associate a
2368type with each grammar symbol whose semantic value is used. These symbols
2369are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2370declarations are augmented with information about their data type (placed
2371between angle brackets).
2372
2373The Bison construct @code{%type} is used for declaring nonterminal
2374symbols, just as @code{%token} is used for declaring token types. We
2375have not used @code{%type} before because nonterminal symbols are
2376normally declared implicitly by the rules that define them. But
2377@code{exp} must be declared explicitly so we can specify its value type.
2378@xref{Type Decl, ,Nonterminal Symbols}.
2379
2380@node Mfcalc Rules
2381@subsection Grammar Rules for @code{mfcalc}
2382
2383Here are the grammar rules for the multi-function calculator.
2384Most of them are copied directly from @code{calc}; three rules,
2385those which mention @code{VAR} or @code{FNCT}, are new.
2386
2387@smallexample
2388@group
2389input: /* empty */
2390 | input line
2391;
2392@end group
2393
2394@group
2395line:
2396 '\n'
2397 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2398 | error '\n' @{ yyerrok; @}
2399;
2400@end group
2401
2402@group
2403exp: NUM @{ $$ = $1; @}
2404 | VAR @{ $$ = $1->value.var; @}
2405 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2406 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2407 | exp '+' exp @{ $$ = $1 + $3; @}
2408 | exp '-' exp @{ $$ = $1 - $3; @}
2409 | exp '*' exp @{ $$ = $1 * $3; @}
2410 | exp '/' exp @{ $$ = $1 / $3; @}
2411 | '-' exp %prec NEG @{ $$ = -$2; @}
2412 | exp '^' exp @{ $$ = pow ($1, $3); @}
2413 | '(' exp ')' @{ $$ = $2; @}
2414;
2415@end group
2416/* End of grammar. */
2417%%
2418@end smallexample
2419
2420@node Mfcalc Symbol Table
2421@subsection The @code{mfcalc} Symbol Table
2422@cindex symbol table example
2423
2424The multi-function calculator requires a symbol table to keep track of the
2425names and meanings of variables and functions. This doesn't affect the
2426grammar rules (except for the actions) or the Bison declarations, but it
2427requires some additional C functions for support.
2428
2429The symbol table itself consists of a linked list of records. Its
2430definition, which is kept in the header @file{calc.h}, is as follows. It
2431provides for either functions or variables to be placed in the table.
2432
2433@smallexample
2434@group
2435/* Function type. */
2436typedef double (*func_t) (double);
2437@end group
2438
2439@group
2440/* Data type for links in the chain of symbols. */
2441struct symrec
2442@{
2443 char *name; /* name of symbol */
2444 int type; /* type of symbol: either VAR or FNCT */
2445 union
2446 @{
2447 double var; /* value of a VAR */
2448 func_t fnctptr; /* value of a FNCT */
2449 @} value;
2450 struct symrec *next; /* link field */
2451@};
2452@end group
2453
2454@group
2455typedef struct symrec symrec;
2456
2457/* The symbol table: a chain of `struct symrec'. */
2458extern symrec *sym_table;
2459
2460symrec *putsym (char const *, int);
2461symrec *getsym (char const *);
2462@end group
2463@end smallexample
2464
2465The new version of @code{main} includes a call to @code{init_table}, a
2466function that initializes the symbol table. Here it is, and
2467@code{init_table} as well:
2468
2469@smallexample
2470#include <stdio.h>
2471
2472@group
2473/* Called by yyparse on error. */
2474void
2475yyerror (char const *s)
2476@{
2477 printf ("%s\n", s);
2478@}
2479@end group
2480
2481@group
2482struct init
2483@{
2484 char const *fname;
2485 double (*fnct) (double);
2486@};
2487@end group
2488
2489@group
2490struct init const arith_fncts[] =
2491@{
2492 "sin", sin,
2493 "cos", cos,
2494 "atan", atan,
2495 "ln", log,
2496 "exp", exp,
2497 "sqrt", sqrt,
2498 0, 0
2499@};
2500@end group
2501
2502@group
2503/* The symbol table: a chain of `struct symrec'. */
2504symrec *sym_table;
2505@end group
2506
2507@group
2508/* Put arithmetic functions in table. */
2509void
2510init_table (void)
2511@{
2512 int i;
2513 symrec *ptr;
2514 for (i = 0; arith_fncts[i].fname != 0; i++)
2515 @{
2516 ptr = putsym (arith_fncts[i].fname, FNCT);
2517 ptr->value.fnctptr = arith_fncts[i].fnct;
2518 @}
2519@}
2520@end group
2521
2522@group
2523int
2524main (void)
2525@{
2526 init_table ();
2527 return yyparse ();
2528@}
2529@end group
2530@end smallexample
2531
2532By simply editing the initialization list and adding the necessary include
2533files, you can add additional functions to the calculator.
2534
2535Two important functions allow look-up and installation of symbols in the
2536symbol table. The function @code{putsym} is passed a name and the type
2537(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2538linked to the front of the list, and a pointer to the object is returned.
2539The function @code{getsym} is passed the name of the symbol to look up. If
2540found, a pointer to that symbol is returned; otherwise zero is returned.
2541
2542@smallexample
2543symrec *
2544putsym (char const *sym_name, int sym_type)
2545@{
2546 symrec *ptr;
2547 ptr = (symrec *) malloc (sizeof (symrec));
2548 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2549 strcpy (ptr->name,sym_name);
2550 ptr->type = sym_type;
2551 ptr->value.var = 0; /* Set value to 0 even if fctn. */
2552 ptr->next = (struct symrec *)sym_table;
2553 sym_table = ptr;
2554 return ptr;
2555@}
2556
2557symrec *
2558getsym (char const *sym_name)
2559@{
2560 symrec *ptr;
2561 for (ptr = sym_table; ptr != (symrec *) 0;
2562 ptr = (symrec *)ptr->next)
2563 if (strcmp (ptr->name,sym_name) == 0)
2564 return ptr;
2565 return 0;
2566@}
2567@end smallexample
2568
2569The function @code{yylex} must now recognize variables, numeric values, and
2570the single-character arithmetic operators. Strings of alphanumeric
2571characters with a leading letter are recognized as either variables or
2572functions depending on what the symbol table says about them.
2573
2574The string is passed to @code{getsym} for look up in the symbol table. If
2575the name appears in the table, a pointer to its location and its type
2576(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2577already in the table, then it is installed as a @code{VAR} using
2578@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
2579returned to @code{yyparse}.
2580
2581No change is needed in the handling of numeric values and arithmetic
2582operators in @code{yylex}.
2583
2584@smallexample
2585@group
2586#include <ctype.h>
2587@end group
2588
2589@group
2590int
2591yylex (void)
2592@{
2593 int c;
2594
2595 /* Ignore white space, get first nonwhite character. */
2596 while ((c = getchar ()) == ' ' || c == '\t');
2597
2598 if (c == EOF)
2599 return 0;
2600@end group
2601
2602@group
2603 /* Char starts a number => parse the number. */
2604 if (c == '.' || isdigit (c))
2605 @{
2606 ungetc (c, stdin);
2607 scanf ("%lf", &yylval.val);
2608 return NUM;
2609 @}
2610@end group
2611
2612@group
2613 /* Char starts an identifier => read the name. */
2614 if (isalpha (c))
2615 @{
2616 symrec *s;
2617 static char *symbuf = 0;
2618 static int length = 0;
2619 int i;
2620@end group
2621
2622@group
2623 /* Initially make the buffer long enough
2624 for a 40-character symbol name. */
2625 if (length == 0)
2626 length = 40, symbuf = (char *)malloc (length + 1);
2627
2628 i = 0;
2629 do
2630@end group
2631@group
2632 @{
2633 /* If buffer is full, make it bigger. */
2634 if (i == length)
2635 @{
2636 length *= 2;
2637 symbuf = (char *) realloc (symbuf, length + 1);
2638 @}
2639 /* Add this character to the buffer. */
2640 symbuf[i++] = c;
2641 /* Get another character. */
2642 c = getchar ();
2643 @}
2644@end group
2645@group
2646 while (isalnum (c));
2647
2648 ungetc (c, stdin);
2649 symbuf[i] = '\0';
2650@end group
2651
2652@group
2653 s = getsym (symbuf);
2654 if (s == 0)
2655 s = putsym (symbuf, VAR);
2656 yylval.tptr = s;
2657 return s->type;
2658 @}
2659
2660 /* Any other character is a token by itself. */
2661 return c;
2662@}
2663@end group
2664@end smallexample
2665
2666This program is both powerful and flexible. You may easily add new
2667functions, and it is a simple job to modify this code to install
2668predefined variables such as @code{pi} or @code{e} as well.
2669
2670@node Exercises
2671@section Exercises
2672@cindex exercises
2673
2674@enumerate
2675@item
2676Add some new functions from @file{math.h} to the initialization list.
2677
2678@item
2679Add another array that contains constants and their values. Then
2680modify @code{init_table} to add these constants to the symbol table.
2681It will be easiest to give the constants type @code{VAR}.
2682
2683@item
2684Make the program report an error if the user refers to an
2685uninitialized variable in any way except to store a value in it.
2686@end enumerate
2687
2688@node Grammar File
2689@chapter Bison Grammar Files
2690
2691Bison takes as input a context-free grammar specification and produces a
2692C-language function that recognizes correct instances of the grammar.
2693
2694The Bison grammar file conventionally has a name ending in @samp{.y}.
2695@xref{Invocation, ,Invoking Bison}.
2696
2697@menu
2698* Grammar Outline:: Overall layout of the grammar file.
2699* Symbols:: Terminal and nonterminal symbols.
2700* Rules:: How to write grammar rules.
2701* Recursion:: Writing recursive rules.
2702* Semantics:: Semantic values and actions.
2703* Locations:: Locations and actions.
2704* Declarations:: All kinds of Bison declarations are described here.
2705* Multiple Parsers:: Putting more than one Bison parser in one program.
2706@end menu
2707
2708@node Grammar Outline
2709@section Outline of a Bison Grammar
2710
2711A Bison grammar file has four main sections, shown here with the
2712appropriate delimiters:
2713
2714@example
2715%@{
2716 @var{Prologue}
2717%@}
2718
2719@var{Bison declarations}
2720
2721%%
2722@var{Grammar rules}
2723%%
2724
2725@var{Epilogue}
2726@end example
2727
2728Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2729As a GNU extension, @samp{//} introduces a comment that
2730continues until end of line.
2731
2732@menu
2733* Prologue:: Syntax and usage of the prologue.
2734* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
2735* Bison Declarations:: Syntax and usage of the Bison declarations section.
2736* Grammar Rules:: Syntax and usage of the grammar rules section.
2737* Epilogue:: Syntax and usage of the epilogue.
2738@end menu
2739
2740@node Prologue
2741@subsection The prologue
2742@cindex declarations section
2743@cindex Prologue
2744@cindex declarations
2745
2746The @var{Prologue} section contains macro definitions and declarations
2747of functions and variables that are used in the actions in the grammar
2748rules. These are copied to the beginning of the parser implementation
2749file so that they precede the definition of @code{yyparse}. You can
2750use @samp{#include} to get the declarations from a header file. If
2751you don't need any C declarations, you may omit the @samp{%@{} and
2752@samp{%@}} delimiters that bracket this section.
2753
2754The @var{Prologue} section is terminated by the first occurrence
2755of @samp{%@}} that is outside a comment, a string literal, or a
2756character constant.
2757
2758You may have more than one @var{Prologue} section, intermixed with the
2759@var{Bison declarations}. This allows you to have C and Bison
2760declarations that refer to each other. For example, the @code{%union}
2761declaration may use types defined in a header file, and you may wish to
2762prototype functions that take arguments of type @code{YYSTYPE}. This
2763can be done with two @var{Prologue} blocks, one before and one after the
2764@code{%union} declaration.
2765
2766@smallexample
2767%@{
2768 #define _GNU_SOURCE
2769 #include <stdio.h>
2770 #include "ptypes.h"
2771%@}
2772
2773%union @{
2774 long int n;
2775 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2776@}
2777
2778%@{
2779 static void print_token_value (FILE *, int, YYSTYPE);
2780 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2781%@}
2782
2783@dots{}
2784@end smallexample
2785
2786When in doubt, it is usually safer to put prologue code before all
2787Bison declarations, rather than after. For example, any definitions
2788of feature test macros like @code{_GNU_SOURCE} or
2789@code{_POSIX_C_SOURCE} should appear before all Bison declarations, as
2790feature test macros can affect the behavior of Bison-generated
2791@code{#include} directives.
2792
2793@node Prologue Alternatives
2794@subsection Prologue Alternatives
2795@cindex Prologue Alternatives
2796
2797@findex %code
2798@findex %code requires
2799@findex %code provides
2800@findex %code top
2801
2802The functionality of @var{Prologue} sections can often be subtle and
2803inflexible. As an alternative, Bison provides a @code{%code}
2804directive with an explicit qualifier field, which identifies the
2805purpose of the code and thus the location(s) where Bison should
2806generate it. For C/C++, the qualifier can be omitted for the default
2807location, or it can be one of @code{requires}, @code{provides},
2808@code{top}. @xref{%code Summary}.
2809
2810Look again at the example of the previous section:
2811
2812@smallexample
2813%@{
2814 #define _GNU_SOURCE
2815 #include <stdio.h>
2816 #include "ptypes.h"
2817%@}
2818
2819%union @{
2820 long int n;
2821 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2822@}
2823
2824%@{
2825 static void print_token_value (FILE *, int, YYSTYPE);
2826 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2827%@}
2828
2829@dots{}
2830@end smallexample
2831
2832@noindent
2833Notice that there are two @var{Prologue} sections here, but there's a
2834subtle distinction between their functionality. For example, if you
2835decide to override Bison's default definition for @code{YYLTYPE}, in
2836which @var{Prologue} section should you write your new definition?
2837You should write it in the first since Bison will insert that code
2838into the parser implementation file @emph{before} the default
2839@code{YYLTYPE} definition. In which @var{Prologue} section should you
2840prototype an internal function, @code{trace_token}, that accepts
2841@code{YYLTYPE} and @code{yytokentype} as arguments? You should
2842prototype it in the second since Bison will insert that code
2843@emph{after} the @code{YYLTYPE} and @code{yytokentype} definitions.
2844
2845This distinction in functionality between the two @var{Prologue} sections is
2846established by the appearance of the @code{%union} between them.
2847This behavior raises a few questions.
2848First, why should the position of a @code{%union} affect definitions related to
2849@code{YYLTYPE} and @code{yytokentype}?
2850Second, what if there is no @code{%union}?
2851In that case, the second kind of @var{Prologue} section is not available.
2852This behavior is not intuitive.
2853
2854To avoid this subtle @code{%union} dependency, rewrite the example using a
2855@code{%code top} and an unqualified @code{%code}.
2856Let's go ahead and add the new @code{YYLTYPE} definition and the
2857@code{trace_token} prototype at the same time:
2858
2859@smallexample
2860%code top @{
2861 #define _GNU_SOURCE
2862 #include <stdio.h>
2863
2864 /* WARNING: The following code really belongs
2865 * in a `%code requires'; see below. */
2866
2867 #include "ptypes.h"
2868 #define YYLTYPE YYLTYPE
2869 typedef struct YYLTYPE
2870 @{
2871 int first_line;
2872 int first_column;
2873 int last_line;
2874 int last_column;
2875 char *filename;
2876 @} YYLTYPE;
2877@}
2878
2879%union @{
2880 long int n;
2881 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2882@}
2883
2884%code @{
2885 static void print_token_value (FILE *, int, YYSTYPE);
2886 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2887 static void trace_token (enum yytokentype token, YYLTYPE loc);
2888@}
2889
2890@dots{}
2891@end smallexample
2892
2893@noindent
2894In this way, @code{%code top} and the unqualified @code{%code} achieve the same
2895functionality as the two kinds of @var{Prologue} sections, but it's always
2896explicit which kind you intend.
2897Moreover, both kinds are always available even in the absence of @code{%union}.
2898
2899The @code{%code top} block above logically contains two parts. The
2900first two lines before the warning need to appear near the top of the
2901parser implementation file. The first line after the warning is
2902required by @code{YYSTYPE} and thus also needs to appear in the parser
2903implementation file. However, if you've instructed Bison to generate
2904a parser header file (@pxref{Decl Summary, ,%defines}), you probably
2905want that line to appear before the @code{YYSTYPE} definition in that
2906header file as well. The @code{YYLTYPE} definition should also appear
2907in the parser header file to override the default @code{YYLTYPE}
2908definition there.
2909
2910In other words, in the @code{%code top} block above, all but the first two
2911lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
2912definitions.
2913Thus, they belong in one or more @code{%code requires}:
2914
2915@smallexample
2916%code top @{
2917 #define _GNU_SOURCE
2918 #include <stdio.h>
2919@}
2920
2921%code requires @{
2922 #include "ptypes.h"
2923@}
2924%union @{
2925 long int n;
2926 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2927@}
2928
2929%code requires @{
2930 #define YYLTYPE YYLTYPE
2931 typedef struct YYLTYPE
2932 @{
2933 int first_line;
2934 int first_column;
2935 int last_line;
2936 int last_column;
2937 char *filename;
2938 @} YYLTYPE;
2939@}
2940
2941%code @{
2942 static void print_token_value (FILE *, int, YYSTYPE);
2943 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2944 static void trace_token (enum yytokentype token, YYLTYPE loc);
2945@}
2946
2947@dots{}
2948@end smallexample
2949
2950@noindent
2951Now Bison will insert @code{#include "ptypes.h"} and the new
2952@code{YYLTYPE} definition before the Bison-generated @code{YYSTYPE}
2953and @code{YYLTYPE} definitions in both the parser implementation file
2954and the parser header file. (By the same reasoning, @code{%code
2955requires} would also be the appropriate place to write your own
2956definition for @code{YYSTYPE}.)
2957
2958When you are writing dependency code for @code{YYSTYPE} and
2959@code{YYLTYPE}, you should prefer @code{%code requires} over
2960@code{%code top} regardless of whether you instruct Bison to generate
2961a parser header file. When you are writing code that you need Bison
2962to insert only into the parser implementation file and that has no
2963special need to appear at the top of that file, you should prefer the
2964unqualified @code{%code} over @code{%code top}. These practices will
2965make the purpose of each block of your code explicit to Bison and to
2966other developers reading your grammar file. Following these
2967practices, we expect the unqualified @code{%code} and @code{%code
2968requires} to be the most important of the four @var{Prologue}
2969alternatives.
2970
2971At some point while developing your parser, you might decide to
2972provide @code{trace_token} to modules that are external to your
2973parser. Thus, you might wish for Bison to insert the prototype into
2974both the parser header file and the parser implementation file. Since
2975this function is not a dependency required by @code{YYSTYPE} or
2976@code{YYLTYPE}, it doesn't make sense to move its prototype to a
2977@code{%code requires}. More importantly, since it depends upon
2978@code{YYLTYPE} and @code{yytokentype}, @code{%code requires} is not
2979sufficient. Instead, move its prototype from the unqualified
2980@code{%code} to a @code{%code provides}:
2981
2982@smallexample
2983%code top @{
2984 #define _GNU_SOURCE
2985 #include <stdio.h>
2986@}
2987
2988%code requires @{
2989 #include "ptypes.h"
2990@}
2991%union @{
2992 long int n;
2993 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2994@}
2995
2996%code requires @{
2997 #define YYLTYPE YYLTYPE
2998 typedef struct YYLTYPE
2999 @{
3000 int first_line;
3001 int first_column;
3002 int last_line;
3003 int last_column;
3004 char *filename;
3005 @} YYLTYPE;
3006@}
3007
3008%code provides @{
3009 void trace_token (enum yytokentype token, YYLTYPE loc);
3010@}
3011
3012%code @{
3013 static void print_token_value (FILE *, int, YYSTYPE);
3014 #define YYPRINT(F, N, L) print_token_value (F, N, L)
3015@}
3016
3017@dots{}
3018@end smallexample
3019
3020@noindent
3021Bison will insert the @code{trace_token} prototype into both the
3022parser header file and the parser implementation file after the
3023definitions for @code{yytokentype}, @code{YYLTYPE}, and
3024@code{YYSTYPE}.
3025
3026The above examples are careful to write directives in an order that
3027reflects the layout of the generated parser implementation and header
3028files: @code{%code top}, @code{%code requires}, @code{%code provides},
3029and then @code{%code}. While your grammar files may generally be
3030easier to read if you also follow this order, Bison does not require
3031it. Instead, Bison lets you choose an organization that makes sense
3032to you.
3033
3034You may declare any of these directives multiple times in the grammar file.
3035In that case, Bison concatenates the contained code in declaration order.
3036This is the only way in which the position of one of these directives within
3037the grammar file affects its functionality.
3038
3039The result of the previous two properties is greater flexibility in how you may
3040organize your grammar file.
3041For example, you may organize semantic-type-related directives by semantic
3042type:
3043
3044@smallexample
3045%code requires @{ #include "type1.h" @}
3046%union @{ type1 field1; @}
3047%destructor @{ type1_free ($$); @} <field1>
3048%printer @{ type1_print ($$); @} <field1>
3049
3050%code requires @{ #include "type2.h" @}
3051%union @{ type2 field2; @}
3052%destructor @{ type2_free ($$); @} <field2>
3053%printer @{ type2_print ($$); @} <field2>
3054@end smallexample
3055
3056@noindent
3057You could even place each of the above directive groups in the rules section of
3058the grammar file next to the set of rules that uses the associated semantic
3059type.
3060(In the rules section, you must terminate each of those directives with a
3061semicolon.)
3062And you don't have to worry that some directive (like a @code{%union}) in the
3063definitions section is going to adversely affect their functionality in some
3064counter-intuitive manner just because it comes first.
3065Such an organization is not possible using @var{Prologue} sections.
3066
3067This section has been concerned with explaining the advantages of the four
3068@var{Prologue} alternatives over the original Yacc @var{Prologue}.
3069However, in most cases when using these directives, you shouldn't need to
3070think about all the low-level ordering issues discussed here.
3071Instead, you should simply use these directives to label each block of your
3072code according to its purpose and let Bison handle the ordering.
3073@code{%code} is the most generic label.
3074Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
3075as needed.
3076
3077@node Bison Declarations
3078@subsection The Bison Declarations Section
3079@cindex Bison declarations (introduction)
3080@cindex declarations, Bison (introduction)
3081
3082The @var{Bison declarations} section contains declarations that define
3083terminal and nonterminal symbols, specify precedence, and so on.
3084In some simple grammars you may not need any declarations.
3085@xref{Declarations, ,Bison Declarations}.
3086
3087@node Grammar Rules
3088@subsection The Grammar Rules Section
3089@cindex grammar rules section
3090@cindex rules section for grammar
3091
3092The @dfn{grammar rules} section contains one or more Bison grammar
3093rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
3094
3095There must always be at least one grammar rule, and the first
3096@samp{%%} (which precedes the grammar rules) may never be omitted even
3097if it is the first thing in the file.
3098
3099@node Epilogue
3100@subsection The epilogue
3101@cindex additional C code section
3102@cindex epilogue
3103@cindex C code, section for additional
3104
3105The @var{Epilogue} is copied verbatim to the end of the parser
3106implementation file, just as the @var{Prologue} is copied to the
3107beginning. This is the most convenient place to put anything that you
3108want to have in the parser implementation file but which need not come
3109before the definition of @code{yyparse}. For example, the definitions
3110of @code{yylex} and @code{yyerror} often go here. Because C requires
3111functions to be declared before being used, you often need to declare
3112functions like @code{yylex} and @code{yyerror} in the Prologue, even
3113if you define them in the Epilogue. @xref{Interface, ,Parser
3114C-Language Interface}.
3115
3116If the last section is empty, you may omit the @samp{%%} that separates it
3117from the grammar rules.
3118
3119The Bison parser itself contains many macros and identifiers whose names
3120start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3121any such names (except those documented in this manual) in the epilogue
3122of the grammar file.
3123
3124@node Symbols
3125@section Symbols, Terminal and Nonterminal
3126@cindex nonterminal symbol
3127@cindex terminal symbol
3128@cindex token type
3129@cindex symbol
3130
3131@dfn{Symbols} in Bison grammars represent the grammatical classifications
3132of the language.
3133
3134A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
3135class of syntactically equivalent tokens. You use the symbol in grammar
3136rules to mean that a token in that class is allowed. The symbol is
3137represented in the Bison parser by a numeric code, and the @code{yylex}
3138function returns a token type code to indicate what kind of token has
3139been read. You don't need to know what the code value is; you can use
3140the symbol to stand for it.
3141
3142A @dfn{nonterminal symbol} stands for a class of syntactically
3143equivalent groupings. The symbol name is used in writing grammar rules.
3144By convention, it should be all lower case.
3145
3146Symbol names can contain letters, underscores, periods, and non-initial
3147digits and dashes. Dashes in symbol names are a GNU extension, incompatible
3148with POSIX Yacc. Periods and dashes make symbol names less convenient to
3149use with named references, which require brackets around such names
3150(@pxref{Named References}). Terminal symbols that contain periods or dashes
3151make little sense: since they are not valid symbols (in most programming
3152languages) they are not exported as token names.
3153
3154There are three ways of writing terminal symbols in the grammar:
3155
3156@itemize @bullet
3157@item
3158A @dfn{named token type} is written with an identifier, like an
3159identifier in C@. By convention, it should be all upper case. Each
3160such name must be defined with a Bison declaration such as
3161@code{%token}. @xref{Token Decl, ,Token Type Names}.
3162
3163@item
3164@cindex character token
3165@cindex literal token
3166@cindex single-character literal
3167A @dfn{character token type} (or @dfn{literal character token}) is
3168written in the grammar using the same syntax used in C for character
3169constants; for example, @code{'+'} is a character token type. A
3170character token type doesn't need to be declared unless you need to
3171specify its semantic value data type (@pxref{Value Type, ,Data Types of
3172Semantic Values}), associativity, or precedence (@pxref{Precedence,
3173,Operator Precedence}).
3174
3175By convention, a character token type is used only to represent a
3176token that consists of that particular character. Thus, the token
3177type @code{'+'} is used to represent the character @samp{+} as a
3178token. Nothing enforces this convention, but if you depart from it,
3179your program will confuse other readers.
3180
3181All the usual escape sequences used in character literals in C can be
3182used in Bison as well, but you must not use the null character as a
3183character literal because its numeric code, zero, signifies
3184end-of-input (@pxref{Calling Convention, ,Calling Convention
3185for @code{yylex}}). Also, unlike standard C, trigraphs have no
3186special meaning in Bison character literals, nor is backslash-newline
3187allowed.
3188
3189@item
3190@cindex string token
3191@cindex literal string token
3192@cindex multicharacter literal
3193A @dfn{literal string token} is written like a C string constant; for
3194example, @code{"<="} is a literal string token. A literal string token
3195doesn't need to be declared unless you need to specify its semantic
3196value data type (@pxref{Value Type}), associativity, or precedence
3197(@pxref{Precedence}).
3198
3199You can associate the literal string token with a symbolic name as an
3200alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
3201Declarations}). If you don't do that, the lexical analyzer has to
3202retrieve the token number for the literal string token from the
3203@code{yytname} table (@pxref{Calling Convention}).
3204
3205@strong{Warning}: literal string tokens do not work in Yacc.
3206
3207By convention, a literal string token is used only to represent a token
3208that consists of that particular string. Thus, you should use the token
3209type @code{"<="} to represent the string @samp{<=} as a token. Bison
3210does not enforce this convention, but if you depart from it, people who
3211read your program will be confused.
3212
3213All the escape sequences used in string literals in C can be used in
3214Bison as well, except that you must not use a null character within a
3215string literal. Also, unlike Standard C, trigraphs have no special
3216meaning in Bison string literals, nor is backslash-newline allowed. A
3217literal string token must contain two or more characters; for a token
3218containing just one character, use a character token (see above).
3219@end itemize
3220
3221How you choose to write a terminal symbol has no effect on its
3222grammatical meaning. That depends only on where it appears in rules and
3223on when the parser function returns that symbol.
3224
3225The value returned by @code{yylex} is always one of the terminal
3226symbols, except that a zero or negative value signifies end-of-input.
3227Whichever way you write the token type in the grammar rules, you write
3228it the same way in the definition of @code{yylex}. The numeric code
3229for a character token type is simply the positive numeric code of the
3230character, so @code{yylex} can use the identical value to generate the
3231requisite code, though you may need to convert it to @code{unsigned
3232char} to avoid sign-extension on hosts where @code{char} is signed.
3233Each named token type becomes a C macro in the parser implementation
3234file, so @code{yylex} can use the name to stand for the code. (This
3235is why periods don't make sense in terminal symbols.) @xref{Calling
3236Convention, ,Calling Convention for @code{yylex}}.
3237
3238If @code{yylex} is defined in a separate file, you need to arrange for the
3239token-type macro definitions to be available there. Use the @samp{-d}
3240option when you run Bison, so that it will write these macro definitions
3241into a separate header file @file{@var{name}.tab.h} which you can include
3242in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
3243
3244If you want to write a grammar that is portable to any Standard C
3245host, you must use only nonnull character tokens taken from the basic
3246execution character set of Standard C@. This set consists of the ten
3247digits, the 52 lower- and upper-case English letters, and the
3248characters in the following C-language string:
3249
3250@example
3251"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3252@end example
3253
3254The @code{yylex} function and Bison must use a consistent character set
3255and encoding for character tokens. For example, if you run Bison in an
3256ASCII environment, but then compile and run the resulting
3257program in an environment that uses an incompatible character set like
3258EBCDIC, the resulting program may not work because the tables
3259generated by Bison will assume ASCII numeric values for
3260character tokens. It is standard practice for software distributions to
3261contain C source files that were generated by Bison in an
3262ASCII environment, so installers on platforms that are
3263incompatible with ASCII must rebuild those files before
3264compiling them.
3265
3266The symbol @code{error} is a terminal symbol reserved for error recovery
3267(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
3268In particular, @code{yylex} should never return this value. The default
3269value of the error token is 256, unless you explicitly assigned 256 to
3270one of your tokens with a @code{%token} declaration.
3271
3272@node Rules
3273@section Syntax of Grammar Rules
3274@cindex rule syntax
3275@cindex grammar rule syntax
3276@cindex syntax of grammar rules
3277
3278A Bison grammar rule has the following general form:
3279
3280@example
3281@group
3282@var{result}: @var{components}@dots{}
3283 ;
3284@end group
3285@end example
3286
3287@noindent
3288where @var{result} is the nonterminal symbol that this rule describes,
3289and @var{components} are various terminal and nonterminal symbols that
3290are put together by this rule (@pxref{Symbols}).
3291
3292For example,
3293
3294@example
3295@group
3296exp: exp '+' exp
3297 ;
3298@end group
3299@end example
3300
3301@noindent
3302says that two groupings of type @code{exp}, with a @samp{+} token in between,
3303can be combined into a larger grouping of type @code{exp}.
3304
3305White space in rules is significant only to separate symbols. You can add
3306extra white space as you wish.
3307
3308Scattered among the components can be @var{actions} that determine
3309the semantics of the rule. An action looks like this:
3310
3311@example
3312@{@var{C statements}@}
3313@end example
3314
3315@noindent
3316@cindex braced code
3317This is an example of @dfn{braced code}, that is, C code surrounded by
3318braces, much like a compound statement in C@. Braced code can contain
3319any sequence of C tokens, so long as its braces are balanced. Bison
3320does not check the braced code for correctness directly; it merely
3321copies the code to the parser implementation file, where the C
3322compiler can check it.
3323
3324Within braced code, the balanced-brace count is not affected by braces
3325within comments, string literals, or character constants, but it is
3326affected by the C digraphs @samp{<%} and @samp{%>} that represent
3327braces. At the top level braced code must be terminated by @samp{@}}
3328and not by a digraph. Bison does not look for trigraphs, so if braced
3329code uses trigraphs you should ensure that they do not affect the
3330nesting of braces or the boundaries of comments, string literals, or
3331character constants.
3332
3333Usually there is only one action and it follows the components.
3334@xref{Actions}.
3335
3336@findex |
3337Multiple rules for the same @var{result} can be written separately or can
3338be joined with the vertical-bar character @samp{|} as follows:
3339
3340@example
3341@group
3342@var{result}: @var{rule1-components}@dots{}
3343 | @var{rule2-components}@dots{}
3344 @dots{}
3345 ;
3346@end group
3347@end example
3348
3349@noindent
3350They are still considered distinct rules even when joined in this way.
3351
3352If @var{components} in a rule is empty, it means that @var{result} can
3353match the empty string. For example, here is how to define a
3354comma-separated sequence of zero or more @code{exp} groupings:
3355
3356@example
3357@group
3358expseq: /* empty */
3359 | expseq1
3360 ;
3361@end group
3362
3363@group
3364expseq1: exp
3365 | expseq1 ',' exp
3366 ;
3367@end group
3368@end example
3369
3370@noindent
3371It is customary to write a comment @samp{/* empty */} in each rule
3372with no components.
3373
3374@node Recursion
3375@section Recursive Rules
3376@cindex recursive rule
3377
3378A rule is called @dfn{recursive} when its @var{result} nonterminal
3379appears also on its right hand side. Nearly all Bison grammars need to
3380use recursion, because that is the only way to define a sequence of any
3381number of a particular thing. Consider this recursive definition of a
3382comma-separated sequence of one or more expressions:
3383
3384@example
3385@group
3386expseq1: exp
3387 | expseq1 ',' exp
3388 ;
3389@end group
3390@end example
3391
3392@cindex left recursion
3393@cindex right recursion
3394@noindent
3395Since the recursive use of @code{expseq1} is the leftmost symbol in the
3396right hand side, we call this @dfn{left recursion}. By contrast, here
3397the same construct is defined using @dfn{right recursion}:
3398
3399@example
3400@group
3401expseq1: exp
3402 | exp ',' expseq1
3403 ;
3404@end group
3405@end example
3406
3407@noindent
3408Any kind of sequence can be defined using either left recursion or right
3409recursion, but you should always use left recursion, because it can
3410parse a sequence of any number of elements with bounded stack space.
3411Right recursion uses up space on the Bison stack in proportion to the
3412number of elements in the sequence, because all the elements must be
3413shifted onto the stack before the rule can be applied even once.
3414@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3415of this.
3416
3417@cindex mutual recursion
3418@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3419rule does not appear directly on its right hand side, but does appear
3420in rules for other nonterminals which do appear on its right hand
3421side.
3422
3423For example:
3424
3425@example
3426@group
3427expr: primary
3428 | primary '+' primary
3429 ;
3430@end group
3431
3432@group
3433primary: constant
3434 | '(' expr ')'
3435 ;
3436@end group
3437@end example
3438
3439@noindent
3440defines two mutually-recursive nonterminals, since each refers to the
3441other.
3442
3443@node Semantics
3444@section Defining Language Semantics
3445@cindex defining language semantics
3446@cindex language semantics, defining
3447
3448The grammar rules for a language determine only the syntax. The semantics
3449are determined by the semantic values associated with various tokens and
3450groupings, and by the actions taken when various groupings are recognized.
3451
3452For example, the calculator calculates properly because the value
3453associated with each expression is the proper number; it adds properly
3454because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3455the numbers associated with @var{x} and @var{y}.
3456
3457@menu
3458* Value Type:: Specifying one data type for all semantic values.
3459* Multiple Types:: Specifying several alternative data types.
3460* Actions:: An action is the semantic definition of a grammar rule.
3461* Action Types:: Specifying data types for actions to operate on.
3462* Mid-Rule Actions:: Most actions go at the end of a rule.
3463 This says when, why and how to use the exceptional
3464 action in the middle of a rule.
3465* Named References:: Using named references in actions.
3466@end menu
3467
3468@node Value Type
3469@subsection Data Types of Semantic Values
3470@cindex semantic value type
3471@cindex value type, semantic
3472@cindex data types of semantic values
3473@cindex default data type
3474
3475In a simple program it may be sufficient to use the same data type for
3476the semantic values of all language constructs. This was true in the
3477RPN and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
3478Notation Calculator}).
3479
3480Bison normally uses the type @code{int} for semantic values if your
3481program uses the same data type for all language constructs. To
3482specify some other type, define @code{YYSTYPE} as a macro, like this:
3483
3484@example
3485#define YYSTYPE double
3486@end example
3487
3488@noindent
3489@code{YYSTYPE}'s replacement list should be a type name
3490that does not contain parentheses or square brackets.
3491This macro definition must go in the prologue of the grammar file
3492(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
3493
3494@node Multiple Types
3495@subsection More Than One Value Type
3496
3497In most programs, you will need different data types for different kinds
3498of tokens and groupings. For example, a numeric constant may need type
3499@code{int} or @code{long int}, while a string constant needs type
3500@code{char *}, and an identifier might need a pointer to an entry in the
3501symbol table.
3502
3503To use more than one data type for semantic values in one parser, Bison
3504requires you to do two things:
3505
3506@itemize @bullet
3507@item
3508Specify the entire collection of possible data types, either by using the
3509@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
3510Value Types}), or by using a @code{typedef} or a @code{#define} to
3511define @code{YYSTYPE} to be a union type whose member names are
3512the type tags.
3513
3514@item
3515Choose one of those types for each symbol (terminal or nonterminal) for
3516which semantic values are used. This is done for tokens with the
3517@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3518and for groupings with the @code{%type} Bison declaration (@pxref{Type
3519Decl, ,Nonterminal Symbols}).
3520@end itemize
3521
3522@node Actions
3523@subsection Actions
3524@cindex action
3525@vindex $$
3526@vindex $@var{n}
3527@vindex $@var{name}
3528@vindex $[@var{name}]
3529
3530An action accompanies a syntactic rule and contains C code to be executed
3531each time an instance of that rule is recognized. The task of most actions
3532is to compute a semantic value for the grouping built by the rule from the
3533semantic values associated with tokens or smaller groupings.
3534
3535An action consists of braced code containing C statements, and can be
3536placed at any position in the rule;
3537it is executed at that position. Most rules have just one action at the
3538end of the rule, following all the components. Actions in the middle of
3539a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3540Actions, ,Actions in Mid-Rule}).
3541
3542The C code in an action can refer to the semantic values of the
3543components matched by the rule with the construct @code{$@var{n}},
3544which stands for the value of the @var{n}th component. The semantic
3545value for the grouping being constructed is @code{$$}. In addition,
3546the semantic values of symbols can be accessed with the named
3547references construct @code{$@var{name}} or @code{$[@var{name}]}.
3548Bison translates both of these constructs into expressions of the
3549appropriate type when it copies the actions into the parser
3550implementation file. @code{$$} (or @code{$@var{name}}, when it stands
3551for the current grouping) is translated to a modifiable lvalue, so it
3552can be assigned to.
3553
3554Here is a typical example:
3555
3556@example
3557@group
3558exp: @dots{}
3559 | exp '+' exp
3560 @{ $$ = $1 + $3; @}
3561@end group
3562@end example
3563
3564Or, in terms of named references:
3565
3566@example
3567@group
3568exp[result]: @dots{}
3569 | exp[left] '+' exp[right]
3570 @{ $result = $left + $right; @}
3571@end group
3572@end example
3573
3574@noindent
3575This rule constructs an @code{exp} from two smaller @code{exp} groupings
3576connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3577(@code{$left} and @code{$right})
3578refer to the semantic values of the two component @code{exp} groupings,
3579which are the first and third symbols on the right hand side of the rule.
3580The sum is stored into @code{$$} (@code{$result}) so that it becomes the
3581semantic value of
3582the addition-expression just recognized by the rule. If there were a
3583useful semantic value associated with the @samp{+} token, it could be
3584referred to as @code{$2}.
3585
3586@xref{Named References,,Using Named References}, for more information
3587about using the named references construct.
3588
3589Note that the vertical-bar character @samp{|} is really a rule
3590separator, and actions are attached to a single rule. This is a
3591difference with tools like Flex, for which @samp{|} stands for either
3592``or'', or ``the same action as that of the next rule''. In the
3593following example, the action is triggered only when @samp{b} is found:
3594
3595@example
3596@group
3597a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3598@end group
3599@end example
3600
3601@cindex default action
3602If you don't specify an action for a rule, Bison supplies a default:
3603@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3604becomes the value of the whole rule. Of course, the default action is
3605valid only if the two data types match. There is no meaningful default
3606action for an empty rule; every empty rule must have an explicit action
3607unless the rule's value does not matter.
3608
3609@code{$@var{n}} with @var{n} zero or negative is allowed for reference
3610to tokens and groupings on the stack @emph{before} those that match the
3611current rule. This is a very risky practice, and to use it reliably
3612you must be certain of the context in which the rule is applied. Here
3613is a case in which you can use this reliably:
3614
3615@example
3616@group
3617foo: expr bar '+' expr @{ @dots{} @}
3618 | expr bar '-' expr @{ @dots{} @}
3619 ;
3620@end group
3621
3622@group
3623bar: /* empty */
3624 @{ previous_expr = $0; @}
3625 ;
3626@end group
3627@end example
3628
3629As long as @code{bar} is used only in the fashion shown here, @code{$0}
3630always refers to the @code{expr} which precedes @code{bar} in the
3631definition of @code{foo}.
3632
3633@vindex yylval
3634It is also possible to access the semantic value of the lookahead token, if
3635any, from a semantic action.
3636This semantic value is stored in @code{yylval}.
3637@xref{Action Features, ,Special Features for Use in Actions}.
3638
3639@node Action Types
3640@subsection Data Types of Values in Actions
3641@cindex action data types
3642@cindex data types in actions
3643
3644If you have chosen a single data type for semantic values, the @code{$$}
3645and @code{$@var{n}} constructs always have that data type.
3646
3647If you have used @code{%union} to specify a variety of data types, then you
3648must declare a choice among these types for each terminal or nonterminal
3649symbol that can have a semantic value. Then each time you use @code{$$} or
3650@code{$@var{n}}, its data type is determined by which symbol it refers to
3651in the rule. In this example,
3652
3653@example
3654@group
3655exp: @dots{}
3656 | exp '+' exp
3657 @{ $$ = $1 + $3; @}
3658@end group
3659@end example
3660
3661@noindent
3662@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3663have the data type declared for the nonterminal symbol @code{exp}. If
3664@code{$2} were used, it would have the data type declared for the
3665terminal symbol @code{'+'}, whatever that might be.
3666
3667Alternatively, you can specify the data type when you refer to the value,
3668by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3669reference. For example, if you have defined types as shown here:
3670
3671@example
3672@group
3673%union @{
3674 int itype;
3675 double dtype;
3676@}
3677@end group
3678@end example
3679
3680@noindent
3681then you can write @code{$<itype>1} to refer to the first subunit of the
3682rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3683
3684@node Mid-Rule Actions
3685@subsection Actions in Mid-Rule
3686@cindex actions in mid-rule
3687@cindex mid-rule actions
3688
3689Occasionally it is useful to put an action in the middle of a rule.
3690These actions are written just like usual end-of-rule actions, but they
3691are executed before the parser even recognizes the following components.
3692
3693A mid-rule action may refer to the components preceding it using
3694@code{$@var{n}}, but it may not refer to subsequent components because
3695it is run before they are parsed.
3696
3697The mid-rule action itself counts as one of the components of the rule.
3698This makes a difference when there is another action later in the same rule
3699(and usually there is another at the end): you have to count the actions
3700along with the symbols when working out which number @var{n} to use in
3701@code{$@var{n}}.
3702
3703The mid-rule action can also have a semantic value. The action can set
3704its value with an assignment to @code{$$}, and actions later in the rule
3705can refer to the value using @code{$@var{n}}. Since there is no symbol
3706to name the action, there is no way to declare a data type for the value
3707in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3708specify a data type each time you refer to this value.
3709
3710There is no way to set the value of the entire rule with a mid-rule
3711action, because assignments to @code{$$} do not have that effect. The
3712only way to set the value for the entire rule is with an ordinary action
3713at the end of the rule.
3714
3715Here is an example from a hypothetical compiler, handling a @code{let}
3716statement that looks like @samp{let (@var{variable}) @var{statement}} and
3717serves to create a variable named @var{variable} temporarily for the
3718duration of @var{statement}. To parse this construct, we must put
3719@var{variable} into the symbol table while @var{statement} is parsed, then
3720remove it afterward. Here is how it is done:
3721
3722@example
3723@group
3724stmt: LET '(' var ')'
3725 @{ $<context>$ = push_context ();
3726 declare_variable ($3); @}
3727 stmt @{ $$ = $6;
3728 pop_context ($<context>5); @}
3729@end group
3730@end example
3731
3732@noindent
3733As soon as @samp{let (@var{variable})} has been recognized, the first
3734action is run. It saves a copy of the current semantic context (the
3735list of accessible variables) as its semantic value, using alternative
3736@code{context} in the data-type union. Then it calls
3737@code{declare_variable} to add the new variable to that list. Once the
3738first action is finished, the embedded statement @code{stmt} can be
3739parsed. Note that the mid-rule action is component number 5, so the
3740@samp{stmt} is component number 6.
3741
3742After the embedded statement is parsed, its semantic value becomes the
3743value of the entire @code{let}-statement. Then the semantic value from the
3744earlier action is used to restore the prior list of variables. This
3745removes the temporary @code{let}-variable from the list so that it won't
3746appear to exist while the rest of the program is parsed.
3747
3748@findex %destructor
3749@cindex discarded symbols, mid-rule actions
3750@cindex error recovery, mid-rule actions
3751In the above example, if the parser initiates error recovery (@pxref{Error
3752Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3753it might discard the previous semantic context @code{$<context>5} without
3754restoring it.
3755Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3756Discarded Symbols}).
3757However, Bison currently provides no means to declare a destructor specific to
3758a particular mid-rule action's semantic value.
3759
3760One solution is to bury the mid-rule action inside a nonterminal symbol and to
3761declare a destructor for that symbol:
3762
3763@example
3764@group
3765%type <context> let
3766%destructor @{ pop_context ($$); @} let
3767
3768%%
3769
3770stmt: let stmt
3771 @{ $$ = $2;
3772 pop_context ($1); @}
3773 ;
3774
3775let: LET '(' var ')'
3776 @{ $$ = push_context ();
3777 declare_variable ($3); @}
3778 ;
3779
3780@end group
3781@end example
3782
3783@noindent
3784Note that the action is now at the end of its rule.
3785Any mid-rule action can be converted to an end-of-rule action in this way, and
3786this is what Bison actually does to implement mid-rule actions.
3787
3788Taking action before a rule is completely recognized often leads to
3789conflicts since the parser must commit to a parse in order to execute the
3790action. For example, the following two rules, without mid-rule actions,
3791can coexist in a working parser because the parser can shift the open-brace
3792token and look at what follows before deciding whether there is a
3793declaration or not:
3794
3795@example
3796@group
3797compound: '@{' declarations statements '@}'
3798 | '@{' statements '@}'
3799 ;
3800@end group
3801@end example
3802
3803@noindent
3804But when we add a mid-rule action as follows, the rules become nonfunctional:
3805
3806@example
3807@group
3808compound: @{ prepare_for_local_variables (); @}
3809 '@{' declarations statements '@}'
3810@end group
3811@group
3812 | '@{' statements '@}'
3813 ;
3814@end group
3815@end example
3816
3817@noindent
3818Now the parser is forced to decide whether to run the mid-rule action
3819when it has read no farther than the open-brace. In other words, it
3820must commit to using one rule or the other, without sufficient
3821information to do it correctly. (The open-brace token is what is called
3822the @dfn{lookahead} token at this time, since the parser is still
3823deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.)
3824
3825You might think that you could correct the problem by putting identical
3826actions into the two rules, like this:
3827
3828@example
3829@group
3830compound: @{ prepare_for_local_variables (); @}
3831 '@{' declarations statements '@}'
3832 | @{ prepare_for_local_variables (); @}
3833 '@{' statements '@}'
3834 ;
3835@end group
3836@end example
3837
3838@noindent
3839But this does not help, because Bison does not realize that the two actions
3840are identical. (Bison never tries to understand the C code in an action.)
3841
3842If the grammar is such that a declaration can be distinguished from a
3843statement by the first token (which is true in C), then one solution which
3844does work is to put the action after the open-brace, like this:
3845
3846@example
3847@group
3848compound: '@{' @{ prepare_for_local_variables (); @}
3849 declarations statements '@}'
3850 | '@{' statements '@}'
3851 ;
3852@end group
3853@end example
3854
3855@noindent
3856Now the first token of the following declaration or statement,
3857which would in any case tell Bison which rule to use, can still do so.
3858
3859Another solution is to bury the action inside a nonterminal symbol which
3860serves as a subroutine:
3861
3862@example
3863@group
3864subroutine: /* empty */
3865 @{ prepare_for_local_variables (); @}
3866 ;
3867
3868@end group
3869
3870@group
3871compound: subroutine
3872 '@{' declarations statements '@}'
3873 | subroutine
3874 '@{' statements '@}'
3875 ;
3876@end group
3877@end example
3878
3879@noindent
3880Now Bison can execute the action in the rule for @code{subroutine} without
3881deciding which rule for @code{compound} it will eventually use.
3882
3883@node Named References
3884@subsection Using Named References
3885@cindex named references
3886
3887While every semantic value can be accessed with positional references
3888@code{$@var{n}} and @code{$$}, it's often much more convenient to refer to
3889them by name. First of all, original symbol names may be used as named
3890references. For example:
3891
3892@example
3893@group
3894invocation: op '(' args ')'
3895 @{ $invocation = new_invocation ($op, $args, @@invocation); @}
3896@end group
3897@end example
3898
3899@noindent
3900The positional @code{$$}, @code{@@$}, @code{$n}, and @code{@@n} can be
3901mixed with @code{$name} and @code{@@name} arbitrarily. For example:
3902
3903@example
3904@group
3905invocation: op '(' args ')'
3906 @{ $$ = new_invocation ($op, $args, @@$); @}
3907@end group
3908@end example
3909
3910@noindent
3911However, sometimes regular symbol names are not sufficient due to
3912ambiguities:
3913
3914@example
3915@group
3916exp: exp '/' exp
3917 @{ $exp = $exp / $exp; @} // $exp is ambiguous.
3918
3919exp: exp '/' exp
3920 @{ $$ = $1 / $exp; @} // One usage is ambiguous.
3921
3922exp: exp '/' exp
3923 @{ $$ = $1 / $3; @} // No error.
3924@end group
3925@end example
3926
3927@noindent
3928When ambiguity occurs, explicitly declared names may be used for values and
3929locations. Explicit names are declared as a bracketed name after a symbol
3930appearance in rule definitions. For example:
3931@example
3932@group
3933exp[result]: exp[left] '/' exp[right]
3934 @{ $result = $left / $right; @}
3935@end group
3936@end example
3937
3938@noindent
3939Explicit names may be declared for RHS and for LHS symbols as well. In order
3940to access a semantic value generated by a mid-rule action, an explicit name
3941may also be declared by putting a bracketed name after the closing brace of
3942the mid-rule action code:
3943@example
3944@group
3945exp[res]: exp[x] '+' @{$left = $x;@}[left] exp[right]
3946 @{ $res = $left + $right; @}
3947@end group
3948@end example
3949
3950@noindent
3951
3952In references, in order to specify names containing dots and dashes, an explicit
3953bracketed syntax @code{$[name]} and @code{@@[name]} must be used:
3954@example
3955@group
3956if-stmt: IF '(' expr ')' THEN then.stmt ';'
3957 @{ $[if-stmt] = new_if_stmt ($expr, $[then.stmt]); @}
3958@end group
3959@end example
3960
3961It often happens that named references are followed by a dot, dash or other
3962C punctuation marks and operators. By default, Bison will read
3963@code{$name.suffix} as a reference to symbol value @code{$name} followed by
3964@samp{.suffix}, i.e., an access to the @samp{suffix} field of the semantic
3965value. In order to force Bison to recognize @code{name.suffix} in its entirety
3966as the name of a semantic value, bracketed syntax @code{$[name.suffix]}
3967must be used.
3968
3969
3970@node Locations
3971@section Tracking Locations
3972@cindex location
3973@cindex textual location
3974@cindex location, textual
3975
3976Though grammar rules and semantic actions are enough to write a fully
3977functional parser, it can be useful to process some additional information,
3978especially symbol locations.
3979
3980The way locations are handled is defined by providing a data type, and
3981actions to take when rules are matched.
3982
3983@menu
3984* Location Type:: Specifying a data type for locations.
3985* Actions and Locations:: Using locations in actions.
3986* Location Default Action:: Defining a general way to compute locations.
3987@end menu
3988
3989@node Location Type
3990@subsection Data Type of Locations
3991@cindex data type of locations
3992@cindex default location type
3993
3994Defining a data type for locations is much simpler than for semantic values,
3995since all tokens and groupings always use the same type.
3996
3997You can specify the type of locations by defining a macro called
3998@code{YYLTYPE}, just as you can specify the semantic value type by
3999defining a @code{YYSTYPE} macro (@pxref{Value Type}).
4000When @code{YYLTYPE} is not defined, Bison uses a default structure type with
4001four members:
4002
4003@example
4004typedef struct YYLTYPE
4005@{
4006 int first_line;
4007 int first_column;
4008 int last_line;
4009 int last_column;
4010@} YYLTYPE;
4011@end example
4012
4013When @code{YYLTYPE} is not defined, at the beginning of the parsing, Bison
4014initializes all these fields to 1 for @code{yylloc}. To initialize
4015@code{yylloc} with a custom location type (or to chose a different
4016initialization), use the @code{%initial-action} directive. @xref{Initial
4017Action Decl, , Performing Actions before Parsing}.
4018
4019@node Actions and Locations
4020@subsection Actions and Locations
4021@cindex location actions
4022@cindex actions, location
4023@vindex @@$
4024@vindex @@@var{n}
4025@vindex @@@var{name}
4026@vindex @@[@var{name}]
4027
4028Actions are not only useful for defining language semantics, but also for
4029describing the behavior of the output parser with locations.
4030
4031The most obvious way for building locations of syntactic groupings is very
4032similar to the way semantic values are computed. In a given rule, several
4033constructs can be used to access the locations of the elements being matched.
4034The location of the @var{n}th component of the right hand side is
4035@code{@@@var{n}}, while the location of the left hand side grouping is
4036@code{@@$}.
4037
4038In addition, the named references construct @code{@@@var{name}} and
4039@code{@@[@var{name}]} may also be used to address the symbol locations.
4040@xref{Named References,,Using Named References}, for more information
4041about using the named references construct.
4042
4043Here is a basic example using the default data type for locations:
4044
4045@example
4046@group
4047exp: @dots{}
4048 | exp '/' exp
4049 @{
4050 @@$.first_column = @@1.first_column;
4051 @@$.first_line = @@1.first_line;
4052 @@$.last_column = @@3.last_column;
4053 @@$.last_line = @@3.last_line;
4054 if ($3)
4055 $$ = $1 / $3;
4056 else
4057 @{
4058 $$ = 1;
4059 fprintf (stderr,
4060 "Division by zero, l%d,c%d-l%d,c%d",
4061 @@3.first_line, @@3.first_column,
4062 @@3.last_line, @@3.last_column);
4063 @}
4064 @}
4065@end group
4066@end example
4067
4068As for semantic values, there is a default action for locations that is
4069run each time a rule is matched. It sets the beginning of @code{@@$} to the
4070beginning of the first symbol, and the end of @code{@@$} to the end of the
4071last symbol.
4072
4073With this default action, the location tracking can be fully automatic. The
4074example above simply rewrites this way:
4075
4076@example
4077@group
4078exp: @dots{}
4079 | exp '/' exp
4080 @{
4081 if ($3)
4082 $$ = $1 / $3;
4083 else
4084 @{
4085 $$ = 1;
4086 fprintf (stderr,
4087 "Division by zero, l%d,c%d-l%d,c%d",
4088 @@3.first_line, @@3.first_column,
4089 @@3.last_line, @@3.last_column);
4090 @}
4091 @}
4092@end group
4093@end example
4094
4095@vindex yylloc
4096It is also possible to access the location of the lookahead token, if any,
4097from a semantic action.
4098This location is stored in @code{yylloc}.
4099@xref{Action Features, ,Special Features for Use in Actions}.
4100
4101@node Location Default Action
4102@subsection Default Action for Locations
4103@vindex YYLLOC_DEFAULT
4104@cindex GLR parsers and @code{YYLLOC_DEFAULT}
4105
4106Actually, actions are not the best place to compute locations. Since
4107locations are much more general than semantic values, there is room in
4108the output parser to redefine the default action to take for each
4109rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
4110matched, before the associated action is run. It is also invoked
4111while processing a syntax error, to compute the error's location.
4112Before reporting an unresolvable syntactic ambiguity, a GLR
4113parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
4114of that ambiguity.
4115
4116Most of the time, this macro is general enough to suppress location
4117dedicated code from semantic actions.
4118
4119The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
4120the location of the grouping (the result of the computation). When a
4121rule is matched, the second parameter identifies locations of
4122all right hand side elements of the rule being matched, and the third
4123parameter is the size of the rule's right hand side.
4124When a GLR parser reports an ambiguity, which of multiple candidate
4125right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
4126When processing a syntax error, the second parameter identifies locations
4127of the symbols that were discarded during error processing, and the third
4128parameter is the number of discarded symbols.
4129
4130By default, @code{YYLLOC_DEFAULT} is defined this way:
4131
4132@smallexample
4133@group
4134# define YYLLOC_DEFAULT(Current, Rhs, N) \
4135 do \
4136 if (N) \
4137 @{ \
4138 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
4139 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
4140 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
4141 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
4142 @} \
4143 else \
4144 @{ \
4145 (Current).first_line = (Current).last_line = \
4146 YYRHSLOC(Rhs, 0).last_line; \
4147 (Current).first_column = (Current).last_column = \
4148 YYRHSLOC(Rhs, 0).last_column; \
4149 @} \
4150 while (0)
4151@end group
4152@end smallexample
4153
4154where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
4155in @var{rhs} when @var{k} is positive, and the location of the symbol
4156just before the reduction when @var{k} and @var{n} are both zero.
4157
4158When defining @code{YYLLOC_DEFAULT}, you should consider that:
4159
4160@itemize @bullet
4161@item
4162All arguments are free of side-effects. However, only the first one (the
4163result) should be modified by @code{YYLLOC_DEFAULT}.
4164
4165@item
4166For consistency with semantic actions, valid indexes within the
4167right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
4168valid index, and it refers to the symbol just before the reduction.
4169During error processing @var{n} is always positive.
4170
4171@item
4172Your macro should parenthesize its arguments, if need be, since the
4173actual arguments may not be surrounded by parentheses. Also, your
4174macro should expand to something that can be used as a single
4175statement when it is followed by a semicolon.
4176@end itemize
4177
4178@node Declarations
4179@section Bison Declarations
4180@cindex declarations, Bison
4181@cindex Bison declarations
4182
4183The @dfn{Bison declarations} section of a Bison grammar defines the symbols
4184used in formulating the grammar and the data types of semantic values.
4185@xref{Symbols}.
4186
4187All token type names (but not single-character literal tokens such as
4188@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
4189declared if you need to specify which data type to use for the semantic
4190value (@pxref{Multiple Types, ,More Than One Value Type}).
4191
4192The first rule in the grammar file also specifies the start symbol, by
4193default. If you want some other symbol to be the start symbol, you
4194must declare it explicitly (@pxref{Language and Grammar, ,Languages
4195and Context-Free Grammars}).
4196
4197@menu
4198* Require Decl:: Requiring a Bison version.
4199* Token Decl:: Declaring terminal symbols.
4200* Precedence Decl:: Declaring terminals with precedence and associativity.
4201* Union Decl:: Declaring the set of all semantic value types.
4202* Type Decl:: Declaring the choice of type for a nonterminal symbol.
4203* Initial Action Decl:: Code run before parsing starts.
4204* Destructor Decl:: Declaring how symbols are freed.
4205* Expect Decl:: Suppressing warnings about parsing conflicts.
4206* Start Decl:: Specifying the start symbol.
4207* Pure Decl:: Requesting a reentrant parser.
4208* Push Decl:: Requesting a push parser.
4209* Decl Summary:: Table of all Bison declarations.
4210* %define Summary:: Defining variables to adjust Bison's behavior.
4211* %code Summary:: Inserting code into the parser source.
4212@end menu
4213
4214@node Require Decl
4215@subsection Require a Version of Bison
4216@cindex version requirement
4217@cindex requiring a version of Bison
4218@findex %require
4219
4220You may require the minimum version of Bison to process the grammar. If
4221the requirement is not met, @command{bison} exits with an error (exit
4222status 63).
4223
4224@example
4225%require "@var{version}"
4226@end example
4227
4228@node Token Decl
4229@subsection Token Type Names
4230@cindex declaring token type names
4231@cindex token type names, declaring
4232@cindex declaring literal string tokens
4233@findex %token
4234
4235The basic way to declare a token type name (terminal symbol) is as follows:
4236
4237@example
4238%token @var{name}
4239@end example
4240
4241Bison will convert this into a @code{#define} directive in
4242the parser, so that the function @code{yylex} (if it is in this file)
4243can use the name @var{name} to stand for this token type's code.
4244
4245Alternatively, you can use @code{%left}, @code{%right},
4246@code{%precedence}, or
4247@code{%nonassoc} instead of @code{%token}, if you wish to specify
4248associativity and precedence. @xref{Precedence Decl, ,Operator
4249Precedence}.
4250
4251You can explicitly specify the numeric code for a token type by appending
4252a nonnegative decimal or hexadecimal integer value in the field immediately
4253following the token name:
4254
4255@example
4256%token NUM 300
4257%token XNUM 0x12d // a GNU extension
4258@end example
4259
4260@noindent
4261It is generally best, however, to let Bison choose the numeric codes for
4262all token types. Bison will automatically select codes that don't conflict
4263with each other or with normal characters.
4264
4265In the event that the stack type is a union, you must augment the
4266@code{%token} or other token declaration to include the data type
4267alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
4268Than One Value Type}).
4269
4270For example:
4271
4272@example
4273@group
4274%union @{ /* define stack type */
4275 double val;
4276 symrec *tptr;
4277@}
4278%token <val> NUM /* define token NUM and its type */
4279@end group
4280@end example
4281
4282You can associate a literal string token with a token type name by
4283writing the literal string at the end of a @code{%token}
4284declaration which declares the name. For example:
4285
4286@example
4287%token arrow "=>"
4288@end example
4289
4290@noindent
4291For example, a grammar for the C language might specify these names with
4292equivalent literal string tokens:
4293
4294@example
4295%token <operator> OR "||"
4296%token <operator> LE 134 "<="
4297%left OR "<="
4298@end example
4299
4300@noindent
4301Once you equate the literal string and the token name, you can use them
4302interchangeably in further declarations or the grammar rules. The
4303@code{yylex} function can use the token name or the literal string to
4304obtain the token type code number (@pxref{Calling Convention}).
4305Syntax error messages passed to @code{yyerror} from the parser will reference
4306the literal string instead of the token name.
4307
4308The token numbered as 0 corresponds to end of file; the following line
4309allows for nicer error messages referring to ``end of file'' instead
4310of ``$end'':
4311
4312@example
4313%token END 0 "end of file"
4314@end example
4315
4316@node Precedence Decl
4317@subsection Operator Precedence
4318@cindex precedence declarations
4319@cindex declaring operator precedence
4320@cindex operator precedence, declaring
4321
4322Use the @code{%left}, @code{%right}, @code{%nonassoc}, or
4323@code{%precedence} declaration to
4324declare a token and specify its precedence and associativity, all at
4325once. These are called @dfn{precedence declarations}.
4326@xref{Precedence, ,Operator Precedence}, for general information on
4327operator precedence.
4328
4329The syntax of a precedence declaration is nearly the same as that of
4330@code{%token}: either
4331
4332@example
4333%left @var{symbols}@dots{}
4334@end example
4335
4336@noindent
4337or
4338
4339@example
4340%left <@var{type}> @var{symbols}@dots{}
4341@end example
4342
4343And indeed any of these declarations serves the purposes of @code{%token}.
4344But in addition, they specify the associativity and relative precedence for
4345all the @var{symbols}:
4346
4347@itemize @bullet
4348@item
4349The associativity of an operator @var{op} determines how repeated uses
4350of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
4351@var{z}} is parsed by grouping @var{x} with @var{y} first or by
4352grouping @var{y} with @var{z} first. @code{%left} specifies
4353left-associativity (grouping @var{x} with @var{y} first) and
4354@code{%right} specifies right-associativity (grouping @var{y} with
4355@var{z} first). @code{%nonassoc} specifies no associativity, which
4356means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
4357considered a syntax error.
4358
4359@code{%precedence} gives only precedence to the @var{symbols}, and
4360defines no associativity at all. Use this to define precedence only,
4361and leave any potential conflict due to associativity enabled.
4362
4363@item
4364The precedence of an operator determines how it nests with other operators.
4365All the tokens declared in a single precedence declaration have equal
4366precedence and nest together according to their associativity.
4367When two tokens declared in different precedence declarations associate,
4368the one declared later has the higher precedence and is grouped first.
4369@end itemize
4370
4371For backward compatibility, there is a confusing difference between the
4372argument lists of @code{%token} and precedence declarations.
4373Only a @code{%token} can associate a literal string with a token type name.
4374A precedence declaration always interprets a literal string as a reference to a
4375separate token.
4376For example:
4377
4378@example
4379%left OR "<=" // Does not declare an alias.
4380%left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
4381@end example
4382
4383@node Union Decl
4384@subsection The Collection of Value Types
4385@cindex declaring value types
4386@cindex value types, declaring
4387@findex %union
4388
4389The @code{%union} declaration specifies the entire collection of
4390possible data types for semantic values. The keyword @code{%union} is
4391followed by braced code containing the same thing that goes inside a
4392@code{union} in C@.
4393
4394For example:
4395
4396@example
4397@group
4398%union @{
4399 double val;
4400 symrec *tptr;
4401@}
4402@end group
4403@end example
4404
4405@noindent
4406This says that the two alternative types are @code{double} and @code{symrec
4407*}. They are given names @code{val} and @code{tptr}; these names are used
4408in the @code{%token} and @code{%type} declarations to pick one of the types
4409for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
4410
4411As an extension to POSIX, a tag is allowed after the
4412@code{union}. For example:
4413
4414@example
4415@group
4416%union value @{
4417 double val;
4418 symrec *tptr;
4419@}
4420@end group
4421@end example
4422
4423@noindent
4424specifies the union tag @code{value}, so the corresponding C type is
4425@code{union value}. If you do not specify a tag, it defaults to
4426@code{YYSTYPE}.
4427
4428As another extension to POSIX, you may specify multiple
4429@code{%union} declarations; their contents are concatenated. However,
4430only the first @code{%union} declaration can specify a tag.
4431
4432Note that, unlike making a @code{union} declaration in C, you need not write
4433a semicolon after the closing brace.
4434
4435Instead of @code{%union}, you can define and use your own union type
4436@code{YYSTYPE} if your grammar contains at least one
4437@samp{<@var{type}>} tag. For example, you can put the following into
4438a header file @file{parser.h}:
4439
4440@example
4441@group
4442union YYSTYPE @{
4443 double val;
4444 symrec *tptr;
4445@};
4446typedef union YYSTYPE YYSTYPE;
4447@end group
4448@end example
4449
4450@noindent
4451and then your grammar can use the following
4452instead of @code{%union}:
4453
4454@example
4455@group
4456%@{
4457#include "parser.h"
4458%@}
4459%type <val> expr
4460%token <tptr> ID
4461@end group
4462@end example
4463
4464@node Type Decl
4465@subsection Nonterminal Symbols
4466@cindex declaring value types, nonterminals
4467@cindex value types, nonterminals, declaring
4468@findex %type
4469
4470@noindent
4471When you use @code{%union} to specify multiple value types, you must
4472declare the value type of each nonterminal symbol for which values are
4473used. This is done with a @code{%type} declaration, like this:
4474
4475@example
4476%type <@var{type}> @var{nonterminal}@dots{}
4477@end example
4478
4479@noindent
4480Here @var{nonterminal} is the name of a nonterminal symbol, and
4481@var{type} is the name given in the @code{%union} to the alternative
4482that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
4483can give any number of nonterminal symbols in the same @code{%type}
4484declaration, if they have the same value type. Use spaces to separate
4485the symbol names.
4486
4487You can also declare the value type of a terminal symbol. To do this,
4488use the same @code{<@var{type}>} construction in a declaration for the
4489terminal symbol. All kinds of token declarations allow
4490@code{<@var{type}>}.
4491
4492@node Initial Action Decl
4493@subsection Performing Actions before Parsing
4494@findex %initial-action
4495
4496Sometimes your parser needs to perform some initializations before
4497parsing. The @code{%initial-action} directive allows for such arbitrary
4498code.
4499
4500@deffn {Directive} %initial-action @{ @var{code} @}
4501@findex %initial-action
4502Declare that the braced @var{code} must be invoked before parsing each time
4503@code{yyparse} is called. The @var{code} may use @code{$$} and
4504@code{@@$} --- initial value and location of the lookahead --- and the
4505@code{%parse-param}.
4506@end deffn
4507
4508For instance, if your locations use a file name, you may use
4509
4510@example
4511%parse-param @{ char const *file_name @};
4512%initial-action
4513@{
4514 @@$.initialize (file_name);
4515@};
4516@end example
4517
4518
4519@node Destructor Decl
4520@subsection Freeing Discarded Symbols
4521@cindex freeing discarded symbols
4522@findex %destructor
4523@findex <*>
4524@findex <>
4525During error recovery (@pxref{Error Recovery}), symbols already pushed
4526on the stack and tokens coming from the rest of the file are discarded
4527until the parser falls on its feet. If the parser runs out of memory,
4528or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
4529symbols on the stack must be discarded. Even if the parser succeeds, it
4530must discard the start symbol.
4531
4532When discarded symbols convey heap based information, this memory is
4533lost. While this behavior can be tolerable for batch parsers, such as
4534in traditional compilers, it is unacceptable for programs like shells or
4535protocol implementations that may parse and execute indefinitely.
4536
4537The @code{%destructor} directive defines code that is called when a
4538symbol is automatically discarded.
4539
4540@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
4541@findex %destructor
4542Invoke the braced @var{code} whenever the parser discards one of the
4543@var{symbols}.
4544Within @var{code}, @code{$$} designates the semantic value associated
4545with the discarded symbol, and @code{@@$} designates its location.
4546The additional parser parameters are also available (@pxref{Parser Function, ,
4547The Parser Function @code{yyparse}}).
4548
4549When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
4550per-symbol @code{%destructor}.
4551You may also define a per-type @code{%destructor} by listing a semantic type
4552tag among @var{symbols}.
4553In that case, the parser will invoke this @var{code} whenever it discards any
4554grammar symbol that has that semantic type tag unless that symbol has its own
4555per-symbol @code{%destructor}.
4556
4557Finally, you can define two different kinds of default @code{%destructor}s.
4558(These default forms are experimental.
4559More user feedback will help to determine whether they should become permanent
4560features.)
4561You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
4562exactly one @code{%destructor} declaration in your grammar file.
4563The parser will invoke the @var{code} associated with one of these whenever it
4564discards any user-defined grammar symbol that has no per-symbol and no per-type
4565@code{%destructor}.
4566The parser uses the @var{code} for @code{<*>} in the case of such a grammar
4567symbol for which you have formally declared a semantic type tag (@code{%type}
4568counts as such a declaration, but @code{$<tag>$} does not).
4569The parser uses the @var{code} for @code{<>} in the case of such a grammar
4570symbol that has no declared semantic type tag.
4571@end deffn
4572
4573@noindent
4574For example:
4575
4576@smallexample
4577%union @{ char *string; @}
4578%token <string> STRING1
4579%token <string> STRING2
4580%type <string> string1
4581%type <string> string2
4582%union @{ char character; @}
4583%token <character> CHR
4584%type <character> chr
4585%token TAGLESS
4586
4587%destructor @{ @} <character>
4588%destructor @{ free ($$); @} <*>
4589%destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
4590%destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
4591@end smallexample
4592
4593@noindent
4594guarantees that, when the parser discards any user-defined symbol that has a
4595semantic type tag other than @code{<character>}, it passes its semantic value
4596to @code{free} by default.
4597However, when the parser discards a @code{STRING1} or a @code{string1}, it also
4598prints its line number to @code{stdout}.
4599It performs only the second @code{%destructor} in this case, so it invokes
4600@code{free} only once.
4601Finally, the parser merely prints a message whenever it discards any symbol,
4602such as @code{TAGLESS}, that has no semantic type tag.
4603
4604A Bison-generated parser invokes the default @code{%destructor}s only for
4605user-defined as opposed to Bison-defined symbols.
4606For example, the parser will not invoke either kind of default
4607@code{%destructor} for the special Bison-defined symbols @code{$accept},
4608@code{$undefined}, or @code{$end} (@pxref{Table of Symbols, ,Bison Symbols}),
4609none of which you can reference in your grammar.
4610It also will not invoke either for the @code{error} token (@pxref{Table of
4611Symbols, ,error}), which is always defined by Bison regardless of whether you
4612reference it in your grammar.
4613However, it may invoke one of them for the end token (token 0) if you
4614redefine it from @code{$end} to, for example, @code{END}:
4615
4616@smallexample
4617%token END 0
4618@end smallexample
4619
4620@cindex actions in mid-rule
4621@cindex mid-rule actions
4622Finally, Bison will never invoke a @code{%destructor} for an unreferenced
4623mid-rule semantic value (@pxref{Mid-Rule Actions,,Actions in Mid-Rule}).
4624That is, Bison does not consider a mid-rule to have a semantic value if you do
4625not reference @code{$$} in the mid-rule's action or @code{$@var{n}} (where
4626@var{n} is the RHS symbol position of the mid-rule) in any later action in that
4627rule.
4628However, if you do reference either, the Bison-generated parser will invoke the
4629@code{<>} @code{%destructor} whenever it discards the mid-rule symbol.
4630
4631@ignore
4632@noindent
4633In the future, it may be possible to redefine the @code{error} token as a
4634nonterminal that captures the discarded symbols.
4635In that case, the parser will invoke the default destructor for it as well.
4636@end ignore
4637
4638@sp 1
4639
4640@cindex discarded symbols
4641@dfn{Discarded symbols} are the following:
4642
4643@itemize
4644@item
4645stacked symbols popped during the first phase of error recovery,
4646@item
4647incoming terminals during the second phase of error recovery,
4648@item
4649the current lookahead and the entire stack (except the current
4650right-hand side symbols) when the parser returns immediately, and
4651@item
4652the start symbol, when the parser succeeds.
4653@end itemize
4654
4655The parser can @dfn{return immediately} because of an explicit call to
4656@code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
4657exhaustion.
4658
4659Right-hand side symbols of a rule that explicitly triggers a syntax
4660error via @code{YYERROR} are not discarded automatically. As a rule
4661of thumb, destructors are invoked only when user actions cannot manage
4662the memory.
4663
4664@node Expect Decl
4665@subsection Suppressing Conflict Warnings
4666@cindex suppressing conflict warnings
4667@cindex preventing warnings about conflicts
4668@cindex warnings, preventing
4669@cindex conflicts, suppressing warnings of
4670@findex %expect
4671@findex %expect-rr
4672
4673Bison normally warns if there are any conflicts in the grammar
4674(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4675have harmless shift/reduce conflicts which are resolved in a predictable
4676way and would be difficult to eliminate. It is desirable to suppress
4677the warning about these conflicts unless the number of conflicts
4678changes. You can do this with the @code{%expect} declaration.
4679
4680The declaration looks like this:
4681
4682@example
4683%expect @var{n}
4684@end example
4685
4686Here @var{n} is a decimal integer. The declaration says there should
4687be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4688Bison reports an error if the number of shift/reduce conflicts differs
4689from @var{n}, or if there are any reduce/reduce conflicts.
4690
4691For deterministic parsers, reduce/reduce conflicts are more
4692serious, and should be eliminated entirely. Bison will always report
4693reduce/reduce conflicts for these parsers. With GLR
4694parsers, however, both kinds of conflicts are routine; otherwise,
4695there would be no need to use GLR parsing. Therefore, it is
4696also possible to specify an expected number of reduce/reduce conflicts
4697in GLR parsers, using the declaration:
4698
4699@example
4700%expect-rr @var{n}
4701@end example
4702
4703In general, using @code{%expect} involves these steps:
4704
4705@itemize @bullet
4706@item
4707Compile your grammar without @code{%expect}. Use the @samp{-v} option
4708to get a verbose list of where the conflicts occur. Bison will also
4709print the number of conflicts.
4710
4711@item
4712Check each of the conflicts to make sure that Bison's default
4713resolution is what you really want. If not, rewrite the grammar and
4714go back to the beginning.
4715
4716@item
4717Add an @code{%expect} declaration, copying the number @var{n} from the
4718number which Bison printed. With GLR parsers, add an
4719@code{%expect-rr} declaration as well.
4720@end itemize
4721
4722Now Bison will report an error if you introduce an unexpected conflict,
4723but will keep silent otherwise.
4724
4725@node Start Decl
4726@subsection The Start-Symbol
4727@cindex declaring the start symbol
4728@cindex start symbol, declaring
4729@cindex default start symbol
4730@findex %start
4731
4732Bison assumes by default that the start symbol for the grammar is the first
4733nonterminal specified in the grammar specification section. The programmer
4734may override this restriction with the @code{%start} declaration as follows:
4735
4736@example
4737%start @var{symbol}
4738@end example
4739
4740@node Pure Decl
4741@subsection A Pure (Reentrant) Parser
4742@cindex reentrant parser
4743@cindex pure parser
4744@findex %define api.pure
4745
4746A @dfn{reentrant} program is one which does not alter in the course of
4747execution; in other words, it consists entirely of @dfn{pure} (read-only)
4748code. Reentrancy is important whenever asynchronous execution is possible;
4749for example, a nonreentrant program may not be safe to call from a signal
4750handler. In systems with multiple threads of control, a nonreentrant
4751program must be called only within interlocks.
4752
4753Normally, Bison generates a parser which is not reentrant. This is
4754suitable for most uses, and it permits compatibility with Yacc. (The
4755standard Yacc interfaces are inherently nonreentrant, because they use
4756statically allocated variables for communication with @code{yylex},
4757including @code{yylval} and @code{yylloc}.)
4758
4759Alternatively, you can generate a pure, reentrant parser. The Bison
4760declaration @samp{%define api.pure} says that you want the parser to be
4761reentrant. It looks like this:
4762
4763@example
4764%define api.pure
4765@end example
4766
4767The result is that the communication variables @code{yylval} and
4768@code{yylloc} become local variables in @code{yyparse}, and a different
4769calling convention is used for the lexical analyzer function
4770@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
4771Parsers}, for the details of this. The variable @code{yynerrs}
4772becomes local in @code{yyparse} in pull mode but it becomes a member
4773of yypstate in push mode. (@pxref{Error Reporting, ,The Error
4774Reporting Function @code{yyerror}}). The convention for calling
4775@code{yyparse} itself is unchanged.
4776
4777Whether the parser is pure has nothing to do with the grammar rules.
4778You can generate either a pure parser or a nonreentrant parser from any
4779valid grammar.
4780
4781@node Push Decl
4782@subsection A Push Parser
4783@cindex push parser
4784@cindex push parser
4785@findex %define api.push-pull
4786
4787(The current push parsing interface is experimental and may evolve.
4788More user feedback will help to stabilize it.)
4789
4790A pull parser is called once and it takes control until all its input
4791is completely parsed. A push parser, on the other hand, is called
4792each time a new token is made available.
4793
4794A push parser is typically useful when the parser is part of a
4795main event loop in the client's application. This is typically
4796a requirement of a GUI, when the main event loop needs to be triggered
4797within a certain time period.
4798
4799Normally, Bison generates a pull parser.
4800The following Bison declaration says that you want the parser to be a push
4801parser (@pxref{%define Summary,,api.push-pull}):
4802
4803@example
4804%define api.push-pull push
4805@end example
4806
4807In almost all cases, you want to ensure that your push parser is also
4808a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). The only
4809time you should create an impure push parser is to have backwards
4810compatibility with the impure Yacc pull mode interface. Unless you know
4811what you are doing, your declarations should look like this:
4812
4813@example
4814%define api.pure
4815%define api.push-pull push
4816@end example
4817
4818There is a major notable functional difference between the pure push parser
4819and the impure push parser. It is acceptable for a pure push parser to have
4820many parser instances, of the same type of parser, in memory at the same time.
4821An impure push parser should only use one parser at a time.
4822
4823When a push parser is selected, Bison will generate some new symbols in
4824the generated parser. @code{yypstate} is a structure that the generated
4825parser uses to store the parser's state. @code{yypstate_new} is the
4826function that will create a new parser instance. @code{yypstate_delete}
4827will free the resources associated with the corresponding parser instance.
4828Finally, @code{yypush_parse} is the function that should be called whenever a
4829token is available to provide the parser. A trivial example
4830of using a pure push parser would look like this:
4831
4832@example
4833int status;
4834yypstate *ps = yypstate_new ();
4835do @{
4836 status = yypush_parse (ps, yylex (), NULL);
4837@} while (status == YYPUSH_MORE);
4838yypstate_delete (ps);
4839@end example
4840
4841If the user decided to use an impure push parser, a few things about
4842the generated parser will change. The @code{yychar} variable becomes
4843a global variable instead of a variable in the @code{yypush_parse} function.
4844For this reason, the signature of the @code{yypush_parse} function is
4845changed to remove the token as a parameter. A nonreentrant push parser
4846example would thus look like this:
4847
4848@example
4849extern int yychar;
4850int status;
4851yypstate *ps = yypstate_new ();
4852do @{
4853 yychar = yylex ();
4854 status = yypush_parse (ps);
4855@} while (status == YYPUSH_MORE);
4856yypstate_delete (ps);
4857@end example
4858
4859That's it. Notice the next token is put into the global variable @code{yychar}
4860for use by the next invocation of the @code{yypush_parse} function.
4861
4862Bison also supports both the push parser interface along with the pull parser
4863interface in the same generated parser. In order to get this functionality,
4864you should replace the @samp{%define api.push-pull push} declaration with the
4865@samp{%define api.push-pull both} declaration. Doing this will create all of
4866the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
4867and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
4868would be used. However, the user should note that it is implemented in the
4869generated parser by calling @code{yypull_parse}.
4870This makes the @code{yyparse} function that is generated with the
4871@samp{%define api.push-pull both} declaration slower than the normal
4872@code{yyparse} function. If the user
4873calls the @code{yypull_parse} function it will parse the rest of the input
4874stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
4875and then @code{yypull_parse} the rest of the input stream. If you would like
4876to switch back and forth between between parsing styles, you would have to
4877write your own @code{yypull_parse} function that knows when to quit looking
4878for input. An example of using the @code{yypull_parse} function would look
4879like this:
4880
4881@example
4882yypstate *ps = yypstate_new ();
4883yypull_parse (ps); /* Will call the lexer */
4884yypstate_delete (ps);
4885@end example
4886
4887Adding the @samp{%define api.pure} declaration does exactly the same thing to
4888the generated parser with @samp{%define api.push-pull both} as it did for
4889@samp{%define api.push-pull push}.
4890
4891@node Decl Summary
4892@subsection Bison Declaration Summary
4893@cindex Bison declaration summary
4894@cindex declaration summary
4895@cindex summary, Bison declaration
4896
4897Here is a summary of the declarations used to define a grammar:
4898
4899@deffn {Directive} %union
4900Declare the collection of data types that semantic values may have
4901(@pxref{Union Decl, ,The Collection of Value Types}).
4902@end deffn
4903
4904@deffn {Directive} %token
4905Declare a terminal symbol (token type name) with no precedence
4906or associativity specified (@pxref{Token Decl, ,Token Type Names}).
4907@end deffn
4908
4909@deffn {Directive} %right
4910Declare a terminal symbol (token type name) that is right-associative
4911(@pxref{Precedence Decl, ,Operator Precedence}).
4912@end deffn
4913
4914@deffn {Directive} %left
4915Declare a terminal symbol (token type name) that is left-associative
4916(@pxref{Precedence Decl, ,Operator Precedence}).
4917@end deffn
4918
4919@deffn {Directive} %nonassoc
4920Declare a terminal symbol (token type name) that is nonassociative
4921(@pxref{Precedence Decl, ,Operator Precedence}).
4922Using it in a way that would be associative is a syntax error.
4923@end deffn
4924
4925@ifset defaultprec
4926@deffn {Directive} %default-prec
4927Assign a precedence to rules lacking an explicit @code{%prec} modifier
4928(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4929@end deffn
4930@end ifset
4931
4932@deffn {Directive} %type
4933Declare the type of semantic values for a nonterminal symbol
4934(@pxref{Type Decl, ,Nonterminal Symbols}).
4935@end deffn
4936
4937@deffn {Directive} %start
4938Specify the grammar's start symbol (@pxref{Start Decl, ,The
4939Start-Symbol}).
4940@end deffn
4941
4942@deffn {Directive} %expect
4943Declare the expected number of shift-reduce conflicts
4944(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
4945@end deffn
4946
4947
4948@sp 1
4949@noindent
4950In order to change the behavior of @command{bison}, use the following
4951directives:
4952
4953@deffn {Directive} %code @{@var{code}@}
4954@deffnx {Directive} %code @var{qualifier} @{@var{code}@}
4955@findex %code
4956Insert @var{code} verbatim into the output parser source at the
4957default location or at the location specified by @var{qualifier}.
4958@xref{%code Summary}.
4959@end deffn
4960
4961@deffn {Directive} %debug
4962Instrument the output parser for traces. Obsoleted by @samp{%define
4963parse.trace}.
4964@xref{Tracing, ,Tracing Your Parser}.
4965@end deffn
4966
4967@deffn {Directive} %define @var{variable}
4968@deffnx {Directive} %define @var{variable} @var{value}
4969@deffnx {Directive} %define @var{variable} "@var{value}"
4970Define a variable to adjust Bison's behavior. @xref{%define Summary}.
4971@end deffn
4972
4973@deffn {Directive} %defines
4974Write a parser header file containing macro definitions for the token
4975type names defined in the grammar as well as a few other declarations.
4976If the parser implementation file is named @file{@var{name}.c} then
4977the parser header file is named @file{@var{name}.h}.
4978
4979For C parsers, the parser header file declares @code{YYSTYPE} unless
4980@code{YYSTYPE} is already defined as a macro or you have used a
4981@code{<@var{type}>} tag without using @code{%union}. Therefore, if
4982you are using a @code{%union} (@pxref{Multiple Types, ,More Than One
4983Value Type}) with components that require other definitions, or if you
4984have defined a @code{YYSTYPE} macro or type definition (@pxref{Value
4985Type, ,Data Types of Semantic Values}), you need to arrange for these
4986definitions to be propagated to all modules, e.g., by putting them in
4987a prerequisite header that is included both by your parser and by any
4988other module that needs @code{YYSTYPE}.
4989
4990Unless your parser is pure, the parser header file declares
4991@code{yylval} as an external variable. @xref{Pure Decl, ,A Pure
4992(Reentrant) Parser}.
4993
4994If you have also used locations, the parser header file declares
4995@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
4996the @code{YYSTYPE} macro and @code{yylval}. @xref{Locations,
4997,Tracking Locations}.
4998
4999This parser header file is normally essential if you wish to put the
5000definition of @code{yylex} in a separate source file, because
5001@code{yylex} typically needs to be able to refer to the
5002above-mentioned declarations and to the token type codes. @xref{Token
5003Values, ,Semantic Values of Tokens}.
5004
5005@findex %code requires
5006@findex %code provides
5007If you have declared @code{%code requires} or @code{%code provides}, the output
5008header also contains their code.
5009@xref{%code Summary}.
5010@end deffn
5011
5012@deffn {Directive} %defines @var{defines-file}
5013Same as above, but save in the file @var{defines-file}.
5014@end deffn
5015
5016@deffn {Directive} %destructor
5017Specify how the parser should reclaim the memory associated to
5018discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
5019@end deffn
5020
5021@deffn {Directive} %file-prefix "@var{prefix}"
5022Specify a prefix to use for all Bison output file names. The names
5023are chosen as if the grammar file were named @file{@var{prefix}.y}.
5024@end deffn
5025
5026@deffn {Directive} %language "@var{language}"
5027Specify the programming language for the generated parser. Currently
5028supported languages include C, C++, and Java.
5029@var{language} is case-insensitive.
5030
5031This directive is experimental and its effect may be modified in future
5032releases.
5033@end deffn
5034
5035@deffn {Directive} %locations
5036Generate the code processing the locations (@pxref{Action Features,
5037,Special Features for Use in Actions}). This mode is enabled as soon as
5038the grammar uses the special @samp{@@@var{n}} tokens, but if your
5039grammar does not use it, using @samp{%locations} allows for more
5040accurate syntax error messages.
5041@end deffn
5042
5043@deffn {Directive} %name-prefix "@var{prefix}"
5044Rename the external symbols used in the parser so that they start with
5045@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
5046in C parsers
5047is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
5048@code{yylval}, @code{yychar}, @code{yydebug}, and
5049(if locations are used) @code{yylloc}. If you use a push parser,
5050@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5051@code{yypstate_new} and @code{yypstate_delete} will
5052also be renamed. For example, if you use @samp{%name-prefix "c_"}, the
5053names become @code{c_parse}, @code{c_lex}, and so on.
5054For C++ parsers, see the @samp{%define api.namespace} documentation in this
5055section.
5056@xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
5057@end deffn
5058
5059@ifset defaultprec
5060@deffn {Directive} %no-default-prec
5061Do not assign a precedence to rules lacking an explicit @code{%prec}
5062modifier (@pxref{Contextual Precedence, ,Context-Dependent
5063Precedence}).
5064@end deffn
5065@end ifset
5066
5067@deffn {Directive} %no-lines
5068Don't generate any @code{#line} preprocessor commands in the parser
5069implementation file. Ordinarily Bison writes these commands in the
5070parser implementation file so that the C compiler and debuggers will
5071associate errors and object code with your source file (the grammar
5072file). This directive causes them to associate errors with the parser
5073implementation file, treating it as an independent source file in its
5074own right.
5075@end deffn
5076
5077@deffn {Directive} %output "@var{file}"
5078Specify @var{file} for the parser implementation file.
5079@end deffn
5080
5081@deffn {Directive} %pure-parser
5082Deprecated version of @samp{%define api.pure} (@pxref{%define
5083Summary,,api.pure}), for which Bison is more careful to warn about
5084unreasonable usage.
5085@end deffn
5086
5087@deffn {Directive} %require "@var{version}"
5088Require version @var{version} or higher of Bison. @xref{Require Decl, ,
5089Require a Version of Bison}.
5090@end deffn
5091
5092@deffn {Directive} %skeleton "@var{file}"
5093Specify the skeleton to use.
5094
5095@c You probably don't need this option unless you are developing Bison.
5096@c You should use @code{%language} if you want to specify the skeleton for a
5097@c different language, because it is clearer and because it will always choose the
5098@c correct skeleton for non-deterministic or push parsers.
5099
5100If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5101file in the Bison installation directory.
5102If it does, @var{file} is an absolute file name or a file name relative to the
5103directory of the grammar file.
5104This is similar to how most shells resolve commands.
5105@end deffn
5106
5107@deffn {Directive} %token-table
5108Generate an array of token names in the parser implementation file.
5109The name of the array is @code{yytname}; @code{yytname[@var{i}]} is
5110the name of the token whose internal Bison token code number is
5111@var{i}. The first three elements of @code{yytname} correspond to the
5112predefined tokens @code{"$end"}, @code{"error"}, and
5113@code{"$undefined"}; after these come the symbols defined in the
5114grammar file.
5115
5116The name in the table includes all the characters needed to represent
5117the token in Bison. For single-character literals and literal
5118strings, this includes the surrounding quoting characters and any
5119escape sequences. For example, the Bison single-character literal
5120@code{'+'} corresponds to a three-character name, represented in C as
5121@code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
5122corresponds to a five-character name, represented in C as
5123@code{"\"\\\\/\""}.
5124
5125When you specify @code{%token-table}, Bison also generates macro
5126definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
5127@code{YYNRULES}, and @code{YYNSTATES}:
5128
5129@table @code
5130@item YYNTOKENS
5131The highest token number, plus one.
5132@item YYNNTS
5133The number of nonterminal symbols.
5134@item YYNRULES
5135The number of grammar rules,
5136@item YYNSTATES
5137The number of parser states (@pxref{Parser States}).
5138@end table
5139@end deffn
5140
5141@deffn {Directive} %verbose
5142Write an extra output file containing verbose descriptions of the
5143parser states and what is done for each type of lookahead token in
5144that state. @xref{Understanding, , Understanding Your Parser}, for more
5145information.
5146@end deffn
5147
5148@deffn {Directive} %yacc
5149Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
5150including its naming conventions. @xref{Bison Options}, for more.
5151@end deffn
5152
5153
5154@node %define Summary
5155@subsection %define Summary
5156
5157There are many features of Bison's behavior that can be controlled by
5158assigning the feature a single value. For historical reasons, some
5159such features are assigned values by dedicated directives, such as
5160@code{%start}, which assigns the start symbol. However, newer such
5161features are associated with variables, which are assigned by the
5162@code{%define} directive:
5163
5164@deffn {Directive} %define @var{variable}
5165@deffnx {Directive} %define @var{variable} @var{value}
5166@deffnx {Directive} %define @var{variable} "@var{value}"
5167Define @var{variable} to @var{value}.
5168
5169@var{value} must be placed in quotation marks if it contains any
5170character other than a letter, underscore, period, or non-initial dash
5171or digit. Omitting @code{"@var{value}"} entirely is always equivalent
5172to specifying @code{""}.
5173
5174It is an error if a @var{variable} is defined by @code{%define}
5175multiple times, but see @ref{Bison Options,,-D
5176@var{name}[=@var{value}]}.
5177@end deffn
5178
5179The rest of this section summarizes variables and values that
5180@code{%define} accepts.
5181
5182Some @var{variable}s take Boolean values. In this case, Bison will
5183complain if the variable definition does not meet one of the following
5184four conditions:
5185
5186@enumerate
5187@item @code{@var{value}} is @code{true}
5188
5189@item @code{@var{value}} is omitted (or @code{""} is specified).
5190This is equivalent to @code{true}.
5191
5192@item @code{@var{value}} is @code{false}.
5193
5194@item @var{variable} is never defined.
5195In this case, Bison selects a default value.
5196@end enumerate
5197
5198What @var{variable}s are accepted, as well as their meanings and default
5199values, depend on the selected target language and/or the parser
5200skeleton (@pxref{Decl Summary,,%language}, @pxref{Decl
5201Summary,,%skeleton}).
5202Unaccepted @var{variable}s produce an error.
5203Some of the accepted @var{variable}s are:
5204
5205@table @code
5206@c ================================================== api.namespace
5207@item api.namespace
5208@findex %define api.namespace
5209@itemize
5210@item Languages(s): C++
5211
5212@item Purpose: Specify the namespace for the parser class.
5213For example, if you specify:
5214
5215@smallexample
5216%define api.namespace "foo::bar"
5217@end smallexample
5218
5219Bison uses @code{foo::bar} verbatim in references such as:
5220
5221@smallexample
5222foo::bar::parser::semantic_type
5223@end smallexample
5224
5225However, to open a namespace, Bison removes any leading @code{::} and then
5226splits on any remaining occurrences:
5227
5228@smallexample
5229namespace foo @{ namespace bar @{
5230 class position;
5231 class location;
5232@} @}
5233@end smallexample
5234
5235@item Accepted Values:
5236Any absolute or relative C++ namespace reference without a trailing
5237@code{"::"}. For example, @code{"foo"} or @code{"::foo::bar"}.
5238
5239@item Default Value:
5240The value specified by @code{%name-prefix}, which defaults to @code{yy}.
5241This usage of @code{%name-prefix} is for backward compatibility and can
5242be confusing since @code{%name-prefix} also specifies the textual prefix
5243for the lexical analyzer function. Thus, if you specify
5244@code{%name-prefix}, it is best to also specify @samp{%define
5245api.namespace} so that @code{%name-prefix} @emph{only} affects the
5246lexical analyzer function. For example, if you specify:
5247
5248@smallexample
5249%define api.namespace "foo"
5250%name-prefix "bar::"
5251@end smallexample
5252
5253The parser namespace is @code{foo} and @code{yylex} is referenced as
5254@code{bar::lex}.
5255@end itemize
5256@c namespace
5257
5258
5259
5260@c ================================================== api.pure
5261@item api.pure
5262@findex %define api.pure
5263
5264@itemize @bullet
5265@item Language(s): C
5266
5267@item Purpose: Request a pure (reentrant) parser program.
5268@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
5269
5270@item Accepted Values: Boolean
5271
5272@item Default Value: @code{false}
5273@end itemize
5274@c api.pure
5275
5276
5277
5278@c ================================================== api.push-pull
5279@item api.push-pull
5280@findex %define api.push-pull
5281
5282@itemize @bullet
5283@item Language(s): C (deterministic parsers only)
5284
5285@item Purpose: Request a pull parser, a push parser, or both.
5286@xref{Push Decl, ,A Push Parser}.
5287(The current push parsing interface is experimental and may evolve.
5288More user feedback will help to stabilize it.)
5289
5290@item Accepted Values: @code{pull}, @code{push}, @code{both}
5291
5292@item Default Value: @code{pull}
5293@end itemize
5294@c api.push-pull
5295
5296
5297
5298@c ================================================== api.tokens.prefix
5299@item api.tokens.prefix
5300@findex %define api.tokens.prefix
5301
5302@itemize
5303@item Languages(s): all
5304
5305@item Purpose:
5306Add a prefix to the token names when generating their definition in the
5307target language. For instance
5308
5309@example
5310%token FILE for ERROR
5311%define api.tokens.prefix "TOK_"
5312%%
5313start: FILE for ERROR;
5314@end example
5315
5316@noindent
5317generates the definition of the symbols @code{TOK_FILE}, @code{TOK_for},
5318and @code{TOK_ERROR} in the generated source files. In particular, the
5319scanner must use these prefixed token names, while the grammar itself
5320may still use the short names (as in the sample rule given above). The
5321generated informational files (@file{*.output}, @file{*.xml},
5322@file{*.dot}) are not modified by this prefix. See @ref{Calc++ Parser}
5323and @ref{Calc++ Scanner}, for a complete example.
5324
5325@item Accepted Values:
5326Any string. Should be a valid identifier prefix in the target language,
5327in other words, it should typically be an identifier itself (sequence of
5328letters, underscores, and ---not at the beginning--- digits).
5329
5330@item Default Value:
5331empty
5332@end itemize
5333@c api.tokens.prefix
5334
5335
5336@c ================================================== lex_symbol
5337@item lex_symbol
5338@findex %define lex_symbol
5339
5340@itemize @bullet
5341@item Language(s):
5342C++
5343
5344@item Purpose:
5345When variant-based semantic values are enabled (@pxref{C++ Variants}),
5346request that symbols be handled as a whole (type, value, and possibly
5347location) in the scanner. @xref{Complete Symbols}, for details.
5348
5349@item Accepted Values:
5350Boolean.
5351
5352@item Default Value:
5353@code{false}
5354@end itemize
5355@c lex_symbol
5356
5357
5358@c ================================================== lr.default-reductions
5359
5360@item lr.default-reductions
5361@findex %define lr.default-reductions
5362
5363@itemize @bullet
5364@item Language(s): all
5365
5366@item Purpose: Specify the kind of states that are permitted to
5367contain default reductions. @xref{Default Reductions}. (The ability to
5368specify where default reductions should be used is experimental. More user
5369feedback will help to stabilize it.)
5370
5371@item Accepted Values: @code{full}, @code{consistent}, @code{accepting}
5372@item Default Value:
5373@itemize
5374@item @code{accepting} if @code{lr.type} is @code{canonical-lr}.
5375@item @code{full} otherwise.
5376@end itemize
5377@end itemize
5378
5379@c ============================================ lr.keep-unreachable-states
5380
5381@item lr.keep-unreachable-states
5382@findex %define lr.keep-unreachable-states
5383
5384@itemize @bullet
5385@item Language(s): all
5386@item Purpose: Request that Bison allow unreachable parser states to
5387remain in the parser tables. @xref{Unreachable States}.
5388@item Accepted Values: Boolean
5389@item Default Value: @code{false}
5390@end itemize
5391@c lr.keep-unreachable-states
5392
5393@c ================================================== lr.type
5394
5395@item lr.type
5396@findex %define lr.type
5397
5398@itemize @bullet
5399@item Language(s): all
5400
5401@item Purpose: Specify the type of parser tables within the
5402LR(1) family. @xref{LR Table Construction}. (This feature is experimental.
5403More user feedback will help to stabilize it.)
5404
5405@item Accepted Values: @code{lalr}, @code{ielr}, @code{canonical-lr}
5406
5407@item Default Value: @code{lalr}
5408@end itemize
5409
5410
5411@c ================================================== namespace
5412@item namespace
5413@findex %define namespace
5414Obsoleted by @code{api.namespace}
5415@c namespace
5416
5417
5418@c ================================================== parse.assert
5419@item parse.assert
5420@findex %define parse.assert
5421
5422@itemize
5423@item Languages(s): C++
5424
5425@item Purpose: Issue runtime assertions to catch invalid uses.
5426In C++, when variants are used (@pxref{C++ Variants}), symbols must be
5427constructed and
5428destroyed properly. This option checks these constraints.
5429
5430@item Accepted Values: Boolean
5431
5432@item Default Value: @code{false}
5433@end itemize
5434@c parse.assert
5435
5436
5437@c ================================================== parse.error
5438@item parse.error
5439@findex %define parse.error
5440@itemize
5441@item Languages(s):
5442all
5443@item Purpose:
5444Control the kind of error messages passed to the error reporting
5445function. @xref{Error Reporting, ,The Error Reporting Function
5446@code{yyerror}}.
5447@item Accepted Values:
5448@itemize
5449@item @code{simple}
5450Error messages passed to @code{yyerror} are simply @w{@code{"syntax
5451error"}}.
5452@item @code{verbose}
5453Error messages report the unexpected token, and possibly the expected ones.
5454However, this report can often be incorrect when LAC is not enabled
5455(@pxref{LAC}).
5456@end itemize
5457
5458@item Default Value:
5459@code{simple}
5460@end itemize
5461@c parse.error
5462
5463
5464@c ================================================== parse.lac
5465@item parse.lac
5466@findex %define parse.lac
5467
5468@itemize
5469@item Languages(s): C (deterministic parsers only)
5470
5471@item Purpose: Enable LAC (lookahead correction) to improve
5472syntax error handling. @xref{LAC}.
5473@item Accepted Values: @code{none}, @code{full}
5474@item Default Value: @code{none}
5475@end itemize
5476@c parse.lac
5477
5478@c ================================================== parse.trace
5479@item parse.trace
5480@findex %define parse.trace
5481
5482@itemize
5483@item Languages(s): C, C++
5484
5485@item Purpose: Require parser instrumentation for tracing.
5486In C/C++, define the macro @code{YYDEBUG} to 1 in the parser implementation
5487file if it is not already defined, so that the debugging facilities are
5488compiled. @xref{Tracing, ,Tracing Your Parser}.
5489
5490@item Accepted Values: Boolean
5491
5492@item Default Value: @code{false}
5493@end itemize
5494@c parse.trace
5495
5496@c ================================================== variant
5497@item variant
5498@findex %define variant
5499
5500@itemize @bullet
5501@item Language(s):
5502C++
5503
5504@item Purpose:
5505Request variant-based semantic values.
5506@xref{C++ Variants}.
5507
5508@item Accepted Values:
5509Boolean.
5510
5511@item Default Value:
5512@code{false}
5513@end itemize
5514@c variant
5515@end table
5516
5517
5518@node %code Summary
5519@subsection %code Summary
5520@findex %code
5521@cindex Prologue
5522
5523The @code{%code} directive inserts code verbatim into the output
5524parser source at any of a predefined set of locations. It thus serves
5525as a flexible and user-friendly alternative to the traditional Yacc
5526prologue, @code{%@{@var{code}%@}}. This section summarizes the
5527functionality of @code{%code} for the various target languages
5528supported by Bison. For a detailed discussion of how to use
5529@code{%code} in place of @code{%@{@var{code}%@}} for C/C++ and why it
5530is advantageous to do so, @pxref{Prologue Alternatives}.
5531
5532@deffn {Directive} %code @{@var{code}@}
5533This is the unqualified form of the @code{%code} directive. It
5534inserts @var{code} verbatim at a language-dependent default location
5535in the parser implementation.
5536
5537For C/C++, the default location is the parser implementation file
5538after the usual contents of the parser header file. Thus, the
5539unqualified form replaces @code{%@{@var{code}%@}} for most purposes.
5540
5541For Java, the default location is inside the parser class.
5542@end deffn
5543
5544@deffn {Directive} %code @var{qualifier} @{@var{code}@}
5545This is the qualified form of the @code{%code} directive.
5546@var{qualifier} identifies the purpose of @var{code} and thus the
5547location(s) where Bison should insert it. That is, if you need to
5548specify location-sensitive @var{code} that does not belong at the
5549default location selected by the unqualified @code{%code} form, use
5550this form instead.
5551@end deffn
5552
5553For any particular qualifier or for the unqualified form, if there are
5554multiple occurrences of the @code{%code} directive, Bison concatenates
5555the specified code in the order in which it appears in the grammar
5556file.
5557
5558Not all qualifiers are accepted for all target languages. Unaccepted
5559qualifiers produce an error. Some of the accepted qualifiers are:
5560
5561@table @code
5562@item requires
5563@findex %code requires
5564
5565@itemize @bullet
5566@item Language(s): C, C++
5567
5568@item Purpose: This is the best place to write dependency code required for
5569@code{YYSTYPE} and @code{YYLTYPE}.
5570In other words, it's the best place to define types referenced in @code{%union}
5571directives, and it's the best place to override Bison's default @code{YYSTYPE}
5572and @code{YYLTYPE} definitions.
5573
5574@item Location(s): The parser header file and the parser implementation file
5575before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
5576definitions.
5577@end itemize
5578
5579@item provides
5580@findex %code provides
5581
5582@itemize @bullet
5583@item Language(s): C, C++
5584
5585@item Purpose: This is the best place to write additional definitions and
5586declarations that should be provided to other modules.
5587
5588@item Location(s): The parser header file and the parser implementation
5589file after the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and
5590token definitions.
5591@end itemize
5592
5593@item top
5594@findex %code top
5595
5596@itemize @bullet
5597@item Language(s): C, C++
5598
5599@item Purpose: The unqualified @code{%code} or @code{%code requires}
5600should usually be more appropriate than @code{%code top}. However,
5601occasionally it is necessary to insert code much nearer the top of the
5602parser implementation file. For example:
5603
5604@smallexample
5605%code top @{
5606 #define _GNU_SOURCE
5607 #include <stdio.h>
5608@}
5609@end smallexample
5610
5611@item Location(s): Near the top of the parser implementation file.
5612@end itemize
5613
5614@item imports
5615@findex %code imports
5616
5617@itemize @bullet
5618@item Language(s): Java
5619
5620@item Purpose: This is the best place to write Java import directives.
5621
5622@item Location(s): The parser Java file after any Java package directive and
5623before any class definitions.
5624@end itemize
5625@end table
5626
5627Though we say the insertion locations are language-dependent, they are
5628technically skeleton-dependent. Writers of non-standard skeletons
5629however should choose their locations consistently with the behavior
5630of the standard Bison skeletons.
5631
5632
5633@node Multiple Parsers
5634@section Multiple Parsers in the Same Program
5635
5636Most programs that use Bison parse only one language and therefore contain
5637only one Bison parser. But what if you want to parse more than one
5638language with the same program? Then you need to avoid a name conflict
5639between different definitions of @code{yyparse}, @code{yylval}, and so on.
5640
5641The easy way to do this is to use the option @samp{-p @var{prefix}}
5642(@pxref{Invocation, ,Invoking Bison}). This renames the interface
5643functions and variables of the Bison parser to start with @var{prefix}
5644instead of @samp{yy}. You can use this to give each parser distinct
5645names that do not conflict.
5646
5647The precise list of symbols renamed is @code{yyparse}, @code{yylex},
5648@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
5649@code{yychar} and @code{yydebug}. If you use a push parser,
5650@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5651@code{yypstate_new} and @code{yypstate_delete} will also be renamed.
5652For example, if you use @samp{-p c}, the names become @code{cparse},
5653@code{clex}, and so on.
5654
5655@strong{All the other variables and macros associated with Bison are not
5656renamed.} These others are not global; there is no conflict if the same
5657name is used in different parsers. For example, @code{YYSTYPE} is not
5658renamed, but defining this in different ways in different parsers causes
5659no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
5660
5661The @samp{-p} option works by adding macro definitions to the
5662beginning of the parser implementation file, defining @code{yyparse}
5663as @code{@var{prefix}parse}, and so on. This effectively substitutes
5664one name for the other in the entire parser implementation file.
5665
5666@node Interface
5667@chapter Parser C-Language Interface
5668@cindex C-language interface
5669@cindex interface
5670
5671The Bison parser is actually a C function named @code{yyparse}. Here we
5672describe the interface conventions of @code{yyparse} and the other
5673functions that it needs to use.
5674
5675Keep in mind that the parser uses many C identifiers starting with
5676@samp{yy} and @samp{YY} for internal purposes. If you use such an
5677identifier (aside from those in this manual) in an action or in epilogue
5678in the grammar file, you are likely to run into trouble.
5679
5680@menu
5681* Parser Function:: How to call @code{yyparse} and what it returns.
5682* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
5683* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
5684* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
5685* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
5686* Lexical:: You must supply a function @code{yylex}
5687 which reads tokens.
5688* Error Reporting:: You must supply a function @code{yyerror}.
5689* Action Features:: Special features for use in actions.
5690* Internationalization:: How to let the parser speak in the user's
5691 native language.
5692@end menu
5693
5694@node Parser Function
5695@section The Parser Function @code{yyparse}
5696@findex yyparse
5697
5698You call the function @code{yyparse} to cause parsing to occur. This
5699function reads tokens, executes actions, and ultimately returns when it
5700encounters end-of-input or an unrecoverable syntax error. You can also
5701write an action which directs @code{yyparse} to return immediately
5702without reading further.
5703
5704
5705@deftypefun int yyparse (void)
5706The value returned by @code{yyparse} is 0 if parsing was successful (return
5707is due to end-of-input).
5708
5709The value is 1 if parsing failed because of invalid input, i.e., input
5710that contains a syntax error or that causes @code{YYABORT} to be
5711invoked.
5712
5713The value is 2 if parsing failed due to memory exhaustion.
5714@end deftypefun
5715
5716In an action, you can cause immediate return from @code{yyparse} by using
5717these macros:
5718
5719@defmac YYACCEPT
5720@findex YYACCEPT
5721Return immediately with value 0 (to report success).
5722@end defmac
5723
5724@defmac YYABORT
5725@findex YYABORT
5726Return immediately with value 1 (to report failure).
5727@end defmac
5728
5729If you use a reentrant parser, you can optionally pass additional
5730parameter information to it in a reentrant way. To do so, use the
5731declaration @code{%parse-param}:
5732
5733@deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
5734@findex %parse-param
5735Declare that one or more
5736@var{argument-declaration} are additional @code{yyparse} arguments.
5737The @var{argument-declaration} is used when declaring
5738functions or prototypes. The last identifier in
5739@var{argument-declaration} must be the argument name.
5740@end deffn
5741
5742Here's an example. Write this in the parser:
5743
5744@example
5745%parse-param @{int *nastiness@} @{int *randomness@}
5746@end example
5747
5748@noindent
5749Then call the parser like this:
5750
5751@example
5752@{
5753 int nastiness, randomness;
5754 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
5755 value = yyparse (&nastiness, &randomness);
5756 @dots{}
5757@}
5758@end example
5759
5760@noindent
5761In the grammar actions, use expressions like this to refer to the data:
5762
5763@example
5764exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
5765@end example
5766
5767@node Push Parser Function
5768@section The Push Parser Function @code{yypush_parse}
5769@findex yypush_parse
5770
5771(The current push parsing interface is experimental and may evolve.
5772More user feedback will help to stabilize it.)
5773
5774You call the function @code{yypush_parse} to parse a single token. This
5775function is available if either the @samp{%define api.push-pull push} or
5776@samp{%define api.push-pull both} declaration is used.
5777@xref{Push Decl, ,A Push Parser}.
5778
5779@deftypefun int yypush_parse (yypstate *yyps)
5780The value returned by @code{yypush_parse} is the same as for yyparse with the
5781following exception. @code{yypush_parse} will return YYPUSH_MORE if more input
5782is required to finish parsing the grammar.
5783@end deftypefun
5784
5785@node Pull Parser Function
5786@section The Pull Parser Function @code{yypull_parse}
5787@findex yypull_parse
5788
5789(The current push parsing interface is experimental and may evolve.
5790More user feedback will help to stabilize it.)
5791
5792You call the function @code{yypull_parse} to parse the rest of the input
5793stream. This function is available if the @samp{%define api.push-pull both}
5794declaration is used.
5795@xref{Push Decl, ,A Push Parser}.
5796
5797@deftypefun int yypull_parse (yypstate *yyps)
5798The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
5799@end deftypefun
5800
5801@node Parser Create Function
5802@section The Parser Create Function @code{yystate_new}
5803@findex yypstate_new
5804
5805(The current push parsing interface is experimental and may evolve.
5806More user feedback will help to stabilize it.)
5807
5808You call the function @code{yypstate_new} to create a new parser instance.
5809This function is available if either the @samp{%define api.push-pull push} or
5810@samp{%define api.push-pull both} declaration is used.
5811@xref{Push Decl, ,A Push Parser}.
5812
5813@deftypefun yypstate *yypstate_new (void)
5814The function will return a valid parser instance if there was memory available
5815or 0 if no memory was available.
5816In impure mode, it will also return 0 if a parser instance is currently
5817allocated.
5818@end deftypefun
5819
5820@node Parser Delete Function
5821@section The Parser Delete Function @code{yystate_delete}
5822@findex yypstate_delete
5823
5824(The current push parsing interface is experimental and may evolve.
5825More user feedback will help to stabilize it.)
5826
5827You call the function @code{yypstate_delete} to delete a parser instance.
5828function is available if either the @samp{%define api.push-pull push} or
5829@samp{%define api.push-pull both} declaration is used.
5830@xref{Push Decl, ,A Push Parser}.
5831
5832@deftypefun void yypstate_delete (yypstate *yyps)
5833This function will reclaim the memory associated with a parser instance.
5834After this call, you should no longer attempt to use the parser instance.
5835@end deftypefun
5836
5837@node Lexical
5838@section The Lexical Analyzer Function @code{yylex}
5839@findex yylex
5840@cindex lexical analyzer
5841
5842The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
5843the input stream and returns them to the parser. Bison does not create
5844this function automatically; you must write it so that @code{yyparse} can
5845call it. The function is sometimes referred to as a lexical scanner.
5846
5847In simple programs, @code{yylex} is often defined at the end of the
5848Bison grammar file. If @code{yylex} is defined in a separate source
5849file, you need to arrange for the token-type macro definitions to be
5850available there. To do this, use the @samp{-d} option when you run
5851Bison, so that it will write these macro definitions into the separate
5852parser header file, @file{@var{name}.tab.h}, which you can include in
5853the other source files that need it. @xref{Invocation, ,Invoking
5854Bison}.
5855
5856@menu
5857* Calling Convention:: How @code{yyparse} calls @code{yylex}.
5858* Token Values:: How @code{yylex} must return the semantic value
5859 of the token it has read.
5860* Token Locations:: How @code{yylex} must return the text location
5861 (line number, etc.) of the token, if the
5862 actions want that.
5863* Pure Calling:: How the calling convention differs in a pure parser
5864 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
5865@end menu
5866
5867@node Calling Convention
5868@subsection Calling Convention for @code{yylex}
5869
5870The value that @code{yylex} returns must be the positive numeric code
5871for the type of token it has just found; a zero or negative value
5872signifies end-of-input.
5873
5874When a token is referred to in the grammar rules by a name, that name
5875in the parser implementation file becomes a C macro whose definition
5876is the proper numeric code for that token type. So @code{yylex} can
5877use the name to indicate that type. @xref{Symbols}.
5878
5879When a token is referred to in the grammar rules by a character literal,
5880the numeric code for that character is also the code for the token type.
5881So @code{yylex} can simply return that character code, possibly converted
5882to @code{unsigned char} to avoid sign-extension. The null character
5883must not be used this way, because its code is zero and that
5884signifies end-of-input.
5885
5886Here is an example showing these things:
5887
5888@example
5889int
5890yylex (void)
5891@{
5892 @dots{}
5893 if (c == EOF) /* Detect end-of-input. */
5894 return 0;
5895 @dots{}
5896 if (c == '+' || c == '-')
5897 return c; /* Assume token type for `+' is '+'. */
5898 @dots{}
5899 return INT; /* Return the type of the token. */
5900 @dots{}
5901@}
5902@end example
5903
5904@noindent
5905This interface has been designed so that the output from the @code{lex}
5906utility can be used without change as the definition of @code{yylex}.
5907
5908If the grammar uses literal string tokens, there are two ways that
5909@code{yylex} can determine the token type codes for them:
5910
5911@itemize @bullet
5912@item
5913If the grammar defines symbolic token names as aliases for the
5914literal string tokens, @code{yylex} can use these symbolic names like
5915all others. In this case, the use of the literal string tokens in
5916the grammar file has no effect on @code{yylex}.
5917
5918@item
5919@code{yylex} can find the multicharacter token in the @code{yytname}
5920table. The index of the token in the table is the token type's code.
5921The name of a multicharacter token is recorded in @code{yytname} with a
5922double-quote, the token's characters, and another double-quote. The
5923token's characters are escaped as necessary to be suitable as input
5924to Bison.
5925
5926Here's code for looking up a multicharacter token in @code{yytname},
5927assuming that the characters of the token are stored in
5928@code{token_buffer}, and assuming that the token does not contain any
5929characters like @samp{"} that require escaping.
5930
5931@smallexample
5932for (i = 0; i < YYNTOKENS; i++)
5933 @{
5934 if (yytname[i] != 0
5935 && yytname[i][0] == '"'
5936 && ! strncmp (yytname[i] + 1, token_buffer,
5937 strlen (token_buffer))
5938 && yytname[i][strlen (token_buffer) + 1] == '"'
5939 && yytname[i][strlen (token_buffer) + 2] == 0)
5940 break;
5941 @}
5942@end smallexample
5943
5944The @code{yytname} table is generated only if you use the
5945@code{%token-table} declaration. @xref{Decl Summary}.
5946@end itemize
5947
5948@node Token Values
5949@subsection Semantic Values of Tokens
5950
5951@vindex yylval
5952In an ordinary (nonreentrant) parser, the semantic value of the token must
5953be stored into the global variable @code{yylval}. When you are using
5954just one data type for semantic values, @code{yylval} has that type.
5955Thus, if the type is @code{int} (the default), you might write this in
5956@code{yylex}:
5957
5958@example
5959@group
5960 @dots{}
5961 yylval = value; /* Put value onto Bison stack. */
5962 return INT; /* Return the type of the token. */
5963 @dots{}
5964@end group
5965@end example
5966
5967When you are using multiple data types, @code{yylval}'s type is a union
5968made from the @code{%union} declaration (@pxref{Union Decl, ,The
5969Collection of Value Types}). So when you store a token's value, you
5970must use the proper member of the union. If the @code{%union}
5971declaration looks like this:
5972
5973@example
5974@group
5975%union @{
5976 int intval;
5977 double val;
5978 symrec *tptr;
5979@}
5980@end group
5981@end example
5982
5983@noindent
5984then the code in @code{yylex} might look like this:
5985
5986@example
5987@group
5988 @dots{}
5989 yylval.intval = value; /* Put value onto Bison stack. */
5990 return INT; /* Return the type of the token. */
5991 @dots{}
5992@end group
5993@end example
5994
5995@node Token Locations
5996@subsection Textual Locations of Tokens
5997
5998@vindex yylloc
5999If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
6000Tracking Locations}) in actions to keep track of the textual locations
6001of tokens and groupings, then you must provide this information in
6002@code{yylex}. The function @code{yyparse} expects to find the textual
6003location of a token just parsed in the global variable @code{yylloc}.
6004So @code{yylex} must store the proper data in that variable.
6005
6006By default, the value of @code{yylloc} is a structure and you need only
6007initialize the members that are going to be used by the actions. The
6008four members are called @code{first_line}, @code{first_column},
6009@code{last_line} and @code{last_column}. Note that the use of this
6010feature makes the parser noticeably slower.
6011
6012@tindex YYLTYPE
6013The data type of @code{yylloc} has the name @code{YYLTYPE}.
6014
6015@node Pure Calling
6016@subsection Calling Conventions for Pure Parsers
6017
6018When you use the Bison declaration @samp{%define api.pure} to request a
6019pure, reentrant parser, the global communication variables @code{yylval}
6020and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
6021Parser}.) In such parsers the two global variables are replaced by
6022pointers passed as arguments to @code{yylex}. You must declare them as
6023shown here, and pass the information back by storing it through those
6024pointers.
6025
6026@example
6027int
6028yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
6029@{
6030 @dots{}
6031 *lvalp = value; /* Put value onto Bison stack. */
6032 return INT; /* Return the type of the token. */
6033 @dots{}
6034@}
6035@end example
6036
6037If the grammar file does not use the @samp{@@} constructs to refer to
6038textual locations, then the type @code{YYLTYPE} will not be defined. In
6039this case, omit the second argument; @code{yylex} will be called with
6040only one argument.
6041
6042If you wish to pass additional arguments to @code{yylex}, use
6043@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
6044Function}). To pass additional arguments to both @code{yylex} and
6045@code{yyparse}, use @code{%param}.
6046
6047@deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
6048@findex %lex-param
6049Specify that @var{argument-declaration} are additional @code{yylex} argument
6050declarations. You may pass one or more such declarations, which is
6051equivalent to repeating @code{%lex-param}.
6052@end deffn
6053
6054@deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
6055@findex %param
6056Specify that @var{argument-declaration} are additional
6057@code{yylex}/@code{yyparse} argument declaration. This is equivalent to
6058@samp{%lex-param @{@var{argument-declaration}@} @dots{} %parse-param
6059@{@var{argument-declaration}@} @dots{}}. You may pass one or more
6060declarations, which is equivalent to repeating @code{%param}.
6061@end deffn
6062
6063For instance:
6064
6065@example
6066%lex-param @{scanner_mode *mode@}
6067%parse-param @{parser_mode *mode@}
6068%param @{environment_type *env@}
6069@end example
6070
6071@noindent
6072results in the following signature:
6073
6074@example
6075int yylex (scanner_mode *mode, environment_type *env);
6076int yyparse (parser_mode *mode, environment_type *env);
6077@end example
6078
6079If @samp{%define api.pure} is added:
6080
6081@example
6082int yylex (YYSTYPE *lvalp, scanner_mode *mode, environment_type *env);
6083int yyparse (parser_mode *mode, environment_type *env);
6084@end example
6085
6086@noindent
6087and finally, if both @samp{%define api.pure} and @code{%locations} are used:
6088
6089@example
6090int yylex (YYSTYPE *lvalp, YYLTYPE *llocp,
6091 scanner_mode *mode, environment_type *env);
6092int yyparse (parser_mode *mode, environment_type *env);
6093@end example
6094
6095@node Error Reporting
6096@section The Error Reporting Function @code{yyerror}
6097@cindex error reporting function
6098@findex yyerror
6099@cindex parse error
6100@cindex syntax error
6101
6102The Bison parser detects a @dfn{syntax error} (or @dfn{parse error})
6103whenever it reads a token which cannot satisfy any syntax rule. An
6104action in the grammar can also explicitly proclaim an error, using the
6105macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
6106in Actions}).
6107
6108The Bison parser expects to report the error by calling an error
6109reporting function named @code{yyerror}, which you must supply. It is
6110called by @code{yyparse} whenever a syntax error is found, and it
6111receives one argument. For a syntax error, the string is normally
6112@w{@code{"syntax error"}}.
6113
6114@findex %define parse.error
6115If you invoke @samp{%define parse.error verbose} in the Bison declarations
6116section (@pxref{Bison Declarations, ,The Bison Declarations Section}), then
6117Bison provides a more verbose and specific error message string instead of
6118just plain @w{@code{"syntax error"}}. However, that message sometimes
6119contains incorrect information if LAC is not enabled (@pxref{LAC}).
6120
6121The parser can detect one other kind of error: memory exhaustion. This
6122can happen when the input contains constructions that are very deeply
6123nested. It isn't likely you will encounter this, since the Bison
6124parser normally extends its stack automatically up to a very large limit. But
6125if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
6126fashion, except that the argument string is @w{@code{"memory exhausted"}}.
6127
6128In some cases diagnostics like @w{@code{"syntax error"}} are
6129translated automatically from English to some other language before
6130they are passed to @code{yyerror}. @xref{Internationalization}.
6131
6132The following definition suffices in simple programs:
6133
6134@example
6135@group
6136void
6137yyerror (char const *s)
6138@{
6139@end group
6140@group
6141 fprintf (stderr, "%s\n", s);
6142@}
6143@end group
6144@end example
6145
6146After @code{yyerror} returns to @code{yyparse}, the latter will attempt
6147error recovery if you have written suitable error recovery grammar rules
6148(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
6149immediately return 1.
6150
6151Obviously, in location tracking pure parsers, @code{yyerror} should have
6152an access to the current location.
6153This is indeed the case for the GLR
6154parsers, but not for the Yacc parser, for historical reasons. I.e., if
6155@samp{%locations %define api.pure} is passed then the prototypes for
6156@code{yyerror} are:
6157
6158@example
6159void yyerror (char const *msg); /* Yacc parsers. */
6160void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
6161@end example
6162
6163If @samp{%parse-param @{int *nastiness@}} is used, then:
6164
6165@example
6166void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
6167void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
6168@end example
6169
6170Finally, GLR and Yacc parsers share the same @code{yyerror} calling
6171convention for absolutely pure parsers, i.e., when the calling
6172convention of @code{yylex} @emph{and} the calling convention of
6173@samp{%define api.pure} are pure.
6174I.e.:
6175
6176@example
6177/* Location tracking. */
6178%locations
6179/* Pure yylex. */
6180%define api.pure
6181%lex-param @{int *nastiness@}
6182/* Pure yyparse. */
6183%parse-param @{int *nastiness@}
6184%parse-param @{int *randomness@}
6185@end example
6186
6187@noindent
6188results in the following signatures for all the parser kinds:
6189
6190@example
6191int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
6192int yyparse (int *nastiness, int *randomness);
6193void yyerror (YYLTYPE *locp,
6194 int *nastiness, int *randomness,
6195 char const *msg);
6196@end example
6197
6198@noindent
6199The prototypes are only indications of how the code produced by Bison
6200uses @code{yyerror}. Bison-generated code always ignores the returned
6201value, so @code{yyerror} can return any type, including @code{void}.
6202Also, @code{yyerror} can be a variadic function; that is why the
6203message is always passed last.
6204
6205Traditionally @code{yyerror} returns an @code{int} that is always
6206ignored, but this is purely for historical reasons, and @code{void} is
6207preferable since it more accurately describes the return type for
6208@code{yyerror}.
6209
6210@vindex yynerrs
6211The variable @code{yynerrs} contains the number of syntax errors
6212reported so far. Normally this variable is global; but if you
6213request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
6214then it is a local variable which only the actions can access.
6215
6216@node Action Features
6217@section Special Features for Use in Actions
6218@cindex summary, action features
6219@cindex action features summary
6220
6221Here is a table of Bison constructs, variables and macros that
6222are useful in actions.
6223
6224@deffn {Variable} $$
6225Acts like a variable that contains the semantic value for the
6226grouping made by the current rule. @xref{Actions}.
6227@end deffn
6228
6229@deffn {Variable} $@var{n}
6230Acts like a variable that contains the semantic value for the
6231@var{n}th component of the current rule. @xref{Actions}.
6232@end deffn
6233
6234@deffn {Variable} $<@var{typealt}>$
6235Like @code{$$} but specifies alternative @var{typealt} in the union
6236specified by the @code{%union} declaration. @xref{Action Types, ,Data
6237Types of Values in Actions}.
6238@end deffn
6239
6240@deffn {Variable} $<@var{typealt}>@var{n}
6241Like @code{$@var{n}} but specifies alternative @var{typealt} in the
6242union specified by the @code{%union} declaration.
6243@xref{Action Types, ,Data Types of Values in Actions}.
6244@end deffn
6245
6246@deffn {Macro} YYABORT;
6247Return immediately from @code{yyparse}, indicating failure.
6248@xref{Parser Function, ,The Parser Function @code{yyparse}}.
6249@end deffn
6250
6251@deffn {Macro} YYACCEPT;
6252Return immediately from @code{yyparse}, indicating success.
6253@xref{Parser Function, ,The Parser Function @code{yyparse}}.
6254@end deffn
6255
6256@deffn {Macro} YYBACKUP (@var{token}, @var{value});
6257@findex YYBACKUP
6258Unshift a token. This macro is allowed only for rules that reduce
6259a single value, and only when there is no lookahead token.
6260It is also disallowed in GLR parsers.
6261It installs a lookahead token with token type @var{token} and
6262semantic value @var{value}; then it discards the value that was
6263going to be reduced by this rule.
6264
6265If the macro is used when it is not valid, such as when there is
6266a lookahead token already, then it reports a syntax error with
6267a message @samp{cannot back up} and performs ordinary error
6268recovery.
6269
6270In either case, the rest of the action is not executed.
6271@end deffn
6272
6273@deffn {Macro} YYEMPTY
6274@vindex YYEMPTY
6275Value stored in @code{yychar} when there is no lookahead token.
6276@end deffn
6277
6278@deffn {Macro} YYEOF
6279@vindex YYEOF
6280Value stored in @code{yychar} when the lookahead is the end of the input
6281stream.
6282@end deffn
6283
6284@deffn {Macro} YYERROR;
6285@findex YYERROR
6286Cause an immediate syntax error. This statement initiates error
6287recovery just as if the parser itself had detected an error; however, it
6288does not call @code{yyerror}, and does not print any message. If you
6289want to print an error message, call @code{yyerror} explicitly before
6290the @samp{YYERROR;} statement. @xref{Error Recovery}.
6291@end deffn
6292
6293@deffn {Macro} YYRECOVERING
6294@findex YYRECOVERING
6295The expression @code{YYRECOVERING ()} yields 1 when the parser
6296is recovering from a syntax error, and 0 otherwise.
6297@xref{Error Recovery}.
6298@end deffn
6299
6300@deffn {Variable} yychar
6301Variable containing either the lookahead token, or @code{YYEOF} when the
6302lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
6303has been performed so the next token is not yet known.
6304Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
6305Actions}).
6306@xref{Lookahead, ,Lookahead Tokens}.
6307@end deffn
6308
6309@deffn {Macro} yyclearin;
6310Discard the current lookahead token. This is useful primarily in
6311error rules.
6312Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
6313Semantic Actions}).
6314@xref{Error Recovery}.
6315@end deffn
6316
6317@deffn {Macro} yyerrok;
6318Resume generating error messages immediately for subsequent syntax
6319errors. This is useful primarily in error rules.
6320@xref{Error Recovery}.
6321@end deffn
6322
6323@deffn {Variable} yylloc
6324Variable containing the lookahead token location when @code{yychar} is not set
6325to @code{YYEMPTY} or @code{YYEOF}.
6326Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
6327Actions}).
6328@xref{Actions and Locations, ,Actions and Locations}.
6329@end deffn
6330
6331@deffn {Variable} yylval
6332Variable containing the lookahead token semantic value when @code{yychar} is
6333not set to @code{YYEMPTY} or @code{YYEOF}.
6334Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
6335Actions}).
6336@xref{Actions, ,Actions}.
6337@end deffn
6338
6339@deffn {Value} @@$
6340@findex @@$
6341Acts like a structure variable containing information on the textual location
6342of the grouping made by the current rule. @xref{Locations, ,
6343Tracking Locations}.
6344
6345@c Check if those paragraphs are still useful or not.
6346
6347@c @example
6348@c struct @{
6349@c int first_line, last_line;
6350@c int first_column, last_column;
6351@c @};
6352@c @end example
6353
6354@c Thus, to get the starting line number of the third component, you would
6355@c use @samp{@@3.first_line}.
6356
6357@c In order for the members of this structure to contain valid information,
6358@c you must make @code{yylex} supply this information about each token.
6359@c If you need only certain members, then @code{yylex} need only fill in
6360@c those members.
6361
6362@c The use of this feature makes the parser noticeably slower.
6363@end deffn
6364
6365@deffn {Value} @@@var{n}
6366@findex @@@var{n}
6367Acts like a structure variable containing information on the textual location
6368of the @var{n}th component of the current rule. @xref{Locations, ,
6369Tracking Locations}.
6370@end deffn
6371
6372@node Internationalization
6373@section Parser Internationalization
6374@cindex internationalization
6375@cindex i18n
6376@cindex NLS
6377@cindex gettext
6378@cindex bison-po
6379
6380A Bison-generated parser can print diagnostics, including error and
6381tracing messages. By default, they appear in English. However, Bison
6382also supports outputting diagnostics in the user's native language. To
6383make this work, the user should set the usual environment variables.
6384@xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
6385For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
6386set the user's locale to French Canadian using the UTF-8
6387encoding. The exact set of available locales depends on the user's
6388installation.
6389
6390The maintainer of a package that uses a Bison-generated parser enables
6391the internationalization of the parser's output through the following
6392steps. Here we assume a package that uses GNU Autoconf and
6393GNU Automake.
6394
6395@enumerate
6396@item
6397@cindex bison-i18n.m4
6398Into the directory containing the GNU Autoconf macros used
6399by the package---often called @file{m4}---copy the
6400@file{bison-i18n.m4} file installed by Bison under
6401@samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
6402For example:
6403
6404@example
6405cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
6406@end example
6407
6408@item
6409@findex BISON_I18N
6410@vindex BISON_LOCALEDIR
6411@vindex YYENABLE_NLS
6412In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
6413invocation, add an invocation of @code{BISON_I18N}. This macro is
6414defined in the file @file{bison-i18n.m4} that you copied earlier. It
6415causes @samp{configure} to find the value of the
6416@code{BISON_LOCALEDIR} variable, and it defines the source-language
6417symbol @code{YYENABLE_NLS} to enable translations in the
6418Bison-generated parser.
6419
6420@item
6421In the @code{main} function of your program, designate the directory
6422containing Bison's runtime message catalog, through a call to
6423@samp{bindtextdomain} with domain name @samp{bison-runtime}.
6424For example:
6425
6426@example
6427bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
6428@end example
6429
6430Typically this appears after any other call @code{bindtextdomain
6431(PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
6432@samp{BISON_LOCALEDIR} to be defined as a string through the
6433@file{Makefile}.
6434
6435@item
6436In the @file{Makefile.am} that controls the compilation of the @code{main}
6437function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
6438either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
6439
6440@example
6441DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6442@end example
6443
6444or:
6445
6446@example
6447AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6448@end example
6449
6450@item
6451Finally, invoke the command @command{autoreconf} to generate the build
6452infrastructure.
6453@end enumerate
6454
6455
6456@node Algorithm
6457@chapter The Bison Parser Algorithm
6458@cindex Bison parser algorithm
6459@cindex algorithm of parser
6460@cindex shifting
6461@cindex reduction
6462@cindex parser stack
6463@cindex stack, parser
6464
6465As Bison reads tokens, it pushes them onto a stack along with their
6466semantic values. The stack is called the @dfn{parser stack}. Pushing a
6467token is traditionally called @dfn{shifting}.
6468
6469For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
6470@samp{3} to come. The stack will have four elements, one for each token
6471that was shifted.
6472
6473But the stack does not always have an element for each token read. When
6474the last @var{n} tokens and groupings shifted match the components of a
6475grammar rule, they can be combined according to that rule. This is called
6476@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
6477single grouping whose symbol is the result (left hand side) of that rule.
6478Running the rule's action is part of the process of reduction, because this
6479is what computes the semantic value of the resulting grouping.
6480
6481For example, if the infix calculator's parser stack contains this:
6482
6483@example
64841 + 5 * 3
6485@end example
6486
6487@noindent
6488and the next input token is a newline character, then the last three
6489elements can be reduced to 15 via the rule:
6490
6491@example
6492expr: expr '*' expr;
6493@end example
6494
6495@noindent
6496Then the stack contains just these three elements:
6497
6498@example
64991 + 15
6500@end example
6501
6502@noindent
6503At this point, another reduction can be made, resulting in the single value
650416. Then the newline token can be shifted.
6505
6506The parser tries, by shifts and reductions, to reduce the entire input down
6507to a single grouping whose symbol is the grammar's start-symbol
6508(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
6509
6510This kind of parser is known in the literature as a bottom-up parser.
6511
6512@menu
6513* Lookahead:: Parser looks one token ahead when deciding what to do.
6514* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
6515* Precedence:: Operator precedence works by resolving conflicts.
6516* Contextual Precedence:: When an operator's precedence depends on context.
6517* Parser States:: The parser is a finite-state-machine with stack.
6518* Reduce/Reduce:: When two rules are applicable in the same situation.
6519* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
6520* Tuning LR:: How to tune fundamental aspects of LR-based parsing.
6521* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
6522* Memory Management:: What happens when memory is exhausted. How to avoid it.
6523@end menu
6524
6525@node Lookahead
6526@section Lookahead Tokens
6527@cindex lookahead token
6528
6529The Bison parser does @emph{not} always reduce immediately as soon as the
6530last @var{n} tokens and groupings match a rule. This is because such a
6531simple strategy is inadequate to handle most languages. Instead, when a
6532reduction is possible, the parser sometimes ``looks ahead'' at the next
6533token in order to decide what to do.
6534
6535When a token is read, it is not immediately shifted; first it becomes the
6536@dfn{lookahead token}, which is not on the stack. Now the parser can
6537perform one or more reductions of tokens and groupings on the stack, while
6538the lookahead token remains off to the side. When no more reductions
6539should take place, the lookahead token is shifted onto the stack. This
6540does not mean that all possible reductions have been done; depending on the
6541token type of the lookahead token, some rules may choose to delay their
6542application.
6543
6544Here is a simple case where lookahead is needed. These three rules define
6545expressions which contain binary addition operators and postfix unary
6546factorial operators (@samp{!}), and allow parentheses for grouping.
6547
6548@example
6549@group
6550expr: term '+' expr
6551 | term
6552 ;
6553@end group
6554
6555@group
6556term: '(' expr ')'
6557 | term '!'
6558 | NUMBER
6559 ;
6560@end group
6561@end example
6562
6563Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
6564should be done? If the following token is @samp{)}, then the first three
6565tokens must be reduced to form an @code{expr}. This is the only valid
6566course, because shifting the @samp{)} would produce a sequence of symbols
6567@w{@code{term ')'}}, and no rule allows this.
6568
6569If the following token is @samp{!}, then it must be shifted immediately so
6570that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
6571parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
6572@code{expr}. It would then be impossible to shift the @samp{!} because
6573doing so would produce on the stack the sequence of symbols @code{expr
6574'!'}. No rule allows that sequence.
6575
6576@vindex yychar
6577@vindex yylval
6578@vindex yylloc
6579The lookahead token is stored in the variable @code{yychar}.
6580Its semantic value and location, if any, are stored in the variables
6581@code{yylval} and @code{yylloc}.
6582@xref{Action Features, ,Special Features for Use in Actions}.
6583
6584@node Shift/Reduce
6585@section Shift/Reduce Conflicts
6586@cindex conflicts
6587@cindex shift/reduce conflicts
6588@cindex dangling @code{else}
6589@cindex @code{else}, dangling
6590
6591Suppose we are parsing a language which has if-then and if-then-else
6592statements, with a pair of rules like this:
6593
6594@example
6595@group
6596if_stmt:
6597 IF expr THEN stmt
6598 | IF expr THEN stmt ELSE stmt
6599 ;
6600@end group
6601@end example
6602
6603@noindent
6604Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
6605terminal symbols for specific keyword tokens.
6606
6607When the @code{ELSE} token is read and becomes the lookahead token, the
6608contents of the stack (assuming the input is valid) are just right for
6609reduction by the first rule. But it is also legitimate to shift the
6610@code{ELSE}, because that would lead to eventual reduction by the second
6611rule.
6612
6613This situation, where either a shift or a reduction would be valid, is
6614called a @dfn{shift/reduce conflict}. Bison is designed to resolve
6615these conflicts by choosing to shift, unless otherwise directed by
6616operator precedence declarations. To see the reason for this, let's
6617contrast it with the other alternative.
6618
6619Since the parser prefers to shift the @code{ELSE}, the result is to attach
6620the else-clause to the innermost if-statement, making these two inputs
6621equivalent:
6622
6623@example
6624if x then if y then win (); else lose;
6625
6626if x then do; if y then win (); else lose; end;
6627@end example
6628
6629But if the parser chose to reduce when possible rather than shift, the
6630result would be to attach the else-clause to the outermost if-statement,
6631making these two inputs equivalent:
6632
6633@example
6634if x then if y then win (); else lose;
6635
6636if x then do; if y then win (); end; else lose;
6637@end example
6638
6639The conflict exists because the grammar as written is ambiguous: either
6640parsing of the simple nested if-statement is legitimate. The established
6641convention is that these ambiguities are resolved by attaching the
6642else-clause to the innermost if-statement; this is what Bison accomplishes
6643by choosing to shift rather than reduce. (It would ideally be cleaner to
6644write an unambiguous grammar, but that is very hard to do in this case.)
6645This particular ambiguity was first encountered in the specifications of
6646Algol 60 and is called the ``dangling @code{else}'' ambiguity.
6647
6648To avoid warnings from Bison about predictable, legitimate shift/reduce
6649conflicts, use the @code{%expect @var{n}} declaration.
6650There will be no warning as long as the number of shift/reduce conflicts
6651is exactly @var{n}, and Bison will report an error if there is a
6652different number.
6653@xref{Expect Decl, ,Suppressing Conflict Warnings}.
6654
6655The definition of @code{if_stmt} above is solely to blame for the
6656conflict, but the conflict does not actually appear without additional
6657rules. Here is a complete Bison grammar file that actually manifests
6658the conflict:
6659
6660@example
6661@group
6662%token IF THEN ELSE variable
6663%%
6664@end group
6665@group
6666stmt: expr
6667 | if_stmt
6668 ;
6669@end group
6670
6671@group
6672if_stmt:
6673 IF expr THEN stmt
6674 | IF expr THEN stmt ELSE stmt
6675 ;
6676@end group
6677
6678expr: variable
6679 ;
6680@end example
6681
6682@node Precedence
6683@section Operator Precedence
6684@cindex operator precedence
6685@cindex precedence of operators
6686
6687Another situation where shift/reduce conflicts appear is in arithmetic
6688expressions. Here shifting is not always the preferred resolution; the
6689Bison declarations for operator precedence allow you to specify when to
6690shift and when to reduce.
6691
6692@menu
6693* Why Precedence:: An example showing why precedence is needed.
6694* Using Precedence:: How to specify precedence and associativity.
6695* Precedence Only:: How to specify precedence only.
6696* Precedence Examples:: How these features are used in the previous example.
6697* How Precedence:: How they work.
6698@end menu
6699
6700@node Why Precedence
6701@subsection When Precedence is Needed
6702
6703Consider the following ambiguous grammar fragment (ambiguous because the
6704input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
6705
6706@example
6707@group
6708expr: expr '-' expr
6709 | expr '*' expr
6710 | expr '<' expr
6711 | '(' expr ')'
6712 @dots{}
6713 ;
6714@end group
6715@end example
6716
6717@noindent
6718Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
6719should it reduce them via the rule for the subtraction operator? It
6720depends on the next token. Of course, if the next token is @samp{)}, we
6721must reduce; shifting is invalid because no single rule can reduce the
6722token sequence @w{@samp{- 2 )}} or anything starting with that. But if
6723the next token is @samp{*} or @samp{<}, we have a choice: either
6724shifting or reduction would allow the parse to complete, but with
6725different results.
6726
6727To decide which one Bison should do, we must consider the results. If
6728the next operator token @var{op} is shifted, then it must be reduced
6729first in order to permit another opportunity to reduce the difference.
6730The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
6731hand, if the subtraction is reduced before shifting @var{op}, the result
6732is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
6733reduce should depend on the relative precedence of the operators
6734@samp{-} and @var{op}: @samp{*} should be shifted first, but not
6735@samp{<}.
6736
6737@cindex associativity
6738What about input such as @w{@samp{1 - 2 - 5}}; should this be
6739@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
6740operators we prefer the former, which is called @dfn{left association}.
6741The latter alternative, @dfn{right association}, is desirable for
6742assignment operators. The choice of left or right association is a
6743matter of whether the parser chooses to shift or reduce when the stack
6744contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
6745makes right-associativity.
6746
6747@node Using Precedence
6748@subsection Specifying Operator Precedence
6749@findex %left
6750@findex %nonassoc
6751@findex %precedence
6752@findex %right
6753
6754Bison allows you to specify these choices with the operator precedence
6755declarations @code{%left} and @code{%right}. Each such declaration
6756contains a list of tokens, which are operators whose precedence and
6757associativity is being declared. The @code{%left} declaration makes all
6758those operators left-associative and the @code{%right} declaration makes
6759them right-associative. A third alternative is @code{%nonassoc}, which
6760declares that it is a syntax error to find the same operator twice ``in a
6761row''.
6762The last alternative, @code{%precedence}, allows to define only
6763precedence and no associativity at all. As a result, any
6764associativity-related conflict that remains will be reported as an
6765compile-time error. The directive @code{%nonassoc} creates run-time
6766error: using the operator in a associative way is a syntax error. The
6767directive @code{%precedence} creates compile-time errors: an operator
6768@emph{can} be involved in an associativity-related conflict, contrary to
6769what expected the grammar author.
6770
6771The relative precedence of different operators is controlled by the
6772order in which they are declared. The first precedence/associativity
6773declaration in the file declares the operators whose
6774precedence is lowest, the next such declaration declares the operators
6775whose precedence is a little higher, and so on.
6776
6777@node Precedence Only
6778@subsection Specifying Precedence Only
6779@findex %precedence
6780
6781Since POSIX Yacc defines only @code{%left}, @code{%right}, and
6782@code{%nonassoc}, which all defines precedence and associativity, little
6783attention is paid to the fact that precedence cannot be defined without
6784defining associativity. Yet, sometimes, when trying to solve a
6785conflict, precedence suffices. In such a case, using @code{%left},
6786@code{%right}, or @code{%nonassoc} might hide future (associativity
6787related) conflicts that would remain hidden.
6788
6789The dangling @code{else} ambiguity (@pxref{Shift/Reduce, , Shift/Reduce
6790Conflicts}) can be solved explicitly. This shift/reduce conflicts occurs
6791in the following situation, where the period denotes the current parsing
6792state:
6793
6794@example
6795if @var{e1} then if @var{e2} then @var{s1} . else @var{s2}
6796@end example
6797
6798The conflict involves the reduction of the rule @samp{IF expr THEN
6799stmt}, which precedence is by default that of its last token
6800(@code{THEN}), and the shifting of the token @code{ELSE}. The usual
6801disambiguation (attach the @code{else} to the closest @code{if}),
6802shifting must be preferred, i.e., the precedence of @code{ELSE} must be
6803higher than that of @code{THEN}. But neither is expected to be involved
6804in an associativity related conflict, which can be specified as follows.
6805
6806@example
6807%precedence THEN
6808%precedence ELSE
6809@end example
6810
6811The unary-minus is another typical example where associativity is
6812usually over-specified, see @ref{Infix Calc, , Infix Notation
6813Calculator: @code{calc}}. The @code{%left} directive is traditionally
6814used to declare the precedence of @code{NEG}, which is more than needed
6815since it also defines its associativity. While this is harmless in the
6816traditional example, who knows how @code{NEG} might be used in future
6817evolutions of the grammar@dots{}
6818
6819@node Precedence Examples
6820@subsection Precedence Examples
6821
6822In our example, we would want the following declarations:
6823
6824@example
6825%left '<'
6826%left '-'
6827%left '*'
6828@end example
6829
6830In a more complete example, which supports other operators as well, we
6831would declare them in groups of equal precedence. For example, @code{'+'} is
6832declared with @code{'-'}:
6833
6834@example
6835%left '<' '>' '=' NE LE GE
6836%left '+' '-'
6837%left '*' '/'
6838@end example
6839
6840@noindent
6841(Here @code{NE} and so on stand for the operators for ``not equal''
6842and so on. We assume that these tokens are more than one character long
6843and therefore are represented by names, not character literals.)
6844
6845@node How Precedence
6846@subsection How Precedence Works
6847
6848The first effect of the precedence declarations is to assign precedence
6849levels to the terminal symbols declared. The second effect is to assign
6850precedence levels to certain rules: each rule gets its precedence from
6851the last terminal symbol mentioned in the components. (You can also
6852specify explicitly the precedence of a rule. @xref{Contextual
6853Precedence, ,Context-Dependent Precedence}.)
6854
6855Finally, the resolution of conflicts works by comparing the precedence
6856of the rule being considered with that of the lookahead token. If the
6857token's precedence is higher, the choice is to shift. If the rule's
6858precedence is higher, the choice is to reduce. If they have equal
6859precedence, the choice is made based on the associativity of that
6860precedence level. The verbose output file made by @samp{-v}
6861(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
6862resolved.
6863
6864Not all rules and not all tokens have precedence. If either the rule or
6865the lookahead token has no precedence, then the default is to shift.
6866
6867@node Contextual Precedence
6868@section Context-Dependent Precedence
6869@cindex context-dependent precedence
6870@cindex unary operator precedence
6871@cindex precedence, context-dependent
6872@cindex precedence, unary operator
6873@findex %prec
6874
6875Often the precedence of an operator depends on the context. This sounds
6876outlandish at first, but it is really very common. For example, a minus
6877sign typically has a very high precedence as a unary operator, and a
6878somewhat lower precedence (lower than multiplication) as a binary operator.
6879
6880The Bison precedence declarations
6881can only be used once for a given token; so a token has
6882only one precedence declared in this way. For context-dependent
6883precedence, you need to use an additional mechanism: the @code{%prec}
6884modifier for rules.
6885
6886The @code{%prec} modifier declares the precedence of a particular rule by
6887specifying a terminal symbol whose precedence should be used for that rule.
6888It's not necessary for that symbol to appear otherwise in the rule. The
6889modifier's syntax is:
6890
6891@example
6892%prec @var{terminal-symbol}
6893@end example
6894
6895@noindent
6896and it is written after the components of the rule. Its effect is to
6897assign the rule the precedence of @var{terminal-symbol}, overriding
6898the precedence that would be deduced for it in the ordinary way. The
6899altered rule precedence then affects how conflicts involving that rule
6900are resolved (@pxref{Precedence, ,Operator Precedence}).
6901
6902Here is how @code{%prec} solves the problem of unary minus. First, declare
6903a precedence for a fictitious terminal symbol named @code{UMINUS}. There
6904are no tokens of this type, but the symbol serves to stand for its
6905precedence:
6906
6907@example
6908@dots{}
6909%left '+' '-'
6910%left '*'
6911%left UMINUS
6912@end example
6913
6914Now the precedence of @code{UMINUS} can be used in specific rules:
6915
6916@example
6917@group
6918exp: @dots{}
6919 | exp '-' exp
6920 @dots{}
6921 | '-' exp %prec UMINUS
6922@end group
6923@end example
6924
6925@ifset defaultprec
6926If you forget to append @code{%prec UMINUS} to the rule for unary
6927minus, Bison silently assumes that minus has its usual precedence.
6928This kind of problem can be tricky to debug, since one typically
6929discovers the mistake only by testing the code.
6930
6931The @code{%no-default-prec;} declaration makes it easier to discover
6932this kind of problem systematically. It causes rules that lack a
6933@code{%prec} modifier to have no precedence, even if the last terminal
6934symbol mentioned in their components has a declared precedence.
6935
6936If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
6937for all rules that participate in precedence conflict resolution.
6938Then you will see any shift/reduce conflict until you tell Bison how
6939to resolve it, either by changing your grammar or by adding an
6940explicit precedence. This will probably add declarations to the
6941grammar, but it helps to protect against incorrect rule precedences.
6942
6943The effect of @code{%no-default-prec;} can be reversed by giving
6944@code{%default-prec;}, which is the default.
6945@end ifset
6946
6947@node Parser States
6948@section Parser States
6949@cindex finite-state machine
6950@cindex parser state
6951@cindex state (of parser)
6952
6953The function @code{yyparse} is implemented using a finite-state machine.
6954The values pushed on the parser stack are not simply token type codes; they
6955represent the entire sequence of terminal and nonterminal symbols at or
6956near the top of the stack. The current state collects all the information
6957about previous input which is relevant to deciding what to do next.
6958
6959Each time a lookahead token is read, the current parser state together
6960with the type of lookahead token are looked up in a table. This table
6961entry can say, ``Shift the lookahead token.'' In this case, it also
6962specifies the new parser state, which is pushed onto the top of the
6963parser stack. Or it can say, ``Reduce using rule number @var{n}.''
6964This means that a certain number of tokens or groupings are taken off
6965the top of the stack, and replaced by one grouping. In other words,
6966that number of states are popped from the stack, and one new state is
6967pushed.
6968
6969There is one other alternative: the table can say that the lookahead token
6970is erroneous in the current state. This causes error processing to begin
6971(@pxref{Error Recovery}).
6972
6973@node Reduce/Reduce
6974@section Reduce/Reduce Conflicts
6975@cindex reduce/reduce conflict
6976@cindex conflicts, reduce/reduce
6977
6978A reduce/reduce conflict occurs if there are two or more rules that apply
6979to the same sequence of input. This usually indicates a serious error
6980in the grammar.
6981
6982For example, here is an erroneous attempt to define a sequence
6983of zero or more @code{word} groupings.
6984
6985@example
6986sequence: /* empty */
6987 @{ printf ("empty sequence\n"); @}
6988 | maybeword
6989 | sequence word
6990 @{ printf ("added word %s\n", $2); @}
6991 ;
6992
6993maybeword: /* empty */
6994 @{ printf ("empty maybeword\n"); @}
6995 | word
6996 @{ printf ("single word %s\n", $1); @}
6997 ;
6998@end example
6999
7000@noindent
7001The error is an ambiguity: there is more than one way to parse a single
7002@code{word} into a @code{sequence}. It could be reduced to a
7003@code{maybeword} and then into a @code{sequence} via the second rule.
7004Alternatively, nothing-at-all could be reduced into a @code{sequence}
7005via the first rule, and this could be combined with the @code{word}
7006using the third rule for @code{sequence}.
7007
7008There is also more than one way to reduce nothing-at-all into a
7009@code{sequence}. This can be done directly via the first rule,
7010or indirectly via @code{maybeword} and then the second rule.
7011
7012You might think that this is a distinction without a difference, because it
7013does not change whether any particular input is valid or not. But it does
7014affect which actions are run. One parsing order runs the second rule's
7015action; the other runs the first rule's action and the third rule's action.
7016In this example, the output of the program changes.
7017
7018Bison resolves a reduce/reduce conflict by choosing to use the rule that
7019appears first in the grammar, but it is very risky to rely on this. Every
7020reduce/reduce conflict must be studied and usually eliminated. Here is the
7021proper way to define @code{sequence}:
7022
7023@example
7024sequence: /* empty */
7025 @{ printf ("empty sequence\n"); @}
7026 | sequence word
7027 @{ printf ("added word %s\n", $2); @}
7028 ;
7029@end example
7030
7031Here is another common error that yields a reduce/reduce conflict:
7032
7033@example
7034sequence: /* empty */
7035 | sequence words
7036 | sequence redirects
7037 ;
7038
7039words: /* empty */
7040 | words word
7041 ;
7042
7043redirects:/* empty */
7044 | redirects redirect
7045 ;
7046@end example
7047
7048@noindent
7049The intention here is to define a sequence which can contain either
7050@code{word} or @code{redirect} groupings. The individual definitions of
7051@code{sequence}, @code{words} and @code{redirects} are error-free, but the
7052three together make a subtle ambiguity: even an empty input can be parsed
7053in infinitely many ways!
7054
7055Consider: nothing-at-all could be a @code{words}. Or it could be two
7056@code{words} in a row, or three, or any number. It could equally well be a
7057@code{redirects}, or two, or any number. Or it could be a @code{words}
7058followed by three @code{redirects} and another @code{words}. And so on.
7059
7060Here are two ways to correct these rules. First, to make it a single level
7061of sequence:
7062
7063@example
7064sequence: /* empty */
7065 | sequence word
7066 | sequence redirect
7067 ;
7068@end example
7069
7070Second, to prevent either a @code{words} or a @code{redirects}
7071from being empty:
7072
7073@example
7074sequence: /* empty */
7075 | sequence words
7076 | sequence redirects
7077 ;
7078
7079words: word
7080 | words word
7081 ;
7082
7083redirects:redirect
7084 | redirects redirect
7085 ;
7086@end example
7087
7088@node Mystery Conflicts
7089@section Mysterious Reduce/Reduce Conflicts
7090@cindex Mysterious Conflicts
7091
7092Sometimes reduce/reduce conflicts can occur that don't look warranted.
7093Here is an example:
7094
7095@example
7096@group
7097%token ID
7098
7099%%
7100def: param_spec return_spec ','
7101 ;
7102param_spec:
7103 type
7104 | name_list ':' type
7105 ;
7106@end group
7107@group
7108return_spec:
7109 type
7110 | name ':' type
7111 ;
7112@end group
7113@group
7114type: ID
7115 ;
7116@end group
7117@group
7118name: ID
7119 ;
7120name_list:
7121 name
7122 | name ',' name_list
7123 ;
7124@end group
7125@end example
7126
7127It would seem that this grammar can be parsed with only a single token
7128of lookahead: when a @code{param_spec} is being read, an @code{ID} is
7129a @code{name} if a comma or colon follows, or a @code{type} if another
7130@code{ID} follows. In other words, this grammar is LR(1).
7131
7132@cindex LR
7133@cindex LALR
7134However, for historical reasons, Bison cannot by default handle all
7135LR(1) grammars.
7136In this grammar, two contexts, that after an @code{ID} at the beginning
7137of a @code{param_spec} and likewise at the beginning of a
7138@code{return_spec}, are similar enough that Bison assumes they are the
7139same.
7140They appear similar because the same set of rules would be
7141active---the rule for reducing to a @code{name} and that for reducing to
7142a @code{type}. Bison is unable to determine at that stage of processing
7143that the rules would require different lookahead tokens in the two
7144contexts, so it makes a single parser state for them both. Combining
7145the two contexts causes a conflict later. In parser terminology, this
7146occurrence means that the grammar is not LALR(1).
7147
7148@cindex IELR
7149@cindex canonical LR
7150For many practical grammars (specifically those that fall into the non-LR(1)
7151class), the limitations of LALR(1) result in difficulties beyond just
7152mysterious reduce/reduce conflicts. The best way to fix all these problems
7153is to select a different parser table construction algorithm. Either
7154IELR(1) or canonical LR(1) would suffice, but the former is more efficient
7155and easier to debug during development. @xref{LR Table Construction}, for
7156details. (Bison's IELR(1) and canonical LR(1) implementations are
7157experimental. More user feedback will help to stabilize them.)
7158
7159If you instead wish to work around LALR(1)'s limitations, you
7160can often fix a mysterious conflict by identifying the two parser states
7161that are being confused, and adding something to make them look
7162distinct. In the above example, adding one rule to
7163@code{return_spec} as follows makes the problem go away:
7164
7165@example
7166@group
7167%token BOGUS
7168@dots{}
7169%%
7170@dots{}
7171return_spec:
7172 type
7173 | name ':' type
7174 /* This rule is never used. */
7175 | ID BOGUS
7176 ;
7177@end group
7178@end example
7179
7180This corrects the problem because it introduces the possibility of an
7181additional active rule in the context after the @code{ID} at the beginning of
7182@code{return_spec}. This rule is not active in the corresponding context
7183in a @code{param_spec}, so the two contexts receive distinct parser states.
7184As long as the token @code{BOGUS} is never generated by @code{yylex},
7185the added rule cannot alter the way actual input is parsed.
7186
7187In this particular example, there is another way to solve the problem:
7188rewrite the rule for @code{return_spec} to use @code{ID} directly
7189instead of via @code{name}. This also causes the two confusing
7190contexts to have different sets of active rules, because the one for
7191@code{return_spec} activates the altered rule for @code{return_spec}
7192rather than the one for @code{name}.
7193
7194@example
7195param_spec:
7196 type
7197 | name_list ':' type
7198 ;
7199return_spec:
7200 type
7201 | ID ':' type
7202 ;
7203@end example
7204
7205For a more detailed exposition of LALR(1) parsers and parser
7206generators, @pxref{Bibliography,,DeRemer 1982}.
7207
7208@node Tuning LR
7209@section Tuning LR
7210
7211The default behavior of Bison's LR-based parsers is chosen mostly for
7212historical reasons, but that behavior is often not robust. For example, in
7213the previous section, we discussed the mysterious conflicts that can be
7214produced by LALR(1), Bison's default parser table construction algorithm.
7215Another example is Bison's @code{%define parse.error verbose} directive,
7216which instructs the generated parser to produce verbose syntax error
7217messages, which can sometimes contain incorrect information.
7218
7219In this section, we explore several modern features of Bison that allow you
7220to tune fundamental aspects of the generated LR-based parsers. Some of
7221these features easily eliminate shortcomings like those mentioned above.
7222Others can be helpful purely for understanding your parser.
7223
7224Most of the features discussed in this section are still experimental. More
7225user feedback will help to stabilize them.
7226
7227@menu
7228* LR Table Construction:: Choose a different construction algorithm.
7229* Default Reductions:: Disable default reductions.
7230* LAC:: Correct lookahead sets in the parser states.
7231* Unreachable States:: Keep unreachable parser states for debugging.
7232@end menu
7233
7234@node LR Table Construction
7235@subsection LR Table Construction
7236@cindex Mysterious Conflict
7237@cindex LALR
7238@cindex IELR
7239@cindex canonical LR
7240@findex %define lr.type
7241
7242For historical reasons, Bison constructs LALR(1) parser tables by default.
7243However, LALR does not possess the full language-recognition power of LR.
7244As a result, the behavior of parsers employing LALR parser tables is often
7245mysterious. We presented a simple example of this effect in @ref{Mystery
7246Conflicts}.
7247
7248As we also demonstrated in that example, the traditional approach to
7249eliminating such mysterious behavior is to restructure the grammar.
7250Unfortunately, doing so correctly is often difficult. Moreover, merely
7251discovering that LALR causes mysterious behavior in your parser can be
7252difficult as well.
7253
7254Fortunately, Bison provides an easy way to eliminate the possibility of such
7255mysterious behavior altogether. You simply need to activate a more powerful
7256parser table construction algorithm by using the @code{%define lr.type}
7257directive.
7258
7259@deffn {Directive} {%define lr.type @var{TYPE}}
7260Specify the type of parser tables within the LR(1) family. The accepted
7261values for @var{TYPE} are:
7262
7263@itemize
7264@item @code{lalr} (default)
7265@item @code{ielr}
7266@item @code{canonical-lr}
7267@end itemize
7268
7269(This feature is experimental. More user feedback will help to stabilize
7270it.)
7271@end deffn
7272
7273For example, to activate IELR, you might add the following directive to you
7274grammar file:
7275
7276@example
7277%define lr.type ielr
7278@end example
7279
7280@noindent For the example in @ref{Mystery Conflicts}, the mysterious
7281conflict is then eliminated, so there is no need to invest time in
7282comprehending the conflict or restructuring the grammar to fix it. If,
7283during future development, the grammar evolves such that all mysterious
7284behavior would have disappeared using just LALR, you need not fear that
7285continuing to use IELR will result in unnecessarily large parser tables.
7286That is, IELR generates LALR tables when LALR (using a deterministic parsing
7287algorithm) is sufficient to support the full language-recognition power of
7288LR. Thus, by enabling IELR at the start of grammar development, you can
7289safely and completely eliminate the need to consider LALR's shortcomings.
7290
7291While IELR is almost always preferable, there are circumstances where LALR
7292or the canonical LR parser tables described by Knuth
7293(@pxref{Bibliography,,Knuth 1965}) can be useful. Here we summarize the
7294relative advantages of each parser table construction algorithm within
7295Bison:
7296
7297@itemize
7298@item LALR
7299
7300There are at least two scenarios where LALR can be worthwhile:
7301
7302@itemize
7303@item GLR without static conflict resolution.
7304
7305@cindex GLR with LALR
7306When employing GLR parsers (@pxref{GLR Parsers}), if you do not resolve any
7307conflicts statically (for example, with @code{%left} or @code{%prec}), then
7308the parser explores all potential parses of any given input. In this case,
7309the choice of parser table construction algorithm is guaranteed not to alter
7310the language accepted by the parser. LALR parser tables are the smallest
7311parser tables Bison can currently construct, so they may then be preferable.
7312Nevertheless, once you begin to resolve conflicts statically, GLR behaves
7313more like a deterministic parser in the syntactic contexts where those
7314conflicts appear, and so either IELR or canonical LR can then be helpful to
7315avoid LALR's mysterious behavior.
7316
7317@item Malformed grammars.
7318
7319Occasionally during development, an especially malformed grammar with a
7320major recurring flaw may severely impede the IELR or canonical LR parser
7321table construction algorithm. LALR can be a quick way to construct parser
7322tables in order to investigate such problems while ignoring the more subtle
7323differences from IELR and canonical LR.
7324@end itemize
7325
7326@item IELR
7327
7328IELR (Inadequacy Elimination LR) is a minimal LR algorithm. That is, given
7329any grammar (LR or non-LR), parsers using IELR or canonical LR parser tables
7330always accept exactly the same set of sentences. However, like LALR, IELR
7331merges parser states during parser table construction so that the number of
7332parser states is often an order of magnitude less than for canonical LR.
7333More importantly, because canonical LR's extra parser states may contain
7334duplicate conflicts in the case of non-LR grammars, the number of conflicts
7335for IELR is often an order of magnitude less as well. This effect can
7336significantly reduce the complexity of developing a grammar.
7337
7338@item Canonical LR
7339
7340@cindex delayed syntax error detection
7341@cindex LAC
7342@findex %nonassoc
7343While inefficient, canonical LR parser tables can be an interesting means to
7344explore a grammar because they possess a property that IELR and LALR tables
7345do not. That is, if @code{%nonassoc} is not used and default reductions are
7346left disabled (@pxref{Default Reductions}), then, for every left context of
7347every canonical LR state, the set of tokens accepted by that state is
7348guaranteed to be the exact set of tokens that is syntactically acceptable in
7349that left context. It might then seem that an advantage of canonical LR
7350parsers in production is that, under the above constraints, they are
7351guaranteed to detect a syntax error as soon as possible without performing
7352any unnecessary reductions. However, IELR parsers that use LAC are also
7353able to achieve this behavior without sacrificing @code{%nonassoc} or
7354default reductions. For details and a few caveats of LAC, @pxref{LAC}.
7355@end itemize
7356
7357For a more detailed exposition of the mysterious behavior in LALR parsers
7358and the benefits of IELR, @pxref{Bibliography,,Denny 2008 March}, and
7359@ref{Bibliography,,Denny 2010 November}.
7360
7361@node Default Reductions
7362@subsection Default Reductions
7363@cindex default reductions
7364@findex %define lr.default-reductions
7365@findex %nonassoc
7366
7367After parser table construction, Bison identifies the reduction with the
7368largest lookahead set in each parser state. To reduce the size of the
7369parser state, traditional Bison behavior is to remove that lookahead set and
7370to assign that reduction to be the default parser action. Such a reduction
7371is known as a @dfn{default reduction}.
7372
7373Default reductions affect more than the size of the parser tables. They
7374also affect the behavior of the parser:
7375
7376@itemize
7377@item Delayed @code{yylex} invocations.
7378
7379@cindex delayed yylex invocations
7380@cindex consistent states
7381@cindex defaulted states
7382A @dfn{consistent state} is a state that has only one possible parser
7383action. If that action is a reduction and is encoded as a default
7384reduction, then that consistent state is called a @dfn{defaulted state}.
7385Upon reaching a defaulted state, a Bison-generated parser does not bother to
7386invoke @code{yylex} to fetch the next token before performing the reduction.
7387In other words, whether default reductions are enabled in consistent states
7388determines how soon a Bison-generated parser invokes @code{yylex} for a
7389token: immediately when it @emph{reaches} that token in the input or when it
7390eventually @emph{needs} that token as a lookahead to determine the next
7391parser action. Traditionally, default reductions are enabled, and so the
7392parser exhibits the latter behavior.
7393
7394The presence of defaulted states is an important consideration when
7395designing @code{yylex} and the grammar file. That is, if the behavior of
7396@code{yylex} can influence or be influenced by the semantic actions
7397associated with the reductions in defaulted states, then the delay of the
7398next @code{yylex} invocation until after those reductions is significant.
7399For example, the semantic actions might pop a scope stack that @code{yylex}
7400uses to determine what token to return. Thus, the delay might be necessary
7401to ensure that @code{yylex} does not look up the next token in a scope that
7402should already be considered closed.
7403
7404@item Delayed syntax error detection.
7405
7406@cindex delayed syntax error detection
7407When the parser fetches a new token by invoking @code{yylex}, it checks
7408whether there is an action for that token in the current parser state. The
7409parser detects a syntax error if and only if either (1) there is no action
7410for that token or (2) the action for that token is the error action (due to
7411the use of @code{%nonassoc}). However, if there is a default reduction in
7412that state (which might or might not be a defaulted state), then it is
7413impossible for condition 1 to exist. That is, all tokens have an action.
7414Thus, the parser sometimes fails to detect the syntax error until it reaches
7415a later state.
7416
7417@cindex LAC
7418@c If there's an infinite loop, default reductions can prevent an incorrect
7419@c sentence from being rejected.
7420While default reductions never cause the parser to accept syntactically
7421incorrect sentences, the delay of syntax error detection can have unexpected
7422effects on the behavior of the parser. However, the delay can be caused
7423anyway by parser state merging and the use of @code{%nonassoc}, and it can
7424be fixed by another Bison feature, LAC. We discuss the effects of delayed
7425syntax error detection and LAC more in the next section (@pxref{LAC}).
7426@end itemize
7427
7428For canonical LR, the only default reduction that Bison enables by default
7429is the accept action, which appears only in the accepting state, which has
7430no other action and is thus a defaulted state. However, the default accept
7431action does not delay any @code{yylex} invocation or syntax error detection
7432because the accept action ends the parse.
7433
7434For LALR and IELR, Bison enables default reductions in nearly all states by
7435default. There are only two exceptions. First, states that have a shift
7436action on the @code{error} token do not have default reductions because
7437delayed syntax error detection could then prevent the @code{error} token
7438from ever being shifted in that state. However, parser state merging can
7439cause the same effect anyway, and LAC fixes it in both cases, so future
7440versions of Bison might drop this exception when LAC is activated. Second,
7441GLR parsers do not record the default reduction as the action on a lookahead
7442token for which there is a conflict. The correct action in this case is to
7443split the parse instead.
7444
7445To adjust which states have default reductions enabled, use the
7446@code{%define lr.default-reductions} directive.
7447
7448@deffn {Directive} {%define lr.default-reductions @var{WHERE}}
7449Specify the kind of states that are permitted to contain default reductions.
7450The accepted values of @var{WHERE} are:
7451@itemize
7452@item @code{full} (default for LALR and IELR)
7453@item @code{consistent}
7454@item @code{accepting} (default for canonical LR)
7455@end itemize
7456
7457(The ability to specify where default reductions are permitted is
7458experimental. More user feedback will help to stabilize it.)
7459@end deffn
7460
7461@node LAC
7462@subsection LAC
7463@findex %define parse.lac
7464@cindex LAC
7465@cindex lookahead correction
7466
7467Canonical LR, IELR, and LALR can suffer from a couple of problems upon
7468encountering a syntax error. First, the parser might perform additional
7469parser stack reductions before discovering the syntax error. Such
7470reductions can perform user semantic actions that are unexpected because
7471they are based on an invalid token, and they cause error recovery to begin
7472in a different syntactic context than the one in which the invalid token was
7473encountered. Second, when verbose error messages are enabled (@pxref{Error
7474Reporting}), the expected token list in the syntax error message can both
7475contain invalid tokens and omit valid tokens.
7476
7477The culprits for the above problems are @code{%nonassoc}, default reductions
7478in inconsistent states (@pxref{Default Reductions}), and parser state
7479merging. Because IELR and LALR merge parser states, they suffer the most.
7480Canonical LR can suffer only if @code{%nonassoc} is used or if default
7481reductions are enabled for inconsistent states.
7482
7483LAC (Lookahead Correction) is a new mechanism within the parsing algorithm
7484that solves these problems for canonical LR, IELR, and LALR without
7485sacrificing @code{%nonassoc}, default reductions, or state merging. You can
7486enable LAC with the @code{%define parse.lac} directive.
7487
7488@deffn {Directive} {%define parse.lac @var{VALUE}}
7489Enable LAC to improve syntax error handling.
7490@itemize
7491@item @code{none} (default)
7492@item @code{full}
7493@end itemize
7494(This feature is experimental. More user feedback will help to stabilize
7495it. Moreover, it is currently only available for deterministic parsers in
7496C.)
7497@end deffn
7498
7499Conceptually, the LAC mechanism is straight-forward. Whenever the parser
7500fetches a new token from the scanner so that it can determine the next
7501parser action, it immediately suspends normal parsing and performs an
7502exploratory parse using a temporary copy of the normal parser state stack.
7503During this exploratory parse, the parser does not perform user semantic
7504actions. If the exploratory parse reaches a shift action, normal parsing
7505then resumes on the normal parser stacks. If the exploratory parse reaches
7506an error instead, the parser reports a syntax error. If verbose syntax
7507error messages are enabled, the parser must then discover the list of
7508expected tokens, so it performs a separate exploratory parse for each token
7509in the grammar.
7510
7511There is one subtlety about the use of LAC. That is, when in a consistent
7512parser state with a default reduction, the parser will not attempt to fetch
7513a token from the scanner because no lookahead is needed to determine the
7514next parser action. Thus, whether default reductions are enabled in
7515consistent states (@pxref{Default Reductions}) affects how soon the parser
7516detects a syntax error: immediately when it @emph{reaches} an erroneous
7517token or when it eventually @emph{needs} that token as a lookahead to
7518determine the next parser action. The latter behavior is probably more
7519intuitive, so Bison currently provides no way to achieve the former behavior
7520while default reductions are enabled in consistent states.
7521
7522Thus, when LAC is in use, for some fixed decision of whether to enable
7523default reductions in consistent states, canonical LR and IELR behave almost
7524exactly the same for both syntactically acceptable and syntactically
7525unacceptable input. While LALR still does not support the full
7526language-recognition power of canonical LR and IELR, LAC at least enables
7527LALR's syntax error handling to correctly reflect LALR's
7528language-recognition power.
7529
7530There are a few caveats to consider when using LAC:
7531
7532@itemize
7533@item Infinite parsing loops.
7534
7535IELR plus LAC does have one shortcoming relative to canonical LR. Some
7536parsers generated by Bison can loop infinitely. LAC does not fix infinite
7537parsing loops that occur between encountering a syntax error and detecting
7538it, but enabling canonical LR or disabling default reductions sometimes
7539does.
7540
7541@item Verbose error message limitations.
7542
7543Because of internationalization considerations, Bison-generated parsers
7544limit the size of the expected token list they are willing to report in a
7545verbose syntax error message. If the number of expected tokens exceeds that
7546limit, the list is simply dropped from the message. Enabling LAC can
7547increase the size of the list and thus cause the parser to drop it. Of
7548course, dropping the list is better than reporting an incorrect list.
7549
7550@item Performance.
7551
7552Because LAC requires many parse actions to be performed twice, it can have a
7553performance penalty. However, not all parse actions must be performed
7554twice. Specifically, during a series of default reductions in consistent
7555states and shift actions, the parser never has to initiate an exploratory
7556parse. Moreover, the most time-consuming tasks in a parse are often the
7557file I/O, the lexical analysis performed by the scanner, and the user's
7558semantic actions, but none of these are performed during the exploratory
7559parse. Finally, the base of the temporary stack used during an exploratory
7560parse is a pointer into the normal parser state stack so that the stack is
7561never physically copied. In our experience, the performance penalty of LAC
7562has proven insignificant for practical grammars.
7563@end itemize
7564
7565@node Unreachable States
7566@subsection Unreachable States
7567@findex %define lr.keep-unreachable-states
7568@cindex unreachable states
7569
7570If there exists no sequence of transitions from the parser's start state to
7571some state @var{s}, then Bison considers @var{s} to be an @dfn{unreachable
7572state}. A state can become unreachable during conflict resolution if Bison
7573disables a shift action leading to it from a predecessor state.
7574
7575By default, Bison removes unreachable states from the parser after conflict
7576resolution because they are useless in the generated parser. However,
7577keeping unreachable states is sometimes useful when trying to understand the
7578relationship between the parser and the grammar.
7579
7580@deffn {Directive} {%define lr.keep-unreachable-states @var{VALUE}}
7581Request that Bison allow unreachable states to remain in the parser tables.
7582@var{VALUE} must be a Boolean. The default is @code{false}.
7583@end deffn
7584
7585There are a few caveats to consider:
7586
7587@itemize @bullet
7588@item Missing or extraneous warnings.
7589
7590Unreachable states may contain conflicts and may use rules not used in any
7591other state. Thus, keeping unreachable states may induce warnings that are
7592irrelevant to your parser's behavior, and it may eliminate warnings that are
7593relevant. Of course, the change in warnings may actually be relevant to a
7594parser table analysis that wants to keep unreachable states, so this
7595behavior will likely remain in future Bison releases.
7596
7597@item Other useless states.
7598
7599While Bison is able to remove unreachable states, it is not guaranteed to
7600remove other kinds of useless states. Specifically, when Bison disables
7601reduce actions during conflict resolution, some goto actions may become
7602useless, and thus some additional states may become useless. If Bison were
7603to compute which goto actions were useless and then disable those actions,
7604it could identify such states as unreachable and then remove those states.
7605However, Bison does not compute which goto actions are useless.
7606@end itemize
7607
7608@node Generalized LR Parsing
7609@section Generalized LR (GLR) Parsing
7610@cindex GLR parsing
7611@cindex generalized LR (GLR) parsing
7612@cindex ambiguous grammars
7613@cindex nondeterministic parsing
7614
7615Bison produces @emph{deterministic} parsers that choose uniquely
7616when to reduce and which reduction to apply
7617based on a summary of the preceding input and on one extra token of lookahead.
7618As a result, normal Bison handles a proper subset of the family of
7619context-free languages.
7620Ambiguous grammars, since they have strings with more than one possible
7621sequence of reductions cannot have deterministic parsers in this sense.
7622The same is true of languages that require more than one symbol of
7623lookahead, since the parser lacks the information necessary to make a
7624decision at the point it must be made in a shift-reduce parser.
7625Finally, as previously mentioned (@pxref{Mystery Conflicts}),
7626there are languages where Bison's default choice of how to
7627summarize the input seen so far loses necessary information.
7628
7629When you use the @samp{%glr-parser} declaration in your grammar file,
7630Bison generates a parser that uses a different algorithm, called
7631Generalized LR (or GLR). A Bison GLR
7632parser uses the same basic
7633algorithm for parsing as an ordinary Bison parser, but behaves
7634differently in cases where there is a shift-reduce conflict that has not
7635been resolved by precedence rules (@pxref{Precedence}) or a
7636reduce-reduce conflict. When a GLR parser encounters such a
7637situation, it
7638effectively @emph{splits} into a several parsers, one for each possible
7639shift or reduction. These parsers then proceed as usual, consuming
7640tokens in lock-step. Some of the stacks may encounter other conflicts
7641and split further, with the result that instead of a sequence of states,
7642a Bison GLR parsing stack is what is in effect a tree of states.
7643
7644In effect, each stack represents a guess as to what the proper parse
7645is. Additional input may indicate that a guess was wrong, in which case
7646the appropriate stack silently disappears. Otherwise, the semantics
7647actions generated in each stack are saved, rather than being executed
7648immediately. When a stack disappears, its saved semantic actions never
7649get executed. When a reduction causes two stacks to become equivalent,
7650their sets of semantic actions are both saved with the state that
7651results from the reduction. We say that two stacks are equivalent
7652when they both represent the same sequence of states,
7653and each pair of corresponding states represents a
7654grammar symbol that produces the same segment of the input token
7655stream.
7656
7657Whenever the parser makes a transition from having multiple
7658states to having one, it reverts to the normal deterministic parsing
7659algorithm, after resolving and executing the saved-up actions.
7660At this transition, some of the states on the stack will have semantic
7661values that are sets (actually multisets) of possible actions. The
7662parser tries to pick one of the actions by first finding one whose rule
7663has the highest dynamic precedence, as set by the @samp{%dprec}
7664declaration. Otherwise, if the alternative actions are not ordered by
7665precedence, but there the same merging function is declared for both
7666rules by the @samp{%merge} declaration,
7667Bison resolves and evaluates both and then calls the merge function on
7668the result. Otherwise, it reports an ambiguity.
7669
7670It is possible to use a data structure for the GLR parsing tree that
7671permits the processing of any LR(1) grammar in linear time (in the
7672size of the input), any unambiguous (not necessarily
7673LR(1)) grammar in
7674quadratic worst-case time, and any general (possibly ambiguous)
7675context-free grammar in cubic worst-case time. However, Bison currently
7676uses a simpler data structure that requires time proportional to the
7677length of the input times the maximum number of stacks required for any
7678prefix of the input. Thus, really ambiguous or nondeterministic
7679grammars can require exponential time and space to process. Such badly
7680behaving examples, however, are not generally of practical interest.
7681Usually, nondeterminism in a grammar is local---the parser is ``in
7682doubt'' only for a few tokens at a time. Therefore, the current data
7683structure should generally be adequate. On LR(1) portions of a
7684grammar, in particular, it is only slightly slower than with the
7685deterministic LR(1) Bison parser.
7686
7687For a more detailed exposition of GLR parsers, @pxref{Bibliography,,Scott
76882000}.
7689
7690@node Memory Management
7691@section Memory Management, and How to Avoid Memory Exhaustion
7692@cindex memory exhaustion
7693@cindex memory management
7694@cindex stack overflow
7695@cindex parser stack overflow
7696@cindex overflow of parser stack
7697
7698The Bison parser stack can run out of memory if too many tokens are shifted and
7699not reduced. When this happens, the parser function @code{yyparse}
7700calls @code{yyerror} and then returns 2.
7701
7702Because Bison parsers have growing stacks, hitting the upper limit
7703usually results from using a right recursion instead of a left
7704recursion, @xref{Recursion, ,Recursive Rules}.
7705
7706@vindex YYMAXDEPTH
7707By defining the macro @code{YYMAXDEPTH}, you can control how deep the
7708parser stack can become before memory is exhausted. Define the
7709macro with a value that is an integer. This value is the maximum number
7710of tokens that can be shifted (and not reduced) before overflow.
7711
7712The stack space allowed is not necessarily allocated. If you specify a
7713large value for @code{YYMAXDEPTH}, the parser normally allocates a small
7714stack at first, and then makes it bigger by stages as needed. This
7715increasing allocation happens automatically and silently. Therefore,
7716you do not need to make @code{YYMAXDEPTH} painfully small merely to save
7717space for ordinary inputs that do not need much stack.
7718
7719However, do not allow @code{YYMAXDEPTH} to be a value so large that
7720arithmetic overflow could occur when calculating the size of the stack
7721space. Also, do not allow @code{YYMAXDEPTH} to be less than
7722@code{YYINITDEPTH}.
7723
7724@cindex default stack limit
7725The default value of @code{YYMAXDEPTH}, if you do not define it, is
772610000.
7727
7728@vindex YYINITDEPTH
7729You can control how much stack is allocated initially by defining the
7730macro @code{YYINITDEPTH} to a positive integer. For the deterministic
7731parser in C, this value must be a compile-time constant
7732unless you are assuming C99 or some other target language or compiler
7733that allows variable-length arrays. The default is 200.
7734
7735Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
7736
7737You can generate a deterministic parser containing C++ user code from
7738the default (C) skeleton, as well as from the C++ skeleton
7739(@pxref{C++ Parsers}). However, if you do use the default skeleton
7740and want to allow the parsing stack to grow,
7741be careful not to use semantic types or location types that require
7742non-trivial copy constructors.
7743The C skeleton bypasses these constructors when copying data to
7744new, larger stacks.
7745
7746@node Error Recovery
7747@chapter Error Recovery
7748@cindex error recovery
7749@cindex recovery from errors
7750
7751It is not usually acceptable to have a program terminate on a syntax
7752error. For example, a compiler should recover sufficiently to parse the
7753rest of the input file and check it for errors; a calculator should accept
7754another expression.
7755
7756In a simple interactive command parser where each input is one line, it may
7757be sufficient to allow @code{yyparse} to return 1 on error and have the
7758caller ignore the rest of the input line when that happens (and then call
7759@code{yyparse} again). But this is inadequate for a compiler, because it
7760forgets all the syntactic context leading up to the error. A syntax error
7761deep within a function in the compiler input should not cause the compiler
7762to treat the following line like the beginning of a source file.
7763
7764@findex error
7765You can define how to recover from a syntax error by writing rules to
7766recognize the special token @code{error}. This is a terminal symbol that
7767is always defined (you need not declare it) and reserved for error
7768handling. The Bison parser generates an @code{error} token whenever a
7769syntax error happens; if you have provided a rule to recognize this token
7770in the current context, the parse can continue.
7771
7772For example:
7773
7774@example
7775stmnts: /* empty string */
7776 | stmnts '\n'
7777 | stmnts exp '\n'
7778 | stmnts error '\n'
7779@end example
7780
7781The fourth rule in this example says that an error followed by a newline
7782makes a valid addition to any @code{stmnts}.
7783
7784What happens if a syntax error occurs in the middle of an @code{exp}? The
7785error recovery rule, interpreted strictly, applies to the precise sequence
7786of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
7787the middle of an @code{exp}, there will probably be some additional tokens
7788and subexpressions on the stack after the last @code{stmnts}, and there
7789will be tokens to read before the next newline. So the rule is not
7790applicable in the ordinary way.
7791
7792But Bison can force the situation to fit the rule, by discarding part of
7793the semantic context and part of the input. First it discards states
7794and objects from the stack until it gets back to a state in which the
7795@code{error} token is acceptable. (This means that the subexpressions
7796already parsed are discarded, back to the last complete @code{stmnts}.)
7797At this point the @code{error} token can be shifted. Then, if the old
7798lookahead token is not acceptable to be shifted next, the parser reads
7799tokens and discards them until it finds a token which is acceptable. In
7800this example, Bison reads and discards input until the next newline so
7801that the fourth rule can apply. Note that discarded symbols are
7802possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
7803Discarded Symbols}, for a means to reclaim this memory.
7804
7805The choice of error rules in the grammar is a choice of strategies for
7806error recovery. A simple and useful strategy is simply to skip the rest of
7807the current input line or current statement if an error is detected:
7808
7809@example
7810stmnt: error ';' /* On error, skip until ';' is read. */
7811@end example
7812
7813It is also useful to recover to the matching close-delimiter of an
7814opening-delimiter that has already been parsed. Otherwise the
7815close-delimiter will probably appear to be unmatched, and generate another,
7816spurious error message:
7817
7818@example
7819primary: '(' expr ')'
7820 | '(' error ')'
7821 @dots{}
7822 ;
7823@end example
7824
7825Error recovery strategies are necessarily guesses. When they guess wrong,
7826one syntax error often leads to another. In the above example, the error
7827recovery rule guesses that an error is due to bad input within one
7828@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
7829middle of a valid @code{stmnt}. After the error recovery rule recovers
7830from the first error, another syntax error will be found straightaway,
7831since the text following the spurious semicolon is also an invalid
7832@code{stmnt}.
7833
7834To prevent an outpouring of error messages, the parser will output no error
7835message for another syntax error that happens shortly after the first; only
7836after three consecutive input tokens have been successfully shifted will
7837error messages resume.
7838
7839Note that rules which accept the @code{error} token may have actions, just
7840as any other rules can.
7841
7842@findex yyerrok
7843You can make error messages resume immediately by using the macro
7844@code{yyerrok} in an action. If you do this in the error rule's action, no
7845error messages will be suppressed. This macro requires no arguments;
7846@samp{yyerrok;} is a valid C statement.
7847
7848@findex yyclearin
7849The previous lookahead token is reanalyzed immediately after an error. If
7850this is unacceptable, then the macro @code{yyclearin} may be used to clear
7851this token. Write the statement @samp{yyclearin;} in the error rule's
7852action.
7853@xref{Action Features, ,Special Features for Use in Actions}.
7854
7855For example, suppose that on a syntax error, an error handling routine is
7856called that advances the input stream to some point where parsing should
7857once again commence. The next symbol returned by the lexical scanner is
7858probably correct. The previous lookahead token ought to be discarded
7859with @samp{yyclearin;}.
7860
7861@vindex YYRECOVERING
7862The expression @code{YYRECOVERING ()} yields 1 when the parser
7863is recovering from a syntax error, and 0 otherwise.
7864Syntax error diagnostics are suppressed while recovering from a syntax
7865error.
7866
7867@node Context Dependency
7868@chapter Handling Context Dependencies
7869
7870The Bison paradigm is to parse tokens first, then group them into larger
7871syntactic units. In many languages, the meaning of a token is affected by
7872its context. Although this violates the Bison paradigm, certain techniques
7873(known as @dfn{kludges}) may enable you to write Bison parsers for such
7874languages.
7875
7876@menu
7877* Semantic Tokens:: Token parsing can depend on the semantic context.
7878* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
7879* Tie-in Recovery:: Lexical tie-ins have implications for how
7880 error recovery rules must be written.
7881@end menu
7882
7883(Actually, ``kludge'' means any technique that gets its job done but is
7884neither clean nor robust.)
7885
7886@node Semantic Tokens
7887@section Semantic Info in Token Types
7888
7889The C language has a context dependency: the way an identifier is used
7890depends on what its current meaning is. For example, consider this:
7891
7892@example
7893foo (x);
7894@end example
7895
7896This looks like a function call statement, but if @code{foo} is a typedef
7897name, then this is actually a declaration of @code{x}. How can a Bison
7898parser for C decide how to parse this input?
7899
7900The method used in GNU C is to have two different token types,
7901@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
7902identifier, it looks up the current declaration of the identifier in order
7903to decide which token type to return: @code{TYPENAME} if the identifier is
7904declared as a typedef, @code{IDENTIFIER} otherwise.
7905
7906The grammar rules can then express the context dependency by the choice of
7907token type to recognize. @code{IDENTIFIER} is accepted as an expression,
7908but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
7909@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
7910is @emph{not} significant, such as in declarations that can shadow a
7911typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
7912accepted---there is one rule for each of the two token types.
7913
7914This technique is simple to use if the decision of which kinds of
7915identifiers to allow is made at a place close to where the identifier is
7916parsed. But in C this is not always so: C allows a declaration to
7917redeclare a typedef name provided an explicit type has been specified
7918earlier:
7919
7920@example
7921typedef int foo, bar;
7922int baz (void)
7923@{
7924 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
7925 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
7926 return foo (bar);
7927@}
7928@end example
7929
7930Unfortunately, the name being declared is separated from the declaration
7931construct itself by a complicated syntactic structure---the ``declarator''.
7932
7933As a result, part of the Bison parser for C needs to be duplicated, with
7934all the nonterminal names changed: once for parsing a declaration in
7935which a typedef name can be redefined, and once for parsing a
7936declaration in which that can't be done. Here is a part of the
7937duplication, with actions omitted for brevity:
7938
7939@example
7940initdcl:
7941 declarator maybeasm '='
7942 init
7943 | declarator maybeasm
7944 ;
7945
7946notype_initdcl:
7947 notype_declarator maybeasm '='
7948 init
7949 | notype_declarator maybeasm
7950 ;
7951@end example
7952
7953@noindent
7954Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
7955cannot. The distinction between @code{declarator} and
7956@code{notype_declarator} is the same sort of thing.
7957
7958There is some similarity between this technique and a lexical tie-in
7959(described next), in that information which alters the lexical analysis is
7960changed during parsing by other parts of the program. The difference is
7961here the information is global, and is used for other purposes in the
7962program. A true lexical tie-in has a special-purpose flag controlled by
7963the syntactic context.
7964
7965@node Lexical Tie-ins
7966@section Lexical Tie-ins
7967@cindex lexical tie-in
7968
7969One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
7970which is set by Bison actions, whose purpose is to alter the way tokens are
7971parsed.
7972
7973For example, suppose we have a language vaguely like C, but with a special
7974construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
7975an expression in parentheses in which all integers are hexadecimal. In
7976particular, the token @samp{a1b} must be treated as an integer rather than
7977as an identifier if it appears in that context. Here is how you can do it:
7978
7979@example
7980@group
7981%@{
7982 int hexflag;
7983 int yylex (void);
7984 void yyerror (char const *);
7985%@}
7986%%
7987@dots{}
7988@end group
7989@group
7990expr: IDENTIFIER
7991 | constant
7992 | HEX '('
7993 @{ hexflag = 1; @}
7994 expr ')'
7995 @{ hexflag = 0;
7996 $$ = $4; @}
7997 | expr '+' expr
7998 @{ $$ = make_sum ($1, $3); @}
7999 @dots{}
8000 ;
8001@end group
8002
8003@group
8004constant:
8005 INTEGER
8006 | STRING
8007 ;
8008@end group
8009@end example
8010
8011@noindent
8012Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
8013it is nonzero, all integers are parsed in hexadecimal, and tokens starting
8014with letters are parsed as integers if possible.
8015
8016The declaration of @code{hexflag} shown in the prologue of the grammar
8017file is needed to make it accessible to the actions (@pxref{Prologue,
8018,The Prologue}). You must also write the code in @code{yylex} to obey
8019the flag.
8020
8021@node Tie-in Recovery
8022@section Lexical Tie-ins and Error Recovery
8023
8024Lexical tie-ins make strict demands on any error recovery rules you have.
8025@xref{Error Recovery}.
8026
8027The reason for this is that the purpose of an error recovery rule is to
8028abort the parsing of one construct and resume in some larger construct.
8029For example, in C-like languages, a typical error recovery rule is to skip
8030tokens until the next semicolon, and then start a new statement, like this:
8031
8032@example
8033stmt: expr ';'
8034 | IF '(' expr ')' stmt @{ @dots{} @}
8035 @dots{}
8036 error ';'
8037 @{ hexflag = 0; @}
8038 ;
8039@end example
8040
8041If there is a syntax error in the middle of a @samp{hex (@var{expr})}
8042construct, this error rule will apply, and then the action for the
8043completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
8044remain set for the entire rest of the input, or until the next @code{hex}
8045keyword, causing identifiers to be misinterpreted as integers.
8046
8047To avoid this problem the error recovery rule itself clears @code{hexflag}.
8048
8049There may also be an error recovery rule that works within expressions.
8050For example, there could be a rule which applies within parentheses
8051and skips to the close-parenthesis:
8052
8053@example
8054@group
8055expr: @dots{}
8056 | '(' expr ')'
8057 @{ $$ = $2; @}
8058 | '(' error ')'
8059 @dots{}
8060@end group
8061@end example
8062
8063If this rule acts within the @code{hex} construct, it is not going to abort
8064that construct (since it applies to an inner level of parentheses within
8065the construct). Therefore, it should not clear the flag: the rest of
8066the @code{hex} construct should be parsed with the flag still in effect.
8067
8068What if there is an error recovery rule which might abort out of the
8069@code{hex} construct or might not, depending on circumstances? There is no
8070way you can write the action to determine whether a @code{hex} construct is
8071being aborted or not. So if you are using a lexical tie-in, you had better
8072make sure your error recovery rules are not of this kind. Each rule must
8073be such that you can be sure that it always will, or always won't, have to
8074clear the flag.
8075
8076@c ================================================== Debugging Your Parser
8077
8078@node Debugging
8079@chapter Debugging Your Parser
8080
8081Developing a parser can be a challenge, especially if you don't
8082understand the algorithm (@pxref{Algorithm, ,The Bison Parser
8083Algorithm}). Even so, sometimes a detailed description of the automaton
8084can help (@pxref{Understanding, , Understanding Your Parser}), or
8085tracing the execution of the parser can give some insight on why it
8086behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
8087
8088@menu
8089* Understanding:: Understanding the structure of your parser.
8090* Tracing:: Tracing the execution of your parser.
8091@end menu
8092
8093@node Understanding
8094@section Understanding Your Parser
8095
8096As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
8097Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
8098frequent than one would hope), looking at this automaton is required to
8099tune or simply fix a parser. Bison provides two different
8100representation of it, either textually or graphically (as a DOT file).
8101
8102The textual file is generated when the options @option{--report} or
8103@option{--verbose} are specified, see @xref{Invocation, , Invoking
8104Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
8105the parser implementation file name, and adding @samp{.output}
8106instead. Therefore, if the grammar file is @file{foo.y}, then the
8107parser implementation file is called @file{foo.tab.c} by default. As
8108a consequence, the verbose output file is called @file{foo.output}.
8109
8110The following grammar file, @file{calc.y}, will be used in the sequel:
8111
8112@example
8113%token NUM STR
8114%left '+' '-'
8115%left '*'
8116%%
8117exp: exp '+' exp
8118 | exp '-' exp
8119 | exp '*' exp
8120 | exp '/' exp
8121 | NUM
8122 ;
8123useless: STR;
8124%%
8125@end example
8126
8127@command{bison} reports:
8128
8129@example
8130calc.y: warning: 1 nonterminal useless in grammar
8131calc.y: warning: 1 rule useless in grammar
8132calc.y:11.1-7: warning: nonterminal useless in grammar: useless
8133calc.y:11.10-12: warning: rule useless in grammar: useless: STR
8134calc.y: conflicts: 7 shift/reduce
8135@end example
8136
8137When given @option{--report=state}, in addition to @file{calc.tab.c}, it
8138creates a file @file{calc.output} with contents detailed below. The
8139order of the output and the exact presentation might vary, but the
8140interpretation is the same.
8141
8142The first section includes details on conflicts that were solved thanks
8143to precedence and/or associativity:
8144
8145@example
8146Conflict in state 8 between rule 2 and token '+' resolved as reduce.
8147Conflict in state 8 between rule 2 and token '-' resolved as reduce.
8148Conflict in state 8 between rule 2 and token '*' resolved as shift.
8149@exdent @dots{}
8150@end example
8151
8152@noindent
8153The next section lists states that still have conflicts.
8154
8155@example
8156State 8 conflicts: 1 shift/reduce
8157State 9 conflicts: 1 shift/reduce
8158State 10 conflicts: 1 shift/reduce
8159State 11 conflicts: 4 shift/reduce
8160@end example
8161
8162@noindent
8163@cindex token, useless
8164@cindex useless token
8165@cindex nonterminal, useless
8166@cindex useless nonterminal
8167@cindex rule, useless
8168@cindex useless rule
8169The next section reports useless tokens, nonterminal and rules. Useless
8170nonterminals and rules are removed in order to produce a smaller parser,
8171but useless tokens are preserved, since they might be used by the
8172scanner (note the difference between ``useless'' and ``unused''
8173below):
8174
8175@example
8176Nonterminals useless in grammar:
8177 useless
8178
8179Terminals unused in grammar:
8180 STR
8181
8182Rules useless in grammar:
8183#6 useless: STR;
8184@end example
8185
8186@noindent
8187The next section reproduces the exact grammar that Bison used:
8188
8189@example
8190Grammar
8191
8192 Number, Line, Rule
8193 0 5 $accept -> exp $end
8194 1 5 exp -> exp '+' exp
8195 2 6 exp -> exp '-' exp
8196 3 7 exp -> exp '*' exp
8197 4 8 exp -> exp '/' exp
8198 5 9 exp -> NUM
8199@end example
8200
8201@noindent
8202and reports the uses of the symbols:
8203
8204@example
8205Terminals, with rules where they appear
8206
8207$end (0) 0
8208'*' (42) 3
8209'+' (43) 1
8210'-' (45) 2
8211'/' (47) 4
8212error (256)
8213NUM (258) 5
8214
8215Nonterminals, with rules where they appear
8216
8217$accept (8)
8218 on left: 0
8219exp (9)
8220 on left: 1 2 3 4 5, on right: 0 1 2 3 4
8221@end example
8222
8223@noindent
8224@cindex item
8225@cindex pointed rule
8226@cindex rule, pointed
8227Bison then proceeds onto the automaton itself, describing each state
8228with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
8229item is a production rule together with a point (marked by @samp{.})
8230that the input cursor.
8231
8232@example
8233state 0
8234
8235 $accept -> . exp $ (rule 0)
8236
8237 NUM shift, and go to state 1
8238
8239 exp go to state 2
8240@end example
8241
8242This reads as follows: ``state 0 corresponds to being at the very
8243beginning of the parsing, in the initial rule, right before the start
8244symbol (here, @code{exp}). When the parser returns to this state right
8245after having reduced a rule that produced an @code{exp}, the control
8246flow jumps to state 2. If there is no such transition on a nonterminal
8247symbol, and the lookahead is a @code{NUM}, then this token is shifted on
8248the parse stack, and the control flow jumps to state 1. Any other
8249lookahead triggers a syntax error.''
8250
8251@cindex core, item set
8252@cindex item set core
8253@cindex kernel, item set
8254@cindex item set core
8255Even though the only active rule in state 0 seems to be rule 0, the
8256report lists @code{NUM} as a lookahead token because @code{NUM} can be
8257at the beginning of any rule deriving an @code{exp}. By default Bison
8258reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
8259you want to see more detail you can invoke @command{bison} with
8260@option{--report=itemset} to list all the items, include those that can
8261be derived:
8262
8263@example
8264state 0
8265
8266 $accept -> . exp $ (rule 0)
8267 exp -> . exp '+' exp (rule 1)
8268 exp -> . exp '-' exp (rule 2)
8269 exp -> . exp '*' exp (rule 3)
8270 exp -> . exp '/' exp (rule 4)
8271 exp -> . NUM (rule 5)
8272
8273 NUM shift, and go to state 1
8274
8275 exp go to state 2
8276@end example
8277
8278@noindent
8279In the state 1...
8280
8281@example
8282state 1
8283
8284 exp -> NUM . (rule 5)
8285
8286 $default reduce using rule 5 (exp)
8287@end example
8288
8289@noindent
8290the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
8291(@samp{$default}), the parser will reduce it. If it was coming from
8292state 0, then, after this reduction it will return to state 0, and will
8293jump to state 2 (@samp{exp: go to state 2}).
8294
8295@example
8296state 2
8297
8298 $accept -> exp . $ (rule 0)
8299 exp -> exp . '+' exp (rule 1)
8300 exp -> exp . '-' exp (rule 2)
8301 exp -> exp . '*' exp (rule 3)
8302 exp -> exp . '/' exp (rule 4)
8303
8304 $ shift, and go to state 3
8305 '+' shift, and go to state 4
8306 '-' shift, and go to state 5
8307 '*' shift, and go to state 6
8308 '/' shift, and go to state 7
8309@end example
8310
8311@noindent
8312In state 2, the automaton can only shift a symbol. For instance,
8313because of the item @samp{exp -> exp . '+' exp}, if the lookahead if
8314@samp{+}, it will be shifted on the parse stack, and the automaton
8315control will jump to state 4, corresponding to the item @samp{exp -> exp
8316'+' . exp}. Since there is no default action, any other token than
8317those listed above will trigger a syntax error.
8318
8319@cindex accepting state
8320The state 3 is named the @dfn{final state}, or the @dfn{accepting
8321state}:
8322
8323@example
8324state 3
8325
8326 $accept -> exp $ . (rule 0)
8327
8328 $default accept
8329@end example
8330
8331@noindent
8332the initial rule is completed (the start symbol and the end
8333of input were read), the parsing exits successfully.
8334
8335The interpretation of states 4 to 7 is straightforward, and is left to
8336the reader.
8337
8338@example
8339state 4
8340
8341 exp -> exp '+' . exp (rule 1)
8342
8343 NUM shift, and go to state 1
8344
8345 exp go to state 8
8346
8347state 5
8348
8349 exp -> exp '-' . exp (rule 2)
8350
8351 NUM shift, and go to state 1
8352
8353 exp go to state 9
8354
8355state 6
8356
8357 exp -> exp '*' . exp (rule 3)
8358
8359 NUM shift, and go to state 1
8360
8361 exp go to state 10
8362
8363state 7
8364
8365 exp -> exp '/' . exp (rule 4)
8366
8367 NUM shift, and go to state 1
8368
8369 exp go to state 11
8370@end example
8371
8372As was announced in beginning of the report, @samp{State 8 conflicts:
83731 shift/reduce}:
8374
8375@example
8376state 8
8377
8378 exp -> exp . '+' exp (rule 1)
8379 exp -> exp '+' exp . (rule 1)
8380 exp -> exp . '-' exp (rule 2)
8381 exp -> exp . '*' exp (rule 3)
8382 exp -> exp . '/' exp (rule 4)
8383
8384 '*' shift, and go to state 6
8385 '/' shift, and go to state 7
8386
8387 '/' [reduce using rule 1 (exp)]
8388 $default reduce using rule 1 (exp)
8389@end example
8390
8391Indeed, there are two actions associated to the lookahead @samp{/}:
8392either shifting (and going to state 7), or reducing rule 1. The
8393conflict means that either the grammar is ambiguous, or the parser lacks
8394information to make the right decision. Indeed the grammar is
8395ambiguous, as, since we did not specify the precedence of @samp{/}, the
8396sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
8397NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
8398NUM}, which corresponds to reducing rule 1.
8399
8400Because in deterministic parsing a single decision can be made, Bison
8401arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
8402Shift/Reduce Conflicts}. Discarded actions are reported in between
8403square brackets.
8404
8405Note that all the previous states had a single possible action: either
8406shifting the next token and going to the corresponding state, or
8407reducing a single rule. In the other cases, i.e., when shifting
8408@emph{and} reducing is possible or when @emph{several} reductions are
8409possible, the lookahead is required to select the action. State 8 is
8410one such state: if the lookahead is @samp{*} or @samp{/} then the action
8411is shifting, otherwise the action is reducing rule 1. In other words,
8412the first two items, corresponding to rule 1, are not eligible when the
8413lookahead token is @samp{*}, since we specified that @samp{*} has higher
8414precedence than @samp{+}. More generally, some items are eligible only
8415with some set of possible lookahead tokens. When run with
8416@option{--report=lookahead}, Bison specifies these lookahead tokens:
8417
8418@example
8419state 8
8420
8421 exp -> exp . '+' exp (rule 1)
8422 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
8423 exp -> exp . '-' exp (rule 2)
8424 exp -> exp . '*' exp (rule 3)
8425 exp -> exp . '/' exp (rule 4)
8426
8427 '*' shift, and go to state 6
8428 '/' shift, and go to state 7
8429
8430 '/' [reduce using rule 1 (exp)]
8431 $default reduce using rule 1 (exp)
8432@end example
8433
8434The remaining states are similar:
8435
8436@example
8437state 9
8438
8439 exp -> exp . '+' exp (rule 1)
8440 exp -> exp . '-' exp (rule 2)
8441 exp -> exp '-' exp . (rule 2)
8442 exp -> exp . '*' exp (rule 3)
8443 exp -> exp . '/' exp (rule 4)
8444
8445 '*' shift, and go to state 6
8446 '/' shift, and go to state 7
8447
8448 '/' [reduce using rule 2 (exp)]
8449 $default reduce using rule 2 (exp)
8450
8451state 10
8452
8453 exp -> exp . '+' exp (rule 1)
8454 exp -> exp . '-' exp (rule 2)
8455 exp -> exp . '*' exp (rule 3)
8456 exp -> exp '*' exp . (rule 3)
8457 exp -> exp . '/' exp (rule 4)
8458
8459 '/' shift, and go to state 7
8460
8461 '/' [reduce using rule 3 (exp)]
8462 $default reduce using rule 3 (exp)
8463
8464state 11
8465
8466 exp -> exp . '+' exp (rule 1)
8467 exp -> exp . '-' exp (rule 2)
8468 exp -> exp . '*' exp (rule 3)
8469 exp -> exp . '/' exp (rule 4)
8470 exp -> exp '/' exp . (rule 4)
8471
8472 '+' shift, and go to state 4
8473 '-' shift, and go to state 5
8474 '*' shift, and go to state 6
8475 '/' shift, and go to state 7
8476
8477 '+' [reduce using rule 4 (exp)]
8478 '-' [reduce using rule 4 (exp)]
8479 '*' [reduce using rule 4 (exp)]
8480 '/' [reduce using rule 4 (exp)]
8481 $default reduce using rule 4 (exp)
8482@end example
8483
8484@noindent
8485Observe that state 11 contains conflicts not only due to the lack of
8486precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
8487@samp{*}, but also because the
8488associativity of @samp{/} is not specified.
8489
8490
8491@node Tracing
8492@section Tracing Your Parser
8493@findex yydebug
8494@cindex debugging
8495@cindex tracing the parser
8496
8497If a Bison grammar compiles properly but doesn't do what you want when it
8498runs, the @code{yydebug} parser-trace feature can help you figure out why.
8499
8500There are several means to enable compilation of trace facilities:
8501
8502@table @asis
8503@item the macro @code{YYDEBUG}
8504@findex YYDEBUG
8505Define the macro @code{YYDEBUG} to a nonzero value when you compile the
8506parser. This is compliant with POSIX Yacc. You could use
8507@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
8508YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
8509Prologue}).
8510
8511@item the option @option{-t}, @option{--debug}
8512Use the @samp{-t} option when you run Bison (@pxref{Invocation,
8513,Invoking Bison}). This is POSIX compliant too.
8514
8515@item the directive @samp{%debug}
8516@findex %debug
8517Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison Declaration
8518Summary}). This Bison extension is maintained for backward
8519compatibility with previous versions of Bison.
8520
8521@item the variable @samp{parse.trace}
8522@findex %define parse.trace
8523Add the @samp{%define parse.trace} directive (@pxref{%define
8524Summary,,parse.trace}), or pass the @option{-Dparse.trace} option
8525(@pxref{Bison Options}). This is a Bison extension, which is especially
8526useful for languages that don't use a preprocessor. Unless POSIX and Yacc
8527portability matter to you, this is the preferred solution.
8528@end table
8529
8530We suggest that you always enable the trace option so that debugging is
8531always possible.
8532
8533The trace facility outputs messages with macro calls of the form
8534@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
8535@var{format} and @var{args} are the usual @code{printf} format and variadic
8536arguments. If you define @code{YYDEBUG} to a nonzero value but do not
8537define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
8538and @code{YYFPRINTF} is defined to @code{fprintf}.
8539
8540Once you have compiled the program with trace facilities, the way to
8541request a trace is to store a nonzero value in the variable @code{yydebug}.
8542You can do this by making the C code do it (in @code{main}, perhaps), or
8543you can alter the value with a C debugger.
8544
8545Each step taken by the parser when @code{yydebug} is nonzero produces a
8546line or two of trace information, written on @code{stderr}. The trace
8547messages tell you these things:
8548
8549@itemize @bullet
8550@item
8551Each time the parser calls @code{yylex}, what kind of token was read.
8552
8553@item
8554Each time a token is shifted, the depth and complete contents of the
8555state stack (@pxref{Parser States}).
8556
8557@item
8558Each time a rule is reduced, which rule it is, and the complete contents
8559of the state stack afterward.
8560@end itemize
8561
8562To make sense of this information, it helps to refer to the listing file
8563produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
8564Bison}). This file shows the meaning of each state in terms of
8565positions in various rules, and also what each state will do with each
8566possible input token. As you read the successive trace messages, you
8567can see that the parser is functioning according to its specification in
8568the listing file. Eventually you will arrive at the place where
8569something undesirable happens, and you will see which parts of the
8570grammar are to blame.
8571
8572The parser implementation file is a C program and you can use C
8573debuggers on it, but it's not easy to interpret what it is doing. The
8574parser function is a finite-state machine interpreter, and aside from
8575the actions it executes the same code over and over. Only the values
8576of variables show where in the grammar it is working.
8577
8578@findex YYPRINT
8579The debugging information normally gives the token type of each token
8580read, but not its semantic value. You can optionally define a macro
8581named @code{YYPRINT} to provide a way to print the value. If you define
8582@code{YYPRINT}, it should take three arguments. The parser will pass a
8583standard I/O stream, the numeric code for the token type, and the token
8584value (from @code{yylval}).
8585
8586Here is an example of @code{YYPRINT} suitable for the multi-function
8587calculator (@pxref{Mfcalc Declarations, ,Declarations for @code{mfcalc}}):
8588
8589@smallexample
8590%@{
8591 static void print_token_value (FILE *, int, YYSTYPE);
8592 #define YYPRINT(file, type, value) print_token_value (file, type, value)
8593%@}
8594
8595@dots{} %% @dots{} %% @dots{}
8596
8597static void
8598print_token_value (FILE *file, int type, YYSTYPE value)
8599@{
8600 if (type == VAR)
8601 fprintf (file, "%s", value.tptr->name);
8602 else if (type == NUM)
8603 fprintf (file, "%d", value.val);
8604@}
8605@end smallexample
8606
8607@c ================================================= Invoking Bison
8608
8609@node Invocation
8610@chapter Invoking Bison
8611@cindex invoking Bison
8612@cindex Bison invocation
8613@cindex options for invoking Bison
8614
8615The usual way to invoke Bison is as follows:
8616
8617@example
8618bison @var{infile}
8619@end example
8620
8621Here @var{infile} is the grammar file name, which usually ends in
8622@samp{.y}. The parser implementation file's name is made by replacing
8623the @samp{.y} with @samp{.tab.c} and removing any leading directory.
8624Thus, the @samp{bison foo.y} file name yields @file{foo.tab.c}, and
8625the @samp{bison hack/foo.y} file name yields @file{foo.tab.c}. It's
8626also possible, in case you are writing C++ code instead of C in your
8627grammar file, to name it @file{foo.ypp} or @file{foo.y++}. Then, the
8628output files will take an extension like the given one as input
8629(respectively @file{foo.tab.cpp} and @file{foo.tab.c++}). This
8630feature takes effect with all options that manipulate file names like
8631@samp{-o} or @samp{-d}.
8632
8633For example :
8634
8635@example
8636bison -d @var{infile.yxx}
8637@end example
8638@noindent
8639will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
8640
8641@example
8642bison -d -o @var{output.c++} @var{infile.y}
8643@end example
8644@noindent
8645will produce @file{output.c++} and @file{outfile.h++}.
8646
8647For compatibility with POSIX, the standard Bison
8648distribution also contains a shell script called @command{yacc} that
8649invokes Bison with the @option{-y} option.
8650
8651@menu
8652* Bison Options:: All the options described in detail,
8653 in alphabetical order by short options.
8654* Option Cross Key:: Alphabetical list of long options.
8655* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
8656@end menu
8657
8658@node Bison Options
8659@section Bison Options
8660
8661Bison supports both traditional single-letter options and mnemonic long
8662option names. Long option names are indicated with @samp{--} instead of
8663@samp{-}. Abbreviations for option names are allowed as long as they
8664are unique. When a long option takes an argument, like
8665@samp{--file-prefix}, connect the option name and the argument with
8666@samp{=}.
8667
8668Here is a list of options that can be used with Bison, alphabetized by
8669short option. It is followed by a cross key alphabetized by long
8670option.
8671
8672@c Please, keep this ordered as in `bison --help'.
8673@noindent
8674Operations modes:
8675@table @option
8676@item -h
8677@itemx --help
8678Print a summary of the command-line options to Bison and exit.
8679
8680@item -V
8681@itemx --version
8682Print the version number of Bison and exit.
8683
8684@item --print-localedir
8685Print the name of the directory containing locale-dependent data.
8686
8687@item --print-datadir
8688Print the name of the directory containing skeletons and XSLT.
8689
8690@item -y
8691@itemx --yacc
8692Act more like the traditional Yacc command. This can cause different
8693diagnostics to be generated, and may change behavior in other minor
8694ways. Most importantly, imitate Yacc's output file name conventions,
8695so that the parser implementation file is called @file{y.tab.c}, and
8696the other outputs are called @file{y.output} and @file{y.tab.h}.
8697Also, if generating a deterministic parser in C, generate
8698@code{#define} statements in addition to an @code{enum} to associate
8699token numbers with token names. Thus, the following shell script can
8700substitute for Yacc, and the Bison distribution contains such a script
8701for compatibility with POSIX:
8702
8703@example
8704#! /bin/sh
8705bison -y "$@@"
8706@end example
8707
8708The @option{-y}/@option{--yacc} option is intended for use with
8709traditional Yacc grammars. If your grammar uses a Bison extension
8710like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
8711this option is specified.
8712
8713@item -W [@var{category}]
8714@itemx --warnings[=@var{category}]
8715Output warnings falling in @var{category}. @var{category} can be one
8716of:
8717@table @code
8718@item midrule-values
8719Warn about mid-rule values that are set but not used within any of the actions
8720of the parent rule.
8721For example, warn about unused @code{$2} in:
8722
8723@example
8724exp: '1' @{ $$ = 1; @} '+' exp @{ $$ = $1 + $4; @};
8725@end example
8726
8727Also warn about mid-rule values that are used but not set.
8728For example, warn about unset @code{$$} in the mid-rule action in:
8729
8730@example
8731 exp: '1' @{ $1 = 1; @} '+' exp @{ $$ = $2 + $4; @};
8732@end example
8733
8734These warnings are not enabled by default since they sometimes prove to
8735be false alarms in existing grammars employing the Yacc constructs
8736@code{$0} or @code{$-@var{n}} (where @var{n} is some positive integer).
8737
8738
8739@item yacc
8740Incompatibilities with POSIX Yacc.
8741
8742@item all
8743All the warnings.
8744@item none
8745Turn off all the warnings.
8746@item error
8747Treat warnings as errors.
8748@end table
8749
8750A category can be turned off by prefixing its name with @samp{no-}. For
8751instance, @option{-Wno-yacc} will hide the warnings about
8752POSIX Yacc incompatibilities.
8753@end table
8754
8755@noindent
8756Tuning the parser:
8757
8758@table @option
8759@item -t
8760@itemx --debug
8761In the parser implementation file, define the macro @code{YYDEBUG} to
87621 if it is not already defined, so that the debugging facilities are
8763compiled. @xref{Tracing, ,Tracing Your Parser}.
8764
8765@item -D @var{name}[=@var{value}]
8766@itemx --define=@var{name}[=@var{value}]
8767@itemx -F @var{name}[=@var{value}]
8768@itemx --force-define=@var{name}[=@var{value}]
8769Each of these is equivalent to @samp{%define @var{name} "@var{value}"}
8770(@pxref{%define Summary}) except that Bison processes multiple
8771definitions for the same @var{name} as follows:
8772
8773@itemize
8774@item
8775Bison quietly ignores all command-line definitions for @var{name} except
8776the last.
8777@item
8778If that command-line definition is specified by a @code{-D} or
8779@code{--define}, Bison reports an error for any @code{%define}
8780definition for @var{name}.
8781@item
8782If that command-line definition is specified by a @code{-F} or
8783@code{--force-define} instead, Bison quietly ignores all @code{%define}
8784definitions for @var{name}.
8785@item
8786Otherwise, Bison reports an error if there are multiple @code{%define}
8787definitions for @var{name}.
8788@end itemize
8789
8790You should avoid using @code{-F} and @code{--force-define} in your
8791make files unless you are confident that it is safe to quietly ignore
8792any conflicting @code{%define} that may be added to the grammar file.
8793
8794@item -L @var{language}
8795@itemx --language=@var{language}
8796Specify the programming language for the generated parser, as if
8797@code{%language} was specified (@pxref{Decl Summary, , Bison Declaration
8798Summary}). Currently supported languages include C, C++, and Java.
8799@var{language} is case-insensitive.
8800
8801This option is experimental and its effect may be modified in future
8802releases.
8803
8804@item --locations
8805Pretend that @code{%locations} was specified. @xref{Decl Summary}.
8806
8807@item -p @var{prefix}
8808@itemx --name-prefix=@var{prefix}
8809Pretend that @code{%name-prefix "@var{prefix}"} was specified.
8810@xref{Decl Summary}.
8811
8812@item -l
8813@itemx --no-lines
8814Don't put any @code{#line} preprocessor commands in the parser
8815implementation file. Ordinarily Bison puts them in the parser
8816implementation file so that the C compiler and debuggers will
8817associate errors with your source file, the grammar file. This option
8818causes them to associate errors with the parser implementation file,
8819treating it as an independent source file in its own right.
8820
8821@item -S @var{file}
8822@itemx --skeleton=@var{file}
8823Specify the skeleton to use, similar to @code{%skeleton}
8824(@pxref{Decl Summary, , Bison Declaration Summary}).
8825
8826@c You probably don't need this option unless you are developing Bison.
8827@c You should use @option{--language} if you want to specify the skeleton for a
8828@c different language, because it is clearer and because it will always
8829@c choose the correct skeleton for non-deterministic or push parsers.
8830
8831If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
8832file in the Bison installation directory.
8833If it does, @var{file} is an absolute file name or a file name relative to the
8834current working directory.
8835This is similar to how most shells resolve commands.
8836
8837@item -k
8838@itemx --token-table
8839Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
8840@end table
8841
8842@noindent
8843Adjust the output:
8844
8845@table @option
8846@item --defines[=@var{file}]
8847Pretend that @code{%defines} was specified, i.e., write an extra output
8848file containing macro definitions for the token type names defined in
8849the grammar, as well as a few other declarations. @xref{Decl Summary}.
8850
8851@item -d
8852This is the same as @code{--defines} except @code{-d} does not accept a
8853@var{file} argument since POSIX Yacc requires that @code{-d} can be bundled
8854with other short options.
8855
8856@item -b @var{file-prefix}
8857@itemx --file-prefix=@var{prefix}
8858Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
8859for all Bison output file names. @xref{Decl Summary}.
8860
8861@item -r @var{things}
8862@itemx --report=@var{things}
8863Write an extra output file containing verbose description of the comma
8864separated list of @var{things} among:
8865
8866@table @code
8867@item state
8868Description of the grammar, conflicts (resolved and unresolved), and
8869parser's automaton.
8870
8871@item lookahead
8872Implies @code{state} and augments the description of the automaton with
8873each rule's lookahead set.
8874
8875@item itemset
8876Implies @code{state} and augments the description of the automaton with
8877the full set of items for each state, instead of its core only.
8878@end table
8879
8880@item --report-file=@var{file}
8881Specify the @var{file} for the verbose description.
8882
8883@item -v
8884@itemx --verbose
8885Pretend that @code{%verbose} was specified, i.e., write an extra output
8886file containing verbose descriptions of the grammar and
8887parser. @xref{Decl Summary}.
8888
8889@item -o @var{file}
8890@itemx --output=@var{file}
8891Specify the @var{file} for the parser implementation file.
8892
8893The other output files' names are constructed from @var{file} as
8894described under the @samp{-v} and @samp{-d} options.
8895
8896@item -g [@var{file}]
8897@itemx --graph[=@var{file}]
8898Output a graphical representation of the parser's
8899automaton computed by Bison, in @uref{http://www.graphviz.org/, Graphviz}
8900@uref{http://www.graphviz.org/doc/info/lang.html, DOT} format.
8901@code{@var{file}} is optional.
8902If omitted and the grammar file is @file{foo.y}, the output file will be
8903@file{foo.dot}.
8904
8905@item -x [@var{file}]
8906@itemx --xml[=@var{file}]
8907Output an XML report of the parser's automaton computed by Bison.
8908@code{@var{file}} is optional.
8909If omitted and the grammar file is @file{foo.y}, the output file will be
8910@file{foo.xml}.
8911(The current XML schema is experimental and may evolve.
8912More user feedback will help to stabilize it.)
8913@end table
8914
8915@node Option Cross Key
8916@section Option Cross Key
8917
8918Here is a list of options, alphabetized by long option, to help you find
8919the corresponding short option and directive.
8920
8921@multitable {@option{--force-define=@var{name}[=@var{value}]}} {@option{-F @var{name}[=@var{value}]}} {@code{%nondeterministic-parser}}
8922@headitem Long Option @tab Short Option @tab Bison Directive
8923@include cross-options.texi
8924@end multitable
8925
8926@node Yacc Library
8927@section Yacc Library
8928
8929The Yacc library contains default implementations of the
8930@code{yyerror} and @code{main} functions. These default
8931implementations are normally not useful, but POSIX requires
8932them. To use the Yacc library, link your program with the
8933@option{-ly} option. Note that Bison's implementation of the Yacc
8934library is distributed under the terms of the GNU General
8935Public License (@pxref{Copying}).
8936
8937If you use the Yacc library's @code{yyerror} function, you should
8938declare @code{yyerror} as follows:
8939
8940@example
8941int yyerror (char const *);
8942@end example
8943
8944Bison ignores the @code{int} value returned by this @code{yyerror}.
8945If you use the Yacc library's @code{main} function, your
8946@code{yyparse} function should have the following type signature:
8947
8948@example
8949int yyparse (void);
8950@end example
8951
8952@c ================================================= C++ Bison
8953
8954@node Other Languages
8955@chapter Parsers Written In Other Languages
8956
8957@menu
8958* C++ Parsers:: The interface to generate C++ parser classes
8959* Java Parsers:: The interface to generate Java parser classes
8960@end menu
8961
8962@node C++ Parsers
8963@section C++ Parsers
8964
8965@menu
8966* C++ Bison Interface:: Asking for C++ parser generation
8967* C++ Semantic Values:: %union vs. C++
8968* C++ Location Values:: The position and location classes
8969* C++ Parser Interface:: Instantiating and running the parser
8970* C++ Scanner Interface:: Exchanges between yylex and parse
8971* A Complete C++ Example:: Demonstrating their use
8972@end menu
8973
8974@node C++ Bison Interface
8975@subsection C++ Bison Interface
8976@c - %skeleton "lalr1.cc"
8977@c - Always pure
8978@c - initial action
8979
8980The C++ deterministic parser is selected using the skeleton directive,
8981@samp{%skeleton "lalr1.cc"}, or the synonymous command-line option
8982@option{--skeleton=lalr1.cc}.
8983@xref{Decl Summary}.
8984
8985When run, @command{bison} will create several entities in the @samp{yy}
8986namespace.
8987@findex %define api.namespace
8988Use the @samp{%define api.namespace} directive to change the namespace name,
8989see @ref{%define Summary,,api.namespace}. The various classes are generated
8990in the following files:
8991
8992@table @file
8993@item position.hh
8994@itemx location.hh
8995The definition of the classes @code{position} and @code{location},
8996used for location tracking when enabled. @xref{C++ Location Values}.
8997
8998@item stack.hh
8999An auxiliary class @code{stack} used by the parser.
9000
9001@item @var{file}.hh
9002@itemx @var{file}.cc
9003(Assuming the extension of the grammar file was @samp{.yy}.) The
9004declaration and implementation of the C++ parser class. The basename
9005and extension of these two files follow the same rules as with regular C
9006parsers (@pxref{Invocation}).
9007
9008The header is @emph{mandatory}; you must either pass
9009@option{-d}/@option{--defines} to @command{bison}, or use the
9010@samp{%defines} directive.
9011@end table
9012
9013All these files are documented using Doxygen; run @command{doxygen}
9014for a complete and accurate documentation.
9015
9016@node C++ Semantic Values
9017@subsection C++ Semantic Values
9018@c - No objects in unions
9019@c - YYSTYPE
9020@c - Printer and destructor
9021
9022Bison supports two different means to handle semantic values in C++. One is
9023alike the C interface, and relies on unions (@pxref{C++ Unions}). As C++
9024practitioners know, unions are inconvenient in C++, therefore another
9025approach is provided, based on variants (@pxref{C++ Variants}).
9026
9027@menu
9028* C++ Unions:: Semantic values cannot be objects
9029* C++ Variants:: Using objects as semantic values
9030@end menu
9031
9032@node C++ Unions
9033@subsubsection C++ Unions
9034
9035The @code{%union} directive works as for C, see @ref{Union Decl, ,The
9036Collection of Value Types}. In particular it produces a genuine
9037@code{union}, which have a few specific features in C++.
9038@itemize @minus
9039@item
9040The type @code{YYSTYPE} is defined but its use is discouraged: rather
9041you should refer to the parser's encapsulated type
9042@code{yy::parser::semantic_type}.
9043@item
9044Non POD (Plain Old Data) types cannot be used. C++ forbids any
9045instance of classes with constructors in unions: only @emph{pointers}
9046to such objects are allowed.
9047@end itemize
9048
9049Because objects have to be stored via pointers, memory is not
9050reclaimed automatically: using the @code{%destructor} directive is the
9051only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
9052Symbols}.
9053
9054@node C++ Variants
9055@subsubsection C++ Variants
9056
9057Starting with version 2.6, Bison provides a @emph{variant} based
9058implementation of semantic values for C++. This alleviates all the
9059limitations reported in the previous section, and in particular, object
9060types can be used without pointers.
9061
9062To enable variant-based semantic values, set @code{%define} variable
9063@code{variant} (@pxref{%define Summary,, variant}). Once this defined,
9064@code{%union} is ignored, and instead of using the name of the fields of the
9065@code{%union} to ``type'' the symbols, use genuine types.
9066
9067For instance, instead of
9068
9069@example
9070%union
9071@{
9072 int ival;
9073 std::string* sval;
9074@}
9075%token <ival> NUMBER;
9076%token <sval> STRING;
9077@end example
9078
9079@noindent
9080write
9081
9082@example
9083%token <int> NUMBER;
9084%token <std::string> STRING;
9085@end example
9086
9087@code{STRING} is no longer a pointer, which should fairly simplify the user
9088actions in the grammar and in the scanner (in particular the memory
9089management).
9090
9091Since C++ features destructors, and since it is customary to specialize
9092@code{operator<<} to support uniform printing of values, variants also
9093typically simplify Bison printers and destructors.
9094
9095Variants are stricter than unions. When based on unions, you may play any
9096dirty game with @code{yylval}, say storing an @code{int}, reading a
9097@code{char*}, and then storing a @code{double} in it. This is no longer
9098possible with variants: they must be initialized, then assigned to, and
9099eventually, destroyed.
9100
9101@deftypemethod {semantic_type} {T&} build<T> ()
9102Initialize, but leave empty. Returns the address where the actual value may
9103be stored. Requires that the variant was not initialized yet.
9104@end deftypemethod
9105
9106@deftypemethod {semantic_type} {T&} build<T> (const T& @var{t})
9107Initialize, and copy-construct from @var{t}.
9108@end deftypemethod
9109
9110
9111@strong{Warning}: We do not use Boost.Variant, for two reasons. First, it
9112appeared unacceptable to require Boost on the user's machine (i.e., the
9113machine on which the generated parser will be compiled, not the machine on
9114which @command{bison} was run). Second, for each possible semantic value,
9115Boost.Variant not only stores the value, but also a tag specifying its
9116type. But the parser already ``knows'' the type of the semantic value, so
9117that would be duplicating the information.
9118
9119Therefore we developed light-weight variants whose type tag is external (so
9120they are really like @code{unions} for C++ actually). But our code is much
9121less mature that Boost.Variant. So there is a number of limitations in
9122(the current implementation of) variants:
9123@itemize
9124@item
9125Alignment must be enforced: values should be aligned in memory according to
9126the most demanding type. Computing the smallest alignment possible requires
9127meta-programming techniques that are not currently implemented in Bison, and
9128therefore, since, as far as we know, @code{double} is the most demanding
9129type on all platforms, alignments are enforced for @code{double} whatever
9130types are actually used. This may waste space in some cases.
9131
9132@item
9133Our implementation is not conforming with strict aliasing rules. Alias
9134analysis is a technique used in optimizing compilers to detect when two
9135pointers are disjoint (they cannot ``meet''). Our implementation breaks
9136some of the rules that G++ 4.4 uses in its alias analysis, so @emph{strict
9137alias analysis must be disabled}. Use the option
9138@option{-fno-strict-aliasing} to compile the generated parser.
9139
9140@item
9141There might be portability issues we are not aware of.
9142@end itemize
9143
9144As far as we know, these limitations @emph{can} be alleviated. All it takes
9145is some time and/or some talented C++ hacker willing to contribute to Bison.
9146
9147@node C++ Location Values
9148@subsection C++ Location Values
9149@c - %locations
9150@c - class Position
9151@c - class Location
9152@c - %define filename_type "const symbol::Symbol"
9153
9154When the directive @code{%locations} is used, the C++ parser supports
9155location tracking, see @ref{Locations, , Locations Overview}. Two
9156auxiliary classes define a @code{position}, a single point in a file,
9157and a @code{location}, a range composed of a pair of
9158@code{position}s (possibly spanning several files).
9159
9160@deftypemethod {position} {std::string*} file
9161The name of the file. It will always be handled as a pointer, the
9162parser will never duplicate nor deallocate it. As an experimental
9163feature you may change it to @samp{@var{type}*} using @samp{%define
9164filename_type "@var{type}"}.
9165@end deftypemethod
9166
9167@deftypemethod {position} {unsigned int} line
9168The line, starting at 1.
9169@end deftypemethod
9170
9171@deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
9172Advance by @var{height} lines, resetting the column number.
9173@end deftypemethod
9174
9175@deftypemethod {position} {unsigned int} column
9176The column, starting at 0.
9177@end deftypemethod
9178
9179@deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
9180Advance by @var{width} columns, without changing the line number.
9181@end deftypemethod
9182
9183@deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
9184@deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
9185@deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
9186@deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
9187Various forms of syntactic sugar for @code{columns}.
9188@end deftypemethod
9189
9190@deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
9191Report @var{p} on @var{o} like this:
9192@samp{@var{file}:@var{line}.@var{column}}, or
9193@samp{@var{line}.@var{column}} if @var{file} is null.
9194@end deftypemethod
9195
9196@deftypemethod {location} {position} begin
9197@deftypemethodx {location} {position} end
9198The first, inclusive, position of the range, and the first beyond.
9199@end deftypemethod
9200
9201@deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
9202@deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
9203Advance the @code{end} position.
9204@end deftypemethod
9205
9206@deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
9207@deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
9208@deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
9209Various forms of syntactic sugar.
9210@end deftypemethod
9211
9212@deftypemethod {location} {void} step ()
9213Move @code{begin} onto @code{end}.
9214@end deftypemethod
9215
9216
9217@node C++ Parser Interface
9218@subsection C++ Parser Interface
9219@c - define parser_class_name
9220@c - Ctor
9221@c - parse, error, set_debug_level, debug_level, set_debug_stream,
9222@c debug_stream.
9223@c - Reporting errors
9224
9225The output files @file{@var{output}.hh} and @file{@var{output}.cc}
9226declare and define the parser class in the namespace @code{yy}. The
9227class name defaults to @code{parser}, but may be changed using
9228@samp{%define parser_class_name "@var{name}"}. The interface of
9229this class is detailed below. It can be extended using the
9230@code{%parse-param} feature: its semantics is slightly changed since
9231it describes an additional member of the parser class, and an
9232additional argument for its constructor.
9233
9234@defcv {Type} {parser} {semantic_type}
9235@defcvx {Type} {parser} {location_type}
9236The types for semantic values and locations (if enabled).
9237@end defcv
9238
9239@defcv {Type} {parser} {token}
9240A structure that contains (only) the definition of the tokens as the
9241@code{yytokentype} enumeration. To refer to the token @code{FOO}, the
9242scanner should use @code{yy::parser::token::FOO}. The scanner can use
9243@samp{typedef yy::parser::token token;} to ``import'' the token enumeration
9244(@pxref{Calc++ Scanner}).
9245@end defcv
9246
9247@defcv {Type} {parser} {syntax_error}
9248This class derives from @code{std::runtime_error}. Throw instances of it
9249from user actions to raise parse errors. This is equivalent with first
9250invoking @code{error} to report the location and message of the syntax
9251error, and then to invoke @code{YYERROR} to enter the error-recovery mode.
9252But contrary to @code{YYERROR} which can only be invoked from user actions
9253(i.e., written in the action itself), the exception can be thrown from
9254function invoked from the user action.
9255@end defcv
9256
9257@deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
9258Build a new parser object. There are no arguments by default, unless
9259@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
9260@end deftypemethod
9261
9262@deftypemethod {syntax_error} {} syntax_error (const location_type& @var{l}, const std::string& @var{m})
9263@deftypemethodx {syntax_error} {} syntax_error (const std::string& @var{m})
9264Instantiate a syntax-error exception.
9265@end deftypemethod
9266
9267@deftypemethod {parser} {int} parse ()
9268Run the syntactic analysis, and return 0 on success, 1 otherwise.
9269@end deftypemethod
9270
9271@deftypemethod {parser} {std::ostream&} debug_stream ()
9272@deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
9273Get or set the stream used for tracing the parsing. It defaults to
9274@code{std::cerr}.
9275@end deftypemethod
9276
9277@deftypemethod {parser} {debug_level_type} debug_level ()
9278@deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
9279Get or set the tracing level. Currently its value is either 0, no trace,
9280or nonzero, full tracing.
9281@end deftypemethod
9282
9283@deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
9284@deftypemethodx {parser} {void} error (const std::string& @var{m})
9285The definition for this member function must be supplied by the user:
9286the parser uses it to report a parser error occurring at @var{l},
9287described by @var{m}. If location tracking is not enabled, the second
9288signature is used.
9289@end deftypemethod
9290
9291
9292@node C++ Scanner Interface
9293@subsection C++ Scanner Interface
9294@c - prefix for yylex.
9295@c - Pure interface to yylex
9296@c - %lex-param
9297
9298The parser invokes the scanner by calling @code{yylex}. Contrary to C
9299parsers, C++ parsers are always pure: there is no point in using the
9300@samp{%define api.pure} directive. The actual interface with @code{yylex}
9301depends whether you use unions, or variants.
9302
9303@menu
9304* Split Symbols:: Passing symbols as two/three components
9305* Complete Symbols:: Making symbols a whole
9306@end menu
9307
9308@node Split Symbols
9309@subsubsection Split Symbols
9310
9311Therefore the interface is as follows.
9312
9313@deftypemethod {parser} {int} yylex (semantic_type* @var{yylval}, location_type* @var{yylloc}, @var{type1} @var{arg1}, ...)
9314@deftypemethodx {parser} {int} yylex (semantic_type* @var{yylval}, @var{type1} @var{arg1}, ...)
9315Return the next token. Its type is the return value, its semantic value and
9316location (if enabled) being @var{yylval} and @var{yylloc}. Invocations of
9317@samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
9318@end deftypemethod
9319
9320Note that when using variants, the interface for @code{yylex} is the same,
9321but @code{yylval} is handled differently.
9322
9323Regular union-based code in Lex scanner typically look like:
9324
9325@example
9326[0-9]+ @{
9327 yylval.ival = text_to_int (yytext);
9328 return yy::parser::INTEGER;
9329 @}
9330[a-z]+ @{
9331 yylval.sval = new std::string (yytext);
9332 return yy::parser::IDENTIFIER;
9333 @}
9334@end example
9335
9336Using variants, @code{yylval} is already constructed, but it is not
9337initialized. So the code would look like:
9338
9339@example
9340[0-9]+ @{
9341 yylval.build<int>() = text_to_int (yytext);
9342 return yy::parser::INTEGER;
9343 @}
9344[a-z]+ @{
9345 yylval.build<std::string> = yytext;
9346 return yy::parser::IDENTIFIER;
9347 @}
9348@end example
9349
9350@noindent
9351or
9352
9353@example
9354[0-9]+ @{
9355 yylval.build(text_to_int (yytext));
9356 return yy::parser::INTEGER;
9357 @}
9358[a-z]+ @{
9359 yylval.build(yytext);
9360 return yy::parser::IDENTIFIER;
9361 @}
9362@end example
9363
9364
9365@node Complete Symbols
9366@subsubsection Complete Symbols
9367
9368If you specified both @code{%define variant} and @code{%define lex_symbol},
9369the @code{parser} class also defines the class @code{parser::symbol_type}
9370which defines a @emph{complete} symbol, aggregating its type (i.e., the
9371traditional value returned by @code{yylex}), its semantic value (i.e., the
9372value passed in @code{yylval}, and possibly its location (@code{yylloc}).
9373
9374@deftypemethod {symbol_type} {} symbol_type (token_type @var{type}, const semantic_type& @var{value}, const location_type& @var{location})
9375Build a complete terminal symbol which token type is @var{type}, and which
9376semantic value is @var{value}. If location tracking is enabled, also pass
9377the @var{location}.
9378@end deftypemethod
9379
9380This interface is low-level and should not be used for two reasons. First,
9381it is inconvenient, as you still have to build the semantic value, which is
9382a variant, and second, because consistency is not enforced: as with unions,
9383it is still possible to give an integer as semantic value for a string.
9384
9385So for each token type, Bison generates named constructors as follows.
9386
9387@deftypemethod {symbol_type} {} make_@var{token} (const @var{value_type}& @var{value}, const location_type& @var{location})
9388@deftypemethodx {symbol_type} {} make_@var{token} (const location_type& @var{location})
9389Build a complete terminal symbol for the token type @var{token} (not
9390including the @code{api.tokens.prefix}) whose possible semantic value is
9391@var{value} of adequate @var{value_type}. If location tracking is enabled,
9392also pass the @var{location}.
9393@end deftypemethod
9394
9395For instance, given the following declarations:
9396
9397@example
9398%define api.tokens.prefix "TOK_"
9399%token <std::string> IDENTIFIER;
9400%token <int> INTEGER;
9401%token COLON;
9402@end example
9403
9404@noindent
9405Bison generates the following functions:
9406
9407@example
9408symbol_type make_IDENTIFIER(const std::string& v,
9409 const location_type& l);
9410symbol_type make_INTEGER(const int& v,
9411 const location_type& loc);
9412symbol_type make_COLON(const location_type& loc);
9413@end example
9414
9415@noindent
9416which should be used in a Lex-scanner as follows.
9417
9418@example
9419[0-9]+ return yy::parser::make_INTEGER(text_to_int (yytext), loc);
9420[a-z]+ return yy::parser::make_IDENTIFIER(yytext, loc);
9421":" return yy::parser::make_COLON(loc);
9422@end example
9423
9424Tokens that do not have an identifier are not accessible: you cannot simply
9425use characters such as @code{':'}, they must be declared with @code{%token}.
9426
9427@node A Complete C++ Example
9428@subsection A Complete C++ Example
9429
9430This section demonstrates the use of a C++ parser with a simple but
9431complete example. This example should be available on your system,
9432ready to compile, in the directory @dfn{.../bison/examples/calc++}. It
9433focuses on the use of Bison, therefore the design of the various C++
9434classes is very naive: no accessors, no encapsulation of members etc.
9435We will use a Lex scanner, and more precisely, a Flex scanner, to
9436demonstrate the various interactions. A hand-written scanner is
9437actually easier to interface with.
9438
9439@menu
9440* Calc++ --- C++ Calculator:: The specifications
9441* Calc++ Parsing Driver:: An active parsing context
9442* Calc++ Parser:: A parser class
9443* Calc++ Scanner:: A pure C++ Flex scanner
9444* Calc++ Top Level:: Conducting the band
9445@end menu
9446
9447@node Calc++ --- C++ Calculator
9448@subsubsection Calc++ --- C++ Calculator
9449
9450Of course the grammar is dedicated to arithmetics, a single
9451expression, possibly preceded by variable assignments. An
9452environment containing possibly predefined variables such as
9453@code{one} and @code{two}, is exchanged with the parser. An example
9454of valid input follows.
9455
9456@example
9457three := 3
9458seven := one + two * three
9459seven * seven
9460@end example
9461
9462@node Calc++ Parsing Driver
9463@subsubsection Calc++ Parsing Driver
9464@c - An env
9465@c - A place to store error messages
9466@c - A place for the result
9467
9468To support a pure interface with the parser (and the scanner) the
9469technique of the ``parsing context'' is convenient: a structure
9470containing all the data to exchange. Since, in addition to simply
9471launch the parsing, there are several auxiliary tasks to execute (open
9472the file for parsing, instantiate the parser etc.), we recommend
9473transforming the simple parsing context structure into a fully blown
9474@dfn{parsing driver} class.
9475
9476The declaration of this driver class, @file{calc++-driver.hh}, is as
9477follows. The first part includes the CPP guard and imports the
9478required standard library components, and the declaration of the parser
9479class.
9480
9481@comment file: calc++-driver.hh
9482@example
9483#ifndef CALCXX_DRIVER_HH
9484# define CALCXX_DRIVER_HH
9485# include <string>
9486# include <map>
9487# include "calc++-parser.hh"
9488@end example
9489
9490
9491@noindent
9492Then comes the declaration of the scanning function. Flex expects
9493the signature of @code{yylex} to be defined in the macro
9494@code{YY_DECL}, and the C++ parser expects it to be declared. We can
9495factor both as follows.
9496
9497@comment file: calc++-driver.hh
9498@example
9499// Tell Flex the lexer's prototype ...
9500# define YY_DECL \
9501 yy::calcxx_parser::symbol_type yylex (calcxx_driver& driver)
9502// ... and declare it for the parser's sake.
9503YY_DECL;
9504@end example
9505
9506@noindent
9507The @code{calcxx_driver} class is then declared with its most obvious
9508members.
9509
9510@comment file: calc++-driver.hh
9511@example
9512// Conducting the whole scanning and parsing of Calc++.
9513class calcxx_driver
9514@{
9515public:
9516 calcxx_driver ();
9517 virtual ~calcxx_driver ();
9518
9519 std::map<std::string, int> variables;
9520
9521 int result;
9522@end example
9523
9524@noindent
9525To encapsulate the coordination with the Flex scanner, it is useful to have
9526member functions to open and close the scanning phase.
9527
9528@comment file: calc++-driver.hh
9529@example
9530 // Handling the scanner.
9531 void scan_begin ();
9532 void scan_end ();
9533 bool trace_scanning;
9534@end example
9535
9536@noindent
9537Similarly for the parser itself.
9538
9539@comment file: calc++-driver.hh
9540@example
9541 // Run the parser on file F.
9542 // Return 0 on success.
9543 int parse (const std::string& f);
9544 // The name of the file being parsed.
9545 // Used later to pass the file name to the location tracker.
9546 std::string file;
9547 // Whether parser traces should be generated.
9548 bool trace_parsing;
9549@end example
9550
9551@noindent
9552To demonstrate pure handling of parse errors, instead of simply
9553dumping them on the standard error output, we will pass them to the
9554compiler driver using the following two member functions. Finally, we
9555close the class declaration and CPP guard.
9556
9557@comment file: calc++-driver.hh
9558@example
9559 // Error handling.
9560 void error (const yy::location& l, const std::string& m);
9561 void error (const std::string& m);
9562@};
9563#endif // ! CALCXX_DRIVER_HH
9564@end example
9565
9566The implementation of the driver is straightforward. The @code{parse}
9567member function deserves some attention. The @code{error} functions
9568are simple stubs, they should actually register the located error
9569messages and set error state.
9570
9571@comment file: calc++-driver.cc
9572@example
9573#include "calc++-driver.hh"
9574#include "calc++-parser.hh"
9575
9576calcxx_driver::calcxx_driver ()
9577 : trace_scanning (false), trace_parsing (false)
9578@{
9579 variables["one"] = 1;
9580 variables["two"] = 2;
9581@}
9582
9583calcxx_driver::~calcxx_driver ()
9584@{
9585@}
9586
9587int
9588calcxx_driver::parse (const std::string &f)
9589@{
9590 file = f;
9591 scan_begin ();
9592 yy::calcxx_parser parser (*this);
9593 parser.set_debug_level (trace_parsing);
9594 int res = parser.parse ();
9595 scan_end ();
9596 return res;
9597@}
9598
9599void
9600calcxx_driver::error (const yy::location& l, const std::string& m)
9601@{
9602 std::cerr << l << ": " << m << std::endl;
9603@}
9604
9605void
9606calcxx_driver::error (const std::string& m)
9607@{
9608 std::cerr << m << std::endl;
9609@}
9610@end example
9611
9612@node Calc++ Parser
9613@subsubsection Calc++ Parser
9614
9615The grammar file @file{calc++-parser.yy} starts by asking for the C++
9616deterministic parser skeleton, the creation of the parser header file,
9617and specifies the name of the parser class. Because the C++ skeleton
9618changed several times, it is safer to require the version you designed
9619the grammar for.
9620
9621@comment file: calc++-parser.yy
9622@example
9623%skeleton "lalr1.cc" /* -*- C++ -*- */
9624%require "@value{VERSION}"
9625%defines
9626%define parser_class_name "calcxx_parser"
9627@end example
9628
9629@noindent
9630@findex %define variant
9631@findex %define lex_symbol
9632This example will use genuine C++ objects as semantic values, therefore, we
9633require the variant-based interface. To make sure we properly use it, we
9634enable assertions. To fully benefit from type-safety and more natural
9635definition of ``symbol'', we enable @code{lex_symbol}.
9636
9637@comment file: calc++-parser.yy
9638@example
9639%define variant
9640%define parse.assert
9641%define lex_symbol
9642@end example
9643
9644@noindent
9645@findex %code requires
9646Then come the declarations/inclusions needed by the semantic values.
9647Because the parser uses the parsing driver and reciprocally, both would like
9648to include the header of the other, which is, of course, insane. This
9649mutual dependency will be broken using forward declarations. Because the
9650driver's header needs detailed knowledge about the parser class (in
9651particular its inner types), it is the parser's header which will use a
9652forward declaration of the driver. @xref{%code Summary}.
9653
9654@comment file: calc++-parser.yy
9655@example
9656%code requires
9657@{
9658# include <string>
9659class calcxx_driver;
9660@}
9661@end example
9662
9663@noindent
9664The driver is passed by reference to the parser and to the scanner.
9665This provides a simple but effective pure interface, not relying on
9666global variables.
9667
9668@comment file: calc++-parser.yy
9669@example
9670// The parsing context.
9671%param @{ calcxx_driver& driver @}
9672@end example
9673
9674@noindent
9675Then we request location tracking, and initialize the
9676first location's file name. Afterward new locations are computed
9677relatively to the previous locations: the file name will be
9678propagated.
9679
9680@comment file: calc++-parser.yy
9681@example
9682%locations
9683%initial-action
9684@{
9685 // Initialize the initial location.
9686 @@$.begin.filename = @@$.end.filename = &driver.file;
9687@};
9688@end example
9689
9690@noindent
9691Use the following two directives to enable parser tracing and verbose error
9692messages. However, verbose error messages can contain incorrect information
9693(@pxref{LAC}).
9694
9695@comment file: calc++-parser.yy
9696@example
9697%define parse.trace
9698%define parse.error verbose
9699@end example
9700
9701@noindent
9702@findex %code
9703The code between @samp{%code @{} and @samp{@}} is output in the
9704@file{*.cc} file; it needs detailed knowledge about the driver.
9705
9706@comment file: calc++-parser.yy
9707@example
9708%code
9709@{
9710# include "calc++-driver.hh"
9711@}
9712@end example
9713
9714
9715@noindent
9716The token numbered as 0 corresponds to end of file; the following line
9717allows for nicer error messages referring to ``end of file'' instead of
9718``$end''. Similarly user friendly names are provided for each symbol. To
9719avoid name clashes in the generated files (@pxref{Calc++ Scanner}), prefix
9720tokens with @code{TOK_} (@pxref{%define Summary,,api.tokens.prefix}).
9721
9722@comment file: calc++-parser.yy
9723@example
9724%define api.tokens.prefix "TOK_"
9725%token
9726 END 0 "end of file"
9727 ASSIGN ":="
9728 MINUS "-"
9729 PLUS "+"
9730 STAR "*"
9731 SLASH "/"
9732 LPAREN "("
9733 RPAREN ")"
9734;
9735@end example
9736
9737@noindent
9738Since we use variant-based semantic values, @code{%union} is not used, and
9739both @code{%type} and @code{%token} expect genuine types, as opposed to type
9740tags.
9741
9742@comment file: calc++-parser.yy
9743@example
9744%token <std::string> IDENTIFIER "identifier"
9745%token <int> NUMBER "number"
9746%type <int> exp
9747@end example
9748
9749@noindent
9750No @code{%destructor} is needed to enable memory deallocation during error
9751recovery; the memory, for strings for instance, will be reclaimed by the
9752regular destructors. All the values are printed using their
9753@code{operator<<}.
9754
9755@c FIXME: Document %printer, and mention that it takes a braced-code operand.
9756@comment file: calc++-parser.yy
9757@example
9758%printer @{ debug_stream () << $$; @} <*>;
9759@end example
9760
9761@noindent
9762The grammar itself is straightforward (@pxref{Location Tracking Calc, ,
9763Location Tracking Calculator: @code{ltcalc}}).
9764
9765@comment file: calc++-parser.yy
9766@example
9767%%
9768%start unit;
9769unit: assignments exp @{ driver.result = $2; @};
9770
9771assignments:
9772 assignments assignment @{@}
9773| /* Nothing. */ @{@};
9774
9775assignment:
9776 "identifier" ":=" exp @{ driver.variables[$1] = $3; @};
9777
9778%left "+" "-";
9779%left "*" "/";
9780exp:
9781 exp "+" exp @{ $$ = $1 + $3; @}
9782| exp "-" exp @{ $$ = $1 - $3; @}
9783| exp "*" exp @{ $$ = $1 * $3; @}
9784| exp "/" exp @{ $$ = $1 / $3; @}
9785| "(" exp ")" @{ std::swap ($$, $2); @}
9786| "identifier" @{ $$ = driver.variables[$1]; @}
9787| "number" @{ std::swap ($$, $1); @};
9788%%
9789@end example
9790
9791@noindent
9792Finally the @code{error} member function registers the errors to the
9793driver.
9794
9795@comment file: calc++-parser.yy
9796@example
9797void
9798yy::calcxx_parser::error (const location_type& l,
9799 const std::string& m)
9800@{
9801 driver.error (l, m);
9802@}
9803@end example
9804
9805@node Calc++ Scanner
9806@subsubsection Calc++ Scanner
9807
9808The Flex scanner first includes the driver declaration, then the
9809parser's to get the set of defined tokens.
9810
9811@comment file: calc++-scanner.ll
9812@example
9813%@{ /* -*- C++ -*- */
9814# include <cerrno>
9815# include <climits>
9816# include <cstdlib>
9817# include <string>
9818# include "calc++-driver.hh"
9819# include "calc++-parser.hh"
9820
9821// Work around an incompatibility in flex (at least versions
9822// 2.5.31 through 2.5.33): it generates code that does
9823// not conform to C89. See Debian bug 333231
9824// <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>.
9825# undef yywrap
9826# define yywrap() 1
9827
9828// The location of the current token.
9829static yy::location loc;
9830%@}
9831@end example
9832
9833@noindent
9834Because there is no @code{#include}-like feature we don't need
9835@code{yywrap}, we don't need @code{unput} either, and we parse an
9836actual file, this is not an interactive session with the user.
9837Finally, we enable scanner tracing.
9838
9839@comment file: calc++-scanner.ll
9840@example
9841%option noyywrap nounput batch debug
9842@end example
9843
9844@noindent
9845Abbreviations allow for more readable rules.
9846
9847@comment file: calc++-scanner.ll
9848@example
9849id [a-zA-Z][a-zA-Z_0-9]*
9850int [0-9]+
9851blank [ \t]
9852@end example
9853
9854@noindent
9855The following paragraph suffices to track locations accurately. Each
9856time @code{yylex} is invoked, the begin position is moved onto the end
9857position. Then when a pattern is matched, its width is added to the end
9858column. When matching ends of lines, the end
9859cursor is adjusted, and each time blanks are matched, the begin cursor
9860is moved onto the end cursor to effectively ignore the blanks
9861preceding tokens. Comments would be treated equally.
9862
9863@comment file: calc++-scanner.ll
9864@example
9865%@{
9866 // Code run each time a pattern is matched.
9867 # define YY_USER_ACTION loc.columns (yyleng);
9868%@}
9869%%
9870%@{
9871 // Code run each time yylex is called.
9872 loc.step ();
9873%@}
9874@{blank@}+ loc.step ();
9875[\n]+ loc.lines (yyleng); loc.step ();
9876@end example
9877
9878@noindent
9879The rules are simple. The driver is used to report errors.
9880
9881@comment file: calc++-scanner.ll
9882@example
9883"-" return yy::calcxx_parser::make_MINUS(loc);
9884"+" return yy::calcxx_parser::make_PLUS(loc);
9885"*" return yy::calcxx_parser::make_STAR(loc);
9886"/" return yy::calcxx_parser::make_SLASH(loc);
9887"(" return yy::calcxx_parser::make_LPAREN(loc);
9888")" return yy::calcxx_parser::make_RPAREN(loc);
9889":=" return yy::calcxx_parser::make_ASSIGN(loc);
9890
9891@{int@} @{
9892 errno = 0;
9893 long n = strtol (yytext, NULL, 10);
9894 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
9895 driver.error (loc, "integer is out of range");
9896 return yy::calcxx_parser::make_NUMBER(n, loc);
9897@}
9898@{id@} return yy::calcxx_parser::make_IDENTIFIER(yytext, loc);
9899. driver.error (loc, "invalid character");
9900<<EOF>> return yy::calcxx_parser::make_END(loc);
9901%%
9902@end example
9903
9904@noindent
9905Finally, because the scanner-related driver's member-functions depend
9906on the scanner's data, it is simpler to implement them in this file.
9907
9908@comment file: calc++-scanner.ll
9909@example
9910void
9911calcxx_driver::scan_begin ()
9912@{
9913 yy_flex_debug = trace_scanning;
9914 if (file == "-")
9915 yyin = stdin;
9916 else if (!(yyin = fopen (file.c_str (), "r")))
9917 @{
9918 error (std::string ("cannot open ") + file + ": " + strerror(errno));
9919 exit (1);
9920 @}
9921@}
9922
9923void
9924calcxx_driver::scan_end ()
9925@{
9926 fclose (yyin);
9927@}
9928@end example
9929
9930@node Calc++ Top Level
9931@subsubsection Calc++ Top Level
9932
9933The top level file, @file{calc++.cc}, poses no problem.
9934
9935@comment file: calc++.cc
9936@example
9937#include <iostream>
9938#include "calc++-driver.hh"
9939
9940int
9941main (int argc, char *argv[])
9942@{
9943 int res = 0;
9944 calcxx_driver driver;
9945 for (++argv; argv[0]; ++argv)
9946 if (*argv == std::string ("-p"))
9947 driver.trace_parsing = true;
9948 else if (*argv == std::string ("-s"))
9949 driver.trace_scanning = true;
9950 else if (!driver.parse (*argv))
9951 std::cout << driver.result << std::endl;
9952 else
9953 res = 1;
9954 return res;
9955@}
9956@end example
9957
9958@node Java Parsers
9959@section Java Parsers
9960
9961@menu
9962* Java Bison Interface:: Asking for Java parser generation
9963* Java Semantic Values:: %type and %token vs. Java
9964* Java Location Values:: The position and location classes
9965* Java Parser Interface:: Instantiating and running the parser
9966* Java Scanner Interface:: Specifying the scanner for the parser
9967* Java Action Features:: Special features for use in actions
9968* Java Differences:: Differences between C/C++ and Java Grammars
9969* Java Declarations Summary:: List of Bison declarations used with Java
9970@end menu
9971
9972@node Java Bison Interface
9973@subsection Java Bison Interface
9974@c - %language "Java"
9975
9976(The current Java interface is experimental and may evolve.
9977More user feedback will help to stabilize it.)
9978
9979The Java parser skeletons are selected using the @code{%language "Java"}
9980directive or the @option{-L java}/@option{--language=java} option.
9981
9982@c FIXME: Documented bug.
9983When generating a Java parser, @code{bison @var{basename}.y} will
9984create a single Java source file named @file{@var{basename}.java}
9985containing the parser implementation. Using a grammar file without a
9986@file{.y} suffix is currently broken. The basename of the parser
9987implementation file can be changed by the @code{%file-prefix}
9988directive or the @option{-p}/@option{--name-prefix} option. The
9989entire parser implementation file name can be changed by the
9990@code{%output} directive or the @option{-o}/@option{--output} option.
9991The parser implementation file contains a single class for the parser.
9992
9993You can create documentation for generated parsers using Javadoc.
9994
9995Contrary to C parsers, Java parsers do not use global variables; the
9996state of the parser is always local to an instance of the parser class.
9997Therefore, all Java parsers are ``pure'', and the @code{%pure-parser}
9998and @samp{%define api.pure} directives does not do anything when used in
9999Java.
10000
10001Push parsers are currently unsupported in Java and @code{%define
10002api.push-pull} have no effect.
10003
10004GLR parsers are currently unsupported in Java. Do not use the
10005@code{glr-parser} directive.
10006
10007No header file can be generated for Java parsers. Do not use the
10008@code{%defines} directive or the @option{-d}/@option{--defines} options.
10009
10010@c FIXME: Possible code change.
10011Currently, support for tracing is always compiled
10012in. Thus the @samp{%define parse.trace} and @samp{%token-table}
10013directives and the
10014@option{-t}/@option{--debug} and @option{-k}/@option{--token-table}
10015options have no effect. This may change in the future to eliminate
10016unused code in the generated parser, so use @samp{%define parse.trace}
10017explicitly
10018if needed. Also, in the future the
10019@code{%token-table} directive might enable a public interface to
10020access the token names and codes.
10021
10022Getting a ``code too large'' error from the Java compiler means the code
10023hit the 64KB bytecode per method limitation of the Java class file.
10024Try reducing the amount of code in actions and static initializers;
10025otherwise, report a bug so that the parser skeleton will be improved.
10026
10027
10028@node Java Semantic Values
10029@subsection Java Semantic Values
10030@c - No %union, specify type in %type/%token.
10031@c - YYSTYPE
10032@c - Printer and destructor
10033
10034There is no @code{%union} directive in Java parsers. Instead, the
10035semantic values' types (class names) should be specified in the
10036@code{%type} or @code{%token} directive:
10037
10038@example
10039%type <Expression> expr assignment_expr term factor
10040%type <Integer> number
10041@end example
10042
10043By default, the semantic stack is declared to have @code{Object} members,
10044which means that the class types you specify can be of any class.
10045To improve the type safety of the parser, you can declare the common
10046superclass of all the semantic values using the @samp{%define stype}
10047directive. For example, after the following declaration:
10048
10049@example
10050%define stype "ASTNode"
10051@end example
10052
10053@noindent
10054any @code{%type} or @code{%token} specifying a semantic type which
10055is not a subclass of ASTNode, will cause a compile-time error.
10056
10057@c FIXME: Documented bug.
10058Types used in the directives may be qualified with a package name.
10059Primitive data types are accepted for Java version 1.5 or later. Note
10060that in this case the autoboxing feature of Java 1.5 will be used.
10061Generic types may not be used; this is due to a limitation in the
10062implementation of Bison, and may change in future releases.
10063
10064Java parsers do not support @code{%destructor}, since the language
10065adopts garbage collection. The parser will try to hold references
10066to semantic values for as little time as needed.
10067
10068Java parsers do not support @code{%printer}, as @code{toString()}
10069can be used to print the semantic values. This however may change
10070(in a backwards-compatible way) in future versions of Bison.
10071
10072
10073@node Java Location Values
10074@subsection Java Location Values
10075@c - %locations
10076@c - class Position
10077@c - class Location
10078
10079When the directive @code{%locations} is used, the Java parser
10080supports location tracking, see @ref{Locations, , Locations Overview}.
10081An auxiliary user-defined class defines a @dfn{position}, a single point
10082in a file; Bison itself defines a class representing a @dfn{location},
10083a range composed of a pair of positions (possibly spanning several
10084files). The location class is an inner class of the parser; the name
10085is @code{Location} by default, and may also be renamed using
10086@samp{%define location_type "@var{class-name}"}.
10087
10088The location class treats the position as a completely opaque value.
10089By default, the class name is @code{Position}, but this can be changed
10090with @samp{%define position_type "@var{class-name}"}. This class must
10091be supplied by the user.
10092
10093
10094@deftypeivar {Location} {Position} begin
10095@deftypeivarx {Location} {Position} end
10096The first, inclusive, position of the range, and the first beyond.
10097@end deftypeivar
10098
10099@deftypeop {Constructor} {Location} {} Location (Position @var{loc})
10100Create a @code{Location} denoting an empty range located at a given point.
10101@end deftypeop
10102
10103@deftypeop {Constructor} {Location} {} Location (Position @var{begin}, Position @var{end})
10104Create a @code{Location} from the endpoints of the range.
10105@end deftypeop
10106
10107@deftypemethod {Location} {String} toString ()
10108Prints the range represented by the location. For this to work
10109properly, the position class should override the @code{equals} and
10110@code{toString} methods appropriately.
10111@end deftypemethod
10112
10113
10114@node Java Parser Interface
10115@subsection Java Parser Interface
10116@c - define parser_class_name
10117@c - Ctor
10118@c - parse, error, set_debug_level, debug_level, set_debug_stream,
10119@c debug_stream.
10120@c - Reporting errors
10121
10122The name of the generated parser class defaults to @code{YYParser}. The
10123@code{YY} prefix may be changed using the @code{%name-prefix} directive
10124or the @option{-p}/@option{--name-prefix} option. Alternatively, use
10125@samp{%define parser_class_name "@var{name}"} to give a custom name to
10126the class. The interface of this class is detailed below.
10127
10128By default, the parser class has package visibility. A declaration
10129@samp{%define public} will change to public visibility. Remember that,
10130according to the Java language specification, the name of the @file{.java}
10131file should match the name of the class in this case. Similarly, you can
10132use @code{abstract}, @code{final} and @code{strictfp} with the
10133@code{%define} declaration to add other modifiers to the parser class.
10134A single @samp{%define annotations "@var{annotations}"} directive can
10135be used to add any number of annotations to the parser class.
10136
10137The Java package name of the parser class can be specified using the
10138@samp{%define package} directive. The superclass and the implemented
10139interfaces of the parser class can be specified with the @code{%define
10140extends} and @samp{%define implements} directives.
10141
10142The parser class defines an inner class, @code{Location}, that is used
10143for location tracking (see @ref{Java Location Values}), and a inner
10144interface, @code{Lexer} (see @ref{Java Scanner Interface}). Other than
10145these inner class/interface, and the members described in the interface
10146below, all the other members and fields are preceded with a @code{yy} or
10147@code{YY} prefix to avoid clashes with user code.
10148
10149The parser class can be extended using the @code{%parse-param}
10150directive. Each occurrence of the directive will add a @code{protected
10151final} field to the parser class, and an argument to its constructor,
10152which initialize them automatically.
10153
10154@deftypeop {Constructor} {YYParser} {} YYParser (@var{lex_param}, @dots{}, @var{parse_param}, @dots{})
10155Build a new parser object with embedded @code{%code lexer}. There are
10156no parameters, unless @code{%param}s and/or @code{%parse-param}s and/or
10157@code{%lex-param}s are used.
10158
10159Use @code{%code init} for code added to the start of the constructor
10160body. This is especially useful to initialize superclasses. Use
10161@samp{%define init_throws} to specify any uncaught exceptions.
10162@end deftypeop
10163
10164@deftypeop {Constructor} {YYParser} {} YYParser (Lexer @var{lexer}, @var{parse_param}, @dots{})
10165Build a new parser object using the specified scanner. There are no
10166additional parameters unless @code{%param}s and/or @code{%parse-param}s are
10167used.
10168
10169If the scanner is defined by @code{%code lexer}, this constructor is
10170declared @code{protected} and is called automatically with a scanner
10171created with the correct @code{%param}s and/or @code{%lex-param}s.
10172
10173Use @code{%code init} for code added to the start of the constructor
10174body. This is especially useful to initialize superclasses. Use
10175@samp{%define init_throws} to specify any uncatch exceptions.
10176@end deftypeop
10177
10178@deftypemethod {YYParser} {boolean} parse ()
10179Run the syntactic analysis, and return @code{true} on success,
10180@code{false} otherwise.
10181@end deftypemethod
10182
10183@deftypemethod {YYParser} {boolean} getErrorVerbose ()
10184@deftypemethodx {YYParser} {void} setErrorVerbose (boolean @var{verbose})
10185Get or set the option to produce verbose error messages. These are only
10186available with @samp{%define parse.error verbose}, which also turns on
10187verbose error messages.
10188@end deftypemethod
10189
10190@deftypemethod {YYParser} {void} yyerror (String @var{msg})
10191@deftypemethodx {YYParser} {void} yyerror (Position @var{pos}, String @var{msg})
10192@deftypemethodx {YYParser} {void} yyerror (Location @var{loc}, String @var{msg})
10193Print an error message using the @code{yyerror} method of the scanner
10194instance in use. The @code{Location} and @code{Position} parameters are
10195available only if location tracking is active.
10196@end deftypemethod
10197
10198@deftypemethod {YYParser} {boolean} recovering ()
10199During the syntactic analysis, return @code{true} if recovering
10200from a syntax error.
10201@xref{Error Recovery}.
10202@end deftypemethod
10203
10204@deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
10205@deftypemethodx {YYParser} {void} setDebugStream (java.io.printStream @var{o})
10206Get or set the stream used for tracing the parsing. It defaults to
10207@code{System.err}.
10208@end deftypemethod
10209
10210@deftypemethod {YYParser} {int} getDebugLevel ()
10211@deftypemethodx {YYParser} {void} setDebugLevel (int @var{l})
10212Get or set the tracing level. Currently its value is either 0, no trace,
10213or nonzero, full tracing.
10214@end deftypemethod
10215
10216@deftypecv {Constant} {YYParser} {String} {bisonVersion}
10217@deftypecvx {Constant} {YYParser} {String} {bisonSkeleton}
10218Identify the Bison version and skeleton used to generate this parser.
10219@end deftypecv
10220
10221
10222@node Java Scanner Interface
10223@subsection Java Scanner Interface
10224@c - %code lexer
10225@c - %lex-param
10226@c - Lexer interface
10227
10228There are two possible ways to interface a Bison-generated Java parser
10229with a scanner: the scanner may be defined by @code{%code lexer}, or
10230defined elsewhere. In either case, the scanner has to implement the
10231@code{Lexer} inner interface of the parser class. This interface also
10232contain constants for all user-defined token names and the predefined
10233@code{EOF} token.
10234
10235In the first case, the body of the scanner class is placed in
10236@code{%code lexer} blocks. If you want to pass parameters from the
10237parser constructor to the scanner constructor, specify them with
10238@code{%lex-param}; they are passed before @code{%parse-param}s to the
10239constructor.
10240
10241In the second case, the scanner has to implement the @code{Lexer} interface,
10242which is defined within the parser class (e.g., @code{YYParser.Lexer}).
10243The constructor of the parser object will then accept an object
10244implementing the interface; @code{%lex-param} is not used in this
10245case.
10246
10247In both cases, the scanner has to implement the following methods.
10248
10249@deftypemethod {Lexer} {void} yyerror (Location @var{loc}, String @var{msg})
10250This method is defined by the user to emit an error message. The first
10251parameter is omitted if location tracking is not active. Its type can be
10252changed using @samp{%define location_type "@var{class-name}".}
10253@end deftypemethod
10254
10255@deftypemethod {Lexer} {int} yylex ()
10256Return the next token. Its type is the return value, its semantic
10257value and location are saved and returned by the their methods in the
10258interface.
10259
10260Use @samp{%define lex_throws} to specify any uncaught exceptions.
10261Default is @code{java.io.IOException}.
10262@end deftypemethod
10263
10264@deftypemethod {Lexer} {Position} getStartPos ()
10265@deftypemethodx {Lexer} {Position} getEndPos ()
10266Return respectively the first position of the last token that
10267@code{yylex} returned, and the first position beyond it. These
10268methods are not needed unless location tracking is active.
10269
10270The return type can be changed using @samp{%define position_type
10271"@var{class-name}".}
10272@end deftypemethod
10273
10274@deftypemethod {Lexer} {Object} getLVal ()
10275Return the semantic value of the last token that yylex returned.
10276
10277The return type can be changed using @samp{%define stype
10278"@var{class-name}".}
10279@end deftypemethod
10280
10281
10282@node Java Action Features
10283@subsection Special Features for Use in Java Actions
10284
10285The following special constructs can be uses in Java actions.
10286Other analogous C action features are currently unavailable for Java.
10287
10288Use @samp{%define throws} to specify any uncaught exceptions from parser
10289actions, and initial actions specified by @code{%initial-action}.
10290
10291@defvar $@var{n}
10292The semantic value for the @var{n}th component of the current rule.
10293This may not be assigned to.
10294@xref{Java Semantic Values}.
10295@end defvar
10296
10297@defvar $<@var{typealt}>@var{n}
10298Like @code{$@var{n}} but specifies a alternative type @var{typealt}.
10299@xref{Java Semantic Values}.
10300@end defvar
10301
10302@defvar $$
10303The semantic value for the grouping made by the current rule. As a
10304value, this is in the base type (@code{Object} or as specified by
10305@samp{%define stype}) as in not cast to the declared subtype because
10306casts are not allowed on the left-hand side of Java assignments.
10307Use an explicit Java cast if the correct subtype is needed.
10308@xref{Java Semantic Values}.
10309@end defvar
10310
10311@defvar $<@var{typealt}>$
10312Same as @code{$$} since Java always allow assigning to the base type.
10313Perhaps we should use this and @code{$<>$} for the value and @code{$$}
10314for setting the value but there is currently no easy way to distinguish
10315these constructs.
10316@xref{Java Semantic Values}.
10317@end defvar
10318
10319@defvar @@@var{n}
10320The location information of the @var{n}th component of the current rule.
10321This may not be assigned to.
10322@xref{Java Location Values}.
10323@end defvar
10324
10325@defvar @@$
10326The location information of the grouping made by the current rule.
10327@xref{Java Location Values}.
10328@end defvar
10329
10330@deffn {Statement} {return YYABORT;}
10331Return immediately from the parser, indicating failure.
10332@xref{Java Parser Interface}.
10333@end deffn
10334
10335@deffn {Statement} {return YYACCEPT;}
10336Return immediately from the parser, indicating success.
10337@xref{Java Parser Interface}.
10338@end deffn
10339
10340@deffn {Statement} {return YYERROR;}
10341Start error recovery without printing an error message.
10342@xref{Error Recovery}.
10343@end deffn
10344
10345@deftypefn {Function} {boolean} recovering ()
10346Return whether error recovery is being done. In this state, the parser
10347reads token until it reaches a known state, and then restarts normal
10348operation.
10349@xref{Error Recovery}.
10350@end deftypefn
10351
10352@deftypefn {Function} {void} yyerror (String @var{msg})
10353@deftypefnx {Function} {void} yyerror (Position @var{loc}, String @var{msg})
10354@deftypefnx {Function} {void} yyerror (Location @var{loc}, String @var{msg})
10355Print an error message using the @code{yyerror} method of the scanner
10356instance in use. The @code{Location} and @code{Position} parameters are
10357available only if location tracking is active.
10358@end deftypefn
10359
10360
10361@node Java Differences
10362@subsection Differences between C/C++ and Java Grammars
10363
10364The different structure of the Java language forces several differences
10365between C/C++ grammars, and grammars designed for Java parsers. This
10366section summarizes these differences.
10367
10368@itemize
10369@item
10370Java lacks a preprocessor, so the @code{YYERROR}, @code{YYACCEPT},
10371@code{YYABORT} symbols (@pxref{Table of Symbols}) cannot obviously be
10372macros. Instead, they should be preceded by @code{return} when they
10373appear in an action. The actual definition of these symbols is
10374opaque to the Bison grammar, and it might change in the future. The
10375only meaningful operation that you can do, is to return them.
10376See @pxref{Java Action Features}.
10377
10378Note that of these three symbols, only @code{YYACCEPT} and
10379@code{YYABORT} will cause a return from the @code{yyparse}
10380method@footnote{Java parsers include the actions in a separate
10381method than @code{yyparse} in order to have an intuitive syntax that
10382corresponds to these C macros.}.
10383
10384@item
10385Java lacks unions, so @code{%union} has no effect. Instead, semantic
10386values have a common base type: @code{Object} or as specified by
10387@samp{%define stype}. Angle brackets on @code{%token}, @code{type},
10388@code{$@var{n}} and @code{$$} specify subtypes rather than fields of
10389an union. The type of @code{$$}, even with angle brackets, is the base
10390type since Java casts are not allow on the left-hand side of assignments.
10391Also, @code{$@var{n}} and @code{@@@var{n}} are not allowed on the
10392left-hand side of assignments. See @pxref{Java Semantic Values} and
10393@pxref{Java Action Features}.
10394
10395@item
10396The prologue declarations have a different meaning than in C/C++ code.
10397@table @asis
10398@item @code{%code imports}
10399blocks are placed at the beginning of the Java source code. They may
10400include copyright notices. For a @code{package} declarations, it is
10401suggested to use @samp{%define package} instead.
10402
10403@item unqualified @code{%code}
10404blocks are placed inside the parser class.
10405
10406@item @code{%code lexer}
10407blocks, if specified, should include the implementation of the
10408scanner. If there is no such block, the scanner can be any class
10409that implements the appropriate interface (see @pxref{Java Scanner
10410Interface}).
10411@end table
10412
10413Other @code{%code} blocks are not supported in Java parsers.
10414In particular, @code{%@{ @dots{} %@}} blocks should not be used
10415and may give an error in future versions of Bison.
10416
10417The epilogue has the same meaning as in C/C++ code and it can
10418be used to define other classes used by the parser @emph{outside}
10419the parser class.
10420@end itemize
10421
10422
10423@node Java Declarations Summary
10424@subsection Java Declarations Summary
10425
10426This summary only include declarations specific to Java or have special
10427meaning when used in a Java parser.
10428
10429@deffn {Directive} {%language "Java"}
10430Generate a Java class for the parser.
10431@end deffn
10432
10433@deffn {Directive} %lex-param @{@var{type} @var{name}@}
10434A parameter for the lexer class defined by @code{%code lexer}
10435@emph{only}, added as parameters to the lexer constructor and the parser
10436constructor that @emph{creates} a lexer. Default is none.
10437@xref{Java Scanner Interface}.
10438@end deffn
10439
10440@deffn {Directive} %name-prefix "@var{prefix}"
10441The prefix of the parser class name @code{@var{prefix}Parser} if
10442@samp{%define parser_class_name} is not used. Default is @code{YY}.
10443@xref{Java Bison Interface}.
10444@end deffn
10445
10446@deffn {Directive} %parse-param @{@var{type} @var{name}@}
10447A parameter for the parser class added as parameters to constructor(s)
10448and as fields initialized by the constructor(s). Default is none.
10449@xref{Java Parser Interface}.
10450@end deffn
10451
10452@deffn {Directive} %token <@var{type}> @var{token} @dots{}
10453Declare tokens. Note that the angle brackets enclose a Java @emph{type}.
10454@xref{Java Semantic Values}.
10455@end deffn
10456
10457@deffn {Directive} %type <@var{type}> @var{nonterminal} @dots{}
10458Declare the type of nonterminals. Note that the angle brackets enclose
10459a Java @emph{type}.
10460@xref{Java Semantic Values}.
10461@end deffn
10462
10463@deffn {Directive} %code @{ @var{code} @dots{} @}
10464Code appended to the inside of the parser class.
10465@xref{Java Differences}.
10466@end deffn
10467
10468@deffn {Directive} {%code imports} @{ @var{code} @dots{} @}
10469Code inserted just after the @code{package} declaration.
10470@xref{Java Differences}.
10471@end deffn
10472
10473@deffn {Directive} {%code init} @{ @var{code} @dots{} @}
10474Code inserted at the beginning of the parser constructor body.
10475@xref{Java Parser Interface}.
10476@end deffn
10477
10478@deffn {Directive} {%code lexer} @{ @var{code} @dots{} @}
10479Code added to the body of a inner lexer class within the parser class.
10480@xref{Java Scanner Interface}.
10481@end deffn
10482
10483@deffn {Directive} %% @var{code} @dots{}
10484Code (after the second @code{%%}) appended to the end of the file,
10485@emph{outside} the parser class.
10486@xref{Java Differences}.
10487@end deffn
10488
10489@deffn {Directive} %@{ @var{code} @dots{} %@}
10490Not supported. Use @code{%code imports} instead.
10491@xref{Java Differences}.
10492@end deffn
10493
10494@deffn {Directive} {%define abstract}
10495Whether the parser class is declared @code{abstract}. Default is false.
10496@xref{Java Bison Interface}.
10497@end deffn
10498
10499@deffn {Directive} {%define annotations} "@var{annotations}"
10500The Java annotations for the parser class. Default is none.
10501@xref{Java Bison Interface}.
10502@end deffn
10503
10504@deffn {Directive} {%define extends} "@var{superclass}"
10505The superclass of the parser class. Default is none.
10506@xref{Java Bison Interface}.
10507@end deffn
10508
10509@deffn {Directive} {%define final}
10510Whether the parser class is declared @code{final}. Default is false.
10511@xref{Java Bison Interface}.
10512@end deffn
10513
10514@deffn {Directive} {%define implements} "@var{interfaces}"
10515The implemented interfaces of the parser class, a comma-separated list.
10516Default is none.
10517@xref{Java Bison Interface}.
10518@end deffn
10519
10520@deffn {Directive} {%define init_throws} "@var{exceptions}"
10521The exceptions thrown by @code{%code init} from the parser class
10522constructor. Default is none.
10523@xref{Java Parser Interface}.
10524@end deffn
10525
10526@deffn {Directive} {%define lex_throws} "@var{exceptions}"
10527The exceptions thrown by the @code{yylex} method of the lexer, a
10528comma-separated list. Default is @code{java.io.IOException}.
10529@xref{Java Scanner Interface}.
10530@end deffn
10531
10532@deffn {Directive} {%define location_type} "@var{class}"
10533The name of the class used for locations (a range between two
10534positions). This class is generated as an inner class of the parser
10535class by @command{bison}. Default is @code{Location}.
10536@xref{Java Location Values}.
10537@end deffn
10538
10539@deffn {Directive} {%define package} "@var{package}"
10540The package to put the parser class in. Default is none.
10541@xref{Java Bison Interface}.
10542@end deffn
10543
10544@deffn {Directive} {%define parser_class_name} "@var{name}"
10545The name of the parser class. Default is @code{YYParser} or
10546@code{@var{name-prefix}Parser}.
10547@xref{Java Bison Interface}.
10548@end deffn
10549
10550@deffn {Directive} {%define position_type} "@var{class}"
10551The name of the class used for positions. This class must be supplied by
10552the user. Default is @code{Position}.
10553@xref{Java Location Values}.
10554@end deffn
10555
10556@deffn {Directive} {%define public}
10557Whether the parser class is declared @code{public}. Default is false.
10558@xref{Java Bison Interface}.
10559@end deffn
10560
10561@deffn {Directive} {%define stype} "@var{class}"
10562The base type of semantic values. Default is @code{Object}.
10563@xref{Java Semantic Values}.
10564@end deffn
10565
10566@deffn {Directive} {%define strictfp}
10567Whether the parser class is declared @code{strictfp}. Default is false.
10568@xref{Java Bison Interface}.
10569@end deffn
10570
10571@deffn {Directive} {%define throws} "@var{exceptions}"
10572The exceptions thrown by user-supplied parser actions and
10573@code{%initial-action}, a comma-separated list. Default is none.
10574@xref{Java Parser Interface}.
10575@end deffn
10576
10577
10578@c ================================================= FAQ
10579
10580@node FAQ
10581@chapter Frequently Asked Questions
10582@cindex frequently asked questions
10583@cindex questions
10584
10585Several questions about Bison come up occasionally. Here some of them
10586are addressed.
10587
10588@menu
10589* Memory Exhausted:: Breaking the Stack Limits
10590* How Can I Reset the Parser:: @code{yyparse} Keeps some State
10591* Strings are Destroyed:: @code{yylval} Loses Track of Strings
10592* Implementing Gotos/Loops:: Control Flow in the Calculator
10593* Multiple start-symbols:: Factoring closely related grammars
10594* Secure? Conform?:: Is Bison POSIX safe?
10595* I can't build Bison:: Troubleshooting
10596* Where can I find help?:: Troubleshouting
10597* Bug Reports:: Troublereporting
10598* More Languages:: Parsers in C++, Java, and so on
10599* Beta Testing:: Experimenting development versions
10600* Mailing Lists:: Meeting other Bison users
10601@end menu
10602
10603@node Memory Exhausted
10604@section Memory Exhausted
10605
10606@display
10607My parser returns with error with a @samp{memory exhausted}
10608message. What can I do?
10609@end display
10610
10611This question is already addressed elsewhere, @xref{Recursion,
10612,Recursive Rules}.
10613
10614@node How Can I Reset the Parser
10615@section How Can I Reset the Parser
10616
10617The following phenomenon has several symptoms, resulting in the
10618following typical questions:
10619
10620@display
10621I invoke @code{yyparse} several times, and on correct input it works
10622properly; but when a parse error is found, all the other calls fail
10623too. How can I reset the error flag of @code{yyparse}?
10624@end display
10625
10626@noindent
10627or
10628
10629@display
10630My parser includes support for an @samp{#include}-like feature, in
10631which case I run @code{yyparse} from @code{yyparse}. This fails
10632although I did specify @samp{%define api.pure}.
10633@end display
10634
10635These problems typically come not from Bison itself, but from
10636Lex-generated scanners. Because these scanners use large buffers for
10637speed, they might not notice a change of input file. As a
10638demonstration, consider the following source file,
10639@file{first-line.l}:
10640
10641@verbatim
10642%{
10643#include <stdio.h>
10644#include <stdlib.h>
10645%}
10646%%
10647.*\n ECHO; return 1;
10648%%
10649int
10650yyparse (char const *file)
10651{
10652 yyin = fopen (file, "r");
10653 if (!yyin)
10654 exit (2);
10655 /* One token only. */
10656 yylex ();
10657 if (fclose (yyin) != 0)
10658 exit (3);
10659 return 0;
10660}
10661
10662int
10663main (void)
10664{
10665 yyparse ("input");
10666 yyparse ("input");
10667 return 0;
10668}
10669@end verbatim
10670
10671@noindent
10672If the file @file{input} contains
10673
10674@verbatim
10675input:1: Hello,
10676input:2: World!
10677@end verbatim
10678
10679@noindent
10680then instead of getting the first line twice, you get:
10681
10682@example
10683$ @kbd{flex -ofirst-line.c first-line.l}
10684$ @kbd{gcc -ofirst-line first-line.c -ll}
10685$ @kbd{./first-line}
10686input:1: Hello,
10687input:2: World!
10688@end example
10689
10690Therefore, whenever you change @code{yyin}, you must tell the
10691Lex-generated scanner to discard its current buffer and switch to the
10692new one. This depends upon your implementation of Lex; see its
10693documentation for more. For Flex, it suffices to call
10694@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
10695Flex-generated scanner needs to read from several input streams to
10696handle features like include files, you might consider using Flex
10697functions like @samp{yy_switch_to_buffer} that manipulate multiple
10698input buffers.
10699
10700If your Flex-generated scanner uses start conditions (@pxref{Start
10701conditions, , Start conditions, flex, The Flex Manual}), you might
10702also want to reset the scanner's state, i.e., go back to the initial
10703start condition, through a call to @samp{BEGIN (0)}.
10704
10705@node Strings are Destroyed
10706@section Strings are Destroyed
10707
10708@display
10709My parser seems to destroy old strings, or maybe it loses track of
10710them. Instead of reporting @samp{"foo", "bar"}, it reports
10711@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
10712@end display
10713
10714This error is probably the single most frequent ``bug report'' sent to
10715Bison lists, but is only concerned with a misunderstanding of the role
10716of the scanner. Consider the following Lex code:
10717
10718@verbatim
10719%{
10720#include <stdio.h>
10721char *yylval = NULL;
10722%}
10723%%
10724.* yylval = yytext; return 1;
10725\n /* IGNORE */
10726%%
10727int
10728main ()
10729{
10730 /* Similar to using $1, $2 in a Bison action. */
10731 char *fst = (yylex (), yylval);
10732 char *snd = (yylex (), yylval);
10733 printf ("\"%s\", \"%s\"\n", fst, snd);
10734 return 0;
10735}
10736@end verbatim
10737
10738If you compile and run this code, you get:
10739
10740@example
10741$ @kbd{flex -osplit-lines.c split-lines.l}
10742$ @kbd{gcc -osplit-lines split-lines.c -ll}
10743$ @kbd{printf 'one\ntwo\n' | ./split-lines}
10744"one
10745two", "two"
10746@end example
10747
10748@noindent
10749this is because @code{yytext} is a buffer provided for @emph{reading}
10750in the action, but if you want to keep it, you have to duplicate it
10751(e.g., using @code{strdup}). Note that the output may depend on how
10752your implementation of Lex handles @code{yytext}. For instance, when
10753given the Lex compatibility option @option{-l} (which triggers the
10754option @samp{%array}) Flex generates a different behavior:
10755
10756@example
10757$ @kbd{flex -l -osplit-lines.c split-lines.l}
10758$ @kbd{gcc -osplit-lines split-lines.c -ll}
10759$ @kbd{printf 'one\ntwo\n' | ./split-lines}
10760"two", "two"
10761@end example
10762
10763
10764@node Implementing Gotos/Loops
10765@section Implementing Gotos/Loops
10766
10767@display
10768My simple calculator supports variables, assignments, and functions,
10769but how can I implement gotos, or loops?
10770@end display
10771
10772Although very pedagogical, the examples included in the document blur
10773the distinction to make between the parser---whose job is to recover
10774the structure of a text and to transmit it to subsequent modules of
10775the program---and the processing (such as the execution) of this
10776structure. This works well with so called straight line programs,
10777i.e., precisely those that have a straightforward execution model:
10778execute simple instructions one after the others.
10779
10780@cindex abstract syntax tree
10781@cindex AST
10782If you want a richer model, you will probably need to use the parser
10783to construct a tree that does represent the structure it has
10784recovered; this tree is usually called the @dfn{abstract syntax tree},
10785or @dfn{AST} for short. Then, walking through this tree,
10786traversing it in various ways, will enable treatments such as its
10787execution or its translation, which will result in an interpreter or a
10788compiler.
10789
10790This topic is way beyond the scope of this manual, and the reader is
10791invited to consult the dedicated literature.
10792
10793
10794@node Multiple start-symbols
10795@section Multiple start-symbols
10796
10797@display
10798I have several closely related grammars, and I would like to share their
10799implementations. In fact, I could use a single grammar but with
10800multiple entry points.
10801@end display
10802
10803Bison does not support multiple start-symbols, but there is a very
10804simple means to simulate them. If @code{foo} and @code{bar} are the two
10805pseudo start-symbols, then introduce two new tokens, say
10806@code{START_FOO} and @code{START_BAR}, and use them as switches from the
10807real start-symbol:
10808
10809@example
10810%token START_FOO START_BAR;
10811%start start;
10812start: START_FOO foo
10813 | START_BAR bar;
10814@end example
10815
10816These tokens prevents the introduction of new conflicts. As far as the
10817parser goes, that is all that is needed.
10818
10819Now the difficult part is ensuring that the scanner will send these
10820tokens first. If your scanner is hand-written, that should be
10821straightforward. If your scanner is generated by Lex, them there is
10822simple means to do it: recall that anything between @samp{%@{ ... %@}}
10823after the first @code{%%} is copied verbatim in the top of the generated
10824@code{yylex} function. Make sure a variable @code{start_token} is
10825available in the scanner (e.g., a global variable or using
10826@code{%lex-param} etc.), and use the following:
10827
10828@example
10829 /* @r{Prologue.} */
10830%%
10831%@{
10832 if (start_token)
10833 @{
10834 int t = start_token;
10835 start_token = 0;
10836 return t;
10837 @}
10838%@}
10839 /* @r{The rules.} */
10840@end example
10841
10842
10843@node Secure? Conform?
10844@section Secure? Conform?
10845
10846@display
10847Is Bison secure? Does it conform to POSIX?
10848@end display
10849
10850If you're looking for a guarantee or certification, we don't provide it.
10851However, Bison is intended to be a reliable program that conforms to the
10852POSIX specification for Yacc. If you run into problems,
10853please send us a bug report.
10854
10855@node I can't build Bison
10856@section I can't build Bison
10857
10858@display
10859I can't build Bison because @command{make} complains that
10860@code{msgfmt} is not found.
10861What should I do?
10862@end display
10863
10864Like most GNU packages with internationalization support, that feature
10865is turned on by default. If you have problems building in the @file{po}
10866subdirectory, it indicates that your system's internationalization
10867support is lacking. You can re-configure Bison with
10868@option{--disable-nls} to turn off this support, or you can install GNU
10869gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
10870Bison. See the file @file{ABOUT-NLS} for more information.
10871
10872
10873@node Where can I find help?
10874@section Where can I find help?
10875
10876@display
10877I'm having trouble using Bison. Where can I find help?
10878@end display
10879
10880First, read this fine manual. Beyond that, you can send mail to
10881@email{help-bison@@gnu.org}. This mailing list is intended to be
10882populated with people who are willing to answer questions about using
10883and installing Bison. Please keep in mind that (most of) the people on
10884the list have aspects of their lives which are not related to Bison (!),
10885so you may not receive an answer to your question right away. This can
10886be frustrating, but please try not to honk them off; remember that any
10887help they provide is purely voluntary and out of the kindness of their
10888hearts.
10889
10890@node Bug Reports
10891@section Bug Reports
10892
10893@display
10894I found a bug. What should I include in the bug report?
10895@end display
10896
10897Before you send a bug report, make sure you are using the latest
10898version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
10899mirrors. Be sure to include the version number in your bug report. If
10900the bug is present in the latest version but not in a previous version,
10901try to determine the most recent version which did not contain the bug.
10902
10903If the bug is parser-related, you should include the smallest grammar
10904you can which demonstrates the bug. The grammar file should also be
10905complete (i.e., I should be able to run it through Bison without having
10906to edit or add anything). The smaller and simpler the grammar, the
10907easier it will be to fix the bug.
10908
10909Include information about your compilation environment, including your
10910operating system's name and version and your compiler's name and
10911version. If you have trouble compiling, you should also include a
10912transcript of the build session, starting with the invocation of
10913`configure'. Depending on the nature of the bug, you may be asked to
10914send additional files as well (such as `config.h' or `config.cache').
10915
10916Patches are most welcome, but not required. That is, do not hesitate to
10917send a bug report just because you can not provide a fix.
10918
10919Send bug reports to @email{bug-bison@@gnu.org}.
10920
10921@node More Languages
10922@section More Languages
10923
10924@display
10925Will Bison ever have C++ and Java support? How about @var{insert your
10926favorite language here}?
10927@end display
10928
10929C++ and Java support is there now, and is documented. We'd love to add other
10930languages; contributions are welcome.
10931
10932@node Beta Testing
10933@section Beta Testing
10934
10935@display
10936What is involved in being a beta tester?
10937@end display
10938
10939It's not terribly involved. Basically, you would download a test
10940release, compile it, and use it to build and run a parser or two. After
10941that, you would submit either a bug report or a message saying that
10942everything is okay. It is important to report successes as well as
10943failures because test releases eventually become mainstream releases,
10944but only if they are adequately tested. If no one tests, development is
10945essentially halted.
10946
10947Beta testers are particularly needed for operating systems to which the
10948developers do not have easy access. They currently have easy access to
10949recent GNU/Linux and Solaris versions. Reports about other operating
10950systems are especially welcome.
10951
10952@node Mailing Lists
10953@section Mailing Lists
10954
10955@display
10956How do I join the help-bison and bug-bison mailing lists?
10957@end display
10958
10959See @url{http://lists.gnu.org/}.
10960
10961@c ================================================= Table of Symbols
10962
10963@node Table of Symbols
10964@appendix Bison Symbols
10965@cindex Bison symbols, table of
10966@cindex symbols in Bison, table of
10967
10968@deffn {Variable} @@$
10969In an action, the location of the left-hand side of the rule.
10970@xref{Locations, , Locations Overview}.
10971@end deffn
10972
10973@deffn {Variable} @@@var{n}
10974In an action, the location of the @var{n}-th symbol of the right-hand
10975side of the rule. @xref{Locations, , Locations Overview}.
10976@end deffn
10977
10978@deffn {Variable} @@@var{name}
10979In an action, the location of a symbol addressed by name.
10980@xref{Locations, , Locations Overview}.
10981@end deffn
10982
10983@deffn {Variable} @@[@var{name}]
10984In an action, the location of a symbol addressed by name.
10985@xref{Locations, , Locations Overview}.
10986@end deffn
10987
10988@deffn {Variable} $$
10989In an action, the semantic value of the left-hand side of the rule.
10990@xref{Actions}.
10991@end deffn
10992
10993@deffn {Variable} $@var{n}
10994In an action, the semantic value of the @var{n}-th symbol of the
10995right-hand side of the rule. @xref{Actions}.
10996@end deffn
10997
10998@deffn {Variable} $@var{name}
10999In an action, the semantic value of a symbol addressed by name.
11000@xref{Actions}.
11001@end deffn
11002
11003@deffn {Variable} $[@var{name}]
11004In an action, the semantic value of a symbol addressed by name.
11005@xref{Actions}.
11006@end deffn
11007
11008@deffn {Delimiter} %%
11009Delimiter used to separate the grammar rule section from the
11010Bison declarations section or the epilogue.
11011@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
11012@end deffn
11013
11014@c Don't insert spaces, or check the DVI output.
11015@deffn {Delimiter} %@{@var{code}%@}
11016All code listed between @samp{%@{} and @samp{%@}} is copied verbatim
11017to the parser implementation file. Such code forms the prologue of
11018the grammar file. @xref{Grammar Outline, ,Outline of a Bison
11019Grammar}.
11020@end deffn
11021
11022@deffn {Directive} %?@{@var{expression}@}
11023Predicate actions. This is a type of action clause that may appear in
11024rules. The expression is evaluated, and if false, causes a syntax error. In
11025GLR parsers during nondeterministic operation,
11026this silently causes an alternative parse to die. During deterministic
11027operation, it is the same as the effect of YYERROR.
11028@xref{Semantic Predicates}.
11029
11030This feature is experimental.
11031More user feedback will help to determine whether it should become a permanent
11032feature.
11033@end deffn
11034
11035@deffn {Construct} /*@dots{}*/
11036Comment delimiters, as in C.
11037@end deffn
11038
11039@deffn {Delimiter} :
11040Separates a rule's result from its components. @xref{Rules, ,Syntax of
11041Grammar Rules}.
11042@end deffn
11043
11044@deffn {Delimiter} ;
11045Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
11046@end deffn
11047
11048@deffn {Delimiter} |
11049Separates alternate rules for the same result nonterminal.
11050@xref{Rules, ,Syntax of Grammar Rules}.
11051@end deffn
11052
11053@deffn {Directive} <*>
11054Used to define a default tagged @code{%destructor} or default tagged
11055@code{%printer}.
11056
11057This feature is experimental.
11058More user feedback will help to determine whether it should become a permanent
11059feature.
11060
11061@xref{Destructor Decl, , Freeing Discarded Symbols}.
11062@end deffn
11063
11064@deffn {Directive} <>
11065Used to define a default tagless @code{%destructor} or default tagless
11066@code{%printer}.
11067
11068This feature is experimental.
11069More user feedback will help to determine whether it should become a permanent
11070feature.
11071
11072@xref{Destructor Decl, , Freeing Discarded Symbols}.
11073@end deffn
11074
11075@deffn {Symbol} $accept
11076The predefined nonterminal whose only rule is @samp{$accept: @var{start}
11077$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
11078Start-Symbol}. It cannot be used in the grammar.
11079@end deffn
11080
11081@deffn {Directive} %code @{@var{code}@}
11082@deffnx {Directive} %code @var{qualifier} @{@var{code}@}
11083Insert @var{code} verbatim into the output parser source at the
11084default location or at the location specified by @var{qualifier}.
11085@xref{%code Summary}.
11086@end deffn
11087
11088@deffn {Directive} %debug
11089Equip the parser for debugging. @xref{Decl Summary}.
11090@end deffn
11091
11092@ifset defaultprec
11093@deffn {Directive} %default-prec
11094Assign a precedence to rules that lack an explicit @samp{%prec}
11095modifier. @xref{Contextual Precedence, ,Context-Dependent
11096Precedence}.
11097@end deffn
11098@end ifset
11099
11100@deffn {Directive} %define @var{variable}
11101@deffnx {Directive} %define @var{variable} @var{value}
11102@deffnx {Directive} %define @var{variable} "@var{value}"
11103Define a variable to adjust Bison's behavior. @xref{%define Summary}.
11104@end deffn
11105
11106@deffn {Directive} %defines
11107Bison declaration to create a parser header file, which is usually
11108meant for the scanner. @xref{Decl Summary}.
11109@end deffn
11110
11111@deffn {Directive} %defines @var{defines-file}
11112Same as above, but save in the file @var{defines-file}.
11113@xref{Decl Summary}.
11114@end deffn
11115
11116@deffn {Directive} %destructor
11117Specify how the parser should reclaim the memory associated to
11118discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
11119@end deffn
11120
11121@deffn {Directive} %dprec
11122Bison declaration to assign a precedence to a rule that is used at parse
11123time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
11124GLR Parsers}.
11125@end deffn
11126
11127@deffn {Symbol} $end
11128The predefined token marking the end of the token stream. It cannot be
11129used in the grammar.
11130@end deffn
11131
11132@deffn {Symbol} error
11133A token name reserved for error recovery. This token may be used in
11134grammar rules so as to allow the Bison parser to recognize an error in
11135the grammar without halting the process. In effect, a sentence
11136containing an error may be recognized as valid. On a syntax error, the
11137token @code{error} becomes the current lookahead token. Actions
11138corresponding to @code{error} are then executed, and the lookahead
11139token is reset to the token that originally caused the violation.
11140@xref{Error Recovery}.
11141@end deffn
11142
11143@deffn {Directive} %error-verbose
11144An obsolete directive standing for @samp{%define parse.error verbose}
11145(@pxref{Error Reporting, ,The Error Reporting Function @code{yyerror}}).
11146@end deffn
11147
11148@deffn {Directive} %file-prefix "@var{prefix}"
11149Bison declaration to set the prefix of the output files. @xref{Decl
11150Summary}.
11151@end deffn
11152
11153@deffn {Directive} %glr-parser
11154Bison declaration to produce a GLR parser. @xref{GLR
11155Parsers, ,Writing GLR Parsers}.
11156@end deffn
11157
11158@deffn {Directive} %initial-action
11159Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
11160@end deffn
11161
11162@deffn {Directive} %language
11163Specify the programming language for the generated parser.
11164@xref{Decl Summary}.
11165@end deffn
11166
11167@deffn {Directive} %left
11168Bison declaration to assign precedence and left associativity to token(s).
11169@xref{Precedence Decl, ,Operator Precedence}.
11170@end deffn
11171
11172@deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
11173Bison declaration to specifying additional arguments that
11174@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
11175for Pure Parsers}.
11176@end deffn
11177
11178@deffn {Directive} %merge
11179Bison declaration to assign a merging function to a rule. If there is a
11180reduce/reduce conflict with a rule having the same merging function, the
11181function is applied to the two semantic values to get a single result.
11182@xref{GLR Parsers, ,Writing GLR Parsers}.
11183@end deffn
11184
11185@deffn {Directive} %name-prefix "@var{prefix}"
11186Bison declaration to rename the external symbols. @xref{Decl Summary}.
11187@end deffn
11188
11189@ifset defaultprec
11190@deffn {Directive} %no-default-prec
11191Do not assign a precedence to rules that lack an explicit @samp{%prec}
11192modifier. @xref{Contextual Precedence, ,Context-Dependent
11193Precedence}.
11194@end deffn
11195@end ifset
11196
11197@deffn {Directive} %no-lines
11198Bison declaration to avoid generating @code{#line} directives in the
11199parser implementation file. @xref{Decl Summary}.
11200@end deffn
11201
11202@deffn {Directive} %nonassoc
11203Bison declaration to assign precedence and nonassociativity to token(s).
11204@xref{Precedence Decl, ,Operator Precedence}.
11205@end deffn
11206
11207@deffn {Directive} %output "@var{file}"
11208Bison declaration to set the name of the parser implementation file.
11209@xref{Decl Summary}.
11210@end deffn
11211
11212@deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
11213Bison declaration to specify additional arguments that both
11214@code{yylex} and @code{yyparse} should accept. @xref{Parser Function,, The
11215Parser Function @code{yyparse}}.
11216@end deffn
11217
11218@deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
11219Bison declaration to specify additional arguments that @code{yyparse}
11220should accept. @xref{Parser Function,, The Parser Function @code{yyparse}}.
11221@end deffn
11222
11223@deffn {Directive} %prec
11224Bison declaration to assign a precedence to a specific rule.
11225@xref{Contextual Precedence, ,Context-Dependent Precedence}.
11226@end deffn
11227
11228@deffn {Directive} %precedence
11229Bison declaration to assign precedence to token(s), but no associativity
11230@xref{Precedence Decl, ,Operator Precedence}.
11231@end deffn
11232
11233@deffn {Directive} %pure-parser
11234Deprecated version of @samp{%define api.pure} (@pxref{%define
11235Summary,,api.pure}), for which Bison is more careful to warn about
11236unreasonable usage.
11237@end deffn
11238
11239@deffn {Directive} %require "@var{version}"
11240Require version @var{version} or higher of Bison. @xref{Require Decl, ,
11241Require a Version of Bison}.
11242@end deffn
11243
11244@deffn {Directive} %right
11245Bison declaration to assign precedence and right associativity to token(s).
11246@xref{Precedence Decl, ,Operator Precedence}.
11247@end deffn
11248
11249@deffn {Directive} %skeleton
11250Specify the skeleton to use; usually for development.
11251@xref{Decl Summary}.
11252@end deffn
11253
11254@deffn {Directive} %start
11255Bison declaration to specify the start symbol. @xref{Start Decl, ,The
11256Start-Symbol}.
11257@end deffn
11258
11259@deffn {Directive} %token
11260Bison declaration to declare token(s) without specifying precedence.
11261@xref{Token Decl, ,Token Type Names}.
11262@end deffn
11263
11264@deffn {Directive} %token-table
11265Bison declaration to include a token name table in the parser
11266implementation file. @xref{Decl Summary}.
11267@end deffn
11268
11269@deffn {Directive} %type
11270Bison declaration to declare nonterminals. @xref{Type Decl,
11271,Nonterminal Symbols}.
11272@end deffn
11273
11274@deffn {Symbol} $undefined
11275The predefined token onto which all undefined values returned by
11276@code{yylex} are mapped. It cannot be used in the grammar, rather, use
11277@code{error}.
11278@end deffn
11279
11280@deffn {Directive} %union
11281Bison declaration to specify several possible data types for semantic
11282values. @xref{Union Decl, ,The Collection of Value Types}.
11283@end deffn
11284
11285@deffn {Macro} YYABORT
11286Macro to pretend that an unrecoverable syntax error has occurred, by
11287making @code{yyparse} return 1 immediately. The error reporting
11288function @code{yyerror} is not called. @xref{Parser Function, ,The
11289Parser Function @code{yyparse}}.
11290
11291For Java parsers, this functionality is invoked using @code{return YYABORT;}
11292instead.
11293@end deffn
11294
11295@deffn {Macro} YYACCEPT
11296Macro to pretend that a complete utterance of the language has been
11297read, by making @code{yyparse} return 0 immediately.
11298@xref{Parser Function, ,The Parser Function @code{yyparse}}.
11299
11300For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
11301instead.
11302@end deffn
11303
11304@deffn {Macro} YYBACKUP
11305Macro to discard a value from the parser stack and fake a lookahead
11306token. @xref{Action Features, ,Special Features for Use in Actions}.
11307@end deffn
11308
11309@deffn {Variable} yychar
11310External integer variable that contains the integer value of the
11311lookahead token. (In a pure parser, it is a local variable within
11312@code{yyparse}.) Error-recovery rule actions may examine this variable.
11313@xref{Action Features, ,Special Features for Use in Actions}.
11314@end deffn
11315
11316@deffn {Variable} yyclearin
11317Macro used in error-recovery rule actions. It clears the previous
11318lookahead token. @xref{Error Recovery}.
11319@end deffn
11320
11321@deffn {Macro} YYDEBUG
11322Macro to define to equip the parser with tracing code. @xref{Tracing,
11323,Tracing Your Parser}.
11324@end deffn
11325
11326@deffn {Variable} yydebug
11327External integer variable set to zero by default. If @code{yydebug}
11328is given a nonzero value, the parser will output information on input
11329symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
11330@end deffn
11331
11332@deffn {Macro} yyerrok
11333Macro to cause parser to recover immediately to its normal mode
11334after a syntax error. @xref{Error Recovery}.
11335@end deffn
11336
11337@deffn {Macro} YYERROR
11338Macro to pretend that a syntax error has just been detected: call
11339@code{yyerror} and then perform normal error recovery if possible
11340(@pxref{Error Recovery}), or (if recovery is impossible) make
11341@code{yyparse} return 1. @xref{Error Recovery}.
11342
11343For Java parsers, this functionality is invoked using @code{return YYERROR;}
11344instead.
11345@end deffn
11346
11347@deffn {Function} yyerror
11348User-supplied function to be called by @code{yyparse} on error.
11349@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
11350@end deffn
11351
11352@deffn {Macro} YYERROR_VERBOSE
11353An obsolete macro used in the @file{yacc.c} skeleton, that you define
11354with @code{#define} in the prologue to request verbose, specific error
11355message strings when @code{yyerror} is called. It doesn't matter what
11356definition you use for @code{YYERROR_VERBOSE}, just whether you define
11357it. Using @samp{%define parse.error verbose} is preferred
11358(@pxref{Error Reporting, ,The Error Reporting Function @code{yyerror}}).
11359@end deffn
11360
11361@deffn {Macro} YYINITDEPTH
11362Macro for specifying the initial size of the parser stack.
11363@xref{Memory Management}.
11364@end deffn
11365
11366@deffn {Function} yylex
11367User-supplied lexical analyzer function, called with no arguments to get
11368the next token. @xref{Lexical, ,The Lexical Analyzer Function
11369@code{yylex}}.
11370@end deffn
11371
11372@deffn {Macro} YYLEX_PARAM
11373An obsolete macro for specifying an extra argument (or list of extra
11374arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
11375macro is deprecated, and is supported only for Yacc like parsers.
11376@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
11377@end deffn
11378
11379@deffn {Variable} yylloc
11380External variable in which @code{yylex} should place the line and column
11381numbers associated with a token. (In a pure parser, it is a local
11382variable within @code{yyparse}, and its address is passed to
11383@code{yylex}.)
11384You can ignore this variable if you don't use the @samp{@@} feature in the
11385grammar actions.
11386@xref{Token Locations, ,Textual Locations of Tokens}.
11387In semantic actions, it stores the location of the lookahead token.
11388@xref{Actions and Locations, ,Actions and Locations}.
11389@end deffn
11390
11391@deffn {Type} YYLTYPE
11392Data type of @code{yylloc}; by default, a structure with four
11393members. @xref{Location Type, , Data Types of Locations}.
11394@end deffn
11395
11396@deffn {Variable} yylval
11397External variable in which @code{yylex} should place the semantic
11398value associated with a token. (In a pure parser, it is a local
11399variable within @code{yyparse}, and its address is passed to
11400@code{yylex}.)
11401@xref{Token Values, ,Semantic Values of Tokens}.
11402In semantic actions, it stores the semantic value of the lookahead token.
11403@xref{Actions, ,Actions}.
11404@end deffn
11405
11406@deffn {Macro} YYMAXDEPTH
11407Macro for specifying the maximum size of the parser stack. @xref{Memory
11408Management}.
11409@end deffn
11410
11411@deffn {Variable} yynerrs
11412Global variable which Bison increments each time it reports a syntax error.
11413(In a pure parser, it is a local variable within @code{yyparse}. In a
11414pure push parser, it is a member of yypstate.)
11415@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
11416@end deffn
11417
11418@deffn {Function} yyparse
11419The parser function produced by Bison; call this function to start
11420parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
11421@end deffn
11422
11423@deffn {Function} yypstate_delete
11424The function to delete a parser instance, produced by Bison in push mode;
11425call this function to delete the memory associated with a parser.
11426@xref{Parser Delete Function, ,The Parser Delete Function
11427@code{yypstate_delete}}.
11428(The current push parsing interface is experimental and may evolve.
11429More user feedback will help to stabilize it.)
11430@end deffn
11431
11432@deffn {Function} yypstate_new
11433The function to create a parser instance, produced by Bison in push mode;
11434call this function to create a new parser.
11435@xref{Parser Create Function, ,The Parser Create Function
11436@code{yypstate_new}}.
11437(The current push parsing interface is experimental and may evolve.
11438More user feedback will help to stabilize it.)
11439@end deffn
11440
11441@deffn {Function} yypull_parse
11442The parser function produced by Bison in push mode; call this function to
11443parse the rest of the input stream.
11444@xref{Pull Parser Function, ,The Pull Parser Function
11445@code{yypull_parse}}.
11446(The current push parsing interface is experimental and may evolve.
11447More user feedback will help to stabilize it.)
11448@end deffn
11449
11450@deffn {Function} yypush_parse
11451The parser function produced by Bison in push mode; call this function to
11452parse a single token. @xref{Push Parser Function, ,The Push Parser Function
11453@code{yypush_parse}}.
11454(The current push parsing interface is experimental and may evolve.
11455More user feedback will help to stabilize it.)
11456@end deffn
11457
11458@deffn {Macro} YYPARSE_PARAM
11459An obsolete macro for specifying the name of a parameter that
11460@code{yyparse} should accept. The use of this macro is deprecated, and
11461is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
11462Conventions for Pure Parsers}.
11463@end deffn
11464
11465@deffn {Macro} YYRECOVERING
11466The expression @code{YYRECOVERING ()} yields 1 when the parser
11467is recovering from a syntax error, and 0 otherwise.
11468@xref{Action Features, ,Special Features for Use in Actions}.
11469@end deffn
11470
11471@deffn {Macro} YYSTACK_USE_ALLOCA
11472Macro used to control the use of @code{alloca} when the
11473deterministic parser in C needs to extend its stacks. If defined to 0,
11474the parser will use @code{malloc} to extend its stacks. If defined to
114751, the parser will use @code{alloca}. Values other than 0 and 1 are
11476reserved for future Bison extensions. If not defined,
11477@code{YYSTACK_USE_ALLOCA} defaults to 0.
11478
11479In the all-too-common case where your code may run on a host with a
11480limited stack and with unreliable stack-overflow checking, you should
11481set @code{YYMAXDEPTH} to a value that cannot possibly result in
11482unchecked stack overflow on any of your target hosts when
11483@code{alloca} is called. You can inspect the code that Bison
11484generates in order to determine the proper numeric values. This will
11485require some expertise in low-level implementation details.
11486@end deffn
11487
11488@deffn {Type} YYSTYPE
11489Data type of semantic values; @code{int} by default.
11490@xref{Value Type, ,Data Types of Semantic Values}.
11491@end deffn
11492
11493@node Glossary
11494@appendix Glossary
11495@cindex glossary
11496
11497@table @asis
11498@item Accepting state
11499A state whose only action is the accept action.
11500The accepting state is thus a consistent state.
11501@xref{Understanding,,}.
11502
11503@item Backus-Naur Form (BNF; also called ``Backus Normal Form'')
11504Formal method of specifying context-free grammars originally proposed
11505by John Backus, and slightly improved by Peter Naur in his 1960-01-02
11506committee document contributing to what became the Algol 60 report.
11507@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11508
11509@item Consistent state
11510A state containing only one possible action. @xref{Default Reductions}.
11511
11512@item Context-free grammars
11513Grammars specified as rules that can be applied regardless of context.
11514Thus, if there is a rule which says that an integer can be used as an
11515expression, integers are allowed @emph{anywhere} an expression is
11516permitted. @xref{Language and Grammar, ,Languages and Context-Free
11517Grammars}.
11518
11519@item Default reduction
11520The reduction that a parser should perform if the current parser state
11521contains no other action for the lookahead token. In permitted parser
11522states, Bison declares the reduction with the largest lookahead set to be
11523the default reduction and removes that lookahead set. @xref{Default
11524Reductions}.
11525
11526@item Defaulted state
11527A consistent state with a default reduction. @xref{Default Reductions}.
11528
11529@item Dynamic allocation
11530Allocation of memory that occurs during execution, rather than at
11531compile time or on entry to a function.
11532
11533@item Empty string
11534Analogous to the empty set in set theory, the empty string is a
11535character string of length zero.
11536
11537@item Finite-state stack machine
11538A ``machine'' that has discrete states in which it is said to exist at
11539each instant in time. As input to the machine is processed, the
11540machine moves from state to state as specified by the logic of the
11541machine. In the case of the parser, the input is the language being
11542parsed, and the states correspond to various stages in the grammar
11543rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
11544
11545@item Generalized LR (GLR)
11546A parsing algorithm that can handle all context-free grammars, including those
11547that are not LR(1). It resolves situations that Bison's
11548deterministic parsing
11549algorithm cannot by effectively splitting off multiple parsers, trying all
11550possible parsers, and discarding those that fail in the light of additional
11551right context. @xref{Generalized LR Parsing, ,Generalized
11552LR Parsing}.
11553
11554@item Grouping
11555A language construct that is (in general) grammatically divisible;
11556for example, `expression' or `declaration' in C@.
11557@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11558
11559@item IELR(1) (Inadequacy Elimination LR(1))
11560A minimal LR(1) parser table construction algorithm. That is, given any
11561context-free grammar, IELR(1) generates parser tables with the full
11562language-recognition power of canonical LR(1) but with nearly the same
11563number of parser states as LALR(1). This reduction in parser states is
11564often an order of magnitude. More importantly, because canonical LR(1)'s
11565extra parser states may contain duplicate conflicts in the case of non-LR(1)
11566grammars, the number of conflicts for IELR(1) is often an order of magnitude
11567less as well. This can significantly reduce the complexity of developing a
11568grammar. @xref{LR Table Construction}.
11569
11570@item Infix operator
11571An arithmetic operator that is placed between the operands on which it
11572performs some operation.
11573
11574@item Input stream
11575A continuous flow of data between devices or programs.
11576
11577@item LAC (Lookahead Correction)
11578A parsing mechanism that fixes the problem of delayed syntax error
11579detection, which is caused by LR state merging, default reductions, and the
11580use of @code{%nonassoc}. Delayed syntax error detection results in
11581unexpected semantic actions, initiation of error recovery in the wrong
11582syntactic context, and an incorrect list of expected tokens in a verbose
11583syntax error message. @xref{LAC}.
11584
11585@item Language construct
11586One of the typical usage schemas of the language. For example, one of
11587the constructs of the C language is the @code{if} statement.
11588@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11589
11590@item Left associativity
11591Operators having left associativity are analyzed from left to right:
11592@samp{a+b+c} first computes @samp{a+b} and then combines with
11593@samp{c}. @xref{Precedence, ,Operator Precedence}.
11594
11595@item Left recursion
11596A rule whose result symbol is also its first component symbol; for
11597example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
11598Rules}.
11599
11600@item Left-to-right parsing
11601Parsing a sentence of a language by analyzing it token by token from
11602left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
11603
11604@item Lexical analyzer (scanner)
11605A function that reads an input stream and returns tokens one by one.
11606@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
11607
11608@item Lexical tie-in
11609A flag, set by actions in the grammar rules, which alters the way
11610tokens are parsed. @xref{Lexical Tie-ins}.
11611
11612@item Literal string token
11613A token which consists of two or more fixed characters. @xref{Symbols}.
11614
11615@item Lookahead token
11616A token already read but not yet shifted. @xref{Lookahead, ,Lookahead
11617Tokens}.
11618
11619@item LALR(1)
11620The class of context-free grammars that Bison (like most other parser
11621generators) can handle by default; a subset of LR(1).
11622@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
11623
11624@item LR(1)
11625The class of context-free grammars in which at most one token of
11626lookahead is needed to disambiguate the parsing of any piece of input.
11627
11628@item Nonterminal symbol
11629A grammar symbol standing for a grammatical construct that can
11630be expressed through rules in terms of smaller constructs; in other
11631words, a construct that is not a token. @xref{Symbols}.
11632
11633@item Parser
11634A function that recognizes valid sentences of a language by analyzing
11635the syntax structure of a set of tokens passed to it from a lexical
11636analyzer.
11637
11638@item Postfix operator
11639An arithmetic operator that is placed after the operands upon which it
11640performs some operation.
11641
11642@item Reduction
11643Replacing a string of nonterminals and/or terminals with a single
11644nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
11645Parser Algorithm}.
11646
11647@item Reentrant
11648A reentrant subprogram is a subprogram which can be in invoked any
11649number of times in parallel, without interference between the various
11650invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
11651
11652@item Reverse polish notation
11653A language in which all operators are postfix operators.
11654
11655@item Right recursion
11656A rule whose result symbol is also its last component symbol; for
11657example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
11658Rules}.
11659
11660@item Semantics
11661In computer languages, the semantics are specified by the actions
11662taken for each instance of the language, i.e., the meaning of
11663each statement. @xref{Semantics, ,Defining Language Semantics}.
11664
11665@item Shift
11666A parser is said to shift when it makes the choice of analyzing
11667further input from the stream rather than reducing immediately some
11668already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
11669
11670@item Single-character literal
11671A single character that is recognized and interpreted as is.
11672@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
11673
11674@item Start symbol
11675The nonterminal symbol that stands for a complete valid utterance in
11676the language being parsed. The start symbol is usually listed as the
11677first nonterminal symbol in a language specification.
11678@xref{Start Decl, ,The Start-Symbol}.
11679
11680@item Symbol table
11681A data structure where symbol names and associated data are stored
11682during parsing to allow for recognition and use of existing
11683information in repeated uses of a symbol. @xref{Multi-function Calc}.
11684
11685@item Syntax error
11686An error encountered during parsing of an input stream due to invalid
11687syntax. @xref{Error Recovery}.
11688
11689@item Token
11690A basic, grammatically indivisible unit of a language. The symbol
11691that describes a token in the grammar is a terminal symbol.
11692The input of the Bison parser is a stream of tokens which comes from
11693the lexical analyzer. @xref{Symbols}.
11694
11695@item Terminal symbol
11696A grammar symbol that has no rules in the grammar and therefore is
11697grammatically indivisible. The piece of text it represents is a token.
11698@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11699
11700@item Unreachable state
11701A parser state to which there does not exist a sequence of transitions from
11702the parser's start state. A state can become unreachable during conflict
11703resolution. @xref{Unreachable States}.
11704@end table
11705
11706@node Copying This Manual
11707@appendix Copying This Manual
11708@include fdl.texi
11709
11710@node Bibliography
11711@unnumbered Bibliography
11712
11713@table @asis
11714@item [Denny 2008]
11715Joel E. Denny and Brian A. Malloy, IELR(1): Practical LR(1) Parser Tables
11716for Non-LR(1) Grammars with Conflict Resolution, in @cite{Proceedings of the
117172008 ACM Symposium on Applied Computing} (SAC'08), ACM, New York, NY, USA,
11718pp.@: 240--245. @uref{http://dx.doi.org/10.1145/1363686.1363747}
11719
11720@item [Denny 2010 May]
11721Joel E. Denny, PSLR(1): Pseudo-Scannerless Minimal LR(1) for the
11722Deterministic Parsing of Composite Languages, Ph.D. Dissertation, Clemson
11723University, Clemson, SC, USA (May 2010).
11724@uref{http://proquest.umi.com/pqdlink?did=2041473591&Fmt=7&clientId=79356&RQT=309&VName=PQD}
11725
11726@item [Denny 2010 November]
11727Joel E. Denny and Brian A. Malloy, The IELR(1) Algorithm for Generating
11728Minimal LR(1) Parser Tables for Non-LR(1) Grammars with Conflict Resolution,
11729in @cite{Science of Computer Programming}, Vol.@: 75, Issue 11 (November
117302010), pp.@: 943--979. @uref{http://dx.doi.org/10.1016/j.scico.2009.08.001}
11731
11732@item [DeRemer 1982]
11733Frank DeRemer and Thomas Pennello, Efficient Computation of LALR(1)
11734Look-Ahead Sets, in @cite{ACM Transactions on Programming Languages and
11735Systems}, Vol.@: 4, No.@: 4 (October 1982), pp.@:
11736615--649. @uref{http://dx.doi.org/10.1145/69622.357187}
11737
11738@item [Knuth 1965]
11739Donald E. Knuth, On the Translation of Languages from Left to Right, in
11740@cite{Information and Control}, Vol.@: 8, Issue 6 (December 1965), pp.@:
11741607--639. @uref{http://dx.doi.org/10.1016/S0019-9958(65)90426-2}
11742
11743@item [Scott 2000]
11744Elizabeth Scott, Adrian Johnstone, and Shamsa Sadaf Hussain,
11745@cite{Tomita-Style Generalised LR Parsers}, Royal Holloway, University of
11746London, Department of Computer Science, TR-00-12 (December 2000).
11747@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps}
11748@end table
11749
11750@node Index
11751@unnumbered Index
11752
11753@printindex cp
11754
11755@bye
11756
11757@c LocalWords: texinfo setfilename settitle setchapternewpage finalout texi FSF
11758@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex FSF's
11759@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry Naur
11760@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa Multi
11761@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc multi
11762@c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex defaultprec Donnelly Gotos
11763@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref yypush
11764@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex lr
11765@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge POSIX
11766@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG yypull
11767@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit nonfree
11768@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok rr
11769@c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln Stallman Destructor
11770@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym enum
11771@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof Lex
11772@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum DOTDOT
11773@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype Unary
11774@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs nonterminal
11775@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES reentrant
11776@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param yypstate
11777@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP subrange
11778@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword loc
11779@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH inline
11780@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype Lookahead
11781@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args Autoconf
11782@c LocalWords: infile ypp yxx outfile itemx tex leaderfill Troubleshouting sqrt
11783@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll lookahead
11784@c LocalWords: nbar yytext fst snd osplit ntwo strdup AST Troublereporting th
11785@c LocalWords: YYSTACK DVI fdl printindex IELR nondeterministic nonterminals ps
11786@c LocalWords: subexpressions declarator nondeferred config libintl postfix LAC
11787@c LocalWords: preprocessor nonpositive unary nonnumeric typedef extern rhs
11788@c LocalWords: yytokentype destructor multicharacter nonnull EBCDIC
11789@c LocalWords: lvalue nonnegative XNUM CHR chr TAGLESS tagless stdout api TOK
11790@c LocalWords: destructors Reentrancy nonreentrant subgrammar nonassociative
11791@c LocalWords: deffnx namespace xml goto lalr ielr runtime lex yacc yyps env
11792@c LocalWords: yystate variadic Unshift NLS gettext po UTF Automake LOCALEDIR
11793@c LocalWords: YYENABLE bindtextdomain Makefile DEFS CPPFLAGS DBISON DeRemer
11794@c LocalWords: autoreconf Pennello multisets nondeterminism Generalised baz
11795@c LocalWords: redeclare automata Dparse localedir datadir XSLT midrule Wno
11796@c LocalWords: Graphviz multitable headitem hh basename Doxygen fno
11797@c LocalWords: doxygen ival sval deftypemethod deallocate pos deftypemethodx
11798@c LocalWords: Ctor defcv defcvx arg accessors arithmetics CPP ifndef CALCXX
11799@c LocalWords: lexer's calcxx bool LPAREN RPAREN deallocation cerrno climits
11800@c LocalWords: cstdlib Debian undef yywrap unput noyywrap nounput zA yyleng
11801@c LocalWords: errno strtol ERANGE str strerror iostream argc argv Javadoc
11802@c LocalWords: bytecode initializers superclass stype ASTNode autoboxing nls
11803@c LocalWords: toString deftypeivar deftypeivarx deftypeop YYParser strictfp
11804@c LocalWords: superclasses boolean getErrorVerbose setErrorVerbose deftypecv
11805@c LocalWords: getDebugStream setDebugStream getDebugLevel setDebugLevel url
11806@c LocalWords: bisonVersion deftypecvx bisonSkeleton getStartPos getEndPos
11807@c LocalWords: getLVal defvar deftypefn deftypefnx gotos msgfmt Corbett
11808@c LocalWords: subdirectory Solaris nonassociativity
11809
11810@c Local Variables:
11811@c ispell-dictionary: "american"
11812@c fill-column: 76
11813@c End: