]> git.saurik.com Git - bison.git/blob - doc/bison.texinfo
doc: don't use @acronym.
[bison.git] / doc / bison.texinfo
1 \input texinfo @c -*-texinfo-*-
2 @comment %**start of header
3 @setfilename bison.info
4 @include version.texi
5 @settitle Bison @value{VERSION}
6 @setchapternewpage odd
7
8 @finalout
9
10 @c SMALL BOOK version
11 @c This edition has been formatted so that you can format and print it in
12 @c the smallbook format.
13 @c @smallbook
14
15 @c Set following if you want to document %default-prec and %no-default-prec.
16 @c This feature is experimental and may change in future Bison versions.
17 @c @set defaultprec
18
19 @ifnotinfo
20 @syncodeindex fn cp
21 @syncodeindex vr cp
22 @syncodeindex tp cp
23 @end ifnotinfo
24 @ifinfo
25 @synindex fn cp
26 @synindex vr cp
27 @synindex tp cp
28 @end ifinfo
29 @comment %**end of header
30
31 @copying
32
33 This manual (@value{UPDATED}) is for GNU Bison (version
34 @value{VERSION}), the GNU parser generator.
35
36 Copyright @copyright{} 1988-1993, 1995, 1998-2011 Free Software
37 Foundation, Inc.
38
39 @quotation
40 Permission is granted to copy, distribute and/or modify this document
41 under the terms of the GNU Free Documentation License,
42 Version 1.3 or any later version published by the Free Software
43 Foundation; with no Invariant Sections, with the Front-Cover texts
44 being ``A GNU Manual,'' and with the Back-Cover Texts as in
45 (a) below. A copy of the license is included in the section entitled
46 ``GNU Free Documentation License.''
47
48 (a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
49 modify this GNU manual. Buying copies from the FSF
50 supports it in developing GNU and promoting software
51 freedom.''
52 @end quotation
53 @end copying
54
55 @dircategory Software development
56 @direntry
57 * bison: (bison). GNU parser generator (Yacc replacement).
58 @end direntry
59
60 @titlepage
61 @title Bison
62 @subtitle The Yacc-compatible Parser Generator
63 @subtitle @value{UPDATED}, Bison Version @value{VERSION}
64
65 @author by Charles Donnelly and Richard Stallman
66
67 @page
68 @vskip 0pt plus 1filll
69 @insertcopying
70 @sp 2
71 Published by the Free Software Foundation @*
72 51 Franklin Street, Fifth Floor @*
73 Boston, MA 02110-1301 USA @*
74 Printed copies are available from the Free Software Foundation.@*
75 ISBN 1-882114-44-2
76 @sp 2
77 Cover art by Etienne Suvasa.
78 @end titlepage
79
80 @contents
81
82 @ifnottex
83 @node Top
84 @top Bison
85 @insertcopying
86 @end ifnottex
87
88 @menu
89 * Introduction::
90 * Conditions::
91 * Copying:: The GNU General Public License says
92 how you can copy and share Bison.
93
94 Tutorial sections:
95 * Concepts:: Basic concepts for understanding Bison.
96 * Examples:: Three simple explained examples of using Bison.
97
98 Reference sections:
99 * Grammar File:: Writing Bison declarations and rules.
100 * Interface:: C-language interface to the parser function @code{yyparse}.
101 * Algorithm:: How the Bison parser works at run-time.
102 * Error Recovery:: Writing rules for error recovery.
103 * Context Dependency:: What to do if your language syntax is too
104 messy for Bison to handle straightforwardly.
105 * Debugging:: Understanding or debugging Bison parsers.
106 * Invocation:: How to run Bison (to produce the parser source file).
107 * Other Languages:: Creating C++ and Java parsers.
108 * FAQ:: Frequently Asked Questions
109 * Table of Symbols:: All the keywords of the Bison language are explained.
110 * Glossary:: Basic concepts are explained.
111 * Copying This Manual:: License for copying this manual.
112 * Index:: Cross-references to the text.
113
114 @detailmenu
115 --- The Detailed Node Listing ---
116
117 The Concepts of Bison
118
119 * Language and Grammar:: Languages and context-free grammars,
120 as mathematical ideas.
121 * Grammar in Bison:: How we represent grammars for Bison's sake.
122 * Semantic Values:: Each token or syntactic grouping can have
123 a semantic value (the value of an integer,
124 the name of an identifier, etc.).
125 * Semantic Actions:: Each rule can have an action containing C code.
126 * GLR Parsers:: Writing parsers for general context-free languages.
127 * Locations Overview:: Tracking Locations.
128 * Bison Parser:: What are Bison's input and output,
129 how is the output used?
130 * Stages:: Stages in writing and running Bison grammars.
131 * Grammar Layout:: Overall structure of a Bison grammar file.
132
133 Writing GLR Parsers
134
135 * Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
136 * Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
137 * GLR Semantic Actions:: Deferred semantic actions have special concerns.
138 * Compiler Requirements:: GLR parsers require a modern C compiler.
139
140 Examples
141
142 * RPN Calc:: Reverse polish notation calculator;
143 a first example with no operator precedence.
144 * Infix Calc:: Infix (algebraic) notation calculator.
145 Operator precedence is introduced.
146 * Simple Error Recovery:: Continuing after syntax errors.
147 * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
148 * Multi-function Calc:: Calculator with memory and trig functions.
149 It uses multiple data-types for semantic values.
150 * Exercises:: Ideas for improving the multi-function calculator.
151
152 Reverse Polish Notation Calculator
153
154 * Rpcalc Declarations:: Prologue (declarations) for rpcalc.
155 * Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
156 * Rpcalc Lexer:: The lexical analyzer.
157 * Rpcalc Main:: The controlling function.
158 * Rpcalc Error:: The error reporting function.
159 * Rpcalc Generate:: Running Bison on the grammar file.
160 * Rpcalc Compile:: Run the C compiler on the output code.
161
162 Grammar Rules for @code{rpcalc}
163
164 * Rpcalc Input::
165 * Rpcalc Line::
166 * Rpcalc Expr::
167
168 Location Tracking Calculator: @code{ltcalc}
169
170 * Ltcalc Declarations:: Bison and C declarations for ltcalc.
171 * Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
172 * Ltcalc Lexer:: The lexical analyzer.
173
174 Multi-Function Calculator: @code{mfcalc}
175
176 * Mfcalc Declarations:: Bison declarations for multi-function calculator.
177 * Mfcalc Rules:: Grammar rules for the calculator.
178 * Mfcalc Symbol Table:: Symbol table management subroutines.
179
180 Bison Grammar Files
181
182 * Grammar Outline:: Overall layout of the grammar file.
183 * Symbols:: Terminal and nonterminal symbols.
184 * Rules:: How to write grammar rules.
185 * Recursion:: Writing recursive rules.
186 * Semantics:: Semantic values and actions.
187 * Locations:: Locations and actions.
188 * Declarations:: All kinds of Bison declarations are described here.
189 * Multiple Parsers:: Putting more than one Bison parser in one program.
190
191 Outline of a Bison Grammar
192
193 * Prologue:: Syntax and usage of the prologue.
194 * Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
195 * Bison Declarations:: Syntax and usage of the Bison declarations section.
196 * Grammar Rules:: Syntax and usage of the grammar rules section.
197 * Epilogue:: Syntax and usage of the epilogue.
198
199 Defining Language Semantics
200
201 * Value Type:: Specifying one data type for all semantic values.
202 * Multiple Types:: Specifying several alternative data types.
203 * Actions:: An action is the semantic definition of a grammar rule.
204 * Action Types:: Specifying data types for actions to operate on.
205 * Mid-Rule Actions:: Most actions go at the end of a rule.
206 This says when, why and how to use the exceptional
207 action in the middle of a rule.
208 * Named References:: Using named references in actions.
209
210 Tracking Locations
211
212 * Location Type:: Specifying a data type for locations.
213 * Actions and Locations:: Using locations in actions.
214 * Location Default Action:: Defining a general way to compute locations.
215
216 Bison Declarations
217
218 * Require Decl:: Requiring a Bison version.
219 * Token Decl:: Declaring terminal symbols.
220 * Precedence Decl:: Declaring terminals with precedence and associativity.
221 * Union Decl:: Declaring the set of all semantic value types.
222 * Type Decl:: Declaring the choice of type for a nonterminal symbol.
223 * Initial Action Decl:: Code run before parsing starts.
224 * Destructor Decl:: Declaring how symbols are freed.
225 * Expect Decl:: Suppressing warnings about parsing conflicts.
226 * Start Decl:: Specifying the start symbol.
227 * Pure Decl:: Requesting a reentrant parser.
228 * Push Decl:: Requesting a push parser.
229 * Decl Summary:: Table of all Bison declarations.
230
231 Parser C-Language Interface
232
233 * Parser Function:: How to call @code{yyparse} and what it returns.
234 * Push Parser Function:: How to call @code{yypush_parse} and what it returns.
235 * Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
236 * Parser Create Function:: How to call @code{yypstate_new} and what it returns.
237 * Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
238 * Lexical:: You must supply a function @code{yylex}
239 which reads tokens.
240 * Error Reporting:: You must supply a function @code{yyerror}.
241 * Action Features:: Special features for use in actions.
242 * Internationalization:: How to let the parser speak in the user's
243 native language.
244
245 The Lexical Analyzer Function @code{yylex}
246
247 * Calling Convention:: How @code{yyparse} calls @code{yylex}.
248 * Token Values:: How @code{yylex} must return the semantic value
249 of the token it has read.
250 * Token Locations:: How @code{yylex} must return the text location
251 (line number, etc.) of the token, if the
252 actions want that.
253 * Pure Calling:: How the calling convention differs in a pure parser
254 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
255
256 The Bison Parser Algorithm
257
258 * Lookahead:: Parser looks one token ahead when deciding what to do.
259 * Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
260 * Precedence:: Operator precedence works by resolving conflicts.
261 * Contextual Precedence:: When an operator's precedence depends on context.
262 * Parser States:: The parser is a finite-state-machine with stack.
263 * Reduce/Reduce:: When two rules are applicable in the same situation.
264 * Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
265 * Generalized LR Parsing:: Parsing arbitrary context-free grammars.
266 * Memory Management:: What happens when memory is exhausted. How to avoid it.
267
268 Operator Precedence
269
270 * Why Precedence:: An example showing why precedence is needed.
271 * Using Precedence:: How to specify precedence in Bison grammars.
272 * Precedence Examples:: How these features are used in the previous example.
273 * How Precedence:: How they work.
274
275 Handling Context Dependencies
276
277 * Semantic Tokens:: Token parsing can depend on the semantic context.
278 * Lexical Tie-ins:: Token parsing can depend on the syntactic context.
279 * Tie-in Recovery:: Lexical tie-ins have implications for how
280 error recovery rules must be written.
281
282 Debugging Your Parser
283
284 * Understanding:: Understanding the structure of your parser.
285 * Tracing:: Tracing the execution of your parser.
286
287 Invoking Bison
288
289 * Bison Options:: All the options described in detail,
290 in alphabetical order by short options.
291 * Option Cross Key:: Alphabetical list of long options.
292 * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
293
294 Parsers Written In Other Languages
295
296 * C++ Parsers:: The interface to generate C++ parser classes
297 * Java Parsers:: The interface to generate Java parser classes
298
299 C++ Parsers
300
301 * C++ Bison Interface:: Asking for C++ parser generation
302 * C++ Semantic Values:: %union vs. C++
303 * C++ Location Values:: The position and location classes
304 * C++ Parser Interface:: Instantiating and running the parser
305 * C++ Scanner Interface:: Exchanges between yylex and parse
306 * A Complete C++ Example:: Demonstrating their use
307
308 A Complete C++ Example
309
310 * Calc++ --- C++ Calculator:: The specifications
311 * Calc++ Parsing Driver:: An active parsing context
312 * Calc++ Parser:: A parser class
313 * Calc++ Scanner:: A pure C++ Flex scanner
314 * Calc++ Top Level:: Conducting the band
315
316 Java Parsers
317
318 * Java Bison Interface:: Asking for Java parser generation
319 * Java Semantic Values:: %type and %token vs. Java
320 * Java Location Values:: The position and location classes
321 * Java Parser Interface:: Instantiating and running the parser
322 * Java Scanner Interface:: Specifying the scanner for the parser
323 * Java Action Features:: Special features for use in actions
324 * Java Differences:: Differences between C/C++ and Java Grammars
325 * Java Declarations Summary:: List of Bison declarations used with Java
326
327 Frequently Asked Questions
328
329 * Memory Exhausted:: Breaking the Stack Limits
330 * How Can I Reset the Parser:: @code{yyparse} Keeps some State
331 * Strings are Destroyed:: @code{yylval} Loses Track of Strings
332 * Implementing Gotos/Loops:: Control Flow in the Calculator
333 * Multiple start-symbols:: Factoring closely related grammars
334 * Secure? Conform?:: Is Bison POSIX safe?
335 * I can't build Bison:: Troubleshooting
336 * Where can I find help?:: Troubleshouting
337 * Bug Reports:: Troublereporting
338 * More Languages:: Parsers in C++, Java, and so on
339 * Beta Testing:: Experimenting development versions
340 * Mailing Lists:: Meeting other Bison users
341
342 Copying This Manual
343
344 * Copying This Manual:: License for copying this manual.
345
346 @end detailmenu
347 @end menu
348
349 @node Introduction
350 @unnumbered Introduction
351 @cindex introduction
352
353 @dfn{Bison} is a general-purpose parser generator that converts an
354 annotated context-free grammar into a deterministic LR or
355 generalized LR (GLR) parser employing
356 LALR(1), IELR(1), or canonical LR(1)
357 parser tables.
358 Once you are proficient with Bison, you can use it to develop a wide
359 range of language parsers, from those used in simple desk calculators to
360 complex programming languages.
361
362 Bison is upward compatible with Yacc: all properly-written Yacc grammars
363 ought to work with Bison with no change. Anyone familiar with Yacc
364 should be able to use Bison with little trouble. You need to be fluent in
365 C or C++ programming in order to use Bison or to understand this manual.
366
367 We begin with tutorial chapters that explain the basic concepts of using
368 Bison and show three explained examples, each building on the last. If you
369 don't know Bison or Yacc, start by reading these chapters. Reference
370 chapters follow which describe specific aspects of Bison in detail.
371
372 Bison was written primarily by Robert Corbett; Richard Stallman made it
373 Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
374 multi-character string literals and other features.
375
376 This edition corresponds to version @value{VERSION} of Bison.
377
378 @node Conditions
379 @unnumbered Conditions for Using Bison
380
381 The distribution terms for Bison-generated parsers permit using the
382 parsers in nonfree programs. Before Bison version 2.2, these extra
383 permissions applied only when Bison was generating LALR(1)
384 parsers in C@. And before Bison version 1.24, Bison-generated
385 parsers could be used only in programs that were free software.
386
387 The other GNU programming tools, such as the GNU C
388 compiler, have never
389 had such a requirement. They could always be used for nonfree
390 software. The reason Bison was different was not due to a special
391 policy decision; it resulted from applying the usual General Public
392 License to all of the Bison source code.
393
394 The output of the Bison utility---the Bison parser file---contains a
395 verbatim copy of a sizable piece of Bison, which is the code for the
396 parser's implementation. (The actions from your grammar are inserted
397 into this implementation at one point, but most of the rest of the
398 implementation is not changed.) When we applied the GPL
399 terms to the skeleton code for the parser's implementation,
400 the effect was to restrict the use of Bison output to free software.
401
402 We didn't change the terms because of sympathy for people who want to
403 make software proprietary. @strong{Software should be free.} But we
404 concluded that limiting Bison's use to free software was doing little to
405 encourage people to make other software free. So we decided to make the
406 practical conditions for using Bison match the practical conditions for
407 using the other GNU tools.
408
409 This exception applies when Bison is generating code for a parser.
410 You can tell whether the exception applies to a Bison output file by
411 inspecting the file for text beginning with ``As a special
412 exception@dots{}''. The text spells out the exact terms of the
413 exception.
414
415 @node Copying
416 @unnumbered GNU GENERAL PUBLIC LICENSE
417 @include gpl-3.0.texi
418
419 @node Concepts
420 @chapter The Concepts of Bison
421
422 This chapter introduces many of the basic concepts without which the
423 details of Bison will not make sense. If you do not already know how to
424 use Bison or Yacc, we suggest you start by reading this chapter carefully.
425
426 @menu
427 * Language and Grammar:: Languages and context-free grammars,
428 as mathematical ideas.
429 * Grammar in Bison:: How we represent grammars for Bison's sake.
430 * Semantic Values:: Each token or syntactic grouping can have
431 a semantic value (the value of an integer,
432 the name of an identifier, etc.).
433 * Semantic Actions:: Each rule can have an action containing C code.
434 * GLR Parsers:: Writing parsers for general context-free languages.
435 * Locations Overview:: Tracking Locations.
436 * Bison Parser:: What are Bison's input and output,
437 how is the output used?
438 * Stages:: Stages in writing and running Bison grammars.
439 * Grammar Layout:: Overall structure of a Bison grammar file.
440 @end menu
441
442 @node Language and Grammar
443 @section Languages and Context-Free Grammars
444
445 @cindex context-free grammar
446 @cindex grammar, context-free
447 In order for Bison to parse a language, it must be described by a
448 @dfn{context-free grammar}. This means that you specify one or more
449 @dfn{syntactic groupings} and give rules for constructing them from their
450 parts. For example, in the C language, one kind of grouping is called an
451 `expression'. One rule for making an expression might be, ``An expression
452 can be made of a minus sign and another expression''. Another would be,
453 ``An expression can be an integer''. As you can see, rules are often
454 recursive, but there must be at least one rule which leads out of the
455 recursion.
456
457 @cindex BNF
458 @cindex Backus-Naur form
459 The most common formal system for presenting such rules for humans to read
460 is @dfn{Backus-Naur Form} or ``BNF'', which was developed in
461 order to specify the language Algol 60. Any grammar expressed in
462 BNF is a context-free grammar. The input to Bison is
463 essentially machine-readable BNF.
464
465 @cindex LALR(1) grammars
466 @cindex IELR(1) grammars
467 @cindex LR(1) grammars
468 There are various important subclasses of context-free grammars.
469 Although it can handle almost all context-free grammars, Bison is
470 optimized for what are called LR(1) grammars.
471 In brief, in these grammars, it must be possible to tell how to parse
472 any portion of an input string with just a single token of lookahead.
473 For historical reasons, Bison by default is limited by the additional
474 restrictions of LALR(1), which is hard to explain simply.
475 @xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
476 more information on this.
477 As an experimental feature, you can escape these additional restrictions by
478 requesting IELR(1) or canonical LR(1) parser tables.
479 @xref{Decl Summary,,lr.type}, to learn how.
480
481 @cindex GLR parsing
482 @cindex generalized LR (GLR) parsing
483 @cindex ambiguous grammars
484 @cindex nondeterministic parsing
485
486 Parsers for LR(1) grammars are @dfn{deterministic}, meaning
487 roughly that the next grammar rule to apply at any point in the input is
488 uniquely determined by the preceding input and a fixed, finite portion
489 (called a @dfn{lookahead}) of the remaining input. A context-free
490 grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
491 apply the grammar rules to get the same inputs. Even unambiguous
492 grammars can be @dfn{nondeterministic}, meaning that no fixed
493 lookahead always suffices to determine the next grammar rule to apply.
494 With the proper declarations, Bison is also able to parse these more
495 general context-free grammars, using a technique known as GLR
496 parsing (for Generalized LR). Bison's GLR parsers
497 are able to handle any context-free grammar for which the number of
498 possible parses of any given string is finite.
499
500 @cindex symbols (abstract)
501 @cindex token
502 @cindex syntactic grouping
503 @cindex grouping, syntactic
504 In the formal grammatical rules for a language, each kind of syntactic
505 unit or grouping is named by a @dfn{symbol}. Those which are built by
506 grouping smaller constructs according to grammatical rules are called
507 @dfn{nonterminal symbols}; those which can't be subdivided are called
508 @dfn{terminal symbols} or @dfn{token types}. We call a piece of input
509 corresponding to a single terminal symbol a @dfn{token}, and a piece
510 corresponding to a single nonterminal symbol a @dfn{grouping}.
511
512 We can use the C language as an example of what symbols, terminal and
513 nonterminal, mean. The tokens of C are identifiers, constants (numeric
514 and string), and the various keywords, arithmetic operators and
515 punctuation marks. So the terminal symbols of a grammar for C include
516 `identifier', `number', `string', plus one symbol for each keyword,
517 operator or punctuation mark: `if', `return', `const', `static', `int',
518 `char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
519 (These tokens can be subdivided into characters, but that is a matter of
520 lexicography, not grammar.)
521
522 Here is a simple C function subdivided into tokens:
523
524 @ifinfo
525 @example
526 int /* @r{keyword `int'} */
527 square (int x) /* @r{identifier, open-paren, keyword `int',}
528 @r{identifier, close-paren} */
529 @{ /* @r{open-brace} */
530 return x * x; /* @r{keyword `return', identifier, asterisk,}
531 @r{identifier, semicolon} */
532 @} /* @r{close-brace} */
533 @end example
534 @end ifinfo
535 @ifnotinfo
536 @example
537 int /* @r{keyword `int'} */
538 square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
539 @{ /* @r{open-brace} */
540 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
541 @} /* @r{close-brace} */
542 @end example
543 @end ifnotinfo
544
545 The syntactic groupings of C include the expression, the statement, the
546 declaration, and the function definition. These are represented in the
547 grammar of C by nonterminal symbols `expression', `statement',
548 `declaration' and `function definition'. The full grammar uses dozens of
549 additional language constructs, each with its own nonterminal symbol, in
550 order to express the meanings of these four. The example above is a
551 function definition; it contains one declaration, and one statement. In
552 the statement, each @samp{x} is an expression and so is @samp{x * x}.
553
554 Each nonterminal symbol must have grammatical rules showing how it is made
555 out of simpler constructs. For example, one kind of C statement is the
556 @code{return} statement; this would be described with a grammar rule which
557 reads informally as follows:
558
559 @quotation
560 A `statement' can be made of a `return' keyword, an `expression' and a
561 `semicolon'.
562 @end quotation
563
564 @noindent
565 There would be many other rules for `statement', one for each kind of
566 statement in C.
567
568 @cindex start symbol
569 One nonterminal symbol must be distinguished as the special one which
570 defines a complete utterance in the language. It is called the @dfn{start
571 symbol}. In a compiler, this means a complete input program. In the C
572 language, the nonterminal symbol `sequence of definitions and declarations'
573 plays this role.
574
575 For example, @samp{1 + 2} is a valid C expression---a valid part of a C
576 program---but it is not valid as an @emph{entire} C program. In the
577 context-free grammar of C, this follows from the fact that `expression' is
578 not the start symbol.
579
580 The Bison parser reads a sequence of tokens as its input, and groups the
581 tokens using the grammar rules. If the input is valid, the end result is
582 that the entire token sequence reduces to a single grouping whose symbol is
583 the grammar's start symbol. If we use a grammar for C, the entire input
584 must be a `sequence of definitions and declarations'. If not, the parser
585 reports a syntax error.
586
587 @node Grammar in Bison
588 @section From Formal Rules to Bison Input
589 @cindex Bison grammar
590 @cindex grammar, Bison
591 @cindex formal grammar
592
593 A formal grammar is a mathematical construct. To define the language
594 for Bison, you must write a file expressing the grammar in Bison syntax:
595 a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
596
597 A nonterminal symbol in the formal grammar is represented in Bison input
598 as an identifier, like an identifier in C@. By convention, it should be
599 in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
600
601 The Bison representation for a terminal symbol is also called a @dfn{token
602 type}. Token types as well can be represented as C-like identifiers. By
603 convention, these identifiers should be upper case to distinguish them from
604 nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
605 @code{RETURN}. A terminal symbol that stands for a particular keyword in
606 the language should be named after that keyword converted to upper case.
607 The terminal symbol @code{error} is reserved for error recovery.
608 @xref{Symbols}.
609
610 A terminal symbol can also be represented as a character literal, just like
611 a C character constant. You should do this whenever a token is just a
612 single character (parenthesis, plus-sign, etc.): use that same character in
613 a literal as the terminal symbol for that token.
614
615 A third way to represent a terminal symbol is with a C string constant
616 containing several characters. @xref{Symbols}, for more information.
617
618 The grammar rules also have an expression in Bison syntax. For example,
619 here is the Bison rule for a C @code{return} statement. The semicolon in
620 quotes is a literal character token, representing part of the C syntax for
621 the statement; the naked semicolon, and the colon, are Bison punctuation
622 used in every rule.
623
624 @example
625 stmt: RETURN expr ';'
626 ;
627 @end example
628
629 @noindent
630 @xref{Rules, ,Syntax of Grammar Rules}.
631
632 @node Semantic Values
633 @section Semantic Values
634 @cindex semantic value
635 @cindex value, semantic
636
637 A formal grammar selects tokens only by their classifications: for example,
638 if a rule mentions the terminal symbol `integer constant', it means that
639 @emph{any} integer constant is grammatically valid in that position. The
640 precise value of the constant is irrelevant to how to parse the input: if
641 @samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
642 grammatical.
643
644 But the precise value is very important for what the input means once it is
645 parsed. A compiler is useless if it fails to distinguish between 4, 1 and
646 3989 as constants in the program! Therefore, each token in a Bison grammar
647 has both a token type and a @dfn{semantic value}. @xref{Semantics,
648 ,Defining Language Semantics},
649 for details.
650
651 The token type is a terminal symbol defined in the grammar, such as
652 @code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
653 you need to know to decide where the token may validly appear and how to
654 group it with other tokens. The grammar rules know nothing about tokens
655 except their types.
656
657 The semantic value has all the rest of the information about the
658 meaning of the token, such as the value of an integer, or the name of an
659 identifier. (A token such as @code{','} which is just punctuation doesn't
660 need to have any semantic value.)
661
662 For example, an input token might be classified as token type
663 @code{INTEGER} and have the semantic value 4. Another input token might
664 have the same token type @code{INTEGER} but value 3989. When a grammar
665 rule says that @code{INTEGER} is allowed, either of these tokens is
666 acceptable because each is an @code{INTEGER}. When the parser accepts the
667 token, it keeps track of the token's semantic value.
668
669 Each grouping can also have a semantic value as well as its nonterminal
670 symbol. For example, in a calculator, an expression typically has a
671 semantic value that is a number. In a compiler for a programming
672 language, an expression typically has a semantic value that is a tree
673 structure describing the meaning of the expression.
674
675 @node Semantic Actions
676 @section Semantic Actions
677 @cindex semantic actions
678 @cindex actions, semantic
679
680 In order to be useful, a program must do more than parse input; it must
681 also produce some output based on the input. In a Bison grammar, a grammar
682 rule can have an @dfn{action} made up of C statements. Each time the
683 parser recognizes a match for that rule, the action is executed.
684 @xref{Actions}.
685
686 Most of the time, the purpose of an action is to compute the semantic value
687 of the whole construct from the semantic values of its parts. For example,
688 suppose we have a rule which says an expression can be the sum of two
689 expressions. When the parser recognizes such a sum, each of the
690 subexpressions has a semantic value which describes how it was built up.
691 The action for this rule should create a similar sort of value for the
692 newly recognized larger expression.
693
694 For example, here is a rule that says an expression can be the sum of
695 two subexpressions:
696
697 @example
698 expr: expr '+' expr @{ $$ = $1 + $3; @}
699 ;
700 @end example
701
702 @noindent
703 The action says how to produce the semantic value of the sum expression
704 from the values of the two subexpressions.
705
706 @node GLR Parsers
707 @section Writing GLR Parsers
708 @cindex GLR parsing
709 @cindex generalized LR (GLR) parsing
710 @findex %glr-parser
711 @cindex conflicts
712 @cindex shift/reduce conflicts
713 @cindex reduce/reduce conflicts
714
715 In some grammars, Bison's deterministic
716 LR(1) parsing algorithm cannot decide whether to apply a
717 certain grammar rule at a given point. That is, it may not be able to
718 decide (on the basis of the input read so far) which of two possible
719 reductions (applications of a grammar rule) applies, or whether to apply
720 a reduction or read more of the input and apply a reduction later in the
721 input. These are known respectively as @dfn{reduce/reduce} conflicts
722 (@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
723 (@pxref{Shift/Reduce}).
724
725 To use a grammar that is not easily modified to be LR(1), a
726 more general parsing algorithm is sometimes necessary. If you include
727 @code{%glr-parser} among the Bison declarations in your file
728 (@pxref{Grammar Outline}), the result is a Generalized LR
729 (GLR) parser. These parsers handle Bison grammars that
730 contain no unresolved conflicts (i.e., after applying precedence
731 declarations) identically to deterministic parsers. However, when
732 faced with unresolved shift/reduce and reduce/reduce conflicts,
733 GLR parsers use the simple expedient of doing both,
734 effectively cloning the parser to follow both possibilities. Each of
735 the resulting parsers can again split, so that at any given time, there
736 can be any number of possible parses being explored. The parsers
737 proceed in lockstep; that is, all of them consume (shift) a given input
738 symbol before any of them proceed to the next. Each of the cloned
739 parsers eventually meets one of two possible fates: either it runs into
740 a parsing error, in which case it simply vanishes, or it merges with
741 another parser, because the two of them have reduced the input to an
742 identical set of symbols.
743
744 During the time that there are multiple parsers, semantic actions are
745 recorded, but not performed. When a parser disappears, its recorded
746 semantic actions disappear as well, and are never performed. When a
747 reduction makes two parsers identical, causing them to merge, Bison
748 records both sets of semantic actions. Whenever the last two parsers
749 merge, reverting to the single-parser case, Bison resolves all the
750 outstanding actions either by precedences given to the grammar rules
751 involved, or by performing both actions, and then calling a designated
752 user-defined function on the resulting values to produce an arbitrary
753 merged result.
754
755 @menu
756 * Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
757 * Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
758 * GLR Semantic Actions:: Deferred semantic actions have special concerns.
759 * Compiler Requirements:: GLR parsers require a modern C compiler.
760 @end menu
761
762 @node Simple GLR Parsers
763 @subsection Using GLR on Unambiguous Grammars
764 @cindex GLR parsing, unambiguous grammars
765 @cindex generalized LR (GLR) parsing, unambiguous grammars
766 @findex %glr-parser
767 @findex %expect-rr
768 @cindex conflicts
769 @cindex reduce/reduce conflicts
770 @cindex shift/reduce conflicts
771
772 In the simplest cases, you can use the GLR algorithm
773 to parse grammars that are unambiguous but fail to be LR(1).
774 Such grammars typically require more than one symbol of lookahead.
775
776 Consider a problem that
777 arises in the declaration of enumerated and subrange types in the
778 programming language Pascal. Here are some examples:
779
780 @example
781 type subrange = lo .. hi;
782 type enum = (a, b, c);
783 @end example
784
785 @noindent
786 The original language standard allows only numeric
787 literals and constant identifiers for the subrange bounds (@samp{lo}
788 and @samp{hi}), but Extended Pascal (ISO/IEC
789 10206) and many other
790 Pascal implementations allow arbitrary expressions there. This gives
791 rise to the following situation, containing a superfluous pair of
792 parentheses:
793
794 @example
795 type subrange = (a) .. b;
796 @end example
797
798 @noindent
799 Compare this to the following declaration of an enumerated
800 type with only one value:
801
802 @example
803 type enum = (a);
804 @end example
805
806 @noindent
807 (These declarations are contrived, but they are syntactically
808 valid, and more-complicated cases can come up in practical programs.)
809
810 These two declarations look identical until the @samp{..} token.
811 With normal LR(1) one-token lookahead it is not
812 possible to decide between the two forms when the identifier
813 @samp{a} is parsed. It is, however, desirable
814 for a parser to decide this, since in the latter case
815 @samp{a} must become a new identifier to represent the enumeration
816 value, while in the former case @samp{a} must be evaluated with its
817 current meaning, which may be a constant or even a function call.
818
819 You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
820 to be resolved later, but this typically requires substantial
821 contortions in both semantic actions and large parts of the
822 grammar, where the parentheses are nested in the recursive rules for
823 expressions.
824
825 You might think of using the lexer to distinguish between the two
826 forms by returning different tokens for currently defined and
827 undefined identifiers. But if these declarations occur in a local
828 scope, and @samp{a} is defined in an outer scope, then both forms
829 are possible---either locally redefining @samp{a}, or using the
830 value of @samp{a} from the outer scope. So this approach cannot
831 work.
832
833 A simple solution to this problem is to declare the parser to
834 use the GLR algorithm.
835 When the GLR parser reaches the critical state, it
836 merely splits into two branches and pursues both syntax rules
837 simultaneously. Sooner or later, one of them runs into a parsing
838 error. If there is a @samp{..} token before the next
839 @samp{;}, the rule for enumerated types fails since it cannot
840 accept @samp{..} anywhere; otherwise, the subrange type rule
841 fails since it requires a @samp{..} token. So one of the branches
842 fails silently, and the other one continues normally, performing
843 all the intermediate actions that were postponed during the split.
844
845 If the input is syntactically incorrect, both branches fail and the parser
846 reports a syntax error as usual.
847
848 The effect of all this is that the parser seems to ``guess'' the
849 correct branch to take, or in other words, it seems to use more
850 lookahead than the underlying LR(1) algorithm actually allows
851 for. In this example, LR(2) would suffice, but also some cases
852 that are not LR(@math{k}) for any @math{k} can be handled this way.
853
854 In general, a GLR parser can take quadratic or cubic worst-case time,
855 and the current Bison parser even takes exponential time and space
856 for some grammars. In practice, this rarely happens, and for many
857 grammars it is possible to prove that it cannot happen.
858 The present example contains only one conflict between two
859 rules, and the type-declaration context containing the conflict
860 cannot be nested. So the number of
861 branches that can exist at any time is limited by the constant 2,
862 and the parsing time is still linear.
863
864 Here is a Bison grammar corresponding to the example above. It
865 parses a vastly simplified form of Pascal type declarations.
866
867 @example
868 %token TYPE DOTDOT ID
869
870 @group
871 %left '+' '-'
872 %left '*' '/'
873 @end group
874
875 %%
876
877 @group
878 type_decl : TYPE ID '=' type ';'
879 ;
880 @end group
881
882 @group
883 type : '(' id_list ')'
884 | expr DOTDOT expr
885 ;
886 @end group
887
888 @group
889 id_list : ID
890 | id_list ',' ID
891 ;
892 @end group
893
894 @group
895 expr : '(' expr ')'
896 | expr '+' expr
897 | expr '-' expr
898 | expr '*' expr
899 | expr '/' expr
900 | ID
901 ;
902 @end group
903 @end example
904
905 When used as a normal LR(1) grammar, Bison correctly complains
906 about one reduce/reduce conflict. In the conflicting situation the
907 parser chooses one of the alternatives, arbitrarily the one
908 declared first. Therefore the following correct input is not
909 recognized:
910
911 @example
912 type t = (a) .. b;
913 @end example
914
915 The parser can be turned into a GLR parser, while also telling Bison
916 to be silent about the one known reduce/reduce conflict, by
917 adding these two declarations to the Bison input file (before the first
918 @samp{%%}):
919
920 @example
921 %glr-parser
922 %expect-rr 1
923 @end example
924
925 @noindent
926 No change in the grammar itself is required. Now the
927 parser recognizes all valid declarations, according to the
928 limited syntax above, transparently. In fact, the user does not even
929 notice when the parser splits.
930
931 So here we have a case where we can use the benefits of GLR,
932 almost without disadvantages. Even in simple cases like this, however,
933 there are at least two potential problems to beware. First, always
934 analyze the conflicts reported by Bison to make sure that GLR
935 splitting is only done where it is intended. A GLR parser
936 splitting inadvertently may cause problems less obvious than an
937 LR parser statically choosing the wrong alternative in a
938 conflict. Second, consider interactions with the lexer (@pxref{Semantic
939 Tokens}) with great care. Since a split parser consumes tokens without
940 performing any actions during the split, the lexer cannot obtain
941 information via parser actions. Some cases of lexer interactions can be
942 eliminated by using GLR to shift the complications from the
943 lexer to the parser. You must check the remaining cases for
944 correctness.
945
946 In our example, it would be safe for the lexer to return tokens based on
947 their current meanings in some symbol table, because no new symbols are
948 defined in the middle of a type declaration. Though it is possible for
949 a parser to define the enumeration constants as they are parsed, before
950 the type declaration is completed, it actually makes no difference since
951 they cannot be used within the same enumerated type declaration.
952
953 @node Merging GLR Parses
954 @subsection Using GLR to Resolve Ambiguities
955 @cindex GLR parsing, ambiguous grammars
956 @cindex generalized LR (GLR) parsing, ambiguous grammars
957 @findex %dprec
958 @findex %merge
959 @cindex conflicts
960 @cindex reduce/reduce conflicts
961
962 Let's consider an example, vastly simplified from a C++ grammar.
963
964 @example
965 %@{
966 #include <stdio.h>
967 #define YYSTYPE char const *
968 int yylex (void);
969 void yyerror (char const *);
970 %@}
971
972 %token TYPENAME ID
973
974 %right '='
975 %left '+'
976
977 %glr-parser
978
979 %%
980
981 prog :
982 | prog stmt @{ printf ("\n"); @}
983 ;
984
985 stmt : expr ';' %dprec 1
986 | decl %dprec 2
987 ;
988
989 expr : ID @{ printf ("%s ", $$); @}
990 | TYPENAME '(' expr ')'
991 @{ printf ("%s <cast> ", $1); @}
992 | expr '+' expr @{ printf ("+ "); @}
993 | expr '=' expr @{ printf ("= "); @}
994 ;
995
996 decl : TYPENAME declarator ';'
997 @{ printf ("%s <declare> ", $1); @}
998 | TYPENAME declarator '=' expr ';'
999 @{ printf ("%s <init-declare> ", $1); @}
1000 ;
1001
1002 declarator : ID @{ printf ("\"%s\" ", $1); @}
1003 | '(' declarator ')'
1004 ;
1005 @end example
1006
1007 @noindent
1008 This models a problematic part of the C++ grammar---the ambiguity between
1009 certain declarations and statements. For example,
1010
1011 @example
1012 T (x) = y+z;
1013 @end example
1014
1015 @noindent
1016 parses as either an @code{expr} or a @code{stmt}
1017 (assuming that @samp{T} is recognized as a @code{TYPENAME} and
1018 @samp{x} as an @code{ID}).
1019 Bison detects this as a reduce/reduce conflict between the rules
1020 @code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
1021 time it encounters @code{x} in the example above. Since this is a
1022 GLR parser, it therefore splits the problem into two parses, one for
1023 each choice of resolving the reduce/reduce conflict.
1024 Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1025 however, neither of these parses ``dies,'' because the grammar as it stands is
1026 ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1027 the other reduces @code{stmt : decl}, after which both parsers are in an
1028 identical state: they've seen @samp{prog stmt} and have the same unprocessed
1029 input remaining. We say that these parses have @dfn{merged.}
1030
1031 At this point, the GLR parser requires a specification in the
1032 grammar of how to choose between the competing parses.
1033 In the example above, the two @code{%dprec}
1034 declarations specify that Bison is to give precedence
1035 to the parse that interprets the example as a
1036 @code{decl}, which implies that @code{x} is a declarator.
1037 The parser therefore prints
1038
1039 @example
1040 "x" y z + T <init-declare>
1041 @end example
1042
1043 The @code{%dprec} declarations only come into play when more than one
1044 parse survives. Consider a different input string for this parser:
1045
1046 @example
1047 T (x) + y;
1048 @end example
1049
1050 @noindent
1051 This is another example of using GLR to parse an unambiguous
1052 construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
1053 Here, there is no ambiguity (this cannot be parsed as a declaration).
1054 However, at the time the Bison parser encounters @code{x}, it does not
1055 have enough information to resolve the reduce/reduce conflict (again,
1056 between @code{x} as an @code{expr} or a @code{declarator}). In this
1057 case, no precedence declaration is used. Again, the parser splits
1058 into two, one assuming that @code{x} is an @code{expr}, and the other
1059 assuming @code{x} is a @code{declarator}. The second of these parsers
1060 then vanishes when it sees @code{+}, and the parser prints
1061
1062 @example
1063 x T <cast> y +
1064 @end example
1065
1066 Suppose that instead of resolving the ambiguity, you wanted to see all
1067 the possibilities. For this purpose, you must merge the semantic
1068 actions of the two possible parsers, rather than choosing one over the
1069 other. To do so, you could change the declaration of @code{stmt} as
1070 follows:
1071
1072 @example
1073 stmt : expr ';' %merge <stmtMerge>
1074 | decl %merge <stmtMerge>
1075 ;
1076 @end example
1077
1078 @noindent
1079 and define the @code{stmtMerge} function as:
1080
1081 @example
1082 static YYSTYPE
1083 stmtMerge (YYSTYPE x0, YYSTYPE x1)
1084 @{
1085 printf ("<OR> ");
1086 return "";
1087 @}
1088 @end example
1089
1090 @noindent
1091 with an accompanying forward declaration
1092 in the C declarations at the beginning of the file:
1093
1094 @example
1095 %@{
1096 #define YYSTYPE char const *
1097 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1098 %@}
1099 @end example
1100
1101 @noindent
1102 With these declarations, the resulting parser parses the first example
1103 as both an @code{expr} and a @code{decl}, and prints
1104
1105 @example
1106 "x" y z + T <init-declare> x T <cast> y z + = <OR>
1107 @end example
1108
1109 Bison requires that all of the
1110 productions that participate in any particular merge have identical
1111 @samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1112 and the parser will report an error during any parse that results in
1113 the offending merge.
1114
1115 @node GLR Semantic Actions
1116 @subsection GLR Semantic Actions
1117
1118 @cindex deferred semantic actions
1119 By definition, a deferred semantic action is not performed at the same time as
1120 the associated reduction.
1121 This raises caveats for several Bison features you might use in a semantic
1122 action in a GLR parser.
1123
1124 @vindex yychar
1125 @cindex GLR parsers and @code{yychar}
1126 @vindex yylval
1127 @cindex GLR parsers and @code{yylval}
1128 @vindex yylloc
1129 @cindex GLR parsers and @code{yylloc}
1130 In any semantic action, you can examine @code{yychar} to determine the type of
1131 the lookahead token present at the time of the associated reduction.
1132 After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1133 you can then examine @code{yylval} and @code{yylloc} to determine the
1134 lookahead token's semantic value and location, if any.
1135 In a nondeferred semantic action, you can also modify any of these variables to
1136 influence syntax analysis.
1137 @xref{Lookahead, ,Lookahead Tokens}.
1138
1139 @findex yyclearin
1140 @cindex GLR parsers and @code{yyclearin}
1141 In a deferred semantic action, it's too late to influence syntax analysis.
1142 In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1143 shallow copies of the values they had at the time of the associated reduction.
1144 For this reason alone, modifying them is dangerous.
1145 Moreover, the result of modifying them is undefined and subject to change with
1146 future versions of Bison.
1147 For example, if a semantic action might be deferred, you should never write it
1148 to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1149 memory referenced by @code{yylval}.
1150
1151 @findex YYERROR
1152 @cindex GLR parsers and @code{YYERROR}
1153 Another Bison feature requiring special consideration is @code{YYERROR}
1154 (@pxref{Action Features}), which you can invoke in a semantic action to
1155 initiate error recovery.
1156 During deterministic GLR operation, the effect of @code{YYERROR} is
1157 the same as its effect in a deterministic parser.
1158 In a deferred semantic action, its effect is undefined.
1159 @c The effect is probably a syntax error at the split point.
1160
1161 Also, see @ref{Location Default Action, ,Default Action for Locations}, which
1162 describes a special usage of @code{YYLLOC_DEFAULT} in GLR parsers.
1163
1164 @node Compiler Requirements
1165 @subsection Considerations when Compiling GLR Parsers
1166 @cindex @code{inline}
1167 @cindex GLR parsers and @code{inline}
1168
1169 The GLR parsers require a compiler for ISO C89 or
1170 later. In addition, they use the @code{inline} keyword, which is not
1171 C89, but is C99 and is a common extension in pre-C99 compilers. It is
1172 up to the user of these parsers to handle
1173 portability issues. For instance, if using Autoconf and the Autoconf
1174 macro @code{AC_C_INLINE}, a mere
1175
1176 @example
1177 %@{
1178 #include <config.h>
1179 %@}
1180 @end example
1181
1182 @noindent
1183 will suffice. Otherwise, we suggest
1184
1185 @example
1186 %@{
1187 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1188 #define inline
1189 #endif
1190 %@}
1191 @end example
1192
1193 @node Locations Overview
1194 @section Locations
1195 @cindex location
1196 @cindex textual location
1197 @cindex location, textual
1198
1199 Many applications, like interpreters or compilers, have to produce verbose
1200 and useful error messages. To achieve this, one must be able to keep track of
1201 the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
1202 Bison provides a mechanism for handling these locations.
1203
1204 Each token has a semantic value. In a similar fashion, each token has an
1205 associated location, but the type of locations is the same for all tokens and
1206 groupings. Moreover, the output parser is equipped with a default data
1207 structure for storing locations (@pxref{Locations}, for more details).
1208
1209 Like semantic values, locations can be reached in actions using a dedicated
1210 set of constructs. In the example above, the location of the whole grouping
1211 is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1212 @code{@@3}.
1213
1214 When a rule is matched, a default action is used to compute the semantic value
1215 of its left hand side (@pxref{Actions}). In the same way, another default
1216 action is used for locations. However, the action for locations is general
1217 enough for most cases, meaning there is usually no need to describe for each
1218 rule how @code{@@$} should be formed. When building a new location for a given
1219 grouping, the default behavior of the output parser is to take the beginning
1220 of the first symbol, and the end of the last symbol.
1221
1222 @node Bison Parser
1223 @section Bison Output: the Parser File
1224 @cindex Bison parser
1225 @cindex Bison utility
1226 @cindex lexical analyzer, purpose
1227 @cindex parser
1228
1229 When you run Bison, you give it a Bison grammar file as input. The output
1230 is a C source file that parses the language described by the grammar.
1231 This file is called a @dfn{Bison parser}. Keep in mind that the Bison
1232 utility and the Bison parser are two distinct programs: the Bison utility
1233 is a program whose output is the Bison parser that becomes part of your
1234 program.
1235
1236 The job of the Bison parser is to group tokens into groupings according to
1237 the grammar rules---for example, to build identifiers and operators into
1238 expressions. As it does this, it runs the actions for the grammar rules it
1239 uses.
1240
1241 The tokens come from a function called the @dfn{lexical analyzer} that
1242 you must supply in some fashion (such as by writing it in C). The Bison
1243 parser calls the lexical analyzer each time it wants a new token. It
1244 doesn't know what is ``inside'' the tokens (though their semantic values
1245 may reflect this). Typically the lexical analyzer makes the tokens by
1246 parsing characters of text, but Bison does not depend on this.
1247 @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
1248
1249 The Bison parser file is C code which defines a function named
1250 @code{yyparse} which implements that grammar. This function does not make
1251 a complete C program: you must supply some additional functions. One is
1252 the lexical analyzer. Another is an error-reporting function which the
1253 parser calls to report an error. In addition, a complete C program must
1254 start with a function called @code{main}; you have to provide this, and
1255 arrange for it to call @code{yyparse} or the parser will never run.
1256 @xref{Interface, ,Parser C-Language Interface}.
1257
1258 Aside from the token type names and the symbols in the actions you
1259 write, all symbols defined in the Bison parser file itself
1260 begin with @samp{yy} or @samp{YY}. This includes interface functions
1261 such as the lexical analyzer function @code{yylex}, the error reporting
1262 function @code{yyerror} and the parser function @code{yyparse} itself.
1263 This also includes numerous identifiers used for internal purposes.
1264 Therefore, you should avoid using C identifiers starting with @samp{yy}
1265 or @samp{YY} in the Bison grammar file except for the ones defined in
1266 this manual. Also, you should avoid using the C identifiers
1267 @samp{malloc} and @samp{free} for anything other than their usual
1268 meanings.
1269
1270 In some cases the Bison parser file includes system headers, and in
1271 those cases your code should respect the identifiers reserved by those
1272 headers. On some non-GNU hosts, @code{<alloca.h>}, @code{<malloc.h>},
1273 @code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
1274 declare memory allocators and related types. @code{<libintl.h>} is
1275 included if message translation is in use
1276 (@pxref{Internationalization}). Other system headers may
1277 be included if you define @code{YYDEBUG} to a nonzero value
1278 (@pxref{Tracing, ,Tracing Your Parser}).
1279
1280 @node Stages
1281 @section Stages in Using Bison
1282 @cindex stages in using Bison
1283 @cindex using Bison
1284
1285 The actual language-design process using Bison, from grammar specification
1286 to a working compiler or interpreter, has these parts:
1287
1288 @enumerate
1289 @item
1290 Formally specify the grammar in a form recognized by Bison
1291 (@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1292 in the language, describe the action that is to be taken when an
1293 instance of that rule is recognized. The action is described by a
1294 sequence of C statements.
1295
1296 @item
1297 Write a lexical analyzer to process input and pass tokens to the parser.
1298 The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1299 Lexical Analyzer Function @code{yylex}}). It could also be produced
1300 using Lex, but the use of Lex is not discussed in this manual.
1301
1302 @item
1303 Write a controlling function that calls the Bison-produced parser.
1304
1305 @item
1306 Write error-reporting routines.
1307 @end enumerate
1308
1309 To turn this source code as written into a runnable program, you
1310 must follow these steps:
1311
1312 @enumerate
1313 @item
1314 Run Bison on the grammar to produce the parser.
1315
1316 @item
1317 Compile the code output by Bison, as well as any other source files.
1318
1319 @item
1320 Link the object files to produce the finished product.
1321 @end enumerate
1322
1323 @node Grammar Layout
1324 @section The Overall Layout of a Bison Grammar
1325 @cindex grammar file
1326 @cindex file format
1327 @cindex format of grammar file
1328 @cindex layout of Bison grammar
1329
1330 The input file for the Bison utility is a @dfn{Bison grammar file}. The
1331 general form of a Bison grammar file is as follows:
1332
1333 @example
1334 %@{
1335 @var{Prologue}
1336 %@}
1337
1338 @var{Bison declarations}
1339
1340 %%
1341 @var{Grammar rules}
1342 %%
1343 @var{Epilogue}
1344 @end example
1345
1346 @noindent
1347 The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1348 in every Bison grammar file to separate the sections.
1349
1350 The prologue may define types and variables used in the actions. You can
1351 also use preprocessor commands to define macros used there, and use
1352 @code{#include} to include header files that do any of these things.
1353 You need to declare the lexical analyzer @code{yylex} and the error
1354 printer @code{yyerror} here, along with any other global identifiers
1355 used by the actions in the grammar rules.
1356
1357 The Bison declarations declare the names of the terminal and nonterminal
1358 symbols, and may also describe operator precedence and the data types of
1359 semantic values of various symbols.
1360
1361 The grammar rules define how to construct each nonterminal symbol from its
1362 parts.
1363
1364 The epilogue can contain any code you want to use. Often the
1365 definitions of functions declared in the prologue go here. In a
1366 simple program, all the rest of the program can go here.
1367
1368 @node Examples
1369 @chapter Examples
1370 @cindex simple examples
1371 @cindex examples, simple
1372
1373 Now we show and explain three sample programs written using Bison: a
1374 reverse polish notation calculator, an algebraic (infix) notation
1375 calculator, and a multi-function calculator. All three have been tested
1376 under BSD Unix 4.3; each produces a usable, though limited, interactive
1377 desk-top calculator.
1378
1379 These examples are simple, but Bison grammars for real programming
1380 languages are written the same way. You can copy these examples into a
1381 source file to try them.
1382
1383 @menu
1384 * RPN Calc:: Reverse polish notation calculator;
1385 a first example with no operator precedence.
1386 * Infix Calc:: Infix (algebraic) notation calculator.
1387 Operator precedence is introduced.
1388 * Simple Error Recovery:: Continuing after syntax errors.
1389 * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
1390 * Multi-function Calc:: Calculator with memory and trig functions.
1391 It uses multiple data-types for semantic values.
1392 * Exercises:: Ideas for improving the multi-function calculator.
1393 @end menu
1394
1395 @node RPN Calc
1396 @section Reverse Polish Notation Calculator
1397 @cindex reverse polish notation
1398 @cindex polish notation calculator
1399 @cindex @code{rpcalc}
1400 @cindex calculator, simple
1401
1402 The first example is that of a simple double-precision @dfn{reverse polish
1403 notation} calculator (a calculator using postfix operators). This example
1404 provides a good starting point, since operator precedence is not an issue.
1405 The second example will illustrate how operator precedence is handled.
1406
1407 The source code for this calculator is named @file{rpcalc.y}. The
1408 @samp{.y} extension is a convention used for Bison input files.
1409
1410 @menu
1411 * Rpcalc Declarations:: Prologue (declarations) for rpcalc.
1412 * Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
1413 * Rpcalc Lexer:: The lexical analyzer.
1414 * Rpcalc Main:: The controlling function.
1415 * Rpcalc Error:: The error reporting function.
1416 * Rpcalc Generate:: Running Bison on the grammar file.
1417 * Rpcalc Compile:: Run the C compiler on the output code.
1418 @end menu
1419
1420 @node Rpcalc Declarations
1421 @subsection Declarations for @code{rpcalc}
1422
1423 Here are the C and Bison declarations for the reverse polish notation
1424 calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1425
1426 @example
1427 /* Reverse polish notation calculator. */
1428
1429 %@{
1430 #define YYSTYPE double
1431 #include <math.h>
1432 int yylex (void);
1433 void yyerror (char const *);
1434 %@}
1435
1436 %token NUM
1437
1438 %% /* Grammar rules and actions follow. */
1439 @end example
1440
1441 The declarations section (@pxref{Prologue, , The prologue}) contains two
1442 preprocessor directives and two forward declarations.
1443
1444 The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1445 specifying the C data type for semantic values of both tokens and
1446 groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1447 Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1448 don't define it, @code{int} is the default. Because we specify
1449 @code{double}, each token and each expression has an associated value,
1450 which is a floating point number.
1451
1452 The @code{#include} directive is used to declare the exponentiation
1453 function @code{pow}.
1454
1455 The forward declarations for @code{yylex} and @code{yyerror} are
1456 needed because the C language requires that functions be declared
1457 before they are used. These functions will be defined in the
1458 epilogue, but the parser calls them so they must be declared in the
1459 prologue.
1460
1461 The second section, Bison declarations, provides information to Bison
1462 about the token types (@pxref{Bison Declarations, ,The Bison
1463 Declarations Section}). Each terminal symbol that is not a
1464 single-character literal must be declared here. (Single-character
1465 literals normally don't need to be declared.) In this example, all the
1466 arithmetic operators are designated by single-character literals, so the
1467 only terminal symbol that needs to be declared is @code{NUM}, the token
1468 type for numeric constants.
1469
1470 @node Rpcalc Rules
1471 @subsection Grammar Rules for @code{rpcalc}
1472
1473 Here are the grammar rules for the reverse polish notation calculator.
1474
1475 @example
1476 input: /* empty */
1477 | input line
1478 ;
1479
1480 line: '\n'
1481 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1482 ;
1483
1484 exp: NUM @{ $$ = $1; @}
1485 | exp exp '+' @{ $$ = $1 + $2; @}
1486 | exp exp '-' @{ $$ = $1 - $2; @}
1487 | exp exp '*' @{ $$ = $1 * $2; @}
1488 | exp exp '/' @{ $$ = $1 / $2; @}
1489 /* Exponentiation */
1490 | exp exp '^' @{ $$ = pow ($1, $2); @}
1491 /* Unary minus */
1492 | exp 'n' @{ $$ = -$1; @}
1493 ;
1494 %%
1495 @end example
1496
1497 The groupings of the rpcalc ``language'' defined here are the expression
1498 (given the name @code{exp}), the line of input (@code{line}), and the
1499 complete input transcript (@code{input}). Each of these nonterminal
1500 symbols has several alternate rules, joined by the vertical bar @samp{|}
1501 which is read as ``or''. The following sections explain what these rules
1502 mean.
1503
1504 The semantics of the language is determined by the actions taken when a
1505 grouping is recognized. The actions are the C code that appears inside
1506 braces. @xref{Actions}.
1507
1508 You must specify these actions in C, but Bison provides the means for
1509 passing semantic values between the rules. In each action, the
1510 pseudo-variable @code{$$} stands for the semantic value for the grouping
1511 that the rule is going to construct. Assigning a value to @code{$$} is the
1512 main job of most actions. The semantic values of the components of the
1513 rule are referred to as @code{$1}, @code{$2}, and so on.
1514
1515 @menu
1516 * Rpcalc Input::
1517 * Rpcalc Line::
1518 * Rpcalc Expr::
1519 @end menu
1520
1521 @node Rpcalc Input
1522 @subsubsection Explanation of @code{input}
1523
1524 Consider the definition of @code{input}:
1525
1526 @example
1527 input: /* empty */
1528 | input line
1529 ;
1530 @end example
1531
1532 This definition reads as follows: ``A complete input is either an empty
1533 string, or a complete input followed by an input line''. Notice that
1534 ``complete input'' is defined in terms of itself. This definition is said
1535 to be @dfn{left recursive} since @code{input} appears always as the
1536 leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1537
1538 The first alternative is empty because there are no symbols between the
1539 colon and the first @samp{|}; this means that @code{input} can match an
1540 empty string of input (no tokens). We write the rules this way because it
1541 is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1542 It's conventional to put an empty alternative first and write the comment
1543 @samp{/* empty */} in it.
1544
1545 The second alternate rule (@code{input line}) handles all nontrivial input.
1546 It means, ``After reading any number of lines, read one more line if
1547 possible.'' The left recursion makes this rule into a loop. Since the
1548 first alternative matches empty input, the loop can be executed zero or
1549 more times.
1550
1551 The parser function @code{yyparse} continues to process input until a
1552 grammatical error is seen or the lexical analyzer says there are no more
1553 input tokens; we will arrange for the latter to happen at end-of-input.
1554
1555 @node Rpcalc Line
1556 @subsubsection Explanation of @code{line}
1557
1558 Now consider the definition of @code{line}:
1559
1560 @example
1561 line: '\n'
1562 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1563 ;
1564 @end example
1565
1566 The first alternative is a token which is a newline character; this means
1567 that rpcalc accepts a blank line (and ignores it, since there is no
1568 action). The second alternative is an expression followed by a newline.
1569 This is the alternative that makes rpcalc useful. The semantic value of
1570 the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1571 question is the first symbol in the alternative. The action prints this
1572 value, which is the result of the computation the user asked for.
1573
1574 This action is unusual because it does not assign a value to @code{$$}. As
1575 a consequence, the semantic value associated with the @code{line} is
1576 uninitialized (its value will be unpredictable). This would be a bug if
1577 that value were ever used, but we don't use it: once rpcalc has printed the
1578 value of the user's input line, that value is no longer needed.
1579
1580 @node Rpcalc Expr
1581 @subsubsection Explanation of @code{expr}
1582
1583 The @code{exp} grouping has several rules, one for each kind of expression.
1584 The first rule handles the simplest expressions: those that are just numbers.
1585 The second handles an addition-expression, which looks like two expressions
1586 followed by a plus-sign. The third handles subtraction, and so on.
1587
1588 @example
1589 exp: NUM
1590 | exp exp '+' @{ $$ = $1 + $2; @}
1591 | exp exp '-' @{ $$ = $1 - $2; @}
1592 @dots{}
1593 ;
1594 @end example
1595
1596 We have used @samp{|} to join all the rules for @code{exp}, but we could
1597 equally well have written them separately:
1598
1599 @example
1600 exp: NUM ;
1601 exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1602 exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1603 @dots{}
1604 @end example
1605
1606 Most of the rules have actions that compute the value of the expression in
1607 terms of the value of its parts. For example, in the rule for addition,
1608 @code{$1} refers to the first component @code{exp} and @code{$2} refers to
1609 the second one. The third component, @code{'+'}, has no meaningful
1610 associated semantic value, but if it had one you could refer to it as
1611 @code{$3}. When @code{yyparse} recognizes a sum expression using this
1612 rule, the sum of the two subexpressions' values is produced as the value of
1613 the entire expression. @xref{Actions}.
1614
1615 You don't have to give an action for every rule. When a rule has no
1616 action, Bison by default copies the value of @code{$1} into @code{$$}.
1617 This is what happens in the first rule (the one that uses @code{NUM}).
1618
1619 The formatting shown here is the recommended convention, but Bison does
1620 not require it. You can add or change white space as much as you wish.
1621 For example, this:
1622
1623 @example
1624 exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
1625 @end example
1626
1627 @noindent
1628 means the same thing as this:
1629
1630 @example
1631 exp: NUM
1632 | exp exp '+' @{ $$ = $1 + $2; @}
1633 | @dots{}
1634 ;
1635 @end example
1636
1637 @noindent
1638 The latter, however, is much more readable.
1639
1640 @node Rpcalc Lexer
1641 @subsection The @code{rpcalc} Lexical Analyzer
1642 @cindex writing a lexical analyzer
1643 @cindex lexical analyzer, writing
1644
1645 The lexical analyzer's job is low-level parsing: converting characters
1646 or sequences of characters into tokens. The Bison parser gets its
1647 tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1648 Analyzer Function @code{yylex}}.
1649
1650 Only a simple lexical analyzer is needed for the RPN
1651 calculator. This
1652 lexical analyzer skips blanks and tabs, then reads in numbers as
1653 @code{double} and returns them as @code{NUM} tokens. Any other character
1654 that isn't part of a number is a separate token. Note that the token-code
1655 for such a single-character token is the character itself.
1656
1657 The return value of the lexical analyzer function is a numeric code which
1658 represents a token type. The same text used in Bison rules to stand for
1659 this token type is also a C expression for the numeric code for the type.
1660 This works in two ways. If the token type is a character literal, then its
1661 numeric code is that of the character; you can use the same
1662 character literal in the lexical analyzer to express the number. If the
1663 token type is an identifier, that identifier is defined by Bison as a C
1664 macro whose definition is the appropriate number. In this example,
1665 therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1666
1667 The semantic value of the token (if it has one) is stored into the
1668 global variable @code{yylval}, which is where the Bison parser will look
1669 for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1670 defined at the beginning of the grammar; @pxref{Rpcalc Declarations,
1671 ,Declarations for @code{rpcalc}}.)
1672
1673 A token type code of zero is returned if the end-of-input is encountered.
1674 (Bison recognizes any nonpositive value as indicating end-of-input.)
1675
1676 Here is the code for the lexical analyzer:
1677
1678 @example
1679 @group
1680 /* The lexical analyzer returns a double floating point
1681 number on the stack and the token NUM, or the numeric code
1682 of the character read if not a number. It skips all blanks
1683 and tabs, and returns 0 for end-of-input. */
1684
1685 #include <ctype.h>
1686 @end group
1687
1688 @group
1689 int
1690 yylex (void)
1691 @{
1692 int c;
1693
1694 /* Skip white space. */
1695 while ((c = getchar ()) == ' ' || c == '\t')
1696 ;
1697 @end group
1698 @group
1699 /* Process numbers. */
1700 if (c == '.' || isdigit (c))
1701 @{
1702 ungetc (c, stdin);
1703 scanf ("%lf", &yylval);
1704 return NUM;
1705 @}
1706 @end group
1707 @group
1708 /* Return end-of-input. */
1709 if (c == EOF)
1710 return 0;
1711 /* Return a single char. */
1712 return c;
1713 @}
1714 @end group
1715 @end example
1716
1717 @node Rpcalc Main
1718 @subsection The Controlling Function
1719 @cindex controlling function
1720 @cindex main function in simple example
1721
1722 In keeping with the spirit of this example, the controlling function is
1723 kept to the bare minimum. The only requirement is that it call
1724 @code{yyparse} to start the process of parsing.
1725
1726 @example
1727 @group
1728 int
1729 main (void)
1730 @{
1731 return yyparse ();
1732 @}
1733 @end group
1734 @end example
1735
1736 @node Rpcalc Error
1737 @subsection The Error Reporting Routine
1738 @cindex error reporting routine
1739
1740 When @code{yyparse} detects a syntax error, it calls the error reporting
1741 function @code{yyerror} to print an error message (usually but not
1742 always @code{"syntax error"}). It is up to the programmer to supply
1743 @code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1744 here is the definition we will use:
1745
1746 @example
1747 @group
1748 #include <stdio.h>
1749
1750 /* Called by yyparse on error. */
1751 void
1752 yyerror (char const *s)
1753 @{
1754 fprintf (stderr, "%s\n", s);
1755 @}
1756 @end group
1757 @end example
1758
1759 After @code{yyerror} returns, the Bison parser may recover from the error
1760 and continue parsing if the grammar contains a suitable error rule
1761 (@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1762 have not written any error rules in this example, so any invalid input will
1763 cause the calculator program to exit. This is not clean behavior for a
1764 real calculator, but it is adequate for the first example.
1765
1766 @node Rpcalc Generate
1767 @subsection Running Bison to Make the Parser
1768 @cindex running Bison (introduction)
1769
1770 Before running Bison to produce a parser, we need to decide how to
1771 arrange all the source code in one or more source files. For such a
1772 simple example, the easiest thing is to put everything in one file. The
1773 definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
1774 end, in the epilogue of the file
1775 (@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
1776
1777 For a large project, you would probably have several source files, and use
1778 @code{make} to arrange to recompile them.
1779
1780 With all the source in a single file, you use the following command to
1781 convert it into a parser file:
1782
1783 @example
1784 bison @var{file}.y
1785 @end example
1786
1787 @noindent
1788 In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
1789 @sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c},
1790 removing the @samp{.y} from the original file name. The file output by
1791 Bison contains the source code for @code{yyparse}. The additional
1792 functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
1793 are copied verbatim to the output.
1794
1795 @node Rpcalc Compile
1796 @subsection Compiling the Parser File
1797 @cindex compiling the parser
1798
1799 Here is how to compile and run the parser file:
1800
1801 @example
1802 @group
1803 # @r{List files in current directory.}
1804 $ @kbd{ls}
1805 rpcalc.tab.c rpcalc.y
1806 @end group
1807
1808 @group
1809 # @r{Compile the Bison parser.}
1810 # @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
1811 $ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
1812 @end group
1813
1814 @group
1815 # @r{List files again.}
1816 $ @kbd{ls}
1817 rpcalc rpcalc.tab.c rpcalc.y
1818 @end group
1819 @end example
1820
1821 The file @file{rpcalc} now contains the executable code. Here is an
1822 example session using @code{rpcalc}.
1823
1824 @example
1825 $ @kbd{rpcalc}
1826 @kbd{4 9 +}
1827 13
1828 @kbd{3 7 + 3 4 5 *+-}
1829 -13
1830 @kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
1831 13
1832 @kbd{5 6 / 4 n +}
1833 -3.166666667
1834 @kbd{3 4 ^} @r{Exponentiation}
1835 81
1836 @kbd{^D} @r{End-of-file indicator}
1837 $
1838 @end example
1839
1840 @node Infix Calc
1841 @section Infix Notation Calculator: @code{calc}
1842 @cindex infix notation calculator
1843 @cindex @code{calc}
1844 @cindex calculator, infix notation
1845
1846 We now modify rpcalc to handle infix operators instead of postfix. Infix
1847 notation involves the concept of operator precedence and the need for
1848 parentheses nested to arbitrary depth. Here is the Bison code for
1849 @file{calc.y}, an infix desk-top calculator.
1850
1851 @example
1852 /* Infix notation calculator. */
1853
1854 %@{
1855 #define YYSTYPE double
1856 #include <math.h>
1857 #include <stdio.h>
1858 int yylex (void);
1859 void yyerror (char const *);
1860 %@}
1861
1862 /* Bison declarations. */
1863 %token NUM
1864 %left '-' '+'
1865 %left '*' '/'
1866 %left NEG /* negation--unary minus */
1867 %right '^' /* exponentiation */
1868
1869 %% /* The grammar follows. */
1870 input: /* empty */
1871 | input line
1872 ;
1873
1874 line: '\n'
1875 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1876 ;
1877
1878 exp: NUM @{ $$ = $1; @}
1879 | exp '+' exp @{ $$ = $1 + $3; @}
1880 | exp '-' exp @{ $$ = $1 - $3; @}
1881 | exp '*' exp @{ $$ = $1 * $3; @}
1882 | exp '/' exp @{ $$ = $1 / $3; @}
1883 | '-' exp %prec NEG @{ $$ = -$2; @}
1884 | exp '^' exp @{ $$ = pow ($1, $3); @}
1885 | '(' exp ')' @{ $$ = $2; @}
1886 ;
1887 %%
1888 @end example
1889
1890 @noindent
1891 The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1892 same as before.
1893
1894 There are two important new features shown in this code.
1895
1896 In the second section (Bison declarations), @code{%left} declares token
1897 types and says they are left-associative operators. The declarations
1898 @code{%left} and @code{%right} (right associativity) take the place of
1899 @code{%token} which is used to declare a token type name without
1900 associativity. (These tokens are single-character literals, which
1901 ordinarily don't need to be declared. We declare them here to specify
1902 the associativity.)
1903
1904 Operator precedence is determined by the line ordering of the
1905 declarations; the higher the line number of the declaration (lower on
1906 the page or screen), the higher the precedence. Hence, exponentiation
1907 has the highest precedence, unary minus (@code{NEG}) is next, followed
1908 by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator
1909 Precedence}.
1910
1911 The other important new feature is the @code{%prec} in the grammar
1912 section for the unary minus operator. The @code{%prec} simply instructs
1913 Bison that the rule @samp{| '-' exp} has the same precedence as
1914 @code{NEG}---in this case the next-to-highest. @xref{Contextual
1915 Precedence, ,Context-Dependent Precedence}.
1916
1917 Here is a sample run of @file{calc.y}:
1918
1919 @need 500
1920 @example
1921 $ @kbd{calc}
1922 @kbd{4 + 4.5 - (34/(8*3+-3))}
1923 6.880952381
1924 @kbd{-56 + 2}
1925 -54
1926 @kbd{3 ^ 2}
1927 9
1928 @end example
1929
1930 @node Simple Error Recovery
1931 @section Simple Error Recovery
1932 @cindex error recovery, simple
1933
1934 Up to this point, this manual has not addressed the issue of @dfn{error
1935 recovery}---how to continue parsing after the parser detects a syntax
1936 error. All we have handled is error reporting with @code{yyerror}.
1937 Recall that by default @code{yyparse} returns after calling
1938 @code{yyerror}. This means that an erroneous input line causes the
1939 calculator program to exit. Now we show how to rectify this deficiency.
1940
1941 The Bison language itself includes the reserved word @code{error}, which
1942 may be included in the grammar rules. In the example below it has
1943 been added to one of the alternatives for @code{line}:
1944
1945 @example
1946 @group
1947 line: '\n'
1948 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1949 | error '\n' @{ yyerrok; @}
1950 ;
1951 @end group
1952 @end example
1953
1954 This addition to the grammar allows for simple error recovery in the
1955 event of a syntax error. If an expression that cannot be evaluated is
1956 read, the error will be recognized by the third rule for @code{line},
1957 and parsing will continue. (The @code{yyerror} function is still called
1958 upon to print its message as well.) The action executes the statement
1959 @code{yyerrok}, a macro defined automatically by Bison; its meaning is
1960 that error recovery is complete (@pxref{Error Recovery}). Note the
1961 difference between @code{yyerrok} and @code{yyerror}; neither one is a
1962 misprint.
1963
1964 This form of error recovery deals with syntax errors. There are other
1965 kinds of errors; for example, division by zero, which raises an exception
1966 signal that is normally fatal. A real calculator program must handle this
1967 signal and use @code{longjmp} to return to @code{main} and resume parsing
1968 input lines; it would also have to discard the rest of the current line of
1969 input. We won't discuss this issue further because it is not specific to
1970 Bison programs.
1971
1972 @node Location Tracking Calc
1973 @section Location Tracking Calculator: @code{ltcalc}
1974 @cindex location tracking calculator
1975 @cindex @code{ltcalc}
1976 @cindex calculator, location tracking
1977
1978 This example extends the infix notation calculator with location
1979 tracking. This feature will be used to improve the error messages. For
1980 the sake of clarity, this example is a simple integer calculator, since
1981 most of the work needed to use locations will be done in the lexical
1982 analyzer.
1983
1984 @menu
1985 * Ltcalc Declarations:: Bison and C declarations for ltcalc.
1986 * Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
1987 * Ltcalc Lexer:: The lexical analyzer.
1988 @end menu
1989
1990 @node Ltcalc Declarations
1991 @subsection Declarations for @code{ltcalc}
1992
1993 The C and Bison declarations for the location tracking calculator are
1994 the same as the declarations for the infix notation calculator.
1995
1996 @example
1997 /* Location tracking calculator. */
1998
1999 %@{
2000 #define YYSTYPE int
2001 #include <math.h>
2002 int yylex (void);
2003 void yyerror (char const *);
2004 %@}
2005
2006 /* Bison declarations. */
2007 %token NUM
2008
2009 %left '-' '+'
2010 %left '*' '/'
2011 %left NEG
2012 %right '^'
2013
2014 %% /* The grammar follows. */
2015 @end example
2016
2017 @noindent
2018 Note there are no declarations specific to locations. Defining a data
2019 type for storing locations is not needed: we will use the type provided
2020 by default (@pxref{Location Type, ,Data Types of Locations}), which is a
2021 four member structure with the following integer fields:
2022 @code{first_line}, @code{first_column}, @code{last_line} and
2023 @code{last_column}. By conventions, and in accordance with the GNU
2024 Coding Standards and common practice, the line and column count both
2025 start at 1.
2026
2027 @node Ltcalc Rules
2028 @subsection Grammar Rules for @code{ltcalc}
2029
2030 Whether handling locations or not has no effect on the syntax of your
2031 language. Therefore, grammar rules for this example will be very close
2032 to those of the previous example: we will only modify them to benefit
2033 from the new information.
2034
2035 Here, we will use locations to report divisions by zero, and locate the
2036 wrong expressions or subexpressions.
2037
2038 @example
2039 @group
2040 input : /* empty */
2041 | input line
2042 ;
2043 @end group
2044
2045 @group
2046 line : '\n'
2047 | exp '\n' @{ printf ("%d\n", $1); @}
2048 ;
2049 @end group
2050
2051 @group
2052 exp : NUM @{ $$ = $1; @}
2053 | exp '+' exp @{ $$ = $1 + $3; @}
2054 | exp '-' exp @{ $$ = $1 - $3; @}
2055 | exp '*' exp @{ $$ = $1 * $3; @}
2056 @end group
2057 @group
2058 | exp '/' exp
2059 @{
2060 if ($3)
2061 $$ = $1 / $3;
2062 else
2063 @{
2064 $$ = 1;
2065 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2066 @@3.first_line, @@3.first_column,
2067 @@3.last_line, @@3.last_column);
2068 @}
2069 @}
2070 @end group
2071 @group
2072 | '-' exp %prec NEG @{ $$ = -$2; @}
2073 | exp '^' exp @{ $$ = pow ($1, $3); @}
2074 | '(' exp ')' @{ $$ = $2; @}
2075 @end group
2076 @end example
2077
2078 This code shows how to reach locations inside of semantic actions, by
2079 using the pseudo-variables @code{@@@var{n}} for rule components, and the
2080 pseudo-variable @code{@@$} for groupings.
2081
2082 We don't need to assign a value to @code{@@$}: the output parser does it
2083 automatically. By default, before executing the C code of each action,
2084 @code{@@$} is set to range from the beginning of @code{@@1} to the end
2085 of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2086 can be redefined (@pxref{Location Default Action, , Default Action for
2087 Locations}), and for very specific rules, @code{@@$} can be computed by
2088 hand.
2089
2090 @node Ltcalc Lexer
2091 @subsection The @code{ltcalc} Lexical Analyzer.
2092
2093 Until now, we relied on Bison's defaults to enable location
2094 tracking. The next step is to rewrite the lexical analyzer, and make it
2095 able to feed the parser with the token locations, as it already does for
2096 semantic values.
2097
2098 To this end, we must take into account every single character of the
2099 input text, to avoid the computed locations of being fuzzy or wrong:
2100
2101 @example
2102 @group
2103 int
2104 yylex (void)
2105 @{
2106 int c;
2107 @end group
2108
2109 @group
2110 /* Skip white space. */
2111 while ((c = getchar ()) == ' ' || c == '\t')
2112 ++yylloc.last_column;
2113 @end group
2114
2115 @group
2116 /* Step. */
2117 yylloc.first_line = yylloc.last_line;
2118 yylloc.first_column = yylloc.last_column;
2119 @end group
2120
2121 @group
2122 /* Process numbers. */
2123 if (isdigit (c))
2124 @{
2125 yylval = c - '0';
2126 ++yylloc.last_column;
2127 while (isdigit (c = getchar ()))
2128 @{
2129 ++yylloc.last_column;
2130 yylval = yylval * 10 + c - '0';
2131 @}
2132 ungetc (c, stdin);
2133 return NUM;
2134 @}
2135 @end group
2136
2137 /* Return end-of-input. */
2138 if (c == EOF)
2139 return 0;
2140
2141 /* Return a single char, and update location. */
2142 if (c == '\n')
2143 @{
2144 ++yylloc.last_line;
2145 yylloc.last_column = 0;
2146 @}
2147 else
2148 ++yylloc.last_column;
2149 return c;
2150 @}
2151 @end example
2152
2153 Basically, the lexical analyzer performs the same processing as before:
2154 it skips blanks and tabs, and reads numbers or single-character tokens.
2155 In addition, it updates @code{yylloc}, the global variable (of type
2156 @code{YYLTYPE}) containing the token's location.
2157
2158 Now, each time this function returns a token, the parser has its number
2159 as well as its semantic value, and its location in the text. The last
2160 needed change is to initialize @code{yylloc}, for example in the
2161 controlling function:
2162
2163 @example
2164 @group
2165 int
2166 main (void)
2167 @{
2168 yylloc.first_line = yylloc.last_line = 1;
2169 yylloc.first_column = yylloc.last_column = 0;
2170 return yyparse ();
2171 @}
2172 @end group
2173 @end example
2174
2175 Remember that computing locations is not a matter of syntax. Every
2176 character must be associated to a location update, whether it is in
2177 valid input, in comments, in literal strings, and so on.
2178
2179 @node Multi-function Calc
2180 @section Multi-Function Calculator: @code{mfcalc}
2181 @cindex multi-function calculator
2182 @cindex @code{mfcalc}
2183 @cindex calculator, multi-function
2184
2185 Now that the basics of Bison have been discussed, it is time to move on to
2186 a more advanced problem. The above calculators provided only five
2187 functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2188 be nice to have a calculator that provides other mathematical functions such
2189 as @code{sin}, @code{cos}, etc.
2190
2191 It is easy to add new operators to the infix calculator as long as they are
2192 only single-character literals. The lexical analyzer @code{yylex} passes
2193 back all nonnumeric characters as tokens, so new grammar rules suffice for
2194 adding a new operator. But we want something more flexible: built-in
2195 functions whose syntax has this form:
2196
2197 @example
2198 @var{function_name} (@var{argument})
2199 @end example
2200
2201 @noindent
2202 At the same time, we will add memory to the calculator, by allowing you
2203 to create named variables, store values in them, and use them later.
2204 Here is a sample session with the multi-function calculator:
2205
2206 @example
2207 $ @kbd{mfcalc}
2208 @kbd{pi = 3.141592653589}
2209 3.1415926536
2210 @kbd{sin(pi)}
2211 0.0000000000
2212 @kbd{alpha = beta1 = 2.3}
2213 2.3000000000
2214 @kbd{alpha}
2215 2.3000000000
2216 @kbd{ln(alpha)}
2217 0.8329091229
2218 @kbd{exp(ln(beta1))}
2219 2.3000000000
2220 $
2221 @end example
2222
2223 Note that multiple assignment and nested function calls are permitted.
2224
2225 @menu
2226 * Mfcalc Declarations:: Bison declarations for multi-function calculator.
2227 * Mfcalc Rules:: Grammar rules for the calculator.
2228 * Mfcalc Symbol Table:: Symbol table management subroutines.
2229 @end menu
2230
2231 @node Mfcalc Declarations
2232 @subsection Declarations for @code{mfcalc}
2233
2234 Here are the C and Bison declarations for the multi-function calculator.
2235
2236 @smallexample
2237 @group
2238 %@{
2239 #include <math.h> /* For math functions, cos(), sin(), etc. */
2240 #include "calc.h" /* Contains definition of `symrec'. */
2241 int yylex (void);
2242 void yyerror (char const *);
2243 %@}
2244 @end group
2245 @group
2246 %union @{
2247 double val; /* For returning numbers. */
2248 symrec *tptr; /* For returning symbol-table pointers. */
2249 @}
2250 @end group
2251 %token <val> NUM /* Simple double precision number. */
2252 %token <tptr> VAR FNCT /* Variable and Function. */
2253 %type <val> exp
2254
2255 @group
2256 %right '='
2257 %left '-' '+'
2258 %left '*' '/'
2259 %left NEG /* negation--unary minus */
2260 %right '^' /* exponentiation */
2261 @end group
2262 %% /* The grammar follows. */
2263 @end smallexample
2264
2265 The above grammar introduces only two new features of the Bison language.
2266 These features allow semantic values to have various data types
2267 (@pxref{Multiple Types, ,More Than One Value Type}).
2268
2269 The @code{%union} declaration specifies the entire list of possible types;
2270 this is instead of defining @code{YYSTYPE}. The allowable types are now
2271 double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2272 the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2273
2274 Since values can now have various types, it is necessary to associate a
2275 type with each grammar symbol whose semantic value is used. These symbols
2276 are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2277 declarations are augmented with information about their data type (placed
2278 between angle brackets).
2279
2280 The Bison construct @code{%type} is used for declaring nonterminal
2281 symbols, just as @code{%token} is used for declaring token types. We
2282 have not used @code{%type} before because nonterminal symbols are
2283 normally declared implicitly by the rules that define them. But
2284 @code{exp} must be declared explicitly so we can specify its value type.
2285 @xref{Type Decl, ,Nonterminal Symbols}.
2286
2287 @node Mfcalc Rules
2288 @subsection Grammar Rules for @code{mfcalc}
2289
2290 Here are the grammar rules for the multi-function calculator.
2291 Most of them are copied directly from @code{calc}; three rules,
2292 those which mention @code{VAR} or @code{FNCT}, are new.
2293
2294 @smallexample
2295 @group
2296 input: /* empty */
2297 | input line
2298 ;
2299 @end group
2300
2301 @group
2302 line:
2303 '\n'
2304 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2305 | error '\n' @{ yyerrok; @}
2306 ;
2307 @end group
2308
2309 @group
2310 exp: NUM @{ $$ = $1; @}
2311 | VAR @{ $$ = $1->value.var; @}
2312 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2313 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2314 | exp '+' exp @{ $$ = $1 + $3; @}
2315 | exp '-' exp @{ $$ = $1 - $3; @}
2316 | exp '*' exp @{ $$ = $1 * $3; @}
2317 | exp '/' exp @{ $$ = $1 / $3; @}
2318 | '-' exp %prec NEG @{ $$ = -$2; @}
2319 | exp '^' exp @{ $$ = pow ($1, $3); @}
2320 | '(' exp ')' @{ $$ = $2; @}
2321 ;
2322 @end group
2323 /* End of grammar. */
2324 %%
2325 @end smallexample
2326
2327 @node Mfcalc Symbol Table
2328 @subsection The @code{mfcalc} Symbol Table
2329 @cindex symbol table example
2330
2331 The multi-function calculator requires a symbol table to keep track of the
2332 names and meanings of variables and functions. This doesn't affect the
2333 grammar rules (except for the actions) or the Bison declarations, but it
2334 requires some additional C functions for support.
2335
2336 The symbol table itself consists of a linked list of records. Its
2337 definition, which is kept in the header @file{calc.h}, is as follows. It
2338 provides for either functions or variables to be placed in the table.
2339
2340 @smallexample
2341 @group
2342 /* Function type. */
2343 typedef double (*func_t) (double);
2344 @end group
2345
2346 @group
2347 /* Data type for links in the chain of symbols. */
2348 struct symrec
2349 @{
2350 char *name; /* name of symbol */
2351 int type; /* type of symbol: either VAR or FNCT */
2352 union
2353 @{
2354 double var; /* value of a VAR */
2355 func_t fnctptr; /* value of a FNCT */
2356 @} value;
2357 struct symrec *next; /* link field */
2358 @};
2359 @end group
2360
2361 @group
2362 typedef struct symrec symrec;
2363
2364 /* The symbol table: a chain of `struct symrec'. */
2365 extern symrec *sym_table;
2366
2367 symrec *putsym (char const *, int);
2368 symrec *getsym (char const *);
2369 @end group
2370 @end smallexample
2371
2372 The new version of @code{main} includes a call to @code{init_table}, a
2373 function that initializes the symbol table. Here it is, and
2374 @code{init_table} as well:
2375
2376 @smallexample
2377 #include <stdio.h>
2378
2379 @group
2380 /* Called by yyparse on error. */
2381 void
2382 yyerror (char const *s)
2383 @{
2384 printf ("%s\n", s);
2385 @}
2386 @end group
2387
2388 @group
2389 struct init
2390 @{
2391 char const *fname;
2392 double (*fnct) (double);
2393 @};
2394 @end group
2395
2396 @group
2397 struct init const arith_fncts[] =
2398 @{
2399 "sin", sin,
2400 "cos", cos,
2401 "atan", atan,
2402 "ln", log,
2403 "exp", exp,
2404 "sqrt", sqrt,
2405 0, 0
2406 @};
2407 @end group
2408
2409 @group
2410 /* The symbol table: a chain of `struct symrec'. */
2411 symrec *sym_table;
2412 @end group
2413
2414 @group
2415 /* Put arithmetic functions in table. */
2416 void
2417 init_table (void)
2418 @{
2419 int i;
2420 symrec *ptr;
2421 for (i = 0; arith_fncts[i].fname != 0; i++)
2422 @{
2423 ptr = putsym (arith_fncts[i].fname, FNCT);
2424 ptr->value.fnctptr = arith_fncts[i].fnct;
2425 @}
2426 @}
2427 @end group
2428
2429 @group
2430 int
2431 main (void)
2432 @{
2433 init_table ();
2434 return yyparse ();
2435 @}
2436 @end group
2437 @end smallexample
2438
2439 By simply editing the initialization list and adding the necessary include
2440 files, you can add additional functions to the calculator.
2441
2442 Two important functions allow look-up and installation of symbols in the
2443 symbol table. The function @code{putsym} is passed a name and the type
2444 (@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2445 linked to the front of the list, and a pointer to the object is returned.
2446 The function @code{getsym} is passed the name of the symbol to look up. If
2447 found, a pointer to that symbol is returned; otherwise zero is returned.
2448
2449 @smallexample
2450 symrec *
2451 putsym (char const *sym_name, int sym_type)
2452 @{
2453 symrec *ptr;
2454 ptr = (symrec *) malloc (sizeof (symrec));
2455 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2456 strcpy (ptr->name,sym_name);
2457 ptr->type = sym_type;
2458 ptr->value.var = 0; /* Set value to 0 even if fctn. */
2459 ptr->next = (struct symrec *)sym_table;
2460 sym_table = ptr;
2461 return ptr;
2462 @}
2463
2464 symrec *
2465 getsym (char const *sym_name)
2466 @{
2467 symrec *ptr;
2468 for (ptr = sym_table; ptr != (symrec *) 0;
2469 ptr = (symrec *)ptr->next)
2470 if (strcmp (ptr->name,sym_name) == 0)
2471 return ptr;
2472 return 0;
2473 @}
2474 @end smallexample
2475
2476 The function @code{yylex} must now recognize variables, numeric values, and
2477 the single-character arithmetic operators. Strings of alphanumeric
2478 characters with a leading letter are recognized as either variables or
2479 functions depending on what the symbol table says about them.
2480
2481 The string is passed to @code{getsym} for look up in the symbol table. If
2482 the name appears in the table, a pointer to its location and its type
2483 (@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2484 already in the table, then it is installed as a @code{VAR} using
2485 @code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
2486 returned to @code{yyparse}.
2487
2488 No change is needed in the handling of numeric values and arithmetic
2489 operators in @code{yylex}.
2490
2491 @smallexample
2492 @group
2493 #include <ctype.h>
2494 @end group
2495
2496 @group
2497 int
2498 yylex (void)
2499 @{
2500 int c;
2501
2502 /* Ignore white space, get first nonwhite character. */
2503 while ((c = getchar ()) == ' ' || c == '\t');
2504
2505 if (c == EOF)
2506 return 0;
2507 @end group
2508
2509 @group
2510 /* Char starts a number => parse the number. */
2511 if (c == '.' || isdigit (c))
2512 @{
2513 ungetc (c, stdin);
2514 scanf ("%lf", &yylval.val);
2515 return NUM;
2516 @}
2517 @end group
2518
2519 @group
2520 /* Char starts an identifier => read the name. */
2521 if (isalpha (c))
2522 @{
2523 symrec *s;
2524 static char *symbuf = 0;
2525 static int length = 0;
2526 int i;
2527 @end group
2528
2529 @group
2530 /* Initially make the buffer long enough
2531 for a 40-character symbol name. */
2532 if (length == 0)
2533 length = 40, symbuf = (char *)malloc (length + 1);
2534
2535 i = 0;
2536 do
2537 @end group
2538 @group
2539 @{
2540 /* If buffer is full, make it bigger. */
2541 if (i == length)
2542 @{
2543 length *= 2;
2544 symbuf = (char *) realloc (symbuf, length + 1);
2545 @}
2546 /* Add this character to the buffer. */
2547 symbuf[i++] = c;
2548 /* Get another character. */
2549 c = getchar ();
2550 @}
2551 @end group
2552 @group
2553 while (isalnum (c));
2554
2555 ungetc (c, stdin);
2556 symbuf[i] = '\0';
2557 @end group
2558
2559 @group
2560 s = getsym (symbuf);
2561 if (s == 0)
2562 s = putsym (symbuf, VAR);
2563 yylval.tptr = s;
2564 return s->type;
2565 @}
2566
2567 /* Any other character is a token by itself. */
2568 return c;
2569 @}
2570 @end group
2571 @end smallexample
2572
2573 This program is both powerful and flexible. You may easily add new
2574 functions, and it is a simple job to modify this code to install
2575 predefined variables such as @code{pi} or @code{e} as well.
2576
2577 @node Exercises
2578 @section Exercises
2579 @cindex exercises
2580
2581 @enumerate
2582 @item
2583 Add some new functions from @file{math.h} to the initialization list.
2584
2585 @item
2586 Add another array that contains constants and their values. Then
2587 modify @code{init_table} to add these constants to the symbol table.
2588 It will be easiest to give the constants type @code{VAR}.
2589
2590 @item
2591 Make the program report an error if the user refers to an
2592 uninitialized variable in any way except to store a value in it.
2593 @end enumerate
2594
2595 @node Grammar File
2596 @chapter Bison Grammar Files
2597
2598 Bison takes as input a context-free grammar specification and produces a
2599 C-language function that recognizes correct instances of the grammar.
2600
2601 The Bison grammar input file conventionally has a name ending in @samp{.y}.
2602 @xref{Invocation, ,Invoking Bison}.
2603
2604 @menu
2605 * Grammar Outline:: Overall layout of the grammar file.
2606 * Symbols:: Terminal and nonterminal symbols.
2607 * Rules:: How to write grammar rules.
2608 * Recursion:: Writing recursive rules.
2609 * Semantics:: Semantic values and actions.
2610 * Locations:: Locations and actions.
2611 * Declarations:: All kinds of Bison declarations are described here.
2612 * Multiple Parsers:: Putting more than one Bison parser in one program.
2613 @end menu
2614
2615 @node Grammar Outline
2616 @section Outline of a Bison Grammar
2617
2618 A Bison grammar file has four main sections, shown here with the
2619 appropriate delimiters:
2620
2621 @example
2622 %@{
2623 @var{Prologue}
2624 %@}
2625
2626 @var{Bison declarations}
2627
2628 %%
2629 @var{Grammar rules}
2630 %%
2631
2632 @var{Epilogue}
2633 @end example
2634
2635 Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2636 As a GNU extension, @samp{//} introduces a comment that
2637 continues until end of line.
2638
2639 @menu
2640 * Prologue:: Syntax and usage of the prologue.
2641 * Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
2642 * Bison Declarations:: Syntax and usage of the Bison declarations section.
2643 * Grammar Rules:: Syntax and usage of the grammar rules section.
2644 * Epilogue:: Syntax and usage of the epilogue.
2645 @end menu
2646
2647 @node Prologue
2648 @subsection The prologue
2649 @cindex declarations section
2650 @cindex Prologue
2651 @cindex declarations
2652
2653 The @var{Prologue} section contains macro definitions and declarations
2654 of functions and variables that are used in the actions in the grammar
2655 rules. These are copied to the beginning of the parser file so that
2656 they precede the definition of @code{yyparse}. You can use
2657 @samp{#include} to get the declarations from a header file. If you
2658 don't need any C declarations, you may omit the @samp{%@{} and
2659 @samp{%@}} delimiters that bracket this section.
2660
2661 The @var{Prologue} section is terminated by the first occurrence
2662 of @samp{%@}} that is outside a comment, a string literal, or a
2663 character constant.
2664
2665 You may have more than one @var{Prologue} section, intermixed with the
2666 @var{Bison declarations}. This allows you to have C and Bison
2667 declarations that refer to each other. For example, the @code{%union}
2668 declaration may use types defined in a header file, and you may wish to
2669 prototype functions that take arguments of type @code{YYSTYPE}. This
2670 can be done with two @var{Prologue} blocks, one before and one after the
2671 @code{%union} declaration.
2672
2673 @smallexample
2674 %@{
2675 #define _GNU_SOURCE
2676 #include <stdio.h>
2677 #include "ptypes.h"
2678 %@}
2679
2680 %union @{
2681 long int n;
2682 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2683 @}
2684
2685 %@{
2686 static void print_token_value (FILE *, int, YYSTYPE);
2687 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2688 %@}
2689
2690 @dots{}
2691 @end smallexample
2692
2693 When in doubt, it is usually safer to put prologue code before all
2694 Bison declarations, rather than after. For example, any definitions
2695 of feature test macros like @code{_GNU_SOURCE} or
2696 @code{_POSIX_C_SOURCE} should appear before all Bison declarations, as
2697 feature test macros can affect the behavior of Bison-generated
2698 @code{#include} directives.
2699
2700 @node Prologue Alternatives
2701 @subsection Prologue Alternatives
2702 @cindex Prologue Alternatives
2703
2704 @findex %code
2705 @findex %code requires
2706 @findex %code provides
2707 @findex %code top
2708
2709 The functionality of @var{Prologue} sections can often be subtle and
2710 inflexible.
2711 As an alternative, Bison provides a %code directive with an explicit qualifier
2712 field, which identifies the purpose of the code and thus the location(s) where
2713 Bison should generate it.
2714 For C/C++, the qualifier can be omitted for the default location, or it can be
2715 one of @code{requires}, @code{provides}, @code{top}.
2716 @xref{Decl Summary,,%code}.
2717
2718 Look again at the example of the previous section:
2719
2720 @smallexample
2721 %@{
2722 #define _GNU_SOURCE
2723 #include <stdio.h>
2724 #include "ptypes.h"
2725 %@}
2726
2727 %union @{
2728 long int n;
2729 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2730 @}
2731
2732 %@{
2733 static void print_token_value (FILE *, int, YYSTYPE);
2734 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2735 %@}
2736
2737 @dots{}
2738 @end smallexample
2739
2740 @noindent
2741 Notice that there are two @var{Prologue} sections here, but there's a subtle
2742 distinction between their functionality.
2743 For example, if you decide to override Bison's default definition for
2744 @code{YYLTYPE}, in which @var{Prologue} section should you write your new
2745 definition?
2746 You should write it in the first since Bison will insert that code into the
2747 parser source code file @emph{before} the default @code{YYLTYPE} definition.
2748 In which @var{Prologue} section should you prototype an internal function,
2749 @code{trace_token}, that accepts @code{YYLTYPE} and @code{yytokentype} as
2750 arguments?
2751 You should prototype it in the second since Bison will insert that code
2752 @emph{after} the @code{YYLTYPE} and @code{yytokentype} definitions.
2753
2754 This distinction in functionality between the two @var{Prologue} sections is
2755 established by the appearance of the @code{%union} between them.
2756 This behavior raises a few questions.
2757 First, why should the position of a @code{%union} affect definitions related to
2758 @code{YYLTYPE} and @code{yytokentype}?
2759 Second, what if there is no @code{%union}?
2760 In that case, the second kind of @var{Prologue} section is not available.
2761 This behavior is not intuitive.
2762
2763 To avoid this subtle @code{%union} dependency, rewrite the example using a
2764 @code{%code top} and an unqualified @code{%code}.
2765 Let's go ahead and add the new @code{YYLTYPE} definition and the
2766 @code{trace_token} prototype at the same time:
2767
2768 @smallexample
2769 %code top @{
2770 #define _GNU_SOURCE
2771 #include <stdio.h>
2772
2773 /* WARNING: The following code really belongs
2774 * in a `%code requires'; see below. */
2775
2776 #include "ptypes.h"
2777 #define YYLTYPE YYLTYPE
2778 typedef struct YYLTYPE
2779 @{
2780 int first_line;
2781 int first_column;
2782 int last_line;
2783 int last_column;
2784 char *filename;
2785 @} YYLTYPE;
2786 @}
2787
2788 %union @{
2789 long int n;
2790 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2791 @}
2792
2793 %code @{
2794 static void print_token_value (FILE *, int, YYSTYPE);
2795 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2796 static void trace_token (enum yytokentype token, YYLTYPE loc);
2797 @}
2798
2799 @dots{}
2800 @end smallexample
2801
2802 @noindent
2803 In this way, @code{%code top} and the unqualified @code{%code} achieve the same
2804 functionality as the two kinds of @var{Prologue} sections, but it's always
2805 explicit which kind you intend.
2806 Moreover, both kinds are always available even in the absence of @code{%union}.
2807
2808 The @code{%code top} block above logically contains two parts.
2809 The first two lines before the warning need to appear near the top of the
2810 parser source code file.
2811 The first line after the warning is required by @code{YYSTYPE} and thus also
2812 needs to appear in the parser source code file.
2813 However, if you've instructed Bison to generate a parser header file
2814 (@pxref{Decl Summary, ,%defines}), you probably want that line to appear before
2815 the @code{YYSTYPE} definition in that header file as well.
2816 The @code{YYLTYPE} definition should also appear in the parser header file to
2817 override the default @code{YYLTYPE} definition there.
2818
2819 In other words, in the @code{%code top} block above, all but the first two
2820 lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
2821 definitions.
2822 Thus, they belong in one or more @code{%code requires}:
2823
2824 @smallexample
2825 %code top @{
2826 #define _GNU_SOURCE
2827 #include <stdio.h>
2828 @}
2829
2830 %code requires @{
2831 #include "ptypes.h"
2832 @}
2833 %union @{
2834 long int n;
2835 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2836 @}
2837
2838 %code requires @{
2839 #define YYLTYPE YYLTYPE
2840 typedef struct YYLTYPE
2841 @{
2842 int first_line;
2843 int first_column;
2844 int last_line;
2845 int last_column;
2846 char *filename;
2847 @} YYLTYPE;
2848 @}
2849
2850 %code @{
2851 static void print_token_value (FILE *, int, YYSTYPE);
2852 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2853 static void trace_token (enum yytokentype token, YYLTYPE loc);
2854 @}
2855
2856 @dots{}
2857 @end smallexample
2858
2859 @noindent
2860 Now Bison will insert @code{#include "ptypes.h"} and the new @code{YYLTYPE}
2861 definition before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
2862 definitions in both the parser source code file and the parser header file.
2863 (By the same reasoning, @code{%code requires} would also be the appropriate
2864 place to write your own definition for @code{YYSTYPE}.)
2865
2866 When you are writing dependency code for @code{YYSTYPE} and @code{YYLTYPE}, you
2867 should prefer @code{%code requires} over @code{%code top} regardless of whether
2868 you instruct Bison to generate a parser header file.
2869 When you are writing code that you need Bison to insert only into the parser
2870 source code file and that has no special need to appear at the top of that
2871 file, you should prefer the unqualified @code{%code} over @code{%code top}.
2872 These practices will make the purpose of each block of your code explicit to
2873 Bison and to other developers reading your grammar file.
2874 Following these practices, we expect the unqualified @code{%code} and
2875 @code{%code requires} to be the most important of the four @var{Prologue}
2876 alternatives.
2877
2878 At some point while developing your parser, you might decide to provide
2879 @code{trace_token} to modules that are external to your parser.
2880 Thus, you might wish for Bison to insert the prototype into both the parser
2881 header file and the parser source code file.
2882 Since this function is not a dependency required by @code{YYSTYPE} or
2883 @code{YYLTYPE}, it doesn't make sense to move its prototype to a
2884 @code{%code requires}.
2885 More importantly, since it depends upon @code{YYLTYPE} and @code{yytokentype},
2886 @code{%code requires} is not sufficient.
2887 Instead, move its prototype from the unqualified @code{%code} to a
2888 @code{%code provides}:
2889
2890 @smallexample
2891 %code top @{
2892 #define _GNU_SOURCE
2893 #include <stdio.h>
2894 @}
2895
2896 %code requires @{
2897 #include "ptypes.h"
2898 @}
2899 %union @{
2900 long int n;
2901 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2902 @}
2903
2904 %code requires @{
2905 #define YYLTYPE YYLTYPE
2906 typedef struct YYLTYPE
2907 @{
2908 int first_line;
2909 int first_column;
2910 int last_line;
2911 int last_column;
2912 char *filename;
2913 @} YYLTYPE;
2914 @}
2915
2916 %code provides @{
2917 void trace_token (enum yytokentype token, YYLTYPE loc);
2918 @}
2919
2920 %code @{
2921 static void print_token_value (FILE *, int, YYSTYPE);
2922 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2923 @}
2924
2925 @dots{}
2926 @end smallexample
2927
2928 @noindent
2929 Bison will insert the @code{trace_token} prototype into both the parser header
2930 file and the parser source code file after the definitions for
2931 @code{yytokentype}, @code{YYLTYPE}, and @code{YYSTYPE}.
2932
2933 The above examples are careful to write directives in an order that reflects
2934 the layout of the generated parser source code and header files:
2935 @code{%code top}, @code{%code requires}, @code{%code provides}, and then
2936 @code{%code}.
2937 While your grammar files may generally be easier to read if you also follow
2938 this order, Bison does not require it.
2939 Instead, Bison lets you choose an organization that makes sense to you.
2940
2941 You may declare any of these directives multiple times in the grammar file.
2942 In that case, Bison concatenates the contained code in declaration order.
2943 This is the only way in which the position of one of these directives within
2944 the grammar file affects its functionality.
2945
2946 The result of the previous two properties is greater flexibility in how you may
2947 organize your grammar file.
2948 For example, you may organize semantic-type-related directives by semantic
2949 type:
2950
2951 @smallexample
2952 %code requires @{ #include "type1.h" @}
2953 %union @{ type1 field1; @}
2954 %destructor @{ type1_free ($$); @} <field1>
2955 %printer @{ type1_print ($$); @} <field1>
2956
2957 %code requires @{ #include "type2.h" @}
2958 %union @{ type2 field2; @}
2959 %destructor @{ type2_free ($$); @} <field2>
2960 %printer @{ type2_print ($$); @} <field2>
2961 @end smallexample
2962
2963 @noindent
2964 You could even place each of the above directive groups in the rules section of
2965 the grammar file next to the set of rules that uses the associated semantic
2966 type.
2967 (In the rules section, you must terminate each of those directives with a
2968 semicolon.)
2969 And you don't have to worry that some directive (like a @code{%union}) in the
2970 definitions section is going to adversely affect their functionality in some
2971 counter-intuitive manner just because it comes first.
2972 Such an organization is not possible using @var{Prologue} sections.
2973
2974 This section has been concerned with explaining the advantages of the four
2975 @var{Prologue} alternatives over the original Yacc @var{Prologue}.
2976 However, in most cases when using these directives, you shouldn't need to
2977 think about all the low-level ordering issues discussed here.
2978 Instead, you should simply use these directives to label each block of your
2979 code according to its purpose and let Bison handle the ordering.
2980 @code{%code} is the most generic label.
2981 Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
2982 as needed.
2983
2984 @node Bison Declarations
2985 @subsection The Bison Declarations Section
2986 @cindex Bison declarations (introduction)
2987 @cindex declarations, Bison (introduction)
2988
2989 The @var{Bison declarations} section contains declarations that define
2990 terminal and nonterminal symbols, specify precedence, and so on.
2991 In some simple grammars you may not need any declarations.
2992 @xref{Declarations, ,Bison Declarations}.
2993
2994 @node Grammar Rules
2995 @subsection The Grammar Rules Section
2996 @cindex grammar rules section
2997 @cindex rules section for grammar
2998
2999 The @dfn{grammar rules} section contains one or more Bison grammar
3000 rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
3001
3002 There must always be at least one grammar rule, and the first
3003 @samp{%%} (which precedes the grammar rules) may never be omitted even
3004 if it is the first thing in the file.
3005
3006 @node Epilogue
3007 @subsection The epilogue
3008 @cindex additional C code section
3009 @cindex epilogue
3010 @cindex C code, section for additional
3011
3012 The @var{Epilogue} is copied verbatim to the end of the parser file, just as
3013 the @var{Prologue} is copied to the beginning. This is the most convenient
3014 place to put anything that you want to have in the parser file but which need
3015 not come before the definition of @code{yyparse}. For example, the
3016 definitions of @code{yylex} and @code{yyerror} often go here. Because
3017 C requires functions to be declared before being used, you often need
3018 to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
3019 even if you define them in the Epilogue.
3020 @xref{Interface, ,Parser C-Language Interface}.
3021
3022 If the last section is empty, you may omit the @samp{%%} that separates it
3023 from the grammar rules.
3024
3025 The Bison parser itself contains many macros and identifiers whose names
3026 start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3027 any such names (except those documented in this manual) in the epilogue
3028 of the grammar file.
3029
3030 @node Symbols
3031 @section Symbols, Terminal and Nonterminal
3032 @cindex nonterminal symbol
3033 @cindex terminal symbol
3034 @cindex token type
3035 @cindex symbol
3036
3037 @dfn{Symbols} in Bison grammars represent the grammatical classifications
3038 of the language.
3039
3040 A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
3041 class of syntactically equivalent tokens. You use the symbol in grammar
3042 rules to mean that a token in that class is allowed. The symbol is
3043 represented in the Bison parser by a numeric code, and the @code{yylex}
3044 function returns a token type code to indicate what kind of token has
3045 been read. You don't need to know what the code value is; you can use
3046 the symbol to stand for it.
3047
3048 A @dfn{nonterminal symbol} stands for a class of syntactically
3049 equivalent groupings. The symbol name is used in writing grammar rules.
3050 By convention, it should be all lower case.
3051
3052 Symbol names can contain letters, underscores, periods, dashes, and (not
3053 at the beginning) digits. Dashes in symbol names are a GNU
3054 extension, incompatible with POSIX Yacc. Terminal symbols
3055 that contain periods or dashes make little sense: since they are not
3056 valid symbols (in most programming languages) they are not exported as
3057 token names.
3058
3059 There are three ways of writing terminal symbols in the grammar:
3060
3061 @itemize @bullet
3062 @item
3063 A @dfn{named token type} is written with an identifier, like an
3064 identifier in C@. By convention, it should be all upper case. Each
3065 such name must be defined with a Bison declaration such as
3066 @code{%token}. @xref{Token Decl, ,Token Type Names}.
3067
3068 @item
3069 @cindex character token
3070 @cindex literal token
3071 @cindex single-character literal
3072 A @dfn{character token type} (or @dfn{literal character token}) is
3073 written in the grammar using the same syntax used in C for character
3074 constants; for example, @code{'+'} is a character token type. A
3075 character token type doesn't need to be declared unless you need to
3076 specify its semantic value data type (@pxref{Value Type, ,Data Types of
3077 Semantic Values}), associativity, or precedence (@pxref{Precedence,
3078 ,Operator Precedence}).
3079
3080 By convention, a character token type is used only to represent a
3081 token that consists of that particular character. Thus, the token
3082 type @code{'+'} is used to represent the character @samp{+} as a
3083 token. Nothing enforces this convention, but if you depart from it,
3084 your program will confuse other readers.
3085
3086 All the usual escape sequences used in character literals in C can be
3087 used in Bison as well, but you must not use the null character as a
3088 character literal because its numeric code, zero, signifies
3089 end-of-input (@pxref{Calling Convention, ,Calling Convention
3090 for @code{yylex}}). Also, unlike standard C, trigraphs have no
3091 special meaning in Bison character literals, nor is backslash-newline
3092 allowed.
3093
3094 @item
3095 @cindex string token
3096 @cindex literal string token
3097 @cindex multicharacter literal
3098 A @dfn{literal string token} is written like a C string constant; for
3099 example, @code{"<="} is a literal string token. A literal string token
3100 doesn't need to be declared unless you need to specify its semantic
3101 value data type (@pxref{Value Type}), associativity, or precedence
3102 (@pxref{Precedence}).
3103
3104 You can associate the literal string token with a symbolic name as an
3105 alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
3106 Declarations}). If you don't do that, the lexical analyzer has to
3107 retrieve the token number for the literal string token from the
3108 @code{yytname} table (@pxref{Calling Convention}).
3109
3110 @strong{Warning}: literal string tokens do not work in Yacc.
3111
3112 By convention, a literal string token is used only to represent a token
3113 that consists of that particular string. Thus, you should use the token
3114 type @code{"<="} to represent the string @samp{<=} as a token. Bison
3115 does not enforce this convention, but if you depart from it, people who
3116 read your program will be confused.
3117
3118 All the escape sequences used in string literals in C can be used in
3119 Bison as well, except that you must not use a null character within a
3120 string literal. Also, unlike Standard C, trigraphs have no special
3121 meaning in Bison string literals, nor is backslash-newline allowed. A
3122 literal string token must contain two or more characters; for a token
3123 containing just one character, use a character token (see above).
3124 @end itemize
3125
3126 How you choose to write a terminal symbol has no effect on its
3127 grammatical meaning. That depends only on where it appears in rules and
3128 on when the parser function returns that symbol.
3129
3130 The value returned by @code{yylex} is always one of the terminal
3131 symbols, except that a zero or negative value signifies end-of-input.
3132 Whichever way you write the token type in the grammar rules, you write
3133 it the same way in the definition of @code{yylex}. The numeric code
3134 for a character token type is simply the positive numeric code of the
3135 character, so @code{yylex} can use the identical value to generate the
3136 requisite code, though you may need to convert it to @code{unsigned
3137 char} to avoid sign-extension on hosts where @code{char} is signed.
3138 Each named token type becomes a C macro in
3139 the parser file, so @code{yylex} can use the name to stand for the code.
3140 (This is why periods don't make sense in terminal symbols.)
3141 @xref{Calling Convention, ,Calling Convention for @code{yylex}}.
3142
3143 If @code{yylex} is defined in a separate file, you need to arrange for the
3144 token-type macro definitions to be available there. Use the @samp{-d}
3145 option when you run Bison, so that it will write these macro definitions
3146 into a separate header file @file{@var{name}.tab.h} which you can include
3147 in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
3148
3149 If you want to write a grammar that is portable to any Standard C
3150 host, you must use only nonnull character tokens taken from the basic
3151 execution character set of Standard C@. This set consists of the ten
3152 digits, the 52 lower- and upper-case English letters, and the
3153 characters in the following C-language string:
3154
3155 @example
3156 "\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3157 @end example
3158
3159 The @code{yylex} function and Bison must use a consistent character set
3160 and encoding for character tokens. For example, if you run Bison in an
3161 ASCII environment, but then compile and run the resulting
3162 program in an environment that uses an incompatible character set like
3163 EBCDIC, the resulting program may not work because the tables
3164 generated by Bison will assume ASCII numeric values for
3165 character tokens. It is standard practice for software distributions to
3166 contain C source files that were generated by Bison in an
3167 ASCII environment, so installers on platforms that are
3168 incompatible with ASCII must rebuild those files before
3169 compiling them.
3170
3171 The symbol @code{error} is a terminal symbol reserved for error recovery
3172 (@pxref{Error Recovery}); you shouldn't use it for any other purpose.
3173 In particular, @code{yylex} should never return this value. The default
3174 value of the error token is 256, unless you explicitly assigned 256 to
3175 one of your tokens with a @code{%token} declaration.
3176
3177 @node Rules
3178 @section Syntax of Grammar Rules
3179 @cindex rule syntax
3180 @cindex grammar rule syntax
3181 @cindex syntax of grammar rules
3182
3183 A Bison grammar rule has the following general form:
3184
3185 @example
3186 @group
3187 @var{result}: @var{components}@dots{}
3188 ;
3189 @end group
3190 @end example
3191
3192 @noindent
3193 where @var{result} is the nonterminal symbol that this rule describes,
3194 and @var{components} are various terminal and nonterminal symbols that
3195 are put together by this rule (@pxref{Symbols}).
3196
3197 For example,
3198
3199 @example
3200 @group
3201 exp: exp '+' exp
3202 ;
3203 @end group
3204 @end example
3205
3206 @noindent
3207 says that two groupings of type @code{exp}, with a @samp{+} token in between,
3208 can be combined into a larger grouping of type @code{exp}.
3209
3210 White space in rules is significant only to separate symbols. You can add
3211 extra white space as you wish.
3212
3213 Scattered among the components can be @var{actions} that determine
3214 the semantics of the rule. An action looks like this:
3215
3216 @example
3217 @{@var{C statements}@}
3218 @end example
3219
3220 @noindent
3221 @cindex braced code
3222 This is an example of @dfn{braced code}, that is, C code surrounded by
3223 braces, much like a compound statement in C@. Braced code can contain
3224 any sequence of C tokens, so long as its braces are balanced. Bison
3225 does not check the braced code for correctness directly; it merely
3226 copies the code to the output file, where the C compiler can check it.
3227
3228 Within braced code, the balanced-brace count is not affected by braces
3229 within comments, string literals, or character constants, but it is
3230 affected by the C digraphs @samp{<%} and @samp{%>} that represent
3231 braces. At the top level braced code must be terminated by @samp{@}}
3232 and not by a digraph. Bison does not look for trigraphs, so if braced
3233 code uses trigraphs you should ensure that they do not affect the
3234 nesting of braces or the boundaries of comments, string literals, or
3235 character constants.
3236
3237 Usually there is only one action and it follows the components.
3238 @xref{Actions}.
3239
3240 @findex |
3241 Multiple rules for the same @var{result} can be written separately or can
3242 be joined with the vertical-bar character @samp{|} as follows:
3243
3244 @example
3245 @group
3246 @var{result}: @var{rule1-components}@dots{}
3247 | @var{rule2-components}@dots{}
3248 @dots{}
3249 ;
3250 @end group
3251 @end example
3252
3253 @noindent
3254 They are still considered distinct rules even when joined in this way.
3255
3256 If @var{components} in a rule is empty, it means that @var{result} can
3257 match the empty string. For example, here is how to define a
3258 comma-separated sequence of zero or more @code{exp} groupings:
3259
3260 @example
3261 @group
3262 expseq: /* empty */
3263 | expseq1
3264 ;
3265 @end group
3266
3267 @group
3268 expseq1: exp
3269 | expseq1 ',' exp
3270 ;
3271 @end group
3272 @end example
3273
3274 @noindent
3275 It is customary to write a comment @samp{/* empty */} in each rule
3276 with no components.
3277
3278 @node Recursion
3279 @section Recursive Rules
3280 @cindex recursive rule
3281
3282 A rule is called @dfn{recursive} when its @var{result} nonterminal
3283 appears also on its right hand side. Nearly all Bison grammars need to
3284 use recursion, because that is the only way to define a sequence of any
3285 number of a particular thing. Consider this recursive definition of a
3286 comma-separated sequence of one or more expressions:
3287
3288 @example
3289 @group
3290 expseq1: exp
3291 | expseq1 ',' exp
3292 ;
3293 @end group
3294 @end example
3295
3296 @cindex left recursion
3297 @cindex right recursion
3298 @noindent
3299 Since the recursive use of @code{expseq1} is the leftmost symbol in the
3300 right hand side, we call this @dfn{left recursion}. By contrast, here
3301 the same construct is defined using @dfn{right recursion}:
3302
3303 @example
3304 @group
3305 expseq1: exp
3306 | exp ',' expseq1
3307 ;
3308 @end group
3309 @end example
3310
3311 @noindent
3312 Any kind of sequence can be defined using either left recursion or right
3313 recursion, but you should always use left recursion, because it can
3314 parse a sequence of any number of elements with bounded stack space.
3315 Right recursion uses up space on the Bison stack in proportion to the
3316 number of elements in the sequence, because all the elements must be
3317 shifted onto the stack before the rule can be applied even once.
3318 @xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3319 of this.
3320
3321 @cindex mutual recursion
3322 @dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3323 rule does not appear directly on its right hand side, but does appear
3324 in rules for other nonterminals which do appear on its right hand
3325 side.
3326
3327 For example:
3328
3329 @example
3330 @group
3331 expr: primary
3332 | primary '+' primary
3333 ;
3334 @end group
3335
3336 @group
3337 primary: constant
3338 | '(' expr ')'
3339 ;
3340 @end group
3341 @end example
3342
3343 @noindent
3344 defines two mutually-recursive nonterminals, since each refers to the
3345 other.
3346
3347 @node Semantics
3348 @section Defining Language Semantics
3349 @cindex defining language semantics
3350 @cindex language semantics, defining
3351
3352 The grammar rules for a language determine only the syntax. The semantics
3353 are determined by the semantic values associated with various tokens and
3354 groupings, and by the actions taken when various groupings are recognized.
3355
3356 For example, the calculator calculates properly because the value
3357 associated with each expression is the proper number; it adds properly
3358 because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3359 the numbers associated with @var{x} and @var{y}.
3360
3361 @menu
3362 * Value Type:: Specifying one data type for all semantic values.
3363 * Multiple Types:: Specifying several alternative data types.
3364 * Actions:: An action is the semantic definition of a grammar rule.
3365 * Action Types:: Specifying data types for actions to operate on.
3366 * Mid-Rule Actions:: Most actions go at the end of a rule.
3367 This says when, why and how to use the exceptional
3368 action in the middle of a rule.
3369 * Named References:: Using named references in actions.
3370 @end menu
3371
3372 @node Value Type
3373 @subsection Data Types of Semantic Values
3374 @cindex semantic value type
3375 @cindex value type, semantic
3376 @cindex data types of semantic values
3377 @cindex default data type
3378
3379 In a simple program it may be sufficient to use the same data type for
3380 the semantic values of all language constructs. This was true in the
3381 RPN and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
3382 Notation Calculator}).
3383
3384 Bison normally uses the type @code{int} for semantic values if your
3385 program uses the same data type for all language constructs. To
3386 specify some other type, define @code{YYSTYPE} as a macro, like this:
3387
3388 @example
3389 #define YYSTYPE double
3390 @end example
3391
3392 @noindent
3393 @code{YYSTYPE}'s replacement list should be a type name
3394 that does not contain parentheses or square brackets.
3395 This macro definition must go in the prologue of the grammar file
3396 (@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
3397
3398 @node Multiple Types
3399 @subsection More Than One Value Type
3400
3401 In most programs, you will need different data types for different kinds
3402 of tokens and groupings. For example, a numeric constant may need type
3403 @code{int} or @code{long int}, while a string constant needs type
3404 @code{char *}, and an identifier might need a pointer to an entry in the
3405 symbol table.
3406
3407 To use more than one data type for semantic values in one parser, Bison
3408 requires you to do two things:
3409
3410 @itemize @bullet
3411 @item
3412 Specify the entire collection of possible data types, either by using the
3413 @code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
3414 Value Types}), or by using a @code{typedef} or a @code{#define} to
3415 define @code{YYSTYPE} to be a union type whose member names are
3416 the type tags.
3417
3418 @item
3419 Choose one of those types for each symbol (terminal or nonterminal) for
3420 which semantic values are used. This is done for tokens with the
3421 @code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3422 and for groupings with the @code{%type} Bison declaration (@pxref{Type
3423 Decl, ,Nonterminal Symbols}).
3424 @end itemize
3425
3426 @node Actions
3427 @subsection Actions
3428 @cindex action
3429 @vindex $$
3430 @vindex $@var{n}
3431 @vindex $@var{name}
3432 @vindex $[@var{name}]
3433
3434 An action accompanies a syntactic rule and contains C code to be executed
3435 each time an instance of that rule is recognized. The task of most actions
3436 is to compute a semantic value for the grouping built by the rule from the
3437 semantic values associated with tokens or smaller groupings.
3438
3439 An action consists of braced code containing C statements, and can be
3440 placed at any position in the rule;
3441 it is executed at that position. Most rules have just one action at the
3442 end of the rule, following all the components. Actions in the middle of
3443 a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3444 Actions, ,Actions in Mid-Rule}).
3445
3446 The C code in an action can refer to the semantic values of the components
3447 matched by the rule with the construct @code{$@var{n}}, which stands for
3448 the value of the @var{n}th component. The semantic value for the grouping
3449 being constructed is @code{$$}. In addition, the semantic values of
3450 symbols can be accessed with the named references construct
3451 @code{$@var{name}} or @code{$[@var{name}]}. Bison translates both of these
3452 constructs into expressions of the appropriate type when it copies the
3453 actions into the parser file. @code{$$} (or @code{$@var{name}}, when it
3454 stands for the current grouping) is translated to a modifiable
3455 lvalue, so it can be assigned to.
3456
3457 Here is a typical example:
3458
3459 @example
3460 @group
3461 exp: @dots{}
3462 | exp '+' exp
3463 @{ $$ = $1 + $3; @}
3464 @end group
3465 @end example
3466
3467 Or, in terms of named references:
3468
3469 @example
3470 @group
3471 exp[result]: @dots{}
3472 | exp[left] '+' exp[right]
3473 @{ $result = $left + $right; @}
3474 @end group
3475 @end example
3476
3477 @noindent
3478 This rule constructs an @code{exp} from two smaller @code{exp} groupings
3479 connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3480 (@code{$left} and @code{$right})
3481 refer to the semantic values of the two component @code{exp} groupings,
3482 which are the first and third symbols on the right hand side of the rule.
3483 The sum is stored into @code{$$} (@code{$result}) so that it becomes the
3484 semantic value of
3485 the addition-expression just recognized by the rule. If there were a
3486 useful semantic value associated with the @samp{+} token, it could be
3487 referred to as @code{$2}.
3488
3489 @xref{Named References,,Using Named References}, for more information
3490 about using the named references construct.
3491
3492 Note that the vertical-bar character @samp{|} is really a rule
3493 separator, and actions are attached to a single rule. This is a
3494 difference with tools like Flex, for which @samp{|} stands for either
3495 ``or'', or ``the same action as that of the next rule''. In the
3496 following example, the action is triggered only when @samp{b} is found:
3497
3498 @example
3499 @group
3500 a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3501 @end group
3502 @end example
3503
3504 @cindex default action
3505 If you don't specify an action for a rule, Bison supplies a default:
3506 @w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3507 becomes the value of the whole rule. Of course, the default action is
3508 valid only if the two data types match. There is no meaningful default
3509 action for an empty rule; every empty rule must have an explicit action
3510 unless the rule's value does not matter.
3511
3512 @code{$@var{n}} with @var{n} zero or negative is allowed for reference
3513 to tokens and groupings on the stack @emph{before} those that match the
3514 current rule. This is a very risky practice, and to use it reliably
3515 you must be certain of the context in which the rule is applied. Here
3516 is a case in which you can use this reliably:
3517
3518 @example
3519 @group
3520 foo: expr bar '+' expr @{ @dots{} @}
3521 | expr bar '-' expr @{ @dots{} @}
3522 ;
3523 @end group
3524
3525 @group
3526 bar: /* empty */
3527 @{ previous_expr = $0; @}
3528 ;
3529 @end group
3530 @end example
3531
3532 As long as @code{bar} is used only in the fashion shown here, @code{$0}
3533 always refers to the @code{expr} which precedes @code{bar} in the
3534 definition of @code{foo}.
3535
3536 @vindex yylval
3537 It is also possible to access the semantic value of the lookahead token, if
3538 any, from a semantic action.
3539 This semantic value is stored in @code{yylval}.
3540 @xref{Action Features, ,Special Features for Use in Actions}.
3541
3542 @node Action Types
3543 @subsection Data Types of Values in Actions
3544 @cindex action data types
3545 @cindex data types in actions
3546
3547 If you have chosen a single data type for semantic values, the @code{$$}
3548 and @code{$@var{n}} constructs always have that data type.
3549
3550 If you have used @code{%union} to specify a variety of data types, then you
3551 must declare a choice among these types for each terminal or nonterminal
3552 symbol that can have a semantic value. Then each time you use @code{$$} or
3553 @code{$@var{n}}, its data type is determined by which symbol it refers to
3554 in the rule. In this example,
3555
3556 @example
3557 @group
3558 exp: @dots{}
3559 | exp '+' exp
3560 @{ $$ = $1 + $3; @}
3561 @end group
3562 @end example
3563
3564 @noindent
3565 @code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3566 have the data type declared for the nonterminal symbol @code{exp}. If
3567 @code{$2} were used, it would have the data type declared for the
3568 terminal symbol @code{'+'}, whatever that might be.
3569
3570 Alternatively, you can specify the data type when you refer to the value,
3571 by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3572 reference. For example, if you have defined types as shown here:
3573
3574 @example
3575 @group
3576 %union @{
3577 int itype;
3578 double dtype;
3579 @}
3580 @end group
3581 @end example
3582
3583 @noindent
3584 then you can write @code{$<itype>1} to refer to the first subunit of the
3585 rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3586
3587 @node Mid-Rule Actions
3588 @subsection Actions in Mid-Rule
3589 @cindex actions in mid-rule
3590 @cindex mid-rule actions
3591
3592 Occasionally it is useful to put an action in the middle of a rule.
3593 These actions are written just like usual end-of-rule actions, but they
3594 are executed before the parser even recognizes the following components.
3595
3596 A mid-rule action may refer to the components preceding it using
3597 @code{$@var{n}}, but it may not refer to subsequent components because
3598 it is run before they are parsed.
3599
3600 The mid-rule action itself counts as one of the components of the rule.
3601 This makes a difference when there is another action later in the same rule
3602 (and usually there is another at the end): you have to count the actions
3603 along with the symbols when working out which number @var{n} to use in
3604 @code{$@var{n}}.
3605
3606 The mid-rule action can also have a semantic value. The action can set
3607 its value with an assignment to @code{$$}, and actions later in the rule
3608 can refer to the value using @code{$@var{n}}. Since there is no symbol
3609 to name the action, there is no way to declare a data type for the value
3610 in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3611 specify a data type each time you refer to this value.
3612
3613 There is no way to set the value of the entire rule with a mid-rule
3614 action, because assignments to @code{$$} do not have that effect. The
3615 only way to set the value for the entire rule is with an ordinary action
3616 at the end of the rule.
3617
3618 Here is an example from a hypothetical compiler, handling a @code{let}
3619 statement that looks like @samp{let (@var{variable}) @var{statement}} and
3620 serves to create a variable named @var{variable} temporarily for the
3621 duration of @var{statement}. To parse this construct, we must put
3622 @var{variable} into the symbol table while @var{statement} is parsed, then
3623 remove it afterward. Here is how it is done:
3624
3625 @example
3626 @group
3627 stmt: LET '(' var ')'
3628 @{ $<context>$ = push_context ();
3629 declare_variable ($3); @}
3630 stmt @{ $$ = $6;
3631 pop_context ($<context>5); @}
3632 @end group
3633 @end example
3634
3635 @noindent
3636 As soon as @samp{let (@var{variable})} has been recognized, the first
3637 action is run. It saves a copy of the current semantic context (the
3638 list of accessible variables) as its semantic value, using alternative
3639 @code{context} in the data-type union. Then it calls
3640 @code{declare_variable} to add the new variable to that list. Once the
3641 first action is finished, the embedded statement @code{stmt} can be
3642 parsed. Note that the mid-rule action is component number 5, so the
3643 @samp{stmt} is component number 6.
3644
3645 After the embedded statement is parsed, its semantic value becomes the
3646 value of the entire @code{let}-statement. Then the semantic value from the
3647 earlier action is used to restore the prior list of variables. This
3648 removes the temporary @code{let}-variable from the list so that it won't
3649 appear to exist while the rest of the program is parsed.
3650
3651 @findex %destructor
3652 @cindex discarded symbols, mid-rule actions
3653 @cindex error recovery, mid-rule actions
3654 In the above example, if the parser initiates error recovery (@pxref{Error
3655 Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3656 it might discard the previous semantic context @code{$<context>5} without
3657 restoring it.
3658 Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3659 Discarded Symbols}).
3660 However, Bison currently provides no means to declare a destructor specific to
3661 a particular mid-rule action's semantic value.
3662
3663 One solution is to bury the mid-rule action inside a nonterminal symbol and to
3664 declare a destructor for that symbol:
3665
3666 @example
3667 @group
3668 %type <context> let
3669 %destructor @{ pop_context ($$); @} let
3670
3671 %%
3672
3673 stmt: let stmt
3674 @{ $$ = $2;
3675 pop_context ($1); @}
3676 ;
3677
3678 let: LET '(' var ')'
3679 @{ $$ = push_context ();
3680 declare_variable ($3); @}
3681 ;
3682
3683 @end group
3684 @end example
3685
3686 @noindent
3687 Note that the action is now at the end of its rule.
3688 Any mid-rule action can be converted to an end-of-rule action in this way, and
3689 this is what Bison actually does to implement mid-rule actions.
3690
3691 Taking action before a rule is completely recognized often leads to
3692 conflicts since the parser must commit to a parse in order to execute the
3693 action. For example, the following two rules, without mid-rule actions,
3694 can coexist in a working parser because the parser can shift the open-brace
3695 token and look at what follows before deciding whether there is a
3696 declaration or not:
3697
3698 @example
3699 @group
3700 compound: '@{' declarations statements '@}'
3701 | '@{' statements '@}'
3702 ;
3703 @end group
3704 @end example
3705
3706 @noindent
3707 But when we add a mid-rule action as follows, the rules become nonfunctional:
3708
3709 @example
3710 @group
3711 compound: @{ prepare_for_local_variables (); @}
3712 '@{' declarations statements '@}'
3713 @end group
3714 @group
3715 | '@{' statements '@}'
3716 ;
3717 @end group
3718 @end example
3719
3720 @noindent
3721 Now the parser is forced to decide whether to run the mid-rule action
3722 when it has read no farther than the open-brace. In other words, it
3723 must commit to using one rule or the other, without sufficient
3724 information to do it correctly. (The open-brace token is what is called
3725 the @dfn{lookahead} token at this time, since the parser is still
3726 deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.)
3727
3728 You might think that you could correct the problem by putting identical
3729 actions into the two rules, like this:
3730
3731 @example
3732 @group
3733 compound: @{ prepare_for_local_variables (); @}
3734 '@{' declarations statements '@}'
3735 | @{ prepare_for_local_variables (); @}
3736 '@{' statements '@}'
3737 ;
3738 @end group
3739 @end example
3740
3741 @noindent
3742 But this does not help, because Bison does not realize that the two actions
3743 are identical. (Bison never tries to understand the C code in an action.)
3744
3745 If the grammar is such that a declaration can be distinguished from a
3746 statement by the first token (which is true in C), then one solution which
3747 does work is to put the action after the open-brace, like this:
3748
3749 @example
3750 @group
3751 compound: '@{' @{ prepare_for_local_variables (); @}
3752 declarations statements '@}'
3753 | '@{' statements '@}'
3754 ;
3755 @end group
3756 @end example
3757
3758 @noindent
3759 Now the first token of the following declaration or statement,
3760 which would in any case tell Bison which rule to use, can still do so.
3761
3762 Another solution is to bury the action inside a nonterminal symbol which
3763 serves as a subroutine:
3764
3765 @example
3766 @group
3767 subroutine: /* empty */
3768 @{ prepare_for_local_variables (); @}
3769 ;
3770
3771 @end group
3772
3773 @group
3774 compound: subroutine
3775 '@{' declarations statements '@}'
3776 | subroutine
3777 '@{' statements '@}'
3778 ;
3779 @end group
3780 @end example
3781
3782 @noindent
3783 Now Bison can execute the action in the rule for @code{subroutine} without
3784 deciding which rule for @code{compound} it will eventually use.
3785
3786 @node Named References
3787 @subsection Using Named References
3788 @cindex named references
3789
3790 While every semantic value can be accessed with positional references
3791 @code{$@var{n}} and @code{$$}, it's often much more convenient to refer to
3792 them by name. First of all, original symbol names may be used as named
3793 references. For example:
3794
3795 @example
3796 @group
3797 invocation: op '(' args ')'
3798 @{ $invocation = new_invocation ($op, $args, @@invocation); @}
3799 @end group
3800 @end example
3801
3802 @noindent
3803 The positional @code{$$}, @code{@@$}, @code{$n}, and @code{@@n} can be
3804 mixed with @code{$name} and @code{@@name} arbitrarily. For example:
3805
3806 @example
3807 @group
3808 invocation: op '(' args ')'
3809 @{ $$ = new_invocation ($op, $args, @@$); @}
3810 @end group
3811 @end example
3812
3813 @noindent
3814 However, sometimes regular symbol names are not sufficient due to
3815 ambiguities:
3816
3817 @example
3818 @group
3819 exp: exp '/' exp
3820 @{ $exp = $exp / $exp; @} // $exp is ambiguous.
3821
3822 exp: exp '/' exp
3823 @{ $$ = $1 / $exp; @} // One usage is ambiguous.
3824
3825 exp: exp '/' exp
3826 @{ $$ = $1 / $3; @} // No error.
3827 @end group
3828 @end example
3829
3830 @noindent
3831 When ambiguity occurs, explicitly declared names may be used for values and
3832 locations. Explicit names are declared as a bracketed name after a symbol
3833 appearance in rule definitions. For example:
3834 @example
3835 @group
3836 exp[result]: exp[left] '/' exp[right]
3837 @{ $result = $left / $right; @}
3838 @end group
3839 @end example
3840
3841 @noindent
3842 Explicit names may be declared for RHS and for LHS symbols as well. In order
3843 to access a semantic value generated by a mid-rule action, an explicit name
3844 may also be declared by putting a bracketed name after the closing brace of
3845 the mid-rule action code:
3846 @example
3847 @group
3848 exp[res]: exp[x] '+' @{$left = $x;@}[left] exp[right]
3849 @{ $res = $left + $right; @}
3850 @end group
3851 @end example
3852
3853 @noindent
3854
3855 In references, in order to specify names containing dots and dashes, an explicit
3856 bracketed syntax @code{$[name]} and @code{@@[name]} must be used:
3857 @example
3858 @group
3859 if-stmt: IF '(' expr ')' THEN then.stmt ';'
3860 @{ $[if-stmt] = new_if_stmt ($expr, $[then.stmt]); @}
3861 @end group
3862 @end example
3863
3864 It often happens that named references are followed by a dot, dash or other
3865 C punctuation marks and operators. By default, Bison will read
3866 @code{$name.suffix} as a reference to symbol value @code{$name} followed by
3867 @samp{.suffix}, i.e., an access to the @samp{suffix} field of the semantic
3868 value. In order to force Bison to recognize @code{name.suffix} in its entirety
3869 as the name of a semantic value, bracketed syntax @code{$[name.suffix]}
3870 must be used.
3871
3872
3873 @node Locations
3874 @section Tracking Locations
3875 @cindex location
3876 @cindex textual location
3877 @cindex location, textual
3878
3879 Though grammar rules and semantic actions are enough to write a fully
3880 functional parser, it can be useful to process some additional information,
3881 especially symbol locations.
3882
3883 The way locations are handled is defined by providing a data type, and
3884 actions to take when rules are matched.
3885
3886 @menu
3887 * Location Type:: Specifying a data type for locations.
3888 * Actions and Locations:: Using locations in actions.
3889 * Location Default Action:: Defining a general way to compute locations.
3890 @end menu
3891
3892 @node Location Type
3893 @subsection Data Type of Locations
3894 @cindex data type of locations
3895 @cindex default location type
3896
3897 Defining a data type for locations is much simpler than for semantic values,
3898 since all tokens and groupings always use the same type.
3899
3900 You can specify the type of locations by defining a macro called
3901 @code{YYLTYPE}, just as you can specify the semantic value type by
3902 defining a @code{YYSTYPE} macro (@pxref{Value Type}).
3903 When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3904 four members:
3905
3906 @example
3907 typedef struct YYLTYPE
3908 @{
3909 int first_line;
3910 int first_column;
3911 int last_line;
3912 int last_column;
3913 @} YYLTYPE;
3914 @end example
3915
3916 When @code{YYLTYPE} is not defined, at the beginning of the parsing, Bison
3917 initializes all these fields to 1 for @code{yylloc}. To initialize
3918 @code{yylloc} with a custom location type (or to chose a different
3919 initialization), use the @code{%initial-action} directive. @xref{Initial
3920 Action Decl, , Performing Actions before Parsing}.
3921
3922 @node Actions and Locations
3923 @subsection Actions and Locations
3924 @cindex location actions
3925 @cindex actions, location
3926 @vindex @@$
3927 @vindex @@@var{n}
3928 @vindex @@@var{name}
3929 @vindex @@[@var{name}]
3930
3931 Actions are not only useful for defining language semantics, but also for
3932 describing the behavior of the output parser with locations.
3933
3934 The most obvious way for building locations of syntactic groupings is very
3935 similar to the way semantic values are computed. In a given rule, several
3936 constructs can be used to access the locations of the elements being matched.
3937 The location of the @var{n}th component of the right hand side is
3938 @code{@@@var{n}}, while the location of the left hand side grouping is
3939 @code{@@$}.
3940
3941 In addition, the named references construct @code{@@@var{name}} and
3942 @code{@@[@var{name}]} may also be used to address the symbol locations.
3943 @xref{Named References,,Using Named References}, for more information
3944 about using the named references construct.
3945
3946 Here is a basic example using the default data type for locations:
3947
3948 @example
3949 @group
3950 exp: @dots{}
3951 | exp '/' exp
3952 @{
3953 @@$.first_column = @@1.first_column;
3954 @@$.first_line = @@1.first_line;
3955 @@$.last_column = @@3.last_column;
3956 @@$.last_line = @@3.last_line;
3957 if ($3)
3958 $$ = $1 / $3;
3959 else
3960 @{
3961 $$ = 1;
3962 fprintf (stderr,
3963 "Division by zero, l%d,c%d-l%d,c%d",
3964 @@3.first_line, @@3.first_column,
3965 @@3.last_line, @@3.last_column);
3966 @}
3967 @}
3968 @end group
3969 @end example
3970
3971 As for semantic values, there is a default action for locations that is
3972 run each time a rule is matched. It sets the beginning of @code{@@$} to the
3973 beginning of the first symbol, and the end of @code{@@$} to the end of the
3974 last symbol.
3975
3976 With this default action, the location tracking can be fully automatic. The
3977 example above simply rewrites this way:
3978
3979 @example
3980 @group
3981 exp: @dots{}
3982 | exp '/' exp
3983 @{
3984 if ($3)
3985 $$ = $1 / $3;
3986 else
3987 @{
3988 $$ = 1;
3989 fprintf (stderr,
3990 "Division by zero, l%d,c%d-l%d,c%d",
3991 @@3.first_line, @@3.first_column,
3992 @@3.last_line, @@3.last_column);
3993 @}
3994 @}
3995 @end group
3996 @end example
3997
3998 @vindex yylloc
3999 It is also possible to access the location of the lookahead token, if any,
4000 from a semantic action.
4001 This location is stored in @code{yylloc}.
4002 @xref{Action Features, ,Special Features for Use in Actions}.
4003
4004 @node Location Default Action
4005 @subsection Default Action for Locations
4006 @vindex YYLLOC_DEFAULT
4007 @cindex GLR parsers and @code{YYLLOC_DEFAULT}
4008
4009 Actually, actions are not the best place to compute locations. Since
4010 locations are much more general than semantic values, there is room in
4011 the output parser to redefine the default action to take for each
4012 rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
4013 matched, before the associated action is run. It is also invoked
4014 while processing a syntax error, to compute the error's location.
4015 Before reporting an unresolvable syntactic ambiguity, a GLR
4016 parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
4017 of that ambiguity.
4018
4019 Most of the time, this macro is general enough to suppress location
4020 dedicated code from semantic actions.
4021
4022 The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
4023 the location of the grouping (the result of the computation). When a
4024 rule is matched, the second parameter identifies locations of
4025 all right hand side elements of the rule being matched, and the third
4026 parameter is the size of the rule's right hand side.
4027 When a GLR parser reports an ambiguity, which of multiple candidate
4028 right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
4029 When processing a syntax error, the second parameter identifies locations
4030 of the symbols that were discarded during error processing, and the third
4031 parameter is the number of discarded symbols.
4032
4033 By default, @code{YYLLOC_DEFAULT} is defined this way:
4034
4035 @smallexample
4036 @group
4037 # define YYLLOC_DEFAULT(Current, Rhs, N) \
4038 do \
4039 if (N) \
4040 @{ \
4041 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
4042 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
4043 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
4044 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
4045 @} \
4046 else \
4047 @{ \
4048 (Current).first_line = (Current).last_line = \
4049 YYRHSLOC(Rhs, 0).last_line; \
4050 (Current).first_column = (Current).last_column = \
4051 YYRHSLOC(Rhs, 0).last_column; \
4052 @} \
4053 while (0)
4054 @end group
4055 @end smallexample
4056
4057 where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
4058 in @var{rhs} when @var{k} is positive, and the location of the symbol
4059 just before the reduction when @var{k} and @var{n} are both zero.
4060
4061 When defining @code{YYLLOC_DEFAULT}, you should consider that:
4062
4063 @itemize @bullet
4064 @item
4065 All arguments are free of side-effects. However, only the first one (the
4066 result) should be modified by @code{YYLLOC_DEFAULT}.
4067
4068 @item
4069 For consistency with semantic actions, valid indexes within the
4070 right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
4071 valid index, and it refers to the symbol just before the reduction.
4072 During error processing @var{n} is always positive.
4073
4074 @item
4075 Your macro should parenthesize its arguments, if need be, since the
4076 actual arguments may not be surrounded by parentheses. Also, your
4077 macro should expand to something that can be used as a single
4078 statement when it is followed by a semicolon.
4079 @end itemize
4080
4081 @node Declarations
4082 @section Bison Declarations
4083 @cindex declarations, Bison
4084 @cindex Bison declarations
4085
4086 The @dfn{Bison declarations} section of a Bison grammar defines the symbols
4087 used in formulating the grammar and the data types of semantic values.
4088 @xref{Symbols}.
4089
4090 All token type names (but not single-character literal tokens such as
4091 @code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
4092 declared if you need to specify which data type to use for the semantic
4093 value (@pxref{Multiple Types, ,More Than One Value Type}).
4094
4095 The first rule in the file also specifies the start symbol, by default.
4096 If you want some other symbol to be the start symbol, you must declare
4097 it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
4098 Grammars}).
4099
4100 @menu
4101 * Require Decl:: Requiring a Bison version.
4102 * Token Decl:: Declaring terminal symbols.
4103 * Precedence Decl:: Declaring terminals with precedence and associativity.
4104 * Union Decl:: Declaring the set of all semantic value types.
4105 * Type Decl:: Declaring the choice of type for a nonterminal symbol.
4106 * Initial Action Decl:: Code run before parsing starts.
4107 * Destructor Decl:: Declaring how symbols are freed.
4108 * Expect Decl:: Suppressing warnings about parsing conflicts.
4109 * Start Decl:: Specifying the start symbol.
4110 * Pure Decl:: Requesting a reentrant parser.
4111 * Push Decl:: Requesting a push parser.
4112 * Decl Summary:: Table of all Bison declarations.
4113 @end menu
4114
4115 @node Require Decl
4116 @subsection Require a Version of Bison
4117 @cindex version requirement
4118 @cindex requiring a version of Bison
4119 @findex %require
4120
4121 You may require the minimum version of Bison to process the grammar. If
4122 the requirement is not met, @command{bison} exits with an error (exit
4123 status 63).
4124
4125 @example
4126 %require "@var{version}"
4127 @end example
4128
4129 @node Token Decl
4130 @subsection Token Type Names
4131 @cindex declaring token type names
4132 @cindex token type names, declaring
4133 @cindex declaring literal string tokens
4134 @findex %token
4135
4136 The basic way to declare a token type name (terminal symbol) is as follows:
4137
4138 @example
4139 %token @var{name}
4140 @end example
4141
4142 Bison will convert this into a @code{#define} directive in
4143 the parser, so that the function @code{yylex} (if it is in this file)
4144 can use the name @var{name} to stand for this token type's code.
4145
4146 Alternatively, you can use @code{%left}, @code{%right}, or
4147 @code{%nonassoc} instead of @code{%token}, if you wish to specify
4148 associativity and precedence. @xref{Precedence Decl, ,Operator
4149 Precedence}.
4150
4151 You can explicitly specify the numeric code for a token type by appending
4152 a nonnegative decimal or hexadecimal integer value in the field immediately
4153 following the token name:
4154
4155 @example
4156 %token NUM 300
4157 %token XNUM 0x12d // a GNU extension
4158 @end example
4159
4160 @noindent
4161 It is generally best, however, to let Bison choose the numeric codes for
4162 all token types. Bison will automatically select codes that don't conflict
4163 with each other or with normal characters.
4164
4165 In the event that the stack type is a union, you must augment the
4166 @code{%token} or other token declaration to include the data type
4167 alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
4168 Than One Value Type}).
4169
4170 For example:
4171
4172 @example
4173 @group
4174 %union @{ /* define stack type */
4175 double val;
4176 symrec *tptr;
4177 @}
4178 %token <val> NUM /* define token NUM and its type */
4179 @end group
4180 @end example
4181
4182 You can associate a literal string token with a token type name by
4183 writing the literal string at the end of a @code{%token}
4184 declaration which declares the name. For example:
4185
4186 @example
4187 %token arrow "=>"
4188 @end example
4189
4190 @noindent
4191 For example, a grammar for the C language might specify these names with
4192 equivalent literal string tokens:
4193
4194 @example
4195 %token <operator> OR "||"
4196 %token <operator> LE 134 "<="
4197 %left OR "<="
4198 @end example
4199
4200 @noindent
4201 Once you equate the literal string and the token name, you can use them
4202 interchangeably in further declarations or the grammar rules. The
4203 @code{yylex} function can use the token name or the literal string to
4204 obtain the token type code number (@pxref{Calling Convention}).
4205 Syntax error messages passed to @code{yyerror} from the parser will reference
4206 the literal string instead of the token name.
4207
4208 The token numbered as 0 corresponds to end of file; the following line
4209 allows for nicer error messages referring to ``end of file'' instead
4210 of ``$end'':
4211
4212 @example
4213 %token END 0 "end of file"
4214 @end example
4215
4216 @node Precedence Decl
4217 @subsection Operator Precedence
4218 @cindex precedence declarations
4219 @cindex declaring operator precedence
4220 @cindex operator precedence, declaring
4221
4222 Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to
4223 declare a token and specify its precedence and associativity, all at
4224 once. These are called @dfn{precedence declarations}.
4225 @xref{Precedence, ,Operator Precedence}, for general information on
4226 operator precedence.
4227
4228 The syntax of a precedence declaration is nearly the same as that of
4229 @code{%token}: either
4230
4231 @example
4232 %left @var{symbols}@dots{}
4233 @end example
4234
4235 @noindent
4236 or
4237
4238 @example
4239 %left <@var{type}> @var{symbols}@dots{}
4240 @end example
4241
4242 And indeed any of these declarations serves the purposes of @code{%token}.
4243 But in addition, they specify the associativity and relative precedence for
4244 all the @var{symbols}:
4245
4246 @itemize @bullet
4247 @item
4248 The associativity of an operator @var{op} determines how repeated uses
4249 of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
4250 @var{z}} is parsed by grouping @var{x} with @var{y} first or by
4251 grouping @var{y} with @var{z} first. @code{%left} specifies
4252 left-associativity (grouping @var{x} with @var{y} first) and
4253 @code{%right} specifies right-associativity (grouping @var{y} with
4254 @var{z} first). @code{%nonassoc} specifies no associativity, which
4255 means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
4256 considered a syntax error.
4257
4258 @item
4259 The precedence of an operator determines how it nests with other operators.
4260 All the tokens declared in a single precedence declaration have equal
4261 precedence and nest together according to their associativity.
4262 When two tokens declared in different precedence declarations associate,
4263 the one declared later has the higher precedence and is grouped first.
4264 @end itemize
4265
4266 For backward compatibility, there is a confusing difference between the
4267 argument lists of @code{%token} and precedence declarations.
4268 Only a @code{%token} can associate a literal string with a token type name.
4269 A precedence declaration always interprets a literal string as a reference to a
4270 separate token.
4271 For example:
4272
4273 @example
4274 %left OR "<=" // Does not declare an alias.
4275 %left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
4276 @end example
4277
4278 @node Union Decl
4279 @subsection The Collection of Value Types
4280 @cindex declaring value types
4281 @cindex value types, declaring
4282 @findex %union
4283
4284 The @code{%union} declaration specifies the entire collection of
4285 possible data types for semantic values. The keyword @code{%union} is
4286 followed by braced code containing the same thing that goes inside a
4287 @code{union} in C@.
4288
4289 For example:
4290
4291 @example
4292 @group
4293 %union @{
4294 double val;
4295 symrec *tptr;
4296 @}
4297 @end group
4298 @end example
4299
4300 @noindent
4301 This says that the two alternative types are @code{double} and @code{symrec
4302 *}. They are given names @code{val} and @code{tptr}; these names are used
4303 in the @code{%token} and @code{%type} declarations to pick one of the types
4304 for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
4305
4306 As an extension to POSIX, a tag is allowed after the
4307 @code{union}. For example:
4308
4309 @example
4310 @group
4311 %union value @{
4312 double val;
4313 symrec *tptr;
4314 @}
4315 @end group
4316 @end example
4317
4318 @noindent
4319 specifies the union tag @code{value}, so the corresponding C type is
4320 @code{union value}. If you do not specify a tag, it defaults to
4321 @code{YYSTYPE}.
4322
4323 As another extension to POSIX, you may specify multiple
4324 @code{%union} declarations; their contents are concatenated. However,
4325 only the first @code{%union} declaration can specify a tag.
4326
4327 Note that, unlike making a @code{union} declaration in C, you need not write
4328 a semicolon after the closing brace.
4329
4330 Instead of @code{%union}, you can define and use your own union type
4331 @code{YYSTYPE} if your grammar contains at least one
4332 @samp{<@var{type}>} tag. For example, you can put the following into
4333 a header file @file{parser.h}:
4334
4335 @example
4336 @group
4337 union YYSTYPE @{
4338 double val;
4339 symrec *tptr;
4340 @};
4341 typedef union YYSTYPE YYSTYPE;
4342 @end group
4343 @end example
4344
4345 @noindent
4346 and then your grammar can use the following
4347 instead of @code{%union}:
4348
4349 @example
4350 @group
4351 %@{
4352 #include "parser.h"
4353 %@}
4354 %type <val> expr
4355 %token <tptr> ID
4356 @end group
4357 @end example
4358
4359 @node Type Decl
4360 @subsection Nonterminal Symbols
4361 @cindex declaring value types, nonterminals
4362 @cindex value types, nonterminals, declaring
4363 @findex %type
4364
4365 @noindent
4366 When you use @code{%union} to specify multiple value types, you must
4367 declare the value type of each nonterminal symbol for which values are
4368 used. This is done with a @code{%type} declaration, like this:
4369
4370 @example
4371 %type <@var{type}> @var{nonterminal}@dots{}
4372 @end example
4373
4374 @noindent
4375 Here @var{nonterminal} is the name of a nonterminal symbol, and
4376 @var{type} is the name given in the @code{%union} to the alternative
4377 that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
4378 can give any number of nonterminal symbols in the same @code{%type}
4379 declaration, if they have the same value type. Use spaces to separate
4380 the symbol names.
4381
4382 You can also declare the value type of a terminal symbol. To do this,
4383 use the same @code{<@var{type}>} construction in a declaration for the
4384 terminal symbol. All kinds of token declarations allow
4385 @code{<@var{type}>}.
4386
4387 @node Initial Action Decl
4388 @subsection Performing Actions before Parsing
4389 @findex %initial-action
4390
4391 Sometimes your parser needs to perform some initializations before
4392 parsing. The @code{%initial-action} directive allows for such arbitrary
4393 code.
4394
4395 @deffn {Directive} %initial-action @{ @var{code} @}
4396 @findex %initial-action
4397 Declare that the braced @var{code} must be invoked before parsing each time
4398 @code{yyparse} is called. The @var{code} may use @code{$$} and
4399 @code{@@$} --- initial value and location of the lookahead --- and the
4400 @code{%parse-param}.
4401 @end deffn
4402
4403 For instance, if your locations use a file name, you may use
4404
4405 @example
4406 %parse-param @{ char const *file_name @};
4407 %initial-action
4408 @{
4409 @@$.initialize (file_name);
4410 @};
4411 @end example
4412
4413
4414 @node Destructor Decl
4415 @subsection Freeing Discarded Symbols
4416 @cindex freeing discarded symbols
4417 @findex %destructor
4418 @findex <*>
4419 @findex <>
4420 During error recovery (@pxref{Error Recovery}), symbols already pushed
4421 on the stack and tokens coming from the rest of the file are discarded
4422 until the parser falls on its feet. If the parser runs out of memory,
4423 or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
4424 symbols on the stack must be discarded. Even if the parser succeeds, it
4425 must discard the start symbol.
4426
4427 When discarded symbols convey heap based information, this memory is
4428 lost. While this behavior can be tolerable for batch parsers, such as
4429 in traditional compilers, it is unacceptable for programs like shells or
4430 protocol implementations that may parse and execute indefinitely.
4431
4432 The @code{%destructor} directive defines code that is called when a
4433 symbol is automatically discarded.
4434
4435 @deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
4436 @findex %destructor
4437 Invoke the braced @var{code} whenever the parser discards one of the
4438 @var{symbols}.
4439 Within @var{code}, @code{$$} designates the semantic value associated
4440 with the discarded symbol, and @code{@@$} designates its location.
4441 The additional parser parameters are also available (@pxref{Parser Function, ,
4442 The Parser Function @code{yyparse}}).
4443
4444 When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
4445 per-symbol @code{%destructor}.
4446 You may also define a per-type @code{%destructor} by listing a semantic type
4447 tag among @var{symbols}.
4448 In that case, the parser will invoke this @var{code} whenever it discards any
4449 grammar symbol that has that semantic type tag unless that symbol has its own
4450 per-symbol @code{%destructor}.
4451
4452 Finally, you can define two different kinds of default @code{%destructor}s.
4453 (These default forms are experimental.
4454 More user feedback will help to determine whether they should become permanent
4455 features.)
4456 You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
4457 exactly one @code{%destructor} declaration in your grammar file.
4458 The parser will invoke the @var{code} associated with one of these whenever it
4459 discards any user-defined grammar symbol that has no per-symbol and no per-type
4460 @code{%destructor}.
4461 The parser uses the @var{code} for @code{<*>} in the case of such a grammar
4462 symbol for which you have formally declared a semantic type tag (@code{%type}
4463 counts as such a declaration, but @code{$<tag>$} does not).
4464 The parser uses the @var{code} for @code{<>} in the case of such a grammar
4465 symbol that has no declared semantic type tag.
4466 @end deffn
4467
4468 @noindent
4469 For example:
4470
4471 @smallexample
4472 %union @{ char *string; @}
4473 %token <string> STRING1
4474 %token <string> STRING2
4475 %type <string> string1
4476 %type <string> string2
4477 %union @{ char character; @}
4478 %token <character> CHR
4479 %type <character> chr
4480 %token TAGLESS
4481
4482 %destructor @{ @} <character>
4483 %destructor @{ free ($$); @} <*>
4484 %destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
4485 %destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
4486 @end smallexample
4487
4488 @noindent
4489 guarantees that, when the parser discards any user-defined symbol that has a
4490 semantic type tag other than @code{<character>}, it passes its semantic value
4491 to @code{free} by default.
4492 However, when the parser discards a @code{STRING1} or a @code{string1}, it also
4493 prints its line number to @code{stdout}.
4494 It performs only the second @code{%destructor} in this case, so it invokes
4495 @code{free} only once.
4496 Finally, the parser merely prints a message whenever it discards any symbol,
4497 such as @code{TAGLESS}, that has no semantic type tag.
4498
4499 A Bison-generated parser invokes the default @code{%destructor}s only for
4500 user-defined as opposed to Bison-defined symbols.
4501 For example, the parser will not invoke either kind of default
4502 @code{%destructor} for the special Bison-defined symbols @code{$accept},
4503 @code{$undefined}, or @code{$end} (@pxref{Table of Symbols, ,Bison Symbols}),
4504 none of which you can reference in your grammar.
4505 It also will not invoke either for the @code{error} token (@pxref{Table of
4506 Symbols, ,error}), which is always defined by Bison regardless of whether you
4507 reference it in your grammar.
4508 However, it may invoke one of them for the end token (token 0) if you
4509 redefine it from @code{$end} to, for example, @code{END}:
4510
4511 @smallexample
4512 %token END 0
4513 @end smallexample
4514
4515 @cindex actions in mid-rule
4516 @cindex mid-rule actions
4517 Finally, Bison will never invoke a @code{%destructor} for an unreferenced
4518 mid-rule semantic value (@pxref{Mid-Rule Actions,,Actions in Mid-Rule}).
4519 That is, Bison does not consider a mid-rule to have a semantic value if you do
4520 not reference @code{$$} in the mid-rule's action or @code{$@var{n}} (where
4521 @var{n} is the RHS symbol position of the mid-rule) in any later action in that
4522 rule.
4523 However, if you do reference either, the Bison-generated parser will invoke the
4524 @code{<>} @code{%destructor} whenever it discards the mid-rule symbol.
4525
4526 @ignore
4527 @noindent
4528 In the future, it may be possible to redefine the @code{error} token as a
4529 nonterminal that captures the discarded symbols.
4530 In that case, the parser will invoke the default destructor for it as well.
4531 @end ignore
4532
4533 @sp 1
4534
4535 @cindex discarded symbols
4536 @dfn{Discarded symbols} are the following:
4537
4538 @itemize
4539 @item
4540 stacked symbols popped during the first phase of error recovery,
4541 @item
4542 incoming terminals during the second phase of error recovery,
4543 @item
4544 the current lookahead and the entire stack (except the current
4545 right-hand side symbols) when the parser returns immediately, and
4546 @item
4547 the start symbol, when the parser succeeds.
4548 @end itemize
4549
4550 The parser can @dfn{return immediately} because of an explicit call to
4551 @code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
4552 exhaustion.
4553
4554 Right-hand side symbols of a rule that explicitly triggers a syntax
4555 error via @code{YYERROR} are not discarded automatically. As a rule
4556 of thumb, destructors are invoked only when user actions cannot manage
4557 the memory.
4558
4559 @node Expect Decl
4560 @subsection Suppressing Conflict Warnings
4561 @cindex suppressing conflict warnings
4562 @cindex preventing warnings about conflicts
4563 @cindex warnings, preventing
4564 @cindex conflicts, suppressing warnings of
4565 @findex %expect
4566 @findex %expect-rr
4567
4568 Bison normally warns if there are any conflicts in the grammar
4569 (@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4570 have harmless shift/reduce conflicts which are resolved in a predictable
4571 way and would be difficult to eliminate. It is desirable to suppress
4572 the warning about these conflicts unless the number of conflicts
4573 changes. You can do this with the @code{%expect} declaration.
4574
4575 The declaration looks like this:
4576
4577 @example
4578 %expect @var{n}
4579 @end example
4580
4581 Here @var{n} is a decimal integer. The declaration says there should
4582 be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4583 Bison reports an error if the number of shift/reduce conflicts differs
4584 from @var{n}, or if there are any reduce/reduce conflicts.
4585
4586 For deterministic parsers, reduce/reduce conflicts are more
4587 serious, and should be eliminated entirely. Bison will always report
4588 reduce/reduce conflicts for these parsers. With GLR
4589 parsers, however, both kinds of conflicts are routine; otherwise,
4590 there would be no need to use GLR parsing. Therefore, it is
4591 also possible to specify an expected number of reduce/reduce conflicts
4592 in GLR parsers, using the declaration:
4593
4594 @example
4595 %expect-rr @var{n}
4596 @end example
4597
4598 In general, using @code{%expect} involves these steps:
4599
4600 @itemize @bullet
4601 @item
4602 Compile your grammar without @code{%expect}. Use the @samp{-v} option
4603 to get a verbose list of where the conflicts occur. Bison will also
4604 print the number of conflicts.
4605
4606 @item
4607 Check each of the conflicts to make sure that Bison's default
4608 resolution is what you really want. If not, rewrite the grammar and
4609 go back to the beginning.
4610
4611 @item
4612 Add an @code{%expect} declaration, copying the number @var{n} from the
4613 number which Bison printed. With GLR parsers, add an
4614 @code{%expect-rr} declaration as well.
4615 @end itemize
4616
4617 Now Bison will report an error if you introduce an unexpected conflict,
4618 but will keep silent otherwise.
4619
4620 @node Start Decl
4621 @subsection The Start-Symbol
4622 @cindex declaring the start symbol
4623 @cindex start symbol, declaring
4624 @cindex default start symbol
4625 @findex %start
4626
4627 Bison assumes by default that the start symbol for the grammar is the first
4628 nonterminal specified in the grammar specification section. The programmer
4629 may override this restriction with the @code{%start} declaration as follows:
4630
4631 @example
4632 %start @var{symbol}
4633 @end example
4634
4635 @node Pure Decl
4636 @subsection A Pure (Reentrant) Parser
4637 @cindex reentrant parser
4638 @cindex pure parser
4639 @findex %define api.pure
4640
4641 A @dfn{reentrant} program is one which does not alter in the course of
4642 execution; in other words, it consists entirely of @dfn{pure} (read-only)
4643 code. Reentrancy is important whenever asynchronous execution is possible;
4644 for example, a nonreentrant program may not be safe to call from a signal
4645 handler. In systems with multiple threads of control, a nonreentrant
4646 program must be called only within interlocks.
4647
4648 Normally, Bison generates a parser which is not reentrant. This is
4649 suitable for most uses, and it permits compatibility with Yacc. (The
4650 standard Yacc interfaces are inherently nonreentrant, because they use
4651 statically allocated variables for communication with @code{yylex},
4652 including @code{yylval} and @code{yylloc}.)
4653
4654 Alternatively, you can generate a pure, reentrant parser. The Bison
4655 declaration @code{%define api.pure} says that you want the parser to be
4656 reentrant. It looks like this:
4657
4658 @example
4659 %define api.pure
4660 @end example
4661
4662 The result is that the communication variables @code{yylval} and
4663 @code{yylloc} become local variables in @code{yyparse}, and a different
4664 calling convention is used for the lexical analyzer function
4665 @code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
4666 Parsers}, for the details of this. The variable @code{yynerrs}
4667 becomes local in @code{yyparse} in pull mode but it becomes a member
4668 of yypstate in push mode. (@pxref{Error Reporting, ,The Error
4669 Reporting Function @code{yyerror}}). The convention for calling
4670 @code{yyparse} itself is unchanged.
4671
4672 Whether the parser is pure has nothing to do with the grammar rules.
4673 You can generate either a pure parser or a nonreentrant parser from any
4674 valid grammar.
4675
4676 @node Push Decl
4677 @subsection A Push Parser
4678 @cindex push parser
4679 @cindex push parser
4680 @findex %define api.push-pull
4681
4682 (The current push parsing interface is experimental and may evolve.
4683 More user feedback will help to stabilize it.)
4684
4685 A pull parser is called once and it takes control until all its input
4686 is completely parsed. A push parser, on the other hand, is called
4687 each time a new token is made available.
4688
4689 A push parser is typically useful when the parser is part of a
4690 main event loop in the client's application. This is typically
4691 a requirement of a GUI, when the main event loop needs to be triggered
4692 within a certain time period.
4693
4694 Normally, Bison generates a pull parser.
4695 The following Bison declaration says that you want the parser to be a push
4696 parser (@pxref{Decl Summary,,%define api.push-pull}):
4697
4698 @example
4699 %define api.push-pull push
4700 @end example
4701
4702 In almost all cases, you want to ensure that your push parser is also
4703 a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). The only
4704 time you should create an impure push parser is to have backwards
4705 compatibility with the impure Yacc pull mode interface. Unless you know
4706 what you are doing, your declarations should look like this:
4707
4708 @example
4709 %define api.pure
4710 %define api.push-pull push
4711 @end example
4712
4713 There is a major notable functional difference between the pure push parser
4714 and the impure push parser. It is acceptable for a pure push parser to have
4715 many parser instances, of the same type of parser, in memory at the same time.
4716 An impure push parser should only use one parser at a time.
4717
4718 When a push parser is selected, Bison will generate some new symbols in
4719 the generated parser. @code{yypstate} is a structure that the generated
4720 parser uses to store the parser's state. @code{yypstate_new} is the
4721 function that will create a new parser instance. @code{yypstate_delete}
4722 will free the resources associated with the corresponding parser instance.
4723 Finally, @code{yypush_parse} is the function that should be called whenever a
4724 token is available to provide the parser. A trivial example
4725 of using a pure push parser would look like this:
4726
4727 @example
4728 int status;
4729 yypstate *ps = yypstate_new ();
4730 do @{
4731 status = yypush_parse (ps, yylex (), NULL);
4732 @} while (status == YYPUSH_MORE);
4733 yypstate_delete (ps);
4734 @end example
4735
4736 If the user decided to use an impure push parser, a few things about
4737 the generated parser will change. The @code{yychar} variable becomes
4738 a global variable instead of a variable in the @code{yypush_parse} function.
4739 For this reason, the signature of the @code{yypush_parse} function is
4740 changed to remove the token as a parameter. A nonreentrant push parser
4741 example would thus look like this:
4742
4743 @example
4744 extern int yychar;
4745 int status;
4746 yypstate *ps = yypstate_new ();
4747 do @{
4748 yychar = yylex ();
4749 status = yypush_parse (ps);
4750 @} while (status == YYPUSH_MORE);
4751 yypstate_delete (ps);
4752 @end example
4753
4754 That's it. Notice the next token is put into the global variable @code{yychar}
4755 for use by the next invocation of the @code{yypush_parse} function.
4756
4757 Bison also supports both the push parser interface along with the pull parser
4758 interface in the same generated parser. In order to get this functionality,
4759 you should replace the @code{%define api.push-pull push} declaration with the
4760 @code{%define api.push-pull both} declaration. Doing this will create all of
4761 the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
4762 and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
4763 would be used. However, the user should note that it is implemented in the
4764 generated parser by calling @code{yypull_parse}.
4765 This makes the @code{yyparse} function that is generated with the
4766 @code{%define api.push-pull both} declaration slower than the normal
4767 @code{yyparse} function. If the user
4768 calls the @code{yypull_parse} function it will parse the rest of the input
4769 stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
4770 and then @code{yypull_parse} the rest of the input stream. If you would like
4771 to switch back and forth between between parsing styles, you would have to
4772 write your own @code{yypull_parse} function that knows when to quit looking
4773 for input. An example of using the @code{yypull_parse} function would look
4774 like this:
4775
4776 @example
4777 yypstate *ps = yypstate_new ();
4778 yypull_parse (ps); /* Will call the lexer */
4779 yypstate_delete (ps);
4780 @end example
4781
4782 Adding the @code{%define api.pure} declaration does exactly the same thing to
4783 the generated parser with @code{%define api.push-pull both} as it did for
4784 @code{%define api.push-pull push}.
4785
4786 @node Decl Summary
4787 @subsection Bison Declaration Summary
4788 @cindex Bison declaration summary
4789 @cindex declaration summary
4790 @cindex summary, Bison declaration
4791
4792 Here is a summary of the declarations used to define a grammar:
4793
4794 @deffn {Directive} %union
4795 Declare the collection of data types that semantic values may have
4796 (@pxref{Union Decl, ,The Collection of Value Types}).
4797 @end deffn
4798
4799 @deffn {Directive} %token
4800 Declare a terminal symbol (token type name) with no precedence
4801 or associativity specified (@pxref{Token Decl, ,Token Type Names}).
4802 @end deffn
4803
4804 @deffn {Directive} %right
4805 Declare a terminal symbol (token type name) that is right-associative
4806 (@pxref{Precedence Decl, ,Operator Precedence}).
4807 @end deffn
4808
4809 @deffn {Directive} %left
4810 Declare a terminal symbol (token type name) that is left-associative
4811 (@pxref{Precedence Decl, ,Operator Precedence}).
4812 @end deffn
4813
4814 @deffn {Directive} %nonassoc
4815 Declare a terminal symbol (token type name) that is nonassociative
4816 (@pxref{Precedence Decl, ,Operator Precedence}).
4817 Using it in a way that would be associative is a syntax error.
4818 @end deffn
4819
4820 @ifset defaultprec
4821 @deffn {Directive} %default-prec
4822 Assign a precedence to rules lacking an explicit @code{%prec} modifier
4823 (@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4824 @end deffn
4825 @end ifset
4826
4827 @deffn {Directive} %type
4828 Declare the type of semantic values for a nonterminal symbol
4829 (@pxref{Type Decl, ,Nonterminal Symbols}).
4830 @end deffn
4831
4832 @deffn {Directive} %start
4833 Specify the grammar's start symbol (@pxref{Start Decl, ,The
4834 Start-Symbol}).
4835 @end deffn
4836
4837 @deffn {Directive} %expect
4838 Declare the expected number of shift-reduce conflicts
4839 (@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
4840 @end deffn
4841
4842
4843 @sp 1
4844 @noindent
4845 In order to change the behavior of @command{bison}, use the following
4846 directives:
4847
4848 @deffn {Directive} %code @{@var{code}@}
4849 @findex %code
4850 This is the unqualified form of the @code{%code} directive.
4851 It inserts @var{code} verbatim at a language-dependent default location in the
4852 output@footnote{The default location is actually skeleton-dependent;
4853 writers of non-standard skeletons however should choose the default location
4854 consistently with the behavior of the standard Bison skeletons.}.
4855
4856 @cindex Prologue
4857 For C/C++, the default location is the parser source code
4858 file after the usual contents of the parser header file.
4859 Thus, @code{%code} replaces the traditional Yacc prologue,
4860 @code{%@{@var{code}%@}}, for most purposes.
4861 For a detailed discussion, see @ref{Prologue Alternatives}.
4862
4863 For Java, the default location is inside the parser class.
4864 @end deffn
4865
4866 @deffn {Directive} %code @var{qualifier} @{@var{code}@}
4867 This is the qualified form of the @code{%code} directive.
4868 If you need to specify location-sensitive verbatim @var{code} that does not
4869 belong at the default location selected by the unqualified @code{%code} form,
4870 use this form instead.
4871
4872 @var{qualifier} identifies the purpose of @var{code} and thus the location(s)
4873 where Bison should generate it.
4874 Not all @var{qualifier}s are accepted for all target languages.
4875 Unaccepted @var{qualifier}s produce an error.
4876 Some of the accepted @var{qualifier}s are:
4877
4878 @itemize @bullet
4879 @item requires
4880 @findex %code requires
4881
4882 @itemize @bullet
4883 @item Language(s): C, C++
4884
4885 @item Purpose: This is the best place to write dependency code required for
4886 @code{YYSTYPE} and @code{YYLTYPE}.
4887 In other words, it's the best place to define types referenced in @code{%union}
4888 directives, and it's the best place to override Bison's default @code{YYSTYPE}
4889 and @code{YYLTYPE} definitions.
4890
4891 @item Location(s): The parser header file and the parser source code file
4892 before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE} definitions.
4893 @end itemize
4894
4895 @item provides
4896 @findex %code provides
4897
4898 @itemize @bullet
4899 @item Language(s): C, C++
4900
4901 @item Purpose: This is the best place to write additional definitions and
4902 declarations that should be provided to other modules.
4903
4904 @item Location(s): The parser header file and the parser source code file after
4905 the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and token definitions.
4906 @end itemize
4907
4908 @item top
4909 @findex %code top
4910
4911 @itemize @bullet
4912 @item Language(s): C, C++
4913
4914 @item Purpose: The unqualified @code{%code} or @code{%code requires} should
4915 usually be more appropriate than @code{%code top}.
4916 However, occasionally it is necessary to insert code much nearer the top of the
4917 parser source code file.
4918 For example:
4919
4920 @smallexample
4921 %code top @{
4922 #define _GNU_SOURCE
4923 #include <stdio.h>
4924 @}
4925 @end smallexample
4926
4927 @item Location(s): Near the top of the parser source code file.
4928 @end itemize
4929
4930 @item imports
4931 @findex %code imports
4932
4933 @itemize @bullet
4934 @item Language(s): Java
4935
4936 @item Purpose: This is the best place to write Java import directives.
4937
4938 @item Location(s): The parser Java file after any Java package directive and
4939 before any class definitions.
4940 @end itemize
4941 @end itemize
4942
4943 @cindex Prologue
4944 For a detailed discussion of how to use @code{%code} in place of the
4945 traditional Yacc prologue for C/C++, see @ref{Prologue Alternatives}.
4946 @end deffn
4947
4948 @deffn {Directive} %debug
4949 In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
4950 already defined, so that the debugging facilities are compiled.
4951 @xref{Tracing, ,Tracing Your Parser}.
4952 @end deffn
4953
4954 @deffn {Directive} %define @var{variable}
4955 @deffnx {Directive} %define @var{variable} @var{value}
4956 @deffnx {Directive} %define @var{variable} "@var{value}"
4957 Define a variable to adjust Bison's behavior.
4958
4959 It is an error if a @var{variable} is defined by @code{%define} multiple
4960 times, but see @ref{Bison Options,,-D @var{name}[=@var{value}]}.
4961
4962 @var{value} must be placed in quotation marks if it contains any
4963 character other than a letter, underscore, period, dash, or non-initial
4964 digit.
4965
4966 Omitting @code{"@var{value}"} entirely is always equivalent to specifying
4967 @code{""}.
4968
4969 Some @var{variable}s take Boolean values.
4970 In this case, Bison will complain if the variable definition does not meet one
4971 of the following four conditions:
4972
4973 @enumerate
4974 @item @code{@var{value}} is @code{true}
4975
4976 @item @code{@var{value}} is omitted (or @code{""} is specified).
4977 This is equivalent to @code{true}.
4978
4979 @item @code{@var{value}} is @code{false}.
4980
4981 @item @var{variable} is never defined.
4982 In this case, Bison selects a default value.
4983 @end enumerate
4984
4985 What @var{variable}s are accepted, as well as their meanings and default
4986 values, depend on the selected target language and/or the parser
4987 skeleton (@pxref{Decl Summary,,%language}, @pxref{Decl
4988 Summary,,%skeleton}).
4989 Unaccepted @var{variable}s produce an error.
4990 Some of the accepted @var{variable}s are:
4991
4992 @itemize @bullet
4993 @item api.pure
4994 @findex %define api.pure
4995
4996 @itemize @bullet
4997 @item Language(s): C
4998
4999 @item Purpose: Request a pure (reentrant) parser program.
5000 @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
5001
5002 @item Accepted Values: Boolean
5003
5004 @item Default Value: @code{false}
5005 @end itemize
5006
5007 @item api.push-pull
5008 @findex %define api.push-pull
5009
5010 @itemize @bullet
5011 @item Language(s): C (deterministic parsers only)
5012
5013 @item Purpose: Request a pull parser, a push parser, or both.
5014 @xref{Push Decl, ,A Push Parser}.
5015 (The current push parsing interface is experimental and may evolve.
5016 More user feedback will help to stabilize it.)
5017
5018 @item Accepted Values: @code{pull}, @code{push}, @code{both}
5019
5020 @item Default Value: @code{pull}
5021 @end itemize
5022
5023 @c ================================================== lr.default-reductions
5024
5025 @item lr.default-reductions
5026 @cindex default reductions
5027 @findex %define lr.default-reductions
5028 @cindex delayed syntax errors
5029 @cindex syntax errors delayed
5030 @cindex LAC
5031 @findex %nonassoc
5032
5033 @itemize @bullet
5034 @item Language(s): all
5035
5036 @item Purpose: Specify the kind of states that are permitted to
5037 contain default reductions.
5038 That is, in such a state, Bison selects the reduction with the largest
5039 lookahead set to be the default parser action and then removes that
5040 lookahead set.
5041 (The ability to specify where default reductions should be used is
5042 experimental.
5043 More user feedback will help to stabilize it.)
5044
5045 @item Accepted Values:
5046 @itemize
5047 @item @code{all}.
5048 This is the traditional Bison behavior.
5049 The main advantage is a significant decrease in the size of the parser
5050 tables.
5051 The disadvantage is that, when the generated parser encounters a
5052 syntactically unacceptable token, the parser might then perform
5053 unnecessary default reductions before it can detect the syntax error.
5054 Such delayed syntax error detection is usually inherent in
5055 LALR and IELR parser tables anyway due to
5056 LR state merging (@pxref{Decl Summary,,lr.type}).
5057 Furthermore, the use of @code{%nonassoc} can contribute to delayed
5058 syntax error detection even in the case of canonical LR.
5059 As an experimental feature, delayed syntax error detection can be
5060 overcome in all cases by enabling LAC (@pxref{Decl
5061 Summary,,parse.lac}, for details, including a discussion of the effects
5062 of delayed syntax error detection).
5063
5064 @item @code{consistent}.
5065 @cindex consistent states
5066 A consistent state is a state that has only one possible action.
5067 If that action is a reduction, then the parser does not need to request
5068 a lookahead token from the scanner before performing that action.
5069 However, the parser recognizes the ability to ignore the lookahead token
5070 in this way only when such a reduction is encoded as a default
5071 reduction.
5072 Thus, if default reductions are permitted only in consistent states,
5073 then a canonical LR parser that does not employ
5074 @code{%nonassoc} detects a syntax error as soon as it @emph{needs} the
5075 syntactically unacceptable token from the scanner.
5076
5077 @item @code{accepting}.
5078 @cindex accepting state
5079 In the accepting state, the default reduction is actually the accept
5080 action.
5081 In this case, a canonical LR parser that does not employ
5082 @code{%nonassoc} detects a syntax error as soon as it @emph{reaches} the
5083 syntactically unacceptable token in the input.
5084 That is, it does not perform any extra reductions.
5085 @end itemize
5086
5087 @item Default Value:
5088 @itemize
5089 @item @code{accepting} if @code{lr.type} is @code{canonical-lr}.
5090 @item @code{all} otherwise.
5091 @end itemize
5092 @end itemize
5093
5094 @c ============================================ lr.keep-unreachable-states
5095
5096 @item lr.keep-unreachable-states
5097 @findex %define lr.keep-unreachable-states
5098
5099 @itemize @bullet
5100 @item Language(s): all
5101
5102 @item Purpose: Request that Bison allow unreachable parser states to
5103 remain in the parser tables.
5104 Bison considers a state to be unreachable if there exists no sequence of
5105 transitions from the start state to that state.
5106 A state can become unreachable during conflict resolution if Bison disables a
5107 shift action leading to it from a predecessor state.
5108 Keeping unreachable states is sometimes useful for analysis purposes, but they
5109 are useless in the generated parser.
5110
5111 @item Accepted Values: Boolean
5112
5113 @item Default Value: @code{false}
5114
5115 @item Caveats:
5116
5117 @itemize @bullet
5118
5119 @item Unreachable states may contain conflicts and may use rules not used in
5120 any other state.
5121 Thus, keeping unreachable states may induce warnings that are irrelevant to
5122 your parser's behavior, and it may eliminate warnings that are relevant.
5123 Of course, the change in warnings may actually be relevant to a parser table
5124 analysis that wants to keep unreachable states, so this behavior will likely
5125 remain in future Bison releases.
5126
5127 @item While Bison is able to remove unreachable states, it is not guaranteed to
5128 remove other kinds of useless states.
5129 Specifically, when Bison disables reduce actions during conflict resolution,
5130 some goto actions may become useless, and thus some additional states may
5131 become useless.
5132 If Bison were to compute which goto actions were useless and then disable those
5133 actions, it could identify such states as unreachable and then remove those
5134 states.
5135 However, Bison does not compute which goto actions are useless.
5136 @end itemize
5137 @end itemize
5138
5139 @c ================================================== lr.type
5140
5141 @item lr.type
5142 @findex %define lr.type
5143 @cindex LALR
5144 @cindex IELR
5145 @cindex LR
5146
5147 @itemize @bullet
5148 @item Language(s): all
5149
5150 @item Purpose: Specify the type of parser tables within the
5151 LR(1) family.
5152 (This feature is experimental.
5153 More user feedback will help to stabilize it.)
5154
5155 @item Accepted Values:
5156 @itemize
5157 @item @code{lalr}.
5158 While Bison generates LALR parser tables by default for
5159 historical reasons, IELR or canonical LR is almost
5160 always preferable for deterministic parsers.
5161 The trouble is that LALR parser tables can suffer from
5162 mysterious conflicts and thus may not accept the full set of sentences
5163 that IELR and canonical LR accept.
5164 @xref{Mystery Conflicts}, for details.
5165 However, there are at least two scenarios where LALR may be
5166 worthwhile:
5167 @itemize
5168 @cindex GLR with LALR
5169 @item When employing GLR parsers (@pxref{GLR Parsers}), if you
5170 do not resolve any conflicts statically (for example, with @code{%left}
5171 or @code{%prec}), then the parser explores all potential parses of any
5172 given input.
5173 In this case, the use of LALR parser tables is guaranteed not
5174 to alter the language accepted by the parser.
5175 LALR parser tables are the smallest parser tables Bison can
5176 currently generate, so they may be preferable.
5177 Nevertheless, once you begin to resolve conflicts statically,
5178 GLR begins to behave more like a deterministic parser, and so
5179 IELR and canonical LR can be helpful to avoid
5180 LALR's mysterious behavior.
5181
5182 @item Occasionally during development, an especially malformed grammar
5183 with a major recurring flaw may severely impede the IELR or
5184 canonical LR parser table generation algorithm.
5185 LALR can be a quick way to generate parser tables in order to
5186 investigate such problems while ignoring the more subtle differences
5187 from IELR and canonical LR.
5188 @end itemize
5189
5190 @item @code{ielr}.
5191 IELR is a minimal LR algorithm.
5192 That is, given any grammar (LR or non-LR),
5193 IELR and canonical LR always accept exactly the same
5194 set of sentences.
5195 However, as for LALR, the number of parser states is often an
5196 order of magnitude less for IELR than for canonical
5197 LR.
5198 More importantly, because canonical LR's extra parser states
5199 may contain duplicate conflicts in the case of non-LR
5200 grammars, the number of conflicts for IELR is often an order
5201 of magnitude less as well.
5202 This can significantly reduce the complexity of developing of a grammar.
5203
5204 @item @code{canonical-lr}.
5205 @cindex delayed syntax errors
5206 @cindex syntax errors delayed
5207 @cindex LAC
5208 @findex %nonassoc
5209 While inefficient, canonical LR parser tables can be an
5210 interesting means to explore a grammar because they have a property that
5211 IELR and LALR tables do not.
5212 That is, if @code{%nonassoc} is not used and default reductions are left
5213 disabled (@pxref{Decl Summary,,lr.default-reductions}), then, for every
5214 left context of every canonical LR state, the set of tokens
5215 accepted by that state is guaranteed to be the exact set of tokens that
5216 is syntactically acceptable in that left context.
5217 It might then seem that an advantage of canonical LR parsers
5218 in production is that, under the above constraints, they are guaranteed
5219 to detect a syntax error as soon as possible without performing any
5220 unnecessary reductions.
5221 However, IELR parsers using LAC (@pxref{Decl
5222 Summary,,parse.lac}) are also able to achieve this behavior without
5223 sacrificing @code{%nonassoc} or default reductions.
5224 @end itemize
5225
5226 @item Default Value: @code{lalr}
5227 @end itemize
5228
5229 @item namespace
5230 @findex %define namespace
5231
5232 @itemize
5233 @item Languages(s): C++
5234
5235 @item Purpose: Specify the namespace for the parser class.
5236 For example, if you specify:
5237
5238 @smallexample
5239 %define namespace "foo::bar"
5240 @end smallexample
5241
5242 Bison uses @code{foo::bar} verbatim in references such as:
5243
5244 @smallexample
5245 foo::bar::parser::semantic_type
5246 @end smallexample
5247
5248 However, to open a namespace, Bison removes any leading @code{::} and then
5249 splits on any remaining occurrences:
5250
5251 @smallexample
5252 namespace foo @{ namespace bar @{
5253 class position;
5254 class location;
5255 @} @}
5256 @end smallexample
5257
5258 @item Accepted Values: Any absolute or relative C++ namespace reference without
5259 a trailing @code{"::"}.
5260 For example, @code{"foo"} or @code{"::foo::bar"}.
5261
5262 @item Default Value: The value specified by @code{%name-prefix}, which defaults
5263 to @code{yy}.
5264 This usage of @code{%name-prefix} is for backward compatibility and can be
5265 confusing since @code{%name-prefix} also specifies the textual prefix for the
5266 lexical analyzer function.
5267 Thus, if you specify @code{%name-prefix}, it is best to also specify
5268 @code{%define namespace} so that @code{%name-prefix} @emph{only} affects the
5269 lexical analyzer function.
5270 For example, if you specify:
5271
5272 @smallexample
5273 %define namespace "foo"
5274 %name-prefix "bar::"
5275 @end smallexample
5276
5277 The parser namespace is @code{foo} and @code{yylex} is referenced as
5278 @code{bar::lex}.
5279 @end itemize
5280
5281 @c ================================================== parse.lac
5282 @item parse.lac
5283 @findex %define parse.lac
5284 @cindex LAC
5285 @cindex lookahead correction
5286
5287 @itemize
5288 @item Languages(s): C
5289
5290 @item Purpose: Enable LAC (lookahead correction) to improve
5291 syntax error handling.
5292
5293 Canonical LR, IELR, and LALR can suffer
5294 from a couple of problems upon encountering a syntax error. First, the
5295 parser might perform additional parser stack reductions before
5296 discovering the syntax error. Such reductions perform user semantic
5297 actions that are unexpected because they are based on an invalid token,
5298 and they cause error recovery to begin in a different syntactic context
5299 than the one in which the invalid token was encountered. Second, when
5300 verbose error messages are enabled (with @code{%error-verbose} or
5301 @code{#define YYERROR_VERBOSE}), the expected token list in the syntax
5302 error message can both contain invalid tokens and omit valid tokens.
5303
5304 The culprits for the above problems are @code{%nonassoc}, default
5305 reductions in inconsistent states, and parser state merging. Thus,
5306 IELR and LALR suffer the most. Canonical
5307 LR can suffer only if @code{%nonassoc} is used or if default
5308 reductions are enabled for inconsistent states.
5309
5310 LAC is a new mechanism within the parsing algorithm that
5311 completely solves these problems for canonical LR,
5312 IELR, and LALR without sacrificing @code{%nonassoc},
5313 default reductions, or state mering. Conceptually, the mechanism is
5314 straight-forward. Whenever the parser fetches a new token from the
5315 scanner so that it can determine the next parser action, it immediately
5316 suspends normal parsing and performs an exploratory parse using a
5317 temporary copy of the normal parser state stack. During this
5318 exploratory parse, the parser does not perform user semantic actions.
5319 If the exploratory parse reaches a shift action, normal parsing then
5320 resumes on the normal parser stacks. If the exploratory parse reaches
5321 an error instead, the parser reports a syntax error. If verbose syntax
5322 error messages are enabled, the parser must then discover the list of
5323 expected tokens, so it performs a separate exploratory parse for each
5324 token in the grammar.
5325
5326 There is one subtlety about the use of LAC. That is, when in
5327 a consistent parser state with a default reduction, the parser will not
5328 attempt to fetch a token from the scanner because no lookahead is needed
5329 to determine the next parser action. Thus, whether default reductions
5330 are enabled in consistent states (@pxref{Decl
5331 Summary,,lr.default-reductions}) affects how soon the parser detects a
5332 syntax error: when it @emph{reaches} an erroneous token or when it
5333 eventually @emph{needs} that token as a lookahead. The latter behavior
5334 is probably more intuitive, so Bison currently provides no way to
5335 achieve the former behavior while default reductions are fully enabled.
5336
5337 Thus, when LAC is in use, for some fixed decision of whether
5338 to enable default reductions in consistent states, canonical
5339 LR and IELR behave exactly the same for both
5340 syntactically acceptable and syntactically unacceptable input. While
5341 LALR still does not support the full language-recognition
5342 power of canonical LR and IELR, LAC at
5343 least enables LALR's syntax error handling to correctly
5344 reflect LALR's language-recognition power.
5345
5346 Because LAC requires many parse actions to be performed twice,
5347 it can have a performance penalty. However, not all parse actions must
5348 be performed twice. Specifically, during a series of default reductions
5349 in consistent states and shift actions, the parser never has to initiate
5350 an exploratory parse. Moreover, the most time-consuming tasks in a
5351 parse are often the file I/O, the lexical analysis performed by the
5352 scanner, and the user's semantic actions, but none of these are
5353 performed during the exploratory parse. Finally, the base of the
5354 temporary stack used during an exploratory parse is a pointer into the
5355 normal parser state stack so that the stack is never physically copied.
5356 In our experience, the performance penalty of LAC has proven
5357 insignificant for practical grammars.
5358
5359 @item Accepted Values: @code{none}, @code{full}
5360
5361 @item Default Value: @code{none}
5362 @end itemize
5363 @end itemize
5364
5365 @end deffn
5366
5367 @deffn {Directive} %defines
5368 Write a header file containing macro definitions for the token type
5369 names defined in the grammar as well as a few other declarations.
5370 If the parser output file is named @file{@var{name}.c} then this file
5371 is named @file{@var{name}.h}.
5372
5373 For C parsers, the output header declares @code{YYSTYPE} unless
5374 @code{YYSTYPE} is already defined as a macro or you have used a
5375 @code{<@var{type}>} tag without using @code{%union}.
5376 Therefore, if you are using a @code{%union}
5377 (@pxref{Multiple Types, ,More Than One Value Type}) with components that
5378 require other definitions, or if you have defined a @code{YYSTYPE} macro
5379 or type definition
5380 (@pxref{Value Type, ,Data Types of Semantic Values}), you need to
5381 arrange for these definitions to be propagated to all modules, e.g., by
5382 putting them in a prerequisite header that is included both by your
5383 parser and by any other module that needs @code{YYSTYPE}.
5384
5385 Unless your parser is pure, the output header declares @code{yylval}
5386 as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
5387 Parser}.
5388
5389 If you have also used locations, the output header declares
5390 @code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
5391 the @code{YYSTYPE} macro and @code{yylval}. @xref{Locations, ,Tracking
5392 Locations}.
5393
5394 This output file is normally essential if you wish to put the definition
5395 of @code{yylex} in a separate source file, because @code{yylex}
5396 typically needs to be able to refer to the above-mentioned declarations
5397 and to the token type codes. @xref{Token Values, ,Semantic Values of
5398 Tokens}.
5399
5400 @findex %code requires
5401 @findex %code provides
5402 If you have declared @code{%code requires} or @code{%code provides}, the output
5403 header also contains their code.
5404 @xref{Decl Summary, ,%code}.
5405 @end deffn
5406
5407 @deffn {Directive} %defines @var{defines-file}
5408 Same as above, but save in the file @var{defines-file}.
5409 @end deffn
5410
5411 @deffn {Directive} %destructor
5412 Specify how the parser should reclaim the memory associated to
5413 discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
5414 @end deffn
5415
5416 @deffn {Directive} %file-prefix "@var{prefix}"
5417 Specify a prefix to use for all Bison output file names. The names are
5418 chosen as if the input file were named @file{@var{prefix}.y}.
5419 @end deffn
5420
5421 @deffn {Directive} %language "@var{language}"
5422 Specify the programming language for the generated parser. Currently
5423 supported languages include C, C++, and Java.
5424 @var{language} is case-insensitive.
5425
5426 This directive is experimental and its effect may be modified in future
5427 releases.
5428 @end deffn
5429
5430 @deffn {Directive} %locations
5431 Generate the code processing the locations (@pxref{Action Features,
5432 ,Special Features for Use in Actions}). This mode is enabled as soon as
5433 the grammar uses the special @samp{@@@var{n}} tokens, but if your
5434 grammar does not use it, using @samp{%locations} allows for more
5435 accurate syntax error messages.
5436 @end deffn
5437
5438 @deffn {Directive} %name-prefix "@var{prefix}"
5439 Rename the external symbols used in the parser so that they start with
5440 @var{prefix} instead of @samp{yy}. The precise list of symbols renamed
5441 in C parsers
5442 is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
5443 @code{yylval}, @code{yychar}, @code{yydebug}, and
5444 (if locations are used) @code{yylloc}. If you use a push parser,
5445 @code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5446 @code{yypstate_new} and @code{yypstate_delete} will
5447 also be renamed. For example, if you use @samp{%name-prefix "c_"}, the
5448 names become @code{c_parse}, @code{c_lex}, and so on.
5449 For C++ parsers, see the @code{%define namespace} documentation in this
5450 section.
5451 @xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
5452 @end deffn
5453
5454 @ifset defaultprec
5455 @deffn {Directive} %no-default-prec
5456 Do not assign a precedence to rules lacking an explicit @code{%prec}
5457 modifier (@pxref{Contextual Precedence, ,Context-Dependent
5458 Precedence}).
5459 @end deffn
5460 @end ifset
5461
5462 @deffn {Directive} %no-lines
5463 Don't generate any @code{#line} preprocessor commands in the parser
5464 file. Ordinarily Bison writes these commands in the parser file so that
5465 the C compiler and debuggers will associate errors and object code with
5466 your source file (the grammar file). This directive causes them to
5467 associate errors with the parser file, treating it an independent source
5468 file in its own right.
5469 @end deffn
5470
5471 @deffn {Directive} %output "@var{file}"
5472 Specify @var{file} for the parser file.
5473 @end deffn
5474
5475 @deffn {Directive} %pure-parser
5476 Deprecated version of @code{%define api.pure} (@pxref{Decl Summary, ,%define}),
5477 for which Bison is more careful to warn about unreasonable usage.
5478 @end deffn
5479
5480 @deffn {Directive} %require "@var{version}"
5481 Require version @var{version} or higher of Bison. @xref{Require Decl, ,
5482 Require a Version of Bison}.
5483 @end deffn
5484
5485 @deffn {Directive} %skeleton "@var{file}"
5486 Specify the skeleton to use.
5487
5488 @c You probably don't need this option unless you are developing Bison.
5489 @c You should use @code{%language} if you want to specify the skeleton for a
5490 @c different language, because it is clearer and because it will always choose the
5491 @c correct skeleton for non-deterministic or push parsers.
5492
5493 If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5494 file in the Bison installation directory.
5495 If it does, @var{file} is an absolute file name or a file name relative to the
5496 directory of the grammar file.
5497 This is similar to how most shells resolve commands.
5498 @end deffn
5499
5500 @deffn {Directive} %token-table
5501 Generate an array of token names in the parser file. The name of the
5502 array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
5503 token whose internal Bison token code number is @var{i}. The first
5504 three elements of @code{yytname} correspond to the predefined tokens
5505 @code{"$end"},
5506 @code{"error"}, and @code{"$undefined"}; after these come the symbols
5507 defined in the grammar file.
5508
5509 The name in the table includes all the characters needed to represent
5510 the token in Bison. For single-character literals and literal
5511 strings, this includes the surrounding quoting characters and any
5512 escape sequences. For example, the Bison single-character literal
5513 @code{'+'} corresponds to a three-character name, represented in C as
5514 @code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
5515 corresponds to a five-character name, represented in C as
5516 @code{"\"\\\\/\""}.
5517
5518 When you specify @code{%token-table}, Bison also generates macro
5519 definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
5520 @code{YYNRULES}, and @code{YYNSTATES}:
5521
5522 @table @code
5523 @item YYNTOKENS
5524 The highest token number, plus one.
5525 @item YYNNTS
5526 The number of nonterminal symbols.
5527 @item YYNRULES
5528 The number of grammar rules,
5529 @item YYNSTATES
5530 The number of parser states (@pxref{Parser States}).
5531 @end table
5532 @end deffn
5533
5534 @deffn {Directive} %verbose
5535 Write an extra output file containing verbose descriptions of the
5536 parser states and what is done for each type of lookahead token in
5537 that state. @xref{Understanding, , Understanding Your Parser}, for more
5538 information.
5539 @end deffn
5540
5541 @deffn {Directive} %yacc
5542 Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
5543 including its naming conventions. @xref{Bison Options}, for more.
5544 @end deffn
5545
5546
5547 @node Multiple Parsers
5548 @section Multiple Parsers in the Same Program
5549
5550 Most programs that use Bison parse only one language and therefore contain
5551 only one Bison parser. But what if you want to parse more than one
5552 language with the same program? Then you need to avoid a name conflict
5553 between different definitions of @code{yyparse}, @code{yylval}, and so on.
5554
5555 The easy way to do this is to use the option @samp{-p @var{prefix}}
5556 (@pxref{Invocation, ,Invoking Bison}). This renames the interface
5557 functions and variables of the Bison parser to start with @var{prefix}
5558 instead of @samp{yy}. You can use this to give each parser distinct
5559 names that do not conflict.
5560
5561 The precise list of symbols renamed is @code{yyparse}, @code{yylex},
5562 @code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
5563 @code{yychar} and @code{yydebug}. If you use a push parser,
5564 @code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5565 @code{yypstate_new} and @code{yypstate_delete} will also be renamed.
5566 For example, if you use @samp{-p c}, the names become @code{cparse},
5567 @code{clex}, and so on.
5568
5569 @strong{All the other variables and macros associated with Bison are not
5570 renamed.} These others are not global; there is no conflict if the same
5571 name is used in different parsers. For example, @code{YYSTYPE} is not
5572 renamed, but defining this in different ways in different parsers causes
5573 no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
5574
5575 The @samp{-p} option works by adding macro definitions to the beginning
5576 of the parser source file, defining @code{yyparse} as
5577 @code{@var{prefix}parse}, and so on. This effectively substitutes one
5578 name for the other in the entire parser file.
5579
5580 @node Interface
5581 @chapter Parser C-Language Interface
5582 @cindex C-language interface
5583 @cindex interface
5584
5585 The Bison parser is actually a C function named @code{yyparse}. Here we
5586 describe the interface conventions of @code{yyparse} and the other
5587 functions that it needs to use.
5588
5589 Keep in mind that the parser uses many C identifiers starting with
5590 @samp{yy} and @samp{YY} for internal purposes. If you use such an
5591 identifier (aside from those in this manual) in an action or in epilogue
5592 in the grammar file, you are likely to run into trouble.
5593
5594 @menu
5595 * Parser Function:: How to call @code{yyparse} and what it returns.
5596 * Push Parser Function:: How to call @code{yypush_parse} and what it returns.
5597 * Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
5598 * Parser Create Function:: How to call @code{yypstate_new} and what it returns.
5599 * Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
5600 * Lexical:: You must supply a function @code{yylex}
5601 which reads tokens.
5602 * Error Reporting:: You must supply a function @code{yyerror}.
5603 * Action Features:: Special features for use in actions.
5604 * Internationalization:: How to let the parser speak in the user's
5605 native language.
5606 @end menu
5607
5608 @node Parser Function
5609 @section The Parser Function @code{yyparse}
5610 @findex yyparse
5611
5612 You call the function @code{yyparse} to cause parsing to occur. This
5613 function reads tokens, executes actions, and ultimately returns when it
5614 encounters end-of-input or an unrecoverable syntax error. You can also
5615 write an action which directs @code{yyparse} to return immediately
5616 without reading further.
5617
5618
5619 @deftypefun int yyparse (void)
5620 The value returned by @code{yyparse} is 0 if parsing was successful (return
5621 is due to end-of-input).
5622
5623 The value is 1 if parsing failed because of invalid input, i.e., input
5624 that contains a syntax error or that causes @code{YYABORT} to be
5625 invoked.
5626
5627 The value is 2 if parsing failed due to memory exhaustion.
5628 @end deftypefun
5629
5630 In an action, you can cause immediate return from @code{yyparse} by using
5631 these macros:
5632
5633 @defmac YYACCEPT
5634 @findex YYACCEPT
5635 Return immediately with value 0 (to report success).
5636 @end defmac
5637
5638 @defmac YYABORT
5639 @findex YYABORT
5640 Return immediately with value 1 (to report failure).
5641 @end defmac
5642
5643 If you use a reentrant parser, you can optionally pass additional
5644 parameter information to it in a reentrant way. To do so, use the
5645 declaration @code{%parse-param}:
5646
5647 @deffn {Directive} %parse-param @{@var{argument-declaration}@}
5648 @findex %parse-param
5649 Declare that an argument declared by the braced-code
5650 @var{argument-declaration} is an additional @code{yyparse} argument.
5651 The @var{argument-declaration} is used when declaring
5652 functions or prototypes. The last identifier in
5653 @var{argument-declaration} must be the argument name.
5654 @end deffn
5655
5656 Here's an example. Write this in the parser:
5657
5658 @example
5659 %parse-param @{int *nastiness@}
5660 %parse-param @{int *randomness@}
5661 @end example
5662
5663 @noindent
5664 Then call the parser like this:
5665
5666 @example
5667 @{
5668 int nastiness, randomness;
5669 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
5670 value = yyparse (&nastiness, &randomness);
5671 @dots{}
5672 @}
5673 @end example
5674
5675 @noindent
5676 In the grammar actions, use expressions like this to refer to the data:
5677
5678 @example
5679 exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
5680 @end example
5681
5682 @node Push Parser Function
5683 @section The Push Parser Function @code{yypush_parse}
5684 @findex yypush_parse
5685
5686 (The current push parsing interface is experimental and may evolve.
5687 More user feedback will help to stabilize it.)
5688
5689 You call the function @code{yypush_parse} to parse a single token. This
5690 function is available if either the @code{%define api.push-pull push} or
5691 @code{%define api.push-pull both} declaration is used.
5692 @xref{Push Decl, ,A Push Parser}.
5693
5694 @deftypefun int yypush_parse (yypstate *yyps)
5695 The value returned by @code{yypush_parse} is the same as for yyparse with the
5696 following exception. @code{yypush_parse} will return YYPUSH_MORE if more input
5697 is required to finish parsing the grammar.
5698 @end deftypefun
5699
5700 @node Pull Parser Function
5701 @section The Pull Parser Function @code{yypull_parse}
5702 @findex yypull_parse
5703
5704 (The current push parsing interface is experimental and may evolve.
5705 More user feedback will help to stabilize it.)
5706
5707 You call the function @code{yypull_parse} to parse the rest of the input
5708 stream. This function is available if the @code{%define api.push-pull both}
5709 declaration is used.
5710 @xref{Push Decl, ,A Push Parser}.
5711
5712 @deftypefun int yypull_parse (yypstate *yyps)
5713 The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
5714 @end deftypefun
5715
5716 @node Parser Create Function
5717 @section The Parser Create Function @code{yystate_new}
5718 @findex yypstate_new
5719
5720 (The current push parsing interface is experimental and may evolve.
5721 More user feedback will help to stabilize it.)
5722
5723 You call the function @code{yypstate_new} to create a new parser instance.
5724 This function is available if either the @code{%define api.push-pull push} or
5725 @code{%define api.push-pull both} declaration is used.
5726 @xref{Push Decl, ,A Push Parser}.
5727
5728 @deftypefun yypstate *yypstate_new (void)
5729 The function will return a valid parser instance if there was memory available
5730 or 0 if no memory was available.
5731 In impure mode, it will also return 0 if a parser instance is currently
5732 allocated.
5733 @end deftypefun
5734
5735 @node Parser Delete Function
5736 @section The Parser Delete Function @code{yystate_delete}
5737 @findex yypstate_delete
5738
5739 (The current push parsing interface is experimental and may evolve.
5740 More user feedback will help to stabilize it.)
5741
5742 You call the function @code{yypstate_delete} to delete a parser instance.
5743 function is available if either the @code{%define api.push-pull push} or
5744 @code{%define api.push-pull both} declaration is used.
5745 @xref{Push Decl, ,A Push Parser}.
5746
5747 @deftypefun void yypstate_delete (yypstate *yyps)
5748 This function will reclaim the memory associated with a parser instance.
5749 After this call, you should no longer attempt to use the parser instance.
5750 @end deftypefun
5751
5752 @node Lexical
5753 @section The Lexical Analyzer Function @code{yylex}
5754 @findex yylex
5755 @cindex lexical analyzer
5756
5757 The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
5758 the input stream and returns them to the parser. Bison does not create
5759 this function automatically; you must write it so that @code{yyparse} can
5760 call it. The function is sometimes referred to as a lexical scanner.
5761
5762 In simple programs, @code{yylex} is often defined at the end of the Bison
5763 grammar file. If @code{yylex} is defined in a separate source file, you
5764 need to arrange for the token-type macro definitions to be available there.
5765 To do this, use the @samp{-d} option when you run Bison, so that it will
5766 write these macro definitions into a separate header file
5767 @file{@var{name}.tab.h} which you can include in the other source files
5768 that need it. @xref{Invocation, ,Invoking Bison}.
5769
5770 @menu
5771 * Calling Convention:: How @code{yyparse} calls @code{yylex}.
5772 * Token Values:: How @code{yylex} must return the semantic value
5773 of the token it has read.
5774 * Token Locations:: How @code{yylex} must return the text location
5775 (line number, etc.) of the token, if the
5776 actions want that.
5777 * Pure Calling:: How the calling convention differs in a pure parser
5778 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
5779 @end menu
5780
5781 @node Calling Convention
5782 @subsection Calling Convention for @code{yylex}
5783
5784 The value that @code{yylex} returns must be the positive numeric code
5785 for the type of token it has just found; a zero or negative value
5786 signifies end-of-input.
5787
5788 When a token is referred to in the grammar rules by a name, that name
5789 in the parser file becomes a C macro whose definition is the proper
5790 numeric code for that token type. So @code{yylex} can use the name
5791 to indicate that type. @xref{Symbols}.
5792
5793 When a token is referred to in the grammar rules by a character literal,
5794 the numeric code for that character is also the code for the token type.
5795 So @code{yylex} can simply return that character code, possibly converted
5796 to @code{unsigned char} to avoid sign-extension. The null character
5797 must not be used this way, because its code is zero and that
5798 signifies end-of-input.
5799
5800 Here is an example showing these things:
5801
5802 @example
5803 int
5804 yylex (void)
5805 @{
5806 @dots{}
5807 if (c == EOF) /* Detect end-of-input. */
5808 return 0;
5809 @dots{}
5810 if (c == '+' || c == '-')
5811 return c; /* Assume token type for `+' is '+'. */
5812 @dots{}
5813 return INT; /* Return the type of the token. */
5814 @dots{}
5815 @}
5816 @end example
5817
5818 @noindent
5819 This interface has been designed so that the output from the @code{lex}
5820 utility can be used without change as the definition of @code{yylex}.
5821
5822 If the grammar uses literal string tokens, there are two ways that
5823 @code{yylex} can determine the token type codes for them:
5824
5825 @itemize @bullet
5826 @item
5827 If the grammar defines symbolic token names as aliases for the
5828 literal string tokens, @code{yylex} can use these symbolic names like
5829 all others. In this case, the use of the literal string tokens in
5830 the grammar file has no effect on @code{yylex}.
5831
5832 @item
5833 @code{yylex} can find the multicharacter token in the @code{yytname}
5834 table. The index of the token in the table is the token type's code.
5835 The name of a multicharacter token is recorded in @code{yytname} with a
5836 double-quote, the token's characters, and another double-quote. The
5837 token's characters are escaped as necessary to be suitable as input
5838 to Bison.
5839
5840 Here's code for looking up a multicharacter token in @code{yytname},
5841 assuming that the characters of the token are stored in
5842 @code{token_buffer}, and assuming that the token does not contain any
5843 characters like @samp{"} that require escaping.
5844
5845 @smallexample
5846 for (i = 0; i < YYNTOKENS; i++)
5847 @{
5848 if (yytname[i] != 0
5849 && yytname[i][0] == '"'
5850 && ! strncmp (yytname[i] + 1, token_buffer,
5851 strlen (token_buffer))
5852 && yytname[i][strlen (token_buffer) + 1] == '"'
5853 && yytname[i][strlen (token_buffer) + 2] == 0)
5854 break;
5855 @}
5856 @end smallexample
5857
5858 The @code{yytname} table is generated only if you use the
5859 @code{%token-table} declaration. @xref{Decl Summary}.
5860 @end itemize
5861
5862 @node Token Values
5863 @subsection Semantic Values of Tokens
5864
5865 @vindex yylval
5866 In an ordinary (nonreentrant) parser, the semantic value of the token must
5867 be stored into the global variable @code{yylval}. When you are using
5868 just one data type for semantic values, @code{yylval} has that type.
5869 Thus, if the type is @code{int} (the default), you might write this in
5870 @code{yylex}:
5871
5872 @example
5873 @group
5874 @dots{}
5875 yylval = value; /* Put value onto Bison stack. */
5876 return INT; /* Return the type of the token. */
5877 @dots{}
5878 @end group
5879 @end example
5880
5881 When you are using multiple data types, @code{yylval}'s type is a union
5882 made from the @code{%union} declaration (@pxref{Union Decl, ,The
5883 Collection of Value Types}). So when you store a token's value, you
5884 must use the proper member of the union. If the @code{%union}
5885 declaration looks like this:
5886
5887 @example
5888 @group
5889 %union @{
5890 int intval;
5891 double val;
5892 symrec *tptr;
5893 @}
5894 @end group
5895 @end example
5896
5897 @noindent
5898 then the code in @code{yylex} might look like this:
5899
5900 @example
5901 @group
5902 @dots{}
5903 yylval.intval = value; /* Put value onto Bison stack. */
5904 return INT; /* Return the type of the token. */
5905 @dots{}
5906 @end group
5907 @end example
5908
5909 @node Token Locations
5910 @subsection Textual Locations of Tokens
5911
5912 @vindex yylloc
5913 If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
5914 Tracking Locations}) in actions to keep track of the textual locations
5915 of tokens and groupings, then you must provide this information in
5916 @code{yylex}. The function @code{yyparse} expects to find the textual
5917 location of a token just parsed in the global variable @code{yylloc}.
5918 So @code{yylex} must store the proper data in that variable.
5919
5920 By default, the value of @code{yylloc} is a structure and you need only
5921 initialize the members that are going to be used by the actions. The
5922 four members are called @code{first_line}, @code{first_column},
5923 @code{last_line} and @code{last_column}. Note that the use of this
5924 feature makes the parser noticeably slower.
5925
5926 @tindex YYLTYPE
5927 The data type of @code{yylloc} has the name @code{YYLTYPE}.
5928
5929 @node Pure Calling
5930 @subsection Calling Conventions for Pure Parsers
5931
5932 When you use the Bison declaration @code{%define api.pure} to request a
5933 pure, reentrant parser, the global communication variables @code{yylval}
5934 and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
5935 Parser}.) In such parsers the two global variables are replaced by
5936 pointers passed as arguments to @code{yylex}. You must declare them as
5937 shown here, and pass the information back by storing it through those
5938 pointers.
5939
5940 @example
5941 int
5942 yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
5943 @{
5944 @dots{}
5945 *lvalp = value; /* Put value onto Bison stack. */
5946 return INT; /* Return the type of the token. */
5947 @dots{}
5948 @}
5949 @end example
5950
5951 If the grammar file does not use the @samp{@@} constructs to refer to
5952 textual locations, then the type @code{YYLTYPE} will not be defined. In
5953 this case, omit the second argument; @code{yylex} will be called with
5954 only one argument.
5955
5956
5957 If you wish to pass the additional parameter data to @code{yylex}, use
5958 @code{%lex-param} just like @code{%parse-param} (@pxref{Parser
5959 Function}).
5960
5961 @deffn {Directive} lex-param @{@var{argument-declaration}@}
5962 @findex %lex-param
5963 Declare that the braced-code @var{argument-declaration} is an
5964 additional @code{yylex} argument declaration.
5965 @end deffn
5966
5967 For instance:
5968
5969 @example
5970 %parse-param @{int *nastiness@}
5971 %lex-param @{int *nastiness@}
5972 %parse-param @{int *randomness@}
5973 @end example
5974
5975 @noindent
5976 results in the following signature:
5977
5978 @example
5979 int yylex (int *nastiness);
5980 int yyparse (int *nastiness, int *randomness);
5981 @end example
5982
5983 If @code{%define api.pure} is added:
5984
5985 @example
5986 int yylex (YYSTYPE *lvalp, int *nastiness);
5987 int yyparse (int *nastiness, int *randomness);
5988 @end example
5989
5990 @noindent
5991 and finally, if both @code{%define api.pure} and @code{%locations} are used:
5992
5993 @example
5994 int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
5995 int yyparse (int *nastiness, int *randomness);
5996 @end example
5997
5998 @node Error Reporting
5999 @section The Error Reporting Function @code{yyerror}
6000 @cindex error reporting function
6001 @findex yyerror
6002 @cindex parse error
6003 @cindex syntax error
6004
6005 The Bison parser detects a @dfn{syntax error} or @dfn{parse error}
6006 whenever it reads a token which cannot satisfy any syntax rule. An
6007 action in the grammar can also explicitly proclaim an error, using the
6008 macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
6009 in Actions}).
6010
6011 The Bison parser expects to report the error by calling an error
6012 reporting function named @code{yyerror}, which you must supply. It is
6013 called by @code{yyparse} whenever a syntax error is found, and it
6014 receives one argument. For a syntax error, the string is normally
6015 @w{@code{"syntax error"}}.
6016
6017 @findex %error-verbose
6018 If you invoke the directive @code{%error-verbose} in the Bison
6019 declarations section (@pxref{Bison Declarations, ,The Bison Declarations
6020 Section}), then Bison provides a more verbose and specific error message
6021 string instead of just plain @w{@code{"syntax error"}}.
6022
6023 The parser can detect one other kind of error: memory exhaustion. This
6024 can happen when the input contains constructions that are very deeply
6025 nested. It isn't likely you will encounter this, since the Bison
6026 parser normally extends its stack automatically up to a very large limit. But
6027 if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
6028 fashion, except that the argument string is @w{@code{"memory exhausted"}}.
6029
6030 In some cases diagnostics like @w{@code{"syntax error"}} are
6031 translated automatically from English to some other language before
6032 they are passed to @code{yyerror}. @xref{Internationalization}.
6033
6034 The following definition suffices in simple programs:
6035
6036 @example
6037 @group
6038 void
6039 yyerror (char const *s)
6040 @{
6041 @end group
6042 @group
6043 fprintf (stderr, "%s\n", s);
6044 @}
6045 @end group
6046 @end example
6047
6048 After @code{yyerror} returns to @code{yyparse}, the latter will attempt
6049 error recovery if you have written suitable error recovery grammar rules
6050 (@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
6051 immediately return 1.
6052
6053 Obviously, in location tracking pure parsers, @code{yyerror} should have
6054 an access to the current location.
6055 This is indeed the case for the GLR
6056 parsers, but not for the Yacc parser, for historical reasons. I.e., if
6057 @samp{%locations %define api.pure} is passed then the prototypes for
6058 @code{yyerror} are:
6059
6060 @example
6061 void yyerror (char const *msg); /* Yacc parsers. */
6062 void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
6063 @end example
6064
6065 If @samp{%parse-param @{int *nastiness@}} is used, then:
6066
6067 @example
6068 void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
6069 void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
6070 @end example
6071
6072 Finally, GLR and Yacc parsers share the same @code{yyerror} calling
6073 convention for absolutely pure parsers, i.e., when the calling
6074 convention of @code{yylex} @emph{and} the calling convention of
6075 @code{%define api.pure} are pure.
6076 I.e.:
6077
6078 @example
6079 /* Location tracking. */
6080 %locations
6081 /* Pure yylex. */
6082 %define api.pure
6083 %lex-param @{int *nastiness@}
6084 /* Pure yyparse. */
6085 %parse-param @{int *nastiness@}
6086 %parse-param @{int *randomness@}
6087 @end example
6088
6089 @noindent
6090 results in the following signatures for all the parser kinds:
6091
6092 @example
6093 int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
6094 int yyparse (int *nastiness, int *randomness);
6095 void yyerror (YYLTYPE *locp,
6096 int *nastiness, int *randomness,
6097 char const *msg);
6098 @end example
6099
6100 @noindent
6101 The prototypes are only indications of how the code produced by Bison
6102 uses @code{yyerror}. Bison-generated code always ignores the returned
6103 value, so @code{yyerror} can return any type, including @code{void}.
6104 Also, @code{yyerror} can be a variadic function; that is why the
6105 message is always passed last.
6106
6107 Traditionally @code{yyerror} returns an @code{int} that is always
6108 ignored, but this is purely for historical reasons, and @code{void} is
6109 preferable since it more accurately describes the return type for
6110 @code{yyerror}.
6111
6112 @vindex yynerrs
6113 The variable @code{yynerrs} contains the number of syntax errors
6114 reported so far. Normally this variable is global; but if you
6115 request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
6116 then it is a local variable which only the actions can access.
6117
6118 @node Action Features
6119 @section Special Features for Use in Actions
6120 @cindex summary, action features
6121 @cindex action features summary
6122
6123 Here is a table of Bison constructs, variables and macros that
6124 are useful in actions.
6125
6126 @deffn {Variable} $$
6127 Acts like a variable that contains the semantic value for the
6128 grouping made by the current rule. @xref{Actions}.
6129 @end deffn
6130
6131 @deffn {Variable} $@var{n}
6132 Acts like a variable that contains the semantic value for the
6133 @var{n}th component of the current rule. @xref{Actions}.
6134 @end deffn
6135
6136 @deffn {Variable} $<@var{typealt}>$
6137 Like @code{$$} but specifies alternative @var{typealt} in the union
6138 specified by the @code{%union} declaration. @xref{Action Types, ,Data
6139 Types of Values in Actions}.
6140 @end deffn
6141
6142 @deffn {Variable} $<@var{typealt}>@var{n}
6143 Like @code{$@var{n}} but specifies alternative @var{typealt} in the
6144 union specified by the @code{%union} declaration.
6145 @xref{Action Types, ,Data Types of Values in Actions}.
6146 @end deffn
6147
6148 @deffn {Macro} YYABORT;
6149 Return immediately from @code{yyparse}, indicating failure.
6150 @xref{Parser Function, ,The Parser Function @code{yyparse}}.
6151 @end deffn
6152
6153 @deffn {Macro} YYACCEPT;
6154 Return immediately from @code{yyparse}, indicating success.
6155 @xref{Parser Function, ,The Parser Function @code{yyparse}}.
6156 @end deffn
6157
6158 @deffn {Macro} YYBACKUP (@var{token}, @var{value});
6159 @findex YYBACKUP
6160 Unshift a token. This macro is allowed only for rules that reduce
6161 a single value, and only when there is no lookahead token.
6162 It is also disallowed in GLR parsers.
6163 It installs a lookahead token with token type @var{token} and
6164 semantic value @var{value}; then it discards the value that was
6165 going to be reduced by this rule.
6166
6167 If the macro is used when it is not valid, such as when there is
6168 a lookahead token already, then it reports a syntax error with
6169 a message @samp{cannot back up} and performs ordinary error
6170 recovery.
6171
6172 In either case, the rest of the action is not executed.
6173 @end deffn
6174
6175 @deffn {Macro} YYEMPTY
6176 @vindex YYEMPTY
6177 Value stored in @code{yychar} when there is no lookahead token.
6178 @end deffn
6179
6180 @deffn {Macro} YYEOF
6181 @vindex YYEOF
6182 Value stored in @code{yychar} when the lookahead is the end of the input
6183 stream.
6184 @end deffn
6185
6186 @deffn {Macro} YYERROR;
6187 @findex YYERROR
6188 Cause an immediate syntax error. This statement initiates error
6189 recovery just as if the parser itself had detected an error; however, it
6190 does not call @code{yyerror}, and does not print any message. If you
6191 want to print an error message, call @code{yyerror} explicitly before
6192 the @samp{YYERROR;} statement. @xref{Error Recovery}.
6193 @end deffn
6194
6195 @deffn {Macro} YYRECOVERING
6196 @findex YYRECOVERING
6197 The expression @code{YYRECOVERING ()} yields 1 when the parser
6198 is recovering from a syntax error, and 0 otherwise.
6199 @xref{Error Recovery}.
6200 @end deffn
6201
6202 @deffn {Variable} yychar
6203 Variable containing either the lookahead token, or @code{YYEOF} when the
6204 lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
6205 has been performed so the next token is not yet known.
6206 Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
6207 Actions}).
6208 @xref{Lookahead, ,Lookahead Tokens}.
6209 @end deffn
6210
6211 @deffn {Macro} yyclearin;
6212 Discard the current lookahead token. This is useful primarily in
6213 error rules.
6214 Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
6215 Semantic Actions}).
6216 @xref{Error Recovery}.
6217 @end deffn
6218
6219 @deffn {Macro} yyerrok;
6220 Resume generating error messages immediately for subsequent syntax
6221 errors. This is useful primarily in error rules.
6222 @xref{Error Recovery}.
6223 @end deffn
6224
6225 @deffn {Variable} yylloc
6226 Variable containing the lookahead token location when @code{yychar} is not set
6227 to @code{YYEMPTY} or @code{YYEOF}.
6228 Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
6229 Actions}).
6230 @xref{Actions and Locations, ,Actions and Locations}.
6231 @end deffn
6232
6233 @deffn {Variable} yylval
6234 Variable containing the lookahead token semantic value when @code{yychar} is
6235 not set to @code{YYEMPTY} or @code{YYEOF}.
6236 Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
6237 Actions}).
6238 @xref{Actions, ,Actions}.
6239 @end deffn
6240
6241 @deffn {Value} @@$
6242 @findex @@$
6243 Acts like a structure variable containing information on the textual location
6244 of the grouping made by the current rule. @xref{Locations, ,
6245 Tracking Locations}.
6246
6247 @c Check if those paragraphs are still useful or not.
6248
6249 @c @example
6250 @c struct @{
6251 @c int first_line, last_line;
6252 @c int first_column, last_column;
6253 @c @};
6254 @c @end example
6255
6256 @c Thus, to get the starting line number of the third component, you would
6257 @c use @samp{@@3.first_line}.
6258
6259 @c In order for the members of this structure to contain valid information,
6260 @c you must make @code{yylex} supply this information about each token.
6261 @c If you need only certain members, then @code{yylex} need only fill in
6262 @c those members.
6263
6264 @c The use of this feature makes the parser noticeably slower.
6265 @end deffn
6266
6267 @deffn {Value} @@@var{n}
6268 @findex @@@var{n}
6269 Acts like a structure variable containing information on the textual location
6270 of the @var{n}th component of the current rule. @xref{Locations, ,
6271 Tracking Locations}.
6272 @end deffn
6273
6274 @node Internationalization
6275 @section Parser Internationalization
6276 @cindex internationalization
6277 @cindex i18n
6278 @cindex NLS
6279 @cindex gettext
6280 @cindex bison-po
6281
6282 A Bison-generated parser can print diagnostics, including error and
6283 tracing messages. By default, they appear in English. However, Bison
6284 also supports outputting diagnostics in the user's native language. To
6285 make this work, the user should set the usual environment variables.
6286 @xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
6287 For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
6288 set the user's locale to French Canadian using the UTF-8
6289 encoding. The exact set of available locales depends on the user's
6290 installation.
6291
6292 The maintainer of a package that uses a Bison-generated parser enables
6293 the internationalization of the parser's output through the following
6294 steps. Here we assume a package that uses GNU Autoconf and
6295 GNU Automake.
6296
6297 @enumerate
6298 @item
6299 @cindex bison-i18n.m4
6300 Into the directory containing the GNU Autoconf macros used
6301 by the package---often called @file{m4}---copy the
6302 @file{bison-i18n.m4} file installed by Bison under
6303 @samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
6304 For example:
6305
6306 @example
6307 cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
6308 @end example
6309
6310 @item
6311 @findex BISON_I18N
6312 @vindex BISON_LOCALEDIR
6313 @vindex YYENABLE_NLS
6314 In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
6315 invocation, add an invocation of @code{BISON_I18N}. This macro is
6316 defined in the file @file{bison-i18n.m4} that you copied earlier. It
6317 causes @samp{configure} to find the value of the
6318 @code{BISON_LOCALEDIR} variable, and it defines the source-language
6319 symbol @code{YYENABLE_NLS} to enable translations in the
6320 Bison-generated parser.
6321
6322 @item
6323 In the @code{main} function of your program, designate the directory
6324 containing Bison's runtime message catalog, through a call to
6325 @samp{bindtextdomain} with domain name @samp{bison-runtime}.
6326 For example:
6327
6328 @example
6329 bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
6330 @end example
6331
6332 Typically this appears after any other call @code{bindtextdomain
6333 (PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
6334 @samp{BISON_LOCALEDIR} to be defined as a string through the
6335 @file{Makefile}.
6336
6337 @item
6338 In the @file{Makefile.am} that controls the compilation of the @code{main}
6339 function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
6340 either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
6341
6342 @example
6343 DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6344 @end example
6345
6346 or:
6347
6348 @example
6349 AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6350 @end example
6351
6352 @item
6353 Finally, invoke the command @command{autoreconf} to generate the build
6354 infrastructure.
6355 @end enumerate
6356
6357
6358 @node Algorithm
6359 @chapter The Bison Parser Algorithm
6360 @cindex Bison parser algorithm
6361 @cindex algorithm of parser
6362 @cindex shifting
6363 @cindex reduction
6364 @cindex parser stack
6365 @cindex stack, parser
6366
6367 As Bison reads tokens, it pushes them onto a stack along with their
6368 semantic values. The stack is called the @dfn{parser stack}. Pushing a
6369 token is traditionally called @dfn{shifting}.
6370
6371 For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
6372 @samp{3} to come. The stack will have four elements, one for each token
6373 that was shifted.
6374
6375 But the stack does not always have an element for each token read. When
6376 the last @var{n} tokens and groupings shifted match the components of a
6377 grammar rule, they can be combined according to that rule. This is called
6378 @dfn{reduction}. Those tokens and groupings are replaced on the stack by a
6379 single grouping whose symbol is the result (left hand side) of that rule.
6380 Running the rule's action is part of the process of reduction, because this
6381 is what computes the semantic value of the resulting grouping.
6382
6383 For example, if the infix calculator's parser stack contains this:
6384
6385 @example
6386 1 + 5 * 3
6387 @end example
6388
6389 @noindent
6390 and the next input token is a newline character, then the last three
6391 elements can be reduced to 15 via the rule:
6392
6393 @example
6394 expr: expr '*' expr;
6395 @end example
6396
6397 @noindent
6398 Then the stack contains just these three elements:
6399
6400 @example
6401 1 + 15
6402 @end example
6403
6404 @noindent
6405 At this point, another reduction can be made, resulting in the single value
6406 16. Then the newline token can be shifted.
6407
6408 The parser tries, by shifts and reductions, to reduce the entire input down
6409 to a single grouping whose symbol is the grammar's start-symbol
6410 (@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
6411
6412 This kind of parser is known in the literature as a bottom-up parser.
6413
6414 @menu
6415 * Lookahead:: Parser looks one token ahead when deciding what to do.
6416 * Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
6417 * Precedence:: Operator precedence works by resolving conflicts.
6418 * Contextual Precedence:: When an operator's precedence depends on context.
6419 * Parser States:: The parser is a finite-state-machine with stack.
6420 * Reduce/Reduce:: When two rules are applicable in the same situation.
6421 * Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
6422 * Generalized LR Parsing:: Parsing arbitrary context-free grammars.
6423 * Memory Management:: What happens when memory is exhausted. How to avoid it.
6424 @end menu
6425
6426 @node Lookahead
6427 @section Lookahead Tokens
6428 @cindex lookahead token
6429
6430 The Bison parser does @emph{not} always reduce immediately as soon as the
6431 last @var{n} tokens and groupings match a rule. This is because such a
6432 simple strategy is inadequate to handle most languages. Instead, when a
6433 reduction is possible, the parser sometimes ``looks ahead'' at the next
6434 token in order to decide what to do.
6435
6436 When a token is read, it is not immediately shifted; first it becomes the
6437 @dfn{lookahead token}, which is not on the stack. Now the parser can
6438 perform one or more reductions of tokens and groupings on the stack, while
6439 the lookahead token remains off to the side. When no more reductions
6440 should take place, the lookahead token is shifted onto the stack. This
6441 does not mean that all possible reductions have been done; depending on the
6442 token type of the lookahead token, some rules may choose to delay their
6443 application.
6444
6445 Here is a simple case where lookahead is needed. These three rules define
6446 expressions which contain binary addition operators and postfix unary
6447 factorial operators (@samp{!}), and allow parentheses for grouping.
6448
6449 @example
6450 @group
6451 expr: term '+' expr
6452 | term
6453 ;
6454 @end group
6455
6456 @group
6457 term: '(' expr ')'
6458 | term '!'
6459 | NUMBER
6460 ;
6461 @end group
6462 @end example
6463
6464 Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
6465 should be done? If the following token is @samp{)}, then the first three
6466 tokens must be reduced to form an @code{expr}. This is the only valid
6467 course, because shifting the @samp{)} would produce a sequence of symbols
6468 @w{@code{term ')'}}, and no rule allows this.
6469
6470 If the following token is @samp{!}, then it must be shifted immediately so
6471 that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
6472 parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
6473 @code{expr}. It would then be impossible to shift the @samp{!} because
6474 doing so would produce on the stack the sequence of symbols @code{expr
6475 '!'}. No rule allows that sequence.
6476
6477 @vindex yychar
6478 @vindex yylval
6479 @vindex yylloc
6480 The lookahead token is stored in the variable @code{yychar}.
6481 Its semantic value and location, if any, are stored in the variables
6482 @code{yylval} and @code{yylloc}.
6483 @xref{Action Features, ,Special Features for Use in Actions}.
6484
6485 @node Shift/Reduce
6486 @section Shift/Reduce Conflicts
6487 @cindex conflicts
6488 @cindex shift/reduce conflicts
6489 @cindex dangling @code{else}
6490 @cindex @code{else}, dangling
6491
6492 Suppose we are parsing a language which has if-then and if-then-else
6493 statements, with a pair of rules like this:
6494
6495 @example
6496 @group
6497 if_stmt:
6498 IF expr THEN stmt
6499 | IF expr THEN stmt ELSE stmt
6500 ;
6501 @end group
6502 @end example
6503
6504 @noindent
6505 Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
6506 terminal symbols for specific keyword tokens.
6507
6508 When the @code{ELSE} token is read and becomes the lookahead token, the
6509 contents of the stack (assuming the input is valid) are just right for
6510 reduction by the first rule. But it is also legitimate to shift the
6511 @code{ELSE}, because that would lead to eventual reduction by the second
6512 rule.
6513
6514 This situation, where either a shift or a reduction would be valid, is
6515 called a @dfn{shift/reduce conflict}. Bison is designed to resolve
6516 these conflicts by choosing to shift, unless otherwise directed by
6517 operator precedence declarations. To see the reason for this, let's
6518 contrast it with the other alternative.
6519
6520 Since the parser prefers to shift the @code{ELSE}, the result is to attach
6521 the else-clause to the innermost if-statement, making these two inputs
6522 equivalent:
6523
6524 @example
6525 if x then if y then win (); else lose;
6526
6527 if x then do; if y then win (); else lose; end;
6528 @end example
6529
6530 But if the parser chose to reduce when possible rather than shift, the
6531 result would be to attach the else-clause to the outermost if-statement,
6532 making these two inputs equivalent:
6533
6534 @example
6535 if x then if y then win (); else lose;
6536
6537 if x then do; if y then win (); end; else lose;
6538 @end example
6539
6540 The conflict exists because the grammar as written is ambiguous: either
6541 parsing of the simple nested if-statement is legitimate. The established
6542 convention is that these ambiguities are resolved by attaching the
6543 else-clause to the innermost if-statement; this is what Bison accomplishes
6544 by choosing to shift rather than reduce. (It would ideally be cleaner to
6545 write an unambiguous grammar, but that is very hard to do in this case.)
6546 This particular ambiguity was first encountered in the specifications of
6547 Algol 60 and is called the ``dangling @code{else}'' ambiguity.
6548
6549 To avoid warnings from Bison about predictable, legitimate shift/reduce
6550 conflicts, use the @code{%expect @var{n}} declaration.
6551 There will be no warning as long as the number of shift/reduce conflicts
6552 is exactly @var{n}, and Bison will report an error if there is a
6553 different number.
6554 @xref{Expect Decl, ,Suppressing Conflict Warnings}.
6555
6556 The definition of @code{if_stmt} above is solely to blame for the
6557 conflict, but the conflict does not actually appear without additional
6558 rules. Here is a complete Bison input file that actually manifests the
6559 conflict:
6560
6561 @example
6562 @group
6563 %token IF THEN ELSE variable
6564 %%
6565 @end group
6566 @group
6567 stmt: expr
6568 | if_stmt
6569 ;
6570 @end group
6571
6572 @group
6573 if_stmt:
6574 IF expr THEN stmt
6575 | IF expr THEN stmt ELSE stmt
6576 ;
6577 @end group
6578
6579 expr: variable
6580 ;
6581 @end example
6582
6583 @node Precedence
6584 @section Operator Precedence
6585 @cindex operator precedence
6586 @cindex precedence of operators
6587
6588 Another situation where shift/reduce conflicts appear is in arithmetic
6589 expressions. Here shifting is not always the preferred resolution; the
6590 Bison declarations for operator precedence allow you to specify when to
6591 shift and when to reduce.
6592
6593 @menu
6594 * Why Precedence:: An example showing why precedence is needed.
6595 * Using Precedence:: How to specify precedence in Bison grammars.
6596 * Precedence Examples:: How these features are used in the previous example.
6597 * How Precedence:: How they work.
6598 @end menu
6599
6600 @node Why Precedence
6601 @subsection When Precedence is Needed
6602
6603 Consider the following ambiguous grammar fragment (ambiguous because the
6604 input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
6605
6606 @example
6607 @group
6608 expr: expr '-' expr
6609 | expr '*' expr
6610 | expr '<' expr
6611 | '(' expr ')'
6612 @dots{}
6613 ;
6614 @end group
6615 @end example
6616
6617 @noindent
6618 Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
6619 should it reduce them via the rule for the subtraction operator? It
6620 depends on the next token. Of course, if the next token is @samp{)}, we
6621 must reduce; shifting is invalid because no single rule can reduce the
6622 token sequence @w{@samp{- 2 )}} or anything starting with that. But if
6623 the next token is @samp{*} or @samp{<}, we have a choice: either
6624 shifting or reduction would allow the parse to complete, but with
6625 different results.
6626
6627 To decide which one Bison should do, we must consider the results. If
6628 the next operator token @var{op} is shifted, then it must be reduced
6629 first in order to permit another opportunity to reduce the difference.
6630 The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
6631 hand, if the subtraction is reduced before shifting @var{op}, the result
6632 is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
6633 reduce should depend on the relative precedence of the operators
6634 @samp{-} and @var{op}: @samp{*} should be shifted first, but not
6635 @samp{<}.
6636
6637 @cindex associativity
6638 What about input such as @w{@samp{1 - 2 - 5}}; should this be
6639 @w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
6640 operators we prefer the former, which is called @dfn{left association}.
6641 The latter alternative, @dfn{right association}, is desirable for
6642 assignment operators. The choice of left or right association is a
6643 matter of whether the parser chooses to shift or reduce when the stack
6644 contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
6645 makes right-associativity.
6646
6647 @node Using Precedence
6648 @subsection Specifying Operator Precedence
6649 @findex %left
6650 @findex %right
6651 @findex %nonassoc
6652
6653 Bison allows you to specify these choices with the operator precedence
6654 declarations @code{%left} and @code{%right}. Each such declaration
6655 contains a list of tokens, which are operators whose precedence and
6656 associativity is being declared. The @code{%left} declaration makes all
6657 those operators left-associative and the @code{%right} declaration makes
6658 them right-associative. A third alternative is @code{%nonassoc}, which
6659 declares that it is a syntax error to find the same operator twice ``in a
6660 row''.
6661
6662 The relative precedence of different operators is controlled by the
6663 order in which they are declared. The first @code{%left} or
6664 @code{%right} declaration in the file declares the operators whose
6665 precedence is lowest, the next such declaration declares the operators
6666 whose precedence is a little higher, and so on.
6667
6668 @node Precedence Examples
6669 @subsection Precedence Examples
6670
6671 In our example, we would want the following declarations:
6672
6673 @example
6674 %left '<'
6675 %left '-'
6676 %left '*'
6677 @end example
6678
6679 In a more complete example, which supports other operators as well, we
6680 would declare them in groups of equal precedence. For example, @code{'+'} is
6681 declared with @code{'-'}:
6682
6683 @example
6684 %left '<' '>' '=' NE LE GE
6685 %left '+' '-'
6686 %left '*' '/'
6687 @end example
6688
6689 @noindent
6690 (Here @code{NE} and so on stand for the operators for ``not equal''
6691 and so on. We assume that these tokens are more than one character long
6692 and therefore are represented by names, not character literals.)
6693
6694 @node How Precedence
6695 @subsection How Precedence Works
6696
6697 The first effect of the precedence declarations is to assign precedence
6698 levels to the terminal symbols declared. The second effect is to assign
6699 precedence levels to certain rules: each rule gets its precedence from
6700 the last terminal symbol mentioned in the components. (You can also
6701 specify explicitly the precedence of a rule. @xref{Contextual
6702 Precedence, ,Context-Dependent Precedence}.)
6703
6704 Finally, the resolution of conflicts works by comparing the precedence
6705 of the rule being considered with that of the lookahead token. If the
6706 token's precedence is higher, the choice is to shift. If the rule's
6707 precedence is higher, the choice is to reduce. If they have equal
6708 precedence, the choice is made based on the associativity of that
6709 precedence level. The verbose output file made by @samp{-v}
6710 (@pxref{Invocation, ,Invoking Bison}) says how each conflict was
6711 resolved.
6712
6713 Not all rules and not all tokens have precedence. If either the rule or
6714 the lookahead token has no precedence, then the default is to shift.
6715
6716 @node Contextual Precedence
6717 @section Context-Dependent Precedence
6718 @cindex context-dependent precedence
6719 @cindex unary operator precedence
6720 @cindex precedence, context-dependent
6721 @cindex precedence, unary operator
6722 @findex %prec
6723
6724 Often the precedence of an operator depends on the context. This sounds
6725 outlandish at first, but it is really very common. For example, a minus
6726 sign typically has a very high precedence as a unary operator, and a
6727 somewhat lower precedence (lower than multiplication) as a binary operator.
6728
6729 The Bison precedence declarations, @code{%left}, @code{%right} and
6730 @code{%nonassoc}, can only be used once for a given token; so a token has
6731 only one precedence declared in this way. For context-dependent
6732 precedence, you need to use an additional mechanism: the @code{%prec}
6733 modifier for rules.
6734
6735 The @code{%prec} modifier declares the precedence of a particular rule by
6736 specifying a terminal symbol whose precedence should be used for that rule.
6737 It's not necessary for that symbol to appear otherwise in the rule. The
6738 modifier's syntax is:
6739
6740 @example
6741 %prec @var{terminal-symbol}
6742 @end example
6743
6744 @noindent
6745 and it is written after the components of the rule. Its effect is to
6746 assign the rule the precedence of @var{terminal-symbol}, overriding
6747 the precedence that would be deduced for it in the ordinary way. The
6748 altered rule precedence then affects how conflicts involving that rule
6749 are resolved (@pxref{Precedence, ,Operator Precedence}).
6750
6751 Here is how @code{%prec} solves the problem of unary minus. First, declare
6752 a precedence for a fictitious terminal symbol named @code{UMINUS}. There
6753 are no tokens of this type, but the symbol serves to stand for its
6754 precedence:
6755
6756 @example
6757 @dots{}
6758 %left '+' '-'
6759 %left '*'
6760 %left UMINUS
6761 @end example
6762
6763 Now the precedence of @code{UMINUS} can be used in specific rules:
6764
6765 @example
6766 @group
6767 exp: @dots{}
6768 | exp '-' exp
6769 @dots{}
6770 | '-' exp %prec UMINUS
6771 @end group
6772 @end example
6773
6774 @ifset defaultprec
6775 If you forget to append @code{%prec UMINUS} to the rule for unary
6776 minus, Bison silently assumes that minus has its usual precedence.
6777 This kind of problem can be tricky to debug, since one typically
6778 discovers the mistake only by testing the code.
6779
6780 The @code{%no-default-prec;} declaration makes it easier to discover
6781 this kind of problem systematically. It causes rules that lack a
6782 @code{%prec} modifier to have no precedence, even if the last terminal
6783 symbol mentioned in their components has a declared precedence.
6784
6785 If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
6786 for all rules that participate in precedence conflict resolution.
6787 Then you will see any shift/reduce conflict until you tell Bison how
6788 to resolve it, either by changing your grammar or by adding an
6789 explicit precedence. This will probably add declarations to the
6790 grammar, but it helps to protect against incorrect rule precedences.
6791
6792 The effect of @code{%no-default-prec;} can be reversed by giving
6793 @code{%default-prec;}, which is the default.
6794 @end ifset
6795
6796 @node Parser States
6797 @section Parser States
6798 @cindex finite-state machine
6799 @cindex parser state
6800 @cindex state (of parser)
6801
6802 The function @code{yyparse} is implemented using a finite-state machine.
6803 The values pushed on the parser stack are not simply token type codes; they
6804 represent the entire sequence of terminal and nonterminal symbols at or
6805 near the top of the stack. The current state collects all the information
6806 about previous input which is relevant to deciding what to do next.
6807
6808 Each time a lookahead token is read, the current parser state together
6809 with the type of lookahead token are looked up in a table. This table
6810 entry can say, ``Shift the lookahead token.'' In this case, it also
6811 specifies the new parser state, which is pushed onto the top of the
6812 parser stack. Or it can say, ``Reduce using rule number @var{n}.''
6813 This means that a certain number of tokens or groupings are taken off
6814 the top of the stack, and replaced by one grouping. In other words,
6815 that number of states are popped from the stack, and one new state is
6816 pushed.
6817
6818 There is one other alternative: the table can say that the lookahead token
6819 is erroneous in the current state. This causes error processing to begin
6820 (@pxref{Error Recovery}).
6821
6822 @node Reduce/Reduce
6823 @section Reduce/Reduce Conflicts
6824 @cindex reduce/reduce conflict
6825 @cindex conflicts, reduce/reduce
6826
6827 A reduce/reduce conflict occurs if there are two or more rules that apply
6828 to the same sequence of input. This usually indicates a serious error
6829 in the grammar.
6830
6831 For example, here is an erroneous attempt to define a sequence
6832 of zero or more @code{word} groupings.
6833
6834 @example
6835 sequence: /* empty */
6836 @{ printf ("empty sequence\n"); @}
6837 | maybeword
6838 | sequence word
6839 @{ printf ("added word %s\n", $2); @}
6840 ;
6841
6842 maybeword: /* empty */
6843 @{ printf ("empty maybeword\n"); @}
6844 | word
6845 @{ printf ("single word %s\n", $1); @}
6846 ;
6847 @end example
6848
6849 @noindent
6850 The error is an ambiguity: there is more than one way to parse a single
6851 @code{word} into a @code{sequence}. It could be reduced to a
6852 @code{maybeword} and then into a @code{sequence} via the second rule.
6853 Alternatively, nothing-at-all could be reduced into a @code{sequence}
6854 via the first rule, and this could be combined with the @code{word}
6855 using the third rule for @code{sequence}.
6856
6857 There is also more than one way to reduce nothing-at-all into a
6858 @code{sequence}. This can be done directly via the first rule,
6859 or indirectly via @code{maybeword} and then the second rule.
6860
6861 You might think that this is a distinction without a difference, because it
6862 does not change whether any particular input is valid or not. But it does
6863 affect which actions are run. One parsing order runs the second rule's
6864 action; the other runs the first rule's action and the third rule's action.
6865 In this example, the output of the program changes.
6866
6867 Bison resolves a reduce/reduce conflict by choosing to use the rule that
6868 appears first in the grammar, but it is very risky to rely on this. Every
6869 reduce/reduce conflict must be studied and usually eliminated. Here is the
6870 proper way to define @code{sequence}:
6871
6872 @example
6873 sequence: /* empty */
6874 @{ printf ("empty sequence\n"); @}
6875 | sequence word
6876 @{ printf ("added word %s\n", $2); @}
6877 ;
6878 @end example
6879
6880 Here is another common error that yields a reduce/reduce conflict:
6881
6882 @example
6883 sequence: /* empty */
6884 | sequence words
6885 | sequence redirects
6886 ;
6887
6888 words: /* empty */
6889 | words word
6890 ;
6891
6892 redirects:/* empty */
6893 | redirects redirect
6894 ;
6895 @end example
6896
6897 @noindent
6898 The intention here is to define a sequence which can contain either
6899 @code{word} or @code{redirect} groupings. The individual definitions of
6900 @code{sequence}, @code{words} and @code{redirects} are error-free, but the
6901 three together make a subtle ambiguity: even an empty input can be parsed
6902 in infinitely many ways!
6903
6904 Consider: nothing-at-all could be a @code{words}. Or it could be two
6905 @code{words} in a row, or three, or any number. It could equally well be a
6906 @code{redirects}, or two, or any number. Or it could be a @code{words}
6907 followed by three @code{redirects} and another @code{words}. And so on.
6908
6909 Here are two ways to correct these rules. First, to make it a single level
6910 of sequence:
6911
6912 @example
6913 sequence: /* empty */
6914 | sequence word
6915 | sequence redirect
6916 ;
6917 @end example
6918
6919 Second, to prevent either a @code{words} or a @code{redirects}
6920 from being empty:
6921
6922 @example
6923 sequence: /* empty */
6924 | sequence words
6925 | sequence redirects
6926 ;
6927
6928 words: word
6929 | words word
6930 ;
6931
6932 redirects:redirect
6933 | redirects redirect
6934 ;
6935 @end example
6936
6937 @node Mystery Conflicts
6938 @section Mysterious Reduce/Reduce Conflicts
6939
6940 Sometimes reduce/reduce conflicts can occur that don't look warranted.
6941 Here is an example:
6942
6943 @example
6944 @group
6945 %token ID
6946
6947 %%
6948 def: param_spec return_spec ','
6949 ;
6950 param_spec:
6951 type
6952 | name_list ':' type
6953 ;
6954 @end group
6955 @group
6956 return_spec:
6957 type
6958 | name ':' type
6959 ;
6960 @end group
6961 @group
6962 type: ID
6963 ;
6964 @end group
6965 @group
6966 name: ID
6967 ;
6968 name_list:
6969 name
6970 | name ',' name_list
6971 ;
6972 @end group
6973 @end example
6974
6975 It would seem that this grammar can be parsed with only a single token
6976 of lookahead: when a @code{param_spec} is being read, an @code{ID} is
6977 a @code{name} if a comma or colon follows, or a @code{type} if another
6978 @code{ID} follows. In other words, this grammar is LR(1).
6979
6980 @cindex LR(1)
6981 @cindex LALR(1)
6982 However, for historical reasons, Bison cannot by default handle all
6983 LR(1) grammars.
6984 In this grammar, two contexts, that after an @code{ID} at the beginning
6985 of a @code{param_spec} and likewise at the beginning of a
6986 @code{return_spec}, are similar enough that Bison assumes they are the
6987 same.
6988 They appear similar because the same set of rules would be
6989 active---the rule for reducing to a @code{name} and that for reducing to
6990 a @code{type}. Bison is unable to determine at that stage of processing
6991 that the rules would require different lookahead tokens in the two
6992 contexts, so it makes a single parser state for them both. Combining
6993 the two contexts causes a conflict later. In parser terminology, this
6994 occurrence means that the grammar is not LALR(1).
6995
6996 For many practical grammars (specifically those that fall into the
6997 non-LR(1) class), the limitations of LALR(1) result in
6998 difficulties beyond just mysterious reduce/reduce conflicts.
6999 The best way to fix all these problems is to select a different parser
7000 table generation algorithm.
7001 Either IELR(1) or canonical LR(1) would suffice, but
7002 the former is more efficient and easier to debug during development.
7003 @xref{Decl Summary,,lr.type}, for details.
7004 (Bison's IELR(1) and canonical LR(1) implementations
7005 are experimental.
7006 More user feedback will help to stabilize them.)
7007
7008 If you instead wish to work around LALR(1)'s limitations, you
7009 can often fix a mysterious conflict by identifying the two parser states
7010 that are being confused, and adding something to make them look
7011 distinct. In the above example, adding one rule to
7012 @code{return_spec} as follows makes the problem go away:
7013
7014 @example
7015 @group
7016 %token BOGUS
7017 @dots{}
7018 %%
7019 @dots{}
7020 return_spec:
7021 type
7022 | name ':' type
7023 /* This rule is never used. */
7024 | ID BOGUS
7025 ;
7026 @end group
7027 @end example
7028
7029 This corrects the problem because it introduces the possibility of an
7030 additional active rule in the context after the @code{ID} at the beginning of
7031 @code{return_spec}. This rule is not active in the corresponding context
7032 in a @code{param_spec}, so the two contexts receive distinct parser states.
7033 As long as the token @code{BOGUS} is never generated by @code{yylex},
7034 the added rule cannot alter the way actual input is parsed.
7035
7036 In this particular example, there is another way to solve the problem:
7037 rewrite the rule for @code{return_spec} to use @code{ID} directly
7038 instead of via @code{name}. This also causes the two confusing
7039 contexts to have different sets of active rules, because the one for
7040 @code{return_spec} activates the altered rule for @code{return_spec}
7041 rather than the one for @code{name}.
7042
7043 @example
7044 param_spec:
7045 type
7046 | name_list ':' type
7047 ;
7048 return_spec:
7049 type
7050 | ID ':' type
7051 ;
7052 @end example
7053
7054 For a more detailed exposition of LALR(1) parsers and parser
7055 generators, please see:
7056 Frank DeRemer and Thomas Pennello, Efficient Computation of
7057 LALR(1) Look-Ahead Sets, @cite{ACM Transactions on
7058 Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
7059 pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
7060
7061 @node Generalized LR Parsing
7062 @section Generalized LR (GLR) Parsing
7063 @cindex GLR parsing
7064 @cindex generalized LR (GLR) parsing
7065 @cindex ambiguous grammars
7066 @cindex nondeterministic parsing
7067
7068 Bison produces @emph{deterministic} parsers that choose uniquely
7069 when to reduce and which reduction to apply
7070 based on a summary of the preceding input and on one extra token of lookahead.
7071 As a result, normal Bison handles a proper subset of the family of
7072 context-free languages.
7073 Ambiguous grammars, since they have strings with more than one possible
7074 sequence of reductions cannot have deterministic parsers in this sense.
7075 The same is true of languages that require more than one symbol of
7076 lookahead, since the parser lacks the information necessary to make a
7077 decision at the point it must be made in a shift-reduce parser.
7078 Finally, as previously mentioned (@pxref{Mystery Conflicts}),
7079 there are languages where Bison's default choice of how to
7080 summarize the input seen so far loses necessary information.
7081
7082 When you use the @samp{%glr-parser} declaration in your grammar file,
7083 Bison generates a parser that uses a different algorithm, called
7084 Generalized LR (or GLR). A Bison GLR
7085 parser uses the same basic
7086 algorithm for parsing as an ordinary Bison parser, but behaves
7087 differently in cases where there is a shift-reduce conflict that has not
7088 been resolved by precedence rules (@pxref{Precedence}) or a
7089 reduce-reduce conflict. When a GLR parser encounters such a
7090 situation, it
7091 effectively @emph{splits} into a several parsers, one for each possible
7092 shift or reduction. These parsers then proceed as usual, consuming
7093 tokens in lock-step. Some of the stacks may encounter other conflicts
7094 and split further, with the result that instead of a sequence of states,
7095 a Bison GLR parsing stack is what is in effect a tree of states.
7096
7097 In effect, each stack represents a guess as to what the proper parse
7098 is. Additional input may indicate that a guess was wrong, in which case
7099 the appropriate stack silently disappears. Otherwise, the semantics
7100 actions generated in each stack are saved, rather than being executed
7101 immediately. When a stack disappears, its saved semantic actions never
7102 get executed. When a reduction causes two stacks to become equivalent,
7103 their sets of semantic actions are both saved with the state that
7104 results from the reduction. We say that two stacks are equivalent
7105 when they both represent the same sequence of states,
7106 and each pair of corresponding states represents a
7107 grammar symbol that produces the same segment of the input token
7108 stream.
7109
7110 Whenever the parser makes a transition from having multiple
7111 states to having one, it reverts to the normal deterministic parsing
7112 algorithm, after resolving and executing the saved-up actions.
7113 At this transition, some of the states on the stack will have semantic
7114 values that are sets (actually multisets) of possible actions. The
7115 parser tries to pick one of the actions by first finding one whose rule
7116 has the highest dynamic precedence, as set by the @samp{%dprec}
7117 declaration. Otherwise, if the alternative actions are not ordered by
7118 precedence, but there the same merging function is declared for both
7119 rules by the @samp{%merge} declaration,
7120 Bison resolves and evaluates both and then calls the merge function on
7121 the result. Otherwise, it reports an ambiguity.
7122
7123 It is possible to use a data structure for the GLR parsing tree that
7124 permits the processing of any LR(1) grammar in linear time (in the
7125 size of the input), any unambiguous (not necessarily
7126 LR(1)) grammar in
7127 quadratic worst-case time, and any general (possibly ambiguous)
7128 context-free grammar in cubic worst-case time. However, Bison currently
7129 uses a simpler data structure that requires time proportional to the
7130 length of the input times the maximum number of stacks required for any
7131 prefix of the input. Thus, really ambiguous or nondeterministic
7132 grammars can require exponential time and space to process. Such badly
7133 behaving examples, however, are not generally of practical interest.
7134 Usually, nondeterminism in a grammar is local---the parser is ``in
7135 doubt'' only for a few tokens at a time. Therefore, the current data
7136 structure should generally be adequate. On LR(1) portions of a
7137 grammar, in particular, it is only slightly slower than with the
7138 deterministic LR(1) Bison parser.
7139
7140 For a more detailed exposition of GLR parsers, please see: Elizabeth
7141 Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
7142 Generalised LR Parsers, Royal Holloway, University of
7143 London, Department of Computer Science, TR-00-12,
7144 @uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
7145 (2000-12-24).
7146
7147 @node Memory Management
7148 @section Memory Management, and How to Avoid Memory Exhaustion
7149 @cindex memory exhaustion
7150 @cindex memory management
7151 @cindex stack overflow
7152 @cindex parser stack overflow
7153 @cindex overflow of parser stack
7154
7155 The Bison parser stack can run out of memory if too many tokens are shifted and
7156 not reduced. When this happens, the parser function @code{yyparse}
7157 calls @code{yyerror} and then returns 2.
7158
7159 Because Bison parsers have growing stacks, hitting the upper limit
7160 usually results from using a right recursion instead of a left
7161 recursion, @xref{Recursion, ,Recursive Rules}.
7162
7163 @vindex YYMAXDEPTH
7164 By defining the macro @code{YYMAXDEPTH}, you can control how deep the
7165 parser stack can become before memory is exhausted. Define the
7166 macro with a value that is an integer. This value is the maximum number
7167 of tokens that can be shifted (and not reduced) before overflow.
7168
7169 The stack space allowed is not necessarily allocated. If you specify a
7170 large value for @code{YYMAXDEPTH}, the parser normally allocates a small
7171 stack at first, and then makes it bigger by stages as needed. This
7172 increasing allocation happens automatically and silently. Therefore,
7173 you do not need to make @code{YYMAXDEPTH} painfully small merely to save
7174 space for ordinary inputs that do not need much stack.
7175
7176 However, do not allow @code{YYMAXDEPTH} to be a value so large that
7177 arithmetic overflow could occur when calculating the size of the stack
7178 space. Also, do not allow @code{YYMAXDEPTH} to be less than
7179 @code{YYINITDEPTH}.
7180
7181 @cindex default stack limit
7182 The default value of @code{YYMAXDEPTH}, if you do not define it, is
7183 10000.
7184
7185 @vindex YYINITDEPTH
7186 You can control how much stack is allocated initially by defining the
7187 macro @code{YYINITDEPTH} to a positive integer. For the deterministic
7188 parser in C, this value must be a compile-time constant
7189 unless you are assuming C99 or some other target language or compiler
7190 that allows variable-length arrays. The default is 200.
7191
7192 Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
7193
7194 @c FIXME: C++ output.
7195 Because of semantic differences between C and C++, the deterministic
7196 parsers in C produced by Bison cannot grow when compiled
7197 by C++ compilers. In this precise case (compiling a C parser as C++) you are
7198 suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix
7199 this deficiency in a future release.
7200
7201 @node Error Recovery
7202 @chapter Error Recovery
7203 @cindex error recovery
7204 @cindex recovery from errors
7205
7206 It is not usually acceptable to have a program terminate on a syntax
7207 error. For example, a compiler should recover sufficiently to parse the
7208 rest of the input file and check it for errors; a calculator should accept
7209 another expression.
7210
7211 In a simple interactive command parser where each input is one line, it may
7212 be sufficient to allow @code{yyparse} to return 1 on error and have the
7213 caller ignore the rest of the input line when that happens (and then call
7214 @code{yyparse} again). But this is inadequate for a compiler, because it
7215 forgets all the syntactic context leading up to the error. A syntax error
7216 deep within a function in the compiler input should not cause the compiler
7217 to treat the following line like the beginning of a source file.
7218
7219 @findex error
7220 You can define how to recover from a syntax error by writing rules to
7221 recognize the special token @code{error}. This is a terminal symbol that
7222 is always defined (you need not declare it) and reserved for error
7223 handling. The Bison parser generates an @code{error} token whenever a
7224 syntax error happens; if you have provided a rule to recognize this token
7225 in the current context, the parse can continue.
7226
7227 For example:
7228
7229 @example
7230 stmnts: /* empty string */
7231 | stmnts '\n'
7232 | stmnts exp '\n'
7233 | stmnts error '\n'
7234 @end example
7235
7236 The fourth rule in this example says that an error followed by a newline
7237 makes a valid addition to any @code{stmnts}.
7238
7239 What happens if a syntax error occurs in the middle of an @code{exp}? The
7240 error recovery rule, interpreted strictly, applies to the precise sequence
7241 of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
7242 the middle of an @code{exp}, there will probably be some additional tokens
7243 and subexpressions on the stack after the last @code{stmnts}, and there
7244 will be tokens to read before the next newline. So the rule is not
7245 applicable in the ordinary way.
7246
7247 But Bison can force the situation to fit the rule, by discarding part of
7248 the semantic context and part of the input. First it discards states
7249 and objects from the stack until it gets back to a state in which the
7250 @code{error} token is acceptable. (This means that the subexpressions
7251 already parsed are discarded, back to the last complete @code{stmnts}.)
7252 At this point the @code{error} token can be shifted. Then, if the old
7253 lookahead token is not acceptable to be shifted next, the parser reads
7254 tokens and discards them until it finds a token which is acceptable. In
7255 this example, Bison reads and discards input until the next newline so
7256 that the fourth rule can apply. Note that discarded symbols are
7257 possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
7258 Discarded Symbols}, for a means to reclaim this memory.
7259
7260 The choice of error rules in the grammar is a choice of strategies for
7261 error recovery. A simple and useful strategy is simply to skip the rest of
7262 the current input line or current statement if an error is detected:
7263
7264 @example
7265 stmnt: error ';' /* On error, skip until ';' is read. */
7266 @end example
7267
7268 It is also useful to recover to the matching close-delimiter of an
7269 opening-delimiter that has already been parsed. Otherwise the
7270 close-delimiter will probably appear to be unmatched, and generate another,
7271 spurious error message:
7272
7273 @example
7274 primary: '(' expr ')'
7275 | '(' error ')'
7276 @dots{}
7277 ;
7278 @end example
7279
7280 Error recovery strategies are necessarily guesses. When they guess wrong,
7281 one syntax error often leads to another. In the above example, the error
7282 recovery rule guesses that an error is due to bad input within one
7283 @code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
7284 middle of a valid @code{stmnt}. After the error recovery rule recovers
7285 from the first error, another syntax error will be found straightaway,
7286 since the text following the spurious semicolon is also an invalid
7287 @code{stmnt}.
7288
7289 To prevent an outpouring of error messages, the parser will output no error
7290 message for another syntax error that happens shortly after the first; only
7291 after three consecutive input tokens have been successfully shifted will
7292 error messages resume.
7293
7294 Note that rules which accept the @code{error} token may have actions, just
7295 as any other rules can.
7296
7297 @findex yyerrok
7298 You can make error messages resume immediately by using the macro
7299 @code{yyerrok} in an action. If you do this in the error rule's action, no
7300 error messages will be suppressed. This macro requires no arguments;
7301 @samp{yyerrok;} is a valid C statement.
7302
7303 @findex yyclearin
7304 The previous lookahead token is reanalyzed immediately after an error. If
7305 this is unacceptable, then the macro @code{yyclearin} may be used to clear
7306 this token. Write the statement @samp{yyclearin;} in the error rule's
7307 action.
7308 @xref{Action Features, ,Special Features for Use in Actions}.
7309
7310 For example, suppose that on a syntax error, an error handling routine is
7311 called that advances the input stream to some point where parsing should
7312 once again commence. The next symbol returned by the lexical scanner is
7313 probably correct. The previous lookahead token ought to be discarded
7314 with @samp{yyclearin;}.
7315
7316 @vindex YYRECOVERING
7317 The expression @code{YYRECOVERING ()} yields 1 when the parser
7318 is recovering from a syntax error, and 0 otherwise.
7319 Syntax error diagnostics are suppressed while recovering from a syntax
7320 error.
7321
7322 @node Context Dependency
7323 @chapter Handling Context Dependencies
7324
7325 The Bison paradigm is to parse tokens first, then group them into larger
7326 syntactic units. In many languages, the meaning of a token is affected by
7327 its context. Although this violates the Bison paradigm, certain techniques
7328 (known as @dfn{kludges}) may enable you to write Bison parsers for such
7329 languages.
7330
7331 @menu
7332 * Semantic Tokens:: Token parsing can depend on the semantic context.
7333 * Lexical Tie-ins:: Token parsing can depend on the syntactic context.
7334 * Tie-in Recovery:: Lexical tie-ins have implications for how
7335 error recovery rules must be written.
7336 @end menu
7337
7338 (Actually, ``kludge'' means any technique that gets its job done but is
7339 neither clean nor robust.)
7340
7341 @node Semantic Tokens
7342 @section Semantic Info in Token Types
7343
7344 The C language has a context dependency: the way an identifier is used
7345 depends on what its current meaning is. For example, consider this:
7346
7347 @example
7348 foo (x);
7349 @end example
7350
7351 This looks like a function call statement, but if @code{foo} is a typedef
7352 name, then this is actually a declaration of @code{x}. How can a Bison
7353 parser for C decide how to parse this input?
7354
7355 The method used in GNU C is to have two different token types,
7356 @code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
7357 identifier, it looks up the current declaration of the identifier in order
7358 to decide which token type to return: @code{TYPENAME} if the identifier is
7359 declared as a typedef, @code{IDENTIFIER} otherwise.
7360
7361 The grammar rules can then express the context dependency by the choice of
7362 token type to recognize. @code{IDENTIFIER} is accepted as an expression,
7363 but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
7364 @code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
7365 is @emph{not} significant, such as in declarations that can shadow a
7366 typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
7367 accepted---there is one rule for each of the two token types.
7368
7369 This technique is simple to use if the decision of which kinds of
7370 identifiers to allow is made at a place close to where the identifier is
7371 parsed. But in C this is not always so: C allows a declaration to
7372 redeclare a typedef name provided an explicit type has been specified
7373 earlier:
7374
7375 @example
7376 typedef int foo, bar;
7377 int baz (void)
7378 @{
7379 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
7380 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
7381 return foo (bar);
7382 @}
7383 @end example
7384
7385 Unfortunately, the name being declared is separated from the declaration
7386 construct itself by a complicated syntactic structure---the ``declarator''.
7387
7388 As a result, part of the Bison parser for C needs to be duplicated, with
7389 all the nonterminal names changed: once for parsing a declaration in
7390 which a typedef name can be redefined, and once for parsing a
7391 declaration in which that can't be done. Here is a part of the
7392 duplication, with actions omitted for brevity:
7393
7394 @example
7395 initdcl:
7396 declarator maybeasm '='
7397 init
7398 | declarator maybeasm
7399 ;
7400
7401 notype_initdcl:
7402 notype_declarator maybeasm '='
7403 init
7404 | notype_declarator maybeasm
7405 ;
7406 @end example
7407
7408 @noindent
7409 Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
7410 cannot. The distinction between @code{declarator} and
7411 @code{notype_declarator} is the same sort of thing.
7412
7413 There is some similarity between this technique and a lexical tie-in
7414 (described next), in that information which alters the lexical analysis is
7415 changed during parsing by other parts of the program. The difference is
7416 here the information is global, and is used for other purposes in the
7417 program. A true lexical tie-in has a special-purpose flag controlled by
7418 the syntactic context.
7419
7420 @node Lexical Tie-ins
7421 @section Lexical Tie-ins
7422 @cindex lexical tie-in
7423
7424 One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
7425 which is set by Bison actions, whose purpose is to alter the way tokens are
7426 parsed.
7427
7428 For example, suppose we have a language vaguely like C, but with a special
7429 construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
7430 an expression in parentheses in which all integers are hexadecimal. In
7431 particular, the token @samp{a1b} must be treated as an integer rather than
7432 as an identifier if it appears in that context. Here is how you can do it:
7433
7434 @example
7435 @group
7436 %@{
7437 int hexflag;
7438 int yylex (void);
7439 void yyerror (char const *);
7440 %@}
7441 %%
7442 @dots{}
7443 @end group
7444 @group
7445 expr: IDENTIFIER
7446 | constant
7447 | HEX '('
7448 @{ hexflag = 1; @}
7449 expr ')'
7450 @{ hexflag = 0;
7451 $$ = $4; @}
7452 | expr '+' expr
7453 @{ $$ = make_sum ($1, $3); @}
7454 @dots{}
7455 ;
7456 @end group
7457
7458 @group
7459 constant:
7460 INTEGER
7461 | STRING
7462 ;
7463 @end group
7464 @end example
7465
7466 @noindent
7467 Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
7468 it is nonzero, all integers are parsed in hexadecimal, and tokens starting
7469 with letters are parsed as integers if possible.
7470
7471 The declaration of @code{hexflag} shown in the prologue of the parser file
7472 is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}).
7473 You must also write the code in @code{yylex} to obey the flag.
7474
7475 @node Tie-in Recovery
7476 @section Lexical Tie-ins and Error Recovery
7477
7478 Lexical tie-ins make strict demands on any error recovery rules you have.
7479 @xref{Error Recovery}.
7480
7481 The reason for this is that the purpose of an error recovery rule is to
7482 abort the parsing of one construct and resume in some larger construct.
7483 For example, in C-like languages, a typical error recovery rule is to skip
7484 tokens until the next semicolon, and then start a new statement, like this:
7485
7486 @example
7487 stmt: expr ';'
7488 | IF '(' expr ')' stmt @{ @dots{} @}
7489 @dots{}
7490 error ';'
7491 @{ hexflag = 0; @}
7492 ;
7493 @end example
7494
7495 If there is a syntax error in the middle of a @samp{hex (@var{expr})}
7496 construct, this error rule will apply, and then the action for the
7497 completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
7498 remain set for the entire rest of the input, or until the next @code{hex}
7499 keyword, causing identifiers to be misinterpreted as integers.
7500
7501 To avoid this problem the error recovery rule itself clears @code{hexflag}.
7502
7503 There may also be an error recovery rule that works within expressions.
7504 For example, there could be a rule which applies within parentheses
7505 and skips to the close-parenthesis:
7506
7507 @example
7508 @group
7509 expr: @dots{}
7510 | '(' expr ')'
7511 @{ $$ = $2; @}
7512 | '(' error ')'
7513 @dots{}
7514 @end group
7515 @end example
7516
7517 If this rule acts within the @code{hex} construct, it is not going to abort
7518 that construct (since it applies to an inner level of parentheses within
7519 the construct). Therefore, it should not clear the flag: the rest of
7520 the @code{hex} construct should be parsed with the flag still in effect.
7521
7522 What if there is an error recovery rule which might abort out of the
7523 @code{hex} construct or might not, depending on circumstances? There is no
7524 way you can write the action to determine whether a @code{hex} construct is
7525 being aborted or not. So if you are using a lexical tie-in, you had better
7526 make sure your error recovery rules are not of this kind. Each rule must
7527 be such that you can be sure that it always will, or always won't, have to
7528 clear the flag.
7529
7530 @c ================================================== Debugging Your Parser
7531
7532 @node Debugging
7533 @chapter Debugging Your Parser
7534
7535 Developing a parser can be a challenge, especially if you don't
7536 understand the algorithm (@pxref{Algorithm, ,The Bison Parser
7537 Algorithm}). Even so, sometimes a detailed description of the automaton
7538 can help (@pxref{Understanding, , Understanding Your Parser}), or
7539 tracing the execution of the parser can give some insight on why it
7540 behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
7541
7542 @menu
7543 * Understanding:: Understanding the structure of your parser.
7544 * Tracing:: Tracing the execution of your parser.
7545 @end menu
7546
7547 @node Understanding
7548 @section Understanding Your Parser
7549
7550 As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
7551 Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
7552 frequent than one would hope), looking at this automaton is required to
7553 tune or simply fix a parser. Bison provides two different
7554 representation of it, either textually or graphically (as a DOT file).
7555
7556 The textual file is generated when the options @option{--report} or
7557 @option{--verbose} are specified, see @xref{Invocation, , Invoking
7558 Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
7559 the parser output file name, and adding @samp{.output} instead.
7560 Therefore, if the input file is @file{foo.y}, then the parser file is
7561 called @file{foo.tab.c} by default. As a consequence, the verbose
7562 output file is called @file{foo.output}.
7563
7564 The following grammar file, @file{calc.y}, will be used in the sequel:
7565
7566 @example
7567 %token NUM STR
7568 %left '+' '-'
7569 %left '*'
7570 %%
7571 exp: exp '+' exp
7572 | exp '-' exp
7573 | exp '*' exp
7574 | exp '/' exp
7575 | NUM
7576 ;
7577 useless: STR;
7578 %%
7579 @end example
7580
7581 @command{bison} reports:
7582
7583 @example
7584 calc.y: warning: 1 nonterminal useless in grammar
7585 calc.y: warning: 1 rule useless in grammar
7586 calc.y:11.1-7: warning: nonterminal useless in grammar: useless
7587 calc.y:11.10-12: warning: rule useless in grammar: useless: STR
7588 calc.y: conflicts: 7 shift/reduce
7589 @end example
7590
7591 When given @option{--report=state}, in addition to @file{calc.tab.c}, it
7592 creates a file @file{calc.output} with contents detailed below. The
7593 order of the output and the exact presentation might vary, but the
7594 interpretation is the same.
7595
7596 The first section includes details on conflicts that were solved thanks
7597 to precedence and/or associativity:
7598
7599 @example
7600 Conflict in state 8 between rule 2 and token '+' resolved as reduce.
7601 Conflict in state 8 between rule 2 and token '-' resolved as reduce.
7602 Conflict in state 8 between rule 2 and token '*' resolved as shift.
7603 @exdent @dots{}
7604 @end example
7605
7606 @noindent
7607 The next section lists states that still have conflicts.
7608
7609 @example
7610 State 8 conflicts: 1 shift/reduce
7611 State 9 conflicts: 1 shift/reduce
7612 State 10 conflicts: 1 shift/reduce
7613 State 11 conflicts: 4 shift/reduce
7614 @end example
7615
7616 @noindent
7617 @cindex token, useless
7618 @cindex useless token
7619 @cindex nonterminal, useless
7620 @cindex useless nonterminal
7621 @cindex rule, useless
7622 @cindex useless rule
7623 The next section reports useless tokens, nonterminal and rules. Useless
7624 nonterminals and rules are removed in order to produce a smaller parser,
7625 but useless tokens are preserved, since they might be used by the
7626 scanner (note the difference between ``useless'' and ``unused''
7627 below):
7628
7629 @example
7630 Nonterminals useless in grammar:
7631 useless
7632
7633 Terminals unused in grammar:
7634 STR
7635
7636 Rules useless in grammar:
7637 #6 useless: STR;
7638 @end example
7639
7640 @noindent
7641 The next section reproduces the exact grammar that Bison used:
7642
7643 @example
7644 Grammar
7645
7646 Number, Line, Rule
7647 0 5 $accept -> exp $end
7648 1 5 exp -> exp '+' exp
7649 2 6 exp -> exp '-' exp
7650 3 7 exp -> exp '*' exp
7651 4 8 exp -> exp '/' exp
7652 5 9 exp -> NUM
7653 @end example
7654
7655 @noindent
7656 and reports the uses of the symbols:
7657
7658 @example
7659 Terminals, with rules where they appear
7660
7661 $end (0) 0
7662 '*' (42) 3
7663 '+' (43) 1
7664 '-' (45) 2
7665 '/' (47) 4
7666 error (256)
7667 NUM (258) 5
7668
7669 Nonterminals, with rules where they appear
7670
7671 $accept (8)
7672 on left: 0
7673 exp (9)
7674 on left: 1 2 3 4 5, on right: 0 1 2 3 4
7675 @end example
7676
7677 @noindent
7678 @cindex item
7679 @cindex pointed rule
7680 @cindex rule, pointed
7681 Bison then proceeds onto the automaton itself, describing each state
7682 with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
7683 item is a production rule together with a point (marked by @samp{.})
7684 that the input cursor.
7685
7686 @example
7687 state 0
7688
7689 $accept -> . exp $ (rule 0)
7690
7691 NUM shift, and go to state 1
7692
7693 exp go to state 2
7694 @end example
7695
7696 This reads as follows: ``state 0 corresponds to being at the very
7697 beginning of the parsing, in the initial rule, right before the start
7698 symbol (here, @code{exp}). When the parser returns to this state right
7699 after having reduced a rule that produced an @code{exp}, the control
7700 flow jumps to state 2. If there is no such transition on a nonterminal
7701 symbol, and the lookahead is a @code{NUM}, then this token is shifted on
7702 the parse stack, and the control flow jumps to state 1. Any other
7703 lookahead triggers a syntax error.''
7704
7705 @cindex core, item set
7706 @cindex item set core
7707 @cindex kernel, item set
7708 @cindex item set core
7709 Even though the only active rule in state 0 seems to be rule 0, the
7710 report lists @code{NUM} as a lookahead token because @code{NUM} can be
7711 at the beginning of any rule deriving an @code{exp}. By default Bison
7712 reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
7713 you want to see more detail you can invoke @command{bison} with
7714 @option{--report=itemset} to list all the items, include those that can
7715 be derived:
7716
7717 @example
7718 state 0
7719
7720 $accept -> . exp $ (rule 0)
7721 exp -> . exp '+' exp (rule 1)
7722 exp -> . exp '-' exp (rule 2)
7723 exp -> . exp '*' exp (rule 3)
7724 exp -> . exp '/' exp (rule 4)
7725 exp -> . NUM (rule 5)
7726
7727 NUM shift, and go to state 1
7728
7729 exp go to state 2
7730 @end example
7731
7732 @noindent
7733 In the state 1...
7734
7735 @example
7736 state 1
7737
7738 exp -> NUM . (rule 5)
7739
7740 $default reduce using rule 5 (exp)
7741 @end example
7742
7743 @noindent
7744 the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
7745 (@samp{$default}), the parser will reduce it. If it was coming from
7746 state 0, then, after this reduction it will return to state 0, and will
7747 jump to state 2 (@samp{exp: go to state 2}).
7748
7749 @example
7750 state 2
7751
7752 $accept -> exp . $ (rule 0)
7753 exp -> exp . '+' exp (rule 1)
7754 exp -> exp . '-' exp (rule 2)
7755 exp -> exp . '*' exp (rule 3)
7756 exp -> exp . '/' exp (rule 4)
7757
7758 $ shift, and go to state 3
7759 '+' shift, and go to state 4
7760 '-' shift, and go to state 5
7761 '*' shift, and go to state 6
7762 '/' shift, and go to state 7
7763 @end example
7764
7765 @noindent
7766 In state 2, the automaton can only shift a symbol. For instance,
7767 because of the item @samp{exp -> exp . '+' exp}, if the lookahead if
7768 @samp{+}, it will be shifted on the parse stack, and the automaton
7769 control will jump to state 4, corresponding to the item @samp{exp -> exp
7770 '+' . exp}. Since there is no default action, any other token than
7771 those listed above will trigger a syntax error.
7772
7773 @cindex accepting state
7774 The state 3 is named the @dfn{final state}, or the @dfn{accepting
7775 state}:
7776
7777 @example
7778 state 3
7779
7780 $accept -> exp $ . (rule 0)
7781
7782 $default accept
7783 @end example
7784
7785 @noindent
7786 the initial rule is completed (the start symbol and the end
7787 of input were read), the parsing exits successfully.
7788
7789 The interpretation of states 4 to 7 is straightforward, and is left to
7790 the reader.
7791
7792 @example
7793 state 4
7794
7795 exp -> exp '+' . exp (rule 1)
7796
7797 NUM shift, and go to state 1
7798
7799 exp go to state 8
7800
7801 state 5
7802
7803 exp -> exp '-' . exp (rule 2)
7804
7805 NUM shift, and go to state 1
7806
7807 exp go to state 9
7808
7809 state 6
7810
7811 exp -> exp '*' . exp (rule 3)
7812
7813 NUM shift, and go to state 1
7814
7815 exp go to state 10
7816
7817 state 7
7818
7819 exp -> exp '/' . exp (rule 4)
7820
7821 NUM shift, and go to state 1
7822
7823 exp go to state 11
7824 @end example
7825
7826 As was announced in beginning of the report, @samp{State 8 conflicts:
7827 1 shift/reduce}:
7828
7829 @example
7830 state 8
7831
7832 exp -> exp . '+' exp (rule 1)
7833 exp -> exp '+' exp . (rule 1)
7834 exp -> exp . '-' exp (rule 2)
7835 exp -> exp . '*' exp (rule 3)
7836 exp -> exp . '/' exp (rule 4)
7837
7838 '*' shift, and go to state 6
7839 '/' shift, and go to state 7
7840
7841 '/' [reduce using rule 1 (exp)]
7842 $default reduce using rule 1 (exp)
7843 @end example
7844
7845 Indeed, there are two actions associated to the lookahead @samp{/}:
7846 either shifting (and going to state 7), or reducing rule 1. The
7847 conflict means that either the grammar is ambiguous, or the parser lacks
7848 information to make the right decision. Indeed the grammar is
7849 ambiguous, as, since we did not specify the precedence of @samp{/}, the
7850 sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
7851 NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
7852 NUM}, which corresponds to reducing rule 1.
7853
7854 Because in deterministic parsing a single decision can be made, Bison
7855 arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
7856 Shift/Reduce Conflicts}. Discarded actions are reported in between
7857 square brackets.
7858
7859 Note that all the previous states had a single possible action: either
7860 shifting the next token and going to the corresponding state, or
7861 reducing a single rule. In the other cases, i.e., when shifting
7862 @emph{and} reducing is possible or when @emph{several} reductions are
7863 possible, the lookahead is required to select the action. State 8 is
7864 one such state: if the lookahead is @samp{*} or @samp{/} then the action
7865 is shifting, otherwise the action is reducing rule 1. In other words,
7866 the first two items, corresponding to rule 1, are not eligible when the
7867 lookahead token is @samp{*}, since we specified that @samp{*} has higher
7868 precedence than @samp{+}. More generally, some items are eligible only
7869 with some set of possible lookahead tokens. When run with
7870 @option{--report=lookahead}, Bison specifies these lookahead tokens:
7871
7872 @example
7873 state 8
7874
7875 exp -> exp . '+' exp (rule 1)
7876 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
7877 exp -> exp . '-' exp (rule 2)
7878 exp -> exp . '*' exp (rule 3)
7879 exp -> exp . '/' exp (rule 4)
7880
7881 '*' shift, and go to state 6
7882 '/' shift, and go to state 7
7883
7884 '/' [reduce using rule 1 (exp)]
7885 $default reduce using rule 1 (exp)
7886 @end example
7887
7888 The remaining states are similar:
7889
7890 @example
7891 state 9
7892
7893 exp -> exp . '+' exp (rule 1)
7894 exp -> exp . '-' exp (rule 2)
7895 exp -> exp '-' exp . (rule 2)
7896 exp -> exp . '*' exp (rule 3)
7897 exp -> exp . '/' exp (rule 4)
7898
7899 '*' shift, and go to state 6
7900 '/' shift, and go to state 7
7901
7902 '/' [reduce using rule 2 (exp)]
7903 $default reduce using rule 2 (exp)
7904
7905 state 10
7906
7907 exp -> exp . '+' exp (rule 1)
7908 exp -> exp . '-' exp (rule 2)
7909 exp -> exp . '*' exp (rule 3)
7910 exp -> exp '*' exp . (rule 3)
7911 exp -> exp . '/' exp (rule 4)
7912
7913 '/' shift, and go to state 7
7914
7915 '/' [reduce using rule 3 (exp)]
7916 $default reduce using rule 3 (exp)
7917
7918 state 11
7919
7920 exp -> exp . '+' exp (rule 1)
7921 exp -> exp . '-' exp (rule 2)
7922 exp -> exp . '*' exp (rule 3)
7923 exp -> exp . '/' exp (rule 4)
7924 exp -> exp '/' exp . (rule 4)
7925
7926 '+' shift, and go to state 4
7927 '-' shift, and go to state 5
7928 '*' shift, and go to state 6
7929 '/' shift, and go to state 7
7930
7931 '+' [reduce using rule 4 (exp)]
7932 '-' [reduce using rule 4 (exp)]
7933 '*' [reduce using rule 4 (exp)]
7934 '/' [reduce using rule 4 (exp)]
7935 $default reduce using rule 4 (exp)
7936 @end example
7937
7938 @noindent
7939 Observe that state 11 contains conflicts not only due to the lack of
7940 precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
7941 @samp{*}, but also because the
7942 associativity of @samp{/} is not specified.
7943
7944
7945 @node Tracing
7946 @section Tracing Your Parser
7947 @findex yydebug
7948 @cindex debugging
7949 @cindex tracing the parser
7950
7951 If a Bison grammar compiles properly but doesn't do what you want when it
7952 runs, the @code{yydebug} parser-trace feature can help you figure out why.
7953
7954 There are several means to enable compilation of trace facilities:
7955
7956 @table @asis
7957 @item the macro @code{YYDEBUG}
7958 @findex YYDEBUG
7959 Define the macro @code{YYDEBUG} to a nonzero value when you compile the
7960 parser. This is compliant with POSIX Yacc. You could use
7961 @samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
7962 YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
7963 Prologue}).
7964
7965 @item the option @option{-t}, @option{--debug}
7966 Use the @samp{-t} option when you run Bison (@pxref{Invocation,
7967 ,Invoking Bison}). This is POSIX compliant too.
7968
7969 @item the directive @samp{%debug}
7970 @findex %debug
7971 Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison
7972 Declaration Summary}). This is a Bison extension, which will prove
7973 useful when Bison will output parsers for languages that don't use a
7974 preprocessor. Unless POSIX and Yacc portability matter to
7975 you, this is
7976 the preferred solution.
7977 @end table
7978
7979 We suggest that you always enable the debug option so that debugging is
7980 always possible.
7981
7982 The trace facility outputs messages with macro calls of the form
7983 @code{YYFPRINTF (stderr, @var{format}, @var{args})} where
7984 @var{format} and @var{args} are the usual @code{printf} format and variadic
7985 arguments. If you define @code{YYDEBUG} to a nonzero value but do not
7986 define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
7987 and @code{YYFPRINTF} is defined to @code{fprintf}.
7988
7989 Once you have compiled the program with trace facilities, the way to
7990 request a trace is to store a nonzero value in the variable @code{yydebug}.
7991 You can do this by making the C code do it (in @code{main}, perhaps), or
7992 you can alter the value with a C debugger.
7993
7994 Each step taken by the parser when @code{yydebug} is nonzero produces a
7995 line or two of trace information, written on @code{stderr}. The trace
7996 messages tell you these things:
7997
7998 @itemize @bullet
7999 @item
8000 Each time the parser calls @code{yylex}, what kind of token was read.
8001
8002 @item
8003 Each time a token is shifted, the depth and complete contents of the
8004 state stack (@pxref{Parser States}).
8005
8006 @item
8007 Each time a rule is reduced, which rule it is, and the complete contents
8008 of the state stack afterward.
8009 @end itemize
8010
8011 To make sense of this information, it helps to refer to the listing file
8012 produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
8013 Bison}). This file shows the meaning of each state in terms of
8014 positions in various rules, and also what each state will do with each
8015 possible input token. As you read the successive trace messages, you
8016 can see that the parser is functioning according to its specification in
8017 the listing file. Eventually you will arrive at the place where
8018 something undesirable happens, and you will see which parts of the
8019 grammar are to blame.
8020
8021 The parser file is a C program and you can use C debuggers on it, but it's
8022 not easy to interpret what it is doing. The parser function is a
8023 finite-state machine interpreter, and aside from the actions it executes
8024 the same code over and over. Only the values of variables show where in
8025 the grammar it is working.
8026
8027 @findex YYPRINT
8028 The debugging information normally gives the token type of each token
8029 read, but not its semantic value. You can optionally define a macro
8030 named @code{YYPRINT} to provide a way to print the value. If you define
8031 @code{YYPRINT}, it should take three arguments. The parser will pass a
8032 standard I/O stream, the numeric code for the token type, and the token
8033 value (from @code{yylval}).
8034
8035 Here is an example of @code{YYPRINT} suitable for the multi-function
8036 calculator (@pxref{Mfcalc Declarations, ,Declarations for @code{mfcalc}}):
8037
8038 @smallexample
8039 %@{
8040 static void print_token_value (FILE *, int, YYSTYPE);
8041 #define YYPRINT(file, type, value) print_token_value (file, type, value)
8042 %@}
8043
8044 @dots{} %% @dots{} %% @dots{}
8045
8046 static void
8047 print_token_value (FILE *file, int type, YYSTYPE value)
8048 @{
8049 if (type == VAR)
8050 fprintf (file, "%s", value.tptr->name);
8051 else if (type == NUM)
8052 fprintf (file, "%d", value.val);
8053 @}
8054 @end smallexample
8055
8056 @c ================================================= Invoking Bison
8057
8058 @node Invocation
8059 @chapter Invoking Bison
8060 @cindex invoking Bison
8061 @cindex Bison invocation
8062 @cindex options for invoking Bison
8063
8064 The usual way to invoke Bison is as follows:
8065
8066 @example
8067 bison @var{infile}
8068 @end example
8069
8070 Here @var{infile} is the grammar file name, which usually ends in
8071 @samp{.y}. The parser file's name is made by replacing the @samp{.y}
8072 with @samp{.tab.c} and removing any leading directory. Thus, the
8073 @samp{bison foo.y} file name yields
8074 @file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields
8075 @file{foo.tab.c}. It's also possible, in case you are writing
8076 C++ code instead of C in your grammar file, to name it @file{foo.ypp}
8077 or @file{foo.y++}. Then, the output files will take an extension like
8078 the given one as input (respectively @file{foo.tab.cpp} and
8079 @file{foo.tab.c++}).
8080 This feature takes effect with all options that manipulate file names like
8081 @samp{-o} or @samp{-d}.
8082
8083 For example :
8084
8085 @example
8086 bison -d @var{infile.yxx}
8087 @end example
8088 @noindent
8089 will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
8090
8091 @example
8092 bison -d -o @var{output.c++} @var{infile.y}
8093 @end example
8094 @noindent
8095 will produce @file{output.c++} and @file{outfile.h++}.
8096
8097 For compatibility with POSIX, the standard Bison
8098 distribution also contains a shell script called @command{yacc} that
8099 invokes Bison with the @option{-y} option.
8100
8101 @menu
8102 * Bison Options:: All the options described in detail,
8103 in alphabetical order by short options.
8104 * Option Cross Key:: Alphabetical list of long options.
8105 * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
8106 @end menu
8107
8108 @node Bison Options
8109 @section Bison Options
8110
8111 Bison supports both traditional single-letter options and mnemonic long
8112 option names. Long option names are indicated with @samp{--} instead of
8113 @samp{-}. Abbreviations for option names are allowed as long as they
8114 are unique. When a long option takes an argument, like
8115 @samp{--file-prefix}, connect the option name and the argument with
8116 @samp{=}.
8117
8118 Here is a list of options that can be used with Bison, alphabetized by
8119 short option. It is followed by a cross key alphabetized by long
8120 option.
8121
8122 @c Please, keep this ordered as in `bison --help'.
8123 @noindent
8124 Operations modes:
8125 @table @option
8126 @item -h
8127 @itemx --help
8128 Print a summary of the command-line options to Bison and exit.
8129
8130 @item -V
8131 @itemx --version
8132 Print the version number of Bison and exit.
8133
8134 @item --print-localedir
8135 Print the name of the directory containing locale-dependent data.
8136
8137 @item --print-datadir
8138 Print the name of the directory containing skeletons and XSLT.
8139
8140 @item -y
8141 @itemx --yacc
8142 Act more like the traditional Yacc command. This can cause
8143 different diagnostics to be generated, and may change behavior in
8144 other minor ways. Most importantly, imitate Yacc's output
8145 file name conventions, so that the parser output file is called
8146 @file{y.tab.c}, and the other outputs are called @file{y.output} and
8147 @file{y.tab.h}.
8148 Also, if generating a deterministic parser in C, generate @code{#define}
8149 statements in addition to an @code{enum} to associate token numbers with token
8150 names.
8151 Thus, the following shell script can substitute for Yacc, and the Bison
8152 distribution contains such a script for compatibility with POSIX:
8153
8154 @example
8155 #! /bin/sh
8156 bison -y "$@@"
8157 @end example
8158
8159 The @option{-y}/@option{--yacc} option is intended for use with
8160 traditional Yacc grammars. If your grammar uses a Bison extension
8161 like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
8162 this option is specified.
8163
8164 @item -W [@var{category}]
8165 @itemx --warnings[=@var{category}]
8166 Output warnings falling in @var{category}. @var{category} can be one
8167 of:
8168 @table @code
8169 @item midrule-values
8170 Warn about mid-rule values that are set but not used within any of the actions
8171 of the parent rule.
8172 For example, warn about unused @code{$2} in:
8173
8174 @example
8175 exp: '1' @{ $$ = 1; @} '+' exp @{ $$ = $1 + $4; @};
8176 @end example
8177
8178 Also warn about mid-rule values that are used but not set.
8179 For example, warn about unset @code{$$} in the mid-rule action in:
8180
8181 @example
8182 exp: '1' @{ $1 = 1; @} '+' exp @{ $$ = $2 + $4; @};
8183 @end example
8184
8185 These warnings are not enabled by default since they sometimes prove to
8186 be false alarms in existing grammars employing the Yacc constructs
8187 @code{$0} or @code{$-@var{n}} (where @var{n} is some positive integer).
8188
8189
8190 @item yacc
8191 Incompatibilities with POSIX Yacc.
8192
8193 @item all
8194 All the warnings.
8195 @item none
8196 Turn off all the warnings.
8197 @item error
8198 Treat warnings as errors.
8199 @end table
8200
8201 A category can be turned off by prefixing its name with @samp{no-}. For
8202 instance, @option{-Wno-yacc} will hide the warnings about
8203 POSIX Yacc incompatibilities.
8204 @end table
8205
8206 @noindent
8207 Tuning the parser:
8208
8209 @table @option
8210 @item -t
8211 @itemx --debug
8212 In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
8213 already defined, so that the debugging facilities are compiled.
8214 @xref{Tracing, ,Tracing Your Parser}.
8215
8216 @item -D @var{name}[=@var{value}]
8217 @itemx --define=@var{name}[=@var{value}]
8218 @itemx -F @var{name}[=@var{value}]
8219 @itemx --force-define=@var{name}[=@var{value}]
8220 Each of these is equivalent to @samp{%define @var{name} "@var{value}"}
8221 (@pxref{Decl Summary, ,%define}) except that Bison processes multiple
8222 definitions for the same @var{name} as follows:
8223
8224 @itemize
8225 @item
8226 Bison quietly ignores all command-line definitions for @var{name} except
8227 the last.
8228 @item
8229 If that command-line definition is specified by a @code{-D} or
8230 @code{--define}, Bison reports an error for any @code{%define}
8231 definition for @var{name}.
8232 @item
8233 If that command-line definition is specified by a @code{-F} or
8234 @code{--force-define} instead, Bison quietly ignores all @code{%define}
8235 definitions for @var{name}.
8236 @item
8237 Otherwise, Bison reports an error if there are multiple @code{%define}
8238 definitions for @var{name}.
8239 @end itemize
8240
8241 You should avoid using @code{-F} and @code{--force-define} in your
8242 makefiles unless you are confident that it is safe to quietly ignore any
8243 conflicting @code{%define} that may be added to the grammar file.
8244
8245 @item -L @var{language}
8246 @itemx --language=@var{language}
8247 Specify the programming language for the generated parser, as if
8248 @code{%language} was specified (@pxref{Decl Summary, , Bison Declaration
8249 Summary}). Currently supported languages include C, C++, and Java.
8250 @var{language} is case-insensitive.
8251
8252 This option is experimental and its effect may be modified in future
8253 releases.
8254
8255 @item --locations
8256 Pretend that @code{%locations} was specified. @xref{Decl Summary}.
8257
8258 @item -p @var{prefix}
8259 @itemx --name-prefix=@var{prefix}
8260 Pretend that @code{%name-prefix "@var{prefix}"} was specified.
8261 @xref{Decl Summary}.
8262
8263 @item -l
8264 @itemx --no-lines
8265 Don't put any @code{#line} preprocessor commands in the parser file.
8266 Ordinarily Bison puts them in the parser file so that the C compiler
8267 and debuggers will associate errors with your source file, the
8268 grammar file. This option causes them to associate errors with the
8269 parser file, treating it as an independent source file in its own right.
8270
8271 @item -S @var{file}
8272 @itemx --skeleton=@var{file}
8273 Specify the skeleton to use, similar to @code{%skeleton}
8274 (@pxref{Decl Summary, , Bison Declaration Summary}).
8275
8276 @c You probably don't need this option unless you are developing Bison.
8277 @c You should use @option{--language} if you want to specify the skeleton for a
8278 @c different language, because it is clearer and because it will always
8279 @c choose the correct skeleton for non-deterministic or push parsers.
8280
8281 If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
8282 file in the Bison installation directory.
8283 If it does, @var{file} is an absolute file name or a file name relative to the
8284 current working directory.
8285 This is similar to how most shells resolve commands.
8286
8287 @item -k
8288 @itemx --token-table
8289 Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
8290 @end table
8291
8292 @noindent
8293 Adjust the output:
8294
8295 @table @option
8296 @item --defines[=@var{file}]
8297 Pretend that @code{%defines} was specified, i.e., write an extra output
8298 file containing macro definitions for the token type names defined in
8299 the grammar, as well as a few other declarations. @xref{Decl Summary}.
8300
8301 @item -d
8302 This is the same as @code{--defines} except @code{-d} does not accept a
8303 @var{file} argument since POSIX Yacc requires that @code{-d} can be bundled
8304 with other short options.
8305
8306 @item -b @var{file-prefix}
8307 @itemx --file-prefix=@var{prefix}
8308 Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
8309 for all Bison output file names. @xref{Decl Summary}.
8310
8311 @item -r @var{things}
8312 @itemx --report=@var{things}
8313 Write an extra output file containing verbose description of the comma
8314 separated list of @var{things} among:
8315
8316 @table @code
8317 @item state
8318 Description of the grammar, conflicts (resolved and unresolved), and
8319 parser's automaton.
8320
8321 @item lookahead
8322 Implies @code{state} and augments the description of the automaton with
8323 each rule's lookahead set.
8324
8325 @item itemset
8326 Implies @code{state} and augments the description of the automaton with
8327 the full set of items for each state, instead of its core only.
8328 @end table
8329
8330 @item --report-file=@var{file}
8331 Specify the @var{file} for the verbose description.
8332
8333 @item -v
8334 @itemx --verbose
8335 Pretend that @code{%verbose} was specified, i.e., write an extra output
8336 file containing verbose descriptions of the grammar and
8337 parser. @xref{Decl Summary}.
8338
8339 @item -o @var{file}
8340 @itemx --output=@var{file}
8341 Specify the @var{file} for the parser file.
8342
8343 The other output files' names are constructed from @var{file} as
8344 described under the @samp{-v} and @samp{-d} options.
8345
8346 @item -g [@var{file}]
8347 @itemx --graph[=@var{file}]
8348 Output a graphical representation of the parser's
8349 automaton computed by Bison, in @uref{http://www.graphviz.org/, Graphviz}
8350 @uref{http://www.graphviz.org/doc/info/lang.html, DOT} format.
8351 @code{@var{file}} is optional.
8352 If omitted and the grammar file is @file{foo.y}, the output file will be
8353 @file{foo.dot}.
8354
8355 @item -x [@var{file}]
8356 @itemx --xml[=@var{file}]
8357 Output an XML report of the parser's automaton computed by Bison.
8358 @code{@var{file}} is optional.
8359 If omitted and the grammar file is @file{foo.y}, the output file will be
8360 @file{foo.xml}.
8361 (The current XML schema is experimental and may evolve.
8362 More user feedback will help to stabilize it.)
8363 @end table
8364
8365 @node Option Cross Key
8366 @section Option Cross Key
8367
8368 Here is a list of options, alphabetized by long option, to help you find
8369 the corresponding short option and directive.
8370
8371 @multitable {@option{--force-define=@var{name}[=@var{value}]}} {@option{-F @var{name}[=@var{value}]}} {@code{%nondeterministic-parser}}
8372 @headitem Long Option @tab Short Option @tab Bison Directive
8373 @include cross-options.texi
8374 @end multitable
8375
8376 @node Yacc Library
8377 @section Yacc Library
8378
8379 The Yacc library contains default implementations of the
8380 @code{yyerror} and @code{main} functions. These default
8381 implementations are normally not useful, but POSIX requires
8382 them. To use the Yacc library, link your program with the
8383 @option{-ly} option. Note that Bison's implementation of the Yacc
8384 library is distributed under the terms of the GNU General
8385 Public License (@pxref{Copying}).
8386
8387 If you use the Yacc library's @code{yyerror} function, you should
8388 declare @code{yyerror} as follows:
8389
8390 @example
8391 int yyerror (char const *);
8392 @end example
8393
8394 Bison ignores the @code{int} value returned by this @code{yyerror}.
8395 If you use the Yacc library's @code{main} function, your
8396 @code{yyparse} function should have the following type signature:
8397
8398 @example
8399 int yyparse (void);
8400 @end example
8401
8402 @c ================================================= C++ Bison
8403
8404 @node Other Languages
8405 @chapter Parsers Written In Other Languages
8406
8407 @menu
8408 * C++ Parsers:: The interface to generate C++ parser classes
8409 * Java Parsers:: The interface to generate Java parser classes
8410 @end menu
8411
8412 @node C++ Parsers
8413 @section C++ Parsers
8414
8415 @menu
8416 * C++ Bison Interface:: Asking for C++ parser generation
8417 * C++ Semantic Values:: %union vs. C++
8418 * C++ Location Values:: The position and location classes
8419 * C++ Parser Interface:: Instantiating and running the parser
8420 * C++ Scanner Interface:: Exchanges between yylex and parse
8421 * A Complete C++ Example:: Demonstrating their use
8422 @end menu
8423
8424 @node C++ Bison Interface
8425 @subsection C++ Bison Interface
8426 @c - %skeleton "lalr1.cc"
8427 @c - Always pure
8428 @c - initial action
8429
8430 The C++ deterministic parser is selected using the skeleton directive,
8431 @samp{%skeleton "lalr1.cc"}, or the synonymous command-line option
8432 @option{--skeleton=lalr1.cc}.
8433 @xref{Decl Summary}.
8434
8435 When run, @command{bison} will create several entities in the @samp{yy}
8436 namespace.
8437 @findex %define namespace
8438 Use the @samp{%define namespace} directive to change the namespace name, see
8439 @ref{Decl Summary}.
8440 The various classes are generated in the following files:
8441
8442 @table @file
8443 @item position.hh
8444 @itemx location.hh
8445 The definition of the classes @code{position} and @code{location},
8446 used for location tracking. @xref{C++ Location Values}.
8447
8448 @item stack.hh
8449 An auxiliary class @code{stack} used by the parser.
8450
8451 @item @var{file}.hh
8452 @itemx @var{file}.cc
8453 (Assuming the extension of the input file was @samp{.yy}.) The
8454 declaration and implementation of the C++ parser class. The basename
8455 and extension of these two files follow the same rules as with regular C
8456 parsers (@pxref{Invocation}).
8457
8458 The header is @emph{mandatory}; you must either pass
8459 @option{-d}/@option{--defines} to @command{bison}, or use the
8460 @samp{%defines} directive.
8461 @end table
8462
8463 All these files are documented using Doxygen; run @command{doxygen}
8464 for a complete and accurate documentation.
8465
8466 @node C++ Semantic Values
8467 @subsection C++ Semantic Values
8468 @c - No objects in unions
8469 @c - YYSTYPE
8470 @c - Printer and destructor
8471
8472 The @code{%union} directive works as for C, see @ref{Union Decl, ,The
8473 Collection of Value Types}. In particular it produces a genuine
8474 @code{union}@footnote{In the future techniques to allow complex types
8475 within pseudo-unions (similar to Boost variants) might be implemented to
8476 alleviate these issues.}, which have a few specific features in C++.
8477 @itemize @minus
8478 @item
8479 The type @code{YYSTYPE} is defined but its use is discouraged: rather
8480 you should refer to the parser's encapsulated type
8481 @code{yy::parser::semantic_type}.
8482 @item
8483 Non POD (Plain Old Data) types cannot be used. C++ forbids any
8484 instance of classes with constructors in unions: only @emph{pointers}
8485 to such objects are allowed.
8486 @end itemize
8487
8488 Because objects have to be stored via pointers, memory is not
8489 reclaimed automatically: using the @code{%destructor} directive is the
8490 only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
8491 Symbols}.
8492
8493
8494 @node C++ Location Values
8495 @subsection C++ Location Values
8496 @c - %locations
8497 @c - class Position
8498 @c - class Location
8499 @c - %define filename_type "const symbol::Symbol"
8500
8501 When the directive @code{%locations} is used, the C++ parser supports
8502 location tracking, see @ref{Locations, , Locations Overview}. Two
8503 auxiliary classes define a @code{position}, a single point in a file,
8504 and a @code{location}, a range composed of a pair of
8505 @code{position}s (possibly spanning several files).
8506
8507 @deftypemethod {position} {std::string*} file
8508 The name of the file. It will always be handled as a pointer, the
8509 parser will never duplicate nor deallocate it. As an experimental
8510 feature you may change it to @samp{@var{type}*} using @samp{%define
8511 filename_type "@var{type}"}.
8512 @end deftypemethod
8513
8514 @deftypemethod {position} {unsigned int} line
8515 The line, starting at 1.
8516 @end deftypemethod
8517
8518 @deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
8519 Advance by @var{height} lines, resetting the column number.
8520 @end deftypemethod
8521
8522 @deftypemethod {position} {unsigned int} column
8523 The column, starting at 0.
8524 @end deftypemethod
8525
8526 @deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
8527 Advance by @var{width} columns, without changing the line number.
8528 @end deftypemethod
8529
8530 @deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
8531 @deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
8532 @deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
8533 @deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
8534 Various forms of syntactic sugar for @code{columns}.
8535 @end deftypemethod
8536
8537 @deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
8538 Report @var{p} on @var{o} like this:
8539 @samp{@var{file}:@var{line}.@var{column}}, or
8540 @samp{@var{line}.@var{column}} if @var{file} is null.
8541 @end deftypemethod
8542
8543 @deftypemethod {location} {position} begin
8544 @deftypemethodx {location} {position} end
8545 The first, inclusive, position of the range, and the first beyond.
8546 @end deftypemethod
8547
8548 @deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
8549 @deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
8550 Advance the @code{end} position.
8551 @end deftypemethod
8552
8553 @deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
8554 @deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
8555 @deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
8556 Various forms of syntactic sugar.
8557 @end deftypemethod
8558
8559 @deftypemethod {location} {void} step ()
8560 Move @code{begin} onto @code{end}.
8561 @end deftypemethod
8562
8563
8564 @node C++ Parser Interface
8565 @subsection C++ Parser Interface
8566 @c - define parser_class_name
8567 @c - Ctor
8568 @c - parse, error, set_debug_level, debug_level, set_debug_stream,
8569 @c debug_stream.
8570 @c - Reporting errors
8571
8572 The output files @file{@var{output}.hh} and @file{@var{output}.cc}
8573 declare and define the parser class in the namespace @code{yy}. The
8574 class name defaults to @code{parser}, but may be changed using
8575 @samp{%define parser_class_name "@var{name}"}. The interface of
8576 this class is detailed below. It can be extended using the
8577 @code{%parse-param} feature: its semantics is slightly changed since
8578 it describes an additional member of the parser class, and an
8579 additional argument for its constructor.
8580
8581 @defcv {Type} {parser} {semantic_type}
8582 @defcvx {Type} {parser} {location_type}
8583 The types for semantics value and locations.
8584 @end defcv
8585
8586 @defcv {Type} {parser} {token}
8587 A structure that contains (only) the definition of the tokens as the
8588 @code{yytokentype} enumeration. To refer to the token @code{FOO}, the
8589 scanner should use @code{yy::parser::token::FOO}. The scanner can use
8590 @samp{typedef yy::parser::token token;} to ``import'' the token enumeration
8591 (@pxref{Calc++ Scanner}).
8592 @end defcv
8593
8594 @deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
8595 Build a new parser object. There are no arguments by default, unless
8596 @samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
8597 @end deftypemethod
8598
8599 @deftypemethod {parser} {int} parse ()
8600 Run the syntactic analysis, and return 0 on success, 1 otherwise.
8601 @end deftypemethod
8602
8603 @deftypemethod {parser} {std::ostream&} debug_stream ()
8604 @deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
8605 Get or set the stream used for tracing the parsing. It defaults to
8606 @code{std::cerr}.
8607 @end deftypemethod
8608
8609 @deftypemethod {parser} {debug_level_type} debug_level ()
8610 @deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
8611 Get or set the tracing level. Currently its value is either 0, no trace,
8612 or nonzero, full tracing.
8613 @end deftypemethod
8614
8615 @deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
8616 The definition for this member function must be supplied by the user:
8617 the parser uses it to report a parser error occurring at @var{l},
8618 described by @var{m}.
8619 @end deftypemethod
8620
8621
8622 @node C++ Scanner Interface
8623 @subsection C++ Scanner Interface
8624 @c - prefix for yylex.
8625 @c - Pure interface to yylex
8626 @c - %lex-param
8627
8628 The parser invokes the scanner by calling @code{yylex}. Contrary to C
8629 parsers, C++ parsers are always pure: there is no point in using the
8630 @code{%define api.pure} directive. Therefore the interface is as follows.
8631
8632 @deftypemethod {parser} {int} yylex (semantic_type* @var{yylval}, location_type* @var{yylloc}, @var{type1} @var{arg1}, ...)
8633 Return the next token. Its type is the return value, its semantic
8634 value and location being @var{yylval} and @var{yylloc}. Invocations of
8635 @samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
8636 @end deftypemethod
8637
8638
8639 @node A Complete C++ Example
8640 @subsection A Complete C++ Example
8641
8642 This section demonstrates the use of a C++ parser with a simple but
8643 complete example. This example should be available on your system,
8644 ready to compile, in the directory @dfn{../bison/examples/calc++}. It
8645 focuses on the use of Bison, therefore the design of the various C++
8646 classes is very naive: no accessors, no encapsulation of members etc.
8647 We will use a Lex scanner, and more precisely, a Flex scanner, to
8648 demonstrate the various interaction. A hand written scanner is
8649 actually easier to interface with.
8650
8651 @menu
8652 * Calc++ --- C++ Calculator:: The specifications
8653 * Calc++ Parsing Driver:: An active parsing context
8654 * Calc++ Parser:: A parser class
8655 * Calc++ Scanner:: A pure C++ Flex scanner
8656 * Calc++ Top Level:: Conducting the band
8657 @end menu
8658
8659 @node Calc++ --- C++ Calculator
8660 @subsubsection Calc++ --- C++ Calculator
8661
8662 Of course the grammar is dedicated to arithmetics, a single
8663 expression, possibly preceded by variable assignments. An
8664 environment containing possibly predefined variables such as
8665 @code{one} and @code{two}, is exchanged with the parser. An example
8666 of valid input follows.
8667
8668 @example
8669 three := 3
8670 seven := one + two * three
8671 seven * seven
8672 @end example
8673
8674 @node Calc++ Parsing Driver
8675 @subsubsection Calc++ Parsing Driver
8676 @c - An env
8677 @c - A place to store error messages
8678 @c - A place for the result
8679
8680 To support a pure interface with the parser (and the scanner) the
8681 technique of the ``parsing context'' is convenient: a structure
8682 containing all the data to exchange. Since, in addition to simply
8683 launch the parsing, there are several auxiliary tasks to execute (open
8684 the file for parsing, instantiate the parser etc.), we recommend
8685 transforming the simple parsing context structure into a fully blown
8686 @dfn{parsing driver} class.
8687
8688 The declaration of this driver class, @file{calc++-driver.hh}, is as
8689 follows. The first part includes the CPP guard and imports the
8690 required standard library components, and the declaration of the parser
8691 class.
8692
8693 @comment file: calc++-driver.hh
8694 @example
8695 #ifndef CALCXX_DRIVER_HH
8696 # define CALCXX_DRIVER_HH
8697 # include <string>
8698 # include <map>
8699 # include "calc++-parser.hh"
8700 @end example
8701
8702
8703 @noindent
8704 Then comes the declaration of the scanning function. Flex expects
8705 the signature of @code{yylex} to be defined in the macro
8706 @code{YY_DECL}, and the C++ parser expects it to be declared. We can
8707 factor both as follows.
8708
8709 @comment file: calc++-driver.hh
8710 @example
8711 // Tell Flex the lexer's prototype ...
8712 # define YY_DECL \
8713 yy::calcxx_parser::token_type \
8714 yylex (yy::calcxx_parser::semantic_type* yylval, \
8715 yy::calcxx_parser::location_type* yylloc, \
8716 calcxx_driver& driver)
8717 // ... and declare it for the parser's sake.
8718 YY_DECL;
8719 @end example
8720
8721 @noindent
8722 The @code{calcxx_driver} class is then declared with its most obvious
8723 members.
8724
8725 @comment file: calc++-driver.hh
8726 @example
8727 // Conducting the whole scanning and parsing of Calc++.
8728 class calcxx_driver
8729 @{
8730 public:
8731 calcxx_driver ();
8732 virtual ~calcxx_driver ();
8733
8734 std::map<std::string, int> variables;
8735
8736 int result;
8737 @end example
8738
8739 @noindent
8740 To encapsulate the coordination with the Flex scanner, it is useful to
8741 have two members function to open and close the scanning phase.
8742
8743 @comment file: calc++-driver.hh
8744 @example
8745 // Handling the scanner.
8746 void scan_begin ();
8747 void scan_end ();
8748 bool trace_scanning;
8749 @end example
8750
8751 @noindent
8752 Similarly for the parser itself.
8753
8754 @comment file: calc++-driver.hh
8755 @example
8756 // Run the parser. Return 0 on success.
8757 int parse (const std::string& f);
8758 std::string file;
8759 bool trace_parsing;
8760 @end example
8761
8762 @noindent
8763 To demonstrate pure handling of parse errors, instead of simply
8764 dumping them on the standard error output, we will pass them to the
8765 compiler driver using the following two member functions. Finally, we
8766 close the class declaration and CPP guard.
8767
8768 @comment file: calc++-driver.hh
8769 @example
8770 // Error handling.
8771 void error (const yy::location& l, const std::string& m);
8772 void error (const std::string& m);
8773 @};
8774 #endif // ! CALCXX_DRIVER_HH
8775 @end example
8776
8777 The implementation of the driver is straightforward. The @code{parse}
8778 member function deserves some attention. The @code{error} functions
8779 are simple stubs, they should actually register the located error
8780 messages and set error state.
8781
8782 @comment file: calc++-driver.cc
8783 @example
8784 #include "calc++-driver.hh"
8785 #include "calc++-parser.hh"
8786
8787 calcxx_driver::calcxx_driver ()
8788 : trace_scanning (false), trace_parsing (false)
8789 @{
8790 variables["one"] = 1;
8791 variables["two"] = 2;
8792 @}
8793
8794 calcxx_driver::~calcxx_driver ()
8795 @{
8796 @}
8797
8798 int
8799 calcxx_driver::parse (const std::string &f)
8800 @{
8801 file = f;
8802 scan_begin ();
8803 yy::calcxx_parser parser (*this);
8804 parser.set_debug_level (trace_parsing);
8805 int res = parser.parse ();
8806 scan_end ();
8807 return res;
8808 @}
8809
8810 void
8811 calcxx_driver::error (const yy::location& l, const std::string& m)
8812 @{
8813 std::cerr << l << ": " << m << std::endl;
8814 @}
8815
8816 void
8817 calcxx_driver::error (const std::string& m)
8818 @{
8819 std::cerr << m << std::endl;
8820 @}
8821 @end example
8822
8823 @node Calc++ Parser
8824 @subsubsection Calc++ Parser
8825
8826 The parser definition file @file{calc++-parser.yy} starts by asking for
8827 the C++ deterministic parser skeleton, the creation of the parser header
8828 file, and specifies the name of the parser class.
8829 Because the C++ skeleton changed several times, it is safer to require
8830 the version you designed the grammar for.
8831
8832 @comment file: calc++-parser.yy
8833 @example
8834 %skeleton "lalr1.cc" /* -*- C++ -*- */
8835 %require "@value{VERSION}"
8836 %defines
8837 %define parser_class_name "calcxx_parser"
8838 @end example
8839
8840 @noindent
8841 @findex %code requires
8842 Then come the declarations/inclusions needed to define the
8843 @code{%union}. Because the parser uses the parsing driver and
8844 reciprocally, both cannot include the header of the other. Because the
8845 driver's header needs detailed knowledge about the parser class (in
8846 particular its inner types), it is the parser's header which will simply
8847 use a forward declaration of the driver.
8848 @xref{Decl Summary, ,%code}.
8849
8850 @comment file: calc++-parser.yy
8851 @example
8852 %code requires @{
8853 # include <string>
8854 class calcxx_driver;
8855 @}
8856 @end example
8857
8858 @noindent
8859 The driver is passed by reference to the parser and to the scanner.
8860 This provides a simple but effective pure interface, not relying on
8861 global variables.
8862
8863 @comment file: calc++-parser.yy
8864 @example
8865 // The parsing context.
8866 %parse-param @{ calcxx_driver& driver @}
8867 %lex-param @{ calcxx_driver& driver @}
8868 @end example
8869
8870 @noindent
8871 Then we request the location tracking feature, and initialize the
8872 first location's file name. Afterward new locations are computed
8873 relatively to the previous locations: the file name will be
8874 automatically propagated.
8875
8876 @comment file: calc++-parser.yy
8877 @example
8878 %locations
8879 %initial-action
8880 @{
8881 // Initialize the initial location.
8882 @@$.begin.filename = @@$.end.filename = &driver.file;
8883 @};
8884 @end example
8885
8886 @noindent
8887 Use the two following directives to enable parser tracing and verbose
8888 error messages.
8889
8890 @comment file: calc++-parser.yy
8891 @example
8892 %debug
8893 %error-verbose
8894 @end example
8895
8896 @noindent
8897 Semantic values cannot use ``real'' objects, but only pointers to
8898 them.
8899
8900 @comment file: calc++-parser.yy
8901 @example
8902 // Symbols.
8903 %union
8904 @{
8905 int ival;
8906 std::string *sval;
8907 @};
8908 @end example
8909
8910 @noindent
8911 @findex %code
8912 The code between @samp{%code @{} and @samp{@}} is output in the
8913 @file{*.cc} file; it needs detailed knowledge about the driver.
8914
8915 @comment file: calc++-parser.yy
8916 @example
8917 %code @{
8918 # include "calc++-driver.hh"
8919 @}
8920 @end example
8921
8922
8923 @noindent
8924 The token numbered as 0 corresponds to end of file; the following line
8925 allows for nicer error messages referring to ``end of file'' instead
8926 of ``$end''. Similarly user friendly named are provided for each
8927 symbol. Note that the tokens names are prefixed by @code{TOKEN_} to
8928 avoid name clashes.
8929
8930 @comment file: calc++-parser.yy
8931 @example
8932 %token END 0 "end of file"
8933 %token ASSIGN ":="
8934 %token <sval> IDENTIFIER "identifier"
8935 %token <ival> NUMBER "number"
8936 %type <ival> exp
8937 @end example
8938
8939 @noindent
8940 To enable memory deallocation during error recovery, use
8941 @code{%destructor}.
8942
8943 @c FIXME: Document %printer, and mention that it takes a braced-code operand.
8944 @comment file: calc++-parser.yy
8945 @example
8946 %printer @{ debug_stream () << *$$; @} "identifier"
8947 %destructor @{ delete $$; @} "identifier"
8948
8949 %printer @{ debug_stream () << $$; @} <ival>
8950 @end example
8951
8952 @noindent
8953 The grammar itself is straightforward.
8954
8955 @comment file: calc++-parser.yy
8956 @example
8957 %%
8958 %start unit;
8959 unit: assignments exp @{ driver.result = $2; @};
8960
8961 assignments: assignments assignment @{@}
8962 | /* Nothing. */ @{@};
8963
8964 assignment:
8965 "identifier" ":=" exp
8966 @{ driver.variables[*$1] = $3; delete $1; @};
8967
8968 %left '+' '-';
8969 %left '*' '/';
8970 exp: exp '+' exp @{ $$ = $1 + $3; @}
8971 | exp '-' exp @{ $$ = $1 - $3; @}
8972 | exp '*' exp @{ $$ = $1 * $3; @}
8973 | exp '/' exp @{ $$ = $1 / $3; @}
8974 | "identifier" @{ $$ = driver.variables[*$1]; delete $1; @}
8975 | "number" @{ $$ = $1; @};
8976 %%
8977 @end example
8978
8979 @noindent
8980 Finally the @code{error} member function registers the errors to the
8981 driver.
8982
8983 @comment file: calc++-parser.yy
8984 @example
8985 void
8986 yy::calcxx_parser::error (const yy::calcxx_parser::location_type& l,
8987 const std::string& m)
8988 @{
8989 driver.error (l, m);
8990 @}
8991 @end example
8992
8993 @node Calc++ Scanner
8994 @subsubsection Calc++ Scanner
8995
8996 The Flex scanner first includes the driver declaration, then the
8997 parser's to get the set of defined tokens.
8998
8999 @comment file: calc++-scanner.ll
9000 @example
9001 %@{ /* -*- C++ -*- */
9002 # include <cstdlib>
9003 # include <cerrno>
9004 # include <climits>
9005 # include <string>
9006 # include "calc++-driver.hh"
9007 # include "calc++-parser.hh"
9008
9009 /* Work around an incompatibility in flex (at least versions
9010 2.5.31 through 2.5.33): it generates code that does
9011 not conform to C89. See Debian bug 333231
9012 <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */
9013 # undef yywrap
9014 # define yywrap() 1
9015
9016 /* By default yylex returns int, we use token_type.
9017 Unfortunately yyterminate by default returns 0, which is
9018 not of token_type. */
9019 #define yyterminate() return token::END
9020 %@}
9021 @end example
9022
9023 @noindent
9024 Because there is no @code{#include}-like feature we don't need
9025 @code{yywrap}, we don't need @code{unput} either, and we parse an
9026 actual file, this is not an interactive session with the user.
9027 Finally we enable the scanner tracing features.
9028
9029 @comment file: calc++-scanner.ll
9030 @example
9031 %option noyywrap nounput batch debug
9032 @end example
9033
9034 @noindent
9035 Abbreviations allow for more readable rules.
9036
9037 @comment file: calc++-scanner.ll
9038 @example
9039 id [a-zA-Z][a-zA-Z_0-9]*
9040 int [0-9]+
9041 blank [ \t]
9042 @end example
9043
9044 @noindent
9045 The following paragraph suffices to track locations accurately. Each
9046 time @code{yylex} is invoked, the begin position is moved onto the end
9047 position. Then when a pattern is matched, the end position is
9048 advanced of its width. In case it matched ends of lines, the end
9049 cursor is adjusted, and each time blanks are matched, the begin cursor
9050 is moved onto the end cursor to effectively ignore the blanks
9051 preceding tokens. Comments would be treated equally.
9052
9053 @comment file: calc++-scanner.ll
9054 @example
9055 %@{
9056 # define YY_USER_ACTION yylloc->columns (yyleng);
9057 %@}
9058 %%
9059 %@{
9060 yylloc->step ();
9061 %@}
9062 @{blank@}+ yylloc->step ();
9063 [\n]+ yylloc->lines (yyleng); yylloc->step ();
9064 @end example
9065
9066 @noindent
9067 The rules are simple, just note the use of the driver to report errors.
9068 It is convenient to use a typedef to shorten
9069 @code{yy::calcxx_parser::token::identifier} into
9070 @code{token::identifier} for instance.
9071
9072 @comment file: calc++-scanner.ll
9073 @example
9074 %@{
9075 typedef yy::calcxx_parser::token token;
9076 %@}
9077 /* Convert ints to the actual type of tokens. */
9078 [-+*/] return yy::calcxx_parser::token_type (yytext[0]);
9079 ":=" return token::ASSIGN;
9080 @{int@} @{
9081 errno = 0;
9082 long n = strtol (yytext, NULL, 10);
9083 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
9084 driver.error (*yylloc, "integer is out of range");
9085 yylval->ival = n;
9086 return token::NUMBER;
9087 @}
9088 @{id@} yylval->sval = new std::string (yytext); return token::IDENTIFIER;
9089 . driver.error (*yylloc, "invalid character");
9090 %%
9091 @end example
9092
9093 @noindent
9094 Finally, because the scanner related driver's member function depend
9095 on the scanner's data, it is simpler to implement them in this file.
9096
9097 @comment file: calc++-scanner.ll
9098 @example
9099 void
9100 calcxx_driver::scan_begin ()
9101 @{
9102 yy_flex_debug = trace_scanning;
9103 if (file == "-")
9104 yyin = stdin;
9105 else if (!(yyin = fopen (file.c_str (), "r")))
9106 @{
9107 error (std::string ("cannot open ") + file);
9108 exit (1);
9109 @}
9110 @}
9111
9112 void
9113 calcxx_driver::scan_end ()
9114 @{
9115 fclose (yyin);
9116 @}
9117 @end example
9118
9119 @node Calc++ Top Level
9120 @subsubsection Calc++ Top Level
9121
9122 The top level file, @file{calc++.cc}, poses no problem.
9123
9124 @comment file: calc++.cc
9125 @example
9126 #include <iostream>
9127 #include "calc++-driver.hh"
9128
9129 int
9130 main (int argc, char *argv[])
9131 @{
9132 calcxx_driver driver;
9133 for (++argv; argv[0]; ++argv)
9134 if (*argv == std::string ("-p"))
9135 driver.trace_parsing = true;
9136 else if (*argv == std::string ("-s"))
9137 driver.trace_scanning = true;
9138 else if (!driver.parse (*argv))
9139 std::cout << driver.result << std::endl;
9140 @}
9141 @end example
9142
9143 @node Java Parsers
9144 @section Java Parsers
9145
9146 @menu
9147 * Java Bison Interface:: Asking for Java parser generation
9148 * Java Semantic Values:: %type and %token vs. Java
9149 * Java Location Values:: The position and location classes
9150 * Java Parser Interface:: Instantiating and running the parser
9151 * Java Scanner Interface:: Specifying the scanner for the parser
9152 * Java Action Features:: Special features for use in actions
9153 * Java Differences:: Differences between C/C++ and Java Grammars
9154 * Java Declarations Summary:: List of Bison declarations used with Java
9155 @end menu
9156
9157 @node Java Bison Interface
9158 @subsection Java Bison Interface
9159 @c - %language "Java"
9160
9161 (The current Java interface is experimental and may evolve.
9162 More user feedback will help to stabilize it.)
9163
9164 The Java parser skeletons are selected using the @code{%language "Java"}
9165 directive or the @option{-L java}/@option{--language=java} option.
9166
9167 @c FIXME: Documented bug.
9168 When generating a Java parser, @code{bison @var{basename}.y} will create
9169 a single Java source file named @file{@var{basename}.java}. Using an
9170 input file without a @file{.y} suffix is currently broken. The basename
9171 of the output file can be changed by the @code{%file-prefix} directive
9172 or the @option{-p}/@option{--name-prefix} option. The entire output file
9173 name can be changed by the @code{%output} directive or the
9174 @option{-o}/@option{--output} option. The output file contains a single
9175 class for the parser.
9176
9177 You can create documentation for generated parsers using Javadoc.
9178
9179 Contrary to C parsers, Java parsers do not use global variables; the
9180 state of the parser is always local to an instance of the parser class.
9181 Therefore, all Java parsers are ``pure'', and the @code{%pure-parser}
9182 and @code{%define api.pure} directives does not do anything when used in
9183 Java.
9184
9185 Push parsers are currently unsupported in Java and @code{%define
9186 api.push-pull} have no effect.
9187
9188 GLR parsers are currently unsupported in Java. Do not use the
9189 @code{glr-parser} directive.
9190
9191 No header file can be generated for Java parsers. Do not use the
9192 @code{%defines} directive or the @option{-d}/@option{--defines} options.
9193
9194 @c FIXME: Possible code change.
9195 Currently, support for debugging and verbose errors are always compiled
9196 in. Thus the @code{%debug} and @code{%token-table} directives and the
9197 @option{-t}/@option{--debug} and @option{-k}/@option{--token-table}
9198 options have no effect. This may change in the future to eliminate
9199 unused code in the generated parser, so use @code{%debug} and
9200 @code{%verbose-error} explicitly if needed. Also, in the future the
9201 @code{%token-table} directive might enable a public interface to
9202 access the token names and codes.
9203
9204 @node Java Semantic Values
9205 @subsection Java Semantic Values
9206 @c - No %union, specify type in %type/%token.
9207 @c - YYSTYPE
9208 @c - Printer and destructor
9209
9210 There is no @code{%union} directive in Java parsers. Instead, the
9211 semantic values' types (class names) should be specified in the
9212 @code{%type} or @code{%token} directive:
9213
9214 @example
9215 %type <Expression> expr assignment_expr term factor
9216 %type <Integer> number
9217 @end example
9218
9219 By default, the semantic stack is declared to have @code{Object} members,
9220 which means that the class types you specify can be of any class.
9221 To improve the type safety of the parser, you can declare the common
9222 superclass of all the semantic values using the @code{%define stype}
9223 directive. For example, after the following declaration:
9224
9225 @example
9226 %define stype "ASTNode"
9227 @end example
9228
9229 @noindent
9230 any @code{%type} or @code{%token} specifying a semantic type which
9231 is not a subclass of ASTNode, will cause a compile-time error.
9232
9233 @c FIXME: Documented bug.
9234 Types used in the directives may be qualified with a package name.
9235 Primitive data types are accepted for Java version 1.5 or later. Note
9236 that in this case the autoboxing feature of Java 1.5 will be used.
9237 Generic types may not be used; this is due to a limitation in the
9238 implementation of Bison, and may change in future releases.
9239
9240 Java parsers do not support @code{%destructor}, since the language
9241 adopts garbage collection. The parser will try to hold references
9242 to semantic values for as little time as needed.
9243
9244 Java parsers do not support @code{%printer}, as @code{toString()}
9245 can be used to print the semantic values. This however may change
9246 (in a backwards-compatible way) in future versions of Bison.
9247
9248
9249 @node Java Location Values
9250 @subsection Java Location Values
9251 @c - %locations
9252 @c - class Position
9253 @c - class Location
9254
9255 When the directive @code{%locations} is used, the Java parser
9256 supports location tracking, see @ref{Locations, , Locations Overview}.
9257 An auxiliary user-defined class defines a @dfn{position}, a single point
9258 in a file; Bison itself defines a class representing a @dfn{location},
9259 a range composed of a pair of positions (possibly spanning several
9260 files). The location class is an inner class of the parser; the name
9261 is @code{Location} by default, and may also be renamed using
9262 @code{%define location_type "@var{class-name}"}.
9263
9264 The location class treats the position as a completely opaque value.
9265 By default, the class name is @code{Position}, but this can be changed
9266 with @code{%define position_type "@var{class-name}"}. This class must
9267 be supplied by the user.
9268
9269
9270 @deftypeivar {Location} {Position} begin
9271 @deftypeivarx {Location} {Position} end
9272 The first, inclusive, position of the range, and the first beyond.
9273 @end deftypeivar
9274
9275 @deftypeop {Constructor} {Location} {} Location (Position @var{loc})
9276 Create a @code{Location} denoting an empty range located at a given point.
9277 @end deftypeop
9278
9279 @deftypeop {Constructor} {Location} {} Location (Position @var{begin}, Position @var{end})
9280 Create a @code{Location} from the endpoints of the range.
9281 @end deftypeop
9282
9283 @deftypemethod {Location} {String} toString ()
9284 Prints the range represented by the location. For this to work
9285 properly, the position class should override the @code{equals} and
9286 @code{toString} methods appropriately.
9287 @end deftypemethod
9288
9289
9290 @node Java Parser Interface
9291 @subsection Java Parser Interface
9292 @c - define parser_class_name
9293 @c - Ctor
9294 @c - parse, error, set_debug_level, debug_level, set_debug_stream,
9295 @c debug_stream.
9296 @c - Reporting errors
9297
9298 The name of the generated parser class defaults to @code{YYParser}. The
9299 @code{YY} prefix may be changed using the @code{%name-prefix} directive
9300 or the @option{-p}/@option{--name-prefix} option. Alternatively, use
9301 @code{%define parser_class_name "@var{name}"} to give a custom name to
9302 the class. The interface of this class is detailed below.
9303
9304 By default, the parser class has package visibility. A declaration
9305 @code{%define public} will change to public visibility. Remember that,
9306 according to the Java language specification, the name of the @file{.java}
9307 file should match the name of the class in this case. Similarly, you can
9308 use @code{abstract}, @code{final} and @code{strictfp} with the
9309 @code{%define} declaration to add other modifiers to the parser class.
9310
9311 The Java package name of the parser class can be specified using the
9312 @code{%define package} directive. The superclass and the implemented
9313 interfaces of the parser class can be specified with the @code{%define
9314 extends} and @code{%define implements} directives.
9315
9316 The parser class defines an inner class, @code{Location}, that is used
9317 for location tracking (see @ref{Java Location Values}), and a inner
9318 interface, @code{Lexer} (see @ref{Java Scanner Interface}). Other than
9319 these inner class/interface, and the members described in the interface
9320 below, all the other members and fields are preceded with a @code{yy} or
9321 @code{YY} prefix to avoid clashes with user code.
9322
9323 @c FIXME: The following constants and variables are still undocumented:
9324 @c @code{bisonVersion}, @code{bisonSkeleton} and @code{errorVerbose}.
9325
9326 The parser class can be extended using the @code{%parse-param}
9327 directive. Each occurrence of the directive will add a @code{protected
9328 final} field to the parser class, and an argument to its constructor,
9329 which initialize them automatically.
9330
9331 Token names defined by @code{%token} and the predefined @code{EOF} token
9332 name are added as constant fields to the parser class.
9333
9334 @deftypeop {Constructor} {YYParser} {} YYParser (@var{lex_param}, @dots{}, @var{parse_param}, @dots{})
9335 Build a new parser object with embedded @code{%code lexer}. There are
9336 no parameters, unless @code{%parse-param}s and/or @code{%lex-param}s are
9337 used.
9338 @end deftypeop
9339
9340 @deftypeop {Constructor} {YYParser} {} YYParser (Lexer @var{lexer}, @var{parse_param}, @dots{})
9341 Build a new parser object using the specified scanner. There are no
9342 additional parameters unless @code{%parse-param}s are used.
9343
9344 If the scanner is defined by @code{%code lexer}, this constructor is
9345 declared @code{protected} and is called automatically with a scanner
9346 created with the correct @code{%lex-param}s.
9347 @end deftypeop
9348
9349 @deftypemethod {YYParser} {boolean} parse ()
9350 Run the syntactic analysis, and return @code{true} on success,
9351 @code{false} otherwise.
9352 @end deftypemethod
9353
9354 @deftypemethod {YYParser} {boolean} recovering ()
9355 During the syntactic analysis, return @code{true} if recovering
9356 from a syntax error.
9357 @xref{Error Recovery}.
9358 @end deftypemethod
9359
9360 @deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
9361 @deftypemethodx {YYParser} {void} setDebugStream (java.io.printStream @var{o})
9362 Get or set the stream used for tracing the parsing. It defaults to
9363 @code{System.err}.
9364 @end deftypemethod
9365
9366 @deftypemethod {YYParser} {int} getDebugLevel ()
9367 @deftypemethodx {YYParser} {void} setDebugLevel (int @var{l})
9368 Get or set the tracing level. Currently its value is either 0, no trace,
9369 or nonzero, full tracing.
9370 @end deftypemethod
9371
9372
9373 @node Java Scanner Interface
9374 @subsection Java Scanner Interface
9375 @c - %code lexer
9376 @c - %lex-param
9377 @c - Lexer interface
9378
9379 There are two possible ways to interface a Bison-generated Java parser
9380 with a scanner: the scanner may be defined by @code{%code lexer}, or
9381 defined elsewhere. In either case, the scanner has to implement the
9382 @code{Lexer} inner interface of the parser class.
9383
9384 In the first case, the body of the scanner class is placed in
9385 @code{%code lexer} blocks. If you want to pass parameters from the
9386 parser constructor to the scanner constructor, specify them with
9387 @code{%lex-param}; they are passed before @code{%parse-param}s to the
9388 constructor.
9389
9390 In the second case, the scanner has to implement the @code{Lexer} interface,
9391 which is defined within the parser class (e.g., @code{YYParser.Lexer}).
9392 The constructor of the parser object will then accept an object
9393 implementing the interface; @code{%lex-param} is not used in this
9394 case.
9395
9396 In both cases, the scanner has to implement the following methods.
9397
9398 @deftypemethod {Lexer} {void} yyerror (Location @var{loc}, String @var{msg})
9399 This method is defined by the user to emit an error message. The first
9400 parameter is omitted if location tracking is not active. Its type can be
9401 changed using @code{%define location_type "@var{class-name}".}
9402 @end deftypemethod
9403
9404 @deftypemethod {Lexer} {int} yylex ()
9405 Return the next token. Its type is the return value, its semantic
9406 value and location are saved and returned by the their methods in the
9407 interface.
9408
9409 Use @code{%define lex_throws} to specify any uncaught exceptions.
9410 Default is @code{java.io.IOException}.
9411 @end deftypemethod
9412
9413 @deftypemethod {Lexer} {Position} getStartPos ()
9414 @deftypemethodx {Lexer} {Position} getEndPos ()
9415 Return respectively the first position of the last token that
9416 @code{yylex} returned, and the first position beyond it. These
9417 methods are not needed unless location tracking is active.
9418
9419 The return type can be changed using @code{%define position_type
9420 "@var{class-name}".}
9421 @end deftypemethod
9422
9423 @deftypemethod {Lexer} {Object} getLVal ()
9424 Return the semantic value of the last token that yylex returned.
9425
9426 The return type can be changed using @code{%define stype
9427 "@var{class-name}".}
9428 @end deftypemethod
9429
9430
9431 @node Java Action Features
9432 @subsection Special Features for Use in Java Actions
9433
9434 The following special constructs can be uses in Java actions.
9435 Other analogous C action features are currently unavailable for Java.
9436
9437 Use @code{%define throws} to specify any uncaught exceptions from parser
9438 actions, and initial actions specified by @code{%initial-action}.
9439
9440 @defvar $@var{n}
9441 The semantic value for the @var{n}th component of the current rule.
9442 This may not be assigned to.
9443 @xref{Java Semantic Values}.
9444 @end defvar
9445
9446 @defvar $<@var{typealt}>@var{n}
9447 Like @code{$@var{n}} but specifies a alternative type @var{typealt}.
9448 @xref{Java Semantic Values}.
9449 @end defvar
9450
9451 @defvar $$
9452 The semantic value for the grouping made by the current rule. As a
9453 value, this is in the base type (@code{Object} or as specified by
9454 @code{%define stype}) as in not cast to the declared subtype because
9455 casts are not allowed on the left-hand side of Java assignments.
9456 Use an explicit Java cast if the correct subtype is needed.
9457 @xref{Java Semantic Values}.
9458 @end defvar
9459
9460 @defvar $<@var{typealt}>$
9461 Same as @code{$$} since Java always allow assigning to the base type.
9462 Perhaps we should use this and @code{$<>$} for the value and @code{$$}
9463 for setting the value but there is currently no easy way to distinguish
9464 these constructs.
9465 @xref{Java Semantic Values}.
9466 @end defvar
9467
9468 @defvar @@@var{n}
9469 The location information of the @var{n}th component of the current rule.
9470 This may not be assigned to.
9471 @xref{Java Location Values}.
9472 @end defvar
9473
9474 @defvar @@$
9475 The location information of the grouping made by the current rule.
9476 @xref{Java Location Values}.
9477 @end defvar
9478
9479 @deffn {Statement} {return YYABORT;}
9480 Return immediately from the parser, indicating failure.
9481 @xref{Java Parser Interface}.
9482 @end deffn
9483
9484 @deffn {Statement} {return YYACCEPT;}
9485 Return immediately from the parser, indicating success.
9486 @xref{Java Parser Interface}.
9487 @end deffn
9488
9489 @deffn {Statement} {return YYERROR;}
9490 Start error recovery without printing an error message.
9491 @xref{Error Recovery}.
9492 @end deffn
9493
9494 @deftypefn {Function} {boolean} recovering ()
9495 Return whether error recovery is being done. In this state, the parser
9496 reads token until it reaches a known state, and then restarts normal
9497 operation.
9498 @xref{Error Recovery}.
9499 @end deftypefn
9500
9501 @deftypefn {Function} {protected void} yyerror (String msg)
9502 @deftypefnx {Function} {protected void} yyerror (Position pos, String msg)
9503 @deftypefnx {Function} {protected void} yyerror (Location loc, String msg)
9504 Print an error message using the @code{yyerror} method of the scanner
9505 instance in use.
9506 @end deftypefn
9507
9508
9509 @node Java Differences
9510 @subsection Differences between C/C++ and Java Grammars
9511
9512 The different structure of the Java language forces several differences
9513 between C/C++ grammars, and grammars designed for Java parsers. This
9514 section summarizes these differences.
9515
9516 @itemize
9517 @item
9518 Java lacks a preprocessor, so the @code{YYERROR}, @code{YYACCEPT},
9519 @code{YYABORT} symbols (@pxref{Table of Symbols}) cannot obviously be
9520 macros. Instead, they should be preceded by @code{return} when they
9521 appear in an action. The actual definition of these symbols is
9522 opaque to the Bison grammar, and it might change in the future. The
9523 only meaningful operation that you can do, is to return them.
9524 See @pxref{Java Action Features}.
9525
9526 Note that of these three symbols, only @code{YYACCEPT} and
9527 @code{YYABORT} will cause a return from the @code{yyparse}
9528 method@footnote{Java parsers include the actions in a separate
9529 method than @code{yyparse} in order to have an intuitive syntax that
9530 corresponds to these C macros.}.
9531
9532 @item
9533 Java lacks unions, so @code{%union} has no effect. Instead, semantic
9534 values have a common base type: @code{Object} or as specified by
9535 @samp{%define stype}. Angle brackets on @code{%token}, @code{type},
9536 @code{$@var{n}} and @code{$$} specify subtypes rather than fields of
9537 an union. The type of @code{$$}, even with angle brackets, is the base
9538 type since Java casts are not allow on the left-hand side of assignments.
9539 Also, @code{$@var{n}} and @code{@@@var{n}} are not allowed on the
9540 left-hand side of assignments. See @pxref{Java Semantic Values} and
9541 @pxref{Java Action Features}.
9542
9543 @item
9544 The prologue declarations have a different meaning than in C/C++ code.
9545 @table @asis
9546 @item @code{%code imports}
9547 blocks are placed at the beginning of the Java source code. They may
9548 include copyright notices. For a @code{package} declarations, it is
9549 suggested to use @code{%define package} instead.
9550
9551 @item unqualified @code{%code}
9552 blocks are placed inside the parser class.
9553
9554 @item @code{%code lexer}
9555 blocks, if specified, should include the implementation of the
9556 scanner. If there is no such block, the scanner can be any class
9557 that implements the appropriate interface (see @pxref{Java Scanner
9558 Interface}).
9559 @end table
9560
9561 Other @code{%code} blocks are not supported in Java parsers.
9562 In particular, @code{%@{ @dots{} %@}} blocks should not be used
9563 and may give an error in future versions of Bison.
9564
9565 The epilogue has the same meaning as in C/C++ code and it can
9566 be used to define other classes used by the parser @emph{outside}
9567 the parser class.
9568 @end itemize
9569
9570
9571 @node Java Declarations Summary
9572 @subsection Java Declarations Summary
9573
9574 This summary only include declarations specific to Java or have special
9575 meaning when used in a Java parser.
9576
9577 @deffn {Directive} {%language "Java"}
9578 Generate a Java class for the parser.
9579 @end deffn
9580
9581 @deffn {Directive} %lex-param @{@var{type} @var{name}@}
9582 A parameter for the lexer class defined by @code{%code lexer}
9583 @emph{only}, added as parameters to the lexer constructor and the parser
9584 constructor that @emph{creates} a lexer. Default is none.
9585 @xref{Java Scanner Interface}.
9586 @end deffn
9587
9588 @deffn {Directive} %name-prefix "@var{prefix}"
9589 The prefix of the parser class name @code{@var{prefix}Parser} if
9590 @code{%define parser_class_name} is not used. Default is @code{YY}.
9591 @xref{Java Bison Interface}.
9592 @end deffn
9593
9594 @deffn {Directive} %parse-param @{@var{type} @var{name}@}
9595 A parameter for the parser class added as parameters to constructor(s)
9596 and as fields initialized by the constructor(s). Default is none.
9597 @xref{Java Parser Interface}.
9598 @end deffn
9599
9600 @deffn {Directive} %token <@var{type}> @var{token} @dots{}
9601 Declare tokens. Note that the angle brackets enclose a Java @emph{type}.
9602 @xref{Java Semantic Values}.
9603 @end deffn
9604
9605 @deffn {Directive} %type <@var{type}> @var{nonterminal} @dots{}
9606 Declare the type of nonterminals. Note that the angle brackets enclose
9607 a Java @emph{type}.
9608 @xref{Java Semantic Values}.
9609 @end deffn
9610
9611 @deffn {Directive} %code @{ @var{code} @dots{} @}
9612 Code appended to the inside of the parser class.
9613 @xref{Java Differences}.
9614 @end deffn
9615
9616 @deffn {Directive} {%code imports} @{ @var{code} @dots{} @}
9617 Code inserted just after the @code{package} declaration.
9618 @xref{Java Differences}.
9619 @end deffn
9620
9621 @deffn {Directive} {%code lexer} @{ @var{code} @dots{} @}
9622 Code added to the body of a inner lexer class within the parser class.
9623 @xref{Java Scanner Interface}.
9624 @end deffn
9625
9626 @deffn {Directive} %% @var{code} @dots{}
9627 Code (after the second @code{%%}) appended to the end of the file,
9628 @emph{outside} the parser class.
9629 @xref{Java Differences}.
9630 @end deffn
9631
9632 @deffn {Directive} %@{ @var{code} @dots{} %@}
9633 Not supported. Use @code{%code import} instead.
9634 @xref{Java Differences}.
9635 @end deffn
9636
9637 @deffn {Directive} {%define abstract}
9638 Whether the parser class is declared @code{abstract}. Default is false.
9639 @xref{Java Bison Interface}.
9640 @end deffn
9641
9642 @deffn {Directive} {%define extends} "@var{superclass}"
9643 The superclass of the parser class. Default is none.
9644 @xref{Java Bison Interface}.
9645 @end deffn
9646
9647 @deffn {Directive} {%define final}
9648 Whether the parser class is declared @code{final}. Default is false.
9649 @xref{Java Bison Interface}.
9650 @end deffn
9651
9652 @deffn {Directive} {%define implements} "@var{interfaces}"
9653 The implemented interfaces of the parser class, a comma-separated list.
9654 Default is none.
9655 @xref{Java Bison Interface}.
9656 @end deffn
9657
9658 @deffn {Directive} {%define lex_throws} "@var{exceptions}"
9659 The exceptions thrown by the @code{yylex} method of the lexer, a
9660 comma-separated list. Default is @code{java.io.IOException}.
9661 @xref{Java Scanner Interface}.
9662 @end deffn
9663
9664 @deffn {Directive} {%define location_type} "@var{class}"
9665 The name of the class used for locations (a range between two
9666 positions). This class is generated as an inner class of the parser
9667 class by @command{bison}. Default is @code{Location}.
9668 @xref{Java Location Values}.
9669 @end deffn
9670
9671 @deffn {Directive} {%define package} "@var{package}"
9672 The package to put the parser class in. Default is none.
9673 @xref{Java Bison Interface}.
9674 @end deffn
9675
9676 @deffn {Directive} {%define parser_class_name} "@var{name}"
9677 The name of the parser class. Default is @code{YYParser} or
9678 @code{@var{name-prefix}Parser}.
9679 @xref{Java Bison Interface}.
9680 @end deffn
9681
9682 @deffn {Directive} {%define position_type} "@var{class}"
9683 The name of the class used for positions. This class must be supplied by
9684 the user. Default is @code{Position}.
9685 @xref{Java Location Values}.
9686 @end deffn
9687
9688 @deffn {Directive} {%define public}
9689 Whether the parser class is declared @code{public}. Default is false.
9690 @xref{Java Bison Interface}.
9691 @end deffn
9692
9693 @deffn {Directive} {%define stype} "@var{class}"
9694 The base type of semantic values. Default is @code{Object}.
9695 @xref{Java Semantic Values}.
9696 @end deffn
9697
9698 @deffn {Directive} {%define strictfp}
9699 Whether the parser class is declared @code{strictfp}. Default is false.
9700 @xref{Java Bison Interface}.
9701 @end deffn
9702
9703 @deffn {Directive} {%define throws} "@var{exceptions}"
9704 The exceptions thrown by user-supplied parser actions and
9705 @code{%initial-action}, a comma-separated list. Default is none.
9706 @xref{Java Parser Interface}.
9707 @end deffn
9708
9709
9710 @c ================================================= FAQ
9711
9712 @node FAQ
9713 @chapter Frequently Asked Questions
9714 @cindex frequently asked questions
9715 @cindex questions
9716
9717 Several questions about Bison come up occasionally. Here some of them
9718 are addressed.
9719
9720 @menu
9721 * Memory Exhausted:: Breaking the Stack Limits
9722 * How Can I Reset the Parser:: @code{yyparse} Keeps some State
9723 * Strings are Destroyed:: @code{yylval} Loses Track of Strings
9724 * Implementing Gotos/Loops:: Control Flow in the Calculator
9725 * Multiple start-symbols:: Factoring closely related grammars
9726 * Secure? Conform?:: Is Bison POSIX safe?
9727 * I can't build Bison:: Troubleshooting
9728 * Where can I find help?:: Troubleshouting
9729 * Bug Reports:: Troublereporting
9730 * More Languages:: Parsers in C++, Java, and so on
9731 * Beta Testing:: Experimenting development versions
9732 * Mailing Lists:: Meeting other Bison users
9733 @end menu
9734
9735 @node Memory Exhausted
9736 @section Memory Exhausted
9737
9738 @display
9739 My parser returns with error with a @samp{memory exhausted}
9740 message. What can I do?
9741 @end display
9742
9743 This question is already addressed elsewhere, @xref{Recursion,
9744 ,Recursive Rules}.
9745
9746 @node How Can I Reset the Parser
9747 @section How Can I Reset the Parser
9748
9749 The following phenomenon has several symptoms, resulting in the
9750 following typical questions:
9751
9752 @display
9753 I invoke @code{yyparse} several times, and on correct input it works
9754 properly; but when a parse error is found, all the other calls fail
9755 too. How can I reset the error flag of @code{yyparse}?
9756 @end display
9757
9758 @noindent
9759 or
9760
9761 @display
9762 My parser includes support for an @samp{#include}-like feature, in
9763 which case I run @code{yyparse} from @code{yyparse}. This fails
9764 although I did specify @code{%define api.pure}.
9765 @end display
9766
9767 These problems typically come not from Bison itself, but from
9768 Lex-generated scanners. Because these scanners use large buffers for
9769 speed, they might not notice a change of input file. As a
9770 demonstration, consider the following source file,
9771 @file{first-line.l}:
9772
9773 @verbatim
9774 %{
9775 #include <stdio.h>
9776 #include <stdlib.h>
9777 %}
9778 %%
9779 .*\n ECHO; return 1;
9780 %%
9781 int
9782 yyparse (char const *file)
9783 {
9784 yyin = fopen (file, "r");
9785 if (!yyin)
9786 exit (2);
9787 /* One token only. */
9788 yylex ();
9789 if (fclose (yyin) != 0)
9790 exit (3);
9791 return 0;
9792 }
9793
9794 int
9795 main (void)
9796 {
9797 yyparse ("input");
9798 yyparse ("input");
9799 return 0;
9800 }
9801 @end verbatim
9802
9803 @noindent
9804 If the file @file{input} contains
9805
9806 @verbatim
9807 input:1: Hello,
9808 input:2: World!
9809 @end verbatim
9810
9811 @noindent
9812 then instead of getting the first line twice, you get:
9813
9814 @example
9815 $ @kbd{flex -ofirst-line.c first-line.l}
9816 $ @kbd{gcc -ofirst-line first-line.c -ll}
9817 $ @kbd{./first-line}
9818 input:1: Hello,
9819 input:2: World!
9820 @end example
9821
9822 Therefore, whenever you change @code{yyin}, you must tell the
9823 Lex-generated scanner to discard its current buffer and switch to the
9824 new one. This depends upon your implementation of Lex; see its
9825 documentation for more. For Flex, it suffices to call
9826 @samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
9827 Flex-generated scanner needs to read from several input streams to
9828 handle features like include files, you might consider using Flex
9829 functions like @samp{yy_switch_to_buffer} that manipulate multiple
9830 input buffers.
9831
9832 If your Flex-generated scanner uses start conditions (@pxref{Start
9833 conditions, , Start conditions, flex, The Flex Manual}), you might
9834 also want to reset the scanner's state, i.e., go back to the initial
9835 start condition, through a call to @samp{BEGIN (0)}.
9836
9837 @node Strings are Destroyed
9838 @section Strings are Destroyed
9839
9840 @display
9841 My parser seems to destroy old strings, or maybe it loses track of
9842 them. Instead of reporting @samp{"foo", "bar"}, it reports
9843 @samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
9844 @end display
9845
9846 This error is probably the single most frequent ``bug report'' sent to
9847 Bison lists, but is only concerned with a misunderstanding of the role
9848 of the scanner. Consider the following Lex code:
9849
9850 @verbatim
9851 %{
9852 #include <stdio.h>
9853 char *yylval = NULL;
9854 %}
9855 %%
9856 .* yylval = yytext; return 1;
9857 \n /* IGNORE */
9858 %%
9859 int
9860 main ()
9861 {
9862 /* Similar to using $1, $2 in a Bison action. */
9863 char *fst = (yylex (), yylval);
9864 char *snd = (yylex (), yylval);
9865 printf ("\"%s\", \"%s\"\n", fst, snd);
9866 return 0;
9867 }
9868 @end verbatim
9869
9870 If you compile and run this code, you get:
9871
9872 @example
9873 $ @kbd{flex -osplit-lines.c split-lines.l}
9874 $ @kbd{gcc -osplit-lines split-lines.c -ll}
9875 $ @kbd{printf 'one\ntwo\n' | ./split-lines}
9876 "one
9877 two", "two"
9878 @end example
9879
9880 @noindent
9881 this is because @code{yytext} is a buffer provided for @emph{reading}
9882 in the action, but if you want to keep it, you have to duplicate it
9883 (e.g., using @code{strdup}). Note that the output may depend on how
9884 your implementation of Lex handles @code{yytext}. For instance, when
9885 given the Lex compatibility option @option{-l} (which triggers the
9886 option @samp{%array}) Flex generates a different behavior:
9887
9888 @example
9889 $ @kbd{flex -l -osplit-lines.c split-lines.l}
9890 $ @kbd{gcc -osplit-lines split-lines.c -ll}
9891 $ @kbd{printf 'one\ntwo\n' | ./split-lines}
9892 "two", "two"
9893 @end example
9894
9895
9896 @node Implementing Gotos/Loops
9897 @section Implementing Gotos/Loops
9898
9899 @display
9900 My simple calculator supports variables, assignments, and functions,
9901 but how can I implement gotos, or loops?
9902 @end display
9903
9904 Although very pedagogical, the examples included in the document blur
9905 the distinction to make between the parser---whose job is to recover
9906 the structure of a text and to transmit it to subsequent modules of
9907 the program---and the processing (such as the execution) of this
9908 structure. This works well with so called straight line programs,
9909 i.e., precisely those that have a straightforward execution model:
9910 execute simple instructions one after the others.
9911
9912 @cindex abstract syntax tree
9913 @cindex AST
9914 If you want a richer model, you will probably need to use the parser
9915 to construct a tree that does represent the structure it has
9916 recovered; this tree is usually called the @dfn{abstract syntax tree},
9917 or @dfn{AST} for short. Then, walking through this tree,
9918 traversing it in various ways, will enable treatments such as its
9919 execution or its translation, which will result in an interpreter or a
9920 compiler.
9921
9922 This topic is way beyond the scope of this manual, and the reader is
9923 invited to consult the dedicated literature.
9924
9925
9926 @node Multiple start-symbols
9927 @section Multiple start-symbols
9928
9929 @display
9930 I have several closely related grammars, and I would like to share their
9931 implementations. In fact, I could use a single grammar but with
9932 multiple entry points.
9933 @end display
9934
9935 Bison does not support multiple start-symbols, but there is a very
9936 simple means to simulate them. If @code{foo} and @code{bar} are the two
9937 pseudo start-symbols, then introduce two new tokens, say
9938 @code{START_FOO} and @code{START_BAR}, and use them as switches from the
9939 real start-symbol:
9940
9941 @example
9942 %token START_FOO START_BAR;
9943 %start start;
9944 start: START_FOO foo
9945 | START_BAR bar;
9946 @end example
9947
9948 These tokens prevents the introduction of new conflicts. As far as the
9949 parser goes, that is all that is needed.
9950
9951 Now the difficult part is ensuring that the scanner will send these
9952 tokens first. If your scanner is hand-written, that should be
9953 straightforward. If your scanner is generated by Lex, them there is
9954 simple means to do it: recall that anything between @samp{%@{ ... %@}}
9955 after the first @code{%%} is copied verbatim in the top of the generated
9956 @code{yylex} function. Make sure a variable @code{start_token} is
9957 available in the scanner (e.g., a global variable or using
9958 @code{%lex-param} etc.), and use the following:
9959
9960 @example
9961 /* @r{Prologue.} */
9962 %%
9963 %@{
9964 if (start_token)
9965 @{
9966 int t = start_token;
9967 start_token = 0;
9968 return t;
9969 @}
9970 %@}
9971 /* @r{The rules.} */
9972 @end example
9973
9974
9975 @node Secure? Conform?
9976 @section Secure? Conform?
9977
9978 @display
9979 Is Bison secure? Does it conform to POSIX?
9980 @end display
9981
9982 If you're looking for a guarantee or certification, we don't provide it.
9983 However, Bison is intended to be a reliable program that conforms to the
9984 POSIX specification for Yacc. If you run into problems,
9985 please send us a bug report.
9986
9987 @node I can't build Bison
9988 @section I can't build Bison
9989
9990 @display
9991 I can't build Bison because @command{make} complains that
9992 @code{msgfmt} is not found.
9993 What should I do?
9994 @end display
9995
9996 Like most GNU packages with internationalization support, that feature
9997 is turned on by default. If you have problems building in the @file{po}
9998 subdirectory, it indicates that your system's internationalization
9999 support is lacking. You can re-configure Bison with
10000 @option{--disable-nls} to turn off this support, or you can install GNU
10001 gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
10002 Bison. See the file @file{ABOUT-NLS} for more information.
10003
10004
10005 @node Where can I find help?
10006 @section Where can I find help?
10007
10008 @display
10009 I'm having trouble using Bison. Where can I find help?
10010 @end display
10011
10012 First, read this fine manual. Beyond that, you can send mail to
10013 @email{help-bison@@gnu.org}. This mailing list is intended to be
10014 populated with people who are willing to answer questions about using
10015 and installing Bison. Please keep in mind that (most of) the people on
10016 the list have aspects of their lives which are not related to Bison (!),
10017 so you may not receive an answer to your question right away. This can
10018 be frustrating, but please try not to honk them off; remember that any
10019 help they provide is purely voluntary and out of the kindness of their
10020 hearts.
10021
10022 @node Bug Reports
10023 @section Bug Reports
10024
10025 @display
10026 I found a bug. What should I include in the bug report?
10027 @end display
10028
10029 Before you send a bug report, make sure you are using the latest
10030 version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
10031 mirrors. Be sure to include the version number in your bug report. If
10032 the bug is present in the latest version but not in a previous version,
10033 try to determine the most recent version which did not contain the bug.
10034
10035 If the bug is parser-related, you should include the smallest grammar
10036 you can which demonstrates the bug. The grammar file should also be
10037 complete (i.e., I should be able to run it through Bison without having
10038 to edit or add anything). The smaller and simpler the grammar, the
10039 easier it will be to fix the bug.
10040
10041 Include information about your compilation environment, including your
10042 operating system's name and version and your compiler's name and
10043 version. If you have trouble compiling, you should also include a
10044 transcript of the build session, starting with the invocation of
10045 `configure'. Depending on the nature of the bug, you may be asked to
10046 send additional files as well (such as `config.h' or `config.cache').
10047
10048 Patches are most welcome, but not required. That is, do not hesitate to
10049 send a bug report just because you can not provide a fix.
10050
10051 Send bug reports to @email{bug-bison@@gnu.org}.
10052
10053 @node More Languages
10054 @section More Languages
10055
10056 @display
10057 Will Bison ever have C++ and Java support? How about @var{insert your
10058 favorite language here}?
10059 @end display
10060
10061 C++ and Java support is there now, and is documented. We'd love to add other
10062 languages; contributions are welcome.
10063
10064 @node Beta Testing
10065 @section Beta Testing
10066
10067 @display
10068 What is involved in being a beta tester?
10069 @end display
10070
10071 It's not terribly involved. Basically, you would download a test
10072 release, compile it, and use it to build and run a parser or two. After
10073 that, you would submit either a bug report or a message saying that
10074 everything is okay. It is important to report successes as well as
10075 failures because test releases eventually become mainstream releases,
10076 but only if they are adequately tested. If no one tests, development is
10077 essentially halted.
10078
10079 Beta testers are particularly needed for operating systems to which the
10080 developers do not have easy access. They currently have easy access to
10081 recent GNU/Linux and Solaris versions. Reports about other operating
10082 systems are especially welcome.
10083
10084 @node Mailing Lists
10085 @section Mailing Lists
10086
10087 @display
10088 How do I join the help-bison and bug-bison mailing lists?
10089 @end display
10090
10091 See @url{http://lists.gnu.org/}.
10092
10093 @c ================================================= Table of Symbols
10094
10095 @node Table of Symbols
10096 @appendix Bison Symbols
10097 @cindex Bison symbols, table of
10098 @cindex symbols in Bison, table of
10099
10100 @deffn {Variable} @@$
10101 In an action, the location of the left-hand side of the rule.
10102 @xref{Locations, , Locations Overview}.
10103 @end deffn
10104
10105 @deffn {Variable} @@@var{n}
10106 In an action, the location of the @var{n}-th symbol of the right-hand
10107 side of the rule. @xref{Locations, , Locations Overview}.
10108 @end deffn
10109
10110 @deffn {Variable} @@@var{name}
10111 In an action, the location of a symbol addressed by name.
10112 @xref{Locations, , Locations Overview}.
10113 @end deffn
10114
10115 @deffn {Variable} @@[@var{name}]
10116 In an action, the location of a symbol addressed by name.
10117 @xref{Locations, , Locations Overview}.
10118 @end deffn
10119
10120 @deffn {Variable} $$
10121 In an action, the semantic value of the left-hand side of the rule.
10122 @xref{Actions}.
10123 @end deffn
10124
10125 @deffn {Variable} $@var{n}
10126 In an action, the semantic value of the @var{n}-th symbol of the
10127 right-hand side of the rule. @xref{Actions}.
10128 @end deffn
10129
10130 @deffn {Variable} $@var{name}
10131 In an action, the semantic value of a symbol addressed by name.
10132 @xref{Actions}.
10133 @end deffn
10134
10135 @deffn {Variable} $[@var{name}]
10136 In an action, the semantic value of a symbol addressed by name.
10137 @xref{Actions}.
10138 @end deffn
10139
10140 @deffn {Delimiter} %%
10141 Delimiter used to separate the grammar rule section from the
10142 Bison declarations section or the epilogue.
10143 @xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
10144 @end deffn
10145
10146 @c Don't insert spaces, or check the DVI output.
10147 @deffn {Delimiter} %@{@var{code}%@}
10148 All code listed between @samp{%@{} and @samp{%@}} is copied directly to
10149 the output file uninterpreted. Such code forms the prologue of the input
10150 file. @xref{Grammar Outline, ,Outline of a Bison
10151 Grammar}.
10152 @end deffn
10153
10154 @deffn {Construct} /*@dots{}*/
10155 Comment delimiters, as in C.
10156 @end deffn
10157
10158 @deffn {Delimiter} :
10159 Separates a rule's result from its components. @xref{Rules, ,Syntax of
10160 Grammar Rules}.
10161 @end deffn
10162
10163 @deffn {Delimiter} ;
10164 Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
10165 @end deffn
10166
10167 @deffn {Delimiter} |
10168 Separates alternate rules for the same result nonterminal.
10169 @xref{Rules, ,Syntax of Grammar Rules}.
10170 @end deffn
10171
10172 @deffn {Directive} <*>
10173 Used to define a default tagged @code{%destructor} or default tagged
10174 @code{%printer}.
10175
10176 This feature is experimental.
10177 More user feedback will help to determine whether it should become a permanent
10178 feature.
10179
10180 @xref{Destructor Decl, , Freeing Discarded Symbols}.
10181 @end deffn
10182
10183 @deffn {Directive} <>
10184 Used to define a default tagless @code{%destructor} or default tagless
10185 @code{%printer}.
10186
10187 This feature is experimental.
10188 More user feedback will help to determine whether it should become a permanent
10189 feature.
10190
10191 @xref{Destructor Decl, , Freeing Discarded Symbols}.
10192 @end deffn
10193
10194 @deffn {Symbol} $accept
10195 The predefined nonterminal whose only rule is @samp{$accept: @var{start}
10196 $end}, where @var{start} is the start symbol. @xref{Start Decl, , The
10197 Start-Symbol}. It cannot be used in the grammar.
10198 @end deffn
10199
10200 @deffn {Directive} %code @{@var{code}@}
10201 @deffnx {Directive} %code @var{qualifier} @{@var{code}@}
10202 Insert @var{code} verbatim into output parser source.
10203 @xref{Decl Summary,,%code}.
10204 @end deffn
10205
10206 @deffn {Directive} %debug
10207 Equip the parser for debugging. @xref{Decl Summary}.
10208 @end deffn
10209
10210 @ifset defaultprec
10211 @deffn {Directive} %default-prec
10212 Assign a precedence to rules that lack an explicit @samp{%prec}
10213 modifier. @xref{Contextual Precedence, ,Context-Dependent
10214 Precedence}.
10215 @end deffn
10216 @end ifset
10217
10218 @deffn {Directive} %define @var{define-variable}
10219 @deffnx {Directive} %define @var{define-variable} @var{value}
10220 @deffnx {Directive} %define @var{define-variable} "@var{value}"
10221 Define a variable to adjust Bison's behavior.
10222 @xref{Decl Summary,,%define}.
10223 @end deffn
10224
10225 @deffn {Directive} %defines
10226 Bison declaration to create a header file meant for the scanner.
10227 @xref{Decl Summary}.
10228 @end deffn
10229
10230 @deffn {Directive} %defines @var{defines-file}
10231 Same as above, but save in the file @var{defines-file}.
10232 @xref{Decl Summary}.
10233 @end deffn
10234
10235 @deffn {Directive} %destructor
10236 Specify how the parser should reclaim the memory associated to
10237 discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
10238 @end deffn
10239
10240 @deffn {Directive} %dprec
10241 Bison declaration to assign a precedence to a rule that is used at parse
10242 time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
10243 GLR Parsers}.
10244 @end deffn
10245
10246 @deffn {Symbol} $end
10247 The predefined token marking the end of the token stream. It cannot be
10248 used in the grammar.
10249 @end deffn
10250
10251 @deffn {Symbol} error
10252 A token name reserved for error recovery. This token may be used in
10253 grammar rules so as to allow the Bison parser to recognize an error in
10254 the grammar without halting the process. In effect, a sentence
10255 containing an error may be recognized as valid. On a syntax error, the
10256 token @code{error} becomes the current lookahead token. Actions
10257 corresponding to @code{error} are then executed, and the lookahead
10258 token is reset to the token that originally caused the violation.
10259 @xref{Error Recovery}.
10260 @end deffn
10261
10262 @deffn {Directive} %error-verbose
10263 Bison declaration to request verbose, specific error message strings
10264 when @code{yyerror} is called.
10265 @end deffn
10266
10267 @deffn {Directive} %file-prefix "@var{prefix}"
10268 Bison declaration to set the prefix of the output files. @xref{Decl
10269 Summary}.
10270 @end deffn
10271
10272 @deffn {Directive} %glr-parser
10273 Bison declaration to produce a GLR parser. @xref{GLR
10274 Parsers, ,Writing GLR Parsers}.
10275 @end deffn
10276
10277 @deffn {Directive} %initial-action
10278 Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
10279 @end deffn
10280
10281 @deffn {Directive} %language
10282 Specify the programming language for the generated parser.
10283 @xref{Decl Summary}.
10284 @end deffn
10285
10286 @deffn {Directive} %left
10287 Bison declaration to assign left associativity to token(s).
10288 @xref{Precedence Decl, ,Operator Precedence}.
10289 @end deffn
10290
10291 @deffn {Directive} %lex-param @{@var{argument-declaration}@}
10292 Bison declaration to specifying an additional parameter that
10293 @code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
10294 for Pure Parsers}.
10295 @end deffn
10296
10297 @deffn {Directive} %merge
10298 Bison declaration to assign a merging function to a rule. If there is a
10299 reduce/reduce conflict with a rule having the same merging function, the
10300 function is applied to the two semantic values to get a single result.
10301 @xref{GLR Parsers, ,Writing GLR Parsers}.
10302 @end deffn
10303
10304 @deffn {Directive} %name-prefix "@var{prefix}"
10305 Bison declaration to rename the external symbols. @xref{Decl Summary}.
10306 @end deffn
10307
10308 @ifset defaultprec
10309 @deffn {Directive} %no-default-prec
10310 Do not assign a precedence to rules that lack an explicit @samp{%prec}
10311 modifier. @xref{Contextual Precedence, ,Context-Dependent
10312 Precedence}.
10313 @end deffn
10314 @end ifset
10315
10316 @deffn {Directive} %no-lines
10317 Bison declaration to avoid generating @code{#line} directives in the
10318 parser file. @xref{Decl Summary}.
10319 @end deffn
10320
10321 @deffn {Directive} %nonassoc
10322 Bison declaration to assign nonassociativity to token(s).
10323 @xref{Precedence Decl, ,Operator Precedence}.
10324 @end deffn
10325
10326 @deffn {Directive} %output "@var{file}"
10327 Bison declaration to set the name of the parser file. @xref{Decl
10328 Summary}.
10329 @end deffn
10330
10331 @deffn {Directive} %parse-param @{@var{argument-declaration}@}
10332 Bison declaration to specifying an additional parameter that
10333 @code{yyparse} should accept. @xref{Parser Function,, The Parser
10334 Function @code{yyparse}}.
10335 @end deffn
10336
10337 @deffn {Directive} %prec
10338 Bison declaration to assign a precedence to a specific rule.
10339 @xref{Contextual Precedence, ,Context-Dependent Precedence}.
10340 @end deffn
10341
10342 @deffn {Directive} %pure-parser
10343 Deprecated version of @code{%define api.pure} (@pxref{Decl Summary, ,%define}),
10344 for which Bison is more careful to warn about unreasonable usage.
10345 @end deffn
10346
10347 @deffn {Directive} %require "@var{version}"
10348 Require version @var{version} or higher of Bison. @xref{Require Decl, ,
10349 Require a Version of Bison}.
10350 @end deffn
10351
10352 @deffn {Directive} %right
10353 Bison declaration to assign right associativity to token(s).
10354 @xref{Precedence Decl, ,Operator Precedence}.
10355 @end deffn
10356
10357 @deffn {Directive} %skeleton
10358 Specify the skeleton to use; usually for development.
10359 @xref{Decl Summary}.
10360 @end deffn
10361
10362 @deffn {Directive} %start
10363 Bison declaration to specify the start symbol. @xref{Start Decl, ,The
10364 Start-Symbol}.
10365 @end deffn
10366
10367 @deffn {Directive} %token
10368 Bison declaration to declare token(s) without specifying precedence.
10369 @xref{Token Decl, ,Token Type Names}.
10370 @end deffn
10371
10372 @deffn {Directive} %token-table
10373 Bison declaration to include a token name table in the parser file.
10374 @xref{Decl Summary}.
10375 @end deffn
10376
10377 @deffn {Directive} %type
10378 Bison declaration to declare nonterminals. @xref{Type Decl,
10379 ,Nonterminal Symbols}.
10380 @end deffn
10381
10382 @deffn {Symbol} $undefined
10383 The predefined token onto which all undefined values returned by
10384 @code{yylex} are mapped. It cannot be used in the grammar, rather, use
10385 @code{error}.
10386 @end deffn
10387
10388 @deffn {Directive} %union
10389 Bison declaration to specify several possible data types for semantic
10390 values. @xref{Union Decl, ,The Collection of Value Types}.
10391 @end deffn
10392
10393 @deffn {Macro} YYABORT
10394 Macro to pretend that an unrecoverable syntax error has occurred, by
10395 making @code{yyparse} return 1 immediately. The error reporting
10396 function @code{yyerror} is not called. @xref{Parser Function, ,The
10397 Parser Function @code{yyparse}}.
10398
10399 For Java parsers, this functionality is invoked using @code{return YYABORT;}
10400 instead.
10401 @end deffn
10402
10403 @deffn {Macro} YYACCEPT
10404 Macro to pretend that a complete utterance of the language has been
10405 read, by making @code{yyparse} return 0 immediately.
10406 @xref{Parser Function, ,The Parser Function @code{yyparse}}.
10407
10408 For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
10409 instead.
10410 @end deffn
10411
10412 @deffn {Macro} YYBACKUP
10413 Macro to discard a value from the parser stack and fake a lookahead
10414 token. @xref{Action Features, ,Special Features for Use in Actions}.
10415 @end deffn
10416
10417 @deffn {Variable} yychar
10418 External integer variable that contains the integer value of the
10419 lookahead token. (In a pure parser, it is a local variable within
10420 @code{yyparse}.) Error-recovery rule actions may examine this variable.
10421 @xref{Action Features, ,Special Features for Use in Actions}.
10422 @end deffn
10423
10424 @deffn {Variable} yyclearin
10425 Macro used in error-recovery rule actions. It clears the previous
10426 lookahead token. @xref{Error Recovery}.
10427 @end deffn
10428
10429 @deffn {Macro} YYDEBUG
10430 Macro to define to equip the parser with tracing code. @xref{Tracing,
10431 ,Tracing Your Parser}.
10432 @end deffn
10433
10434 @deffn {Variable} yydebug
10435 External integer variable set to zero by default. If @code{yydebug}
10436 is given a nonzero value, the parser will output information on input
10437 symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
10438 @end deffn
10439
10440 @deffn {Macro} yyerrok
10441 Macro to cause parser to recover immediately to its normal mode
10442 after a syntax error. @xref{Error Recovery}.
10443 @end deffn
10444
10445 @deffn {Macro} YYERROR
10446 Macro to pretend that a syntax error has just been detected: call
10447 @code{yyerror} and then perform normal error recovery if possible
10448 (@pxref{Error Recovery}), or (if recovery is impossible) make
10449 @code{yyparse} return 1. @xref{Error Recovery}.
10450
10451 For Java parsers, this functionality is invoked using @code{return YYERROR;}
10452 instead.
10453 @end deffn
10454
10455 @deffn {Function} yyerror
10456 User-supplied function to be called by @code{yyparse} on error.
10457 @xref{Error Reporting, ,The Error
10458 Reporting Function @code{yyerror}}.
10459 @end deffn
10460
10461 @deffn {Macro} YYERROR_VERBOSE
10462 An obsolete macro that you define with @code{#define} in the prologue
10463 to request verbose, specific error message strings
10464 when @code{yyerror} is called. It doesn't matter what definition you
10465 use for @code{YYERROR_VERBOSE}, just whether you define it. Using
10466 @code{%error-verbose} is preferred.
10467 @end deffn
10468
10469 @deffn {Macro} YYINITDEPTH
10470 Macro for specifying the initial size of the parser stack.
10471 @xref{Memory Management}.
10472 @end deffn
10473
10474 @deffn {Function} yylex
10475 User-supplied lexical analyzer function, called with no arguments to get
10476 the next token. @xref{Lexical, ,The Lexical Analyzer Function
10477 @code{yylex}}.
10478 @end deffn
10479
10480 @deffn {Macro} YYLEX_PARAM
10481 An obsolete macro for specifying an extra argument (or list of extra
10482 arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
10483 macro is deprecated, and is supported only for Yacc like parsers.
10484 @xref{Pure Calling,, Calling Conventions for Pure Parsers}.
10485 @end deffn
10486
10487 @deffn {Variable} yylloc
10488 External variable in which @code{yylex} should place the line and column
10489 numbers associated with a token. (In a pure parser, it is a local
10490 variable within @code{yyparse}, and its address is passed to
10491 @code{yylex}.)
10492 You can ignore this variable if you don't use the @samp{@@} feature in the
10493 grammar actions.
10494 @xref{Token Locations, ,Textual Locations of Tokens}.
10495 In semantic actions, it stores the location of the lookahead token.
10496 @xref{Actions and Locations, ,Actions and Locations}.
10497 @end deffn
10498
10499 @deffn {Type} YYLTYPE
10500 Data type of @code{yylloc}; by default, a structure with four
10501 members. @xref{Location Type, , Data Types of Locations}.
10502 @end deffn
10503
10504 @deffn {Variable} yylval
10505 External variable in which @code{yylex} should place the semantic
10506 value associated with a token. (In a pure parser, it is a local
10507 variable within @code{yyparse}, and its address is passed to
10508 @code{yylex}.)
10509 @xref{Token Values, ,Semantic Values of Tokens}.
10510 In semantic actions, it stores the semantic value of the lookahead token.
10511 @xref{Actions, ,Actions}.
10512 @end deffn
10513
10514 @deffn {Macro} YYMAXDEPTH
10515 Macro for specifying the maximum size of the parser stack. @xref{Memory
10516 Management}.
10517 @end deffn
10518
10519 @deffn {Variable} yynerrs
10520 Global variable which Bison increments each time it reports a syntax error.
10521 (In a pure parser, it is a local variable within @code{yyparse}. In a
10522 pure push parser, it is a member of yypstate.)
10523 @xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
10524 @end deffn
10525
10526 @deffn {Function} yyparse
10527 The parser function produced by Bison; call this function to start
10528 parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
10529 @end deffn
10530
10531 @deffn {Function} yypstate_delete
10532 The function to delete a parser instance, produced by Bison in push mode;
10533 call this function to delete the memory associated with a parser.
10534 @xref{Parser Delete Function, ,The Parser Delete Function
10535 @code{yypstate_delete}}.
10536 (The current push parsing interface is experimental and may evolve.
10537 More user feedback will help to stabilize it.)
10538 @end deffn
10539
10540 @deffn {Function} yypstate_new
10541 The function to create a parser instance, produced by Bison in push mode;
10542 call this function to create a new parser.
10543 @xref{Parser Create Function, ,The Parser Create Function
10544 @code{yypstate_new}}.
10545 (The current push parsing interface is experimental and may evolve.
10546 More user feedback will help to stabilize it.)
10547 @end deffn
10548
10549 @deffn {Function} yypull_parse
10550 The parser function produced by Bison in push mode; call this function to
10551 parse the rest of the input stream.
10552 @xref{Pull Parser Function, ,The Pull Parser Function
10553 @code{yypull_parse}}.
10554 (The current push parsing interface is experimental and may evolve.
10555 More user feedback will help to stabilize it.)
10556 @end deffn
10557
10558 @deffn {Function} yypush_parse
10559 The parser function produced by Bison in push mode; call this function to
10560 parse a single token. @xref{Push Parser Function, ,The Push Parser Function
10561 @code{yypush_parse}}.
10562 (The current push parsing interface is experimental and may evolve.
10563 More user feedback will help to stabilize it.)
10564 @end deffn
10565
10566 @deffn {Macro} YYPARSE_PARAM
10567 An obsolete macro for specifying the name of a parameter that
10568 @code{yyparse} should accept. The use of this macro is deprecated, and
10569 is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
10570 Conventions for Pure Parsers}.
10571 @end deffn
10572
10573 @deffn {Macro} YYRECOVERING
10574 The expression @code{YYRECOVERING ()} yields 1 when the parser
10575 is recovering from a syntax error, and 0 otherwise.
10576 @xref{Action Features, ,Special Features for Use in Actions}.
10577 @end deffn
10578
10579 @deffn {Macro} YYSTACK_USE_ALLOCA
10580 Macro used to control the use of @code{alloca} when the
10581 deterministic parser in C needs to extend its stacks. If defined to 0,
10582 the parser will use @code{malloc} to extend its stacks. If defined to
10583 1, the parser will use @code{alloca}. Values other than 0 and 1 are
10584 reserved for future Bison extensions. If not defined,
10585 @code{YYSTACK_USE_ALLOCA} defaults to 0.
10586
10587 In the all-too-common case where your code may run on a host with a
10588 limited stack and with unreliable stack-overflow checking, you should
10589 set @code{YYMAXDEPTH} to a value that cannot possibly result in
10590 unchecked stack overflow on any of your target hosts when
10591 @code{alloca} is called. You can inspect the code that Bison
10592 generates in order to determine the proper numeric values. This will
10593 require some expertise in low-level implementation details.
10594 @end deffn
10595
10596 @deffn {Type} YYSTYPE
10597 Data type of semantic values; @code{int} by default.
10598 @xref{Value Type, ,Data Types of Semantic Values}.
10599 @end deffn
10600
10601 @node Glossary
10602 @appendix Glossary
10603 @cindex glossary
10604
10605 @table @asis
10606 @item Accepting State
10607 A state whose only action is the accept action.
10608 The accepting state is thus a consistent state.
10609 @xref{Understanding,,}.
10610
10611 @item Backus-Naur Form (BNF; also called ``Backus Normal Form'')
10612 Formal method of specifying context-free grammars originally proposed
10613 by John Backus, and slightly improved by Peter Naur in his 1960-01-02
10614 committee document contributing to what became the Algol 60 report.
10615 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10616
10617 @item Consistent State
10618 A state containing only one possible action.
10619 @xref{Decl Summary,,lr.default-reductions}.
10620
10621 @item Context-free grammars
10622 Grammars specified as rules that can be applied regardless of context.
10623 Thus, if there is a rule which says that an integer can be used as an
10624 expression, integers are allowed @emph{anywhere} an expression is
10625 permitted. @xref{Language and Grammar, ,Languages and Context-Free
10626 Grammars}.
10627
10628 @item Default Reduction
10629 The reduction that a parser should perform if the current parser state
10630 contains no other action for the lookahead token.
10631 In permitted parser states, Bison declares the reduction with the
10632 largest lookahead set to be the default reduction and removes that
10633 lookahead set.
10634 @xref{Decl Summary,,lr.default-reductions}.
10635
10636 @item Dynamic allocation
10637 Allocation of memory that occurs during execution, rather than at
10638 compile time or on entry to a function.
10639
10640 @item Empty string
10641 Analogous to the empty set in set theory, the empty string is a
10642 character string of length zero.
10643
10644 @item Finite-state stack machine
10645 A ``machine'' that has discrete states in which it is said to exist at
10646 each instant in time. As input to the machine is processed, the
10647 machine moves from state to state as specified by the logic of the
10648 machine. In the case of the parser, the input is the language being
10649 parsed, and the states correspond to various stages in the grammar
10650 rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
10651
10652 @item Generalized LR (GLR)
10653 A parsing algorithm that can handle all context-free grammars, including those
10654 that are not LR(1). It resolves situations that Bison's
10655 deterministic parsing
10656 algorithm cannot by effectively splitting off multiple parsers, trying all
10657 possible parsers, and discarding those that fail in the light of additional
10658 right context. @xref{Generalized LR Parsing, ,Generalized
10659 LR Parsing}.
10660
10661 @item Grouping
10662 A language construct that is (in general) grammatically divisible;
10663 for example, `expression' or `declaration' in C@.
10664 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10665
10666 @item IELR(1)
10667 A minimal LR(1) parser table generation algorithm.
10668 That is, given any context-free grammar, IELR(1) generates
10669 parser tables with the full language recognition power of canonical
10670 LR(1) but with nearly the same number of parser states as
10671 LALR(1).
10672 This reduction in parser states is often an order of magnitude.
10673 More importantly, because canonical LR(1)'s extra parser
10674 states may contain duplicate conflicts in the case of
10675 non-LR(1) grammars, the number of conflicts for
10676 IELR(1) is often an order of magnitude less as well.
10677 This can significantly reduce the complexity of developing of a grammar.
10678 @xref{Decl Summary,,lr.type}.
10679
10680 @item Infix operator
10681 An arithmetic operator that is placed between the operands on which it
10682 performs some operation.
10683
10684 @item Input stream
10685 A continuous flow of data between devices or programs.
10686
10687 @item LAC (Lookahead Correction)
10688 A parsing mechanism that fixes the problem of delayed syntax error
10689 detection, which is caused by LR state merging, default reductions, and
10690 the use of @code{%nonassoc}. Delayed syntax error detection results in
10691 unexpected semantic actions, initiation of error recovery in the wrong
10692 syntactic context, and an incorrect list of expected tokens in a verbose
10693 syntax error message. @xref{Decl Summary,,parse.lac}.
10694
10695 @item Language construct
10696 One of the typical usage schemas of the language. For example, one of
10697 the constructs of the C language is the @code{if} statement.
10698 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10699
10700 @item Left associativity
10701 Operators having left associativity are analyzed from left to right:
10702 @samp{a+b+c} first computes @samp{a+b} and then combines with
10703 @samp{c}. @xref{Precedence, ,Operator Precedence}.
10704
10705 @item Left recursion
10706 A rule whose result symbol is also its first component symbol; for
10707 example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
10708 Rules}.
10709
10710 @item Left-to-right parsing
10711 Parsing a sentence of a language by analyzing it token by token from
10712 left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
10713
10714 @item Lexical analyzer (scanner)
10715 A function that reads an input stream and returns tokens one by one.
10716 @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
10717
10718 @item Lexical tie-in
10719 A flag, set by actions in the grammar rules, which alters the way
10720 tokens are parsed. @xref{Lexical Tie-ins}.
10721
10722 @item Literal string token
10723 A token which consists of two or more fixed characters. @xref{Symbols}.
10724
10725 @item Lookahead token
10726 A token already read but not yet shifted. @xref{Lookahead, ,Lookahead
10727 Tokens}.
10728
10729 @item LALR(1)
10730 The class of context-free grammars that Bison (like most other parser
10731 generators) can handle by default; a subset of LR(1).
10732 @xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
10733
10734 @item LR(1)
10735 The class of context-free grammars in which at most one token of
10736 lookahead is needed to disambiguate the parsing of any piece of input.
10737
10738 @item Nonterminal symbol
10739 A grammar symbol standing for a grammatical construct that can
10740 be expressed through rules in terms of smaller constructs; in other
10741 words, a construct that is not a token. @xref{Symbols}.
10742
10743 @item Parser
10744 A function that recognizes valid sentences of a language by analyzing
10745 the syntax structure of a set of tokens passed to it from a lexical
10746 analyzer.
10747
10748 @item Postfix operator
10749 An arithmetic operator that is placed after the operands upon which it
10750 performs some operation.
10751
10752 @item Reduction
10753 Replacing a string of nonterminals and/or terminals with a single
10754 nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
10755 Parser Algorithm}.
10756
10757 @item Reentrant
10758 A reentrant subprogram is a subprogram which can be in invoked any
10759 number of times in parallel, without interference between the various
10760 invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
10761
10762 @item Reverse polish notation
10763 A language in which all operators are postfix operators.
10764
10765 @item Right recursion
10766 A rule whose result symbol is also its last component symbol; for
10767 example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
10768 Rules}.
10769
10770 @item Semantics
10771 In computer languages, the semantics are specified by the actions
10772 taken for each instance of the language, i.e., the meaning of
10773 each statement. @xref{Semantics, ,Defining Language Semantics}.
10774
10775 @item Shift
10776 A parser is said to shift when it makes the choice of analyzing
10777 further input from the stream rather than reducing immediately some
10778 already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
10779
10780 @item Single-character literal
10781 A single character that is recognized and interpreted as is.
10782 @xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
10783
10784 @item Start symbol
10785 The nonterminal symbol that stands for a complete valid utterance in
10786 the language being parsed. The start symbol is usually listed as the
10787 first nonterminal symbol in a language specification.
10788 @xref{Start Decl, ,The Start-Symbol}.
10789
10790 @item Symbol table
10791 A data structure where symbol names and associated data are stored
10792 during parsing to allow for recognition and use of existing
10793 information in repeated uses of a symbol. @xref{Multi-function Calc}.
10794
10795 @item Syntax error
10796 An error encountered during parsing of an input stream due to invalid
10797 syntax. @xref{Error Recovery}.
10798
10799 @item Token
10800 A basic, grammatically indivisible unit of a language. The symbol
10801 that describes a token in the grammar is a terminal symbol.
10802 The input of the Bison parser is a stream of tokens which comes from
10803 the lexical analyzer. @xref{Symbols}.
10804
10805 @item Terminal symbol
10806 A grammar symbol that has no rules in the grammar and therefore is
10807 grammatically indivisible. The piece of text it represents is a token.
10808 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10809 @end table
10810
10811 @node Copying This Manual
10812 @appendix Copying This Manual
10813 @include fdl.texi
10814
10815 @node Index
10816 @unnumbered Index
10817
10818 @printindex cp
10819
10820 @bye
10821
10822 @c Local Variables:
10823 @c fill-column: 76
10824 @c End:
10825
10826 @c LocalWords: texinfo setfilename settitle setchapternewpage finalout texi FSF
10827 @c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex FSF's
10828 @c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry Naur
10829 @c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa Multi
10830 @c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc multi
10831 @c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex defaultprec Donnelly Gotos
10832 @c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref yypush
10833 @c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex lr
10834 @c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge POSIX
10835 @c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG yypull
10836 @c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit nonfree
10837 @c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok rr
10838 @c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln Stallman Destructor
10839 @c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym enum
10840 @c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof Lex
10841 @c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum DOTDOT
10842 @c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype Unary
10843 @c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs nonterminal
10844 @c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES reentrant
10845 @c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param yypstate
10846 @c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP subrange
10847 @c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword loc
10848 @c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH inline
10849 @c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype Lookahead
10850 @c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args Autoconf
10851 @c LocalWords: infile ypp yxx outfile itemx tex leaderfill Troubleshouting sqrt
10852 @c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll lookahead
10853 @c LocalWords: nbar yytext fst snd osplit ntwo strdup AST Troublereporting th
10854 @c LocalWords: YYSTACK DVI fdl printindex IELR nondeterministic nonterminals ps
10855 @c LocalWords: subexpressions declarator nondeferred config libintl postfix LAC
10856 @c LocalWords: preprocessor nonpositive unary nonnumeric typedef extern rhs
10857 @c LocalWords: yytokentype filename destructor multicharacter nonnull EBCDIC
10858 @c LocalWords: lvalue nonnegative XNUM CHR chr TAGLESS tagless stdout api TOK
10859 @c LocalWords: destructors Reentrancy nonreentrant subgrammar nonassociative
10860 @c LocalWords: deffnx namespace xml goto lalr ielr runtime lex yacc yyps env
10861 @c LocalWords: yystate variadic Unshift NLS gettext po UTF Automake LOCALEDIR
10862 @c LocalWords: YYENABLE bindtextdomain Makefile DEFS CPPFLAGS DBISON DeRemer
10863 @c LocalWords: autoreconf Pennello multisets nondeterminism Generalised baz
10864 @c LocalWords: redeclare automata Dparse localedir datadir XSLT midrule Wno
10865 @c LocalWords: makefiles Graphviz multitable headitem hh basename Doxygen fno
10866 @c LocalWords: doxygen ival sval deftypemethod deallocate pos deftypemethodx
10867 @c LocalWords: Ctor defcv defcvx arg accessors arithmetics CPP ifndef CALCXX
10868 @c LocalWords: lexer's calcxx bool LPAREN RPAREN deallocation cerrno climits
10869 @c LocalWords: cstdlib Debian undef yywrap unput noyywrap nounput zA yyleng
10870 @c LocalWords: errno strtol ERANGE str strerror iostream argc argv Javadoc
10871 @c LocalWords: bytecode initializers superclass stype ASTNode autoboxing nls
10872 @c LocalWords: toString deftypeivar deftypeivarx deftypeop YYParser strictfp
10873 @c LocalWords: superclasses boolean getErrorVerbose setErrorVerbose deftypecv
10874 @c LocalWords: getDebugStream setDebugStream getDebugLevel setDebugLevel url
10875 @c LocalWords: bisonVersion deftypecvx bisonSkeleton getStartPos getEndPos
10876 @c LocalWords: getLVal defvar deftypefn deftypefnx gotos msgfmt
10877 @c LocalWords: subdirectory Solaris nonassociativity