Convert underscores to dashes in some %define variable names.
[bison.git] / doc / bison.texinfo
1 \input texinfo @c -*-texinfo-*-
2 @comment %**start of header
3 @setfilename bison.info
4 @include version.texi
5 @settitle Bison @value{VERSION}
6 @setchapternewpage odd
7
8 @finalout
9
10 @c SMALL BOOK version
11 @c This edition has been formatted so that you can format and print it in
12 @c the smallbook format.
13 @c @smallbook
14
15 @c Set following if you want to document %default-prec and %no-default-prec.
16 @c This feature is experimental and may change in future Bison versions.
17 @c @set defaultprec
18
19 @ifnotinfo
20 @syncodeindex fn cp
21 @syncodeindex vr cp
22 @syncodeindex tp cp
23 @end ifnotinfo
24 @ifinfo
25 @synindex fn cp
26 @synindex vr cp
27 @synindex tp cp
28 @end ifinfo
29 @comment %**end of header
30
31 @copying
32
33 This manual (@value{UPDATED}) is for @acronym{GNU} Bison (version
34 @value{VERSION}), the @acronym{GNU} parser generator.
35
36 Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
37 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Free
38 Software Foundation, Inc.
39
40 @quotation
41 Permission is granted to copy, distribute and/or modify this document
42 under the terms of the @acronym{GNU} Free Documentation License,
43 Version 1.2 or any later version published by the Free Software
44 Foundation; with no Invariant Sections, with the Front-Cover texts
45 being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in
46 (a) below. A copy of the license is included in the section entitled
47 ``@acronym{GNU} Free Documentation License.''
48
49 (a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
50 modify this @acronym{GNU} manual. Buying copies from the @acronym{FSF}
51 supports it in developing @acronym{GNU} and promoting software
52 freedom.''
53 @end quotation
54 @end copying
55
56 @dircategory Software development
57 @direntry
58 * bison: (bison). @acronym{GNU} parser generator (Yacc replacement).
59 @end direntry
60
61 @titlepage
62 @title Bison
63 @subtitle The Yacc-compatible Parser Generator
64 @subtitle @value{UPDATED}, Bison Version @value{VERSION}
65
66 @author by Charles Donnelly and Richard Stallman
67
68 @page
69 @vskip 0pt plus 1filll
70 @insertcopying
71 @sp 2
72 Published by the Free Software Foundation @*
73 51 Franklin Street, Fifth Floor @*
74 Boston, MA 02110-1301 USA @*
75 Printed copies are available from the Free Software Foundation.@*
76 @acronym{ISBN} 1-882114-44-2
77 @sp 2
78 Cover art by Etienne Suvasa.
79 @end titlepage
80
81 @contents
82
83 @ifnottex
84 @node Top
85 @top Bison
86 @insertcopying
87 @end ifnottex
88
89 @menu
90 * Introduction::
91 * Conditions::
92 * Copying:: The @acronym{GNU} General Public License says
93 how you can copy and share Bison.
94
95 Tutorial sections:
96 * Concepts:: Basic concepts for understanding Bison.
97 * Examples:: Three simple explained examples of using Bison.
98
99 Reference sections:
100 * Grammar File:: Writing Bison declarations and rules.
101 * Interface:: C-language interface to the parser function @code{yyparse}.
102 * Algorithm:: How the Bison parser works at run-time.
103 * Error Recovery:: Writing rules for error recovery.
104 * Context Dependency:: What to do if your language syntax is too
105 messy for Bison to handle straightforwardly.
106 * Debugging:: Understanding or debugging Bison parsers.
107 * Invocation:: How to run Bison (to produce the parser source file).
108 * Other Languages:: Creating C++ and Java parsers.
109 * FAQ:: Frequently Asked Questions
110 * Table of Symbols:: All the keywords of the Bison language are explained.
111 * Glossary:: Basic concepts are explained.
112 * Copying This Manual:: License for copying this manual.
113 * Index:: Cross-references to the text.
114
115 @detailmenu
116 --- The Detailed Node Listing ---
117
118 The Concepts of Bison
119
120 * Language and Grammar:: Languages and context-free grammars,
121 as mathematical ideas.
122 * Grammar in Bison:: How we represent grammars for Bison's sake.
123 * Semantic Values:: Each token or syntactic grouping can have
124 a semantic value (the value of an integer,
125 the name of an identifier, etc.).
126 * Semantic Actions:: Each rule can have an action containing C code.
127 * GLR Parsers:: Writing parsers for general context-free languages.
128 * Locations Overview:: Tracking Locations.
129 * Bison Parser:: What are Bison's input and output,
130 how is the output used?
131 * Stages:: Stages in writing and running Bison grammars.
132 * Grammar Layout:: Overall structure of a Bison grammar file.
133
134 Writing @acronym{GLR} Parsers
135
136 * Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
137 * Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
138 * GLR Semantic Actions:: Deferred semantic actions have special concerns.
139 * Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
140
141 Examples
142
143 * RPN Calc:: Reverse polish notation calculator;
144 a first example with no operator precedence.
145 * Infix Calc:: Infix (algebraic) notation calculator.
146 Operator precedence is introduced.
147 * Simple Error Recovery:: Continuing after syntax errors.
148 * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
149 * Multi-function Calc:: Calculator with memory and trig functions.
150 It uses multiple data-types for semantic values.
151 * Exercises:: Ideas for improving the multi-function calculator.
152
153 Reverse Polish Notation Calculator
154
155 * Rpcalc Declarations:: Prologue (declarations) for rpcalc.
156 * Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
157 * Rpcalc Lexer:: The lexical analyzer.
158 * Rpcalc Main:: The controlling function.
159 * Rpcalc Error:: The error reporting function.
160 * Rpcalc Generate:: Running Bison on the grammar file.
161 * Rpcalc Compile:: Run the C compiler on the output code.
162
163 Grammar Rules for @code{rpcalc}
164
165 * Rpcalc Input::
166 * Rpcalc Line::
167 * Rpcalc Expr::
168
169 Location Tracking Calculator: @code{ltcalc}
170
171 * Ltcalc Declarations:: Bison and C declarations for ltcalc.
172 * Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
173 * Ltcalc Lexer:: The lexical analyzer.
174
175 Multi-Function Calculator: @code{mfcalc}
176
177 * Mfcalc Declarations:: Bison declarations for multi-function calculator.
178 * Mfcalc Rules:: Grammar rules for the calculator.
179 * Mfcalc Symbol Table:: Symbol table management subroutines.
180
181 Bison Grammar Files
182
183 * Grammar Outline:: Overall layout of the grammar file.
184 * Symbols:: Terminal and nonterminal symbols.
185 * Rules:: How to write grammar rules.
186 * Recursion:: Writing recursive rules.
187 * Semantics:: Semantic values and actions.
188 * Locations:: Locations and actions.
189 * Declarations:: All kinds of Bison declarations are described here.
190 * Multiple Parsers:: Putting more than one Bison parser in one program.
191
192 Outline of a Bison Grammar
193
194 * Prologue:: Syntax and usage of the prologue.
195 * Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
196 * Bison Declarations:: Syntax and usage of the Bison declarations section.
197 * Grammar Rules:: Syntax and usage of the grammar rules section.
198 * Epilogue:: Syntax and usage of the epilogue.
199
200 Defining Language Semantics
201
202 * Value Type:: Specifying one data type for all semantic values.
203 * Multiple Types:: Specifying several alternative data types.
204 * Actions:: An action is the semantic definition of a grammar rule.
205 * Action Types:: Specifying data types for actions to operate on.
206 * Mid-Rule Actions:: Most actions go at the end of a rule.
207 This says when, why and how to use the exceptional
208 action in the middle of a rule.
209
210 Tracking Locations
211
212 * Location Type:: Specifying a data type for locations.
213 * Actions and Locations:: Using locations in actions.
214 * Location Default Action:: Defining a general way to compute locations.
215
216 Bison Declarations
217
218 * Require Decl:: Requiring a Bison version.
219 * Token Decl:: Declaring terminal symbols.
220 * Precedence Decl:: Declaring terminals with precedence and associativity.
221 * Union Decl:: Declaring the set of all semantic value types.
222 * Type Decl:: Declaring the choice of type for a nonterminal symbol.
223 * Initial Action Decl:: Code run before parsing starts.
224 * Destructor Decl:: Declaring how symbols are freed.
225 * Expect Decl:: Suppressing warnings about parsing conflicts.
226 * Start Decl:: Specifying the start symbol.
227 * Pure Decl:: Requesting a reentrant parser.
228 * Push Decl:: Requesting a push parser.
229 * Decl Summary:: Table of all Bison declarations.
230
231 Parser C-Language Interface
232
233 * Parser Function:: How to call @code{yyparse} and what it returns.
234 * Push Parser Function:: How to call @code{yypush_parse} and what it returns.
235 * Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
236 * Parser Create Function:: How to call @code{yypstate_new} and what it returns.
237 * Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
238 * Lexical:: You must supply a function @code{yylex}
239 which reads tokens.
240 * Error Reporting:: You must supply a function @code{yyerror}.
241 * Action Features:: Special features for use in actions.
242 * Internationalization:: How to let the parser speak in the user's
243 native language.
244
245 The Lexical Analyzer Function @code{yylex}
246
247 * Calling Convention:: How @code{yyparse} calls @code{yylex}.
248 * Token Values:: How @code{yylex} must return the semantic value
249 of the token it has read.
250 * Token Locations:: How @code{yylex} must return the text location
251 (line number, etc.) of the token, if the
252 actions want that.
253 * Pure Calling:: How the calling convention differs in a pure parser
254 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
255
256 The Bison Parser Algorithm
257
258 * Lookahead:: Parser looks one token ahead when deciding what to do.
259 * Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
260 * Precedence:: Operator precedence works by resolving conflicts.
261 * Contextual Precedence:: When an operator's precedence depends on context.
262 * Parser States:: The parser is a finite-state-machine with stack.
263 * Reduce/Reduce:: When two rules are applicable in the same situation.
264 * Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
265 * Generalized LR Parsing:: Parsing arbitrary context-free grammars.
266 * Memory Management:: What happens when memory is exhausted. How to avoid it.
267
268 Operator Precedence
269
270 * Why Precedence:: An example showing why precedence is needed.
271 * Using Precedence:: How to specify precedence and associativity.
272 * Precedence Only:: How to specify precedence only.
273 * Precedence Examples:: How these features are used in the previous example.
274 * How Precedence:: How they work.
275
276 Handling Context Dependencies
277
278 * Semantic Tokens:: Token parsing can depend on the semantic context.
279 * Lexical Tie-ins:: Token parsing can depend on the syntactic context.
280 * Tie-in Recovery:: Lexical tie-ins have implications for how
281 error recovery rules must be written.
282
283 Debugging Your Parser
284
285 * Understanding:: Understanding the structure of your parser.
286 * Tracing:: Tracing the execution of your parser.
287
288 Invoking Bison
289
290 * Bison Options:: All the options described in detail,
291 in alphabetical order by short options.
292 * Option Cross Key:: Alphabetical list of long options.
293 * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
294
295 Parsers Written In Other Languages
296
297 * C++ Parsers:: The interface to generate C++ parser classes
298 * Java Parsers:: The interface to generate Java parser classes
299
300 C++ Parsers
301
302 * C++ Bison Interface:: Asking for C++ parser generation
303 * C++ Semantic Values:: %union vs. C++
304 * C++ Location Values:: The position and location classes
305 * C++ Parser Interface:: Instantiating and running the parser
306 * C++ Scanner Interface:: Exchanges between yylex and parse
307 * A Complete C++ Example:: Demonstrating their use
308
309 A Complete C++ Example
310
311 * Calc++ --- C++ Calculator:: The specifications
312 * Calc++ Parsing Driver:: An active parsing context
313 * Calc++ Parser:: A parser class
314 * Calc++ Scanner:: A pure C++ Flex scanner
315 * Calc++ Top Level:: Conducting the band
316
317 Java Parsers
318
319 * Java Bison Interface:: Asking for Java parser generation
320 * Java Semantic Values:: %type and %token vs. Java
321 * Java Location Values:: The position and location classes
322 * Java Parser Interface:: Instantiating and running the parser
323 * Java Scanner Interface:: Specifying the scanner for the parser
324 * Java Action Features:: Special features for use in actions
325 * Java Differences:: Differences between C/C++ and Java Grammars
326 * Java Declarations Summary:: List of Bison declarations used with Java
327
328 Frequently Asked Questions
329
330 * Memory Exhausted:: Breaking the Stack Limits
331 * How Can I Reset the Parser:: @code{yyparse} Keeps some State
332 * Strings are Destroyed:: @code{yylval} Loses Track of Strings
333 * Implementing Gotos/Loops:: Control Flow in the Calculator
334 * Multiple start-symbols:: Factoring closely related grammars
335 * Secure? Conform?:: Is Bison @acronym{POSIX} safe?
336 * I can't build Bison:: Troubleshooting
337 * Where can I find help?:: Troubleshouting
338 * Bug Reports:: Troublereporting
339 * More Languages:: Parsers in C++, Java, and so on
340 * Beta Testing:: Experimenting development versions
341 * Mailing Lists:: Meeting other Bison users
342
343 Copying This Manual
344
345 * Copying This Manual:: License for copying this manual.
346
347 @end detailmenu
348 @end menu
349
350 @node Introduction
351 @unnumbered Introduction
352 @cindex introduction
353
354 @dfn{Bison} is a general-purpose parser generator that converts an
355 annotated context-free grammar into a deterministic or @acronym{GLR}
356 parser employing @acronym{LALR}(1), @acronym{IELR}(1), or canonical
357 @acronym{LR}(1) parser tables.
358 Once you are proficient with Bison, you can use it to develop a wide
359 range of language parsers, from those used in simple desk calculators to
360 complex programming languages.
361
362 Bison is upward compatible with Yacc: all properly-written Yacc grammars
363 ought to work with Bison with no change. Anyone familiar with Yacc
364 should be able to use Bison with little trouble. You need to be fluent in
365 C or C++ programming in order to use Bison or to understand this manual.
366
367 We begin with tutorial chapters that explain the basic concepts of using
368 Bison and show three explained examples, each building on the last. If you
369 don't know Bison or Yacc, start by reading these chapters. Reference
370 chapters follow which describe specific aspects of Bison in detail.
371
372 Bison was written primarily by Robert Corbett; Richard Stallman made it
373 Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
374 multi-character string literals and other features.
375
376 This edition corresponds to version @value{VERSION} of Bison.
377
378 @node Conditions
379 @unnumbered Conditions for Using Bison
380
381 The distribution terms for Bison-generated parsers permit using the
382 parsers in nonfree programs. Before Bison version 2.2, these extra
383 permissions applied only when Bison was generating @acronym{LALR}(1)
384 parsers in C@. And before Bison version 1.24, Bison-generated
385 parsers could be used only in programs that were free software.
386
387 The other @acronym{GNU} programming tools, such as the @acronym{GNU} C
388 compiler, have never
389 had such a requirement. They could always be used for nonfree
390 software. The reason Bison was different was not due to a special
391 policy decision; it resulted from applying the usual General Public
392 License to all of the Bison source code.
393
394 The output of the Bison utility---the Bison parser file---contains a
395 verbatim copy of a sizable piece of Bison, which is the code for the
396 parser's implementation. (The actions from your grammar are inserted
397 into this implementation at one point, but most of the rest of the
398 implementation is not changed.) When we applied the @acronym{GPL}
399 terms to the skeleton code for the parser's implementation,
400 the effect was to restrict the use of Bison output to free software.
401
402 We didn't change the terms because of sympathy for people who want to
403 make software proprietary. @strong{Software should be free.} But we
404 concluded that limiting Bison's use to free software was doing little to
405 encourage people to make other software free. So we decided to make the
406 practical conditions for using Bison match the practical conditions for
407 using the other @acronym{GNU} tools.
408
409 This exception applies when Bison is generating code for a parser.
410 You can tell whether the exception applies to a Bison output file by
411 inspecting the file for text beginning with ``As a special
412 exception@dots{}''. The text spells out the exact terms of the
413 exception.
414
415 @node Copying
416 @unnumbered GNU GENERAL PUBLIC LICENSE
417 @include gpl-3.0.texi
418
419 @node Concepts
420 @chapter The Concepts of Bison
421
422 This chapter introduces many of the basic concepts without which the
423 details of Bison will not make sense. If you do not already know how to
424 use Bison or Yacc, we suggest you start by reading this chapter carefully.
425
426 @menu
427 * Language and Grammar:: Languages and context-free grammars,
428 as mathematical ideas.
429 * Grammar in Bison:: How we represent grammars for Bison's sake.
430 * Semantic Values:: Each token or syntactic grouping can have
431 a semantic value (the value of an integer,
432 the name of an identifier, etc.).
433 * Semantic Actions:: Each rule can have an action containing C code.
434 * GLR Parsers:: Writing parsers for general context-free languages.
435 * Locations Overview:: Tracking Locations.
436 * Bison Parser:: What are Bison's input and output,
437 how is the output used?
438 * Stages:: Stages in writing and running Bison grammars.
439 * Grammar Layout:: Overall structure of a Bison grammar file.
440 @end menu
441
442 @node Language and Grammar
443 @section Languages and Context-Free Grammars
444
445 @cindex context-free grammar
446 @cindex grammar, context-free
447 In order for Bison to parse a language, it must be described by a
448 @dfn{context-free grammar}. This means that you specify one or more
449 @dfn{syntactic groupings} and give rules for constructing them from their
450 parts. For example, in the C language, one kind of grouping is called an
451 `expression'. One rule for making an expression might be, ``An expression
452 can be made of a minus sign and another expression''. Another would be,
453 ``An expression can be an integer''. As you can see, rules are often
454 recursive, but there must be at least one rule which leads out of the
455 recursion.
456
457 @cindex @acronym{BNF}
458 @cindex Backus-Naur form
459 The most common formal system for presenting such rules for humans to read
460 is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in
461 order to specify the language Algol 60. Any grammar expressed in
462 @acronym{BNF} is a context-free grammar. The input to Bison is
463 essentially machine-readable @acronym{BNF}.
464
465 @cindex @acronym{LALR}(1) grammars
466 @cindex @acronym{IELR}(1) grammars
467 @cindex @acronym{LR}(1) grammars
468 There are various important subclasses of context-free grammars.
469 Although it can handle almost all context-free grammars, Bison is
470 optimized for what are called @acronym{LR}(1) grammars.
471 In brief, in these grammars, it must be possible to tell how to parse
472 any portion of an input string with just a single token of lookahead.
473 For historical reasons, Bison by default is limited by the additional
474 restrictions of @acronym{LALR}(1), which is hard to explain simply.
475 @xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
476 more information on this.
477 To escape these additional restrictions, you can request
478 @acronym{IELR}(1) or canonical @acronym{LR}(1) parser tables.
479 @xref{Decl Summary,,lr.type}, to learn how.
480
481 @cindex @acronym{GLR} parsing
482 @cindex generalized @acronym{LR} (@acronym{GLR}) parsing
483 @cindex ambiguous grammars
484 @cindex nondeterministic parsing
485
486 Parsers for @acronym{LR}(1) grammars are @dfn{deterministic}, meaning
487 roughly that the next grammar rule to apply at any point in the input is
488 uniquely determined by the preceding input and a fixed, finite portion
489 (called a @dfn{lookahead}) of the remaining input. A context-free
490 grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
491 apply the grammar rules to get the same inputs. Even unambiguous
492 grammars can be @dfn{nondeterministic}, meaning that no fixed
493 lookahead always suffices to determine the next grammar rule to apply.
494 With the proper declarations, Bison is also able to parse these more
495 general context-free grammars, using a technique known as @acronym{GLR}
496 parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers
497 are able to handle any context-free grammar for which the number of
498 possible parses of any given string is finite.
499
500 @cindex symbols (abstract)
501 @cindex token
502 @cindex syntactic grouping
503 @cindex grouping, syntactic
504 In the formal grammatical rules for a language, each kind of syntactic
505 unit or grouping is named by a @dfn{symbol}. Those which are built by
506 grouping smaller constructs according to grammatical rules are called
507 @dfn{nonterminal symbols}; those which can't be subdivided are called
508 @dfn{terminal symbols} or @dfn{token types}. We call a piece of input
509 corresponding to a single terminal symbol a @dfn{token}, and a piece
510 corresponding to a single nonterminal symbol a @dfn{grouping}.
511
512 We can use the C language as an example of what symbols, terminal and
513 nonterminal, mean. The tokens of C are identifiers, constants (numeric
514 and string), and the various keywords, arithmetic operators and
515 punctuation marks. So the terminal symbols of a grammar for C include
516 `identifier', `number', `string', plus one symbol for each keyword,
517 operator or punctuation mark: `if', `return', `const', `static', `int',
518 `char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
519 (These tokens can be subdivided into characters, but that is a matter of
520 lexicography, not grammar.)
521
522 Here is a simple C function subdivided into tokens:
523
524 @ifinfo
525 @example
526 int /* @r{keyword `int'} */
527 square (int x) /* @r{identifier, open-paren, keyword `int',}
528 @r{identifier, close-paren} */
529 @{ /* @r{open-brace} */
530 return x * x; /* @r{keyword `return', identifier, asterisk,}
531 @r{identifier, semicolon} */
532 @} /* @r{close-brace} */
533 @end example
534 @end ifinfo
535 @ifnotinfo
536 @example
537 int /* @r{keyword `int'} */
538 square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
539 @{ /* @r{open-brace} */
540 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
541 @} /* @r{close-brace} */
542 @end example
543 @end ifnotinfo
544
545 The syntactic groupings of C include the expression, the statement, the
546 declaration, and the function definition. These are represented in the
547 grammar of C by nonterminal symbols `expression', `statement',
548 `declaration' and `function definition'. The full grammar uses dozens of
549 additional language constructs, each with its own nonterminal symbol, in
550 order to express the meanings of these four. The example above is a
551 function definition; it contains one declaration, and one statement. In
552 the statement, each @samp{x} is an expression and so is @samp{x * x}.
553
554 Each nonterminal symbol must have grammatical rules showing how it is made
555 out of simpler constructs. For example, one kind of C statement is the
556 @code{return} statement; this would be described with a grammar rule which
557 reads informally as follows:
558
559 @quotation
560 A `statement' can be made of a `return' keyword, an `expression' and a
561 `semicolon'.
562 @end quotation
563
564 @noindent
565 There would be many other rules for `statement', one for each kind of
566 statement in C.
567
568 @cindex start symbol
569 One nonterminal symbol must be distinguished as the special one which
570 defines a complete utterance in the language. It is called the @dfn{start
571 symbol}. In a compiler, this means a complete input program. In the C
572 language, the nonterminal symbol `sequence of definitions and declarations'
573 plays this role.
574
575 For example, @samp{1 + 2} is a valid C expression---a valid part of a C
576 program---but it is not valid as an @emph{entire} C program. In the
577 context-free grammar of C, this follows from the fact that `expression' is
578 not the start symbol.
579
580 The Bison parser reads a sequence of tokens as its input, and groups the
581 tokens using the grammar rules. If the input is valid, the end result is
582 that the entire token sequence reduces to a single grouping whose symbol is
583 the grammar's start symbol. If we use a grammar for C, the entire input
584 must be a `sequence of definitions and declarations'. If not, the parser
585 reports a syntax error.
586
587 @node Grammar in Bison
588 @section From Formal Rules to Bison Input
589 @cindex Bison grammar
590 @cindex grammar, Bison
591 @cindex formal grammar
592
593 A formal grammar is a mathematical construct. To define the language
594 for Bison, you must write a file expressing the grammar in Bison syntax:
595 a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
596
597 A nonterminal symbol in the formal grammar is represented in Bison input
598 as an identifier, like an identifier in C@. By convention, it should be
599 in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
600
601 The Bison representation for a terminal symbol is also called a @dfn{token
602 type}. Token types as well can be represented as C-like identifiers. By
603 convention, these identifiers should be upper case to distinguish them from
604 nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
605 @code{RETURN}. A terminal symbol that stands for a particular keyword in
606 the language should be named after that keyword converted to upper case.
607 The terminal symbol @code{error} is reserved for error recovery.
608 @xref{Symbols}.
609
610 A terminal symbol can also be represented as a character literal, just like
611 a C character constant. You should do this whenever a token is just a
612 single character (parenthesis, plus-sign, etc.): use that same character in
613 a literal as the terminal symbol for that token.
614
615 A third way to represent a terminal symbol is with a C string constant
616 containing several characters. @xref{Symbols}, for more information.
617
618 The grammar rules also have an expression in Bison syntax. For example,
619 here is the Bison rule for a C @code{return} statement. The semicolon in
620 quotes is a literal character token, representing part of the C syntax for
621 the statement; the naked semicolon, and the colon, are Bison punctuation
622 used in every rule.
623
624 @example
625 stmt: RETURN expr ';'
626 ;
627 @end example
628
629 @noindent
630 @xref{Rules, ,Syntax of Grammar Rules}.
631
632 @node Semantic Values
633 @section Semantic Values
634 @cindex semantic value
635 @cindex value, semantic
636
637 A formal grammar selects tokens only by their classifications: for example,
638 if a rule mentions the terminal symbol `integer constant', it means that
639 @emph{any} integer constant is grammatically valid in that position. The
640 precise value of the constant is irrelevant to how to parse the input: if
641 @samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
642 grammatical.
643
644 But the precise value is very important for what the input means once it is
645 parsed. A compiler is useless if it fails to distinguish between 4, 1 and
646 3989 as constants in the program! Therefore, each token in a Bison grammar
647 has both a token type and a @dfn{semantic value}. @xref{Semantics,
648 ,Defining Language Semantics},
649 for details.
650
651 The token type is a terminal symbol defined in the grammar, such as
652 @code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
653 you need to know to decide where the token may validly appear and how to
654 group it with other tokens. The grammar rules know nothing about tokens
655 except their types.
656
657 The semantic value has all the rest of the information about the
658 meaning of the token, such as the value of an integer, or the name of an
659 identifier. (A token such as @code{','} which is just punctuation doesn't
660 need to have any semantic value.)
661
662 For example, an input token might be classified as token type
663 @code{INTEGER} and have the semantic value 4. Another input token might
664 have the same token type @code{INTEGER} but value 3989. When a grammar
665 rule says that @code{INTEGER} is allowed, either of these tokens is
666 acceptable because each is an @code{INTEGER}. When the parser accepts the
667 token, it keeps track of the token's semantic value.
668
669 Each grouping can also have a semantic value as well as its nonterminal
670 symbol. For example, in a calculator, an expression typically has a
671 semantic value that is a number. In a compiler for a programming
672 language, an expression typically has a semantic value that is a tree
673 structure describing the meaning of the expression.
674
675 @node Semantic Actions
676 @section Semantic Actions
677 @cindex semantic actions
678 @cindex actions, semantic
679
680 In order to be useful, a program must do more than parse input; it must
681 also produce some output based on the input. In a Bison grammar, a grammar
682 rule can have an @dfn{action} made up of C statements. Each time the
683 parser recognizes a match for that rule, the action is executed.
684 @xref{Actions}.
685
686 Most of the time, the purpose of an action is to compute the semantic value
687 of the whole construct from the semantic values of its parts. For example,
688 suppose we have a rule which says an expression can be the sum of two
689 expressions. When the parser recognizes such a sum, each of the
690 subexpressions has a semantic value which describes how it was built up.
691 The action for this rule should create a similar sort of value for the
692 newly recognized larger expression.
693
694 For example, here is a rule that says an expression can be the sum of
695 two subexpressions:
696
697 @example
698 expr: expr '+' expr @{ $$ = $1 + $3; @}
699 ;
700 @end example
701
702 @noindent
703 The action says how to produce the semantic value of the sum expression
704 from the values of the two subexpressions.
705
706 @node GLR Parsers
707 @section Writing @acronym{GLR} Parsers
708 @cindex @acronym{GLR} parsing
709 @cindex generalized @acronym{LR} (@acronym{GLR}) parsing
710 @findex %glr-parser
711 @cindex conflicts
712 @cindex shift/reduce conflicts
713 @cindex reduce/reduce conflicts
714
715 In some grammars, Bison's deterministic
716 @acronym{LR}(1) parsing algorithm cannot decide whether to apply a
717 certain grammar rule at a given point. That is, it may not be able to
718 decide (on the basis of the input read so far) which of two possible
719 reductions (applications of a grammar rule) applies, or whether to apply
720 a reduction or read more of the input and apply a reduction later in the
721 input. These are known respectively as @dfn{reduce/reduce} conflicts
722 (@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
723 (@pxref{Shift/Reduce}).
724
725 To use a grammar that is not easily modified to be @acronym{LR}(1), a
726 more general parsing algorithm is sometimes necessary. If you include
727 @code{%glr-parser} among the Bison declarations in your file
728 (@pxref{Grammar Outline}), the result is a Generalized @acronym{LR}
729 (@acronym{GLR}) parser. These parsers handle Bison grammars that
730 contain no unresolved conflicts (i.e., after applying precedence
731 declarations) identically to deterministic parsers. However, when
732 faced with unresolved shift/reduce and reduce/reduce conflicts,
733 @acronym{GLR} parsers use the simple expedient of doing both,
734 effectively cloning the parser to follow both possibilities. Each of
735 the resulting parsers can again split, so that at any given time, there
736 can be any number of possible parses being explored. The parsers
737 proceed in lockstep; that is, all of them consume (shift) a given input
738 symbol before any of them proceed to the next. Each of the cloned
739 parsers eventually meets one of two possible fates: either it runs into
740 a parsing error, in which case it simply vanishes, or it merges with
741 another parser, because the two of them have reduced the input to an
742 identical set of symbols.
743
744 During the time that there are multiple parsers, semantic actions are
745 recorded, but not performed. When a parser disappears, its recorded
746 semantic actions disappear as well, and are never performed. When a
747 reduction makes two parsers identical, causing them to merge, Bison
748 records both sets of semantic actions. Whenever the last two parsers
749 merge, reverting to the single-parser case, Bison resolves all the
750 outstanding actions either by precedences given to the grammar rules
751 involved, or by performing both actions, and then calling a designated
752 user-defined function on the resulting values to produce an arbitrary
753 merged result.
754
755 @menu
756 * Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
757 * Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
758 * GLR Semantic Actions:: Deferred semantic actions have special concerns.
759 * Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
760 @end menu
761
762 @node Simple GLR Parsers
763 @subsection Using @acronym{GLR} on Unambiguous Grammars
764 @cindex @acronym{GLR} parsing, unambiguous grammars
765 @cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars
766 @findex %glr-parser
767 @findex %expect-rr
768 @cindex conflicts
769 @cindex reduce/reduce conflicts
770 @cindex shift/reduce conflicts
771
772 In the simplest cases, you can use the @acronym{GLR} algorithm
773 to parse grammars that are unambiguous but fail to be @acronym{LR}(1).
774 Such grammars typically require more than one symbol of lookahead.
775
776 Consider a problem that
777 arises in the declaration of enumerated and subrange types in the
778 programming language Pascal. Here are some examples:
779
780 @example
781 type subrange = lo .. hi;
782 type enum = (a, b, c);
783 @end example
784
785 @noindent
786 The original language standard allows only numeric
787 literals and constant identifiers for the subrange bounds (@samp{lo}
788 and @samp{hi}), but Extended Pascal (@acronym{ISO}/@acronym{IEC}
789 10206) and many other
790 Pascal implementations allow arbitrary expressions there. This gives
791 rise to the following situation, containing a superfluous pair of
792 parentheses:
793
794 @example
795 type subrange = (a) .. b;
796 @end example
797
798 @noindent
799 Compare this to the following declaration of an enumerated
800 type with only one value:
801
802 @example
803 type enum = (a);
804 @end example
805
806 @noindent
807 (These declarations are contrived, but they are syntactically
808 valid, and more-complicated cases can come up in practical programs.)
809
810 These two declarations look identical until the @samp{..} token.
811 With normal @acronym{LR}(1) one-token lookahead it is not
812 possible to decide between the two forms when the identifier
813 @samp{a} is parsed. It is, however, desirable
814 for a parser to decide this, since in the latter case
815 @samp{a} must become a new identifier to represent the enumeration
816 value, while in the former case @samp{a} must be evaluated with its
817 current meaning, which may be a constant or even a function call.
818
819 You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
820 to be resolved later, but this typically requires substantial
821 contortions in both semantic actions and large parts of the
822 grammar, where the parentheses are nested in the recursive rules for
823 expressions.
824
825 You might think of using the lexer to distinguish between the two
826 forms by returning different tokens for currently defined and
827 undefined identifiers. But if these declarations occur in a local
828 scope, and @samp{a} is defined in an outer scope, then both forms
829 are possible---either locally redefining @samp{a}, or using the
830 value of @samp{a} from the outer scope. So this approach cannot
831 work.
832
833 A simple solution to this problem is to declare the parser to
834 use the @acronym{GLR} algorithm.
835 When the @acronym{GLR} parser reaches the critical state, it
836 merely splits into two branches and pursues both syntax rules
837 simultaneously. Sooner or later, one of them runs into a parsing
838 error. If there is a @samp{..} token before the next
839 @samp{;}, the rule for enumerated types fails since it cannot
840 accept @samp{..} anywhere; otherwise, the subrange type rule
841 fails since it requires a @samp{..} token. So one of the branches
842 fails silently, and the other one continues normally, performing
843 all the intermediate actions that were postponed during the split.
844
845 If the input is syntactically incorrect, both branches fail and the parser
846 reports a syntax error as usual.
847
848 The effect of all this is that the parser seems to ``guess'' the
849 correct branch to take, or in other words, it seems to use more
850 lookahead than the underlying @acronym{LR}(1) algorithm actually allows
851 for. In this example, @acronym{LR}(2) would suffice, but also some cases
852 that are not @acronym{LR}(@math{k}) for any @math{k} can be handled this way.
853
854 In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time,
855 and the current Bison parser even takes exponential time and space
856 for some grammars. In practice, this rarely happens, and for many
857 grammars it is possible to prove that it cannot happen.
858 The present example contains only one conflict between two
859 rules, and the type-declaration context containing the conflict
860 cannot be nested. So the number of
861 branches that can exist at any time is limited by the constant 2,
862 and the parsing time is still linear.
863
864 Here is a Bison grammar corresponding to the example above. It
865 parses a vastly simplified form of Pascal type declarations.
866
867 @example
868 %token TYPE DOTDOT ID
869
870 @group
871 %left '+' '-'
872 %left '*' '/'
873 @end group
874
875 %%
876
877 @group
878 type_decl : TYPE ID '=' type ';'
879 ;
880 @end group
881
882 @group
883 type : '(' id_list ')'
884 | expr DOTDOT expr
885 ;
886 @end group
887
888 @group
889 id_list : ID
890 | id_list ',' ID
891 ;
892 @end group
893
894 @group
895 expr : '(' expr ')'
896 | expr '+' expr
897 | expr '-' expr
898 | expr '*' expr
899 | expr '/' expr
900 | ID
901 ;
902 @end group
903 @end example
904
905 When used as a normal @acronym{LR}(1) grammar, Bison correctly complains
906 about one reduce/reduce conflict. In the conflicting situation the
907 parser chooses one of the alternatives, arbitrarily the one
908 declared first. Therefore the following correct input is not
909 recognized:
910
911 @example
912 type t = (a) .. b;
913 @end example
914
915 The parser can be turned into a @acronym{GLR} parser, while also telling Bison
916 to be silent about the one known reduce/reduce conflict, by
917 adding these two declarations to the Bison input file (before the first
918 @samp{%%}):
919
920 @example
921 %glr-parser
922 %expect-rr 1
923 @end example
924
925 @noindent
926 No change in the grammar itself is required. Now the
927 parser recognizes all valid declarations, according to the
928 limited syntax above, transparently. In fact, the user does not even
929 notice when the parser splits.
930
931 So here we have a case where we can use the benefits of @acronym{GLR},
932 almost without disadvantages. Even in simple cases like this, however,
933 there are at least two potential problems to beware. First, always
934 analyze the conflicts reported by Bison to make sure that @acronym{GLR}
935 splitting is only done where it is intended. A @acronym{GLR} parser
936 splitting inadvertently may cause problems less obvious than an
937 @acronym{LR} parser statically choosing the wrong alternative in a
938 conflict. Second, consider interactions with the lexer (@pxref{Semantic
939 Tokens}) with great care. Since a split parser consumes tokens without
940 performing any actions during the split, the lexer cannot obtain
941 information via parser actions. Some cases of lexer interactions can be
942 eliminated by using @acronym{GLR} to shift the complications from the
943 lexer to the parser. You must check the remaining cases for
944 correctness.
945
946 In our example, it would be safe for the lexer to return tokens based on
947 their current meanings in some symbol table, because no new symbols are
948 defined in the middle of a type declaration. Though it is possible for
949 a parser to define the enumeration constants as they are parsed, before
950 the type declaration is completed, it actually makes no difference since
951 they cannot be used within the same enumerated type declaration.
952
953 @node Merging GLR Parses
954 @subsection Using @acronym{GLR} to Resolve Ambiguities
955 @cindex @acronym{GLR} parsing, ambiguous grammars
956 @cindex generalized @acronym{LR} (@acronym{GLR}) parsing, ambiguous grammars
957 @findex %dprec
958 @findex %merge
959 @cindex conflicts
960 @cindex reduce/reduce conflicts
961
962 Let's consider an example, vastly simplified from a C++ grammar.
963
964 @example
965 %@{
966 #include <stdio.h>
967 #define YYSTYPE char const *
968 int yylex (void);
969 void yyerror (char const *);
970 %@}
971
972 %token TYPENAME ID
973
974 %right '='
975 %left '+'
976
977 %glr-parser
978
979 %%
980
981 prog :
982 | prog stmt @{ printf ("\n"); @}
983 ;
984
985 stmt : expr ';' %dprec 1
986 | decl %dprec 2
987 ;
988
989 expr : ID @{ printf ("%s ", $$); @}
990 | TYPENAME '(' expr ')'
991 @{ printf ("%s <cast> ", $1); @}
992 | expr '+' expr @{ printf ("+ "); @}
993 | expr '=' expr @{ printf ("= "); @}
994 ;
995
996 decl : TYPENAME declarator ';'
997 @{ printf ("%s <declare> ", $1); @}
998 | TYPENAME declarator '=' expr ';'
999 @{ printf ("%s <init-declare> ", $1); @}
1000 ;
1001
1002 declarator : ID @{ printf ("\"%s\" ", $1); @}
1003 | '(' declarator ')'
1004 ;
1005 @end example
1006
1007 @noindent
1008 This models a problematic part of the C++ grammar---the ambiguity between
1009 certain declarations and statements. For example,
1010
1011 @example
1012 T (x) = y+z;
1013 @end example
1014
1015 @noindent
1016 parses as either an @code{expr} or a @code{stmt}
1017 (assuming that @samp{T} is recognized as a @code{TYPENAME} and
1018 @samp{x} as an @code{ID}).
1019 Bison detects this as a reduce/reduce conflict between the rules
1020 @code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
1021 time it encounters @code{x} in the example above. Since this is a
1022 @acronym{GLR} parser, it therefore splits the problem into two parses, one for
1023 each choice of resolving the reduce/reduce conflict.
1024 Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1025 however, neither of these parses ``dies,'' because the grammar as it stands is
1026 ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1027 the other reduces @code{stmt : decl}, after which both parsers are in an
1028 identical state: they've seen @samp{prog stmt} and have the same unprocessed
1029 input remaining. We say that these parses have @dfn{merged.}
1030
1031 At this point, the @acronym{GLR} parser requires a specification in the
1032 grammar of how to choose between the competing parses.
1033 In the example above, the two @code{%dprec}
1034 declarations specify that Bison is to give precedence
1035 to the parse that interprets the example as a
1036 @code{decl}, which implies that @code{x} is a declarator.
1037 The parser therefore prints
1038
1039 @example
1040 "x" y z + T <init-declare>
1041 @end example
1042
1043 The @code{%dprec} declarations only come into play when more than one
1044 parse survives. Consider a different input string for this parser:
1045
1046 @example
1047 T (x) + y;
1048 @end example
1049
1050 @noindent
1051 This is another example of using @acronym{GLR} to parse an unambiguous
1052 construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
1053 Here, there is no ambiguity (this cannot be parsed as a declaration).
1054 However, at the time the Bison parser encounters @code{x}, it does not
1055 have enough information to resolve the reduce/reduce conflict (again,
1056 between @code{x} as an @code{expr} or a @code{declarator}). In this
1057 case, no precedence declaration is used. Again, the parser splits
1058 into two, one assuming that @code{x} is an @code{expr}, and the other
1059 assuming @code{x} is a @code{declarator}. The second of these parsers
1060 then vanishes when it sees @code{+}, and the parser prints
1061
1062 @example
1063 x T <cast> y +
1064 @end example
1065
1066 Suppose that instead of resolving the ambiguity, you wanted to see all
1067 the possibilities. For this purpose, you must merge the semantic
1068 actions of the two possible parsers, rather than choosing one over the
1069 other. To do so, you could change the declaration of @code{stmt} as
1070 follows:
1071
1072 @example
1073 stmt : expr ';' %merge <stmtMerge>
1074 | decl %merge <stmtMerge>
1075 ;
1076 @end example
1077
1078 @noindent
1079 and define the @code{stmtMerge} function as:
1080
1081 @example
1082 static YYSTYPE
1083 stmtMerge (YYSTYPE x0, YYSTYPE x1)
1084 @{
1085 printf ("<OR> ");
1086 return "";
1087 @}
1088 @end example
1089
1090 @noindent
1091 with an accompanying forward declaration
1092 in the C declarations at the beginning of the file:
1093
1094 @example
1095 %@{
1096 #define YYSTYPE char const *
1097 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1098 %@}
1099 @end example
1100
1101 @noindent
1102 With these declarations, the resulting parser parses the first example
1103 as both an @code{expr} and a @code{decl}, and prints
1104
1105 @example
1106 "x" y z + T <init-declare> x T <cast> y z + = <OR>
1107 @end example
1108
1109 Bison requires that all of the
1110 productions that participate in any particular merge have identical
1111 @samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1112 and the parser will report an error during any parse that results in
1113 the offending merge.
1114
1115 @node GLR Semantic Actions
1116 @subsection GLR Semantic Actions
1117
1118 @cindex deferred semantic actions
1119 By definition, a deferred semantic action is not performed at the same time as
1120 the associated reduction.
1121 This raises caveats for several Bison features you might use in a semantic
1122 action in a @acronym{GLR} parser.
1123
1124 @vindex yychar
1125 @cindex @acronym{GLR} parsers and @code{yychar}
1126 @vindex yylval
1127 @cindex @acronym{GLR} parsers and @code{yylval}
1128 @vindex yylloc
1129 @cindex @acronym{GLR} parsers and @code{yylloc}
1130 In any semantic action, you can examine @code{yychar} to determine the type of
1131 the lookahead token present at the time of the associated reduction.
1132 After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1133 you can then examine @code{yylval} and @code{yylloc} to determine the
1134 lookahead token's semantic value and location, if any.
1135 In a nondeferred semantic action, you can also modify any of these variables to
1136 influence syntax analysis.
1137 @xref{Lookahead, ,Lookahead Tokens}.
1138
1139 @findex yyclearin
1140 @cindex @acronym{GLR} parsers and @code{yyclearin}
1141 In a deferred semantic action, it's too late to influence syntax analysis.
1142 In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1143 shallow copies of the values they had at the time of the associated reduction.
1144 For this reason alone, modifying them is dangerous.
1145 Moreover, the result of modifying them is undefined and subject to change with
1146 future versions of Bison.
1147 For example, if a semantic action might be deferred, you should never write it
1148 to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1149 memory referenced by @code{yylval}.
1150
1151 @findex YYERROR
1152 @cindex @acronym{GLR} parsers and @code{YYERROR}
1153 Another Bison feature requiring special consideration is @code{YYERROR}
1154 (@pxref{Action Features}), which you can invoke in a semantic action to
1155 initiate error recovery.
1156 During deterministic @acronym{GLR} operation, the effect of @code{YYERROR} is
1157 the same as its effect in a deterministic parser.
1158 In a deferred semantic action, its effect is undefined.
1159 @c The effect is probably a syntax error at the split point.
1160
1161 Also, see @ref{Location Default Action, ,Default Action for Locations}, which
1162 describes a special usage of @code{YYLLOC_DEFAULT} in @acronym{GLR} parsers.
1163
1164 @node Compiler Requirements
1165 @subsection Considerations when Compiling @acronym{GLR} Parsers
1166 @cindex @code{inline}
1167 @cindex @acronym{GLR} parsers and @code{inline}
1168
1169 The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or
1170 later. In addition, they use the @code{inline} keyword, which is not
1171 C89, but is C99 and is a common extension in pre-C99 compilers. It is
1172 up to the user of these parsers to handle
1173 portability issues. For instance, if using Autoconf and the Autoconf
1174 macro @code{AC_C_INLINE}, a mere
1175
1176 @example
1177 %@{
1178 #include <config.h>
1179 %@}
1180 @end example
1181
1182 @noindent
1183 will suffice. Otherwise, we suggest
1184
1185 @example
1186 %@{
1187 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1188 #define inline
1189 #endif
1190 %@}
1191 @end example
1192
1193 @node Locations Overview
1194 @section Locations
1195 @cindex location
1196 @cindex textual location
1197 @cindex location, textual
1198
1199 Many applications, like interpreters or compilers, have to produce verbose
1200 and useful error messages. To achieve this, one must be able to keep track of
1201 the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
1202 Bison provides a mechanism for handling these locations.
1203
1204 Each token has a semantic value. In a similar fashion, each token has an
1205 associated location, but the type of locations is the same for all tokens and
1206 groupings. Moreover, the output parser is equipped with a default data
1207 structure for storing locations (@pxref{Locations}, for more details).
1208
1209 Like semantic values, locations can be reached in actions using a dedicated
1210 set of constructs. In the example above, the location of the whole grouping
1211 is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1212 @code{@@3}.
1213
1214 When a rule is matched, a default action is used to compute the semantic value
1215 of its left hand side (@pxref{Actions}). In the same way, another default
1216 action is used for locations. However, the action for locations is general
1217 enough for most cases, meaning there is usually no need to describe for each
1218 rule how @code{@@$} should be formed. When building a new location for a given
1219 grouping, the default behavior of the output parser is to take the beginning
1220 of the first symbol, and the end of the last symbol.
1221
1222 @node Bison Parser
1223 @section Bison Output: the Parser File
1224 @cindex Bison parser
1225 @cindex Bison utility
1226 @cindex lexical analyzer, purpose
1227 @cindex parser
1228
1229 When you run Bison, you give it a Bison grammar file as input. The output
1230 is a C source file that parses the language described by the grammar.
1231 This file is called a @dfn{Bison parser}. Keep in mind that the Bison
1232 utility and the Bison parser are two distinct programs: the Bison utility
1233 is a program whose output is the Bison parser that becomes part of your
1234 program.
1235
1236 The job of the Bison parser is to group tokens into groupings according to
1237 the grammar rules---for example, to build identifiers and operators into
1238 expressions. As it does this, it runs the actions for the grammar rules it
1239 uses.
1240
1241 The tokens come from a function called the @dfn{lexical analyzer} that
1242 you must supply in some fashion (such as by writing it in C). The Bison
1243 parser calls the lexical analyzer each time it wants a new token. It
1244 doesn't know what is ``inside'' the tokens (though their semantic values
1245 may reflect this). Typically the lexical analyzer makes the tokens by
1246 parsing characters of text, but Bison does not depend on this.
1247 @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
1248
1249 The Bison parser file is C code which defines a function named
1250 @code{yyparse} which implements that grammar. This function does not make
1251 a complete C program: you must supply some additional functions. One is
1252 the lexical analyzer. Another is an error-reporting function which the
1253 parser calls to report an error. In addition, a complete C program must
1254 start with a function called @code{main}; you have to provide this, and
1255 arrange for it to call @code{yyparse} or the parser will never run.
1256 @xref{Interface, ,Parser C-Language Interface}.
1257
1258 Aside from the token type names and the symbols in the actions you
1259 write, all symbols defined in the Bison parser file itself
1260 begin with @samp{yy} or @samp{YY}. This includes interface functions
1261 such as the lexical analyzer function @code{yylex}, the error reporting
1262 function @code{yyerror} and the parser function @code{yyparse} itself.
1263 This also includes numerous identifiers used for internal purposes.
1264 Therefore, you should avoid using C identifiers starting with @samp{yy}
1265 or @samp{YY} in the Bison grammar file except for the ones defined in
1266 this manual. Also, you should avoid using the C identifiers
1267 @samp{malloc} and @samp{free} for anything other than their usual
1268 meanings.
1269
1270 In some cases the Bison parser file includes system headers, and in
1271 those cases your code should respect the identifiers reserved by those
1272 headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>}, @code{<malloc.h>},
1273 @code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
1274 declare memory allocators and related types. @code{<libintl.h>} is
1275 included if message translation is in use
1276 (@pxref{Internationalization}). Other system headers may
1277 be included if you define @code{YYDEBUG} to a nonzero value
1278 (@pxref{Tracing, ,Tracing Your Parser}).
1279
1280 @node Stages
1281 @section Stages in Using Bison
1282 @cindex stages in using Bison
1283 @cindex using Bison
1284
1285 The actual language-design process using Bison, from grammar specification
1286 to a working compiler or interpreter, has these parts:
1287
1288 @enumerate
1289 @item
1290 Formally specify the grammar in a form recognized by Bison
1291 (@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1292 in the language, describe the action that is to be taken when an
1293 instance of that rule is recognized. The action is described by a
1294 sequence of C statements.
1295
1296 @item
1297 Write a lexical analyzer to process input and pass tokens to the parser.
1298 The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1299 Lexical Analyzer Function @code{yylex}}). It could also be produced
1300 using Lex, but the use of Lex is not discussed in this manual.
1301
1302 @item
1303 Write a controlling function that calls the Bison-produced parser.
1304
1305 @item
1306 Write error-reporting routines.
1307 @end enumerate
1308
1309 To turn this source code as written into a runnable program, you
1310 must follow these steps:
1311
1312 @enumerate
1313 @item
1314 Run Bison on the grammar to produce the parser.
1315
1316 @item
1317 Compile the code output by Bison, as well as any other source files.
1318
1319 @item
1320 Link the object files to produce the finished product.
1321 @end enumerate
1322
1323 @node Grammar Layout
1324 @section The Overall Layout of a Bison Grammar
1325 @cindex grammar file
1326 @cindex file format
1327 @cindex format of grammar file
1328 @cindex layout of Bison grammar
1329
1330 The input file for the Bison utility is a @dfn{Bison grammar file}. The
1331 general form of a Bison grammar file is as follows:
1332
1333 @example
1334 %@{
1335 @var{Prologue}
1336 %@}
1337
1338 @var{Bison declarations}
1339
1340 %%
1341 @var{Grammar rules}
1342 %%
1343 @var{Epilogue}
1344 @end example
1345
1346 @noindent
1347 The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1348 in every Bison grammar file to separate the sections.
1349
1350 The prologue may define types and variables used in the actions. You can
1351 also use preprocessor commands to define macros used there, and use
1352 @code{#include} to include header files that do any of these things.
1353 You need to declare the lexical analyzer @code{yylex} and the error
1354 printer @code{yyerror} here, along with any other global identifiers
1355 used by the actions in the grammar rules.
1356
1357 The Bison declarations declare the names of the terminal and nonterminal
1358 symbols, and may also describe operator precedence and the data types of
1359 semantic values of various symbols.
1360
1361 The grammar rules define how to construct each nonterminal symbol from its
1362 parts.
1363
1364 The epilogue can contain any code you want to use. Often the
1365 definitions of functions declared in the prologue go here. In a
1366 simple program, all the rest of the program can go here.
1367
1368 @node Examples
1369 @chapter Examples
1370 @cindex simple examples
1371 @cindex examples, simple
1372
1373 Now we show and explain three sample programs written using Bison: a
1374 reverse polish notation calculator, an algebraic (infix) notation
1375 calculator, and a multi-function calculator. All three have been tested
1376 under BSD Unix 4.3; each produces a usable, though limited, interactive
1377 desk-top calculator.
1378
1379 These examples are simple, but Bison grammars for real programming
1380 languages are written the same way. You can copy these examples into a
1381 source file to try them.
1382
1383 @menu
1384 * RPN Calc:: Reverse polish notation calculator;
1385 a first example with no operator precedence.
1386 * Infix Calc:: Infix (algebraic) notation calculator.
1387 Operator precedence is introduced.
1388 * Simple Error Recovery:: Continuing after syntax errors.
1389 * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
1390 * Multi-function Calc:: Calculator with memory and trig functions.
1391 It uses multiple data-types for semantic values.
1392 * Exercises:: Ideas for improving the multi-function calculator.
1393 @end menu
1394
1395 @node RPN Calc
1396 @section Reverse Polish Notation Calculator
1397 @cindex reverse polish notation
1398 @cindex polish notation calculator
1399 @cindex @code{rpcalc}
1400 @cindex calculator, simple
1401
1402 The first example is that of a simple double-precision @dfn{reverse polish
1403 notation} calculator (a calculator using postfix operators). This example
1404 provides a good starting point, since operator precedence is not an issue.
1405 The second example will illustrate how operator precedence is handled.
1406
1407 The source code for this calculator is named @file{rpcalc.y}. The
1408 @samp{.y} extension is a convention used for Bison input files.
1409
1410 @menu
1411 * Rpcalc Declarations:: Prologue (declarations) for rpcalc.
1412 * Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
1413 * Rpcalc Lexer:: The lexical analyzer.
1414 * Rpcalc Main:: The controlling function.
1415 * Rpcalc Error:: The error reporting function.
1416 * Rpcalc Generate:: Running Bison on the grammar file.
1417 * Rpcalc Compile:: Run the C compiler on the output code.
1418 @end menu
1419
1420 @node Rpcalc Declarations
1421 @subsection Declarations for @code{rpcalc}
1422
1423 Here are the C and Bison declarations for the reverse polish notation
1424 calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1425
1426 @example
1427 /* Reverse polish notation calculator. */
1428
1429 %@{
1430 #define YYSTYPE double
1431 #include <math.h>
1432 int yylex (void);
1433 void yyerror (char const *);
1434 %@}
1435
1436 %token NUM
1437
1438 %% /* Grammar rules and actions follow. */
1439 @end example
1440
1441 The declarations section (@pxref{Prologue, , The prologue}) contains two
1442 preprocessor directives and two forward declarations.
1443
1444 The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1445 specifying the C data type for semantic values of both tokens and
1446 groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1447 Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1448 don't define it, @code{int} is the default. Because we specify
1449 @code{double}, each token and each expression has an associated value,
1450 which is a floating point number.
1451
1452 The @code{#include} directive is used to declare the exponentiation
1453 function @code{pow}.
1454
1455 The forward declarations for @code{yylex} and @code{yyerror} are
1456 needed because the C language requires that functions be declared
1457 before they are used. These functions will be defined in the
1458 epilogue, but the parser calls them so they must be declared in the
1459 prologue.
1460
1461 The second section, Bison declarations, provides information to Bison
1462 about the token types (@pxref{Bison Declarations, ,The Bison
1463 Declarations Section}). Each terminal symbol that is not a
1464 single-character literal must be declared here. (Single-character
1465 literals normally don't need to be declared.) In this example, all the
1466 arithmetic operators are designated by single-character literals, so the
1467 only terminal symbol that needs to be declared is @code{NUM}, the token
1468 type for numeric constants.
1469
1470 @node Rpcalc Rules
1471 @subsection Grammar Rules for @code{rpcalc}
1472
1473 Here are the grammar rules for the reverse polish notation calculator.
1474
1475 @example
1476 input: /* empty */
1477 | input line
1478 ;
1479
1480 line: '\n'
1481 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1482 ;
1483
1484 exp: NUM @{ $$ = $1; @}
1485 | exp exp '+' @{ $$ = $1 + $2; @}
1486 | exp exp '-' @{ $$ = $1 - $2; @}
1487 | exp exp '*' @{ $$ = $1 * $2; @}
1488 | exp exp '/' @{ $$ = $1 / $2; @}
1489 /* Exponentiation */
1490 | exp exp '^' @{ $$ = pow ($1, $2); @}
1491 /* Unary minus */
1492 | exp 'n' @{ $$ = -$1; @}
1493 ;
1494 %%
1495 @end example
1496
1497 The groupings of the rpcalc ``language'' defined here are the expression
1498 (given the name @code{exp}), the line of input (@code{line}), and the
1499 complete input transcript (@code{input}). Each of these nonterminal
1500 symbols has several alternate rules, joined by the vertical bar @samp{|}
1501 which is read as ``or''. The following sections explain what these rules
1502 mean.
1503
1504 The semantics of the language is determined by the actions taken when a
1505 grouping is recognized. The actions are the C code that appears inside
1506 braces. @xref{Actions}.
1507
1508 You must specify these actions in C, but Bison provides the means for
1509 passing semantic values between the rules. In each action, the
1510 pseudo-variable @code{$$} stands for the semantic value for the grouping
1511 that the rule is going to construct. Assigning a value to @code{$$} is the
1512 main job of most actions. The semantic values of the components of the
1513 rule are referred to as @code{$1}, @code{$2}, and so on.
1514
1515 @menu
1516 * Rpcalc Input::
1517 * Rpcalc Line::
1518 * Rpcalc Expr::
1519 @end menu
1520
1521 @node Rpcalc Input
1522 @subsubsection Explanation of @code{input}
1523
1524 Consider the definition of @code{input}:
1525
1526 @example
1527 input: /* empty */
1528 | input line
1529 ;
1530 @end example
1531
1532 This definition reads as follows: ``A complete input is either an empty
1533 string, or a complete input followed by an input line''. Notice that
1534 ``complete input'' is defined in terms of itself. This definition is said
1535 to be @dfn{left recursive} since @code{input} appears always as the
1536 leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1537
1538 The first alternative is empty because there are no symbols between the
1539 colon and the first @samp{|}; this means that @code{input} can match an
1540 empty string of input (no tokens). We write the rules this way because it
1541 is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1542 It's conventional to put an empty alternative first and write the comment
1543 @samp{/* empty */} in it.
1544
1545 The second alternate rule (@code{input line}) handles all nontrivial input.
1546 It means, ``After reading any number of lines, read one more line if
1547 possible.'' The left recursion makes this rule into a loop. Since the
1548 first alternative matches empty input, the loop can be executed zero or
1549 more times.
1550
1551 The parser function @code{yyparse} continues to process input until a
1552 grammatical error is seen or the lexical analyzer says there are no more
1553 input tokens; we will arrange for the latter to happen at end-of-input.
1554
1555 @node Rpcalc Line
1556 @subsubsection Explanation of @code{line}
1557
1558 Now consider the definition of @code{line}:
1559
1560 @example
1561 line: '\n'
1562 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1563 ;
1564 @end example
1565
1566 The first alternative is a token which is a newline character; this means
1567 that rpcalc accepts a blank line (and ignores it, since there is no
1568 action). The second alternative is an expression followed by a newline.
1569 This is the alternative that makes rpcalc useful. The semantic value of
1570 the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1571 question is the first symbol in the alternative. The action prints this
1572 value, which is the result of the computation the user asked for.
1573
1574 This action is unusual because it does not assign a value to @code{$$}. As
1575 a consequence, the semantic value associated with the @code{line} is
1576 uninitialized (its value will be unpredictable). This would be a bug if
1577 that value were ever used, but we don't use it: once rpcalc has printed the
1578 value of the user's input line, that value is no longer needed.
1579
1580 @node Rpcalc Expr
1581 @subsubsection Explanation of @code{expr}
1582
1583 The @code{exp} grouping has several rules, one for each kind of expression.
1584 The first rule handles the simplest expressions: those that are just numbers.
1585 The second handles an addition-expression, which looks like two expressions
1586 followed by a plus-sign. The third handles subtraction, and so on.
1587
1588 @example
1589 exp: NUM
1590 | exp exp '+' @{ $$ = $1 + $2; @}
1591 | exp exp '-' @{ $$ = $1 - $2; @}
1592 @dots{}
1593 ;
1594 @end example
1595
1596 We have used @samp{|} to join all the rules for @code{exp}, but we could
1597 equally well have written them separately:
1598
1599 @example
1600 exp: NUM ;
1601 exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1602 exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1603 @dots{}
1604 @end example
1605
1606 Most of the rules have actions that compute the value of the expression in
1607 terms of the value of its parts. For example, in the rule for addition,
1608 @code{$1} refers to the first component @code{exp} and @code{$2} refers to
1609 the second one. The third component, @code{'+'}, has no meaningful
1610 associated semantic value, but if it had one you could refer to it as
1611 @code{$3}. When @code{yyparse} recognizes a sum expression using this
1612 rule, the sum of the two subexpressions' values is produced as the value of
1613 the entire expression. @xref{Actions}.
1614
1615 You don't have to give an action for every rule. When a rule has no
1616 action, Bison by default copies the value of @code{$1} into @code{$$}.
1617 This is what happens in the first rule (the one that uses @code{NUM}).
1618
1619 The formatting shown here is the recommended convention, but Bison does
1620 not require it. You can add or change white space as much as you wish.
1621 For example, this:
1622
1623 @example
1624 exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
1625 @end example
1626
1627 @noindent
1628 means the same thing as this:
1629
1630 @example
1631 exp: NUM
1632 | exp exp '+' @{ $$ = $1 + $2; @}
1633 | @dots{}
1634 ;
1635 @end example
1636
1637 @noindent
1638 The latter, however, is much more readable.
1639
1640 @node Rpcalc Lexer
1641 @subsection The @code{rpcalc} Lexical Analyzer
1642 @cindex writing a lexical analyzer
1643 @cindex lexical analyzer, writing
1644
1645 The lexical analyzer's job is low-level parsing: converting characters
1646 or sequences of characters into tokens. The Bison parser gets its
1647 tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1648 Analyzer Function @code{yylex}}.
1649
1650 Only a simple lexical analyzer is needed for the @acronym{RPN}
1651 calculator. This
1652 lexical analyzer skips blanks and tabs, then reads in numbers as
1653 @code{double} and returns them as @code{NUM} tokens. Any other character
1654 that isn't part of a number is a separate token. Note that the token-code
1655 for such a single-character token is the character itself.
1656
1657 The return value of the lexical analyzer function is a numeric code which
1658 represents a token type. The same text used in Bison rules to stand for
1659 this token type is also a C expression for the numeric code for the type.
1660 This works in two ways. If the token type is a character literal, then its
1661 numeric code is that of the character; you can use the same
1662 character literal in the lexical analyzer to express the number. If the
1663 token type is an identifier, that identifier is defined by Bison as a C
1664 macro whose definition is the appropriate number. In this example,
1665 therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1666
1667 The semantic value of the token (if it has one) is stored into the
1668 global variable @code{yylval}, which is where the Bison parser will look
1669 for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1670 defined at the beginning of the grammar; @pxref{Rpcalc Declarations,
1671 ,Declarations for @code{rpcalc}}.)
1672
1673 A token type code of zero is returned if the end-of-input is encountered.
1674 (Bison recognizes any nonpositive value as indicating end-of-input.)
1675
1676 Here is the code for the lexical analyzer:
1677
1678 @example
1679 @group
1680 /* The lexical analyzer returns a double floating point
1681 number on the stack and the token NUM, or the numeric code
1682 of the character read if not a number. It skips all blanks
1683 and tabs, and returns 0 for end-of-input. */
1684
1685 #include <ctype.h>
1686 @end group
1687
1688 @group
1689 int
1690 yylex (void)
1691 @{
1692 int c;
1693
1694 /* Skip white space. */
1695 while ((c = getchar ()) == ' ' || c == '\t')
1696 ;
1697 @end group
1698 @group
1699 /* Process numbers. */
1700 if (c == '.' || isdigit (c))
1701 @{
1702 ungetc (c, stdin);
1703 scanf ("%lf", &yylval);
1704 return NUM;
1705 @}
1706 @end group
1707 @group
1708 /* Return end-of-input. */
1709 if (c == EOF)
1710 return 0;
1711 /* Return a single char. */
1712 return c;
1713 @}
1714 @end group
1715 @end example
1716
1717 @node Rpcalc Main
1718 @subsection The Controlling Function
1719 @cindex controlling function
1720 @cindex main function in simple example
1721
1722 In keeping with the spirit of this example, the controlling function is
1723 kept to the bare minimum. The only requirement is that it call
1724 @code{yyparse} to start the process of parsing.
1725
1726 @example
1727 @group
1728 int
1729 main (void)
1730 @{
1731 return yyparse ();
1732 @}
1733 @end group
1734 @end example
1735
1736 @node Rpcalc Error
1737 @subsection The Error Reporting Routine
1738 @cindex error reporting routine
1739
1740 When @code{yyparse} detects a syntax error, it calls the error reporting
1741 function @code{yyerror} to print an error message (usually but not
1742 always @code{"syntax error"}). It is up to the programmer to supply
1743 @code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1744 here is the definition we will use:
1745
1746 @example
1747 @group
1748 #include <stdio.h>
1749
1750 /* Called by yyparse on error. */
1751 void
1752 yyerror (char const *s)
1753 @{
1754 fprintf (stderr, "%s\n", s);
1755 @}
1756 @end group
1757 @end example
1758
1759 After @code{yyerror} returns, the Bison parser may recover from the error
1760 and continue parsing if the grammar contains a suitable error rule
1761 (@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1762 have not written any error rules in this example, so any invalid input will
1763 cause the calculator program to exit. This is not clean behavior for a
1764 real calculator, but it is adequate for the first example.
1765
1766 @node Rpcalc Generate
1767 @subsection Running Bison to Make the Parser
1768 @cindex running Bison (introduction)
1769
1770 Before running Bison to produce a parser, we need to decide how to
1771 arrange all the source code in one or more source files. For such a
1772 simple example, the easiest thing is to put everything in one file. The
1773 definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
1774 end, in the epilogue of the file
1775 (@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
1776
1777 For a large project, you would probably have several source files, and use
1778 @code{make} to arrange to recompile them.
1779
1780 With all the source in a single file, you use the following command to
1781 convert it into a parser file:
1782
1783 @example
1784 bison @var{file}.y
1785 @end example
1786
1787 @noindent
1788 In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
1789 @sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c},
1790 removing the @samp{.y} from the original file name. The file output by
1791 Bison contains the source code for @code{yyparse}. The additional
1792 functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
1793 are copied verbatim to the output.
1794
1795 @node Rpcalc Compile
1796 @subsection Compiling the Parser File
1797 @cindex compiling the parser
1798
1799 Here is how to compile and run the parser file:
1800
1801 @example
1802 @group
1803 # @r{List files in current directory.}
1804 $ @kbd{ls}
1805 rpcalc.tab.c rpcalc.y
1806 @end group
1807
1808 @group
1809 # @r{Compile the Bison parser.}
1810 # @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
1811 $ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
1812 @end group
1813
1814 @group
1815 # @r{List files again.}
1816 $ @kbd{ls}
1817 rpcalc rpcalc.tab.c rpcalc.y
1818 @end group
1819 @end example
1820
1821 The file @file{rpcalc} now contains the executable code. Here is an
1822 example session using @code{rpcalc}.
1823
1824 @example
1825 $ @kbd{rpcalc}
1826 @kbd{4 9 +}
1827 13
1828 @kbd{3 7 + 3 4 5 *+-}
1829 -13
1830 @kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
1831 13
1832 @kbd{5 6 / 4 n +}
1833 -3.166666667
1834 @kbd{3 4 ^} @r{Exponentiation}
1835 81
1836 @kbd{^D} @r{End-of-file indicator}
1837 $
1838 @end example
1839
1840 @node Infix Calc
1841 @section Infix Notation Calculator: @code{calc}
1842 @cindex infix notation calculator
1843 @cindex @code{calc}
1844 @cindex calculator, infix notation
1845
1846 We now modify rpcalc to handle infix operators instead of postfix. Infix
1847 notation involves the concept of operator precedence and the need for
1848 parentheses nested to arbitrary depth. Here is the Bison code for
1849 @file{calc.y}, an infix desk-top calculator.
1850
1851 @example
1852 /* Infix notation calculator. */
1853
1854 %@{
1855 #define YYSTYPE double
1856 #include <math.h>
1857 #include <stdio.h>
1858 int yylex (void);
1859 void yyerror (char const *);
1860 %@}
1861
1862 /* Bison declarations. */
1863 %token NUM
1864 %left '-' '+'
1865 %left '*' '/'
1866 %precedence NEG /* negation--unary minus */
1867 %right '^' /* exponentiation */
1868
1869 %% /* The grammar follows. */
1870 input: /* empty */
1871 | input line
1872 ;
1873
1874 line: '\n'
1875 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1876 ;
1877
1878 exp: NUM @{ $$ = $1; @}
1879 | exp '+' exp @{ $$ = $1 + $3; @}
1880 | exp '-' exp @{ $$ = $1 - $3; @}
1881 | exp '*' exp @{ $$ = $1 * $3; @}
1882 | exp '/' exp @{ $$ = $1 / $3; @}
1883 | '-' exp %prec NEG @{ $$ = -$2; @}
1884 | exp '^' exp @{ $$ = pow ($1, $3); @}
1885 | '(' exp ')' @{ $$ = $2; @}
1886 ;
1887 %%
1888 @end example
1889
1890 @noindent
1891 The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1892 same as before.
1893
1894 There are two important new features shown in this code.
1895
1896 In the second section (Bison declarations), @code{%left} declares token
1897 types and says they are left-associative operators. The declarations
1898 @code{%left} and @code{%right} (right associativity) take the place of
1899 @code{%token} which is used to declare a token type name without
1900 associativity/precedence. (These tokens are single-character literals, which
1901 ordinarily don't need to be declared. We declare them here to specify
1902 the associativity/precedence.)
1903
1904 Operator precedence is determined by the line ordering of the
1905 declarations; the higher the line number of the declaration (lower on
1906 the page or screen), the higher the precedence. Hence, exponentiation
1907 has the highest precedence, unary minus (@code{NEG}) is next, followed
1908 by @samp{*} and @samp{/}, and so on. Unary minus is not associative,
1909 only precedence matters (@code{%precedence}. @xref{Precedence, ,Operator
1910 Precedence}.
1911
1912 The other important new feature is the @code{%prec} in the grammar
1913 section for the unary minus operator. The @code{%prec} simply instructs
1914 Bison that the rule @samp{| '-' exp} has the same precedence as
1915 @code{NEG}---in this case the next-to-highest. @xref{Contextual
1916 Precedence, ,Context-Dependent Precedence}.
1917
1918 Here is a sample run of @file{calc.y}:
1919
1920 @need 500
1921 @example
1922 $ @kbd{calc}
1923 @kbd{4 + 4.5 - (34/(8*3+-3))}
1924 6.880952381
1925 @kbd{-56 + 2}
1926 -54
1927 @kbd{3 ^ 2}
1928 9
1929 @end example
1930
1931 @node Simple Error Recovery
1932 @section Simple Error Recovery
1933 @cindex error recovery, simple
1934
1935 Up to this point, this manual has not addressed the issue of @dfn{error
1936 recovery}---how to continue parsing after the parser detects a syntax
1937 error. All we have handled is error reporting with @code{yyerror}.
1938 Recall that by default @code{yyparse} returns after calling
1939 @code{yyerror}. This means that an erroneous input line causes the
1940 calculator program to exit. Now we show how to rectify this deficiency.
1941
1942 The Bison language itself includes the reserved word @code{error}, which
1943 may be included in the grammar rules. In the example below it has
1944 been added to one of the alternatives for @code{line}:
1945
1946 @example
1947 @group
1948 line: '\n'
1949 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1950 | error '\n' @{ yyerrok; @}
1951 ;
1952 @end group
1953 @end example
1954
1955 This addition to the grammar allows for simple error recovery in the
1956 event of a syntax error. If an expression that cannot be evaluated is
1957 read, the error will be recognized by the third rule for @code{line},
1958 and parsing will continue. (The @code{yyerror} function is still called
1959 upon to print its message as well.) The action executes the statement
1960 @code{yyerrok}, a macro defined automatically by Bison; its meaning is
1961 that error recovery is complete (@pxref{Error Recovery}). Note the
1962 difference between @code{yyerrok} and @code{yyerror}; neither one is a
1963 misprint.
1964
1965 This form of error recovery deals with syntax errors. There are other
1966 kinds of errors; for example, division by zero, which raises an exception
1967 signal that is normally fatal. A real calculator program must handle this
1968 signal and use @code{longjmp} to return to @code{main} and resume parsing
1969 input lines; it would also have to discard the rest of the current line of
1970 input. We won't discuss this issue further because it is not specific to
1971 Bison programs.
1972
1973 @node Location Tracking Calc
1974 @section Location Tracking Calculator: @code{ltcalc}
1975 @cindex location tracking calculator
1976 @cindex @code{ltcalc}
1977 @cindex calculator, location tracking
1978
1979 This example extends the infix notation calculator with location
1980 tracking. This feature will be used to improve the error messages. For
1981 the sake of clarity, this example is a simple integer calculator, since
1982 most of the work needed to use locations will be done in the lexical
1983 analyzer.
1984
1985 @menu
1986 * Ltcalc Declarations:: Bison and C declarations for ltcalc.
1987 * Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
1988 * Ltcalc Lexer:: The lexical analyzer.
1989 @end menu
1990
1991 @node Ltcalc Declarations
1992 @subsection Declarations for @code{ltcalc}
1993
1994 The C and Bison declarations for the location tracking calculator are
1995 the same as the declarations for the infix notation calculator.
1996
1997 @example
1998 /* Location tracking calculator. */
1999
2000 %@{
2001 #define YYSTYPE int
2002 #include <math.h>
2003 int yylex (void);
2004 void yyerror (char const *);
2005 %@}
2006
2007 /* Bison declarations. */
2008 %token NUM
2009
2010 %left '-' '+'
2011 %left '*' '/'
2012 %precedence NEG
2013 %right '^'
2014
2015 %% /* The grammar follows. */
2016 @end example
2017
2018 @noindent
2019 Note there are no declarations specific to locations. Defining a data
2020 type for storing locations is not needed: we will use the type provided
2021 by default (@pxref{Location Type, ,Data Types of Locations}), which is a
2022 four member structure with the following integer fields:
2023 @code{first_line}, @code{first_column}, @code{last_line} and
2024 @code{last_column}. By conventions, and in accordance with the GNU
2025 Coding Standards and common practice, the line and column count both
2026 start at 1.
2027
2028 @node Ltcalc Rules
2029 @subsection Grammar Rules for @code{ltcalc}
2030
2031 Whether handling locations or not has no effect on the syntax of your
2032 language. Therefore, grammar rules for this example will be very close
2033 to those of the previous example: we will only modify them to benefit
2034 from the new information.
2035
2036 Here, we will use locations to report divisions by zero, and locate the
2037 wrong expressions or subexpressions.
2038
2039 @example
2040 @group
2041 input : /* empty */
2042 | input line
2043 ;
2044 @end group
2045
2046 @group
2047 line : '\n'
2048 | exp '\n' @{ printf ("%d\n", $1); @}
2049 ;
2050 @end group
2051
2052 @group
2053 exp : NUM @{ $$ = $1; @}
2054 | exp '+' exp @{ $$ = $1 + $3; @}
2055 | exp '-' exp @{ $$ = $1 - $3; @}
2056 | exp '*' exp @{ $$ = $1 * $3; @}
2057 @end group
2058 @group
2059 | exp '/' exp
2060 @{
2061 if ($3)
2062 $$ = $1 / $3;
2063 else
2064 @{
2065 $$ = 1;
2066 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2067 @@3.first_line, @@3.first_column,
2068 @@3.last_line, @@3.last_column);
2069 @}
2070 @}
2071 @end group
2072 @group
2073 | '-' exp %prec NEG @{ $$ = -$2; @}
2074 | exp '^' exp @{ $$ = pow ($1, $3); @}
2075 | '(' exp ')' @{ $$ = $2; @}
2076 @end group
2077 @end example
2078
2079 This code shows how to reach locations inside of semantic actions, by
2080 using the pseudo-variables @code{@@@var{n}} for rule components, and the
2081 pseudo-variable @code{@@$} for groupings.
2082
2083 We don't need to assign a value to @code{@@$}: the output parser does it
2084 automatically. By default, before executing the C code of each action,
2085 @code{@@$} is set to range from the beginning of @code{@@1} to the end
2086 of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2087 can be redefined (@pxref{Location Default Action, , Default Action for
2088 Locations}), and for very specific rules, @code{@@$} can be computed by
2089 hand.
2090
2091 @node Ltcalc Lexer
2092 @subsection The @code{ltcalc} Lexical Analyzer.
2093
2094 Until now, we relied on Bison's defaults to enable location
2095 tracking. The next step is to rewrite the lexical analyzer, and make it
2096 able to feed the parser with the token locations, as it already does for
2097 semantic values.
2098
2099 To this end, we must take into account every single character of the
2100 input text, to avoid the computed locations of being fuzzy or wrong:
2101
2102 @example
2103 @group
2104 int
2105 yylex (void)
2106 @{
2107 int c;
2108 @end group
2109
2110 @group
2111 /* Skip white space. */
2112 while ((c = getchar ()) == ' ' || c == '\t')
2113 ++yylloc.last_column;
2114 @end group
2115
2116 @group
2117 /* Step. */
2118 yylloc.first_line = yylloc.last_line;
2119 yylloc.first_column = yylloc.last_column;
2120 @end group
2121
2122 @group
2123 /* Process numbers. */
2124 if (isdigit (c))
2125 @{
2126 yylval = c - '0';
2127 ++yylloc.last_column;
2128 while (isdigit (c = getchar ()))
2129 @{
2130 ++yylloc.last_column;
2131 yylval = yylval * 10 + c - '0';
2132 @}
2133 ungetc (c, stdin);
2134 return NUM;
2135 @}
2136 @end group
2137
2138 /* Return end-of-input. */
2139 if (c == EOF)
2140 return 0;
2141
2142 /* Return a single char, and update location. */
2143 if (c == '\n')
2144 @{
2145 ++yylloc.last_line;
2146 yylloc.last_column = 0;
2147 @}
2148 else
2149 ++yylloc.last_column;
2150 return c;
2151 @}
2152 @end example
2153
2154 Basically, the lexical analyzer performs the same processing as before:
2155 it skips blanks and tabs, and reads numbers or single-character tokens.
2156 In addition, it updates @code{yylloc}, the global variable (of type
2157 @code{YYLTYPE}) containing the token's location.
2158
2159 Now, each time this function returns a token, the parser has its number
2160 as well as its semantic value, and its location in the text. The last
2161 needed change is to initialize @code{yylloc}, for example in the
2162 controlling function:
2163
2164 @example
2165 @group
2166 int
2167 main (void)
2168 @{
2169 yylloc.first_line = yylloc.last_line = 1;
2170 yylloc.first_column = yylloc.last_column = 0;
2171 return yyparse ();
2172 @}
2173 @end group
2174 @end example
2175
2176 Remember that computing locations is not a matter of syntax. Every
2177 character must be associated to a location update, whether it is in
2178 valid input, in comments, in literal strings, and so on.
2179
2180 @node Multi-function Calc
2181 @section Multi-Function Calculator: @code{mfcalc}
2182 @cindex multi-function calculator
2183 @cindex @code{mfcalc}
2184 @cindex calculator, multi-function
2185
2186 Now that the basics of Bison have been discussed, it is time to move on to
2187 a more advanced problem. The above calculators provided only five
2188 functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2189 be nice to have a calculator that provides other mathematical functions such
2190 as @code{sin}, @code{cos}, etc.
2191
2192 It is easy to add new operators to the infix calculator as long as they are
2193 only single-character literals. The lexical analyzer @code{yylex} passes
2194 back all nonnumeric characters as tokens, so new grammar rules suffice for
2195 adding a new operator. But we want something more flexible: built-in
2196 functions whose syntax has this form:
2197
2198 @example
2199 @var{function_name} (@var{argument})
2200 @end example
2201
2202 @noindent
2203 At the same time, we will add memory to the calculator, by allowing you
2204 to create named variables, store values in them, and use them later.
2205 Here is a sample session with the multi-function calculator:
2206
2207 @example
2208 $ @kbd{mfcalc}
2209 @kbd{pi = 3.141592653589}
2210 3.1415926536
2211 @kbd{sin(pi)}
2212 0.0000000000
2213 @kbd{alpha = beta1 = 2.3}
2214 2.3000000000
2215 @kbd{alpha}
2216 2.3000000000
2217 @kbd{ln(alpha)}
2218 0.8329091229
2219 @kbd{exp(ln(beta1))}
2220 2.3000000000
2221 $
2222 @end example
2223
2224 Note that multiple assignment and nested function calls are permitted.
2225
2226 @menu
2227 * Mfcalc Declarations:: Bison declarations for multi-function calculator.
2228 * Mfcalc Rules:: Grammar rules for the calculator.
2229 * Mfcalc Symbol Table:: Symbol table management subroutines.
2230 @end menu
2231
2232 @node Mfcalc Declarations
2233 @subsection Declarations for @code{mfcalc}
2234
2235 Here are the C and Bison declarations for the multi-function calculator.
2236
2237 @smallexample
2238 @group
2239 %@{
2240 #include <math.h> /* For math functions, cos(), sin(), etc. */
2241 #include "calc.h" /* Contains definition of `symrec'. */
2242 int yylex (void);
2243 void yyerror (char const *);
2244 %@}
2245 @end group
2246 @group
2247 %union @{
2248 double val; /* For returning numbers. */
2249 symrec *tptr; /* For returning symbol-table pointers. */
2250 @}
2251 @end group
2252 %token <val> NUM /* Simple double precision number. */
2253 %token <tptr> VAR FNCT /* Variable and Function. */
2254 %type <val> exp
2255
2256 @group
2257 %right '='
2258 %left '-' '+'
2259 %left '*' '/'
2260 %precedence NEG /* negation--unary minus */
2261 %right '^' /* exponentiation */
2262 @end group
2263 %% /* The grammar follows. */
2264 @end smallexample
2265
2266 The above grammar introduces only two new features of the Bison language.
2267 These features allow semantic values to have various data types
2268 (@pxref{Multiple Types, ,More Than One Value Type}).
2269
2270 The @code{%union} declaration specifies the entire list of possible types;
2271 this is instead of defining @code{YYSTYPE}. The allowable types are now
2272 double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2273 the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2274
2275 Since values can now have various types, it is necessary to associate a
2276 type with each grammar symbol whose semantic value is used. These symbols
2277 are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2278 declarations are augmented with information about their data type (placed
2279 between angle brackets).
2280
2281 The Bison construct @code{%type} is used for declaring nonterminal
2282 symbols, just as @code{%token} is used for declaring token types. We
2283 have not used @code{%type} before because nonterminal symbols are
2284 normally declared implicitly by the rules that define them. But
2285 @code{exp} must be declared explicitly so we can specify its value type.
2286 @xref{Type Decl, ,Nonterminal Symbols}.
2287
2288 @node Mfcalc Rules
2289 @subsection Grammar Rules for @code{mfcalc}
2290
2291 Here are the grammar rules for the multi-function calculator.
2292 Most of them are copied directly from @code{calc}; three rules,
2293 those which mention @code{VAR} or @code{FNCT}, are new.
2294
2295 @smallexample
2296 @group
2297 input: /* empty */
2298 | input line
2299 ;
2300 @end group
2301
2302 @group
2303 line:
2304 '\n'
2305 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2306 | error '\n' @{ yyerrok; @}
2307 ;
2308 @end group
2309
2310 @group
2311 exp: NUM @{ $$ = $1; @}
2312 | VAR @{ $$ = $1->value.var; @}
2313 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2314 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2315 | exp '+' exp @{ $$ = $1 + $3; @}
2316 | exp '-' exp @{ $$ = $1 - $3; @}
2317 | exp '*' exp @{ $$ = $1 * $3; @}
2318 | exp '/' exp @{ $$ = $1 / $3; @}
2319 | '-' exp %prec NEG @{ $$ = -$2; @}
2320 | exp '^' exp @{ $$ = pow ($1, $3); @}
2321 | '(' exp ')' @{ $$ = $2; @}
2322 ;
2323 @end group
2324 /* End of grammar. */
2325 %%
2326 @end smallexample
2327
2328 @node Mfcalc Symbol Table
2329 @subsection The @code{mfcalc} Symbol Table
2330 @cindex symbol table example
2331
2332 The multi-function calculator requires a symbol table to keep track of the
2333 names and meanings of variables and functions. This doesn't affect the
2334 grammar rules (except for the actions) or the Bison declarations, but it
2335 requires some additional C functions for support.
2336
2337 The symbol table itself consists of a linked list of records. Its
2338 definition, which is kept in the header @file{calc.h}, is as follows. It
2339 provides for either functions or variables to be placed in the table.
2340
2341 @smallexample
2342 @group
2343 /* Function type. */
2344 typedef double (*func_t) (double);
2345 @end group
2346
2347 @group
2348 /* Data type for links in the chain of symbols. */
2349 struct symrec
2350 @{
2351 char *name; /* name of symbol */
2352 int type; /* type of symbol: either VAR or FNCT */
2353 union
2354 @{
2355 double var; /* value of a VAR */
2356 func_t fnctptr; /* value of a FNCT */
2357 @} value;
2358 struct symrec *next; /* link field */
2359 @};
2360 @end group
2361
2362 @group
2363 typedef struct symrec symrec;
2364
2365 /* The symbol table: a chain of `struct symrec'. */
2366 extern symrec *sym_table;
2367
2368 symrec *putsym (char const *, int);
2369 symrec *getsym (char const *);
2370 @end group
2371 @end smallexample
2372
2373 The new version of @code{main} includes a call to @code{init_table}, a
2374 function that initializes the symbol table. Here it is, and
2375 @code{init_table} as well:
2376
2377 @smallexample
2378 #include <stdio.h>
2379
2380 @group
2381 /* Called by yyparse on error. */
2382 void
2383 yyerror (char const *s)
2384 @{
2385 printf ("%s\n", s);
2386 @}
2387 @end group
2388
2389 @group
2390 struct init
2391 @{
2392 char const *fname;
2393 double (*fnct) (double);
2394 @};
2395 @end group
2396
2397 @group
2398 struct init const arith_fncts[] =
2399 @{
2400 "sin", sin,
2401 "cos", cos,
2402 "atan", atan,
2403 "ln", log,
2404 "exp", exp,
2405 "sqrt", sqrt,
2406 0, 0
2407 @};
2408 @end group
2409
2410 @group
2411 /* The symbol table: a chain of `struct symrec'. */
2412 symrec *sym_table;
2413 @end group
2414
2415 @group
2416 /* Put arithmetic functions in table. */
2417 void
2418 init_table (void)
2419 @{
2420 int i;
2421 symrec *ptr;
2422 for (i = 0; arith_fncts[i].fname != 0; i++)
2423 @{
2424 ptr = putsym (arith_fncts[i].fname, FNCT);
2425 ptr->value.fnctptr = arith_fncts[i].fnct;
2426 @}
2427 @}
2428 @end group
2429
2430 @group
2431 int
2432 main (void)
2433 @{
2434 init_table ();
2435 return yyparse ();
2436 @}
2437 @end group
2438 @end smallexample
2439
2440 By simply editing the initialization list and adding the necessary include
2441 files, you can add additional functions to the calculator.
2442
2443 Two important functions allow look-up and installation of symbols in the
2444 symbol table. The function @code{putsym} is passed a name and the type
2445 (@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2446 linked to the front of the list, and a pointer to the object is returned.
2447 The function @code{getsym} is passed the name of the symbol to look up. If
2448 found, a pointer to that symbol is returned; otherwise zero is returned.
2449
2450 @smallexample
2451 symrec *
2452 putsym (char const *sym_name, int sym_type)
2453 @{
2454 symrec *ptr;
2455 ptr = (symrec *) malloc (sizeof (symrec));
2456 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2457 strcpy (ptr->name,sym_name);
2458 ptr->type = sym_type;
2459 ptr->value.var = 0; /* Set value to 0 even if fctn. */
2460 ptr->next = (struct symrec *)sym_table;
2461 sym_table = ptr;
2462 return ptr;
2463 @}
2464
2465 symrec *
2466 getsym (char const *sym_name)
2467 @{
2468 symrec *ptr;
2469 for (ptr = sym_table; ptr != (symrec *) 0;
2470 ptr = (symrec *)ptr->next)
2471 if (strcmp (ptr->name,sym_name) == 0)
2472 return ptr;
2473 return 0;
2474 @}
2475 @end smallexample
2476
2477 The function @code{yylex} must now recognize variables, numeric values, and
2478 the single-character arithmetic operators. Strings of alphanumeric
2479 characters with a leading letter are recognized as either variables or
2480 functions depending on what the symbol table says about them.
2481
2482 The string is passed to @code{getsym} for look up in the symbol table. If
2483 the name appears in the table, a pointer to its location and its type
2484 (@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2485 already in the table, then it is installed as a @code{VAR} using
2486 @code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
2487 returned to @code{yyparse}.
2488
2489 No change is needed in the handling of numeric values and arithmetic
2490 operators in @code{yylex}.
2491
2492 @smallexample
2493 @group
2494 #include <ctype.h>
2495 @end group
2496
2497 @group
2498 int
2499 yylex (void)
2500 @{
2501 int c;
2502
2503 /* Ignore white space, get first nonwhite character. */
2504 while ((c = getchar ()) == ' ' || c == '\t');
2505
2506 if (c == EOF)
2507 return 0;
2508 @end group
2509
2510 @group
2511 /* Char starts a number => parse the number. */
2512 if (c == '.' || isdigit (c))
2513 @{
2514 ungetc (c, stdin);
2515 scanf ("%lf", &yylval.val);
2516 return NUM;
2517 @}
2518 @end group
2519
2520 @group
2521 /* Char starts an identifier => read the name. */
2522 if (isalpha (c))
2523 @{
2524 symrec *s;
2525 static char *symbuf = 0;
2526 static int length = 0;
2527 int i;
2528 @end group
2529
2530 @group
2531 /* Initially make the buffer long enough
2532 for a 40-character symbol name. */
2533 if (length == 0)
2534 length = 40, symbuf = (char *)malloc (length + 1);
2535
2536 i = 0;
2537 do
2538 @end group
2539 @group
2540 @{
2541 /* If buffer is full, make it bigger. */
2542 if (i == length)
2543 @{
2544 length *= 2;
2545 symbuf = (char *) realloc (symbuf, length + 1);
2546 @}
2547 /* Add this character to the buffer. */
2548 symbuf[i++] = c;
2549 /* Get another character. */
2550 c = getchar ();
2551 @}
2552 @end group
2553 @group
2554 while (isalnum (c));
2555
2556 ungetc (c, stdin);
2557 symbuf[i] = '\0';
2558 @end group
2559
2560 @group
2561 s = getsym (symbuf);
2562 if (s == 0)
2563 s = putsym (symbuf, VAR);
2564 yylval.tptr = s;
2565 return s->type;
2566 @}
2567
2568 /* Any other character is a token by itself. */
2569 return c;
2570 @}
2571 @end group
2572 @end smallexample
2573
2574 This program is both powerful and flexible. You may easily add new
2575 functions, and it is a simple job to modify this code to install
2576 predefined variables such as @code{pi} or @code{e} as well.
2577
2578 @node Exercises
2579 @section Exercises
2580 @cindex exercises
2581
2582 @enumerate
2583 @item
2584 Add some new functions from @file{math.h} to the initialization list.
2585
2586 @item
2587 Add another array that contains constants and their values. Then
2588 modify @code{init_table} to add these constants to the symbol table.
2589 It will be easiest to give the constants type @code{VAR}.
2590
2591 @item
2592 Make the program report an error if the user refers to an
2593 uninitialized variable in any way except to store a value in it.
2594 @end enumerate
2595
2596 @node Grammar File
2597 @chapter Bison Grammar Files
2598
2599 Bison takes as input a context-free grammar specification and produces a
2600 C-language function that recognizes correct instances of the grammar.
2601
2602 The Bison grammar input file conventionally has a name ending in @samp{.y}.
2603 @xref{Invocation, ,Invoking Bison}.
2604
2605 @menu
2606 * Grammar Outline:: Overall layout of the grammar file.
2607 * Symbols:: Terminal and nonterminal symbols.
2608 * Rules:: How to write grammar rules.
2609 * Recursion:: Writing recursive rules.
2610 * Semantics:: Semantic values and actions.
2611 * Locations:: Locations and actions.
2612 * Declarations:: All kinds of Bison declarations are described here.
2613 * Multiple Parsers:: Putting more than one Bison parser in one program.
2614 @end menu
2615
2616 @node Grammar Outline
2617 @section Outline of a Bison Grammar
2618
2619 A Bison grammar file has four main sections, shown here with the
2620 appropriate delimiters:
2621
2622 @example
2623 %@{
2624 @var{Prologue}
2625 %@}
2626
2627 @var{Bison declarations}
2628
2629 %%
2630 @var{Grammar rules}
2631 %%
2632
2633 @var{Epilogue}
2634 @end example
2635
2636 Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2637 As a @acronym{GNU} extension, @samp{//} introduces a comment that
2638 continues until end of line.
2639
2640 @menu
2641 * Prologue:: Syntax and usage of the prologue.
2642 * Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
2643 * Bison Declarations:: Syntax and usage of the Bison declarations section.
2644 * Grammar Rules:: Syntax and usage of the grammar rules section.
2645 * Epilogue:: Syntax and usage of the epilogue.
2646 @end menu
2647
2648 @node Prologue
2649 @subsection The prologue
2650 @cindex declarations section
2651 @cindex Prologue
2652 @cindex declarations
2653
2654 The @var{Prologue} section contains macro definitions and declarations
2655 of functions and variables that are used in the actions in the grammar
2656 rules. These are copied to the beginning of the parser file so that
2657 they precede the definition of @code{yyparse}. You can use
2658 @samp{#include} to get the declarations from a header file. If you
2659 don't need any C declarations, you may omit the @samp{%@{} and
2660 @samp{%@}} delimiters that bracket this section.
2661
2662 The @var{Prologue} section is terminated by the first occurrence
2663 of @samp{%@}} that is outside a comment, a string literal, or a
2664 character constant.
2665
2666 You may have more than one @var{Prologue} section, intermixed with the
2667 @var{Bison declarations}. This allows you to have C and Bison
2668 declarations that refer to each other. For example, the @code{%union}
2669 declaration may use types defined in a header file, and you may wish to
2670 prototype functions that take arguments of type @code{YYSTYPE}. This
2671 can be done with two @var{Prologue} blocks, one before and one after the
2672 @code{%union} declaration.
2673
2674 @smallexample
2675 %@{
2676 #define _GNU_SOURCE
2677 #include <stdio.h>
2678 #include "ptypes.h"
2679 %@}
2680
2681 %union @{
2682 long int n;
2683 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2684 @}
2685
2686 %@{
2687 static void print_token_value (FILE *, int, YYSTYPE);
2688 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2689 %@}
2690
2691 @dots{}
2692 @end smallexample
2693
2694 When in doubt, it is usually safer to put prologue code before all
2695 Bison declarations, rather than after. For example, any definitions
2696 of feature test macros like @code{_GNU_SOURCE} or
2697 @code{_POSIX_C_SOURCE} should appear before all Bison declarations, as
2698 feature test macros can affect the behavior of Bison-generated
2699 @code{#include} directives.
2700
2701 @node Prologue Alternatives
2702 @subsection Prologue Alternatives
2703 @cindex Prologue Alternatives
2704
2705 @findex %code
2706 @findex %code requires
2707 @findex %code provides
2708 @findex %code top
2709 (The prologue alternatives described here are experimental.
2710 More user feedback will help to determine whether they should become permanent
2711 features.)
2712
2713 The functionality of @var{Prologue} sections can often be subtle and
2714 inflexible.
2715 As an alternative, Bison provides a %code directive with an explicit qualifier
2716 field, which identifies the purpose of the code and thus the location(s) where
2717 Bison should generate it.
2718 For C/C++, the qualifier can be omitted for the default location, or it can be
2719 one of @code{requires}, @code{provides}, @code{top}.
2720 @xref{Decl Summary,,%code}.
2721
2722 Look again at the example of the previous section:
2723
2724 @smallexample
2725 %@{
2726 #define _GNU_SOURCE
2727 #include <stdio.h>
2728 #include "ptypes.h"
2729 %@}
2730
2731 %union @{
2732 long int n;
2733 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2734 @}
2735
2736 %@{
2737 static void print_token_value (FILE *, int, YYSTYPE);
2738 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2739 %@}
2740
2741 @dots{}
2742 @end smallexample
2743
2744 @noindent
2745 Notice that there are two @var{Prologue} sections here, but there's a subtle
2746 distinction between their functionality.
2747 For example, if you decide to override Bison's default definition for
2748 @code{YYLTYPE}, in which @var{Prologue} section should you write your new
2749 definition?
2750 You should write it in the first since Bison will insert that code into the
2751 parser source code file @emph{before} the default @code{YYLTYPE} definition.
2752 In which @var{Prologue} section should you prototype an internal function,
2753 @code{trace_token}, that accepts @code{YYLTYPE} and @code{yytokentype} as
2754 arguments?
2755 You should prototype it in the second since Bison will insert that code
2756 @emph{after} the @code{YYLTYPE} and @code{yytokentype} definitions.
2757
2758 This distinction in functionality between the two @var{Prologue} sections is
2759 established by the appearance of the @code{%union} between them.
2760 This behavior raises a few questions.
2761 First, why should the position of a @code{%union} affect definitions related to
2762 @code{YYLTYPE} and @code{yytokentype}?
2763 Second, what if there is no @code{%union}?
2764 In that case, the second kind of @var{Prologue} section is not available.
2765 This behavior is not intuitive.
2766
2767 To avoid this subtle @code{%union} dependency, rewrite the example using a
2768 @code{%code top} and an unqualified @code{%code}.
2769 Let's go ahead and add the new @code{YYLTYPE} definition and the
2770 @code{trace_token} prototype at the same time:
2771
2772 @smallexample
2773 %code top @{
2774 #define _GNU_SOURCE
2775 #include <stdio.h>
2776
2777 /* WARNING: The following code really belongs
2778 * in a `%code requires'; see below. */
2779
2780 #include "ptypes.h"
2781 #define YYLTYPE YYLTYPE
2782 typedef struct YYLTYPE
2783 @{
2784 int first_line;
2785 int first_column;
2786 int last_line;
2787 int last_column;
2788 char *filename;
2789 @} YYLTYPE;
2790 @}
2791
2792 %union @{
2793 long int n;
2794 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2795 @}
2796
2797 %code @{
2798 static void print_token_value (FILE *, int, YYSTYPE);
2799 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2800 static void trace_token (enum yytokentype token, YYLTYPE loc);
2801 @}
2802
2803 @dots{}
2804 @end smallexample
2805
2806 @noindent
2807 In this way, @code{%code top} and the unqualified @code{%code} achieve the same
2808 functionality as the two kinds of @var{Prologue} sections, but it's always
2809 explicit which kind you intend.
2810 Moreover, both kinds are always available even in the absence of @code{%union}.
2811
2812 The @code{%code top} block above logically contains two parts.
2813 The first two lines before the warning need to appear near the top of the
2814 parser source code file.
2815 The first line after the warning is required by @code{YYSTYPE} and thus also
2816 needs to appear in the parser source code file.
2817 However, if you've instructed Bison to generate a parser header file
2818 (@pxref{Decl Summary, ,%defines}), you probably want that line to appear before
2819 the @code{YYSTYPE} definition in that header file as well.
2820 The @code{YYLTYPE} definition should also appear in the parser header file to
2821 override the default @code{YYLTYPE} definition there.
2822
2823 In other words, in the @code{%code top} block above, all but the first two
2824 lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
2825 definitions.
2826 Thus, they belong in one or more @code{%code requires}:
2827
2828 @smallexample
2829 %code top @{
2830 #define _GNU_SOURCE
2831 #include <stdio.h>
2832 @}
2833
2834 %code requires @{
2835 #include "ptypes.h"
2836 @}
2837 %union @{
2838 long int n;
2839 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2840 @}
2841
2842 %code requires @{
2843 #define YYLTYPE YYLTYPE
2844 typedef struct YYLTYPE
2845 @{
2846 int first_line;
2847 int first_column;
2848 int last_line;
2849 int last_column;
2850 char *filename;
2851 @} YYLTYPE;
2852 @}
2853
2854 %code @{
2855 static void print_token_value (FILE *, int, YYSTYPE);
2856 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2857 static void trace_token (enum yytokentype token, YYLTYPE loc);
2858 @}
2859
2860 @dots{}
2861 @end smallexample
2862
2863 @noindent
2864 Now Bison will insert @code{#include "ptypes.h"} and the new @code{YYLTYPE}
2865 definition before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
2866 definitions in both the parser source code file and the parser header file.
2867 (By the same reasoning, @code{%code requires} would also be the appropriate
2868 place to write your own definition for @code{YYSTYPE}.)
2869
2870 When you are writing dependency code for @code{YYSTYPE} and @code{YYLTYPE}, you
2871 should prefer @code{%code requires} over @code{%code top} regardless of whether
2872 you instruct Bison to generate a parser header file.
2873 When you are writing code that you need Bison to insert only into the parser
2874 source code file and that has no special need to appear at the top of that
2875 file, you should prefer the unqualified @code{%code} over @code{%code top}.
2876 These practices will make the purpose of each block of your code explicit to
2877 Bison and to other developers reading your grammar file.
2878 Following these practices, we expect the unqualified @code{%code} and
2879 @code{%code requires} to be the most important of the four @var{Prologue}
2880 alternatives.
2881
2882 At some point while developing your parser, you might decide to provide
2883 @code{trace_token} to modules that are external to your parser.
2884 Thus, you might wish for Bison to insert the prototype into both the parser
2885 header file and the parser source code file.
2886 Since this function is not a dependency required by @code{YYSTYPE} or
2887 @code{YYLTYPE}, it doesn't make sense to move its prototype to a
2888 @code{%code requires}.
2889 More importantly, since it depends upon @code{YYLTYPE} and @code{yytokentype},
2890 @code{%code requires} is not sufficient.
2891 Instead, move its prototype from the unqualified @code{%code} to a
2892 @code{%code provides}:
2893
2894 @smallexample
2895 %code top @{
2896 #define _GNU_SOURCE
2897 #include <stdio.h>
2898 @}
2899
2900 %code requires @{
2901 #include "ptypes.h"
2902 @}
2903 %union @{
2904 long int n;
2905 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2906 @}
2907
2908 %code requires @{
2909 #define YYLTYPE YYLTYPE
2910 typedef struct YYLTYPE
2911 @{
2912 int first_line;
2913 int first_column;
2914 int last_line;
2915 int last_column;
2916 char *filename;
2917 @} YYLTYPE;
2918 @}
2919
2920 %code provides @{
2921 void trace_token (enum yytokentype token, YYLTYPE loc);
2922 @}
2923
2924 %code @{
2925 static void print_token_value (FILE *, int, YYSTYPE);
2926 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2927 @}
2928
2929 @dots{}
2930 @end smallexample
2931
2932 @noindent
2933 Bison will insert the @code{trace_token} prototype into both the parser header
2934 file and the parser source code file after the definitions for
2935 @code{yytokentype}, @code{YYLTYPE}, and @code{YYSTYPE}.
2936
2937 The above examples are careful to write directives in an order that reflects
2938 the layout of the generated parser source code and header files:
2939 @code{%code top}, @code{%code requires}, @code{%code provides}, and then
2940 @code{%code}.
2941 While your grammar files may generally be easier to read if you also follow
2942 this order, Bison does not require it.
2943 Instead, Bison lets you choose an organization that makes sense to you.
2944
2945 You may declare any of these directives multiple times in the grammar file.
2946 In that case, Bison concatenates the contained code in declaration order.
2947 This is the only way in which the position of one of these directives within
2948 the grammar file affects its functionality.
2949
2950 The result of the previous two properties is greater flexibility in how you may
2951 organize your grammar file.
2952 For example, you may organize semantic-type-related directives by semantic
2953 type:
2954
2955 @smallexample
2956 %code requires @{ #include "type1.h" @}
2957 %union @{ type1 field1; @}
2958 %destructor @{ type1_free ($$); @} <field1>
2959 %printer @{ type1_print ($$); @} <field1>
2960
2961 %code requires @{ #include "type2.h" @}
2962 %union @{ type2 field2; @}
2963 %destructor @{ type2_free ($$); @} <field2>
2964 %printer @{ type2_print ($$); @} <field2>
2965 @end smallexample
2966
2967 @noindent
2968 You could even place each of the above directive groups in the rules section of
2969 the grammar file next to the set of rules that uses the associated semantic
2970 type.
2971 (In the rules section, you must terminate each of those directives with a
2972 semicolon.)
2973 And you don't have to worry that some directive (like a @code{%union}) in the
2974 definitions section is going to adversely affect their functionality in some
2975 counter-intuitive manner just because it comes first.
2976 Such an organization is not possible using @var{Prologue} sections.
2977
2978 This section has been concerned with explaining the advantages of the four
2979 @var{Prologue} alternatives over the original Yacc @var{Prologue}.
2980 However, in most cases when using these directives, you shouldn't need to
2981 think about all the low-level ordering issues discussed here.
2982 Instead, you should simply use these directives to label each block of your
2983 code according to its purpose and let Bison handle the ordering.
2984 @code{%code} is the most generic label.
2985 Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
2986 as needed.
2987
2988 @node Bison Declarations
2989 @subsection The Bison Declarations Section
2990 @cindex Bison declarations (introduction)
2991 @cindex declarations, Bison (introduction)
2992
2993 The @var{Bison declarations} section contains declarations that define
2994 terminal and nonterminal symbols, specify precedence, and so on.
2995 In some simple grammars you may not need any declarations.
2996 @xref{Declarations, ,Bison Declarations}.
2997
2998 @node Grammar Rules
2999 @subsection The Grammar Rules Section
3000 @cindex grammar rules section
3001 @cindex rules section for grammar
3002
3003 The @dfn{grammar rules} section contains one or more Bison grammar
3004 rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
3005
3006 There must always be at least one grammar rule, and the first
3007 @samp{%%} (which precedes the grammar rules) may never be omitted even
3008 if it is the first thing in the file.
3009
3010 @node Epilogue
3011 @subsection The epilogue
3012 @cindex additional C code section
3013 @cindex epilogue
3014 @cindex C code, section for additional
3015
3016 The @var{Epilogue} is copied verbatim to the end of the parser file, just as
3017 the @var{Prologue} is copied to the beginning. This is the most convenient
3018 place to put anything that you want to have in the parser file but which need
3019 not come before the definition of @code{yyparse}. For example, the
3020 definitions of @code{yylex} and @code{yyerror} often go here. Because
3021 C requires functions to be declared before being used, you often need
3022 to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
3023 even if you define them in the Epilogue.
3024 @xref{Interface, ,Parser C-Language Interface}.
3025
3026 If the last section is empty, you may omit the @samp{%%} that separates it
3027 from the grammar rules.
3028
3029 The Bison parser itself contains many macros and identifiers whose names
3030 start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3031 any such names (except those documented in this manual) in the epilogue
3032 of the grammar file.
3033
3034 @node Symbols
3035 @section Symbols, Terminal and Nonterminal
3036 @cindex nonterminal symbol
3037 @cindex terminal symbol
3038 @cindex token type
3039 @cindex symbol
3040
3041 @dfn{Symbols} in Bison grammars represent the grammatical classifications
3042 of the language.
3043
3044 A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
3045 class of syntactically equivalent tokens. You use the symbol in grammar
3046 rules to mean that a token in that class is allowed. The symbol is
3047 represented in the Bison parser by a numeric code, and the @code{yylex}
3048 function returns a token type code to indicate what kind of token has
3049 been read. You don't need to know what the code value is; you can use
3050 the symbol to stand for it.
3051
3052 A @dfn{nonterminal symbol} stands for a class of syntactically
3053 equivalent groupings. The symbol name is used in writing grammar rules.
3054 By convention, it should be all lower case.
3055
3056 Symbol names can contain letters, underscores, period, and (not at the
3057 beginning) digits and dashes. Dashes in symbol names are a GNU
3058 extension, incompatible with @acronym{POSIX} Yacc. Terminal symbols
3059 that contain periods or dashes make little sense: since they are not
3060 valid symbols (in most programming languages) they are not exported as
3061 token names.
3062
3063 There are three ways of writing terminal symbols in the grammar:
3064
3065 @itemize @bullet
3066 @item
3067 A @dfn{named token type} is written with an identifier, like an
3068 identifier in C@. By convention, it should be all upper case. Each
3069 such name must be defined with a Bison declaration such as
3070 @code{%token}. @xref{Token Decl, ,Token Type Names}.
3071
3072 @item
3073 @cindex character token
3074 @cindex literal token
3075 @cindex single-character literal
3076 A @dfn{character token type} (or @dfn{literal character token}) is
3077 written in the grammar using the same syntax used in C for character
3078 constants; for example, @code{'+'} is a character token type. A
3079 character token type doesn't need to be declared unless you need to
3080 specify its semantic value data type (@pxref{Value Type, ,Data Types of
3081 Semantic Values}), associativity, or precedence (@pxref{Precedence,
3082 ,Operator Precedence}).
3083
3084 By convention, a character token type is used only to represent a
3085 token that consists of that particular character. Thus, the token
3086 type @code{'+'} is used to represent the character @samp{+} as a
3087 token. Nothing enforces this convention, but if you depart from it,
3088 your program will confuse other readers.
3089
3090 All the usual escape sequences used in character literals in C can be
3091 used in Bison as well, but you must not use the null character as a
3092 character literal because its numeric code, zero, signifies
3093 end-of-input (@pxref{Calling Convention, ,Calling Convention
3094 for @code{yylex}}). Also, unlike standard C, trigraphs have no
3095 special meaning in Bison character literals, nor is backslash-newline
3096 allowed.
3097
3098 @item
3099 @cindex string token
3100 @cindex literal string token
3101 @cindex multicharacter literal
3102 A @dfn{literal string token} is written like a C string constant; for
3103 example, @code{"<="} is a literal string token. A literal string token
3104 doesn't need to be declared unless you need to specify its semantic
3105 value data type (@pxref{Value Type}), associativity, or precedence
3106 (@pxref{Precedence}).
3107
3108 You can associate the literal string token with a symbolic name as an
3109 alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
3110 Declarations}). If you don't do that, the lexical analyzer has to
3111 retrieve the token number for the literal string token from the
3112 @code{yytname} table (@pxref{Calling Convention}).
3113
3114 @strong{Warning}: literal string tokens do not work in Yacc.
3115
3116 By convention, a literal string token is used only to represent a token
3117 that consists of that particular string. Thus, you should use the token
3118 type @code{"<="} to represent the string @samp{<=} as a token. Bison
3119 does not enforce this convention, but if you depart from it, people who
3120 read your program will be confused.
3121
3122 All the escape sequences used in string literals in C can be used in
3123 Bison as well, except that you must not use a null character within a
3124 string literal. Also, unlike Standard C, trigraphs have no special
3125 meaning in Bison string literals, nor is backslash-newline allowed. A
3126 literal string token must contain two or more characters; for a token
3127 containing just one character, use a character token (see above).
3128 @end itemize
3129
3130 How you choose to write a terminal symbol has no effect on its
3131 grammatical meaning. That depends only on where it appears in rules and
3132 on when the parser function returns that symbol.
3133
3134 The value returned by @code{yylex} is always one of the terminal
3135 symbols, except that a zero or negative value signifies end-of-input.
3136 Whichever way you write the token type in the grammar rules, you write
3137 it the same way in the definition of @code{yylex}. The numeric code
3138 for a character token type is simply the positive numeric code of the
3139 character, so @code{yylex} can use the identical value to generate the
3140 requisite code, though you may need to convert it to @code{unsigned
3141 char} to avoid sign-extension on hosts where @code{char} is signed.
3142 Each named token type becomes a C macro in
3143 the parser file, so @code{yylex} can use the name to stand for the code.
3144 (This is why periods don't make sense in terminal symbols.)
3145 @xref{Calling Convention, ,Calling Convention for @code{yylex}}.
3146
3147 If @code{yylex} is defined in a separate file, you need to arrange for the
3148 token-type macro definitions to be available there. Use the @samp{-d}
3149 option when you run Bison, so that it will write these macro definitions
3150 into a separate header file @file{@var{name}.tab.h} which you can include
3151 in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
3152
3153 If you want to write a grammar that is portable to any Standard C
3154 host, you must use only nonnull character tokens taken from the basic
3155 execution character set of Standard C@. This set consists of the ten
3156 digits, the 52 lower- and upper-case English letters, and the
3157 characters in the following C-language string:
3158
3159 @example
3160 "\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3161 @end example
3162
3163 The @code{yylex} function and Bison must use a consistent character set
3164 and encoding for character tokens. For example, if you run Bison in an
3165 @acronym{ASCII} environment, but then compile and run the resulting
3166 program in an environment that uses an incompatible character set like
3167 @acronym{EBCDIC}, the resulting program may not work because the tables
3168 generated by Bison will assume @acronym{ASCII} numeric values for
3169 character tokens. It is standard practice for software distributions to
3170 contain C source files that were generated by Bison in an
3171 @acronym{ASCII} environment, so installers on platforms that are
3172 incompatible with @acronym{ASCII} must rebuild those files before
3173 compiling them.
3174
3175 The symbol @code{error} is a terminal symbol reserved for error recovery
3176 (@pxref{Error Recovery}); you shouldn't use it for any other purpose.
3177 In particular, @code{yylex} should never return this value. The default
3178 value of the error token is 256, unless you explicitly assigned 256 to
3179 one of your tokens with a @code{%token} declaration.
3180
3181 @node Rules
3182 @section Syntax of Grammar Rules
3183 @cindex rule syntax
3184 @cindex grammar rule syntax
3185 @cindex syntax of grammar rules
3186
3187 A Bison grammar rule has the following general form:
3188
3189 @example
3190 @group
3191 @var{result}: @var{components}@dots{}
3192 ;
3193 @end group
3194 @end example
3195
3196 @noindent
3197 where @var{result} is the nonterminal symbol that this rule describes,
3198 and @var{components} are various terminal and nonterminal symbols that
3199 are put together by this rule (@pxref{Symbols}).
3200
3201 For example,
3202
3203 @example
3204 @group
3205 exp: exp '+' exp
3206 ;
3207 @end group
3208 @end example
3209
3210 @noindent
3211 says that two groupings of type @code{exp}, with a @samp{+} token in between,
3212 can be combined into a larger grouping of type @code{exp}.
3213
3214 White space in rules is significant only to separate symbols. You can add
3215 extra white space as you wish.
3216
3217 Scattered among the components can be @var{actions} that determine
3218 the semantics of the rule. An action looks like this:
3219
3220 @example
3221 @{@var{C statements}@}
3222 @end example
3223
3224 @noindent
3225 @cindex braced code
3226 This is an example of @dfn{braced code}, that is, C code surrounded by
3227 braces, much like a compound statement in C@. Braced code can contain
3228 any sequence of C tokens, so long as its braces are balanced. Bison
3229 does not check the braced code for correctness directly; it merely
3230 copies the code to the output file, where the C compiler can check it.
3231
3232 Within braced code, the balanced-brace count is not affected by braces
3233 within comments, string literals, or character constants, but it is
3234 affected by the C digraphs @samp{<%} and @samp{%>} that represent
3235 braces. At the top level braced code must be terminated by @samp{@}}
3236 and not by a digraph. Bison does not look for trigraphs, so if braced
3237 code uses trigraphs you should ensure that they do not affect the
3238 nesting of braces or the boundaries of comments, string literals, or
3239 character constants.
3240
3241 Usually there is only one action and it follows the components.
3242 @xref{Actions}.
3243
3244 @findex |
3245 Multiple rules for the same @var{result} can be written separately or can
3246 be joined with the vertical-bar character @samp{|} as follows:
3247
3248 @example
3249 @group
3250 @var{result}: @var{rule1-components}@dots{}
3251 | @var{rule2-components}@dots{}
3252 @dots{}
3253 ;
3254 @end group
3255 @end example
3256
3257 @noindent
3258 They are still considered distinct rules even when joined in this way.
3259
3260 If @var{components} in a rule is empty, it means that @var{result} can
3261 match the empty string. For example, here is how to define a
3262 comma-separated sequence of zero or more @code{exp} groupings:
3263
3264 @example
3265 @group
3266 expseq: /* empty */
3267 | expseq1
3268 ;
3269 @end group
3270
3271 @group
3272 expseq1: exp
3273 | expseq1 ',' exp
3274 ;
3275 @end group
3276 @end example
3277
3278 @noindent
3279 It is customary to write a comment @samp{/* empty */} in each rule
3280 with no components.
3281
3282 @node Recursion
3283 @section Recursive Rules
3284 @cindex recursive rule
3285
3286 A rule is called @dfn{recursive} when its @var{result} nonterminal
3287 appears also on its right hand side. Nearly all Bison grammars need to
3288 use recursion, because that is the only way to define a sequence of any
3289 number of a particular thing. Consider this recursive definition of a
3290 comma-separated sequence of one or more expressions:
3291
3292 @example
3293 @group
3294 expseq1: exp
3295 | expseq1 ',' exp
3296 ;
3297 @end group
3298 @end example
3299
3300 @cindex left recursion
3301 @cindex right recursion
3302 @noindent
3303 Since the recursive use of @code{expseq1} is the leftmost symbol in the
3304 right hand side, we call this @dfn{left recursion}. By contrast, here
3305 the same construct is defined using @dfn{right recursion}:
3306
3307 @example
3308 @group
3309 expseq1: exp
3310 | exp ',' expseq1
3311 ;
3312 @end group
3313 @end example
3314
3315 @noindent
3316 Any kind of sequence can be defined using either left recursion or right
3317 recursion, but you should always use left recursion, because it can
3318 parse a sequence of any number of elements with bounded stack space.
3319 Right recursion uses up space on the Bison stack in proportion to the
3320 number of elements in the sequence, because all the elements must be
3321 shifted onto the stack before the rule can be applied even once.
3322 @xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3323 of this.
3324
3325 @cindex mutual recursion
3326 @dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3327 rule does not appear directly on its right hand side, but does appear
3328 in rules for other nonterminals which do appear on its right hand
3329 side.
3330
3331 For example:
3332
3333 @example
3334 @group
3335 expr: primary
3336 | primary '+' primary
3337 ;
3338 @end group
3339
3340 @group
3341 primary: constant
3342 | '(' expr ')'
3343 ;
3344 @end group
3345 @end example
3346
3347 @noindent
3348 defines two mutually-recursive nonterminals, since each refers to the
3349 other.
3350
3351 @node Semantics
3352 @section Defining Language Semantics
3353 @cindex defining language semantics
3354 @cindex language semantics, defining
3355
3356 The grammar rules for a language determine only the syntax. The semantics
3357 are determined by the semantic values associated with various tokens and
3358 groupings, and by the actions taken when various groupings are recognized.
3359
3360 For example, the calculator calculates properly because the value
3361 associated with each expression is the proper number; it adds properly
3362 because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3363 the numbers associated with @var{x} and @var{y}.
3364
3365 @menu
3366 * Value Type:: Specifying one data type for all semantic values.
3367 * Multiple Types:: Specifying several alternative data types.
3368 * Actions:: An action is the semantic definition of a grammar rule.
3369 * Action Types:: Specifying data types for actions to operate on.
3370 * Mid-Rule Actions:: Most actions go at the end of a rule.
3371 This says when, why and how to use the exceptional
3372 action in the middle of a rule.
3373 @end menu
3374
3375 @node Value Type
3376 @subsection Data Types of Semantic Values
3377 @cindex semantic value type
3378 @cindex value type, semantic
3379 @cindex data types of semantic values
3380 @cindex default data type
3381
3382 In a simple program it may be sufficient to use the same data type for
3383 the semantic values of all language constructs. This was true in the
3384 @acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
3385 Notation Calculator}).
3386
3387 Bison normally uses the type @code{int} for semantic values if your
3388 program uses the same data type for all language constructs. To
3389 specify some other type, define @code{YYSTYPE} as a macro, like this:
3390
3391 @example
3392 #define YYSTYPE double
3393 @end example
3394
3395 @noindent
3396 @code{YYSTYPE}'s replacement list should be a type name
3397 that does not contain parentheses or square brackets.
3398 This macro definition must go in the prologue of the grammar file
3399 (@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
3400
3401 @node Multiple Types
3402 @subsection More Than One Value Type
3403
3404 In most programs, you will need different data types for different kinds
3405 of tokens and groupings. For example, a numeric constant may need type
3406 @code{int} or @code{long int}, while a string constant needs type
3407 @code{char *}, and an identifier might need a pointer to an entry in the
3408 symbol table.
3409
3410 To use more than one data type for semantic values in one parser, Bison
3411 requires you to do two things:
3412
3413 @itemize @bullet
3414 @item
3415 Specify the entire collection of possible data types, either by using the
3416 @code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
3417 Value Types}), or by using a @code{typedef} or a @code{#define} to
3418 define @code{YYSTYPE} to be a union type whose member names are
3419 the type tags.
3420
3421 @item
3422 Choose one of those types for each symbol (terminal or nonterminal) for
3423 which semantic values are used. This is done for tokens with the
3424 @code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3425 and for groupings with the @code{%type} Bison declaration (@pxref{Type
3426 Decl, ,Nonterminal Symbols}).
3427 @end itemize
3428
3429 @node Actions
3430 @subsection Actions
3431 @cindex action
3432 @vindex $$
3433 @vindex $@var{n}
3434
3435 An action accompanies a syntactic rule and contains C code to be executed
3436 each time an instance of that rule is recognized. The task of most actions
3437 is to compute a semantic value for the grouping built by the rule from the
3438 semantic values associated with tokens or smaller groupings.
3439
3440 An action consists of braced code containing C statements, and can be
3441 placed at any position in the rule;
3442 it is executed at that position. Most rules have just one action at the
3443 end of the rule, following all the components. Actions in the middle of
3444 a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3445 Actions, ,Actions in Mid-Rule}).
3446
3447 The C code in an action can refer to the semantic values of the components
3448 matched by the rule with the construct @code{$@var{n}}, which stands for
3449 the value of the @var{n}th component. The semantic value for the grouping
3450 being constructed is @code{$$}. Bison translates both of these
3451 constructs into expressions of the appropriate type when it copies the
3452 actions into the parser file. @code{$$} is translated to a modifiable
3453 lvalue, so it can be assigned to.
3454
3455 Here is a typical example:
3456
3457 @example
3458 @group
3459 exp: @dots{}
3460 | exp '+' exp
3461 @{ $$ = $1 + $3; @}
3462 @end group
3463 @end example
3464
3465 @noindent
3466 This rule constructs an @code{exp} from two smaller @code{exp} groupings
3467 connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3468 refer to the semantic values of the two component @code{exp} groupings,
3469 which are the first and third symbols on the right hand side of the rule.
3470 The sum is stored into @code{$$} so that it becomes the semantic value of
3471 the addition-expression just recognized by the rule. If there were a
3472 useful semantic value associated with the @samp{+} token, it could be
3473 referred to as @code{$2}.
3474
3475 Note that the vertical-bar character @samp{|} is really a rule
3476 separator, and actions are attached to a single rule. This is a
3477 difference with tools like Flex, for which @samp{|} stands for either
3478 ``or'', or ``the same action as that of the next rule''. In the
3479 following example, the action is triggered only when @samp{b} is found:
3480
3481 @example
3482 @group
3483 a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3484 @end group
3485 @end example
3486
3487 @cindex default action
3488 If you don't specify an action for a rule, Bison supplies a default:
3489 @w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3490 becomes the value of the whole rule. Of course, the default action is
3491 valid only if the two data types match. There is no meaningful default
3492 action for an empty rule; every empty rule must have an explicit action
3493 unless the rule's value does not matter.
3494
3495 @code{$@var{n}} with @var{n} zero or negative is allowed for reference
3496 to tokens and groupings on the stack @emph{before} those that match the
3497 current rule. This is a very risky practice, and to use it reliably
3498 you must be certain of the context in which the rule is applied. Here
3499 is a case in which you can use this reliably:
3500
3501 @example
3502 @group
3503 foo: expr bar '+' expr @{ @dots{} @}
3504 | expr bar '-' expr @{ @dots{} @}
3505 ;
3506 @end group
3507
3508 @group
3509 bar: /* empty */
3510 @{ previous_expr = $0; @}
3511 ;
3512 @end group
3513 @end example
3514
3515 As long as @code{bar} is used only in the fashion shown here, @code{$0}
3516 always refers to the @code{expr} which precedes @code{bar} in the
3517 definition of @code{foo}.
3518
3519 @vindex yylval
3520 It is also possible to access the semantic value of the lookahead token, if
3521 any, from a semantic action.
3522 This semantic value is stored in @code{yylval}.
3523 @xref{Action Features, ,Special Features for Use in Actions}.
3524
3525 @node Action Types
3526 @subsection Data Types of Values in Actions
3527 @cindex action data types
3528 @cindex data types in actions
3529
3530 If you have chosen a single data type for semantic values, the @code{$$}
3531 and @code{$@var{n}} constructs always have that data type.
3532
3533 If you have used @code{%union} to specify a variety of data types, then you
3534 must declare a choice among these types for each terminal or nonterminal
3535 symbol that can have a semantic value. Then each time you use @code{$$} or
3536 @code{$@var{n}}, its data type is determined by which symbol it refers to
3537 in the rule. In this example,
3538
3539 @example
3540 @group
3541 exp: @dots{}
3542 | exp '+' exp
3543 @{ $$ = $1 + $3; @}
3544 @end group
3545 @end example
3546
3547 @noindent
3548 @code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3549 have the data type declared for the nonterminal symbol @code{exp}. If
3550 @code{$2} were used, it would have the data type declared for the
3551 terminal symbol @code{'+'}, whatever that might be.
3552
3553 Alternatively, you can specify the data type when you refer to the value,
3554 by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3555 reference. For example, if you have defined types as shown here:
3556
3557 @example
3558 @group
3559 %union @{
3560 int itype;
3561 double dtype;
3562 @}
3563 @end group
3564 @end example
3565
3566 @noindent
3567 then you can write @code{$<itype>1} to refer to the first subunit of the
3568 rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3569
3570 @node Mid-Rule Actions
3571 @subsection Actions in Mid-Rule
3572 @cindex actions in mid-rule
3573 @cindex mid-rule actions
3574
3575 Occasionally it is useful to put an action in the middle of a rule.
3576 These actions are written just like usual end-of-rule actions, but they
3577 are executed before the parser even recognizes the following components.
3578
3579 A mid-rule action may refer to the components preceding it using
3580 @code{$@var{n}}, but it may not refer to subsequent components because
3581 it is run before they are parsed.
3582
3583 The mid-rule action itself counts as one of the components of the rule.
3584 This makes a difference when there is another action later in the same rule
3585 (and usually there is another at the end): you have to count the actions
3586 along with the symbols when working out which number @var{n} to use in
3587 @code{$@var{n}}.
3588
3589 The mid-rule action can also have a semantic value. The action can set
3590 its value with an assignment to @code{$$}, and actions later in the rule
3591 can refer to the value using @code{$@var{n}}. Since there is no symbol
3592 to name the action, there is no way to declare a data type for the value
3593 in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3594 specify a data type each time you refer to this value.
3595
3596 There is no way to set the value of the entire rule with a mid-rule
3597 action, because assignments to @code{$$} do not have that effect. The
3598 only way to set the value for the entire rule is with an ordinary action
3599 at the end of the rule.
3600
3601 Here is an example from a hypothetical compiler, handling a @code{let}
3602 statement that looks like @samp{let (@var{variable}) @var{statement}} and
3603 serves to create a variable named @var{variable} temporarily for the
3604 duration of @var{statement}. To parse this construct, we must put
3605 @var{variable} into the symbol table while @var{statement} is parsed, then
3606 remove it afterward. Here is how it is done:
3607
3608 @example
3609 @group
3610 stmt: LET '(' var ')'
3611 @{ $<context>$ = push_context ();
3612 declare_variable ($3); @}
3613 stmt @{ $$ = $6;
3614 pop_context ($<context>5); @}
3615 @end group
3616 @end example
3617
3618 @noindent
3619 As soon as @samp{let (@var{variable})} has been recognized, the first
3620 action is run. It saves a copy of the current semantic context (the
3621 list of accessible variables) as its semantic value, using alternative
3622 @code{context} in the data-type union. Then it calls
3623 @code{declare_variable} to add the new variable to that list. Once the
3624 first action is finished, the embedded statement @code{stmt} can be
3625 parsed. Note that the mid-rule action is component number 5, so the
3626 @samp{stmt} is component number 6.
3627
3628 After the embedded statement is parsed, its semantic value becomes the
3629 value of the entire @code{let}-statement. Then the semantic value from the
3630 earlier action is used to restore the prior list of variables. This
3631 removes the temporary @code{let}-variable from the list so that it won't
3632 appear to exist while the rest of the program is parsed.
3633
3634 @findex %destructor
3635 @cindex discarded symbols, mid-rule actions
3636 @cindex error recovery, mid-rule actions
3637 In the above example, if the parser initiates error recovery (@pxref{Error
3638 Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3639 it might discard the previous semantic context @code{$<context>5} without
3640 restoring it.
3641 Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3642 Discarded Symbols}).
3643 However, Bison currently provides no means to declare a destructor specific to
3644 a particular mid-rule action's semantic value.
3645
3646 One solution is to bury the mid-rule action inside a nonterminal symbol and to
3647 declare a destructor for that symbol:
3648
3649 @example
3650 @group
3651 %type <context> let
3652 %destructor @{ pop_context ($$); @} let
3653
3654 %%
3655
3656 stmt: let stmt
3657 @{ $$ = $2;
3658 pop_context ($1); @}
3659 ;
3660
3661 let: LET '(' var ')'
3662 @{ $$ = push_context ();
3663 declare_variable ($3); @}
3664 ;
3665
3666 @end group
3667 @end example
3668
3669 @noindent
3670 Note that the action is now at the end of its rule.
3671 Any mid-rule action can be converted to an end-of-rule action in this way, and
3672 this is what Bison actually does to implement mid-rule actions.
3673
3674 Taking action before a rule is completely recognized often leads to
3675 conflicts since the parser must commit to a parse in order to execute the
3676 action. For example, the following two rules, without mid-rule actions,
3677 can coexist in a working parser because the parser can shift the open-brace
3678 token and look at what follows before deciding whether there is a
3679 declaration or not:
3680
3681 @example
3682 @group
3683 compound: '@{' declarations statements '@}'
3684 | '@{' statements '@}'
3685 ;
3686 @end group
3687 @end example
3688
3689 @noindent
3690 But when we add a mid-rule action as follows, the rules become nonfunctional:
3691
3692 @example
3693 @group
3694 compound: @{ prepare_for_local_variables (); @}
3695 '@{' declarations statements '@}'
3696 @end group
3697 @group
3698 | '@{' statements '@}'
3699 ;
3700 @end group
3701 @end example
3702
3703 @noindent
3704 Now the parser is forced to decide whether to run the mid-rule action
3705 when it has read no farther than the open-brace. In other words, it
3706 must commit to using one rule or the other, without sufficient
3707 information to do it correctly. (The open-brace token is what is called
3708 the @dfn{lookahead} token at this time, since the parser is still
3709 deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.)
3710
3711 You might think that you could correct the problem by putting identical
3712 actions into the two rules, like this:
3713
3714 @example
3715 @group
3716 compound: @{ prepare_for_local_variables (); @}
3717 '@{' declarations statements '@}'
3718 | @{ prepare_for_local_variables (); @}
3719 '@{' statements '@}'
3720 ;
3721 @end group
3722 @end example
3723
3724 @noindent
3725 But this does not help, because Bison does not realize that the two actions
3726 are identical. (Bison never tries to understand the C code in an action.)
3727
3728 If the grammar is such that a declaration can be distinguished from a
3729 statement by the first token (which is true in C), then one solution which
3730 does work is to put the action after the open-brace, like this:
3731
3732 @example
3733 @group
3734 compound: '@{' @{ prepare_for_local_variables (); @}
3735 declarations statements '@}'
3736 | '@{' statements '@}'
3737 ;
3738 @end group
3739 @end example
3740
3741 @noindent
3742 Now the first token of the following declaration or statement,
3743 which would in any case tell Bison which rule to use, can still do so.
3744
3745 Another solution is to bury the action inside a nonterminal symbol which
3746 serves as a subroutine:
3747
3748 @example
3749 @group
3750 subroutine: /* empty */
3751 @{ prepare_for_local_variables (); @}
3752 ;
3753
3754 @end group
3755
3756 @group
3757 compound: subroutine
3758 '@{' declarations statements '@}'
3759 | subroutine
3760 '@{' statements '@}'
3761 ;
3762 @end group
3763 @end example
3764
3765 @noindent
3766 Now Bison can execute the action in the rule for @code{subroutine} without
3767 deciding which rule for @code{compound} it will eventually use.
3768
3769 @node Locations
3770 @section Tracking Locations
3771 @cindex location
3772 @cindex textual location
3773 @cindex location, textual
3774
3775 Though grammar rules and semantic actions are enough to write a fully
3776 functional parser, it can be useful to process some additional information,
3777 especially symbol locations.
3778
3779 The way locations are handled is defined by providing a data type, and
3780 actions to take when rules are matched.
3781
3782 @menu
3783 * Location Type:: Specifying a data type for locations.
3784 * Actions and Locations:: Using locations in actions.
3785 * Location Default Action:: Defining a general way to compute locations.
3786 @end menu
3787
3788 @node Location Type
3789 @subsection Data Type of Locations
3790 @cindex data type of locations
3791 @cindex default location type
3792
3793 Defining a data type for locations is much simpler than for semantic values,
3794 since all tokens and groupings always use the same type.
3795
3796 You can specify the type of locations by defining a macro called
3797 @code{YYLTYPE}, just as you can specify the semantic value type by
3798 defining a @code{YYSTYPE} macro (@pxref{Value Type}).
3799 When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3800 four members:
3801
3802 @example
3803 typedef struct YYLTYPE
3804 @{
3805 int first_line;
3806 int first_column;
3807 int last_line;
3808 int last_column;
3809 @} YYLTYPE;
3810 @end example
3811
3812 At the beginning of the parsing, Bison initializes all these fields to 1
3813 for @code{yylloc}.
3814
3815 @node Actions and Locations
3816 @subsection Actions and Locations
3817 @cindex location actions
3818 @cindex actions, location
3819 @vindex @@$
3820 @vindex @@@var{n}
3821
3822 Actions are not only useful for defining language semantics, but also for
3823 describing the behavior of the output parser with locations.
3824
3825 The most obvious way for building locations of syntactic groupings is very
3826 similar to the way semantic values are computed. In a given rule, several
3827 constructs can be used to access the locations of the elements being matched.
3828 The location of the @var{n}th component of the right hand side is
3829 @code{@@@var{n}}, while the location of the left hand side grouping is
3830 @code{@@$}.
3831
3832 Here is a basic example using the default data type for locations:
3833
3834 @example
3835 @group
3836 exp: @dots{}
3837 | exp '/' exp
3838 @{
3839 @@$.first_column = @@1.first_column;
3840 @@$.first_line = @@1.first_line;
3841 @@$.last_column = @@3.last_column;
3842 @@$.last_line = @@3.last_line;
3843 if ($3)
3844 $$ = $1 / $3;
3845 else
3846 @{
3847 $$ = 1;
3848 fprintf (stderr,
3849 "Division by zero, l%d,c%d-l%d,c%d",
3850 @@3.first_line, @@3.first_column,
3851 @@3.last_line, @@3.last_column);
3852 @}
3853 @}
3854 @end group
3855 @end example
3856
3857 As for semantic values, there is a default action for locations that is
3858 run each time a rule is matched. It sets the beginning of @code{@@$} to the
3859 beginning of the first symbol, and the end of @code{@@$} to the end of the
3860 last symbol.
3861
3862 With this default action, the location tracking can be fully automatic. The
3863 example above simply rewrites this way:
3864
3865 @example
3866 @group
3867 exp: @dots{}
3868 | exp '/' exp
3869 @{
3870 if ($3)
3871 $$ = $1 / $3;
3872 else
3873 @{
3874 $$ = 1;
3875 fprintf (stderr,
3876 "Division by zero, l%d,c%d-l%d,c%d",
3877 @@3.first_line, @@3.first_column,
3878 @@3.last_line, @@3.last_column);
3879 @}
3880 @}
3881 @end group
3882 @end example
3883
3884 @vindex yylloc
3885 It is also possible to access the location of the lookahead token, if any,
3886 from a semantic action.
3887 This location is stored in @code{yylloc}.
3888 @xref{Action Features, ,Special Features for Use in Actions}.
3889
3890 @node Location Default Action
3891 @subsection Default Action for Locations
3892 @vindex YYLLOC_DEFAULT
3893 @cindex @acronym{GLR} parsers and @code{YYLLOC_DEFAULT}
3894
3895 Actually, actions are not the best place to compute locations. Since
3896 locations are much more general than semantic values, there is room in
3897 the output parser to redefine the default action to take for each
3898 rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
3899 matched, before the associated action is run. It is also invoked
3900 while processing a syntax error, to compute the error's location.
3901 Before reporting an unresolvable syntactic ambiguity, a @acronym{GLR}
3902 parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
3903 of that ambiguity.
3904
3905 Most of the time, this macro is general enough to suppress location
3906 dedicated code from semantic actions.
3907
3908 The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
3909 the location of the grouping (the result of the computation). When a
3910 rule is matched, the second parameter identifies locations of
3911 all right hand side elements of the rule being matched, and the third
3912 parameter is the size of the rule's right hand side.
3913 When a @acronym{GLR} parser reports an ambiguity, which of multiple candidate
3914 right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
3915 When processing a syntax error, the second parameter identifies locations
3916 of the symbols that were discarded during error processing, and the third
3917 parameter is the number of discarded symbols.
3918
3919 By default, @code{YYLLOC_DEFAULT} is defined this way:
3920
3921 @smallexample
3922 @group
3923 # define YYLLOC_DEFAULT(Current, Rhs, N) \
3924 do \
3925 if (N) \
3926 @{ \
3927 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
3928 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
3929 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
3930 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
3931 @} \
3932 else \
3933 @{ \
3934 (Current).first_line = (Current).last_line = \
3935 YYRHSLOC(Rhs, 0).last_line; \
3936 (Current).first_column = (Current).last_column = \
3937 YYRHSLOC(Rhs, 0).last_column; \
3938 @} \
3939 while (0)
3940 @end group
3941 @end smallexample
3942
3943 where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
3944 in @var{rhs} when @var{k} is positive, and the location of the symbol
3945 just before the reduction when @var{k} and @var{n} are both zero.
3946
3947 When defining @code{YYLLOC_DEFAULT}, you should consider that:
3948
3949 @itemize @bullet
3950 @item
3951 All arguments are free of side-effects. However, only the first one (the
3952 result) should be modified by @code{YYLLOC_DEFAULT}.
3953
3954 @item
3955 For consistency with semantic actions, valid indexes within the
3956 right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
3957 valid index, and it refers to the symbol just before the reduction.
3958 During error processing @var{n} is always positive.
3959
3960 @item
3961 Your macro should parenthesize its arguments, if need be, since the
3962 actual arguments may not be surrounded by parentheses. Also, your
3963 macro should expand to something that can be used as a single
3964 statement when it is followed by a semicolon.
3965 @end itemize
3966
3967 @node Declarations
3968 @section Bison Declarations
3969 @cindex declarations, Bison
3970 @cindex Bison declarations
3971
3972 The @dfn{Bison declarations} section of a Bison grammar defines the symbols
3973 used in formulating the grammar and the data types of semantic values.
3974 @xref{Symbols}.
3975
3976 All token type names (but not single-character literal tokens such as
3977 @code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
3978 declared if you need to specify which data type to use for the semantic
3979 value (@pxref{Multiple Types, ,More Than One Value Type}).
3980
3981 The first rule in the file also specifies the start symbol, by default.
3982 If you want some other symbol to be the start symbol, you must declare
3983 it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
3984 Grammars}).
3985
3986 @menu
3987 * Require Decl:: Requiring a Bison version.
3988 * Token Decl:: Declaring terminal symbols.
3989 * Precedence Decl:: Declaring terminals with precedence and associativity.
3990 * Union Decl:: Declaring the set of all semantic value types.
3991 * Type Decl:: Declaring the choice of type for a nonterminal symbol.
3992 * Initial Action Decl:: Code run before parsing starts.
3993 * Destructor Decl:: Declaring how symbols are freed.
3994 * Expect Decl:: Suppressing warnings about parsing conflicts.
3995 * Start Decl:: Specifying the start symbol.
3996 * Pure Decl:: Requesting a reentrant parser.
3997 * Push Decl:: Requesting a push parser.
3998 * Decl Summary:: Table of all Bison declarations.
3999 @end menu
4000
4001 @node Require Decl
4002 @subsection Require a Version of Bison
4003 @cindex version requirement
4004 @cindex requiring a version of Bison
4005 @findex %require
4006
4007 You may require the minimum version of Bison to process the grammar. If
4008 the requirement is not met, @command{bison} exits with an error (exit
4009 status 63).
4010
4011 @example
4012 %require "@var{version}"
4013 @end example
4014
4015 @node Token Decl
4016 @subsection Token Type Names
4017 @cindex declaring token type names
4018 @cindex token type names, declaring
4019 @cindex declaring literal string tokens
4020 @findex %token
4021
4022 The basic way to declare a token type name (terminal symbol) is as follows:
4023
4024 @example
4025 %token @var{name}
4026 @end example
4027
4028 Bison will convert this into a @code{#define} directive in
4029 the parser, so that the function @code{yylex} (if it is in this file)
4030 can use the name @var{name} to stand for this token type's code.
4031
4032 Alternatively, you can use @code{%left}, @code{%right},
4033 @code{%precedence}, or
4034 @code{%nonassoc} instead of @code{%token}, if you wish to specify
4035 associativity and precedence. @xref{Precedence Decl, ,Operator
4036 Precedence}.
4037
4038 You can explicitly specify the numeric code for a token type by appending
4039 a nonnegative decimal or hexadecimal integer value in the field immediately
4040 following the token name:
4041
4042 @example
4043 %token NUM 300
4044 %token XNUM 0x12d // a GNU extension
4045 @end example
4046
4047 @noindent
4048 It is generally best, however, to let Bison choose the numeric codes for
4049 all token types. Bison will automatically select codes that don't conflict
4050 with each other or with normal characters.
4051
4052 In the event that the stack type is a union, you must augment the
4053 @code{%token} or other token declaration to include the data type
4054 alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
4055 Than One Value Type}).
4056
4057 For example:
4058
4059 @example
4060 @group
4061 %union @{ /* define stack type */
4062 double val;
4063 symrec *tptr;
4064 @}
4065 %token <val> NUM /* define token NUM and its type */
4066 @end group
4067 @end example
4068
4069 You can associate a literal string token with a token type name by
4070 writing the literal string at the end of a @code{%token}
4071 declaration which declares the name. For example:
4072
4073 @example
4074 %token arrow "=>"
4075 @end example
4076
4077 @noindent
4078 For example, a grammar for the C language might specify these names with
4079 equivalent literal string tokens:
4080
4081 @example
4082 %token <operator> OR "||"
4083 %token <operator> LE 134 "<="
4084 %left OR "<="
4085 @end example
4086
4087 @noindent
4088 Once you equate the literal string and the token name, you can use them
4089 interchangeably in further declarations or the grammar rules. The
4090 @code{yylex} function can use the token name or the literal string to
4091 obtain the token type code number (@pxref{Calling Convention}).
4092 Syntax error messages passed to @code{yyerror} from the parser will reference
4093 the literal string instead of the token name.
4094
4095 The token numbered as 0 corresponds to end of file; the following line
4096 allows for nicer error messages referring to ``end of file'' instead
4097 of ``$end'':
4098
4099 @example
4100 %token END 0 "end of file"
4101 @end example
4102
4103 @node Precedence Decl
4104 @subsection Operator Precedence
4105 @cindex precedence declarations
4106 @cindex declaring operator precedence
4107 @cindex operator precedence, declaring
4108
4109 Use the @code{%left}, @code{%right}, @code{%nonassoc}, or
4110 @code{%precedence} declaration to
4111 declare a token and specify its precedence and associativity, all at
4112 once. These are called @dfn{precedence declarations}.
4113 @xref{Precedence, ,Operator Precedence}, for general information on
4114 operator precedence.
4115
4116 The syntax of a precedence declaration is nearly the same as that of
4117 @code{%token}: either
4118
4119 @example
4120 %left @var{symbols}@dots{}
4121 @end example
4122
4123 @noindent
4124 or
4125
4126 @example
4127 %left <@var{type}> @var{symbols}@dots{}
4128 @end example
4129
4130 And indeed any of these declarations serves the purposes of @code{%token}.
4131 But in addition, they specify the associativity and relative precedence for
4132 all the @var{symbols}:
4133
4134 @itemize @bullet
4135 @item
4136 The associativity of an operator @var{op} determines how repeated uses
4137 of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
4138 @var{z}} is parsed by grouping @var{x} with @var{y} first or by
4139 grouping @var{y} with @var{z} first. @code{%left} specifies
4140 left-associativity (grouping @var{x} with @var{y} first) and
4141 @code{%right} specifies right-associativity (grouping @var{y} with
4142 @var{z} first). @code{%nonassoc} specifies no associativity, which
4143 means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
4144 considered a syntax error.
4145
4146 @code{%precedence} gives only precedence to the @var{symbols}, and
4147 defines no associativity at all. Use this to define precedence only,
4148 and leave any potential conflict due to associativity enabled.
4149
4150 @item
4151 The precedence of an operator determines how it nests with other operators.
4152 All the tokens declared in a single precedence declaration have equal
4153 precedence and nest together according to their associativity.
4154 When two tokens declared in different precedence declarations associate,
4155 the one declared later has the higher precedence and is grouped first.
4156 @end itemize
4157
4158 For backward compatibility, there is a confusing difference between the
4159 argument lists of @code{%token} and precedence declarations.
4160 Only a @code{%token} can associate a literal string with a token type name.
4161 A precedence declaration always interprets a literal string as a reference to a
4162 separate token.
4163 For example:
4164
4165 @example
4166 %left OR "<=" // Does not declare an alias.
4167 %left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
4168 @end example
4169
4170 @node Union Decl
4171 @subsection The Collection of Value Types
4172 @cindex declaring value types
4173 @cindex value types, declaring
4174 @findex %union
4175
4176 The @code{%union} declaration specifies the entire collection of
4177 possible data types for semantic values. The keyword @code{%union} is
4178 followed by braced code containing the same thing that goes inside a
4179 @code{union} in C@.
4180
4181 For example:
4182
4183 @example
4184 @group
4185 %union @{
4186 double val;
4187 symrec *tptr;
4188 @}
4189 @end group
4190 @end example
4191
4192 @noindent
4193 This says that the two alternative types are @code{double} and @code{symrec
4194 *}. They are given names @code{val} and @code{tptr}; these names are used
4195 in the @code{%token} and @code{%type} declarations to pick one of the types
4196 for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
4197
4198 As an extension to @acronym{POSIX}, a tag is allowed after the
4199 @code{union}. For example:
4200
4201 @example
4202 @group
4203 %union value @{
4204 double val;
4205 symrec *tptr;
4206 @}
4207 @end group
4208 @end example
4209
4210 @noindent
4211 specifies the union tag @code{value}, so the corresponding C type is
4212 @code{union value}. If you do not specify a tag, it defaults to
4213 @code{YYSTYPE}.
4214
4215 As another extension to @acronym{POSIX}, you may specify multiple
4216 @code{%union} declarations; their contents are concatenated. However,
4217 only the first @code{%union} declaration can specify a tag.
4218
4219 Note that, unlike making a @code{union} declaration in C, you need not write
4220 a semicolon after the closing brace.
4221
4222 Instead of @code{%union}, you can define and use your own union type
4223 @code{YYSTYPE} if your grammar contains at least one
4224 @samp{<@var{type}>} tag. For example, you can put the following into
4225 a header file @file{parser.h}:
4226
4227 @example
4228 @group
4229 union YYSTYPE @{
4230 double val;
4231 symrec *tptr;
4232 @};
4233 typedef union YYSTYPE YYSTYPE;
4234 @end group
4235 @end example
4236
4237 @noindent
4238 and then your grammar can use the following
4239 instead of @code{%union}:
4240
4241 @example
4242 @group
4243 %@{
4244 #include "parser.h"
4245 %@}
4246 %type <val> expr
4247 %token <tptr> ID
4248 @end group
4249 @end example
4250
4251 @node Type Decl
4252 @subsection Nonterminal Symbols
4253 @cindex declaring value types, nonterminals
4254 @cindex value types, nonterminals, declaring
4255 @findex %type
4256
4257 @noindent
4258 When you use @code{%union} to specify multiple value types, you must
4259 declare the value type of each nonterminal symbol for which values are
4260 used. This is done with a @code{%type} declaration, like this:
4261
4262 @example
4263 %type <@var{type}> @var{nonterminal}@dots{}
4264 @end example
4265
4266 @noindent
4267 Here @var{nonterminal} is the name of a nonterminal symbol, and
4268 @var{type} is the name given in the @code{%union} to the alternative
4269 that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
4270 can give any number of nonterminal symbols in the same @code{%type}
4271 declaration, if they have the same value type. Use spaces to separate
4272 the symbol names.
4273
4274 You can also declare the value type of a terminal symbol. To do this,
4275 use the same @code{<@var{type}>} construction in a declaration for the
4276 terminal symbol. All kinds of token declarations allow
4277 @code{<@var{type}>}.
4278
4279 @node Initial Action Decl
4280 @subsection Performing Actions before Parsing
4281 @findex %initial-action
4282
4283 Sometimes your parser needs to perform some initializations before
4284 parsing. The @code{%initial-action} directive allows for such arbitrary
4285 code.
4286
4287 @deffn {Directive} %initial-action @{ @var{code} @}
4288 @findex %initial-action
4289 Declare that the braced @var{code} must be invoked before parsing each time
4290 @code{yyparse} is called. The @var{code} may use @code{$$} and
4291 @code{@@$} --- initial value and location of the lookahead --- and the
4292 @code{%parse-param}.
4293 @end deffn
4294
4295 For instance, if your locations use a file name, you may use
4296
4297 @example
4298 %parse-param @{ char const *file_name @};
4299 %initial-action
4300 @{
4301 @@$.initialize (file_name);
4302 @};
4303 @end example
4304
4305
4306 @node Destructor Decl
4307 @subsection Freeing Discarded Symbols
4308 @cindex freeing discarded symbols
4309 @findex %destructor
4310 @findex <*>
4311 @findex <>
4312 During error recovery (@pxref{Error Recovery}), symbols already pushed
4313 on the stack and tokens coming from the rest of the file are discarded
4314 until the parser falls on its feet. If the parser runs out of memory,
4315 or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
4316 symbols on the stack must be discarded. Even if the parser succeeds, it
4317 must discard the start symbol.
4318
4319 When discarded symbols convey heap based information, this memory is
4320 lost. While this behavior can be tolerable for batch parsers, such as
4321 in traditional compilers, it is unacceptable for programs like shells or
4322 protocol implementations that may parse and execute indefinitely.
4323
4324 The @code{%destructor} directive defines code that is called when a
4325 symbol is automatically discarded.
4326
4327 @deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
4328 @findex %destructor
4329 Invoke the braced @var{code} whenever the parser discards one of the
4330 @var{symbols}.
4331 Within @var{code}, @code{$$} designates the semantic value associated
4332 with the discarded symbol, and @code{@@$} designates its location.
4333 The additional parser parameters are also available (@pxref{Parser Function, ,
4334 The Parser Function @code{yyparse}}).
4335
4336 When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
4337 per-symbol @code{%destructor}.
4338 You may also define a per-type @code{%destructor} by listing a semantic type
4339 tag among @var{symbols}.
4340 In that case, the parser will invoke this @var{code} whenever it discards any
4341 grammar symbol that has that semantic type tag unless that symbol has its own
4342 per-symbol @code{%destructor}.
4343
4344 Finally, you can define two different kinds of default @code{%destructor}s.
4345 (These default forms are experimental.
4346 More user feedback will help to determine whether they should become permanent
4347 features.)
4348 You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
4349 exactly one @code{%destructor} declaration in your grammar file.
4350 The parser will invoke the @var{code} associated with one of these whenever it
4351 discards any user-defined grammar symbol that has no per-symbol and no per-type
4352 @code{%destructor}.
4353 The parser uses the @var{code} for @code{<*>} in the case of such a grammar
4354 symbol for which you have formally declared a semantic type tag (@code{%type}
4355 counts as such a declaration, but @code{$<tag>$} does not).
4356 The parser uses the @var{code} for @code{<>} in the case of such a grammar
4357 symbol that has no declared semantic type tag.
4358 @end deffn
4359
4360 @noindent
4361 For example:
4362
4363 @smallexample
4364 %union @{ char *string; @}
4365 %token <string> STRING1
4366 %token <string> STRING2
4367 %type <string> string1
4368 %type <string> string2
4369 %union @{ char character; @}
4370 %token <character> CHR
4371 %type <character> chr
4372 %token TAGLESS
4373
4374 %destructor @{ @} <character>
4375 %destructor @{ free ($$); @} <*>
4376 %destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
4377 %destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
4378 @end smallexample
4379
4380 @noindent
4381 guarantees that, when the parser discards any user-defined symbol that has a
4382 semantic type tag other than @code{<character>}, it passes its semantic value
4383 to @code{free} by default.
4384 However, when the parser discards a @code{STRING1} or a @code{string1}, it also
4385 prints its line number to @code{stdout}.
4386 It performs only the second @code{%destructor} in this case, so it invokes
4387 @code{free} only once.
4388 Finally, the parser merely prints a message whenever it discards any symbol,
4389 such as @code{TAGLESS}, that has no semantic type tag.
4390
4391 A Bison-generated parser invokes the default @code{%destructor}s only for
4392 user-defined as opposed to Bison-defined symbols.
4393 For example, the parser will not invoke either kind of default
4394 @code{%destructor} for the special Bison-defined symbols @code{$accept},
4395 @code{$undefined}, or @code{$end} (@pxref{Table of Symbols, ,Bison Symbols}),
4396 none of which you can reference in your grammar.
4397 It also will not invoke either for the @code{error} token (@pxref{Table of
4398 Symbols, ,error}), which is always defined by Bison regardless of whether you
4399 reference it in your grammar.
4400 However, it may invoke one of them for the end token (token 0) if you
4401 redefine it from @code{$end} to, for example, @code{END}:
4402
4403 @smallexample
4404 %token END 0
4405 @end smallexample
4406
4407 @cindex actions in mid-rule
4408 @cindex mid-rule actions
4409 Finally, Bison will never invoke a @code{%destructor} for an unreferenced
4410 mid-rule semantic value (@pxref{Mid-Rule Actions,,Actions in Mid-Rule}).
4411 That is, Bison does not consider a mid-rule to have a semantic value if you do
4412 not reference @code{$$} in the mid-rule's action or @code{$@var{n}} (where
4413 @var{n} is the RHS symbol position of the mid-rule) in any later action in that
4414 rule.
4415 However, if you do reference either, the Bison-generated parser will invoke the
4416 @code{<>} @code{%destructor} whenever it discards the mid-rule symbol.
4417
4418 @ignore
4419 @noindent
4420 In the future, it may be possible to redefine the @code{error} token as a
4421 nonterminal that captures the discarded symbols.
4422 In that case, the parser will invoke the default destructor for it as well.
4423 @end ignore
4424
4425 @sp 1
4426
4427 @cindex discarded symbols
4428 @dfn{Discarded symbols} are the following:
4429
4430 @itemize
4431 @item
4432 stacked symbols popped during the first phase of error recovery,
4433 @item
4434 incoming terminals during the second phase of error recovery,
4435 @item
4436 the current lookahead and the entire stack (except the current
4437 right-hand side symbols) when the parser returns immediately, and
4438 @item
4439 the start symbol, when the parser succeeds.
4440 @end itemize
4441
4442 The parser can @dfn{return immediately} because of an explicit call to
4443 @code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
4444 exhaustion.
4445
4446 Right-hand side symbols of a rule that explicitly triggers a syntax
4447 error via @code{YYERROR} are not discarded automatically. As a rule
4448 of thumb, destructors are invoked only when user actions cannot manage
4449 the memory.
4450
4451 @node Expect Decl
4452 @subsection Suppressing Conflict Warnings
4453 @cindex suppressing conflict warnings
4454 @cindex preventing warnings about conflicts
4455 @cindex warnings, preventing
4456 @cindex conflicts, suppressing warnings of
4457 @findex %expect
4458 @findex %expect-rr
4459
4460 Bison normally warns if there are any conflicts in the grammar
4461 (@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4462 have harmless shift/reduce conflicts which are resolved in a predictable
4463 way and would be difficult to eliminate. It is desirable to suppress
4464 the warning about these conflicts unless the number of conflicts
4465 changes. You can do this with the @code{%expect} declaration.
4466
4467 The declaration looks like this:
4468
4469 @example
4470 %expect @var{n}
4471 @end example
4472
4473 Here @var{n} is a decimal integer. The declaration says there should
4474 be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4475 Bison reports an error if the number of shift/reduce conflicts differs
4476 from @var{n}, or if there are any reduce/reduce conflicts.
4477
4478 For deterministic parsers, reduce/reduce conflicts are more
4479 serious, and should be eliminated entirely. Bison will always report
4480 reduce/reduce conflicts for these parsers. With @acronym{GLR}
4481 parsers, however, both kinds of conflicts are routine; otherwise,
4482 there would be no need to use @acronym{GLR} parsing. Therefore, it is
4483 also possible to specify an expected number of reduce/reduce conflicts
4484 in @acronym{GLR} parsers, using the declaration:
4485
4486 @example
4487 %expect-rr @var{n}
4488 @end example
4489
4490 In general, using @code{%expect} involves these steps:
4491
4492 @itemize @bullet
4493 @item
4494 Compile your grammar without @code{%expect}. Use the @samp{-v} option
4495 to get a verbose list of where the conflicts occur. Bison will also
4496 print the number of conflicts.
4497
4498 @item
4499 Check each of the conflicts to make sure that Bison's default
4500 resolution is what you really want. If not, rewrite the grammar and
4501 go back to the beginning.
4502
4503 @item
4504 Add an @code{%expect} declaration, copying the number @var{n} from the
4505 number which Bison printed. With @acronym{GLR} parsers, add an
4506 @code{%expect-rr} declaration as well.
4507 @end itemize
4508
4509 Now Bison will warn you if you introduce an unexpected conflict, but
4510 will keep silent otherwise.
4511
4512 @node Start Decl
4513 @subsection The Start-Symbol
4514 @cindex declaring the start symbol
4515 @cindex start symbol, declaring
4516 @cindex default start symbol
4517 @findex %start
4518
4519 Bison assumes by default that the start symbol for the grammar is the first
4520 nonterminal specified in the grammar specification section. The programmer
4521 may override this restriction with the @code{%start} declaration as follows:
4522
4523 @example
4524 %start @var{symbol}
4525 @end example
4526
4527 @node Pure Decl
4528 @subsection A Pure (Reentrant) Parser
4529 @cindex reentrant parser
4530 @cindex pure parser
4531 @findex %define api.pure
4532
4533 A @dfn{reentrant} program is one which does not alter in the course of
4534 execution; in other words, it consists entirely of @dfn{pure} (read-only)
4535 code. Reentrancy is important whenever asynchronous execution is possible;
4536 for example, a nonreentrant program may not be safe to call from a signal
4537 handler. In systems with multiple threads of control, a nonreentrant
4538 program must be called only within interlocks.
4539
4540 Normally, Bison generates a parser which is not reentrant. This is
4541 suitable for most uses, and it permits compatibility with Yacc. (The
4542 standard Yacc interfaces are inherently nonreentrant, because they use
4543 statically allocated variables for communication with @code{yylex},
4544 including @code{yylval} and @code{yylloc}.)
4545
4546 Alternatively, you can generate a pure, reentrant parser. The Bison
4547 declaration @code{%define api.pure} says that you want the parser to be
4548 reentrant. It looks like this:
4549
4550 @example
4551 %define api.pure
4552 @end example
4553
4554 The result is that the communication variables @code{yylval} and
4555 @code{yylloc} become local variables in @code{yyparse}, and a different
4556 calling convention is used for the lexical analyzer function
4557 @code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
4558 Parsers}, for the details of this. The variable @code{yynerrs}
4559 becomes local in @code{yyparse} in pull mode but it becomes a member
4560 of yypstate in push mode. (@pxref{Error Reporting, ,The Error
4561 Reporting Function @code{yyerror}}). The convention for calling
4562 @code{yyparse} itself is unchanged.
4563
4564 Whether the parser is pure has nothing to do with the grammar rules.
4565 You can generate either a pure parser or a nonreentrant parser from any
4566 valid grammar.
4567
4568 @node Push Decl
4569 @subsection A Push Parser
4570 @cindex push parser
4571 @cindex push parser
4572 @findex %define api.push-pull
4573
4574 (The current push parsing interface is experimental and may evolve.
4575 More user feedback will help to stabilize it.)
4576
4577 A pull parser is called once and it takes control until all its input
4578 is completely parsed. A push parser, on the other hand, is called
4579 each time a new token is made available.
4580
4581 A push parser is typically useful when the parser is part of a
4582 main event loop in the client's application. This is typically
4583 a requirement of a GUI, when the main event loop needs to be triggered
4584 within a certain time period.
4585
4586 Normally, Bison generates a pull parser.
4587 The following Bison declaration says that you want the parser to be a push
4588 parser (@pxref{Decl Summary,,%define api.push-pull}):
4589
4590 @example
4591 %define api.push-pull "push"
4592 @end example
4593
4594 In almost all cases, you want to ensure that your push parser is also
4595 a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). The only
4596 time you should create an impure push parser is to have backwards
4597 compatibility with the impure Yacc pull mode interface. Unless you know
4598 what you are doing, your declarations should look like this:
4599
4600 @example
4601 %define api.pure
4602 %define api.push-pull "push"
4603 @end example
4604
4605 There is a major notable functional difference between the pure push parser
4606 and the impure push parser. It is acceptable for a pure push parser to have
4607 many parser instances, of the same type of parser, in memory at the same time.
4608 An impure push parser should only use one parser at a time.
4609
4610 When a push parser is selected, Bison will generate some new symbols in
4611 the generated parser. @code{yypstate} is a structure that the generated
4612 parser uses to store the parser's state. @code{yypstate_new} is the
4613 function that will create a new parser instance. @code{yypstate_delete}
4614 will free the resources associated with the corresponding parser instance.
4615 Finally, @code{yypush_parse} is the function that should be called whenever a
4616 token is available to provide the parser. A trivial example
4617 of using a pure push parser would look like this:
4618
4619 @example
4620 int status;
4621 yypstate *ps = yypstate_new ();
4622 do @{
4623 status = yypush_parse (ps, yylex (), NULL);
4624 @} while (status == YYPUSH_MORE);
4625 yypstate_delete (ps);
4626 @end example
4627
4628 If the user decided to use an impure push parser, a few things about
4629 the generated parser will change. The @code{yychar} variable becomes
4630 a global variable instead of a variable in the @code{yypush_parse} function.
4631 For this reason, the signature of the @code{yypush_parse} function is
4632 changed to remove the token as a parameter. A nonreentrant push parser
4633 example would thus look like this:
4634
4635 @example
4636 extern int yychar;
4637 int status;
4638 yypstate *ps = yypstate_new ();
4639 do @{
4640 yychar = yylex ();
4641 status = yypush_parse (ps);
4642 @} while (status == YYPUSH_MORE);
4643 yypstate_delete (ps);
4644 @end example
4645
4646 That's it. Notice the next token is put into the global variable @code{yychar}
4647 for use by the next invocation of the @code{yypush_parse} function.
4648
4649 Bison also supports both the push parser interface along with the pull parser
4650 interface in the same generated parser. In order to get this functionality,
4651 you should replace the @code{%define api.push-pull "push"} declaration with the
4652 @code{%define api.push-pull "both"} declaration. Doing this will create all of
4653 the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
4654 and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
4655 would be used. However, the user should note that it is implemented in the
4656 generated parser by calling @code{yypull_parse}.
4657 This makes the @code{yyparse} function that is generated with the
4658 @code{%define api.push-pull "both"} declaration slower than the normal
4659 @code{yyparse} function. If the user
4660 calls the @code{yypull_parse} function it will parse the rest of the input
4661 stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
4662 and then @code{yypull_parse} the rest of the input stream. If you would like
4663 to switch back and forth between between parsing styles, you would have to
4664 write your own @code{yypull_parse} function that knows when to quit looking
4665 for input. An example of using the @code{yypull_parse} function would look
4666 like this:
4667
4668 @example
4669 yypstate *ps = yypstate_new ();
4670 yypull_parse (ps); /* Will call the lexer */
4671 yypstate_delete (ps);
4672 @end example
4673
4674 Adding the @code{%define api.pure} declaration does exactly the same thing to
4675 the generated parser with @code{%define api.push-pull "both"} as it did for
4676 @code{%define api.push-pull "push"}.
4677
4678 @node Decl Summary
4679 @subsection Bison Declaration Summary
4680 @cindex Bison declaration summary
4681 @cindex declaration summary
4682 @cindex summary, Bison declaration
4683
4684 Here is a summary of the declarations used to define a grammar:
4685
4686 @deffn {Directive} %union
4687 Declare the collection of data types that semantic values may have
4688 (@pxref{Union Decl, ,The Collection of Value Types}).
4689 @end deffn
4690
4691 @deffn {Directive} %token
4692 Declare a terminal symbol (token type name) with no precedence
4693 or associativity specified (@pxref{Token Decl, ,Token Type Names}).
4694 @end deffn
4695
4696 @deffn {Directive} %right
4697 Declare a terminal symbol (token type name) that is right-associative
4698 (@pxref{Precedence Decl, ,Operator Precedence}).
4699 @end deffn
4700
4701 @deffn {Directive} %left
4702 Declare a terminal symbol (token type name) that is left-associative
4703 (@pxref{Precedence Decl, ,Operator Precedence}).
4704 @end deffn
4705
4706 @deffn {Directive} %nonassoc
4707 Declare a terminal symbol (token type name) that is nonassociative
4708 (@pxref{Precedence Decl, ,Operator Precedence}).
4709 Using it in a way that would be associative is a syntax error.
4710 @end deffn
4711
4712 @ifset defaultprec
4713 @deffn {Directive} %default-prec
4714 Assign a precedence to rules lacking an explicit @code{%prec} modifier
4715 (@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4716 @end deffn
4717 @end ifset
4718
4719 @deffn {Directive} %type
4720 Declare the type of semantic values for a nonterminal symbol
4721 (@pxref{Type Decl, ,Nonterminal Symbols}).
4722 @end deffn
4723
4724 @deffn {Directive} %start
4725 Specify the grammar's start symbol (@pxref{Start Decl, ,The
4726 Start-Symbol}).
4727 @end deffn
4728
4729 @deffn {Directive} %expect
4730 Declare the expected number of shift-reduce conflicts
4731 (@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
4732 @end deffn
4733
4734
4735 @sp 1
4736 @noindent
4737 In order to change the behavior of @command{bison}, use the following
4738 directives:
4739
4740 @deffn {Directive} %code @{@var{code}@}
4741 @findex %code
4742 This is the unqualified form of the @code{%code} directive.
4743 It inserts @var{code} verbatim at a language-dependent default location in the
4744 output@footnote{The default location is actually skeleton-dependent;
4745 writers of non-standard skeletons however should choose the default location
4746 consistently with the behavior of the standard Bison skeletons.}.
4747
4748 @cindex Prologue
4749 For C/C++, the default location is the parser source code
4750 file after the usual contents of the parser header file.
4751 Thus, @code{%code} replaces the traditional Yacc prologue,
4752 @code{%@{@var{code}%@}}, for most purposes.
4753 For a detailed discussion, see @ref{Prologue Alternatives}.
4754
4755 For Java, the default location is inside the parser class.
4756
4757 (Like all the Yacc prologue alternatives, this directive is experimental.
4758 More user feedback will help to determine whether it should become a permanent
4759 feature.)
4760 @end deffn
4761
4762 @deffn {Directive} %code @var{qualifier} @{@var{code}@}
4763 This is the qualified form of the @code{%code} directive.
4764 If you need to specify location-sensitive verbatim @var{code} that does not
4765 belong at the default location selected by the unqualified @code{%code} form,
4766 use this form instead.
4767
4768 @var{qualifier} identifies the purpose of @var{code} and thus the location(s)
4769 where Bison should generate it.
4770 Not all values of @var{qualifier} are available for all target languages:
4771
4772 @itemize @bullet
4773 @item requires
4774 @findex %code requires
4775
4776 @itemize @bullet
4777 @item Language(s): C, C++
4778
4779 @item Purpose: This is the best place to write dependency code required for
4780 @code{YYSTYPE} and @code{YYLTYPE}.
4781 In other words, it's the best place to define types referenced in @code{%union}
4782 directives, and it's the best place to override Bison's default @code{YYSTYPE}
4783 and @code{YYLTYPE} definitions.
4784
4785 @item Location(s): The parser header file and the parser source code file
4786 before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE} definitions.
4787 @end itemize
4788
4789 @item provides
4790 @findex %code provides
4791
4792 @itemize @bullet
4793 @item Language(s): C, C++
4794
4795 @item Purpose: This is the best place to write additional definitions and
4796 declarations that should be provided to other modules.
4797
4798 @item Location(s): The parser header file and the parser source code file after
4799 the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and token definitions.
4800 @end itemize
4801
4802 @item top
4803 @findex %code top
4804
4805 @itemize @bullet
4806 @item Language(s): C, C++
4807
4808 @item Purpose: The unqualified @code{%code} or @code{%code requires} should
4809 usually be more appropriate than @code{%code top}.
4810 However, occasionally it is necessary to insert code much nearer the top of the
4811 parser source code file.
4812 For example:
4813
4814 @smallexample
4815 %code top @{
4816 #define _GNU_SOURCE
4817 #include <stdio.h>
4818 @}
4819 @end smallexample
4820
4821 @item Location(s): Near the top of the parser source code file.
4822 @end itemize
4823
4824 @item imports
4825 @findex %code imports
4826
4827 @itemize @bullet
4828 @item Language(s): Java
4829
4830 @item Purpose: This is the best place to write Java import directives.
4831
4832 @item Location(s): The parser Java file after any Java package directive and
4833 before any class definitions.
4834 @end itemize
4835 @end itemize
4836
4837 (Like all the Yacc prologue alternatives, this directive is experimental.
4838 More user feedback will help to determine whether it should become a permanent
4839 feature.)
4840
4841 @cindex Prologue
4842 For a detailed discussion of how to use @code{%code} in place of the
4843 traditional Yacc prologue for C/C++, see @ref{Prologue Alternatives}.
4844 @end deffn
4845
4846 @deffn {Directive} %debug
4847 Instrument the output parser for traces. Obsoleted by @samp{%define
4848 parse.trace}.
4849 @xref{Tracing, ,Tracing Your Parser}.
4850 @end deffn
4851
4852 @deffn {Directive} %define @var{variable}
4853 @deffnx {Directive} %define @var{variable} "@var{value}"
4854 Define a variable to adjust Bison's behavior.
4855 The possible choices for @var{variable}, as well as their meanings, depend on
4856 the selected target language and/or the parser skeleton (@pxref{Decl
4857 Summary,,%language}, @pxref{Decl Summary,,%skeleton}).
4858
4859 Bison will warn if a @var{variable} is defined multiple times.
4860
4861 Omitting @code{"@var{value}"} is always equivalent to specifying it as
4862 @code{""}.
4863
4864 Some @var{variable}s may be used as Booleans.
4865 In this case, Bison will complain if the variable definition does not meet one
4866 of the following four conditions:
4867
4868 @enumerate
4869 @item @code{"@var{value}"} is @code{"true"}
4870
4871 @item @code{"@var{value}"} is omitted (or is @code{""}).
4872 This is equivalent to @code{"true"}.
4873
4874 @item @code{"@var{value}"} is @code{"false"}.
4875
4876 @item @var{variable} is never defined.
4877 In this case, Bison selects a default value, which may depend on the selected
4878 target language and/or parser skeleton.
4879 @end enumerate
4880
4881 Some of the accepted @var{variable}s are:
4882
4883 @table @code
4884 @item api.pure
4885 @findex %define api.pure
4886
4887 @itemize @bullet
4888 @item Language(s): C
4889
4890 @item Purpose: Request a pure (reentrant) parser program.
4891 @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
4892
4893 @item Accepted Values: Boolean
4894
4895 @item Default Value: @code{"false"}
4896 @end itemize
4897 @c api.pure
4898
4899 @item api.push-pull
4900 @findex %define api.push-pull
4901
4902 @itemize @bullet
4903 @item Language(s): C (deterministic parsers only)
4904
4905 @item Purpose: Requests a pull parser, a push parser, or both.
4906 @xref{Push Decl, ,A Push Parser}.
4907 (The current push parsing interface is experimental and may evolve.
4908 More user feedback will help to stabilize it.)
4909
4910 @item Accepted Values: @code{"pull"}, @code{"push"}, @code{"both"}
4911
4912 @item Default Value: @code{"pull"}
4913 @end itemize
4914 @c api.push-pull
4915
4916 @item error-verbose
4917 @findex %define error-verbose
4918 @itemize
4919 @item Languages(s):
4920 all.
4921 @item Purpose:
4922 Enable the generation of more verbose error messages than a instead of
4923 just plain @w{@code{"syntax error"}}. @xref{Error Reporting, ,The Error
4924 Reporting Function @code{yyerror}}.
4925 @item Accepted Values:
4926 Boolean
4927 @item Default Value:
4928 @code{false}
4929 @end itemize
4930 @c error-verbose
4931
4932
4933 @item lr.default-reductions
4934 @cindex default reductions
4935 @findex %define lr.default-reductions
4936 @cindex delayed syntax errors
4937 @cindex syntax errors delayed
4938
4939 @itemize @bullet
4940 @item Language(s): all
4941
4942 @item Purpose: Specifies the kind of states that are permitted to
4943 contain default reductions.
4944 That is, in such a state, Bison declares the reduction with the largest
4945 lookahead set to be the default reduction and then removes that
4946 lookahead set.
4947 The advantages of default reductions are discussed below.
4948 The disadvantage is that, when the generated parser encounters a
4949 syntactically unacceptable token, the parser might then perform
4950 unnecessary default reductions before it can detect the syntax error.
4951
4952 (This feature is experimental.
4953 More user feedback will help to stabilize it.)
4954
4955 @item Accepted Values:
4956 @itemize
4957 @item @code{"all"}.
4958 For @acronym{LALR} and @acronym{IELR} parsers (@pxref{Decl
4959 Summary,,lr.type}) by default, all states are permitted to contain
4960 default reductions.
4961 The advantage is that parser table sizes can be significantly reduced.
4962 The reason Bison does not by default attempt to address the disadvantage
4963 of delayed syntax error detection is that this disadvantage is already
4964 inherent in @acronym{LALR} and @acronym{IELR} parser tables.
4965 That is, unlike in a canonical @acronym{LR} state, the lookahead sets of
4966 reductions in an @acronym{LALR} or @acronym{IELR} state can contain
4967 tokens that are syntactically incorrect for some left contexts.
4968
4969 @item @code{"consistent"}.
4970 @cindex consistent states
4971 A consistent state is a state that has only one possible action.
4972 If that action is a reduction, then the parser does not need to request
4973 a lookahead token from the scanner before performing that action.
4974 However, the parser only recognizes the ability to ignore the lookahead
4975 token when such a reduction is encoded as a default reduction.
4976 Thus, if default reductions are permitted in and only in consistent
4977 states, then a canonical @acronym{LR} parser reports a syntax error as
4978 soon as it @emph{needs} the syntactically unacceptable token from the
4979 scanner.
4980
4981 @item @code{"accepting"}.
4982 @cindex accepting state
4983 By default, the only default reduction permitted in a canonical
4984 @acronym{LR} parser is the accept action in the accepting state, which
4985 the parser reaches only after reading all tokens from the input.
4986 Thus, the default canonical @acronym{LR} parser reports a syntax error
4987 as soon as it @emph{reaches} the syntactically unacceptable token
4988 without performing any extra reductions.
4989 @end itemize
4990
4991 @item Default Value:
4992 @itemize
4993 @item @code{"accepting"} if @code{lr.type} is @code{"canonical LR"}.
4994 @item @code{"all"} otherwise.
4995 @end itemize
4996 @end itemize
4997
4998 @item lr.keep-unreachable-states
4999 @findex %define lr.keep-unreachable-states
5000
5001 @itemize @bullet
5002 @item Language(s): all
5003
5004 @item Purpose: Requests that Bison allow unreachable parser states to remain in
5005 the parser tables.
5006 Bison considers a state to be unreachable if there exists no sequence of
5007 transitions from the start state to that state.
5008 A state can become unreachable during conflict resolution if Bison disables a
5009 shift action leading to it from a predecessor state.
5010 Keeping unreachable states is sometimes useful for analysis purposes, but they
5011 are useless in the generated parser.
5012
5013 @item Accepted Values: Boolean
5014
5015 @item Default Value: @code{"false"}
5016
5017 @item Caveats:
5018
5019 @itemize @bullet
5020
5021 @item Unreachable states may contain conflicts and may use rules not used in
5022 any other state.
5023 Thus, keeping unreachable states may induce warnings that are irrelevant to
5024 your parser's behavior, and it may eliminate warnings that are relevant.
5025 Of course, the change in warnings may actually be relevant to a parser table
5026 analysis that wants to keep unreachable states, so this behavior will likely
5027 remain in future Bison releases.
5028
5029 @item While Bison is able to remove unreachable states, it is not guaranteed to
5030 remove other kinds of useless states.
5031 Specifically, when Bison disables reduce actions during conflict resolution,
5032 some goto actions may become useless, and thus some additional states may
5033 become useless.
5034 If Bison were to compute which goto actions were useless and then disable those
5035 actions, it could identify such states as unreachable and then remove those
5036 states.
5037 However, Bison does not compute which goto actions are useless.
5038 @end itemize
5039 @end itemize
5040 @c lr.keep-unreachable-states
5041
5042 @item lr.type
5043 @findex %define lr.type
5044 @cindex @acronym{LALR}
5045 @cindex @acronym{IELR}
5046 @cindex @acronym{LR}
5047
5048 @itemize @bullet
5049 @item Language(s): all
5050
5051 @item Purpose: Specifies the type of parser tables within the
5052 @acronym{LR}(1) family.
5053 (This feature is experimental.
5054 More user feedback will help to stabilize it.)
5055
5056 @item Accepted Values:
5057 @itemize
5058 @item @code{"LALR"}.
5059 While Bison generates @acronym{LALR} parser tables by default for
5060 historical reasons, @acronym{IELR} or canonical @acronym{LR} is almost
5061 always preferable for deterministic parsers.
5062 The trouble is that @acronym{LALR} parser tables can suffer from
5063 mysterious conflicts and thus may not accept the full set of sentences
5064 that @acronym{IELR} and canonical @acronym{LR} accept.
5065 @xref{Mystery Conflicts}, for details.
5066 However, there are at least two scenarios where @acronym{LALR} may be
5067 worthwhile:
5068 @itemize
5069 @cindex @acronym{GLR} with @acronym{LALR}
5070 @item When employing @acronym{GLR} parsers (@pxref{GLR Parsers}), if you
5071 do not resolve any conflicts statically (for example, with @code{%left}
5072 or @code{%prec}), then the parser explores all potential parses of any
5073 given input.
5074 In this case, the use of @acronym{LALR} parser tables is guaranteed not
5075 to alter the language accepted by the parser.
5076 @acronym{LALR} parser tables are the smallest parser tables Bison can
5077 currently generate, so they may be preferable.
5078
5079 @item Occasionally during development, an especially malformed grammar
5080 with a major recurring flaw may severely impede the @acronym{IELR} or
5081 canonical @acronym{LR} parser table generation algorithm.
5082 @acronym{LALR} can be a quick way to generate parser tables in order to
5083 investigate such problems while ignoring the more subtle differences
5084 from @acronym{IELR} and canonical @acronym{LR}.
5085 @end itemize
5086
5087 @item @code{"IELR"}.
5088 @acronym{IELR} is a minimal @acronym{LR} algorithm.
5089 That is, given any grammar (@acronym{LR} or non-@acronym{LR}),
5090 @acronym{IELR} and canonical @acronym{LR} always accept exactly the same
5091 set of sentences.
5092 However, as for @acronym{LALR}, the number of parser states is often an
5093 order of magnitude less for @acronym{IELR} than for canonical
5094 @acronym{LR}.
5095 More importantly, because canonical @acronym{LR}'s extra parser states
5096 may contain duplicate conflicts in the case of non-@acronym{LR}
5097 grammars, the number of conflicts for @acronym{IELR} is often an order
5098 of magnitude less as well.
5099 This can significantly reduce the complexity of developing of a grammar.
5100
5101 @item @code{"canonical LR"}.
5102 @cindex delayed syntax errors
5103 @cindex syntax errors delayed
5104 The only advantage of canonical @acronym{LR} over @acronym{IELR} is
5105 that, for every left context of every canonical @acronym{LR} state, the
5106 set of tokens accepted by that state is the exact set of tokens that is
5107 syntactically acceptable in that left context.
5108 Thus, the only difference in parsing behavior is that the canonical
5109 @acronym{LR} parser can report a syntax error as soon as possible
5110 without performing any unnecessary reductions.
5111 @xref{Decl Summary,,lr.default-reductions}, for further details.
5112 Even when canonical @acronym{LR} behavior is ultimately desired,
5113 @acronym{IELR}'s elimination of duplicate conflicts should still
5114 facilitate the development of a grammar.
5115 @end itemize
5116
5117 @item Default Value: @code{"LALR"}
5118 @end itemize
5119
5120 @item namespace
5121 @findex %define namespace
5122
5123 @itemize
5124 @item Languages(s): C++
5125
5126 @item Purpose: Specifies the namespace for the parser class.
5127 For example, if you specify:
5128
5129 @smallexample
5130 %define namespace "foo::bar"
5131 @end smallexample
5132
5133 Bison uses @code{foo::bar} verbatim in references such as:
5134
5135 @smallexample
5136 foo::bar::parser::semantic_type
5137 @end smallexample
5138
5139 However, to open a namespace, Bison removes any leading @code{::} and then
5140 splits on any remaining occurrences:
5141
5142 @smallexample
5143 namespace foo @{ namespace bar @{
5144 class position;
5145 class location;
5146 @} @}
5147 @end smallexample
5148
5149 @item Accepted Values: Any absolute or relative C++ namespace reference without
5150 a trailing @code{"::"}.
5151 For example, @code{"foo"} or @code{"::foo::bar"}.
5152
5153 @item Default Value: The value specified by @code{%name-prefix}, which defaults
5154 to @code{yy}.
5155 This usage of @code{%name-prefix} is for backward compatibility and can be
5156 confusing since @code{%name-prefix} also specifies the textual prefix for the
5157 lexical analyzer function.
5158 Thus, if you specify @code{%name-prefix}, it is best to also specify
5159 @code{%define namespace} so that @code{%name-prefix} @emph{only} affects the
5160 lexical analyzer function.
5161 For example, if you specify:
5162
5163 @smallexample
5164 %define namespace "foo"
5165 %name-prefix "bar::"
5166 @end smallexample
5167
5168 The parser namespace is @code{foo} and @code{yylex} is referenced as
5169 @code{bar::lex}.
5170 @end itemize
5171 @c namespace
5172
5173 @item parse.assert
5174 @findex %define parse.assert
5175
5176 @itemize
5177 @item Languages(s): C++
5178
5179 @item Purpose: Issue runtime assertions to catch invalid uses.
5180 In C++, when variants are used, symbols must be constructed and
5181 destroyed properly. This option checks these constraints.
5182
5183 @item Accepted Values: Boolean
5184
5185 @item Default Value: @code{false}
5186 @end itemize
5187 @c parse.assert
5188
5189 @item parse.trace
5190 @findex %define parse.trace
5191
5192 @itemize
5193 @item Languages(s): C, C++
5194
5195 @item Purpose: Require parser instrumentation for tracing.
5196 In C/C++, define the macro @code{YYDEBUG} to 1 in the parser file if it
5197 is not already defined, so that the debugging facilities are compiled.
5198 @xref{Tracing, ,Tracing Your Parser}.
5199
5200 @item Accepted Values: Boolean
5201
5202 @item Default Value: @code{false}
5203 @end itemize
5204 @end table
5205 @c parse.trace
5206 @end deffn
5207 @c %define
5208
5209 @deffn {Directive} %defines
5210 Write a header file containing macro definitions for the token type
5211 names defined in the grammar as well as a few other declarations.
5212 If the parser output file is named @file{@var{name}.c} then this file
5213 is named @file{@var{name}.h}.
5214
5215 For C parsers, the output header declares @code{YYSTYPE} unless
5216 @code{YYSTYPE} is already defined as a macro or you have used a
5217 @code{<@var{type}>} tag without using @code{%union}.
5218 Therefore, if you are using a @code{%union}
5219 (@pxref{Multiple Types, ,More Than One Value Type}) with components that
5220 require other definitions, or if you have defined a @code{YYSTYPE} macro
5221 or type definition
5222 (@pxref{Value Type, ,Data Types of Semantic Values}), you need to
5223 arrange for these definitions to be propagated to all modules, e.g., by
5224 putting them in a prerequisite header that is included both by your
5225 parser and by any other module that needs @code{YYSTYPE}.
5226
5227 Unless your parser is pure, the output header declares @code{yylval}
5228 as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
5229 Parser}.
5230
5231 If you have also used locations, the output header declares
5232 @code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
5233 the @code{YYSTYPE} macro and @code{yylval}. @xref{Locations, ,Tracking
5234 Locations}.
5235
5236 This output file is normally essential if you wish to put the definition
5237 of @code{yylex} in a separate source file, because @code{yylex}
5238 typically needs to be able to refer to the above-mentioned declarations
5239 and to the token type codes. @xref{Token Values, ,Semantic Values of
5240 Tokens}.
5241
5242 @findex %code requires
5243 @findex %code provides
5244 If you have declared @code{%code requires} or @code{%code provides}, the output
5245 header also contains their code.
5246 @xref{Decl Summary, ,%code}.
5247 @end deffn
5248
5249 @deffn {Directive} %defines @var{defines-file}
5250 Same as above, but save in the file @var{defines-file}.
5251 @end deffn
5252
5253 @deffn {Directive} %destructor
5254 Specify how the parser should reclaim the memory associated to
5255 discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
5256 @end deffn
5257
5258 @deffn {Directive} %file-prefix "@var{prefix}"
5259 Specify a prefix to use for all Bison output file names. The names are
5260 chosen as if the input file were named @file{@var{prefix}.y}.
5261 @end deffn
5262
5263 @deffn {Directive} %language "@var{language}"
5264 Specify the programming language for the generated parser. Currently
5265 supported languages include C, C++, and Java.
5266 @var{language} is case-insensitive.
5267
5268 This directive is experimental and its effect may be modified in future
5269 releases.
5270 @end deffn
5271
5272 @deffn {Directive} %locations
5273 Generate the code processing the locations (@pxref{Action Features,
5274 ,Special Features for Use in Actions}). This mode is enabled as soon as
5275 the grammar uses the special @samp{@@@var{n}} tokens, but if your
5276 grammar does not use it, using @samp{%locations} allows for more
5277 accurate syntax error messages.
5278 @end deffn
5279
5280 @deffn {Directive} %name-prefix "@var{prefix}"
5281 Rename the external symbols used in the parser so that they start with
5282 @var{prefix} instead of @samp{yy}. The precise list of symbols renamed
5283 in C parsers
5284 is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
5285 @code{yylval}, @code{yychar}, @code{yydebug}, and
5286 (if locations are used) @code{yylloc}. If you use a push parser,
5287 @code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5288 @code{yypstate_new} and @code{yypstate_delete} will
5289 also be renamed. For example, if you use @samp{%name-prefix "c_"}, the
5290 names become @code{c_parse}, @code{c_lex}, and so on.
5291 For C++ parsers, see the @code{%define namespace} documentation in this
5292 section.
5293 @xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
5294 @end deffn
5295
5296 @ifset defaultprec
5297 @deffn {Directive} %no-default-prec
5298 Do not assign a precedence to rules lacking an explicit @code{%prec}
5299 modifier (@pxref{Contextual Precedence, ,Context-Dependent
5300 Precedence}).
5301 @end deffn
5302 @end ifset
5303
5304 @deffn {Directive} %no-lines
5305 Don't generate any @code{#line} preprocessor commands in the parser
5306 file. Ordinarily Bison writes these commands in the parser file so that
5307 the C compiler and debuggers will associate errors and object code with
5308 your source file (the grammar file). This directive causes them to
5309 associate errors with the parser file, treating it an independent source
5310 file in its own right.
5311 @end deffn
5312
5313 @deffn {Directive} %output "@var{file}"
5314 Specify @var{file} for the parser file.
5315 @end deffn
5316
5317 @deffn {Directive} %pure-parser
5318 Deprecated version of @code{%define api.pure} (@pxref{Decl Summary, ,%define}),
5319 for which Bison is more careful to warn about unreasonable usage.
5320 @end deffn
5321
5322 @deffn {Directive} %require "@var{version}"
5323 Require version @var{version} or higher of Bison. @xref{Require Decl, ,
5324 Require a Version of Bison}.
5325 @end deffn
5326
5327 @deffn {Directive} %skeleton "@var{file}"
5328 Specify the skeleton to use.
5329
5330 @c You probably don't need this option unless you are developing Bison.
5331 @c You should use @code{%language} if you want to specify the skeleton for a
5332 @c different language, because it is clearer and because it will always choose the
5333 @c correct skeleton for non-deterministic or push parsers.
5334
5335 If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5336 file in the Bison installation directory.
5337 If it does, @var{file} is an absolute file name or a file name relative to the
5338 directory of the grammar file.
5339 This is similar to how most shells resolve commands.
5340 @end deffn
5341
5342 @deffn {Directive} %token-table
5343 Generate an array of token names in the parser file. The name of the
5344 array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
5345 token whose internal Bison token code number is @var{i}. The first
5346 three elements of @code{yytname} correspond to the predefined tokens
5347 @code{"$end"},
5348 @code{"error"}, and @code{"$undefined"}; after these come the symbols
5349 defined in the grammar file.
5350
5351 The name in the table includes all the characters needed to represent
5352 the token in Bison. For single-character literals and literal
5353 strings, this includes the surrounding quoting characters and any
5354 escape sequences. For example, the Bison single-character literal
5355 @code{'+'} corresponds to a three-character name, represented in C as
5356 @code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
5357 corresponds to a five-character name, represented in C as
5358 @code{"\"\\\\/\""}.
5359
5360 When you specify @code{%token-table}, Bison also generates macro
5361 definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
5362 @code{YYNRULES}, and @code{YYNSTATES}:
5363
5364 @table @code
5365 @item YYNTOKENS
5366 The highest token number, plus one.
5367 @item YYNNTS
5368 The number of nonterminal symbols.
5369 @item YYNRULES
5370 The number of grammar rules,
5371 @item YYNSTATES
5372 The number of parser states (@pxref{Parser States}).
5373 @end table
5374 @end deffn
5375
5376 @deffn {Directive} %verbose
5377 Write an extra output file containing verbose descriptions of the
5378 parser states and what is done for each type of lookahead token in
5379 that state. @xref{Understanding, , Understanding Your Parser}, for more
5380 information.
5381 @end deffn
5382
5383 @deffn {Directive} %yacc
5384 Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
5385 including its naming conventions. @xref{Bison Options}, for more.
5386 @end deffn
5387
5388
5389 @node Multiple Parsers
5390 @section Multiple Parsers in the Same Program
5391
5392 Most programs that use Bison parse only one language and therefore contain
5393 only one Bison parser. But what if you want to parse more than one
5394 language with the same program? Then you need to avoid a name conflict
5395 between different definitions of @code{yyparse}, @code{yylval}, and so on.
5396
5397 The easy way to do this is to use the option @samp{-p @var{prefix}}
5398 (@pxref{Invocation, ,Invoking Bison}). This renames the interface
5399 functions and variables of the Bison parser to start with @var{prefix}
5400 instead of @samp{yy}. You can use this to give each parser distinct
5401 names that do not conflict.
5402
5403 The precise list of symbols renamed is @code{yyparse}, @code{yylex},
5404 @code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
5405 @code{yychar} and @code{yydebug}. If you use a push parser,
5406 @code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5407 @code{yypstate_new} and @code{yypstate_delete} will also be renamed.
5408 For example, if you use @samp{-p c}, the names become @code{cparse},
5409 @code{clex}, and so on.
5410
5411 @strong{All the other variables and macros associated with Bison are not
5412 renamed.} These others are not global; there is no conflict if the same
5413 name is used in different parsers. For example, @code{YYSTYPE} is not
5414 renamed, but defining this in different ways in different parsers causes
5415 no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
5416
5417 The @samp{-p} option works by adding macro definitions to the beginning
5418 of the parser source file, defining @code{yyparse} as
5419 @code{@var{prefix}parse}, and so on. This effectively substitutes one
5420 name for the other in the entire parser file.
5421
5422 @node Interface
5423 @chapter Parser C-Language Interface
5424 @cindex C-language interface
5425 @cindex interface
5426
5427 The Bison parser is actually a C function named @code{yyparse}. Here we
5428 describe the interface conventions of @code{yyparse} and the other
5429 functions that it needs to use.
5430
5431 Keep in mind that the parser uses many C identifiers starting with
5432 @samp{yy} and @samp{YY} for internal purposes. If you use such an
5433 identifier (aside from those in this manual) in an action or in epilogue
5434 in the grammar file, you are likely to run into trouble.
5435
5436 @menu
5437 * Parser Function:: How to call @code{yyparse} and what it returns.
5438 * Push Parser Function:: How to call @code{yypush_parse} and what it returns.
5439 * Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
5440 * Parser Create Function:: How to call @code{yypstate_new} and what it returns.
5441 * Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
5442 * Lexical:: You must supply a function @code{yylex}
5443 which reads tokens.
5444 * Error Reporting:: You must supply a function @code{yyerror}.
5445 * Action Features:: Special features for use in actions.
5446 * Internationalization:: How to let the parser speak in the user's
5447 native language.
5448 @end menu
5449
5450 @node Parser Function
5451 @section The Parser Function @code{yyparse}
5452 @findex yyparse
5453
5454 You call the function @code{yyparse} to cause parsing to occur. This
5455 function reads tokens, executes actions, and ultimately returns when it
5456 encounters end-of-input or an unrecoverable syntax error. You can also
5457 write an action which directs @code{yyparse} to return immediately
5458 without reading further.
5459
5460
5461 @deftypefun int yyparse (void)
5462 The value returned by @code{yyparse} is 0 if parsing was successful (return
5463 is due to end-of-input).
5464
5465 The value is 1 if parsing failed because of invalid input, i.e., input
5466 that contains a syntax error or that causes @code{YYABORT} to be
5467 invoked.
5468
5469 The value is 2 if parsing failed due to memory exhaustion.
5470 @end deftypefun
5471
5472 In an action, you can cause immediate return from @code{yyparse} by using
5473 these macros:
5474
5475 @defmac YYACCEPT
5476 @findex YYACCEPT
5477 Return immediately with value 0 (to report success).
5478 @end defmac
5479
5480 @defmac YYABORT
5481 @findex YYABORT
5482 Return immediately with value 1 (to report failure).
5483 @end defmac
5484
5485 If you use a reentrant parser, you can optionally pass additional
5486 parameter information to it in a reentrant way. To do so, use the
5487 declaration @code{%parse-param}:
5488
5489 @deffn {Directive} %parse-param @{@var{argument-declaration}@}
5490 @findex %parse-param
5491 Declare that an argument declared by the braced-code
5492 @var{argument-declaration} is an additional @code{yyparse} argument.
5493 The @var{argument-declaration} is used when declaring
5494 functions or prototypes. The last identifier in
5495 @var{argument-declaration} must be the argument name.
5496 @end deffn
5497
5498 Here's an example. Write this in the parser:
5499
5500 @example
5501 %parse-param @{int *nastiness@}
5502 %parse-param @{int *randomness@}
5503 @end example
5504
5505 @noindent
5506 Then call the parser like this:
5507
5508 @example
5509 @{
5510 int nastiness, randomness;
5511 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
5512 value = yyparse (&nastiness, &randomness);
5513 @dots{}
5514 @}
5515 @end example
5516
5517 @noindent
5518 In the grammar actions, use expressions like this to refer to the data:
5519
5520 @example
5521 exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
5522 @end example
5523
5524 @node Push Parser Function
5525 @section The Push Parser Function @code{yypush_parse}
5526 @findex yypush_parse
5527
5528 (The current push parsing interface is experimental and may evolve.
5529 More user feedback will help to stabilize it.)
5530
5531 You call the function @code{yypush_parse} to parse a single token. This
5532 function is available if either the @code{%define api.push-pull "push"} or
5533 @code{%define api.push-pull "both"} declaration is used.
5534 @xref{Push Decl, ,A Push Parser}.
5535
5536 @deftypefun int yypush_parse (yypstate *yyps)
5537 The value returned by @code{yypush_parse} is the same as for yyparse with the
5538 following exception. @code{yypush_parse} will return YYPUSH_MORE if more input
5539 is required to finish parsing the grammar.
5540 @end deftypefun
5541
5542 @node Pull Parser Function
5543 @section The Pull Parser Function @code{yypull_parse}
5544 @findex yypull_parse
5545
5546 (The current push parsing interface is experimental and may evolve.
5547 More user feedback will help to stabilize it.)
5548
5549 You call the function @code{yypull_parse} to parse the rest of the input
5550 stream. This function is available if the @code{%define api.push-pull "both"}
5551 declaration is used.
5552 @xref{Push Decl, ,A Push Parser}.
5553
5554 @deftypefun int yypull_parse (yypstate *yyps)
5555 The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
5556 @end deftypefun
5557
5558 @node Parser Create Function
5559 @section The Parser Create Function @code{yystate_new}
5560 @findex yypstate_new
5561
5562 (The current push parsing interface is experimental and may evolve.
5563 More user feedback will help to stabilize it.)
5564
5565 You call the function @code{yypstate_new} to create a new parser instance.
5566 This function is available if either the @code{%define api.push-pull "push"} or
5567 @code{%define api.push-pull "both"} declaration is used.
5568 @xref{Push Decl, ,A Push Parser}.
5569
5570 @deftypefun yypstate *yypstate_new (void)
5571 The fuction will return a valid parser instance if there was memory available
5572 or 0 if no memory was available.
5573 In impure mode, it will also return 0 if a parser instance is currently
5574 allocated.
5575 @end deftypefun
5576
5577 @node Parser Delete Function
5578 @section The Parser Delete Function @code{yystate_delete}
5579 @findex yypstate_delete
5580
5581 (The current push parsing interface is experimental and may evolve.
5582 More user feedback will help to stabilize it.)
5583
5584 You call the function @code{yypstate_delete} to delete a parser instance.
5585 function is available if either the @code{%define api.push-pull "push"} or
5586 @code{%define api.push-pull "both"} declaration is used.
5587 @xref{Push Decl, ,A Push Parser}.
5588
5589 @deftypefun void yypstate_delete (yypstate *yyps)
5590 This function will reclaim the memory associated with a parser instance.
5591 After this call, you should no longer attempt to use the parser instance.
5592 @end deftypefun
5593
5594 @node Lexical
5595 @section The Lexical Analyzer Function @code{yylex}
5596 @findex yylex
5597 @cindex lexical analyzer
5598
5599 The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
5600 the input stream and returns them to the parser. Bison does not create
5601 this function automatically; you must write it so that @code{yyparse} can
5602 call it. The function is sometimes referred to as a lexical scanner.
5603
5604 In simple programs, @code{yylex} is often defined at the end of the Bison
5605 grammar file. If @code{yylex} is defined in a separate source file, you
5606 need to arrange for the token-type macro definitions to be available there.
5607 To do this, use the @samp{-d} option when you run Bison, so that it will
5608 write these macro definitions into a separate header file
5609 @file{@var{name}.tab.h} which you can include in the other source files
5610 that need it. @xref{Invocation, ,Invoking Bison}.
5611
5612 @menu
5613 * Calling Convention:: How @code{yyparse} calls @code{yylex}.
5614 * Token Values:: How @code{yylex} must return the semantic value
5615 of the token it has read.
5616 * Token Locations:: How @code{yylex} must return the text location
5617 (line number, etc.) of the token, if the
5618 actions want that.
5619 * Pure Calling:: How the calling convention differs in a pure parser
5620 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
5621 @end menu
5622
5623 @node Calling Convention
5624 @subsection Calling Convention for @code{yylex}
5625
5626 The value that @code{yylex} returns must be the positive numeric code
5627 for the type of token it has just found; a zero or negative value
5628 signifies end-of-input.
5629
5630 When a token is referred to in the grammar rules by a name, that name
5631 in the parser file becomes a C macro whose definition is the proper
5632 numeric code for that token type. So @code{yylex} can use the name
5633 to indicate that type. @xref{Symbols}.
5634
5635 When a token is referred to in the grammar rules by a character literal,
5636 the numeric code for that character is also the code for the token type.
5637 So @code{yylex} can simply return that character code, possibly converted
5638 to @code{unsigned char} to avoid sign-extension. The null character
5639 must not be used this way, because its code is zero and that
5640 signifies end-of-input.
5641
5642 Here is an example showing these things:
5643
5644 @example
5645 int
5646 yylex (void)
5647 @{
5648 @dots{}
5649 if (c == EOF) /* Detect end-of-input. */
5650 return 0;
5651 @dots{}
5652 if (c == '+' || c == '-')
5653 return c; /* Assume token type for `+' is '+'. */
5654 @dots{}
5655 return INT; /* Return the type of the token. */
5656 @dots{}
5657 @}
5658 @end example
5659
5660 @noindent
5661 This interface has been designed so that the output from the @code{lex}
5662 utility can be used without change as the definition of @code{yylex}.
5663
5664 If the grammar uses literal string tokens, there are two ways that
5665 @code{yylex} can determine the token type codes for them:
5666
5667 @itemize @bullet
5668 @item
5669 If the grammar defines symbolic token names as aliases for the
5670 literal string tokens, @code{yylex} can use these symbolic names like
5671 all others. In this case, the use of the literal string tokens in
5672 the grammar file has no effect on @code{yylex}.
5673
5674 @item
5675 @code{yylex} can find the multicharacter token in the @code{yytname}
5676 table. The index of the token in the table is the token type's code.
5677 The name of a multicharacter token is recorded in @code{yytname} with a
5678 double-quote, the token's characters, and another double-quote. The
5679 token's characters are escaped as necessary to be suitable as input
5680 to Bison.
5681
5682 Here's code for looking up a multicharacter token in @code{yytname},
5683 assuming that the characters of the token are stored in
5684 @code{token_buffer}, and assuming that the token does not contain any
5685 characters like @samp{"} that require escaping.
5686
5687 @smallexample
5688 for (i = 0; i < YYNTOKENS; i++)
5689 @{
5690 if (yytname[i] != 0
5691 && yytname[i][0] == '"'
5692 && ! strncmp (yytname[i] + 1, token_buffer,
5693 strlen (token_buffer))
5694 && yytname[i][strlen (token_buffer) + 1] == '"'
5695 && yytname[i][strlen (token_buffer) + 2] == 0)
5696 break;
5697 @}
5698 @end smallexample
5699
5700 The @code{yytname} table is generated only if you use the
5701 @code{%token-table} declaration. @xref{Decl Summary}.
5702 @end itemize
5703
5704 @node Token Values
5705 @subsection Semantic Values of Tokens
5706
5707 @vindex yylval
5708 In an ordinary (nonreentrant) parser, the semantic value of the token must
5709 be stored into the global variable @code{yylval}. When you are using
5710 just one data type for semantic values, @code{yylval} has that type.
5711 Thus, if the type is @code{int} (the default), you might write this in
5712 @code{yylex}:
5713
5714 @example
5715 @group
5716 @dots{}
5717 yylval = value; /* Put value onto Bison stack. */
5718 return INT; /* Return the type of the token. */
5719 @dots{}
5720 @end group
5721 @end example
5722
5723 When you are using multiple data types, @code{yylval}'s type is a union
5724 made from the @code{%union} declaration (@pxref{Union Decl, ,The
5725 Collection of Value Types}). So when you store a token's value, you
5726 must use the proper member of the union. If the @code{%union}
5727 declaration looks like this:
5728
5729 @example
5730 @group
5731 %union @{
5732 int intval;
5733 double val;
5734 symrec *tptr;
5735 @}
5736 @end group
5737 @end example
5738
5739 @noindent
5740 then the code in @code{yylex} might look like this:
5741
5742 @example
5743 @group
5744 @dots{}
5745 yylval.intval = value; /* Put value onto Bison stack. */
5746 return INT; /* Return the type of the token. */
5747 @dots{}
5748 @end group
5749 @end example
5750
5751 @node Token Locations
5752 @subsection Textual Locations of Tokens
5753
5754 @vindex yylloc
5755 If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
5756 Tracking Locations}) in actions to keep track of the textual locations
5757 of tokens and groupings, then you must provide this information in
5758 @code{yylex}. The function @code{yyparse} expects to find the textual
5759 location of a token just parsed in the global variable @code{yylloc}.
5760 So @code{yylex} must store the proper data in that variable.
5761
5762 By default, the value of @code{yylloc} is a structure and you need only
5763 initialize the members that are going to be used by the actions. The
5764 four members are called @code{first_line}, @code{first_column},
5765 @code{last_line} and @code{last_column}. Note that the use of this
5766 feature makes the parser noticeably slower.
5767
5768 @tindex YYLTYPE
5769 The data type of @code{yylloc} has the name @code{YYLTYPE}.
5770
5771 @node Pure Calling
5772 @subsection Calling Conventions for Pure Parsers
5773
5774 When you use the Bison declaration @code{%define api.pure} to request a
5775 pure, reentrant parser, the global communication variables @code{yylval}
5776 and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
5777 Parser}.) In such parsers the two global variables are replaced by
5778 pointers passed as arguments to @code{yylex}. You must declare them as
5779 shown here, and pass the information back by storing it through those
5780 pointers.
5781
5782 @example
5783 int
5784 yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
5785 @{
5786 @dots{}
5787 *lvalp = value; /* Put value onto Bison stack. */
5788 return INT; /* Return the type of the token. */
5789 @dots{}
5790 @}
5791 @end example
5792
5793 If the grammar file does not use the @samp{@@} constructs to refer to
5794 textual locations, then the type @code{YYLTYPE} will not be defined. In
5795 this case, omit the second argument; @code{yylex} will be called with
5796 only one argument.
5797
5798
5799 If you wish to pass the additional parameter data to @code{yylex}, use
5800 @code{%lex-param} just like @code{%parse-param} (@pxref{Parser
5801 Function}).
5802
5803 @deffn {Directive} lex-param @{@var{argument-declaration}@}
5804 @findex %lex-param
5805 Declare that the braced-code @var{argument-declaration} is an
5806 additional @code{yylex} argument declaration.
5807 @end deffn
5808
5809 For instance:
5810
5811 @example
5812 %parse-param @{int *nastiness@}
5813 %lex-param @{int *nastiness@}
5814 %parse-param @{int *randomness@}
5815 @end example
5816
5817 @noindent
5818 results in the following signature:
5819
5820 @example
5821 int yylex (int *nastiness);
5822 int yyparse (int *nastiness, int *randomness);
5823 @end example
5824
5825 If @code{%define api.pure} is added:
5826
5827 @example
5828 int yylex (YYSTYPE *lvalp, int *nastiness);
5829 int yyparse (int *nastiness, int *randomness);
5830 @end example
5831
5832 @noindent
5833 and finally, if both @code{%define api.pure} and @code{%locations} are used:
5834
5835 @example
5836 int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
5837 int yyparse (int *nastiness, int *randomness);
5838 @end example
5839
5840 @node Error Reporting
5841 @section The Error Reporting Function @code{yyerror}
5842 @cindex error reporting function
5843 @findex yyerror
5844 @cindex parse error
5845 @cindex syntax error
5846
5847 The Bison parser detects a @dfn{syntax error} or @dfn{parse error}
5848 whenever it reads a token which cannot satisfy any syntax rule. An
5849 action in the grammar can also explicitly proclaim an error, using the
5850 macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
5851 in Actions}).
5852
5853 The Bison parser expects to report the error by calling an error
5854 reporting function named @code{yyerror}, which you must supply. It is
5855 called by @code{yyparse} whenever a syntax error is found, and it
5856 receives one argument. For a syntax error, the string is normally
5857 @w{@code{"syntax error"}}.
5858
5859 @findex %define error-verbose
5860 If you invoke the directive @code{%define error-verbose} in the Bison
5861 declarations section (@pxref{Bison Declarations, ,The Bison Declarations
5862 Section}), then Bison provides a more verbose and specific error message
5863 string instead of just plain @w{@code{"syntax error"}}.
5864
5865 The parser can detect one other kind of error: memory exhaustion. This
5866 can happen when the input contains constructions that are very deeply
5867 nested. It isn't likely you will encounter this, since the Bison
5868 parser normally extends its stack automatically up to a very large limit. But
5869 if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
5870 fashion, except that the argument string is @w{@code{"memory exhausted"}}.
5871
5872 In some cases diagnostics like @w{@code{"syntax error"}} are
5873 translated automatically from English to some other language before
5874 they are passed to @code{yyerror}. @xref{Internationalization}.
5875
5876 The following definition suffices in simple programs:
5877
5878 @example
5879 @group
5880 void
5881 yyerror (char const *s)
5882 @{
5883 @end group
5884 @group
5885 fprintf (stderr, "%s\n", s);
5886 @}
5887 @end group
5888 @end example
5889
5890 After @code{yyerror} returns to @code{yyparse}, the latter will attempt
5891 error recovery if you have written suitable error recovery grammar rules
5892 (@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
5893 immediately return 1.
5894
5895 Obviously, in location tracking pure parsers, @code{yyerror} should have
5896 an access to the current location.
5897 This is indeed the case for the @acronym{GLR}
5898 parsers, but not for the Yacc parser, for historical reasons. I.e., if
5899 @samp{%locations %define api.pure} is passed then the prototypes for
5900 @code{yyerror} are:
5901
5902 @example
5903 void yyerror (char const *msg); /* Yacc parsers. */
5904 void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
5905 @end example
5906
5907 If @samp{%parse-param @{int *nastiness@}} is used, then:
5908
5909 @example
5910 void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
5911 void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
5912 @end example
5913
5914 Finally, @acronym{GLR} and Yacc parsers share the same @code{yyerror} calling
5915 convention for absolutely pure parsers, i.e., when the calling
5916 convention of @code{yylex} @emph{and} the calling convention of
5917 @code{%define api.pure} are pure.
5918 I.e.:
5919
5920 @example
5921 /* Location tracking. */
5922 %locations
5923 /* Pure yylex. */
5924 %define api.pure
5925 %lex-param @{int *nastiness@}
5926 /* Pure yyparse. */
5927 %parse-param @{int *nastiness@}
5928 %parse-param @{int *randomness@}
5929 @end example
5930
5931 @noindent
5932 results in the following signatures for all the parser kinds:
5933
5934 @example
5935 int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
5936 int yyparse (int *nastiness, int *randomness);
5937 void yyerror (YYLTYPE *locp,
5938 int *nastiness, int *randomness,
5939 char const *msg);
5940 @end example
5941
5942 @noindent
5943 The prototypes are only indications of how the code produced by Bison
5944 uses @code{yyerror}. Bison-generated code always ignores the returned
5945 value, so @code{yyerror} can return any type, including @code{void}.
5946 Also, @code{yyerror} can be a variadic function; that is why the
5947 message is always passed last.
5948
5949 Traditionally @code{yyerror} returns an @code{int} that is always
5950 ignored, but this is purely for historical reasons, and @code{void} is
5951 preferable since it more accurately describes the return type for
5952 @code{yyerror}.
5953
5954 @vindex yynerrs
5955 The variable @code{yynerrs} contains the number of syntax errors
5956 reported so far. Normally this variable is global; but if you
5957 request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
5958 then it is a local variable which only the actions can access.
5959
5960 @node Action Features
5961 @section Special Features for Use in Actions
5962 @cindex summary, action features
5963 @cindex action features summary
5964
5965 Here is a table of Bison constructs, variables and macros that
5966 are useful in actions.
5967
5968 @deffn {Variable} $$
5969 Acts like a variable that contains the semantic value for the
5970 grouping made by the current rule. @xref{Actions}.
5971 @end deffn
5972
5973 @deffn {Variable} $@var{n}
5974 Acts like a variable that contains the semantic value for the
5975 @var{n}th component of the current rule. @xref{Actions}.
5976 @end deffn
5977
5978 @deffn {Variable} $<@var{typealt}>$
5979 Like @code{$$} but specifies alternative @var{typealt} in the union
5980 specified by the @code{%union} declaration. @xref{Action Types, ,Data
5981 Types of Values in Actions}.
5982 @end deffn
5983
5984 @deffn {Variable} $<@var{typealt}>@var{n}
5985 Like @code{$@var{n}} but specifies alternative @var{typealt} in the
5986 union specified by the @code{%union} declaration.
5987 @xref{Action Types, ,Data Types of Values in Actions}.
5988 @end deffn
5989
5990 @deffn {Macro} YYABORT;
5991 Return immediately from @code{yyparse}, indicating failure.
5992 @xref{Parser Function, ,The Parser Function @code{yyparse}}.
5993 @end deffn
5994
5995 @deffn {Macro} YYACCEPT;
5996 Return immediately from @code{yyparse}, indicating success.
5997 @xref{Parser Function, ,The Parser Function @code{yyparse}}.
5998 @end deffn
5999
6000 @deffn {Macro} YYBACKUP (@var{token}, @var{value});
6001 @findex YYBACKUP
6002 Unshift a token. This macro is allowed only for rules that reduce
6003 a single value, and only when there is no lookahead token.
6004 It is also disallowed in @acronym{GLR} parsers.
6005 It installs a lookahead token with token type @var{token} and
6006 semantic value @var{value}; then it discards the value that was
6007 going to be reduced by this rule.
6008
6009 If the macro is used when it is not valid, such as when there is
6010 a lookahead token already, then it reports a syntax error with
6011 a message @samp{cannot back up} and performs ordinary error
6012 recovery.
6013
6014 In either case, the rest of the action is not executed.
6015 @end deffn
6016
6017 @deffn {Macro} YYEMPTY
6018 @vindex YYEMPTY
6019 Value stored in @code{yychar} when there is no lookahead token.
6020 @end deffn
6021
6022 @deffn {Macro} YYEOF
6023 @vindex YYEOF
6024 Value stored in @code{yychar} when the lookahead is the end of the input
6025 stream.
6026 @end deffn
6027
6028 @deffn {Macro} YYERROR;
6029 @findex YYERROR
6030 Cause an immediate syntax error. This statement initiates error
6031 recovery just as if the parser itself had detected an error; however, it
6032 does not call @code{yyerror}, and does not print any message. If you
6033 want to print an error message, call @code{yyerror} explicitly before
6034 the @samp{YYERROR;} statement. @xref{Error Recovery}.
6035 @end deffn
6036
6037 @deffn {Macro} YYRECOVERING
6038 @findex YYRECOVERING
6039 The expression @code{YYRECOVERING ()} yields 1 when the parser
6040 is recovering from a syntax error, and 0 otherwise.
6041 @xref{Error Recovery}.
6042 @end deffn
6043
6044 @deffn {Variable} yychar
6045 Variable containing either the lookahead token, or @code{YYEOF} when the
6046 lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
6047 has been performed so the next token is not yet known.
6048 Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
6049 Actions}).
6050 @xref{Lookahead, ,Lookahead Tokens}.
6051 @end deffn
6052
6053 @deffn {Macro} yyclearin;
6054 Discard the current lookahead token. This is useful primarily in
6055 error rules.
6056 Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
6057 Semantic Actions}).
6058 @xref{Error Recovery}.
6059 @end deffn
6060
6061 @deffn {Macro} yyerrok;
6062 Resume generating error messages immediately for subsequent syntax
6063 errors. This is useful primarily in error rules.
6064 @xref{Error Recovery}.
6065 @end deffn
6066
6067 @deffn {Variable} yylloc
6068 Variable containing the lookahead token location when @code{yychar} is not set
6069 to @code{YYEMPTY} or @code{YYEOF}.
6070 Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
6071 Actions}).
6072 @xref{Actions and Locations, ,Actions and Locations}.
6073 @end deffn
6074
6075 @deffn {Variable} yylval
6076 Variable containing the lookahead token semantic value when @code{yychar} is
6077 not set to @code{YYEMPTY} or @code{YYEOF}.
6078 Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
6079 Actions}).
6080 @xref{Actions, ,Actions}.
6081 @end deffn
6082
6083 @deffn {Value} @@$
6084 @findex @@$
6085 Acts like a structure variable containing information on the textual location
6086 of the grouping made by the current rule. @xref{Locations, ,
6087 Tracking Locations}.
6088
6089 @c Check if those paragraphs are still useful or not.
6090
6091 @c @example
6092 @c struct @{
6093 @c int first_line, last_line;
6094 @c int first_column, last_column;
6095 @c @};
6096 @c @end example
6097
6098 @c Thus, to get the starting line number of the third component, you would
6099 @c use @samp{@@3.first_line}.
6100
6101 @c In order for the members of this structure to contain valid information,
6102 @c you must make @code{yylex} supply this information about each token.
6103 @c If you need only certain members, then @code{yylex} need only fill in
6104 @c those members.
6105
6106 @c The use of this feature makes the parser noticeably slower.
6107 @end deffn
6108
6109 @deffn {Value} @@@var{n}
6110 @findex @@@var{n}
6111 Acts like a structure variable containing information on the textual location
6112 of the @var{n}th component of the current rule. @xref{Locations, ,
6113 Tracking Locations}.
6114 @end deffn
6115
6116 @node Internationalization
6117 @section Parser Internationalization
6118 @cindex internationalization
6119 @cindex i18n
6120 @cindex NLS
6121 @cindex gettext
6122 @cindex bison-po
6123
6124 A Bison-generated parser can print diagnostics, including error and
6125 tracing messages. By default, they appear in English. However, Bison
6126 also supports outputting diagnostics in the user's native language. To
6127 make this work, the user should set the usual environment variables.
6128 @xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
6129 For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
6130 set the user's locale to French Canadian using the @acronym{UTF}-8
6131 encoding. The exact set of available locales depends on the user's
6132 installation.
6133
6134 The maintainer of a package that uses a Bison-generated parser enables
6135 the internationalization of the parser's output through the following
6136 steps. Here we assume a package that uses @acronym{GNU} Autoconf and
6137 @acronym{GNU} Automake.
6138
6139 @enumerate
6140 @item
6141 @cindex bison-i18n.m4
6142 Into the directory containing the @acronym{GNU} Autoconf macros used
6143 by the package---often called @file{m4}---copy the
6144 @file{bison-i18n.m4} file installed by Bison under
6145 @samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
6146 For example:
6147
6148 @example
6149 cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
6150 @end example
6151
6152 @item
6153 @findex BISON_I18N
6154 @vindex BISON_LOCALEDIR
6155 @vindex YYENABLE_NLS
6156 In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
6157 invocation, add an invocation of @code{BISON_I18N}. This macro is
6158 defined in the file @file{bison-i18n.m4} that you copied earlier. It
6159 causes @samp{configure} to find the value of the
6160 @code{BISON_LOCALEDIR} variable, and it defines the source-language
6161 symbol @code{YYENABLE_NLS} to enable translations in the
6162 Bison-generated parser.
6163
6164 @item
6165 In the @code{main} function of your program, designate the directory
6166 containing Bison's runtime message catalog, through a call to
6167 @samp{bindtextdomain} with domain name @samp{bison-runtime}.
6168 For example:
6169
6170 @example
6171 bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
6172 @end example
6173
6174 Typically this appears after any other call @code{bindtextdomain
6175 (PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
6176 @samp{BISON_LOCALEDIR} to be defined as a string through the
6177 @file{Makefile}.
6178
6179 @item
6180 In the @file{Makefile.am} that controls the compilation of the @code{main}
6181 function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
6182 either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
6183
6184 @example
6185 DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6186 @end example
6187
6188 or:
6189
6190 @example
6191 AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6192 @end example
6193
6194 @item
6195 Finally, invoke the command @command{autoreconf} to generate the build
6196 infrastructure.
6197 @end enumerate
6198
6199
6200 @node Algorithm
6201 @chapter The Bison Parser Algorithm
6202 @cindex Bison parser algorithm
6203 @cindex algorithm of parser
6204 @cindex shifting
6205 @cindex reduction
6206 @cindex parser stack
6207 @cindex stack, parser
6208
6209 As Bison reads tokens, it pushes them onto a stack along with their
6210 semantic values. The stack is called the @dfn{parser stack}. Pushing a
6211 token is traditionally called @dfn{shifting}.
6212
6213 For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
6214 @samp{3} to come. The stack will have four elements, one for each token
6215 that was shifted.
6216
6217 But the stack does not always have an element for each token read. When
6218 the last @var{n} tokens and groupings shifted match the components of a
6219 grammar rule, they can be combined according to that rule. This is called
6220 @dfn{reduction}. Those tokens and groupings are replaced on the stack by a
6221 single grouping whose symbol is the result (left hand side) of that rule.
6222 Running the rule's action is part of the process of reduction, because this
6223 is what computes the semantic value of the resulting grouping.
6224
6225 For example, if the infix calculator's parser stack contains this:
6226
6227 @example
6228 1 + 5 * 3
6229 @end example
6230
6231 @noindent
6232 and the next input token is a newline character, then the last three
6233 elements can be reduced to 15 via the rule:
6234
6235 @example
6236 expr: expr '*' expr;
6237 @end example
6238
6239 @noindent
6240 Then the stack contains just these three elements:
6241
6242 @example
6243 1 + 15
6244 @end example
6245
6246 @noindent
6247 At this point, another reduction can be made, resulting in the single value
6248 16. Then the newline token can be shifted.
6249
6250 The parser tries, by shifts and reductions, to reduce the entire input down
6251 to a single grouping whose symbol is the grammar's start-symbol
6252 (@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
6253
6254 This kind of parser is known in the literature as a bottom-up parser.
6255
6256 @menu
6257 * Lookahead:: Parser looks one token ahead when deciding what to do.
6258 * Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
6259 * Precedence:: Operator precedence works by resolving conflicts.
6260 * Contextual Precedence:: When an operator's precedence depends on context.
6261 * Parser States:: The parser is a finite-state-machine with stack.
6262 * Reduce/Reduce:: When two rules are applicable in the same situation.
6263 * Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
6264 * Generalized LR Parsing:: Parsing arbitrary context-free grammars.
6265 * Memory Management:: What happens when memory is exhausted. How to avoid it.
6266 @end menu
6267
6268 @node Lookahead
6269 @section Lookahead Tokens
6270 @cindex lookahead token
6271
6272 The Bison parser does @emph{not} always reduce immediately as soon as the
6273 last @var{n} tokens and groupings match a rule. This is because such a
6274 simple strategy is inadequate to handle most languages. Instead, when a
6275 reduction is possible, the parser sometimes ``looks ahead'' at the next
6276 token in order to decide what to do.
6277
6278 When a token is read, it is not immediately shifted; first it becomes the
6279 @dfn{lookahead token}, which is not on the stack. Now the parser can
6280 perform one or more reductions of tokens and groupings on the stack, while
6281 the lookahead token remains off to the side. When no more reductions
6282 should take place, the lookahead token is shifted onto the stack. This
6283 does not mean that all possible reductions have been done; depending on the
6284 token type of the lookahead token, some rules may choose to delay their
6285 application.
6286
6287 Here is a simple case where lookahead is needed. These three rules define
6288 expressions which contain binary addition operators and postfix unary
6289 factorial operators (@samp{!}), and allow parentheses for grouping.
6290
6291 @example
6292 @group
6293 expr: term '+' expr
6294 | term
6295 ;
6296 @end group
6297
6298 @group
6299 term: '(' expr ')'
6300 | term '!'
6301 | NUMBER
6302 ;
6303 @end group
6304 @end example
6305
6306 Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
6307 should be done? If the following token is @samp{)}, then the first three
6308 tokens must be reduced to form an @code{expr}. This is the only valid
6309 course, because shifting the @samp{)} would produce a sequence of symbols
6310 @w{@code{term ')'}}, and no rule allows this.
6311
6312 If the following token is @samp{!}, then it must be shifted immediately so
6313 that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
6314 parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
6315 @code{expr}. It would then be impossible to shift the @samp{!} because
6316 doing so would produce on the stack the sequence of symbols @code{expr
6317 '!'}. No rule allows that sequence.
6318
6319 @vindex yychar
6320 @vindex yylval
6321 @vindex yylloc
6322 The lookahead token is stored in the variable @code{yychar}.
6323 Its semantic value and location, if any, are stored in the variables
6324 @code{yylval} and @code{yylloc}.
6325 @xref{Action Features, ,Special Features for Use in Actions}.
6326
6327 @node Shift/Reduce
6328 @section Shift/Reduce Conflicts
6329 @cindex conflicts
6330 @cindex shift/reduce conflicts
6331 @cindex dangling @code{else}
6332 @cindex @code{else}, dangling
6333
6334 Suppose we are parsing a language which has if-then and if-then-else
6335 statements, with a pair of rules like this:
6336
6337 @example
6338 @group
6339 if_stmt:
6340 IF expr THEN stmt
6341 | IF expr THEN stmt ELSE stmt
6342 ;
6343 @end group
6344 @end example
6345
6346 @noindent
6347 Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
6348 terminal symbols for specific keyword tokens.
6349
6350 When the @code{ELSE} token is read and becomes the lookahead token, the
6351 contents of the stack (assuming the input is valid) are just right for
6352 reduction by the first rule. But it is also legitimate to shift the
6353 @code{ELSE}, because that would lead to eventual reduction by the second
6354 rule.
6355
6356 This situation, where either a shift or a reduction would be valid, is
6357 called a @dfn{shift/reduce conflict}. Bison is designed to resolve
6358 these conflicts by choosing to shift, unless otherwise directed by
6359 operator precedence declarations. To see the reason for this, let's
6360 contrast it with the other alternative.
6361
6362 Since the parser prefers to shift the @code{ELSE}, the result is to attach
6363 the else-clause to the innermost if-statement, making these two inputs
6364 equivalent:
6365
6366 @example
6367 if x then if y then win (); else lose;
6368
6369 if x then do; if y then win (); else lose; end;
6370 @end example
6371
6372 But if the parser chose to reduce when possible rather than shift, the
6373 result would be to attach the else-clause to the outermost if-statement,
6374 making these two inputs equivalent:
6375
6376 @example
6377 if x then if y then win (); else lose;
6378
6379 if x then do; if y then win (); end; else lose;
6380 @end example
6381
6382 The conflict exists because the grammar as written is ambiguous: either
6383 parsing of the simple nested if-statement is legitimate. The established
6384 convention is that these ambiguities are resolved by attaching the
6385 else-clause to the innermost if-statement; this is what Bison accomplishes
6386 by choosing to shift rather than reduce. (It would ideally be cleaner to
6387 write an unambiguous grammar, but that is very hard to do in this case.)
6388 This particular ambiguity was first encountered in the specifications of
6389 Algol 60 and is called the ``dangling @code{else}'' ambiguity.
6390
6391 To avoid warnings from Bison about predictable, legitimate shift/reduce
6392 conflicts, use the @code{%expect @var{n}} declaration. There will be no
6393 warning as long as the number of shift/reduce conflicts is exactly @var{n}.
6394 @xref{Expect Decl, ,Suppressing Conflict Warnings}.
6395
6396 The definition of @code{if_stmt} above is solely to blame for the
6397 conflict, but the conflict does not actually appear without additional
6398 rules. Here is a complete Bison input file that actually manifests the
6399 conflict:
6400
6401 @example
6402 @group
6403 %token IF THEN ELSE variable
6404 %%
6405 @end group
6406 @group
6407 stmt: expr
6408 | if_stmt
6409 ;
6410 @end group
6411
6412 @group
6413 if_stmt:
6414 IF expr THEN stmt
6415 | IF expr THEN stmt ELSE stmt
6416 ;
6417 @end group
6418
6419 expr: variable
6420 ;
6421 @end example
6422
6423 @node Precedence
6424 @section Operator Precedence
6425 @cindex operator precedence
6426 @cindex precedence of operators
6427
6428 Another situation where shift/reduce conflicts appear is in arithmetic
6429 expressions. Here shifting is not always the preferred resolution; the
6430 Bison declarations for operator precedence allow you to specify when to
6431 shift and when to reduce.
6432
6433 @menu
6434 * Why Precedence:: An example showing why precedence is needed.
6435 * Using Precedence:: How to specify precedence and associativity.
6436 * Precedence Only:: How to specify precedence only.
6437 * Precedence Examples:: How these features are used in the previous example.
6438 * How Precedence:: How they work.
6439 @end menu
6440
6441 @node Why Precedence
6442 @subsection When Precedence is Needed
6443
6444 Consider the following ambiguous grammar fragment (ambiguous because the
6445 input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
6446
6447 @example
6448 @group
6449 expr: expr '-' expr
6450 | expr '*' expr
6451 | expr '<' expr
6452 | '(' expr ')'
6453 @dots{}
6454 ;
6455 @end group
6456 @end example
6457
6458 @noindent
6459 Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
6460 should it reduce them via the rule for the subtraction operator? It
6461 depends on the next token. Of course, if the next token is @samp{)}, we
6462 must reduce; shifting is invalid because no single rule can reduce the
6463 token sequence @w{@samp{- 2 )}} or anything starting with that. But if
6464 the next token is @samp{*} or @samp{<}, we have a choice: either
6465 shifting or reduction would allow the parse to complete, but with
6466 different results.
6467
6468 To decide which one Bison should do, we must consider the results. If
6469 the next operator token @var{op} is shifted, then it must be reduced
6470 first in order to permit another opportunity to reduce the difference.
6471 The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
6472 hand, if the subtraction is reduced before shifting @var{op}, the result
6473 is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
6474 reduce should depend on the relative precedence of the operators
6475 @samp{-} and @var{op}: @samp{*} should be shifted first, but not
6476 @samp{<}.
6477
6478 @cindex associativity
6479 What about input such as @w{@samp{1 - 2 - 5}}; should this be
6480 @w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
6481 operators we prefer the former, which is called @dfn{left association}.
6482 The latter alternative, @dfn{right association}, is desirable for
6483 assignment operators. The choice of left or right association is a
6484 matter of whether the parser chooses to shift or reduce when the stack
6485 contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
6486 makes right-associativity.
6487
6488 @node Using Precedence
6489 @subsection Specifying Operator Precedence
6490 @findex %left
6491 @findex %nonassoc
6492 @findex %precedence
6493 @findex %right
6494
6495 Bison allows you to specify these choices with the operator precedence
6496 declarations @code{%left} and @code{%right}. Each such declaration
6497 contains a list of tokens, which are operators whose precedence and
6498 associativity is being declared. The @code{%left} declaration makes all
6499 those operators left-associative and the @code{%right} declaration makes
6500 them right-associative. A third alternative is @code{%nonassoc}, which
6501 declares that it is a syntax error to find the same operator twice ``in a
6502 row''.
6503 The last alternative, @code{%precedence}, allows to define only
6504 precedence and no associativity at all. As a result, any
6505 associativity-related conflict that remains will be reported as an
6506 compile-time error. The directive @code{%nonassoc} creates run-time
6507 error: using the operator in a associative way is a syntax error. The
6508 directive @code{%precedence} creates compile-time errors: an operator
6509 @emph{can} be involved in an associativity-related conflict, contrary to
6510 what expected the grammar author.
6511
6512 The relative precedence of different operators is controlled by the
6513 order in which they are declared. The first precedence/associativity
6514 declaration in the file declares the operators whose
6515 precedence is lowest, the next such declaration declares the operators
6516 whose precedence is a little higher, and so on.
6517
6518 @node Precedence Only
6519 @subsection Specifying Precedence Only
6520 @findex %precedence
6521
6522 Since @acronym{POSIX} Yacc defines only @code{%left}, @code{%right}, and
6523 @code{%nonassoc}, which all defines precedence and associativity, little
6524 attention is paid to the fact that precedence cannot be defined without
6525 defining associativity. Yet, sometimes, when trying to solve a
6526 conflict, precedence suffices. In such a case, using @code{%left},
6527 @code{%right}, or @code{%nonassoc} might hide future (associativity
6528 related) conflicts that would remain hidden.
6529
6530 The dangling @code{else} ambiguity (@pxref{Shift/Reduce, , Shift/Reduce
6531 Conflicts}) can be solved explictly. This shift/reduce conflicts occurs
6532 in the following situation, where the period denotes the current parsing
6533 state:
6534
6535 @example
6536 if @var{e1} then if @var{e2} then @var{s1} . else @var{s2}
6537 @end example
6538
6539 The conflict involves the reduction of the rule @samp{IF expr THEN
6540 stmt}, which precedence is by default that of its last token
6541 (@code{THEN}), and the shifting of the token @code{ELSE}. The usual
6542 disambiguation (attach the @code{else} to the closest @code{if}),
6543 shifting must be preferred, i.e., the precedence of @code{ELSE} must be
6544 higher than that of @code{THEN}. But neither is expected to be involved
6545 in an associativity related conflict, which can be specified as follows.
6546
6547 @example
6548 %precedence THEN
6549 %precedence ELSE
6550 @end example
6551
6552 The unary-minus is another typical example where associativity is
6553 usually over-specified, see @ref{Infix Calc, , Infix Notation
6554 Calculator: @code{calc}}. The @code{%left} directive is traditionaly
6555 used to declare the precedence of @code{NEG}, which is more than needed
6556 since it also defines its associativity. While this is harmless in the
6557 traditional example, who knows how @code{NEG} might be used in future
6558 evolutions of the grammar@dots{}
6559
6560 @node Precedence Examples
6561 @subsection Precedence Examples
6562
6563 In our example, we would want the following declarations:
6564
6565 @example
6566 %left '<'
6567 %left '-'
6568 %left '*'
6569 @end example
6570
6571 In a more complete example, which supports other operators as well, we
6572 would declare them in groups of equal precedence. For example, @code{'+'} is
6573 declared with @code{'-'}:
6574
6575 @example
6576 %left '<' '>' '=' NE LE GE
6577 %left '+' '-'
6578 %left '*' '/'
6579 @end example
6580
6581 @noindent
6582 (Here @code{NE} and so on stand for the operators for ``not equal''
6583 and so on. We assume that these tokens are more than one character long
6584 and therefore are represented by names, not character literals.)
6585
6586 @node How Precedence
6587 @subsection How Precedence Works
6588
6589 The first effect of the precedence declarations is to assign precedence
6590 levels to the terminal symbols declared. The second effect is to assign
6591 precedence levels to certain rules: each rule gets its precedence from
6592 the last terminal symbol mentioned in the components. (You can also
6593 specify explicitly the precedence of a rule. @xref{Contextual
6594 Precedence, ,Context-Dependent Precedence}.)
6595
6596 Finally, the resolution of conflicts works by comparing the precedence
6597 of the rule being considered with that of the lookahead token. If the
6598 token's precedence is higher, the choice is to shift. If the rule's
6599 precedence is higher, the choice is to reduce. If they have equal
6600 precedence, the choice is made based on the associativity of that
6601 precedence level. The verbose output file made by @samp{-v}
6602 (@pxref{Invocation, ,Invoking Bison}) says how each conflict was
6603 resolved.
6604
6605 Not all rules and not all tokens have precedence. If either the rule or
6606 the lookahead token has no precedence, then the default is to shift.
6607
6608 @node Contextual Precedence
6609 @section Context-Dependent Precedence
6610 @cindex context-dependent precedence
6611 @cindex unary operator precedence
6612 @cindex precedence, context-dependent
6613 @cindex precedence, unary operator
6614 @findex %prec
6615
6616 Often the precedence of an operator depends on the context. This sounds
6617 outlandish at first, but it is really very common. For example, a minus
6618 sign typically has a very high precedence as a unary operator, and a
6619 somewhat lower precedence (lower than multiplication) as a binary operator.
6620
6621 The Bison precedence declarations
6622 can only be used once for a given token; so a token has
6623 only one precedence declared in this way. For context-dependent
6624 precedence, you need to use an additional mechanism: the @code{%prec}
6625 modifier for rules.
6626
6627 The @code{%prec} modifier declares the precedence of a particular rule by
6628 specifying a terminal symbol whose precedence should be used for that rule.
6629 It's not necessary for that symbol to appear otherwise in the rule. The
6630 modifier's syntax is:
6631
6632 @example
6633 %prec @var{terminal-symbol}
6634 @end example
6635
6636 @noindent
6637 and it is written after the components of the rule. Its effect is to
6638 assign the rule the precedence of @var{terminal-symbol}, overriding
6639 the precedence that would be deduced for it in the ordinary way. The
6640 altered rule precedence then affects how conflicts involving that rule
6641 are resolved (@pxref{Precedence, ,Operator Precedence}).
6642
6643 Here is how @code{%prec} solves the problem of unary minus. First, declare
6644 a precedence for a fictitious terminal symbol named @code{UMINUS}. There
6645 are no tokens of this type, but the symbol serves to stand for its
6646 precedence:
6647
6648 @example
6649 @dots{}
6650 %left '+' '-'
6651 %left '*'
6652 %left UMINUS
6653 @end example
6654
6655 Now the precedence of @code{UMINUS} can be used in specific rules:
6656
6657 @example
6658 @group
6659 exp: @dots{}
6660 | exp '-' exp
6661 @dots{}
6662 | '-' exp %prec UMINUS
6663 @end group
6664 @end example
6665
6666 @ifset defaultprec
6667 If you forget to append @code{%prec UMINUS} to the rule for unary
6668 minus, Bison silently assumes that minus has its usual precedence.
6669 This kind of problem can be tricky to debug, since one typically
6670 discovers the mistake only by testing the code.
6671
6672 The @code{%no-default-prec;} declaration makes it easier to discover
6673 this kind of problem systematically. It causes rules that lack a
6674 @code{%prec} modifier to have no precedence, even if the last terminal
6675 symbol mentioned in their components has a declared precedence.
6676
6677 If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
6678 for all rules that participate in precedence conflict resolution.
6679 Then you will see any shift/reduce conflict until you tell Bison how
6680 to resolve it, either by changing your grammar or by adding an
6681 explicit precedence. This will probably add declarations to the
6682 grammar, but it helps to protect against incorrect rule precedences.
6683
6684 The effect of @code{%no-default-prec;} can be reversed by giving
6685 @code{%default-prec;}, which is the default.
6686 @end ifset
6687
6688 @node Parser States
6689 @section Parser States
6690 @cindex finite-state machine
6691 @cindex parser state
6692 @cindex state (of parser)
6693
6694 The function @code{yyparse} is implemented using a finite-state machine.
6695 The values pushed on the parser stack are not simply token type codes; they
6696 represent the entire sequence of terminal and nonterminal symbols at or
6697 near the top of the stack. The current state collects all the information
6698 about previous input which is relevant to deciding what to do next.
6699
6700 Each time a lookahead token is read, the current parser state together
6701 with the type of lookahead token are looked up in a table. This table
6702 entry can say, ``Shift the lookahead token.'' In this case, it also
6703 specifies the new parser state, which is pushed onto the top of the
6704 parser stack. Or it can say, ``Reduce using rule number @var{n}.''
6705 This means that a certain number of tokens or groupings are taken off
6706 the top of the stack, and replaced by one grouping. In other words,
6707 that number of states are popped from the stack, and one new state is
6708 pushed.
6709
6710 There is one other alternative: the table can say that the lookahead token
6711 is erroneous in the current state. This causes error processing to begin
6712 (@pxref{Error Recovery}).
6713
6714 @node Reduce/Reduce
6715 @section Reduce/Reduce Conflicts
6716 @cindex reduce/reduce conflict
6717 @cindex conflicts, reduce/reduce
6718
6719 A reduce/reduce conflict occurs if there are two or more rules that apply
6720 to the same sequence of input. This usually indicates a serious error
6721 in the grammar.
6722
6723 For example, here is an erroneous attempt to define a sequence
6724 of zero or more @code{word} groupings.
6725
6726 @example
6727 sequence: /* empty */
6728 @{ printf ("empty sequence\n"); @}
6729 | maybeword
6730 | sequence word
6731 @{ printf ("added word %s\n", $2); @}
6732 ;
6733
6734 maybeword: /* empty */
6735 @{ printf ("empty maybeword\n"); @}
6736 | word
6737 @{ printf ("single word %s\n", $1); @}
6738 ;
6739 @end example
6740
6741 @noindent
6742 The error is an ambiguity: there is more than one way to parse a single
6743 @code{word} into a @code{sequence}. It could be reduced to a
6744 @code{maybeword} and then into a @code{sequence} via the second rule.
6745 Alternatively, nothing-at-all could be reduced into a @code{sequence}
6746 via the first rule, and this could be combined with the @code{word}
6747 using the third rule for @code{sequence}.
6748
6749 There is also more than one way to reduce nothing-at-all into a
6750 @code{sequence}. This can be done directly via the first rule,
6751 or indirectly via @code{maybeword} and then the second rule.
6752
6753 You might think that this is a distinction without a difference, because it
6754 does not change whether any particular input is valid or not. But it does
6755 affect which actions are run. One parsing order runs the second rule's
6756 action; the other runs the first rule's action and the third rule's action.
6757 In this example, the output of the program changes.
6758
6759 Bison resolves a reduce/reduce conflict by choosing to use the rule that
6760 appears first in the grammar, but it is very risky to rely on this. Every
6761 reduce/reduce conflict must be studied and usually eliminated. Here is the
6762 proper way to define @code{sequence}:
6763
6764 @example
6765 sequence: /* empty */
6766 @{ printf ("empty sequence\n"); @}
6767 | sequence word
6768 @{ printf ("added word %s\n", $2); @}
6769 ;
6770 @end example
6771
6772 Here is another common error that yields a reduce/reduce conflict:
6773
6774 @example
6775 sequence: /* empty */
6776 | sequence words
6777 | sequence redirects
6778 ;
6779
6780 words: /* empty */
6781 | words word
6782 ;
6783
6784 redirects:/* empty */
6785 | redirects redirect
6786 ;
6787 @end example
6788
6789 @noindent
6790 The intention here is to define a sequence which can contain either
6791 @code{word} or @code{redirect} groupings. The individual definitions of
6792 @code{sequence}, @code{words} and @code{redirects} are error-free, but the
6793 three together make a subtle ambiguity: even an empty input can be parsed
6794 in infinitely many ways!
6795
6796 Consider: nothing-at-all could be a @code{words}. Or it could be two
6797 @code{words} in a row, or three, or any number. It could equally well be a
6798 @code{redirects}, or two, or any number. Or it could be a @code{words}
6799 followed by three @code{redirects} and another @code{words}. And so on.
6800
6801 Here are two ways to correct these rules. First, to make it a single level
6802 of sequence:
6803
6804 @example
6805 sequence: /* empty */
6806 | sequence word
6807 | sequence redirect
6808 ;
6809 @end example
6810
6811 Second, to prevent either a @code{words} or a @code{redirects}
6812 from being empty:
6813
6814 @example
6815 sequence: /* empty */
6816 | sequence words
6817 | sequence redirects
6818 ;
6819
6820 words: word
6821 | words word
6822 ;
6823
6824 redirects:redirect
6825 | redirects redirect
6826 ;
6827 @end example
6828
6829 @node Mystery Conflicts
6830 @section Mysterious Reduce/Reduce Conflicts
6831
6832 Sometimes reduce/reduce conflicts can occur that don't look warranted.
6833 Here is an example:
6834
6835 @example
6836 @group
6837 %token ID
6838
6839 %%
6840 def: param_spec return_spec ','
6841 ;
6842 param_spec:
6843 type
6844 | name_list ':' type
6845 ;
6846 @end group
6847 @group
6848 return_spec:
6849 type
6850 | name ':' type
6851 ;
6852 @end group
6853 @group
6854 type: ID
6855 ;
6856 @end group
6857 @group
6858 name: ID
6859 ;
6860 name_list:
6861 name
6862 | name ',' name_list
6863 ;
6864 @end group
6865 @end example
6866
6867 It would seem that this grammar can be parsed with only a single token
6868 of lookahead: when a @code{param_spec} is being read, an @code{ID} is
6869 a @code{name} if a comma or colon follows, or a @code{type} if another
6870 @code{ID} follows. In other words, this grammar is @acronym{LR}(1).
6871
6872 @cindex @acronym{LR}(1)
6873 @cindex @acronym{LALR}(1)
6874 However, for historical reasons, Bison cannot by default handle all
6875 @acronym{LR}(1) grammars.
6876 In this grammar, two contexts, that after an @code{ID} at the beginning
6877 of a @code{param_spec} and likewise at the beginning of a
6878 @code{return_spec}, are similar enough that Bison assumes they are the
6879 same.
6880 They appear similar because the same set of rules would be
6881 active---the rule for reducing to a @code{name} and that for reducing to
6882 a @code{type}. Bison is unable to determine at that stage of processing
6883 that the rules would require different lookahead tokens in the two
6884 contexts, so it makes a single parser state for them both. Combining
6885 the two contexts causes a conflict later. In parser terminology, this
6886 occurrence means that the grammar is not @acronym{LALR}(1).
6887
6888 For many practical grammars (specifically those that fall into the
6889 non-@acronym{LR}(1) class), the limitations of @acronym{LALR}(1) result in
6890 difficulties beyond just mysterious reduce/reduce conflicts.
6891 The best way to fix all these problems is to select a different parser
6892 table generation algorithm.
6893 Either @acronym{IELR}(1) or canonical @acronym{LR}(1) would suffice, but
6894 the former is more efficient and easier to debug during development.
6895 @xref{Decl Summary,,lr.type}, for details.
6896 (Bison's @acronym{IELR}(1) and canonical @acronym{LR}(1) implementations
6897 are experimental.
6898 More user feedback will help to stabilize them.)
6899
6900 If you instead wish to work around @acronym{LALR}(1)'s limitations, you
6901 can often fix a mysterious conflict by identifying the two parser states
6902 that are being confused, and adding something to make them look
6903 distinct. In the above example, adding one rule to
6904 @code{return_spec} as follows makes the problem go away:
6905
6906 @example
6907 @group
6908 %token BOGUS
6909 @dots{}
6910 %%
6911 @dots{}
6912 return_spec:
6913 type
6914 | name ':' type
6915 /* This rule is never used. */
6916 | ID BOGUS
6917 ;
6918 @end group
6919 @end example
6920
6921 This corrects the problem because it introduces the possibility of an
6922 additional active rule in the context after the @code{ID} at the beginning of
6923 @code{return_spec}. This rule is not active in the corresponding context
6924 in a @code{param_spec}, so the two contexts receive distinct parser states.
6925 As long as the token @code{BOGUS} is never generated by @code{yylex},
6926 the added rule cannot alter the way actual input is parsed.
6927
6928 In this particular example, there is another way to solve the problem:
6929 rewrite the rule for @code{return_spec} to use @code{ID} directly
6930 instead of via @code{name}. This also causes the two confusing
6931 contexts to have different sets of active rules, because the one for
6932 @code{return_spec} activates the altered rule for @code{return_spec}
6933 rather than the one for @code{name}.
6934
6935 @example
6936 param_spec:
6937 type
6938 | name_list ':' type
6939 ;
6940 return_spec:
6941 type
6942 | ID ':' type
6943 ;
6944 @end example
6945
6946 For a more detailed exposition of @acronym{LALR}(1) parsers and parser
6947 generators, please see:
6948 Frank DeRemer and Thomas Pennello, Efficient Computation of
6949 @acronym{LALR}(1) Look-Ahead Sets, @cite{@acronym{ACM} Transactions on
6950 Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
6951 pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
6952
6953 @node Generalized LR Parsing
6954 @section Generalized @acronym{LR} (@acronym{GLR}) Parsing
6955 @cindex @acronym{GLR} parsing
6956 @cindex generalized @acronym{LR} (@acronym{GLR}) parsing
6957 @cindex ambiguous grammars
6958 @cindex nondeterministic parsing
6959
6960 Bison produces @emph{deterministic} parsers that choose uniquely
6961 when to reduce and which reduction to apply
6962 based on a summary of the preceding input and on one extra token of lookahead.
6963 As a result, normal Bison handles a proper subset of the family of
6964 context-free languages.
6965 Ambiguous grammars, since they have strings with more than one possible
6966 sequence of reductions cannot have deterministic parsers in this sense.
6967 The same is true of languages that require more than one symbol of
6968 lookahead, since the parser lacks the information necessary to make a
6969 decision at the point it must be made in a shift-reduce parser.
6970 Finally, as previously mentioned (@pxref{Mystery Conflicts}),
6971 there are languages where Bison's default choice of how to
6972 summarize the input seen so far loses necessary information.
6973
6974 When you use the @samp{%glr-parser} declaration in your grammar file,
6975 Bison generates a parser that uses a different algorithm, called
6976 Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR}
6977 parser uses the same basic
6978 algorithm for parsing as an ordinary Bison parser, but behaves
6979 differently in cases where there is a shift-reduce conflict that has not
6980 been resolved by precedence rules (@pxref{Precedence}) or a
6981 reduce-reduce conflict. When a @acronym{GLR} parser encounters such a
6982 situation, it
6983 effectively @emph{splits} into a several parsers, one for each possible
6984 shift or reduction. These parsers then proceed as usual, consuming
6985 tokens in lock-step. Some of the stacks may encounter other conflicts
6986 and split further, with the result that instead of a sequence of states,
6987 a Bison @acronym{GLR} parsing stack is what is in effect a tree of states.
6988
6989 In effect, each stack represents a guess as to what the proper parse
6990 is. Additional input may indicate that a guess was wrong, in which case
6991 the appropriate stack silently disappears. Otherwise, the semantics
6992 actions generated in each stack are saved, rather than being executed
6993 immediately. When a stack disappears, its saved semantic actions never
6994 get executed. When a reduction causes two stacks to become equivalent,
6995 their sets of semantic actions are both saved with the state that
6996 results from the reduction. We say that two stacks are equivalent
6997 when they both represent the same sequence of states,
6998 and each pair of corresponding states represents a
6999 grammar symbol that produces the same segment of the input token
7000 stream.
7001
7002 Whenever the parser makes a transition from having multiple
7003 states to having one, it reverts to the normal deterministic parsing
7004 algorithm, after resolving and executing the saved-up actions.
7005 At this transition, some of the states on the stack will have semantic
7006 values that are sets (actually multisets) of possible actions. The
7007 parser tries to pick one of the actions by first finding one whose rule
7008 has the highest dynamic precedence, as set by the @samp{%dprec}
7009 declaration. Otherwise, if the alternative actions are not ordered by
7010 precedence, but there the same merging function is declared for both
7011 rules by the @samp{%merge} declaration,
7012 Bison resolves and evaluates both and then calls the merge function on
7013 the result. Otherwise, it reports an ambiguity.
7014
7015 It is possible to use a data structure for the @acronym{GLR} parsing tree that
7016 permits the processing of any @acronym{LR}(1) grammar in linear time (in the
7017 size of the input), any unambiguous (not necessarily
7018 @acronym{LR}(1)) grammar in
7019 quadratic worst-case time, and any general (possibly ambiguous)
7020 context-free grammar in cubic worst-case time. However, Bison currently
7021 uses a simpler data structure that requires time proportional to the
7022 length of the input times the maximum number of stacks required for any
7023 prefix of the input. Thus, really ambiguous or nondeterministic
7024 grammars can require exponential time and space to process. Such badly
7025 behaving examples, however, are not generally of practical interest.
7026 Usually, nondeterminism in a grammar is local---the parser is ``in
7027 doubt'' only for a few tokens at a time. Therefore, the current data
7028 structure should generally be adequate. On @acronym{LR}(1) portions of a
7029 grammar, in particular, it is only slightly slower than with the
7030 deterministic @acronym{LR}(1) Bison parser.
7031
7032 For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth
7033 Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
7034 Generalised @acronym{LR} Parsers, Royal Holloway, University of
7035 London, Department of Computer Science, TR-00-12,
7036 @uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
7037 (2000-12-24).
7038
7039 @node Memory Management
7040 @section Memory Management, and How to Avoid Memory Exhaustion
7041 @cindex memory exhaustion
7042 @cindex memory management
7043 @cindex stack overflow
7044 @cindex parser stack overflow
7045 @cindex overflow of parser stack
7046
7047 The Bison parser stack can run out of memory if too many tokens are shifted and
7048 not reduced. When this happens, the parser function @code{yyparse}
7049 calls @code{yyerror} and then returns 2.
7050
7051 Because Bison parsers have growing stacks, hitting the upper limit
7052 usually results from using a right recursion instead of a left
7053 recursion, @xref{Recursion, ,Recursive Rules}.
7054
7055 @vindex YYMAXDEPTH
7056 By defining the macro @code{YYMAXDEPTH}, you can control how deep the
7057 parser stack can become before memory is exhausted. Define the
7058 macro with a value that is an integer. This value is the maximum number
7059 of tokens that can be shifted (and not reduced) before overflow.
7060
7061 The stack space allowed is not necessarily allocated. If you specify a
7062 large value for @code{YYMAXDEPTH}, the parser normally allocates a small
7063 stack at first, and then makes it bigger by stages as needed. This
7064 increasing allocation happens automatically and silently. Therefore,
7065 you do not need to make @code{YYMAXDEPTH} painfully small merely to save
7066 space for ordinary inputs that do not need much stack.
7067
7068 However, do not allow @code{YYMAXDEPTH} to be a value so large that
7069 arithmetic overflow could occur when calculating the size of the stack
7070 space. Also, do not allow @code{YYMAXDEPTH} to be less than
7071 @code{YYINITDEPTH}.
7072
7073 @cindex default stack limit
7074 The default value of @code{YYMAXDEPTH}, if you do not define it, is
7075 10000.
7076
7077 @vindex YYINITDEPTH
7078 You can control how much stack is allocated initially by defining the
7079 macro @code{YYINITDEPTH} to a positive integer. For the deterministic
7080 parser in C, this value must be a compile-time constant
7081 unless you are assuming C99 or some other target language or compiler
7082 that allows variable-length arrays. The default is 200.
7083
7084 Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
7085
7086 @c FIXME: C++ output.
7087 Because of semantical differences between C and C++, the deterministic
7088 parsers in C produced by Bison cannot grow when compiled
7089 by C++ compilers. In this precise case (compiling a C parser as C++) you are
7090 suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix
7091 this deficiency in a future release.
7092
7093 @node Error Recovery
7094 @chapter Error Recovery
7095 @cindex error recovery
7096 @cindex recovery from errors
7097
7098 It is not usually acceptable to have a program terminate on a syntax
7099 error. For example, a compiler should recover sufficiently to parse the
7100 rest of the input file and check it for errors; a calculator should accept
7101 another expression.
7102
7103 In a simple interactive command parser where each input is one line, it may
7104 be sufficient to allow @code{yyparse} to return 1 on error and have the
7105 caller ignore the rest of the input line when that happens (and then call
7106 @code{yyparse} again). But this is inadequate for a compiler, because it
7107 forgets all the syntactic context leading up to the error. A syntax error
7108 deep within a function in the compiler input should not cause the compiler
7109 to treat the following line like the beginning of a source file.
7110
7111 @findex error
7112 You can define how to recover from a syntax error by writing rules to
7113 recognize the special token @code{error}. This is a terminal symbol that
7114 is always defined (you need not declare it) and reserved for error
7115 handling. The Bison parser generates an @code{error} token whenever a
7116 syntax error happens; if you have provided a rule to recognize this token
7117 in the current context, the parse can continue.
7118
7119 For example:
7120
7121 @example
7122 stmnts: /* empty string */
7123 | stmnts '\n'
7124 | stmnts exp '\n'
7125 | stmnts error '\n'
7126 @end example
7127
7128 The fourth rule in this example says that an error followed by a newline
7129 makes a valid addition to any @code{stmnts}.
7130
7131 What happens if a syntax error occurs in the middle of an @code{exp}? The
7132 error recovery rule, interpreted strictly, applies to the precise sequence
7133 of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
7134 the middle of an @code{exp}, there will probably be some additional tokens
7135 and subexpressions on the stack after the last @code{stmnts}, and there
7136 will be tokens to read before the next newline. So the rule is not
7137 applicable in the ordinary way.
7138
7139 But Bison can force the situation to fit the rule, by discarding part of
7140 the semantic context and part of the input. First it discards states
7141 and objects from the stack until it gets back to a state in which the
7142 @code{error} token is acceptable. (This means that the subexpressions
7143 already parsed are discarded, back to the last complete @code{stmnts}.)
7144 At this point the @code{error} token can be shifted. Then, if the old
7145 lookahead token is not acceptable to be shifted next, the parser reads
7146 tokens and discards them until it finds a token which is acceptable. In
7147 this example, Bison reads and discards input until the next newline so
7148 that the fourth rule can apply. Note that discarded symbols are
7149 possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
7150 Discarded Symbols}, for a means to reclaim this memory.
7151
7152 The choice of error rules in the grammar is a choice of strategies for
7153 error recovery. A simple and useful strategy is simply to skip the rest of
7154 the current input line or current statement if an error is detected:
7155
7156 @example
7157 stmnt: error ';' /* On error, skip until ';' is read. */
7158 @end example
7159
7160 It is also useful to recover to the matching close-delimiter of an
7161 opening-delimiter that has already been parsed. Otherwise the
7162 close-delimiter will probably appear to be unmatched, and generate another,
7163 spurious error message:
7164
7165 @example
7166 primary: '(' expr ')'
7167 | '(' error ')'
7168 @dots{}
7169 ;
7170 @end example
7171
7172 Error recovery strategies are necessarily guesses. When they guess wrong,
7173 one syntax error often leads to another. In the above example, the error
7174 recovery rule guesses that an error is due to bad input within one
7175 @code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
7176 middle of a valid @code{stmnt}. After the error recovery rule recovers
7177 from the first error, another syntax error will be found straightaway,
7178 since the text following the spurious semicolon is also an invalid
7179 @code{stmnt}.
7180
7181 To prevent an outpouring of error messages, the parser will output no error
7182 message for another syntax error that happens shortly after the first; only
7183 after three consecutive input tokens have been successfully shifted will
7184 error messages resume.
7185
7186 Note that rules which accept the @code{error} token may have actions, just
7187 as any other rules can.
7188
7189 @findex yyerrok
7190 You can make error messages resume immediately by using the macro
7191 @code{yyerrok} in an action. If you do this in the error rule's action, no
7192 error messages will be suppressed. This macro requires no arguments;
7193 @samp{yyerrok;} is a valid C statement.
7194
7195 @findex yyclearin
7196 The previous lookahead token is reanalyzed immediately after an error. If
7197 this is unacceptable, then the macro @code{yyclearin} may be used to clear
7198 this token. Write the statement @samp{yyclearin;} in the error rule's
7199 action.
7200 @xref{Action Features, ,Special Features for Use in Actions}.
7201
7202 For example, suppose that on a syntax error, an error handling routine is
7203 called that advances the input stream to some point where parsing should
7204 once again commence. The next symbol returned by the lexical scanner is
7205 probably correct. The previous lookahead token ought to be discarded
7206 with @samp{yyclearin;}.
7207
7208 @vindex YYRECOVERING
7209 The expression @code{YYRECOVERING ()} yields 1 when the parser
7210 is recovering from a syntax error, and 0 otherwise.
7211 Syntax error diagnostics are suppressed while recovering from a syntax
7212 error.
7213
7214 @node Context Dependency
7215 @chapter Handling Context Dependencies
7216
7217 The Bison paradigm is to parse tokens first, then group them into larger
7218 syntactic units. In many languages, the meaning of a token is affected by
7219 its context. Although this violates the Bison paradigm, certain techniques
7220 (known as @dfn{kludges}) may enable you to write Bison parsers for such
7221 languages.
7222
7223 @menu
7224 * Semantic Tokens:: Token parsing can depend on the semantic context.
7225 * Lexical Tie-ins:: Token parsing can depend on the syntactic context.
7226 * Tie-in Recovery:: Lexical tie-ins have implications for how
7227 error recovery rules must be written.
7228 @end menu
7229
7230 (Actually, ``kludge'' means any technique that gets its job done but is
7231 neither clean nor robust.)
7232
7233 @node Semantic Tokens
7234 @section Semantic Info in Token Types
7235
7236 The C language has a context dependency: the way an identifier is used
7237 depends on what its current meaning is. For example, consider this:
7238
7239 @example
7240 foo (x);
7241 @end example
7242
7243 This looks like a function call statement, but if @code{foo} is a typedef
7244 name, then this is actually a declaration of @code{x}. How can a Bison
7245 parser for C decide how to parse this input?
7246
7247 The method used in @acronym{GNU} C is to have two different token types,
7248 @code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
7249 identifier, it looks up the current declaration of the identifier in order
7250 to decide which token type to return: @code{TYPENAME} if the identifier is
7251 declared as a typedef, @code{IDENTIFIER} otherwise.
7252
7253 The grammar rules can then express the context dependency by the choice of
7254 token type to recognize. @code{IDENTIFIER} is accepted as an expression,
7255 but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
7256 @code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
7257 is @emph{not} significant, such as in declarations that can shadow a
7258 typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
7259 accepted---there is one rule for each of the two token types.
7260
7261 This technique is simple to use if the decision of which kinds of
7262 identifiers to allow is made at a place close to where the identifier is
7263 parsed. But in C this is not always so: C allows a declaration to
7264 redeclare a typedef name provided an explicit type has been specified
7265 earlier:
7266
7267 @example
7268 typedef int foo, bar;
7269 int baz (void)
7270 @{
7271 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
7272 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
7273 return foo (bar);
7274 @}
7275 @end example
7276
7277 Unfortunately, the name being declared is separated from the declaration
7278 construct itself by a complicated syntactic structure---the ``declarator''.
7279
7280 As a result, part of the Bison parser for C needs to be duplicated, with
7281 all the nonterminal names changed: once for parsing a declaration in
7282 which a typedef name can be redefined, and once for parsing a
7283 declaration in which that can't be done. Here is a part of the
7284 duplication, with actions omitted for brevity:
7285
7286 @example
7287 initdcl:
7288 declarator maybeasm '='
7289 init
7290 | declarator maybeasm
7291 ;
7292
7293 notype_initdcl:
7294 notype_declarator maybeasm '='
7295 init
7296 | notype_declarator maybeasm
7297 ;
7298 @end example
7299
7300 @noindent
7301 Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
7302 cannot. The distinction between @code{declarator} and
7303 @code{notype_declarator} is the same sort of thing.
7304
7305 There is some similarity between this technique and a lexical tie-in
7306 (described next), in that information which alters the lexical analysis is
7307 changed during parsing by other parts of the program. The difference is
7308 here the information is global, and is used for other purposes in the
7309 program. A true lexical tie-in has a special-purpose flag controlled by
7310 the syntactic context.
7311
7312 @node Lexical Tie-ins
7313 @section Lexical Tie-ins
7314 @cindex lexical tie-in
7315
7316 One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
7317 which is set by Bison actions, whose purpose is to alter the way tokens are
7318 parsed.
7319
7320 For example, suppose we have a language vaguely like C, but with a special
7321 construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
7322 an expression in parentheses in which all integers are hexadecimal. In
7323 particular, the token @samp{a1b} must be treated as an integer rather than
7324 as an identifier if it appears in that context. Here is how you can do it:
7325
7326 @example
7327 @group
7328 %@{
7329 int hexflag;
7330 int yylex (void);
7331 void yyerror (char const *);
7332 %@}
7333 %%
7334 @dots{}
7335 @end group
7336 @group
7337 expr: IDENTIFIER
7338 | constant
7339 | HEX '('
7340 @{ hexflag = 1; @}
7341 expr ')'
7342 @{ hexflag = 0;
7343 $$ = $4; @}
7344 | expr '+' expr
7345 @{ $$ = make_sum ($1, $3); @}
7346 @dots{}
7347 ;
7348 @end group
7349
7350 @group
7351 constant:
7352 INTEGER
7353 | STRING
7354 ;
7355 @end group
7356 @end example
7357
7358 @noindent
7359 Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
7360 it is nonzero, all integers are parsed in hexadecimal, and tokens starting
7361 with letters are parsed as integers if possible.
7362
7363 The declaration of @code{hexflag} shown in the prologue of the parser file
7364 is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}).
7365 You must also write the code in @code{yylex} to obey the flag.
7366
7367 @node Tie-in Recovery
7368 @section Lexical Tie-ins and Error Recovery
7369
7370 Lexical tie-ins make strict demands on any error recovery rules you have.
7371 @xref{Error Recovery}.
7372
7373 The reason for this is that the purpose of an error recovery rule is to
7374 abort the parsing of one construct and resume in some larger construct.
7375 For example, in C-like languages, a typical error recovery rule is to skip
7376 tokens until the next semicolon, and then start a new statement, like this:
7377
7378 @example
7379 stmt: expr ';'
7380 | IF '(' expr ')' stmt @{ @dots{} @}
7381 @dots{}
7382 error ';'
7383 @{ hexflag = 0; @}
7384 ;
7385 @end example
7386
7387 If there is a syntax error in the middle of a @samp{hex (@var{expr})}
7388 construct, this error rule will apply, and then the action for the
7389 completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
7390 remain set for the entire rest of the input, or until the next @code{hex}
7391 keyword, causing identifiers to be misinterpreted as integers.
7392
7393 To avoid this problem the error recovery rule itself clears @code{hexflag}.
7394
7395 There may also be an error recovery rule that works within expressions.
7396 For example, there could be a rule which applies within parentheses
7397 and skips to the close-parenthesis:
7398
7399 @example
7400 @group
7401 expr: @dots{}
7402 | '(' expr ')'
7403 @{ $$ = $2; @}
7404 | '(' error ')'
7405 @dots{}
7406 @end group
7407 @end example
7408
7409 If this rule acts within the @code{hex} construct, it is not going to abort
7410 that construct (since it applies to an inner level of parentheses within
7411 the construct). Therefore, it should not clear the flag: the rest of
7412 the @code{hex} construct should be parsed with the flag still in effect.
7413
7414 What if there is an error recovery rule which might abort out of the
7415 @code{hex} construct or might not, depending on circumstances? There is no
7416 way you can write the action to determine whether a @code{hex} construct is
7417 being aborted or not. So if you are using a lexical tie-in, you had better
7418 make sure your error recovery rules are not of this kind. Each rule must
7419 be such that you can be sure that it always will, or always won't, have to
7420 clear the flag.
7421
7422 @c ================================================== Debugging Your Parser
7423
7424 @node Debugging
7425 @chapter Debugging Your Parser
7426
7427 Developing a parser can be a challenge, especially if you don't
7428 understand the algorithm (@pxref{Algorithm, ,The Bison Parser
7429 Algorithm}). Even so, sometimes a detailed description of the automaton
7430 can help (@pxref{Understanding, , Understanding Your Parser}), or
7431 tracing the execution of the parser can give some insight on why it
7432 behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
7433
7434 @menu
7435 * Understanding:: Understanding the structure of your parser.
7436 * Tracing:: Tracing the execution of your parser.
7437 @end menu
7438
7439 @node Understanding
7440 @section Understanding Your Parser
7441
7442 As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
7443 Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
7444 frequent than one would hope), looking at this automaton is required to
7445 tune or simply fix a parser. Bison provides two different
7446 representation of it, either textually or graphically (as a DOT file).
7447
7448 The textual file is generated when the options @option{--report} or
7449 @option{--verbose} are specified, see @xref{Invocation, , Invoking
7450 Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
7451 the parser output file name, and adding @samp{.output} instead.
7452 Therefore, if the input file is @file{foo.y}, then the parser file is
7453 called @file{foo.tab.c} by default. As a consequence, the verbose
7454 output file is called @file{foo.output}.
7455
7456 The following grammar file, @file{calc.y}, will be used in the sequel:
7457
7458 @example
7459 %token NUM STR
7460 %left '+' '-'
7461 %left '*'
7462 %%
7463 exp: exp '+' exp
7464 | exp '-' exp
7465 | exp '*' exp
7466 | exp '/' exp
7467 | NUM
7468 ;
7469 useless: STR;
7470 %%
7471 @end example
7472
7473 @command{bison} reports:
7474
7475 @example
7476 calc.y: warning: 1 nonterminal useless in grammar
7477 calc.y: warning: 1 rule useless in grammar
7478 calc.y:11.1-7: warning: nonterminal useless in grammar: useless
7479 calc.y:11.10-12: warning: rule useless in grammar: useless: STR
7480 calc.y: conflicts: 7 shift/reduce
7481 @end example
7482
7483 When given @option{--report=state}, in addition to @file{calc.tab.c}, it
7484 creates a file @file{calc.output} with contents detailed below. The
7485 order of the output and the exact presentation might vary, but the
7486 interpretation is the same.
7487
7488 The first section includes details on conflicts that were solved thanks
7489 to precedence and/or associativity:
7490
7491 @example
7492 Conflict in state 8 between rule 2 and token '+' resolved as reduce.
7493 Conflict in state 8 between rule 2 and token '-' resolved as reduce.
7494 Conflict in state 8 between rule 2 and token '*' resolved as shift.
7495 @exdent @dots{}
7496 @end example
7497
7498 @noindent
7499 The next section lists states that still have conflicts.
7500
7501 @example
7502 State 8 conflicts: 1 shift/reduce
7503 State 9 conflicts: 1 shift/reduce
7504 State 10 conflicts: 1 shift/reduce
7505 State 11 conflicts: 4 shift/reduce
7506 @end example
7507
7508 @noindent
7509 @cindex token, useless
7510 @cindex useless token
7511 @cindex nonterminal, useless
7512 @cindex useless nonterminal
7513 @cindex rule, useless
7514 @cindex useless rule
7515 The next section reports useless tokens, nonterminal and rules. Useless
7516 nonterminals and rules are removed in order to produce a smaller parser,
7517 but useless tokens are preserved, since they might be used by the
7518 scanner (note the difference between ``useless'' and ``unused''
7519 below):
7520
7521 @example
7522 Nonterminals useless in grammar:
7523 useless
7524
7525 Terminals unused in grammar:
7526 STR
7527
7528 Rules useless in grammar:
7529 #6 useless: STR;
7530 @end example
7531
7532 @noindent
7533 The next section reproduces the exact grammar that Bison used:
7534
7535 @example
7536 Grammar
7537
7538 Number, Line, Rule
7539 0 5 $accept -> exp $end
7540 1 5 exp -> exp '+' exp
7541 2 6 exp -> exp '-' exp
7542 3 7 exp -> exp '*' exp
7543 4 8 exp -> exp '/' exp
7544 5 9 exp -> NUM
7545 @end example
7546
7547 @noindent
7548 and reports the uses of the symbols:
7549
7550 @example
7551 Terminals, with rules where they appear
7552
7553 $end (0) 0
7554 '*' (42) 3
7555 '+' (43) 1
7556 '-' (45) 2
7557 '/' (47) 4
7558 error (256)
7559 NUM (258) 5
7560
7561 Nonterminals, with rules where they appear
7562
7563 $accept (8)
7564 on left: 0
7565 exp (9)
7566 on left: 1 2 3 4 5, on right: 0 1 2 3 4
7567 @end example
7568
7569 @noindent
7570 @cindex item
7571 @cindex pointed rule
7572 @cindex rule, pointed
7573 Bison then proceeds onto the automaton itself, describing each state
7574 with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
7575 item is a production rule together with a point (marked by @samp{.})
7576 that the input cursor.
7577
7578 @example
7579 state 0
7580
7581 $accept -> . exp $ (rule 0)
7582
7583 NUM shift, and go to state 1
7584
7585 exp go to state 2
7586 @end example
7587
7588 This reads as follows: ``state 0 corresponds to being at the very
7589 beginning of the parsing, in the initial rule, right before the start
7590 symbol (here, @code{exp}). When the parser returns to this state right
7591 after having reduced a rule that produced an @code{exp}, the control
7592 flow jumps to state 2. If there is no such transition on a nonterminal
7593 symbol, and the lookahead is a @code{NUM}, then this token is shifted on
7594 the parse stack, and the control flow jumps to state 1. Any other
7595 lookahead triggers a syntax error.''
7596
7597 @cindex core, item set
7598 @cindex item set core
7599 @cindex kernel, item set
7600 @cindex item set core
7601 Even though the only active rule in state 0 seems to be rule 0, the
7602 report lists @code{NUM} as a lookahead token because @code{NUM} can be
7603 at the beginning of any rule deriving an @code{exp}. By default Bison
7604 reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
7605 you want to see more detail you can invoke @command{bison} with
7606 @option{--report=itemset} to list all the items, include those that can
7607 be derived:
7608
7609 @example
7610 state 0
7611
7612 $accept -> . exp $ (rule 0)
7613 exp -> . exp '+' exp (rule 1)
7614 exp -> . exp '-' exp (rule 2)
7615 exp -> . exp '*' exp (rule 3)
7616 exp -> . exp '/' exp (rule 4)
7617 exp -> . NUM (rule 5)
7618
7619 NUM shift, and go to state 1
7620
7621 exp go to state 2
7622 @end example
7623
7624 @noindent
7625 In the state 1...
7626
7627 @example
7628 state 1
7629
7630 exp -> NUM . (rule 5)
7631
7632 $default reduce using rule 5 (exp)
7633 @end example
7634
7635 @noindent
7636 the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
7637 (@samp{$default}), the parser will reduce it. If it was coming from
7638 state 0, then, after this reduction it will return to state 0, and will
7639 jump to state 2 (@samp{exp: go to state 2}).
7640
7641 @example
7642 state 2
7643
7644 $accept -> exp . $ (rule 0)
7645 exp -> exp . '+' exp (rule 1)
7646 exp -> exp . '-' exp (rule 2)
7647 exp -> exp . '*' exp (rule 3)
7648 exp -> exp . '/' exp (rule 4)
7649
7650 $ shift, and go to state 3
7651 '+' shift, and go to state 4
7652 '-' shift, and go to state 5
7653 '*' shift, and go to state 6
7654 '/' shift, and go to state 7
7655 @end example
7656
7657 @noindent
7658 In state 2, the automaton can only shift a symbol. For instance,
7659 because of the item @samp{exp -> exp . '+' exp}, if the lookahead if
7660 @samp{+}, it will be shifted on the parse stack, and the automaton
7661 control will jump to state 4, corresponding to the item @samp{exp -> exp
7662 '+' . exp}. Since there is no default action, any other token than
7663 those listed above will trigger a syntax error.
7664
7665 @cindex accepting state
7666 The state 3 is named the @dfn{final state}, or the @dfn{accepting
7667 state}:
7668
7669 @example
7670 state 3
7671
7672 $accept -> exp $ . (rule 0)
7673
7674 $default accept
7675 @end example
7676
7677 @noindent
7678 the initial rule is completed (the start symbol and the end
7679 of input were read), the parsing exits successfully.
7680
7681 The interpretation of states 4 to 7 is straightforward, and is left to
7682 the reader.
7683
7684 @example
7685 state 4
7686
7687 exp -> exp '+' . exp (rule 1)
7688
7689 NUM shift, and go to state 1
7690
7691 exp go to state 8
7692
7693 state 5
7694
7695 exp -> exp '-' . exp (rule 2)
7696
7697 NUM shift, and go to state 1
7698
7699 exp go to state 9
7700
7701 state 6
7702
7703 exp -> exp '*' . exp (rule 3)
7704
7705 NUM shift, and go to state 1
7706
7707 exp go to state 10
7708
7709 state 7
7710
7711 exp -> exp '/' . exp (rule 4)
7712
7713 NUM shift, and go to state 1
7714
7715 exp go to state 11
7716 @end example
7717
7718 As was announced in beginning of the report, @samp{State 8 conflicts:
7719 1 shift/reduce}:
7720
7721 @example
7722 state 8
7723
7724 exp -> exp . '+' exp (rule 1)
7725 exp -> exp '+' exp . (rule 1)
7726 exp -> exp . '-' exp (rule 2)
7727 exp -> exp . '*' exp (rule 3)
7728 exp -> exp . '/' exp (rule 4)
7729
7730 '*' shift, and go to state 6
7731 '/' shift, and go to state 7
7732
7733 '/' [reduce using rule 1 (exp)]
7734 $default reduce using rule 1 (exp)
7735 @end example
7736
7737 Indeed, there are two actions associated to the lookahead @samp{/}:
7738 either shifting (and going to state 7), or reducing rule 1. The
7739 conflict means that either the grammar is ambiguous, or the parser lacks
7740 information to make the right decision. Indeed the grammar is
7741 ambiguous, as, since we did not specify the precedence of @samp{/}, the
7742 sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
7743 NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
7744 NUM}, which corresponds to reducing rule 1.
7745
7746 Because in deterministic parsing a single decision can be made, Bison
7747 arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
7748 Shift/Reduce Conflicts}. Discarded actions are reported in between
7749 square brackets.
7750
7751 Note that all the previous states had a single possible action: either
7752 shifting the next token and going to the corresponding state, or
7753 reducing a single rule. In the other cases, i.e., when shifting
7754 @emph{and} reducing is possible or when @emph{several} reductions are
7755 possible, the lookahead is required to select the action. State 8 is
7756 one such state: if the lookahead is @samp{*} or @samp{/} then the action
7757 is shifting, otherwise the action is reducing rule 1. In other words,
7758 the first two items, corresponding to rule 1, are not eligible when the
7759 lookahead token is @samp{*}, since we specified that @samp{*} has higher
7760 precedence than @samp{+}. More generally, some items are eligible only
7761 with some set of possible lookahead tokens. When run with
7762 @option{--report=lookahead}, Bison specifies these lookahead tokens:
7763
7764 @example
7765 state 8
7766
7767 exp -> exp . '+' exp (rule 1)
7768 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
7769 exp -> exp . '-' exp (rule 2)
7770 exp -> exp . '*' exp (rule 3)
7771 exp -> exp . '/' exp (rule 4)
7772
7773 '*' shift, and go to state 6
7774 '/' shift, and go to state 7
7775
7776 '/' [reduce using rule 1 (exp)]
7777 $default reduce using rule 1 (exp)
7778 @end example
7779
7780 The remaining states are similar:
7781
7782 @example
7783 state 9
7784
7785 exp -> exp . '+' exp (rule 1)
7786 exp -> exp . '-' exp (rule 2)
7787 exp -> exp '-' exp . (rule 2)
7788 exp -> exp . '*' exp (rule 3)
7789 exp -> exp . '/' exp (rule 4)
7790
7791 '*' shift, and go to state 6
7792 '/' shift, and go to state 7
7793
7794 '/' [reduce using rule 2 (exp)]
7795 $default reduce using rule 2 (exp)
7796
7797 state 10
7798
7799 exp -> exp . '+' exp (rule 1)
7800 exp -> exp . '-' exp (rule 2)
7801 exp -> exp . '*' exp (rule 3)
7802 exp -> exp '*' exp . (rule 3)
7803 exp -> exp . '/' exp (rule 4)
7804
7805 '/' shift, and go to state 7
7806
7807 '/' [reduce using rule 3 (exp)]
7808 $default reduce using rule 3 (exp)
7809
7810 state 11
7811
7812 exp -> exp . '+' exp (rule 1)
7813 exp -> exp . '-' exp (rule 2)
7814 exp -> exp . '*' exp (rule 3)
7815 exp -> exp . '/' exp (rule 4)
7816 exp -> exp '/' exp . (rule 4)
7817
7818 '+' shift, and go to state 4
7819 '-' shift, and go to state 5
7820 '*' shift, and go to state 6
7821 '/' shift, and go to state 7
7822
7823 '+' [reduce using rule 4 (exp)]
7824 '-' [reduce using rule 4 (exp)]
7825 '*' [reduce using rule 4 (exp)]
7826 '/' [reduce using rule 4 (exp)]
7827 $default reduce using rule 4 (exp)
7828 @end example
7829
7830 @noindent
7831 Observe that state 11 contains conflicts not only due to the lack of
7832 precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
7833 @samp{*}, but also because the
7834 associativity of @samp{/} is not specified.
7835
7836
7837 @node Tracing
7838 @section Tracing Your Parser
7839 @findex yydebug
7840 @cindex debugging
7841 @cindex tracing the parser
7842
7843 If a Bison grammar compiles properly but doesn't do what you want when it
7844 runs, the @code{yydebug} parser-trace feature can help you figure out why.
7845
7846 There are several means to enable compilation of trace facilities:
7847
7848 @table @asis
7849 @item the macro @code{YYDEBUG}
7850 @findex YYDEBUG
7851 Define the macro @code{YYDEBUG} to a nonzero value when you compile the
7852 parser. This is compliant with @acronym{POSIX} Yacc. You could use
7853 @samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
7854 YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
7855 Prologue}).
7856
7857 @item the option @option{-t}, @option{--debug}
7858 Use the @samp{-t} option when you run Bison (@pxref{Invocation,
7859 ,Invoking Bison}). This is @acronym{POSIX} compliant too.
7860
7861 @item the directive @samp{%debug}
7862 @findex %debug
7863 Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison Declaration
7864 Summary}). This Bison extension is maintained for backward
7865 compatibility with previous versions of Bison.
7866
7867 @item the variable @samp{parse.trace}
7868 @findex %define parse.trace
7869 Add the @samp{%define parse.trace} directive (@pxref{Decl Summary,
7870 ,Bison Declaration Summary}), or pass the @option{-Dparse.trace} option
7871 (@pxref{Bison Options}). This is a Bison extension, which is especially
7872 useful for languages that don't use a preprocessor. Unless
7873 @acronym{POSIX} and Yacc portability matter to you, this is the
7874 preferred solution.
7875 @end table
7876
7877 We suggest that you always enable the trace option so that debugging is
7878 always possible.
7879
7880 The trace facility outputs messages with macro calls of the form
7881 @code{YYFPRINTF (stderr, @var{format}, @var{args})} where
7882 @var{format} and @var{args} are the usual @code{printf} format and variadic
7883 arguments. If you define @code{YYDEBUG} to a nonzero value but do not
7884 define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
7885 and @code{YYFPRINTF} is defined to @code{fprintf}.
7886
7887 Once you have compiled the program with trace facilities, the way to
7888 request a trace is to store a nonzero value in the variable @code{yydebug}.
7889 You can do this by making the C code do it (in @code{main}, perhaps), or
7890 you can alter the value with a C debugger.
7891
7892 Each step taken by the parser when @code{yydebug} is nonzero produces a
7893 line or two of trace information, written on @code{stderr}. The trace
7894 messages tell you these things:
7895
7896 @itemize @bullet
7897 @item
7898 Each time the parser calls @code{yylex}, what kind of token was read.
7899
7900 @item
7901 Each time a token is shifted, the depth and complete contents of the
7902 state stack (@pxref{Parser States}).
7903
7904 @item
7905 Each time a rule is reduced, which rule it is, and the complete contents
7906 of the state stack afterward.
7907 @end itemize
7908
7909 To make sense of this information, it helps to refer to the listing file
7910 produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
7911 Bison}). This file shows the meaning of each state in terms of
7912 positions in various rules, and also what each state will do with each
7913 possible input token. As you read the successive trace messages, you
7914 can see that the parser is functioning according to its specification in
7915 the listing file. Eventually you will arrive at the place where
7916 something undesirable happens, and you will see which parts of the
7917 grammar are to blame.
7918
7919 The parser file is a C program and you can use C debuggers on it, but it's
7920 not easy to interpret what it is doing. The parser function is a
7921 finite-state machine interpreter, and aside from the actions it executes
7922 the same code over and over. Only the values of variables show where in
7923 the grammar it is working.
7924
7925 @findex YYPRINT
7926 The debugging information normally gives the token type of each token
7927 read, but not its semantic value. You can optionally define a macro
7928 named @code{YYPRINT} to provide a way to print the value. If you define
7929 @code{YYPRINT}, it should take three arguments. The parser will pass a
7930 standard I/O stream, the numeric code for the token type, and the token
7931 value (from @code{yylval}).
7932
7933 Here is an example of @code{YYPRINT} suitable for the multi-function
7934 calculator (@pxref{Mfcalc Declarations, ,Declarations for @code{mfcalc}}):
7935
7936 @smallexample
7937 %@{
7938 static void print_token_value (FILE *, int, YYSTYPE);
7939 #define YYPRINT(file, type, value) print_token_value (file, type, value)
7940 %@}
7941
7942 @dots{} %% @dots{} %% @dots{}
7943
7944 static void
7945 print_token_value (FILE *file, int type, YYSTYPE value)
7946 @{
7947 if (type == VAR)
7948 fprintf (file, "%s", value.tptr->name);
7949 else if (type == NUM)
7950 fprintf (file, "%d", value.val);
7951 @}
7952 @end smallexample
7953
7954 @c ================================================= Invoking Bison
7955
7956 @node Invocation
7957 @chapter Invoking Bison
7958 @cindex invoking Bison
7959 @cindex Bison invocation
7960 @cindex options for invoking Bison
7961
7962 The usual way to invoke Bison is as follows:
7963
7964 @example
7965 bison @var{infile}
7966 @end example
7967
7968 Here @var{infile} is the grammar file name, which usually ends in
7969 @samp{.y}. The parser file's name is made by replacing the @samp{.y}
7970 with @samp{.tab.c} and removing any leading directory. Thus, the
7971 @samp{bison foo.y} file name yields
7972 @file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields
7973 @file{foo.tab.c}. It's also possible, in case you are writing
7974 C++ code instead of C in your grammar file, to name it @file{foo.ypp}
7975 or @file{foo.y++}. Then, the output files will take an extension like
7976 the given one as input (respectively @file{foo.tab.cpp} and
7977 @file{foo.tab.c++}).
7978 This feature takes effect with all options that manipulate file names like
7979 @samp{-o} or @samp{-d}.
7980
7981 For example :
7982
7983 @example
7984 bison -d @var{infile.yxx}
7985 @end example
7986 @noindent
7987 will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
7988
7989 @example
7990 bison -d -o @var{output.c++} @var{infile.y}
7991 @end example
7992 @noindent
7993 will produce @file{output.c++} and @file{outfile.h++}.
7994
7995 For compatibility with @acronym{POSIX}, the standard Bison
7996 distribution also contains a shell script called @command{yacc} that
7997 invokes Bison with the @option{-y} option.
7998
7999 @menu
8000 * Bison Options:: All the options described in detail,
8001 in alphabetical order by short options.
8002 * Option Cross Key:: Alphabetical list of long options.
8003 * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
8004 @end menu
8005
8006 @node Bison Options
8007 @section Bison Options
8008
8009 Bison supports both traditional single-letter options and mnemonic long
8010 option names. Long option names are indicated with @samp{--} instead of
8011 @samp{-}. Abbreviations for option names are allowed as long as they
8012 are unique. When a long option takes an argument, like
8013 @samp{--file-prefix}, connect the option name and the argument with
8014 @samp{=}.
8015
8016 Here is a list of options that can be used with Bison, alphabetized by
8017 short option. It is followed by a cross key alphabetized by long
8018 option.
8019
8020 @c Please, keep this ordered as in `bison --help'.
8021 @noindent
8022 Operations modes:
8023 @table @option
8024 @item -h
8025 @itemx --help
8026 Print a summary of the command-line options to Bison and exit.
8027
8028 @item -V
8029 @itemx --version
8030 Print the version number of Bison and exit.
8031
8032 @item --print-localedir
8033 Print the name of the directory containing locale-dependent data.
8034
8035 @item --print-datadir
8036 Print the name of the directory containing skeletons and XSLT.
8037
8038 @item -y
8039 @itemx --yacc
8040 Act more like the traditional Yacc command. This can cause
8041 different diagnostics to be generated, and may change behavior in
8042 other minor ways. Most importantly, imitate Yacc's output
8043 file name conventions, so that the parser output file is called
8044 @file{y.tab.c}, and the other outputs are called @file{y.output} and
8045 @file{y.tab.h}.
8046 Also, if generating a deterministic parser in C, generate @code{#define}
8047 statements in addition to an @code{enum} to associate token numbers with token
8048 names.
8049 Thus, the following shell script can substitute for Yacc, and the Bison
8050 distribution contains such a script for compatibility with @acronym{POSIX}:
8051
8052 @example
8053 #! /bin/sh
8054 bison -y "$@@"
8055 @end example
8056
8057 The @option{-y}/@option{--yacc} option is intended for use with
8058 traditional Yacc grammars. If your grammar uses a Bison extension
8059 like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
8060 this option is specified.
8061
8062 @item -W [@var{category}]
8063 @itemx --warnings[=@var{category}]
8064 Output warnings falling in @var{category}. @var{category} can be one
8065 of:
8066 @table @code
8067 @item midrule-values
8068 Warn about mid-rule values that are set but not used within any of the actions
8069 of the parent rule.
8070 For example, warn about unused @code{$2} in:
8071
8072 @example
8073 exp: '1' @{ $$ = 1; @} '+' exp @{ $$ = $1 + $4; @};
8074 @end example
8075
8076 Also warn about mid-rule values that are used but not set.
8077 For example, warn about unset @code{$$} in the mid-rule action in:
8078
8079 @example
8080 exp: '1' @{ $1 = 1; @} '+' exp @{ $$ = $2 + $4; @};
8081 @end example
8082
8083 These warnings are not enabled by default since they sometimes prove to
8084 be false alarms in existing grammars employing the Yacc constructs
8085 @code{$0} or @code{$-@var{n}} (where @var{n} is some positive integer).
8086
8087
8088 @item yacc
8089 Incompatibilities with @acronym{POSIX} Yacc.
8090
8091 @item all
8092 All the warnings.
8093 @item none
8094 Turn off all the warnings.
8095 @item error
8096 Treat warnings as errors.
8097 @end table
8098
8099 A category can be turned off by prefixing its name with @samp{no-}. For
8100 instance, @option{-Wno-syntax} will hide the warnings about unused
8101 variables.
8102 @end table
8103
8104 @noindent
8105 Tuning the parser:
8106
8107 @table @option
8108 @item -t
8109 @itemx --debug
8110 In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
8111 already defined, so that the debugging facilities are compiled.
8112 @xref{Tracing, ,Tracing Your Parser}.
8113
8114 @item -D @var{name}[=@var{value}]
8115 @itemx --define=@var{name}[=@var{value}]
8116 Same as running @samp{%define @var{name} "@var{value}"} (@pxref{Decl
8117 Summary, ,%define}).
8118
8119 @item -L @var{language}
8120 @itemx --language=@var{language}
8121 Specify the programming language for the generated parser, as if
8122 @code{%language} was specified (@pxref{Decl Summary, , Bison Declaration
8123 Summary}). Currently supported languages include C, C++, and Java.
8124 @var{language} is case-insensitive.
8125
8126 This option is experimental and its effect may be modified in future
8127 releases.
8128
8129 @item --locations
8130 Pretend that @code{%locations} was specified. @xref{Decl Summary}.
8131
8132 @item -p @var{prefix}
8133 @itemx --name-prefix=@var{prefix}
8134 Pretend that @code{%name-prefix "@var{prefix}"} was specified.
8135 @xref{Decl Summary}.
8136
8137 @item -l
8138 @itemx --no-lines
8139 Don't put any @code{#line} preprocessor commands in the parser file.
8140 Ordinarily Bison puts them in the parser file so that the C compiler
8141 and debuggers will associate errors with your source file, the
8142 grammar file. This option causes them to associate errors with the
8143 parser file, treating it as an independent source file in its own right.
8144
8145 @item -S @var{file}
8146 @itemx --skeleton=@var{file}
8147 Specify the skeleton to use, similar to @code{%skeleton}
8148 (@pxref{Decl Summary, , Bison Declaration Summary}).
8149
8150 @c You probably don't need this option unless you are developing Bison.
8151 @c You should use @option{--language} if you want to specify the skeleton for a
8152 @c different language, because it is clearer and because it will always
8153 @c choose the correct skeleton for non-deterministic or push parsers.
8154
8155 If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
8156 file in the Bison installation directory.
8157 If it does, @var{file} is an absolute file name or a file name relative to the
8158 current working directory.
8159 This is similar to how most shells resolve commands.
8160
8161 @item -k
8162 @itemx --token-table
8163 Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
8164 @end table
8165
8166 @noindent
8167 Adjust the output:
8168
8169 @table @option
8170 @item --defines[=@var{file}]
8171 Pretend that @code{%defines} was specified, i.e., write an extra output
8172 file containing macro definitions for the token type names defined in
8173 the grammar, as well as a few other declarations. @xref{Decl Summary}.
8174
8175 @item -d
8176 This is the same as @code{--defines} except @code{-d} does not accept a
8177 @var{file} argument since POSIX Yacc requires that @code{-d} can be bundled
8178 with other short options.
8179
8180 @item -b @var{file-prefix}
8181 @itemx --file-prefix=@var{prefix}
8182 Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
8183 for all Bison output file names. @xref{Decl Summary}.
8184
8185 @item -r @var{things}
8186 @itemx --report=@var{things}
8187 Write an extra output file containing verbose description of the comma
8188 separated list of @var{things} among:
8189
8190 @table @code
8191 @item state
8192 Description of the grammar, conflicts (resolved and unresolved), and
8193 parser's automaton.
8194
8195 @item lookahead
8196 Implies @code{state} and augments the description of the automaton with
8197 each rule's lookahead set.
8198
8199 @item itemset
8200 Implies @code{state} and augments the description of the automaton with
8201 the full set of items for each state, instead of its core only.
8202 @end table
8203
8204 @item --report-file=@var{file}
8205 Specify the @var{file} for the verbose description.
8206
8207 @item -v
8208 @itemx --verbose
8209 Pretend that @code{%verbose} was specified, i.e., write an extra output
8210 file containing verbose descriptions of the grammar and
8211 parser. @xref{Decl Summary}.
8212
8213 @item -o @var{file}
8214 @itemx --output=@var{file}
8215 Specify the @var{file} for the parser file.
8216
8217 The other output files' names are constructed from @var{file} as
8218 described under the @samp{-v} and @samp{-d} options.
8219
8220 @item -g [@var{file}]
8221 @itemx --graph[=@var{file}]
8222 Output a graphical representation of the parser's
8223 automaton computed by Bison, in @uref{http://www.graphviz.org/, Graphviz}
8224 @uref{http://www.graphviz.org/doc/info/lang.html, @acronym{DOT}} format.
8225 @code{@var{file}} is optional.
8226 If omitted and the grammar file is @file{foo.y}, the output file will be
8227 @file{foo.dot}.
8228
8229 @item -x [@var{file}]
8230 @itemx --xml[=@var{file}]
8231 Output an XML report of the parser's automaton computed by Bison.
8232 @code{@var{file}} is optional.
8233 If omitted and the grammar file is @file{foo.y}, the output file will be
8234 @file{foo.xml}.
8235 (The current XML schema is experimental and may evolve.
8236 More user feedback will help to stabilize it.)
8237 @end table
8238
8239 @node Option Cross Key
8240 @section Option Cross Key
8241
8242 Here is a list of options, alphabetized by long option, to help you find
8243 the corresponding short option.
8244
8245 @multitable {@option{--defines=@var{defines-file}}} {@option{-D @var{name}[=@var{value}]}} {@code{%nondeterministic-parser}}
8246 @headitem Long Option @tab Short Option @tab Bison Directive
8247 @include cross-options.texi
8248 @end multitable
8249
8250 @node Yacc Library
8251 @section Yacc Library
8252
8253 The Yacc library contains default implementations of the
8254 @code{yyerror} and @code{main} functions. These default
8255 implementations are normally not useful, but @acronym{POSIX} requires
8256 them. To use the Yacc library, link your program with the
8257 @option{-ly} option. Note that Bison's implementation of the Yacc
8258 library is distributed under the terms of the @acronym{GNU} General
8259 Public License (@pxref{Copying}).
8260
8261 If you use the Yacc library's @code{yyerror} function, you should
8262 declare @code{yyerror} as follows:
8263
8264 @example
8265 int yyerror (char const *);
8266 @end example
8267
8268 Bison ignores the @code{int} value returned by this @code{yyerror}.
8269 If you use the Yacc library's @code{main} function, your
8270 @code{yyparse} function should have the following type signature:
8271
8272 @example
8273 int yyparse (void);
8274 @end example
8275
8276 @c ================================================= C++ Bison
8277
8278 @node Other Languages
8279 @chapter Parsers Written In Other Languages
8280
8281 @menu
8282 * C++ Parsers:: The interface to generate C++ parser classes
8283 * Java Parsers:: The interface to generate Java parser classes
8284 @end menu
8285
8286 @node C++ Parsers
8287 @section C++ Parsers
8288
8289 @menu
8290 * C++ Bison Interface:: Asking for C++ parser generation
8291 * C++ Semantic Values:: %union vs. C++
8292 * C++ Location Values:: The position and location classes
8293 * C++ Parser Interface:: Instantiating and running the parser
8294 * C++ Scanner Interface:: Exchanges between yylex and parse
8295 * A Complete C++ Example:: Demonstrating their use
8296 @end menu
8297
8298 @node C++ Bison Interface
8299 @subsection C++ Bison Interface
8300 @c - %skeleton "lalr1.cc"
8301 @c - Always pure
8302 @c - initial action
8303
8304 The C++ deterministic parser is selected using the skeleton directive,
8305 @samp{%skeleton "lalr1.c"}, or the synonymous command-line option
8306 @option{--skeleton=lalr1.c}.
8307 @xref{Decl Summary}.
8308
8309 When run, @command{bison} will create several entities in the @samp{yy}
8310 namespace.
8311 @findex %define namespace
8312 Use the @samp{%define namespace} directive to change the namespace name, see
8313 @ref{Decl Summary}.
8314 The various classes are generated in the following files:
8315
8316 @table @file
8317 @item position.hh
8318 @itemx location.hh
8319 The definition of the classes @code{position} and @code{location},
8320 used for location tracking. @xref{C++ Location Values}.
8321
8322 @item stack.hh
8323 An auxiliary class @code{stack} used by the parser.
8324
8325 @item @var{file}.hh
8326 @itemx @var{file}.cc
8327 (Assuming the extension of the input file was @samp{.yy}.) The
8328 declaration and implementation of the C++ parser class. The basename
8329 and extension of these two files follow the same rules as with regular C
8330 parsers (@pxref{Invocation}).
8331
8332 The header is @emph{mandatory}; you must either pass
8333 @option{-d}/@option{--defines} to @command{bison}, or use the
8334 @samp{%defines} directive.
8335 @end table
8336
8337 All these files are documented using Doxygen; run @command{doxygen}
8338 for a complete and accurate documentation.
8339
8340 @node C++ Semantic Values
8341 @subsection C++ Semantic Values
8342 @c - No objects in unions
8343 @c - YYSTYPE
8344 @c - Printer and destructor
8345
8346 The @code{%union} directive works as for C, see @ref{Union Decl, ,The
8347 Collection of Value Types}. In particular it produces a genuine
8348 @code{union}@footnote{In the future techniques to allow complex types
8349 within pseudo-unions (similar to Boost variants) might be implemented to
8350 alleviate these issues.}, which have a few specific features in C++.
8351 @itemize @minus
8352 @item
8353 The type @code{YYSTYPE} is defined but its use is discouraged: rather
8354 you should refer to the parser's encapsulated type
8355 @code{yy::parser::semantic_type}.
8356 @item
8357 Non POD (Plain Old Data) types cannot be used. C++ forbids any
8358 instance of classes with constructors in unions: only @emph{pointers}
8359 to such objects are allowed.
8360 @end itemize
8361
8362 Because objects have to be stored via pointers, memory is not
8363 reclaimed automatically: using the @code{%destructor} directive is the
8364 only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
8365 Symbols}.
8366
8367
8368 @node C++ Location Values
8369 @subsection C++ Location Values
8370 @c - %locations
8371 @c - class Position
8372 @c - class Location
8373 @c - %define filename_type "const symbol::Symbol"
8374
8375 When the directive @code{%locations} is used, the C++ parser supports
8376 location tracking, see @ref{Locations, , Locations Overview}. Two
8377 auxiliary classes define a @code{position}, a single point in a file,
8378 and a @code{location}, a range composed of a pair of
8379 @code{position}s (possibly spanning several files).
8380
8381 @deftypemethod {position} {std::string*} file
8382 The name of the file. It will always be handled as a pointer, the
8383 parser will never duplicate nor deallocate it. As an experimental
8384 feature you may change it to @samp{@var{type}*} using @samp{%define
8385 filename_type "@var{type}"}.
8386 @end deftypemethod
8387
8388 @deftypemethod {position} {unsigned int} line
8389 The line, starting at 1.
8390 @end deftypemethod
8391
8392 @deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
8393 Advance by @var{height} lines, resetting the column number.
8394 @end deftypemethod
8395
8396 @deftypemethod {position} {unsigned int} column
8397 The column, starting at 0.
8398 @end deftypemethod
8399
8400 @deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
8401 Advance by @var{width} columns, without changing the line number.
8402 @end deftypemethod
8403
8404 @deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
8405 @deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
8406 @deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
8407 @deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
8408 Various forms of syntactic sugar for @code{columns}.
8409 @end deftypemethod
8410
8411 @deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
8412 Report @var{p} on @var{o} like this:
8413 @samp{@var{file}:@var{line}.@var{column}}, or
8414 @samp{@var{line}.@var{column}} if @var{file} is null.
8415 @end deftypemethod
8416
8417 @deftypemethod {location} {position} begin
8418 @deftypemethodx {location} {position} end
8419 The first, inclusive, position of the range, and the first beyond.
8420 @end deftypemethod
8421
8422 @deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
8423 @deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
8424 Advance the @code{end} position.
8425 @end deftypemethod
8426
8427 @deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
8428 @deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
8429 @deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
8430 Various forms of syntactic sugar.
8431 @end deftypemethod
8432
8433 @deftypemethod {location} {void} step ()
8434 Move @code{begin} onto @code{end}.
8435 @end deftypemethod
8436
8437
8438 @node C++ Parser Interface
8439 @subsection C++ Parser Interface
8440 @c - define parser_class_name
8441 @c - Ctor
8442 @c - parse, error, set_debug_level, debug_level, set_debug_stream,
8443 @c debug_stream.
8444 @c - Reporting errors
8445
8446 The output files @file{@var{output}.hh} and @file{@var{output}.cc}
8447 declare and define the parser class in the namespace @code{yy}. The
8448 class name defaults to @code{parser}, but may be changed using
8449 @samp{%define parser_class_name "@var{name}"}. The interface of
8450 this class is detailed below. It can be extended using the
8451 @code{%parse-param} feature: its semantics is slightly changed since
8452 it describes an additional member of the parser class, and an
8453 additional argument for its constructor.
8454
8455 @defcv {Type} {parser} {semantic_value_type}
8456 @defcvx {Type} {parser} {location_value_type}
8457 The types for semantics value and locations.
8458 @end defcv
8459
8460 @deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
8461 Build a new parser object. There are no arguments by default, unless
8462 @samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
8463 @end deftypemethod
8464
8465 @deftypemethod {parser} {int} parse ()
8466 Run the syntactic analysis, and return 0 on success, 1 otherwise.
8467 @end deftypemethod
8468
8469 @deftypemethod {parser} {std::ostream&} debug_stream ()
8470 @deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
8471 Get or set the stream used for tracing the parsing. It defaults to
8472 @code{std::cerr}.
8473 @end deftypemethod
8474
8475 @deftypemethod {parser} {debug_level_type} debug_level ()
8476 @deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
8477 Get or set the tracing level. Currently its value is either 0, no trace,
8478 or nonzero, full tracing.
8479 @end deftypemethod
8480
8481 @deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
8482 The definition for this member function must be supplied by the user:
8483 the parser uses it to report a parser error occurring at @var{l},
8484 described by @var{m}.
8485 @end deftypemethod
8486
8487
8488 @node C++ Scanner Interface
8489 @subsection C++ Scanner Interface
8490 @c - prefix for yylex.
8491 @c - Pure interface to yylex
8492 @c - %lex-param
8493
8494 The parser invokes the scanner by calling @code{yylex}. Contrary to C
8495 parsers, C++ parsers are always pure: there is no point in using the
8496 @code{%define api.pure} directive. Therefore the interface is as follows.
8497
8498 @deftypemethod {parser} {int} yylex (semantic_value_type& @var{yylval}, location_type& @var{yylloc}, @var{type1} @var{arg1}, ...)
8499 Return the next token. Its type is the return value, its semantic
8500 value and location being @var{yylval} and @var{yylloc}. Invocations of
8501 @samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
8502 @end deftypemethod
8503
8504
8505 @node A Complete C++ Example
8506 @subsection A Complete C++ Example
8507
8508 This section demonstrates the use of a C++ parser with a simple but
8509 complete example. This example should be available on your system,
8510 ready to compile, in the directory @dfn{../bison/examples/calc++}. It
8511 focuses on the use of Bison, therefore the design of the various C++
8512 classes is very naive: no accessors, no encapsulation of members etc.
8513 We will use a Lex scanner, and more precisely, a Flex scanner, to
8514 demonstrate the various interaction. A hand written scanner is
8515 actually easier to interface with.
8516
8517 @menu
8518 * Calc++ --- C++ Calculator:: The specifications
8519 * Calc++ Parsing Driver:: An active parsing context
8520 * Calc++ Parser:: A parser class
8521 * Calc++ Scanner:: A pure C++ Flex scanner
8522 * Calc++ Top Level:: Conducting the band
8523 @end menu
8524
8525 @node Calc++ --- C++ Calculator
8526 @subsubsection Calc++ --- C++ Calculator
8527
8528 Of course the grammar is dedicated to arithmetics, a single
8529 expression, possibly preceded by variable assignments. An
8530 environment containing possibly predefined variables such as
8531 @code{one} and @code{two}, is exchanged with the parser. An example
8532 of valid input follows.
8533
8534 @example
8535 three := 3
8536 seven := one + two * three
8537 seven * seven
8538 @end example
8539
8540 @node Calc++ Parsing Driver
8541 @subsubsection Calc++ Parsing Driver
8542 @c - An env
8543 @c - A place to store error messages
8544 @c - A place for the result
8545
8546 To support a pure interface with the parser (and the scanner) the
8547 technique of the ``parsing context'' is convenient: a structure
8548 containing all the data to exchange. Since, in addition to simply
8549 launch the parsing, there are several auxiliary tasks to execute (open
8550 the file for parsing, instantiate the parser etc.), we recommend
8551 transforming the simple parsing context structure into a fully blown
8552 @dfn{parsing driver} class.
8553
8554 The declaration of this driver class, @file{calc++-driver.hh}, is as
8555 follows. The first part includes the CPP guard and imports the
8556 required standard library components, and the declaration of the parser
8557 class.
8558
8559 @comment file: calc++-driver.hh
8560 @example
8561 #ifndef CALCXX_DRIVER_HH
8562 # define CALCXX_DRIVER_HH
8563 # include <string>
8564 # include <map>
8565 # include "calc++-parser.hh"
8566 @end example
8567
8568
8569 @noindent
8570 Then comes the declaration of the scanning function. Flex expects
8571 the signature of @code{yylex} to be defined in the macro
8572 @code{YY_DECL}, and the C++ parser expects it to be declared. We can
8573 factor both as follows.
8574
8575 @comment file: calc++-driver.hh
8576 @example
8577 // Tell Flex the lexer's prototype ...
8578 # define YY_DECL \
8579 yy::calcxx_parser::token_type \
8580 yylex (yy::calcxx_parser::semantic_type* yylval, \
8581 yy::calcxx_parser::location_type* yylloc, \
8582 calcxx_driver& driver)
8583 // ... and declare it for the parser's sake.
8584 YY_DECL;
8585 @end example
8586
8587 @noindent
8588 The @code{calcxx_driver} class is then declared with its most obvious
8589 members.
8590
8591 @comment file: calc++-driver.hh
8592 @example
8593 // Conducting the whole scanning and parsing of Calc++.
8594 class calcxx_driver
8595 @{
8596 public:
8597 calcxx_driver ();
8598 virtual ~calcxx_driver ();
8599
8600 std::map<std::string, int> variables;
8601
8602 int result;
8603 @end example
8604
8605 @noindent
8606 To encapsulate the coordination with the Flex scanner, it is useful to
8607 have two members function to open and close the scanning phase.
8608
8609 @comment file: calc++-driver.hh
8610 @example
8611 // Handling the scanner.
8612 void scan_begin ();
8613 void scan_end ();
8614 bool trace_scanning;
8615 @end example
8616
8617 @noindent
8618 Similarly for the parser itself.
8619
8620 @comment file: calc++-driver.hh
8621 @example
8622 // Run the parser. Return 0 on success.
8623 int parse (const std::string& f);
8624 std::string file;
8625 bool trace_parsing;
8626 @end example
8627
8628 @noindent
8629 To demonstrate pure handling of parse errors, instead of simply
8630 dumping them on the standard error output, we will pass them to the
8631 compiler driver using the following two member functions. Finally, we
8632 close the class declaration and CPP guard.
8633
8634 @comment file: calc++-driver.hh
8635 @example
8636 // Error handling.
8637 void error (const yy::location& l, const std::string& m);
8638 void error (const std::string& m);
8639 @};
8640 #endif // ! CALCXX_DRIVER_HH
8641 @end example
8642
8643 The implementation of the driver is straightforward. The @code{parse}
8644 member function deserves some attention. The @code{error} functions
8645 are simple stubs, they should actually register the located error
8646 messages and set error state.
8647
8648 @comment file: calc++-driver.cc
8649 @example
8650 #include "calc++-driver.hh"
8651 #include "calc++-parser.hh"
8652
8653 calcxx_driver::calcxx_driver ()
8654 : trace_scanning (false), trace_parsing (false)
8655 @{
8656 variables["one"] = 1;
8657 variables["two"] = 2;
8658 @}
8659
8660 calcxx_driver::~calcxx_driver ()
8661 @{
8662 @}
8663
8664 int
8665 calcxx_driver::parse (const std::string &f)
8666 @{
8667 file = f;
8668 scan_begin ();
8669 yy::calcxx_parser parser (*this);
8670 parser.set_debug_level (trace_parsing);
8671 int res = parser.parse ();
8672 scan_end ();
8673 return res;
8674 @}
8675
8676 void
8677 calcxx_driver::error (const yy::location& l, const std::string& m)
8678 @{
8679 std::cerr << l << ": " << m << std::endl;
8680 @}
8681
8682 void
8683 calcxx_driver::error (const std::string& m)
8684 @{
8685 std::cerr << m << std::endl;
8686 @}
8687 @end example
8688
8689 @node Calc++ Parser
8690 @subsubsection Calc++ Parser
8691
8692 The parser definition file @file{calc++-parser.yy} starts by asking for
8693 the C++ deterministic parser skeleton, the creation of the parser header
8694 file, and specifies the name of the parser class.
8695 Because the C++ skeleton changed several times, it is safer to require
8696 the version you designed the grammar for.
8697
8698 @comment file: calc++-parser.yy
8699 @example
8700 %skeleton "lalr1.cc" /* -*- C++ -*- */
8701 %require "@value{VERSION}"
8702 %defines
8703 %define parser_class_name "calcxx_parser"
8704 @end example
8705
8706 @noindent
8707 @findex %code requires
8708 Then come the declarations/inclusions needed to define the
8709 @code{%union}. Because the parser uses the parsing driver and
8710 reciprocally, both cannot include the header of the other. Because the
8711 driver's header needs detailed knowledge about the parser class (in
8712 particular its inner types), it is the parser's header which will simply
8713 use a forward declaration of the driver.
8714 @xref{Decl Summary, ,%code}.
8715
8716 @comment file: calc++-parser.yy
8717 @example
8718 %code requires @{
8719 # include <string>
8720 class calcxx_driver;
8721 @}
8722 @end example
8723
8724 @noindent
8725 The driver is passed by reference to the parser and to the scanner.
8726 This provides a simple but effective pure interface, not relying on
8727 global variables.
8728
8729 @comment file: calc++-parser.yy
8730 @example
8731 // The parsing context.
8732 %parse-param @{ calcxx_driver& driver @}
8733 %lex-param @{ calcxx_driver& driver @}
8734 @end example
8735
8736 @noindent
8737 Then we request the location tracking feature, and initialize the
8738 first location's file name. Afterwards new locations are computed
8739 relatively to the previous locations: the file name will be
8740 automatically propagated.
8741
8742 @comment file: calc++-parser.yy
8743 @example
8744 %locations
8745 %initial-action
8746 @{
8747 // Initialize the initial location.
8748 @@$.begin.filename = @@$.end.filename = &driver.file;
8749 @};
8750 @end example
8751
8752 @noindent
8753 Use the two following directives to enable parser tracing and verbose
8754 error messages.
8755
8756 @comment file: calc++-parser.yy
8757 @example
8758 %define parse.trace
8759 %define error-verbose
8760 @end example
8761
8762 @noindent
8763 Semantic values cannot use ``real'' objects, but only pointers to
8764 them.
8765
8766 @comment file: calc++-parser.yy
8767 @example
8768 // Symbols.
8769 %union
8770 @{
8771 int ival;
8772 std::string *sval;
8773 @};
8774 @end example
8775
8776 @noindent
8777 @findex %code
8778 The code between @samp{%code @{} and @samp{@}} is output in the
8779 @file{*.cc} file; it needs detailed knowledge about the driver.
8780
8781 @comment file: calc++-parser.yy
8782 @example
8783 %code @{
8784 # include "calc++-driver.hh"
8785 @}
8786 @end example
8787
8788
8789 @noindent
8790 The token numbered as 0 corresponds to end of file; the following line
8791 allows for nicer error messages referring to ``end of file'' instead
8792 of ``$end''. Similarly user friendly named are provided for each
8793 symbol. Note that the tokens names are prefixed by @code{TOKEN_} to
8794 avoid name clashes.
8795
8796 @comment file: calc++-parser.yy
8797 @example
8798 %token END 0 "end of file"
8799 %token ASSIGN ":="
8800 %token <sval> IDENTIFIER "identifier"
8801 %token <ival> NUMBER "number"
8802 %type <ival> exp
8803 @end example
8804
8805 @noindent
8806 To enable memory deallocation during error recovery, use
8807 @code{%destructor}.
8808
8809 @c FIXME: Document %printer, and mention that it takes a braced-code operand.
8810 @comment file: calc++-parser.yy
8811 @example
8812 %printer @{ debug_stream () << *$$; @} "identifier"
8813 %destructor @{ delete $$; @} "identifier"
8814
8815 %printer @{ debug_stream () << $$; @} <ival>
8816 @end example
8817
8818 @noindent
8819 The grammar itself is straightforward.
8820
8821 @comment file: calc++-parser.yy
8822 @example
8823 %%
8824 %start unit;
8825 unit: assignments exp @{ driver.result = $2; @};
8826
8827 assignments: assignments assignment @{@}
8828 | /* Nothing. */ @{@};
8829
8830 assignment:
8831 "identifier" ":=" exp
8832 @{ driver.variables[*$1] = $3; delete $1; @};
8833
8834 %left '+' '-';
8835 %left '*' '/';
8836 exp: exp '+' exp @{ $$ = $1 + $3; @}
8837 | exp '-' exp @{ $$ = $1 - $3; @}
8838 | exp '*' exp @{ $$ = $1 * $3; @}
8839 | exp '/' exp @{ $$ = $1 / $3; @}
8840 | '(' exp ')' @{ $$ = $2; @}
8841 | "identifier" @{ $$ = driver.variables[*$1]; delete $1; @}
8842 | "number" @{ $$ = $1; @};
8843 %%
8844 @end example
8845
8846 @noindent
8847 Finally the @code{error} member function registers the errors to the
8848 driver.
8849
8850 @comment file: calc++-parser.yy
8851 @example
8852 void
8853 yy::calcxx_parser::error (const yy::calcxx_parser::location_type& l,
8854 const std::string& m)
8855 @{
8856 driver.error (l, m);
8857 @}
8858 @end example
8859
8860 @node Calc++ Scanner
8861 @subsubsection Calc++ Scanner
8862
8863 The Flex scanner first includes the driver declaration, then the
8864 parser's to get the set of defined tokens.
8865
8866 @comment file: calc++-scanner.ll
8867 @example
8868 %@{ /* -*- C++ -*- */
8869 # include <cstdlib>
8870 # include <errno.h>
8871 # include <limits.h>
8872 # include <string>
8873 # include "calc++-driver.hh"
8874 # include "calc++-parser.hh"
8875
8876 /* Work around an incompatibility in flex (at least versions
8877 2.5.31 through 2.5.33): it generates code that does
8878 not conform to C89. See Debian bug 333231
8879 <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */
8880 # undef yywrap
8881 # define yywrap() 1
8882
8883 /* By default yylex returns int, we use token_type.
8884 Unfortunately yyterminate by default returns 0, which is
8885 not of token_type. */
8886 #define yyterminate() return token::END
8887 %@}
8888 @end example
8889
8890 @noindent
8891 Because there is no @code{#include}-like feature we don't need
8892 @code{yywrap}, we don't need @code{unput} either, and we parse an
8893 actual file, this is not an interactive session with the user.
8894 Finally we enable the scanner tracing features.
8895
8896 @comment file: calc++-scanner.ll
8897 @example
8898 %option noyywrap nounput batch debug
8899 @end example
8900
8901 @noindent
8902 Abbreviations allow for more readable rules.
8903
8904 @comment file: calc++-scanner.ll
8905 @example
8906 id [a-zA-Z][a-zA-Z_0-9]*
8907 int [0-9]+
8908 blank [ \t]
8909 @end example
8910
8911 @noindent
8912 The following paragraph suffices to track locations accurately. Each
8913 time @code{yylex} is invoked, the begin position is moved onto the end
8914 position. Then when a pattern is matched, the end position is
8915 advanced of its width. In case it matched ends of lines, the end
8916 cursor is adjusted, and each time blanks are matched, the begin cursor
8917 is moved onto the end cursor to effectively ignore the blanks
8918 preceding tokens. Comments would be treated equally.
8919
8920 @comment file: calc++-scanner.ll
8921 @example
8922 %@{
8923 # define YY_USER_ACTION yylloc->columns (yyleng);
8924 %@}
8925 %%
8926 %@{
8927 yylloc->step ();
8928 %@}
8929 @{blank@}+ yylloc->step ();
8930 [\n]+ yylloc->lines (yyleng); yylloc->step ();
8931 @end example
8932
8933 @noindent
8934 The rules are simple, just note the use of the driver to report errors.
8935 It is convenient to use a typedef to shorten
8936 @code{yy::calcxx_parser::token::identifier} into
8937 @code{token::identifier} for instance.
8938
8939 @comment file: calc++-scanner.ll
8940 @example
8941 %@{
8942 typedef yy::calcxx_parser::token token;
8943 %@}
8944 /* Convert ints to the actual type of tokens. */
8945 [-+*/()] return yy::calcxx_parser::token_type (yytext[0]);
8946 ":=" return token::ASSIGN;
8947 @{int@} @{
8948 errno = 0;
8949 long n = strtol (yytext, NULL, 10);
8950 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
8951 driver.error (*yylloc, "integer is out of range");
8952 yylval->ival = n;
8953 return token::NUMBER;
8954 @}
8955 @{id@} yylval->sval = new std::string (yytext); return token::IDENTIFIER;
8956 . driver.error (*yylloc, "invalid character");
8957 %%
8958 @end example
8959
8960 @noindent
8961 Finally, because the scanner related driver's member function depend
8962 on the scanner's data, it is simpler to implement them in this file.
8963
8964 @comment file: calc++-scanner.ll
8965 @example
8966 void
8967 calcxx_driver::scan_begin ()
8968 @{
8969 yy_flex_debug = trace_scanning;
8970 if (file == "-")
8971 yyin = stdin;
8972 else if (!(yyin = fopen (file.c_str (), "r")))
8973 @{
8974 error (std::string ("cannot open ") + file);
8975 exit (1);
8976 @}
8977 @}
8978
8979 void
8980 calcxx_driver::scan_end ()
8981 @{
8982 fclose (yyin);
8983 @}
8984 @end example
8985
8986 @node Calc++ Top Level
8987 @subsubsection Calc++ Top Level
8988
8989 The top level file, @file{calc++.cc}, poses no problem.
8990
8991 @comment file: calc++.cc
8992 @example
8993 #include <iostream>
8994 #include "calc++-driver.hh"
8995
8996 int
8997 main (int argc, char *argv[])
8998 @{
8999 int res = 0;
9000 calcxx_driver driver;
9001 for (++argv; argv[0]; ++argv)
9002 if (*argv == std::string ("-p"))
9003 driver.trace_parsing = true;
9004 else if (*argv == std::string ("-s"))
9005 driver.trace_scanning = true;
9006 else if (!driver.parse (*argv))
9007 std::cout << driver.result << std::endl;
9008 else
9009 res = 1;
9010 return res;
9011 @}
9012 @end example
9013
9014 @node Java Parsers
9015 @section Java Parsers
9016
9017 @menu
9018 * Java Bison Interface:: Asking for Java parser generation
9019 * Java Semantic Values:: %type and %token vs. Java
9020 * Java Location Values:: The position and location classes
9021 * Java Parser Interface:: Instantiating and running the parser
9022 * Java Scanner Interface:: Specifying the scanner for the parser
9023 * Java Action Features:: Special features for use in actions
9024 * Java Differences:: Differences between C/C++ and Java Grammars
9025 * Java Declarations Summary:: List of Bison declarations used with Java
9026 @end menu
9027
9028 @node Java Bison Interface
9029 @subsection Java Bison Interface
9030 @c - %language "Java"
9031
9032 (The current Java interface is experimental and may evolve.
9033 More user feedback will help to stabilize it.)
9034
9035 The Java parser skeletons are selected using the @code{%language "Java"}
9036 directive or the @option{-L java}/@option{--language=java} option.
9037
9038 @c FIXME: Documented bug.
9039 When generating a Java parser, @code{bison @var{basename}.y} will create
9040 a single Java source file named @file{@var{basename}.java}. Using an
9041 input file without a @file{.y} suffix is currently broken. The basename
9042 of the output file can be changed by the @code{%file-prefix} directive
9043 or the @option{-p}/@option{--name-prefix} option. The entire output file
9044 name can be changed by the @code{%output} directive or the
9045 @option{-o}/@option{--output} option. The output file contains a single
9046 class for the parser.
9047
9048 You can create documentation for generated parsers using Javadoc.
9049
9050 Contrary to C parsers, Java parsers do not use global variables; the
9051 state of the parser is always local to an instance of the parser class.
9052 Therefore, all Java parsers are ``pure'', and the @code{%pure-parser}
9053 and @code{%define api.pure} directives does not do anything when used in
9054 Java.
9055
9056 Push parsers are currently unsupported in Java and @code{%define
9057 api.push-pull} have no effect.
9058
9059 @acronym{GLR} parsers are currently unsupported in Java. Do not use the
9060 @code{glr-parser} directive.
9061
9062 No header file can be generated for Java parsers. Do not use the
9063 @code{%defines} directive or the @option{-d}/@option{--defines} options.
9064
9065 @c FIXME: Possible code change.
9066 Currently, support for tracing is always compiled
9067 in. Thus the @samp{%define parse.trace} and @samp{%token-table}
9068 directives and the
9069 @option{-t}/@option{--debug} and @option{-k}/@option{--token-table}
9070 options have no effect. This may change in the future to eliminate
9071 unused code in the generated parser, so use @samp{%define parse.trace}
9072 explicitly
9073 if needed. Also, in the future the
9074 @code{%token-table} directive might enable a public interface to
9075 access the token names and codes.
9076
9077 Getting a ``code too large'' error from the Java compiler means the code
9078 hit the 64KB bytecode per method limination of the Java class file.
9079 Try reducing the amount of code in actions and static initializers;
9080 otherwise, report a bug so that the parser skeleton will be improved.
9081
9082
9083 @node Java Semantic Values
9084 @subsection Java Semantic Values
9085 @c - No %union, specify type in %type/%token.
9086 @c - YYSTYPE
9087 @c - Printer and destructor
9088
9089 There is no @code{%union} directive in Java parsers. Instead, the
9090 semantic values' types (class names) should be specified in the
9091 @code{%type} or @code{%token} directive:
9092
9093 @example
9094 %type <Expression> expr assignment_expr term factor
9095 %type <Integer> number
9096 @end example
9097
9098 By default, the semantic stack is declared to have @code{Object} members,
9099 which means that the class types you specify can be of any class.
9100 To improve the type safety of the parser, you can declare the common
9101 superclass of all the semantic values using the @code{%define stype}
9102 directive. For example, after the following declaration:
9103
9104 @example
9105 %define stype "ASTNode"
9106 @end example
9107
9108 @noindent
9109 any @code{%type} or @code{%token} specifying a semantic type which
9110 is not a subclass of ASTNode, will cause a compile-time error.
9111
9112 @c FIXME: Documented bug.
9113 Types used in the directives may be qualified with a package name.
9114 Primitive data types are accepted for Java version 1.5 or later. Note
9115 that in this case the autoboxing feature of Java 1.5 will be used.
9116 Generic types may not be used; this is due to a limitation in the
9117 implementation of Bison, and may change in future releases.
9118
9119 Java parsers do not support @code{%destructor}, since the language
9120 adopts garbage collection. The parser will try to hold references
9121 to semantic values for as little time as needed.
9122
9123 Java parsers do not support @code{%printer}, as @code{toString()}
9124 can be used to print the semantic values. This however may change
9125 (in a backwards-compatible way) in future versions of Bison.
9126
9127
9128 @node Java Location Values
9129 @subsection Java Location Values
9130 @c - %locations
9131 @c - class Position
9132 @c - class Location
9133
9134 When the directive @code{%locations} is used, the Java parser
9135 supports location tracking, see @ref{Locations, , Locations Overview}.
9136 An auxiliary user-defined class defines a @dfn{position}, a single point
9137 in a file; Bison itself defines a class representing a @dfn{location},
9138 a range composed of a pair of positions (possibly spanning several
9139 files). The location class is an inner class of the parser; the name
9140 is @code{Location} by default, and may also be renamed using
9141 @code{%define location_type "@var{class-name}}.
9142
9143 The location class treats the position as a completely opaque value.
9144 By default, the class name is @code{Position}, but this can be changed
9145 with @code{%define position_type "@var{class-name}"}. This class must
9146 be supplied by the user.
9147
9148
9149 @deftypeivar {Location} {Position} begin
9150 @deftypeivarx {Location} {Position} end
9151 The first, inclusive, position of the range, and the first beyond.
9152 @end deftypeivar
9153
9154 @deftypeop {Constructor} {Location} {} Location (Position @var{loc})
9155 Create a @code{Location} denoting an empty range located at a given point.
9156 @end deftypeop
9157
9158 @deftypeop {Constructor} {Location} {} Location (Position @var{begin}, Position @var{end})
9159 Create a @code{Location} from the endpoints of the range.
9160 @end deftypeop
9161
9162 @deftypemethod {Location} {String} toString ()
9163 Prints the range represented by the location. For this to work
9164 properly, the position class should override the @code{equals} and
9165 @code{toString} methods appropriately.
9166 @end deftypemethod
9167
9168
9169 @node Java Parser Interface
9170 @subsection Java Parser Interface
9171 @c - define parser_class_name
9172 @c - Ctor
9173 @c - parse, error, set_debug_level, debug_level, set_debug_stream,
9174 @c debug_stream.
9175 @c - Reporting errors
9176
9177 The name of the generated parser class defaults to @code{YYParser}. The
9178 @code{YY} prefix may be changed using the @code{%name-prefix} directive
9179 or the @option{-p}/@option{--name-prefix} option. Alternatively, use
9180 @code{%define parser_class_name "@var{name}"} to give a custom name to
9181 the class. The interface of this class is detailed below.
9182
9183 By default, the parser class has package visibility. A declaration
9184 @code{%define public} will change to public visibility. Remember that,
9185 according to the Java language specification, the name of the @file{.java}
9186 file should match the name of the class in this case. Similarly, you can
9187 use @code{abstract}, @code{final} and @code{strictfp} with the
9188 @code{%define} declaration to add other modifiers to the parser class.
9189 A single @code{%define annotations "@var{annotations}"} directive can
9190 be used to add any number of annotations to the parser class.
9191
9192 The Java package name of the parser class can be specified using the
9193 @code{%define package} directive. The superclass and the implemented
9194 interfaces of the parser class can be specified with the @code{%define
9195 extends} and @code{%define implements} directives.
9196
9197 The parser class defines an inner class, @code{Location}, that is used
9198 for location tracking (see @ref{Java Location Values}), and a inner
9199 interface, @code{Lexer} (see @ref{Java Scanner Interface}). Other than
9200 these inner class/interface, and the members described in the interface
9201 below, all the other members and fields are preceded with a @code{yy} or
9202 @code{YY} prefix to avoid clashes with user code.
9203
9204 The parser class can be extended using the @code{%parse-param}
9205 directive. Each occurrence of the directive will add a @code{protected
9206 final} field to the parser class, and an argument to its constructor,
9207 which initialize them automatically.
9208
9209 @deftypeop {Constructor} {YYParser} {} YYParser (@var{lex_param}, @dots{}, @var{parse_param}, @dots{})
9210 Build a new parser object with embedded @code{%code lexer}. There are
9211 no parameters, unless @code{%parse-param}s and/or @code{%lex-param}s are
9212 used.
9213
9214 Use @code{%code init} for code added to the start of the constructor
9215 body. This is especially useful to initialize superclasses. Use
9216 @code{%define init_throws} to specify any uncatch exceptions.
9217 @end deftypeop
9218
9219 @deftypeop {Constructor} {YYParser} {} YYParser (Lexer @var{lexer}, @var{parse_param}, @dots{})
9220 Build a new parser object using the specified scanner. There are no
9221 additional parameters unless @code{%parse-param}s are used.
9222
9223 If the scanner is defined by @code{%code lexer}, this constructor is
9224 declared @code{protected} and is called automatically with a scanner
9225 created with the correct @code{%lex-param}s.
9226
9227 Use @code{%code init} for code added to the start of the constructor
9228 body. This is especially useful to initialize superclasses. Use
9229 @code{%define init_throws} to specify any uncatch exceptions.
9230 @end deftypeop
9231
9232 @deftypemethod {YYParser} {boolean} parse ()
9233 Run the syntactic analysis, and return @code{true} on success,
9234 @code{false} otherwise.
9235 @end deftypemethod
9236
9237 @deftypemethod {YYParser} {boolean} getErrorVerbose ()
9238 @deftypemethodx {YYParser} {void} setErrorVerbose (boolean @var{verbose})
9239 Get or set the option to produce verbose error messages. These are only
9240 available with the @code{%define error-verbose} directive, which also turn on
9241 verbose error messages.
9242 @end deftypemethod
9243
9244 @deftypemethod {YYParser} {void} yyerror (String @var{msg})
9245 @deftypemethodx {YYParser} {void} yyerror (Position @var{pos}, String @var{msg})
9246 @deftypemethodx {YYParser} {void} yyerror (Location @var{loc}, String @var{msg})
9247 Print an error message using the @code{yyerror} method of the scanner
9248 instance in use. The @code{Location} and @code{Position} parameters are
9249 available only if location tracking is active.
9250 @end deftypemethod
9251
9252 @deftypemethod {YYParser} {boolean} recovering ()
9253 During the syntactic analysis, return @code{true} if recovering
9254 from a syntax error.
9255 @xref{Error Recovery}.
9256 @end deftypemethod
9257
9258 @deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
9259 @deftypemethodx {YYParser} {void} setDebugStream (java.io.printStream @var{o})
9260 Get or set the stream used for tracing the parsing. It defaults to
9261 @code{System.err}.
9262 @end deftypemethod
9263
9264 @deftypemethod {YYParser} {int} getDebugLevel ()
9265 @deftypemethodx {YYParser} {void} setDebugLevel (int @var{l})
9266 Get or set the tracing level. Currently its value is either 0, no trace,
9267 or nonzero, full tracing.
9268 @end deftypemethod
9269
9270 @deftypecv {Constant} {YYParser} {String} {bisonVersion}
9271 @deftypecvx {Constant} {YYParser} {String} {bisonSkeleton}
9272 Identify the Bison version and skeleton used to generate this parser.
9273 @end deftypecv
9274
9275
9276 @node Java Scanner Interface
9277 @subsection Java Scanner Interface
9278 @c - %code lexer
9279 @c - %lex-param
9280 @c - Lexer interface
9281
9282 There are two possible ways to interface a Bison-generated Java parser
9283 with a scanner: the scanner may be defined by @code{%code lexer}, or
9284 defined elsewhere. In either case, the scanner has to implement the
9285 @code{Lexer} inner interface of the parser class. This interface also
9286 contain constants for all user-defined token names and the predefined
9287 @code{EOF} token.
9288
9289 In the first case, the body of the scanner class is placed in
9290 @code{%code lexer} blocks. If you want to pass parameters from the
9291 parser constructor to the scanner constructor, specify them with
9292 @code{%lex-param}; they are passed before @code{%parse-param}s to the
9293 constructor.
9294
9295 In the second case, the scanner has to implement the @code{Lexer} interface,
9296 which is defined within the parser class (e.g., @code{YYParser.Lexer}).
9297 The constructor of the parser object will then accept an object
9298 implementing the interface; @code{%lex-param} is not used in this
9299 case.
9300
9301 In both cases, the scanner has to implement the following methods.
9302
9303 @deftypemethod {Lexer} {void} yyerror (Location @var{loc}, String @var{msg})
9304 This method is defined by the user to emit an error message. The first
9305 parameter is omitted if location tracking is not active. Its type can be
9306 changed using @code{%define location_type "@var{class-name}".}
9307 @end deftypemethod
9308
9309 @deftypemethod {Lexer} {int} yylex ()
9310 Return the next token. Its type is the return value, its semantic
9311 value and location are saved and returned by the ther methods in the
9312 interface.
9313
9314 Use @code{%define lex_throws} to specify any uncaught exceptions.
9315 Default is @code{java.io.IOException}.
9316 @end deftypemethod
9317
9318 @deftypemethod {Lexer} {Position} getStartPos ()
9319 @deftypemethodx {Lexer} {Position} getEndPos ()
9320 Return respectively the first position of the last token that
9321 @code{yylex} returned, and the first position beyond it. These
9322 methods are not needed unless location tracking is active.
9323
9324 The return type can be changed using @code{%define position_type
9325 "@var{class-name}".}
9326 @end deftypemethod
9327
9328 @deftypemethod {Lexer} {Object} getLVal ()
9329 Return the semantical value of the last token that yylex returned.
9330
9331 The return type can be changed using @code{%define stype
9332 "@var{class-name}".}
9333 @end deftypemethod
9334
9335
9336 @node Java Action Features
9337 @subsection Special Features for Use in Java Actions
9338
9339 The following special constructs can be uses in Java actions.
9340 Other analogous C action features are currently unavailable for Java.
9341
9342 Use @code{%define throws} to specify any uncaught exceptions from parser
9343 actions, and initial actions specified by @code{%initial-action}.
9344
9345 @defvar $@var{n}
9346 The semantic value for the @var{n}th component of the current rule.
9347 This may not be assigned to.
9348 @xref{Java Semantic Values}.
9349 @end defvar
9350
9351 @defvar $<@var{typealt}>@var{n}
9352 Like @code{$@var{n}} but specifies a alternative type @var{typealt}.
9353 @xref{Java Semantic Values}.
9354 @end defvar
9355
9356 @defvar $$
9357 The semantic value for the grouping made by the current rule. As a
9358 value, this is in the base type (@code{Object} or as specified by
9359 @code{%define stype}) as in not cast to the declared subtype because
9360 casts are not allowed on the left-hand side of Java assignments.
9361 Use an explicit Java cast if the correct subtype is needed.
9362 @xref{Java Semantic Values}.
9363 @end defvar
9364
9365 @defvar $<@var{typealt}>$
9366 Same as @code{$$} since Java always allow assigning to the base type.
9367 Perhaps we should use this and @code{$<>$} for the value and @code{$$}
9368 for setting the value but there is currently no easy way to distinguish
9369 these constructs.
9370 @xref{Java Semantic Values}.
9371 @end defvar
9372
9373 @defvar @@@var{n}
9374 The location information of the @var{n}th component of the current rule.
9375 This may not be assigned to.
9376 @xref{Java Location Values}.
9377 @end defvar
9378
9379 @defvar @@$
9380 The location information of the grouping made by the current rule.
9381 @xref{Java Location Values}.
9382 @end defvar
9383
9384 @deffn {Statement} {return YYABORT;}
9385 Return immediately from the parser, indicating failure.
9386 @xref{Java Parser Interface}.
9387 @end deffn
9388
9389 @deffn {Statement} {return YYACCEPT;}
9390 Return immediately from the parser, indicating success.
9391 @xref{Java Parser Interface}.
9392 @end deffn
9393
9394 @deffn {Statement} {return YYERROR;}
9395 Start error recovery without printing an error message.
9396 @xref{Error Recovery}.
9397 @end deffn
9398
9399 @deffn {Statement} {return YYFAIL;}
9400 Print an error message and start error recovery.
9401 @xref{Error Recovery}.
9402 @end deffn
9403
9404 @deftypefn {Function} {boolean} recovering ()
9405 Return whether error recovery is being done. In this state, the parser
9406 reads token until it reaches a known state, and then restarts normal
9407 operation.
9408 @xref{Error Recovery}.
9409 @end deftypefn
9410
9411 @deftypefn {Function} {void} yyerror (String @var{msg})
9412 @deftypefnx {Function} {void} yyerror (Position @var{loc}, String @var{msg})
9413 @deftypefnx {Function} {void} yyerror (Location @var{loc}, String @var{msg})
9414 Print an error message using the @code{yyerror} method of the scanner
9415 instance in use. The @code{Location} and @code{Position} parameters are
9416 available only if location tracking is active.
9417 @end deftypefn
9418
9419
9420 @node Java Differences
9421 @subsection Differences between C/C++ and Java Grammars
9422
9423 The different structure of the Java language forces several differences
9424 between C/C++ grammars, and grammars designed for Java parsers. This
9425 section summarizes these differences.
9426
9427 @itemize
9428 @item
9429 Java lacks a preprocessor, so the @code{YYERROR}, @code{YYACCEPT},
9430 @code{YYABORT} symbols (@pxref{Table of Symbols}) cannot obviously be
9431 macros. Instead, they should be preceded by @code{return} when they
9432 appear in an action. The actual definition of these symbols is
9433 opaque to the Bison grammar, and it might change in the future. The
9434 only meaningful operation that you can do, is to return them.
9435 See @pxref{Java Action Features}.
9436
9437 Note that of these three symbols, only @code{YYACCEPT} and
9438 @code{YYABORT} will cause a return from the @code{yyparse}
9439 method@footnote{Java parsers include the actions in a separate
9440 method than @code{yyparse} in order to have an intuitive syntax that
9441 corresponds to these C macros.}.
9442
9443 @item
9444 Java lacks unions, so @code{%union} has no effect. Instead, semantic
9445 values have a common base type: @code{Object} or as specified by
9446 @code{%define stype}. Angle backets on @code{%token}, @code{type},
9447 @code{$@var{n}} and @code{$$} specify subtypes rather than fields of
9448 an union. The type of @code{$$}, even with angle brackets, is the base
9449 type since Java casts are not allow on the left-hand side of assignments.
9450 Also, @code{$@var{n}} and @code{@@@var{n}} are not allowed on the
9451 left-hand side of assignments. See @pxref{Java Semantic Values} and
9452 @pxref{Java Action Features}.
9453
9454 @item
9455 The prolog declarations have a different meaning than in C/C++ code.
9456 @table @asis
9457 @item @code{%code imports}
9458 blocks are placed at the beginning of the Java source code. They may
9459 include copyright notices. For a @code{package} declarations, it is
9460 suggested to use @code{%define package} instead.
9461
9462 @item unqualified @code{%code}
9463 blocks are placed inside the parser class.
9464
9465 @item @code{%code lexer}
9466 blocks, if specified, should include the implementation of the
9467 scanner. If there is no such block, the scanner can be any class
9468 that implements the appropriate interface (see @pxref{Java Scanner
9469 Interface}).
9470 @end table
9471
9472 Other @code{%code} blocks are not supported in Java parsers.
9473 In particular, @code{%@{ @dots{} %@}} blocks should not be used
9474 and may give an error in future versions of Bison.
9475
9476 The epilogue has the same meaning as in C/C++ code and it can
9477 be used to define other classes used by the parser @emph{outside}
9478 the parser class.
9479 @end itemize
9480
9481
9482 @node Java Declarations Summary
9483 @subsection Java Declarations Summary
9484
9485 This summary only include declarations specific to Java or have special
9486 meaning when used in a Java parser.
9487
9488 @deffn {Directive} {%language "Java"}
9489 Generate a Java class for the parser.
9490 @end deffn
9491
9492 @deffn {Directive} %lex-param @{@var{type} @var{name}@}
9493 A parameter for the lexer class defined by @code{%code lexer}
9494 @emph{only}, added as parameters to the lexer constructor and the parser
9495 constructor that @emph{creates} a lexer. Default is none.
9496 @xref{Java Scanner Interface}.
9497 @end deffn
9498
9499 @deffn {Directive} %name-prefix "@var{prefix}"
9500 The prefix of the parser class name @code{@var{prefix}Parser} if
9501 @code{%define parser_class_name} is not used. Default is @code{YY}.
9502 @xref{Java Bison Interface}.
9503 @end deffn
9504
9505 @deffn {Directive} %parse-param @{@var{type} @var{name}@}
9506 A parameter for the parser class added as parameters to constructor(s)
9507 and as fields initialized by the constructor(s). Default is none.
9508 @xref{Java Parser Interface}.
9509 @end deffn
9510
9511 @deffn {Directive} %token <@var{type}> @var{token} @dots{}
9512 Declare tokens. Note that the angle brackets enclose a Java @emph{type}.
9513 @xref{Java Semantic Values}.
9514 @end deffn
9515
9516 @deffn {Directive} %type <@var{type}> @var{nonterminal} @dots{}
9517 Declare the type of nonterminals. Note that the angle brackets enclose
9518 a Java @emph{type}.
9519 @xref{Java Semantic Values}.
9520 @end deffn
9521
9522 @deffn {Directive} %code @{ @var{code} @dots{} @}
9523 Code appended to the inside of the parser class.
9524 @xref{Java Differences}.
9525 @end deffn
9526
9527 @deffn {Directive} {%code imports} @{ @var{code} @dots{} @}
9528 Code inserted just after the @code{package} declaration.
9529 @xref{Java Differences}.
9530 @end deffn
9531
9532 @deffn {Directive} {%code init} @{ @var{code} @dots{} @}
9533 Code inserted at the beginning of the parser constructor body.
9534 @xref{Java Parser Interface}.
9535 @end deffn
9536
9537 @deffn {Directive} {%code lexer} @{ @var{code} @dots{} @}
9538 Code added to the body of a inner lexer class within the parser class.
9539 @xref{Java Scanner Interface}.
9540 @end deffn
9541
9542 @deffn {Directive} %% @var{code} @dots{}
9543 Code (after the second @code{%%}) appended to the end of the file,
9544 @emph{outside} the parser class.
9545 @xref{Java Differences}.
9546 @end deffn
9547
9548 @deffn {Directive} %@{ @var{code} @dots{} %@}
9549 Not supported. Use @code{%code imports} instead.
9550 @xref{Java Differences}.
9551 @end deffn
9552
9553 @deffn {Directive} {%define abstract}
9554 Whether the parser class is declared @code{abstract}. Default is false.
9555 @xref{Java Bison Interface}.
9556 @end deffn
9557
9558 @deffn {Directive} {%define annotations} "@var{annotations}"
9559 The Java annotations for the parser class. Default is none.
9560 @xref{Java Bison Interface}.
9561 @end deffn
9562
9563 @deffn {Directive} {%define extends} "@var{superclass}"
9564 The superclass of the parser class. Default is none.
9565 @xref{Java Bison Interface}.
9566 @end deffn
9567
9568 @deffn {Directive} {%define final}
9569 Whether the parser class is declared @code{final}. Default is false.
9570 @xref{Java Bison Interface}.
9571 @end deffn
9572
9573 @deffn {Directive} {%define implements} "@var{interfaces}"
9574 The implemented interfaces of the parser class, a comma-separated list.
9575 Default is none.
9576 @xref{Java Bison Interface}.
9577 @end deffn
9578
9579 @deffn {Directive} {%define init_throws} "@var{exceptions}"
9580 The exceptions thrown by @code{%code init} from the parser class
9581 constructor. Default is none.
9582 @xref{Java Parser Interface}.
9583 @end deffn
9584
9585 @deffn {Directive} {%define lex_throws} "@var{exceptions}"
9586 The exceptions thrown by the @code{yylex} method of the lexer, a
9587 comma-separated list. Default is @code{java.io.IOException}.
9588 @xref{Java Scanner Interface}.
9589 @end deffn
9590
9591 @deffn {Directive} {%define location_type} "@var{class}"
9592 The name of the class used for locations (a range between two
9593 positions). This class is generated as an inner class of the parser
9594 class by @command{bison}. Default is @code{Location}.
9595 @xref{Java Location Values}.
9596 @end deffn
9597
9598 @deffn {Directive} {%define package} "@var{package}"
9599 The package to put the parser class in. Default is none.
9600 @xref{Java Bison Interface}.
9601 @end deffn
9602
9603 @deffn {Directive} {%define parser_class_name} "@var{name}"
9604 The name of the parser class. Default is @code{YYParser} or
9605 @code{@var{name-prefix}Parser}.
9606 @xref{Java Bison Interface}.
9607 @end deffn
9608
9609 @deffn {Directive} {%define position_type} "@var{class}"
9610 The name of the class used for positions. This class must be supplied by
9611 the user. Default is @code{Position}.
9612 @xref{Java Location Values}.
9613 @end deffn
9614
9615 @deffn {Directive} {%define public}
9616 Whether the parser class is declared @code{public}. Default is false.
9617 @xref{Java Bison Interface}.
9618 @end deffn
9619
9620 @deffn {Directive} {%define stype} "@var{class}"
9621 The base type of semantic values. Default is @code{Object}.
9622 @xref{Java Semantic Values}.
9623 @end deffn
9624
9625 @deffn {Directive} {%define strictfp}
9626 Whether the parser class is declared @code{strictfp}. Default is false.
9627 @xref{Java Bison Interface}.
9628 @end deffn
9629
9630 @deffn {Directive} {%define throws} "@var{exceptions}"
9631 The exceptions thrown by user-supplied parser actions and
9632 @code{%initial-action}, a comma-separated list. Default is none.
9633 @xref{Java Parser Interface}.
9634 @end deffn
9635
9636
9637 @c ================================================= FAQ
9638
9639 @node FAQ
9640 @chapter Frequently Asked Questions
9641 @cindex frequently asked questions
9642 @cindex questions
9643
9644 Several questions about Bison come up occasionally. Here some of them
9645 are addressed.
9646
9647 @menu
9648 * Memory Exhausted:: Breaking the Stack Limits
9649 * How Can I Reset the Parser:: @code{yyparse} Keeps some State
9650 * Strings are Destroyed:: @code{yylval} Loses Track of Strings
9651 * Implementing Gotos/Loops:: Control Flow in the Calculator
9652 * Multiple start-symbols:: Factoring closely related grammars
9653 * Secure? Conform?:: Is Bison @acronym{POSIX} safe?
9654 * I can't build Bison:: Troubleshooting
9655 * Where can I find help?:: Troubleshouting
9656 * Bug Reports:: Troublereporting
9657 * More Languages:: Parsers in C++, Java, and so on
9658 * Beta Testing:: Experimenting development versions
9659 * Mailing Lists:: Meeting other Bison users
9660 @end menu
9661
9662 @node Memory Exhausted
9663 @section Memory Exhausted
9664
9665 @display
9666 My parser returns with error with a @samp{memory exhausted}
9667 message. What can I do?
9668 @end display
9669
9670 This question is already addressed elsewhere, @xref{Recursion,
9671 ,Recursive Rules}.
9672
9673 @node How Can I Reset the Parser
9674 @section How Can I Reset the Parser
9675
9676 The following phenomenon has several symptoms, resulting in the
9677 following typical questions:
9678
9679 @display
9680 I invoke @code{yyparse} several times, and on correct input it works
9681 properly; but when a parse error is found, all the other calls fail
9682 too. How can I reset the error flag of @code{yyparse}?
9683 @end display
9684
9685 @noindent
9686 or
9687
9688 @display
9689 My parser includes support for an @samp{#include}-like feature, in
9690 which case I run @code{yyparse} from @code{yyparse}. This fails
9691 although I did specify @code{%define api.pure}.
9692 @end display
9693
9694 These problems typically come not from Bison itself, but from
9695 Lex-generated scanners. Because these scanners use large buffers for
9696 speed, they might not notice a change of input file. As a
9697 demonstration, consider the following source file,
9698 @file{first-line.l}:
9699
9700 @verbatim
9701 %{
9702 #include <stdio.h>
9703 #include <stdlib.h>
9704 %}
9705 %%
9706 .*\n ECHO; return 1;
9707 %%
9708 int
9709 yyparse (char const *file)
9710 {
9711 yyin = fopen (file, "r");
9712 if (!yyin)
9713 exit (2);
9714 /* One token only. */
9715 yylex ();
9716 if (fclose (yyin) != 0)
9717 exit (3);
9718 return 0;
9719 }
9720
9721 int
9722 main (void)
9723 {
9724 yyparse ("input");
9725 yyparse ("input");
9726 return 0;
9727 }
9728 @end verbatim
9729
9730 @noindent
9731 If the file @file{input} contains
9732
9733 @verbatim
9734 input:1: Hello,
9735 input:2: World!
9736 @end verbatim
9737
9738 @noindent
9739 then instead of getting the first line twice, you get:
9740
9741 @example
9742 $ @kbd{flex -ofirst-line.c first-line.l}
9743 $ @kbd{gcc -ofirst-line first-line.c -ll}
9744 $ @kbd{./first-line}
9745 input:1: Hello,
9746 input:2: World!
9747 @end example
9748
9749 Therefore, whenever you change @code{yyin}, you must tell the
9750 Lex-generated scanner to discard its current buffer and switch to the
9751 new one. This depends upon your implementation of Lex; see its
9752 documentation for more. For Flex, it suffices to call
9753 @samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
9754 Flex-generated scanner needs to read from several input streams to
9755 handle features like include files, you might consider using Flex
9756 functions like @samp{yy_switch_to_buffer} that manipulate multiple
9757 input buffers.
9758
9759 If your Flex-generated scanner uses start conditions (@pxref{Start
9760 conditions, , Start conditions, flex, The Flex Manual}), you might
9761 also want to reset the scanner's state, i.e., go back to the initial
9762 start condition, through a call to @samp{BEGIN (0)}.
9763
9764 @node Strings are Destroyed
9765 @section Strings are Destroyed
9766
9767 @display
9768 My parser seems to destroy old strings, or maybe it loses track of
9769 them. Instead of reporting @samp{"foo", "bar"}, it reports
9770 @samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
9771 @end display
9772
9773 This error is probably the single most frequent ``bug report'' sent to
9774 Bison lists, but is only concerned with a misunderstanding of the role
9775 of the scanner. Consider the following Lex code:
9776
9777 @verbatim
9778 %{
9779 #include <stdio.h>
9780 char *yylval = NULL;
9781 %}
9782 %%
9783 .* yylval = yytext; return 1;
9784 \n /* IGNORE */
9785 %%
9786 int
9787 main ()
9788 {
9789 /* Similar to using $1, $2 in a Bison action. */
9790 char *fst = (yylex (), yylval);
9791 char *snd = (yylex (), yylval);
9792 printf ("\"%s\", \"%s\"\n", fst, snd);
9793 return 0;
9794 }
9795 @end verbatim
9796
9797 If you compile and run this code, you get:
9798
9799 @example
9800 $ @kbd{flex -osplit-lines.c split-lines.l}
9801 $ @kbd{gcc -osplit-lines split-lines.c -ll}
9802 $ @kbd{printf 'one\ntwo\n' | ./split-lines}
9803 "one
9804 two", "two"
9805 @end example
9806
9807 @noindent
9808 this is because @code{yytext} is a buffer provided for @emph{reading}
9809 in the action, but if you want to keep it, you have to duplicate it
9810 (e.g., using @code{strdup}). Note that the output may depend on how
9811 your implementation of Lex handles @code{yytext}. For instance, when
9812 given the Lex compatibility option @option{-l} (which triggers the
9813 option @samp{%array}) Flex generates a different behavior:
9814
9815 @example
9816 $ @kbd{flex -l -osplit-lines.c split-lines.l}
9817 $ @kbd{gcc -osplit-lines split-lines.c -ll}
9818 $ @kbd{printf 'one\ntwo\n' | ./split-lines}
9819 "two", "two"
9820 @end example
9821
9822
9823 @node Implementing Gotos/Loops
9824 @section Implementing Gotos/Loops
9825
9826 @display
9827 My simple calculator supports variables, assignments, and functions,
9828 but how can I implement gotos, or loops?
9829 @end display
9830
9831 Although very pedagogical, the examples included in the document blur
9832 the distinction to make between the parser---whose job is to recover
9833 the structure of a text and to transmit it to subsequent modules of
9834 the program---and the processing (such as the execution) of this
9835 structure. This works well with so called straight line programs,
9836 i.e., precisely those that have a straightforward execution model:
9837 execute simple instructions one after the others.
9838
9839 @cindex abstract syntax tree
9840 @cindex @acronym{AST}
9841 If you want a richer model, you will probably need to use the parser
9842 to construct a tree that does represent the structure it has
9843 recovered; this tree is usually called the @dfn{abstract syntax tree},
9844 or @dfn{@acronym{AST}} for short. Then, walking through this tree,
9845 traversing it in various ways, will enable treatments such as its
9846 execution or its translation, which will result in an interpreter or a
9847 compiler.
9848
9849 This topic is way beyond the scope of this manual, and the reader is
9850 invited to consult the dedicated literature.
9851
9852
9853 @node Multiple start-symbols
9854 @section Multiple start-symbols
9855
9856 @display
9857 I have several closely related grammars, and I would like to share their
9858 implementations. In fact, I could use a single grammar but with
9859 multiple entry points.
9860 @end display
9861
9862 Bison does not support multiple start-symbols, but there is a very
9863 simple means to simulate them. If @code{foo} and @code{bar} are the two
9864 pseudo start-symbols, then introduce two new tokens, say
9865 @code{START_FOO} and @code{START_BAR}, and use them as switches from the
9866 real start-symbol:
9867
9868 @example
9869 %token START_FOO START_BAR;
9870 %start start;
9871 start: START_FOO foo
9872 | START_BAR bar;
9873 @end example
9874
9875 These tokens prevents the introduction of new conflicts. As far as the
9876 parser goes, that is all that is needed.
9877
9878 Now the difficult part is ensuring that the scanner will send these
9879 tokens first. If your scanner is hand-written, that should be
9880 straightforward. If your scanner is generated by Lex, them there is
9881 simple means to do it: recall that anything between @samp{%@{ ... %@}}
9882 after the first @code{%%} is copied verbatim in the top of the generated
9883 @code{yylex} function. Make sure a variable @code{start_token} is
9884 available in the scanner (e.g., a global variable or using
9885 @code{%lex-param} etc.), and use the following:
9886
9887 @example
9888 /* @r{Prologue.} */
9889 %%
9890 %@{
9891 if (start_token)
9892 @{
9893 int t = start_token;
9894 start_token = 0;
9895 return t;
9896 @}
9897 %@}
9898 /* @r{The rules.} */
9899 @end example
9900
9901
9902 @node Secure? Conform?
9903 @section Secure? Conform?
9904
9905 @display
9906 Is Bison secure? Does it conform to POSIX?
9907 @end display
9908
9909 If you're looking for a guarantee or certification, we don't provide it.
9910 However, Bison is intended to be a reliable program that conforms to the
9911 @acronym{POSIX} specification for Yacc. If you run into problems,
9912 please send us a bug report.
9913
9914 @node I can't build Bison
9915 @section I can't build Bison
9916
9917 @display
9918 I can't build Bison because @command{make} complains that
9919 @code{msgfmt} is not found.
9920 What should I do?
9921 @end display
9922
9923 Like most GNU packages with internationalization support, that feature
9924 is turned on by default. If you have problems building in the @file{po}
9925 subdirectory, it indicates that your system's internationalization
9926 support is lacking. You can re-configure Bison with
9927 @option{--disable-nls} to turn off this support, or you can install GNU
9928 gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
9929 Bison. See the file @file{ABOUT-NLS} for more information.
9930
9931
9932 @node Where can I find help?
9933 @section Where can I find help?
9934
9935 @display
9936 I'm having trouble using Bison. Where can I find help?
9937 @end display
9938
9939 First, read this fine manual. Beyond that, you can send mail to
9940 @email{help-bison@@gnu.org}. This mailing list is intended to be
9941 populated with people who are willing to answer questions about using
9942 and installing Bison. Please keep in mind that (most of) the people on
9943 the list have aspects of their lives which are not related to Bison (!),
9944 so you may not receive an answer to your question right away. This can
9945 be frustrating, but please try not to honk them off; remember that any
9946 help they provide is purely voluntary and out of the kindness of their
9947 hearts.
9948
9949 @node Bug Reports
9950 @section Bug Reports
9951
9952 @display
9953 I found a bug. What should I include in the bug report?
9954 @end display
9955
9956 Before you send a bug report, make sure you are using the latest
9957 version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
9958 mirrors. Be sure to include the version number in your bug report. If
9959 the bug is present in the latest version but not in a previous version,
9960 try to determine the most recent version which did not contain the bug.
9961
9962 If the bug is parser-related, you should include the smallest grammar
9963 you can which demonstrates the bug. The grammar file should also be
9964 complete (i.e., I should be able to run it through Bison without having
9965 to edit or add anything). The smaller and simpler the grammar, the
9966 easier it will be to fix the bug.
9967
9968 Include information about your compilation environment, including your
9969 operating system's name and version and your compiler's name and
9970 version. If you have trouble compiling, you should also include a
9971 transcript of the build session, starting with the invocation of
9972 `configure'. Depending on the nature of the bug, you may be asked to
9973 send additional files as well (such as `config.h' or `config.cache').
9974
9975 Patches are most welcome, but not required. That is, do not hesitate to
9976 send a bug report just because you can not provide a fix.
9977
9978 Send bug reports to @email{bug-bison@@gnu.org}.
9979
9980 @node More Languages
9981 @section More Languages
9982
9983 @display
9984 Will Bison ever have C++ and Java support? How about @var{insert your
9985 favorite language here}?
9986 @end display
9987
9988 C++ and Java support is there now, and is documented. We'd love to add other
9989 languages; contributions are welcome.
9990
9991 @node Beta Testing
9992 @section Beta Testing
9993
9994 @display
9995 What is involved in being a beta tester?
9996 @end display
9997
9998 It's not terribly involved. Basically, you would download a test
9999 release, compile it, and use it to build and run a parser or two. After
10000 that, you would submit either a bug report or a message saying that
10001 everything is okay. It is important to report successes as well as
10002 failures because test releases eventually become mainstream releases,
10003 but only if they are adequately tested. If no one tests, development is
10004 essentially halted.
10005
10006 Beta testers are particularly needed for operating systems to which the
10007 developers do not have easy access. They currently have easy access to
10008 recent GNU/Linux and Solaris versions. Reports about other operating
10009 systems are especially welcome.
10010
10011 @node Mailing Lists
10012 @section Mailing Lists
10013
10014 @display
10015 How do I join the help-bison and bug-bison mailing lists?
10016 @end display
10017
10018 See @url{http://lists.gnu.org/}.
10019
10020 @c ================================================= Table of Symbols
10021
10022 @node Table of Symbols
10023 @appendix Bison Symbols
10024 @cindex Bison symbols, table of
10025 @cindex symbols in Bison, table of
10026
10027 @deffn {Variable} @@$
10028 In an action, the location of the left-hand side of the rule.
10029 @xref{Locations, , Locations Overview}.
10030 @end deffn
10031
10032 @deffn {Variable} @@@var{n}
10033 In an action, the location of the @var{n}-th symbol of the right-hand
10034 side of the rule. @xref{Locations, , Locations Overview}.
10035 @end deffn
10036
10037 @deffn {Variable} $$
10038 In an action, the semantic value of the left-hand side of the rule.
10039 @xref{Actions}.
10040 @end deffn
10041
10042 @deffn {Variable} $@var{n}
10043 In an action, the semantic value of the @var{n}-th symbol of the
10044 right-hand side of the rule. @xref{Actions}.
10045 @end deffn
10046
10047 @deffn {Delimiter} %%
10048 Delimiter used to separate the grammar rule section from the
10049 Bison declarations section or the epilogue.
10050 @xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
10051 @end deffn
10052
10053 @c Don't insert spaces, or check the DVI output.
10054 @deffn {Delimiter} %@{@var{code}%@}
10055 All code listed between @samp{%@{} and @samp{%@}} is copied directly to
10056 the output file uninterpreted. Such code forms the prologue of the input
10057 file. @xref{Grammar Outline, ,Outline of a Bison
10058 Grammar}.
10059 @end deffn
10060
10061 @deffn {Construct} /*@dots{}*/
10062 Comment delimiters, as in C.
10063 @end deffn
10064
10065 @deffn {Delimiter} :
10066 Separates a rule's result from its components. @xref{Rules, ,Syntax of
10067 Grammar Rules}.
10068 @end deffn
10069
10070 @deffn {Delimiter} ;
10071 Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
10072 @end deffn
10073
10074 @deffn {Delimiter} |
10075 Separates alternate rules for the same result nonterminal.
10076 @xref{Rules, ,Syntax of Grammar Rules}.
10077 @end deffn
10078
10079 @deffn {Directive} <*>
10080 Used to define a default tagged @code{%destructor} or default tagged
10081 @code{%printer}.
10082
10083 This feature is experimental.
10084 More user feedback will help to determine whether it should become a permanent
10085 feature.
10086
10087 @xref{Destructor Decl, , Freeing Discarded Symbols}.
10088 @end deffn
10089
10090 @deffn {Directive} <>
10091 Used to define a default tagless @code{%destructor} or default tagless
10092 @code{%printer}.
10093
10094 This feature is experimental.
10095 More user feedback will help to determine whether it should become a permanent
10096 feature.
10097
10098 @xref{Destructor Decl, , Freeing Discarded Symbols}.
10099 @end deffn
10100
10101 @deffn {Symbol} $accept
10102 The predefined nonterminal whose only rule is @samp{$accept: @var{start}
10103 $end}, where @var{start} is the start symbol. @xref{Start Decl, , The
10104 Start-Symbol}. It cannot be used in the grammar.
10105 @end deffn
10106
10107 @deffn {Directive} %code @{@var{code}@}
10108 @deffnx {Directive} %code @var{qualifier} @{@var{code}@}
10109 Insert @var{code} verbatim into output parser source.
10110 @xref{Decl Summary,,%code}.
10111 @end deffn
10112
10113 @deffn {Directive} %debug
10114 Equip the parser for debugging. @xref{Decl Summary}.
10115 @end deffn
10116
10117 @ifset defaultprec
10118 @deffn {Directive} %default-prec
10119 Assign a precedence to rules that lack an explicit @samp{%prec}
10120 modifier. @xref{Contextual Precedence, ,Context-Dependent
10121 Precedence}.
10122 @end deffn
10123 @end ifset
10124
10125 @deffn {Directive} %define @var{define-variable}
10126 @deffnx {Directive} %define @var{define-variable} @var{value}
10127 Define a variable to adjust Bison's behavior.
10128 @xref{Decl Summary,,%define}.
10129 @end deffn
10130
10131 @deffn {Directive} %defines
10132 Bison declaration to create a header file meant for the scanner.
10133 @xref{Decl Summary}.
10134 @end deffn
10135
10136 @deffn {Directive} %defines @var{defines-file}
10137 Same as above, but save in the file @var{defines-file}.
10138 @xref{Decl Summary}.
10139 @end deffn
10140
10141 @deffn {Directive} %destructor
10142 Specify how the parser should reclaim the memory associated to
10143 discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
10144 @end deffn
10145
10146 @deffn {Directive} %dprec
10147 Bison declaration to assign a precedence to a rule that is used at parse
10148 time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
10149 @acronym{GLR} Parsers}.
10150 @end deffn
10151
10152 @deffn {Symbol} $end
10153 The predefined token marking the end of the token stream. It cannot be
10154 used in the grammar.
10155 @end deffn
10156
10157 @deffn {Symbol} error
10158 A token name reserved for error recovery. This token may be used in
10159 grammar rules so as to allow the Bison parser to recognize an error in
10160 the grammar without halting the process. In effect, a sentence
10161 containing an error may be recognized as valid. On a syntax error, the
10162 token @code{error} becomes the current lookahead token. Actions
10163 corresponding to @code{error} are then executed, and the lookahead
10164 token is reset to the token that originally caused the violation.
10165 @xref{Error Recovery}.
10166 @end deffn
10167
10168 @deffn {Directive} %error-verbose
10169 An obsolete directive standing for @samp{%define error-verbose}.
10170 @end deffn
10171
10172 @deffn {Directive} %file-prefix "@var{prefix}"
10173 Bison declaration to set the prefix of the output files. @xref{Decl
10174 Summary}.
10175 @end deffn
10176
10177 @deffn {Directive} %glr-parser
10178 Bison declaration to produce a @acronym{GLR} parser. @xref{GLR
10179 Parsers, ,Writing @acronym{GLR} Parsers}.
10180 @end deffn
10181
10182 @deffn {Directive} %initial-action
10183 Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
10184 @end deffn
10185
10186 @deffn {Directive} %language
10187 Specify the programming language for the generated parser.
10188 @xref{Decl Summary}.
10189 @end deffn
10190
10191 @deffn {Directive} %left
10192 Bison declaration to assign precedence and left associativity to token(s).
10193 @xref{Precedence Decl, ,Operator Precedence}.
10194 @end deffn
10195
10196 @deffn {Directive} %lex-param @{@var{argument-declaration}@}
10197 Bison declaration to specifying an additional parameter that
10198 @code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
10199 for Pure Parsers}.
10200 @end deffn
10201
10202 @deffn {Directive} %merge
10203 Bison declaration to assign a merging function to a rule. If there is a
10204 reduce/reduce conflict with a rule having the same merging function, the
10205 function is applied to the two semantic values to get a single result.
10206 @xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}.
10207 @end deffn
10208
10209 @deffn {Directive} %name-prefix "@var{prefix}"
10210 Bison declaration to rename the external symbols. @xref{Decl Summary}.
10211 @end deffn
10212
10213 @ifset defaultprec
10214 @deffn {Directive} %no-default-prec
10215 Do not assign a precedence to rules that lack an explicit @samp{%prec}
10216 modifier. @xref{Contextual Precedence, ,Context-Dependent
10217 Precedence}.
10218 @end deffn
10219 @end ifset
10220
10221 @deffn {Directive} %no-lines
10222 Bison declaration to avoid generating @code{#line} directives in the
10223 parser file. @xref{Decl Summary}.
10224 @end deffn
10225
10226 @deffn {Directive} %nonassoc
10227 Bison declaration to assign precedence and nonassociativity to token(s).
10228 @xref{Precedence Decl, ,Operator Precedence}.
10229 @end deffn
10230
10231 @deffn {Directive} %output "@var{file}"
10232 Bison declaration to set the name of the parser file. @xref{Decl
10233 Summary}.
10234 @end deffn
10235
10236 @deffn {Directive} %parse-param @{@var{argument-declaration}@}
10237 Bison declaration to specifying an additional parameter that
10238 @code{yyparse} should accept. @xref{Parser Function,, The Parser
10239 Function @code{yyparse}}.
10240 @end deffn
10241
10242 @deffn {Directive} %prec
10243 Bison declaration to assign a precedence to a specific rule.
10244 @xref{Contextual Precedence, ,Context-Dependent Precedence}.
10245 @end deffn
10246
10247 @deffn {Directive} %precedence
10248 Bison declaration to assign precedence to token(s), but no associativity
10249 @xref{Precedence Decl, ,Operator Precedence}.
10250 @end deffn
10251
10252 @deffn {Directive} %pure-parser
10253 Deprecated version of @code{%define api.pure} (@pxref{Decl Summary, ,%define}),
10254 for which Bison is more careful to warn about unreasonable usage.
10255 @end deffn
10256
10257 @deffn {Directive} %require "@var{version}"
10258 Require version @var{version} or higher of Bison. @xref{Require Decl, ,
10259 Require a Version of Bison}.
10260 @end deffn
10261
10262 @deffn {Directive} %right
10263 Bison declaration to assign precedence and right associativity to token(s).
10264 @xref{Precedence Decl, ,Operator Precedence}.
10265 @end deffn
10266
10267 @deffn {Directive} %skeleton
10268 Specify the skeleton to use; usually for development.
10269 @xref{Decl Summary}.
10270 @end deffn
10271
10272 @deffn {Directive} %start
10273 Bison declaration to specify the start symbol. @xref{Start Decl, ,The
10274 Start-Symbol}.
10275 @end deffn
10276
10277 @deffn {Directive} %token
10278 Bison declaration to declare token(s) without specifying precedence.
10279 @xref{Token Decl, ,Token Type Names}.
10280 @end deffn
10281
10282 @deffn {Directive} %token-table
10283 Bison declaration to include a token name table in the parser file.
10284 @xref{Decl Summary}.
10285 @end deffn
10286
10287 @deffn {Directive} %type
10288 Bison declaration to declare nonterminals. @xref{Type Decl,
10289 ,Nonterminal Symbols}.
10290 @end deffn
10291
10292 @deffn {Symbol} $undefined
10293 The predefined token onto which all undefined values returned by
10294 @code{yylex} are mapped. It cannot be used in the grammar, rather, use
10295 @code{error}.
10296 @end deffn
10297
10298 @deffn {Directive} %union
10299 Bison declaration to specify several possible data types for semantic
10300 values. @xref{Union Decl, ,The Collection of Value Types}.
10301 @end deffn
10302
10303 @deffn {Macro} YYABORT
10304 Macro to pretend that an unrecoverable syntax error has occurred, by
10305 making @code{yyparse} return 1 immediately. The error reporting
10306 function @code{yyerror} is not called. @xref{Parser Function, ,The
10307 Parser Function @code{yyparse}}.
10308
10309 For Java parsers, this functionality is invoked using @code{return YYABORT;}
10310 instead.
10311 @end deffn
10312
10313 @deffn {Macro} YYACCEPT
10314 Macro to pretend that a complete utterance of the language has been
10315 read, by making @code{yyparse} return 0 immediately.
10316 @xref{Parser Function, ,The Parser Function @code{yyparse}}.
10317
10318 For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
10319 instead.
10320 @end deffn
10321
10322 @deffn {Macro} YYBACKUP
10323 Macro to discard a value from the parser stack and fake a lookahead
10324 token. @xref{Action Features, ,Special Features for Use in Actions}.
10325 @end deffn
10326
10327 @deffn {Variable} yychar
10328 External integer variable that contains the integer value of the
10329 lookahead token. (In a pure parser, it is a local variable within
10330 @code{yyparse}.) Error-recovery rule actions may examine this variable.
10331 @xref{Action Features, ,Special Features for Use in Actions}.
10332 @end deffn
10333
10334 @deffn {Variable} yyclearin
10335 Macro used in error-recovery rule actions. It clears the previous
10336 lookahead token. @xref{Error Recovery}.
10337 @end deffn
10338
10339 @deffn {Macro} YYDEBUG
10340 Macro to define to equip the parser with tracing code. @xref{Tracing,
10341 ,Tracing Your Parser}.
10342 @end deffn
10343
10344 @deffn {Variable} yydebug
10345 External integer variable set to zero by default. If @code{yydebug}
10346 is given a nonzero value, the parser will output information on input
10347 symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
10348 @end deffn
10349
10350 @deffn {Macro} yyerrok
10351 Macro to cause parser to recover immediately to its normal mode
10352 after a syntax error. @xref{Error Recovery}.
10353 @end deffn
10354
10355 @deffn {Macro} YYERROR
10356 Macro to pretend that a syntax error has just been detected: call
10357 @code{yyerror} and then perform normal error recovery if possible
10358 (@pxref{Error Recovery}), or (if recovery is impossible) make
10359 @code{yyparse} return 1. @xref{Error Recovery}.
10360
10361 For Java parsers, this functionality is invoked using @code{return YYERROR;}
10362 instead.
10363 @end deffn
10364
10365 @deffn {Function} yyerror
10366 User-supplied function to be called by @code{yyparse} on error.
10367 @xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
10368 @end deffn
10369
10370 @deffn {Macro} YYERROR_VERBOSE
10371 An obsolete macro used in the @file{yacc.c} skeleton, that you define
10372 with @code{#define} in the prologue to request verbose, specific error
10373 message strings when @code{yyerror} is called. It doesn't matter what
10374 definition you use for @code{YYERROR_VERBOSE}, just whether you define
10375 it. Using @code{%define error-verbose} is preferred (@pxref{Error
10376 Reporting, ,The Error Reporting Function @code{yyerror}}).
10377 @end deffn
10378
10379 @deffn {Macro} YYINITDEPTH
10380 Macro for specifying the initial size of the parser stack.
10381 @xref{Memory Management}.
10382 @end deffn
10383
10384 @deffn {Function} yylex
10385 User-supplied lexical analyzer function, called with no arguments to get
10386 the next token. @xref{Lexical, ,The Lexical Analyzer Function
10387 @code{yylex}}.
10388 @end deffn
10389
10390 @deffn {Macro} YYLEX_PARAM
10391 An obsolete macro for specifying an extra argument (or list of extra
10392 arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
10393 macro is deprecated, and is supported only for Yacc like parsers.
10394 @xref{Pure Calling,, Calling Conventions for Pure Parsers}.
10395 @end deffn
10396
10397 @deffn {Variable} yylloc
10398 External variable in which @code{yylex} should place the line and column
10399 numbers associated with a token. (In a pure parser, it is a local
10400 variable within @code{yyparse}, and its address is passed to
10401 @code{yylex}.)
10402 You can ignore this variable if you don't use the @samp{@@} feature in the
10403 grammar actions.
10404 @xref{Token Locations, ,Textual Locations of Tokens}.
10405 In semantic actions, it stores the location of the lookahead token.
10406 @xref{Actions and Locations, ,Actions and Locations}.
10407 @end deffn
10408
10409 @deffn {Type} YYLTYPE
10410 Data type of @code{yylloc}; by default, a structure with four
10411 members. @xref{Location Type, , Data Types of Locations}.
10412 @end deffn
10413
10414 @deffn {Variable} yylval
10415 External variable in which @code{yylex} should place the semantic
10416 value associated with a token. (In a pure parser, it is a local
10417 variable within @code{yyparse}, and its address is passed to
10418 @code{yylex}.)
10419 @xref{Token Values, ,Semantic Values of Tokens}.
10420 In semantic actions, it stores the semantic value of the lookahead token.
10421 @xref{Actions, ,Actions}.
10422 @end deffn
10423
10424 @deffn {Macro} YYMAXDEPTH
10425 Macro for specifying the maximum size of the parser stack. @xref{Memory
10426 Management}.
10427 @end deffn
10428
10429 @deffn {Variable} yynerrs
10430 Global variable which Bison increments each time it reports a syntax error.
10431 (In a pure parser, it is a local variable within @code{yyparse}. In a
10432 pure push parser, it is a member of yypstate.)
10433 @xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
10434 @end deffn
10435
10436 @deffn {Function} yyparse
10437 The parser function produced by Bison; call this function to start
10438 parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
10439 @end deffn
10440
10441 @deffn {Function} yypstate_delete
10442 The function to delete a parser instance, produced by Bison in push mode;
10443 call this function to delete the memory associated with a parser.
10444 @xref{Parser Delete Function, ,The Parser Delete Function
10445 @code{yypstate_delete}}.
10446 (The current push parsing interface is experimental and may evolve.
10447 More user feedback will help to stabilize it.)
10448 @end deffn
10449
10450 @deffn {Function} yypstate_new
10451 The function to create a parser instance, produced by Bison in push mode;
10452 call this function to create a new parser.
10453 @xref{Parser Create Function, ,The Parser Create Function
10454 @code{yypstate_new}}.
10455 (The current push parsing interface is experimental and may evolve.
10456 More user feedback will help to stabilize it.)
10457 @end deffn
10458
10459 @deffn {Function} yypull_parse
10460 The parser function produced by Bison in push mode; call this function to
10461 parse the rest of the input stream.
10462 @xref{Pull Parser Function, ,The Pull Parser Function
10463 @code{yypull_parse}}.
10464 (The current push parsing interface is experimental and may evolve.
10465 More user feedback will help to stabilize it.)
10466 @end deffn
10467
10468 @deffn {Function} yypush_parse
10469 The parser function produced by Bison in push mode; call this function to
10470 parse a single token. @xref{Push Parser Function, ,The Push Parser Function
10471 @code{yypush_parse}}.
10472 (The current push parsing interface is experimental and may evolve.
10473 More user feedback will help to stabilize it.)
10474 @end deffn
10475
10476 @deffn {Macro} YYPARSE_PARAM
10477 An obsolete macro for specifying the name of a parameter that
10478 @code{yyparse} should accept. The use of this macro is deprecated, and
10479 is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
10480 Conventions for Pure Parsers}.
10481 @end deffn
10482
10483 @deffn {Macro} YYRECOVERING
10484 The expression @code{YYRECOVERING ()} yields 1 when the parser
10485 is recovering from a syntax error, and 0 otherwise.
10486 @xref{Action Features, ,Special Features for Use in Actions}.
10487 @end deffn
10488
10489 @deffn {Macro} YYSTACK_USE_ALLOCA
10490 Macro used to control the use of @code{alloca} when the
10491 deterministic parser in C needs to extend its stacks. If defined to 0,
10492 the parser will use @code{malloc} to extend its stacks. If defined to
10493 1, the parser will use @code{alloca}. Values other than 0 and 1 are
10494 reserved for future Bison extensions. If not defined,
10495 @code{YYSTACK_USE_ALLOCA} defaults to 0.
10496
10497 In the all-too-common case where your code may run on a host with a
10498 limited stack and with unreliable stack-overflow checking, you should
10499 set @code{YYMAXDEPTH} to a value that cannot possibly result in
10500 unchecked stack overflow on any of your target hosts when
10501 @code{alloca} is called. You can inspect the code that Bison
10502 generates in order to determine the proper numeric values. This will
10503 require some expertise in low-level implementation details.
10504 @end deffn
10505
10506 @deffn {Type} YYSTYPE
10507 Data type of semantic values; @code{int} by default.
10508 @xref{Value Type, ,Data Types of Semantic Values}.
10509 @end deffn
10510
10511 @node Glossary
10512 @appendix Glossary
10513 @cindex glossary
10514
10515 @table @asis
10516 @item Accepting State
10517 A state whose only action is the accept action.
10518 The accepting state is thus a consistent state.
10519 @xref{Understanding,,}.
10520
10521 @item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'')
10522 Formal method of specifying context-free grammars originally proposed
10523 by John Backus, and slightly improved by Peter Naur in his 1960-01-02
10524 committee document contributing to what became the Algol 60 report.
10525 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10526
10527 @item Consistent State
10528 A state containing only one possible action.
10529 @xref{Decl Summary,,lr.default-reductions}.
10530
10531 @item Context-free grammars
10532 Grammars specified as rules that can be applied regardless of context.
10533 Thus, if there is a rule which says that an integer can be used as an
10534 expression, integers are allowed @emph{anywhere} an expression is
10535 permitted. @xref{Language and Grammar, ,Languages and Context-Free
10536 Grammars}.
10537
10538 @item Default Reduction
10539 The reduction that a parser should perform if the current parser state
10540 contains no other action for the lookahead token.
10541 In permitted parser states, Bison declares the reduction with the
10542 largest lookahead set to be the default reduction and removes that
10543 lookahead set.
10544 @xref{Decl Summary,,lr.default-reductions}.
10545
10546 @item Dynamic allocation
10547 Allocation of memory that occurs during execution, rather than at
10548 compile time or on entry to a function.
10549
10550 @item Empty string
10551 Analogous to the empty set in set theory, the empty string is a
10552 character string of length zero.
10553
10554 @item Finite-state stack machine
10555 A ``machine'' that has discrete states in which it is said to exist at
10556 each instant in time. As input to the machine is processed, the
10557 machine moves from state to state as specified by the logic of the
10558 machine. In the case of the parser, the input is the language being
10559 parsed, and the states correspond to various stages in the grammar
10560 rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
10561
10562 @item Generalized @acronym{LR} (@acronym{GLR})
10563 A parsing algorithm that can handle all context-free grammars, including those
10564 that are not @acronym{LR}(1). It resolves situations that Bison's
10565 deterministic parsing
10566 algorithm cannot by effectively splitting off multiple parsers, trying all
10567 possible parsers, and discarding those that fail in the light of additional
10568 right context. @xref{Generalized LR Parsing, ,Generalized
10569 @acronym{LR} Parsing}.
10570
10571 @item Grouping
10572 A language construct that is (in general) grammatically divisible;
10573 for example, `expression' or `declaration' in C@.
10574 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10575
10576 @item @acronym{IELR}(1)
10577 A minimal @acronym{LR}(1) parser table generation algorithm.
10578 That is, given any context-free grammar, @acronym{IELR}(1) generates
10579 parser tables with the full language recognition power of canonical
10580 @acronym{LR}(1) but with nearly the same number of parser states as
10581 @acronym{LALR}(1).
10582 This reduction in parser states is often an order of magnitude.
10583 More importantly, because canonical @acronym{LR}(1)'s extra parser
10584 states may contain duplicate conflicts in the case of
10585 non-@acronym{LR}(1) grammars, the number of conflicts for
10586 @acronym{IELR}(1) is often an order of magnitude less as well.
10587 This can significantly reduce the complexity of developing of a grammar.
10588 @xref{Decl Summary,,lr.type}.
10589
10590 @item Infix operator
10591 An arithmetic operator that is placed between the operands on which it
10592 performs some operation.
10593
10594 @item Input stream
10595 A continuous flow of data between devices or programs.
10596
10597 @item Language construct
10598 One of the typical usage schemas of the language. For example, one of
10599 the constructs of the C language is the @code{if} statement.
10600 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10601
10602 @item Left associativity
10603 Operators having left associativity are analyzed from left to right:
10604 @samp{a+b+c} first computes @samp{a+b} and then combines with
10605 @samp{c}. @xref{Precedence, ,Operator Precedence}.
10606
10607 @item Left recursion
10608 A rule whose result symbol is also its first component symbol; for
10609 example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
10610 Rules}.
10611
10612 @item Left-to-right parsing
10613 Parsing a sentence of a language by analyzing it token by token from
10614 left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
10615
10616 @item Lexical analyzer (scanner)
10617 A function that reads an input stream and returns tokens one by one.
10618 @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
10619
10620 @item Lexical tie-in
10621 A flag, set by actions in the grammar rules, which alters the way
10622 tokens are parsed. @xref{Lexical Tie-ins}.
10623
10624 @item Literal string token
10625 A token which consists of two or more fixed characters. @xref{Symbols}.
10626
10627 @item Lookahead token
10628 A token already read but not yet shifted. @xref{Lookahead, ,Lookahead
10629 Tokens}.
10630
10631 @item @acronym{LALR}(1)
10632 The class of context-free grammars that Bison (like most other parser
10633 generators) can handle by default; a subset of @acronym{LR}(1).
10634 @xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
10635
10636 @item @acronym{LR}(1)
10637 The class of context-free grammars in which at most one token of
10638 lookahead is needed to disambiguate the parsing of any piece of input.
10639
10640 @item Nonterminal symbol
10641 A grammar symbol standing for a grammatical construct that can
10642 be expressed through rules in terms of smaller constructs; in other
10643 words, a construct that is not a token. @xref{Symbols}.
10644
10645 @item Parser
10646 A function that recognizes valid sentences of a language by analyzing
10647 the syntax structure of a set of tokens passed to it from a lexical
10648 analyzer.
10649
10650 @item Postfix operator
10651 An arithmetic operator that is placed after the operands upon which it
10652 performs some operation.
10653
10654 @item Reduction
10655 Replacing a string of nonterminals and/or terminals with a single
10656 nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
10657 Parser Algorithm}.
10658
10659 @item Reentrant
10660 A reentrant subprogram is a subprogram which can be in invoked any
10661 number of times in parallel, without interference between the various
10662 invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
10663
10664 @item Reverse polish notation
10665 A language in which all operators are postfix operators.
10666
10667 @item Right recursion
10668 A rule whose result symbol is also its last component symbol; for
10669 example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
10670 Rules}.
10671
10672 @item Semantics
10673 In computer languages, the semantics are specified by the actions
10674 taken for each instance of the language, i.e., the meaning of
10675 each statement. @xref{Semantics, ,Defining Language Semantics}.
10676
10677 @item Shift
10678 A parser is said to shift when it makes the choice of analyzing
10679 further input from the stream rather than reducing immediately some
10680 already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
10681
10682 @item Single-character literal
10683 A single character that is recognized and interpreted as is.
10684 @xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
10685
10686 @item Start symbol
10687 The nonterminal symbol that stands for a complete valid utterance in
10688 the language being parsed. The start symbol is usually listed as the
10689 first nonterminal symbol in a language specification.
10690 @xref{Start Decl, ,The Start-Symbol}.
10691
10692 @item Symbol table
10693 A data structure where symbol names and associated data are stored
10694 during parsing to allow for recognition and use of existing
10695 information in repeated uses of a symbol. @xref{Multi-function Calc}.
10696
10697 @item Syntax error
10698 An error encountered during parsing of an input stream due to invalid
10699 syntax. @xref{Error Recovery}.
10700
10701 @item Token
10702 A basic, grammatically indivisible unit of a language. The symbol
10703 that describes a token in the grammar is a terminal symbol.
10704 The input of the Bison parser is a stream of tokens which comes from
10705 the lexical analyzer. @xref{Symbols}.
10706
10707 @item Terminal symbol
10708 A grammar symbol that has no rules in the grammar and therefore is
10709 grammatically indivisible. The piece of text it represents is a token.
10710 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10711 @end table
10712
10713 @node Copying This Manual
10714 @appendix Copying This Manual
10715 @include fdl.texi
10716
10717 @node Index
10718 @unnumbered Index
10719
10720 @printindex cp
10721
10722 @bye
10723
10724 @c LocalWords: texinfo setfilename settitle setchapternewpage finalout
10725 @c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex
10726 @c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry
10727 @c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa
10728 @c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc
10729 @c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex
10730 @c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref
10731 @c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex
10732 @c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge
10733 @c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG
10734 @c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit
10735 @c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok
10736 @c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln
10737 @c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym
10738 @c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof
10739 @c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum
10740 @c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype
10741 @c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs
10742 @c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES
10743 @c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param
10744 @c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP
10745 @c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword
10746 @c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH
10747 @c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype
10748 @c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args
10749 @c LocalWords: infile ypp yxx outfile itemx tex leaderfill
10750 @c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll
10751 @c LocalWords: nbar yytext fst snd osplit ntwo strdup AST
10752 @c LocalWords: YYSTACK DVI fdl printindex IELR