]> git.saurik.com Git - bison.git/blame_incremental - doc/bison.texinfo
maint: update for gnulib's recent update-copyright changes
[bison.git] / doc / bison.texinfo
... / ...
CommitLineData
1\input texinfo @c -*-texinfo-*-
2@comment %**start of header
3@setfilename bison.info
4@include version.texi
5@settitle Bison @value{VERSION}
6@setchapternewpage odd
7
8@finalout
9
10@c SMALL BOOK version
11@c This edition has been formatted so that you can format and print it in
12@c the smallbook format.
13@c @smallbook
14
15@c Set following if you want to document %default-prec and %no-default-prec.
16@c This feature is experimental and may change in future Bison versions.
17@c @set defaultprec
18
19@ifnotinfo
20@syncodeindex fn cp
21@syncodeindex vr cp
22@syncodeindex tp cp
23@end ifnotinfo
24@ifinfo
25@synindex fn cp
26@synindex vr cp
27@synindex tp cp
28@end ifinfo
29@comment %**end of header
30
31@copying
32
33This manual (@value{UPDATED}) is for @acronym{GNU} Bison (version
34@value{VERSION}), the @acronym{GNU} parser generator.
35
36Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
371999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Free
38Software Foundation, Inc.
39
40@quotation
41Permission is granted to copy, distribute and/or modify this document
42under the terms of the @acronym{GNU} Free Documentation License,
43Version 1.2 or any later version published by the Free Software
44Foundation; with no Invariant Sections, with the Front-Cover texts
45being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in
46(a) below. A copy of the license is included in the section entitled
47``@acronym{GNU} Free Documentation License.''
48
49(a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
50modify this @acronym{GNU} manual. Buying copies from the @acronym{FSF}
51supports it in developing @acronym{GNU} and promoting software
52freedom.''
53@end quotation
54@end copying
55
56@dircategory Software development
57@direntry
58* bison: (bison). @acronym{GNU} parser generator (Yacc replacement).
59@end direntry
60
61@titlepage
62@title Bison
63@subtitle The Yacc-compatible Parser Generator
64@subtitle @value{UPDATED}, Bison Version @value{VERSION}
65
66@author by Charles Donnelly and Richard Stallman
67
68@page
69@vskip 0pt plus 1filll
70@insertcopying
71@sp 2
72Published by the Free Software Foundation @*
7351 Franklin Street, Fifth Floor @*
74Boston, MA 02110-1301 USA @*
75Printed copies are available from the Free Software Foundation.@*
76@acronym{ISBN} 1-882114-44-2
77@sp 2
78Cover art by Etienne Suvasa.
79@end titlepage
80
81@contents
82
83@ifnottex
84@node Top
85@top Bison
86@insertcopying
87@end ifnottex
88
89@menu
90* Introduction::
91* Conditions::
92* Copying:: The @acronym{GNU} General Public License says
93 how you can copy and share Bison.
94
95Tutorial sections:
96* Concepts:: Basic concepts for understanding Bison.
97* Examples:: Three simple explained examples of using Bison.
98
99Reference sections:
100* Grammar File:: Writing Bison declarations and rules.
101* Interface:: C-language interface to the parser function @code{yyparse}.
102* Algorithm:: How the Bison parser works at run-time.
103* Error Recovery:: Writing rules for error recovery.
104* Context Dependency:: What to do if your language syntax is too
105 messy for Bison to handle straightforwardly.
106* Debugging:: Understanding or debugging Bison parsers.
107* Invocation:: How to run Bison (to produce the parser source file).
108* Other Languages:: Creating C++ and Java parsers.
109* FAQ:: Frequently Asked Questions
110* Table of Symbols:: All the keywords of the Bison language are explained.
111* Glossary:: Basic concepts are explained.
112* Copying This Manual:: License for copying this manual.
113* Index:: Cross-references to the text.
114
115@detailmenu
116 --- The Detailed Node Listing ---
117
118The Concepts of Bison
119
120* Language and Grammar:: Languages and context-free grammars,
121 as mathematical ideas.
122* Grammar in Bison:: How we represent grammars for Bison's sake.
123* Semantic Values:: Each token or syntactic grouping can have
124 a semantic value (the value of an integer,
125 the name of an identifier, etc.).
126* Semantic Actions:: Each rule can have an action containing C code.
127* GLR Parsers:: Writing parsers for general context-free languages.
128* Locations Overview:: Tracking Locations.
129* Bison Parser:: What are Bison's input and output,
130 how is the output used?
131* Stages:: Stages in writing and running Bison grammars.
132* Grammar Layout:: Overall structure of a Bison grammar file.
133
134Writing @acronym{GLR} Parsers
135
136* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
137* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
138* GLR Semantic Actions:: Deferred semantic actions have special concerns.
139* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
140
141Examples
142
143* RPN Calc:: Reverse polish notation calculator;
144 a first example with no operator precedence.
145* Infix Calc:: Infix (algebraic) notation calculator.
146 Operator precedence is introduced.
147* Simple Error Recovery:: Continuing after syntax errors.
148* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
149* Multi-function Calc:: Calculator with memory and trig functions.
150 It uses multiple data-types for semantic values.
151* Exercises:: Ideas for improving the multi-function calculator.
152
153Reverse Polish Notation Calculator
154
155* Rpcalc Declarations:: Prologue (declarations) for rpcalc.
156* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
157* Rpcalc Lexer:: The lexical analyzer.
158* Rpcalc Main:: The controlling function.
159* Rpcalc Error:: The error reporting function.
160* Rpcalc Generate:: Running Bison on the grammar file.
161* Rpcalc Compile:: Run the C compiler on the output code.
162
163Grammar Rules for @code{rpcalc}
164
165* Rpcalc Input::
166* Rpcalc Line::
167* Rpcalc Expr::
168
169Location Tracking Calculator: @code{ltcalc}
170
171* Ltcalc Declarations:: Bison and C declarations for ltcalc.
172* Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
173* Ltcalc Lexer:: The lexical analyzer.
174
175Multi-Function Calculator: @code{mfcalc}
176
177* Mfcalc Declarations:: Bison declarations for multi-function calculator.
178* Mfcalc Rules:: Grammar rules for the calculator.
179* Mfcalc Symbol Table:: Symbol table management subroutines.
180
181Bison Grammar Files
182
183* Grammar Outline:: Overall layout of the grammar file.
184* Symbols:: Terminal and nonterminal symbols.
185* Rules:: How to write grammar rules.
186* Recursion:: Writing recursive rules.
187* Semantics:: Semantic values and actions.
188* Locations:: Locations and actions.
189* Declarations:: All kinds of Bison declarations are described here.
190* Multiple Parsers:: Putting more than one Bison parser in one program.
191
192Outline of a Bison Grammar
193
194* Prologue:: Syntax and usage of the prologue.
195* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
196* Bison Declarations:: Syntax and usage of the Bison declarations section.
197* Grammar Rules:: Syntax and usage of the grammar rules section.
198* Epilogue:: Syntax and usage of the epilogue.
199
200Defining Language Semantics
201
202* Value Type:: Specifying one data type for all semantic values.
203* Multiple Types:: Specifying several alternative data types.
204* Actions:: An action is the semantic definition of a grammar rule.
205* Action Types:: Specifying data types for actions to operate on.
206* Mid-Rule Actions:: Most actions go at the end of a rule.
207 This says when, why and how to use the exceptional
208 action in the middle of a rule.
209
210Tracking Locations
211
212* Location Type:: Specifying a data type for locations.
213* Actions and Locations:: Using locations in actions.
214* Location Default Action:: Defining a general way to compute locations.
215
216Bison Declarations
217
218* Require Decl:: Requiring a Bison version.
219* Token Decl:: Declaring terminal symbols.
220* Precedence Decl:: Declaring terminals with precedence and associativity.
221* Union Decl:: Declaring the set of all semantic value types.
222* Type Decl:: Declaring the choice of type for a nonterminal symbol.
223* Initial Action Decl:: Code run before parsing starts.
224* Destructor Decl:: Declaring how symbols are freed.
225* Expect Decl:: Suppressing warnings about parsing conflicts.
226* Start Decl:: Specifying the start symbol.
227* Pure Decl:: Requesting a reentrant parser.
228* Push Decl:: Requesting a push parser.
229* Decl Summary:: Table of all Bison declarations.
230
231Parser C-Language Interface
232
233* Parser Function:: How to call @code{yyparse} and what it returns.
234* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
235* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
236* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
237* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
238* Lexical:: You must supply a function @code{yylex}
239 which reads tokens.
240* Error Reporting:: You must supply a function @code{yyerror}.
241* Action Features:: Special features for use in actions.
242* Internationalization:: How to let the parser speak in the user's
243 native language.
244
245The Lexical Analyzer Function @code{yylex}
246
247* Calling Convention:: How @code{yyparse} calls @code{yylex}.
248* Token Values:: How @code{yylex} must return the semantic value
249 of the token it has read.
250* Token Locations:: How @code{yylex} must return the text location
251 (line number, etc.) of the token, if the
252 actions want that.
253* Pure Calling:: How the calling convention differs in a pure parser
254 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
255
256The Bison Parser Algorithm
257
258* Lookahead:: Parser looks one token ahead when deciding what to do.
259* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
260* Precedence:: Operator precedence works by resolving conflicts.
261* Contextual Precedence:: When an operator's precedence depends on context.
262* Parser States:: The parser is a finite-state-machine with stack.
263* Reduce/Reduce:: When two rules are applicable in the same situation.
264* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
265* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
266* Memory Management:: What happens when memory is exhausted. How to avoid it.
267
268Operator Precedence
269
270* Why Precedence:: An example showing why precedence is needed.
271* Using Precedence:: How to specify precedence in Bison grammars.
272* Precedence Examples:: How these features are used in the previous example.
273* How Precedence:: How they work.
274
275Handling Context Dependencies
276
277* Semantic Tokens:: Token parsing can depend on the semantic context.
278* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
279* Tie-in Recovery:: Lexical tie-ins have implications for how
280 error recovery rules must be written.
281
282Debugging Your Parser
283
284* Understanding:: Understanding the structure of your parser.
285* Tracing:: Tracing the execution of your parser.
286
287Invoking Bison
288
289* Bison Options:: All the options described in detail,
290 in alphabetical order by short options.
291* Option Cross Key:: Alphabetical list of long options.
292* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
293
294Parsers Written In Other Languages
295
296* C++ Parsers:: The interface to generate C++ parser classes
297* Java Parsers:: The interface to generate Java parser classes
298
299C++ Parsers
300
301* C++ Bison Interface:: Asking for C++ parser generation
302* C++ Semantic Values:: %union vs. C++
303* C++ Location Values:: The position and location classes
304* C++ Parser Interface:: Instantiating and running the parser
305* C++ Scanner Interface:: Exchanges between yylex and parse
306* A Complete C++ Example:: Demonstrating their use
307
308A Complete C++ Example
309
310* Calc++ --- C++ Calculator:: The specifications
311* Calc++ Parsing Driver:: An active parsing context
312* Calc++ Parser:: A parser class
313* Calc++ Scanner:: A pure C++ Flex scanner
314* Calc++ Top Level:: Conducting the band
315
316Java Parsers
317
318* Java Bison Interface:: Asking for Java parser generation
319* Java Semantic Values:: %type and %token vs. Java
320* Java Location Values:: The position and location classes
321* Java Parser Interface:: Instantiating and running the parser
322* Java Scanner Interface:: Specifying the scanner for the parser
323* Java Action Features:: Special features for use in actions
324* Java Differences:: Differences between C/C++ and Java Grammars
325* Java Declarations Summary:: List of Bison declarations used with Java
326
327Frequently Asked Questions
328
329* Memory Exhausted:: Breaking the Stack Limits
330* How Can I Reset the Parser:: @code{yyparse} Keeps some State
331* Strings are Destroyed:: @code{yylval} Loses Track of Strings
332* Implementing Gotos/Loops:: Control Flow in the Calculator
333* Multiple start-symbols:: Factoring closely related grammars
334* Secure? Conform?:: Is Bison @acronym{POSIX} safe?
335* I can't build Bison:: Troubleshooting
336* Where can I find help?:: Troubleshouting
337* Bug Reports:: Troublereporting
338* More Languages:: Parsers in C++, Java, and so on
339* Beta Testing:: Experimenting development versions
340* Mailing Lists:: Meeting other Bison users
341
342Copying This Manual
343
344* Copying This Manual:: License for copying this manual.
345
346@end detailmenu
347@end menu
348
349@node Introduction
350@unnumbered Introduction
351@cindex introduction
352
353@dfn{Bison} is a general-purpose parser generator that converts an
354annotated context-free grammar into an @acronym{LALR}(1) or
355@acronym{GLR} parser for that grammar. Once you are proficient with
356Bison, you can use it to develop a wide range of language parsers, from those
357used in simple desk calculators to complex programming languages.
358
359Bison is upward compatible with Yacc: all properly-written Yacc grammars
360ought to work with Bison with no change. Anyone familiar with Yacc
361should be able to use Bison with little trouble. You need to be fluent in
362C or C++ programming in order to use Bison or to understand this manual.
363
364We begin with tutorial chapters that explain the basic concepts of using
365Bison and show three explained examples, each building on the last. If you
366don't know Bison or Yacc, start by reading these chapters. Reference
367chapters follow which describe specific aspects of Bison in detail.
368
369Bison was written primarily by Robert Corbett; Richard Stallman made it
370Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
371multi-character string literals and other features.
372
373This edition corresponds to version @value{VERSION} of Bison.
374
375@node Conditions
376@unnumbered Conditions for Using Bison
377
378The distribution terms for Bison-generated parsers permit using the
379parsers in nonfree programs. Before Bison version 2.2, these extra
380permissions applied only when Bison was generating @acronym{LALR}(1)
381parsers in C@. And before Bison version 1.24, Bison-generated
382parsers could be used only in programs that were free software.
383
384The other @acronym{GNU} programming tools, such as the @acronym{GNU} C
385compiler, have never
386had such a requirement. They could always be used for nonfree
387software. The reason Bison was different was not due to a special
388policy decision; it resulted from applying the usual General Public
389License to all of the Bison source code.
390
391The output of the Bison utility---the Bison parser file---contains a
392verbatim copy of a sizable piece of Bison, which is the code for the
393parser's implementation. (The actions from your grammar are inserted
394into this implementation at one point, but most of the rest of the
395implementation is not changed.) When we applied the @acronym{GPL}
396terms to the skeleton code for the parser's implementation,
397the effect was to restrict the use of Bison output to free software.
398
399We didn't change the terms because of sympathy for people who want to
400make software proprietary. @strong{Software should be free.} But we
401concluded that limiting Bison's use to free software was doing little to
402encourage people to make other software free. So we decided to make the
403practical conditions for using Bison match the practical conditions for
404using the other @acronym{GNU} tools.
405
406This exception applies when Bison is generating code for a parser.
407You can tell whether the exception applies to a Bison output file by
408inspecting the file for text beginning with ``As a special
409exception@dots{}''. The text spells out the exact terms of the
410exception.
411
412@node Copying
413@unnumbered GNU GENERAL PUBLIC LICENSE
414@include gpl-3.0.texi
415
416@node Concepts
417@chapter The Concepts of Bison
418
419This chapter introduces many of the basic concepts without which the
420details of Bison will not make sense. If you do not already know how to
421use Bison or Yacc, we suggest you start by reading this chapter carefully.
422
423@menu
424* Language and Grammar:: Languages and context-free grammars,
425 as mathematical ideas.
426* Grammar in Bison:: How we represent grammars for Bison's sake.
427* Semantic Values:: Each token or syntactic grouping can have
428 a semantic value (the value of an integer,
429 the name of an identifier, etc.).
430* Semantic Actions:: Each rule can have an action containing C code.
431* GLR Parsers:: Writing parsers for general context-free languages.
432* Locations Overview:: Tracking Locations.
433* Bison Parser:: What are Bison's input and output,
434 how is the output used?
435* Stages:: Stages in writing and running Bison grammars.
436* Grammar Layout:: Overall structure of a Bison grammar file.
437@end menu
438
439@node Language and Grammar
440@section Languages and Context-Free Grammars
441
442@cindex context-free grammar
443@cindex grammar, context-free
444In order for Bison to parse a language, it must be described by a
445@dfn{context-free grammar}. This means that you specify one or more
446@dfn{syntactic groupings} and give rules for constructing them from their
447parts. For example, in the C language, one kind of grouping is called an
448`expression'. One rule for making an expression might be, ``An expression
449can be made of a minus sign and another expression''. Another would be,
450``An expression can be an integer''. As you can see, rules are often
451recursive, but there must be at least one rule which leads out of the
452recursion.
453
454@cindex @acronym{BNF}
455@cindex Backus-Naur form
456The most common formal system for presenting such rules for humans to read
457is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in
458order to specify the language Algol 60. Any grammar expressed in
459@acronym{BNF} is a context-free grammar. The input to Bison is
460essentially machine-readable @acronym{BNF}.
461
462@cindex @acronym{LALR}(1) grammars
463@cindex @acronym{LR}(1) grammars
464There are various important subclasses of context-free grammar. Although it
465can handle almost all context-free grammars, Bison is optimized for what
466are called @acronym{LALR}(1) grammars.
467In brief, in these grammars, it must be possible to
468tell how to parse any portion of an input string with just a single
469token of lookahead. Strictly speaking, that is a description of an
470@acronym{LR}(1) grammar, and @acronym{LALR}(1) involves additional
471restrictions that are
472hard to explain simply; but it is rare in actual practice to find an
473@acronym{LR}(1) grammar that fails to be @acronym{LALR}(1).
474@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
475more information on this.
476
477@cindex @acronym{GLR} parsing
478@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
479@cindex ambiguous grammars
480@cindex nondeterministic parsing
481
482Parsers for @acronym{LALR}(1) grammars are @dfn{deterministic}, meaning
483roughly that the next grammar rule to apply at any point in the input is
484uniquely determined by the preceding input and a fixed, finite portion
485(called a @dfn{lookahead}) of the remaining input. A context-free
486grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
487apply the grammar rules to get the same inputs. Even unambiguous
488grammars can be @dfn{nondeterministic}, meaning that no fixed
489lookahead always suffices to determine the next grammar rule to apply.
490With the proper declarations, Bison is also able to parse these more
491general context-free grammars, using a technique known as @acronym{GLR}
492parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers
493are able to handle any context-free grammar for which the number of
494possible parses of any given string is finite.
495
496@cindex symbols (abstract)
497@cindex token
498@cindex syntactic grouping
499@cindex grouping, syntactic
500In the formal grammatical rules for a language, each kind of syntactic
501unit or grouping is named by a @dfn{symbol}. Those which are built by
502grouping smaller constructs according to grammatical rules are called
503@dfn{nonterminal symbols}; those which can't be subdivided are called
504@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
505corresponding to a single terminal symbol a @dfn{token}, and a piece
506corresponding to a single nonterminal symbol a @dfn{grouping}.
507
508We can use the C language as an example of what symbols, terminal and
509nonterminal, mean. The tokens of C are identifiers, constants (numeric
510and string), and the various keywords, arithmetic operators and
511punctuation marks. So the terminal symbols of a grammar for C include
512`identifier', `number', `string', plus one symbol for each keyword,
513operator or punctuation mark: `if', `return', `const', `static', `int',
514`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
515(These tokens can be subdivided into characters, but that is a matter of
516lexicography, not grammar.)
517
518Here is a simple C function subdivided into tokens:
519
520@ifinfo
521@example
522int /* @r{keyword `int'} */
523square (int x) /* @r{identifier, open-paren, keyword `int',}
524 @r{identifier, close-paren} */
525@{ /* @r{open-brace} */
526 return x * x; /* @r{keyword `return', identifier, asterisk,}
527 @r{identifier, semicolon} */
528@} /* @r{close-brace} */
529@end example
530@end ifinfo
531@ifnotinfo
532@example
533int /* @r{keyword `int'} */
534square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
535@{ /* @r{open-brace} */
536 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
537@} /* @r{close-brace} */
538@end example
539@end ifnotinfo
540
541The syntactic groupings of C include the expression, the statement, the
542declaration, and the function definition. These are represented in the
543grammar of C by nonterminal symbols `expression', `statement',
544`declaration' and `function definition'. The full grammar uses dozens of
545additional language constructs, each with its own nonterminal symbol, in
546order to express the meanings of these four. The example above is a
547function definition; it contains one declaration, and one statement. In
548the statement, each @samp{x} is an expression and so is @samp{x * x}.
549
550Each nonterminal symbol must have grammatical rules showing how it is made
551out of simpler constructs. For example, one kind of C statement is the
552@code{return} statement; this would be described with a grammar rule which
553reads informally as follows:
554
555@quotation
556A `statement' can be made of a `return' keyword, an `expression' and a
557`semicolon'.
558@end quotation
559
560@noindent
561There would be many other rules for `statement', one for each kind of
562statement in C.
563
564@cindex start symbol
565One nonterminal symbol must be distinguished as the special one which
566defines a complete utterance in the language. It is called the @dfn{start
567symbol}. In a compiler, this means a complete input program. In the C
568language, the nonterminal symbol `sequence of definitions and declarations'
569plays this role.
570
571For example, @samp{1 + 2} is a valid C expression---a valid part of a C
572program---but it is not valid as an @emph{entire} C program. In the
573context-free grammar of C, this follows from the fact that `expression' is
574not the start symbol.
575
576The Bison parser reads a sequence of tokens as its input, and groups the
577tokens using the grammar rules. If the input is valid, the end result is
578that the entire token sequence reduces to a single grouping whose symbol is
579the grammar's start symbol. If we use a grammar for C, the entire input
580must be a `sequence of definitions and declarations'. If not, the parser
581reports a syntax error.
582
583@node Grammar in Bison
584@section From Formal Rules to Bison Input
585@cindex Bison grammar
586@cindex grammar, Bison
587@cindex formal grammar
588
589A formal grammar is a mathematical construct. To define the language
590for Bison, you must write a file expressing the grammar in Bison syntax:
591a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
592
593A nonterminal symbol in the formal grammar is represented in Bison input
594as an identifier, like an identifier in C@. By convention, it should be
595in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
596
597The Bison representation for a terminal symbol is also called a @dfn{token
598type}. Token types as well can be represented as C-like identifiers. By
599convention, these identifiers should be upper case to distinguish them from
600nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
601@code{RETURN}. A terminal symbol that stands for a particular keyword in
602the language should be named after that keyword converted to upper case.
603The terminal symbol @code{error} is reserved for error recovery.
604@xref{Symbols}.
605
606A terminal symbol can also be represented as a character literal, just like
607a C character constant. You should do this whenever a token is just a
608single character (parenthesis, plus-sign, etc.): use that same character in
609a literal as the terminal symbol for that token.
610
611A third way to represent a terminal symbol is with a C string constant
612containing several characters. @xref{Symbols}, for more information.
613
614The grammar rules also have an expression in Bison syntax. For example,
615here is the Bison rule for a C @code{return} statement. The semicolon in
616quotes is a literal character token, representing part of the C syntax for
617the statement; the naked semicolon, and the colon, are Bison punctuation
618used in every rule.
619
620@example
621stmt: RETURN expr ';'
622 ;
623@end example
624
625@noindent
626@xref{Rules, ,Syntax of Grammar Rules}.
627
628@node Semantic Values
629@section Semantic Values
630@cindex semantic value
631@cindex value, semantic
632
633A formal grammar selects tokens only by their classifications: for example,
634if a rule mentions the terminal symbol `integer constant', it means that
635@emph{any} integer constant is grammatically valid in that position. The
636precise value of the constant is irrelevant to how to parse the input: if
637@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
638grammatical.
639
640But the precise value is very important for what the input means once it is
641parsed. A compiler is useless if it fails to distinguish between 4, 1 and
6423989 as constants in the program! Therefore, each token in a Bison grammar
643has both a token type and a @dfn{semantic value}. @xref{Semantics,
644,Defining Language Semantics},
645for details.
646
647The token type is a terminal symbol defined in the grammar, such as
648@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
649you need to know to decide where the token may validly appear and how to
650group it with other tokens. The grammar rules know nothing about tokens
651except their types.
652
653The semantic value has all the rest of the information about the
654meaning of the token, such as the value of an integer, or the name of an
655identifier. (A token such as @code{','} which is just punctuation doesn't
656need to have any semantic value.)
657
658For example, an input token might be classified as token type
659@code{INTEGER} and have the semantic value 4. Another input token might
660have the same token type @code{INTEGER} but value 3989. When a grammar
661rule says that @code{INTEGER} is allowed, either of these tokens is
662acceptable because each is an @code{INTEGER}. When the parser accepts the
663token, it keeps track of the token's semantic value.
664
665Each grouping can also have a semantic value as well as its nonterminal
666symbol. For example, in a calculator, an expression typically has a
667semantic value that is a number. In a compiler for a programming
668language, an expression typically has a semantic value that is a tree
669structure describing the meaning of the expression.
670
671@node Semantic Actions
672@section Semantic Actions
673@cindex semantic actions
674@cindex actions, semantic
675
676In order to be useful, a program must do more than parse input; it must
677also produce some output based on the input. In a Bison grammar, a grammar
678rule can have an @dfn{action} made up of C statements. Each time the
679parser recognizes a match for that rule, the action is executed.
680@xref{Actions}.
681
682Most of the time, the purpose of an action is to compute the semantic value
683of the whole construct from the semantic values of its parts. For example,
684suppose we have a rule which says an expression can be the sum of two
685expressions. When the parser recognizes such a sum, each of the
686subexpressions has a semantic value which describes how it was built up.
687The action for this rule should create a similar sort of value for the
688newly recognized larger expression.
689
690For example, here is a rule that says an expression can be the sum of
691two subexpressions:
692
693@example
694expr: expr '+' expr @{ $$ = $1 + $3; @}
695 ;
696@end example
697
698@noindent
699The action says how to produce the semantic value of the sum expression
700from the values of the two subexpressions.
701
702@node GLR Parsers
703@section Writing @acronym{GLR} Parsers
704@cindex @acronym{GLR} parsing
705@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
706@findex %glr-parser
707@cindex conflicts
708@cindex shift/reduce conflicts
709@cindex reduce/reduce conflicts
710
711In some grammars, Bison's standard
712@acronym{LALR}(1) parsing algorithm cannot decide whether to apply a
713certain grammar rule at a given point. That is, it may not be able to
714decide (on the basis of the input read so far) which of two possible
715reductions (applications of a grammar rule) applies, or whether to apply
716a reduction or read more of the input and apply a reduction later in the
717input. These are known respectively as @dfn{reduce/reduce} conflicts
718(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
719(@pxref{Shift/Reduce}).
720
721To use a grammar that is not easily modified to be @acronym{LALR}(1), a
722more general parsing algorithm is sometimes necessary. If you include
723@code{%glr-parser} among the Bison declarations in your file
724(@pxref{Grammar Outline}), the result is a Generalized @acronym{LR}
725(@acronym{GLR}) parser. These parsers handle Bison grammars that
726contain no unresolved conflicts (i.e., after applying precedence
727declarations) identically to @acronym{LALR}(1) parsers. However, when
728faced with unresolved shift/reduce and reduce/reduce conflicts,
729@acronym{GLR} parsers use the simple expedient of doing both,
730effectively cloning the parser to follow both possibilities. Each of
731the resulting parsers can again split, so that at any given time, there
732can be any number of possible parses being explored. The parsers
733proceed in lockstep; that is, all of them consume (shift) a given input
734symbol before any of them proceed to the next. Each of the cloned
735parsers eventually meets one of two possible fates: either it runs into
736a parsing error, in which case it simply vanishes, or it merges with
737another parser, because the two of them have reduced the input to an
738identical set of symbols.
739
740During the time that there are multiple parsers, semantic actions are
741recorded, but not performed. When a parser disappears, its recorded
742semantic actions disappear as well, and are never performed. When a
743reduction makes two parsers identical, causing them to merge, Bison
744records both sets of semantic actions. Whenever the last two parsers
745merge, reverting to the single-parser case, Bison resolves all the
746outstanding actions either by precedences given to the grammar rules
747involved, or by performing both actions, and then calling a designated
748user-defined function on the resulting values to produce an arbitrary
749merged result.
750
751@menu
752* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
753* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
754* GLR Semantic Actions:: Deferred semantic actions have special concerns.
755* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
756@end menu
757
758@node Simple GLR Parsers
759@subsection Using @acronym{GLR} on Unambiguous Grammars
760@cindex @acronym{GLR} parsing, unambiguous grammars
761@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars
762@findex %glr-parser
763@findex %expect-rr
764@cindex conflicts
765@cindex reduce/reduce conflicts
766@cindex shift/reduce conflicts
767
768In the simplest cases, you can use the @acronym{GLR} algorithm
769to parse grammars that are unambiguous, but fail to be @acronym{LALR}(1).
770Such grammars typically require more than one symbol of lookahead,
771or (in rare cases) fall into the category of grammars in which the
772@acronym{LALR}(1) algorithm throws away too much information (they are in
773@acronym{LR}(1), but not @acronym{LALR}(1), @ref{Mystery Conflicts}).
774
775Consider a problem that
776arises in the declaration of enumerated and subrange types in the
777programming language Pascal. Here are some examples:
778
779@example
780type subrange = lo .. hi;
781type enum = (a, b, c);
782@end example
783
784@noindent
785The original language standard allows only numeric
786literals and constant identifiers for the subrange bounds (@samp{lo}
787and @samp{hi}), but Extended Pascal (@acronym{ISO}/@acronym{IEC}
78810206) and many other
789Pascal implementations allow arbitrary expressions there. This gives
790rise to the following situation, containing a superfluous pair of
791parentheses:
792
793@example
794type subrange = (a) .. b;
795@end example
796
797@noindent
798Compare this to the following declaration of an enumerated
799type with only one value:
800
801@example
802type enum = (a);
803@end example
804
805@noindent
806(These declarations are contrived, but they are syntactically
807valid, and more-complicated cases can come up in practical programs.)
808
809These two declarations look identical until the @samp{..} token.
810With normal @acronym{LALR}(1) one-token lookahead it is not
811possible to decide between the two forms when the identifier
812@samp{a} is parsed. It is, however, desirable
813for a parser to decide this, since in the latter case
814@samp{a} must become a new identifier to represent the enumeration
815value, while in the former case @samp{a} must be evaluated with its
816current meaning, which may be a constant or even a function call.
817
818You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
819to be resolved later, but this typically requires substantial
820contortions in both semantic actions and large parts of the
821grammar, where the parentheses are nested in the recursive rules for
822expressions.
823
824You might think of using the lexer to distinguish between the two
825forms by returning different tokens for currently defined and
826undefined identifiers. But if these declarations occur in a local
827scope, and @samp{a} is defined in an outer scope, then both forms
828are possible---either locally redefining @samp{a}, or using the
829value of @samp{a} from the outer scope. So this approach cannot
830work.
831
832A simple solution to this problem is to declare the parser to
833use the @acronym{GLR} algorithm.
834When the @acronym{GLR} parser reaches the critical state, it
835merely splits into two branches and pursues both syntax rules
836simultaneously. Sooner or later, one of them runs into a parsing
837error. If there is a @samp{..} token before the next
838@samp{;}, the rule for enumerated types fails since it cannot
839accept @samp{..} anywhere; otherwise, the subrange type rule
840fails since it requires a @samp{..} token. So one of the branches
841fails silently, and the other one continues normally, performing
842all the intermediate actions that were postponed during the split.
843
844If the input is syntactically incorrect, both branches fail and the parser
845reports a syntax error as usual.
846
847The effect of all this is that the parser seems to ``guess'' the
848correct branch to take, or in other words, it seems to use more
849lookahead than the underlying @acronym{LALR}(1) algorithm actually allows
850for. In this example, @acronym{LALR}(2) would suffice, but also some cases
851that are not @acronym{LALR}(@math{k}) for any @math{k} can be handled this way.
852
853In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time,
854and the current Bison parser even takes exponential time and space
855for some grammars. In practice, this rarely happens, and for many
856grammars it is possible to prove that it cannot happen.
857The present example contains only one conflict between two
858rules, and the type-declaration context containing the conflict
859cannot be nested. So the number of
860branches that can exist at any time is limited by the constant 2,
861and the parsing time is still linear.
862
863Here is a Bison grammar corresponding to the example above. It
864parses a vastly simplified form of Pascal type declarations.
865
866@example
867%token TYPE DOTDOT ID
868
869@group
870%left '+' '-'
871%left '*' '/'
872@end group
873
874%%
875
876@group
877type_decl : TYPE ID '=' type ';'
878 ;
879@end group
880
881@group
882type : '(' id_list ')'
883 | expr DOTDOT expr
884 ;
885@end group
886
887@group
888id_list : ID
889 | id_list ',' ID
890 ;
891@end group
892
893@group
894expr : '(' expr ')'
895 | expr '+' expr
896 | expr '-' expr
897 | expr '*' expr
898 | expr '/' expr
899 | ID
900 ;
901@end group
902@end example
903
904When used as a normal @acronym{LALR}(1) grammar, Bison correctly complains
905about one reduce/reduce conflict. In the conflicting situation the
906parser chooses one of the alternatives, arbitrarily the one
907declared first. Therefore the following correct input is not
908recognized:
909
910@example
911type t = (a) .. b;
912@end example
913
914The parser can be turned into a @acronym{GLR} parser, while also telling Bison
915to be silent about the one known reduce/reduce conflict, by
916adding these two declarations to the Bison input file (before the first
917@samp{%%}):
918
919@example
920%glr-parser
921%expect-rr 1
922@end example
923
924@noindent
925No change in the grammar itself is required. Now the
926parser recognizes all valid declarations, according to the
927limited syntax above, transparently. In fact, the user does not even
928notice when the parser splits.
929
930So here we have a case where we can use the benefits of @acronym{GLR},
931almost without disadvantages. Even in simple cases like this, however,
932there are at least two potential problems to beware. First, always
933analyze the conflicts reported by Bison to make sure that @acronym{GLR}
934splitting is only done where it is intended. A @acronym{GLR} parser
935splitting inadvertently may cause problems less obvious than an
936@acronym{LALR} parser statically choosing the wrong alternative in a
937conflict. Second, consider interactions with the lexer (@pxref{Semantic
938Tokens}) with great care. Since a split parser consumes tokens without
939performing any actions during the split, the lexer cannot obtain
940information via parser actions. Some cases of lexer interactions can be
941eliminated by using @acronym{GLR} to shift the complications from the
942lexer to the parser. You must check the remaining cases for
943correctness.
944
945In our example, it would be safe for the lexer to return tokens based on
946their current meanings in some symbol table, because no new symbols are
947defined in the middle of a type declaration. Though it is possible for
948a parser to define the enumeration constants as they are parsed, before
949the type declaration is completed, it actually makes no difference since
950they cannot be used within the same enumerated type declaration.
951
952@node Merging GLR Parses
953@subsection Using @acronym{GLR} to Resolve Ambiguities
954@cindex @acronym{GLR} parsing, ambiguous grammars
955@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, ambiguous grammars
956@findex %dprec
957@findex %merge
958@cindex conflicts
959@cindex reduce/reduce conflicts
960
961Let's consider an example, vastly simplified from a C++ grammar.
962
963@example
964%@{
965 #include <stdio.h>
966 #define YYSTYPE char const *
967 int yylex (void);
968 void yyerror (char const *);
969%@}
970
971%token TYPENAME ID
972
973%right '='
974%left '+'
975
976%glr-parser
977
978%%
979
980prog :
981 | prog stmt @{ printf ("\n"); @}
982 ;
983
984stmt : expr ';' %dprec 1
985 | decl %dprec 2
986 ;
987
988expr : ID @{ printf ("%s ", $$); @}
989 | TYPENAME '(' expr ')'
990 @{ printf ("%s <cast> ", $1); @}
991 | expr '+' expr @{ printf ("+ "); @}
992 | expr '=' expr @{ printf ("= "); @}
993 ;
994
995decl : TYPENAME declarator ';'
996 @{ printf ("%s <declare> ", $1); @}
997 | TYPENAME declarator '=' expr ';'
998 @{ printf ("%s <init-declare> ", $1); @}
999 ;
1000
1001declarator : ID @{ printf ("\"%s\" ", $1); @}
1002 | '(' declarator ')'
1003 ;
1004@end example
1005
1006@noindent
1007This models a problematic part of the C++ grammar---the ambiguity between
1008certain declarations and statements. For example,
1009
1010@example
1011T (x) = y+z;
1012@end example
1013
1014@noindent
1015parses as either an @code{expr} or a @code{stmt}
1016(assuming that @samp{T} is recognized as a @code{TYPENAME} and
1017@samp{x} as an @code{ID}).
1018Bison detects this as a reduce/reduce conflict between the rules
1019@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
1020time it encounters @code{x} in the example above. Since this is a
1021@acronym{GLR} parser, it therefore splits the problem into two parses, one for
1022each choice of resolving the reduce/reduce conflict.
1023Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1024however, neither of these parses ``dies,'' because the grammar as it stands is
1025ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1026the other reduces @code{stmt : decl}, after which both parsers are in an
1027identical state: they've seen @samp{prog stmt} and have the same unprocessed
1028input remaining. We say that these parses have @dfn{merged.}
1029
1030At this point, the @acronym{GLR} parser requires a specification in the
1031grammar of how to choose between the competing parses.
1032In the example above, the two @code{%dprec}
1033declarations specify that Bison is to give precedence
1034to the parse that interprets the example as a
1035@code{decl}, which implies that @code{x} is a declarator.
1036The parser therefore prints
1037
1038@example
1039"x" y z + T <init-declare>
1040@end example
1041
1042The @code{%dprec} declarations only come into play when more than one
1043parse survives. Consider a different input string for this parser:
1044
1045@example
1046T (x) + y;
1047@end example
1048
1049@noindent
1050This is another example of using @acronym{GLR} to parse an unambiguous
1051construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
1052Here, there is no ambiguity (this cannot be parsed as a declaration).
1053However, at the time the Bison parser encounters @code{x}, it does not
1054have enough information to resolve the reduce/reduce conflict (again,
1055between @code{x} as an @code{expr} or a @code{declarator}). In this
1056case, no precedence declaration is used. Again, the parser splits
1057into two, one assuming that @code{x} is an @code{expr}, and the other
1058assuming @code{x} is a @code{declarator}. The second of these parsers
1059then vanishes when it sees @code{+}, and the parser prints
1060
1061@example
1062x T <cast> y +
1063@end example
1064
1065Suppose that instead of resolving the ambiguity, you wanted to see all
1066the possibilities. For this purpose, you must merge the semantic
1067actions of the two possible parsers, rather than choosing one over the
1068other. To do so, you could change the declaration of @code{stmt} as
1069follows:
1070
1071@example
1072stmt : expr ';' %merge <stmtMerge>
1073 | decl %merge <stmtMerge>
1074 ;
1075@end example
1076
1077@noindent
1078and define the @code{stmtMerge} function as:
1079
1080@example
1081static YYSTYPE
1082stmtMerge (YYSTYPE x0, YYSTYPE x1)
1083@{
1084 printf ("<OR> ");
1085 return "";
1086@}
1087@end example
1088
1089@noindent
1090with an accompanying forward declaration
1091in the C declarations at the beginning of the file:
1092
1093@example
1094%@{
1095 #define YYSTYPE char const *
1096 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1097%@}
1098@end example
1099
1100@noindent
1101With these declarations, the resulting parser parses the first example
1102as both an @code{expr} and a @code{decl}, and prints
1103
1104@example
1105"x" y z + T <init-declare> x T <cast> y z + = <OR>
1106@end example
1107
1108Bison requires that all of the
1109productions that participate in any particular merge have identical
1110@samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1111and the parser will report an error during any parse that results in
1112the offending merge.
1113
1114@node GLR Semantic Actions
1115@subsection GLR Semantic Actions
1116
1117@cindex deferred semantic actions
1118By definition, a deferred semantic action is not performed at the same time as
1119the associated reduction.
1120This raises caveats for several Bison features you might use in a semantic
1121action in a @acronym{GLR} parser.
1122
1123@vindex yychar
1124@cindex @acronym{GLR} parsers and @code{yychar}
1125@vindex yylval
1126@cindex @acronym{GLR} parsers and @code{yylval}
1127@vindex yylloc
1128@cindex @acronym{GLR} parsers and @code{yylloc}
1129In any semantic action, you can examine @code{yychar} to determine the type of
1130the lookahead token present at the time of the associated reduction.
1131After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1132you can then examine @code{yylval} and @code{yylloc} to determine the
1133lookahead token's semantic value and location, if any.
1134In a nondeferred semantic action, you can also modify any of these variables to
1135influence syntax analysis.
1136@xref{Lookahead, ,Lookahead Tokens}.
1137
1138@findex yyclearin
1139@cindex @acronym{GLR} parsers and @code{yyclearin}
1140In a deferred semantic action, it's too late to influence syntax analysis.
1141In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1142shallow copies of the values they had at the time of the associated reduction.
1143For this reason alone, modifying them is dangerous.
1144Moreover, the result of modifying them is undefined and subject to change with
1145future versions of Bison.
1146For example, if a semantic action might be deferred, you should never write it
1147to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1148memory referenced by @code{yylval}.
1149
1150@findex YYERROR
1151@cindex @acronym{GLR} parsers and @code{YYERROR}
1152Another Bison feature requiring special consideration is @code{YYERROR}
1153(@pxref{Action Features}), which you can invoke in a semantic action to
1154initiate error recovery.
1155During deterministic @acronym{GLR} operation, the effect of @code{YYERROR} is
1156the same as its effect in an @acronym{LALR}(1) parser.
1157In a deferred semantic action, its effect is undefined.
1158@c The effect is probably a syntax error at the split point.
1159
1160Also, see @ref{Location Default Action, ,Default Action for Locations}, which
1161describes a special usage of @code{YYLLOC_DEFAULT} in @acronym{GLR} parsers.
1162
1163@node Compiler Requirements
1164@subsection Considerations when Compiling @acronym{GLR} Parsers
1165@cindex @code{inline}
1166@cindex @acronym{GLR} parsers and @code{inline}
1167
1168The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or
1169later. In addition, they use the @code{inline} keyword, which is not
1170C89, but is C99 and is a common extension in pre-C99 compilers. It is
1171up to the user of these parsers to handle
1172portability issues. For instance, if using Autoconf and the Autoconf
1173macro @code{AC_C_INLINE}, a mere
1174
1175@example
1176%@{
1177 #include <config.h>
1178%@}
1179@end example
1180
1181@noindent
1182will suffice. Otherwise, we suggest
1183
1184@example
1185%@{
1186 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1187 #define inline
1188 #endif
1189%@}
1190@end example
1191
1192@node Locations Overview
1193@section Locations
1194@cindex location
1195@cindex textual location
1196@cindex location, textual
1197
1198Many applications, like interpreters or compilers, have to produce verbose
1199and useful error messages. To achieve this, one must be able to keep track of
1200the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
1201Bison provides a mechanism for handling these locations.
1202
1203Each token has a semantic value. In a similar fashion, each token has an
1204associated location, but the type of locations is the same for all tokens and
1205groupings. Moreover, the output parser is equipped with a default data
1206structure for storing locations (@pxref{Locations}, for more details).
1207
1208Like semantic values, locations can be reached in actions using a dedicated
1209set of constructs. In the example above, the location of the whole grouping
1210is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1211@code{@@3}.
1212
1213When a rule is matched, a default action is used to compute the semantic value
1214of its left hand side (@pxref{Actions}). In the same way, another default
1215action is used for locations. However, the action for locations is general
1216enough for most cases, meaning there is usually no need to describe for each
1217rule how @code{@@$} should be formed. When building a new location for a given
1218grouping, the default behavior of the output parser is to take the beginning
1219of the first symbol, and the end of the last symbol.
1220
1221@node Bison Parser
1222@section Bison Output: the Parser File
1223@cindex Bison parser
1224@cindex Bison utility
1225@cindex lexical analyzer, purpose
1226@cindex parser
1227
1228When you run Bison, you give it a Bison grammar file as input. The output
1229is a C source file that parses the language described by the grammar.
1230This file is called a @dfn{Bison parser}. Keep in mind that the Bison
1231utility and the Bison parser are two distinct programs: the Bison utility
1232is a program whose output is the Bison parser that becomes part of your
1233program.
1234
1235The job of the Bison parser is to group tokens into groupings according to
1236the grammar rules---for example, to build identifiers and operators into
1237expressions. As it does this, it runs the actions for the grammar rules it
1238uses.
1239
1240The tokens come from a function called the @dfn{lexical analyzer} that
1241you must supply in some fashion (such as by writing it in C). The Bison
1242parser calls the lexical analyzer each time it wants a new token. It
1243doesn't know what is ``inside'' the tokens (though their semantic values
1244may reflect this). Typically the lexical analyzer makes the tokens by
1245parsing characters of text, but Bison does not depend on this.
1246@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
1247
1248The Bison parser file is C code which defines a function named
1249@code{yyparse} which implements that grammar. This function does not make
1250a complete C program: you must supply some additional functions. One is
1251the lexical analyzer. Another is an error-reporting function which the
1252parser calls to report an error. In addition, a complete C program must
1253start with a function called @code{main}; you have to provide this, and
1254arrange for it to call @code{yyparse} or the parser will never run.
1255@xref{Interface, ,Parser C-Language Interface}.
1256
1257Aside from the token type names and the symbols in the actions you
1258write, all symbols defined in the Bison parser file itself
1259begin with @samp{yy} or @samp{YY}. This includes interface functions
1260such as the lexical analyzer function @code{yylex}, the error reporting
1261function @code{yyerror} and the parser function @code{yyparse} itself.
1262This also includes numerous identifiers used for internal purposes.
1263Therefore, you should avoid using C identifiers starting with @samp{yy}
1264or @samp{YY} in the Bison grammar file except for the ones defined in
1265this manual. Also, you should avoid using the C identifiers
1266@samp{malloc} and @samp{free} for anything other than their usual
1267meanings.
1268
1269In some cases the Bison parser file includes system headers, and in
1270those cases your code should respect the identifiers reserved by those
1271headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>}, @code{<malloc.h>},
1272@code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
1273declare memory allocators and related types. @code{<libintl.h>} is
1274included if message translation is in use
1275(@pxref{Internationalization}). Other system headers may
1276be included if you define @code{YYDEBUG} to a nonzero value
1277(@pxref{Tracing, ,Tracing Your Parser}).
1278
1279@node Stages
1280@section Stages in Using Bison
1281@cindex stages in using Bison
1282@cindex using Bison
1283
1284The actual language-design process using Bison, from grammar specification
1285to a working compiler or interpreter, has these parts:
1286
1287@enumerate
1288@item
1289Formally specify the grammar in a form recognized by Bison
1290(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1291in the language, describe the action that is to be taken when an
1292instance of that rule is recognized. The action is described by a
1293sequence of C statements.
1294
1295@item
1296Write a lexical analyzer to process input and pass tokens to the parser.
1297The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1298Lexical Analyzer Function @code{yylex}}). It could also be produced
1299using Lex, but the use of Lex is not discussed in this manual.
1300
1301@item
1302Write a controlling function that calls the Bison-produced parser.
1303
1304@item
1305Write error-reporting routines.
1306@end enumerate
1307
1308To turn this source code as written into a runnable program, you
1309must follow these steps:
1310
1311@enumerate
1312@item
1313Run Bison on the grammar to produce the parser.
1314
1315@item
1316Compile the code output by Bison, as well as any other source files.
1317
1318@item
1319Link the object files to produce the finished product.
1320@end enumerate
1321
1322@node Grammar Layout
1323@section The Overall Layout of a Bison Grammar
1324@cindex grammar file
1325@cindex file format
1326@cindex format of grammar file
1327@cindex layout of Bison grammar
1328
1329The input file for the Bison utility is a @dfn{Bison grammar file}. The
1330general form of a Bison grammar file is as follows:
1331
1332@example
1333%@{
1334@var{Prologue}
1335%@}
1336
1337@var{Bison declarations}
1338
1339%%
1340@var{Grammar rules}
1341%%
1342@var{Epilogue}
1343@end example
1344
1345@noindent
1346The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1347in every Bison grammar file to separate the sections.
1348
1349The prologue may define types and variables used in the actions. You can
1350also use preprocessor commands to define macros used there, and use
1351@code{#include} to include header files that do any of these things.
1352You need to declare the lexical analyzer @code{yylex} and the error
1353printer @code{yyerror} here, along with any other global identifiers
1354used by the actions in the grammar rules.
1355
1356The Bison declarations declare the names of the terminal and nonterminal
1357symbols, and may also describe operator precedence and the data types of
1358semantic values of various symbols.
1359
1360The grammar rules define how to construct each nonterminal symbol from its
1361parts.
1362
1363The epilogue can contain any code you want to use. Often the
1364definitions of functions declared in the prologue go here. In a
1365simple program, all the rest of the program can go here.
1366
1367@node Examples
1368@chapter Examples
1369@cindex simple examples
1370@cindex examples, simple
1371
1372Now we show and explain three sample programs written using Bison: a
1373reverse polish notation calculator, an algebraic (infix) notation
1374calculator, and a multi-function calculator. All three have been tested
1375under BSD Unix 4.3; each produces a usable, though limited, interactive
1376desk-top calculator.
1377
1378These examples are simple, but Bison grammars for real programming
1379languages are written the same way. You can copy these examples into a
1380source file to try them.
1381
1382@menu
1383* RPN Calc:: Reverse polish notation calculator;
1384 a first example with no operator precedence.
1385* Infix Calc:: Infix (algebraic) notation calculator.
1386 Operator precedence is introduced.
1387* Simple Error Recovery:: Continuing after syntax errors.
1388* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
1389* Multi-function Calc:: Calculator with memory and trig functions.
1390 It uses multiple data-types for semantic values.
1391* Exercises:: Ideas for improving the multi-function calculator.
1392@end menu
1393
1394@node RPN Calc
1395@section Reverse Polish Notation Calculator
1396@cindex reverse polish notation
1397@cindex polish notation calculator
1398@cindex @code{rpcalc}
1399@cindex calculator, simple
1400
1401The first example is that of a simple double-precision @dfn{reverse polish
1402notation} calculator (a calculator using postfix operators). This example
1403provides a good starting point, since operator precedence is not an issue.
1404The second example will illustrate how operator precedence is handled.
1405
1406The source code for this calculator is named @file{rpcalc.y}. The
1407@samp{.y} extension is a convention used for Bison input files.
1408
1409@menu
1410* Rpcalc Declarations:: Prologue (declarations) for rpcalc.
1411* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
1412* Rpcalc Lexer:: The lexical analyzer.
1413* Rpcalc Main:: The controlling function.
1414* Rpcalc Error:: The error reporting function.
1415* Rpcalc Generate:: Running Bison on the grammar file.
1416* Rpcalc Compile:: Run the C compiler on the output code.
1417@end menu
1418
1419@node Rpcalc Declarations
1420@subsection Declarations for @code{rpcalc}
1421
1422Here are the C and Bison declarations for the reverse polish notation
1423calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1424
1425@example
1426/* Reverse polish notation calculator. */
1427
1428%@{
1429 #define YYSTYPE double
1430 #include <math.h>
1431 int yylex (void);
1432 void yyerror (char const *);
1433%@}
1434
1435%token NUM
1436
1437%% /* Grammar rules and actions follow. */
1438@end example
1439
1440The declarations section (@pxref{Prologue, , The prologue}) contains two
1441preprocessor directives and two forward declarations.
1442
1443The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1444specifying the C data type for semantic values of both tokens and
1445groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1446Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1447don't define it, @code{int} is the default. Because we specify
1448@code{double}, each token and each expression has an associated value,
1449which is a floating point number.
1450
1451The @code{#include} directive is used to declare the exponentiation
1452function @code{pow}.
1453
1454The forward declarations for @code{yylex} and @code{yyerror} are
1455needed because the C language requires that functions be declared
1456before they are used. These functions will be defined in the
1457epilogue, but the parser calls them so they must be declared in the
1458prologue.
1459
1460The second section, Bison declarations, provides information to Bison
1461about the token types (@pxref{Bison Declarations, ,The Bison
1462Declarations Section}). Each terminal symbol that is not a
1463single-character literal must be declared here. (Single-character
1464literals normally don't need to be declared.) In this example, all the
1465arithmetic operators are designated by single-character literals, so the
1466only terminal symbol that needs to be declared is @code{NUM}, the token
1467type for numeric constants.
1468
1469@node Rpcalc Rules
1470@subsection Grammar Rules for @code{rpcalc}
1471
1472Here are the grammar rules for the reverse polish notation calculator.
1473
1474@example
1475input: /* empty */
1476 | input line
1477;
1478
1479line: '\n'
1480 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1481;
1482
1483exp: NUM @{ $$ = $1; @}
1484 | exp exp '+' @{ $$ = $1 + $2; @}
1485 | exp exp '-' @{ $$ = $1 - $2; @}
1486 | exp exp '*' @{ $$ = $1 * $2; @}
1487 | exp exp '/' @{ $$ = $1 / $2; @}
1488 /* Exponentiation */
1489 | exp exp '^' @{ $$ = pow ($1, $2); @}
1490 /* Unary minus */
1491 | exp 'n' @{ $$ = -$1; @}
1492;
1493%%
1494@end example
1495
1496The groupings of the rpcalc ``language'' defined here are the expression
1497(given the name @code{exp}), the line of input (@code{line}), and the
1498complete input transcript (@code{input}). Each of these nonterminal
1499symbols has several alternate rules, joined by the vertical bar @samp{|}
1500which is read as ``or''. The following sections explain what these rules
1501mean.
1502
1503The semantics of the language is determined by the actions taken when a
1504grouping is recognized. The actions are the C code that appears inside
1505braces. @xref{Actions}.
1506
1507You must specify these actions in C, but Bison provides the means for
1508passing semantic values between the rules. In each action, the
1509pseudo-variable @code{$$} stands for the semantic value for the grouping
1510that the rule is going to construct. Assigning a value to @code{$$} is the
1511main job of most actions. The semantic values of the components of the
1512rule are referred to as @code{$1}, @code{$2}, and so on.
1513
1514@menu
1515* Rpcalc Input::
1516* Rpcalc Line::
1517* Rpcalc Expr::
1518@end menu
1519
1520@node Rpcalc Input
1521@subsubsection Explanation of @code{input}
1522
1523Consider the definition of @code{input}:
1524
1525@example
1526input: /* empty */
1527 | input line
1528;
1529@end example
1530
1531This definition reads as follows: ``A complete input is either an empty
1532string, or a complete input followed by an input line''. Notice that
1533``complete input'' is defined in terms of itself. This definition is said
1534to be @dfn{left recursive} since @code{input} appears always as the
1535leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1536
1537The first alternative is empty because there are no symbols between the
1538colon and the first @samp{|}; this means that @code{input} can match an
1539empty string of input (no tokens). We write the rules this way because it
1540is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1541It's conventional to put an empty alternative first and write the comment
1542@samp{/* empty */} in it.
1543
1544The second alternate rule (@code{input line}) handles all nontrivial input.
1545It means, ``After reading any number of lines, read one more line if
1546possible.'' The left recursion makes this rule into a loop. Since the
1547first alternative matches empty input, the loop can be executed zero or
1548more times.
1549
1550The parser function @code{yyparse} continues to process input until a
1551grammatical error is seen or the lexical analyzer says there are no more
1552input tokens; we will arrange for the latter to happen at end-of-input.
1553
1554@node Rpcalc Line
1555@subsubsection Explanation of @code{line}
1556
1557Now consider the definition of @code{line}:
1558
1559@example
1560line: '\n'
1561 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1562;
1563@end example
1564
1565The first alternative is a token which is a newline character; this means
1566that rpcalc accepts a blank line (and ignores it, since there is no
1567action). The second alternative is an expression followed by a newline.
1568This is the alternative that makes rpcalc useful. The semantic value of
1569the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1570question is the first symbol in the alternative. The action prints this
1571value, which is the result of the computation the user asked for.
1572
1573This action is unusual because it does not assign a value to @code{$$}. As
1574a consequence, the semantic value associated with the @code{line} is
1575uninitialized (its value will be unpredictable). This would be a bug if
1576that value were ever used, but we don't use it: once rpcalc has printed the
1577value of the user's input line, that value is no longer needed.
1578
1579@node Rpcalc Expr
1580@subsubsection Explanation of @code{expr}
1581
1582The @code{exp} grouping has several rules, one for each kind of expression.
1583The first rule handles the simplest expressions: those that are just numbers.
1584The second handles an addition-expression, which looks like two expressions
1585followed by a plus-sign. The third handles subtraction, and so on.
1586
1587@example
1588exp: NUM
1589 | exp exp '+' @{ $$ = $1 + $2; @}
1590 | exp exp '-' @{ $$ = $1 - $2; @}
1591 @dots{}
1592 ;
1593@end example
1594
1595We have used @samp{|} to join all the rules for @code{exp}, but we could
1596equally well have written them separately:
1597
1598@example
1599exp: NUM ;
1600exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1601exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1602 @dots{}
1603@end example
1604
1605Most of the rules have actions that compute the value of the expression in
1606terms of the value of its parts. For example, in the rule for addition,
1607@code{$1} refers to the first component @code{exp} and @code{$2} refers to
1608the second one. The third component, @code{'+'}, has no meaningful
1609associated semantic value, but if it had one you could refer to it as
1610@code{$3}. When @code{yyparse} recognizes a sum expression using this
1611rule, the sum of the two subexpressions' values is produced as the value of
1612the entire expression. @xref{Actions}.
1613
1614You don't have to give an action for every rule. When a rule has no
1615action, Bison by default copies the value of @code{$1} into @code{$$}.
1616This is what happens in the first rule (the one that uses @code{NUM}).
1617
1618The formatting shown here is the recommended convention, but Bison does
1619not require it. You can add or change white space as much as you wish.
1620For example, this:
1621
1622@example
1623exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
1624@end example
1625
1626@noindent
1627means the same thing as this:
1628
1629@example
1630exp: NUM
1631 | exp exp '+' @{ $$ = $1 + $2; @}
1632 | @dots{}
1633;
1634@end example
1635
1636@noindent
1637The latter, however, is much more readable.
1638
1639@node Rpcalc Lexer
1640@subsection The @code{rpcalc} Lexical Analyzer
1641@cindex writing a lexical analyzer
1642@cindex lexical analyzer, writing
1643
1644The lexical analyzer's job is low-level parsing: converting characters
1645or sequences of characters into tokens. The Bison parser gets its
1646tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1647Analyzer Function @code{yylex}}.
1648
1649Only a simple lexical analyzer is needed for the @acronym{RPN}
1650calculator. This
1651lexical analyzer skips blanks and tabs, then reads in numbers as
1652@code{double} and returns them as @code{NUM} tokens. Any other character
1653that isn't part of a number is a separate token. Note that the token-code
1654for such a single-character token is the character itself.
1655
1656The return value of the lexical analyzer function is a numeric code which
1657represents a token type. The same text used in Bison rules to stand for
1658this token type is also a C expression for the numeric code for the type.
1659This works in two ways. If the token type is a character literal, then its
1660numeric code is that of the character; you can use the same
1661character literal in the lexical analyzer to express the number. If the
1662token type is an identifier, that identifier is defined by Bison as a C
1663macro whose definition is the appropriate number. In this example,
1664therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1665
1666The semantic value of the token (if it has one) is stored into the
1667global variable @code{yylval}, which is where the Bison parser will look
1668for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1669defined at the beginning of the grammar; @pxref{Rpcalc Declarations,
1670,Declarations for @code{rpcalc}}.)
1671
1672A token type code of zero is returned if the end-of-input is encountered.
1673(Bison recognizes any nonpositive value as indicating end-of-input.)
1674
1675Here is the code for the lexical analyzer:
1676
1677@example
1678@group
1679/* The lexical analyzer returns a double floating point
1680 number on the stack and the token NUM, or the numeric code
1681 of the character read if not a number. It skips all blanks
1682 and tabs, and returns 0 for end-of-input. */
1683
1684#include <ctype.h>
1685@end group
1686
1687@group
1688int
1689yylex (void)
1690@{
1691 int c;
1692
1693 /* Skip white space. */
1694 while ((c = getchar ()) == ' ' || c == '\t')
1695 ;
1696@end group
1697@group
1698 /* Process numbers. */
1699 if (c == '.' || isdigit (c))
1700 @{
1701 ungetc (c, stdin);
1702 scanf ("%lf", &yylval);
1703 return NUM;
1704 @}
1705@end group
1706@group
1707 /* Return end-of-input. */
1708 if (c == EOF)
1709 return 0;
1710 /* Return a single char. */
1711 return c;
1712@}
1713@end group
1714@end example
1715
1716@node Rpcalc Main
1717@subsection The Controlling Function
1718@cindex controlling function
1719@cindex main function in simple example
1720
1721In keeping with the spirit of this example, the controlling function is
1722kept to the bare minimum. The only requirement is that it call
1723@code{yyparse} to start the process of parsing.
1724
1725@example
1726@group
1727int
1728main (void)
1729@{
1730 return yyparse ();
1731@}
1732@end group
1733@end example
1734
1735@node Rpcalc Error
1736@subsection The Error Reporting Routine
1737@cindex error reporting routine
1738
1739When @code{yyparse} detects a syntax error, it calls the error reporting
1740function @code{yyerror} to print an error message (usually but not
1741always @code{"syntax error"}). It is up to the programmer to supply
1742@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1743here is the definition we will use:
1744
1745@example
1746@group
1747#include <stdio.h>
1748
1749/* Called by yyparse on error. */
1750void
1751yyerror (char const *s)
1752@{
1753 fprintf (stderr, "%s\n", s);
1754@}
1755@end group
1756@end example
1757
1758After @code{yyerror} returns, the Bison parser may recover from the error
1759and continue parsing if the grammar contains a suitable error rule
1760(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1761have not written any error rules in this example, so any invalid input will
1762cause the calculator program to exit. This is not clean behavior for a
1763real calculator, but it is adequate for the first example.
1764
1765@node Rpcalc Generate
1766@subsection Running Bison to Make the Parser
1767@cindex running Bison (introduction)
1768
1769Before running Bison to produce a parser, we need to decide how to
1770arrange all the source code in one or more source files. For such a
1771simple example, the easiest thing is to put everything in one file. The
1772definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
1773end, in the epilogue of the file
1774(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
1775
1776For a large project, you would probably have several source files, and use
1777@code{make} to arrange to recompile them.
1778
1779With all the source in a single file, you use the following command to
1780convert it into a parser file:
1781
1782@example
1783bison @var{file}.y
1784@end example
1785
1786@noindent
1787In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
1788@sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c},
1789removing the @samp{.y} from the original file name. The file output by
1790Bison contains the source code for @code{yyparse}. The additional
1791functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
1792are copied verbatim to the output.
1793
1794@node Rpcalc Compile
1795@subsection Compiling the Parser File
1796@cindex compiling the parser
1797
1798Here is how to compile and run the parser file:
1799
1800@example
1801@group
1802# @r{List files in current directory.}
1803$ @kbd{ls}
1804rpcalc.tab.c rpcalc.y
1805@end group
1806
1807@group
1808# @r{Compile the Bison parser.}
1809# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
1810$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
1811@end group
1812
1813@group
1814# @r{List files again.}
1815$ @kbd{ls}
1816rpcalc rpcalc.tab.c rpcalc.y
1817@end group
1818@end example
1819
1820The file @file{rpcalc} now contains the executable code. Here is an
1821example session using @code{rpcalc}.
1822
1823@example
1824$ @kbd{rpcalc}
1825@kbd{4 9 +}
182613
1827@kbd{3 7 + 3 4 5 *+-}
1828-13
1829@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
183013
1831@kbd{5 6 / 4 n +}
1832-3.166666667
1833@kbd{3 4 ^} @r{Exponentiation}
183481
1835@kbd{^D} @r{End-of-file indicator}
1836$
1837@end example
1838
1839@node Infix Calc
1840@section Infix Notation Calculator: @code{calc}
1841@cindex infix notation calculator
1842@cindex @code{calc}
1843@cindex calculator, infix notation
1844
1845We now modify rpcalc to handle infix operators instead of postfix. Infix
1846notation involves the concept of operator precedence and the need for
1847parentheses nested to arbitrary depth. Here is the Bison code for
1848@file{calc.y}, an infix desk-top calculator.
1849
1850@example
1851/* Infix notation calculator. */
1852
1853%@{
1854 #define YYSTYPE double
1855 #include <math.h>
1856 #include <stdio.h>
1857 int yylex (void);
1858 void yyerror (char const *);
1859%@}
1860
1861/* Bison declarations. */
1862%token NUM
1863%left '-' '+'
1864%left '*' '/'
1865%left NEG /* negation--unary minus */
1866%right '^' /* exponentiation */
1867
1868%% /* The grammar follows. */
1869input: /* empty */
1870 | input line
1871;
1872
1873line: '\n'
1874 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1875;
1876
1877exp: NUM @{ $$ = $1; @}
1878 | exp '+' exp @{ $$ = $1 + $3; @}
1879 | exp '-' exp @{ $$ = $1 - $3; @}
1880 | exp '*' exp @{ $$ = $1 * $3; @}
1881 | exp '/' exp @{ $$ = $1 / $3; @}
1882 | '-' exp %prec NEG @{ $$ = -$2; @}
1883 | exp '^' exp @{ $$ = pow ($1, $3); @}
1884 | '(' exp ')' @{ $$ = $2; @}
1885;
1886%%
1887@end example
1888
1889@noindent
1890The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1891same as before.
1892
1893There are two important new features shown in this code.
1894
1895In the second section (Bison declarations), @code{%left} declares token
1896types and says they are left-associative operators. The declarations
1897@code{%left} and @code{%right} (right associativity) take the place of
1898@code{%token} which is used to declare a token type name without
1899associativity. (These tokens are single-character literals, which
1900ordinarily don't need to be declared. We declare them here to specify
1901the associativity.)
1902
1903Operator precedence is determined by the line ordering of the
1904declarations; the higher the line number of the declaration (lower on
1905the page or screen), the higher the precedence. Hence, exponentiation
1906has the highest precedence, unary minus (@code{NEG}) is next, followed
1907by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator
1908Precedence}.
1909
1910The other important new feature is the @code{%prec} in the grammar
1911section for the unary minus operator. The @code{%prec} simply instructs
1912Bison that the rule @samp{| '-' exp} has the same precedence as
1913@code{NEG}---in this case the next-to-highest. @xref{Contextual
1914Precedence, ,Context-Dependent Precedence}.
1915
1916Here is a sample run of @file{calc.y}:
1917
1918@need 500
1919@example
1920$ @kbd{calc}
1921@kbd{4 + 4.5 - (34/(8*3+-3))}
19226.880952381
1923@kbd{-56 + 2}
1924-54
1925@kbd{3 ^ 2}
19269
1927@end example
1928
1929@node Simple Error Recovery
1930@section Simple Error Recovery
1931@cindex error recovery, simple
1932
1933Up to this point, this manual has not addressed the issue of @dfn{error
1934recovery}---how to continue parsing after the parser detects a syntax
1935error. All we have handled is error reporting with @code{yyerror}.
1936Recall that by default @code{yyparse} returns after calling
1937@code{yyerror}. This means that an erroneous input line causes the
1938calculator program to exit. Now we show how to rectify this deficiency.
1939
1940The Bison language itself includes the reserved word @code{error}, which
1941may be included in the grammar rules. In the example below it has
1942been added to one of the alternatives for @code{line}:
1943
1944@example
1945@group
1946line: '\n'
1947 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1948 | error '\n' @{ yyerrok; @}
1949;
1950@end group
1951@end example
1952
1953This addition to the grammar allows for simple error recovery in the
1954event of a syntax error. If an expression that cannot be evaluated is
1955read, the error will be recognized by the third rule for @code{line},
1956and parsing will continue. (The @code{yyerror} function is still called
1957upon to print its message as well.) The action executes the statement
1958@code{yyerrok}, a macro defined automatically by Bison; its meaning is
1959that error recovery is complete (@pxref{Error Recovery}). Note the
1960difference between @code{yyerrok} and @code{yyerror}; neither one is a
1961misprint.
1962
1963This form of error recovery deals with syntax errors. There are other
1964kinds of errors; for example, division by zero, which raises an exception
1965signal that is normally fatal. A real calculator program must handle this
1966signal and use @code{longjmp} to return to @code{main} and resume parsing
1967input lines; it would also have to discard the rest of the current line of
1968input. We won't discuss this issue further because it is not specific to
1969Bison programs.
1970
1971@node Location Tracking Calc
1972@section Location Tracking Calculator: @code{ltcalc}
1973@cindex location tracking calculator
1974@cindex @code{ltcalc}
1975@cindex calculator, location tracking
1976
1977This example extends the infix notation calculator with location
1978tracking. This feature will be used to improve the error messages. For
1979the sake of clarity, this example is a simple integer calculator, since
1980most of the work needed to use locations will be done in the lexical
1981analyzer.
1982
1983@menu
1984* Ltcalc Declarations:: Bison and C declarations for ltcalc.
1985* Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
1986* Ltcalc Lexer:: The lexical analyzer.
1987@end menu
1988
1989@node Ltcalc Declarations
1990@subsection Declarations for @code{ltcalc}
1991
1992The C and Bison declarations for the location tracking calculator are
1993the same as the declarations for the infix notation calculator.
1994
1995@example
1996/* Location tracking calculator. */
1997
1998%@{
1999 #define YYSTYPE int
2000 #include <math.h>
2001 int yylex (void);
2002 void yyerror (char const *);
2003%@}
2004
2005/* Bison declarations. */
2006%token NUM
2007
2008%left '-' '+'
2009%left '*' '/'
2010%left NEG
2011%right '^'
2012
2013%% /* The grammar follows. */
2014@end example
2015
2016@noindent
2017Note there are no declarations specific to locations. Defining a data
2018type for storing locations is not needed: we will use the type provided
2019by default (@pxref{Location Type, ,Data Types of Locations}), which is a
2020four member structure with the following integer fields:
2021@code{first_line}, @code{first_column}, @code{last_line} and
2022@code{last_column}. By conventions, and in accordance with the GNU
2023Coding Standards and common practice, the line and column count both
2024start at 1.
2025
2026@node Ltcalc Rules
2027@subsection Grammar Rules for @code{ltcalc}
2028
2029Whether handling locations or not has no effect on the syntax of your
2030language. Therefore, grammar rules for this example will be very close
2031to those of the previous example: we will only modify them to benefit
2032from the new information.
2033
2034Here, we will use locations to report divisions by zero, and locate the
2035wrong expressions or subexpressions.
2036
2037@example
2038@group
2039input : /* empty */
2040 | input line
2041;
2042@end group
2043
2044@group
2045line : '\n'
2046 | exp '\n' @{ printf ("%d\n", $1); @}
2047;
2048@end group
2049
2050@group
2051exp : NUM @{ $$ = $1; @}
2052 | exp '+' exp @{ $$ = $1 + $3; @}
2053 | exp '-' exp @{ $$ = $1 - $3; @}
2054 | exp '*' exp @{ $$ = $1 * $3; @}
2055@end group
2056@group
2057 | exp '/' exp
2058 @{
2059 if ($3)
2060 $$ = $1 / $3;
2061 else
2062 @{
2063 $$ = 1;
2064 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2065 @@3.first_line, @@3.first_column,
2066 @@3.last_line, @@3.last_column);
2067 @}
2068 @}
2069@end group
2070@group
2071 | '-' exp %prec NEG @{ $$ = -$2; @}
2072 | exp '^' exp @{ $$ = pow ($1, $3); @}
2073 | '(' exp ')' @{ $$ = $2; @}
2074@end group
2075@end example
2076
2077This code shows how to reach locations inside of semantic actions, by
2078using the pseudo-variables @code{@@@var{n}} for rule components, and the
2079pseudo-variable @code{@@$} for groupings.
2080
2081We don't need to assign a value to @code{@@$}: the output parser does it
2082automatically. By default, before executing the C code of each action,
2083@code{@@$} is set to range from the beginning of @code{@@1} to the end
2084of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2085can be redefined (@pxref{Location Default Action, , Default Action for
2086Locations}), and for very specific rules, @code{@@$} can be computed by
2087hand.
2088
2089@node Ltcalc Lexer
2090@subsection The @code{ltcalc} Lexical Analyzer.
2091
2092Until now, we relied on Bison's defaults to enable location
2093tracking. The next step is to rewrite the lexical analyzer, and make it
2094able to feed the parser with the token locations, as it already does for
2095semantic values.
2096
2097To this end, we must take into account every single character of the
2098input text, to avoid the computed locations of being fuzzy or wrong:
2099
2100@example
2101@group
2102int
2103yylex (void)
2104@{
2105 int c;
2106@end group
2107
2108@group
2109 /* Skip white space. */
2110 while ((c = getchar ()) == ' ' || c == '\t')
2111 ++yylloc.last_column;
2112@end group
2113
2114@group
2115 /* Step. */
2116 yylloc.first_line = yylloc.last_line;
2117 yylloc.first_column = yylloc.last_column;
2118@end group
2119
2120@group
2121 /* Process numbers. */
2122 if (isdigit (c))
2123 @{
2124 yylval = c - '0';
2125 ++yylloc.last_column;
2126 while (isdigit (c = getchar ()))
2127 @{
2128 ++yylloc.last_column;
2129 yylval = yylval * 10 + c - '0';
2130 @}
2131 ungetc (c, stdin);
2132 return NUM;
2133 @}
2134@end group
2135
2136 /* Return end-of-input. */
2137 if (c == EOF)
2138 return 0;
2139
2140 /* Return a single char, and update location. */
2141 if (c == '\n')
2142 @{
2143 ++yylloc.last_line;
2144 yylloc.last_column = 0;
2145 @}
2146 else
2147 ++yylloc.last_column;
2148 return c;
2149@}
2150@end example
2151
2152Basically, the lexical analyzer performs the same processing as before:
2153it skips blanks and tabs, and reads numbers or single-character tokens.
2154In addition, it updates @code{yylloc}, the global variable (of type
2155@code{YYLTYPE}) containing the token's location.
2156
2157Now, each time this function returns a token, the parser has its number
2158as well as its semantic value, and its location in the text. The last
2159needed change is to initialize @code{yylloc}, for example in the
2160controlling function:
2161
2162@example
2163@group
2164int
2165main (void)
2166@{
2167 yylloc.first_line = yylloc.last_line = 1;
2168 yylloc.first_column = yylloc.last_column = 0;
2169 return yyparse ();
2170@}
2171@end group
2172@end example
2173
2174Remember that computing locations is not a matter of syntax. Every
2175character must be associated to a location update, whether it is in
2176valid input, in comments, in literal strings, and so on.
2177
2178@node Multi-function Calc
2179@section Multi-Function Calculator: @code{mfcalc}
2180@cindex multi-function calculator
2181@cindex @code{mfcalc}
2182@cindex calculator, multi-function
2183
2184Now that the basics of Bison have been discussed, it is time to move on to
2185a more advanced problem. The above calculators provided only five
2186functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2187be nice to have a calculator that provides other mathematical functions such
2188as @code{sin}, @code{cos}, etc.
2189
2190It is easy to add new operators to the infix calculator as long as they are
2191only single-character literals. The lexical analyzer @code{yylex} passes
2192back all nonnumeric characters as tokens, so new grammar rules suffice for
2193adding a new operator. But we want something more flexible: built-in
2194functions whose syntax has this form:
2195
2196@example
2197@var{function_name} (@var{argument})
2198@end example
2199
2200@noindent
2201At the same time, we will add memory to the calculator, by allowing you
2202to create named variables, store values in them, and use them later.
2203Here is a sample session with the multi-function calculator:
2204
2205@example
2206$ @kbd{mfcalc}
2207@kbd{pi = 3.141592653589}
22083.1415926536
2209@kbd{sin(pi)}
22100.0000000000
2211@kbd{alpha = beta1 = 2.3}
22122.3000000000
2213@kbd{alpha}
22142.3000000000
2215@kbd{ln(alpha)}
22160.8329091229
2217@kbd{exp(ln(beta1))}
22182.3000000000
2219$
2220@end example
2221
2222Note that multiple assignment and nested function calls are permitted.
2223
2224@menu
2225* Mfcalc Declarations:: Bison declarations for multi-function calculator.
2226* Mfcalc Rules:: Grammar rules for the calculator.
2227* Mfcalc Symbol Table:: Symbol table management subroutines.
2228@end menu
2229
2230@node Mfcalc Declarations
2231@subsection Declarations for @code{mfcalc}
2232
2233Here are the C and Bison declarations for the multi-function calculator.
2234
2235@smallexample
2236@group
2237%@{
2238 #include <math.h> /* For math functions, cos(), sin(), etc. */
2239 #include "calc.h" /* Contains definition of `symrec'. */
2240 int yylex (void);
2241 void yyerror (char const *);
2242%@}
2243@end group
2244@group
2245%union @{
2246 double val; /* For returning numbers. */
2247 symrec *tptr; /* For returning symbol-table pointers. */
2248@}
2249@end group
2250%token <val> NUM /* Simple double precision number. */
2251%token <tptr> VAR FNCT /* Variable and Function. */
2252%type <val> exp
2253
2254@group
2255%right '='
2256%left '-' '+'
2257%left '*' '/'
2258%left NEG /* negation--unary minus */
2259%right '^' /* exponentiation */
2260@end group
2261%% /* The grammar follows. */
2262@end smallexample
2263
2264The above grammar introduces only two new features of the Bison language.
2265These features allow semantic values to have various data types
2266(@pxref{Multiple Types, ,More Than One Value Type}).
2267
2268The @code{%union} declaration specifies the entire list of possible types;
2269this is instead of defining @code{YYSTYPE}. The allowable types are now
2270double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2271the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2272
2273Since values can now have various types, it is necessary to associate a
2274type with each grammar symbol whose semantic value is used. These symbols
2275are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2276declarations are augmented with information about their data type (placed
2277between angle brackets).
2278
2279The Bison construct @code{%type} is used for declaring nonterminal
2280symbols, just as @code{%token} is used for declaring token types. We
2281have not used @code{%type} before because nonterminal symbols are
2282normally declared implicitly by the rules that define them. But
2283@code{exp} must be declared explicitly so we can specify its value type.
2284@xref{Type Decl, ,Nonterminal Symbols}.
2285
2286@node Mfcalc Rules
2287@subsection Grammar Rules for @code{mfcalc}
2288
2289Here are the grammar rules for the multi-function calculator.
2290Most of them are copied directly from @code{calc}; three rules,
2291those which mention @code{VAR} or @code{FNCT}, are new.
2292
2293@smallexample
2294@group
2295input: /* empty */
2296 | input line
2297;
2298@end group
2299
2300@group
2301line:
2302 '\n'
2303 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2304 | error '\n' @{ yyerrok; @}
2305;
2306@end group
2307
2308@group
2309exp: NUM @{ $$ = $1; @}
2310 | VAR @{ $$ = $1->value.var; @}
2311 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2312 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2313 | exp '+' exp @{ $$ = $1 + $3; @}
2314 | exp '-' exp @{ $$ = $1 - $3; @}
2315 | exp '*' exp @{ $$ = $1 * $3; @}
2316 | exp '/' exp @{ $$ = $1 / $3; @}
2317 | '-' exp %prec NEG @{ $$ = -$2; @}
2318 | exp '^' exp @{ $$ = pow ($1, $3); @}
2319 | '(' exp ')' @{ $$ = $2; @}
2320;
2321@end group
2322/* End of grammar. */
2323%%
2324@end smallexample
2325
2326@node Mfcalc Symbol Table
2327@subsection The @code{mfcalc} Symbol Table
2328@cindex symbol table example
2329
2330The multi-function calculator requires a symbol table to keep track of the
2331names and meanings of variables and functions. This doesn't affect the
2332grammar rules (except for the actions) or the Bison declarations, but it
2333requires some additional C functions for support.
2334
2335The symbol table itself consists of a linked list of records. Its
2336definition, which is kept in the header @file{calc.h}, is as follows. It
2337provides for either functions or variables to be placed in the table.
2338
2339@smallexample
2340@group
2341/* Function type. */
2342typedef double (*func_t) (double);
2343@end group
2344
2345@group
2346/* Data type for links in the chain of symbols. */
2347struct symrec
2348@{
2349 char *name; /* name of symbol */
2350 int type; /* type of symbol: either VAR or FNCT */
2351 union
2352 @{
2353 double var; /* value of a VAR */
2354 func_t fnctptr; /* value of a FNCT */
2355 @} value;
2356 struct symrec *next; /* link field */
2357@};
2358@end group
2359
2360@group
2361typedef struct symrec symrec;
2362
2363/* The symbol table: a chain of `struct symrec'. */
2364extern symrec *sym_table;
2365
2366symrec *putsym (char const *, int);
2367symrec *getsym (char const *);
2368@end group
2369@end smallexample
2370
2371The new version of @code{main} includes a call to @code{init_table}, a
2372function that initializes the symbol table. Here it is, and
2373@code{init_table} as well:
2374
2375@smallexample
2376#include <stdio.h>
2377
2378@group
2379/* Called by yyparse on error. */
2380void
2381yyerror (char const *s)
2382@{
2383 printf ("%s\n", s);
2384@}
2385@end group
2386
2387@group
2388struct init
2389@{
2390 char const *fname;
2391 double (*fnct) (double);
2392@};
2393@end group
2394
2395@group
2396struct init const arith_fncts[] =
2397@{
2398 "sin", sin,
2399 "cos", cos,
2400 "atan", atan,
2401 "ln", log,
2402 "exp", exp,
2403 "sqrt", sqrt,
2404 0, 0
2405@};
2406@end group
2407
2408@group
2409/* The symbol table: a chain of `struct symrec'. */
2410symrec *sym_table;
2411@end group
2412
2413@group
2414/* Put arithmetic functions in table. */
2415void
2416init_table (void)
2417@{
2418 int i;
2419 symrec *ptr;
2420 for (i = 0; arith_fncts[i].fname != 0; i++)
2421 @{
2422 ptr = putsym (arith_fncts[i].fname, FNCT);
2423 ptr->value.fnctptr = arith_fncts[i].fnct;
2424 @}
2425@}
2426@end group
2427
2428@group
2429int
2430main (void)
2431@{
2432 init_table ();
2433 return yyparse ();
2434@}
2435@end group
2436@end smallexample
2437
2438By simply editing the initialization list and adding the necessary include
2439files, you can add additional functions to the calculator.
2440
2441Two important functions allow look-up and installation of symbols in the
2442symbol table. The function @code{putsym} is passed a name and the type
2443(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2444linked to the front of the list, and a pointer to the object is returned.
2445The function @code{getsym} is passed the name of the symbol to look up. If
2446found, a pointer to that symbol is returned; otherwise zero is returned.
2447
2448@smallexample
2449symrec *
2450putsym (char const *sym_name, int sym_type)
2451@{
2452 symrec *ptr;
2453 ptr = (symrec *) malloc (sizeof (symrec));
2454 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2455 strcpy (ptr->name,sym_name);
2456 ptr->type = sym_type;
2457 ptr->value.var = 0; /* Set value to 0 even if fctn. */
2458 ptr->next = (struct symrec *)sym_table;
2459 sym_table = ptr;
2460 return ptr;
2461@}
2462
2463symrec *
2464getsym (char const *sym_name)
2465@{
2466 symrec *ptr;
2467 for (ptr = sym_table; ptr != (symrec *) 0;
2468 ptr = (symrec *)ptr->next)
2469 if (strcmp (ptr->name,sym_name) == 0)
2470 return ptr;
2471 return 0;
2472@}
2473@end smallexample
2474
2475The function @code{yylex} must now recognize variables, numeric values, and
2476the single-character arithmetic operators. Strings of alphanumeric
2477characters with a leading letter are recognized as either variables or
2478functions depending on what the symbol table says about them.
2479
2480The string is passed to @code{getsym} for look up in the symbol table. If
2481the name appears in the table, a pointer to its location and its type
2482(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2483already in the table, then it is installed as a @code{VAR} using
2484@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
2485returned to @code{yyparse}.
2486
2487No change is needed in the handling of numeric values and arithmetic
2488operators in @code{yylex}.
2489
2490@smallexample
2491@group
2492#include <ctype.h>
2493@end group
2494
2495@group
2496int
2497yylex (void)
2498@{
2499 int c;
2500
2501 /* Ignore white space, get first nonwhite character. */
2502 while ((c = getchar ()) == ' ' || c == '\t');
2503
2504 if (c == EOF)
2505 return 0;
2506@end group
2507
2508@group
2509 /* Char starts a number => parse the number. */
2510 if (c == '.' || isdigit (c))
2511 @{
2512 ungetc (c, stdin);
2513 scanf ("%lf", &yylval.val);
2514 return NUM;
2515 @}
2516@end group
2517
2518@group
2519 /* Char starts an identifier => read the name. */
2520 if (isalpha (c))
2521 @{
2522 symrec *s;
2523 static char *symbuf = 0;
2524 static int length = 0;
2525 int i;
2526@end group
2527
2528@group
2529 /* Initially make the buffer long enough
2530 for a 40-character symbol name. */
2531 if (length == 0)
2532 length = 40, symbuf = (char *)malloc (length + 1);
2533
2534 i = 0;
2535 do
2536@end group
2537@group
2538 @{
2539 /* If buffer is full, make it bigger. */
2540 if (i == length)
2541 @{
2542 length *= 2;
2543 symbuf = (char *) realloc (symbuf, length + 1);
2544 @}
2545 /* Add this character to the buffer. */
2546 symbuf[i++] = c;
2547 /* Get another character. */
2548 c = getchar ();
2549 @}
2550@end group
2551@group
2552 while (isalnum (c));
2553
2554 ungetc (c, stdin);
2555 symbuf[i] = '\0';
2556@end group
2557
2558@group
2559 s = getsym (symbuf);
2560 if (s == 0)
2561 s = putsym (symbuf, VAR);
2562 yylval.tptr = s;
2563 return s->type;
2564 @}
2565
2566 /* Any other character is a token by itself. */
2567 return c;
2568@}
2569@end group
2570@end smallexample
2571
2572This program is both powerful and flexible. You may easily add new
2573functions, and it is a simple job to modify this code to install
2574predefined variables such as @code{pi} or @code{e} as well.
2575
2576@node Exercises
2577@section Exercises
2578@cindex exercises
2579
2580@enumerate
2581@item
2582Add some new functions from @file{math.h} to the initialization list.
2583
2584@item
2585Add another array that contains constants and their values. Then
2586modify @code{init_table} to add these constants to the symbol table.
2587It will be easiest to give the constants type @code{VAR}.
2588
2589@item
2590Make the program report an error if the user refers to an
2591uninitialized variable in any way except to store a value in it.
2592@end enumerate
2593
2594@node Grammar File
2595@chapter Bison Grammar Files
2596
2597Bison takes as input a context-free grammar specification and produces a
2598C-language function that recognizes correct instances of the grammar.
2599
2600The Bison grammar input file conventionally has a name ending in @samp{.y}.
2601@xref{Invocation, ,Invoking Bison}.
2602
2603@menu
2604* Grammar Outline:: Overall layout of the grammar file.
2605* Symbols:: Terminal and nonterminal symbols.
2606* Rules:: How to write grammar rules.
2607* Recursion:: Writing recursive rules.
2608* Semantics:: Semantic values and actions.
2609* Locations:: Locations and actions.
2610* Declarations:: All kinds of Bison declarations are described here.
2611* Multiple Parsers:: Putting more than one Bison parser in one program.
2612@end menu
2613
2614@node Grammar Outline
2615@section Outline of a Bison Grammar
2616
2617A Bison grammar file has four main sections, shown here with the
2618appropriate delimiters:
2619
2620@example
2621%@{
2622 @var{Prologue}
2623%@}
2624
2625@var{Bison declarations}
2626
2627%%
2628@var{Grammar rules}
2629%%
2630
2631@var{Epilogue}
2632@end example
2633
2634Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2635As a @acronym{GNU} extension, @samp{//} introduces a comment that
2636continues until end of line.
2637
2638@menu
2639* Prologue:: Syntax and usage of the prologue.
2640* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
2641* Bison Declarations:: Syntax and usage of the Bison declarations section.
2642* Grammar Rules:: Syntax and usage of the grammar rules section.
2643* Epilogue:: Syntax and usage of the epilogue.
2644@end menu
2645
2646@node Prologue
2647@subsection The prologue
2648@cindex declarations section
2649@cindex Prologue
2650@cindex declarations
2651
2652The @var{Prologue} section contains macro definitions and declarations
2653of functions and variables that are used in the actions in the grammar
2654rules. These are copied to the beginning of the parser file so that
2655they precede the definition of @code{yyparse}. You can use
2656@samp{#include} to get the declarations from a header file. If you
2657don't need any C declarations, you may omit the @samp{%@{} and
2658@samp{%@}} delimiters that bracket this section.
2659
2660The @var{Prologue} section is terminated by the first occurrence
2661of @samp{%@}} that is outside a comment, a string literal, or a
2662character constant.
2663
2664You may have more than one @var{Prologue} section, intermixed with the
2665@var{Bison declarations}. This allows you to have C and Bison
2666declarations that refer to each other. For example, the @code{%union}
2667declaration may use types defined in a header file, and you may wish to
2668prototype functions that take arguments of type @code{YYSTYPE}. This
2669can be done with two @var{Prologue} blocks, one before and one after the
2670@code{%union} declaration.
2671
2672@smallexample
2673%@{
2674 #define _GNU_SOURCE
2675 #include <stdio.h>
2676 #include "ptypes.h"
2677%@}
2678
2679%union @{
2680 long int n;
2681 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2682@}
2683
2684%@{
2685 static void print_token_value (FILE *, int, YYSTYPE);
2686 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2687%@}
2688
2689@dots{}
2690@end smallexample
2691
2692When in doubt, it is usually safer to put prologue code before all
2693Bison declarations, rather than after. For example, any definitions
2694of feature test macros like @code{_GNU_SOURCE} or
2695@code{_POSIX_C_SOURCE} should appear before all Bison declarations, as
2696feature test macros can affect the behavior of Bison-generated
2697@code{#include} directives.
2698
2699@node Prologue Alternatives
2700@subsection Prologue Alternatives
2701@cindex Prologue Alternatives
2702
2703@findex %code
2704@findex %code requires
2705@findex %code provides
2706@findex %code top
2707
2708The functionality of @var{Prologue} sections can often be subtle and
2709inflexible.
2710As an alternative, Bison provides a %code directive with an explicit qualifier
2711field, which identifies the purpose of the code and thus the location(s) where
2712Bison should generate it.
2713For C/C++, the qualifier can be omitted for the default location, or it can be
2714one of @code{requires}, @code{provides}, @code{top}.
2715@xref{Decl Summary,,%code}.
2716
2717Look again at the example of the previous section:
2718
2719@smallexample
2720%@{
2721 #define _GNU_SOURCE
2722 #include <stdio.h>
2723 #include "ptypes.h"
2724%@}
2725
2726%union @{
2727 long int n;
2728 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2729@}
2730
2731%@{
2732 static void print_token_value (FILE *, int, YYSTYPE);
2733 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2734%@}
2735
2736@dots{}
2737@end smallexample
2738
2739@noindent
2740Notice that there are two @var{Prologue} sections here, but there's a subtle
2741distinction between their functionality.
2742For example, if you decide to override Bison's default definition for
2743@code{YYLTYPE}, in which @var{Prologue} section should you write your new
2744definition?
2745You should write it in the first since Bison will insert that code into the
2746parser source code file @emph{before} the default @code{YYLTYPE} definition.
2747In which @var{Prologue} section should you prototype an internal function,
2748@code{trace_token}, that accepts @code{YYLTYPE} and @code{yytokentype} as
2749arguments?
2750You should prototype it in the second since Bison will insert that code
2751@emph{after} the @code{YYLTYPE} and @code{yytokentype} definitions.
2752
2753This distinction in functionality between the two @var{Prologue} sections is
2754established by the appearance of the @code{%union} between them.
2755This behavior raises a few questions.
2756First, why should the position of a @code{%union} affect definitions related to
2757@code{YYLTYPE} and @code{yytokentype}?
2758Second, what if there is no @code{%union}?
2759In that case, the second kind of @var{Prologue} section is not available.
2760This behavior is not intuitive.
2761
2762To avoid this subtle @code{%union} dependency, rewrite the example using a
2763@code{%code top} and an unqualified @code{%code}.
2764Let's go ahead and add the new @code{YYLTYPE} definition and the
2765@code{trace_token} prototype at the same time:
2766
2767@smallexample
2768%code top @{
2769 #define _GNU_SOURCE
2770 #include <stdio.h>
2771
2772 /* WARNING: The following code really belongs
2773 * in a `%code requires'; see below. */
2774
2775 #include "ptypes.h"
2776 #define YYLTYPE YYLTYPE
2777 typedef struct YYLTYPE
2778 @{
2779 int first_line;
2780 int first_column;
2781 int last_line;
2782 int last_column;
2783 char *filename;
2784 @} YYLTYPE;
2785@}
2786
2787%union @{
2788 long int n;
2789 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2790@}
2791
2792%code @{
2793 static void print_token_value (FILE *, int, YYSTYPE);
2794 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2795 static void trace_token (enum yytokentype token, YYLTYPE loc);
2796@}
2797
2798@dots{}
2799@end smallexample
2800
2801@noindent
2802In this way, @code{%code top} and the unqualified @code{%code} achieve the same
2803functionality as the two kinds of @var{Prologue} sections, but it's always
2804explicit which kind you intend.
2805Moreover, both kinds are always available even in the absence of @code{%union}.
2806
2807The @code{%code top} block above logically contains two parts.
2808The first two lines before the warning need to appear near the top of the
2809parser source code file.
2810The first line after the warning is required by @code{YYSTYPE} and thus also
2811needs to appear in the parser source code file.
2812However, if you've instructed Bison to generate a parser header file
2813(@pxref{Decl Summary, ,%defines}), you probably want that line to appear before
2814the @code{YYSTYPE} definition in that header file as well.
2815The @code{YYLTYPE} definition should also appear in the parser header file to
2816override the default @code{YYLTYPE} definition there.
2817
2818In other words, in the @code{%code top} block above, all but the first two
2819lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
2820definitions.
2821Thus, they belong in one or more @code{%code requires}:
2822
2823@smallexample
2824%code top @{
2825 #define _GNU_SOURCE
2826 #include <stdio.h>
2827@}
2828
2829%code requires @{
2830 #include "ptypes.h"
2831@}
2832%union @{
2833 long int n;
2834 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2835@}
2836
2837%code requires @{
2838 #define YYLTYPE YYLTYPE
2839 typedef struct YYLTYPE
2840 @{
2841 int first_line;
2842 int first_column;
2843 int last_line;
2844 int last_column;
2845 char *filename;
2846 @} YYLTYPE;
2847@}
2848
2849%code @{
2850 static void print_token_value (FILE *, int, YYSTYPE);
2851 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2852 static void trace_token (enum yytokentype token, YYLTYPE loc);
2853@}
2854
2855@dots{}
2856@end smallexample
2857
2858@noindent
2859Now Bison will insert @code{#include "ptypes.h"} and the new @code{YYLTYPE}
2860definition before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
2861definitions in both the parser source code file and the parser header file.
2862(By the same reasoning, @code{%code requires} would also be the appropriate
2863place to write your own definition for @code{YYSTYPE}.)
2864
2865When you are writing dependency code for @code{YYSTYPE} and @code{YYLTYPE}, you
2866should prefer @code{%code requires} over @code{%code top} regardless of whether
2867you instruct Bison to generate a parser header file.
2868When you are writing code that you need Bison to insert only into the parser
2869source code file and that has no special need to appear at the top of that
2870file, you should prefer the unqualified @code{%code} over @code{%code top}.
2871These practices will make the purpose of each block of your code explicit to
2872Bison and to other developers reading your grammar file.
2873Following these practices, we expect the unqualified @code{%code} and
2874@code{%code requires} to be the most important of the four @var{Prologue}
2875alternatives.
2876
2877At some point while developing your parser, you might decide to provide
2878@code{trace_token} to modules that are external to your parser.
2879Thus, you might wish for Bison to insert the prototype into both the parser
2880header file and the parser source code file.
2881Since this function is not a dependency required by @code{YYSTYPE} or
2882@code{YYLTYPE}, it doesn't make sense to move its prototype to a
2883@code{%code requires}.
2884More importantly, since it depends upon @code{YYLTYPE} and @code{yytokentype},
2885@code{%code requires} is not sufficient.
2886Instead, move its prototype from the unqualified @code{%code} to a
2887@code{%code provides}:
2888
2889@smallexample
2890%code top @{
2891 #define _GNU_SOURCE
2892 #include <stdio.h>
2893@}
2894
2895%code requires @{
2896 #include "ptypes.h"
2897@}
2898%union @{
2899 long int n;
2900 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2901@}
2902
2903%code requires @{
2904 #define YYLTYPE YYLTYPE
2905 typedef struct YYLTYPE
2906 @{
2907 int first_line;
2908 int first_column;
2909 int last_line;
2910 int last_column;
2911 char *filename;
2912 @} YYLTYPE;
2913@}
2914
2915%code provides @{
2916 void trace_token (enum yytokentype token, YYLTYPE loc);
2917@}
2918
2919%code @{
2920 static void print_token_value (FILE *, int, YYSTYPE);
2921 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2922@}
2923
2924@dots{}
2925@end smallexample
2926
2927@noindent
2928Bison will insert the @code{trace_token} prototype into both the parser header
2929file and the parser source code file after the definitions for
2930@code{yytokentype}, @code{YYLTYPE}, and @code{YYSTYPE}.
2931
2932The above examples are careful to write directives in an order that reflects
2933the layout of the generated parser source code and header files:
2934@code{%code top}, @code{%code requires}, @code{%code provides}, and then
2935@code{%code}.
2936While your grammar files may generally be easier to read if you also follow
2937this order, Bison does not require it.
2938Instead, Bison lets you choose an organization that makes sense to you.
2939
2940You may declare any of these directives multiple times in the grammar file.
2941In that case, Bison concatenates the contained code in declaration order.
2942This is the only way in which the position of one of these directives within
2943the grammar file affects its functionality.
2944
2945The result of the previous two properties is greater flexibility in how you may
2946organize your grammar file.
2947For example, you may organize semantic-type-related directives by semantic
2948type:
2949
2950@smallexample
2951%code requires @{ #include "type1.h" @}
2952%union @{ type1 field1; @}
2953%destructor @{ type1_free ($$); @} <field1>
2954%printer @{ type1_print ($$); @} <field1>
2955
2956%code requires @{ #include "type2.h" @}
2957%union @{ type2 field2; @}
2958%destructor @{ type2_free ($$); @} <field2>
2959%printer @{ type2_print ($$); @} <field2>
2960@end smallexample
2961
2962@noindent
2963You could even place each of the above directive groups in the rules section of
2964the grammar file next to the set of rules that uses the associated semantic
2965type.
2966(In the rules section, you must terminate each of those directives with a
2967semicolon.)
2968And you don't have to worry that some directive (like a @code{%union}) in the
2969definitions section is going to adversely affect their functionality in some
2970counter-intuitive manner just because it comes first.
2971Such an organization is not possible using @var{Prologue} sections.
2972
2973This section has been concerned with explaining the advantages of the four
2974@var{Prologue} alternatives over the original Yacc @var{Prologue}.
2975However, in most cases when using these directives, you shouldn't need to
2976think about all the low-level ordering issues discussed here.
2977Instead, you should simply use these directives to label each block of your
2978code according to its purpose and let Bison handle the ordering.
2979@code{%code} is the most generic label.
2980Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
2981as needed.
2982
2983@node Bison Declarations
2984@subsection The Bison Declarations Section
2985@cindex Bison declarations (introduction)
2986@cindex declarations, Bison (introduction)
2987
2988The @var{Bison declarations} section contains declarations that define
2989terminal and nonterminal symbols, specify precedence, and so on.
2990In some simple grammars you may not need any declarations.
2991@xref{Declarations, ,Bison Declarations}.
2992
2993@node Grammar Rules
2994@subsection The Grammar Rules Section
2995@cindex grammar rules section
2996@cindex rules section for grammar
2997
2998The @dfn{grammar rules} section contains one or more Bison grammar
2999rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
3000
3001There must always be at least one grammar rule, and the first
3002@samp{%%} (which precedes the grammar rules) may never be omitted even
3003if it is the first thing in the file.
3004
3005@node Epilogue
3006@subsection The epilogue
3007@cindex additional C code section
3008@cindex epilogue
3009@cindex C code, section for additional
3010
3011The @var{Epilogue} is copied verbatim to the end of the parser file, just as
3012the @var{Prologue} is copied to the beginning. This is the most convenient
3013place to put anything that you want to have in the parser file but which need
3014not come before the definition of @code{yyparse}. For example, the
3015definitions of @code{yylex} and @code{yyerror} often go here. Because
3016C requires functions to be declared before being used, you often need
3017to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
3018even if you define them in the Epilogue.
3019@xref{Interface, ,Parser C-Language Interface}.
3020
3021If the last section is empty, you may omit the @samp{%%} that separates it
3022from the grammar rules.
3023
3024The Bison parser itself contains many macros and identifiers whose names
3025start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3026any such names (except those documented in this manual) in the epilogue
3027of the grammar file.
3028
3029@node Symbols
3030@section Symbols, Terminal and Nonterminal
3031@cindex nonterminal symbol
3032@cindex terminal symbol
3033@cindex token type
3034@cindex symbol
3035
3036@dfn{Symbols} in Bison grammars represent the grammatical classifications
3037of the language.
3038
3039A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
3040class of syntactically equivalent tokens. You use the symbol in grammar
3041rules to mean that a token in that class is allowed. The symbol is
3042represented in the Bison parser by a numeric code, and the @code{yylex}
3043function returns a token type code to indicate what kind of token has
3044been read. You don't need to know what the code value is; you can use
3045the symbol to stand for it.
3046
3047A @dfn{nonterminal symbol} stands for a class of syntactically
3048equivalent groupings. The symbol name is used in writing grammar rules.
3049By convention, it should be all lower case.
3050
3051Symbol names can contain letters, digits (not at the beginning),
3052underscores and periods. Periods make sense only in nonterminals.
3053
3054There are three ways of writing terminal symbols in the grammar:
3055
3056@itemize @bullet
3057@item
3058A @dfn{named token type} is written with an identifier, like an
3059identifier in C@. By convention, it should be all upper case. Each
3060such name must be defined with a Bison declaration such as
3061@code{%token}. @xref{Token Decl, ,Token Type Names}.
3062
3063@item
3064@cindex character token
3065@cindex literal token
3066@cindex single-character literal
3067A @dfn{character token type} (or @dfn{literal character token}) is
3068written in the grammar using the same syntax used in C for character
3069constants; for example, @code{'+'} is a character token type. A
3070character token type doesn't need to be declared unless you need to
3071specify its semantic value data type (@pxref{Value Type, ,Data Types of
3072Semantic Values}), associativity, or precedence (@pxref{Precedence,
3073,Operator Precedence}).
3074
3075By convention, a character token type is used only to represent a
3076token that consists of that particular character. Thus, the token
3077type @code{'+'} is used to represent the character @samp{+} as a
3078token. Nothing enforces this convention, but if you depart from it,
3079your program will confuse other readers.
3080
3081All the usual escape sequences used in character literals in C can be
3082used in Bison as well, but you must not use the null character as a
3083character literal because its numeric code, zero, signifies
3084end-of-input (@pxref{Calling Convention, ,Calling Convention
3085for @code{yylex}}). Also, unlike standard C, trigraphs have no
3086special meaning in Bison character literals, nor is backslash-newline
3087allowed.
3088
3089@item
3090@cindex string token
3091@cindex literal string token
3092@cindex multicharacter literal
3093A @dfn{literal string token} is written like a C string constant; for
3094example, @code{"<="} is a literal string token. A literal string token
3095doesn't need to be declared unless you need to specify its semantic
3096value data type (@pxref{Value Type}), associativity, or precedence
3097(@pxref{Precedence}).
3098
3099You can associate the literal string token with a symbolic name as an
3100alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
3101Declarations}). If you don't do that, the lexical analyzer has to
3102retrieve the token number for the literal string token from the
3103@code{yytname} table (@pxref{Calling Convention}).
3104
3105@strong{Warning}: literal string tokens do not work in Yacc.
3106
3107By convention, a literal string token is used only to represent a token
3108that consists of that particular string. Thus, you should use the token
3109type @code{"<="} to represent the string @samp{<=} as a token. Bison
3110does not enforce this convention, but if you depart from it, people who
3111read your program will be confused.
3112
3113All the escape sequences used in string literals in C can be used in
3114Bison as well, except that you must not use a null character within a
3115string literal. Also, unlike Standard C, trigraphs have no special
3116meaning in Bison string literals, nor is backslash-newline allowed. A
3117literal string token must contain two or more characters; for a token
3118containing just one character, use a character token (see above).
3119@end itemize
3120
3121How you choose to write a terminal symbol has no effect on its
3122grammatical meaning. That depends only on where it appears in rules and
3123on when the parser function returns that symbol.
3124
3125The value returned by @code{yylex} is always one of the terminal
3126symbols, except that a zero or negative value signifies end-of-input.
3127Whichever way you write the token type in the grammar rules, you write
3128it the same way in the definition of @code{yylex}. The numeric code
3129for a character token type is simply the positive numeric code of the
3130character, so @code{yylex} can use the identical value to generate the
3131requisite code, though you may need to convert it to @code{unsigned
3132char} to avoid sign-extension on hosts where @code{char} is signed.
3133Each named token type becomes a C macro in
3134the parser file, so @code{yylex} can use the name to stand for the code.
3135(This is why periods don't make sense in terminal symbols.)
3136@xref{Calling Convention, ,Calling Convention for @code{yylex}}.
3137
3138If @code{yylex} is defined in a separate file, you need to arrange for the
3139token-type macro definitions to be available there. Use the @samp{-d}
3140option when you run Bison, so that it will write these macro definitions
3141into a separate header file @file{@var{name}.tab.h} which you can include
3142in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
3143
3144If you want to write a grammar that is portable to any Standard C
3145host, you must use only nonnull character tokens taken from the basic
3146execution character set of Standard C@. This set consists of the ten
3147digits, the 52 lower- and upper-case English letters, and the
3148characters in the following C-language string:
3149
3150@example
3151"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3152@end example
3153
3154The @code{yylex} function and Bison must use a consistent character set
3155and encoding for character tokens. For example, if you run Bison in an
3156@acronym{ASCII} environment, but then compile and run the resulting
3157program in an environment that uses an incompatible character set like
3158@acronym{EBCDIC}, the resulting program may not work because the tables
3159generated by Bison will assume @acronym{ASCII} numeric values for
3160character tokens. It is standard practice for software distributions to
3161contain C source files that were generated by Bison in an
3162@acronym{ASCII} environment, so installers on platforms that are
3163incompatible with @acronym{ASCII} must rebuild those files before
3164compiling them.
3165
3166The symbol @code{error} is a terminal symbol reserved for error recovery
3167(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
3168In particular, @code{yylex} should never return this value. The default
3169value of the error token is 256, unless you explicitly assigned 256 to
3170one of your tokens with a @code{%token} declaration.
3171
3172@node Rules
3173@section Syntax of Grammar Rules
3174@cindex rule syntax
3175@cindex grammar rule syntax
3176@cindex syntax of grammar rules
3177
3178A Bison grammar rule has the following general form:
3179
3180@example
3181@group
3182@var{result}: @var{components}@dots{}
3183 ;
3184@end group
3185@end example
3186
3187@noindent
3188where @var{result} is the nonterminal symbol that this rule describes,
3189and @var{components} are various terminal and nonterminal symbols that
3190are put together by this rule (@pxref{Symbols}).
3191
3192For example,
3193
3194@example
3195@group
3196exp: exp '+' exp
3197 ;
3198@end group
3199@end example
3200
3201@noindent
3202says that two groupings of type @code{exp}, with a @samp{+} token in between,
3203can be combined into a larger grouping of type @code{exp}.
3204
3205White space in rules is significant only to separate symbols. You can add
3206extra white space as you wish.
3207
3208Scattered among the components can be @var{actions} that determine
3209the semantics of the rule. An action looks like this:
3210
3211@example
3212@{@var{C statements}@}
3213@end example
3214
3215@noindent
3216@cindex braced code
3217This is an example of @dfn{braced code}, that is, C code surrounded by
3218braces, much like a compound statement in C@. Braced code can contain
3219any sequence of C tokens, so long as its braces are balanced. Bison
3220does not check the braced code for correctness directly; it merely
3221copies the code to the output file, where the C compiler can check it.
3222
3223Within braced code, the balanced-brace count is not affected by braces
3224within comments, string literals, or character constants, but it is
3225affected by the C digraphs @samp{<%} and @samp{%>} that represent
3226braces. At the top level braced code must be terminated by @samp{@}}
3227and not by a digraph. Bison does not look for trigraphs, so if braced
3228code uses trigraphs you should ensure that they do not affect the
3229nesting of braces or the boundaries of comments, string literals, or
3230character constants.
3231
3232Usually there is only one action and it follows the components.
3233@xref{Actions}.
3234
3235@findex |
3236Multiple rules for the same @var{result} can be written separately or can
3237be joined with the vertical-bar character @samp{|} as follows:
3238
3239@example
3240@group
3241@var{result}: @var{rule1-components}@dots{}
3242 | @var{rule2-components}@dots{}
3243 @dots{}
3244 ;
3245@end group
3246@end example
3247
3248@noindent
3249They are still considered distinct rules even when joined in this way.
3250
3251If @var{components} in a rule is empty, it means that @var{result} can
3252match the empty string. For example, here is how to define a
3253comma-separated sequence of zero or more @code{exp} groupings:
3254
3255@example
3256@group
3257expseq: /* empty */
3258 | expseq1
3259 ;
3260@end group
3261
3262@group
3263expseq1: exp
3264 | expseq1 ',' exp
3265 ;
3266@end group
3267@end example
3268
3269@noindent
3270It is customary to write a comment @samp{/* empty */} in each rule
3271with no components.
3272
3273@node Recursion
3274@section Recursive Rules
3275@cindex recursive rule
3276
3277A rule is called @dfn{recursive} when its @var{result} nonterminal
3278appears also on its right hand side. Nearly all Bison grammars need to
3279use recursion, because that is the only way to define a sequence of any
3280number of a particular thing. Consider this recursive definition of a
3281comma-separated sequence of one or more expressions:
3282
3283@example
3284@group
3285expseq1: exp
3286 | expseq1 ',' exp
3287 ;
3288@end group
3289@end example
3290
3291@cindex left recursion
3292@cindex right recursion
3293@noindent
3294Since the recursive use of @code{expseq1} is the leftmost symbol in the
3295right hand side, we call this @dfn{left recursion}. By contrast, here
3296the same construct is defined using @dfn{right recursion}:
3297
3298@example
3299@group
3300expseq1: exp
3301 | exp ',' expseq1
3302 ;
3303@end group
3304@end example
3305
3306@noindent
3307Any kind of sequence can be defined using either left recursion or right
3308recursion, but you should always use left recursion, because it can
3309parse a sequence of any number of elements with bounded stack space.
3310Right recursion uses up space on the Bison stack in proportion to the
3311number of elements in the sequence, because all the elements must be
3312shifted onto the stack before the rule can be applied even once.
3313@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3314of this.
3315
3316@cindex mutual recursion
3317@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3318rule does not appear directly on its right hand side, but does appear
3319in rules for other nonterminals which do appear on its right hand
3320side.
3321
3322For example:
3323
3324@example
3325@group
3326expr: primary
3327 | primary '+' primary
3328 ;
3329@end group
3330
3331@group
3332primary: constant
3333 | '(' expr ')'
3334 ;
3335@end group
3336@end example
3337
3338@noindent
3339defines two mutually-recursive nonterminals, since each refers to the
3340other.
3341
3342@node Semantics
3343@section Defining Language Semantics
3344@cindex defining language semantics
3345@cindex language semantics, defining
3346
3347The grammar rules for a language determine only the syntax. The semantics
3348are determined by the semantic values associated with various tokens and
3349groupings, and by the actions taken when various groupings are recognized.
3350
3351For example, the calculator calculates properly because the value
3352associated with each expression is the proper number; it adds properly
3353because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3354the numbers associated with @var{x} and @var{y}.
3355
3356@menu
3357* Value Type:: Specifying one data type for all semantic values.
3358* Multiple Types:: Specifying several alternative data types.
3359* Actions:: An action is the semantic definition of a grammar rule.
3360* Action Types:: Specifying data types for actions to operate on.
3361* Mid-Rule Actions:: Most actions go at the end of a rule.
3362 This says when, why and how to use the exceptional
3363 action in the middle of a rule.
3364@end menu
3365
3366@node Value Type
3367@subsection Data Types of Semantic Values
3368@cindex semantic value type
3369@cindex value type, semantic
3370@cindex data types of semantic values
3371@cindex default data type
3372
3373In a simple program it may be sufficient to use the same data type for
3374the semantic values of all language constructs. This was true in the
3375@acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
3376Notation Calculator}).
3377
3378Bison normally uses the type @code{int} for semantic values if your
3379program uses the same data type for all language constructs. To
3380specify some other type, define @code{YYSTYPE} as a macro, like this:
3381
3382@example
3383#define YYSTYPE double
3384@end example
3385
3386@noindent
3387@code{YYSTYPE}'s replacement list should be a type name
3388that does not contain parentheses or square brackets.
3389This macro definition must go in the prologue of the grammar file
3390(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
3391
3392@node Multiple Types
3393@subsection More Than One Value Type
3394
3395In most programs, you will need different data types for different kinds
3396of tokens and groupings. For example, a numeric constant may need type
3397@code{int} or @code{long int}, while a string constant needs type
3398@code{char *}, and an identifier might need a pointer to an entry in the
3399symbol table.
3400
3401To use more than one data type for semantic values in one parser, Bison
3402requires you to do two things:
3403
3404@itemize @bullet
3405@item
3406Specify the entire collection of possible data types, either by using the
3407@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
3408Value Types}), or by using a @code{typedef} or a @code{#define} to
3409define @code{YYSTYPE} to be a union type whose member names are
3410the type tags.
3411
3412@item
3413Choose one of those types for each symbol (terminal or nonterminal) for
3414which semantic values are used. This is done for tokens with the
3415@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3416and for groupings with the @code{%type} Bison declaration (@pxref{Type
3417Decl, ,Nonterminal Symbols}).
3418@end itemize
3419
3420@node Actions
3421@subsection Actions
3422@cindex action
3423@vindex $$
3424@vindex $@var{n}
3425
3426An action accompanies a syntactic rule and contains C code to be executed
3427each time an instance of that rule is recognized. The task of most actions
3428is to compute a semantic value for the grouping built by the rule from the
3429semantic values associated with tokens or smaller groupings.
3430
3431An action consists of braced code containing C statements, and can be
3432placed at any position in the rule;
3433it is executed at that position. Most rules have just one action at the
3434end of the rule, following all the components. Actions in the middle of
3435a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3436Actions, ,Actions in Mid-Rule}).
3437
3438The C code in an action can refer to the semantic values of the components
3439matched by the rule with the construct @code{$@var{n}}, which stands for
3440the value of the @var{n}th component. The semantic value for the grouping
3441being constructed is @code{$$}. Bison translates both of these
3442constructs into expressions of the appropriate type when it copies the
3443actions into the parser file. @code{$$} is translated to a modifiable
3444lvalue, so it can be assigned to.
3445
3446Here is a typical example:
3447
3448@example
3449@group
3450exp: @dots{}
3451 | exp '+' exp
3452 @{ $$ = $1 + $3; @}
3453@end group
3454@end example
3455
3456@noindent
3457This rule constructs an @code{exp} from two smaller @code{exp} groupings
3458connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3459refer to the semantic values of the two component @code{exp} groupings,
3460which are the first and third symbols on the right hand side of the rule.
3461The sum is stored into @code{$$} so that it becomes the semantic value of
3462the addition-expression just recognized by the rule. If there were a
3463useful semantic value associated with the @samp{+} token, it could be
3464referred to as @code{$2}.
3465
3466Note that the vertical-bar character @samp{|} is really a rule
3467separator, and actions are attached to a single rule. This is a
3468difference with tools like Flex, for which @samp{|} stands for either
3469``or'', or ``the same action as that of the next rule''. In the
3470following example, the action is triggered only when @samp{b} is found:
3471
3472@example
3473@group
3474a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3475@end group
3476@end example
3477
3478@cindex default action
3479If you don't specify an action for a rule, Bison supplies a default:
3480@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3481becomes the value of the whole rule. Of course, the default action is
3482valid only if the two data types match. There is no meaningful default
3483action for an empty rule; every empty rule must have an explicit action
3484unless the rule's value does not matter.
3485
3486@code{$@var{n}} with @var{n} zero or negative is allowed for reference
3487to tokens and groupings on the stack @emph{before} those that match the
3488current rule. This is a very risky practice, and to use it reliably
3489you must be certain of the context in which the rule is applied. Here
3490is a case in which you can use this reliably:
3491
3492@example
3493@group
3494foo: expr bar '+' expr @{ @dots{} @}
3495 | expr bar '-' expr @{ @dots{} @}
3496 ;
3497@end group
3498
3499@group
3500bar: /* empty */
3501 @{ previous_expr = $0; @}
3502 ;
3503@end group
3504@end example
3505
3506As long as @code{bar} is used only in the fashion shown here, @code{$0}
3507always refers to the @code{expr} which precedes @code{bar} in the
3508definition of @code{foo}.
3509
3510@vindex yylval
3511It is also possible to access the semantic value of the lookahead token, if
3512any, from a semantic action.
3513This semantic value is stored in @code{yylval}.
3514@xref{Action Features, ,Special Features for Use in Actions}.
3515
3516@node Action Types
3517@subsection Data Types of Values in Actions
3518@cindex action data types
3519@cindex data types in actions
3520
3521If you have chosen a single data type for semantic values, the @code{$$}
3522and @code{$@var{n}} constructs always have that data type.
3523
3524If you have used @code{%union} to specify a variety of data types, then you
3525must declare a choice among these types for each terminal or nonterminal
3526symbol that can have a semantic value. Then each time you use @code{$$} or
3527@code{$@var{n}}, its data type is determined by which symbol it refers to
3528in the rule. In this example,
3529
3530@example
3531@group
3532exp: @dots{}
3533 | exp '+' exp
3534 @{ $$ = $1 + $3; @}
3535@end group
3536@end example
3537
3538@noindent
3539@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3540have the data type declared for the nonterminal symbol @code{exp}. If
3541@code{$2} were used, it would have the data type declared for the
3542terminal symbol @code{'+'}, whatever that might be.
3543
3544Alternatively, you can specify the data type when you refer to the value,
3545by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3546reference. For example, if you have defined types as shown here:
3547
3548@example
3549@group
3550%union @{
3551 int itype;
3552 double dtype;
3553@}
3554@end group
3555@end example
3556
3557@noindent
3558then you can write @code{$<itype>1} to refer to the first subunit of the
3559rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3560
3561@node Mid-Rule Actions
3562@subsection Actions in Mid-Rule
3563@cindex actions in mid-rule
3564@cindex mid-rule actions
3565
3566Occasionally it is useful to put an action in the middle of a rule.
3567These actions are written just like usual end-of-rule actions, but they
3568are executed before the parser even recognizes the following components.
3569
3570A mid-rule action may refer to the components preceding it using
3571@code{$@var{n}}, but it may not refer to subsequent components because
3572it is run before they are parsed.
3573
3574The mid-rule action itself counts as one of the components of the rule.
3575This makes a difference when there is another action later in the same rule
3576(and usually there is another at the end): you have to count the actions
3577along with the symbols when working out which number @var{n} to use in
3578@code{$@var{n}}.
3579
3580The mid-rule action can also have a semantic value. The action can set
3581its value with an assignment to @code{$$}, and actions later in the rule
3582can refer to the value using @code{$@var{n}}. Since there is no symbol
3583to name the action, there is no way to declare a data type for the value
3584in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3585specify a data type each time you refer to this value.
3586
3587There is no way to set the value of the entire rule with a mid-rule
3588action, because assignments to @code{$$} do not have that effect. The
3589only way to set the value for the entire rule is with an ordinary action
3590at the end of the rule.
3591
3592Here is an example from a hypothetical compiler, handling a @code{let}
3593statement that looks like @samp{let (@var{variable}) @var{statement}} and
3594serves to create a variable named @var{variable} temporarily for the
3595duration of @var{statement}. To parse this construct, we must put
3596@var{variable} into the symbol table while @var{statement} is parsed, then
3597remove it afterward. Here is how it is done:
3598
3599@example
3600@group
3601stmt: LET '(' var ')'
3602 @{ $<context>$ = push_context ();
3603 declare_variable ($3); @}
3604 stmt @{ $$ = $6;
3605 pop_context ($<context>5); @}
3606@end group
3607@end example
3608
3609@noindent
3610As soon as @samp{let (@var{variable})} has been recognized, the first
3611action is run. It saves a copy of the current semantic context (the
3612list of accessible variables) as its semantic value, using alternative
3613@code{context} in the data-type union. Then it calls
3614@code{declare_variable} to add the new variable to that list. Once the
3615first action is finished, the embedded statement @code{stmt} can be
3616parsed. Note that the mid-rule action is component number 5, so the
3617@samp{stmt} is component number 6.
3618
3619After the embedded statement is parsed, its semantic value becomes the
3620value of the entire @code{let}-statement. Then the semantic value from the
3621earlier action is used to restore the prior list of variables. This
3622removes the temporary @code{let}-variable from the list so that it won't
3623appear to exist while the rest of the program is parsed.
3624
3625@findex %destructor
3626@cindex discarded symbols, mid-rule actions
3627@cindex error recovery, mid-rule actions
3628In the above example, if the parser initiates error recovery (@pxref{Error
3629Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3630it might discard the previous semantic context @code{$<context>5} without
3631restoring it.
3632Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3633Discarded Symbols}).
3634However, Bison currently provides no means to declare a destructor specific to
3635a particular mid-rule action's semantic value.
3636
3637One solution is to bury the mid-rule action inside a nonterminal symbol and to
3638declare a destructor for that symbol:
3639
3640@example
3641@group
3642%type <context> let
3643%destructor @{ pop_context ($$); @} let
3644
3645%%
3646
3647stmt: let stmt
3648 @{ $$ = $2;
3649 pop_context ($1); @}
3650 ;
3651
3652let: LET '(' var ')'
3653 @{ $$ = push_context ();
3654 declare_variable ($3); @}
3655 ;
3656
3657@end group
3658@end example
3659
3660@noindent
3661Note that the action is now at the end of its rule.
3662Any mid-rule action can be converted to an end-of-rule action in this way, and
3663this is what Bison actually does to implement mid-rule actions.
3664
3665Taking action before a rule is completely recognized often leads to
3666conflicts since the parser must commit to a parse in order to execute the
3667action. For example, the following two rules, without mid-rule actions,
3668can coexist in a working parser because the parser can shift the open-brace
3669token and look at what follows before deciding whether there is a
3670declaration or not:
3671
3672@example
3673@group
3674compound: '@{' declarations statements '@}'
3675 | '@{' statements '@}'
3676 ;
3677@end group
3678@end example
3679
3680@noindent
3681But when we add a mid-rule action as follows, the rules become nonfunctional:
3682
3683@example
3684@group
3685compound: @{ prepare_for_local_variables (); @}
3686 '@{' declarations statements '@}'
3687@end group
3688@group
3689 | '@{' statements '@}'
3690 ;
3691@end group
3692@end example
3693
3694@noindent
3695Now the parser is forced to decide whether to run the mid-rule action
3696when it has read no farther than the open-brace. In other words, it
3697must commit to using one rule or the other, without sufficient
3698information to do it correctly. (The open-brace token is what is called
3699the @dfn{lookahead} token at this time, since the parser is still
3700deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.)
3701
3702You might think that you could correct the problem by putting identical
3703actions into the two rules, like this:
3704
3705@example
3706@group
3707compound: @{ prepare_for_local_variables (); @}
3708 '@{' declarations statements '@}'
3709 | @{ prepare_for_local_variables (); @}
3710 '@{' statements '@}'
3711 ;
3712@end group
3713@end example
3714
3715@noindent
3716But this does not help, because Bison does not realize that the two actions
3717are identical. (Bison never tries to understand the C code in an action.)
3718
3719If the grammar is such that a declaration can be distinguished from a
3720statement by the first token (which is true in C), then one solution which
3721does work is to put the action after the open-brace, like this:
3722
3723@example
3724@group
3725compound: '@{' @{ prepare_for_local_variables (); @}
3726 declarations statements '@}'
3727 | '@{' statements '@}'
3728 ;
3729@end group
3730@end example
3731
3732@noindent
3733Now the first token of the following declaration or statement,
3734which would in any case tell Bison which rule to use, can still do so.
3735
3736Another solution is to bury the action inside a nonterminal symbol which
3737serves as a subroutine:
3738
3739@example
3740@group
3741subroutine: /* empty */
3742 @{ prepare_for_local_variables (); @}
3743 ;
3744
3745@end group
3746
3747@group
3748compound: subroutine
3749 '@{' declarations statements '@}'
3750 | subroutine
3751 '@{' statements '@}'
3752 ;
3753@end group
3754@end example
3755
3756@noindent
3757Now Bison can execute the action in the rule for @code{subroutine} without
3758deciding which rule for @code{compound} it will eventually use.
3759
3760@node Locations
3761@section Tracking Locations
3762@cindex location
3763@cindex textual location
3764@cindex location, textual
3765
3766Though grammar rules and semantic actions are enough to write a fully
3767functional parser, it can be useful to process some additional information,
3768especially symbol locations.
3769
3770The way locations are handled is defined by providing a data type, and
3771actions to take when rules are matched.
3772
3773@menu
3774* Location Type:: Specifying a data type for locations.
3775* Actions and Locations:: Using locations in actions.
3776* Location Default Action:: Defining a general way to compute locations.
3777@end menu
3778
3779@node Location Type
3780@subsection Data Type of Locations
3781@cindex data type of locations
3782@cindex default location type
3783
3784Defining a data type for locations is much simpler than for semantic values,
3785since all tokens and groupings always use the same type.
3786
3787You can specify the type of locations by defining a macro called
3788@code{YYLTYPE}, just as you can specify the semantic value type by
3789defining a @code{YYSTYPE} macro (@pxref{Value Type}).
3790When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3791four members:
3792
3793@example
3794typedef struct YYLTYPE
3795@{
3796 int first_line;
3797 int first_column;
3798 int last_line;
3799 int last_column;
3800@} YYLTYPE;
3801@end example
3802
3803At the beginning of the parsing, Bison initializes all these fields to 1
3804for @code{yylloc}.
3805
3806@node Actions and Locations
3807@subsection Actions and Locations
3808@cindex location actions
3809@cindex actions, location
3810@vindex @@$
3811@vindex @@@var{n}
3812
3813Actions are not only useful for defining language semantics, but also for
3814describing the behavior of the output parser with locations.
3815
3816The most obvious way for building locations of syntactic groupings is very
3817similar to the way semantic values are computed. In a given rule, several
3818constructs can be used to access the locations of the elements being matched.
3819The location of the @var{n}th component of the right hand side is
3820@code{@@@var{n}}, while the location of the left hand side grouping is
3821@code{@@$}.
3822
3823Here is a basic example using the default data type for locations:
3824
3825@example
3826@group
3827exp: @dots{}
3828 | exp '/' exp
3829 @{
3830 @@$.first_column = @@1.first_column;
3831 @@$.first_line = @@1.first_line;
3832 @@$.last_column = @@3.last_column;
3833 @@$.last_line = @@3.last_line;
3834 if ($3)
3835 $$ = $1 / $3;
3836 else
3837 @{
3838 $$ = 1;
3839 fprintf (stderr,
3840 "Division by zero, l%d,c%d-l%d,c%d",
3841 @@3.first_line, @@3.first_column,
3842 @@3.last_line, @@3.last_column);
3843 @}
3844 @}
3845@end group
3846@end example
3847
3848As for semantic values, there is a default action for locations that is
3849run each time a rule is matched. It sets the beginning of @code{@@$} to the
3850beginning of the first symbol, and the end of @code{@@$} to the end of the
3851last symbol.
3852
3853With this default action, the location tracking can be fully automatic. The
3854example above simply rewrites this way:
3855
3856@example
3857@group
3858exp: @dots{}
3859 | exp '/' exp
3860 @{
3861 if ($3)
3862 $$ = $1 / $3;
3863 else
3864 @{
3865 $$ = 1;
3866 fprintf (stderr,
3867 "Division by zero, l%d,c%d-l%d,c%d",
3868 @@3.first_line, @@3.first_column,
3869 @@3.last_line, @@3.last_column);
3870 @}
3871 @}
3872@end group
3873@end example
3874
3875@vindex yylloc
3876It is also possible to access the location of the lookahead token, if any,
3877from a semantic action.
3878This location is stored in @code{yylloc}.
3879@xref{Action Features, ,Special Features for Use in Actions}.
3880
3881@node Location Default Action
3882@subsection Default Action for Locations
3883@vindex YYLLOC_DEFAULT
3884@cindex @acronym{GLR} parsers and @code{YYLLOC_DEFAULT}
3885
3886Actually, actions are not the best place to compute locations. Since
3887locations are much more general than semantic values, there is room in
3888the output parser to redefine the default action to take for each
3889rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
3890matched, before the associated action is run. It is also invoked
3891while processing a syntax error, to compute the error's location.
3892Before reporting an unresolvable syntactic ambiguity, a @acronym{GLR}
3893parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
3894of that ambiguity.
3895
3896Most of the time, this macro is general enough to suppress location
3897dedicated code from semantic actions.
3898
3899The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
3900the location of the grouping (the result of the computation). When a
3901rule is matched, the second parameter identifies locations of
3902all right hand side elements of the rule being matched, and the third
3903parameter is the size of the rule's right hand side.
3904When a @acronym{GLR} parser reports an ambiguity, which of multiple candidate
3905right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
3906When processing a syntax error, the second parameter identifies locations
3907of the symbols that were discarded during error processing, and the third
3908parameter is the number of discarded symbols.
3909
3910By default, @code{YYLLOC_DEFAULT} is defined this way:
3911
3912@smallexample
3913@group
3914# define YYLLOC_DEFAULT(Current, Rhs, N) \
3915 do \
3916 if (N) \
3917 @{ \
3918 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
3919 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
3920 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
3921 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
3922 @} \
3923 else \
3924 @{ \
3925 (Current).first_line = (Current).last_line = \
3926 YYRHSLOC(Rhs, 0).last_line; \
3927 (Current).first_column = (Current).last_column = \
3928 YYRHSLOC(Rhs, 0).last_column; \
3929 @} \
3930 while (0)
3931@end group
3932@end smallexample
3933
3934where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
3935in @var{rhs} when @var{k} is positive, and the location of the symbol
3936just before the reduction when @var{k} and @var{n} are both zero.
3937
3938When defining @code{YYLLOC_DEFAULT}, you should consider that:
3939
3940@itemize @bullet
3941@item
3942All arguments are free of side-effects. However, only the first one (the
3943result) should be modified by @code{YYLLOC_DEFAULT}.
3944
3945@item
3946For consistency with semantic actions, valid indexes within the
3947right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
3948valid index, and it refers to the symbol just before the reduction.
3949During error processing @var{n} is always positive.
3950
3951@item
3952Your macro should parenthesize its arguments, if need be, since the
3953actual arguments may not be surrounded by parentheses. Also, your
3954macro should expand to something that can be used as a single
3955statement when it is followed by a semicolon.
3956@end itemize
3957
3958@node Declarations
3959@section Bison Declarations
3960@cindex declarations, Bison
3961@cindex Bison declarations
3962
3963The @dfn{Bison declarations} section of a Bison grammar defines the symbols
3964used in formulating the grammar and the data types of semantic values.
3965@xref{Symbols}.
3966
3967All token type names (but not single-character literal tokens such as
3968@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
3969declared if you need to specify which data type to use for the semantic
3970value (@pxref{Multiple Types, ,More Than One Value Type}).
3971
3972The first rule in the file also specifies the start symbol, by default.
3973If you want some other symbol to be the start symbol, you must declare
3974it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
3975Grammars}).
3976
3977@menu
3978* Require Decl:: Requiring a Bison version.
3979* Token Decl:: Declaring terminal symbols.
3980* Precedence Decl:: Declaring terminals with precedence and associativity.
3981* Union Decl:: Declaring the set of all semantic value types.
3982* Type Decl:: Declaring the choice of type for a nonterminal symbol.
3983* Initial Action Decl:: Code run before parsing starts.
3984* Destructor Decl:: Declaring how symbols are freed.
3985* Expect Decl:: Suppressing warnings about parsing conflicts.
3986* Start Decl:: Specifying the start symbol.
3987* Pure Decl:: Requesting a reentrant parser.
3988* Push Decl:: Requesting a push parser.
3989* Decl Summary:: Table of all Bison declarations.
3990@end menu
3991
3992@node Require Decl
3993@subsection Require a Version of Bison
3994@cindex version requirement
3995@cindex requiring a version of Bison
3996@findex %require
3997
3998You may require the minimum version of Bison to process the grammar. If
3999the requirement is not met, @command{bison} exits with an error (exit
4000status 63).
4001
4002@example
4003%require "@var{version}"
4004@end example
4005
4006@node Token Decl
4007@subsection Token Type Names
4008@cindex declaring token type names
4009@cindex token type names, declaring
4010@cindex declaring literal string tokens
4011@findex %token
4012
4013The basic way to declare a token type name (terminal symbol) is as follows:
4014
4015@example
4016%token @var{name}
4017@end example
4018
4019Bison will convert this into a @code{#define} directive in
4020the parser, so that the function @code{yylex} (if it is in this file)
4021can use the name @var{name} to stand for this token type's code.
4022
4023Alternatively, you can use @code{%left}, @code{%right}, or
4024@code{%nonassoc} instead of @code{%token}, if you wish to specify
4025associativity and precedence. @xref{Precedence Decl, ,Operator
4026Precedence}.
4027
4028You can explicitly specify the numeric code for a token type by appending
4029a nonnegative decimal or hexadecimal integer value in the field immediately
4030following the token name:
4031
4032@example
4033%token NUM 300
4034%token XNUM 0x12d // a GNU extension
4035@end example
4036
4037@noindent
4038It is generally best, however, to let Bison choose the numeric codes for
4039all token types. Bison will automatically select codes that don't conflict
4040with each other or with normal characters.
4041
4042In the event that the stack type is a union, you must augment the
4043@code{%token} or other token declaration to include the data type
4044alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
4045Than One Value Type}).
4046
4047For example:
4048
4049@example
4050@group
4051%union @{ /* define stack type */
4052 double val;
4053 symrec *tptr;
4054@}
4055%token <val> NUM /* define token NUM and its type */
4056@end group
4057@end example
4058
4059You can associate a literal string token with a token type name by
4060writing the literal string at the end of a @code{%token}
4061declaration which declares the name. For example:
4062
4063@example
4064%token arrow "=>"
4065@end example
4066
4067@noindent
4068For example, a grammar for the C language might specify these names with
4069equivalent literal string tokens:
4070
4071@example
4072%token <operator> OR "||"
4073%token <operator> LE 134 "<="
4074%left OR "<="
4075@end example
4076
4077@noindent
4078Once you equate the literal string and the token name, you can use them
4079interchangeably in further declarations or the grammar rules. The
4080@code{yylex} function can use the token name or the literal string to
4081obtain the token type code number (@pxref{Calling Convention}).
4082Syntax error messages passed to @code{yyerror} from the parser will reference
4083the literal string instead of the token name.
4084
4085The token numbered as 0 corresponds to end of file; the following line
4086allows for nicer error messages referring to ``end of file'' instead
4087of ``$end'':
4088
4089@example
4090%token END 0 "end of file"
4091@end example
4092
4093@node Precedence Decl
4094@subsection Operator Precedence
4095@cindex precedence declarations
4096@cindex declaring operator precedence
4097@cindex operator precedence, declaring
4098
4099Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to
4100declare a token and specify its precedence and associativity, all at
4101once. These are called @dfn{precedence declarations}.
4102@xref{Precedence, ,Operator Precedence}, for general information on
4103operator precedence.
4104
4105The syntax of a precedence declaration is nearly the same as that of
4106@code{%token}: either
4107
4108@example
4109%left @var{symbols}@dots{}
4110@end example
4111
4112@noindent
4113or
4114
4115@example
4116%left <@var{type}> @var{symbols}@dots{}
4117@end example
4118
4119And indeed any of these declarations serves the purposes of @code{%token}.
4120But in addition, they specify the associativity and relative precedence for
4121all the @var{symbols}:
4122
4123@itemize @bullet
4124@item
4125The associativity of an operator @var{op} determines how repeated uses
4126of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
4127@var{z}} is parsed by grouping @var{x} with @var{y} first or by
4128grouping @var{y} with @var{z} first. @code{%left} specifies
4129left-associativity (grouping @var{x} with @var{y} first) and
4130@code{%right} specifies right-associativity (grouping @var{y} with
4131@var{z} first). @code{%nonassoc} specifies no associativity, which
4132means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
4133considered a syntax error.
4134
4135@item
4136The precedence of an operator determines how it nests with other operators.
4137All the tokens declared in a single precedence declaration have equal
4138precedence and nest together according to their associativity.
4139When two tokens declared in different precedence declarations associate,
4140the one declared later has the higher precedence and is grouped first.
4141@end itemize
4142
4143For backward compatibility, there is a confusing difference between the
4144argument lists of @code{%token} and precedence declarations.
4145Only a @code{%token} can associate a literal string with a token type name.
4146A precedence declaration always interprets a literal string as a reference to a
4147separate token.
4148For example:
4149
4150@example
4151%left OR "<=" // Does not declare an alias.
4152%left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
4153@end example
4154
4155@node Union Decl
4156@subsection The Collection of Value Types
4157@cindex declaring value types
4158@cindex value types, declaring
4159@findex %union
4160
4161The @code{%union} declaration specifies the entire collection of
4162possible data types for semantic values. The keyword @code{%union} is
4163followed by braced code containing the same thing that goes inside a
4164@code{union} in C@.
4165
4166For example:
4167
4168@example
4169@group
4170%union @{
4171 double val;
4172 symrec *tptr;
4173@}
4174@end group
4175@end example
4176
4177@noindent
4178This says that the two alternative types are @code{double} and @code{symrec
4179*}. They are given names @code{val} and @code{tptr}; these names are used
4180in the @code{%token} and @code{%type} declarations to pick one of the types
4181for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
4182
4183As an extension to @acronym{POSIX}, a tag is allowed after the
4184@code{union}. For example:
4185
4186@example
4187@group
4188%union value @{
4189 double val;
4190 symrec *tptr;
4191@}
4192@end group
4193@end example
4194
4195@noindent
4196specifies the union tag @code{value}, so the corresponding C type is
4197@code{union value}. If you do not specify a tag, it defaults to
4198@code{YYSTYPE}.
4199
4200As another extension to @acronym{POSIX}, you may specify multiple
4201@code{%union} declarations; their contents are concatenated. However,
4202only the first @code{%union} declaration can specify a tag.
4203
4204Note that, unlike making a @code{union} declaration in C, you need not write
4205a semicolon after the closing brace.
4206
4207Instead of @code{%union}, you can define and use your own union type
4208@code{YYSTYPE} if your grammar contains at least one
4209@samp{<@var{type}>} tag. For example, you can put the following into
4210a header file @file{parser.h}:
4211
4212@example
4213@group
4214union YYSTYPE @{
4215 double val;
4216 symrec *tptr;
4217@};
4218typedef union YYSTYPE YYSTYPE;
4219@end group
4220@end example
4221
4222@noindent
4223and then your grammar can use the following
4224instead of @code{%union}:
4225
4226@example
4227@group
4228%@{
4229#include "parser.h"
4230%@}
4231%type <val> expr
4232%token <tptr> ID
4233@end group
4234@end example
4235
4236@node Type Decl
4237@subsection Nonterminal Symbols
4238@cindex declaring value types, nonterminals
4239@cindex value types, nonterminals, declaring
4240@findex %type
4241
4242@noindent
4243When you use @code{%union} to specify multiple value types, you must
4244declare the value type of each nonterminal symbol for which values are
4245used. This is done with a @code{%type} declaration, like this:
4246
4247@example
4248%type <@var{type}> @var{nonterminal}@dots{}
4249@end example
4250
4251@noindent
4252Here @var{nonterminal} is the name of a nonterminal symbol, and
4253@var{type} is the name given in the @code{%union} to the alternative
4254that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
4255can give any number of nonterminal symbols in the same @code{%type}
4256declaration, if they have the same value type. Use spaces to separate
4257the symbol names.
4258
4259You can also declare the value type of a terminal symbol. To do this,
4260use the same @code{<@var{type}>} construction in a declaration for the
4261terminal symbol. All kinds of token declarations allow
4262@code{<@var{type}>}.
4263
4264@node Initial Action Decl
4265@subsection Performing Actions before Parsing
4266@findex %initial-action
4267
4268Sometimes your parser needs to perform some initializations before
4269parsing. The @code{%initial-action} directive allows for such arbitrary
4270code.
4271
4272@deffn {Directive} %initial-action @{ @var{code} @}
4273@findex %initial-action
4274Declare that the braced @var{code} must be invoked before parsing each time
4275@code{yyparse} is called. The @var{code} may use @code{$$} and
4276@code{@@$} --- initial value and location of the lookahead --- and the
4277@code{%parse-param}.
4278@end deffn
4279
4280For instance, if your locations use a file name, you may use
4281
4282@example
4283%parse-param @{ char const *file_name @};
4284%initial-action
4285@{
4286 @@$.initialize (file_name);
4287@};
4288@end example
4289
4290
4291@node Destructor Decl
4292@subsection Freeing Discarded Symbols
4293@cindex freeing discarded symbols
4294@findex %destructor
4295@findex <*>
4296@findex <>
4297During error recovery (@pxref{Error Recovery}), symbols already pushed
4298on the stack and tokens coming from the rest of the file are discarded
4299until the parser falls on its feet. If the parser runs out of memory,
4300or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
4301symbols on the stack must be discarded. Even if the parser succeeds, it
4302must discard the start symbol.
4303
4304When discarded symbols convey heap based information, this memory is
4305lost. While this behavior can be tolerable for batch parsers, such as
4306in traditional compilers, it is unacceptable for programs like shells or
4307protocol implementations that may parse and execute indefinitely.
4308
4309The @code{%destructor} directive defines code that is called when a
4310symbol is automatically discarded.
4311
4312@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
4313@findex %destructor
4314Invoke the braced @var{code} whenever the parser discards one of the
4315@var{symbols}.
4316Within @var{code}, @code{$$} designates the semantic value associated
4317with the discarded symbol, and @code{@@$} designates its location.
4318The additional parser parameters are also available (@pxref{Parser Function, ,
4319The Parser Function @code{yyparse}}).
4320
4321When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
4322per-symbol @code{%destructor}.
4323You may also define a per-type @code{%destructor} by listing a semantic type
4324tag among @var{symbols}.
4325In that case, the parser will invoke this @var{code} whenever it discards any
4326grammar symbol that has that semantic type tag unless that symbol has its own
4327per-symbol @code{%destructor}.
4328
4329Finally, you can define two different kinds of default @code{%destructor}s.
4330(These default forms are experimental.
4331More user feedback will help to determine whether they should become permanent
4332features.)
4333You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
4334exactly one @code{%destructor} declaration in your grammar file.
4335The parser will invoke the @var{code} associated with one of these whenever it
4336discards any user-defined grammar symbol that has no per-symbol and no per-type
4337@code{%destructor}.
4338The parser uses the @var{code} for @code{<*>} in the case of such a grammar
4339symbol for which you have formally declared a semantic type tag (@code{%type}
4340counts as such a declaration, but @code{$<tag>$} does not).
4341The parser uses the @var{code} for @code{<>} in the case of such a grammar
4342symbol that has no declared semantic type tag.
4343@end deffn
4344
4345@noindent
4346For example:
4347
4348@smallexample
4349%union @{ char *string; @}
4350%token <string> STRING1
4351%token <string> STRING2
4352%type <string> string1
4353%type <string> string2
4354%union @{ char character; @}
4355%token <character> CHR
4356%type <character> chr
4357%token TAGLESS
4358
4359%destructor @{ @} <character>
4360%destructor @{ free ($$); @} <*>
4361%destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
4362%destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
4363@end smallexample
4364
4365@noindent
4366guarantees that, when the parser discards any user-defined symbol that has a
4367semantic type tag other than @code{<character>}, it passes its semantic value
4368to @code{free} by default.
4369However, when the parser discards a @code{STRING1} or a @code{string1}, it also
4370prints its line number to @code{stdout}.
4371It performs only the second @code{%destructor} in this case, so it invokes
4372@code{free} only once.
4373Finally, the parser merely prints a message whenever it discards any symbol,
4374such as @code{TAGLESS}, that has no semantic type tag.
4375
4376A Bison-generated parser invokes the default @code{%destructor}s only for
4377user-defined as opposed to Bison-defined symbols.
4378For example, the parser will not invoke either kind of default
4379@code{%destructor} for the special Bison-defined symbols @code{$accept},
4380@code{$undefined}, or @code{$end} (@pxref{Table of Symbols, ,Bison Symbols}),
4381none of which you can reference in your grammar.
4382It also will not invoke either for the @code{error} token (@pxref{Table of
4383Symbols, ,error}), which is always defined by Bison regardless of whether you
4384reference it in your grammar.
4385However, it may invoke one of them for the end token (token 0) if you
4386redefine it from @code{$end} to, for example, @code{END}:
4387
4388@smallexample
4389%token END 0
4390@end smallexample
4391
4392@cindex actions in mid-rule
4393@cindex mid-rule actions
4394Finally, Bison will never invoke a @code{%destructor} for an unreferenced
4395mid-rule semantic value (@pxref{Mid-Rule Actions,,Actions in Mid-Rule}).
4396That is, Bison does not consider a mid-rule to have a semantic value if you do
4397not reference @code{$$} in the mid-rule's action or @code{$@var{n}} (where
4398@var{n} is the RHS symbol position of the mid-rule) in any later action in that
4399rule.
4400However, if you do reference either, the Bison-generated parser will invoke the
4401@code{<>} @code{%destructor} whenever it discards the mid-rule symbol.
4402
4403@ignore
4404@noindent
4405In the future, it may be possible to redefine the @code{error} token as a
4406nonterminal that captures the discarded symbols.
4407In that case, the parser will invoke the default destructor for it as well.
4408@end ignore
4409
4410@sp 1
4411
4412@cindex discarded symbols
4413@dfn{Discarded symbols} are the following:
4414
4415@itemize
4416@item
4417stacked symbols popped during the first phase of error recovery,
4418@item
4419incoming terminals during the second phase of error recovery,
4420@item
4421the current lookahead and the entire stack (except the current
4422right-hand side symbols) when the parser returns immediately, and
4423@item
4424the start symbol, when the parser succeeds.
4425@end itemize
4426
4427The parser can @dfn{return immediately} because of an explicit call to
4428@code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
4429exhaustion.
4430
4431Right-hand side symbols of a rule that explicitly triggers a syntax
4432error via @code{YYERROR} are not discarded automatically. As a rule
4433of thumb, destructors are invoked only when user actions cannot manage
4434the memory.
4435
4436@node Expect Decl
4437@subsection Suppressing Conflict Warnings
4438@cindex suppressing conflict warnings
4439@cindex preventing warnings about conflicts
4440@cindex warnings, preventing
4441@cindex conflicts, suppressing warnings of
4442@findex %expect
4443@findex %expect-rr
4444
4445Bison normally warns if there are any conflicts in the grammar
4446(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4447have harmless shift/reduce conflicts which are resolved in a predictable
4448way and would be difficult to eliminate. It is desirable to suppress
4449the warning about these conflicts unless the number of conflicts
4450changes. You can do this with the @code{%expect} declaration.
4451
4452The declaration looks like this:
4453
4454@example
4455%expect @var{n}
4456@end example
4457
4458Here @var{n} is a decimal integer. The declaration says there should
4459be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4460Bison reports an error if the number of shift/reduce conflicts differs
4461from @var{n}, or if there are any reduce/reduce conflicts.
4462
4463For normal @acronym{LALR}(1) parsers, reduce/reduce conflicts are more
4464serious, and should be eliminated entirely. Bison will always report
4465reduce/reduce conflicts for these parsers. With @acronym{GLR}
4466parsers, however, both kinds of conflicts are routine; otherwise,
4467there would be no need to use @acronym{GLR} parsing. Therefore, it is
4468also possible to specify an expected number of reduce/reduce conflicts
4469in @acronym{GLR} parsers, using the declaration:
4470
4471@example
4472%expect-rr @var{n}
4473@end example
4474
4475In general, using @code{%expect} involves these steps:
4476
4477@itemize @bullet
4478@item
4479Compile your grammar without @code{%expect}. Use the @samp{-v} option
4480to get a verbose list of where the conflicts occur. Bison will also
4481print the number of conflicts.
4482
4483@item
4484Check each of the conflicts to make sure that Bison's default
4485resolution is what you really want. If not, rewrite the grammar and
4486go back to the beginning.
4487
4488@item
4489Add an @code{%expect} declaration, copying the number @var{n} from the
4490number which Bison printed. With @acronym{GLR} parsers, add an
4491@code{%expect-rr} declaration as well.
4492@end itemize
4493
4494Now Bison will warn you if you introduce an unexpected conflict, but
4495will keep silent otherwise.
4496
4497@node Start Decl
4498@subsection The Start-Symbol
4499@cindex declaring the start symbol
4500@cindex start symbol, declaring
4501@cindex default start symbol
4502@findex %start
4503
4504Bison assumes by default that the start symbol for the grammar is the first
4505nonterminal specified in the grammar specification section. The programmer
4506may override this restriction with the @code{%start} declaration as follows:
4507
4508@example
4509%start @var{symbol}
4510@end example
4511
4512@node Pure Decl
4513@subsection A Pure (Reentrant) Parser
4514@cindex reentrant parser
4515@cindex pure parser
4516@findex %define api.pure
4517
4518A @dfn{reentrant} program is one which does not alter in the course of
4519execution; in other words, it consists entirely of @dfn{pure} (read-only)
4520code. Reentrancy is important whenever asynchronous execution is possible;
4521for example, a nonreentrant program may not be safe to call from a signal
4522handler. In systems with multiple threads of control, a nonreentrant
4523program must be called only within interlocks.
4524
4525Normally, Bison generates a parser which is not reentrant. This is
4526suitable for most uses, and it permits compatibility with Yacc. (The
4527standard Yacc interfaces are inherently nonreentrant, because they use
4528statically allocated variables for communication with @code{yylex},
4529including @code{yylval} and @code{yylloc}.)
4530
4531Alternatively, you can generate a pure, reentrant parser. The Bison
4532declaration @code{%define api.pure} says that you want the parser to be
4533reentrant. It looks like this:
4534
4535@example
4536%define api.pure
4537@end example
4538
4539The result is that the communication variables @code{yylval} and
4540@code{yylloc} become local variables in @code{yyparse}, and a different
4541calling convention is used for the lexical analyzer function
4542@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
4543Parsers}, for the details of this. The variable @code{yynerrs}
4544becomes local in @code{yyparse} in pull mode but it becomes a member
4545of yypstate in push mode. (@pxref{Error Reporting, ,The Error
4546Reporting Function @code{yyerror}}). The convention for calling
4547@code{yyparse} itself is unchanged.
4548
4549Whether the parser is pure has nothing to do with the grammar rules.
4550You can generate either a pure parser or a nonreentrant parser from any
4551valid grammar.
4552
4553@node Push Decl
4554@subsection A Push Parser
4555@cindex push parser
4556@cindex push parser
4557@findex %define api.push_pull
4558
4559(The current push parsing interface is experimental and may evolve.
4560More user feedback will help to stabilize it.)
4561
4562A pull parser is called once and it takes control until all its input
4563is completely parsed. A push parser, on the other hand, is called
4564each time a new token is made available.
4565
4566A push parser is typically useful when the parser is part of a
4567main event loop in the client's application. This is typically
4568a requirement of a GUI, when the main event loop needs to be triggered
4569within a certain time period.
4570
4571Normally, Bison generates a pull parser.
4572The following Bison declaration says that you want the parser to be a push
4573parser (@pxref{Decl Summary,,%define api.push_pull}):
4574
4575@example
4576%define api.push_pull "push"
4577@end example
4578
4579In almost all cases, you want to ensure that your push parser is also
4580a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). The only
4581time you should create an impure push parser is to have backwards
4582compatibility with the impure Yacc pull mode interface. Unless you know
4583what you are doing, your declarations should look like this:
4584
4585@example
4586%define api.pure
4587%define api.push_pull "push"
4588@end example
4589
4590There is a major notable functional difference between the pure push parser
4591and the impure push parser. It is acceptable for a pure push parser to have
4592many parser instances, of the same type of parser, in memory at the same time.
4593An impure push parser should only use one parser at a time.
4594
4595When a push parser is selected, Bison will generate some new symbols in
4596the generated parser. @code{yypstate} is a structure that the generated
4597parser uses to store the parser's state. @code{yypstate_new} is the
4598function that will create a new parser instance. @code{yypstate_delete}
4599will free the resources associated with the corresponding parser instance.
4600Finally, @code{yypush_parse} is the function that should be called whenever a
4601token is available to provide the parser. A trivial example
4602of using a pure push parser would look like this:
4603
4604@example
4605int status;
4606yypstate *ps = yypstate_new ();
4607do @{
4608 status = yypush_parse (ps, yylex (), NULL);
4609@} while (status == YYPUSH_MORE);
4610yypstate_delete (ps);
4611@end example
4612
4613If the user decided to use an impure push parser, a few things about
4614the generated parser will change. The @code{yychar} variable becomes
4615a global variable instead of a variable in the @code{yypush_parse} function.
4616For this reason, the signature of the @code{yypush_parse} function is
4617changed to remove the token as a parameter. A nonreentrant push parser
4618example would thus look like this:
4619
4620@example
4621extern int yychar;
4622int status;
4623yypstate *ps = yypstate_new ();
4624do @{
4625 yychar = yylex ();
4626 status = yypush_parse (ps);
4627@} while (status == YYPUSH_MORE);
4628yypstate_delete (ps);
4629@end example
4630
4631That's it. Notice the next token is put into the global variable @code{yychar}
4632for use by the next invocation of the @code{yypush_parse} function.
4633
4634Bison also supports both the push parser interface along with the pull parser
4635interface in the same generated parser. In order to get this functionality,
4636you should replace the @code{%define api.push_pull "push"} declaration with the
4637@code{%define api.push_pull "both"} declaration. Doing this will create all of
4638the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
4639and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
4640would be used. However, the user should note that it is implemented in the
4641generated parser by calling @code{yypull_parse}.
4642This makes the @code{yyparse} function that is generated with the
4643@code{%define api.push_pull "both"} declaration slower than the normal
4644@code{yyparse} function. If the user
4645calls the @code{yypull_parse} function it will parse the rest of the input
4646stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
4647and then @code{yypull_parse} the rest of the input stream. If you would like
4648to switch back and forth between between parsing styles, you would have to
4649write your own @code{yypull_parse} function that knows when to quit looking
4650for input. An example of using the @code{yypull_parse} function would look
4651like this:
4652
4653@example
4654yypstate *ps = yypstate_new ();
4655yypull_parse (ps); /* Will call the lexer */
4656yypstate_delete (ps);
4657@end example
4658
4659Adding the @code{%define api.pure} declaration does exactly the same thing to
4660the generated parser with @code{%define api.push_pull "both"} as it did for
4661@code{%define api.push_pull "push"}.
4662
4663@node Decl Summary
4664@subsection Bison Declaration Summary
4665@cindex Bison declaration summary
4666@cindex declaration summary
4667@cindex summary, Bison declaration
4668
4669Here is a summary of the declarations used to define a grammar:
4670
4671@deffn {Directive} %union
4672Declare the collection of data types that semantic values may have
4673(@pxref{Union Decl, ,The Collection of Value Types}).
4674@end deffn
4675
4676@deffn {Directive} %token
4677Declare a terminal symbol (token type name) with no precedence
4678or associativity specified (@pxref{Token Decl, ,Token Type Names}).
4679@end deffn
4680
4681@deffn {Directive} %right
4682Declare a terminal symbol (token type name) that is right-associative
4683(@pxref{Precedence Decl, ,Operator Precedence}).
4684@end deffn
4685
4686@deffn {Directive} %left
4687Declare a terminal symbol (token type name) that is left-associative
4688(@pxref{Precedence Decl, ,Operator Precedence}).
4689@end deffn
4690
4691@deffn {Directive} %nonassoc
4692Declare a terminal symbol (token type name) that is nonassociative
4693(@pxref{Precedence Decl, ,Operator Precedence}).
4694Using it in a way that would be associative is a syntax error.
4695@end deffn
4696
4697@ifset defaultprec
4698@deffn {Directive} %default-prec
4699Assign a precedence to rules lacking an explicit @code{%prec} modifier
4700(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4701@end deffn
4702@end ifset
4703
4704@deffn {Directive} %type
4705Declare the type of semantic values for a nonterminal symbol
4706(@pxref{Type Decl, ,Nonterminal Symbols}).
4707@end deffn
4708
4709@deffn {Directive} %start
4710Specify the grammar's start symbol (@pxref{Start Decl, ,The
4711Start-Symbol}).
4712@end deffn
4713
4714@deffn {Directive} %expect
4715Declare the expected number of shift-reduce conflicts
4716(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
4717@end deffn
4718
4719
4720@sp 1
4721@noindent
4722In order to change the behavior of @command{bison}, use the following
4723directives:
4724
4725@deffn {Directive} %code @{@var{code}@}
4726@findex %code
4727This is the unqualified form of the @code{%code} directive.
4728It inserts @var{code} verbatim at a language-dependent default location in the
4729output@footnote{The default location is actually skeleton-dependent;
4730 writers of non-standard skeletons however should choose the default location
4731 consistently with the behavior of the standard Bison skeletons.}.
4732
4733@cindex Prologue
4734For C/C++, the default location is the parser source code
4735file after the usual contents of the parser header file.
4736Thus, @code{%code} replaces the traditional Yacc prologue,
4737@code{%@{@var{code}%@}}, for most purposes.
4738For a detailed discussion, see @ref{Prologue Alternatives}.
4739
4740For Java, the default location is inside the parser class.
4741@end deffn
4742
4743@deffn {Directive} %code @var{qualifier} @{@var{code}@}
4744This is the qualified form of the @code{%code} directive.
4745If you need to specify location-sensitive verbatim @var{code} that does not
4746belong at the default location selected by the unqualified @code{%code} form,
4747use this form instead.
4748
4749@var{qualifier} identifies the purpose of @var{code} and thus the location(s)
4750where Bison should generate it.
4751Not all values of @var{qualifier} are available for all target languages:
4752
4753@itemize @bullet
4754@item requires
4755@findex %code requires
4756
4757@itemize @bullet
4758@item Language(s): C, C++
4759
4760@item Purpose: This is the best place to write dependency code required for
4761@code{YYSTYPE} and @code{YYLTYPE}.
4762In other words, it's the best place to define types referenced in @code{%union}
4763directives, and it's the best place to override Bison's default @code{YYSTYPE}
4764and @code{YYLTYPE} definitions.
4765
4766@item Location(s): The parser header file and the parser source code file
4767before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE} definitions.
4768@end itemize
4769
4770@item provides
4771@findex %code provides
4772
4773@itemize @bullet
4774@item Language(s): C, C++
4775
4776@item Purpose: This is the best place to write additional definitions and
4777declarations that should be provided to other modules.
4778
4779@item Location(s): The parser header file and the parser source code file after
4780the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and token definitions.
4781@end itemize
4782
4783@item top
4784@findex %code top
4785
4786@itemize @bullet
4787@item Language(s): C, C++
4788
4789@item Purpose: The unqualified @code{%code} or @code{%code requires} should
4790usually be more appropriate than @code{%code top}.
4791However, occasionally it is necessary to insert code much nearer the top of the
4792parser source code file.
4793For example:
4794
4795@smallexample
4796%code top @{
4797 #define _GNU_SOURCE
4798 #include <stdio.h>
4799@}
4800@end smallexample
4801
4802@item Location(s): Near the top of the parser source code file.
4803@end itemize
4804
4805@item imports
4806@findex %code imports
4807
4808@itemize @bullet
4809@item Language(s): Java
4810
4811@item Purpose: This is the best place to write Java import directives.
4812
4813@item Location(s): The parser Java file after any Java package directive and
4814before any class definitions.
4815@end itemize
4816@end itemize
4817
4818@cindex Prologue
4819For a detailed discussion of how to use @code{%code} in place of the
4820traditional Yacc prologue for C/C++, see @ref{Prologue Alternatives}.
4821@end deffn
4822
4823@deffn {Directive} %debug
4824In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
4825already defined, so that the debugging facilities are compiled.
4826@xref{Tracing, ,Tracing Your Parser}.
4827@end deffn
4828
4829@deffn {Directive} %define @var{variable}
4830@deffnx {Directive} %define @var{variable} "@var{value}"
4831Define a variable to adjust Bison's behavior.
4832The possible choices for @var{variable}, as well as their meanings, depend on
4833the selected target language and/or the parser skeleton (@pxref{Decl
4834Summary,,%language}, @pxref{Decl Summary,,%skeleton}).
4835
4836Bison will warn if a @var{variable} is defined multiple times.
4837
4838Omitting @code{"@var{value}"} is always equivalent to specifying it as
4839@code{""}.
4840
4841Some @var{variable}s may be used as Booleans.
4842In this case, Bison will complain if the variable definition does not meet one
4843of the following four conditions:
4844
4845@enumerate
4846@item @code{"@var{value}"} is @code{"true"}
4847
4848@item @code{"@var{value}"} is omitted (or is @code{""}).
4849This is equivalent to @code{"true"}.
4850
4851@item @code{"@var{value}"} is @code{"false"}.
4852
4853@item @var{variable} is never defined.
4854In this case, Bison selects a default value, which may depend on the selected
4855target language and/or parser skeleton.
4856@end enumerate
4857
4858Some of the accepted @var{variable}s are:
4859
4860@itemize @bullet
4861@item api.pure
4862@findex %define api.pure
4863
4864@itemize @bullet
4865@item Language(s): C
4866
4867@item Purpose: Request a pure (reentrant) parser program.
4868@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
4869
4870@item Accepted Values: Boolean
4871
4872@item Default Value: @code{"false"}
4873@end itemize
4874
4875@item api.push_pull
4876@findex %define api.push_pull
4877
4878@itemize @bullet
4879@item Language(s): C (LALR(1) only)
4880
4881@item Purpose: Requests a pull parser, a push parser, or both.
4882@xref{Push Decl, ,A Push Parser}.
4883(The current push parsing interface is experimental and may evolve.
4884More user feedback will help to stabilize it.)
4885
4886@item Accepted Values: @code{"pull"}, @code{"push"}, @code{"both"}
4887
4888@item Default Value: @code{"pull"}
4889@end itemize
4890
4891@item lr.keep_unreachable_states
4892@findex %define lr.keep_unreachable_states
4893
4894@itemize @bullet
4895@item Language(s): all
4896
4897@item Purpose: Requests that Bison allow unreachable parser states to remain in
4898the parser tables.
4899Bison considers a state to be unreachable if there exists no sequence of
4900transitions from the start state to that state.
4901A state can become unreachable during conflict resolution if Bison disables a
4902shift action leading to it from a predecessor state.
4903Keeping unreachable states is sometimes useful for analysis purposes, but they
4904are useless in the generated parser.
4905
4906@item Accepted Values: Boolean
4907
4908@item Default Value: @code{"false"}
4909
4910@item Caveats:
4911
4912@itemize @bullet
4913
4914@item Unreachable states may contain conflicts and may use rules not used in
4915any other state.
4916Thus, keeping unreachable states may induce warnings that are irrelevant to
4917your parser's behavior, and it may eliminate warnings that are relevant.
4918Of course, the change in warnings may actually be relevant to a parser table
4919analysis that wants to keep unreachable states, so this behavior will likely
4920remain in future Bison releases.
4921
4922@item While Bison is able to remove unreachable states, it is not guaranteed to
4923remove other kinds of useless states.
4924Specifically, when Bison disables reduce actions during conflict resolution,
4925some goto actions may become useless, and thus some additional states may
4926become useless.
4927If Bison were to compute which goto actions were useless and then disable those
4928actions, it could identify such states as unreachable and then remove those
4929states.
4930However, Bison does not compute which goto actions are useless.
4931@end itemize
4932@end itemize
4933
4934@item namespace
4935@findex %define namespace
4936
4937@itemize
4938@item Languages(s): C++
4939
4940@item Purpose: Specifies the namespace for the parser class.
4941For example, if you specify:
4942
4943@smallexample
4944%define namespace "foo::bar"
4945@end smallexample
4946
4947Bison uses @code{foo::bar} verbatim in references such as:
4948
4949@smallexample
4950foo::bar::parser::semantic_type
4951@end smallexample
4952
4953However, to open a namespace, Bison removes any leading @code{::} and then
4954splits on any remaining occurrences:
4955
4956@smallexample
4957namespace foo @{ namespace bar @{
4958 class position;
4959 class location;
4960@} @}
4961@end smallexample
4962
4963@item Accepted Values: Any absolute or relative C++ namespace reference without
4964a trailing @code{"::"}.
4965For example, @code{"foo"} or @code{"::foo::bar"}.
4966
4967@item Default Value: The value specified by @code{%name-prefix}, which defaults
4968to @code{yy}.
4969This usage of @code{%name-prefix} is for backward compatibility and can be
4970confusing since @code{%name-prefix} also specifies the textual prefix for the
4971lexical analyzer function.
4972Thus, if you specify @code{%name-prefix}, it is best to also specify
4973@code{%define namespace} so that @code{%name-prefix} @emph{only} affects the
4974lexical analyzer function.
4975For example, if you specify:
4976
4977@smallexample
4978%define namespace "foo"
4979%name-prefix "bar::"
4980@end smallexample
4981
4982The parser namespace is @code{foo} and @code{yylex} is referenced as
4983@code{bar::lex}.
4984@end itemize
4985@end itemize
4986
4987@end deffn
4988
4989@deffn {Directive} %defines
4990Write a header file containing macro definitions for the token type
4991names defined in the grammar as well as a few other declarations.
4992If the parser output file is named @file{@var{name}.c} then this file
4993is named @file{@var{name}.h}.
4994
4995For C parsers, the output header declares @code{YYSTYPE} unless
4996@code{YYSTYPE} is already defined as a macro or you have used a
4997@code{<@var{type}>} tag without using @code{%union}.
4998Therefore, if you are using a @code{%union}
4999(@pxref{Multiple Types, ,More Than One Value Type}) with components that
5000require other definitions, or if you have defined a @code{YYSTYPE} macro
5001or type definition
5002(@pxref{Value Type, ,Data Types of Semantic Values}), you need to
5003arrange for these definitions to be propagated to all modules, e.g., by
5004putting them in a prerequisite header that is included both by your
5005parser and by any other module that needs @code{YYSTYPE}.
5006
5007Unless your parser is pure, the output header declares @code{yylval}
5008as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
5009Parser}.
5010
5011If you have also used locations, the output header declares
5012@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
5013the @code{YYSTYPE} macro and @code{yylval}. @xref{Locations, ,Tracking
5014Locations}.
5015
5016This output file is normally essential if you wish to put the definition
5017of @code{yylex} in a separate source file, because @code{yylex}
5018typically needs to be able to refer to the above-mentioned declarations
5019and to the token type codes. @xref{Token Values, ,Semantic Values of
5020Tokens}.
5021
5022@findex %code requires
5023@findex %code provides
5024If you have declared @code{%code requires} or @code{%code provides}, the output
5025header also contains their code.
5026@xref{Decl Summary, ,%code}.
5027@end deffn
5028
5029@deffn {Directive} %defines @var{defines-file}
5030Same as above, but save in the file @var{defines-file}.
5031@end deffn
5032
5033@deffn {Directive} %destructor
5034Specify how the parser should reclaim the memory associated to
5035discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
5036@end deffn
5037
5038@deffn {Directive} %file-prefix "@var{prefix}"
5039Specify a prefix to use for all Bison output file names. The names are
5040chosen as if the input file were named @file{@var{prefix}.y}.
5041@end deffn
5042
5043@deffn {Directive} %language "@var{language}"
5044Specify the programming language for the generated parser. Currently
5045supported languages include C, C++, and Java.
5046@var{language} is case-insensitive.
5047
5048This directive is experimental and its effect may be modified in future
5049releases.
5050@end deffn
5051
5052@deffn {Directive} %locations
5053Generate the code processing the locations (@pxref{Action Features,
5054,Special Features for Use in Actions}). This mode is enabled as soon as
5055the grammar uses the special @samp{@@@var{n}} tokens, but if your
5056grammar does not use it, using @samp{%locations} allows for more
5057accurate syntax error messages.
5058@end deffn
5059
5060@deffn {Directive} %name-prefix "@var{prefix}"
5061Rename the external symbols used in the parser so that they start with
5062@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
5063in C parsers
5064is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
5065@code{yylval}, @code{yychar}, @code{yydebug}, and
5066(if locations are used) @code{yylloc}. If you use a push parser,
5067@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5068@code{yypstate_new} and @code{yypstate_delete} will
5069also be renamed. For example, if you use @samp{%name-prefix "c_"}, the
5070names become @code{c_parse}, @code{c_lex}, and so on.
5071For C++ parsers, see the @code{%define namespace} documentation in this
5072section.
5073@xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
5074@end deffn
5075
5076@ifset defaultprec
5077@deffn {Directive} %no-default-prec
5078Do not assign a precedence to rules lacking an explicit @code{%prec}
5079modifier (@pxref{Contextual Precedence, ,Context-Dependent
5080Precedence}).
5081@end deffn
5082@end ifset
5083
5084@deffn {Directive} %no-lines
5085Don't generate any @code{#line} preprocessor commands in the parser
5086file. Ordinarily Bison writes these commands in the parser file so that
5087the C compiler and debuggers will associate errors and object code with
5088your source file (the grammar file). This directive causes them to
5089associate errors with the parser file, treating it an independent source
5090file in its own right.
5091@end deffn
5092
5093@deffn {Directive} %output "@var{file}"
5094Specify @var{file} for the parser file.
5095@end deffn
5096
5097@deffn {Directive} %pure-parser
5098Deprecated version of @code{%define api.pure} (@pxref{Decl Summary, ,%define}),
5099for which Bison is more careful to warn about unreasonable usage.
5100@end deffn
5101
5102@deffn {Directive} %require "@var{version}"
5103Require version @var{version} or higher of Bison. @xref{Require Decl, ,
5104Require a Version of Bison}.
5105@end deffn
5106
5107@deffn {Directive} %skeleton "@var{file}"
5108Specify the skeleton to use.
5109
5110@c You probably don't need this option unless you are developing Bison.
5111@c You should use @code{%language} if you want to specify the skeleton for a
5112@c different language, because it is clearer and because it will always choose the
5113@c correct skeleton for non-deterministic or push parsers.
5114
5115If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5116file in the Bison installation directory.
5117If it does, @var{file} is an absolute file name or a file name relative to the
5118directory of the grammar file.
5119This is similar to how most shells resolve commands.
5120@end deffn
5121
5122@deffn {Directive} %token-table
5123Generate an array of token names in the parser file. The name of the
5124array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
5125token whose internal Bison token code number is @var{i}. The first
5126three elements of @code{yytname} correspond to the predefined tokens
5127@code{"$end"},
5128@code{"error"}, and @code{"$undefined"}; after these come the symbols
5129defined in the grammar file.
5130
5131The name in the table includes all the characters needed to represent
5132the token in Bison. For single-character literals and literal
5133strings, this includes the surrounding quoting characters and any
5134escape sequences. For example, the Bison single-character literal
5135@code{'+'} corresponds to a three-character name, represented in C as
5136@code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
5137corresponds to a five-character name, represented in C as
5138@code{"\"\\\\/\""}.
5139
5140When you specify @code{%token-table}, Bison also generates macro
5141definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
5142@code{YYNRULES}, and @code{YYNSTATES}:
5143
5144@table @code
5145@item YYNTOKENS
5146The highest token number, plus one.
5147@item YYNNTS
5148The number of nonterminal symbols.
5149@item YYNRULES
5150The number of grammar rules,
5151@item YYNSTATES
5152The number of parser states (@pxref{Parser States}).
5153@end table
5154@end deffn
5155
5156@deffn {Directive} %verbose
5157Write an extra output file containing verbose descriptions of the
5158parser states and what is done for each type of lookahead token in
5159that state. @xref{Understanding, , Understanding Your Parser}, for more
5160information.
5161@end deffn
5162
5163@deffn {Directive} %yacc
5164Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
5165including its naming conventions. @xref{Bison Options}, for more.
5166@end deffn
5167
5168
5169@node Multiple Parsers
5170@section Multiple Parsers in the Same Program
5171
5172Most programs that use Bison parse only one language and therefore contain
5173only one Bison parser. But what if you want to parse more than one
5174language with the same program? Then you need to avoid a name conflict
5175between different definitions of @code{yyparse}, @code{yylval}, and so on.
5176
5177The easy way to do this is to use the option @samp{-p @var{prefix}}
5178(@pxref{Invocation, ,Invoking Bison}). This renames the interface
5179functions and variables of the Bison parser to start with @var{prefix}
5180instead of @samp{yy}. You can use this to give each parser distinct
5181names that do not conflict.
5182
5183The precise list of symbols renamed is @code{yyparse}, @code{yylex},
5184@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
5185@code{yychar} and @code{yydebug}. If you use a push parser,
5186@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5187@code{yypstate_new} and @code{yypstate_delete} will also be renamed.
5188For example, if you use @samp{-p c}, the names become @code{cparse},
5189@code{clex}, and so on.
5190
5191@strong{All the other variables and macros associated with Bison are not
5192renamed.} These others are not global; there is no conflict if the same
5193name is used in different parsers. For example, @code{YYSTYPE} is not
5194renamed, but defining this in different ways in different parsers causes
5195no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
5196
5197The @samp{-p} option works by adding macro definitions to the beginning
5198of the parser source file, defining @code{yyparse} as
5199@code{@var{prefix}parse}, and so on. This effectively substitutes one
5200name for the other in the entire parser file.
5201
5202@node Interface
5203@chapter Parser C-Language Interface
5204@cindex C-language interface
5205@cindex interface
5206
5207The Bison parser is actually a C function named @code{yyparse}. Here we
5208describe the interface conventions of @code{yyparse} and the other
5209functions that it needs to use.
5210
5211Keep in mind that the parser uses many C identifiers starting with
5212@samp{yy} and @samp{YY} for internal purposes. If you use such an
5213identifier (aside from those in this manual) in an action or in epilogue
5214in the grammar file, you are likely to run into trouble.
5215
5216@menu
5217* Parser Function:: How to call @code{yyparse} and what it returns.
5218* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
5219* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
5220* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
5221* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
5222* Lexical:: You must supply a function @code{yylex}
5223 which reads tokens.
5224* Error Reporting:: You must supply a function @code{yyerror}.
5225* Action Features:: Special features for use in actions.
5226* Internationalization:: How to let the parser speak in the user's
5227 native language.
5228@end menu
5229
5230@node Parser Function
5231@section The Parser Function @code{yyparse}
5232@findex yyparse
5233
5234You call the function @code{yyparse} to cause parsing to occur. This
5235function reads tokens, executes actions, and ultimately returns when it
5236encounters end-of-input or an unrecoverable syntax error. You can also
5237write an action which directs @code{yyparse} to return immediately
5238without reading further.
5239
5240
5241@deftypefun int yyparse (void)
5242The value returned by @code{yyparse} is 0 if parsing was successful (return
5243is due to end-of-input).
5244
5245The value is 1 if parsing failed because of invalid input, i.e., input
5246that contains a syntax error or that causes @code{YYABORT} to be
5247invoked.
5248
5249The value is 2 if parsing failed due to memory exhaustion.
5250@end deftypefun
5251
5252In an action, you can cause immediate return from @code{yyparse} by using
5253these macros:
5254
5255@defmac YYACCEPT
5256@findex YYACCEPT
5257Return immediately with value 0 (to report success).
5258@end defmac
5259
5260@defmac YYABORT
5261@findex YYABORT
5262Return immediately with value 1 (to report failure).
5263@end defmac
5264
5265If you use a reentrant parser, you can optionally pass additional
5266parameter information to it in a reentrant way. To do so, use the
5267declaration @code{%parse-param}:
5268
5269@deffn {Directive} %parse-param @{@var{argument-declaration}@}
5270@findex %parse-param
5271Declare that an argument declared by the braced-code
5272@var{argument-declaration} is an additional @code{yyparse} argument.
5273The @var{argument-declaration} is used when declaring
5274functions or prototypes. The last identifier in
5275@var{argument-declaration} must be the argument name.
5276@end deffn
5277
5278Here's an example. Write this in the parser:
5279
5280@example
5281%parse-param @{int *nastiness@}
5282%parse-param @{int *randomness@}
5283@end example
5284
5285@noindent
5286Then call the parser like this:
5287
5288@example
5289@{
5290 int nastiness, randomness;
5291 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
5292 value = yyparse (&nastiness, &randomness);
5293 @dots{}
5294@}
5295@end example
5296
5297@noindent
5298In the grammar actions, use expressions like this to refer to the data:
5299
5300@example
5301exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
5302@end example
5303
5304@node Push Parser Function
5305@section The Push Parser Function @code{yypush_parse}
5306@findex yypush_parse
5307
5308(The current push parsing interface is experimental and may evolve.
5309More user feedback will help to stabilize it.)
5310
5311You call the function @code{yypush_parse} to parse a single token. This
5312function is available if either the @code{%define api.push_pull "push"} or
5313@code{%define api.push_pull "both"} declaration is used.
5314@xref{Push Decl, ,A Push Parser}.
5315
5316@deftypefun int yypush_parse (yypstate *yyps)
5317The value returned by @code{yypush_parse} is the same as for yyparse with the
5318following exception. @code{yypush_parse} will return YYPUSH_MORE if more input
5319is required to finish parsing the grammar.
5320@end deftypefun
5321
5322@node Pull Parser Function
5323@section The Pull Parser Function @code{yypull_parse}
5324@findex yypull_parse
5325
5326(The current push parsing interface is experimental and may evolve.
5327More user feedback will help to stabilize it.)
5328
5329You call the function @code{yypull_parse} to parse the rest of the input
5330stream. This function is available if the @code{%define api.push_pull "both"}
5331declaration is used.
5332@xref{Push Decl, ,A Push Parser}.
5333
5334@deftypefun int yypull_parse (yypstate *yyps)
5335The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
5336@end deftypefun
5337
5338@node Parser Create Function
5339@section The Parser Create Function @code{yystate_new}
5340@findex yypstate_new
5341
5342(The current push parsing interface is experimental and may evolve.
5343More user feedback will help to stabilize it.)
5344
5345You call the function @code{yypstate_new} to create a new parser instance.
5346This function is available if either the @code{%define api.push_pull "push"} or
5347@code{%define api.push_pull "both"} declaration is used.
5348@xref{Push Decl, ,A Push Parser}.
5349
5350@deftypefun yypstate *yypstate_new (void)
5351The fuction will return a valid parser instance if there was memory available
5352or 0 if no memory was available.
5353In impure mode, it will also return 0 if a parser instance is currently
5354allocated.
5355@end deftypefun
5356
5357@node Parser Delete Function
5358@section The Parser Delete Function @code{yystate_delete}
5359@findex yypstate_delete
5360
5361(The current push parsing interface is experimental and may evolve.
5362More user feedback will help to stabilize it.)
5363
5364You call the function @code{yypstate_delete} to delete a parser instance.
5365function is available if either the @code{%define api.push_pull "push"} or
5366@code{%define api.push_pull "both"} declaration is used.
5367@xref{Push Decl, ,A Push Parser}.
5368
5369@deftypefun void yypstate_delete (yypstate *yyps)
5370This function will reclaim the memory associated with a parser instance.
5371After this call, you should no longer attempt to use the parser instance.
5372@end deftypefun
5373
5374@node Lexical
5375@section The Lexical Analyzer Function @code{yylex}
5376@findex yylex
5377@cindex lexical analyzer
5378
5379The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
5380the input stream and returns them to the parser. Bison does not create
5381this function automatically; you must write it so that @code{yyparse} can
5382call it. The function is sometimes referred to as a lexical scanner.
5383
5384In simple programs, @code{yylex} is often defined at the end of the Bison
5385grammar file. If @code{yylex} is defined in a separate source file, you
5386need to arrange for the token-type macro definitions to be available there.
5387To do this, use the @samp{-d} option when you run Bison, so that it will
5388write these macro definitions into a separate header file
5389@file{@var{name}.tab.h} which you can include in the other source files
5390that need it. @xref{Invocation, ,Invoking Bison}.
5391
5392@menu
5393* Calling Convention:: How @code{yyparse} calls @code{yylex}.
5394* Token Values:: How @code{yylex} must return the semantic value
5395 of the token it has read.
5396* Token Locations:: How @code{yylex} must return the text location
5397 (line number, etc.) of the token, if the
5398 actions want that.
5399* Pure Calling:: How the calling convention differs in a pure parser
5400 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
5401@end menu
5402
5403@node Calling Convention
5404@subsection Calling Convention for @code{yylex}
5405
5406The value that @code{yylex} returns must be the positive numeric code
5407for the type of token it has just found; a zero or negative value
5408signifies end-of-input.
5409
5410When a token is referred to in the grammar rules by a name, that name
5411in the parser file becomes a C macro whose definition is the proper
5412numeric code for that token type. So @code{yylex} can use the name
5413to indicate that type. @xref{Symbols}.
5414
5415When a token is referred to in the grammar rules by a character literal,
5416the numeric code for that character is also the code for the token type.
5417So @code{yylex} can simply return that character code, possibly converted
5418to @code{unsigned char} to avoid sign-extension. The null character
5419must not be used this way, because its code is zero and that
5420signifies end-of-input.
5421
5422Here is an example showing these things:
5423
5424@example
5425int
5426yylex (void)
5427@{
5428 @dots{}
5429 if (c == EOF) /* Detect end-of-input. */
5430 return 0;
5431 @dots{}
5432 if (c == '+' || c == '-')
5433 return c; /* Assume token type for `+' is '+'. */
5434 @dots{}
5435 return INT; /* Return the type of the token. */
5436 @dots{}
5437@}
5438@end example
5439
5440@noindent
5441This interface has been designed so that the output from the @code{lex}
5442utility can be used without change as the definition of @code{yylex}.
5443
5444If the grammar uses literal string tokens, there are two ways that
5445@code{yylex} can determine the token type codes for them:
5446
5447@itemize @bullet
5448@item
5449If the grammar defines symbolic token names as aliases for the
5450literal string tokens, @code{yylex} can use these symbolic names like
5451all others. In this case, the use of the literal string tokens in
5452the grammar file has no effect on @code{yylex}.
5453
5454@item
5455@code{yylex} can find the multicharacter token in the @code{yytname}
5456table. The index of the token in the table is the token type's code.
5457The name of a multicharacter token is recorded in @code{yytname} with a
5458double-quote, the token's characters, and another double-quote. The
5459token's characters are escaped as necessary to be suitable as input
5460to Bison.
5461
5462Here's code for looking up a multicharacter token in @code{yytname},
5463assuming that the characters of the token are stored in
5464@code{token_buffer}, and assuming that the token does not contain any
5465characters like @samp{"} that require escaping.
5466
5467@smallexample
5468for (i = 0; i < YYNTOKENS; i++)
5469 @{
5470 if (yytname[i] != 0
5471 && yytname[i][0] == '"'
5472 && ! strncmp (yytname[i] + 1, token_buffer,
5473 strlen (token_buffer))
5474 && yytname[i][strlen (token_buffer) + 1] == '"'
5475 && yytname[i][strlen (token_buffer) + 2] == 0)
5476 break;
5477 @}
5478@end smallexample
5479
5480The @code{yytname} table is generated only if you use the
5481@code{%token-table} declaration. @xref{Decl Summary}.
5482@end itemize
5483
5484@node Token Values
5485@subsection Semantic Values of Tokens
5486
5487@vindex yylval
5488In an ordinary (nonreentrant) parser, the semantic value of the token must
5489be stored into the global variable @code{yylval}. When you are using
5490just one data type for semantic values, @code{yylval} has that type.
5491Thus, if the type is @code{int} (the default), you might write this in
5492@code{yylex}:
5493
5494@example
5495@group
5496 @dots{}
5497 yylval = value; /* Put value onto Bison stack. */
5498 return INT; /* Return the type of the token. */
5499 @dots{}
5500@end group
5501@end example
5502
5503When you are using multiple data types, @code{yylval}'s type is a union
5504made from the @code{%union} declaration (@pxref{Union Decl, ,The
5505Collection of Value Types}). So when you store a token's value, you
5506must use the proper member of the union. If the @code{%union}
5507declaration looks like this:
5508
5509@example
5510@group
5511%union @{
5512 int intval;
5513 double val;
5514 symrec *tptr;
5515@}
5516@end group
5517@end example
5518
5519@noindent
5520then the code in @code{yylex} might look like this:
5521
5522@example
5523@group
5524 @dots{}
5525 yylval.intval = value; /* Put value onto Bison stack. */
5526 return INT; /* Return the type of the token. */
5527 @dots{}
5528@end group
5529@end example
5530
5531@node Token Locations
5532@subsection Textual Locations of Tokens
5533
5534@vindex yylloc
5535If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
5536Tracking Locations}) in actions to keep track of the textual locations
5537of tokens and groupings, then you must provide this information in
5538@code{yylex}. The function @code{yyparse} expects to find the textual
5539location of a token just parsed in the global variable @code{yylloc}.
5540So @code{yylex} must store the proper data in that variable.
5541
5542By default, the value of @code{yylloc} is a structure and you need only
5543initialize the members that are going to be used by the actions. The
5544four members are called @code{first_line}, @code{first_column},
5545@code{last_line} and @code{last_column}. Note that the use of this
5546feature makes the parser noticeably slower.
5547
5548@tindex YYLTYPE
5549The data type of @code{yylloc} has the name @code{YYLTYPE}.
5550
5551@node Pure Calling
5552@subsection Calling Conventions for Pure Parsers
5553
5554When you use the Bison declaration @code{%define api.pure} to request a
5555pure, reentrant parser, the global communication variables @code{yylval}
5556and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
5557Parser}.) In such parsers the two global variables are replaced by
5558pointers passed as arguments to @code{yylex}. You must declare them as
5559shown here, and pass the information back by storing it through those
5560pointers.
5561
5562@example
5563int
5564yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
5565@{
5566 @dots{}
5567 *lvalp = value; /* Put value onto Bison stack. */
5568 return INT; /* Return the type of the token. */
5569 @dots{}
5570@}
5571@end example
5572
5573If the grammar file does not use the @samp{@@} constructs to refer to
5574textual locations, then the type @code{YYLTYPE} will not be defined. In
5575this case, omit the second argument; @code{yylex} will be called with
5576only one argument.
5577
5578
5579If you wish to pass the additional parameter data to @code{yylex}, use
5580@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
5581Function}).
5582
5583@deffn {Directive} lex-param @{@var{argument-declaration}@}
5584@findex %lex-param
5585Declare that the braced-code @var{argument-declaration} is an
5586additional @code{yylex} argument declaration.
5587@end deffn
5588
5589For instance:
5590
5591@example
5592%parse-param @{int *nastiness@}
5593%lex-param @{int *nastiness@}
5594%parse-param @{int *randomness@}
5595@end example
5596
5597@noindent
5598results in the following signature:
5599
5600@example
5601int yylex (int *nastiness);
5602int yyparse (int *nastiness, int *randomness);
5603@end example
5604
5605If @code{%define api.pure} is added:
5606
5607@example
5608int yylex (YYSTYPE *lvalp, int *nastiness);
5609int yyparse (int *nastiness, int *randomness);
5610@end example
5611
5612@noindent
5613and finally, if both @code{%define api.pure} and @code{%locations} are used:
5614
5615@example
5616int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
5617int yyparse (int *nastiness, int *randomness);
5618@end example
5619
5620@node Error Reporting
5621@section The Error Reporting Function @code{yyerror}
5622@cindex error reporting function
5623@findex yyerror
5624@cindex parse error
5625@cindex syntax error
5626
5627The Bison parser detects a @dfn{syntax error} or @dfn{parse error}
5628whenever it reads a token which cannot satisfy any syntax rule. An
5629action in the grammar can also explicitly proclaim an error, using the
5630macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
5631in Actions}).
5632
5633The Bison parser expects to report the error by calling an error
5634reporting function named @code{yyerror}, which you must supply. It is
5635called by @code{yyparse} whenever a syntax error is found, and it
5636receives one argument. For a syntax error, the string is normally
5637@w{@code{"syntax error"}}.
5638
5639@findex %error-verbose
5640If you invoke the directive @code{%error-verbose} in the Bison
5641declarations section (@pxref{Bison Declarations, ,The Bison Declarations
5642Section}), then Bison provides a more verbose and specific error message
5643string instead of just plain @w{@code{"syntax error"}}.
5644
5645The parser can detect one other kind of error: memory exhaustion. This
5646can happen when the input contains constructions that are very deeply
5647nested. It isn't likely you will encounter this, since the Bison
5648parser normally extends its stack automatically up to a very large limit. But
5649if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
5650fashion, except that the argument string is @w{@code{"memory exhausted"}}.
5651
5652In some cases diagnostics like @w{@code{"syntax error"}} are
5653translated automatically from English to some other language before
5654they are passed to @code{yyerror}. @xref{Internationalization}.
5655
5656The following definition suffices in simple programs:
5657
5658@example
5659@group
5660void
5661yyerror (char const *s)
5662@{
5663@end group
5664@group
5665 fprintf (stderr, "%s\n", s);
5666@}
5667@end group
5668@end example
5669
5670After @code{yyerror} returns to @code{yyparse}, the latter will attempt
5671error recovery if you have written suitable error recovery grammar rules
5672(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
5673immediately return 1.
5674
5675Obviously, in location tracking pure parsers, @code{yyerror} should have
5676an access to the current location.
5677This is indeed the case for the @acronym{GLR}
5678parsers, but not for the Yacc parser, for historical reasons. I.e., if
5679@samp{%locations %define api.pure} is passed then the prototypes for
5680@code{yyerror} are:
5681
5682@example
5683void yyerror (char const *msg); /* Yacc parsers. */
5684void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
5685@end example
5686
5687If @samp{%parse-param @{int *nastiness@}} is used, then:
5688
5689@example
5690void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
5691void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
5692@end example
5693
5694Finally, @acronym{GLR} and Yacc parsers share the same @code{yyerror} calling
5695convention for absolutely pure parsers, i.e., when the calling
5696convention of @code{yylex} @emph{and} the calling convention of
5697@code{%define api.pure} are pure.
5698I.e.:
5699
5700@example
5701/* Location tracking. */
5702%locations
5703/* Pure yylex. */
5704%define api.pure
5705%lex-param @{int *nastiness@}
5706/* Pure yyparse. */
5707%parse-param @{int *nastiness@}
5708%parse-param @{int *randomness@}
5709@end example
5710
5711@noindent
5712results in the following signatures for all the parser kinds:
5713
5714@example
5715int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
5716int yyparse (int *nastiness, int *randomness);
5717void yyerror (YYLTYPE *locp,
5718 int *nastiness, int *randomness,
5719 char const *msg);
5720@end example
5721
5722@noindent
5723The prototypes are only indications of how the code produced by Bison
5724uses @code{yyerror}. Bison-generated code always ignores the returned
5725value, so @code{yyerror} can return any type, including @code{void}.
5726Also, @code{yyerror} can be a variadic function; that is why the
5727message is always passed last.
5728
5729Traditionally @code{yyerror} returns an @code{int} that is always
5730ignored, but this is purely for historical reasons, and @code{void} is
5731preferable since it more accurately describes the return type for
5732@code{yyerror}.
5733
5734@vindex yynerrs
5735The variable @code{yynerrs} contains the number of syntax errors
5736reported so far. Normally this variable is global; but if you
5737request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
5738then it is a local variable which only the actions can access.
5739
5740@node Action Features
5741@section Special Features for Use in Actions
5742@cindex summary, action features
5743@cindex action features summary
5744
5745Here is a table of Bison constructs, variables and macros that
5746are useful in actions.
5747
5748@deffn {Variable} $$
5749Acts like a variable that contains the semantic value for the
5750grouping made by the current rule. @xref{Actions}.
5751@end deffn
5752
5753@deffn {Variable} $@var{n}
5754Acts like a variable that contains the semantic value for the
5755@var{n}th component of the current rule. @xref{Actions}.
5756@end deffn
5757
5758@deffn {Variable} $<@var{typealt}>$
5759Like @code{$$} but specifies alternative @var{typealt} in the union
5760specified by the @code{%union} declaration. @xref{Action Types, ,Data
5761Types of Values in Actions}.
5762@end deffn
5763
5764@deffn {Variable} $<@var{typealt}>@var{n}
5765Like @code{$@var{n}} but specifies alternative @var{typealt} in the
5766union specified by the @code{%union} declaration.
5767@xref{Action Types, ,Data Types of Values in Actions}.
5768@end deffn
5769
5770@deffn {Macro} YYABORT;
5771Return immediately from @code{yyparse}, indicating failure.
5772@xref{Parser Function, ,The Parser Function @code{yyparse}}.
5773@end deffn
5774
5775@deffn {Macro} YYACCEPT;
5776Return immediately from @code{yyparse}, indicating success.
5777@xref{Parser Function, ,The Parser Function @code{yyparse}}.
5778@end deffn
5779
5780@deffn {Macro} YYBACKUP (@var{token}, @var{value});
5781@findex YYBACKUP
5782Unshift a token. This macro is allowed only for rules that reduce
5783a single value, and only when there is no lookahead token.
5784It is also disallowed in @acronym{GLR} parsers.
5785It installs a lookahead token with token type @var{token} and
5786semantic value @var{value}; then it discards the value that was
5787going to be reduced by this rule.
5788
5789If the macro is used when it is not valid, such as when there is
5790a lookahead token already, then it reports a syntax error with
5791a message @samp{cannot back up} and performs ordinary error
5792recovery.
5793
5794In either case, the rest of the action is not executed.
5795@end deffn
5796
5797@deffn {Macro} YYEMPTY
5798@vindex YYEMPTY
5799Value stored in @code{yychar} when there is no lookahead token.
5800@end deffn
5801
5802@deffn {Macro} YYEOF
5803@vindex YYEOF
5804Value stored in @code{yychar} when the lookahead is the end of the input
5805stream.
5806@end deffn
5807
5808@deffn {Macro} YYERROR;
5809@findex YYERROR
5810Cause an immediate syntax error. This statement initiates error
5811recovery just as if the parser itself had detected an error; however, it
5812does not call @code{yyerror}, and does not print any message. If you
5813want to print an error message, call @code{yyerror} explicitly before
5814the @samp{YYERROR;} statement. @xref{Error Recovery}.
5815@end deffn
5816
5817@deffn {Macro} YYRECOVERING
5818@findex YYRECOVERING
5819The expression @code{YYRECOVERING ()} yields 1 when the parser
5820is recovering from a syntax error, and 0 otherwise.
5821@xref{Error Recovery}.
5822@end deffn
5823
5824@deffn {Variable} yychar
5825Variable containing either the lookahead token, or @code{YYEOF} when the
5826lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
5827has been performed so the next token is not yet known.
5828Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
5829Actions}).
5830@xref{Lookahead, ,Lookahead Tokens}.
5831@end deffn
5832
5833@deffn {Macro} yyclearin;
5834Discard the current lookahead token. This is useful primarily in
5835error rules.
5836Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
5837Semantic Actions}).
5838@xref{Error Recovery}.
5839@end deffn
5840
5841@deffn {Macro} yyerrok;
5842Resume generating error messages immediately for subsequent syntax
5843errors. This is useful primarily in error rules.
5844@xref{Error Recovery}.
5845@end deffn
5846
5847@deffn {Variable} yylloc
5848Variable containing the lookahead token location when @code{yychar} is not set
5849to @code{YYEMPTY} or @code{YYEOF}.
5850Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
5851Actions}).
5852@xref{Actions and Locations, ,Actions and Locations}.
5853@end deffn
5854
5855@deffn {Variable} yylval
5856Variable containing the lookahead token semantic value when @code{yychar} is
5857not set to @code{YYEMPTY} or @code{YYEOF}.
5858Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
5859Actions}).
5860@xref{Actions, ,Actions}.
5861@end deffn
5862
5863@deffn {Value} @@$
5864@findex @@$
5865Acts like a structure variable containing information on the textual location
5866of the grouping made by the current rule. @xref{Locations, ,
5867Tracking Locations}.
5868
5869@c Check if those paragraphs are still useful or not.
5870
5871@c @example
5872@c struct @{
5873@c int first_line, last_line;
5874@c int first_column, last_column;
5875@c @};
5876@c @end example
5877
5878@c Thus, to get the starting line number of the third component, you would
5879@c use @samp{@@3.first_line}.
5880
5881@c In order for the members of this structure to contain valid information,
5882@c you must make @code{yylex} supply this information about each token.
5883@c If you need only certain members, then @code{yylex} need only fill in
5884@c those members.
5885
5886@c The use of this feature makes the parser noticeably slower.
5887@end deffn
5888
5889@deffn {Value} @@@var{n}
5890@findex @@@var{n}
5891Acts like a structure variable containing information on the textual location
5892of the @var{n}th component of the current rule. @xref{Locations, ,
5893Tracking Locations}.
5894@end deffn
5895
5896@node Internationalization
5897@section Parser Internationalization
5898@cindex internationalization
5899@cindex i18n
5900@cindex NLS
5901@cindex gettext
5902@cindex bison-po
5903
5904A Bison-generated parser can print diagnostics, including error and
5905tracing messages. By default, they appear in English. However, Bison
5906also supports outputting diagnostics in the user's native language. To
5907make this work, the user should set the usual environment variables.
5908@xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
5909For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
5910set the user's locale to French Canadian using the @acronym{UTF}-8
5911encoding. The exact set of available locales depends on the user's
5912installation.
5913
5914The maintainer of a package that uses a Bison-generated parser enables
5915the internationalization of the parser's output through the following
5916steps. Here we assume a package that uses @acronym{GNU} Autoconf and
5917@acronym{GNU} Automake.
5918
5919@enumerate
5920@item
5921@cindex bison-i18n.m4
5922Into the directory containing the @acronym{GNU} Autoconf macros used
5923by the package---often called @file{m4}---copy the
5924@file{bison-i18n.m4} file installed by Bison under
5925@samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
5926For example:
5927
5928@example
5929cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
5930@end example
5931
5932@item
5933@findex BISON_I18N
5934@vindex BISON_LOCALEDIR
5935@vindex YYENABLE_NLS
5936In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
5937invocation, add an invocation of @code{BISON_I18N}. This macro is
5938defined in the file @file{bison-i18n.m4} that you copied earlier. It
5939causes @samp{configure} to find the value of the
5940@code{BISON_LOCALEDIR} variable, and it defines the source-language
5941symbol @code{YYENABLE_NLS} to enable translations in the
5942Bison-generated parser.
5943
5944@item
5945In the @code{main} function of your program, designate the directory
5946containing Bison's runtime message catalog, through a call to
5947@samp{bindtextdomain} with domain name @samp{bison-runtime}.
5948For example:
5949
5950@example
5951bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
5952@end example
5953
5954Typically this appears after any other call @code{bindtextdomain
5955(PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
5956@samp{BISON_LOCALEDIR} to be defined as a string through the
5957@file{Makefile}.
5958
5959@item
5960In the @file{Makefile.am} that controls the compilation of the @code{main}
5961function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
5962either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
5963
5964@example
5965DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5966@end example
5967
5968or:
5969
5970@example
5971AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5972@end example
5973
5974@item
5975Finally, invoke the command @command{autoreconf} to generate the build
5976infrastructure.
5977@end enumerate
5978
5979
5980@node Algorithm
5981@chapter The Bison Parser Algorithm
5982@cindex Bison parser algorithm
5983@cindex algorithm of parser
5984@cindex shifting
5985@cindex reduction
5986@cindex parser stack
5987@cindex stack, parser
5988
5989As Bison reads tokens, it pushes them onto a stack along with their
5990semantic values. The stack is called the @dfn{parser stack}. Pushing a
5991token is traditionally called @dfn{shifting}.
5992
5993For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
5994@samp{3} to come. The stack will have four elements, one for each token
5995that was shifted.
5996
5997But the stack does not always have an element for each token read. When
5998the last @var{n} tokens and groupings shifted match the components of a
5999grammar rule, they can be combined according to that rule. This is called
6000@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
6001single grouping whose symbol is the result (left hand side) of that rule.
6002Running the rule's action is part of the process of reduction, because this
6003is what computes the semantic value of the resulting grouping.
6004
6005For example, if the infix calculator's parser stack contains this:
6006
6007@example
60081 + 5 * 3
6009@end example
6010
6011@noindent
6012and the next input token is a newline character, then the last three
6013elements can be reduced to 15 via the rule:
6014
6015@example
6016expr: expr '*' expr;
6017@end example
6018
6019@noindent
6020Then the stack contains just these three elements:
6021
6022@example
60231 + 15
6024@end example
6025
6026@noindent
6027At this point, another reduction can be made, resulting in the single value
602816. Then the newline token can be shifted.
6029
6030The parser tries, by shifts and reductions, to reduce the entire input down
6031to a single grouping whose symbol is the grammar's start-symbol
6032(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
6033
6034This kind of parser is known in the literature as a bottom-up parser.
6035
6036@menu
6037* Lookahead:: Parser looks one token ahead when deciding what to do.
6038* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
6039* Precedence:: Operator precedence works by resolving conflicts.
6040* Contextual Precedence:: When an operator's precedence depends on context.
6041* Parser States:: The parser is a finite-state-machine with stack.
6042* Reduce/Reduce:: When two rules are applicable in the same situation.
6043* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
6044* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
6045* Memory Management:: What happens when memory is exhausted. How to avoid it.
6046@end menu
6047
6048@node Lookahead
6049@section Lookahead Tokens
6050@cindex lookahead token
6051
6052The Bison parser does @emph{not} always reduce immediately as soon as the
6053last @var{n} tokens and groupings match a rule. This is because such a
6054simple strategy is inadequate to handle most languages. Instead, when a
6055reduction is possible, the parser sometimes ``looks ahead'' at the next
6056token in order to decide what to do.
6057
6058When a token is read, it is not immediately shifted; first it becomes the
6059@dfn{lookahead token}, which is not on the stack. Now the parser can
6060perform one or more reductions of tokens and groupings on the stack, while
6061the lookahead token remains off to the side. When no more reductions
6062should take place, the lookahead token is shifted onto the stack. This
6063does not mean that all possible reductions have been done; depending on the
6064token type of the lookahead token, some rules may choose to delay their
6065application.
6066
6067Here is a simple case where lookahead is needed. These three rules define
6068expressions which contain binary addition operators and postfix unary
6069factorial operators (@samp{!}), and allow parentheses for grouping.
6070
6071@example
6072@group
6073expr: term '+' expr
6074 | term
6075 ;
6076@end group
6077
6078@group
6079term: '(' expr ')'
6080 | term '!'
6081 | NUMBER
6082 ;
6083@end group
6084@end example
6085
6086Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
6087should be done? If the following token is @samp{)}, then the first three
6088tokens must be reduced to form an @code{expr}. This is the only valid
6089course, because shifting the @samp{)} would produce a sequence of symbols
6090@w{@code{term ')'}}, and no rule allows this.
6091
6092If the following token is @samp{!}, then it must be shifted immediately so
6093that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
6094parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
6095@code{expr}. It would then be impossible to shift the @samp{!} because
6096doing so would produce on the stack the sequence of symbols @code{expr
6097'!'}. No rule allows that sequence.
6098
6099@vindex yychar
6100@vindex yylval
6101@vindex yylloc
6102The lookahead token is stored in the variable @code{yychar}.
6103Its semantic value and location, if any, are stored in the variables
6104@code{yylval} and @code{yylloc}.
6105@xref{Action Features, ,Special Features for Use in Actions}.
6106
6107@node Shift/Reduce
6108@section Shift/Reduce Conflicts
6109@cindex conflicts
6110@cindex shift/reduce conflicts
6111@cindex dangling @code{else}
6112@cindex @code{else}, dangling
6113
6114Suppose we are parsing a language which has if-then and if-then-else
6115statements, with a pair of rules like this:
6116
6117@example
6118@group
6119if_stmt:
6120 IF expr THEN stmt
6121 | IF expr THEN stmt ELSE stmt
6122 ;
6123@end group
6124@end example
6125
6126@noindent
6127Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
6128terminal symbols for specific keyword tokens.
6129
6130When the @code{ELSE} token is read and becomes the lookahead token, the
6131contents of the stack (assuming the input is valid) are just right for
6132reduction by the first rule. But it is also legitimate to shift the
6133@code{ELSE}, because that would lead to eventual reduction by the second
6134rule.
6135
6136This situation, where either a shift or a reduction would be valid, is
6137called a @dfn{shift/reduce conflict}. Bison is designed to resolve
6138these conflicts by choosing to shift, unless otherwise directed by
6139operator precedence declarations. To see the reason for this, let's
6140contrast it with the other alternative.
6141
6142Since the parser prefers to shift the @code{ELSE}, the result is to attach
6143the else-clause to the innermost if-statement, making these two inputs
6144equivalent:
6145
6146@example
6147if x then if y then win (); else lose;
6148
6149if x then do; if y then win (); else lose; end;
6150@end example
6151
6152But if the parser chose to reduce when possible rather than shift, the
6153result would be to attach the else-clause to the outermost if-statement,
6154making these two inputs equivalent:
6155
6156@example
6157if x then if y then win (); else lose;
6158
6159if x then do; if y then win (); end; else lose;
6160@end example
6161
6162The conflict exists because the grammar as written is ambiguous: either
6163parsing of the simple nested if-statement is legitimate. The established
6164convention is that these ambiguities are resolved by attaching the
6165else-clause to the innermost if-statement; this is what Bison accomplishes
6166by choosing to shift rather than reduce. (It would ideally be cleaner to
6167write an unambiguous grammar, but that is very hard to do in this case.)
6168This particular ambiguity was first encountered in the specifications of
6169Algol 60 and is called the ``dangling @code{else}'' ambiguity.
6170
6171To avoid warnings from Bison about predictable, legitimate shift/reduce
6172conflicts, use the @code{%expect @var{n}} declaration. There will be no
6173warning as long as the number of shift/reduce conflicts is exactly @var{n}.
6174@xref{Expect Decl, ,Suppressing Conflict Warnings}.
6175
6176The definition of @code{if_stmt} above is solely to blame for the
6177conflict, but the conflict does not actually appear without additional
6178rules. Here is a complete Bison input file that actually manifests the
6179conflict:
6180
6181@example
6182@group
6183%token IF THEN ELSE variable
6184%%
6185@end group
6186@group
6187stmt: expr
6188 | if_stmt
6189 ;
6190@end group
6191
6192@group
6193if_stmt:
6194 IF expr THEN stmt
6195 | IF expr THEN stmt ELSE stmt
6196 ;
6197@end group
6198
6199expr: variable
6200 ;
6201@end example
6202
6203@node Precedence
6204@section Operator Precedence
6205@cindex operator precedence
6206@cindex precedence of operators
6207
6208Another situation where shift/reduce conflicts appear is in arithmetic
6209expressions. Here shifting is not always the preferred resolution; the
6210Bison declarations for operator precedence allow you to specify when to
6211shift and when to reduce.
6212
6213@menu
6214* Why Precedence:: An example showing why precedence is needed.
6215* Using Precedence:: How to specify precedence in Bison grammars.
6216* Precedence Examples:: How these features are used in the previous example.
6217* How Precedence:: How they work.
6218@end menu
6219
6220@node Why Precedence
6221@subsection When Precedence is Needed
6222
6223Consider the following ambiguous grammar fragment (ambiguous because the
6224input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
6225
6226@example
6227@group
6228expr: expr '-' expr
6229 | expr '*' expr
6230 | expr '<' expr
6231 | '(' expr ')'
6232 @dots{}
6233 ;
6234@end group
6235@end example
6236
6237@noindent
6238Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
6239should it reduce them via the rule for the subtraction operator? It
6240depends on the next token. Of course, if the next token is @samp{)}, we
6241must reduce; shifting is invalid because no single rule can reduce the
6242token sequence @w{@samp{- 2 )}} or anything starting with that. But if
6243the next token is @samp{*} or @samp{<}, we have a choice: either
6244shifting or reduction would allow the parse to complete, but with
6245different results.
6246
6247To decide which one Bison should do, we must consider the results. If
6248the next operator token @var{op} is shifted, then it must be reduced
6249first in order to permit another opportunity to reduce the difference.
6250The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
6251hand, if the subtraction is reduced before shifting @var{op}, the result
6252is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
6253reduce should depend on the relative precedence of the operators
6254@samp{-} and @var{op}: @samp{*} should be shifted first, but not
6255@samp{<}.
6256
6257@cindex associativity
6258What about input such as @w{@samp{1 - 2 - 5}}; should this be
6259@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
6260operators we prefer the former, which is called @dfn{left association}.
6261The latter alternative, @dfn{right association}, is desirable for
6262assignment operators. The choice of left or right association is a
6263matter of whether the parser chooses to shift or reduce when the stack
6264contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
6265makes right-associativity.
6266
6267@node Using Precedence
6268@subsection Specifying Operator Precedence
6269@findex %left
6270@findex %right
6271@findex %nonassoc
6272
6273Bison allows you to specify these choices with the operator precedence
6274declarations @code{%left} and @code{%right}. Each such declaration
6275contains a list of tokens, which are operators whose precedence and
6276associativity is being declared. The @code{%left} declaration makes all
6277those operators left-associative and the @code{%right} declaration makes
6278them right-associative. A third alternative is @code{%nonassoc}, which
6279declares that it is a syntax error to find the same operator twice ``in a
6280row''.
6281
6282The relative precedence of different operators is controlled by the
6283order in which they are declared. The first @code{%left} or
6284@code{%right} declaration in the file declares the operators whose
6285precedence is lowest, the next such declaration declares the operators
6286whose precedence is a little higher, and so on.
6287
6288@node Precedence Examples
6289@subsection Precedence Examples
6290
6291In our example, we would want the following declarations:
6292
6293@example
6294%left '<'
6295%left '-'
6296%left '*'
6297@end example
6298
6299In a more complete example, which supports other operators as well, we
6300would declare them in groups of equal precedence. For example, @code{'+'} is
6301declared with @code{'-'}:
6302
6303@example
6304%left '<' '>' '=' NE LE GE
6305%left '+' '-'
6306%left '*' '/'
6307@end example
6308
6309@noindent
6310(Here @code{NE} and so on stand for the operators for ``not equal''
6311and so on. We assume that these tokens are more than one character long
6312and therefore are represented by names, not character literals.)
6313
6314@node How Precedence
6315@subsection How Precedence Works
6316
6317The first effect of the precedence declarations is to assign precedence
6318levels to the terminal symbols declared. The second effect is to assign
6319precedence levels to certain rules: each rule gets its precedence from
6320the last terminal symbol mentioned in the components. (You can also
6321specify explicitly the precedence of a rule. @xref{Contextual
6322Precedence, ,Context-Dependent Precedence}.)
6323
6324Finally, the resolution of conflicts works by comparing the precedence
6325of the rule being considered with that of the lookahead token. If the
6326token's precedence is higher, the choice is to shift. If the rule's
6327precedence is higher, the choice is to reduce. If they have equal
6328precedence, the choice is made based on the associativity of that
6329precedence level. The verbose output file made by @samp{-v}
6330(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
6331resolved.
6332
6333Not all rules and not all tokens have precedence. If either the rule or
6334the lookahead token has no precedence, then the default is to shift.
6335
6336@node Contextual Precedence
6337@section Context-Dependent Precedence
6338@cindex context-dependent precedence
6339@cindex unary operator precedence
6340@cindex precedence, context-dependent
6341@cindex precedence, unary operator
6342@findex %prec
6343
6344Often the precedence of an operator depends on the context. This sounds
6345outlandish at first, but it is really very common. For example, a minus
6346sign typically has a very high precedence as a unary operator, and a
6347somewhat lower precedence (lower than multiplication) as a binary operator.
6348
6349The Bison precedence declarations, @code{%left}, @code{%right} and
6350@code{%nonassoc}, can only be used once for a given token; so a token has
6351only one precedence declared in this way. For context-dependent
6352precedence, you need to use an additional mechanism: the @code{%prec}
6353modifier for rules.
6354
6355The @code{%prec} modifier declares the precedence of a particular rule by
6356specifying a terminal symbol whose precedence should be used for that rule.
6357It's not necessary for that symbol to appear otherwise in the rule. The
6358modifier's syntax is:
6359
6360@example
6361%prec @var{terminal-symbol}
6362@end example
6363
6364@noindent
6365and it is written after the components of the rule. Its effect is to
6366assign the rule the precedence of @var{terminal-symbol}, overriding
6367the precedence that would be deduced for it in the ordinary way. The
6368altered rule precedence then affects how conflicts involving that rule
6369are resolved (@pxref{Precedence, ,Operator Precedence}).
6370
6371Here is how @code{%prec} solves the problem of unary minus. First, declare
6372a precedence for a fictitious terminal symbol named @code{UMINUS}. There
6373are no tokens of this type, but the symbol serves to stand for its
6374precedence:
6375
6376@example
6377@dots{}
6378%left '+' '-'
6379%left '*'
6380%left UMINUS
6381@end example
6382
6383Now the precedence of @code{UMINUS} can be used in specific rules:
6384
6385@example
6386@group
6387exp: @dots{}
6388 | exp '-' exp
6389 @dots{}
6390 | '-' exp %prec UMINUS
6391@end group
6392@end example
6393
6394@ifset defaultprec
6395If you forget to append @code{%prec UMINUS} to the rule for unary
6396minus, Bison silently assumes that minus has its usual precedence.
6397This kind of problem can be tricky to debug, since one typically
6398discovers the mistake only by testing the code.
6399
6400The @code{%no-default-prec;} declaration makes it easier to discover
6401this kind of problem systematically. It causes rules that lack a
6402@code{%prec} modifier to have no precedence, even if the last terminal
6403symbol mentioned in their components has a declared precedence.
6404
6405If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
6406for all rules that participate in precedence conflict resolution.
6407Then you will see any shift/reduce conflict until you tell Bison how
6408to resolve it, either by changing your grammar or by adding an
6409explicit precedence. This will probably add declarations to the
6410grammar, but it helps to protect against incorrect rule precedences.
6411
6412The effect of @code{%no-default-prec;} can be reversed by giving
6413@code{%default-prec;}, which is the default.
6414@end ifset
6415
6416@node Parser States
6417@section Parser States
6418@cindex finite-state machine
6419@cindex parser state
6420@cindex state (of parser)
6421
6422The function @code{yyparse} is implemented using a finite-state machine.
6423The values pushed on the parser stack are not simply token type codes; they
6424represent the entire sequence of terminal and nonterminal symbols at or
6425near the top of the stack. The current state collects all the information
6426about previous input which is relevant to deciding what to do next.
6427
6428Each time a lookahead token is read, the current parser state together
6429with the type of lookahead token are looked up in a table. This table
6430entry can say, ``Shift the lookahead token.'' In this case, it also
6431specifies the new parser state, which is pushed onto the top of the
6432parser stack. Or it can say, ``Reduce using rule number @var{n}.''
6433This means that a certain number of tokens or groupings are taken off
6434the top of the stack, and replaced by one grouping. In other words,
6435that number of states are popped from the stack, and one new state is
6436pushed.
6437
6438There is one other alternative: the table can say that the lookahead token
6439is erroneous in the current state. This causes error processing to begin
6440(@pxref{Error Recovery}).
6441
6442@node Reduce/Reduce
6443@section Reduce/Reduce Conflicts
6444@cindex reduce/reduce conflict
6445@cindex conflicts, reduce/reduce
6446
6447A reduce/reduce conflict occurs if there are two or more rules that apply
6448to the same sequence of input. This usually indicates a serious error
6449in the grammar.
6450
6451For example, here is an erroneous attempt to define a sequence
6452of zero or more @code{word} groupings.
6453
6454@example
6455sequence: /* empty */
6456 @{ printf ("empty sequence\n"); @}
6457 | maybeword
6458 | sequence word
6459 @{ printf ("added word %s\n", $2); @}
6460 ;
6461
6462maybeword: /* empty */
6463 @{ printf ("empty maybeword\n"); @}
6464 | word
6465 @{ printf ("single word %s\n", $1); @}
6466 ;
6467@end example
6468
6469@noindent
6470The error is an ambiguity: there is more than one way to parse a single
6471@code{word} into a @code{sequence}. It could be reduced to a
6472@code{maybeword} and then into a @code{sequence} via the second rule.
6473Alternatively, nothing-at-all could be reduced into a @code{sequence}
6474via the first rule, and this could be combined with the @code{word}
6475using the third rule for @code{sequence}.
6476
6477There is also more than one way to reduce nothing-at-all into a
6478@code{sequence}. This can be done directly via the first rule,
6479or indirectly via @code{maybeword} and then the second rule.
6480
6481You might think that this is a distinction without a difference, because it
6482does not change whether any particular input is valid or not. But it does
6483affect which actions are run. One parsing order runs the second rule's
6484action; the other runs the first rule's action and the third rule's action.
6485In this example, the output of the program changes.
6486
6487Bison resolves a reduce/reduce conflict by choosing to use the rule that
6488appears first in the grammar, but it is very risky to rely on this. Every
6489reduce/reduce conflict must be studied and usually eliminated. Here is the
6490proper way to define @code{sequence}:
6491
6492@example
6493sequence: /* empty */
6494 @{ printf ("empty sequence\n"); @}
6495 | sequence word
6496 @{ printf ("added word %s\n", $2); @}
6497 ;
6498@end example
6499
6500Here is another common error that yields a reduce/reduce conflict:
6501
6502@example
6503sequence: /* empty */
6504 | sequence words
6505 | sequence redirects
6506 ;
6507
6508words: /* empty */
6509 | words word
6510 ;
6511
6512redirects:/* empty */
6513 | redirects redirect
6514 ;
6515@end example
6516
6517@noindent
6518The intention here is to define a sequence which can contain either
6519@code{word} or @code{redirect} groupings. The individual definitions of
6520@code{sequence}, @code{words} and @code{redirects} are error-free, but the
6521three together make a subtle ambiguity: even an empty input can be parsed
6522in infinitely many ways!
6523
6524Consider: nothing-at-all could be a @code{words}. Or it could be two
6525@code{words} in a row, or three, or any number. It could equally well be a
6526@code{redirects}, or two, or any number. Or it could be a @code{words}
6527followed by three @code{redirects} and another @code{words}. And so on.
6528
6529Here are two ways to correct these rules. First, to make it a single level
6530of sequence:
6531
6532@example
6533sequence: /* empty */
6534 | sequence word
6535 | sequence redirect
6536 ;
6537@end example
6538
6539Second, to prevent either a @code{words} or a @code{redirects}
6540from being empty:
6541
6542@example
6543sequence: /* empty */
6544 | sequence words
6545 | sequence redirects
6546 ;
6547
6548words: word
6549 | words word
6550 ;
6551
6552redirects:redirect
6553 | redirects redirect
6554 ;
6555@end example
6556
6557@node Mystery Conflicts
6558@section Mysterious Reduce/Reduce Conflicts
6559
6560Sometimes reduce/reduce conflicts can occur that don't look warranted.
6561Here is an example:
6562
6563@example
6564@group
6565%token ID
6566
6567%%
6568def: param_spec return_spec ','
6569 ;
6570param_spec:
6571 type
6572 | name_list ':' type
6573 ;
6574@end group
6575@group
6576return_spec:
6577 type
6578 | name ':' type
6579 ;
6580@end group
6581@group
6582type: ID
6583 ;
6584@end group
6585@group
6586name: ID
6587 ;
6588name_list:
6589 name
6590 | name ',' name_list
6591 ;
6592@end group
6593@end example
6594
6595It would seem that this grammar can be parsed with only a single token
6596of lookahead: when a @code{param_spec} is being read, an @code{ID} is
6597a @code{name} if a comma or colon follows, or a @code{type} if another
6598@code{ID} follows. In other words, this grammar is @acronym{LR}(1).
6599
6600@cindex @acronym{LR}(1)
6601@cindex @acronym{LALR}(1)
6602However, Bison, like most parser generators, cannot actually handle all
6603@acronym{LR}(1) grammars. In this grammar, two contexts, that after
6604an @code{ID}
6605at the beginning of a @code{param_spec} and likewise at the beginning of
6606a @code{return_spec}, are similar enough that Bison assumes they are the
6607same. They appear similar because the same set of rules would be
6608active---the rule for reducing to a @code{name} and that for reducing to
6609a @code{type}. Bison is unable to determine at that stage of processing
6610that the rules would require different lookahead tokens in the two
6611contexts, so it makes a single parser state for them both. Combining
6612the two contexts causes a conflict later. In parser terminology, this
6613occurrence means that the grammar is not @acronym{LALR}(1).
6614
6615In general, it is better to fix deficiencies than to document them. But
6616this particular deficiency is intrinsically hard to fix; parser
6617generators that can handle @acronym{LR}(1) grammars are hard to write
6618and tend to
6619produce parsers that are very large. In practice, Bison is more useful
6620as it is now.
6621
6622When the problem arises, you can often fix it by identifying the two
6623parser states that are being confused, and adding something to make them
6624look distinct. In the above example, adding one rule to
6625@code{return_spec} as follows makes the problem go away:
6626
6627@example
6628@group
6629%token BOGUS
6630@dots{}
6631%%
6632@dots{}
6633return_spec:
6634 type
6635 | name ':' type
6636 /* This rule is never used. */
6637 | ID BOGUS
6638 ;
6639@end group
6640@end example
6641
6642This corrects the problem because it introduces the possibility of an
6643additional active rule in the context after the @code{ID} at the beginning of
6644@code{return_spec}. This rule is not active in the corresponding context
6645in a @code{param_spec}, so the two contexts receive distinct parser states.
6646As long as the token @code{BOGUS} is never generated by @code{yylex},
6647the added rule cannot alter the way actual input is parsed.
6648
6649In this particular example, there is another way to solve the problem:
6650rewrite the rule for @code{return_spec} to use @code{ID} directly
6651instead of via @code{name}. This also causes the two confusing
6652contexts to have different sets of active rules, because the one for
6653@code{return_spec} activates the altered rule for @code{return_spec}
6654rather than the one for @code{name}.
6655
6656@example
6657param_spec:
6658 type
6659 | name_list ':' type
6660 ;
6661return_spec:
6662 type
6663 | ID ':' type
6664 ;
6665@end example
6666
6667For a more detailed exposition of @acronym{LALR}(1) parsers and parser
6668generators, please see:
6669Frank DeRemer and Thomas Pennello, Efficient Computation of
6670@acronym{LALR}(1) Look-Ahead Sets, @cite{@acronym{ACM} Transactions on
6671Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
6672pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
6673
6674@node Generalized LR Parsing
6675@section Generalized @acronym{LR} (@acronym{GLR}) Parsing
6676@cindex @acronym{GLR} parsing
6677@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
6678@cindex ambiguous grammars
6679@cindex nondeterministic parsing
6680
6681Bison produces @emph{deterministic} parsers that choose uniquely
6682when to reduce and which reduction to apply
6683based on a summary of the preceding input and on one extra token of lookahead.
6684As a result, normal Bison handles a proper subset of the family of
6685context-free languages.
6686Ambiguous grammars, since they have strings with more than one possible
6687sequence of reductions cannot have deterministic parsers in this sense.
6688The same is true of languages that require more than one symbol of
6689lookahead, since the parser lacks the information necessary to make a
6690decision at the point it must be made in a shift-reduce parser.
6691Finally, as previously mentioned (@pxref{Mystery Conflicts}),
6692there are languages where Bison's particular choice of how to
6693summarize the input seen so far loses necessary information.
6694
6695When you use the @samp{%glr-parser} declaration in your grammar file,
6696Bison generates a parser that uses a different algorithm, called
6697Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR}
6698parser uses the same basic
6699algorithm for parsing as an ordinary Bison parser, but behaves
6700differently in cases where there is a shift-reduce conflict that has not
6701been resolved by precedence rules (@pxref{Precedence}) or a
6702reduce-reduce conflict. When a @acronym{GLR} parser encounters such a
6703situation, it
6704effectively @emph{splits} into a several parsers, one for each possible
6705shift or reduction. These parsers then proceed as usual, consuming
6706tokens in lock-step. Some of the stacks may encounter other conflicts
6707and split further, with the result that instead of a sequence of states,
6708a Bison @acronym{GLR} parsing stack is what is in effect a tree of states.
6709
6710In effect, each stack represents a guess as to what the proper parse
6711is. Additional input may indicate that a guess was wrong, in which case
6712the appropriate stack silently disappears. Otherwise, the semantics
6713actions generated in each stack are saved, rather than being executed
6714immediately. When a stack disappears, its saved semantic actions never
6715get executed. When a reduction causes two stacks to become equivalent,
6716their sets of semantic actions are both saved with the state that
6717results from the reduction. We say that two stacks are equivalent
6718when they both represent the same sequence of states,
6719and each pair of corresponding states represents a
6720grammar symbol that produces the same segment of the input token
6721stream.
6722
6723Whenever the parser makes a transition from having multiple
6724states to having one, it reverts to the normal @acronym{LALR}(1) parsing
6725algorithm, after resolving and executing the saved-up actions.
6726At this transition, some of the states on the stack will have semantic
6727values that are sets (actually multisets) of possible actions. The
6728parser tries to pick one of the actions by first finding one whose rule
6729has the highest dynamic precedence, as set by the @samp{%dprec}
6730declaration. Otherwise, if the alternative actions are not ordered by
6731precedence, but there the same merging function is declared for both
6732rules by the @samp{%merge} declaration,
6733Bison resolves and evaluates both and then calls the merge function on
6734the result. Otherwise, it reports an ambiguity.
6735
6736It is possible to use a data structure for the @acronym{GLR} parsing tree that
6737permits the processing of any @acronym{LALR}(1) grammar in linear time (in the
6738size of the input), any unambiguous (not necessarily
6739@acronym{LALR}(1)) grammar in
6740quadratic worst-case time, and any general (possibly ambiguous)
6741context-free grammar in cubic worst-case time. However, Bison currently
6742uses a simpler data structure that requires time proportional to the
6743length of the input times the maximum number of stacks required for any
6744prefix of the input. Thus, really ambiguous or nondeterministic
6745grammars can require exponential time and space to process. Such badly
6746behaving examples, however, are not generally of practical interest.
6747Usually, nondeterminism in a grammar is local---the parser is ``in
6748doubt'' only for a few tokens at a time. Therefore, the current data
6749structure should generally be adequate. On @acronym{LALR}(1) portions of a
6750grammar, in particular, it is only slightly slower than with the default
6751Bison parser.
6752
6753For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth
6754Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
6755Generalised @acronym{LR} Parsers, Royal Holloway, University of
6756London, Department of Computer Science, TR-00-12,
6757@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
6758(2000-12-24).
6759
6760@node Memory Management
6761@section Memory Management, and How to Avoid Memory Exhaustion
6762@cindex memory exhaustion
6763@cindex memory management
6764@cindex stack overflow
6765@cindex parser stack overflow
6766@cindex overflow of parser stack
6767
6768The Bison parser stack can run out of memory if too many tokens are shifted and
6769not reduced. When this happens, the parser function @code{yyparse}
6770calls @code{yyerror} and then returns 2.
6771
6772Because Bison parsers have growing stacks, hitting the upper limit
6773usually results from using a right recursion instead of a left
6774recursion, @xref{Recursion, ,Recursive Rules}.
6775
6776@vindex YYMAXDEPTH
6777By defining the macro @code{YYMAXDEPTH}, you can control how deep the
6778parser stack can become before memory is exhausted. Define the
6779macro with a value that is an integer. This value is the maximum number
6780of tokens that can be shifted (and not reduced) before overflow.
6781
6782The stack space allowed is not necessarily allocated. If you specify a
6783large value for @code{YYMAXDEPTH}, the parser normally allocates a small
6784stack at first, and then makes it bigger by stages as needed. This
6785increasing allocation happens automatically and silently. Therefore,
6786you do not need to make @code{YYMAXDEPTH} painfully small merely to save
6787space for ordinary inputs that do not need much stack.
6788
6789However, do not allow @code{YYMAXDEPTH} to be a value so large that
6790arithmetic overflow could occur when calculating the size of the stack
6791space. Also, do not allow @code{YYMAXDEPTH} to be less than
6792@code{YYINITDEPTH}.
6793
6794@cindex default stack limit
6795The default value of @code{YYMAXDEPTH}, if you do not define it, is
679610000.
6797
6798@vindex YYINITDEPTH
6799You can control how much stack is allocated initially by defining the
6800macro @code{YYINITDEPTH} to a positive integer. For the C
6801@acronym{LALR}(1) parser, this value must be a compile-time constant
6802unless you are assuming C99 or some other target language or compiler
6803that allows variable-length arrays. The default is 200.
6804
6805Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
6806
6807@c FIXME: C++ output.
6808Because of semantical differences between C and C++, the
6809@acronym{LALR}(1) parsers in C produced by Bison cannot grow when compiled
6810by C++ compilers. In this precise case (compiling a C parser as C++) you are
6811suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix
6812this deficiency in a future release.
6813
6814@node Error Recovery
6815@chapter Error Recovery
6816@cindex error recovery
6817@cindex recovery from errors
6818
6819It is not usually acceptable to have a program terminate on a syntax
6820error. For example, a compiler should recover sufficiently to parse the
6821rest of the input file and check it for errors; a calculator should accept
6822another expression.
6823
6824In a simple interactive command parser where each input is one line, it may
6825be sufficient to allow @code{yyparse} to return 1 on error and have the
6826caller ignore the rest of the input line when that happens (and then call
6827@code{yyparse} again). But this is inadequate for a compiler, because it
6828forgets all the syntactic context leading up to the error. A syntax error
6829deep within a function in the compiler input should not cause the compiler
6830to treat the following line like the beginning of a source file.
6831
6832@findex error
6833You can define how to recover from a syntax error by writing rules to
6834recognize the special token @code{error}. This is a terminal symbol that
6835is always defined (you need not declare it) and reserved for error
6836handling. The Bison parser generates an @code{error} token whenever a
6837syntax error happens; if you have provided a rule to recognize this token
6838in the current context, the parse can continue.
6839
6840For example:
6841
6842@example
6843stmnts: /* empty string */
6844 | stmnts '\n'
6845 | stmnts exp '\n'
6846 | stmnts error '\n'
6847@end example
6848
6849The fourth rule in this example says that an error followed by a newline
6850makes a valid addition to any @code{stmnts}.
6851
6852What happens if a syntax error occurs in the middle of an @code{exp}? The
6853error recovery rule, interpreted strictly, applies to the precise sequence
6854of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
6855the middle of an @code{exp}, there will probably be some additional tokens
6856and subexpressions on the stack after the last @code{stmnts}, and there
6857will be tokens to read before the next newline. So the rule is not
6858applicable in the ordinary way.
6859
6860But Bison can force the situation to fit the rule, by discarding part of
6861the semantic context and part of the input. First it discards states
6862and objects from the stack until it gets back to a state in which the
6863@code{error} token is acceptable. (This means that the subexpressions
6864already parsed are discarded, back to the last complete @code{stmnts}.)
6865At this point the @code{error} token can be shifted. Then, if the old
6866lookahead token is not acceptable to be shifted next, the parser reads
6867tokens and discards them until it finds a token which is acceptable. In
6868this example, Bison reads and discards input until the next newline so
6869that the fourth rule can apply. Note that discarded symbols are
6870possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
6871Discarded Symbols}, for a means to reclaim this memory.
6872
6873The choice of error rules in the grammar is a choice of strategies for
6874error recovery. A simple and useful strategy is simply to skip the rest of
6875the current input line or current statement if an error is detected:
6876
6877@example
6878stmnt: error ';' /* On error, skip until ';' is read. */
6879@end example
6880
6881It is also useful to recover to the matching close-delimiter of an
6882opening-delimiter that has already been parsed. Otherwise the
6883close-delimiter will probably appear to be unmatched, and generate another,
6884spurious error message:
6885
6886@example
6887primary: '(' expr ')'
6888 | '(' error ')'
6889 @dots{}
6890 ;
6891@end example
6892
6893Error recovery strategies are necessarily guesses. When they guess wrong,
6894one syntax error often leads to another. In the above example, the error
6895recovery rule guesses that an error is due to bad input within one
6896@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
6897middle of a valid @code{stmnt}. After the error recovery rule recovers
6898from the first error, another syntax error will be found straightaway,
6899since the text following the spurious semicolon is also an invalid
6900@code{stmnt}.
6901
6902To prevent an outpouring of error messages, the parser will output no error
6903message for another syntax error that happens shortly after the first; only
6904after three consecutive input tokens have been successfully shifted will
6905error messages resume.
6906
6907Note that rules which accept the @code{error} token may have actions, just
6908as any other rules can.
6909
6910@findex yyerrok
6911You can make error messages resume immediately by using the macro
6912@code{yyerrok} in an action. If you do this in the error rule's action, no
6913error messages will be suppressed. This macro requires no arguments;
6914@samp{yyerrok;} is a valid C statement.
6915
6916@findex yyclearin
6917The previous lookahead token is reanalyzed immediately after an error. If
6918this is unacceptable, then the macro @code{yyclearin} may be used to clear
6919this token. Write the statement @samp{yyclearin;} in the error rule's
6920action.
6921@xref{Action Features, ,Special Features for Use in Actions}.
6922
6923For example, suppose that on a syntax error, an error handling routine is
6924called that advances the input stream to some point where parsing should
6925once again commence. The next symbol returned by the lexical scanner is
6926probably correct. The previous lookahead token ought to be discarded
6927with @samp{yyclearin;}.
6928
6929@vindex YYRECOVERING
6930The expression @code{YYRECOVERING ()} yields 1 when the parser
6931is recovering from a syntax error, and 0 otherwise.
6932Syntax error diagnostics are suppressed while recovering from a syntax
6933error.
6934
6935@node Context Dependency
6936@chapter Handling Context Dependencies
6937
6938The Bison paradigm is to parse tokens first, then group them into larger
6939syntactic units. In many languages, the meaning of a token is affected by
6940its context. Although this violates the Bison paradigm, certain techniques
6941(known as @dfn{kludges}) may enable you to write Bison parsers for such
6942languages.
6943
6944@menu
6945* Semantic Tokens:: Token parsing can depend on the semantic context.
6946* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
6947* Tie-in Recovery:: Lexical tie-ins have implications for how
6948 error recovery rules must be written.
6949@end menu
6950
6951(Actually, ``kludge'' means any technique that gets its job done but is
6952neither clean nor robust.)
6953
6954@node Semantic Tokens
6955@section Semantic Info in Token Types
6956
6957The C language has a context dependency: the way an identifier is used
6958depends on what its current meaning is. For example, consider this:
6959
6960@example
6961foo (x);
6962@end example
6963
6964This looks like a function call statement, but if @code{foo} is a typedef
6965name, then this is actually a declaration of @code{x}. How can a Bison
6966parser for C decide how to parse this input?
6967
6968The method used in @acronym{GNU} C is to have two different token types,
6969@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
6970identifier, it looks up the current declaration of the identifier in order
6971to decide which token type to return: @code{TYPENAME} if the identifier is
6972declared as a typedef, @code{IDENTIFIER} otherwise.
6973
6974The grammar rules can then express the context dependency by the choice of
6975token type to recognize. @code{IDENTIFIER} is accepted as an expression,
6976but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
6977@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
6978is @emph{not} significant, such as in declarations that can shadow a
6979typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
6980accepted---there is one rule for each of the two token types.
6981
6982This technique is simple to use if the decision of which kinds of
6983identifiers to allow is made at a place close to where the identifier is
6984parsed. But in C this is not always so: C allows a declaration to
6985redeclare a typedef name provided an explicit type has been specified
6986earlier:
6987
6988@example
6989typedef int foo, bar;
6990int baz (void)
6991@{
6992 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
6993 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
6994 return foo (bar);
6995@}
6996@end example
6997
6998Unfortunately, the name being declared is separated from the declaration
6999construct itself by a complicated syntactic structure---the ``declarator''.
7000
7001As a result, part of the Bison parser for C needs to be duplicated, with
7002all the nonterminal names changed: once for parsing a declaration in
7003which a typedef name can be redefined, and once for parsing a
7004declaration in which that can't be done. Here is a part of the
7005duplication, with actions omitted for brevity:
7006
7007@example
7008initdcl:
7009 declarator maybeasm '='
7010 init
7011 | declarator maybeasm
7012 ;
7013
7014notype_initdcl:
7015 notype_declarator maybeasm '='
7016 init
7017 | notype_declarator maybeasm
7018 ;
7019@end example
7020
7021@noindent
7022Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
7023cannot. The distinction between @code{declarator} and
7024@code{notype_declarator} is the same sort of thing.
7025
7026There is some similarity between this technique and a lexical tie-in
7027(described next), in that information which alters the lexical analysis is
7028changed during parsing by other parts of the program. The difference is
7029here the information is global, and is used for other purposes in the
7030program. A true lexical tie-in has a special-purpose flag controlled by
7031the syntactic context.
7032
7033@node Lexical Tie-ins
7034@section Lexical Tie-ins
7035@cindex lexical tie-in
7036
7037One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
7038which is set by Bison actions, whose purpose is to alter the way tokens are
7039parsed.
7040
7041For example, suppose we have a language vaguely like C, but with a special
7042construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
7043an expression in parentheses in which all integers are hexadecimal. In
7044particular, the token @samp{a1b} must be treated as an integer rather than
7045as an identifier if it appears in that context. Here is how you can do it:
7046
7047@example
7048@group
7049%@{
7050 int hexflag;
7051 int yylex (void);
7052 void yyerror (char const *);
7053%@}
7054%%
7055@dots{}
7056@end group
7057@group
7058expr: IDENTIFIER
7059 | constant
7060 | HEX '('
7061 @{ hexflag = 1; @}
7062 expr ')'
7063 @{ hexflag = 0;
7064 $$ = $4; @}
7065 | expr '+' expr
7066 @{ $$ = make_sum ($1, $3); @}
7067 @dots{}
7068 ;
7069@end group
7070
7071@group
7072constant:
7073 INTEGER
7074 | STRING
7075 ;
7076@end group
7077@end example
7078
7079@noindent
7080Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
7081it is nonzero, all integers are parsed in hexadecimal, and tokens starting
7082with letters are parsed as integers if possible.
7083
7084The declaration of @code{hexflag} shown in the prologue of the parser file
7085is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}).
7086You must also write the code in @code{yylex} to obey the flag.
7087
7088@node Tie-in Recovery
7089@section Lexical Tie-ins and Error Recovery
7090
7091Lexical tie-ins make strict demands on any error recovery rules you have.
7092@xref{Error Recovery}.
7093
7094The reason for this is that the purpose of an error recovery rule is to
7095abort the parsing of one construct and resume in some larger construct.
7096For example, in C-like languages, a typical error recovery rule is to skip
7097tokens until the next semicolon, and then start a new statement, like this:
7098
7099@example
7100stmt: expr ';'
7101 | IF '(' expr ')' stmt @{ @dots{} @}
7102 @dots{}
7103 error ';'
7104 @{ hexflag = 0; @}
7105 ;
7106@end example
7107
7108If there is a syntax error in the middle of a @samp{hex (@var{expr})}
7109construct, this error rule will apply, and then the action for the
7110completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
7111remain set for the entire rest of the input, or until the next @code{hex}
7112keyword, causing identifiers to be misinterpreted as integers.
7113
7114To avoid this problem the error recovery rule itself clears @code{hexflag}.
7115
7116There may also be an error recovery rule that works within expressions.
7117For example, there could be a rule which applies within parentheses
7118and skips to the close-parenthesis:
7119
7120@example
7121@group
7122expr: @dots{}
7123 | '(' expr ')'
7124 @{ $$ = $2; @}
7125 | '(' error ')'
7126 @dots{}
7127@end group
7128@end example
7129
7130If this rule acts within the @code{hex} construct, it is not going to abort
7131that construct (since it applies to an inner level of parentheses within
7132the construct). Therefore, it should not clear the flag: the rest of
7133the @code{hex} construct should be parsed with the flag still in effect.
7134
7135What if there is an error recovery rule which might abort out of the
7136@code{hex} construct or might not, depending on circumstances? There is no
7137way you can write the action to determine whether a @code{hex} construct is
7138being aborted or not. So if you are using a lexical tie-in, you had better
7139make sure your error recovery rules are not of this kind. Each rule must
7140be such that you can be sure that it always will, or always won't, have to
7141clear the flag.
7142
7143@c ================================================== Debugging Your Parser
7144
7145@node Debugging
7146@chapter Debugging Your Parser
7147
7148Developing a parser can be a challenge, especially if you don't
7149understand the algorithm (@pxref{Algorithm, ,The Bison Parser
7150Algorithm}). Even so, sometimes a detailed description of the automaton
7151can help (@pxref{Understanding, , Understanding Your Parser}), or
7152tracing the execution of the parser can give some insight on why it
7153behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
7154
7155@menu
7156* Understanding:: Understanding the structure of your parser.
7157* Tracing:: Tracing the execution of your parser.
7158@end menu
7159
7160@node Understanding
7161@section Understanding Your Parser
7162
7163As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
7164Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
7165frequent than one would hope), looking at this automaton is required to
7166tune or simply fix a parser. Bison provides two different
7167representation of it, either textually or graphically (as a DOT file).
7168
7169The textual file is generated when the options @option{--report} or
7170@option{--verbose} are specified, see @xref{Invocation, , Invoking
7171Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
7172the parser output file name, and adding @samp{.output} instead.
7173Therefore, if the input file is @file{foo.y}, then the parser file is
7174called @file{foo.tab.c} by default. As a consequence, the verbose
7175output file is called @file{foo.output}.
7176
7177The following grammar file, @file{calc.y}, will be used in the sequel:
7178
7179@example
7180%token NUM STR
7181%left '+' '-'
7182%left '*'
7183%%
7184exp: exp '+' exp
7185 | exp '-' exp
7186 | exp '*' exp
7187 | exp '/' exp
7188 | NUM
7189 ;
7190useless: STR;
7191%%
7192@end example
7193
7194@command{bison} reports:
7195
7196@example
7197calc.y: warning: 1 nonterminal and 1 rule useless in grammar
7198calc.y:11.1-7: warning: nonterminal useless in grammar: useless
7199calc.y:11.10-12: warning: rule useless in grammar: useless: STR
7200calc.y: conflicts: 7 shift/reduce
7201@end example
7202
7203When given @option{--report=state}, in addition to @file{calc.tab.c}, it
7204creates a file @file{calc.output} with contents detailed below. The
7205order of the output and the exact presentation might vary, but the
7206interpretation is the same.
7207
7208The first section includes details on conflicts that were solved thanks
7209to precedence and/or associativity:
7210
7211@example
7212Conflict in state 8 between rule 2 and token '+' resolved as reduce.
7213Conflict in state 8 between rule 2 and token '-' resolved as reduce.
7214Conflict in state 8 between rule 2 and token '*' resolved as shift.
7215@exdent @dots{}
7216@end example
7217
7218@noindent
7219The next section lists states that still have conflicts.
7220
7221@example
7222State 8 conflicts: 1 shift/reduce
7223State 9 conflicts: 1 shift/reduce
7224State 10 conflicts: 1 shift/reduce
7225State 11 conflicts: 4 shift/reduce
7226@end example
7227
7228@noindent
7229@cindex token, useless
7230@cindex useless token
7231@cindex nonterminal, useless
7232@cindex useless nonterminal
7233@cindex rule, useless
7234@cindex useless rule
7235The next section reports useless tokens, nonterminal and rules. Useless
7236nonterminals and rules are removed in order to produce a smaller parser,
7237but useless tokens are preserved, since they might be used by the
7238scanner (note the difference between ``useless'' and ``unused''
7239below):
7240
7241@example
7242Nonterminals useless in grammar:
7243 useless
7244
7245Terminals unused in grammar:
7246 STR
7247
7248Rules useless in grammar:
7249#6 useless: STR;
7250@end example
7251
7252@noindent
7253The next section reproduces the exact grammar that Bison used:
7254
7255@example
7256Grammar
7257
7258 Number, Line, Rule
7259 0 5 $accept -> exp $end
7260 1 5 exp -> exp '+' exp
7261 2 6 exp -> exp '-' exp
7262 3 7 exp -> exp '*' exp
7263 4 8 exp -> exp '/' exp
7264 5 9 exp -> NUM
7265@end example
7266
7267@noindent
7268and reports the uses of the symbols:
7269
7270@example
7271Terminals, with rules where they appear
7272
7273$end (0) 0
7274'*' (42) 3
7275'+' (43) 1
7276'-' (45) 2
7277'/' (47) 4
7278error (256)
7279NUM (258) 5
7280
7281Nonterminals, with rules where they appear
7282
7283$accept (8)
7284 on left: 0
7285exp (9)
7286 on left: 1 2 3 4 5, on right: 0 1 2 3 4
7287@end example
7288
7289@noindent
7290@cindex item
7291@cindex pointed rule
7292@cindex rule, pointed
7293Bison then proceeds onto the automaton itself, describing each state
7294with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
7295item is a production rule together with a point (marked by @samp{.})
7296that the input cursor.
7297
7298@example
7299state 0
7300
7301 $accept -> . exp $ (rule 0)
7302
7303 NUM shift, and go to state 1
7304
7305 exp go to state 2
7306@end example
7307
7308This reads as follows: ``state 0 corresponds to being at the very
7309beginning of the parsing, in the initial rule, right before the start
7310symbol (here, @code{exp}). When the parser returns to this state right
7311after having reduced a rule that produced an @code{exp}, the control
7312flow jumps to state 2. If there is no such transition on a nonterminal
7313symbol, and the lookahead is a @code{NUM}, then this token is shifted on
7314the parse stack, and the control flow jumps to state 1. Any other
7315lookahead triggers a syntax error.''
7316
7317@cindex core, item set
7318@cindex item set core
7319@cindex kernel, item set
7320@cindex item set core
7321Even though the only active rule in state 0 seems to be rule 0, the
7322report lists @code{NUM} as a lookahead token because @code{NUM} can be
7323at the beginning of any rule deriving an @code{exp}. By default Bison
7324reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
7325you want to see more detail you can invoke @command{bison} with
7326@option{--report=itemset} to list all the items, include those that can
7327be derived:
7328
7329@example
7330state 0
7331
7332 $accept -> . exp $ (rule 0)
7333 exp -> . exp '+' exp (rule 1)
7334 exp -> . exp '-' exp (rule 2)
7335 exp -> . exp '*' exp (rule 3)
7336 exp -> . exp '/' exp (rule 4)
7337 exp -> . NUM (rule 5)
7338
7339 NUM shift, and go to state 1
7340
7341 exp go to state 2
7342@end example
7343
7344@noindent
7345In the state 1...
7346
7347@example
7348state 1
7349
7350 exp -> NUM . (rule 5)
7351
7352 $default reduce using rule 5 (exp)
7353@end example
7354
7355@noindent
7356the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
7357(@samp{$default}), the parser will reduce it. If it was coming from
7358state 0, then, after this reduction it will return to state 0, and will
7359jump to state 2 (@samp{exp: go to state 2}).
7360
7361@example
7362state 2
7363
7364 $accept -> exp . $ (rule 0)
7365 exp -> exp . '+' exp (rule 1)
7366 exp -> exp . '-' exp (rule 2)
7367 exp -> exp . '*' exp (rule 3)
7368 exp -> exp . '/' exp (rule 4)
7369
7370 $ shift, and go to state 3
7371 '+' shift, and go to state 4
7372 '-' shift, and go to state 5
7373 '*' shift, and go to state 6
7374 '/' shift, and go to state 7
7375@end example
7376
7377@noindent
7378In state 2, the automaton can only shift a symbol. For instance,
7379because of the item @samp{exp -> exp . '+' exp}, if the lookahead if
7380@samp{+}, it will be shifted on the parse stack, and the automaton
7381control will jump to state 4, corresponding to the item @samp{exp -> exp
7382'+' . exp}. Since there is no default action, any other token than
7383those listed above will trigger a syntax error.
7384
7385The state 3 is named the @dfn{final state}, or the @dfn{accepting
7386state}:
7387
7388@example
7389state 3
7390
7391 $accept -> exp $ . (rule 0)
7392
7393 $default accept
7394@end example
7395
7396@noindent
7397the initial rule is completed (the start symbol and the end
7398of input were read), the parsing exits successfully.
7399
7400The interpretation of states 4 to 7 is straightforward, and is left to
7401the reader.
7402
7403@example
7404state 4
7405
7406 exp -> exp '+' . exp (rule 1)
7407
7408 NUM shift, and go to state 1
7409
7410 exp go to state 8
7411
7412state 5
7413
7414 exp -> exp '-' . exp (rule 2)
7415
7416 NUM shift, and go to state 1
7417
7418 exp go to state 9
7419
7420state 6
7421
7422 exp -> exp '*' . exp (rule 3)
7423
7424 NUM shift, and go to state 1
7425
7426 exp go to state 10
7427
7428state 7
7429
7430 exp -> exp '/' . exp (rule 4)
7431
7432 NUM shift, and go to state 1
7433
7434 exp go to state 11
7435@end example
7436
7437As was announced in beginning of the report, @samp{State 8 conflicts:
74381 shift/reduce}:
7439
7440@example
7441state 8
7442
7443 exp -> exp . '+' exp (rule 1)
7444 exp -> exp '+' exp . (rule 1)
7445 exp -> exp . '-' exp (rule 2)
7446 exp -> exp . '*' exp (rule 3)
7447 exp -> exp . '/' exp (rule 4)
7448
7449 '*' shift, and go to state 6
7450 '/' shift, and go to state 7
7451
7452 '/' [reduce using rule 1 (exp)]
7453 $default reduce using rule 1 (exp)
7454@end example
7455
7456Indeed, there are two actions associated to the lookahead @samp{/}:
7457either shifting (and going to state 7), or reducing rule 1. The
7458conflict means that either the grammar is ambiguous, or the parser lacks
7459information to make the right decision. Indeed the grammar is
7460ambiguous, as, since we did not specify the precedence of @samp{/}, the
7461sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
7462NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
7463NUM}, which corresponds to reducing rule 1.
7464
7465Because in @acronym{LALR}(1) parsing a single decision can be made, Bison
7466arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
7467Shift/Reduce Conflicts}. Discarded actions are reported in between
7468square brackets.
7469
7470Note that all the previous states had a single possible action: either
7471shifting the next token and going to the corresponding state, or
7472reducing a single rule. In the other cases, i.e., when shifting
7473@emph{and} reducing is possible or when @emph{several} reductions are
7474possible, the lookahead is required to select the action. State 8 is
7475one such state: if the lookahead is @samp{*} or @samp{/} then the action
7476is shifting, otherwise the action is reducing rule 1. In other words,
7477the first two items, corresponding to rule 1, are not eligible when the
7478lookahead token is @samp{*}, since we specified that @samp{*} has higher
7479precedence than @samp{+}. More generally, some items are eligible only
7480with some set of possible lookahead tokens. When run with
7481@option{--report=lookahead}, Bison specifies these lookahead tokens:
7482
7483@example
7484state 8
7485
7486 exp -> exp . '+' exp (rule 1)
7487 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
7488 exp -> exp . '-' exp (rule 2)
7489 exp -> exp . '*' exp (rule 3)
7490 exp -> exp . '/' exp (rule 4)
7491
7492 '*' shift, and go to state 6
7493 '/' shift, and go to state 7
7494
7495 '/' [reduce using rule 1 (exp)]
7496 $default reduce using rule 1 (exp)
7497@end example
7498
7499The remaining states are similar:
7500
7501@example
7502state 9
7503
7504 exp -> exp . '+' exp (rule 1)
7505 exp -> exp . '-' exp (rule 2)
7506 exp -> exp '-' exp . (rule 2)
7507 exp -> exp . '*' exp (rule 3)
7508 exp -> exp . '/' exp (rule 4)
7509
7510 '*' shift, and go to state 6
7511 '/' shift, and go to state 7
7512
7513 '/' [reduce using rule 2 (exp)]
7514 $default reduce using rule 2 (exp)
7515
7516state 10
7517
7518 exp -> exp . '+' exp (rule 1)
7519 exp -> exp . '-' exp (rule 2)
7520 exp -> exp . '*' exp (rule 3)
7521 exp -> exp '*' exp . (rule 3)
7522 exp -> exp . '/' exp (rule 4)
7523
7524 '/' shift, and go to state 7
7525
7526 '/' [reduce using rule 3 (exp)]
7527 $default reduce using rule 3 (exp)
7528
7529state 11
7530
7531 exp -> exp . '+' exp (rule 1)
7532 exp -> exp . '-' exp (rule 2)
7533 exp -> exp . '*' exp (rule 3)
7534 exp -> exp . '/' exp (rule 4)
7535 exp -> exp '/' exp . (rule 4)
7536
7537 '+' shift, and go to state 4
7538 '-' shift, and go to state 5
7539 '*' shift, and go to state 6
7540 '/' shift, and go to state 7
7541
7542 '+' [reduce using rule 4 (exp)]
7543 '-' [reduce using rule 4 (exp)]
7544 '*' [reduce using rule 4 (exp)]
7545 '/' [reduce using rule 4 (exp)]
7546 $default reduce using rule 4 (exp)
7547@end example
7548
7549@noindent
7550Observe that state 11 contains conflicts not only due to the lack of
7551precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
7552@samp{*}, but also because the
7553associativity of @samp{/} is not specified.
7554
7555
7556@node Tracing
7557@section Tracing Your Parser
7558@findex yydebug
7559@cindex debugging
7560@cindex tracing the parser
7561
7562If a Bison grammar compiles properly but doesn't do what you want when it
7563runs, the @code{yydebug} parser-trace feature can help you figure out why.
7564
7565There are several means to enable compilation of trace facilities:
7566
7567@table @asis
7568@item the macro @code{YYDEBUG}
7569@findex YYDEBUG
7570Define the macro @code{YYDEBUG} to a nonzero value when you compile the
7571parser. This is compliant with @acronym{POSIX} Yacc. You could use
7572@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
7573YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
7574Prologue}).
7575
7576@item the option @option{-t}, @option{--debug}
7577Use the @samp{-t} option when you run Bison (@pxref{Invocation,
7578,Invoking Bison}). This is @acronym{POSIX} compliant too.
7579
7580@item the directive @samp{%debug}
7581@findex %debug
7582Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison
7583Declaration Summary}). This is a Bison extension, which will prove
7584useful when Bison will output parsers for languages that don't use a
7585preprocessor. Unless @acronym{POSIX} and Yacc portability matter to
7586you, this is
7587the preferred solution.
7588@end table
7589
7590We suggest that you always enable the debug option so that debugging is
7591always possible.
7592
7593The trace facility outputs messages with macro calls of the form
7594@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
7595@var{format} and @var{args} are the usual @code{printf} format and variadic
7596arguments. If you define @code{YYDEBUG} to a nonzero value but do not
7597define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
7598and @code{YYFPRINTF} is defined to @code{fprintf}.
7599
7600Once you have compiled the program with trace facilities, the way to
7601request a trace is to store a nonzero value in the variable @code{yydebug}.
7602You can do this by making the C code do it (in @code{main}, perhaps), or
7603you can alter the value with a C debugger.
7604
7605Each step taken by the parser when @code{yydebug} is nonzero produces a
7606line or two of trace information, written on @code{stderr}. The trace
7607messages tell you these things:
7608
7609@itemize @bullet
7610@item
7611Each time the parser calls @code{yylex}, what kind of token was read.
7612
7613@item
7614Each time a token is shifted, the depth and complete contents of the
7615state stack (@pxref{Parser States}).
7616
7617@item
7618Each time a rule is reduced, which rule it is, and the complete contents
7619of the state stack afterward.
7620@end itemize
7621
7622To make sense of this information, it helps to refer to the listing file
7623produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
7624Bison}). This file shows the meaning of each state in terms of
7625positions in various rules, and also what each state will do with each
7626possible input token. As you read the successive trace messages, you
7627can see that the parser is functioning according to its specification in
7628the listing file. Eventually you will arrive at the place where
7629something undesirable happens, and you will see which parts of the
7630grammar are to blame.
7631
7632The parser file is a C program and you can use C debuggers on it, but it's
7633not easy to interpret what it is doing. The parser function is a
7634finite-state machine interpreter, and aside from the actions it executes
7635the same code over and over. Only the values of variables show where in
7636the grammar it is working.
7637
7638@findex YYPRINT
7639The debugging information normally gives the token type of each token
7640read, but not its semantic value. You can optionally define a macro
7641named @code{YYPRINT} to provide a way to print the value. If you define
7642@code{YYPRINT}, it should take three arguments. The parser will pass a
7643standard I/O stream, the numeric code for the token type, and the token
7644value (from @code{yylval}).
7645
7646Here is an example of @code{YYPRINT} suitable for the multi-function
7647calculator (@pxref{Mfcalc Declarations, ,Declarations for @code{mfcalc}}):
7648
7649@smallexample
7650%@{
7651 static void print_token_value (FILE *, int, YYSTYPE);
7652 #define YYPRINT(file, type, value) print_token_value (file, type, value)
7653%@}
7654
7655@dots{} %% @dots{} %% @dots{}
7656
7657static void
7658print_token_value (FILE *file, int type, YYSTYPE value)
7659@{
7660 if (type == VAR)
7661 fprintf (file, "%s", value.tptr->name);
7662 else if (type == NUM)
7663 fprintf (file, "%d", value.val);
7664@}
7665@end smallexample
7666
7667@c ================================================= Invoking Bison
7668
7669@node Invocation
7670@chapter Invoking Bison
7671@cindex invoking Bison
7672@cindex Bison invocation
7673@cindex options for invoking Bison
7674
7675The usual way to invoke Bison is as follows:
7676
7677@example
7678bison @var{infile}
7679@end example
7680
7681Here @var{infile} is the grammar file name, which usually ends in
7682@samp{.y}. The parser file's name is made by replacing the @samp{.y}
7683with @samp{.tab.c} and removing any leading directory. Thus, the
7684@samp{bison foo.y} file name yields
7685@file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields
7686@file{foo.tab.c}. It's also possible, in case you are writing
7687C++ code instead of C in your grammar file, to name it @file{foo.ypp}
7688or @file{foo.y++}. Then, the output files will take an extension like
7689the given one as input (respectively @file{foo.tab.cpp} and
7690@file{foo.tab.c++}).
7691This feature takes effect with all options that manipulate file names like
7692@samp{-o} or @samp{-d}.
7693
7694For example :
7695
7696@example
7697bison -d @var{infile.yxx}
7698@end example
7699@noindent
7700will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
7701
7702@example
7703bison -d -o @var{output.c++} @var{infile.y}
7704@end example
7705@noindent
7706will produce @file{output.c++} and @file{outfile.h++}.
7707
7708For compatibility with @acronym{POSIX}, the standard Bison
7709distribution also contains a shell script called @command{yacc} that
7710invokes Bison with the @option{-y} option.
7711
7712@menu
7713* Bison Options:: All the options described in detail,
7714 in alphabetical order by short options.
7715* Option Cross Key:: Alphabetical list of long options.
7716* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
7717@end menu
7718
7719@node Bison Options
7720@section Bison Options
7721
7722Bison supports both traditional single-letter options and mnemonic long
7723option names. Long option names are indicated with @samp{--} instead of
7724@samp{-}. Abbreviations for option names are allowed as long as they
7725are unique. When a long option takes an argument, like
7726@samp{--file-prefix}, connect the option name and the argument with
7727@samp{=}.
7728
7729Here is a list of options that can be used with Bison, alphabetized by
7730short option. It is followed by a cross key alphabetized by long
7731option.
7732
7733@c Please, keep this ordered as in `bison --help'.
7734@noindent
7735Operations modes:
7736@table @option
7737@item -h
7738@itemx --help
7739Print a summary of the command-line options to Bison and exit.
7740
7741@item -V
7742@itemx --version
7743Print the version number of Bison and exit.
7744
7745@item --print-localedir
7746Print the name of the directory containing locale-dependent data.
7747
7748@item --print-datadir
7749Print the name of the directory containing skeletons and XSLT.
7750
7751@item -y
7752@itemx --yacc
7753Act more like the traditional Yacc command. This can cause
7754different diagnostics to be generated, and may change behavior in
7755other minor ways. Most importantly, imitate Yacc's output
7756file name conventions, so that the parser output file is called
7757@file{y.tab.c}, and the other outputs are called @file{y.output} and
7758@file{y.tab.h}.
7759Also, if generating an @acronym{LALR}(1) parser in C, generate @code{#define}
7760statements in addition to an @code{enum} to associate token numbers with token
7761names.
7762Thus, the following shell script can substitute for Yacc, and the Bison
7763distribution contains such a script for compatibility with @acronym{POSIX}:
7764
7765@example
7766#! /bin/sh
7767bison -y "$@@"
7768@end example
7769
7770The @option{-y}/@option{--yacc} option is intended for use with
7771traditional Yacc grammars. If your grammar uses a Bison extension
7772like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
7773this option is specified.
7774
7775@item -W [@var{category}]
7776@itemx --warnings[=@var{category}]
7777Output warnings falling in @var{category}. @var{category} can be one
7778of:
7779@table @code
7780@item midrule-values
7781Warn about mid-rule values that are set but not used within any of the actions
7782of the parent rule.
7783For example, warn about unused @code{$2} in:
7784
7785@example
7786exp: '1' @{ $$ = 1; @} '+' exp @{ $$ = $1 + $4; @};
7787@end example
7788
7789Also warn about mid-rule values that are used but not set.
7790For example, warn about unset @code{$$} in the mid-rule action in:
7791
7792@example
7793 exp: '1' @{ $1 = 1; @} '+' exp @{ $$ = $2 + $4; @};
7794@end example
7795
7796These warnings are not enabled by default since they sometimes prove to
7797be false alarms in existing grammars employing the Yacc constructs
7798@code{$0} or @code{$-@var{n}} (where @var{n} is some positive integer).
7799
7800
7801@item yacc
7802Incompatibilities with @acronym{POSIX} Yacc.
7803
7804@item all
7805All the warnings.
7806@item none
7807Turn off all the warnings.
7808@item error
7809Treat warnings as errors.
7810@end table
7811
7812A category can be turned off by prefixing its name with @samp{no-}. For
7813instance, @option{-Wno-syntax} will hide the warnings about unused
7814variables.
7815@end table
7816
7817@noindent
7818Tuning the parser:
7819
7820@table @option
7821@item -t
7822@itemx --debug
7823In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
7824already defined, so that the debugging facilities are compiled.
7825@xref{Tracing, ,Tracing Your Parser}.
7826
7827@item -L @var{language}
7828@itemx --language=@var{language}
7829Specify the programming language for the generated parser, as if
7830@code{%language} was specified (@pxref{Decl Summary, , Bison Declaration
7831Summary}). Currently supported languages include C, C++, and Java.
7832@var{language} is case-insensitive.
7833
7834This option is experimental and its effect may be modified in future
7835releases.
7836
7837@item --locations
7838Pretend that @code{%locations} was specified. @xref{Decl Summary}.
7839
7840@item -p @var{prefix}
7841@itemx --name-prefix=@var{prefix}
7842Pretend that @code{%name-prefix "@var{prefix}"} was specified.
7843@xref{Decl Summary}.
7844
7845@item -l
7846@itemx --no-lines
7847Don't put any @code{#line} preprocessor commands in the parser file.
7848Ordinarily Bison puts them in the parser file so that the C compiler
7849and debuggers will associate errors with your source file, the
7850grammar file. This option causes them to associate errors with the
7851parser file, treating it as an independent source file in its own right.
7852
7853@item -S @var{file}
7854@itemx --skeleton=@var{file}
7855Specify the skeleton to use, similar to @code{%skeleton}
7856(@pxref{Decl Summary, , Bison Declaration Summary}).
7857
7858@c You probably don't need this option unless you are developing Bison.
7859@c You should use @option{--language} if you want to specify the skeleton for a
7860@c different language, because it is clearer and because it will always
7861@c choose the correct skeleton for non-deterministic or push parsers.
7862
7863If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
7864file in the Bison installation directory.
7865If it does, @var{file} is an absolute file name or a file name relative to the
7866current working directory.
7867This is similar to how most shells resolve commands.
7868
7869@item -k
7870@itemx --token-table
7871Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
7872@end table
7873
7874@noindent
7875Adjust the output:
7876
7877@table @option
7878@item --defines[=@var{file}]
7879Pretend that @code{%defines} was specified, i.e., write an extra output
7880file containing macro definitions for the token type names defined in
7881the grammar, as well as a few other declarations. @xref{Decl Summary}.
7882
7883@item -d
7884This is the same as @code{--defines} except @code{-d} does not accept a
7885@var{file} argument since POSIX Yacc requires that @code{-d} can be bundled
7886with other short options.
7887
7888@item -b @var{file-prefix}
7889@itemx --file-prefix=@var{prefix}
7890Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
7891for all Bison output file names. @xref{Decl Summary}.
7892
7893@item -r @var{things}
7894@itemx --report=@var{things}
7895Write an extra output file containing verbose description of the comma
7896separated list of @var{things} among:
7897
7898@table @code
7899@item state
7900Description of the grammar, conflicts (resolved and unresolved), and
7901@acronym{LALR} automaton.
7902
7903@item lookahead
7904Implies @code{state} and augments the description of the automaton with
7905each rule's lookahead set.
7906
7907@item itemset
7908Implies @code{state} and augments the description of the automaton with
7909the full set of items for each state, instead of its core only.
7910@end table
7911
7912@item --report-file=@var{file}
7913Specify the @var{file} for the verbose description.
7914
7915@item -v
7916@itemx --verbose
7917Pretend that @code{%verbose} was specified, i.e., write an extra output
7918file containing verbose descriptions of the grammar and
7919parser. @xref{Decl Summary}.
7920
7921@item -o @var{file}
7922@itemx --output=@var{file}
7923Specify the @var{file} for the parser file.
7924
7925The other output files' names are constructed from @var{file} as
7926described under the @samp{-v} and @samp{-d} options.
7927
7928@item -g[@var{file}]
7929@itemx --graph[=@var{file}]
7930Output a graphical representation of the @acronym{LALR}(1) grammar
7931automaton computed by Bison, in @uref{http://www.graphviz.org/, Graphviz}
7932@uref{http://www.graphviz.org/doc/info/lang.html, @acronym{DOT}} format.
7933@code{@var{file}} is optional.
7934If omitted and the grammar file is @file{foo.y}, the output file will be
7935@file{foo.dot}.
7936
7937@item -x[@var{file}]
7938@itemx --xml[=@var{file}]
7939Output an XML report of the @acronym{LALR}(1) automaton computed by Bison.
7940@code{@var{file}} is optional.
7941If omitted and the grammar file is @file{foo.y}, the output file will be
7942@file{foo.xml}.
7943(The current XML schema is experimental and may evolve.
7944More user feedback will help to stabilize it.)
7945@end table
7946
7947@node Option Cross Key
7948@section Option Cross Key
7949
7950@c FIXME: How about putting the directives too?
7951Here is a list of options, alphabetized by long option, to help you find
7952the corresponding short option.
7953
7954@multitable {@option{--defines=@var{defines-file}}} {@option{-b @var{file-prefix}XXX}}
7955@headitem Long Option @tab Short Option
7956@include cross-options.texi
7957@end multitable
7958
7959@node Yacc Library
7960@section Yacc Library
7961
7962The Yacc library contains default implementations of the
7963@code{yyerror} and @code{main} functions. These default
7964implementations are normally not useful, but @acronym{POSIX} requires
7965them. To use the Yacc library, link your program with the
7966@option{-ly} option. Note that Bison's implementation of the Yacc
7967library is distributed under the terms of the @acronym{GNU} General
7968Public License (@pxref{Copying}).
7969
7970If you use the Yacc library's @code{yyerror} function, you should
7971declare @code{yyerror} as follows:
7972
7973@example
7974int yyerror (char const *);
7975@end example
7976
7977Bison ignores the @code{int} value returned by this @code{yyerror}.
7978If you use the Yacc library's @code{main} function, your
7979@code{yyparse} function should have the following type signature:
7980
7981@example
7982int yyparse (void);
7983@end example
7984
7985@c ================================================= C++ Bison
7986
7987@node Other Languages
7988@chapter Parsers Written In Other Languages
7989
7990@menu
7991* C++ Parsers:: The interface to generate C++ parser classes
7992* Java Parsers:: The interface to generate Java parser classes
7993@end menu
7994
7995@node C++ Parsers
7996@section C++ Parsers
7997
7998@menu
7999* C++ Bison Interface:: Asking for C++ parser generation
8000* C++ Semantic Values:: %union vs. C++
8001* C++ Location Values:: The position and location classes
8002* C++ Parser Interface:: Instantiating and running the parser
8003* C++ Scanner Interface:: Exchanges between yylex and parse
8004* A Complete C++ Example:: Demonstrating their use
8005@end menu
8006
8007@node C++ Bison Interface
8008@subsection C++ Bison Interface
8009@c - %skeleton "lalr1.cc"
8010@c - Always pure
8011@c - initial action
8012
8013The C++ @acronym{LALR}(1) parser is selected using the skeleton directive,
8014@samp{%skeleton "lalr1.c"}, or the synonymous command-line option
8015@option{--skeleton=lalr1.c}.
8016@xref{Decl Summary}.
8017
8018When run, @command{bison} will create several entities in the @samp{yy}
8019namespace.
8020@findex %define namespace
8021Use the @samp{%define namespace} directive to change the namespace name, see
8022@ref{Decl Summary}.
8023The various classes are generated in the following files:
8024
8025@table @file
8026@item position.hh
8027@itemx location.hh
8028The definition of the classes @code{position} and @code{location},
8029used for location tracking. @xref{C++ Location Values}.
8030
8031@item stack.hh
8032An auxiliary class @code{stack} used by the parser.
8033
8034@item @var{file}.hh
8035@itemx @var{file}.cc
8036(Assuming the extension of the input file was @samp{.yy}.) The
8037declaration and implementation of the C++ parser class. The basename
8038and extension of these two files follow the same rules as with regular C
8039parsers (@pxref{Invocation}).
8040
8041The header is @emph{mandatory}; you must either pass
8042@option{-d}/@option{--defines} to @command{bison}, or use the
8043@samp{%defines} directive.
8044@end table
8045
8046All these files are documented using Doxygen; run @command{doxygen}
8047for a complete and accurate documentation.
8048
8049@node C++ Semantic Values
8050@subsection C++ Semantic Values
8051@c - No objects in unions
8052@c - YYSTYPE
8053@c - Printer and destructor
8054
8055The @code{%union} directive works as for C, see @ref{Union Decl, ,The
8056Collection of Value Types}. In particular it produces a genuine
8057@code{union}@footnote{In the future techniques to allow complex types
8058within pseudo-unions (similar to Boost variants) might be implemented to
8059alleviate these issues.}, which have a few specific features in C++.
8060@itemize @minus
8061@item
8062The type @code{YYSTYPE} is defined but its use is discouraged: rather
8063you should refer to the parser's encapsulated type
8064@code{yy::parser::semantic_type}.
8065@item
8066Non POD (Plain Old Data) types cannot be used. C++ forbids any
8067instance of classes with constructors in unions: only @emph{pointers}
8068to such objects are allowed.
8069@end itemize
8070
8071Because objects have to be stored via pointers, memory is not
8072reclaimed automatically: using the @code{%destructor} directive is the
8073only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
8074Symbols}.
8075
8076
8077@node C++ Location Values
8078@subsection C++ Location Values
8079@c - %locations
8080@c - class Position
8081@c - class Location
8082@c - %define filename_type "const symbol::Symbol"
8083
8084When the directive @code{%locations} is used, the C++ parser supports
8085location tracking, see @ref{Locations, , Locations Overview}. Two
8086auxiliary classes define a @code{position}, a single point in a file,
8087and a @code{location}, a range composed of a pair of
8088@code{position}s (possibly spanning several files).
8089
8090@deftypemethod {position} {std::string*} file
8091The name of the file. It will always be handled as a pointer, the
8092parser will never duplicate nor deallocate it. As an experimental
8093feature you may change it to @samp{@var{type}*} using @samp{%define
8094filename_type "@var{type}"}.
8095@end deftypemethod
8096
8097@deftypemethod {position} {unsigned int} line
8098The line, starting at 1.
8099@end deftypemethod
8100
8101@deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
8102Advance by @var{height} lines, resetting the column number.
8103@end deftypemethod
8104
8105@deftypemethod {position} {unsigned int} column
8106The column, starting at 0.
8107@end deftypemethod
8108
8109@deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
8110Advance by @var{width} columns, without changing the line number.
8111@end deftypemethod
8112
8113@deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
8114@deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
8115@deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
8116@deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
8117Various forms of syntactic sugar for @code{columns}.
8118@end deftypemethod
8119
8120@deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
8121Report @var{p} on @var{o} like this:
8122@samp{@var{file}:@var{line}.@var{column}}, or
8123@samp{@var{line}.@var{column}} if @var{file} is null.
8124@end deftypemethod
8125
8126@deftypemethod {location} {position} begin
8127@deftypemethodx {location} {position} end
8128The first, inclusive, position of the range, and the first beyond.
8129@end deftypemethod
8130
8131@deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
8132@deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
8133Advance the @code{end} position.
8134@end deftypemethod
8135
8136@deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
8137@deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
8138@deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
8139Various forms of syntactic sugar.
8140@end deftypemethod
8141
8142@deftypemethod {location} {void} step ()
8143Move @code{begin} onto @code{end}.
8144@end deftypemethod
8145
8146
8147@node C++ Parser Interface
8148@subsection C++ Parser Interface
8149@c - define parser_class_name
8150@c - Ctor
8151@c - parse, error, set_debug_level, debug_level, set_debug_stream,
8152@c debug_stream.
8153@c - Reporting errors
8154
8155The output files @file{@var{output}.hh} and @file{@var{output}.cc}
8156declare and define the parser class in the namespace @code{yy}. The
8157class name defaults to @code{parser}, but may be changed using
8158@samp{%define parser_class_name "@var{name}"}. The interface of
8159this class is detailed below. It can be extended using the
8160@code{%parse-param} feature: its semantics is slightly changed since
8161it describes an additional member of the parser class, and an
8162additional argument for its constructor.
8163
8164@defcv {Type} {parser} {semantic_value_type}
8165@defcvx {Type} {parser} {location_value_type}
8166The types for semantics value and locations.
8167@end defcv
8168
8169@deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
8170Build a new parser object. There are no arguments by default, unless
8171@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
8172@end deftypemethod
8173
8174@deftypemethod {parser} {int} parse ()
8175Run the syntactic analysis, and return 0 on success, 1 otherwise.
8176@end deftypemethod
8177
8178@deftypemethod {parser} {std::ostream&} debug_stream ()
8179@deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
8180Get or set the stream used for tracing the parsing. It defaults to
8181@code{std::cerr}.
8182@end deftypemethod
8183
8184@deftypemethod {parser} {debug_level_type} debug_level ()
8185@deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
8186Get or set the tracing level. Currently its value is either 0, no trace,
8187or nonzero, full tracing.
8188@end deftypemethod
8189
8190@deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
8191The definition for this member function must be supplied by the user:
8192the parser uses it to report a parser error occurring at @var{l},
8193described by @var{m}.
8194@end deftypemethod
8195
8196
8197@node C++ Scanner Interface
8198@subsection C++ Scanner Interface
8199@c - prefix for yylex.
8200@c - Pure interface to yylex
8201@c - %lex-param
8202
8203The parser invokes the scanner by calling @code{yylex}. Contrary to C
8204parsers, C++ parsers are always pure: there is no point in using the
8205@code{%define api.pure} directive. Therefore the interface is as follows.
8206
8207@deftypemethod {parser} {int} yylex (semantic_value_type& @var{yylval}, location_type& @var{yylloc}, @var{type1} @var{arg1}, ...)
8208Return the next token. Its type is the return value, its semantic
8209value and location being @var{yylval} and @var{yylloc}. Invocations of
8210@samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
8211@end deftypemethod
8212
8213
8214@node A Complete C++ Example
8215@subsection A Complete C++ Example
8216
8217This section demonstrates the use of a C++ parser with a simple but
8218complete example. This example should be available on your system,
8219ready to compile, in the directory @dfn{../bison/examples/calc++}. It
8220focuses on the use of Bison, therefore the design of the various C++
8221classes is very naive: no accessors, no encapsulation of members etc.
8222We will use a Lex scanner, and more precisely, a Flex scanner, to
8223demonstrate the various interaction. A hand written scanner is
8224actually easier to interface with.
8225
8226@menu
8227* Calc++ --- C++ Calculator:: The specifications
8228* Calc++ Parsing Driver:: An active parsing context
8229* Calc++ Parser:: A parser class
8230* Calc++ Scanner:: A pure C++ Flex scanner
8231* Calc++ Top Level:: Conducting the band
8232@end menu
8233
8234@node Calc++ --- C++ Calculator
8235@subsubsection Calc++ --- C++ Calculator
8236
8237Of course the grammar is dedicated to arithmetics, a single
8238expression, possibly preceded by variable assignments. An
8239environment containing possibly predefined variables such as
8240@code{one} and @code{two}, is exchanged with the parser. An example
8241of valid input follows.
8242
8243@example
8244three := 3
8245seven := one + two * three
8246seven * seven
8247@end example
8248
8249@node Calc++ Parsing Driver
8250@subsubsection Calc++ Parsing Driver
8251@c - An env
8252@c - A place to store error messages
8253@c - A place for the result
8254
8255To support a pure interface with the parser (and the scanner) the
8256technique of the ``parsing context'' is convenient: a structure
8257containing all the data to exchange. Since, in addition to simply
8258launch the parsing, there are several auxiliary tasks to execute (open
8259the file for parsing, instantiate the parser etc.), we recommend
8260transforming the simple parsing context structure into a fully blown
8261@dfn{parsing driver} class.
8262
8263The declaration of this driver class, @file{calc++-driver.hh}, is as
8264follows. The first part includes the CPP guard and imports the
8265required standard library components, and the declaration of the parser
8266class.
8267
8268@comment file: calc++-driver.hh
8269@example
8270#ifndef CALCXX_DRIVER_HH
8271# define CALCXX_DRIVER_HH
8272# include <string>
8273# include <map>
8274# include "calc++-parser.hh"
8275@end example
8276
8277
8278@noindent
8279Then comes the declaration of the scanning function. Flex expects
8280the signature of @code{yylex} to be defined in the macro
8281@code{YY_DECL}, and the C++ parser expects it to be declared. We can
8282factor both as follows.
8283
8284@comment file: calc++-driver.hh
8285@example
8286// Tell Flex the lexer's prototype ...
8287# define YY_DECL \
8288 yy::calcxx_parser::token_type \
8289 yylex (yy::calcxx_parser::semantic_type* yylval, \
8290 yy::calcxx_parser::location_type* yylloc, \
8291 calcxx_driver& driver)
8292// ... and declare it for the parser's sake.
8293YY_DECL;
8294@end example
8295
8296@noindent
8297The @code{calcxx_driver} class is then declared with its most obvious
8298members.
8299
8300@comment file: calc++-driver.hh
8301@example
8302// Conducting the whole scanning and parsing of Calc++.
8303class calcxx_driver
8304@{
8305public:
8306 calcxx_driver ();
8307 virtual ~calcxx_driver ();
8308
8309 std::map<std::string, int> variables;
8310
8311 int result;
8312@end example
8313
8314@noindent
8315To encapsulate the coordination with the Flex scanner, it is useful to
8316have two members function to open and close the scanning phase.
8317
8318@comment file: calc++-driver.hh
8319@example
8320 // Handling the scanner.
8321 void scan_begin ();
8322 void scan_end ();
8323 bool trace_scanning;
8324@end example
8325
8326@noindent
8327Similarly for the parser itself.
8328
8329@comment file: calc++-driver.hh
8330@example
8331 // Run the parser. Return 0 on success.
8332 int parse (const std::string& f);
8333 std::string file;
8334 bool trace_parsing;
8335@end example
8336
8337@noindent
8338To demonstrate pure handling of parse errors, instead of simply
8339dumping them on the standard error output, we will pass them to the
8340compiler driver using the following two member functions. Finally, we
8341close the class declaration and CPP guard.
8342
8343@comment file: calc++-driver.hh
8344@example
8345 // Error handling.
8346 void error (const yy::location& l, const std::string& m);
8347 void error (const std::string& m);
8348@};
8349#endif // ! CALCXX_DRIVER_HH
8350@end example
8351
8352The implementation of the driver is straightforward. The @code{parse}
8353member function deserves some attention. The @code{error} functions
8354are simple stubs, they should actually register the located error
8355messages and set error state.
8356
8357@comment file: calc++-driver.cc
8358@example
8359#include "calc++-driver.hh"
8360#include "calc++-parser.hh"
8361
8362calcxx_driver::calcxx_driver ()
8363 : trace_scanning (false), trace_parsing (false)
8364@{
8365 variables["one"] = 1;
8366 variables["two"] = 2;
8367@}
8368
8369calcxx_driver::~calcxx_driver ()
8370@{
8371@}
8372
8373int
8374calcxx_driver::parse (const std::string &f)
8375@{
8376 file = f;
8377 scan_begin ();
8378 yy::calcxx_parser parser (*this);
8379 parser.set_debug_level (trace_parsing);
8380 int res = parser.parse ();
8381 scan_end ();
8382 return res;
8383@}
8384
8385void
8386calcxx_driver::error (const yy::location& l, const std::string& m)
8387@{
8388 std::cerr << l << ": " << m << std::endl;
8389@}
8390
8391void
8392calcxx_driver::error (const std::string& m)
8393@{
8394 std::cerr << m << std::endl;
8395@}
8396@end example
8397
8398@node Calc++ Parser
8399@subsubsection Calc++ Parser
8400
8401The parser definition file @file{calc++-parser.yy} starts by asking for
8402the C++ LALR(1) skeleton, the creation of the parser header file, and
8403specifies the name of the parser class. Because the C++ skeleton
8404changed several times, it is safer to require the version you designed
8405the grammar for.
8406
8407@comment file: calc++-parser.yy
8408@example
8409%skeleton "lalr1.cc" /* -*- C++ -*- */
8410%require "@value{VERSION}"
8411%defines
8412%define parser_class_name "calcxx_parser"
8413@end example
8414
8415@noindent
8416@findex %code requires
8417Then come the declarations/inclusions needed to define the
8418@code{%union}. Because the parser uses the parsing driver and
8419reciprocally, both cannot include the header of the other. Because the
8420driver's header needs detailed knowledge about the parser class (in
8421particular its inner types), it is the parser's header which will simply
8422use a forward declaration of the driver.
8423@xref{Decl Summary, ,%code}.
8424
8425@comment file: calc++-parser.yy
8426@example
8427%code requires @{
8428# include <string>
8429class calcxx_driver;
8430@}
8431@end example
8432
8433@noindent
8434The driver is passed by reference to the parser and to the scanner.
8435This provides a simple but effective pure interface, not relying on
8436global variables.
8437
8438@comment file: calc++-parser.yy
8439@example
8440// The parsing context.
8441%parse-param @{ calcxx_driver& driver @}
8442%lex-param @{ calcxx_driver& driver @}
8443@end example
8444
8445@noindent
8446Then we request the location tracking feature, and initialize the
8447first location's file name. Afterwards new locations are computed
8448relatively to the previous locations: the file name will be
8449automatically propagated.
8450
8451@comment file: calc++-parser.yy
8452@example
8453%locations
8454%initial-action
8455@{
8456 // Initialize the initial location.
8457 @@$.begin.filename = @@$.end.filename = &driver.file;
8458@};
8459@end example
8460
8461@noindent
8462Use the two following directives to enable parser tracing and verbose
8463error messages.
8464
8465@comment file: calc++-parser.yy
8466@example
8467%debug
8468%error-verbose
8469@end example
8470
8471@noindent
8472Semantic values cannot use ``real'' objects, but only pointers to
8473them.
8474
8475@comment file: calc++-parser.yy
8476@example
8477// Symbols.
8478%union
8479@{
8480 int ival;
8481 std::string *sval;
8482@};
8483@end example
8484
8485@noindent
8486@findex %code
8487The code between @samp{%code @{} and @samp{@}} is output in the
8488@file{*.cc} file; it needs detailed knowledge about the driver.
8489
8490@comment file: calc++-parser.yy
8491@example
8492%code @{
8493# include "calc++-driver.hh"
8494@}
8495@end example
8496
8497
8498@noindent
8499The token numbered as 0 corresponds to end of file; the following line
8500allows for nicer error messages referring to ``end of file'' instead
8501of ``$end''. Similarly user friendly named are provided for each
8502symbol. Note that the tokens names are prefixed by @code{TOKEN_} to
8503avoid name clashes.
8504
8505@comment file: calc++-parser.yy
8506@example
8507%token END 0 "end of file"
8508%token ASSIGN ":="
8509%token <sval> IDENTIFIER "identifier"
8510%token <ival> NUMBER "number"
8511%type <ival> exp
8512@end example
8513
8514@noindent
8515To enable memory deallocation during error recovery, use
8516@code{%destructor}.
8517
8518@c FIXME: Document %printer, and mention that it takes a braced-code operand.
8519@comment file: calc++-parser.yy
8520@example
8521%printer @{ debug_stream () << *$$; @} "identifier"
8522%destructor @{ delete $$; @} "identifier"
8523
8524%printer @{ debug_stream () << $$; @} <ival>
8525@end example
8526
8527@noindent
8528The grammar itself is straightforward.
8529
8530@comment file: calc++-parser.yy
8531@example
8532%%
8533%start unit;
8534unit: assignments exp @{ driver.result = $2; @};
8535
8536assignments: assignments assignment @{@}
8537 | /* Nothing. */ @{@};
8538
8539assignment:
8540 "identifier" ":=" exp
8541 @{ driver.variables[*$1] = $3; delete $1; @};
8542
8543%left '+' '-';
8544%left '*' '/';
8545exp: exp '+' exp @{ $$ = $1 + $3; @}
8546 | exp '-' exp @{ $$ = $1 - $3; @}
8547 | exp '*' exp @{ $$ = $1 * $3; @}
8548 | exp '/' exp @{ $$ = $1 / $3; @}
8549 | "identifier" @{ $$ = driver.variables[*$1]; delete $1; @}
8550 | "number" @{ $$ = $1; @};
8551%%
8552@end example
8553
8554@noindent
8555Finally the @code{error} member function registers the errors to the
8556driver.
8557
8558@comment file: calc++-parser.yy
8559@example
8560void
8561yy::calcxx_parser::error (const yy::calcxx_parser::location_type& l,
8562 const std::string& m)
8563@{
8564 driver.error (l, m);
8565@}
8566@end example
8567
8568@node Calc++ Scanner
8569@subsubsection Calc++ Scanner
8570
8571The Flex scanner first includes the driver declaration, then the
8572parser's to get the set of defined tokens.
8573
8574@comment file: calc++-scanner.ll
8575@example
8576%@{ /* -*- C++ -*- */
8577# include <cstdlib>
8578# include <cerrno>
8579# include <climits>
8580# include <string>
8581# include "calc++-driver.hh"
8582# include "calc++-parser.hh"
8583
8584/* Work around an incompatibility in flex (at least versions
8585 2.5.31 through 2.5.33): it generates code that does
8586 not conform to C89. See Debian bug 333231
8587 <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */
8588# undef yywrap
8589# define yywrap() 1
8590
8591/* By default yylex returns int, we use token_type.
8592 Unfortunately yyterminate by default returns 0, which is
8593 not of token_type. */
8594#define yyterminate() return token::END
8595%@}
8596@end example
8597
8598@noindent
8599Because there is no @code{#include}-like feature we don't need
8600@code{yywrap}, we don't need @code{unput} either, and we parse an
8601actual file, this is not an interactive session with the user.
8602Finally we enable the scanner tracing features.
8603
8604@comment file: calc++-scanner.ll
8605@example
8606%option noyywrap nounput batch debug
8607@end example
8608
8609@noindent
8610Abbreviations allow for more readable rules.
8611
8612@comment file: calc++-scanner.ll
8613@example
8614id [a-zA-Z][a-zA-Z_0-9]*
8615int [0-9]+
8616blank [ \t]
8617@end example
8618
8619@noindent
8620The following paragraph suffices to track locations accurately. Each
8621time @code{yylex} is invoked, the begin position is moved onto the end
8622position. Then when a pattern is matched, the end position is
8623advanced of its width. In case it matched ends of lines, the end
8624cursor is adjusted, and each time blanks are matched, the begin cursor
8625is moved onto the end cursor to effectively ignore the blanks
8626preceding tokens. Comments would be treated equally.
8627
8628@comment file: calc++-scanner.ll
8629@example
8630%@{
8631# define YY_USER_ACTION yylloc->columns (yyleng);
8632%@}
8633%%
8634%@{
8635 yylloc->step ();
8636%@}
8637@{blank@}+ yylloc->step ();
8638[\n]+ yylloc->lines (yyleng); yylloc->step ();
8639@end example
8640
8641@noindent
8642The rules are simple, just note the use of the driver to report errors.
8643It is convenient to use a typedef to shorten
8644@code{yy::calcxx_parser::token::identifier} into
8645@code{token::identifier} for instance.
8646
8647@comment file: calc++-scanner.ll
8648@example
8649%@{
8650 typedef yy::calcxx_parser::token token;
8651%@}
8652 /* Convert ints to the actual type of tokens. */
8653[-+*/] return yy::calcxx_parser::token_type (yytext[0]);
8654":=" return token::ASSIGN;
8655@{int@} @{
8656 errno = 0;
8657 long n = strtol (yytext, NULL, 10);
8658 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
8659 driver.error (*yylloc, "integer is out of range");
8660 yylval->ival = n;
8661 return token::NUMBER;
8662@}
8663@{id@} yylval->sval = new std::string (yytext); return token::IDENTIFIER;
8664. driver.error (*yylloc, "invalid character");
8665%%
8666@end example
8667
8668@noindent
8669Finally, because the scanner related driver's member function depend
8670on the scanner's data, it is simpler to implement them in this file.
8671
8672@comment file: calc++-scanner.ll
8673@example
8674void
8675calcxx_driver::scan_begin ()
8676@{
8677 yy_flex_debug = trace_scanning;
8678 if (file == "-")
8679 yyin = stdin;
8680 else if (!(yyin = fopen (file.c_str (), "r")))
8681 @{
8682 error (std::string ("cannot open ") + file);
8683 exit (1);
8684 @}
8685@}
8686
8687void
8688calcxx_driver::scan_end ()
8689@{
8690 fclose (yyin);
8691@}
8692@end example
8693
8694@node Calc++ Top Level
8695@subsubsection Calc++ Top Level
8696
8697The top level file, @file{calc++.cc}, poses no problem.
8698
8699@comment file: calc++.cc
8700@example
8701#include <iostream>
8702#include "calc++-driver.hh"
8703
8704int
8705main (int argc, char *argv[])
8706@{
8707 calcxx_driver driver;
8708 for (++argv; argv[0]; ++argv)
8709 if (*argv == std::string ("-p"))
8710 driver.trace_parsing = true;
8711 else if (*argv == std::string ("-s"))
8712 driver.trace_scanning = true;
8713 else if (!driver.parse (*argv))
8714 std::cout << driver.result << std::endl;
8715@}
8716@end example
8717
8718@node Java Parsers
8719@section Java Parsers
8720
8721@menu
8722* Java Bison Interface:: Asking for Java parser generation
8723* Java Semantic Values:: %type and %token vs. Java
8724* Java Location Values:: The position and location classes
8725* Java Parser Interface:: Instantiating and running the parser
8726* Java Scanner Interface:: Specifying the scanner for the parser
8727* Java Action Features:: Special features for use in actions
8728* Java Differences:: Differences between C/C++ and Java Grammars
8729* Java Declarations Summary:: List of Bison declarations used with Java
8730@end menu
8731
8732@node Java Bison Interface
8733@subsection Java Bison Interface
8734@c - %language "Java"
8735
8736(The current Java interface is experimental and may evolve.
8737More user feedback will help to stabilize it.)
8738
8739The Java parser skeletons are selected using the @code{%language "Java"}
8740directive or the @option{-L java}/@option{--language=java} option.
8741
8742@c FIXME: Documented bug.
8743When generating a Java parser, @code{bison @var{basename}.y} will create
8744a single Java source file named @file{@var{basename}.java}. Using an
8745input file without a @file{.y} suffix is currently broken. The basename
8746of the output file can be changed by the @code{%file-prefix} directive
8747or the @option{-p}/@option{--name-prefix} option. The entire output file
8748name can be changed by the @code{%output} directive or the
8749@option{-o}/@option{--output} option. The output file contains a single
8750class for the parser.
8751
8752You can create documentation for generated parsers using Javadoc.
8753
8754Contrary to C parsers, Java parsers do not use global variables; the
8755state of the parser is always local to an instance of the parser class.
8756Therefore, all Java parsers are ``pure'', and the @code{%pure-parser}
8757and @code{%define api.pure} directives does not do anything when used in
8758Java.
8759
8760Push parsers are currently unsupported in Java and @code{%define
8761api.push_pull} have no effect.
8762
8763@acronym{GLR} parsers are currently unsupported in Java. Do not use the
8764@code{glr-parser} directive.
8765
8766No header file can be generated for Java parsers. Do not use the
8767@code{%defines} directive or the @option{-d}/@option{--defines} options.
8768
8769@c FIXME: Possible code change.
8770Currently, support for debugging and verbose errors are always compiled
8771in. Thus the @code{%debug} and @code{%token-table} directives and the
8772@option{-t}/@option{--debug} and @option{-k}/@option{--token-table}
8773options have no effect. This may change in the future to eliminate
8774unused code in the generated parser, so use @code{%debug} and
8775@code{%verbose-error} explicitly if needed. Also, in the future the
8776@code{%token-table} directive might enable a public interface to
8777access the token names and codes.
8778
8779@node Java Semantic Values
8780@subsection Java Semantic Values
8781@c - No %union, specify type in %type/%token.
8782@c - YYSTYPE
8783@c - Printer and destructor
8784
8785There is no @code{%union} directive in Java parsers. Instead, the
8786semantic values' types (class names) should be specified in the
8787@code{%type} or @code{%token} directive:
8788
8789@example
8790%type <Expression> expr assignment_expr term factor
8791%type <Integer> number
8792@end example
8793
8794By default, the semantic stack is declared to have @code{Object} members,
8795which means that the class types you specify can be of any class.
8796To improve the type safety of the parser, you can declare the common
8797superclass of all the semantic values using the @code{%define stype}
8798directive. For example, after the following declaration:
8799
8800@example
8801%define stype "ASTNode"
8802@end example
8803
8804@noindent
8805any @code{%type} or @code{%token} specifying a semantic type which
8806is not a subclass of ASTNode, will cause a compile-time error.
8807
8808@c FIXME: Documented bug.
8809Types used in the directives may be qualified with a package name.
8810Primitive data types are accepted for Java version 1.5 or later. Note
8811that in this case the autoboxing feature of Java 1.5 will be used.
8812Generic types may not be used; this is due to a limitation in the
8813implementation of Bison, and may change in future releases.
8814
8815Java parsers do not support @code{%destructor}, since the language
8816adopts garbage collection. The parser will try to hold references
8817to semantic values for as little time as needed.
8818
8819Java parsers do not support @code{%printer}, as @code{toString()}
8820can be used to print the semantic values. This however may change
8821(in a backwards-compatible way) in future versions of Bison.
8822
8823
8824@node Java Location Values
8825@subsection Java Location Values
8826@c - %locations
8827@c - class Position
8828@c - class Location
8829
8830When the directive @code{%locations} is used, the Java parser
8831supports location tracking, see @ref{Locations, , Locations Overview}.
8832An auxiliary user-defined class defines a @dfn{position}, a single point
8833in a file; Bison itself defines a class representing a @dfn{location},
8834a range composed of a pair of positions (possibly spanning several
8835files). The location class is an inner class of the parser; the name
8836is @code{Location} by default, and may also be renamed using
8837@code{%define location_type "@var{class-name}}.
8838
8839The location class treats the position as a completely opaque value.
8840By default, the class name is @code{Position}, but this can be changed
8841with @code{%define position_type "@var{class-name}"}. This class must
8842be supplied by the user.
8843
8844
8845@deftypeivar {Location} {Position} begin
8846@deftypeivarx {Location} {Position} end
8847The first, inclusive, position of the range, and the first beyond.
8848@end deftypeivar
8849
8850@deftypeop {Constructor} {Location} {} Location (Position @var{loc})
8851Create a @code{Location} denoting an empty range located at a given point.
8852@end deftypeop
8853
8854@deftypeop {Constructor} {Location} {} Location (Position @var{begin}, Position @var{end})
8855Create a @code{Location} from the endpoints of the range.
8856@end deftypeop
8857
8858@deftypemethod {Location} {String} toString ()
8859Prints the range represented by the location. For this to work
8860properly, the position class should override the @code{equals} and
8861@code{toString} methods appropriately.
8862@end deftypemethod
8863
8864
8865@node Java Parser Interface
8866@subsection Java Parser Interface
8867@c - define parser_class_name
8868@c - Ctor
8869@c - parse, error, set_debug_level, debug_level, set_debug_stream,
8870@c debug_stream.
8871@c - Reporting errors
8872
8873The name of the generated parser class defaults to @code{YYParser}. The
8874@code{YY} prefix may be changed using the @code{%name-prefix} directive
8875or the @option{-p}/@option{--name-prefix} option. Alternatively, use
8876@code{%define parser_class_name "@var{name}"} to give a custom name to
8877the class. The interface of this class is detailed below.
8878
8879By default, the parser class has package visibility. A declaration
8880@code{%define public} will change to public visibility. Remember that,
8881according to the Java language specification, the name of the @file{.java}
8882file should match the name of the class in this case. Similarly, you can
8883use @code{abstract}, @code{final} and @code{strictfp} with the
8884@code{%define} declaration to add other modifiers to the parser class.
8885
8886The Java package name of the parser class can be specified using the
8887@code{%define package} directive. The superclass and the implemented
8888interfaces of the parser class can be specified with the @code{%define
8889extends} and @code{%define implements} directives.
8890
8891The parser class defines an inner class, @code{Location}, that is used
8892for location tracking (see @ref{Java Location Values}), and a inner
8893interface, @code{Lexer} (see @ref{Java Scanner Interface}). Other than
8894these inner class/interface, and the members described in the interface
8895below, all the other members and fields are preceded with a @code{yy} or
8896@code{YY} prefix to avoid clashes with user code.
8897
8898@c FIXME: The following constants and variables are still undocumented:
8899@c @code{bisonVersion}, @code{bisonSkeleton} and @code{errorVerbose}.
8900
8901The parser class can be extended using the @code{%parse-param}
8902directive. Each occurrence of the directive will add a @code{protected
8903final} field to the parser class, and an argument to its constructor,
8904which initialize them automatically.
8905
8906Token names defined by @code{%token} and the predefined @code{EOF} token
8907name are added as constant fields to the parser class.
8908
8909@deftypeop {Constructor} {YYParser} {} YYParser (@var{lex_param}, @dots{}, @var{parse_param}, @dots{})
8910Build a new parser object with embedded @code{%code lexer}. There are
8911no parameters, unless @code{%parse-param}s and/or @code{%lex-param}s are
8912used.
8913@end deftypeop
8914
8915@deftypeop {Constructor} {YYParser} {} YYParser (Lexer @var{lexer}, @var{parse_param}, @dots{})
8916Build a new parser object using the specified scanner. There are no
8917additional parameters unless @code{%parse-param}s are used.
8918
8919If the scanner is defined by @code{%code lexer}, this constructor is
8920declared @code{protected} and is called automatically with a scanner
8921created with the correct @code{%lex-param}s.
8922@end deftypeop
8923
8924@deftypemethod {YYParser} {boolean} parse ()
8925Run the syntactic analysis, and return @code{true} on success,
8926@code{false} otherwise.
8927@end deftypemethod
8928
8929@deftypemethod {YYParser} {boolean} recovering ()
8930During the syntactic analysis, return @code{true} if recovering
8931from a syntax error.
8932@xref{Error Recovery}.
8933@end deftypemethod
8934
8935@deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
8936@deftypemethodx {YYParser} {void} setDebugStream (java.io.printStream @var{o})
8937Get or set the stream used for tracing the parsing. It defaults to
8938@code{System.err}.
8939@end deftypemethod
8940
8941@deftypemethod {YYParser} {int} getDebugLevel ()
8942@deftypemethodx {YYParser} {void} setDebugLevel (int @var{l})
8943Get or set the tracing level. Currently its value is either 0, no trace,
8944or nonzero, full tracing.
8945@end deftypemethod
8946
8947
8948@node Java Scanner Interface
8949@subsection Java Scanner Interface
8950@c - %code lexer
8951@c - %lex-param
8952@c - Lexer interface
8953
8954There are two possible ways to interface a Bison-generated Java parser
8955with a scanner: the scanner may be defined by @code{%code lexer}, or
8956defined elsewhere. In either case, the scanner has to implement the
8957@code{Lexer} inner interface of the parser class.
8958
8959In the first case, the body of the scanner class is placed in
8960@code{%code lexer} blocks. If you want to pass parameters from the
8961parser constructor to the scanner constructor, specify them with
8962@code{%lex-param}; they are passed before @code{%parse-param}s to the
8963constructor.
8964
8965In the second case, the scanner has to implement the @code{Lexer} interface,
8966which is defined within the parser class (e.g., @code{YYParser.Lexer}).
8967The constructor of the parser object will then accept an object
8968implementing the interface; @code{%lex-param} is not used in this
8969case.
8970
8971In both cases, the scanner has to implement the following methods.
8972
8973@deftypemethod {Lexer} {void} yyerror (Location @var{loc}, String @var{msg})
8974This method is defined by the user to emit an error message. The first
8975parameter is omitted if location tracking is not active. Its type can be
8976changed using @code{%define location_type "@var{class-name}".}
8977@end deftypemethod
8978
8979@deftypemethod {Lexer} {int} yylex ()
8980Return the next token. Its type is the return value, its semantic
8981value and location are saved and returned by the ther methods in the
8982interface.
8983
8984Use @code{%define lex_throws} to specify any uncaught exceptions.
8985Default is @code{java.io.IOException}.
8986@end deftypemethod
8987
8988@deftypemethod {Lexer} {Position} getStartPos ()
8989@deftypemethodx {Lexer} {Position} getEndPos ()
8990Return respectively the first position of the last token that
8991@code{yylex} returned, and the first position beyond it. These
8992methods are not needed unless location tracking is active.
8993
8994The return type can be changed using @code{%define position_type
8995"@var{class-name}".}
8996@end deftypemethod
8997
8998@deftypemethod {Lexer} {Object} getLVal ()
8999Return the semantical value of the last token that yylex returned.
9000
9001The return type can be changed using @code{%define stype
9002"@var{class-name}".}
9003@end deftypemethod
9004
9005
9006@node Java Action Features
9007@subsection Special Features for Use in Java Actions
9008
9009The following special constructs can be uses in Java actions.
9010Other analogous C action features are currently unavailable for Java.
9011
9012Use @code{%define throws} to specify any uncaught exceptions from parser
9013actions, and initial actions specified by @code{%initial-action}.
9014
9015@defvar $@var{n}
9016The semantic value for the @var{n}th component of the current rule.
9017This may not be assigned to.
9018@xref{Java Semantic Values}.
9019@end defvar
9020
9021@defvar $<@var{typealt}>@var{n}
9022Like @code{$@var{n}} but specifies a alternative type @var{typealt}.
9023@xref{Java Semantic Values}.
9024@end defvar
9025
9026@defvar $$
9027The semantic value for the grouping made by the current rule. As a
9028value, this is in the base type (@code{Object} or as specified by
9029@code{%define stype}) as in not cast to the declared subtype because
9030casts are not allowed on the left-hand side of Java assignments.
9031Use an explicit Java cast if the correct subtype is needed.
9032@xref{Java Semantic Values}.
9033@end defvar
9034
9035@defvar $<@var{typealt}>$
9036Same as @code{$$} since Java always allow assigning to the base type.
9037Perhaps we should use this and @code{$<>$} for the value and @code{$$}
9038for setting the value but there is currently no easy way to distinguish
9039these constructs.
9040@xref{Java Semantic Values}.
9041@end defvar
9042
9043@defvar @@@var{n}
9044The location information of the @var{n}th component of the current rule.
9045This may not be assigned to.
9046@xref{Java Location Values}.
9047@end defvar
9048
9049@defvar @@$
9050The location information of the grouping made by the current rule.
9051@xref{Java Location Values}.
9052@end defvar
9053
9054@deffn {Statement} {return YYABORT;}
9055Return immediately from the parser, indicating failure.
9056@xref{Java Parser Interface}.
9057@end deffn
9058
9059@deffn {Statement} {return YYACCEPT;}
9060Return immediately from the parser, indicating success.
9061@xref{Java Parser Interface}.
9062@end deffn
9063
9064@deffn {Statement} {return YYERROR;}
9065Start error recovery without printing an error message.
9066@xref{Error Recovery}.
9067@end deffn
9068
9069@deffn {Statement} {return YYFAIL;}
9070Print an error message and start error recovery.
9071@xref{Error Recovery}.
9072@end deffn
9073
9074@deftypefn {Function} {boolean} recovering ()
9075Return whether error recovery is being done. In this state, the parser
9076reads token until it reaches a known state, and then restarts normal
9077operation.
9078@xref{Error Recovery}.
9079@end deftypefn
9080
9081@deftypefn {Function} {protected void} yyerror (String msg)
9082@deftypefnx {Function} {protected void} yyerror (Position pos, String msg)
9083@deftypefnx {Function} {protected void} yyerror (Location loc, String msg)
9084Print an error message using the @code{yyerror} method of the scanner
9085instance in use.
9086@end deftypefn
9087
9088
9089@node Java Differences
9090@subsection Differences between C/C++ and Java Grammars
9091
9092The different structure of the Java language forces several differences
9093between C/C++ grammars, and grammars designed for Java parsers. This
9094section summarizes these differences.
9095
9096@itemize
9097@item
9098Java lacks a preprocessor, so the @code{YYERROR}, @code{YYACCEPT},
9099@code{YYABORT} symbols (@pxref{Table of Symbols}) cannot obviously be
9100macros. Instead, they should be preceded by @code{return} when they
9101appear in an action. The actual definition of these symbols is
9102opaque to the Bison grammar, and it might change in the future. The
9103only meaningful operation that you can do, is to return them.
9104See @pxref{Java Action Features}.
9105
9106Note that of these three symbols, only @code{YYACCEPT} and
9107@code{YYABORT} will cause a return from the @code{yyparse}
9108method@footnote{Java parsers include the actions in a separate
9109method than @code{yyparse} in order to have an intuitive syntax that
9110corresponds to these C macros.}.
9111
9112@item
9113Java lacks unions, so @code{%union} has no effect. Instead, semantic
9114values have a common base type: @code{Object} or as specified by
9115@code{%define stype}. Angle backets on @code{%token}, @code{type},
9116@code{$@var{n}} and @code{$$} specify subtypes rather than fields of
9117an union. The type of @code{$$}, even with angle brackets, is the base
9118type since Java casts are not allow on the left-hand side of assignments.
9119Also, @code{$@var{n}} and @code{@@@var{n}} are not allowed on the
9120left-hand side of assignments. See @pxref{Java Semantic Values} and
9121@pxref{Java Action Features}.
9122
9123@item
9124The prolog declarations have a different meaning than in C/C++ code.
9125@table @asis
9126@item @code{%code imports}
9127blocks are placed at the beginning of the Java source code. They may
9128include copyright notices. For a @code{package} declarations, it is
9129suggested to use @code{%define package} instead.
9130
9131@item unqualified @code{%code}
9132blocks are placed inside the parser class.
9133
9134@item @code{%code lexer}
9135blocks, if specified, should include the implementation of the
9136scanner. If there is no such block, the scanner can be any class
9137that implements the appropriate interface (see @pxref{Java Scanner
9138Interface}).
9139@end table
9140
9141Other @code{%code} blocks are not supported in Java parsers.
9142In particular, @code{%@{ @dots{} %@}} blocks should not be used
9143and may give an error in future versions of Bison.
9144
9145The epilogue has the same meaning as in C/C++ code and it can
9146be used to define other classes used by the parser @emph{outside}
9147the parser class.
9148@end itemize
9149
9150
9151@node Java Declarations Summary
9152@subsection Java Declarations Summary
9153
9154This summary only include declarations specific to Java or have special
9155meaning when used in a Java parser.
9156
9157@deffn {Directive} {%language "Java"}
9158Generate a Java class for the parser.
9159@end deffn
9160
9161@deffn {Directive} %lex-param @{@var{type} @var{name}@}
9162A parameter for the lexer class defined by @code{%code lexer}
9163@emph{only}, added as parameters to the lexer constructor and the parser
9164constructor that @emph{creates} a lexer. Default is none.
9165@xref{Java Scanner Interface}.
9166@end deffn
9167
9168@deffn {Directive} %name-prefix "@var{prefix}"
9169The prefix of the parser class name @code{@var{prefix}Parser} if
9170@code{%define parser_class_name} is not used. Default is @code{YY}.
9171@xref{Java Bison Interface}.
9172@end deffn
9173
9174@deffn {Directive} %parse-param @{@var{type} @var{name}@}
9175A parameter for the parser class added as parameters to constructor(s)
9176and as fields initialized by the constructor(s). Default is none.
9177@xref{Java Parser Interface}.
9178@end deffn
9179
9180@deffn {Directive} %token <@var{type}> @var{token} @dots{}
9181Declare tokens. Note that the angle brackets enclose a Java @emph{type}.
9182@xref{Java Semantic Values}.
9183@end deffn
9184
9185@deffn {Directive} %type <@var{type}> @var{nonterminal} @dots{}
9186Declare the type of nonterminals. Note that the angle brackets enclose
9187a Java @emph{type}.
9188@xref{Java Semantic Values}.
9189@end deffn
9190
9191@deffn {Directive} %code @{ @var{code} @dots{} @}
9192Code appended to the inside of the parser class.
9193@xref{Java Differences}.
9194@end deffn
9195
9196@deffn {Directive} {%code imports} @{ @var{code} @dots{} @}
9197Code inserted just after the @code{package} declaration.
9198@xref{Java Differences}.
9199@end deffn
9200
9201@deffn {Directive} {%code lexer} @{ @var{code} @dots{} @}
9202Code added to the body of a inner lexer class within the parser class.
9203@xref{Java Scanner Interface}.
9204@end deffn
9205
9206@deffn {Directive} %% @var{code} @dots{}
9207Code (after the second @code{%%}) appended to the end of the file,
9208@emph{outside} the parser class.
9209@xref{Java Differences}.
9210@end deffn
9211
9212@deffn {Directive} %@{ @var{code} @dots{} %@}
9213Not supported. Use @code{%code import} instead.
9214@xref{Java Differences}.
9215@end deffn
9216
9217@deffn {Directive} {%define abstract}
9218Whether the parser class is declared @code{abstract}. Default is false.
9219@xref{Java Bison Interface}.
9220@end deffn
9221
9222@deffn {Directive} {%define extends} "@var{superclass}"
9223The superclass of the parser class. Default is none.
9224@xref{Java Bison Interface}.
9225@end deffn
9226
9227@deffn {Directive} {%define final}
9228Whether the parser class is declared @code{final}. Default is false.
9229@xref{Java Bison Interface}.
9230@end deffn
9231
9232@deffn {Directive} {%define implements} "@var{interfaces}"
9233The implemented interfaces of the parser class, a comma-separated list.
9234Default is none.
9235@xref{Java Bison Interface}.
9236@end deffn
9237
9238@deffn {Directive} {%define lex_throws} "@var{exceptions}"
9239The exceptions thrown by the @code{yylex} method of the lexer, a
9240comma-separated list. Default is @code{java.io.IOException}.
9241@xref{Java Scanner Interface}.
9242@end deffn
9243
9244@deffn {Directive} {%define location_type} "@var{class}"
9245The name of the class used for locations (a range between two
9246positions). This class is generated as an inner class of the parser
9247class by @command{bison}. Default is @code{Location}.
9248@xref{Java Location Values}.
9249@end deffn
9250
9251@deffn {Directive} {%define package} "@var{package}"
9252The package to put the parser class in. Default is none.
9253@xref{Java Bison Interface}.
9254@end deffn
9255
9256@deffn {Directive} {%define parser_class_name} "@var{name}"
9257The name of the parser class. Default is @code{YYParser} or
9258@code{@var{name-prefix}Parser}.
9259@xref{Java Bison Interface}.
9260@end deffn
9261
9262@deffn {Directive} {%define position_type} "@var{class}"
9263The name of the class used for positions. This class must be supplied by
9264the user. Default is @code{Position}.
9265@xref{Java Location Values}.
9266@end deffn
9267
9268@deffn {Directive} {%define public}
9269Whether the parser class is declared @code{public}. Default is false.
9270@xref{Java Bison Interface}.
9271@end deffn
9272
9273@deffn {Directive} {%define stype} "@var{class}"
9274The base type of semantic values. Default is @code{Object}.
9275@xref{Java Semantic Values}.
9276@end deffn
9277
9278@deffn {Directive} {%define strictfp}
9279Whether the parser class is declared @code{strictfp}. Default is false.
9280@xref{Java Bison Interface}.
9281@end deffn
9282
9283@deffn {Directive} {%define throws} "@var{exceptions}"
9284The exceptions thrown by user-supplied parser actions and
9285@code{%initial-action}, a comma-separated list. Default is none.
9286@xref{Java Parser Interface}.
9287@end deffn
9288
9289
9290@c ================================================= FAQ
9291
9292@node FAQ
9293@chapter Frequently Asked Questions
9294@cindex frequently asked questions
9295@cindex questions
9296
9297Several questions about Bison come up occasionally. Here some of them
9298are addressed.
9299
9300@menu
9301* Memory Exhausted:: Breaking the Stack Limits
9302* How Can I Reset the Parser:: @code{yyparse} Keeps some State
9303* Strings are Destroyed:: @code{yylval} Loses Track of Strings
9304* Implementing Gotos/Loops:: Control Flow in the Calculator
9305* Multiple start-symbols:: Factoring closely related grammars
9306* Secure? Conform?:: Is Bison @acronym{POSIX} safe?
9307* I can't build Bison:: Troubleshooting
9308* Where can I find help?:: Troubleshouting
9309* Bug Reports:: Troublereporting
9310* More Languages:: Parsers in C++, Java, and so on
9311* Beta Testing:: Experimenting development versions
9312* Mailing Lists:: Meeting other Bison users
9313@end menu
9314
9315@node Memory Exhausted
9316@section Memory Exhausted
9317
9318@display
9319My parser returns with error with a @samp{memory exhausted}
9320message. What can I do?
9321@end display
9322
9323This question is already addressed elsewhere, @xref{Recursion,
9324,Recursive Rules}.
9325
9326@node How Can I Reset the Parser
9327@section How Can I Reset the Parser
9328
9329The following phenomenon has several symptoms, resulting in the
9330following typical questions:
9331
9332@display
9333I invoke @code{yyparse} several times, and on correct input it works
9334properly; but when a parse error is found, all the other calls fail
9335too. How can I reset the error flag of @code{yyparse}?
9336@end display
9337
9338@noindent
9339or
9340
9341@display
9342My parser includes support for an @samp{#include}-like feature, in
9343which case I run @code{yyparse} from @code{yyparse}. This fails
9344although I did specify @code{%define api.pure}.
9345@end display
9346
9347These problems typically come not from Bison itself, but from
9348Lex-generated scanners. Because these scanners use large buffers for
9349speed, they might not notice a change of input file. As a
9350demonstration, consider the following source file,
9351@file{first-line.l}:
9352
9353@verbatim
9354%{
9355#include <stdio.h>
9356#include <stdlib.h>
9357%}
9358%%
9359.*\n ECHO; return 1;
9360%%
9361int
9362yyparse (char const *file)
9363{
9364 yyin = fopen (file, "r");
9365 if (!yyin)
9366 exit (2);
9367 /* One token only. */
9368 yylex ();
9369 if (fclose (yyin) != 0)
9370 exit (3);
9371 return 0;
9372}
9373
9374int
9375main (void)
9376{
9377 yyparse ("input");
9378 yyparse ("input");
9379 return 0;
9380}
9381@end verbatim
9382
9383@noindent
9384If the file @file{input} contains
9385
9386@verbatim
9387input:1: Hello,
9388input:2: World!
9389@end verbatim
9390
9391@noindent
9392then instead of getting the first line twice, you get:
9393
9394@example
9395$ @kbd{flex -ofirst-line.c first-line.l}
9396$ @kbd{gcc -ofirst-line first-line.c -ll}
9397$ @kbd{./first-line}
9398input:1: Hello,
9399input:2: World!
9400@end example
9401
9402Therefore, whenever you change @code{yyin}, you must tell the
9403Lex-generated scanner to discard its current buffer and switch to the
9404new one. This depends upon your implementation of Lex; see its
9405documentation for more. For Flex, it suffices to call
9406@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
9407Flex-generated scanner needs to read from several input streams to
9408handle features like include files, you might consider using Flex
9409functions like @samp{yy_switch_to_buffer} that manipulate multiple
9410input buffers.
9411
9412If your Flex-generated scanner uses start conditions (@pxref{Start
9413conditions, , Start conditions, flex, The Flex Manual}), you might
9414also want to reset the scanner's state, i.e., go back to the initial
9415start condition, through a call to @samp{BEGIN (0)}.
9416
9417@node Strings are Destroyed
9418@section Strings are Destroyed
9419
9420@display
9421My parser seems to destroy old strings, or maybe it loses track of
9422them. Instead of reporting @samp{"foo", "bar"}, it reports
9423@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
9424@end display
9425
9426This error is probably the single most frequent ``bug report'' sent to
9427Bison lists, but is only concerned with a misunderstanding of the role
9428of the scanner. Consider the following Lex code:
9429
9430@verbatim
9431%{
9432#include <stdio.h>
9433char *yylval = NULL;
9434%}
9435%%
9436.* yylval = yytext; return 1;
9437\n /* IGNORE */
9438%%
9439int
9440main ()
9441{
9442 /* Similar to using $1, $2 in a Bison action. */
9443 char *fst = (yylex (), yylval);
9444 char *snd = (yylex (), yylval);
9445 printf ("\"%s\", \"%s\"\n", fst, snd);
9446 return 0;
9447}
9448@end verbatim
9449
9450If you compile and run this code, you get:
9451
9452@example
9453$ @kbd{flex -osplit-lines.c split-lines.l}
9454$ @kbd{gcc -osplit-lines split-lines.c -ll}
9455$ @kbd{printf 'one\ntwo\n' | ./split-lines}
9456"one
9457two", "two"
9458@end example
9459
9460@noindent
9461this is because @code{yytext} is a buffer provided for @emph{reading}
9462in the action, but if you want to keep it, you have to duplicate it
9463(e.g., using @code{strdup}). Note that the output may depend on how
9464your implementation of Lex handles @code{yytext}. For instance, when
9465given the Lex compatibility option @option{-l} (which triggers the
9466option @samp{%array}) Flex generates a different behavior:
9467
9468@example
9469$ @kbd{flex -l -osplit-lines.c split-lines.l}
9470$ @kbd{gcc -osplit-lines split-lines.c -ll}
9471$ @kbd{printf 'one\ntwo\n' | ./split-lines}
9472"two", "two"
9473@end example
9474
9475
9476@node Implementing Gotos/Loops
9477@section Implementing Gotos/Loops
9478
9479@display
9480My simple calculator supports variables, assignments, and functions,
9481but how can I implement gotos, or loops?
9482@end display
9483
9484Although very pedagogical, the examples included in the document blur
9485the distinction to make between the parser---whose job is to recover
9486the structure of a text and to transmit it to subsequent modules of
9487the program---and the processing (such as the execution) of this
9488structure. This works well with so called straight line programs,
9489i.e., precisely those that have a straightforward execution model:
9490execute simple instructions one after the others.
9491
9492@cindex abstract syntax tree
9493@cindex @acronym{AST}
9494If you want a richer model, you will probably need to use the parser
9495to construct a tree that does represent the structure it has
9496recovered; this tree is usually called the @dfn{abstract syntax tree},
9497or @dfn{@acronym{AST}} for short. Then, walking through this tree,
9498traversing it in various ways, will enable treatments such as its
9499execution or its translation, which will result in an interpreter or a
9500compiler.
9501
9502This topic is way beyond the scope of this manual, and the reader is
9503invited to consult the dedicated literature.
9504
9505
9506@node Multiple start-symbols
9507@section Multiple start-symbols
9508
9509@display
9510I have several closely related grammars, and I would like to share their
9511implementations. In fact, I could use a single grammar but with
9512multiple entry points.
9513@end display
9514
9515Bison does not support multiple start-symbols, but there is a very
9516simple means to simulate them. If @code{foo} and @code{bar} are the two
9517pseudo start-symbols, then introduce two new tokens, say
9518@code{START_FOO} and @code{START_BAR}, and use them as switches from the
9519real start-symbol:
9520
9521@example
9522%token START_FOO START_BAR;
9523%start start;
9524start: START_FOO foo
9525 | START_BAR bar;
9526@end example
9527
9528These tokens prevents the introduction of new conflicts. As far as the
9529parser goes, that is all that is needed.
9530
9531Now the difficult part is ensuring that the scanner will send these
9532tokens first. If your scanner is hand-written, that should be
9533straightforward. If your scanner is generated by Lex, them there is
9534simple means to do it: recall that anything between @samp{%@{ ... %@}}
9535after the first @code{%%} is copied verbatim in the top of the generated
9536@code{yylex} function. Make sure a variable @code{start_token} is
9537available in the scanner (e.g., a global variable or using
9538@code{%lex-param} etc.), and use the following:
9539
9540@example
9541 /* @r{Prologue.} */
9542%%
9543%@{
9544 if (start_token)
9545 @{
9546 int t = start_token;
9547 start_token = 0;
9548 return t;
9549 @}
9550%@}
9551 /* @r{The rules.} */
9552@end example
9553
9554
9555@node Secure? Conform?
9556@section Secure? Conform?
9557
9558@display
9559Is Bison secure? Does it conform to POSIX?
9560@end display
9561
9562If you're looking for a guarantee or certification, we don't provide it.
9563However, Bison is intended to be a reliable program that conforms to the
9564@acronym{POSIX} specification for Yacc. If you run into problems,
9565please send us a bug report.
9566
9567@node I can't build Bison
9568@section I can't build Bison
9569
9570@display
9571I can't build Bison because @command{make} complains that
9572@code{msgfmt} is not found.
9573What should I do?
9574@end display
9575
9576Like most GNU packages with internationalization support, that feature
9577is turned on by default. If you have problems building in the @file{po}
9578subdirectory, it indicates that your system's internationalization
9579support is lacking. You can re-configure Bison with
9580@option{--disable-nls} to turn off this support, or you can install GNU
9581gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
9582Bison. See the file @file{ABOUT-NLS} for more information.
9583
9584
9585@node Where can I find help?
9586@section Where can I find help?
9587
9588@display
9589I'm having trouble using Bison. Where can I find help?
9590@end display
9591
9592First, read this fine manual. Beyond that, you can send mail to
9593@email{help-bison@@gnu.org}. This mailing list is intended to be
9594populated with people who are willing to answer questions about using
9595and installing Bison. Please keep in mind that (most of) the people on
9596the list have aspects of their lives which are not related to Bison (!),
9597so you may not receive an answer to your question right away. This can
9598be frustrating, but please try not to honk them off; remember that any
9599help they provide is purely voluntary and out of the kindness of their
9600hearts.
9601
9602@node Bug Reports
9603@section Bug Reports
9604
9605@display
9606I found a bug. What should I include in the bug report?
9607@end display
9608
9609Before you send a bug report, make sure you are using the latest
9610version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
9611mirrors. Be sure to include the version number in your bug report. If
9612the bug is present in the latest version but not in a previous version,
9613try to determine the most recent version which did not contain the bug.
9614
9615If the bug is parser-related, you should include the smallest grammar
9616you can which demonstrates the bug. The grammar file should also be
9617complete (i.e., I should be able to run it through Bison without having
9618to edit or add anything). The smaller and simpler the grammar, the
9619easier it will be to fix the bug.
9620
9621Include information about your compilation environment, including your
9622operating system's name and version and your compiler's name and
9623version. If you have trouble compiling, you should also include a
9624transcript of the build session, starting with the invocation of
9625`configure'. Depending on the nature of the bug, you may be asked to
9626send additional files as well (such as `config.h' or `config.cache').
9627
9628Patches are most welcome, but not required. That is, do not hesitate to
9629send a bug report just because you can not provide a fix.
9630
9631Send bug reports to @email{bug-bison@@gnu.org}.
9632
9633@node More Languages
9634@section More Languages
9635
9636@display
9637Will Bison ever have C++ and Java support? How about @var{insert your
9638favorite language here}?
9639@end display
9640
9641C++ and Java support is there now, and is documented. We'd love to add other
9642languages; contributions are welcome.
9643
9644@node Beta Testing
9645@section Beta Testing
9646
9647@display
9648What is involved in being a beta tester?
9649@end display
9650
9651It's not terribly involved. Basically, you would download a test
9652release, compile it, and use it to build and run a parser or two. After
9653that, you would submit either a bug report or a message saying that
9654everything is okay. It is important to report successes as well as
9655failures because test releases eventually become mainstream releases,
9656but only if they are adequately tested. If no one tests, development is
9657essentially halted.
9658
9659Beta testers are particularly needed for operating systems to which the
9660developers do not have easy access. They currently have easy access to
9661recent GNU/Linux and Solaris versions. Reports about other operating
9662systems are especially welcome.
9663
9664@node Mailing Lists
9665@section Mailing Lists
9666
9667@display
9668How do I join the help-bison and bug-bison mailing lists?
9669@end display
9670
9671See @url{http://lists.gnu.org/}.
9672
9673@c ================================================= Table of Symbols
9674
9675@node Table of Symbols
9676@appendix Bison Symbols
9677@cindex Bison symbols, table of
9678@cindex symbols in Bison, table of
9679
9680@deffn {Variable} @@$
9681In an action, the location of the left-hand side of the rule.
9682@xref{Locations, , Locations Overview}.
9683@end deffn
9684
9685@deffn {Variable} @@@var{n}
9686In an action, the location of the @var{n}-th symbol of the right-hand
9687side of the rule. @xref{Locations, , Locations Overview}.
9688@end deffn
9689
9690@deffn {Variable} $$
9691In an action, the semantic value of the left-hand side of the rule.
9692@xref{Actions}.
9693@end deffn
9694
9695@deffn {Variable} $@var{n}
9696In an action, the semantic value of the @var{n}-th symbol of the
9697right-hand side of the rule. @xref{Actions}.
9698@end deffn
9699
9700@deffn {Delimiter} %%
9701Delimiter used to separate the grammar rule section from the
9702Bison declarations section or the epilogue.
9703@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
9704@end deffn
9705
9706@c Don't insert spaces, or check the DVI output.
9707@deffn {Delimiter} %@{@var{code}%@}
9708All code listed between @samp{%@{} and @samp{%@}} is copied directly to
9709the output file uninterpreted. Such code forms the prologue of the input
9710file. @xref{Grammar Outline, ,Outline of a Bison
9711Grammar}.
9712@end deffn
9713
9714@deffn {Construct} /*@dots{}*/
9715Comment delimiters, as in C.
9716@end deffn
9717
9718@deffn {Delimiter} :
9719Separates a rule's result from its components. @xref{Rules, ,Syntax of
9720Grammar Rules}.
9721@end deffn
9722
9723@deffn {Delimiter} ;
9724Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
9725@end deffn
9726
9727@deffn {Delimiter} |
9728Separates alternate rules for the same result nonterminal.
9729@xref{Rules, ,Syntax of Grammar Rules}.
9730@end deffn
9731
9732@deffn {Directive} <*>
9733Used to define a default tagged @code{%destructor} or default tagged
9734@code{%printer}.
9735
9736This feature is experimental.
9737More user feedback will help to determine whether it should become a permanent
9738feature.
9739
9740@xref{Destructor Decl, , Freeing Discarded Symbols}.
9741@end deffn
9742
9743@deffn {Directive} <>
9744Used to define a default tagless @code{%destructor} or default tagless
9745@code{%printer}.
9746
9747This feature is experimental.
9748More user feedback will help to determine whether it should become a permanent
9749feature.
9750
9751@xref{Destructor Decl, , Freeing Discarded Symbols}.
9752@end deffn
9753
9754@deffn {Symbol} $accept
9755The predefined nonterminal whose only rule is @samp{$accept: @var{start}
9756$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
9757Start-Symbol}. It cannot be used in the grammar.
9758@end deffn
9759
9760@deffn {Directive} %code @{@var{code}@}
9761@deffnx {Directive} %code @var{qualifier} @{@var{code}@}
9762Insert @var{code} verbatim into output parser source.
9763@xref{Decl Summary,,%code}.
9764@end deffn
9765
9766@deffn {Directive} %debug
9767Equip the parser for debugging. @xref{Decl Summary}.
9768@end deffn
9769
9770@ifset defaultprec
9771@deffn {Directive} %default-prec
9772Assign a precedence to rules that lack an explicit @samp{%prec}
9773modifier. @xref{Contextual Precedence, ,Context-Dependent
9774Precedence}.
9775@end deffn
9776@end ifset
9777
9778@deffn {Directive} %define @var{define-variable}
9779@deffnx {Directive} %define @var{define-variable} @var{value}
9780Define a variable to adjust Bison's behavior.
9781@xref{Decl Summary,,%define}.
9782@end deffn
9783
9784@deffn {Directive} %defines
9785Bison declaration to create a header file meant for the scanner.
9786@xref{Decl Summary}.
9787@end deffn
9788
9789@deffn {Directive} %defines @var{defines-file}
9790Same as above, but save in the file @var{defines-file}.
9791@xref{Decl Summary}.
9792@end deffn
9793
9794@deffn {Directive} %destructor
9795Specify how the parser should reclaim the memory associated to
9796discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
9797@end deffn
9798
9799@deffn {Directive} %dprec
9800Bison declaration to assign a precedence to a rule that is used at parse
9801time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
9802@acronym{GLR} Parsers}.
9803@end deffn
9804
9805@deffn {Symbol} $end
9806The predefined token marking the end of the token stream. It cannot be
9807used in the grammar.
9808@end deffn
9809
9810@deffn {Symbol} error
9811A token name reserved for error recovery. This token may be used in
9812grammar rules so as to allow the Bison parser to recognize an error in
9813the grammar without halting the process. In effect, a sentence
9814containing an error may be recognized as valid. On a syntax error, the
9815token @code{error} becomes the current lookahead token. Actions
9816corresponding to @code{error} are then executed, and the lookahead
9817token is reset to the token that originally caused the violation.
9818@xref{Error Recovery}.
9819@end deffn
9820
9821@deffn {Directive} %error-verbose
9822Bison declaration to request verbose, specific error message strings
9823when @code{yyerror} is called.
9824@end deffn
9825
9826@deffn {Directive} %file-prefix "@var{prefix}"
9827Bison declaration to set the prefix of the output files. @xref{Decl
9828Summary}.
9829@end deffn
9830
9831@deffn {Directive} %glr-parser
9832Bison declaration to produce a @acronym{GLR} parser. @xref{GLR
9833Parsers, ,Writing @acronym{GLR} Parsers}.
9834@end deffn
9835
9836@deffn {Directive} %initial-action
9837Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
9838@end deffn
9839
9840@deffn {Directive} %language
9841Specify the programming language for the generated parser.
9842@xref{Decl Summary}.
9843@end deffn
9844
9845@deffn {Directive} %left
9846Bison declaration to assign left associativity to token(s).
9847@xref{Precedence Decl, ,Operator Precedence}.
9848@end deffn
9849
9850@deffn {Directive} %lex-param @{@var{argument-declaration}@}
9851Bison declaration to specifying an additional parameter that
9852@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
9853for Pure Parsers}.
9854@end deffn
9855
9856@deffn {Directive} %merge
9857Bison declaration to assign a merging function to a rule. If there is a
9858reduce/reduce conflict with a rule having the same merging function, the
9859function is applied to the two semantic values to get a single result.
9860@xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}.
9861@end deffn
9862
9863@deffn {Directive} %name-prefix "@var{prefix}"
9864Bison declaration to rename the external symbols. @xref{Decl Summary}.
9865@end deffn
9866
9867@ifset defaultprec
9868@deffn {Directive} %no-default-prec
9869Do not assign a precedence to rules that lack an explicit @samp{%prec}
9870modifier. @xref{Contextual Precedence, ,Context-Dependent
9871Precedence}.
9872@end deffn
9873@end ifset
9874
9875@deffn {Directive} %no-lines
9876Bison declaration to avoid generating @code{#line} directives in the
9877parser file. @xref{Decl Summary}.
9878@end deffn
9879
9880@deffn {Directive} %nonassoc
9881Bison declaration to assign nonassociativity to token(s).
9882@xref{Precedence Decl, ,Operator Precedence}.
9883@end deffn
9884
9885@deffn {Directive} %output "@var{file}"
9886Bison declaration to set the name of the parser file. @xref{Decl
9887Summary}.
9888@end deffn
9889
9890@deffn {Directive} %parse-param @{@var{argument-declaration}@}
9891Bison declaration to specifying an additional parameter that
9892@code{yyparse} should accept. @xref{Parser Function,, The Parser
9893Function @code{yyparse}}.
9894@end deffn
9895
9896@deffn {Directive} %prec
9897Bison declaration to assign a precedence to a specific rule.
9898@xref{Contextual Precedence, ,Context-Dependent Precedence}.
9899@end deffn
9900
9901@deffn {Directive} %pure-parser
9902Deprecated version of @code{%define api.pure} (@pxref{Decl Summary, ,%define}),
9903for which Bison is more careful to warn about unreasonable usage.
9904@end deffn
9905
9906@deffn {Directive} %require "@var{version}"
9907Require version @var{version} or higher of Bison. @xref{Require Decl, ,
9908Require a Version of Bison}.
9909@end deffn
9910
9911@deffn {Directive} %right
9912Bison declaration to assign right associativity to token(s).
9913@xref{Precedence Decl, ,Operator Precedence}.
9914@end deffn
9915
9916@deffn {Directive} %skeleton
9917Specify the skeleton to use; usually for development.
9918@xref{Decl Summary}.
9919@end deffn
9920
9921@deffn {Directive} %start
9922Bison declaration to specify the start symbol. @xref{Start Decl, ,The
9923Start-Symbol}.
9924@end deffn
9925
9926@deffn {Directive} %token
9927Bison declaration to declare token(s) without specifying precedence.
9928@xref{Token Decl, ,Token Type Names}.
9929@end deffn
9930
9931@deffn {Directive} %token-table
9932Bison declaration to include a token name table in the parser file.
9933@xref{Decl Summary}.
9934@end deffn
9935
9936@deffn {Directive} %type
9937Bison declaration to declare nonterminals. @xref{Type Decl,
9938,Nonterminal Symbols}.
9939@end deffn
9940
9941@deffn {Symbol} $undefined
9942The predefined token onto which all undefined values returned by
9943@code{yylex} are mapped. It cannot be used in the grammar, rather, use
9944@code{error}.
9945@end deffn
9946
9947@deffn {Directive} %union
9948Bison declaration to specify several possible data types for semantic
9949values. @xref{Union Decl, ,The Collection of Value Types}.
9950@end deffn
9951
9952@deffn {Macro} YYABORT
9953Macro to pretend that an unrecoverable syntax error has occurred, by
9954making @code{yyparse} return 1 immediately. The error reporting
9955function @code{yyerror} is not called. @xref{Parser Function, ,The
9956Parser Function @code{yyparse}}.
9957
9958For Java parsers, this functionality is invoked using @code{return YYABORT;}
9959instead.
9960@end deffn
9961
9962@deffn {Macro} YYACCEPT
9963Macro to pretend that a complete utterance of the language has been
9964read, by making @code{yyparse} return 0 immediately.
9965@xref{Parser Function, ,The Parser Function @code{yyparse}}.
9966
9967For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
9968instead.
9969@end deffn
9970
9971@deffn {Macro} YYBACKUP
9972Macro to discard a value from the parser stack and fake a lookahead
9973token. @xref{Action Features, ,Special Features for Use in Actions}.
9974@end deffn
9975
9976@deffn {Variable} yychar
9977External integer variable that contains the integer value of the
9978lookahead token. (In a pure parser, it is a local variable within
9979@code{yyparse}.) Error-recovery rule actions may examine this variable.
9980@xref{Action Features, ,Special Features for Use in Actions}.
9981@end deffn
9982
9983@deffn {Variable} yyclearin
9984Macro used in error-recovery rule actions. It clears the previous
9985lookahead token. @xref{Error Recovery}.
9986@end deffn
9987
9988@deffn {Macro} YYDEBUG
9989Macro to define to equip the parser with tracing code. @xref{Tracing,
9990,Tracing Your Parser}.
9991@end deffn
9992
9993@deffn {Variable} yydebug
9994External integer variable set to zero by default. If @code{yydebug}
9995is given a nonzero value, the parser will output information on input
9996symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
9997@end deffn
9998
9999@deffn {Macro} yyerrok
10000Macro to cause parser to recover immediately to its normal mode
10001after a syntax error. @xref{Error Recovery}.
10002@end deffn
10003
10004@deffn {Macro} YYERROR
10005Macro to pretend that a syntax error has just been detected: call
10006@code{yyerror} and then perform normal error recovery if possible
10007(@pxref{Error Recovery}), or (if recovery is impossible) make
10008@code{yyparse} return 1. @xref{Error Recovery}.
10009
10010For Java parsers, this functionality is invoked using @code{return YYERROR;}
10011instead.
10012@end deffn
10013
10014@deffn {Function} yyerror
10015User-supplied function to be called by @code{yyparse} on error.
10016@xref{Error Reporting, ,The Error
10017Reporting Function @code{yyerror}}.
10018@end deffn
10019
10020@deffn {Macro} YYERROR_VERBOSE
10021An obsolete macro that you define with @code{#define} in the prologue
10022to request verbose, specific error message strings
10023when @code{yyerror} is called. It doesn't matter what definition you
10024use for @code{YYERROR_VERBOSE}, just whether you define it. Using
10025@code{%error-verbose} is preferred.
10026@end deffn
10027
10028@deffn {Macro} YYINITDEPTH
10029Macro for specifying the initial size of the parser stack.
10030@xref{Memory Management}.
10031@end deffn
10032
10033@deffn {Function} yylex
10034User-supplied lexical analyzer function, called with no arguments to get
10035the next token. @xref{Lexical, ,The Lexical Analyzer Function
10036@code{yylex}}.
10037@end deffn
10038
10039@deffn {Macro} YYLEX_PARAM
10040An obsolete macro for specifying an extra argument (or list of extra
10041arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
10042macro is deprecated, and is supported only for Yacc like parsers.
10043@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
10044@end deffn
10045
10046@deffn {Variable} yylloc
10047External variable in which @code{yylex} should place the line and column
10048numbers associated with a token. (In a pure parser, it is a local
10049variable within @code{yyparse}, and its address is passed to
10050@code{yylex}.)
10051You can ignore this variable if you don't use the @samp{@@} feature in the
10052grammar actions.
10053@xref{Token Locations, ,Textual Locations of Tokens}.
10054In semantic actions, it stores the location of the lookahead token.
10055@xref{Actions and Locations, ,Actions and Locations}.
10056@end deffn
10057
10058@deffn {Type} YYLTYPE
10059Data type of @code{yylloc}; by default, a structure with four
10060members. @xref{Location Type, , Data Types of Locations}.
10061@end deffn
10062
10063@deffn {Variable} yylval
10064External variable in which @code{yylex} should place the semantic
10065value associated with a token. (In a pure parser, it is a local
10066variable within @code{yyparse}, and its address is passed to
10067@code{yylex}.)
10068@xref{Token Values, ,Semantic Values of Tokens}.
10069In semantic actions, it stores the semantic value of the lookahead token.
10070@xref{Actions, ,Actions}.
10071@end deffn
10072
10073@deffn {Macro} YYMAXDEPTH
10074Macro for specifying the maximum size of the parser stack. @xref{Memory
10075Management}.
10076@end deffn
10077
10078@deffn {Variable} yynerrs
10079Global variable which Bison increments each time it reports a syntax error.
10080(In a pure parser, it is a local variable within @code{yyparse}. In a
10081pure push parser, it is a member of yypstate.)
10082@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
10083@end deffn
10084
10085@deffn {Function} yyparse
10086The parser function produced by Bison; call this function to start
10087parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
10088@end deffn
10089
10090@deffn {Function} yypstate_delete
10091The function to delete a parser instance, produced by Bison in push mode;
10092call this function to delete the memory associated with a parser.
10093@xref{Parser Delete Function, ,The Parser Delete Function
10094@code{yypstate_delete}}.
10095(The current push parsing interface is experimental and may evolve.
10096More user feedback will help to stabilize it.)
10097@end deffn
10098
10099@deffn {Function} yypstate_new
10100The function to create a parser instance, produced by Bison in push mode;
10101call this function to create a new parser.
10102@xref{Parser Create Function, ,The Parser Create Function
10103@code{yypstate_new}}.
10104(The current push parsing interface is experimental and may evolve.
10105More user feedback will help to stabilize it.)
10106@end deffn
10107
10108@deffn {Function} yypull_parse
10109The parser function produced by Bison in push mode; call this function to
10110parse the rest of the input stream.
10111@xref{Pull Parser Function, ,The Pull Parser Function
10112@code{yypull_parse}}.
10113(The current push parsing interface is experimental and may evolve.
10114More user feedback will help to stabilize it.)
10115@end deffn
10116
10117@deffn {Function} yypush_parse
10118The parser function produced by Bison in push mode; call this function to
10119parse a single token. @xref{Push Parser Function, ,The Push Parser Function
10120@code{yypush_parse}}.
10121(The current push parsing interface is experimental and may evolve.
10122More user feedback will help to stabilize it.)
10123@end deffn
10124
10125@deffn {Macro} YYPARSE_PARAM
10126An obsolete macro for specifying the name of a parameter that
10127@code{yyparse} should accept. The use of this macro is deprecated, and
10128is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
10129Conventions for Pure Parsers}.
10130@end deffn
10131
10132@deffn {Macro} YYRECOVERING
10133The expression @code{YYRECOVERING ()} yields 1 when the parser
10134is recovering from a syntax error, and 0 otherwise.
10135@xref{Action Features, ,Special Features for Use in Actions}.
10136@end deffn
10137
10138@deffn {Macro} YYSTACK_USE_ALLOCA
10139Macro used to control the use of @code{alloca} when the C
10140@acronym{LALR}(1) parser needs to extend its stacks. If defined to 0,
10141the parser will use @code{malloc} to extend its stacks. If defined to
101421, the parser will use @code{alloca}. Values other than 0 and 1 are
10143reserved for future Bison extensions. If not defined,
10144@code{YYSTACK_USE_ALLOCA} defaults to 0.
10145
10146In the all-too-common case where your code may run on a host with a
10147limited stack and with unreliable stack-overflow checking, you should
10148set @code{YYMAXDEPTH} to a value that cannot possibly result in
10149unchecked stack overflow on any of your target hosts when
10150@code{alloca} is called. You can inspect the code that Bison
10151generates in order to determine the proper numeric values. This will
10152require some expertise in low-level implementation details.
10153@end deffn
10154
10155@deffn {Type} YYSTYPE
10156Data type of semantic values; @code{int} by default.
10157@xref{Value Type, ,Data Types of Semantic Values}.
10158@end deffn
10159
10160@node Glossary
10161@appendix Glossary
10162@cindex glossary
10163
10164@table @asis
10165@item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'')
10166Formal method of specifying context-free grammars originally proposed
10167by John Backus, and slightly improved by Peter Naur in his 1960-01-02
10168committee document contributing to what became the Algol 60 report.
10169@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10170
10171@item Context-free grammars
10172Grammars specified as rules that can be applied regardless of context.
10173Thus, if there is a rule which says that an integer can be used as an
10174expression, integers are allowed @emph{anywhere} an expression is
10175permitted. @xref{Language and Grammar, ,Languages and Context-Free
10176Grammars}.
10177
10178@item Dynamic allocation
10179Allocation of memory that occurs during execution, rather than at
10180compile time or on entry to a function.
10181
10182@item Empty string
10183Analogous to the empty set in set theory, the empty string is a
10184character string of length zero.
10185
10186@item Finite-state stack machine
10187A ``machine'' that has discrete states in which it is said to exist at
10188each instant in time. As input to the machine is processed, the
10189machine moves from state to state as specified by the logic of the
10190machine. In the case of the parser, the input is the language being
10191parsed, and the states correspond to various stages in the grammar
10192rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
10193
10194@item Generalized @acronym{LR} (@acronym{GLR})
10195A parsing algorithm that can handle all context-free grammars, including those
10196that are not @acronym{LALR}(1). It resolves situations that Bison's
10197usual @acronym{LALR}(1)
10198algorithm cannot by effectively splitting off multiple parsers, trying all
10199possible parsers, and discarding those that fail in the light of additional
10200right context. @xref{Generalized LR Parsing, ,Generalized
10201@acronym{LR} Parsing}.
10202
10203@item Grouping
10204A language construct that is (in general) grammatically divisible;
10205for example, `expression' or `declaration' in C@.
10206@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10207
10208@item Infix operator
10209An arithmetic operator that is placed between the operands on which it
10210performs some operation.
10211
10212@item Input stream
10213A continuous flow of data between devices or programs.
10214
10215@item Language construct
10216One of the typical usage schemas of the language. For example, one of
10217the constructs of the C language is the @code{if} statement.
10218@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10219
10220@item Left associativity
10221Operators having left associativity are analyzed from left to right:
10222@samp{a+b+c} first computes @samp{a+b} and then combines with
10223@samp{c}. @xref{Precedence, ,Operator Precedence}.
10224
10225@item Left recursion
10226A rule whose result symbol is also its first component symbol; for
10227example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
10228Rules}.
10229
10230@item Left-to-right parsing
10231Parsing a sentence of a language by analyzing it token by token from
10232left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
10233
10234@item Lexical analyzer (scanner)
10235A function that reads an input stream and returns tokens one by one.
10236@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
10237
10238@item Lexical tie-in
10239A flag, set by actions in the grammar rules, which alters the way
10240tokens are parsed. @xref{Lexical Tie-ins}.
10241
10242@item Literal string token
10243A token which consists of two or more fixed characters. @xref{Symbols}.
10244
10245@item Lookahead token
10246A token already read but not yet shifted. @xref{Lookahead, ,Lookahead
10247Tokens}.
10248
10249@item @acronym{LALR}(1)
10250The class of context-free grammars that Bison (like most other parser
10251generators) can handle; a subset of @acronym{LR}(1). @xref{Mystery
10252Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
10253
10254@item @acronym{LR}(1)
10255The class of context-free grammars in which at most one token of
10256lookahead is needed to disambiguate the parsing of any piece of input.
10257
10258@item Nonterminal symbol
10259A grammar symbol standing for a grammatical construct that can
10260be expressed through rules in terms of smaller constructs; in other
10261words, a construct that is not a token. @xref{Symbols}.
10262
10263@item Parser
10264A function that recognizes valid sentences of a language by analyzing
10265the syntax structure of a set of tokens passed to it from a lexical
10266analyzer.
10267
10268@item Postfix operator
10269An arithmetic operator that is placed after the operands upon which it
10270performs some operation.
10271
10272@item Reduction
10273Replacing a string of nonterminals and/or terminals with a single
10274nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
10275Parser Algorithm}.
10276
10277@item Reentrant
10278A reentrant subprogram is a subprogram which can be in invoked any
10279number of times in parallel, without interference between the various
10280invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
10281
10282@item Reverse polish notation
10283A language in which all operators are postfix operators.
10284
10285@item Right recursion
10286A rule whose result symbol is also its last component symbol; for
10287example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
10288Rules}.
10289
10290@item Semantics
10291In computer languages, the semantics are specified by the actions
10292taken for each instance of the language, i.e., the meaning of
10293each statement. @xref{Semantics, ,Defining Language Semantics}.
10294
10295@item Shift
10296A parser is said to shift when it makes the choice of analyzing
10297further input from the stream rather than reducing immediately some
10298already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
10299
10300@item Single-character literal
10301A single character that is recognized and interpreted as is.
10302@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
10303
10304@item Start symbol
10305The nonterminal symbol that stands for a complete valid utterance in
10306the language being parsed. The start symbol is usually listed as the
10307first nonterminal symbol in a language specification.
10308@xref{Start Decl, ,The Start-Symbol}.
10309
10310@item Symbol table
10311A data structure where symbol names and associated data are stored
10312during parsing to allow for recognition and use of existing
10313information in repeated uses of a symbol. @xref{Multi-function Calc}.
10314
10315@item Syntax error
10316An error encountered during parsing of an input stream due to invalid
10317syntax. @xref{Error Recovery}.
10318
10319@item Token
10320A basic, grammatically indivisible unit of a language. The symbol
10321that describes a token in the grammar is a terminal symbol.
10322The input of the Bison parser is a stream of tokens which comes from
10323the lexical analyzer. @xref{Symbols}.
10324
10325@item Terminal symbol
10326A grammar symbol that has no rules in the grammar and therefore is
10327grammatically indivisible. The piece of text it represents is a token.
10328@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10329@end table
10330
10331@node Copying This Manual
10332@appendix Copying This Manual
10333@include fdl.texi
10334
10335@node Index
10336@unnumbered Index
10337
10338@printindex cp
10339
10340@bye
10341
10342@c LocalWords: texinfo setfilename settitle setchapternewpage finalout
10343@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex
10344@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry
10345@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa
10346@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc
10347@c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex
10348@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref
10349@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex
10350@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge
10351@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG
10352@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit
10353@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok
10354@c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln
10355@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym
10356@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof
10357@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum
10358@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype
10359@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs
10360@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES
10361@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param
10362@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP
10363@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword
10364@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH
10365@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype
10366@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args
10367@c LocalWords: infile ypp yxx outfile itemx tex leaderfill
10368@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll
10369@c LocalWords: nbar yytext fst snd osplit ntwo strdup AST
10370@c LocalWords: YYSTACK DVI fdl printindex