* tests/glr-regression.at (Leaked semantic values when reporting
[bison.git] / doc / bison.texinfo
CommitLineData
bfa74976
RS
1\input texinfo @c -*-texinfo-*-
2@comment %**start of header
3@setfilename bison.info
df1af54c
JT
4@include version.texi
5@settitle Bison @value{VERSION}
bfa74976
RS
6@setchapternewpage odd
7
5378c3e7 8@finalout
5378c3e7 9
13863333 10@c SMALL BOOK version
bfa74976 11@c This edition has been formatted so that you can format and print it in
13863333 12@c the smallbook format.
bfa74976
RS
13@c @smallbook
14
bfa74976
RS
15@c Set following if you have the new `shorttitlepage' command
16@c @clear shorttitlepage-enabled
17@c @set shorttitlepage-enabled
18
91d2c560
PE
19@c Set following if you want to document %default-prec and %no-default-prec.
20@c This feature is experimental and may change in future Bison versions.
21@c @set defaultprec
22
bfa74976
RS
23@c ISPELL CHECK: done, 14 Jan 1993 --bob
24
25@c Check COPYRIGHT dates. should be updated in the titlepage, ifinfo
26@c titlepage; should NOT be changed in the GPL. --mew
27
ec3bc396 28@c FIXME: I don't understand this `iftex'. Obsolete? --akim.
bfa74976
RS
29@iftex
30@syncodeindex fn cp
31@syncodeindex vr cp
32@syncodeindex tp cp
33@end iftex
34@ifinfo
35@synindex fn cp
36@synindex vr cp
37@synindex tp cp
38@end ifinfo
39@comment %**end of header
40
fae437e8 41@copying
bd773d73 42
c827f760
PE
43This manual is for @acronym{GNU} Bison (version @value{VERSION},
44@value{UPDATED}), the @acronym{GNU} parser generator.
fae437e8 45
a06ea4aa 46Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
32c29292 471999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation, Inc.
fae437e8
AD
48
49@quotation
50Permission is granted to copy, distribute and/or modify this document
c827f760 51under the terms of the @acronym{GNU} Free Documentation License,
592fde95 52Version 1.2 or any later version published by the Free Software
c827f760
PE
53Foundation; with no Invariant Sections, with the Front-Cover texts
54being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in
55(a) below. A copy of the license is included in the section entitled
56``@acronym{GNU} Free Documentation License.''
57
58(a) The @acronym{FSF}'s Back-Cover Text is: ``You have freedom to copy
59and modify this @acronym{GNU} Manual, like @acronym{GNU} software.
60Copies published by the Free Software Foundation raise funds for
61@acronym{GNU} development.''
fae437e8
AD
62@end quotation
63@end copying
64
e62f1a89 65@dircategory Software development
fae437e8 66@direntry
c827f760 67* bison: (bison). @acronym{GNU} parser generator (Yacc replacement).
fae437e8 68@end direntry
bfa74976
RS
69
70@ifset shorttitlepage-enabled
71@shorttitlepage Bison
72@end ifset
73@titlepage
74@title Bison
c827f760 75@subtitle The Yacc-compatible Parser Generator
df1af54c 76@subtitle @value{UPDATED}, Bison Version @value{VERSION}
bfa74976
RS
77
78@author by Charles Donnelly and Richard Stallman
79
80@page
81@vskip 0pt plus 1filll
fae437e8 82@insertcopying
bfa74976
RS
83@sp 2
84Published by the Free Software Foundation @*
0fb669f9
PE
8551 Franklin Street, Fifth Floor @*
86Boston, MA 02110-1301 USA @*
9ecbd125 87Printed copies are available from the Free Software Foundation.@*
c827f760 88@acronym{ISBN} 1-882114-44-2
bfa74976
RS
89@sp 2
90Cover art by Etienne Suvasa.
91@end titlepage
d5796688
JT
92
93@contents
bfa74976 94
342b8b6e
AD
95@ifnottex
96@node Top
97@top Bison
fae437e8 98@insertcopying
342b8b6e 99@end ifnottex
bfa74976
RS
100
101@menu
13863333
AD
102* Introduction::
103* Conditions::
c827f760 104* Copying:: The @acronym{GNU} General Public License says
bfa74976
RS
105 how you can copy and share Bison
106
107Tutorial sections:
108* Concepts:: Basic concepts for understanding Bison.
109* Examples:: Three simple explained examples of using Bison.
110
111Reference sections:
112* Grammar File:: Writing Bison declarations and rules.
113* Interface:: C-language interface to the parser function @code{yyparse}.
114* Algorithm:: How the Bison parser works at run-time.
115* Error Recovery:: Writing rules for error recovery.
116* Context Dependency:: What to do if your language syntax is too
117 messy for Bison to handle straightforwardly.
ec3bc396 118* Debugging:: Understanding or debugging Bison parsers.
bfa74976 119* Invocation:: How to run Bison (to produce the parser source file).
12545799
AD
120* C++ Language Interface:: Creating C++ parser objects.
121* FAQ:: Frequently Asked Questions
bfa74976
RS
122* Table of Symbols:: All the keywords of the Bison language are explained.
123* Glossary:: Basic concepts are explained.
f2b5126e 124* Copying This Manual:: License for copying this manual.
bfa74976
RS
125* Index:: Cross-references to the text.
126
93dd49ab
PE
127@detailmenu
128 --- The Detailed Node Listing ---
bfa74976
RS
129
130The Concepts of Bison
131
132* Language and Grammar:: Languages and context-free grammars,
133 as mathematical ideas.
134* Grammar in Bison:: How we represent grammars for Bison's sake.
135* Semantic Values:: Each token or syntactic grouping can have
136 a semantic value (the value of an integer,
137 the name of an identifier, etc.).
138* Semantic Actions:: Each rule can have an action containing C code.
99a9344e 139* GLR Parsers:: Writing parsers for general context-free languages.
93dd49ab 140* Locations Overview:: Tracking Locations.
bfa74976
RS
141* Bison Parser:: What are Bison's input and output,
142 how is the output used?
143* Stages:: Stages in writing and running Bison grammars.
144* Grammar Layout:: Overall structure of a Bison grammar file.
145
fa7e68c3
PE
146Writing @acronym{GLR} Parsers
147
32c29292
JD
148* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
149* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
150* GLR Semantic Actions:: Deferred semantic actions have special concerns.
151* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
fa7e68c3 152
bfa74976
RS
153Examples
154
155* RPN Calc:: Reverse polish notation calculator;
156 a first example with no operator precedence.
157* Infix Calc:: Infix (algebraic) notation calculator.
158 Operator precedence is introduced.
159* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 160* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
93dd49ab
PE
161* Multi-function Calc:: Calculator with memory and trig functions.
162 It uses multiple data-types for semantic values.
bfa74976
RS
163* Exercises:: Ideas for improving the multi-function calculator.
164
165Reverse Polish Notation Calculator
166
75f5aaea 167* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
bfa74976
RS
168* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
169* Lexer: Rpcalc Lexer. The lexical analyzer.
170* Main: Rpcalc Main. The controlling function.
171* Error: Rpcalc Error. The error reporting function.
172* Gen: Rpcalc Gen. Running Bison on the grammar file.
173* Comp: Rpcalc Compile. Run the C compiler on the output code.
174
175Grammar Rules for @code{rpcalc}
176
13863333
AD
177* Rpcalc Input::
178* Rpcalc Line::
179* Rpcalc Expr::
bfa74976 180
342b8b6e
AD
181Location Tracking Calculator: @code{ltcalc}
182
183* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
184* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
185* Lexer: Ltcalc Lexer. The lexical analyzer.
186
bfa74976
RS
187Multi-Function Calculator: @code{mfcalc}
188
189* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
190* Rules: Mfcalc Rules. Grammar rules for the calculator.
191* Symtab: Mfcalc Symtab. Symbol table management subroutines.
192
193Bison Grammar Files
194
195* Grammar Outline:: Overall layout of the grammar file.
196* Symbols:: Terminal and nonterminal symbols.
197* Rules:: How to write grammar rules.
198* Recursion:: Writing recursive rules.
199* Semantics:: Semantic values and actions.
93dd49ab 200* Locations:: Locations and actions.
bfa74976
RS
201* Declarations:: All kinds of Bison declarations are described here.
202* Multiple Parsers:: Putting more than one Bison parser in one program.
203
204Outline of a Bison Grammar
205
93dd49ab 206* Prologue:: Syntax and usage of the prologue.
bfa74976
RS
207* Bison Declarations:: Syntax and usage of the Bison declarations section.
208* Grammar Rules:: Syntax and usage of the grammar rules section.
93dd49ab 209* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
210
211Defining Language Semantics
212
213* Value Type:: Specifying one data type for all semantic values.
214* Multiple Types:: Specifying several alternative data types.
215* Actions:: An action is the semantic definition of a grammar rule.
216* Action Types:: Specifying data types for actions to operate on.
217* Mid-Rule Actions:: Most actions go at the end of a rule.
218 This says when, why and how to use the exceptional
219 action in the middle of a rule.
220
93dd49ab
PE
221Tracking Locations
222
223* Location Type:: Specifying a data type for locations.
224* Actions and Locations:: Using locations in actions.
225* Location Default Action:: Defining a general way to compute locations.
226
bfa74976
RS
227Bison Declarations
228
b50d2359 229* Require Decl:: Requiring a Bison version.
bfa74976
RS
230* Token Decl:: Declaring terminal symbols.
231* Precedence Decl:: Declaring terminals with precedence and associativity.
232* Union Decl:: Declaring the set of all semantic value types.
233* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 234* Initial Action Decl:: Code run before parsing starts.
72f889cc 235* Destructor Decl:: Declaring how symbols are freed.
d6328241 236* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
237* Start Decl:: Specifying the start symbol.
238* Pure Decl:: Requesting a reentrant parser.
239* Decl Summary:: Table of all Bison declarations.
240
241Parser C-Language Interface
242
243* Parser Function:: How to call @code{yyparse} and what it returns.
13863333 244* Lexical:: You must supply a function @code{yylex}
bfa74976
RS
245 which reads tokens.
246* Error Reporting:: You must supply a function @code{yyerror}.
247* Action Features:: Special features for use in actions.
f7ab6a50
PE
248* Internationalization:: How to let the parser speak in the user's
249 native language.
bfa74976
RS
250
251The Lexical Analyzer Function @code{yylex}
252
253* Calling Convention:: How @code{yyparse} calls @code{yylex}.
254* Token Values:: How @code{yylex} must return the semantic value
255 of the token it has read.
95923bd6 256* Token Locations:: How @code{yylex} must return the text location
bfa74976 257 (line number, etc.) of the token, if the
93dd49ab 258 actions want that.
bfa74976
RS
259* Pure Calling:: How the calling convention differs
260 in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
261
13863333 262The Bison Parser Algorithm
bfa74976
RS
263
264* Look-Ahead:: Parser looks one token ahead when deciding what to do.
265* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
266* Precedence:: Operator precedence works by resolving conflicts.
267* Contextual Precedence:: When an operator's precedence depends on context.
268* Parser States:: The parser is a finite-state-machine with stack.
269* Reduce/Reduce:: When two rules are applicable in the same situation.
270* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 271* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 272* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
273
274Operator Precedence
275
276* Why Precedence:: An example showing why precedence is needed.
277* Using Precedence:: How to specify precedence in Bison grammars.
278* Precedence Examples:: How these features are used in the previous example.
279* How Precedence:: How they work.
280
281Handling Context Dependencies
282
283* Semantic Tokens:: Token parsing can depend on the semantic context.
284* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
285* Tie-in Recovery:: Lexical tie-ins have implications for how
286 error recovery rules must be written.
287
93dd49ab 288Debugging Your Parser
ec3bc396
AD
289
290* Understanding:: Understanding the structure of your parser.
291* Tracing:: Tracing the execution of your parser.
292
bfa74976
RS
293Invoking Bison
294
13863333 295* Bison Options:: All the options described in detail,
c827f760 296 in alphabetical order by short options.
bfa74976 297* Option Cross Key:: Alphabetical list of long options.
93dd49ab 298* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
f2b5126e 299
12545799
AD
300C++ Language Interface
301
302* C++ Parsers:: The interface to generate C++ parser classes
303* A Complete C++ Example:: Demonstrating their use
304
305C++ Parsers
306
307* C++ Bison Interface:: Asking for C++ parser generation
308* C++ Semantic Values:: %union vs. C++
309* C++ Location Values:: The position and location classes
310* C++ Parser Interface:: Instantiating and running the parser
311* C++ Scanner Interface:: Exchanges between yylex and parse
312
313A Complete C++ Example
314
315* Calc++ --- C++ Calculator:: The specifications
316* Calc++ Parsing Driver:: An active parsing context
317* Calc++ Parser:: A parser class
318* Calc++ Scanner:: A pure C++ Flex scanner
319* Calc++ Top Level:: Conducting the band
320
d1a1114f
AD
321Frequently Asked Questions
322
1a059451 323* Memory Exhausted:: Breaking the Stack Limits
e64fec0a 324* How Can I Reset the Parser:: @code{yyparse} Keeps some State
fef4cb51 325* Strings are Destroyed:: @code{yylval} Loses Track of Strings
2fa09258 326* Implementing Gotos/Loops:: Control Flow in the Calculator
d1a1114f 327
f2b5126e
PB
328Copying This Manual
329
330* GNU Free Documentation License:: License for copying this manual.
331
342b8b6e 332@end detailmenu
bfa74976
RS
333@end menu
334
342b8b6e 335@node Introduction
bfa74976
RS
336@unnumbered Introduction
337@cindex introduction
338
339@dfn{Bison} is a general-purpose parser generator that converts a
c827f760 340grammar description for an @acronym{LALR}(1) context-free grammar into a C
bfa74976
RS
341program to parse that grammar. Once you are proficient with Bison,
342you may use it to develop a wide range of language parsers, from those
343used in simple desk calculators to complex programming languages.
344
345Bison is upward compatible with Yacc: all properly-written Yacc grammars
346ought to work with Bison with no change. Anyone familiar with Yacc
347should be able to use Bison with little trouble. You need to be fluent in
348C programming in order to use Bison or to understand this manual.
349
350We begin with tutorial chapters that explain the basic concepts of using
351Bison and show three explained examples, each building on the last. If you
352don't know Bison or Yacc, start by reading these chapters. Reference
353chapters follow which describe specific aspects of Bison in detail.
354
931c7513
RS
355Bison was written primarily by Robert Corbett; Richard Stallman made it
356Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
14ded682 357multi-character string literals and other features.
931c7513 358
df1af54c 359This edition corresponds to version @value{VERSION} of Bison.
bfa74976 360
342b8b6e 361@node Conditions
bfa74976
RS
362@unnumbered Conditions for Using Bison
363
a31239f1 364As of Bison version 1.24, we have changed the distribution terms for
262aa8dd 365@code{yyparse} to permit using Bison's output in nonfree programs when
c827f760 366Bison is generating C code for @acronym{LALR}(1) parsers. Formerly, these
262aa8dd 367parsers could be used only in programs that were free software.
a31239f1 368
c827f760
PE
369The other @acronym{GNU} programming tools, such as the @acronym{GNU} C
370compiler, have never
9ecbd125 371had such a requirement. They could always be used for nonfree
a31239f1
RS
372software. The reason Bison was different was not due to a special
373policy decision; it resulted from applying the usual General Public
374License to all of the Bison source code.
375
376The output of the Bison utility---the Bison parser file---contains a
377verbatim copy of a sizable piece of Bison, which is the code for the
378@code{yyparse} function. (The actions from your grammar are inserted
379into this function at one point, but the rest of the function is not
c827f760
PE
380changed.) When we applied the @acronym{GPL} terms to the code for
381@code{yyparse},
a31239f1
RS
382the effect was to restrict the use of Bison output to free software.
383
384We didn't change the terms because of sympathy for people who want to
385make software proprietary. @strong{Software should be free.} But we
386concluded that limiting Bison's use to free software was doing little to
387encourage people to make other software free. So we decided to make the
388practical conditions for using Bison match the practical conditions for
c827f760 389using the other @acronym{GNU} tools.
bfa74976 390
eda42934 391This exception applies only when Bison is generating C code for an
c827f760
PE
392@acronym{LALR}(1) parser; otherwise, the @acronym{GPL} terms operate
393as usual. You can
262aa8dd
PE
394tell whether the exception applies to your @samp{.c} output file by
395inspecting it to see whether it says ``As a special exception, when
396this file is copied by Bison into a Bison output file, you may use
397that output file without restriction.''
398
c67a198d 399@include gpl.texi
bfa74976 400
342b8b6e 401@node Concepts
bfa74976
RS
402@chapter The Concepts of Bison
403
404This chapter introduces many of the basic concepts without which the
405details of Bison will not make sense. If you do not already know how to
406use Bison or Yacc, we suggest you start by reading this chapter carefully.
407
408@menu
409* Language and Grammar:: Languages and context-free grammars,
410 as mathematical ideas.
411* Grammar in Bison:: How we represent grammars for Bison's sake.
412* Semantic Values:: Each token or syntactic grouping can have
413 a semantic value (the value of an integer,
414 the name of an identifier, etc.).
415* Semantic Actions:: Each rule can have an action containing C code.
99a9344e 416* GLR Parsers:: Writing parsers for general context-free languages.
847bf1f5 417* Locations Overview:: Tracking Locations.
bfa74976
RS
418* Bison Parser:: What are Bison's input and output,
419 how is the output used?
420* Stages:: Stages in writing and running Bison grammars.
421* Grammar Layout:: Overall structure of a Bison grammar file.
422@end menu
423
342b8b6e 424@node Language and Grammar
bfa74976
RS
425@section Languages and Context-Free Grammars
426
bfa74976
RS
427@cindex context-free grammar
428@cindex grammar, context-free
429In order for Bison to parse a language, it must be described by a
430@dfn{context-free grammar}. This means that you specify one or more
431@dfn{syntactic groupings} and give rules for constructing them from their
432parts. For example, in the C language, one kind of grouping is called an
433`expression'. One rule for making an expression might be, ``An expression
434can be made of a minus sign and another expression''. Another would be,
435``An expression can be an integer''. As you can see, rules are often
436recursive, but there must be at least one rule which leads out of the
437recursion.
438
c827f760 439@cindex @acronym{BNF}
bfa74976
RS
440@cindex Backus-Naur form
441The most common formal system for presenting such rules for humans to read
c827f760
PE
442is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in
443order to specify the language Algol 60. Any grammar expressed in
444@acronym{BNF} is a context-free grammar. The input to Bison is
445essentially machine-readable @acronym{BNF}.
bfa74976 446
c827f760
PE
447@cindex @acronym{LALR}(1) grammars
448@cindex @acronym{LR}(1) grammars
676385e2
PH
449There are various important subclasses of context-free grammar. Although it
450can handle almost all context-free grammars, Bison is optimized for what
c827f760 451are called @acronym{LALR}(1) grammars.
676385e2 452In brief, in these grammars, it must be possible to
bfa74976
RS
453tell how to parse any portion of an input string with just a single
454token of look-ahead. Strictly speaking, that is a description of an
c827f760
PE
455@acronym{LR}(1) grammar, and @acronym{LALR}(1) involves additional
456restrictions that are
bfa74976 457hard to explain simply; but it is rare in actual practice to find an
c827f760
PE
458@acronym{LR}(1) grammar that fails to be @acronym{LALR}(1).
459@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
460more information on this.
bfa74976 461
c827f760
PE
462@cindex @acronym{GLR} parsing
463@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2 464@cindex ambiguous grammars
9d9b8b70 465@cindex nondeterministic parsing
9501dc6e
AD
466
467Parsers for @acronym{LALR}(1) grammars are @dfn{deterministic}, meaning
468roughly that the next grammar rule to apply at any point in the input is
469uniquely determined by the preceding input and a fixed, finite portion
470(called a @dfn{look-ahead}) of the remaining input. A context-free
471grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
e4f85c39 472apply the grammar rules to get the same inputs. Even unambiguous
9d9b8b70 473grammars can be @dfn{nondeterministic}, meaning that no fixed
9501dc6e
AD
474look-ahead always suffices to determine the next grammar rule to apply.
475With the proper declarations, Bison is also able to parse these more
476general context-free grammars, using a technique known as @acronym{GLR}
477parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers
478are able to handle any context-free grammar for which the number of
479possible parses of any given string is finite.
676385e2 480
bfa74976
RS
481@cindex symbols (abstract)
482@cindex token
483@cindex syntactic grouping
484@cindex grouping, syntactic
9501dc6e
AD
485In the formal grammatical rules for a language, each kind of syntactic
486unit or grouping is named by a @dfn{symbol}. Those which are built by
487grouping smaller constructs according to grammatical rules are called
bfa74976
RS
488@dfn{nonterminal symbols}; those which can't be subdivided are called
489@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
490corresponding to a single terminal symbol a @dfn{token}, and a piece
e0c471a9 491corresponding to a single nonterminal symbol a @dfn{grouping}.
bfa74976
RS
492
493We can use the C language as an example of what symbols, terminal and
9501dc6e
AD
494nonterminal, mean. The tokens of C are identifiers, constants (numeric
495and string), and the various keywords, arithmetic operators and
496punctuation marks. So the terminal symbols of a grammar for C include
497`identifier', `number', `string', plus one symbol for each keyword,
498operator or punctuation mark: `if', `return', `const', `static', `int',
499`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
500(These tokens can be subdivided into characters, but that is a matter of
bfa74976
RS
501lexicography, not grammar.)
502
503Here is a simple C function subdivided into tokens:
504
9edcd895
AD
505@ifinfo
506@example
507int /* @r{keyword `int'} */
14d4662b 508square (int x) /* @r{identifier, open-paren, keyword `int',}
9edcd895
AD
509 @r{identifier, close-paren} */
510@{ /* @r{open-brace} */
511 return x * x; /* @r{keyword `return', identifier, asterisk,
512 identifier, semicolon} */
513@} /* @r{close-brace} */
514@end example
515@end ifinfo
516@ifnotinfo
bfa74976
RS
517@example
518int /* @r{keyword `int'} */
14d4662b 519square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
bfa74976 520@{ /* @r{open-brace} */
9edcd895 521 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
bfa74976
RS
522@} /* @r{close-brace} */
523@end example
9edcd895 524@end ifnotinfo
bfa74976
RS
525
526The syntactic groupings of C include the expression, the statement, the
527declaration, and the function definition. These are represented in the
528grammar of C by nonterminal symbols `expression', `statement',
529`declaration' and `function definition'. The full grammar uses dozens of
530additional language constructs, each with its own nonterminal symbol, in
531order to express the meanings of these four. The example above is a
532function definition; it contains one declaration, and one statement. In
533the statement, each @samp{x} is an expression and so is @samp{x * x}.
534
535Each nonterminal symbol must have grammatical rules showing how it is made
536out of simpler constructs. For example, one kind of C statement is the
537@code{return} statement; this would be described with a grammar rule which
538reads informally as follows:
539
540@quotation
541A `statement' can be made of a `return' keyword, an `expression' and a
542`semicolon'.
543@end quotation
544
545@noindent
546There would be many other rules for `statement', one for each kind of
547statement in C.
548
549@cindex start symbol
550One nonterminal symbol must be distinguished as the special one which
551defines a complete utterance in the language. It is called the @dfn{start
552symbol}. In a compiler, this means a complete input program. In the C
553language, the nonterminal symbol `sequence of definitions and declarations'
554plays this role.
555
556For example, @samp{1 + 2} is a valid C expression---a valid part of a C
557program---but it is not valid as an @emph{entire} C program. In the
558context-free grammar of C, this follows from the fact that `expression' is
559not the start symbol.
560
561The Bison parser reads a sequence of tokens as its input, and groups the
562tokens using the grammar rules. If the input is valid, the end result is
563that the entire token sequence reduces to a single grouping whose symbol is
564the grammar's start symbol. If we use a grammar for C, the entire input
565must be a `sequence of definitions and declarations'. If not, the parser
566reports a syntax error.
567
342b8b6e 568@node Grammar in Bison
bfa74976
RS
569@section From Formal Rules to Bison Input
570@cindex Bison grammar
571@cindex grammar, Bison
572@cindex formal grammar
573
574A formal grammar is a mathematical construct. To define the language
575for Bison, you must write a file expressing the grammar in Bison syntax:
576a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
577
578A nonterminal symbol in the formal grammar is represented in Bison input
c827f760 579as an identifier, like an identifier in C@. By convention, it should be
bfa74976
RS
580in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
581
582The Bison representation for a terminal symbol is also called a @dfn{token
583type}. Token types as well can be represented as C-like identifiers. By
584convention, these identifiers should be upper case to distinguish them from
585nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
586@code{RETURN}. A terminal symbol that stands for a particular keyword in
587the language should be named after that keyword converted to upper case.
588The terminal symbol @code{error} is reserved for error recovery.
931c7513 589@xref{Symbols}.
bfa74976
RS
590
591A terminal symbol can also be represented as a character literal, just like
592a C character constant. You should do this whenever a token is just a
593single character (parenthesis, plus-sign, etc.): use that same character in
594a literal as the terminal symbol for that token.
595
931c7513
RS
596A third way to represent a terminal symbol is with a C string constant
597containing several characters. @xref{Symbols}, for more information.
598
bfa74976
RS
599The grammar rules also have an expression in Bison syntax. For example,
600here is the Bison rule for a C @code{return} statement. The semicolon in
601quotes is a literal character token, representing part of the C syntax for
602the statement; the naked semicolon, and the colon, are Bison punctuation
603used in every rule.
604
605@example
606stmt: RETURN expr ';'
607 ;
608@end example
609
610@noindent
611@xref{Rules, ,Syntax of Grammar Rules}.
612
342b8b6e 613@node Semantic Values
bfa74976
RS
614@section Semantic Values
615@cindex semantic value
616@cindex value, semantic
617
618A formal grammar selects tokens only by their classifications: for example,
619if a rule mentions the terminal symbol `integer constant', it means that
620@emph{any} integer constant is grammatically valid in that position. The
621precise value of the constant is irrelevant to how to parse the input: if
622@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
e0c471a9 623grammatical.
bfa74976
RS
624
625But the precise value is very important for what the input means once it is
626parsed. A compiler is useless if it fails to distinguish between 4, 1 and
6273989 as constants in the program! Therefore, each token in a Bison grammar
c827f760
PE
628has both a token type and a @dfn{semantic value}. @xref{Semantics,
629,Defining Language Semantics},
bfa74976
RS
630for details.
631
632The token type is a terminal symbol defined in the grammar, such as
633@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
634you need to know to decide where the token may validly appear and how to
635group it with other tokens. The grammar rules know nothing about tokens
e0c471a9 636except their types.
bfa74976
RS
637
638The semantic value has all the rest of the information about the
639meaning of the token, such as the value of an integer, or the name of an
640identifier. (A token such as @code{','} which is just punctuation doesn't
641need to have any semantic value.)
642
643For example, an input token might be classified as token type
644@code{INTEGER} and have the semantic value 4. Another input token might
645have the same token type @code{INTEGER} but value 3989. When a grammar
646rule says that @code{INTEGER} is allowed, either of these tokens is
647acceptable because each is an @code{INTEGER}. When the parser accepts the
648token, it keeps track of the token's semantic value.
649
650Each grouping can also have a semantic value as well as its nonterminal
651symbol. For example, in a calculator, an expression typically has a
652semantic value that is a number. In a compiler for a programming
653language, an expression typically has a semantic value that is a tree
654structure describing the meaning of the expression.
655
342b8b6e 656@node Semantic Actions
bfa74976
RS
657@section Semantic Actions
658@cindex semantic actions
659@cindex actions, semantic
660
661In order to be useful, a program must do more than parse input; it must
662also produce some output based on the input. In a Bison grammar, a grammar
663rule can have an @dfn{action} made up of C statements. Each time the
664parser recognizes a match for that rule, the action is executed.
665@xref{Actions}.
13863333 666
bfa74976
RS
667Most of the time, the purpose of an action is to compute the semantic value
668of the whole construct from the semantic values of its parts. For example,
669suppose we have a rule which says an expression can be the sum of two
670expressions. When the parser recognizes such a sum, each of the
671subexpressions has a semantic value which describes how it was built up.
672The action for this rule should create a similar sort of value for the
673newly recognized larger expression.
674
675For example, here is a rule that says an expression can be the sum of
676two subexpressions:
677
678@example
679expr: expr '+' expr @{ $$ = $1 + $3; @}
680 ;
681@end example
682
683@noindent
684The action says how to produce the semantic value of the sum expression
685from the values of the two subexpressions.
686
676385e2 687@node GLR Parsers
c827f760
PE
688@section Writing @acronym{GLR} Parsers
689@cindex @acronym{GLR} parsing
690@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2
PH
691@findex %glr-parser
692@cindex conflicts
693@cindex shift/reduce conflicts
fa7e68c3 694@cindex reduce/reduce conflicts
676385e2 695
fa7e68c3 696In some grammars, Bison's standard
9501dc6e
AD
697@acronym{LALR}(1) parsing algorithm cannot decide whether to apply a
698certain grammar rule at a given point. That is, it may not be able to
699decide (on the basis of the input read so far) which of two possible
700reductions (applications of a grammar rule) applies, or whether to apply
701a reduction or read more of the input and apply a reduction later in the
702input. These are known respectively as @dfn{reduce/reduce} conflicts
703(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
704(@pxref{Shift/Reduce}).
705
706To use a grammar that is not easily modified to be @acronym{LALR}(1), a
707more general parsing algorithm is sometimes necessary. If you include
676385e2 708@code{%glr-parser} among the Bison declarations in your file
fa7e68c3 709(@pxref{Grammar Outline}), the result is a Generalized @acronym{LR}
9501dc6e
AD
710(@acronym{GLR}) parser. These parsers handle Bison grammars that
711contain no unresolved conflicts (i.e., after applying precedence
712declarations) identically to @acronym{LALR}(1) parsers. However, when
713faced with unresolved shift/reduce and reduce/reduce conflicts,
714@acronym{GLR} parsers use the simple expedient of doing both,
715effectively cloning the parser to follow both possibilities. Each of
716the resulting parsers can again split, so that at any given time, there
717can be any number of possible parses being explored. The parsers
676385e2
PH
718proceed in lockstep; that is, all of them consume (shift) a given input
719symbol before any of them proceed to the next. Each of the cloned
720parsers eventually meets one of two possible fates: either it runs into
721a parsing error, in which case it simply vanishes, or it merges with
722another parser, because the two of them have reduced the input to an
723identical set of symbols.
724
725During the time that there are multiple parsers, semantic actions are
726recorded, but not performed. When a parser disappears, its recorded
727semantic actions disappear as well, and are never performed. When a
728reduction makes two parsers identical, causing them to merge, Bison
729records both sets of semantic actions. Whenever the last two parsers
730merge, reverting to the single-parser case, Bison resolves all the
731outstanding actions either by precedences given to the grammar rules
732involved, or by performing both actions, and then calling a designated
733user-defined function on the resulting values to produce an arbitrary
734merged result.
735
fa7e68c3 736@menu
32c29292
JD
737* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
738* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
739* GLR Semantic Actions:: Deferred semantic actions have special concerns.
740* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
fa7e68c3
PE
741@end menu
742
743@node Simple GLR Parsers
744@subsection Using @acronym{GLR} on Unambiguous Grammars
745@cindex @acronym{GLR} parsing, unambiguous grammars
746@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars
747@findex %glr-parser
748@findex %expect-rr
749@cindex conflicts
750@cindex reduce/reduce conflicts
751@cindex shift/reduce conflicts
752
753In the simplest cases, you can use the @acronym{GLR} algorithm
754to parse grammars that are unambiguous, but fail to be @acronym{LALR}(1).
755Such grammars typically require more than one symbol of look-ahead,
756or (in rare cases) fall into the category of grammars in which the
757@acronym{LALR}(1) algorithm throws away too much information (they are in
758@acronym{LR}(1), but not @acronym{LALR}(1), @ref{Mystery Conflicts}).
759
760Consider a problem that
761arises in the declaration of enumerated and subrange types in the
762programming language Pascal. Here are some examples:
763
764@example
765type subrange = lo .. hi;
766type enum = (a, b, c);
767@end example
768
769@noindent
770The original language standard allows only numeric
771literals and constant identifiers for the subrange bounds (@samp{lo}
772and @samp{hi}), but Extended Pascal (@acronym{ISO}/@acronym{IEC}
77310206) and many other
774Pascal implementations allow arbitrary expressions there. This gives
775rise to the following situation, containing a superfluous pair of
776parentheses:
777
778@example
779type subrange = (a) .. b;
780@end example
781
782@noindent
783Compare this to the following declaration of an enumerated
784type with only one value:
785
786@example
787type enum = (a);
788@end example
789
790@noindent
791(These declarations are contrived, but they are syntactically
792valid, and more-complicated cases can come up in practical programs.)
793
794These two declarations look identical until the @samp{..} token.
795With normal @acronym{LALR}(1) one-token look-ahead it is not
796possible to decide between the two forms when the identifier
797@samp{a} is parsed. It is, however, desirable
798for a parser to decide this, since in the latter case
799@samp{a} must become a new identifier to represent the enumeration
800value, while in the former case @samp{a} must be evaluated with its
801current meaning, which may be a constant or even a function call.
802
803You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
804to be resolved later, but this typically requires substantial
805contortions in both semantic actions and large parts of the
806grammar, where the parentheses are nested in the recursive rules for
807expressions.
808
809You might think of using the lexer to distinguish between the two
810forms by returning different tokens for currently defined and
811undefined identifiers. But if these declarations occur in a local
812scope, and @samp{a} is defined in an outer scope, then both forms
813are possible---either locally redefining @samp{a}, or using the
814value of @samp{a} from the outer scope. So this approach cannot
815work.
816
e757bb10 817A simple solution to this problem is to declare the parser to
fa7e68c3
PE
818use the @acronym{GLR} algorithm.
819When the @acronym{GLR} parser reaches the critical state, it
820merely splits into two branches and pursues both syntax rules
821simultaneously. Sooner or later, one of them runs into a parsing
822error. If there is a @samp{..} token before the next
823@samp{;}, the rule for enumerated types fails since it cannot
824accept @samp{..} anywhere; otherwise, the subrange type rule
825fails since it requires a @samp{..} token. So one of the branches
826fails silently, and the other one continues normally, performing
827all the intermediate actions that were postponed during the split.
828
829If the input is syntactically incorrect, both branches fail and the parser
830reports a syntax error as usual.
831
832The effect of all this is that the parser seems to ``guess'' the
833correct branch to take, or in other words, it seems to use more
834look-ahead than the underlying @acronym{LALR}(1) algorithm actually allows
835for. In this example, @acronym{LALR}(2) would suffice, but also some cases
836that are not @acronym{LALR}(@math{k}) for any @math{k} can be handled this way.
837
838In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time,
839and the current Bison parser even takes exponential time and space
840for some grammars. In practice, this rarely happens, and for many
841grammars it is possible to prove that it cannot happen.
842The present example contains only one conflict between two
843rules, and the type-declaration context containing the conflict
844cannot be nested. So the number of
845branches that can exist at any time is limited by the constant 2,
846and the parsing time is still linear.
847
848Here is a Bison grammar corresponding to the example above. It
849parses a vastly simplified form of Pascal type declarations.
850
851@example
852%token TYPE DOTDOT ID
853
854@group
855%left '+' '-'
856%left '*' '/'
857@end group
858
859%%
860
861@group
862type_decl : TYPE ID '=' type ';'
863 ;
864@end group
865
866@group
867type : '(' id_list ')'
868 | expr DOTDOT expr
869 ;
870@end group
871
872@group
873id_list : ID
874 | id_list ',' ID
875 ;
876@end group
877
878@group
879expr : '(' expr ')'
880 | expr '+' expr
881 | expr '-' expr
882 | expr '*' expr
883 | expr '/' expr
884 | ID
885 ;
886@end group
887@end example
888
889When used as a normal @acronym{LALR}(1) grammar, Bison correctly complains
890about one reduce/reduce conflict. In the conflicting situation the
891parser chooses one of the alternatives, arbitrarily the one
892declared first. Therefore the following correct input is not
893recognized:
894
895@example
896type t = (a) .. b;
897@end example
898
899The parser can be turned into a @acronym{GLR} parser, while also telling Bison
900to be silent about the one known reduce/reduce conflict, by
e757bb10 901adding these two declarations to the Bison input file (before the first
fa7e68c3
PE
902@samp{%%}):
903
904@example
905%glr-parser
906%expect-rr 1
907@end example
908
909@noindent
910No change in the grammar itself is required. Now the
911parser recognizes all valid declarations, according to the
912limited syntax above, transparently. In fact, the user does not even
913notice when the parser splits.
914
f8e1c9e5
AD
915So here we have a case where we can use the benefits of @acronym{GLR},
916almost without disadvantages. Even in simple cases like this, however,
917there are at least two potential problems to beware. First, always
918analyze the conflicts reported by Bison to make sure that @acronym{GLR}
919splitting is only done where it is intended. A @acronym{GLR} parser
920splitting inadvertently may cause problems less obvious than an
921@acronym{LALR} parser statically choosing the wrong alternative in a
922conflict. Second, consider interactions with the lexer (@pxref{Semantic
923Tokens}) with great care. Since a split parser consumes tokens without
924performing any actions during the split, the lexer cannot obtain
925information via parser actions. Some cases of lexer interactions can be
926eliminated by using @acronym{GLR} to shift the complications from the
927lexer to the parser. You must check the remaining cases for
928correctness.
929
930In our example, it would be safe for the lexer to return tokens based on
931their current meanings in some symbol table, because no new symbols are
932defined in the middle of a type declaration. Though it is possible for
933a parser to define the enumeration constants as they are parsed, before
934the type declaration is completed, it actually makes no difference since
935they cannot be used within the same enumerated type declaration.
fa7e68c3
PE
936
937@node Merging GLR Parses
938@subsection Using @acronym{GLR} to Resolve Ambiguities
939@cindex @acronym{GLR} parsing, ambiguous grammars
940@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, ambiguous grammars
941@findex %dprec
942@findex %merge
943@cindex conflicts
944@cindex reduce/reduce conflicts
945
2a8d363a 946Let's consider an example, vastly simplified from a C++ grammar.
676385e2
PH
947
948@example
949%@{
38a92d50
PE
950 #include <stdio.h>
951 #define YYSTYPE char const *
952 int yylex (void);
953 void yyerror (char const *);
676385e2
PH
954%@}
955
956%token TYPENAME ID
957
958%right '='
959%left '+'
960
961%glr-parser
962
963%%
964
fae437e8 965prog :
676385e2
PH
966 | prog stmt @{ printf ("\n"); @}
967 ;
968
969stmt : expr ';' %dprec 1
970 | decl %dprec 2
971 ;
972
2a8d363a 973expr : ID @{ printf ("%s ", $$); @}
fae437e8 974 | TYPENAME '(' expr ')'
2a8d363a
AD
975 @{ printf ("%s <cast> ", $1); @}
976 | expr '+' expr @{ printf ("+ "); @}
977 | expr '=' expr @{ printf ("= "); @}
676385e2
PH
978 ;
979
fae437e8 980decl : TYPENAME declarator ';'
2a8d363a 981 @{ printf ("%s <declare> ", $1); @}
676385e2 982 | TYPENAME declarator '=' expr ';'
2a8d363a 983 @{ printf ("%s <init-declare> ", $1); @}
676385e2
PH
984 ;
985
2a8d363a 986declarator : ID @{ printf ("\"%s\" ", $1); @}
676385e2
PH
987 | '(' declarator ')'
988 ;
989@end example
990
991@noindent
992This models a problematic part of the C++ grammar---the ambiguity between
993certain declarations and statements. For example,
994
995@example
996T (x) = y+z;
997@end example
998
999@noindent
1000parses as either an @code{expr} or a @code{stmt}
c827f760
PE
1001(assuming that @samp{T} is recognized as a @code{TYPENAME} and
1002@samp{x} as an @code{ID}).
676385e2 1003Bison detects this as a reduce/reduce conflict between the rules
fae437e8 1004@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
e757bb10
AD
1005time it encounters @code{x} in the example above. Since this is a
1006@acronym{GLR} parser, it therefore splits the problem into two parses, one for
fa7e68c3
PE
1007each choice of resolving the reduce/reduce conflict.
1008Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1009however, neither of these parses ``dies,'' because the grammar as it stands is
e757bb10
AD
1010ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1011the other reduces @code{stmt : decl}, after which both parsers are in an
1012identical state: they've seen @samp{prog stmt} and have the same unprocessed
1013input remaining. We say that these parses have @dfn{merged.}
fa7e68c3
PE
1014
1015At this point, the @acronym{GLR} parser requires a specification in the
1016grammar of how to choose between the competing parses.
1017In the example above, the two @code{%dprec}
e757bb10 1018declarations specify that Bison is to give precedence
fa7e68c3 1019to the parse that interprets the example as a
676385e2
PH
1020@code{decl}, which implies that @code{x} is a declarator.
1021The parser therefore prints
1022
1023@example
fae437e8 1024"x" y z + T <init-declare>
676385e2
PH
1025@end example
1026
fa7e68c3
PE
1027The @code{%dprec} declarations only come into play when more than one
1028parse survives. Consider a different input string for this parser:
676385e2
PH
1029
1030@example
1031T (x) + y;
1032@end example
1033
1034@noindent
e757bb10 1035This is another example of using @acronym{GLR} to parse an unambiguous
fa7e68c3 1036construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
676385e2
PH
1037Here, there is no ambiguity (this cannot be parsed as a declaration).
1038However, at the time the Bison parser encounters @code{x}, it does not
1039have enough information to resolve the reduce/reduce conflict (again,
1040between @code{x} as an @code{expr} or a @code{declarator}). In this
fa7e68c3 1041case, no precedence declaration is used. Again, the parser splits
676385e2
PH
1042into two, one assuming that @code{x} is an @code{expr}, and the other
1043assuming @code{x} is a @code{declarator}. The second of these parsers
1044then vanishes when it sees @code{+}, and the parser prints
1045
1046@example
fae437e8 1047x T <cast> y +
676385e2
PH
1048@end example
1049
1050Suppose that instead of resolving the ambiguity, you wanted to see all
fa7e68c3 1051the possibilities. For this purpose, you must merge the semantic
676385e2
PH
1052actions of the two possible parsers, rather than choosing one over the
1053other. To do so, you could change the declaration of @code{stmt} as
1054follows:
1055
1056@example
1057stmt : expr ';' %merge <stmtMerge>
1058 | decl %merge <stmtMerge>
1059 ;
1060@end example
1061
1062@noindent
676385e2
PH
1063and define the @code{stmtMerge} function as:
1064
1065@example
38a92d50
PE
1066static YYSTYPE
1067stmtMerge (YYSTYPE x0, YYSTYPE x1)
676385e2
PH
1068@{
1069 printf ("<OR> ");
1070 return "";
1071@}
1072@end example
1073
1074@noindent
1075with an accompanying forward declaration
1076in the C declarations at the beginning of the file:
1077
1078@example
1079%@{
38a92d50 1080 #define YYSTYPE char const *
676385e2
PH
1081 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1082%@}
1083@end example
1084
1085@noindent
fa7e68c3
PE
1086With these declarations, the resulting parser parses the first example
1087as both an @code{expr} and a @code{decl}, and prints
676385e2
PH
1088
1089@example
fae437e8 1090"x" y z + T <init-declare> x T <cast> y z + = <OR>
676385e2
PH
1091@end example
1092
fa7e68c3 1093Bison requires that all of the
e757bb10 1094productions that participate in any particular merge have identical
fa7e68c3
PE
1095@samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1096and the parser will report an error during any parse that results in
1097the offending merge.
9501dc6e 1098
32c29292
JD
1099@node GLR Semantic Actions
1100@subsection GLR Semantic Actions
1101
1102@cindex deferred semantic actions
1103By definition, a deferred semantic action is not performed at the same time as
1104the associated reduction.
1105This raises caveats for several Bison features you might use in a semantic
1106action in a @acronym{GLR} parser.
1107
1108@vindex yychar
1109@cindex @acronym{GLR} parsers and @code{yychar}
1110@vindex yylval
1111@cindex @acronym{GLR} parsers and @code{yylval}
1112@vindex yylloc
1113@cindex @acronym{GLR} parsers and @code{yylloc}
1114In any semantic action, you can examine @code{yychar} to determine the type of
1115the look-ahead token present at the time of the associated reduction.
1116After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1117you can then examine @code{yylval} and @code{yylloc} to determine the
1118look-ahead token's semantic value and location, if any.
1119In a nondeferred semantic action, you can also modify any of these variables to
1120influence syntax analysis.
1121@xref{Look-Ahead, ,Look-Ahead Tokens}.
1122
1123@findex yyclearin
1124@cindex @acronym{GLR} parsers and @code{yyclearin}
1125In a deferred semantic action, it's too late to influence syntax analysis.
1126In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1127shallow copies of the values they had at the time of the associated reduction.
1128For this reason alone, modifying them is dangerous.
1129Moreover, the result of modifying them is undefined and subject to change with
1130future versions of Bison.
1131For example, if a semantic action might be deferred, you should never write it
1132to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1133memory referenced by @code{yylval}.
1134
1135@findex YYERROR
1136@cindex @acronym{GLR} parsers and @code{YYERROR}
1137Another Bison feature requiring special consideration is @code{YYERROR}
1138(@pxref{Action Features}), which you can invoke in any semantic action to
1139initiate error recovery.
1140During deterministic @acronym{GLR} operation, the effect of @code{YYERROR} is
1141the same as its effect in an @acronym{LALR}(1) parser.
1142In a deferred semantic action, its effect is undefined.
1143@c The effect is probably a syntax error at the split point.
1144
fa7e68c3
PE
1145@node Compiler Requirements
1146@subsection Considerations when Compiling @acronym{GLR} Parsers
1147@cindex @code{inline}
9501dc6e 1148@cindex @acronym{GLR} parsers and @code{inline}
fa7e68c3 1149
38a92d50
PE
1150The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or
1151later. In addition, they use the @code{inline} keyword, which is not
1152C89, but is C99 and is a common extension in pre-C99 compilers. It is
1153up to the user of these parsers to handle
9501dc6e
AD
1154portability issues. For instance, if using Autoconf and the Autoconf
1155macro @code{AC_C_INLINE}, a mere
1156
1157@example
1158%@{
38a92d50 1159 #include <config.h>
9501dc6e
AD
1160%@}
1161@end example
1162
1163@noindent
1164will suffice. Otherwise, we suggest
1165
1166@example
1167%@{
38a92d50
PE
1168 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1169 #define inline
1170 #endif
9501dc6e
AD
1171%@}
1172@end example
676385e2 1173
342b8b6e 1174@node Locations Overview
847bf1f5
AD
1175@section Locations
1176@cindex location
95923bd6
AD
1177@cindex textual location
1178@cindex location, textual
847bf1f5
AD
1179
1180Many applications, like interpreters or compilers, have to produce verbose
72d2299c 1181and useful error messages. To achieve this, one must be able to keep track of
95923bd6 1182the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
847bf1f5
AD
1183Bison provides a mechanism for handling these locations.
1184
72d2299c 1185Each token has a semantic value. In a similar fashion, each token has an
847bf1f5 1186associated location, but the type of locations is the same for all tokens and
72d2299c 1187groupings. Moreover, the output parser is equipped with a default data
847bf1f5
AD
1188structure for storing locations (@pxref{Locations}, for more details).
1189
1190Like semantic values, locations can be reached in actions using a dedicated
72d2299c 1191set of constructs. In the example above, the location of the whole grouping
847bf1f5
AD
1192is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1193@code{@@3}.
1194
1195When a rule is matched, a default action is used to compute the semantic value
72d2299c
PE
1196of its left hand side (@pxref{Actions}). In the same way, another default
1197action is used for locations. However, the action for locations is general
847bf1f5 1198enough for most cases, meaning there is usually no need to describe for each
72d2299c 1199rule how @code{@@$} should be formed. When building a new location for a given
847bf1f5
AD
1200grouping, the default behavior of the output parser is to take the beginning
1201of the first symbol, and the end of the last symbol.
1202
342b8b6e 1203@node Bison Parser
bfa74976
RS
1204@section Bison Output: the Parser File
1205@cindex Bison parser
1206@cindex Bison utility
1207@cindex lexical analyzer, purpose
1208@cindex parser
1209
1210When you run Bison, you give it a Bison grammar file as input. The output
1211is a C source file that parses the language described by the grammar.
1212This file is called a @dfn{Bison parser}. Keep in mind that the Bison
1213utility and the Bison parser are two distinct programs: the Bison utility
1214is a program whose output is the Bison parser that becomes part of your
1215program.
1216
1217The job of the Bison parser is to group tokens into groupings according to
1218the grammar rules---for example, to build identifiers and operators into
1219expressions. As it does this, it runs the actions for the grammar rules it
1220uses.
1221
704a47c4
AD
1222The tokens come from a function called the @dfn{lexical analyzer} that
1223you must supply in some fashion (such as by writing it in C). The Bison
1224parser calls the lexical analyzer each time it wants a new token. It
1225doesn't know what is ``inside'' the tokens (though their semantic values
1226may reflect this). Typically the lexical analyzer makes the tokens by
1227parsing characters of text, but Bison does not depend on this.
1228@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
bfa74976
RS
1229
1230The Bison parser file is C code which defines a function named
1231@code{yyparse} which implements that grammar. This function does not make
1232a complete C program: you must supply some additional functions. One is
1233the lexical analyzer. Another is an error-reporting function which the
1234parser calls to report an error. In addition, a complete C program must
1235start with a function called @code{main}; you have to provide this, and
1236arrange for it to call @code{yyparse} or the parser will never run.
1237@xref{Interface, ,Parser C-Language Interface}.
1238
f7ab6a50 1239Aside from the token type names and the symbols in the actions you
7093d0f5 1240write, all symbols defined in the Bison parser file itself
bfa74976
RS
1241begin with @samp{yy} or @samp{YY}. This includes interface functions
1242such as the lexical analyzer function @code{yylex}, the error reporting
1243function @code{yyerror} and the parser function @code{yyparse} itself.
1244This also includes numerous identifiers used for internal purposes.
1245Therefore, you should avoid using C identifiers starting with @samp{yy}
1246or @samp{YY} in the Bison grammar file except for the ones defined in
55289366
PE
1247this manual. Also, you should avoid using the C identifiers
1248@samp{malloc} and @samp{free} for anything other than their usual
1249meanings.
bfa74976 1250
7093d0f5
AD
1251In some cases the Bison parser file includes system headers, and in
1252those cases your code should respect the identifiers reserved by those
55289366 1253headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>}, @code{<malloc.h>},
7093d0f5 1254@code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
30757c8c
PE
1255declare memory allocators and related types. @code{<libintl.h>} is
1256included if message translation is in use
1257(@pxref{Internationalization}). Other system headers may
ec3bc396
AD
1258be included if you define @code{YYDEBUG} to a nonzero value
1259(@pxref{Tracing, ,Tracing Your Parser}).
7093d0f5 1260
342b8b6e 1261@node Stages
bfa74976
RS
1262@section Stages in Using Bison
1263@cindex stages in using Bison
1264@cindex using Bison
1265
1266The actual language-design process using Bison, from grammar specification
1267to a working compiler or interpreter, has these parts:
1268
1269@enumerate
1270@item
1271Formally specify the grammar in a form recognized by Bison
704a47c4
AD
1272(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1273in the language, describe the action that is to be taken when an
1274instance of that rule is recognized. The action is described by a
1275sequence of C statements.
bfa74976
RS
1276
1277@item
704a47c4
AD
1278Write a lexical analyzer to process input and pass tokens to the parser.
1279The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1280Lexical Analyzer Function @code{yylex}}). It could also be produced
1281using Lex, but the use of Lex is not discussed in this manual.
bfa74976
RS
1282
1283@item
1284Write a controlling function that calls the Bison-produced parser.
1285
1286@item
1287Write error-reporting routines.
1288@end enumerate
1289
1290To turn this source code as written into a runnable program, you
1291must follow these steps:
1292
1293@enumerate
1294@item
1295Run Bison on the grammar to produce the parser.
1296
1297@item
1298Compile the code output by Bison, as well as any other source files.
1299
1300@item
1301Link the object files to produce the finished product.
1302@end enumerate
1303
342b8b6e 1304@node Grammar Layout
bfa74976
RS
1305@section The Overall Layout of a Bison Grammar
1306@cindex grammar file
1307@cindex file format
1308@cindex format of grammar file
1309@cindex layout of Bison grammar
1310
1311The input file for the Bison utility is a @dfn{Bison grammar file}. The
1312general form of a Bison grammar file is as follows:
1313
1314@example
1315%@{
08e49d20 1316@var{Prologue}
bfa74976
RS
1317%@}
1318
1319@var{Bison declarations}
1320
1321%%
1322@var{Grammar rules}
1323%%
08e49d20 1324@var{Epilogue}
bfa74976
RS
1325@end example
1326
1327@noindent
1328The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1329in every Bison grammar file to separate the sections.
1330
72d2299c 1331The prologue may define types and variables used in the actions. You can
342b8b6e 1332also use preprocessor commands to define macros used there, and use
bfa74976 1333@code{#include} to include header files that do any of these things.
38a92d50
PE
1334You need to declare the lexical analyzer @code{yylex} and the error
1335printer @code{yyerror} here, along with any other global identifiers
1336used by the actions in the grammar rules.
bfa74976
RS
1337
1338The Bison declarations declare the names of the terminal and nonterminal
1339symbols, and may also describe operator precedence and the data types of
1340semantic values of various symbols.
1341
1342The grammar rules define how to construct each nonterminal symbol from its
1343parts.
1344
38a92d50
PE
1345The epilogue can contain any code you want to use. Often the
1346definitions of functions declared in the prologue go here. In a
1347simple program, all the rest of the program can go here.
bfa74976 1348
342b8b6e 1349@node Examples
bfa74976
RS
1350@chapter Examples
1351@cindex simple examples
1352@cindex examples, simple
1353
1354Now we show and explain three sample programs written using Bison: a
1355reverse polish notation calculator, an algebraic (infix) notation
1356calculator, and a multi-function calculator. All three have been tested
1357under BSD Unix 4.3; each produces a usable, though limited, interactive
1358desk-top calculator.
1359
1360These examples are simple, but Bison grammars for real programming
1361languages are written the same way.
1362@ifinfo
1363You can copy these examples out of the Info file and into a source file
1364to try them.
1365@end ifinfo
1366
1367@menu
1368* RPN Calc:: Reverse polish notation calculator;
1369 a first example with no operator precedence.
1370* Infix Calc:: Infix (algebraic) notation calculator.
1371 Operator precedence is introduced.
1372* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 1373* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
bfa74976
RS
1374* Multi-function Calc:: Calculator with memory and trig functions.
1375 It uses multiple data-types for semantic values.
1376* Exercises:: Ideas for improving the multi-function calculator.
1377@end menu
1378
342b8b6e 1379@node RPN Calc
bfa74976
RS
1380@section Reverse Polish Notation Calculator
1381@cindex reverse polish notation
1382@cindex polish notation calculator
1383@cindex @code{rpcalc}
1384@cindex calculator, simple
1385
1386The first example is that of a simple double-precision @dfn{reverse polish
1387notation} calculator (a calculator using postfix operators). This example
1388provides a good starting point, since operator precedence is not an issue.
1389The second example will illustrate how operator precedence is handled.
1390
1391The source code for this calculator is named @file{rpcalc.y}. The
1392@samp{.y} extension is a convention used for Bison input files.
1393
1394@menu
75f5aaea 1395* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
bfa74976
RS
1396* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
1397* Lexer: Rpcalc Lexer. The lexical analyzer.
1398* Main: Rpcalc Main. The controlling function.
1399* Error: Rpcalc Error. The error reporting function.
1400* Gen: Rpcalc Gen. Running Bison on the grammar file.
1401* Comp: Rpcalc Compile. Run the C compiler on the output code.
1402@end menu
1403
342b8b6e 1404@node Rpcalc Decls
bfa74976
RS
1405@subsection Declarations for @code{rpcalc}
1406
1407Here are the C and Bison declarations for the reverse polish notation
1408calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1409
1410@example
72d2299c 1411/* Reverse polish notation calculator. */
bfa74976
RS
1412
1413%@{
38a92d50
PE
1414 #define YYSTYPE double
1415 #include <math.h>
1416 int yylex (void);
1417 void yyerror (char const *);
bfa74976
RS
1418%@}
1419
1420%token NUM
1421
72d2299c 1422%% /* Grammar rules and actions follow. */
bfa74976
RS
1423@end example
1424
75f5aaea 1425The declarations section (@pxref{Prologue, , The prologue}) contains two
38a92d50 1426preprocessor directives and two forward declarations.
bfa74976
RS
1427
1428The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1964ad8c
AD
1429specifying the C data type for semantic values of both tokens and
1430groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1431Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1432don't define it, @code{int} is the default. Because we specify
1433@code{double}, each token and each expression has an associated value,
1434which is a floating point number.
bfa74976
RS
1435
1436The @code{#include} directive is used to declare the exponentiation
1437function @code{pow}.
1438
38a92d50
PE
1439The forward declarations for @code{yylex} and @code{yyerror} are
1440needed because the C language requires that functions be declared
1441before they are used. These functions will be defined in the
1442epilogue, but the parser calls them so they must be declared in the
1443prologue.
1444
704a47c4
AD
1445The second section, Bison declarations, provides information to Bison
1446about the token types (@pxref{Bison Declarations, ,The Bison
1447Declarations Section}). Each terminal symbol that is not a
1448single-character literal must be declared here. (Single-character
bfa74976
RS
1449literals normally don't need to be declared.) In this example, all the
1450arithmetic operators are designated by single-character literals, so the
1451only terminal symbol that needs to be declared is @code{NUM}, the token
1452type for numeric constants.
1453
342b8b6e 1454@node Rpcalc Rules
bfa74976
RS
1455@subsection Grammar Rules for @code{rpcalc}
1456
1457Here are the grammar rules for the reverse polish notation calculator.
1458
1459@example
1460input: /* empty */
1461 | input line
1462;
1463
1464line: '\n'
18b519c0 1465 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
bfa74976
RS
1466;
1467
18b519c0
AD
1468exp: NUM @{ $$ = $1; @}
1469 | exp exp '+' @{ $$ = $1 + $2; @}
1470 | exp exp '-' @{ $$ = $1 - $2; @}
1471 | exp exp '*' @{ $$ = $1 * $2; @}
1472 | exp exp '/' @{ $$ = $1 / $2; @}
1473 /* Exponentiation */
1474 | exp exp '^' @{ $$ = pow ($1, $2); @}
1475 /* Unary minus */
1476 | exp 'n' @{ $$ = -$1; @}
bfa74976
RS
1477;
1478%%
1479@end example
1480
1481The groupings of the rpcalc ``language'' defined here are the expression
1482(given the name @code{exp}), the line of input (@code{line}), and the
1483complete input transcript (@code{input}). Each of these nonterminal
1484symbols has several alternate rules, joined by the @samp{|} punctuator
1485which is read as ``or''. The following sections explain what these rules
1486mean.
1487
1488The semantics of the language is determined by the actions taken when a
1489grouping is recognized. The actions are the C code that appears inside
1490braces. @xref{Actions}.
1491
1492You must specify these actions in C, but Bison provides the means for
1493passing semantic values between the rules. In each action, the
1494pseudo-variable @code{$$} stands for the semantic value for the grouping
1495that the rule is going to construct. Assigning a value to @code{$$} is the
1496main job of most actions. The semantic values of the components of the
1497rule are referred to as @code{$1}, @code{$2}, and so on.
1498
1499@menu
13863333
AD
1500* Rpcalc Input::
1501* Rpcalc Line::
1502* Rpcalc Expr::
bfa74976
RS
1503@end menu
1504
342b8b6e 1505@node Rpcalc Input
bfa74976
RS
1506@subsubsection Explanation of @code{input}
1507
1508Consider the definition of @code{input}:
1509
1510@example
1511input: /* empty */
1512 | input line
1513;
1514@end example
1515
1516This definition reads as follows: ``A complete input is either an empty
1517string, or a complete input followed by an input line''. Notice that
1518``complete input'' is defined in terms of itself. This definition is said
1519to be @dfn{left recursive} since @code{input} appears always as the
1520leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1521
1522The first alternative is empty because there are no symbols between the
1523colon and the first @samp{|}; this means that @code{input} can match an
1524empty string of input (no tokens). We write the rules this way because it
1525is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1526It's conventional to put an empty alternative first and write the comment
1527@samp{/* empty */} in it.
1528
1529The second alternate rule (@code{input line}) handles all nontrivial input.
1530It means, ``After reading any number of lines, read one more line if
1531possible.'' The left recursion makes this rule into a loop. Since the
1532first alternative matches empty input, the loop can be executed zero or
1533more times.
1534
1535The parser function @code{yyparse} continues to process input until a
1536grammatical error is seen or the lexical analyzer says there are no more
72d2299c 1537input tokens; we will arrange for the latter to happen at end-of-input.
bfa74976 1538
342b8b6e 1539@node Rpcalc Line
bfa74976
RS
1540@subsubsection Explanation of @code{line}
1541
1542Now consider the definition of @code{line}:
1543
1544@example
1545line: '\n'
1546 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1547;
1548@end example
1549
1550The first alternative is a token which is a newline character; this means
1551that rpcalc accepts a blank line (and ignores it, since there is no
1552action). The second alternative is an expression followed by a newline.
1553This is the alternative that makes rpcalc useful. The semantic value of
1554the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1555question is the first symbol in the alternative. The action prints this
1556value, which is the result of the computation the user asked for.
1557
1558This action is unusual because it does not assign a value to @code{$$}. As
1559a consequence, the semantic value associated with the @code{line} is
1560uninitialized (its value will be unpredictable). This would be a bug if
1561that value were ever used, but we don't use it: once rpcalc has printed the
1562value of the user's input line, that value is no longer needed.
1563
342b8b6e 1564@node Rpcalc Expr
bfa74976
RS
1565@subsubsection Explanation of @code{expr}
1566
1567The @code{exp} grouping has several rules, one for each kind of expression.
1568The first rule handles the simplest expressions: those that are just numbers.
1569The second handles an addition-expression, which looks like two expressions
1570followed by a plus-sign. The third handles subtraction, and so on.
1571
1572@example
1573exp: NUM
1574 | exp exp '+' @{ $$ = $1 + $2; @}
1575 | exp exp '-' @{ $$ = $1 - $2; @}
1576 @dots{}
1577 ;
1578@end example
1579
1580We have used @samp{|} to join all the rules for @code{exp}, but we could
1581equally well have written them separately:
1582
1583@example
1584exp: NUM ;
1585exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1586exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1587 @dots{}
1588@end example
1589
1590Most of the rules have actions that compute the value of the expression in
1591terms of the value of its parts. For example, in the rule for addition,
1592@code{$1} refers to the first component @code{exp} and @code{$2} refers to
1593the second one. The third component, @code{'+'}, has no meaningful
1594associated semantic value, but if it had one you could refer to it as
1595@code{$3}. When @code{yyparse} recognizes a sum expression using this
1596rule, the sum of the two subexpressions' values is produced as the value of
1597the entire expression. @xref{Actions}.
1598
1599You don't have to give an action for every rule. When a rule has no
1600action, Bison by default copies the value of @code{$1} into @code{$$}.
1601This is what happens in the first rule (the one that uses @code{NUM}).
1602
1603The formatting shown here is the recommended convention, but Bison does
72d2299c 1604not require it. You can add or change white space as much as you wish.
bfa74976
RS
1605For example, this:
1606
1607@example
99a9344e 1608exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
bfa74976
RS
1609@end example
1610
1611@noindent
1612means the same thing as this:
1613
1614@example
1615exp: NUM
1616 | exp exp '+' @{ $$ = $1 + $2; @}
1617 | @dots{}
99a9344e 1618;
bfa74976
RS
1619@end example
1620
1621@noindent
1622The latter, however, is much more readable.
1623
342b8b6e 1624@node Rpcalc Lexer
bfa74976
RS
1625@subsection The @code{rpcalc} Lexical Analyzer
1626@cindex writing a lexical analyzer
1627@cindex lexical analyzer, writing
1628
704a47c4
AD
1629The lexical analyzer's job is low-level parsing: converting characters
1630or sequences of characters into tokens. The Bison parser gets its
1631tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1632Analyzer Function @code{yylex}}.
bfa74976 1633
c827f760
PE
1634Only a simple lexical analyzer is needed for the @acronym{RPN}
1635calculator. This
bfa74976
RS
1636lexical analyzer skips blanks and tabs, then reads in numbers as
1637@code{double} and returns them as @code{NUM} tokens. Any other character
1638that isn't part of a number is a separate token. Note that the token-code
1639for such a single-character token is the character itself.
1640
1641The return value of the lexical analyzer function is a numeric code which
1642represents a token type. The same text used in Bison rules to stand for
1643this token type is also a C expression for the numeric code for the type.
1644This works in two ways. If the token type is a character literal, then its
e966383b 1645numeric code is that of the character; you can use the same
bfa74976
RS
1646character literal in the lexical analyzer to express the number. If the
1647token type is an identifier, that identifier is defined by Bison as a C
1648macro whose definition is the appropriate number. In this example,
1649therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1650
1964ad8c
AD
1651The semantic value of the token (if it has one) is stored into the
1652global variable @code{yylval}, which is where the Bison parser will look
1653for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1654defined at the beginning of the grammar; @pxref{Rpcalc Decls,
1655,Declarations for @code{rpcalc}}.)
bfa74976 1656
72d2299c
PE
1657A token type code of zero is returned if the end-of-input is encountered.
1658(Bison recognizes any nonpositive value as indicating end-of-input.)
bfa74976
RS
1659
1660Here is the code for the lexical analyzer:
1661
1662@example
1663@group
72d2299c 1664/* The lexical analyzer returns a double floating point
e966383b 1665 number on the stack and the token NUM, or the numeric code
72d2299c
PE
1666 of the character read if not a number. It skips all blanks
1667 and tabs, and returns 0 for end-of-input. */
bfa74976
RS
1668
1669#include <ctype.h>
1670@end group
1671
1672@group
13863333
AD
1673int
1674yylex (void)
bfa74976
RS
1675@{
1676 int c;
1677
72d2299c 1678 /* Skip white space. */
13863333 1679 while ((c = getchar ()) == ' ' || c == '\t')
bfa74976
RS
1680 ;
1681@end group
1682@group
72d2299c 1683 /* Process numbers. */
13863333 1684 if (c == '.' || isdigit (c))
bfa74976
RS
1685 @{
1686 ungetc (c, stdin);
1687 scanf ("%lf", &yylval);
1688 return NUM;
1689 @}
1690@end group
1691@group
72d2299c 1692 /* Return end-of-input. */
13863333 1693 if (c == EOF)
bfa74976 1694 return 0;
72d2299c 1695 /* Return a single char. */
13863333 1696 return c;
bfa74976
RS
1697@}
1698@end group
1699@end example
1700
342b8b6e 1701@node Rpcalc Main
bfa74976
RS
1702@subsection The Controlling Function
1703@cindex controlling function
1704@cindex main function in simple example
1705
1706In keeping with the spirit of this example, the controlling function is
1707kept to the bare minimum. The only requirement is that it call
1708@code{yyparse} to start the process of parsing.
1709
1710@example
1711@group
13863333
AD
1712int
1713main (void)
bfa74976 1714@{
13863333 1715 return yyparse ();
bfa74976
RS
1716@}
1717@end group
1718@end example
1719
342b8b6e 1720@node Rpcalc Error
bfa74976
RS
1721@subsection The Error Reporting Routine
1722@cindex error reporting routine
1723
1724When @code{yyparse} detects a syntax error, it calls the error reporting
13863333 1725function @code{yyerror} to print an error message (usually but not
6e649e65 1726always @code{"syntax error"}). It is up to the programmer to supply
13863333
AD
1727@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1728here is the definition we will use:
bfa74976
RS
1729
1730@example
1731@group
1732#include <stdio.h>
1733
38a92d50 1734/* Called by yyparse on error. */
13863333 1735void
38a92d50 1736yyerror (char const *s)
bfa74976 1737@{
4e03e201 1738 fprintf (stderr, "%s\n", s);
bfa74976
RS
1739@}
1740@end group
1741@end example
1742
1743After @code{yyerror} returns, the Bison parser may recover from the error
1744and continue parsing if the grammar contains a suitable error rule
1745(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1746have not written any error rules in this example, so any invalid input will
1747cause the calculator program to exit. This is not clean behavior for a
9ecbd125 1748real calculator, but it is adequate for the first example.
bfa74976 1749
342b8b6e 1750@node Rpcalc Gen
bfa74976
RS
1751@subsection Running Bison to Make the Parser
1752@cindex running Bison (introduction)
1753
ceed8467
AD
1754Before running Bison to produce a parser, we need to decide how to
1755arrange all the source code in one or more source files. For such a
1756simple example, the easiest thing is to put everything in one file. The
1757definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
342b8b6e 1758end, in the epilogue of the file
75f5aaea 1759(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
bfa74976
RS
1760
1761For a large project, you would probably have several source files, and use
1762@code{make} to arrange to recompile them.
1763
1764With all the source in a single file, you use the following command to
1765convert it into a parser file:
1766
1767@example
fa4d969f 1768bison @var{file}.y
bfa74976
RS
1769@end example
1770
1771@noindent
1772In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
fa4d969f 1773@sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c},
72d2299c 1774removing the @samp{.y} from the original file name. The file output by
bfa74976
RS
1775Bison contains the source code for @code{yyparse}. The additional
1776functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
1777are copied verbatim to the output.
1778
342b8b6e 1779@node Rpcalc Compile
bfa74976
RS
1780@subsection Compiling the Parser File
1781@cindex compiling the parser
1782
1783Here is how to compile and run the parser file:
1784
1785@example
1786@group
1787# @r{List files in current directory.}
9edcd895 1788$ @kbd{ls}
bfa74976
RS
1789rpcalc.tab.c rpcalc.y
1790@end group
1791
1792@group
1793# @r{Compile the Bison parser.}
1794# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
b56471a6 1795$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
bfa74976
RS
1796@end group
1797
1798@group
1799# @r{List files again.}
9edcd895 1800$ @kbd{ls}
bfa74976
RS
1801rpcalc rpcalc.tab.c rpcalc.y
1802@end group
1803@end example
1804
1805The file @file{rpcalc} now contains the executable code. Here is an
1806example session using @code{rpcalc}.
1807
1808@example
9edcd895
AD
1809$ @kbd{rpcalc}
1810@kbd{4 9 +}
bfa74976 181113
9edcd895 1812@kbd{3 7 + 3 4 5 *+-}
bfa74976 1813-13
9edcd895 1814@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
bfa74976 181513
9edcd895 1816@kbd{5 6 / 4 n +}
bfa74976 1817-3.166666667
9edcd895 1818@kbd{3 4 ^} @r{Exponentiation}
bfa74976 181981
9edcd895
AD
1820@kbd{^D} @r{End-of-file indicator}
1821$
bfa74976
RS
1822@end example
1823
342b8b6e 1824@node Infix Calc
bfa74976
RS
1825@section Infix Notation Calculator: @code{calc}
1826@cindex infix notation calculator
1827@cindex @code{calc}
1828@cindex calculator, infix notation
1829
1830We now modify rpcalc to handle infix operators instead of postfix. Infix
1831notation involves the concept of operator precedence and the need for
1832parentheses nested to arbitrary depth. Here is the Bison code for
1833@file{calc.y}, an infix desk-top calculator.
1834
1835@example
38a92d50 1836/* Infix notation calculator. */
bfa74976
RS
1837
1838%@{
38a92d50
PE
1839 #define YYSTYPE double
1840 #include <math.h>
1841 #include <stdio.h>
1842 int yylex (void);
1843 void yyerror (char const *);
bfa74976
RS
1844%@}
1845
38a92d50 1846/* Bison declarations. */
bfa74976
RS
1847%token NUM
1848%left '-' '+'
1849%left '*' '/'
1850%left NEG /* negation--unary minus */
38a92d50 1851%right '^' /* exponentiation */
bfa74976 1852
38a92d50
PE
1853%% /* The grammar follows. */
1854input: /* empty */
bfa74976
RS
1855 | input line
1856;
1857
1858line: '\n'
1859 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1860;
1861
1862exp: NUM @{ $$ = $1; @}
1863 | exp '+' exp @{ $$ = $1 + $3; @}
1864 | exp '-' exp @{ $$ = $1 - $3; @}
1865 | exp '*' exp @{ $$ = $1 * $3; @}
1866 | exp '/' exp @{ $$ = $1 / $3; @}
1867 | '-' exp %prec NEG @{ $$ = -$2; @}
1868 | exp '^' exp @{ $$ = pow ($1, $3); @}
1869 | '(' exp ')' @{ $$ = $2; @}
1870;
1871%%
1872@end example
1873
1874@noindent
ceed8467
AD
1875The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1876same as before.
bfa74976
RS
1877
1878There are two important new features shown in this code.
1879
1880In the second section (Bison declarations), @code{%left} declares token
1881types and says they are left-associative operators. The declarations
1882@code{%left} and @code{%right} (right associativity) take the place of
1883@code{%token} which is used to declare a token type name without
1884associativity. (These tokens are single-character literals, which
1885ordinarily don't need to be declared. We declare them here to specify
1886the associativity.)
1887
1888Operator precedence is determined by the line ordering of the
1889declarations; the higher the line number of the declaration (lower on
1890the page or screen), the higher the precedence. Hence, exponentiation
1891has the highest precedence, unary minus (@code{NEG}) is next, followed
704a47c4
AD
1892by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator
1893Precedence}.
bfa74976 1894
704a47c4
AD
1895The other important new feature is the @code{%prec} in the grammar
1896section for the unary minus operator. The @code{%prec} simply instructs
1897Bison that the rule @samp{| '-' exp} has the same precedence as
1898@code{NEG}---in this case the next-to-highest. @xref{Contextual
1899Precedence, ,Context-Dependent Precedence}.
bfa74976
RS
1900
1901Here is a sample run of @file{calc.y}:
1902
1903@need 500
1904@example
9edcd895
AD
1905$ @kbd{calc}
1906@kbd{4 + 4.5 - (34/(8*3+-3))}
bfa74976 19076.880952381
9edcd895 1908@kbd{-56 + 2}
bfa74976 1909-54
9edcd895 1910@kbd{3 ^ 2}
bfa74976
RS
19119
1912@end example
1913
342b8b6e 1914@node Simple Error Recovery
bfa74976
RS
1915@section Simple Error Recovery
1916@cindex error recovery, simple
1917
1918Up to this point, this manual has not addressed the issue of @dfn{error
1919recovery}---how to continue parsing after the parser detects a syntax
ceed8467
AD
1920error. All we have handled is error reporting with @code{yyerror}.
1921Recall that by default @code{yyparse} returns after calling
1922@code{yyerror}. This means that an erroneous input line causes the
1923calculator program to exit. Now we show how to rectify this deficiency.
bfa74976
RS
1924
1925The Bison language itself includes the reserved word @code{error}, which
1926may be included in the grammar rules. In the example below it has
1927been added to one of the alternatives for @code{line}:
1928
1929@example
1930@group
1931line: '\n'
1932 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1933 | error '\n' @{ yyerrok; @}
1934;
1935@end group
1936@end example
1937
ceed8467 1938This addition to the grammar allows for simple error recovery in the
6e649e65 1939event of a syntax error. If an expression that cannot be evaluated is
ceed8467
AD
1940read, the error will be recognized by the third rule for @code{line},
1941and parsing will continue. (The @code{yyerror} function is still called
1942upon to print its message as well.) The action executes the statement
1943@code{yyerrok}, a macro defined automatically by Bison; its meaning is
1944that error recovery is complete (@pxref{Error Recovery}). Note the
1945difference between @code{yyerrok} and @code{yyerror}; neither one is a
e0c471a9 1946misprint.
bfa74976
RS
1947
1948This form of error recovery deals with syntax errors. There are other
1949kinds of errors; for example, division by zero, which raises an exception
1950signal that is normally fatal. A real calculator program must handle this
1951signal and use @code{longjmp} to return to @code{main} and resume parsing
1952input lines; it would also have to discard the rest of the current line of
1953input. We won't discuss this issue further because it is not specific to
1954Bison programs.
1955
342b8b6e
AD
1956@node Location Tracking Calc
1957@section Location Tracking Calculator: @code{ltcalc}
1958@cindex location tracking calculator
1959@cindex @code{ltcalc}
1960@cindex calculator, location tracking
1961
9edcd895
AD
1962This example extends the infix notation calculator with location
1963tracking. This feature will be used to improve the error messages. For
1964the sake of clarity, this example is a simple integer calculator, since
1965most of the work needed to use locations will be done in the lexical
72d2299c 1966analyzer.
342b8b6e
AD
1967
1968@menu
1969* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
1970* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
1971* Lexer: Ltcalc Lexer. The lexical analyzer.
1972@end menu
1973
1974@node Ltcalc Decls
1975@subsection Declarations for @code{ltcalc}
1976
9edcd895
AD
1977The C and Bison declarations for the location tracking calculator are
1978the same as the declarations for the infix notation calculator.
342b8b6e
AD
1979
1980@example
1981/* Location tracking calculator. */
1982
1983%@{
38a92d50
PE
1984 #define YYSTYPE int
1985 #include <math.h>
1986 int yylex (void);
1987 void yyerror (char const *);
342b8b6e
AD
1988%@}
1989
1990/* Bison declarations. */
1991%token NUM
1992
1993%left '-' '+'
1994%left '*' '/'
1995%left NEG
1996%right '^'
1997
38a92d50 1998%% /* The grammar follows. */
342b8b6e
AD
1999@end example
2000
9edcd895
AD
2001@noindent
2002Note there are no declarations specific to locations. Defining a data
2003type for storing locations is not needed: we will use the type provided
2004by default (@pxref{Location Type, ,Data Types of Locations}), which is a
2005four member structure with the following integer fields:
2006@code{first_line}, @code{first_column}, @code{last_line} and
2007@code{last_column}.
342b8b6e
AD
2008
2009@node Ltcalc Rules
2010@subsection Grammar Rules for @code{ltcalc}
2011
9edcd895
AD
2012Whether handling locations or not has no effect on the syntax of your
2013language. Therefore, grammar rules for this example will be very close
2014to those of the previous example: we will only modify them to benefit
2015from the new information.
342b8b6e 2016
9edcd895
AD
2017Here, we will use locations to report divisions by zero, and locate the
2018wrong expressions or subexpressions.
342b8b6e
AD
2019
2020@example
2021@group
2022input : /* empty */
2023 | input line
2024;
2025@end group
2026
2027@group
2028line : '\n'
2029 | exp '\n' @{ printf ("%d\n", $1); @}
2030;
2031@end group
2032
2033@group
2034exp : NUM @{ $$ = $1; @}
2035 | exp '+' exp @{ $$ = $1 + $3; @}
2036 | exp '-' exp @{ $$ = $1 - $3; @}
2037 | exp '*' exp @{ $$ = $1 * $3; @}
2038@end group
342b8b6e 2039@group
9edcd895 2040 | exp '/' exp
342b8b6e
AD
2041 @{
2042 if ($3)
2043 $$ = $1 / $3;
2044 else
2045 @{
2046 $$ = 1;
9edcd895
AD
2047 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2048 @@3.first_line, @@3.first_column,
2049 @@3.last_line, @@3.last_column);
342b8b6e
AD
2050 @}
2051 @}
2052@end group
2053@group
2054 | '-' exp %preg NEG @{ $$ = -$2; @}
2055 | exp '^' exp @{ $$ = pow ($1, $3); @}
2056 | '(' exp ')' @{ $$ = $2; @}
2057@end group
2058@end example
2059
2060This code shows how to reach locations inside of semantic actions, by
2061using the pseudo-variables @code{@@@var{n}} for rule components, and the
2062pseudo-variable @code{@@$} for groupings.
2063
9edcd895
AD
2064We don't need to assign a value to @code{@@$}: the output parser does it
2065automatically. By default, before executing the C code of each action,
2066@code{@@$} is set to range from the beginning of @code{@@1} to the end
2067of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2068can be redefined (@pxref{Location Default Action, , Default Action for
2069Locations}), and for very specific rules, @code{@@$} can be computed by
2070hand.
342b8b6e
AD
2071
2072@node Ltcalc Lexer
2073@subsection The @code{ltcalc} Lexical Analyzer.
2074
9edcd895 2075Until now, we relied on Bison's defaults to enable location
72d2299c 2076tracking. The next step is to rewrite the lexical analyzer, and make it
9edcd895
AD
2077able to feed the parser with the token locations, as it already does for
2078semantic values.
342b8b6e 2079
9edcd895
AD
2080To this end, we must take into account every single character of the
2081input text, to avoid the computed locations of being fuzzy or wrong:
342b8b6e
AD
2082
2083@example
2084@group
2085int
2086yylex (void)
2087@{
2088 int c;
18b519c0 2089@end group
342b8b6e 2090
18b519c0 2091@group
72d2299c 2092 /* Skip white space. */
342b8b6e
AD
2093 while ((c = getchar ()) == ' ' || c == '\t')
2094 ++yylloc.last_column;
18b519c0 2095@end group
342b8b6e 2096
18b519c0 2097@group
72d2299c 2098 /* Step. */
342b8b6e
AD
2099 yylloc.first_line = yylloc.last_line;
2100 yylloc.first_column = yylloc.last_column;
2101@end group
2102
2103@group
72d2299c 2104 /* Process numbers. */
342b8b6e
AD
2105 if (isdigit (c))
2106 @{
2107 yylval = c - '0';
2108 ++yylloc.last_column;
2109 while (isdigit (c = getchar ()))
2110 @{
2111 ++yylloc.last_column;
2112 yylval = yylval * 10 + c - '0';
2113 @}
2114 ungetc (c, stdin);
2115 return NUM;
2116 @}
2117@end group
2118
72d2299c 2119 /* Return end-of-input. */
342b8b6e
AD
2120 if (c == EOF)
2121 return 0;
2122
72d2299c 2123 /* Return a single char, and update location. */
342b8b6e
AD
2124 if (c == '\n')
2125 @{
2126 ++yylloc.last_line;
2127 yylloc.last_column = 0;
2128 @}
2129 else
2130 ++yylloc.last_column;
2131 return c;
2132@}
2133@end example
2134
9edcd895
AD
2135Basically, the lexical analyzer performs the same processing as before:
2136it skips blanks and tabs, and reads numbers or single-character tokens.
2137In addition, it updates @code{yylloc}, the global variable (of type
2138@code{YYLTYPE}) containing the token's location.
342b8b6e 2139
9edcd895 2140Now, each time this function returns a token, the parser has its number
72d2299c 2141as well as its semantic value, and its location in the text. The last
9edcd895
AD
2142needed change is to initialize @code{yylloc}, for example in the
2143controlling function:
342b8b6e
AD
2144
2145@example
9edcd895 2146@group
342b8b6e
AD
2147int
2148main (void)
2149@{
2150 yylloc.first_line = yylloc.last_line = 1;
2151 yylloc.first_column = yylloc.last_column = 0;
2152 return yyparse ();
2153@}
9edcd895 2154@end group
342b8b6e
AD
2155@end example
2156
9edcd895
AD
2157Remember that computing locations is not a matter of syntax. Every
2158character must be associated to a location update, whether it is in
2159valid input, in comments, in literal strings, and so on.
342b8b6e
AD
2160
2161@node Multi-function Calc
bfa74976
RS
2162@section Multi-Function Calculator: @code{mfcalc}
2163@cindex multi-function calculator
2164@cindex @code{mfcalc}
2165@cindex calculator, multi-function
2166
2167Now that the basics of Bison have been discussed, it is time to move on to
2168a more advanced problem. The above calculators provided only five
2169functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2170be nice to have a calculator that provides other mathematical functions such
2171as @code{sin}, @code{cos}, etc.
2172
2173It is easy to add new operators to the infix calculator as long as they are
2174only single-character literals. The lexical analyzer @code{yylex} passes
9d9b8b70 2175back all nonnumeric characters as tokens, so new grammar rules suffice for
bfa74976
RS
2176adding a new operator. But we want something more flexible: built-in
2177functions whose syntax has this form:
2178
2179@example
2180@var{function_name} (@var{argument})
2181@end example
2182
2183@noindent
2184At the same time, we will add memory to the calculator, by allowing you
2185to create named variables, store values in them, and use them later.
2186Here is a sample session with the multi-function calculator:
2187
2188@example
9edcd895
AD
2189$ @kbd{mfcalc}
2190@kbd{pi = 3.141592653589}
bfa74976 21913.1415926536
9edcd895 2192@kbd{sin(pi)}
bfa74976 21930.0000000000
9edcd895 2194@kbd{alpha = beta1 = 2.3}
bfa74976 21952.3000000000
9edcd895 2196@kbd{alpha}
bfa74976 21972.3000000000
9edcd895 2198@kbd{ln(alpha)}
bfa74976 21990.8329091229
9edcd895 2200@kbd{exp(ln(beta1))}
bfa74976 22012.3000000000
9edcd895 2202$
bfa74976
RS
2203@end example
2204
2205Note that multiple assignment and nested function calls are permitted.
2206
2207@menu
2208* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
2209* Rules: Mfcalc Rules. Grammar rules for the calculator.
2210* Symtab: Mfcalc Symtab. Symbol table management subroutines.
2211@end menu
2212
342b8b6e 2213@node Mfcalc Decl
bfa74976
RS
2214@subsection Declarations for @code{mfcalc}
2215
2216Here are the C and Bison declarations for the multi-function calculator.
2217
2218@smallexample
18b519c0 2219@group
bfa74976 2220%@{
38a92d50
PE
2221 #include <math.h> /* For math functions, cos(), sin(), etc. */
2222 #include "calc.h" /* Contains definition of `symrec'. */
2223 int yylex (void);
2224 void yyerror (char const *);
bfa74976 2225%@}
18b519c0
AD
2226@end group
2227@group
bfa74976 2228%union @{
38a92d50
PE
2229 double val; /* For returning numbers. */
2230 symrec *tptr; /* For returning symbol-table pointers. */
bfa74976 2231@}
18b519c0 2232@end group
38a92d50
PE
2233%token <val> NUM /* Simple double precision number. */
2234%token <tptr> VAR FNCT /* Variable and Function. */
bfa74976
RS
2235%type <val> exp
2236
18b519c0 2237@group
bfa74976
RS
2238%right '='
2239%left '-' '+'
2240%left '*' '/'
38a92d50
PE
2241%left NEG /* negation--unary minus */
2242%right '^' /* exponentiation */
18b519c0 2243@end group
38a92d50 2244%% /* The grammar follows. */
bfa74976
RS
2245@end smallexample
2246
2247The above grammar introduces only two new features of the Bison language.
2248These features allow semantic values to have various data types
2249(@pxref{Multiple Types, ,More Than One Value Type}).
2250
2251The @code{%union} declaration specifies the entire list of possible types;
2252this is instead of defining @code{YYSTYPE}. The allowable types are now
2253double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2254the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2255
2256Since values can now have various types, it is necessary to associate a
2257type with each grammar symbol whose semantic value is used. These symbols
2258are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2259declarations are augmented with information about their data type (placed
2260between angle brackets).
2261
704a47c4
AD
2262The Bison construct @code{%type} is used for declaring nonterminal
2263symbols, just as @code{%token} is used for declaring token types. We
2264have not used @code{%type} before because nonterminal symbols are
2265normally declared implicitly by the rules that define them. But
2266@code{exp} must be declared explicitly so we can specify its value type.
2267@xref{Type Decl, ,Nonterminal Symbols}.
bfa74976 2268
342b8b6e 2269@node Mfcalc Rules
bfa74976
RS
2270@subsection Grammar Rules for @code{mfcalc}
2271
2272Here are the grammar rules for the multi-function calculator.
2273Most of them are copied directly from @code{calc}; three rules,
2274those which mention @code{VAR} or @code{FNCT}, are new.
2275
2276@smallexample
18b519c0 2277@group
bfa74976
RS
2278input: /* empty */
2279 | input line
2280;
18b519c0 2281@end group
bfa74976 2282
18b519c0 2283@group
bfa74976
RS
2284line:
2285 '\n'
2286 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2287 | error '\n' @{ yyerrok; @}
2288;
18b519c0 2289@end group
bfa74976 2290
18b519c0 2291@group
bfa74976
RS
2292exp: NUM @{ $$ = $1; @}
2293 | VAR @{ $$ = $1->value.var; @}
2294 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2295 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2296 | exp '+' exp @{ $$ = $1 + $3; @}
2297 | exp '-' exp @{ $$ = $1 - $3; @}
2298 | exp '*' exp @{ $$ = $1 * $3; @}
2299 | exp '/' exp @{ $$ = $1 / $3; @}
2300 | '-' exp %prec NEG @{ $$ = -$2; @}
2301 | exp '^' exp @{ $$ = pow ($1, $3); @}
2302 | '(' exp ')' @{ $$ = $2; @}
2303;
18b519c0 2304@end group
38a92d50 2305/* End of grammar. */
bfa74976
RS
2306%%
2307@end smallexample
2308
342b8b6e 2309@node Mfcalc Symtab
bfa74976
RS
2310@subsection The @code{mfcalc} Symbol Table
2311@cindex symbol table example
2312
2313The multi-function calculator requires a symbol table to keep track of the
2314names and meanings of variables and functions. This doesn't affect the
2315grammar rules (except for the actions) or the Bison declarations, but it
2316requires some additional C functions for support.
2317
2318The symbol table itself consists of a linked list of records. Its
2319definition, which is kept in the header @file{calc.h}, is as follows. It
2320provides for either functions or variables to be placed in the table.
2321
2322@smallexample
2323@group
38a92d50 2324/* Function type. */
32dfccf8 2325typedef double (*func_t) (double);
72f889cc 2326@end group
32dfccf8 2327
72f889cc 2328@group
38a92d50 2329/* Data type for links in the chain of symbols. */
bfa74976
RS
2330struct symrec
2331@{
38a92d50 2332 char *name; /* name of symbol */
bfa74976 2333 int type; /* type of symbol: either VAR or FNCT */
32dfccf8
AD
2334 union
2335 @{
38a92d50
PE
2336 double var; /* value of a VAR */
2337 func_t fnctptr; /* value of a FNCT */
bfa74976 2338 @} value;
38a92d50 2339 struct symrec *next; /* link field */
bfa74976
RS
2340@};
2341@end group
2342
2343@group
2344typedef struct symrec symrec;
2345
38a92d50 2346/* The symbol table: a chain of `struct symrec'. */
bfa74976
RS
2347extern symrec *sym_table;
2348
a730d142 2349symrec *putsym (char const *, int);
38a92d50 2350symrec *getsym (char const *);
bfa74976
RS
2351@end group
2352@end smallexample
2353
2354The new version of @code{main} includes a call to @code{init_table}, a
2355function that initializes the symbol table. Here it is, and
2356@code{init_table} as well:
2357
2358@smallexample
bfa74976
RS
2359#include <stdio.h>
2360
18b519c0 2361@group
38a92d50 2362/* Called by yyparse on error. */
13863333 2363void
38a92d50 2364yyerror (char const *s)
bfa74976
RS
2365@{
2366 printf ("%s\n", s);
2367@}
18b519c0 2368@end group
bfa74976 2369
18b519c0 2370@group
bfa74976
RS
2371struct init
2372@{
38a92d50
PE
2373 char const *fname;
2374 double (*fnct) (double);
bfa74976
RS
2375@};
2376@end group
2377
2378@group
38a92d50 2379struct init const arith_fncts[] =
13863333 2380@{
32dfccf8
AD
2381 "sin", sin,
2382 "cos", cos,
13863333 2383 "atan", atan,
32dfccf8
AD
2384 "ln", log,
2385 "exp", exp,
13863333
AD
2386 "sqrt", sqrt,
2387 0, 0
2388@};
18b519c0 2389@end group
bfa74976 2390
18b519c0 2391@group
bfa74976 2392/* The symbol table: a chain of `struct symrec'. */
38a92d50 2393symrec *sym_table;
bfa74976
RS
2394@end group
2395
2396@group
72d2299c 2397/* Put arithmetic functions in table. */
13863333
AD
2398void
2399init_table (void)
bfa74976
RS
2400@{
2401 int i;
2402 symrec *ptr;
2403 for (i = 0; arith_fncts[i].fname != 0; i++)
2404 @{
2405 ptr = putsym (arith_fncts[i].fname, FNCT);
2406 ptr->value.fnctptr = arith_fncts[i].fnct;
2407 @}
2408@}
2409@end group
38a92d50
PE
2410
2411@group
2412int
2413main (void)
2414@{
2415 init_table ();
2416 return yyparse ();
2417@}
2418@end group
bfa74976
RS
2419@end smallexample
2420
2421By simply editing the initialization list and adding the necessary include
2422files, you can add additional functions to the calculator.
2423
2424Two important functions allow look-up and installation of symbols in the
2425symbol table. The function @code{putsym} is passed a name and the type
2426(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2427linked to the front of the list, and a pointer to the object is returned.
2428The function @code{getsym} is passed the name of the symbol to look up. If
2429found, a pointer to that symbol is returned; otherwise zero is returned.
2430
2431@smallexample
2432symrec *
38a92d50 2433putsym (char const *sym_name, int sym_type)
bfa74976
RS
2434@{
2435 symrec *ptr;
2436 ptr = (symrec *) malloc (sizeof (symrec));
2437 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2438 strcpy (ptr->name,sym_name);
2439 ptr->type = sym_type;
72d2299c 2440 ptr->value.var = 0; /* Set value to 0 even if fctn. */
bfa74976
RS
2441 ptr->next = (struct symrec *)sym_table;
2442 sym_table = ptr;
2443 return ptr;
2444@}
2445
2446symrec *
38a92d50 2447getsym (char const *sym_name)
bfa74976
RS
2448@{
2449 symrec *ptr;
2450 for (ptr = sym_table; ptr != (symrec *) 0;
2451 ptr = (symrec *)ptr->next)
2452 if (strcmp (ptr->name,sym_name) == 0)
2453 return ptr;
2454 return 0;
2455@}
2456@end smallexample
2457
2458The function @code{yylex} must now recognize variables, numeric values, and
2459the single-character arithmetic operators. Strings of alphanumeric
9d9b8b70 2460characters with a leading letter are recognized as either variables or
bfa74976
RS
2461functions depending on what the symbol table says about them.
2462
2463The string is passed to @code{getsym} for look up in the symbol table. If
2464the name appears in the table, a pointer to its location and its type
2465(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2466already in the table, then it is installed as a @code{VAR} using
2467@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
e0c471a9 2468returned to @code{yyparse}.
bfa74976
RS
2469
2470No change is needed in the handling of numeric values and arithmetic
2471operators in @code{yylex}.
2472
2473@smallexample
2474@group
2475#include <ctype.h>
18b519c0 2476@end group
13863333 2477
18b519c0 2478@group
13863333
AD
2479int
2480yylex (void)
bfa74976
RS
2481@{
2482 int c;
2483
72d2299c 2484 /* Ignore white space, get first nonwhite character. */
bfa74976
RS
2485 while ((c = getchar ()) == ' ' || c == '\t');
2486
2487 if (c == EOF)
2488 return 0;
2489@end group
2490
2491@group
2492 /* Char starts a number => parse the number. */
2493 if (c == '.' || isdigit (c))
2494 @{
2495 ungetc (c, stdin);
2496 scanf ("%lf", &yylval.val);
2497 return NUM;
2498 @}
2499@end group
2500
2501@group
2502 /* Char starts an identifier => read the name. */
2503 if (isalpha (c))
2504 @{
2505 symrec *s;
2506 static char *symbuf = 0;
2507 static int length = 0;
2508 int i;
2509@end group
2510
2511@group
2512 /* Initially make the buffer long enough
2513 for a 40-character symbol name. */
2514 if (length == 0)
2515 length = 40, symbuf = (char *)malloc (length + 1);
2516
2517 i = 0;
2518 do
2519@end group
2520@group
2521 @{
2522 /* If buffer is full, make it bigger. */
2523 if (i == length)
2524 @{
2525 length *= 2;
18b519c0 2526 symbuf = (char *) realloc (symbuf, length + 1);
bfa74976
RS
2527 @}
2528 /* Add this character to the buffer. */
2529 symbuf[i++] = c;
2530 /* Get another character. */
2531 c = getchar ();
2532 @}
2533@end group
2534@group
72d2299c 2535 while (isalnum (c));
bfa74976
RS
2536
2537 ungetc (c, stdin);
2538 symbuf[i] = '\0';
2539@end group
2540
2541@group
2542 s = getsym (symbuf);
2543 if (s == 0)
2544 s = putsym (symbuf, VAR);
2545 yylval.tptr = s;
2546 return s->type;
2547 @}
2548
2549 /* Any other character is a token by itself. */
2550 return c;
2551@}
2552@end group
2553@end smallexample
2554
72d2299c 2555This program is both powerful and flexible. You may easily add new
704a47c4
AD
2556functions, and it is a simple job to modify this code to install
2557predefined variables such as @code{pi} or @code{e} as well.
bfa74976 2558
342b8b6e 2559@node Exercises
bfa74976
RS
2560@section Exercises
2561@cindex exercises
2562
2563@enumerate
2564@item
2565Add some new functions from @file{math.h} to the initialization list.
2566
2567@item
2568Add another array that contains constants and their values. Then
2569modify @code{init_table} to add these constants to the symbol table.
2570It will be easiest to give the constants type @code{VAR}.
2571
2572@item
2573Make the program report an error if the user refers to an
2574uninitialized variable in any way except to store a value in it.
2575@end enumerate
2576
342b8b6e 2577@node Grammar File
bfa74976
RS
2578@chapter Bison Grammar Files
2579
2580Bison takes as input a context-free grammar specification and produces a
2581C-language function that recognizes correct instances of the grammar.
2582
2583The Bison grammar input file conventionally has a name ending in @samp{.y}.
234a3be3 2584@xref{Invocation, ,Invoking Bison}.
bfa74976
RS
2585
2586@menu
2587* Grammar Outline:: Overall layout of the grammar file.
2588* Symbols:: Terminal and nonterminal symbols.
2589* Rules:: How to write grammar rules.
2590* Recursion:: Writing recursive rules.
2591* Semantics:: Semantic values and actions.
847bf1f5 2592* Locations:: Locations and actions.
bfa74976
RS
2593* Declarations:: All kinds of Bison declarations are described here.
2594* Multiple Parsers:: Putting more than one Bison parser in one program.
2595@end menu
2596
342b8b6e 2597@node Grammar Outline
bfa74976
RS
2598@section Outline of a Bison Grammar
2599
2600A Bison grammar file has four main sections, shown here with the
2601appropriate delimiters:
2602
2603@example
2604%@{
38a92d50 2605 @var{Prologue}
bfa74976
RS
2606%@}
2607
2608@var{Bison declarations}
2609
2610%%
2611@var{Grammar rules}
2612%%
2613
75f5aaea 2614@var{Epilogue}
bfa74976
RS
2615@end example
2616
2617Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2bfc2e2a
PE
2618As a @acronym{GNU} extension, @samp{//} introduces a comment that
2619continues until end of line.
bfa74976
RS
2620
2621@menu
75f5aaea 2622* Prologue:: Syntax and usage of the prologue.
bfa74976
RS
2623* Bison Declarations:: Syntax and usage of the Bison declarations section.
2624* Grammar Rules:: Syntax and usage of the grammar rules section.
75f5aaea 2625* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
2626@end menu
2627
38a92d50 2628@node Prologue
75f5aaea
MA
2629@subsection The prologue
2630@cindex declarations section
2631@cindex Prologue
2632@cindex declarations
bfa74976 2633
f8e1c9e5
AD
2634The @var{Prologue} section contains macro definitions and declarations
2635of functions and variables that are used in the actions in the grammar
2636rules. These are copied to the beginning of the parser file so that
2637they precede the definition of @code{yyparse}. You can use
2638@samp{#include} to get the declarations from a header file. If you
2639don't need any C declarations, you may omit the @samp{%@{} and
2640@samp{%@}} delimiters that bracket this section.
bfa74976 2641
287c78f6
PE
2642The @var{Prologue} section is terminated by the the first occurrence
2643of @samp{%@}} that is outside a comment, a string literal, or a
2644character constant.
2645
c732d2c6
AD
2646You may have more than one @var{Prologue} section, intermixed with the
2647@var{Bison declarations}. This allows you to have C and Bison
2648declarations that refer to each other. For example, the @code{%union}
2649declaration may use types defined in a header file, and you may wish to
2650prototype functions that take arguments of type @code{YYSTYPE}. This
2651can be done with two @var{Prologue} blocks, one before and one after the
2652@code{%union} declaration.
2653
2654@smallexample
2655%@{
38a92d50
PE
2656 #include <stdio.h>
2657 #include "ptypes.h"
c732d2c6
AD
2658%@}
2659
2660%union @{
779e7ceb 2661 long int n;
c732d2c6
AD
2662 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2663@}
2664
2665%@{
38a92d50
PE
2666 static void print_token_value (FILE *, int, YYSTYPE);
2667 #define YYPRINT(F, N, L) print_token_value (F, N, L)
c732d2c6
AD
2668%@}
2669
2670@dots{}
2671@end smallexample
2672
342b8b6e 2673@node Bison Declarations
bfa74976
RS
2674@subsection The Bison Declarations Section
2675@cindex Bison declarations (introduction)
2676@cindex declarations, Bison (introduction)
2677
2678The @var{Bison declarations} section contains declarations that define
2679terminal and nonterminal symbols, specify precedence, and so on.
2680In some simple grammars you may not need any declarations.
2681@xref{Declarations, ,Bison Declarations}.
2682
342b8b6e 2683@node Grammar Rules
bfa74976
RS
2684@subsection The Grammar Rules Section
2685@cindex grammar rules section
2686@cindex rules section for grammar
2687
2688The @dfn{grammar rules} section contains one or more Bison grammar
2689rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
2690
2691There must always be at least one grammar rule, and the first
2692@samp{%%} (which precedes the grammar rules) may never be omitted even
2693if it is the first thing in the file.
2694
38a92d50 2695@node Epilogue
75f5aaea 2696@subsection The epilogue
bfa74976 2697@cindex additional C code section
75f5aaea 2698@cindex epilogue
bfa74976
RS
2699@cindex C code, section for additional
2700
08e49d20
PE
2701The @var{Epilogue} is copied verbatim to the end of the parser file, just as
2702the @var{Prologue} is copied to the beginning. This is the most convenient
342b8b6e
AD
2703place to put anything that you want to have in the parser file but which need
2704not come before the definition of @code{yyparse}. For example, the
38a92d50
PE
2705definitions of @code{yylex} and @code{yyerror} often go here. Because
2706C requires functions to be declared before being used, you often need
2707to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
e4f85c39 2708even if you define them in the Epilogue.
75f5aaea 2709@xref{Interface, ,Parser C-Language Interface}.
bfa74976
RS
2710
2711If the last section is empty, you may omit the @samp{%%} that separates it
2712from the grammar rules.
2713
f8e1c9e5
AD
2714The Bison parser itself contains many macros and identifiers whose names
2715start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
2716any such names (except those documented in this manual) in the epilogue
2717of the grammar file.
bfa74976 2718
342b8b6e 2719@node Symbols
bfa74976
RS
2720@section Symbols, Terminal and Nonterminal
2721@cindex nonterminal symbol
2722@cindex terminal symbol
2723@cindex token type
2724@cindex symbol
2725
2726@dfn{Symbols} in Bison grammars represent the grammatical classifications
2727of the language.
2728
2729A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
2730class of syntactically equivalent tokens. You use the symbol in grammar
2731rules to mean that a token in that class is allowed. The symbol is
2732represented in the Bison parser by a numeric code, and the @code{yylex}
f8e1c9e5
AD
2733function returns a token type code to indicate what kind of token has
2734been read. You don't need to know what the code value is; you can use
2735the symbol to stand for it.
bfa74976 2736
f8e1c9e5
AD
2737A @dfn{nonterminal symbol} stands for a class of syntactically
2738equivalent groupings. The symbol name is used in writing grammar rules.
2739By convention, it should be all lower case.
bfa74976
RS
2740
2741Symbol names can contain letters, digits (not at the beginning),
2742underscores and periods. Periods make sense only in nonterminals.
2743
931c7513 2744There are three ways of writing terminal symbols in the grammar:
bfa74976
RS
2745
2746@itemize @bullet
2747@item
2748A @dfn{named token type} is written with an identifier, like an
c827f760 2749identifier in C@. By convention, it should be all upper case. Each
bfa74976
RS
2750such name must be defined with a Bison declaration such as
2751@code{%token}. @xref{Token Decl, ,Token Type Names}.
2752
2753@item
2754@cindex character token
2755@cindex literal token
2756@cindex single-character literal
931c7513
RS
2757A @dfn{character token type} (or @dfn{literal character token}) is
2758written in the grammar using the same syntax used in C for character
2759constants; for example, @code{'+'} is a character token type. A
2760character token type doesn't need to be declared unless you need to
2761specify its semantic value data type (@pxref{Value Type, ,Data Types of
2762Semantic Values}), associativity, or precedence (@pxref{Precedence,
2763,Operator Precedence}).
bfa74976
RS
2764
2765By convention, a character token type is used only to represent a
2766token that consists of that particular character. Thus, the token
2767type @code{'+'} is used to represent the character @samp{+} as a
2768token. Nothing enforces this convention, but if you depart from it,
2769your program will confuse other readers.
2770
2771All the usual escape sequences used in character literals in C can be
2772used in Bison as well, but you must not use the null character as a
72d2299c
PE
2773character literal because its numeric code, zero, signifies
2774end-of-input (@pxref{Calling Convention, ,Calling Convention
2bfc2e2a
PE
2775for @code{yylex}}). Also, unlike standard C, trigraphs have no
2776special meaning in Bison character literals, nor is backslash-newline
2777allowed.
931c7513
RS
2778
2779@item
2780@cindex string token
2781@cindex literal string token
9ecbd125 2782@cindex multicharacter literal
931c7513
RS
2783A @dfn{literal string token} is written like a C string constant; for
2784example, @code{"<="} is a literal string token. A literal string token
2785doesn't need to be declared unless you need to specify its semantic
14ded682 2786value data type (@pxref{Value Type}), associativity, or precedence
931c7513
RS
2787(@pxref{Precedence}).
2788
2789You can associate the literal string token with a symbolic name as an
2790alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
2791Declarations}). If you don't do that, the lexical analyzer has to
2792retrieve the token number for the literal string token from the
2793@code{yytname} table (@pxref{Calling Convention}).
2794
c827f760 2795@strong{Warning}: literal string tokens do not work in Yacc.
931c7513
RS
2796
2797By convention, a literal string token is used only to represent a token
2798that consists of that particular string. Thus, you should use the token
2799type @code{"<="} to represent the string @samp{<=} as a token. Bison
9ecbd125 2800does not enforce this convention, but if you depart from it, people who
931c7513
RS
2801read your program will be confused.
2802
2803All the escape sequences used in string literals in C can be used in
92ac3705
PE
2804Bison as well, except that you must not use a null character within a
2805string literal. Also, unlike Standard C, trigraphs have no special
2bfc2e2a
PE
2806meaning in Bison string literals, nor is backslash-newline allowed. A
2807literal string token must contain two or more characters; for a token
2808containing just one character, use a character token (see above).
bfa74976
RS
2809@end itemize
2810
2811How you choose to write a terminal symbol has no effect on its
2812grammatical meaning. That depends only on where it appears in rules and
2813on when the parser function returns that symbol.
2814
72d2299c
PE
2815The value returned by @code{yylex} is always one of the terminal
2816symbols, except that a zero or negative value signifies end-of-input.
2817Whichever way you write the token type in the grammar rules, you write
2818it the same way in the definition of @code{yylex}. The numeric code
2819for a character token type is simply the positive numeric code of the
2820character, so @code{yylex} can use the identical value to generate the
2821requisite code, though you may need to convert it to @code{unsigned
2822char} to avoid sign-extension on hosts where @code{char} is signed.
2823Each named token type becomes a C macro in
bfa74976 2824the parser file, so @code{yylex} can use the name to stand for the code.
13863333 2825(This is why periods don't make sense in terminal symbols.)
bfa74976
RS
2826@xref{Calling Convention, ,Calling Convention for @code{yylex}}.
2827
2828If @code{yylex} is defined in a separate file, you need to arrange for the
2829token-type macro definitions to be available there. Use the @samp{-d}
2830option when you run Bison, so that it will write these macro definitions
2831into a separate header file @file{@var{name}.tab.h} which you can include
2832in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
2833
72d2299c 2834If you want to write a grammar that is portable to any Standard C
9d9b8b70 2835host, you must use only nonnull character tokens taken from the basic
c827f760 2836execution character set of Standard C@. This set consists of the ten
72d2299c
PE
2837digits, the 52 lower- and upper-case English letters, and the
2838characters in the following C-language string:
2839
2840@example
2841"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
2842@end example
2843
f8e1c9e5
AD
2844The @code{yylex} function and Bison must use a consistent character set
2845and encoding for character tokens. For example, if you run Bison in an
2846@acronym{ASCII} environment, but then compile and run the resulting
2847program in an environment that uses an incompatible character set like
2848@acronym{EBCDIC}, the resulting program may not work because the tables
2849generated by Bison will assume @acronym{ASCII} numeric values for
2850character tokens. It is standard practice for software distributions to
2851contain C source files that were generated by Bison in an
2852@acronym{ASCII} environment, so installers on platforms that are
2853incompatible with @acronym{ASCII} must rebuild those files before
2854compiling them.
e966383b 2855
bfa74976
RS
2856The symbol @code{error} is a terminal symbol reserved for error recovery
2857(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
23c5a174
AD
2858In particular, @code{yylex} should never return this value. The default
2859value of the error token is 256, unless you explicitly assigned 256 to
2860one of your tokens with a @code{%token} declaration.
bfa74976 2861
342b8b6e 2862@node Rules
bfa74976
RS
2863@section Syntax of Grammar Rules
2864@cindex rule syntax
2865@cindex grammar rule syntax
2866@cindex syntax of grammar rules
2867
2868A Bison grammar rule has the following general form:
2869
2870@example
e425e872 2871@group
bfa74976
RS
2872@var{result}: @var{components}@dots{}
2873 ;
e425e872 2874@end group
bfa74976
RS
2875@end example
2876
2877@noindent
9ecbd125 2878where @var{result} is the nonterminal symbol that this rule describes,
bfa74976 2879and @var{components} are various terminal and nonterminal symbols that
13863333 2880are put together by this rule (@pxref{Symbols}).
bfa74976
RS
2881
2882For example,
2883
2884@example
2885@group
2886exp: exp '+' exp
2887 ;
2888@end group
2889@end example
2890
2891@noindent
2892says that two groupings of type @code{exp}, with a @samp{+} token in between,
2893can be combined into a larger grouping of type @code{exp}.
2894
72d2299c
PE
2895White space in rules is significant only to separate symbols. You can add
2896extra white space as you wish.
bfa74976
RS
2897
2898Scattered among the components can be @var{actions} that determine
2899the semantics of the rule. An action looks like this:
2900
2901@example
2902@{@var{C statements}@}
2903@end example
2904
2905@noindent
287c78f6
PE
2906@cindex braced code
2907This is an example of @dfn{braced code}, that is, C code surrounded by
2908braces, much like a compound statement in C@. Braced code can contain
2909any sequence of C tokens, so long as its braces are balanced. Bison
2910does not check the braced code for correctness directly; it merely
2911copies the code to the output file, where the C compiler can check it.
2912
2913Within braced code, the balanced-brace count is not affected by braces
2914within comments, string literals, or character constants, but it is
2915affected by the C digraphs @samp{<%} and @samp{%>} that represent
2916braces. At the top level braced code must be terminated by @samp{@}}
2917and not by a digraph. Bison does not look for trigraphs, so if braced
2918code uses trigraphs you should ensure that they do not affect the
2919nesting of braces or the boundaries of comments, string literals, or
2920character constants.
2921
bfa74976
RS
2922Usually there is only one action and it follows the components.
2923@xref{Actions}.
2924
2925@findex |
2926Multiple rules for the same @var{result} can be written separately or can
2927be joined with the vertical-bar character @samp{|} as follows:
2928
2929@ifinfo
2930@example
2931@var{result}: @var{rule1-components}@dots{}
2932 | @var{rule2-components}@dots{}
2933 @dots{}
2934 ;
2935@end example
2936@end ifinfo
2937@iftex
2938@example
2939@group
2940@var{result}: @var{rule1-components}@dots{}
2941 | @var{rule2-components}@dots{}
2942 @dots{}
2943 ;
2944@end group
2945@end example
2946@end iftex
2947
2948@noindent
2949They are still considered distinct rules even when joined in this way.
2950
2951If @var{components} in a rule is empty, it means that @var{result} can
2952match the empty string. For example, here is how to define a
2953comma-separated sequence of zero or more @code{exp} groupings:
2954
2955@example
2956@group
2957expseq: /* empty */
2958 | expseq1
2959 ;
2960@end group
2961
2962@group
2963expseq1: exp
2964 | expseq1 ',' exp
2965 ;
2966@end group
2967@end example
2968
2969@noindent
2970It is customary to write a comment @samp{/* empty */} in each rule
2971with no components.
2972
342b8b6e 2973@node Recursion
bfa74976
RS
2974@section Recursive Rules
2975@cindex recursive rule
2976
f8e1c9e5
AD
2977A rule is called @dfn{recursive} when its @var{result} nonterminal
2978appears also on its right hand side. Nearly all Bison grammars need to
2979use recursion, because that is the only way to define a sequence of any
2980number of a particular thing. Consider this recursive definition of a
9ecbd125 2981comma-separated sequence of one or more expressions:
bfa74976
RS
2982
2983@example
2984@group
2985expseq1: exp
2986 | expseq1 ',' exp
2987 ;
2988@end group
2989@end example
2990
2991@cindex left recursion
2992@cindex right recursion
2993@noindent
2994Since the recursive use of @code{expseq1} is the leftmost symbol in the
2995right hand side, we call this @dfn{left recursion}. By contrast, here
2996the same construct is defined using @dfn{right recursion}:
2997
2998@example
2999@group
3000expseq1: exp
3001 | exp ',' expseq1
3002 ;
3003@end group
3004@end example
3005
3006@noindent
ec3bc396
AD
3007Any kind of sequence can be defined using either left recursion or right
3008recursion, but you should always use left recursion, because it can
3009parse a sequence of any number of elements with bounded stack space.
3010Right recursion uses up space on the Bison stack in proportion to the
3011number of elements in the sequence, because all the elements must be
3012shifted onto the stack before the rule can be applied even once.
3013@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3014of this.
bfa74976
RS
3015
3016@cindex mutual recursion
3017@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3018rule does not appear directly on its right hand side, but does appear
3019in rules for other nonterminals which do appear on its right hand
13863333 3020side.
bfa74976
RS
3021
3022For example:
3023
3024@example
3025@group
3026expr: primary
3027 | primary '+' primary
3028 ;
3029@end group
3030
3031@group
3032primary: constant
3033 | '(' expr ')'
3034 ;
3035@end group
3036@end example
3037
3038@noindent
3039defines two mutually-recursive nonterminals, since each refers to the
3040other.
3041
342b8b6e 3042@node Semantics
bfa74976
RS
3043@section Defining Language Semantics
3044@cindex defining language semantics
13863333 3045@cindex language semantics, defining
bfa74976
RS
3046
3047The grammar rules for a language determine only the syntax. The semantics
3048are determined by the semantic values associated with various tokens and
3049groupings, and by the actions taken when various groupings are recognized.
3050
3051For example, the calculator calculates properly because the value
3052associated with each expression is the proper number; it adds properly
3053because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3054the numbers associated with @var{x} and @var{y}.
3055
3056@menu
3057* Value Type:: Specifying one data type for all semantic values.
3058* Multiple Types:: Specifying several alternative data types.
3059* Actions:: An action is the semantic definition of a grammar rule.
3060* Action Types:: Specifying data types for actions to operate on.
3061* Mid-Rule Actions:: Most actions go at the end of a rule.
3062 This says when, why and how to use the exceptional
3063 action in the middle of a rule.
3064@end menu
3065
342b8b6e 3066@node Value Type
bfa74976
RS
3067@subsection Data Types of Semantic Values
3068@cindex semantic value type
3069@cindex value type, semantic
3070@cindex data types of semantic values
3071@cindex default data type
3072
3073In a simple program it may be sufficient to use the same data type for
3074the semantic values of all language constructs. This was true in the
c827f760 3075@acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
1964ad8c 3076Notation Calculator}).
bfa74976
RS
3077
3078Bison's default is to use type @code{int} for all semantic values. To
3079specify some other type, define @code{YYSTYPE} as a macro, like this:
3080
3081@example
3082#define YYSTYPE double
3083@end example
3084
3085@noindent
342b8b6e 3086This macro definition must go in the prologue of the grammar file
75f5aaea 3087(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
bfa74976 3088
342b8b6e 3089@node Multiple Types
bfa74976
RS
3090@subsection More Than One Value Type
3091
3092In most programs, you will need different data types for different kinds
3093of tokens and groupings. For example, a numeric constant may need type
f8e1c9e5
AD
3094@code{int} or @code{long int}, while a string constant needs type
3095@code{char *}, and an identifier might need a pointer to an entry in the
3096symbol table.
bfa74976
RS
3097
3098To use more than one data type for semantic values in one parser, Bison
3099requires you to do two things:
3100
3101@itemize @bullet
3102@item
3103Specify the entire collection of possible data types, with the
704a47c4
AD
3104@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
3105Value Types}).
bfa74976
RS
3106
3107@item
14ded682
AD
3108Choose one of those types for each symbol (terminal or nonterminal) for
3109which semantic values are used. This is done for tokens with the
3110@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3111and for groupings with the @code{%type} Bison declaration (@pxref{Type
3112Decl, ,Nonterminal Symbols}).
bfa74976
RS
3113@end itemize
3114
342b8b6e 3115@node Actions
bfa74976
RS
3116@subsection Actions
3117@cindex action
3118@vindex $$
3119@vindex $@var{n}
3120
3121An action accompanies a syntactic rule and contains C code to be executed
3122each time an instance of that rule is recognized. The task of most actions
3123is to compute a semantic value for the grouping built by the rule from the
3124semantic values associated with tokens or smaller groupings.
3125
287c78f6
PE
3126An action consists of braced code containing C statements, and can be
3127placed at any position in the rule;
704a47c4
AD
3128it is executed at that position. Most rules have just one action at the
3129end of the rule, following all the components. Actions in the middle of
3130a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3131Actions, ,Actions in Mid-Rule}).
bfa74976
RS
3132
3133The C code in an action can refer to the semantic values of the components
3134matched by the rule with the construct @code{$@var{n}}, which stands for
3135the value of the @var{n}th component. The semantic value for the grouping
0cc3da3a
PE
3136being constructed is @code{$$}. Bison translates both of these
3137constructs into expressions of the appropriate type when it copies the
3138actions into the parser file. @code{$$} is translated to a modifiable
3139lvalue, so it can be assigned to.
bfa74976
RS
3140
3141Here is a typical example:
3142
3143@example
3144@group
3145exp: @dots{}
3146 | exp '+' exp
3147 @{ $$ = $1 + $3; @}
3148@end group
3149@end example
3150
3151@noindent
3152This rule constructs an @code{exp} from two smaller @code{exp} groupings
3153connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3154refer to the semantic values of the two component @code{exp} groupings,
3155which are the first and third symbols on the right hand side of the rule.
3156The sum is stored into @code{$$} so that it becomes the semantic value of
3157the addition-expression just recognized by the rule. If there were a
3158useful semantic value associated with the @samp{+} token, it could be
e0c471a9 3159referred to as @code{$2}.
bfa74976 3160
3ded9a63
AD
3161Note that the vertical-bar character @samp{|} is really a rule
3162separator, and actions are attached to a single rule. This is a
3163difference with tools like Flex, for which @samp{|} stands for either
3164``or'', or ``the same action as that of the next rule''. In the
3165following example, the action is triggered only when @samp{b} is found:
3166
3167@example
3168@group
3169a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3170@end group
3171@end example
3172
bfa74976
RS
3173@cindex default action
3174If you don't specify an action for a rule, Bison supplies a default:
72f889cc
AD
3175@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3176becomes the value of the whole rule. Of course, the default action is
3177valid only if the two data types match. There is no meaningful default
3178action for an empty rule; every empty rule must have an explicit action
3179unless the rule's value does not matter.
bfa74976
RS
3180
3181@code{$@var{n}} with @var{n} zero or negative is allowed for reference
3182to tokens and groupings on the stack @emph{before} those that match the
3183current rule. This is a very risky practice, and to use it reliably
3184you must be certain of the context in which the rule is applied. Here
3185is a case in which you can use this reliably:
3186
3187@example
3188@group
3189foo: expr bar '+' expr @{ @dots{} @}
3190 | expr bar '-' expr @{ @dots{} @}
3191 ;
3192@end group
3193
3194@group
3195bar: /* empty */
3196 @{ previous_expr = $0; @}
3197 ;
3198@end group
3199@end example
3200
3201As long as @code{bar} is used only in the fashion shown here, @code{$0}
3202always refers to the @code{expr} which precedes @code{bar} in the
3203definition of @code{foo}.
3204
32c29292
JD
3205@vindex yylval
3206It is also possible to access the semantic value of the look-ahead token, if
3207any, from a semantic action.
3208This semantic value is stored in @code{yylval}.
3209@xref{Action Features, ,Special Features for Use in Actions}.
3210
342b8b6e 3211@node Action Types
bfa74976
RS
3212@subsection Data Types of Values in Actions
3213@cindex action data types
3214@cindex data types in actions
3215
3216If you have chosen a single data type for semantic values, the @code{$$}
3217and @code{$@var{n}} constructs always have that data type.
3218
3219If you have used @code{%union} to specify a variety of data types, then you
3220must declare a choice among these types for each terminal or nonterminal
3221symbol that can have a semantic value. Then each time you use @code{$$} or
3222@code{$@var{n}}, its data type is determined by which symbol it refers to
e0c471a9 3223in the rule. In this example,
bfa74976
RS
3224
3225@example
3226@group
3227exp: @dots{}
3228 | exp '+' exp
3229 @{ $$ = $1 + $3; @}
3230@end group
3231@end example
3232
3233@noindent
3234@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3235have the data type declared for the nonterminal symbol @code{exp}. If
3236@code{$2} were used, it would have the data type declared for the
e0c471a9 3237terminal symbol @code{'+'}, whatever that might be.
bfa74976
RS
3238
3239Alternatively, you can specify the data type when you refer to the value,
3240by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3241reference. For example, if you have defined types as shown here:
3242
3243@example
3244@group
3245%union @{
3246 int itype;
3247 double dtype;
3248@}
3249@end group
3250@end example
3251
3252@noindent
3253then you can write @code{$<itype>1} to refer to the first subunit of the
3254rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3255
342b8b6e 3256@node Mid-Rule Actions
bfa74976
RS
3257@subsection Actions in Mid-Rule
3258@cindex actions in mid-rule
3259@cindex mid-rule actions
3260
3261Occasionally it is useful to put an action in the middle of a rule.
3262These actions are written just like usual end-of-rule actions, but they
3263are executed before the parser even recognizes the following components.
3264
3265A mid-rule action may refer to the components preceding it using
3266@code{$@var{n}}, but it may not refer to subsequent components because
3267it is run before they are parsed.
3268
3269The mid-rule action itself counts as one of the components of the rule.
3270This makes a difference when there is another action later in the same rule
3271(and usually there is another at the end): you have to count the actions
3272along with the symbols when working out which number @var{n} to use in
3273@code{$@var{n}}.
3274
3275The mid-rule action can also have a semantic value. The action can set
3276its value with an assignment to @code{$$}, and actions later in the rule
3277can refer to the value using @code{$@var{n}}. Since there is no symbol
3278to name the action, there is no way to declare a data type for the value
fdc6758b
MA
3279in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3280specify a data type each time you refer to this value.
bfa74976
RS
3281
3282There is no way to set the value of the entire rule with a mid-rule
3283action, because assignments to @code{$$} do not have that effect. The
3284only way to set the value for the entire rule is with an ordinary action
3285at the end of the rule.
3286
3287Here is an example from a hypothetical compiler, handling a @code{let}
3288statement that looks like @samp{let (@var{variable}) @var{statement}} and
3289serves to create a variable named @var{variable} temporarily for the
3290duration of @var{statement}. To parse this construct, we must put
3291@var{variable} into the symbol table while @var{statement} is parsed, then
3292remove it afterward. Here is how it is done:
3293
3294@example
3295@group
3296stmt: LET '(' var ')'
3297 @{ $<context>$ = push_context ();
3298 declare_variable ($3); @}
3299 stmt @{ $$ = $6;
3300 pop_context ($<context>5); @}
3301@end group
3302@end example
3303
3304@noindent
3305As soon as @samp{let (@var{variable})} has been recognized, the first
3306action is run. It saves a copy of the current semantic context (the
3307list of accessible variables) as its semantic value, using alternative
3308@code{context} in the data-type union. Then it calls
3309@code{declare_variable} to add the new variable to that list. Once the
3310first action is finished, the embedded statement @code{stmt} can be
3311parsed. Note that the mid-rule action is component number 5, so the
3312@samp{stmt} is component number 6.
3313
3314After the embedded statement is parsed, its semantic value becomes the
3315value of the entire @code{let}-statement. Then the semantic value from the
3316earlier action is used to restore the prior list of variables. This
3317removes the temporary @code{let}-variable from the list so that it won't
3318appear to exist while the rest of the program is parsed.
3319
841a7737
JD
3320@findex %destructor
3321@cindex discarded symbols, mid-rule actions
3322@cindex error recovery, mid-rule actions
3323In the above example, if the parser initiates error recovery (@pxref{Error
3324Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3325it might discard the previous semantic context @code{$<context>5} without
3326restoring it.
3327Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3328Discarded Symbols}).
3329However, Bison currently provides no means to declare a destructor for a
3330mid-rule action's semantic value.
3331
3332One solution is to bury the mid-rule action inside a nonterminal symbol and to
3333declare a destructor for that symbol:
3334
3335@example
3336@group
3337%type <context> let
3338%destructor @{ pop_context ($$); @} let
3339
3340%%
3341
3342stmt: let stmt
3343 @{ $$ = $2;
3344 pop_context ($1); @}
3345 ;
3346
3347let: LET '(' var ')'
3348 @{ $$ = push_context ();
3349 declare_variable ($3); @}
3350 ;
3351
3352@end group
3353@end example
3354
3355@noindent
3356Note that the action is now at the end of its rule.
3357Any mid-rule action can be converted to an end-of-rule action in this way, and
3358this is what Bison actually does to implement mid-rule actions.
3359
bfa74976
RS
3360Taking action before a rule is completely recognized often leads to
3361conflicts since the parser must commit to a parse in order to execute the
3362action. For example, the following two rules, without mid-rule actions,
3363can coexist in a working parser because the parser can shift the open-brace
3364token and look at what follows before deciding whether there is a
3365declaration or not:
3366
3367@example
3368@group
3369compound: '@{' declarations statements '@}'
3370 | '@{' statements '@}'
3371 ;
3372@end group
3373@end example
3374
3375@noindent
3376But when we add a mid-rule action as follows, the rules become nonfunctional:
3377
3378@example
3379@group
3380compound: @{ prepare_for_local_variables (); @}
3381 '@{' declarations statements '@}'
3382@end group
3383@group
3384 | '@{' statements '@}'
3385 ;
3386@end group
3387@end example
3388
3389@noindent
3390Now the parser is forced to decide whether to run the mid-rule action
3391when it has read no farther than the open-brace. In other words, it
3392must commit to using one rule or the other, without sufficient
3393information to do it correctly. (The open-brace token is what is called
3394the @dfn{look-ahead} token at this time, since the parser is still
3395deciding what to do about it. @xref{Look-Ahead, ,Look-Ahead Tokens}.)
3396
3397You might think that you could correct the problem by putting identical
3398actions into the two rules, like this:
3399
3400@example
3401@group
3402compound: @{ prepare_for_local_variables (); @}
3403 '@{' declarations statements '@}'
3404 | @{ prepare_for_local_variables (); @}
3405 '@{' statements '@}'
3406 ;
3407@end group
3408@end example
3409
3410@noindent
3411But this does not help, because Bison does not realize that the two actions
3412are identical. (Bison never tries to understand the C code in an action.)
3413
3414If the grammar is such that a declaration can be distinguished from a
3415statement by the first token (which is true in C), then one solution which
3416does work is to put the action after the open-brace, like this:
3417
3418@example
3419@group
3420compound: '@{' @{ prepare_for_local_variables (); @}
3421 declarations statements '@}'
3422 | '@{' statements '@}'
3423 ;
3424@end group
3425@end example
3426
3427@noindent
3428Now the first token of the following declaration or statement,
3429which would in any case tell Bison which rule to use, can still do so.
3430
3431Another solution is to bury the action inside a nonterminal symbol which
3432serves as a subroutine:
3433
3434@example
3435@group
3436subroutine: /* empty */
3437 @{ prepare_for_local_variables (); @}
3438 ;
3439
3440@end group
3441
3442@group
3443compound: subroutine
3444 '@{' declarations statements '@}'
3445 | subroutine
3446 '@{' statements '@}'
3447 ;
3448@end group
3449@end example
3450
3451@noindent
3452Now Bison can execute the action in the rule for @code{subroutine} without
841a7737 3453deciding which rule for @code{compound} it will eventually use.
bfa74976 3454
342b8b6e 3455@node Locations
847bf1f5
AD
3456@section Tracking Locations
3457@cindex location
95923bd6
AD
3458@cindex textual location
3459@cindex location, textual
847bf1f5
AD
3460
3461Though grammar rules and semantic actions are enough to write a fully
72d2299c 3462functional parser, it can be useful to process some additional information,
3e259915
MA
3463especially symbol locations.
3464
704a47c4
AD
3465The way locations are handled is defined by providing a data type, and
3466actions to take when rules are matched.
847bf1f5
AD
3467
3468@menu
3469* Location Type:: Specifying a data type for locations.
3470* Actions and Locations:: Using locations in actions.
3471* Location Default Action:: Defining a general way to compute locations.
3472@end menu
3473
342b8b6e 3474@node Location Type
847bf1f5
AD
3475@subsection Data Type of Locations
3476@cindex data type of locations
3477@cindex default location type
3478
3479Defining a data type for locations is much simpler than for semantic values,
3480since all tokens and groupings always use the same type.
3481
3482The type of locations is specified by defining a macro called @code{YYLTYPE}.
3483When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3484four members:
3485
3486@example
6273355b 3487typedef struct YYLTYPE
847bf1f5
AD
3488@{
3489 int first_line;
3490 int first_column;
3491 int last_line;
3492 int last_column;
6273355b 3493@} YYLTYPE;
847bf1f5
AD
3494@end example
3495
342b8b6e 3496@node Actions and Locations
847bf1f5
AD
3497@subsection Actions and Locations
3498@cindex location actions
3499@cindex actions, location
3500@vindex @@$
3501@vindex @@@var{n}
3502
3503Actions are not only useful for defining language semantics, but also for
3504describing the behavior of the output parser with locations.
3505
3506The most obvious way for building locations of syntactic groupings is very
72d2299c 3507similar to the way semantic values are computed. In a given rule, several
847bf1f5
AD
3508constructs can be used to access the locations of the elements being matched.
3509The location of the @var{n}th component of the right hand side is
3510@code{@@@var{n}}, while the location of the left hand side grouping is
3511@code{@@$}.
3512
3e259915 3513Here is a basic example using the default data type for locations:
847bf1f5
AD
3514
3515@example
3516@group
3517exp: @dots{}
3e259915 3518 | exp '/' exp
847bf1f5 3519 @{
3e259915
MA
3520 @@$.first_column = @@1.first_column;
3521 @@$.first_line = @@1.first_line;
847bf1f5
AD
3522 @@$.last_column = @@3.last_column;
3523 @@$.last_line = @@3.last_line;
3e259915
MA
3524 if ($3)
3525 $$ = $1 / $3;
3526 else
3527 @{
3528 $$ = 1;
4e03e201
AD
3529 fprintf (stderr,
3530 "Division by zero, l%d,c%d-l%d,c%d",
3531 @@3.first_line, @@3.first_column,
3532 @@3.last_line, @@3.last_column);
3e259915 3533 @}
847bf1f5
AD
3534 @}
3535@end group
3536@end example
3537
3e259915 3538As for semantic values, there is a default action for locations that is
72d2299c 3539run each time a rule is matched. It sets the beginning of @code{@@$} to the
3e259915 3540beginning of the first symbol, and the end of @code{@@$} to the end of the
79282c6c 3541last symbol.
3e259915 3542
72d2299c 3543With this default action, the location tracking can be fully automatic. The
3e259915
MA
3544example above simply rewrites this way:
3545
3546@example
3547@group
3548exp: @dots{}
3549 | exp '/' exp
3550 @{
3551 if ($3)
3552 $$ = $1 / $3;
3553 else
3554 @{
3555 $$ = 1;
4e03e201
AD
3556 fprintf (stderr,
3557 "Division by zero, l%d,c%d-l%d,c%d",
3558 @@3.first_line, @@3.first_column,
3559 @@3.last_line, @@3.last_column);
3e259915
MA
3560 @}
3561 @}
3562@end group
3563@end example
847bf1f5 3564
32c29292
JD
3565@vindex yylloc
3566It is also possible to access the location of the look-ahead token, if any,
3567from a semantic action.
3568This location is stored in @code{yylloc}.
3569@xref{Action Features, ,Special Features for Use in Actions}.
3570
342b8b6e 3571@node Location Default Action
847bf1f5
AD
3572@subsection Default Action for Locations
3573@vindex YYLLOC_DEFAULT
3574
72d2299c 3575Actually, actions are not the best place to compute locations. Since
704a47c4
AD
3576locations are much more general than semantic values, there is room in
3577the output parser to redefine the default action to take for each
72d2299c 3578rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
96b93a3d
PE
3579matched, before the associated action is run. It is also invoked
3580while processing a syntax error, to compute the error's location.
847bf1f5 3581
3e259915 3582Most of the time, this macro is general enough to suppress location
79282c6c 3583dedicated code from semantic actions.
847bf1f5 3584
72d2299c 3585The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
96b93a3d 3586the location of the grouping (the result of the computation). When a
766de5eb 3587rule is matched, the second parameter identifies locations of
96b93a3d
PE
3588all right hand side elements of the rule being matched, and the third
3589parameter is the size of the rule's right hand side. When processing
766de5eb 3590a syntax error, the second parameter identifies locations of
96b93a3d
PE
3591the symbols that were discarded during error processing, and the third
3592parameter is the number of discarded symbols.
847bf1f5 3593
766de5eb 3594By default, @code{YYLLOC_DEFAULT} is defined this way:
847bf1f5 3595
766de5eb 3596@smallexample
847bf1f5 3597@group
766de5eb
PE
3598# define YYLLOC_DEFAULT(Current, Rhs, N) \
3599 do \
3600 if (N) \
3601 @{ \
3602 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
3603 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
3604 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
3605 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
3606 @} \
3607 else \
3608 @{ \
3609 (Current).first_line = (Current).last_line = \
3610 YYRHSLOC(Rhs, 0).last_line; \
3611 (Current).first_column = (Current).last_column = \
3612 YYRHSLOC(Rhs, 0).last_column; \
3613 @} \
3614 while (0)
847bf1f5 3615@end group
766de5eb 3616@end smallexample
676385e2 3617
766de5eb
PE
3618where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
3619in @var{rhs} when @var{k} is positive, and the location of the symbol
f28ac696 3620just before the reduction when @var{k} and @var{n} are both zero.
676385e2 3621
3e259915 3622When defining @code{YYLLOC_DEFAULT}, you should consider that:
847bf1f5 3623
3e259915 3624@itemize @bullet
79282c6c 3625@item
72d2299c 3626All arguments are free of side-effects. However, only the first one (the
3e259915 3627result) should be modified by @code{YYLLOC_DEFAULT}.
847bf1f5 3628
3e259915 3629@item
766de5eb
PE
3630For consistency with semantic actions, valid indexes within the
3631right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
3632valid index, and it refers to the symbol just before the reduction.
3633During error processing @var{n} is always positive.
0ae99356
PE
3634
3635@item
3636Your macro should parenthesize its arguments, if need be, since the
3637actual arguments may not be surrounded by parentheses. Also, your
3638macro should expand to something that can be used as a single
3639statement when it is followed by a semicolon.
3e259915 3640@end itemize
847bf1f5 3641
342b8b6e 3642@node Declarations
bfa74976
RS
3643@section Bison Declarations
3644@cindex declarations, Bison
3645@cindex Bison declarations
3646
3647The @dfn{Bison declarations} section of a Bison grammar defines the symbols
3648used in formulating the grammar and the data types of semantic values.
3649@xref{Symbols}.
3650
3651All token type names (but not single-character literal tokens such as
3652@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
3653declared if you need to specify which data type to use for the semantic
3654value (@pxref{Multiple Types, ,More Than One Value Type}).
3655
3656The first rule in the file also specifies the start symbol, by default.
3657If you want some other symbol to be the start symbol, you must declare
704a47c4
AD
3658it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
3659Grammars}).
bfa74976
RS
3660
3661@menu
b50d2359 3662* Require Decl:: Requiring a Bison version.
bfa74976
RS
3663* Token Decl:: Declaring terminal symbols.
3664* Precedence Decl:: Declaring terminals with precedence and associativity.
3665* Union Decl:: Declaring the set of all semantic value types.
3666* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 3667* Initial Action Decl:: Code run before parsing starts.
72f889cc 3668* Destructor Decl:: Declaring how symbols are freed.
d6328241 3669* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
3670* Start Decl:: Specifying the start symbol.
3671* Pure Decl:: Requesting a reentrant parser.
3672* Decl Summary:: Table of all Bison declarations.
3673@end menu
3674
b50d2359
AD
3675@node Require Decl
3676@subsection Require a Version of Bison
3677@cindex version requirement
3678@cindex requiring a version of Bison
3679@findex %require
3680
3681You may require the minimum version of Bison to process the grammar. If
9b8a5ce0
AD
3682the requirement is not met, @command{bison} exits with an error (exit
3683status 63).
b50d2359
AD
3684
3685@example
3686%require "@var{version}"
3687@end example
3688
342b8b6e 3689@node Token Decl
bfa74976
RS
3690@subsection Token Type Names
3691@cindex declaring token type names
3692@cindex token type names, declaring
931c7513 3693@cindex declaring literal string tokens
bfa74976
RS
3694@findex %token
3695
3696The basic way to declare a token type name (terminal symbol) is as follows:
3697
3698@example
3699%token @var{name}
3700@end example
3701
3702Bison will convert this into a @code{#define} directive in
3703the parser, so that the function @code{yylex} (if it is in this file)
3704can use the name @var{name} to stand for this token type's code.
3705
14ded682
AD
3706Alternatively, you can use @code{%left}, @code{%right}, or
3707@code{%nonassoc} instead of @code{%token}, if you wish to specify
3708associativity and precedence. @xref{Precedence Decl, ,Operator
3709Precedence}.
bfa74976
RS
3710
3711You can explicitly specify the numeric code for a token type by appending
1452af69
PE
3712a decimal or hexadecimal integer value in the field immediately
3713following the token name:
bfa74976
RS
3714
3715@example
3716%token NUM 300
1452af69 3717%token XNUM 0x12d // a GNU extension
bfa74976
RS
3718@end example
3719
3720@noindent
3721It is generally best, however, to let Bison choose the numeric codes for
3722all token types. Bison will automatically select codes that don't conflict
e966383b 3723with each other or with normal characters.
bfa74976
RS
3724
3725In the event that the stack type is a union, you must augment the
3726@code{%token} or other token declaration to include the data type
704a47c4
AD
3727alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
3728Than One Value Type}).
bfa74976
RS
3729
3730For example:
3731
3732@example
3733@group
3734%union @{ /* define stack type */
3735 double val;
3736 symrec *tptr;
3737@}
3738%token <val> NUM /* define token NUM and its type */
3739@end group
3740@end example
3741
931c7513
RS
3742You can associate a literal string token with a token type name by
3743writing the literal string at the end of a @code{%token}
3744declaration which declares the name. For example:
3745
3746@example
3747%token arrow "=>"
3748@end example
3749
3750@noindent
3751For example, a grammar for the C language might specify these names with
3752equivalent literal string tokens:
3753
3754@example
3755%token <operator> OR "||"
3756%token <operator> LE 134 "<="
3757%left OR "<="
3758@end example
3759
3760@noindent
3761Once you equate the literal string and the token name, you can use them
3762interchangeably in further declarations or the grammar rules. The
3763@code{yylex} function can use the token name or the literal string to
3764obtain the token type code number (@pxref{Calling Convention}).
3765
342b8b6e 3766@node Precedence Decl
bfa74976
RS
3767@subsection Operator Precedence
3768@cindex precedence declarations
3769@cindex declaring operator precedence
3770@cindex operator precedence, declaring
3771
3772Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to
3773declare a token and specify its precedence and associativity, all at
3774once. These are called @dfn{precedence declarations}.
704a47c4
AD
3775@xref{Precedence, ,Operator Precedence}, for general information on
3776operator precedence.
bfa74976
RS
3777
3778The syntax of a precedence declaration is the same as that of
3779@code{%token}: either
3780
3781@example
3782%left @var{symbols}@dots{}
3783@end example
3784
3785@noindent
3786or
3787
3788@example
3789%left <@var{type}> @var{symbols}@dots{}
3790@end example
3791
3792And indeed any of these declarations serves the purposes of @code{%token}.
3793But in addition, they specify the associativity and relative precedence for
3794all the @var{symbols}:
3795
3796@itemize @bullet
3797@item
3798The associativity of an operator @var{op} determines how repeated uses
3799of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
3800@var{z}} is parsed by grouping @var{x} with @var{y} first or by
3801grouping @var{y} with @var{z} first. @code{%left} specifies
3802left-associativity (grouping @var{x} with @var{y} first) and
3803@code{%right} specifies right-associativity (grouping @var{y} with
3804@var{z} first). @code{%nonassoc} specifies no associativity, which
3805means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
3806considered a syntax error.
3807
3808@item
3809The precedence of an operator determines how it nests with other operators.
3810All the tokens declared in a single precedence declaration have equal
3811precedence and nest together according to their associativity.
3812When two tokens declared in different precedence declarations associate,
3813the one declared later has the higher precedence and is grouped first.
3814@end itemize
3815
342b8b6e 3816@node Union Decl
bfa74976
RS
3817@subsection The Collection of Value Types
3818@cindex declaring value types
3819@cindex value types, declaring
3820@findex %union
3821
287c78f6
PE
3822The @code{%union} declaration specifies the entire collection of
3823possible data types for semantic values. The keyword @code{%union} is
3824followed by braced code containing the same thing that goes inside a
3825@code{union} in C@.
bfa74976
RS
3826
3827For example:
3828
3829@example
3830@group
3831%union @{
3832 double val;
3833 symrec *tptr;
3834@}
3835@end group
3836@end example
3837
3838@noindent
3839This says that the two alternative types are @code{double} and @code{symrec
3840*}. They are given names @code{val} and @code{tptr}; these names are used
3841in the @code{%token} and @code{%type} declarations to pick one of the types
3842for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
3843
6273355b
PE
3844As an extension to @acronym{POSIX}, a tag is allowed after the
3845@code{union}. For example:
3846
3847@example
3848@group
3849%union value @{
3850 double val;
3851 symrec *tptr;
3852@}
3853@end group
3854@end example
3855
d6ca7905 3856@noindent
6273355b
PE
3857specifies the union tag @code{value}, so the corresponding C type is
3858@code{union value}. If you do not specify a tag, it defaults to
3859@code{YYSTYPE}.
3860
d6ca7905
PE
3861As another extension to @acronym{POSIX}, you may specify multiple
3862@code{%union} declarations; their contents are concatenated. However,
3863only the first @code{%union} declaration can specify a tag.
3864
6273355b 3865Note that, unlike making a @code{union} declaration in C, you need not write
bfa74976
RS
3866a semicolon after the closing brace.
3867
342b8b6e 3868@node Type Decl
bfa74976
RS
3869@subsection Nonterminal Symbols
3870@cindex declaring value types, nonterminals
3871@cindex value types, nonterminals, declaring
3872@findex %type
3873
3874@noindent
3875When you use @code{%union} to specify multiple value types, you must
3876declare the value type of each nonterminal symbol for which values are
3877used. This is done with a @code{%type} declaration, like this:
3878
3879@example
3880%type <@var{type}> @var{nonterminal}@dots{}
3881@end example
3882
3883@noindent
704a47c4
AD
3884Here @var{nonterminal} is the name of a nonterminal symbol, and
3885@var{type} is the name given in the @code{%union} to the alternative
3886that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
3887can give any number of nonterminal symbols in the same @code{%type}
3888declaration, if they have the same value type. Use spaces to separate
3889the symbol names.
bfa74976 3890
931c7513
RS
3891You can also declare the value type of a terminal symbol. To do this,
3892use the same @code{<@var{type}>} construction in a declaration for the
3893terminal symbol. All kinds of token declarations allow
3894@code{<@var{type}>}.
3895
18d192f0
AD
3896@node Initial Action Decl
3897@subsection Performing Actions before Parsing
3898@findex %initial-action
3899
3900Sometimes your parser needs to perform some initializations before
3901parsing. The @code{%initial-action} directive allows for such arbitrary
3902code.
3903
3904@deffn {Directive} %initial-action @{ @var{code} @}
3905@findex %initial-action
287c78f6 3906Declare that the braced @var{code} must be invoked before parsing each time
451364ed
AD
3907@code{yyparse} is called. The @var{code} may use @code{$$} and
3908@code{@@$} --- initial value and location of the look-ahead --- and the
3909@code{%parse-param}.
18d192f0
AD
3910@end deffn
3911
451364ed
AD
3912For instance, if your locations use a file name, you may use
3913
3914@example
48b16bbc 3915%parse-param @{ char const *file_name @};
451364ed
AD
3916%initial-action
3917@{
4626a15d 3918 @@$.initialize (file_name);
451364ed
AD
3919@};
3920@end example
3921
18d192f0 3922
72f889cc
AD
3923@node Destructor Decl
3924@subsection Freeing Discarded Symbols
3925@cindex freeing discarded symbols
3926@findex %destructor
3927
a85284cf
AD
3928During error recovery (@pxref{Error Recovery}), symbols already pushed
3929on the stack and tokens coming from the rest of the file are discarded
3930until the parser falls on its feet. If the parser runs out of memory,
9d9b8b70 3931or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
a85284cf
AD
3932symbols on the stack must be discarded. Even if the parser succeeds, it
3933must discard the start symbol.
258b75ca
PE
3934
3935When discarded symbols convey heap based information, this memory is
3936lost. While this behavior can be tolerable for batch parsers, such as
4b367315
AD
3937in traditional compilers, it is unacceptable for programs like shells or
3938protocol implementations that may parse and execute indefinitely.
258b75ca 3939
a85284cf
AD
3940The @code{%destructor} directive defines code that is called when a
3941symbol is automatically discarded.
72f889cc
AD
3942
3943@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
3944@findex %destructor
287c78f6
PE
3945Invoke the braced @var{code} whenever the parser discards one of the
3946@var{symbols}.
4b367315
AD
3947Within @var{code}, @code{$$} designates the semantic value associated
3948with the discarded symbol. The additional parser parameters are also
3949available (@pxref{Parser Function, , The Parser Function
3950@code{yyparse}}).
72f889cc
AD
3951@end deffn
3952
3953For instance:
3954
3955@smallexample
3956%union
3957@{
3958 char *string;
3959@}
3960%token <string> STRING
3961%type <string> string
3962%destructor @{ free ($$); @} STRING string
3963@end smallexample
3964
3965@noindent
258b75ca 3966guarantees that when a @code{STRING} or a @code{string} is discarded,
72f889cc
AD
3967its associated memory will be freed.
3968
e757bb10
AD
3969@sp 1
3970
3971@cindex discarded symbols
3972@dfn{Discarded symbols} are the following:
3973
3974@itemize
3975@item
3976stacked symbols popped during the first phase of error recovery,
3977@item
3978incoming terminals during the second phase of error recovery,
3979@item
a85284cf 3980the current look-ahead and the entire stack (except the current
9d9b8b70 3981right-hand side symbols) when the parser returns immediately, and
258b75ca
PE
3982@item
3983the start symbol, when the parser succeeds.
e757bb10
AD
3984@end itemize
3985
9d9b8b70
PE
3986The parser can @dfn{return immediately} because of an explicit call to
3987@code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
3988exhaustion.
3989
3990Right-hand size symbols of a rule that explicitly triggers a syntax
3991error via @code{YYERROR} are not discarded automatically. As a rule
3992of thumb, destructors are invoked only when user actions cannot manage
a85284cf 3993the memory.
e757bb10 3994
342b8b6e 3995@node Expect Decl
bfa74976
RS
3996@subsection Suppressing Conflict Warnings
3997@cindex suppressing conflict warnings
3998@cindex preventing warnings about conflicts
3999@cindex warnings, preventing
4000@cindex conflicts, suppressing warnings of
4001@findex %expect
d6328241 4002@findex %expect-rr
bfa74976
RS
4003
4004Bison normally warns if there are any conflicts in the grammar
7da99ede
AD
4005(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4006have harmless shift/reduce conflicts which are resolved in a predictable
4007way and would be difficult to eliminate. It is desirable to suppress
4008the warning about these conflicts unless the number of conflicts
4009changes. You can do this with the @code{%expect} declaration.
bfa74976
RS
4010
4011The declaration looks like this:
4012
4013@example
4014%expect @var{n}
4015@end example
4016
035aa4a0
PE
4017Here @var{n} is a decimal integer. The declaration says there should
4018be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4019Bison reports an error if the number of shift/reduce conflicts differs
4020from @var{n}, or if there are any reduce/reduce conflicts.
bfa74976 4021
035aa4a0
PE
4022For normal @acronym{LALR}(1) parsers, reduce/reduce conflicts are more
4023serious, and should be eliminated entirely. Bison will always report
4024reduce/reduce conflicts for these parsers. With @acronym{GLR}
4025parsers, however, both kinds of conflicts are routine; otherwise,
4026there would be no need to use @acronym{GLR} parsing. Therefore, it is
4027also possible to specify an expected number of reduce/reduce conflicts
4028in @acronym{GLR} parsers, using the declaration:
d6328241
PH
4029
4030@example
4031%expect-rr @var{n}
4032@end example
4033
bfa74976
RS
4034In general, using @code{%expect} involves these steps:
4035
4036@itemize @bullet
4037@item
4038Compile your grammar without @code{%expect}. Use the @samp{-v} option
4039to get a verbose list of where the conflicts occur. Bison will also
4040print the number of conflicts.
4041
4042@item
4043Check each of the conflicts to make sure that Bison's default
4044resolution is what you really want. If not, rewrite the grammar and
4045go back to the beginning.
4046
4047@item
4048Add an @code{%expect} declaration, copying the number @var{n} from the
035aa4a0
PE
4049number which Bison printed. With @acronym{GLR} parsers, add an
4050@code{%expect-rr} declaration as well.
bfa74976
RS
4051@end itemize
4052
035aa4a0
PE
4053Now Bison will warn you if you introduce an unexpected conflict, but
4054will keep silent otherwise.
bfa74976 4055
342b8b6e 4056@node Start Decl
bfa74976
RS
4057@subsection The Start-Symbol
4058@cindex declaring the start symbol
4059@cindex start symbol, declaring
4060@cindex default start symbol
4061@findex %start
4062
4063Bison assumes by default that the start symbol for the grammar is the first
4064nonterminal specified in the grammar specification section. The programmer
4065may override this restriction with the @code{%start} declaration as follows:
4066
4067@example
4068%start @var{symbol}
4069@end example
4070
342b8b6e 4071@node Pure Decl
bfa74976
RS
4072@subsection A Pure (Reentrant) Parser
4073@cindex reentrant parser
4074@cindex pure parser
8c9a50be 4075@findex %pure-parser
bfa74976
RS
4076
4077A @dfn{reentrant} program is one which does not alter in the course of
4078execution; in other words, it consists entirely of @dfn{pure} (read-only)
4079code. Reentrancy is important whenever asynchronous execution is possible;
9d9b8b70
PE
4080for example, a nonreentrant program may not be safe to call from a signal
4081handler. In systems with multiple threads of control, a nonreentrant
bfa74976
RS
4082program must be called only within interlocks.
4083
70811b85 4084Normally, Bison generates a parser which is not reentrant. This is
c827f760
PE
4085suitable for most uses, and it permits compatibility with Yacc. (The
4086standard Yacc interfaces are inherently nonreentrant, because they use
70811b85
RS
4087statically allocated variables for communication with @code{yylex},
4088including @code{yylval} and @code{yylloc}.)
bfa74976 4089
70811b85 4090Alternatively, you can generate a pure, reentrant parser. The Bison
8c9a50be 4091declaration @code{%pure-parser} says that you want the parser to be
70811b85 4092reentrant. It looks like this:
bfa74976
RS
4093
4094@example
8c9a50be 4095%pure-parser
bfa74976
RS
4096@end example
4097
70811b85
RS
4098The result is that the communication variables @code{yylval} and
4099@code{yylloc} become local variables in @code{yyparse}, and a different
4100calling convention is used for the lexical analyzer function
4101@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
4102Parsers}, for the details of this. The variable @code{yynerrs} also
4103becomes local in @code{yyparse} (@pxref{Error Reporting, ,The Error
4104Reporting Function @code{yyerror}}). The convention for calling
4105@code{yyparse} itself is unchanged.
4106
4107Whether the parser is pure has nothing to do with the grammar rules.
4108You can generate either a pure parser or a nonreentrant parser from any
4109valid grammar.
bfa74976 4110
342b8b6e 4111@node Decl Summary
bfa74976
RS
4112@subsection Bison Declaration Summary
4113@cindex Bison declaration summary
4114@cindex declaration summary
4115@cindex summary, Bison declaration
4116
d8988b2f 4117Here is a summary of the declarations used to define a grammar:
bfa74976 4118
18b519c0 4119@deffn {Directive} %union
bfa74976
RS
4120Declare the collection of data types that semantic values may have
4121(@pxref{Union Decl, ,The Collection of Value Types}).
18b519c0 4122@end deffn
bfa74976 4123
18b519c0 4124@deffn {Directive} %token
bfa74976
RS
4125Declare a terminal symbol (token type name) with no precedence
4126or associativity specified (@pxref{Token Decl, ,Token Type Names}).
18b519c0 4127@end deffn
bfa74976 4128
18b519c0 4129@deffn {Directive} %right
bfa74976
RS
4130Declare a terminal symbol (token type name) that is right-associative
4131(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4132@end deffn
bfa74976 4133
18b519c0 4134@deffn {Directive} %left
bfa74976
RS
4135Declare a terminal symbol (token type name) that is left-associative
4136(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4137@end deffn
bfa74976 4138
18b519c0 4139@deffn {Directive} %nonassoc
bfa74976 4140Declare a terminal symbol (token type name) that is nonassociative
bfa74976 4141(@pxref{Precedence Decl, ,Operator Precedence}).
39a06c25
PE
4142Using it in a way that would be associative is a syntax error.
4143@end deffn
4144
91d2c560 4145@ifset defaultprec
39a06c25 4146@deffn {Directive} %default-prec
22fccf95 4147Assign a precedence to rules lacking an explicit @code{%prec} modifier
39a06c25
PE
4148(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4149@end deffn
91d2c560 4150@end ifset
bfa74976 4151
18b519c0 4152@deffn {Directive} %type
bfa74976
RS
4153Declare the type of semantic values for a nonterminal symbol
4154(@pxref{Type Decl, ,Nonterminal Symbols}).
18b519c0 4155@end deffn
bfa74976 4156
18b519c0 4157@deffn {Directive} %start
89cab50d
AD
4158Specify the grammar's start symbol (@pxref{Start Decl, ,The
4159Start-Symbol}).
18b519c0 4160@end deffn
bfa74976 4161
18b519c0 4162@deffn {Directive} %expect
bfa74976
RS
4163Declare the expected number of shift-reduce conflicts
4164(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
18b519c0
AD
4165@end deffn
4166
bfa74976 4167
d8988b2f
AD
4168@sp 1
4169@noindent
4170In order to change the behavior of @command{bison}, use the following
4171directives:
4172
18b519c0 4173@deffn {Directive} %debug
4947ebdb
PE
4174In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
4175already defined, so that the debugging facilities are compiled.
18b519c0 4176@end deffn
ec3bc396 4177@xref{Tracing, ,Tracing Your Parser}.
d8988b2f 4178
18b519c0 4179@deffn {Directive} %defines
4bfd5e4e
PE
4180Write a header file containing macro definitions for the token type
4181names defined in the grammar as well as a few other declarations.
d8988b2f 4182If the parser output file is named @file{@var{name}.c} then this file
e0c471a9 4183is named @file{@var{name}.h}.
d8988b2f 4184
4bfd5e4e 4185Unless @code{YYSTYPE} is already defined as a macro, the output header
5c9be03d 4186declares @code{YYSTYPE}. Therefore, if you are using a @code{%union}
f8e1c9e5
AD
4187(@pxref{Multiple Types, ,More Than One Value Type}) with components that
4188require other definitions, or if you have defined a @code{YYSTYPE} macro
4189(@pxref{Value Type, ,Data Types of Semantic Values}), you need to
4190arrange for these definitions to be propagated to all modules, e.g., by
4191putting them in a prerequisite header that is included both by your
4192parser and by any other module that needs @code{YYSTYPE}.
4bfd5e4e
PE
4193
4194Unless your parser is pure, the output header declares @code{yylval}
4195as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
4196Parser}.
4197
4198If you have also used locations, the output header declares
4199@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
4200@code{YYSTYPE} and @code{yylval}. @xref{Locations, ,Tracking
4201Locations}.
4202
f8e1c9e5
AD
4203This output file is normally essential if you wish to put the definition
4204of @code{yylex} in a separate source file, because @code{yylex}
4205typically needs to be able to refer to the above-mentioned declarations
4206and to the token type codes. @xref{Token Values, ,Semantic Values of
4207Tokens}.
18b519c0 4208@end deffn
d8988b2f 4209
18b519c0 4210@deffn {Directive} %destructor
258b75ca 4211Specify how the parser should reclaim the memory associated to
fa7e68c3 4212discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 4213@end deffn
72f889cc 4214
18b519c0 4215@deffn {Directive} %file-prefix="@var{prefix}"
d8988b2f
AD
4216Specify a prefix to use for all Bison output file names. The names are
4217chosen as if the input file were named @file{@var{prefix}.y}.
18b519c0 4218@end deffn
d8988b2f 4219
18b519c0 4220@deffn {Directive} %locations
89cab50d
AD
4221Generate the code processing the locations (@pxref{Action Features,
4222,Special Features for Use in Actions}). This mode is enabled as soon as
4223the grammar uses the special @samp{@@@var{n}} tokens, but if your
4224grammar does not use it, using @samp{%locations} allows for more
6e649e65 4225accurate syntax error messages.
18b519c0 4226@end deffn
89cab50d 4227
18b519c0 4228@deffn {Directive} %name-prefix="@var{prefix}"
d8988b2f
AD
4229Rename the external symbols used in the parser so that they start with
4230@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
4231is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
2a8d363a
AD
4232@code{yylval}, @code{yylloc}, @code{yychar}, @code{yydebug}, and
4233possible @code{yylloc}. For example, if you use
4234@samp{%name-prefix="c_"}, the names become @code{c_parse}, @code{c_lex},
4235and so on. @xref{Multiple Parsers, ,Multiple Parsers in the Same
4236Program}.
18b519c0 4237@end deffn
931c7513 4238
91d2c560 4239@ifset defaultprec
22fccf95
PE
4240@deffn {Directive} %no-default-prec
4241Do not assign a precedence to rules lacking an explicit @code{%prec}
4242modifier (@pxref{Contextual Precedence, ,Context-Dependent
4243Precedence}).
4244@end deffn
91d2c560 4245@end ifset
22fccf95 4246
18b519c0 4247@deffn {Directive} %no-parser
6deb4447
AD
4248Do not include any C code in the parser file; generate tables only. The
4249parser file contains just @code{#define} directives and static variable
4250declarations.
4251
4252This option also tells Bison to write the C code for the grammar actions
fa4d969f 4253into a file named @file{@var{file}.act}, in the form of a
6deb4447 4254brace-surrounded body fit for a @code{switch} statement.
18b519c0 4255@end deffn
6deb4447 4256
18b519c0 4257@deffn {Directive} %no-lines
931c7513
RS
4258Don't generate any @code{#line} preprocessor commands in the parser
4259file. Ordinarily Bison writes these commands in the parser file so that
4260the C compiler and debuggers will associate errors and object code with
4261your source file (the grammar file). This directive causes them to
4262associate errors with the parser file, treating it an independent source
4263file in its own right.
18b519c0 4264@end deffn
931c7513 4265
fa4d969f
PE
4266@deffn {Directive} %output="@var{file}"
4267Specify @var{file} for the parser file.
18b519c0 4268@end deffn
6deb4447 4269
18b519c0 4270@deffn {Directive} %pure-parser
d8988b2f
AD
4271Request a pure (reentrant) parser program (@pxref{Pure Decl, ,A Pure
4272(Reentrant) Parser}).
18b519c0 4273@end deffn
6deb4447 4274
b50d2359 4275@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
4276Require version @var{version} or higher of Bison. @xref{Require Decl, ,
4277Require a Version of Bison}.
b50d2359
AD
4278@end deffn
4279
18b519c0 4280@deffn {Directive} %token-table
931c7513
RS
4281Generate an array of token names in the parser file. The name of the
4282array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
3650b4b8 4283token whose internal Bison token code number is @var{i}. The first
f67ad422
PE
4284three elements of @code{yytname} correspond to the predefined tokens
4285@code{"$end"},
88bce5a2
AD
4286@code{"error"}, and @code{"$undefined"}; after these come the symbols
4287defined in the grammar file.
931c7513 4288
9e0876fb
PE
4289The name in the table includes all the characters needed to represent
4290the token in Bison. For single-character literals and literal
4291strings, this includes the surrounding quoting characters and any
4292escape sequences. For example, the Bison single-character literal
4293@code{'+'} corresponds to a three-character name, represented in C as
4294@code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
4295corresponds to a five-character name, represented in C as
4296@code{"\"\\\\/\""}.
931c7513 4297
8c9a50be 4298When you specify @code{%token-table}, Bison also generates macro
931c7513
RS
4299definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
4300@code{YYNRULES}, and @code{YYNSTATES}:
4301
4302@table @code
4303@item YYNTOKENS
4304The highest token number, plus one.
4305@item YYNNTS
9ecbd125 4306The number of nonterminal symbols.
931c7513
RS
4307@item YYNRULES
4308The number of grammar rules,
4309@item YYNSTATES
4310The number of parser states (@pxref{Parser States}).
4311@end table
18b519c0 4312@end deffn
d8988b2f 4313
18b519c0 4314@deffn {Directive} %verbose
d8988b2f
AD
4315Write an extra output file containing verbose descriptions of the
4316parser states and what is done for each type of look-ahead token in
72d2299c 4317that state. @xref{Understanding, , Understanding Your Parser}, for more
ec3bc396 4318information.
18b519c0 4319@end deffn
d8988b2f 4320
18b519c0 4321@deffn {Directive} %yacc
d8988b2f
AD
4322Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
4323including its naming conventions. @xref{Bison Options}, for more.
18b519c0 4324@end deffn
d8988b2f
AD
4325
4326
342b8b6e 4327@node Multiple Parsers
bfa74976
RS
4328@section Multiple Parsers in the Same Program
4329
4330Most programs that use Bison parse only one language and therefore contain
4331only one Bison parser. But what if you want to parse more than one
4332language with the same program? Then you need to avoid a name conflict
4333between different definitions of @code{yyparse}, @code{yylval}, and so on.
4334
4335The easy way to do this is to use the option @samp{-p @var{prefix}}
704a47c4
AD
4336(@pxref{Invocation, ,Invoking Bison}). This renames the interface
4337functions and variables of the Bison parser to start with @var{prefix}
4338instead of @samp{yy}. You can use this to give each parser distinct
4339names that do not conflict.
bfa74976
RS
4340
4341The precise list of symbols renamed is @code{yyparse}, @code{yylex},
2a8d363a
AD
4342@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
4343@code{yychar} and @code{yydebug}. For example, if you use @samp{-p c},
4344the names become @code{cparse}, @code{clex}, and so on.
bfa74976
RS
4345
4346@strong{All the other variables and macros associated with Bison are not
4347renamed.} These others are not global; there is no conflict if the same
4348name is used in different parsers. For example, @code{YYSTYPE} is not
4349renamed, but defining this in different ways in different parsers causes
4350no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
4351
4352The @samp{-p} option works by adding macro definitions to the beginning
4353of the parser source file, defining @code{yyparse} as
4354@code{@var{prefix}parse}, and so on. This effectively substitutes one
4355name for the other in the entire parser file.
4356
342b8b6e 4357@node Interface
bfa74976
RS
4358@chapter Parser C-Language Interface
4359@cindex C-language interface
4360@cindex interface
4361
4362The Bison parser is actually a C function named @code{yyparse}. Here we
4363describe the interface conventions of @code{yyparse} and the other
4364functions that it needs to use.
4365
4366Keep in mind that the parser uses many C identifiers starting with
4367@samp{yy} and @samp{YY} for internal purposes. If you use such an
75f5aaea
MA
4368identifier (aside from those in this manual) in an action or in epilogue
4369in the grammar file, you are likely to run into trouble.
bfa74976
RS
4370
4371@menu
4372* Parser Function:: How to call @code{yyparse} and what it returns.
13863333 4373* Lexical:: You must supply a function @code{yylex}
bfa74976
RS
4374 which reads tokens.
4375* Error Reporting:: You must supply a function @code{yyerror}.
4376* Action Features:: Special features for use in actions.
f7ab6a50
PE
4377* Internationalization:: How to let the parser speak in the user's
4378 native language.
bfa74976
RS
4379@end menu
4380
342b8b6e 4381@node Parser Function
bfa74976
RS
4382@section The Parser Function @code{yyparse}
4383@findex yyparse
4384
4385You call the function @code{yyparse} to cause parsing to occur. This
4386function reads tokens, executes actions, and ultimately returns when it
4387encounters end-of-input or an unrecoverable syntax error. You can also
14ded682
AD
4388write an action which directs @code{yyparse} to return immediately
4389without reading further.
bfa74976 4390
2a8d363a
AD
4391
4392@deftypefun int yyparse (void)
bfa74976
RS
4393The value returned by @code{yyparse} is 0 if parsing was successful (return
4394is due to end-of-input).
4395
b47dbebe
PE
4396The value is 1 if parsing failed because of invalid input, i.e., input
4397that contains a syntax error or that causes @code{YYABORT} to be
4398invoked.
4399
4400The value is 2 if parsing failed due to memory exhaustion.
2a8d363a 4401@end deftypefun
bfa74976
RS
4402
4403In an action, you can cause immediate return from @code{yyparse} by using
4404these macros:
4405
2a8d363a 4406@defmac YYACCEPT
bfa74976
RS
4407@findex YYACCEPT
4408Return immediately with value 0 (to report success).
2a8d363a 4409@end defmac
bfa74976 4410
2a8d363a 4411@defmac YYABORT
bfa74976
RS
4412@findex YYABORT
4413Return immediately with value 1 (to report failure).
2a8d363a
AD
4414@end defmac
4415
4416If you use a reentrant parser, you can optionally pass additional
4417parameter information to it in a reentrant way. To do so, use the
4418declaration @code{%parse-param}:
4419
feeb0eda 4420@deffn {Directive} %parse-param @{@var{argument-declaration}@}
2a8d363a 4421@findex %parse-param
287c78f6
PE
4422Declare that an argument declared by the braced-code
4423@var{argument-declaration} is an additional @code{yyparse} argument.
94175978 4424The @var{argument-declaration} is used when declaring
feeb0eda
PE
4425functions or prototypes. The last identifier in
4426@var{argument-declaration} must be the argument name.
2a8d363a
AD
4427@end deffn
4428
4429Here's an example. Write this in the parser:
4430
4431@example
feeb0eda
PE
4432%parse-param @{int *nastiness@}
4433%parse-param @{int *randomness@}
2a8d363a
AD
4434@end example
4435
4436@noindent
4437Then call the parser like this:
4438
4439@example
4440@{
4441 int nastiness, randomness;
4442 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
4443 value = yyparse (&nastiness, &randomness);
4444 @dots{}
4445@}
4446@end example
4447
4448@noindent
4449In the grammar actions, use expressions like this to refer to the data:
4450
4451@example
4452exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
4453@end example
4454
bfa74976 4455
342b8b6e 4456@node Lexical
bfa74976
RS
4457@section The Lexical Analyzer Function @code{yylex}
4458@findex yylex
4459@cindex lexical analyzer
4460
4461The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
4462the input stream and returns them to the parser. Bison does not create
4463this function automatically; you must write it so that @code{yyparse} can
4464call it. The function is sometimes referred to as a lexical scanner.
4465
4466In simple programs, @code{yylex} is often defined at the end of the Bison
4467grammar file. If @code{yylex} is defined in a separate source file, you
4468need to arrange for the token-type macro definitions to be available there.
4469To do this, use the @samp{-d} option when you run Bison, so that it will
4470write these macro definitions into a separate header file
4471@file{@var{name}.tab.h} which you can include in the other source files
e0c471a9 4472that need it. @xref{Invocation, ,Invoking Bison}.
bfa74976
RS
4473
4474@menu
4475* Calling Convention:: How @code{yyparse} calls @code{yylex}.
4476* Token Values:: How @code{yylex} must return the semantic value
4477 of the token it has read.
95923bd6 4478* Token Locations:: How @code{yylex} must return the text location
bfa74976
RS
4479 (line number, etc.) of the token, if the
4480 actions want that.
4481* Pure Calling:: How the calling convention differs
4482 in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
4483@end menu
4484
342b8b6e 4485@node Calling Convention
bfa74976
RS
4486@subsection Calling Convention for @code{yylex}
4487
72d2299c
PE
4488The value that @code{yylex} returns must be the positive numeric code
4489for the type of token it has just found; a zero or negative value
4490signifies end-of-input.
bfa74976
RS
4491
4492When a token is referred to in the grammar rules by a name, that name
4493in the parser file becomes a C macro whose definition is the proper
4494numeric code for that token type. So @code{yylex} can use the name
4495to indicate that type. @xref{Symbols}.
4496
4497When a token is referred to in the grammar rules by a character literal,
4498the numeric code for that character is also the code for the token type.
72d2299c
PE
4499So @code{yylex} can simply return that character code, possibly converted
4500to @code{unsigned char} to avoid sign-extension. The null character
4501must not be used this way, because its code is zero and that
bfa74976
RS
4502signifies end-of-input.
4503
4504Here is an example showing these things:
4505
4506@example
13863333
AD
4507int
4508yylex (void)
bfa74976
RS
4509@{
4510 @dots{}
72d2299c 4511 if (c == EOF) /* Detect end-of-input. */
bfa74976
RS
4512 return 0;
4513 @dots{}
4514 if (c == '+' || c == '-')
72d2299c 4515 return c; /* Assume token type for `+' is '+'. */
bfa74976 4516 @dots{}
72d2299c 4517 return INT; /* Return the type of the token. */
bfa74976
RS
4518 @dots{}
4519@}
4520@end example
4521
4522@noindent
4523This interface has been designed so that the output from the @code{lex}
4524utility can be used without change as the definition of @code{yylex}.
4525
931c7513
RS
4526If the grammar uses literal string tokens, there are two ways that
4527@code{yylex} can determine the token type codes for them:
4528
4529@itemize @bullet
4530@item
4531If the grammar defines symbolic token names as aliases for the
4532literal string tokens, @code{yylex} can use these symbolic names like
4533all others. In this case, the use of the literal string tokens in
4534the grammar file has no effect on @code{yylex}.
4535
4536@item
9ecbd125 4537@code{yylex} can find the multicharacter token in the @code{yytname}
931c7513 4538table. The index of the token in the table is the token type's code.
9ecbd125 4539The name of a multicharacter token is recorded in @code{yytname} with a
931c7513 4540double-quote, the token's characters, and another double-quote. The
9e0876fb
PE
4541token's characters are escaped as necessary to be suitable as input
4542to Bison.
931c7513 4543
9e0876fb
PE
4544Here's code for looking up a multicharacter token in @code{yytname},
4545assuming that the characters of the token are stored in
4546@code{token_buffer}, and assuming that the token does not contain any
4547characters like @samp{"} that require escaping.
931c7513
RS
4548
4549@smallexample
4550for (i = 0; i < YYNTOKENS; i++)
4551 @{
4552 if (yytname[i] != 0
4553 && yytname[i][0] == '"'
68449b3a
PE
4554 && ! strncmp (yytname[i] + 1, token_buffer,
4555 strlen (token_buffer))
931c7513
RS
4556 && yytname[i][strlen (token_buffer) + 1] == '"'
4557 && yytname[i][strlen (token_buffer) + 2] == 0)
4558 break;
4559 @}
4560@end smallexample
4561
4562The @code{yytname} table is generated only if you use the
8c9a50be 4563@code{%token-table} declaration. @xref{Decl Summary}.
931c7513
RS
4564@end itemize
4565
342b8b6e 4566@node Token Values
bfa74976
RS
4567@subsection Semantic Values of Tokens
4568
4569@vindex yylval
9d9b8b70 4570In an ordinary (nonreentrant) parser, the semantic value of the token must
bfa74976
RS
4571be stored into the global variable @code{yylval}. When you are using
4572just one data type for semantic values, @code{yylval} has that type.
4573Thus, if the type is @code{int} (the default), you might write this in
4574@code{yylex}:
4575
4576@example
4577@group
4578 @dots{}
72d2299c
PE
4579 yylval = value; /* Put value onto Bison stack. */
4580 return INT; /* Return the type of the token. */
bfa74976
RS
4581 @dots{}
4582@end group
4583@end example
4584
4585When you are using multiple data types, @code{yylval}'s type is a union
704a47c4
AD
4586made from the @code{%union} declaration (@pxref{Union Decl, ,The
4587Collection of Value Types}). So when you store a token's value, you
4588must use the proper member of the union. If the @code{%union}
4589declaration looks like this:
bfa74976
RS
4590
4591@example
4592@group
4593%union @{
4594 int intval;
4595 double val;
4596 symrec *tptr;
4597@}
4598@end group
4599@end example
4600
4601@noindent
4602then the code in @code{yylex} might look like this:
4603
4604@example
4605@group
4606 @dots{}
72d2299c
PE
4607 yylval.intval = value; /* Put value onto Bison stack. */
4608 return INT; /* Return the type of the token. */
bfa74976
RS
4609 @dots{}
4610@end group
4611@end example
4612
95923bd6
AD
4613@node Token Locations
4614@subsection Textual Locations of Tokens
bfa74976
RS
4615
4616@vindex yylloc
847bf1f5 4617If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
f8e1c9e5
AD
4618Tracking Locations}) in actions to keep track of the textual locations
4619of tokens and groupings, then you must provide this information in
4620@code{yylex}. The function @code{yyparse} expects to find the textual
4621location of a token just parsed in the global variable @code{yylloc}.
4622So @code{yylex} must store the proper data in that variable.
847bf1f5
AD
4623
4624By default, the value of @code{yylloc} is a structure and you need only
89cab50d
AD
4625initialize the members that are going to be used by the actions. The
4626four members are called @code{first_line}, @code{first_column},
4627@code{last_line} and @code{last_column}. Note that the use of this
4628feature makes the parser noticeably slower.
bfa74976
RS
4629
4630@tindex YYLTYPE
4631The data type of @code{yylloc} has the name @code{YYLTYPE}.
4632
342b8b6e 4633@node Pure Calling
c656404a 4634@subsection Calling Conventions for Pure Parsers
bfa74976 4635
8c9a50be 4636When you use the Bison declaration @code{%pure-parser} to request a
e425e872
RS
4637pure, reentrant parser, the global communication variables @code{yylval}
4638and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
4639Parser}.) In such parsers the two global variables are replaced by
4640pointers passed as arguments to @code{yylex}. You must declare them as
4641shown here, and pass the information back by storing it through those
4642pointers.
bfa74976
RS
4643
4644@example
13863333
AD
4645int
4646yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
bfa74976
RS
4647@{
4648 @dots{}
4649 *lvalp = value; /* Put value onto Bison stack. */
4650 return INT; /* Return the type of the token. */
4651 @dots{}
4652@}
4653@end example
4654
4655If the grammar file does not use the @samp{@@} constructs to refer to
95923bd6 4656textual locations, then the type @code{YYLTYPE} will not be defined. In
bfa74976
RS
4657this case, omit the second argument; @code{yylex} will be called with
4658only one argument.
4659
e425e872 4660
2a8d363a
AD
4661If you wish to pass the additional parameter data to @code{yylex}, use
4662@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
4663Function}).
e425e872 4664
feeb0eda 4665@deffn {Directive} lex-param @{@var{argument-declaration}@}
2a8d363a 4666@findex %lex-param
287c78f6
PE
4667Declare that the braced-code @var{argument-declaration} is an
4668additional @code{yylex} argument declaration.
2a8d363a 4669@end deffn
e425e872 4670
2a8d363a 4671For instance:
e425e872
RS
4672
4673@example
feeb0eda
PE
4674%parse-param @{int *nastiness@}
4675%lex-param @{int *nastiness@}
4676%parse-param @{int *randomness@}
e425e872
RS
4677@end example
4678
4679@noindent
2a8d363a 4680results in the following signature:
e425e872
RS
4681
4682@example
2a8d363a
AD
4683int yylex (int *nastiness);
4684int yyparse (int *nastiness, int *randomness);
e425e872
RS
4685@end example
4686
2a8d363a 4687If @code{%pure-parser} is added:
c656404a
RS
4688
4689@example
2a8d363a
AD
4690int yylex (YYSTYPE *lvalp, int *nastiness);
4691int yyparse (int *nastiness, int *randomness);
c656404a
RS
4692@end example
4693
2a8d363a
AD
4694@noindent
4695and finally, if both @code{%pure-parser} and @code{%locations} are used:
c656404a 4696
2a8d363a
AD
4697@example
4698int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
4699int yyparse (int *nastiness, int *randomness);
4700@end example
931c7513 4701
342b8b6e 4702@node Error Reporting
bfa74976
RS
4703@section The Error Reporting Function @code{yyerror}
4704@cindex error reporting function
4705@findex yyerror
4706@cindex parse error
4707@cindex syntax error
4708
6e649e65 4709The Bison parser detects a @dfn{syntax error} or @dfn{parse error}
9ecbd125 4710whenever it reads a token which cannot satisfy any syntax rule. An
bfa74976 4711action in the grammar can also explicitly proclaim an error, using the
ceed8467
AD
4712macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
4713in Actions}).
bfa74976
RS
4714
4715The Bison parser expects to report the error by calling an error
4716reporting function named @code{yyerror}, which you must supply. It is
4717called by @code{yyparse} whenever a syntax error is found, and it
6e649e65
PE
4718receives one argument. For a syntax error, the string is normally
4719@w{@code{"syntax error"}}.
bfa74976 4720
2a8d363a
AD
4721@findex %error-verbose
4722If you invoke the directive @code{%error-verbose} in the Bison
4723declarations section (@pxref{Bison Declarations, ,The Bison Declarations
4724Section}), then Bison provides a more verbose and specific error message
6e649e65 4725string instead of just plain @w{@code{"syntax error"}}.
bfa74976 4726
1a059451
PE
4727The parser can detect one other kind of error: memory exhaustion. This
4728can happen when the input contains constructions that are very deeply
bfa74976 4729nested. It isn't likely you will encounter this, since the Bison
1a059451
PE
4730parser normally extends its stack automatically up to a very large limit. But
4731if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
4732fashion, except that the argument string is @w{@code{"memory exhausted"}}.
4733
4734In some cases diagnostics like @w{@code{"syntax error"}} are
4735translated automatically from English to some other language before
4736they are passed to @code{yyerror}. @xref{Internationalization}.
bfa74976
RS
4737
4738The following definition suffices in simple programs:
4739
4740@example
4741@group
13863333 4742void
38a92d50 4743yyerror (char const *s)
bfa74976
RS
4744@{
4745@end group
4746@group
4747 fprintf (stderr, "%s\n", s);
4748@}
4749@end group
4750@end example
4751
4752After @code{yyerror} returns to @code{yyparse}, the latter will attempt
4753error recovery if you have written suitable error recovery grammar rules
4754(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
4755immediately return 1.
4756
93724f13 4757Obviously, in location tracking pure parsers, @code{yyerror} should have
fa7e68c3
PE
4758an access to the current location.
4759This is indeed the case for the @acronym{GLR}
2a8d363a
AD
4760parsers, but not for the Yacc parser, for historical reasons. I.e., if
4761@samp{%locations %pure-parser} is passed then the prototypes for
4762@code{yyerror} are:
4763
4764@example
38a92d50
PE
4765void yyerror (char const *msg); /* Yacc parsers. */
4766void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
2a8d363a
AD
4767@end example
4768
feeb0eda 4769If @samp{%parse-param @{int *nastiness@}} is used, then:
2a8d363a
AD
4770
4771@example
b317297e
PE
4772void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
4773void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
2a8d363a
AD
4774@end example
4775
fa7e68c3 4776Finally, @acronym{GLR} and Yacc parsers share the same @code{yyerror} calling
2a8d363a
AD
4777convention for absolutely pure parsers, i.e., when the calling
4778convention of @code{yylex} @emph{and} the calling convention of
4779@code{%pure-parser} are pure. I.e.:
4780
4781@example
4782/* Location tracking. */
4783%locations
4784/* Pure yylex. */
4785%pure-parser
feeb0eda 4786%lex-param @{int *nastiness@}
2a8d363a 4787/* Pure yyparse. */
feeb0eda
PE
4788%parse-param @{int *nastiness@}
4789%parse-param @{int *randomness@}
2a8d363a
AD
4790@end example
4791
4792@noindent
4793results in the following signatures for all the parser kinds:
4794
4795@example
4796int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
4797int yyparse (int *nastiness, int *randomness);
93724f13
AD
4798void yyerror (YYLTYPE *locp,
4799 int *nastiness, int *randomness,
38a92d50 4800 char const *msg);
2a8d363a
AD
4801@end example
4802
1c0c3e95 4803@noindent
38a92d50
PE
4804The prototypes are only indications of how the code produced by Bison
4805uses @code{yyerror}. Bison-generated code always ignores the returned
4806value, so @code{yyerror} can return any type, including @code{void}.
4807Also, @code{yyerror} can be a variadic function; that is why the
4808message is always passed last.
4809
4810Traditionally @code{yyerror} returns an @code{int} that is always
4811ignored, but this is purely for historical reasons, and @code{void} is
4812preferable since it more accurately describes the return type for
4813@code{yyerror}.
93724f13 4814
bfa74976
RS
4815@vindex yynerrs
4816The variable @code{yynerrs} contains the number of syntax errors
8a2800e7 4817reported so far. Normally this variable is global; but if you
704a47c4
AD
4818request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
4819then it is a local variable which only the actions can access.
bfa74976 4820
342b8b6e 4821@node Action Features
bfa74976
RS
4822@section Special Features for Use in Actions
4823@cindex summary, action features
4824@cindex action features summary
4825
4826Here is a table of Bison constructs, variables and macros that
4827are useful in actions.
4828
18b519c0 4829@deffn {Variable} $$
bfa74976
RS
4830Acts like a variable that contains the semantic value for the
4831grouping made by the current rule. @xref{Actions}.
18b519c0 4832@end deffn
bfa74976 4833
18b519c0 4834@deffn {Variable} $@var{n}
bfa74976
RS
4835Acts like a variable that contains the semantic value for the
4836@var{n}th component of the current rule. @xref{Actions}.
18b519c0 4837@end deffn
bfa74976 4838
18b519c0 4839@deffn {Variable} $<@var{typealt}>$
bfa74976 4840Like @code{$$} but specifies alternative @var{typealt} in the union
704a47c4
AD
4841specified by the @code{%union} declaration. @xref{Action Types, ,Data
4842Types of Values in Actions}.
18b519c0 4843@end deffn
bfa74976 4844
18b519c0 4845@deffn {Variable} $<@var{typealt}>@var{n}
bfa74976 4846Like @code{$@var{n}} but specifies alternative @var{typealt} in the
13863333 4847union specified by the @code{%union} declaration.
e0c471a9 4848@xref{Action Types, ,Data Types of Values in Actions}.
18b519c0 4849@end deffn
bfa74976 4850
18b519c0 4851@deffn {Macro} YYABORT;
bfa74976
RS
4852Return immediately from @code{yyparse}, indicating failure.
4853@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 4854@end deffn
bfa74976 4855
18b519c0 4856@deffn {Macro} YYACCEPT;
bfa74976
RS
4857Return immediately from @code{yyparse}, indicating success.
4858@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 4859@end deffn
bfa74976 4860
18b519c0 4861@deffn {Macro} YYBACKUP (@var{token}, @var{value});
bfa74976
RS
4862@findex YYBACKUP
4863Unshift a token. This macro is allowed only for rules that reduce
4864a single value, and only when there is no look-ahead token.
c827f760 4865It is also disallowed in @acronym{GLR} parsers.
bfa74976
RS
4866It installs a look-ahead token with token type @var{token} and
4867semantic value @var{value}; then it discards the value that was
4868going to be reduced by this rule.
4869
4870If the macro is used when it is not valid, such as when there is
4871a look-ahead token already, then it reports a syntax error with
4872a message @samp{cannot back up} and performs ordinary error
4873recovery.
4874
4875In either case, the rest of the action is not executed.
18b519c0 4876@end deffn
bfa74976 4877
18b519c0 4878@deffn {Macro} YYEMPTY
bfa74976
RS
4879@vindex YYEMPTY
4880Value stored in @code{yychar} when there is no look-ahead token.
18b519c0 4881@end deffn
bfa74976 4882
32c29292
JD
4883@deffn {Macro} YYEOF
4884@vindex YYEOF
4885Value stored in @code{yychar} when the look-ahead is the end of the input
4886stream.
4887@end deffn
4888
18b519c0 4889@deffn {Macro} YYERROR;
bfa74976
RS
4890@findex YYERROR
4891Cause an immediate syntax error. This statement initiates error
4892recovery just as if the parser itself had detected an error; however, it
4893does not call @code{yyerror}, and does not print any message. If you
4894want to print an error message, call @code{yyerror} explicitly before
4895the @samp{YYERROR;} statement. @xref{Error Recovery}.
18b519c0 4896@end deffn
bfa74976 4897
18b519c0 4898@deffn {Macro} YYRECOVERING
bfa74976
RS
4899This macro stands for an expression that has the value 1 when the parser
4900is recovering from a syntax error, and 0 the rest of the time.
4901@xref{Error Recovery}.
18b519c0 4902@end deffn
bfa74976 4903
18b519c0 4904@deffn {Variable} yychar
32c29292
JD
4905Variable containing either the look-ahead token, or @code{YYEOF} when the
4906look-ahead is the end of the input stream, or @code{YYEMPTY} when no look-ahead
4907has been performed so the next token is not yet known.
4908Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
4909Actions}).
bfa74976 4910@xref{Look-Ahead, ,Look-Ahead Tokens}.
18b519c0 4911@end deffn
bfa74976 4912
18b519c0 4913@deffn {Macro} yyclearin;
bfa74976 4914Discard the current look-ahead token. This is useful primarily in
32c29292
JD
4915error rules.
4916Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
4917Semantic Actions}).
4918@xref{Error Recovery}.
18b519c0 4919@end deffn
bfa74976 4920
18b519c0 4921@deffn {Macro} yyerrok;
bfa74976 4922Resume generating error messages immediately for subsequent syntax
13863333 4923errors. This is useful primarily in error rules.
bfa74976 4924@xref{Error Recovery}.
18b519c0 4925@end deffn
bfa74976 4926
32c29292
JD
4927@deffn {Variable} yylloc
4928Variable containing the look-ahead token location when @code{yychar} is not set
4929to @code{YYEMPTY} or @code{YYEOF}.
4930Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
4931Actions}).
4932@xref{Actions and Locations, ,Actions and Locations}.
4933@end deffn
4934
4935@deffn {Variable} yylval
4936Variable containing the look-ahead token semantic value when @code{yychar} is
4937not set to @code{YYEMPTY} or @code{YYEOF}.
4938Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
4939Actions}).
4940@xref{Actions, ,Actions}.
4941@end deffn
4942
18b519c0 4943@deffn {Value} @@$
847bf1f5 4944@findex @@$
95923bd6 4945Acts like a structure variable containing information on the textual location
847bf1f5
AD
4946of the grouping made by the current rule. @xref{Locations, ,
4947Tracking Locations}.
bfa74976 4948
847bf1f5
AD
4949@c Check if those paragraphs are still useful or not.
4950
4951@c @example
4952@c struct @{
4953@c int first_line, last_line;
4954@c int first_column, last_column;
4955@c @};
4956@c @end example
4957
4958@c Thus, to get the starting line number of the third component, you would
4959@c use @samp{@@3.first_line}.
bfa74976 4960
847bf1f5
AD
4961@c In order for the members of this structure to contain valid information,
4962@c you must make @code{yylex} supply this information about each token.
4963@c If you need only certain members, then @code{yylex} need only fill in
4964@c those members.
bfa74976 4965
847bf1f5 4966@c The use of this feature makes the parser noticeably slower.
18b519c0 4967@end deffn
847bf1f5 4968
18b519c0 4969@deffn {Value} @@@var{n}
847bf1f5 4970@findex @@@var{n}
95923bd6 4971Acts like a structure variable containing information on the textual location
847bf1f5
AD
4972of the @var{n}th component of the current rule. @xref{Locations, ,
4973Tracking Locations}.
18b519c0 4974@end deffn
bfa74976 4975
f7ab6a50
PE
4976@node Internationalization
4977@section Parser Internationalization
4978@cindex internationalization
4979@cindex i18n
4980@cindex NLS
4981@cindex gettext
4982@cindex bison-po
4983
4984A Bison-generated parser can print diagnostics, including error and
4985tracing messages. By default, they appear in English. However, Bison
f8e1c9e5
AD
4986also supports outputting diagnostics in the user's native language. To
4987make this work, the user should set the usual environment variables.
4988@xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
4989For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
4990set the user's locale to French Canadian using the @acronym{UTF}-8
f7ab6a50
PE
4991encoding. The exact set of available locales depends on the user's
4992installation.
4993
4994The maintainer of a package that uses a Bison-generated parser enables
4995the internationalization of the parser's output through the following
4996steps. Here we assume a package that uses @acronym{GNU} Autoconf and
4997@acronym{GNU} Automake.
4998
4999@enumerate
5000@item
30757c8c 5001@cindex bison-i18n.m4
f7ab6a50
PE
5002Into the directory containing the @acronym{GNU} Autoconf macros used
5003by the package---often called @file{m4}---copy the
5004@file{bison-i18n.m4} file installed by Bison under
5005@samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
5006For example:
5007
5008@example
5009cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
5010@end example
5011
5012@item
30757c8c
PE
5013@findex BISON_I18N
5014@vindex BISON_LOCALEDIR
5015@vindex YYENABLE_NLS
f7ab6a50
PE
5016In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
5017invocation, add an invocation of @code{BISON_I18N}. This macro is
5018defined in the file @file{bison-i18n.m4} that you copied earlier. It
5019causes @samp{configure} to find the value of the
30757c8c
PE
5020@code{BISON_LOCALEDIR} variable, and it defines the source-language
5021symbol @code{YYENABLE_NLS} to enable translations in the
5022Bison-generated parser.
f7ab6a50
PE
5023
5024@item
5025In the @code{main} function of your program, designate the directory
5026containing Bison's runtime message catalog, through a call to
5027@samp{bindtextdomain} with domain name @samp{bison-runtime}.
5028For example:
5029
5030@example
5031bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
5032@end example
5033
5034Typically this appears after any other call @code{bindtextdomain
5035(PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
5036@samp{BISON_LOCALEDIR} to be defined as a string through the
5037@file{Makefile}.
5038
5039@item
5040In the @file{Makefile.am} that controls the compilation of the @code{main}
5041function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
5042either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
5043
5044@example
5045DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5046@end example
5047
5048or:
5049
5050@example
5051AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5052@end example
5053
5054@item
5055Finally, invoke the command @command{autoreconf} to generate the build
5056infrastructure.
5057@end enumerate
5058
bfa74976 5059
342b8b6e 5060@node Algorithm
13863333
AD
5061@chapter The Bison Parser Algorithm
5062@cindex Bison parser algorithm
bfa74976
RS
5063@cindex algorithm of parser
5064@cindex shifting
5065@cindex reduction
5066@cindex parser stack
5067@cindex stack, parser
5068
5069As Bison reads tokens, it pushes them onto a stack along with their
5070semantic values. The stack is called the @dfn{parser stack}. Pushing a
5071token is traditionally called @dfn{shifting}.
5072
5073For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
5074@samp{3} to come. The stack will have four elements, one for each token
5075that was shifted.
5076
5077But the stack does not always have an element for each token read. When
5078the last @var{n} tokens and groupings shifted match the components of a
5079grammar rule, they can be combined according to that rule. This is called
5080@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
5081single grouping whose symbol is the result (left hand side) of that rule.
5082Running the rule's action is part of the process of reduction, because this
5083is what computes the semantic value of the resulting grouping.
5084
5085For example, if the infix calculator's parser stack contains this:
5086
5087@example
50881 + 5 * 3
5089@end example
5090
5091@noindent
5092and the next input token is a newline character, then the last three
5093elements can be reduced to 15 via the rule:
5094
5095@example
5096expr: expr '*' expr;
5097@end example
5098
5099@noindent
5100Then the stack contains just these three elements:
5101
5102@example
51031 + 15
5104@end example
5105
5106@noindent
5107At this point, another reduction can be made, resulting in the single value
510816. Then the newline token can be shifted.
5109
5110The parser tries, by shifts and reductions, to reduce the entire input down
5111to a single grouping whose symbol is the grammar's start-symbol
5112(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
5113
5114This kind of parser is known in the literature as a bottom-up parser.
5115
5116@menu
5117* Look-Ahead:: Parser looks one token ahead when deciding what to do.
5118* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
5119* Precedence:: Operator precedence works by resolving conflicts.
5120* Contextual Precedence:: When an operator's precedence depends on context.
5121* Parser States:: The parser is a finite-state-machine with stack.
5122* Reduce/Reduce:: When two rules are applicable in the same situation.
5123* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 5124* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 5125* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
5126@end menu
5127
342b8b6e 5128@node Look-Ahead
bfa74976
RS
5129@section Look-Ahead Tokens
5130@cindex look-ahead token
5131
5132The Bison parser does @emph{not} always reduce immediately as soon as the
5133last @var{n} tokens and groupings match a rule. This is because such a
5134simple strategy is inadequate to handle most languages. Instead, when a
5135reduction is possible, the parser sometimes ``looks ahead'' at the next
5136token in order to decide what to do.
5137
5138When a token is read, it is not immediately shifted; first it becomes the
5139@dfn{look-ahead token}, which is not on the stack. Now the parser can
5140perform one or more reductions of tokens and groupings on the stack, while
5141the look-ahead token remains off to the side. When no more reductions
5142should take place, the look-ahead token is shifted onto the stack. This
5143does not mean that all possible reductions have been done; depending on the
5144token type of the look-ahead token, some rules may choose to delay their
5145application.
5146
5147Here is a simple case where look-ahead is needed. These three rules define
5148expressions which contain binary addition operators and postfix unary
5149factorial operators (@samp{!}), and allow parentheses for grouping.
5150
5151@example
5152@group
5153expr: term '+' expr
5154 | term
5155 ;
5156@end group
5157
5158@group
5159term: '(' expr ')'
5160 | term '!'
5161 | NUMBER
5162 ;
5163@end group
5164@end example
5165
5166Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
5167should be done? If the following token is @samp{)}, then the first three
5168tokens must be reduced to form an @code{expr}. This is the only valid
5169course, because shifting the @samp{)} would produce a sequence of symbols
5170@w{@code{term ')'}}, and no rule allows this.
5171
5172If the following token is @samp{!}, then it must be shifted immediately so
5173that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
5174parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
5175@code{expr}. It would then be impossible to shift the @samp{!} because
5176doing so would produce on the stack the sequence of symbols @code{expr
5177'!'}. No rule allows that sequence.
5178
5179@vindex yychar
32c29292
JD
5180@vindex yylval
5181@vindex yylloc
5182The look-ahead token is stored in the variable @code{yychar}.
5183Its semantic value and location, if any, are stored in the variables
5184@code{yylval} and @code{yylloc}.
bfa74976
RS
5185@xref{Action Features, ,Special Features for Use in Actions}.
5186
342b8b6e 5187@node Shift/Reduce
bfa74976
RS
5188@section Shift/Reduce Conflicts
5189@cindex conflicts
5190@cindex shift/reduce conflicts
5191@cindex dangling @code{else}
5192@cindex @code{else}, dangling
5193
5194Suppose we are parsing a language which has if-then and if-then-else
5195statements, with a pair of rules like this:
5196
5197@example
5198@group
5199if_stmt:
5200 IF expr THEN stmt
5201 | IF expr THEN stmt ELSE stmt
5202 ;
5203@end group
5204@end example
5205
5206@noindent
5207Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
5208terminal symbols for specific keyword tokens.
5209
5210When the @code{ELSE} token is read and becomes the look-ahead token, the
5211contents of the stack (assuming the input is valid) are just right for
5212reduction by the first rule. But it is also legitimate to shift the
5213@code{ELSE}, because that would lead to eventual reduction by the second
5214rule.
5215
5216This situation, where either a shift or a reduction would be valid, is
5217called a @dfn{shift/reduce conflict}. Bison is designed to resolve
5218these conflicts by choosing to shift, unless otherwise directed by
5219operator precedence declarations. To see the reason for this, let's
5220contrast it with the other alternative.
5221
5222Since the parser prefers to shift the @code{ELSE}, the result is to attach
5223the else-clause to the innermost if-statement, making these two inputs
5224equivalent:
5225
5226@example
5227if x then if y then win (); else lose;
5228
5229if x then do; if y then win (); else lose; end;
5230@end example
5231
5232But if the parser chose to reduce when possible rather than shift, the
5233result would be to attach the else-clause to the outermost if-statement,
5234making these two inputs equivalent:
5235
5236@example
5237if x then if y then win (); else lose;
5238
5239if x then do; if y then win (); end; else lose;
5240@end example
5241
5242The conflict exists because the grammar as written is ambiguous: either
5243parsing of the simple nested if-statement is legitimate. The established
5244convention is that these ambiguities are resolved by attaching the
5245else-clause to the innermost if-statement; this is what Bison accomplishes
5246by choosing to shift rather than reduce. (It would ideally be cleaner to
5247write an unambiguous grammar, but that is very hard to do in this case.)
5248This particular ambiguity was first encountered in the specifications of
5249Algol 60 and is called the ``dangling @code{else}'' ambiguity.
5250
5251To avoid warnings from Bison about predictable, legitimate shift/reduce
5252conflicts, use the @code{%expect @var{n}} declaration. There will be no
5253warning as long as the number of shift/reduce conflicts is exactly @var{n}.
5254@xref{Expect Decl, ,Suppressing Conflict Warnings}.
5255
5256The definition of @code{if_stmt} above is solely to blame for the
5257conflict, but the conflict does not actually appear without additional
5258rules. Here is a complete Bison input file that actually manifests the
5259conflict:
5260
5261@example
5262@group
5263%token IF THEN ELSE variable
5264%%
5265@end group
5266@group
5267stmt: expr
5268 | if_stmt
5269 ;
5270@end group
5271
5272@group
5273if_stmt:
5274 IF expr THEN stmt
5275 | IF expr THEN stmt ELSE stmt
5276 ;
5277@end group
5278
5279expr: variable
5280 ;
5281@end example
5282
342b8b6e 5283@node Precedence
bfa74976
RS
5284@section Operator Precedence
5285@cindex operator precedence
5286@cindex precedence of operators
5287
5288Another situation where shift/reduce conflicts appear is in arithmetic
5289expressions. Here shifting is not always the preferred resolution; the
5290Bison declarations for operator precedence allow you to specify when to
5291shift and when to reduce.
5292
5293@menu
5294* Why Precedence:: An example showing why precedence is needed.
5295* Using Precedence:: How to specify precedence in Bison grammars.
5296* Precedence Examples:: How these features are used in the previous example.
5297* How Precedence:: How they work.
5298@end menu
5299
342b8b6e 5300@node Why Precedence
bfa74976
RS
5301@subsection When Precedence is Needed
5302
5303Consider the following ambiguous grammar fragment (ambiguous because the
5304input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
5305
5306@example
5307@group
5308expr: expr '-' expr
5309 | expr '*' expr
5310 | expr '<' expr
5311 | '(' expr ')'
5312 @dots{}
5313 ;
5314@end group
5315@end example
5316
5317@noindent
5318Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
14ded682
AD
5319should it reduce them via the rule for the subtraction operator? It
5320depends on the next token. Of course, if the next token is @samp{)}, we
5321must reduce; shifting is invalid because no single rule can reduce the
5322token sequence @w{@samp{- 2 )}} or anything starting with that. But if
5323the next token is @samp{*} or @samp{<}, we have a choice: either
5324shifting or reduction would allow the parse to complete, but with
5325different results.
5326
5327To decide which one Bison should do, we must consider the results. If
5328the next operator token @var{op} is shifted, then it must be reduced
5329first in order to permit another opportunity to reduce the difference.
5330The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
5331hand, if the subtraction is reduced before shifting @var{op}, the result
5332is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
5333reduce should depend on the relative precedence of the operators
5334@samp{-} and @var{op}: @samp{*} should be shifted first, but not
5335@samp{<}.
bfa74976
RS
5336
5337@cindex associativity
5338What about input such as @w{@samp{1 - 2 - 5}}; should this be
14ded682
AD
5339@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
5340operators we prefer the former, which is called @dfn{left association}.
5341The latter alternative, @dfn{right association}, is desirable for
5342assignment operators. The choice of left or right association is a
5343matter of whether the parser chooses to shift or reduce when the stack
5344contains @w{@samp{1 - 2}} and the look-ahead token is @samp{-}: shifting
5345makes right-associativity.
bfa74976 5346
342b8b6e 5347@node Using Precedence
bfa74976
RS
5348@subsection Specifying Operator Precedence
5349@findex %left
5350@findex %right
5351@findex %nonassoc
5352
5353Bison allows you to specify these choices with the operator precedence
5354declarations @code{%left} and @code{%right}. Each such declaration
5355contains a list of tokens, which are operators whose precedence and
5356associativity is being declared. The @code{%left} declaration makes all
5357those operators left-associative and the @code{%right} declaration makes
5358them right-associative. A third alternative is @code{%nonassoc}, which
5359declares that it is a syntax error to find the same operator twice ``in a
5360row''.
5361
5362The relative precedence of different operators is controlled by the
5363order in which they are declared. The first @code{%left} or
5364@code{%right} declaration in the file declares the operators whose
5365precedence is lowest, the next such declaration declares the operators
5366whose precedence is a little higher, and so on.
5367
342b8b6e 5368@node Precedence Examples
bfa74976
RS
5369@subsection Precedence Examples
5370
5371In our example, we would want the following declarations:
5372
5373@example
5374%left '<'
5375%left '-'
5376%left '*'
5377@end example
5378
5379In a more complete example, which supports other operators as well, we
5380would declare them in groups of equal precedence. For example, @code{'+'} is
5381declared with @code{'-'}:
5382
5383@example
5384%left '<' '>' '=' NE LE GE
5385%left '+' '-'
5386%left '*' '/'
5387@end example
5388
5389@noindent
5390(Here @code{NE} and so on stand for the operators for ``not equal''
5391and so on. We assume that these tokens are more than one character long
5392and therefore are represented by names, not character literals.)
5393
342b8b6e 5394@node How Precedence
bfa74976
RS
5395@subsection How Precedence Works
5396
5397The first effect of the precedence declarations is to assign precedence
5398levels to the terminal symbols declared. The second effect is to assign
704a47c4
AD
5399precedence levels to certain rules: each rule gets its precedence from
5400the last terminal symbol mentioned in the components. (You can also
5401specify explicitly the precedence of a rule. @xref{Contextual
5402Precedence, ,Context-Dependent Precedence}.)
5403
5404Finally, the resolution of conflicts works by comparing the precedence
5405of the rule being considered with that of the look-ahead token. If the
5406token's precedence is higher, the choice is to shift. If the rule's
5407precedence is higher, the choice is to reduce. If they have equal
5408precedence, the choice is made based on the associativity of that
5409precedence level. The verbose output file made by @samp{-v}
5410(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
5411resolved.
bfa74976
RS
5412
5413Not all rules and not all tokens have precedence. If either the rule or
5414the look-ahead token has no precedence, then the default is to shift.
5415
342b8b6e 5416@node Contextual Precedence
bfa74976
RS
5417@section Context-Dependent Precedence
5418@cindex context-dependent precedence
5419@cindex unary operator precedence
5420@cindex precedence, context-dependent
5421@cindex precedence, unary operator
5422@findex %prec
5423
5424Often the precedence of an operator depends on the context. This sounds
5425outlandish at first, but it is really very common. For example, a minus
5426sign typically has a very high precedence as a unary operator, and a
5427somewhat lower precedence (lower than multiplication) as a binary operator.
5428
5429The Bison precedence declarations, @code{%left}, @code{%right} and
5430@code{%nonassoc}, can only be used once for a given token; so a token has
5431only one precedence declared in this way. For context-dependent
5432precedence, you need to use an additional mechanism: the @code{%prec}
e0c471a9 5433modifier for rules.
bfa74976
RS
5434
5435The @code{%prec} modifier declares the precedence of a particular rule by
5436specifying a terminal symbol whose precedence should be used for that rule.
5437It's not necessary for that symbol to appear otherwise in the rule. The
5438modifier's syntax is:
5439
5440@example
5441%prec @var{terminal-symbol}
5442@end example
5443
5444@noindent
5445and it is written after the components of the rule. Its effect is to
5446assign the rule the precedence of @var{terminal-symbol}, overriding
5447the precedence that would be deduced for it in the ordinary way. The
5448altered rule precedence then affects how conflicts involving that rule
5449are resolved (@pxref{Precedence, ,Operator Precedence}).
5450
5451Here is how @code{%prec} solves the problem of unary minus. First, declare
5452a precedence for a fictitious terminal symbol named @code{UMINUS}. There
5453are no tokens of this type, but the symbol serves to stand for its
5454precedence:
5455
5456@example
5457@dots{}
5458%left '+' '-'
5459%left '*'
5460%left UMINUS
5461@end example
5462
5463Now the precedence of @code{UMINUS} can be used in specific rules:
5464
5465@example
5466@group
5467exp: @dots{}
5468 | exp '-' exp
5469 @dots{}
5470 | '-' exp %prec UMINUS
5471@end group
5472@end example
5473
91d2c560 5474@ifset defaultprec
39a06c25
PE
5475If you forget to append @code{%prec UMINUS} to the rule for unary
5476minus, Bison silently assumes that minus has its usual precedence.
5477This kind of problem can be tricky to debug, since one typically
5478discovers the mistake only by testing the code.
5479
22fccf95 5480The @code{%no-default-prec;} declaration makes it easier to discover
39a06c25
PE
5481this kind of problem systematically. It causes rules that lack a
5482@code{%prec} modifier to have no precedence, even if the last terminal
5483symbol mentioned in their components has a declared precedence.
5484
22fccf95 5485If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
39a06c25
PE
5486for all rules that participate in precedence conflict resolution.
5487Then you will see any shift/reduce conflict until you tell Bison how
5488to resolve it, either by changing your grammar or by adding an
5489explicit precedence. This will probably add declarations to the
5490grammar, but it helps to protect against incorrect rule precedences.
5491
22fccf95
PE
5492The effect of @code{%no-default-prec;} can be reversed by giving
5493@code{%default-prec;}, which is the default.
91d2c560 5494@end ifset
39a06c25 5495
342b8b6e 5496@node Parser States
bfa74976
RS
5497@section Parser States
5498@cindex finite-state machine
5499@cindex parser state
5500@cindex state (of parser)
5501
5502The function @code{yyparse} is implemented using a finite-state machine.
5503The values pushed on the parser stack are not simply token type codes; they
5504represent the entire sequence of terminal and nonterminal symbols at or
5505near the top of the stack. The current state collects all the information
5506about previous input which is relevant to deciding what to do next.
5507
5508Each time a look-ahead token is read, the current parser state together
5509with the type of look-ahead token are looked up in a table. This table
5510entry can say, ``Shift the look-ahead token.'' In this case, it also
5511specifies the new parser state, which is pushed onto the top of the
5512parser stack. Or it can say, ``Reduce using rule number @var{n}.''
5513This means that a certain number of tokens or groupings are taken off
5514the top of the stack, and replaced by one grouping. In other words,
5515that number of states are popped from the stack, and one new state is
5516pushed.
5517
5518There is one other alternative: the table can say that the look-ahead token
5519is erroneous in the current state. This causes error processing to begin
5520(@pxref{Error Recovery}).
5521
342b8b6e 5522@node Reduce/Reduce
bfa74976
RS
5523@section Reduce/Reduce Conflicts
5524@cindex reduce/reduce conflict
5525@cindex conflicts, reduce/reduce
5526
5527A reduce/reduce conflict occurs if there are two or more rules that apply
5528to the same sequence of input. This usually indicates a serious error
5529in the grammar.
5530
5531For example, here is an erroneous attempt to define a sequence
5532of zero or more @code{word} groupings.
5533
5534@example
5535sequence: /* empty */
5536 @{ printf ("empty sequence\n"); @}
5537 | maybeword
5538 | sequence word
5539 @{ printf ("added word %s\n", $2); @}
5540 ;
5541
5542maybeword: /* empty */
5543 @{ printf ("empty maybeword\n"); @}
5544 | word
5545 @{ printf ("single word %s\n", $1); @}
5546 ;
5547@end example
5548
5549@noindent
5550The error is an ambiguity: there is more than one way to parse a single
5551@code{word} into a @code{sequence}. It could be reduced to a
5552@code{maybeword} and then into a @code{sequence} via the second rule.
5553Alternatively, nothing-at-all could be reduced into a @code{sequence}
5554via the first rule, and this could be combined with the @code{word}
5555using the third rule for @code{sequence}.
5556
5557There is also more than one way to reduce nothing-at-all into a
5558@code{sequence}. This can be done directly via the first rule,
5559or indirectly via @code{maybeword} and then the second rule.
5560
5561You might think that this is a distinction without a difference, because it
5562does not change whether any particular input is valid or not. But it does
5563affect which actions are run. One parsing order runs the second rule's
5564action; the other runs the first rule's action and the third rule's action.
5565In this example, the output of the program changes.
5566
5567Bison resolves a reduce/reduce conflict by choosing to use the rule that
5568appears first in the grammar, but it is very risky to rely on this. Every
5569reduce/reduce conflict must be studied and usually eliminated. Here is the
5570proper way to define @code{sequence}:
5571
5572@example
5573sequence: /* empty */
5574 @{ printf ("empty sequence\n"); @}
5575 | sequence word
5576 @{ printf ("added word %s\n", $2); @}
5577 ;
5578@end example
5579
5580Here is another common error that yields a reduce/reduce conflict:
5581
5582@example
5583sequence: /* empty */
5584 | sequence words
5585 | sequence redirects
5586 ;
5587
5588words: /* empty */
5589 | words word
5590 ;
5591
5592redirects:/* empty */
5593 | redirects redirect
5594 ;
5595@end example
5596
5597@noindent
5598The intention here is to define a sequence which can contain either
5599@code{word} or @code{redirect} groupings. The individual definitions of
5600@code{sequence}, @code{words} and @code{redirects} are error-free, but the
5601three together make a subtle ambiguity: even an empty input can be parsed
5602in infinitely many ways!
5603
5604Consider: nothing-at-all could be a @code{words}. Or it could be two
5605@code{words} in a row, or three, or any number. It could equally well be a
5606@code{redirects}, or two, or any number. Or it could be a @code{words}
5607followed by three @code{redirects} and another @code{words}. And so on.
5608
5609Here are two ways to correct these rules. First, to make it a single level
5610of sequence:
5611
5612@example
5613sequence: /* empty */
5614 | sequence word
5615 | sequence redirect
5616 ;
5617@end example
5618
5619Second, to prevent either a @code{words} or a @code{redirects}
5620from being empty:
5621
5622@example
5623sequence: /* empty */
5624 | sequence words
5625 | sequence redirects
5626 ;
5627
5628words: word
5629 | words word
5630 ;
5631
5632redirects:redirect
5633 | redirects redirect
5634 ;
5635@end example
5636
342b8b6e 5637@node Mystery Conflicts
bfa74976
RS
5638@section Mysterious Reduce/Reduce Conflicts
5639
5640Sometimes reduce/reduce conflicts can occur that don't look warranted.
5641Here is an example:
5642
5643@example
5644@group
5645%token ID
5646
5647%%
5648def: param_spec return_spec ','
5649 ;
5650param_spec:
5651 type
5652 | name_list ':' type
5653 ;
5654@end group
5655@group
5656return_spec:
5657 type
5658 | name ':' type
5659 ;
5660@end group
5661@group
5662type: ID
5663 ;
5664@end group
5665@group
5666name: ID
5667 ;
5668name_list:
5669 name
5670 | name ',' name_list
5671 ;
5672@end group
5673@end example
5674
5675It would seem that this grammar can be parsed with only a single token
13863333 5676of look-ahead: when a @code{param_spec} is being read, an @code{ID} is
bfa74976 5677a @code{name} if a comma or colon follows, or a @code{type} if another
c827f760 5678@code{ID} follows. In other words, this grammar is @acronym{LR}(1).
bfa74976 5679
c827f760
PE
5680@cindex @acronym{LR}(1)
5681@cindex @acronym{LALR}(1)
bfa74976 5682However, Bison, like most parser generators, cannot actually handle all
c827f760
PE
5683@acronym{LR}(1) grammars. In this grammar, two contexts, that after
5684an @code{ID}
bfa74976
RS
5685at the beginning of a @code{param_spec} and likewise at the beginning of
5686a @code{return_spec}, are similar enough that Bison assumes they are the
5687same. They appear similar because the same set of rules would be
5688active---the rule for reducing to a @code{name} and that for reducing to
5689a @code{type}. Bison is unable to determine at that stage of processing
5690that the rules would require different look-ahead tokens in the two
5691contexts, so it makes a single parser state for them both. Combining
5692the two contexts causes a conflict later. In parser terminology, this
c827f760 5693occurrence means that the grammar is not @acronym{LALR}(1).
bfa74976
RS
5694
5695In general, it is better to fix deficiencies than to document them. But
5696this particular deficiency is intrinsically hard to fix; parser
c827f760
PE
5697generators that can handle @acronym{LR}(1) grammars are hard to write
5698and tend to
bfa74976
RS
5699produce parsers that are very large. In practice, Bison is more useful
5700as it is now.
5701
5702When the problem arises, you can often fix it by identifying the two
a220f555
MA
5703parser states that are being confused, and adding something to make them
5704look distinct. In the above example, adding one rule to
bfa74976
RS
5705@code{return_spec} as follows makes the problem go away:
5706
5707@example
5708@group
5709%token BOGUS
5710@dots{}
5711%%
5712@dots{}
5713return_spec:
5714 type
5715 | name ':' type
5716 /* This rule is never used. */
5717 | ID BOGUS
5718 ;
5719@end group
5720@end example
5721
5722This corrects the problem because it introduces the possibility of an
5723additional active rule in the context after the @code{ID} at the beginning of
5724@code{return_spec}. This rule is not active in the corresponding context
5725in a @code{param_spec}, so the two contexts receive distinct parser states.
5726As long as the token @code{BOGUS} is never generated by @code{yylex},
5727the added rule cannot alter the way actual input is parsed.
5728
5729In this particular example, there is another way to solve the problem:
5730rewrite the rule for @code{return_spec} to use @code{ID} directly
5731instead of via @code{name}. This also causes the two confusing
5732contexts to have different sets of active rules, because the one for
5733@code{return_spec} activates the altered rule for @code{return_spec}
5734rather than the one for @code{name}.
5735
5736@example
5737param_spec:
5738 type
5739 | name_list ':' type
5740 ;
5741return_spec:
5742 type
5743 | ID ':' type
5744 ;
5745@end example
5746
e054b190
PE
5747For a more detailed exposition of @acronym{LALR}(1) parsers and parser
5748generators, please see:
5749Frank DeRemer and Thomas Pennello, Efficient Computation of
5750@acronym{LALR}(1) Look-Ahead Sets, @cite{@acronym{ACM} Transactions on
5751Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
5752pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
5753
fae437e8 5754@node Generalized LR Parsing
c827f760
PE
5755@section Generalized @acronym{LR} (@acronym{GLR}) Parsing
5756@cindex @acronym{GLR} parsing
5757@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2 5758@cindex ambiguous grammars
9d9b8b70 5759@cindex nondeterministic parsing
676385e2 5760
fae437e8
AD
5761Bison produces @emph{deterministic} parsers that choose uniquely
5762when to reduce and which reduction to apply
8dd162d3 5763based on a summary of the preceding input and on one extra token of look-ahead.
676385e2
PH
5764As a result, normal Bison handles a proper subset of the family of
5765context-free languages.
fae437e8 5766Ambiguous grammars, since they have strings with more than one possible
676385e2
PH
5767sequence of reductions cannot have deterministic parsers in this sense.
5768The same is true of languages that require more than one symbol of
8dd162d3 5769look-ahead, since the parser lacks the information necessary to make a
676385e2 5770decision at the point it must be made in a shift-reduce parser.
fae437e8 5771Finally, as previously mentioned (@pxref{Mystery Conflicts}),
676385e2
PH
5772there are languages where Bison's particular choice of how to
5773summarize the input seen so far loses necessary information.
5774
5775When you use the @samp{%glr-parser} declaration in your grammar file,
5776Bison generates a parser that uses a different algorithm, called
c827f760
PE
5777Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR}
5778parser uses the same basic
676385e2
PH
5779algorithm for parsing as an ordinary Bison parser, but behaves
5780differently in cases where there is a shift-reduce conflict that has not
fae437e8 5781been resolved by precedence rules (@pxref{Precedence}) or a
c827f760
PE
5782reduce-reduce conflict. When a @acronym{GLR} parser encounters such a
5783situation, it
fae437e8 5784effectively @emph{splits} into a several parsers, one for each possible
676385e2
PH
5785shift or reduction. These parsers then proceed as usual, consuming
5786tokens in lock-step. Some of the stacks may encounter other conflicts
fae437e8 5787and split further, with the result that instead of a sequence of states,
c827f760 5788a Bison @acronym{GLR} parsing stack is what is in effect a tree of states.
676385e2
PH
5789
5790In effect, each stack represents a guess as to what the proper parse
5791is. Additional input may indicate that a guess was wrong, in which case
5792the appropriate stack silently disappears. Otherwise, the semantics
fae437e8 5793actions generated in each stack are saved, rather than being executed
676385e2 5794immediately. When a stack disappears, its saved semantic actions never
fae437e8 5795get executed. When a reduction causes two stacks to become equivalent,
676385e2
PH
5796their sets of semantic actions are both saved with the state that
5797results from the reduction. We say that two stacks are equivalent
fae437e8 5798when they both represent the same sequence of states,
676385e2
PH
5799and each pair of corresponding states represents a
5800grammar symbol that produces the same segment of the input token
5801stream.
5802
5803Whenever the parser makes a transition from having multiple
c827f760 5804states to having one, it reverts to the normal @acronym{LALR}(1) parsing
676385e2
PH
5805algorithm, after resolving and executing the saved-up actions.
5806At this transition, some of the states on the stack will have semantic
5807values that are sets (actually multisets) of possible actions. The
5808parser tries to pick one of the actions by first finding one whose rule
5809has the highest dynamic precedence, as set by the @samp{%dprec}
fae437e8 5810declaration. Otherwise, if the alternative actions are not ordered by
676385e2 5811precedence, but there the same merging function is declared for both
fae437e8 5812rules by the @samp{%merge} declaration,
676385e2
PH
5813Bison resolves and evaluates both and then calls the merge function on
5814the result. Otherwise, it reports an ambiguity.
5815
c827f760
PE
5816It is possible to use a data structure for the @acronym{GLR} parsing tree that
5817permits the processing of any @acronym{LALR}(1) grammar in linear time (in the
5818size of the input), any unambiguous (not necessarily
5819@acronym{LALR}(1)) grammar in
fae437e8 5820quadratic worst-case time, and any general (possibly ambiguous)
676385e2
PH
5821context-free grammar in cubic worst-case time. However, Bison currently
5822uses a simpler data structure that requires time proportional to the
5823length of the input times the maximum number of stacks required for any
9d9b8b70 5824prefix of the input. Thus, really ambiguous or nondeterministic
676385e2
PH
5825grammars can require exponential time and space to process. Such badly
5826behaving examples, however, are not generally of practical interest.
9d9b8b70 5827Usually, nondeterminism in a grammar is local---the parser is ``in
676385e2 5828doubt'' only for a few tokens at a time. Therefore, the current data
c827f760 5829structure should generally be adequate. On @acronym{LALR}(1) portions of a
676385e2
PH
5830grammar, in particular, it is only slightly slower than with the default
5831Bison parser.
5832
fa7e68c3 5833For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth
f6481e2f
PE
5834Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
5835Generalised @acronym{LR} Parsers, Royal Holloway, University of
5836London, Department of Computer Science, TR-00-12,
5837@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
5838(2000-12-24).
5839
1a059451
PE
5840@node Memory Management
5841@section Memory Management, and How to Avoid Memory Exhaustion
5842@cindex memory exhaustion
5843@cindex memory management
bfa74976
RS
5844@cindex stack overflow
5845@cindex parser stack overflow
5846@cindex overflow of parser stack
5847
1a059451 5848The Bison parser stack can run out of memory if too many tokens are shifted and
bfa74976 5849not reduced. When this happens, the parser function @code{yyparse}
1a059451 5850calls @code{yyerror} and then returns 2.
bfa74976 5851
c827f760 5852Because Bison parsers have growing stacks, hitting the upper limit
d1a1114f
AD
5853usually results from using a right recursion instead of a left
5854recursion, @xref{Recursion, ,Recursive Rules}.
5855
bfa74976
RS
5856@vindex YYMAXDEPTH
5857By defining the macro @code{YYMAXDEPTH}, you can control how deep the
1a059451 5858parser stack can become before memory is exhausted. Define the
bfa74976
RS
5859macro with a value that is an integer. This value is the maximum number
5860of tokens that can be shifted (and not reduced) before overflow.
bfa74976
RS
5861
5862The stack space allowed is not necessarily allocated. If you specify a
1a059451 5863large value for @code{YYMAXDEPTH}, the parser normally allocates a small
bfa74976
RS
5864stack at first, and then makes it bigger by stages as needed. This
5865increasing allocation happens automatically and silently. Therefore,
5866you do not need to make @code{YYMAXDEPTH} painfully small merely to save
5867space for ordinary inputs that do not need much stack.
5868
d7e14fc0
PE
5869However, do not allow @code{YYMAXDEPTH} to be a value so large that
5870arithmetic overflow could occur when calculating the size of the stack
5871space. Also, do not allow @code{YYMAXDEPTH} to be less than
5872@code{YYINITDEPTH}.
5873
bfa74976
RS
5874@cindex default stack limit
5875The default value of @code{YYMAXDEPTH}, if you do not define it, is
587610000.
5877
5878@vindex YYINITDEPTH
5879You can control how much stack is allocated initially by defining the
d7e14fc0
PE
5880macro @code{YYINITDEPTH} to a positive integer. For the C
5881@acronym{LALR}(1) parser, this value must be a compile-time constant
5882unless you are assuming C99 or some other target language or compiler
5883that allows variable-length arrays. The default is 200.
5884
1a059451 5885Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
bfa74976 5886
d1a1114f 5887@c FIXME: C++ output.
c827f760 5888Because of semantical differences between C and C++, the
1a059451
PE
5889@acronym{LALR}(1) parsers in C produced by Bison cannot grow when compiled
5890by C++ compilers. In this precise case (compiling a C parser as C++) you are
5891suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix
5892this deficiency in a future release.
d1a1114f 5893
342b8b6e 5894@node Error Recovery
bfa74976
RS
5895@chapter Error Recovery
5896@cindex error recovery
5897@cindex recovery from errors
5898
6e649e65 5899It is not usually acceptable to have a program terminate on a syntax
bfa74976
RS
5900error. For example, a compiler should recover sufficiently to parse the
5901rest of the input file and check it for errors; a calculator should accept
5902another expression.
5903
5904In a simple interactive command parser where each input is one line, it may
5905be sufficient to allow @code{yyparse} to return 1 on error and have the
5906caller ignore the rest of the input line when that happens (and then call
5907@code{yyparse} again). But this is inadequate for a compiler, because it
5908forgets all the syntactic context leading up to the error. A syntax error
5909deep within a function in the compiler input should not cause the compiler
5910to treat the following line like the beginning of a source file.
5911
5912@findex error
5913You can define how to recover from a syntax error by writing rules to
5914recognize the special token @code{error}. This is a terminal symbol that
5915is always defined (you need not declare it) and reserved for error
5916handling. The Bison parser generates an @code{error} token whenever a
5917syntax error happens; if you have provided a rule to recognize this token
13863333 5918in the current context, the parse can continue.
bfa74976
RS
5919
5920For example:
5921
5922@example
5923stmnts: /* empty string */
5924 | stmnts '\n'
5925 | stmnts exp '\n'
5926 | stmnts error '\n'
5927@end example
5928
5929The fourth rule in this example says that an error followed by a newline
5930makes a valid addition to any @code{stmnts}.
5931
5932What happens if a syntax error occurs in the middle of an @code{exp}? The
5933error recovery rule, interpreted strictly, applies to the precise sequence
5934of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
5935the middle of an @code{exp}, there will probably be some additional tokens
5936and subexpressions on the stack after the last @code{stmnts}, and there
5937will be tokens to read before the next newline. So the rule is not
5938applicable in the ordinary way.
5939
5940But Bison can force the situation to fit the rule, by discarding part of
72f889cc
AD
5941the semantic context and part of the input. First it discards states
5942and objects from the stack until it gets back to a state in which the
bfa74976 5943@code{error} token is acceptable. (This means that the subexpressions
72f889cc
AD
5944already parsed are discarded, back to the last complete @code{stmnts}.)
5945At this point the @code{error} token can be shifted. Then, if the old
bfa74976
RS
5946look-ahead token is not acceptable to be shifted next, the parser reads
5947tokens and discards them until it finds a token which is acceptable. In
72f889cc
AD
5948this example, Bison reads and discards input until the next newline so
5949that the fourth rule can apply. Note that discarded symbols are
5950possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
5951Discarded Symbols}, for a means to reclaim this memory.
bfa74976
RS
5952
5953The choice of error rules in the grammar is a choice of strategies for
5954error recovery. A simple and useful strategy is simply to skip the rest of
5955the current input line or current statement if an error is detected:
5956
5957@example
72d2299c 5958stmnt: error ';' /* On error, skip until ';' is read. */
bfa74976
RS
5959@end example
5960
5961It is also useful to recover to the matching close-delimiter of an
5962opening-delimiter that has already been parsed. Otherwise the
5963close-delimiter will probably appear to be unmatched, and generate another,
5964spurious error message:
5965
5966@example
5967primary: '(' expr ')'
5968 | '(' error ')'
5969 @dots{}
5970 ;
5971@end example
5972
5973Error recovery strategies are necessarily guesses. When they guess wrong,
5974one syntax error often leads to another. In the above example, the error
5975recovery rule guesses that an error is due to bad input within one
5976@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
5977middle of a valid @code{stmnt}. After the error recovery rule recovers
5978from the first error, another syntax error will be found straightaway,
5979since the text following the spurious semicolon is also an invalid
5980@code{stmnt}.
5981
5982To prevent an outpouring of error messages, the parser will output no error
5983message for another syntax error that happens shortly after the first; only
5984after three consecutive input tokens have been successfully shifted will
5985error messages resume.
5986
5987Note that rules which accept the @code{error} token may have actions, just
5988as any other rules can.
5989
5990@findex yyerrok
5991You can make error messages resume immediately by using the macro
5992@code{yyerrok} in an action. If you do this in the error rule's action, no
5993error messages will be suppressed. This macro requires no arguments;
5994@samp{yyerrok;} is a valid C statement.
5995
5996@findex yyclearin
5997The previous look-ahead token is reanalyzed immediately after an error. If
5998this is unacceptable, then the macro @code{yyclearin} may be used to clear
5999this token. Write the statement @samp{yyclearin;} in the error rule's
6000action.
32c29292 6001@xref{Action Features, ,Special Features for Use in Actions}.
bfa74976 6002
6e649e65 6003For example, suppose that on a syntax error, an error handling routine is
bfa74976
RS
6004called that advances the input stream to some point where parsing should
6005once again commence. The next symbol returned by the lexical scanner is
6006probably correct. The previous look-ahead token ought to be discarded
6007with @samp{yyclearin;}.
6008
6009@vindex YYRECOVERING
6010The macro @code{YYRECOVERING} stands for an expression that has the
6011value 1 when the parser is recovering from a syntax error, and 0 the
6012rest of the time. A value of 1 indicates that error messages are
6013currently suppressed for new syntax errors.
6014
342b8b6e 6015@node Context Dependency
bfa74976
RS
6016@chapter Handling Context Dependencies
6017
6018The Bison paradigm is to parse tokens first, then group them into larger
6019syntactic units. In many languages, the meaning of a token is affected by
6020its context. Although this violates the Bison paradigm, certain techniques
6021(known as @dfn{kludges}) may enable you to write Bison parsers for such
6022languages.
6023
6024@menu
6025* Semantic Tokens:: Token parsing can depend on the semantic context.
6026* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
6027* Tie-in Recovery:: Lexical tie-ins have implications for how
6028 error recovery rules must be written.
6029@end menu
6030
6031(Actually, ``kludge'' means any technique that gets its job done but is
6032neither clean nor robust.)
6033
342b8b6e 6034@node Semantic Tokens
bfa74976
RS
6035@section Semantic Info in Token Types
6036
6037The C language has a context dependency: the way an identifier is used
6038depends on what its current meaning is. For example, consider this:
6039
6040@example
6041foo (x);
6042@end example
6043
6044This looks like a function call statement, but if @code{foo} is a typedef
6045name, then this is actually a declaration of @code{x}. How can a Bison
6046parser for C decide how to parse this input?
6047
c827f760 6048The method used in @acronym{GNU} C is to have two different token types,
bfa74976
RS
6049@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
6050identifier, it looks up the current declaration of the identifier in order
6051to decide which token type to return: @code{TYPENAME} if the identifier is
6052declared as a typedef, @code{IDENTIFIER} otherwise.
6053
6054The grammar rules can then express the context dependency by the choice of
6055token type to recognize. @code{IDENTIFIER} is accepted as an expression,
6056but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
6057@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
6058is @emph{not} significant, such as in declarations that can shadow a
6059typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
6060accepted---there is one rule for each of the two token types.
6061
6062This technique is simple to use if the decision of which kinds of
6063identifiers to allow is made at a place close to where the identifier is
6064parsed. But in C this is not always so: C allows a declaration to
6065redeclare a typedef name provided an explicit type has been specified
6066earlier:
6067
6068@example
3a4f411f
PE
6069typedef int foo, bar;
6070int baz (void)
6071@{
6072 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
6073 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
6074 return foo (bar);
6075@}
bfa74976
RS
6076@end example
6077
6078Unfortunately, the name being declared is separated from the declaration
6079construct itself by a complicated syntactic structure---the ``declarator''.
6080
9ecbd125 6081As a result, part of the Bison parser for C needs to be duplicated, with
14ded682
AD
6082all the nonterminal names changed: once for parsing a declaration in
6083which a typedef name can be redefined, and once for parsing a
6084declaration in which that can't be done. Here is a part of the
6085duplication, with actions omitted for brevity:
bfa74976
RS
6086
6087@example
6088initdcl:
6089 declarator maybeasm '='
6090 init
6091 | declarator maybeasm
6092 ;
6093
6094notype_initdcl:
6095 notype_declarator maybeasm '='
6096 init
6097 | notype_declarator maybeasm
6098 ;
6099@end example
6100
6101@noindent
6102Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
6103cannot. The distinction between @code{declarator} and
6104@code{notype_declarator} is the same sort of thing.
6105
6106There is some similarity between this technique and a lexical tie-in
6107(described next), in that information which alters the lexical analysis is
6108changed during parsing by other parts of the program. The difference is
6109here the information is global, and is used for other purposes in the
6110program. A true lexical tie-in has a special-purpose flag controlled by
6111the syntactic context.
6112
342b8b6e 6113@node Lexical Tie-ins
bfa74976
RS
6114@section Lexical Tie-ins
6115@cindex lexical tie-in
6116
6117One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
6118which is set by Bison actions, whose purpose is to alter the way tokens are
6119parsed.
6120
6121For example, suppose we have a language vaguely like C, but with a special
6122construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
6123an expression in parentheses in which all integers are hexadecimal. In
6124particular, the token @samp{a1b} must be treated as an integer rather than
6125as an identifier if it appears in that context. Here is how you can do it:
6126
6127@example
6128@group
6129%@{
38a92d50
PE
6130 int hexflag;
6131 int yylex (void);
6132 void yyerror (char const *);
bfa74976
RS
6133%@}
6134%%
6135@dots{}
6136@end group
6137@group
6138expr: IDENTIFIER
6139 | constant
6140 | HEX '('
6141 @{ hexflag = 1; @}
6142 expr ')'
6143 @{ hexflag = 0;
6144 $$ = $4; @}
6145 | expr '+' expr
6146 @{ $$ = make_sum ($1, $3); @}
6147 @dots{}
6148 ;
6149@end group
6150
6151@group
6152constant:
6153 INTEGER
6154 | STRING
6155 ;
6156@end group
6157@end example
6158
6159@noindent
6160Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
6161it is nonzero, all integers are parsed in hexadecimal, and tokens starting
6162with letters are parsed as integers if possible.
6163
342b8b6e
AD
6164The declaration of @code{hexflag} shown in the prologue of the parser file
6165is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}).
75f5aaea 6166You must also write the code in @code{yylex} to obey the flag.
bfa74976 6167
342b8b6e 6168@node Tie-in Recovery
bfa74976
RS
6169@section Lexical Tie-ins and Error Recovery
6170
6171Lexical tie-ins make strict demands on any error recovery rules you have.
6172@xref{Error Recovery}.
6173
6174The reason for this is that the purpose of an error recovery rule is to
6175abort the parsing of one construct and resume in some larger construct.
6176For example, in C-like languages, a typical error recovery rule is to skip
6177tokens until the next semicolon, and then start a new statement, like this:
6178
6179@example
6180stmt: expr ';'
6181 | IF '(' expr ')' stmt @{ @dots{} @}
6182 @dots{}
6183 error ';'
6184 @{ hexflag = 0; @}
6185 ;
6186@end example
6187
6188If there is a syntax error in the middle of a @samp{hex (@var{expr})}
6189construct, this error rule will apply, and then the action for the
6190completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
6191remain set for the entire rest of the input, or until the next @code{hex}
6192keyword, causing identifiers to be misinterpreted as integers.
6193
6194To avoid this problem the error recovery rule itself clears @code{hexflag}.
6195
6196There may also be an error recovery rule that works within expressions.
6197For example, there could be a rule which applies within parentheses
6198and skips to the close-parenthesis:
6199
6200@example
6201@group
6202expr: @dots{}
6203 | '(' expr ')'
6204 @{ $$ = $2; @}
6205 | '(' error ')'
6206 @dots{}
6207@end group
6208@end example
6209
6210If this rule acts within the @code{hex} construct, it is not going to abort
6211that construct (since it applies to an inner level of parentheses within
6212the construct). Therefore, it should not clear the flag: the rest of
6213the @code{hex} construct should be parsed with the flag still in effect.
6214
6215What if there is an error recovery rule which might abort out of the
6216@code{hex} construct or might not, depending on circumstances? There is no
6217way you can write the action to determine whether a @code{hex} construct is
6218being aborted or not. So if you are using a lexical tie-in, you had better
6219make sure your error recovery rules are not of this kind. Each rule must
6220be such that you can be sure that it always will, or always won't, have to
6221clear the flag.
6222
ec3bc396
AD
6223@c ================================================== Debugging Your Parser
6224
342b8b6e 6225@node Debugging
bfa74976 6226@chapter Debugging Your Parser
ec3bc396
AD
6227
6228Developing a parser can be a challenge, especially if you don't
6229understand the algorithm (@pxref{Algorithm, ,The Bison Parser
6230Algorithm}). Even so, sometimes a detailed description of the automaton
6231can help (@pxref{Understanding, , Understanding Your Parser}), or
6232tracing the execution of the parser can give some insight on why it
6233behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
6234
6235@menu
6236* Understanding:: Understanding the structure of your parser.
6237* Tracing:: Tracing the execution of your parser.
6238@end menu
6239
6240@node Understanding
6241@section Understanding Your Parser
6242
6243As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
6244Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
6245frequent than one would hope), looking at this automaton is required to
6246tune or simply fix a parser. Bison provides two different
c827f760 6247representation of it, either textually or graphically (as a @acronym{VCG}
ec3bc396
AD
6248file).
6249
6250The textual file is generated when the options @option{--report} or
6251@option{--verbose} are specified, see @xref{Invocation, , Invoking
6252Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
6253the parser output file name, and adding @samp{.output} instead.
6254Therefore, if the input file is @file{foo.y}, then the parser file is
6255called @file{foo.tab.c} by default. As a consequence, the verbose
6256output file is called @file{foo.output}.
6257
6258The following grammar file, @file{calc.y}, will be used in the sequel:
6259
6260@example
6261%token NUM STR
6262%left '+' '-'
6263%left '*'
6264%%
6265exp: exp '+' exp
6266 | exp '-' exp
6267 | exp '*' exp
6268 | exp '/' exp
6269 | NUM
6270 ;
6271useless: STR;
6272%%
6273@end example
6274
88bce5a2
AD
6275@command{bison} reports:
6276
6277@example
6278calc.y: warning: 1 useless nonterminal and 1 useless rule
6279calc.y:11.1-7: warning: useless nonterminal: useless
5a99098d
PE
6280calc.y:11.10-12: warning: useless rule: useless: STR
6281calc.y: conflicts: 7 shift/reduce
88bce5a2
AD
6282@end example
6283
6284When given @option{--report=state}, in addition to @file{calc.tab.c}, it
6285creates a file @file{calc.output} with contents detailed below. The
6286order of the output and the exact presentation might vary, but the
6287interpretation is the same.
ec3bc396
AD
6288
6289The first section includes details on conflicts that were solved thanks
6290to precedence and/or associativity:
6291
6292@example
6293Conflict in state 8 between rule 2 and token '+' resolved as reduce.
6294Conflict in state 8 between rule 2 and token '-' resolved as reduce.
6295Conflict in state 8 between rule 2 and token '*' resolved as shift.
6296@exdent @dots{}
6297@end example
6298
6299@noindent
6300The next section lists states that still have conflicts.
6301
6302@example
5a99098d
PE
6303State 8 conflicts: 1 shift/reduce
6304State 9 conflicts: 1 shift/reduce
6305State 10 conflicts: 1 shift/reduce
6306State 11 conflicts: 4 shift/reduce
ec3bc396
AD
6307@end example
6308
6309@noindent
6310@cindex token, useless
6311@cindex useless token
6312@cindex nonterminal, useless
6313@cindex useless nonterminal
6314@cindex rule, useless
6315@cindex useless rule
6316The next section reports useless tokens, nonterminal and rules. Useless
6317nonterminals and rules are removed in order to produce a smaller parser,
6318but useless tokens are preserved, since they might be used by the
6319scanner (note the difference between ``useless'' and ``not used''
6320below):
6321
6322@example
6323Useless nonterminals:
6324 useless
6325
6326Terminals which are not used:
6327 STR
6328
6329Useless rules:
6330#6 useless: STR;
6331@end example
6332
6333@noindent
6334The next section reproduces the exact grammar that Bison used:
6335
6336@example
6337Grammar
6338
6339 Number, Line, Rule
88bce5a2 6340 0 5 $accept -> exp $end
ec3bc396
AD
6341 1 5 exp -> exp '+' exp
6342 2 6 exp -> exp '-' exp
6343 3 7 exp -> exp '*' exp
6344 4 8 exp -> exp '/' exp
6345 5 9 exp -> NUM
6346@end example
6347
6348@noindent
6349and reports the uses of the symbols:
6350
6351@example
6352Terminals, with rules where they appear
6353
88bce5a2 6354$end (0) 0
ec3bc396
AD
6355'*' (42) 3
6356'+' (43) 1
6357'-' (45) 2
6358'/' (47) 4
6359error (256)
6360NUM (258) 5
6361
6362Nonterminals, with rules where they appear
6363
88bce5a2 6364$accept (8)
ec3bc396
AD
6365 on left: 0
6366exp (9)
6367 on left: 1 2 3 4 5, on right: 0 1 2 3 4
6368@end example
6369
6370@noindent
6371@cindex item
6372@cindex pointed rule
6373@cindex rule, pointed
6374Bison then proceeds onto the automaton itself, describing each state
6375with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
6376item is a production rule together with a point (marked by @samp{.})
6377that the input cursor.
6378
6379@example
6380state 0
6381
88bce5a2 6382 $accept -> . exp $ (rule 0)
ec3bc396 6383
2a8d363a 6384 NUM shift, and go to state 1
ec3bc396 6385
2a8d363a 6386 exp go to state 2
ec3bc396
AD
6387@end example
6388
6389This reads as follows: ``state 0 corresponds to being at the very
6390beginning of the parsing, in the initial rule, right before the start
6391symbol (here, @code{exp}). When the parser returns to this state right
6392after having reduced a rule that produced an @code{exp}, the control
6393flow jumps to state 2. If there is no such transition on a nonterminal
8dd162d3 6394symbol, and the look-ahead is a @code{NUM}, then this token is shifted on
ec3bc396 6395the parse stack, and the control flow jumps to state 1. Any other
8dd162d3 6396look-ahead triggers a syntax error.''
ec3bc396
AD
6397
6398@cindex core, item set
6399@cindex item set core
6400@cindex kernel, item set
6401@cindex item set core
6402Even though the only active rule in state 0 seems to be rule 0, the
8dd162d3 6403report lists @code{NUM} as a look-ahead token because @code{NUM} can be
ec3bc396
AD
6404at the beginning of any rule deriving an @code{exp}. By default Bison
6405reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
6406you want to see more detail you can invoke @command{bison} with
6407@option{--report=itemset} to list all the items, include those that can
6408be derived:
6409
6410@example
6411state 0
6412
88bce5a2 6413 $accept -> . exp $ (rule 0)
ec3bc396
AD
6414 exp -> . exp '+' exp (rule 1)
6415 exp -> . exp '-' exp (rule 2)
6416 exp -> . exp '*' exp (rule 3)
6417 exp -> . exp '/' exp (rule 4)
6418 exp -> . NUM (rule 5)
6419
6420 NUM shift, and go to state 1
6421
6422 exp go to state 2
6423@end example
6424
6425@noindent
6426In the state 1...
6427
6428@example
6429state 1
6430
6431 exp -> NUM . (rule 5)
6432
2a8d363a 6433 $default reduce using rule 5 (exp)
ec3bc396
AD
6434@end example
6435
6436@noindent
8dd162d3 6437the rule 5, @samp{exp: NUM;}, is completed. Whatever the look-ahead token
ec3bc396
AD
6438(@samp{$default}), the parser will reduce it. If it was coming from
6439state 0, then, after this reduction it will return to state 0, and will
6440jump to state 2 (@samp{exp: go to state 2}).
6441
6442@example
6443state 2
6444
88bce5a2 6445 $accept -> exp . $ (rule 0)
ec3bc396
AD
6446 exp -> exp . '+' exp (rule 1)
6447 exp -> exp . '-' exp (rule 2)
6448 exp -> exp . '*' exp (rule 3)
6449 exp -> exp . '/' exp (rule 4)
6450
2a8d363a
AD
6451 $ shift, and go to state 3
6452 '+' shift, and go to state 4
6453 '-' shift, and go to state 5
6454 '*' shift, and go to state 6
6455 '/' shift, and go to state 7
ec3bc396
AD
6456@end example
6457
6458@noindent
6459In state 2, the automaton can only shift a symbol. For instance,
8dd162d3 6460because of the item @samp{exp -> exp . '+' exp}, if the look-ahead if
ec3bc396
AD
6461@samp{+}, it will be shifted on the parse stack, and the automaton
6462control will jump to state 4, corresponding to the item @samp{exp -> exp
6463'+' . exp}. Since there is no default action, any other token than
6e649e65 6464those listed above will trigger a syntax error.
ec3bc396
AD
6465
6466The state 3 is named the @dfn{final state}, or the @dfn{accepting
6467state}:
6468
6469@example
6470state 3
6471
88bce5a2 6472 $accept -> exp $ . (rule 0)
ec3bc396 6473
2a8d363a 6474 $default accept
ec3bc396
AD
6475@end example
6476
6477@noindent
6478the initial rule is completed (the start symbol and the end
6479of input were read), the parsing exits successfully.
6480
6481The interpretation of states 4 to 7 is straightforward, and is left to
6482the reader.
6483
6484@example
6485state 4
6486
6487 exp -> exp '+' . exp (rule 1)
6488
2a8d363a 6489 NUM shift, and go to state 1
ec3bc396 6490
2a8d363a 6491 exp go to state 8
ec3bc396
AD
6492
6493state 5
6494
6495 exp -> exp '-' . exp (rule 2)
6496
2a8d363a 6497 NUM shift, and go to state 1
ec3bc396 6498
2a8d363a 6499 exp go to state 9
ec3bc396
AD
6500
6501state 6
6502
6503 exp -> exp '*' . exp (rule 3)
6504
2a8d363a 6505 NUM shift, and go to state 1
ec3bc396 6506
2a8d363a 6507 exp go to state 10
ec3bc396
AD
6508
6509state 7
6510
6511 exp -> exp '/' . exp (rule 4)
6512
2a8d363a 6513 NUM shift, and go to state 1
ec3bc396 6514
2a8d363a 6515 exp go to state 11
ec3bc396
AD
6516@end example
6517
5a99098d
PE
6518As was announced in beginning of the report, @samp{State 8 conflicts:
65191 shift/reduce}:
ec3bc396
AD
6520
6521@example
6522state 8
6523
6524 exp -> exp . '+' exp (rule 1)
6525 exp -> exp '+' exp . (rule 1)
6526 exp -> exp . '-' exp (rule 2)
6527 exp -> exp . '*' exp (rule 3)
6528 exp -> exp . '/' exp (rule 4)
6529
2a8d363a
AD
6530 '*' shift, and go to state 6
6531 '/' shift, and go to state 7
ec3bc396 6532
2a8d363a
AD
6533 '/' [reduce using rule 1 (exp)]
6534 $default reduce using rule 1 (exp)
ec3bc396
AD
6535@end example
6536
8dd162d3 6537Indeed, there are two actions associated to the look-ahead @samp{/}:
ec3bc396
AD
6538either shifting (and going to state 7), or reducing rule 1. The
6539conflict means that either the grammar is ambiguous, or the parser lacks
6540information to make the right decision. Indeed the grammar is
6541ambiguous, as, since we did not specify the precedence of @samp{/}, the
6542sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
6543NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
6544NUM}, which corresponds to reducing rule 1.
6545
c827f760 6546Because in @acronym{LALR}(1) parsing a single decision can be made, Bison
ec3bc396
AD
6547arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
6548Shift/Reduce Conflicts}. Discarded actions are reported in between
6549square brackets.
6550
6551Note that all the previous states had a single possible action: either
6552shifting the next token and going to the corresponding state, or
6553reducing a single rule. In the other cases, i.e., when shifting
6554@emph{and} reducing is possible or when @emph{several} reductions are
8dd162d3
PE
6555possible, the look-ahead is required to select the action. State 8 is
6556one such state: if the look-ahead is @samp{*} or @samp{/} then the action
ec3bc396
AD
6557is shifting, otherwise the action is reducing rule 1. In other words,
6558the first two items, corresponding to rule 1, are not eligible when the
8dd162d3
PE
6559look-ahead token is @samp{*}, since we specified that @samp{*} has higher
6560precedence than @samp{+}. More generally, some items are eligible only
6561with some set of possible look-ahead tokens. When run with
6562@option{--report=look-ahead}, Bison specifies these look-ahead tokens:
ec3bc396
AD
6563
6564@example
6565state 8
6566
6567 exp -> exp . '+' exp [$, '+', '-', '/'] (rule 1)
6568 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
6569 exp -> exp . '-' exp (rule 2)
6570 exp -> exp . '*' exp (rule 3)
6571 exp -> exp . '/' exp (rule 4)
6572
6573 '*' shift, and go to state 6
6574 '/' shift, and go to state 7
6575
6576 '/' [reduce using rule 1 (exp)]
6577 $default reduce using rule 1 (exp)
6578@end example
6579
6580The remaining states are similar:
6581
6582@example
6583state 9
6584
6585 exp -> exp . '+' exp (rule 1)
6586 exp -> exp . '-' exp (rule 2)
6587 exp -> exp '-' exp . (rule 2)
6588 exp -> exp . '*' exp (rule 3)
6589 exp -> exp . '/' exp (rule 4)
6590
2a8d363a
AD
6591 '*' shift, and go to state 6
6592 '/' shift, and go to state 7
ec3bc396 6593
2a8d363a
AD
6594 '/' [reduce using rule 2 (exp)]
6595 $default reduce using rule 2 (exp)
ec3bc396
AD
6596
6597state 10
6598
6599 exp -> exp . '+' exp (rule 1)
6600 exp -> exp . '-' exp (rule 2)
6601 exp -> exp . '*' exp (rule 3)
6602 exp -> exp '*' exp . (rule 3)
6603 exp -> exp . '/' exp (rule 4)
6604
2a8d363a 6605 '/' shift, and go to state 7
ec3bc396 6606
2a8d363a
AD
6607 '/' [reduce using rule 3 (exp)]
6608 $default reduce using rule 3 (exp)
ec3bc396
AD
6609
6610state 11
6611
6612 exp -> exp . '+' exp (rule 1)
6613 exp -> exp . '-' exp (rule 2)
6614 exp -> exp . '*' exp (rule 3)
6615 exp -> exp . '/' exp (rule 4)
6616 exp -> exp '/' exp . (rule 4)
6617
2a8d363a
AD
6618 '+' shift, and go to state 4
6619 '-' shift, and go to state 5
6620 '*' shift, and go to state 6
6621 '/' shift, and go to state 7
ec3bc396 6622
2a8d363a
AD
6623 '+' [reduce using rule 4 (exp)]
6624 '-' [reduce using rule 4 (exp)]
6625 '*' [reduce using rule 4 (exp)]
6626 '/' [reduce using rule 4 (exp)]
6627 $default reduce using rule 4 (exp)
ec3bc396
AD
6628@end example
6629
6630@noindent
fa7e68c3
PE
6631Observe that state 11 contains conflicts not only due to the lack of
6632precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
6633@samp{*}, but also because the
ec3bc396
AD
6634associativity of @samp{/} is not specified.
6635
6636
6637@node Tracing
6638@section Tracing Your Parser
bfa74976
RS
6639@findex yydebug
6640@cindex debugging
6641@cindex tracing the parser
6642
6643If a Bison grammar compiles properly but doesn't do what you want when it
6644runs, the @code{yydebug} parser-trace feature can help you figure out why.
6645
3ded9a63
AD
6646There are several means to enable compilation of trace facilities:
6647
6648@table @asis
6649@item the macro @code{YYDEBUG}
6650@findex YYDEBUG
6651Define the macro @code{YYDEBUG} to a nonzero value when you compile the
c827f760 6652parser. This is compliant with @acronym{POSIX} Yacc. You could use
3ded9a63
AD
6653@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
6654YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
6655Prologue}).
6656
6657@item the option @option{-t}, @option{--debug}
6658Use the @samp{-t} option when you run Bison (@pxref{Invocation,
c827f760 6659,Invoking Bison}). This is @acronym{POSIX} compliant too.
3ded9a63
AD
6660
6661@item the directive @samp{%debug}
6662@findex %debug
6663Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison
6664Declaration Summary}). This is a Bison extension, which will prove
6665useful when Bison will output parsers for languages that don't use a
c827f760
PE
6666preprocessor. Unless @acronym{POSIX} and Yacc portability matter to
6667you, this is
3ded9a63
AD
6668the preferred solution.
6669@end table
6670
6671We suggest that you always enable the debug option so that debugging is
6672always possible.
bfa74976 6673
02a81e05 6674The trace facility outputs messages with macro calls of the form
e2742e46 6675@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
02a81e05 6676@var{format} and @var{args} are the usual @code{printf} format and
4947ebdb
PE
6677arguments. If you define @code{YYDEBUG} to a nonzero value but do not
6678define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
e4e1a4dc 6679and @code{YYPRINTF} is defined to @code{fprintf}.
bfa74976
RS
6680
6681Once you have compiled the program with trace facilities, the way to
6682request a trace is to store a nonzero value in the variable @code{yydebug}.
6683You can do this by making the C code do it (in @code{main}, perhaps), or
6684you can alter the value with a C debugger.
6685
6686Each step taken by the parser when @code{yydebug} is nonzero produces a
6687line or two of trace information, written on @code{stderr}. The trace
6688messages tell you these things:
6689
6690@itemize @bullet
6691@item
6692Each time the parser calls @code{yylex}, what kind of token was read.
6693
6694@item
6695Each time a token is shifted, the depth and complete contents of the
6696state stack (@pxref{Parser States}).
6697
6698@item
6699Each time a rule is reduced, which rule it is, and the complete contents
6700of the state stack afterward.
6701@end itemize
6702
6703To make sense of this information, it helps to refer to the listing file
704a47c4
AD
6704produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
6705Bison}). This file shows the meaning of each state in terms of
6706positions in various rules, and also what each state will do with each
6707possible input token. As you read the successive trace messages, you
6708can see that the parser is functioning according to its specification in
6709the listing file. Eventually you will arrive at the place where
6710something undesirable happens, and you will see which parts of the
6711grammar are to blame.
bfa74976
RS
6712
6713The parser file is a C program and you can use C debuggers on it, but it's
6714not easy to interpret what it is doing. The parser function is a
6715finite-state machine interpreter, and aside from the actions it executes
6716the same code over and over. Only the values of variables show where in
6717the grammar it is working.
6718
6719@findex YYPRINT
6720The debugging information normally gives the token type of each token
6721read, but not its semantic value. You can optionally define a macro
6722named @code{YYPRINT} to provide a way to print the value. If you define
6723@code{YYPRINT}, it should take three arguments. The parser will pass a
6724standard I/O stream, the numeric code for the token type, and the token
6725value (from @code{yylval}).
6726
6727Here is an example of @code{YYPRINT} suitable for the multi-function
6728calculator (@pxref{Mfcalc Decl, ,Declarations for @code{mfcalc}}):
6729
6730@smallexample
38a92d50
PE
6731%@{
6732 static void print_token_value (FILE *, int, YYSTYPE);
6733 #define YYPRINT(file, type, value) print_token_value (file, type, value)
6734%@}
6735
6736@dots{} %% @dots{} %% @dots{}
bfa74976
RS
6737
6738static void
831d3c99 6739print_token_value (FILE *file, int type, YYSTYPE value)
bfa74976
RS
6740@{
6741 if (type == VAR)
d3c4e709 6742 fprintf (file, "%s", value.tptr->name);
bfa74976 6743 else if (type == NUM)
d3c4e709 6744 fprintf (file, "%d", value.val);
bfa74976
RS
6745@}
6746@end smallexample
6747
ec3bc396
AD
6748@c ================================================= Invoking Bison
6749
342b8b6e 6750@node Invocation
bfa74976
RS
6751@chapter Invoking Bison
6752@cindex invoking Bison
6753@cindex Bison invocation
6754@cindex options for invoking Bison
6755
6756The usual way to invoke Bison is as follows:
6757
6758@example
6759bison @var{infile}
6760@end example
6761
6762Here @var{infile} is the grammar file name, which usually ends in
6763@samp{.y}. The parser file's name is made by replacing the @samp{.y}
fa4d969f
PE
6764with @samp{.tab.c} and removing any leading directory. Thus, the
6765@samp{bison foo.y} file name yields
6766@file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields
6767@file{foo.tab.c}. It's also possible, in case you are writing
79282c6c 6768C++ code instead of C in your grammar file, to name it @file{foo.ypp}
72d2299c
PE
6769or @file{foo.y++}. Then, the output files will take an extension like
6770the given one as input (respectively @file{foo.tab.cpp} and
6771@file{foo.tab.c++}).
fa4d969f 6772This feature takes effect with all options that manipulate file names like
234a3be3
AD
6773@samp{-o} or @samp{-d}.
6774
6775For example :
6776
6777@example
6778bison -d @var{infile.yxx}
6779@end example
84163231 6780@noindent
72d2299c 6781will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
234a3be3
AD
6782
6783@example
b56471a6 6784bison -d -o @var{output.c++} @var{infile.y}
234a3be3 6785@end example
84163231 6786@noindent
234a3be3
AD
6787will produce @file{output.c++} and @file{outfile.h++}.
6788
397ec073
PE
6789For compatibility with @acronym{POSIX}, the standard Bison
6790distribution also contains a shell script called @command{yacc} that
6791invokes Bison with the @option{-y} option.
6792
bfa74976 6793@menu
13863333 6794* Bison Options:: All the options described in detail,
c827f760 6795 in alphabetical order by short options.
bfa74976 6796* Option Cross Key:: Alphabetical list of long options.
93dd49ab 6797* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
bfa74976
RS
6798@end menu
6799
342b8b6e 6800@node Bison Options
bfa74976
RS
6801@section Bison Options
6802
6803Bison supports both traditional single-letter options and mnemonic long
6804option names. Long option names are indicated with @samp{--} instead of
6805@samp{-}. Abbreviations for option names are allowed as long as they
6806are unique. When a long option takes an argument, like
6807@samp{--file-prefix}, connect the option name and the argument with
6808@samp{=}.
6809
6810Here is a list of options that can be used with Bison, alphabetized by
6811short option. It is followed by a cross key alphabetized by long
6812option.
6813
89cab50d
AD
6814@c Please, keep this ordered as in `bison --help'.
6815@noindent
6816Operations modes:
6817@table @option
6818@item -h
6819@itemx --help
6820Print a summary of the command-line options to Bison and exit.
bfa74976 6821
89cab50d
AD
6822@item -V
6823@itemx --version
6824Print the version number of Bison and exit.
bfa74976 6825
f7ab6a50
PE
6826@item --print-localedir
6827Print the name of the directory containing locale-dependent data.
6828
89cab50d
AD
6829@item -y
6830@itemx --yacc
54662697
PE
6831Act more like the traditional Yacc command. This can cause
6832different diagnostics to be generated, and may change behavior in
6833other minor ways. Most importantly, imitate Yacc's output
6834file name conventions, so that the parser output file is called
89cab50d 6835@file{y.tab.c}, and the other outputs are called @file{y.output} and
54662697 6836@file{y.tab.h}. Thus, the following shell script can substitute
397ec073
PE
6837for Yacc, and the Bison distribution contains such a script for
6838compatibility with @acronym{POSIX}:
bfa74976 6839
89cab50d 6840@example
397ec073 6841#! /bin/sh
26e06a21 6842bison -y "$@@"
89cab50d 6843@end example
54662697
PE
6844
6845The @option{-y}/@option{--yacc} option is intended for use with
6846traditional Yacc grammars. If your grammar uses a Bison extension
6847like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
6848this option is specified.
6849
89cab50d
AD
6850@end table
6851
6852@noindent
6853Tuning the parser:
6854
6855@table @option
cd5bd6ac
AD
6856@item -S @var{file}
6857@itemx --skeleton=@var{file}
6858Specify the skeleton to use. You probably don't need this option unless
6859you are developing Bison.
6860
89cab50d
AD
6861@item -t
6862@itemx --debug
4947ebdb
PE
6863In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
6864already defined, so that the debugging facilities are compiled.
ec3bc396 6865@xref{Tracing, ,Tracing Your Parser}.
89cab50d
AD
6866
6867@item --locations
d8988b2f 6868Pretend that @code{%locations} was specified. @xref{Decl Summary}.
89cab50d
AD
6869
6870@item -p @var{prefix}
6871@itemx --name-prefix=@var{prefix}
d8988b2f
AD
6872Pretend that @code{%name-prefix="@var{prefix}"} was specified.
6873@xref{Decl Summary}.
bfa74976
RS
6874
6875@item -l
6876@itemx --no-lines
6877Don't put any @code{#line} preprocessor commands in the parser file.
6878Ordinarily Bison puts them in the parser file so that the C compiler
6879and debuggers will associate errors with your source file, the
6880grammar file. This option causes them to associate errors with the
95e742f7 6881parser file, treating it as an independent source file in its own right.
bfa74976 6882
931c7513
RS
6883@item -n
6884@itemx --no-parser
d8988b2f 6885Pretend that @code{%no-parser} was specified. @xref{Decl Summary}.
931c7513 6886
89cab50d
AD
6887@item -k
6888@itemx --token-table
d8988b2f 6889Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
89cab50d 6890@end table
bfa74976 6891
89cab50d
AD
6892@noindent
6893Adjust the output:
bfa74976 6894
89cab50d
AD
6895@table @option
6896@item -d
d8988b2f
AD
6897@itemx --defines
6898Pretend that @code{%defines} was specified, i.e., write an extra output
6deb4447 6899file containing macro definitions for the token type names defined in
4bfd5e4e 6900the grammar, as well as a few other declarations. @xref{Decl Summary}.
931c7513 6901
342b8b6e 6902@item --defines=@var{defines-file}
d8988b2f 6903Same as above, but save in the file @var{defines-file}.
342b8b6e 6904
89cab50d
AD
6905@item -b @var{file-prefix}
6906@itemx --file-prefix=@var{prefix}
d8988b2f 6907Pretend that @code{%verbose} was specified, i.e, specify prefix to use
72d2299c 6908for all Bison output file names. @xref{Decl Summary}.
bfa74976 6909
ec3bc396
AD
6910@item -r @var{things}
6911@itemx --report=@var{things}
6912Write an extra output file containing verbose description of the comma
6913separated list of @var{things} among:
6914
6915@table @code
6916@item state
6917Description of the grammar, conflicts (resolved and unresolved), and
c827f760 6918@acronym{LALR} automaton.
ec3bc396 6919
8dd162d3 6920@item look-ahead
ec3bc396 6921Implies @code{state} and augments the description of the automaton with
8dd162d3 6922each rule's look-ahead set.
ec3bc396
AD
6923
6924@item itemset
6925Implies @code{state} and augments the description of the automaton with
6926the full set of items for each state, instead of its core only.
6927@end table
6928
bfa74976
RS
6929@item -v
6930@itemx --verbose
6deb4447
AD
6931Pretend that @code{%verbose} was specified, i.e, write an extra output
6932file containing verbose descriptions of the grammar and
72d2299c 6933parser. @xref{Decl Summary}.
bfa74976 6934
fa4d969f
PE
6935@item -o @var{file}
6936@itemx --output=@var{file}
6937Specify the @var{file} for the parser file.
bfa74976 6938
fa4d969f 6939The other output files' names are constructed from @var{file} as
d8988b2f 6940described under the @samp{-v} and @samp{-d} options.
342b8b6e
AD
6941
6942@item -g
c827f760
PE
6943Output a @acronym{VCG} definition of the @acronym{LALR}(1) grammar
6944automaton computed by Bison. If the grammar file is @file{foo.y}, the
6945@acronym{VCG} output file will
342b8b6e
AD
6946be @file{foo.vcg}.
6947
6948@item --graph=@var{graph-file}
72d2299c
PE
6949The behavior of @var{--graph} is the same than @samp{-g}. The only
6950difference is that it has an optional argument which is the name of
fa4d969f 6951the output graph file.
bfa74976
RS
6952@end table
6953
342b8b6e 6954@node Option Cross Key
bfa74976
RS
6955@section Option Cross Key
6956
6957Here is a list of options, alphabetized by long option, to help you find
6958the corresponding short option.
6959
6960@tex
6961\def\leaderfill{\leaders\hbox to 1em{\hss.\hss}\hfill}
6962
6963{\tt
6964\line{ --debug \leaderfill -t}
6965\line{ --defines \leaderfill -d}
6966\line{ --file-prefix \leaderfill -b}
342b8b6e 6967\line{ --graph \leaderfill -g}
ff51d159 6968\line{ --help \leaderfill -h}
bfa74976
RS
6969\line{ --name-prefix \leaderfill -p}
6970\line{ --no-lines \leaderfill -l}
931c7513 6971\line{ --no-parser \leaderfill -n}
d8988b2f 6972\line{ --output \leaderfill -o}
f7ab6a50 6973\line{ --print-localedir}
931c7513 6974\line{ --token-table \leaderfill -k}
bfa74976
RS
6975\line{ --verbose \leaderfill -v}
6976\line{ --version \leaderfill -V}
6977\line{ --yacc \leaderfill -y}
6978}
6979@end tex
6980
6981@ifinfo
6982@example
6983--debug -t
342b8b6e 6984--defines=@var{defines-file} -d
bfa74976 6985--file-prefix=@var{prefix} -b @var{file-prefix}
342b8b6e 6986--graph=@var{graph-file} -d
ff51d159 6987--help -h
931c7513 6988--name-prefix=@var{prefix} -p @var{name-prefix}
bfa74976 6989--no-lines -l
931c7513 6990--no-parser -n
d8988b2f 6991--output=@var{outfile} -o @var{outfile}
f7ab6a50 6992--print-localedir
931c7513 6993--token-table -k
bfa74976
RS
6994--verbose -v
6995--version -V
8c9a50be 6996--yacc -y
bfa74976
RS
6997@end example
6998@end ifinfo
6999
93dd49ab
PE
7000@node Yacc Library
7001@section Yacc Library
7002
7003The Yacc library contains default implementations of the
7004@code{yyerror} and @code{main} functions. These default
7005implementations are normally not useful, but @acronym{POSIX} requires
7006them. To use the Yacc library, link your program with the
7007@option{-ly} option. Note that Bison's implementation of the Yacc
7008library is distributed under the terms of the @acronym{GNU} General
7009Public License (@pxref{Copying}).
7010
7011If you use the Yacc library's @code{yyerror} function, you should
7012declare @code{yyerror} as follows:
7013
7014@example
7015int yyerror (char const *);
7016@end example
7017
7018Bison ignores the @code{int} value returned by this @code{yyerror}.
7019If you use the Yacc library's @code{main} function, your
7020@code{yyparse} function should have the following type signature:
7021
7022@example
7023int yyparse (void);
7024@end example
7025
12545799
AD
7026@c ================================================= C++ Bison
7027
7028@node C++ Language Interface
7029@chapter C++ Language Interface
7030
7031@menu
7032* C++ Parsers:: The interface to generate C++ parser classes
7033* A Complete C++ Example:: Demonstrating their use
7034@end menu
7035
7036@node C++ Parsers
7037@section C++ Parsers
7038
7039@menu
7040* C++ Bison Interface:: Asking for C++ parser generation
7041* C++ Semantic Values:: %union vs. C++
7042* C++ Location Values:: The position and location classes
7043* C++ Parser Interface:: Instantiating and running the parser
7044* C++ Scanner Interface:: Exchanges between yylex and parse
7045@end menu
7046
7047@node C++ Bison Interface
7048@subsection C++ Bison Interface
7049@c - %skeleton "lalr1.cc"
7050@c - Always pure
7051@c - initial action
7052
e054b190 7053The C++ parser @acronym{LALR}(1) skeleton is named @file{lalr1.cc}. To select
12545799
AD
7054it, you may either pass the option @option{--skeleton=lalr1.cc} to
7055Bison, or include the directive @samp{%skeleton "lalr1.cc"} in the
7056grammar preamble. When run, @command{bison} will create several
7057files:
7058@table @file
7059@item position.hh
7060@itemx location.hh
7061The definition of the classes @code{position} and @code{location},
7062used for location tracking. @xref{C++ Location Values}.
7063
7064@item stack.hh
7065An auxiliary class @code{stack} used by the parser.
7066
fa4d969f
PE
7067@item @var{file}.hh
7068@itemx @var{file}.cc
12545799 7069The declaration and implementation of the C++ parser class.
fa4d969f 7070@var{file} is the name of the output file. It follows the same
12545799
AD
7071rules as with regular C parsers.
7072
fa4d969f 7073Note that @file{@var{file}.hh} is @emph{mandatory}, the C++ cannot
12545799
AD
7074work without the parser class declaration. Therefore, you must either
7075pass @option{-d}/@option{--defines} to @command{bison}, or use the
7076@samp{%defines} directive.
7077@end table
7078
7079All these files are documented using Doxygen; run @command{doxygen}
7080for a complete and accurate documentation.
7081
7082@node C++ Semantic Values
7083@subsection C++ Semantic Values
7084@c - No objects in unions
7085@c - YSTYPE
7086@c - Printer and destructor
7087
7088The @code{%union} directive works as for C, see @ref{Union Decl, ,The
7089Collection of Value Types}. In particular it produces a genuine
7090@code{union}@footnote{In the future techniques to allow complex types
fb9712a9
AD
7091within pseudo-unions (similar to Boost variants) might be implemented to
7092alleviate these issues.}, which have a few specific features in C++.
12545799
AD
7093@itemize @minus
7094@item
fb9712a9
AD
7095The type @code{YYSTYPE} is defined but its use is discouraged: rather
7096you should refer to the parser's encapsulated type
7097@code{yy::parser::semantic_type}.
12545799
AD
7098@item
7099Non POD (Plain Old Data) types cannot be used. C++ forbids any
7100instance of classes with constructors in unions: only @emph{pointers}
7101to such objects are allowed.
7102@end itemize
7103
7104Because objects have to be stored via pointers, memory is not
7105reclaimed automatically: using the @code{%destructor} directive is the
7106only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
7107Symbols}.
7108
7109
7110@node C++ Location Values
7111@subsection C++ Location Values
7112@c - %locations
7113@c - class Position
7114@c - class Location
b47dbebe 7115@c - %define "filename_type" "const symbol::Symbol"
12545799
AD
7116
7117When the directive @code{%locations} is used, the C++ parser supports
7118location tracking, see @ref{Locations, , Locations Overview}. Two
7119auxiliary classes define a @code{position}, a single point in a file,
7120and a @code{location}, a range composed of a pair of
7121@code{position}s (possibly spanning several files).
7122
fa4d969f 7123@deftypemethod {position} {std::string*} file
12545799
AD
7124The name of the file. It will always be handled as a pointer, the
7125parser will never duplicate nor deallocate it. As an experimental
7126feature you may change it to @samp{@var{type}*} using @samp{%define
b47dbebe 7127"filename_type" "@var{type}"}.
12545799
AD
7128@end deftypemethod
7129
7130@deftypemethod {position} {unsigned int} line
7131The line, starting at 1.
7132@end deftypemethod
7133
7134@deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
7135Advance by @var{height} lines, resetting the column number.
7136@end deftypemethod
7137
7138@deftypemethod {position} {unsigned int} column
7139The column, starting at 0.
7140@end deftypemethod
7141
7142@deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
7143Advance by @var{width} columns, without changing the line number.
7144@end deftypemethod
7145
7146@deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
7147@deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
7148@deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
7149@deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
7150Various forms of syntactic sugar for @code{columns}.
7151@end deftypemethod
7152
7153@deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
7154Report @var{p} on @var{o} like this:
fa4d969f
PE
7155@samp{@var{file}:@var{line}.@var{column}}, or
7156@samp{@var{line}.@var{column}} if @var{file} is null.
12545799
AD
7157@end deftypemethod
7158
7159@deftypemethod {location} {position} begin
7160@deftypemethodx {location} {position} end
7161The first, inclusive, position of the range, and the first beyond.
7162@end deftypemethod
7163
7164@deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
7165@deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
7166Advance the @code{end} position.
7167@end deftypemethod
7168
7169@deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
7170@deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
7171@deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
7172Various forms of syntactic sugar.
7173@end deftypemethod
7174
7175@deftypemethod {location} {void} step ()
7176Move @code{begin} onto @code{end}.
7177@end deftypemethod
7178
7179
7180@node C++ Parser Interface
7181@subsection C++ Parser Interface
7182@c - define parser_class_name
7183@c - Ctor
7184@c - parse, error, set_debug_level, debug_level, set_debug_stream,
7185@c debug_stream.
7186@c - Reporting errors
7187
7188The output files @file{@var{output}.hh} and @file{@var{output}.cc}
7189declare and define the parser class in the namespace @code{yy}. The
7190class name defaults to @code{parser}, but may be changed using
7191@samp{%define "parser_class_name" "@var{name}"}. The interface of
9d9b8b70 7192this class is detailed below. It can be extended using the
12545799
AD
7193@code{%parse-param} feature: its semantics is slightly changed since
7194it describes an additional member of the parser class, and an
7195additional argument for its constructor.
7196
8a0adb01
AD
7197@defcv {Type} {parser} {semantic_value_type}
7198@defcvx {Type} {parser} {location_value_type}
12545799 7199The types for semantics value and locations.
8a0adb01 7200@end defcv
12545799
AD
7201
7202@deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
7203Build a new parser object. There are no arguments by default, unless
7204@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
7205@end deftypemethod
7206
7207@deftypemethod {parser} {int} parse ()
7208Run the syntactic analysis, and return 0 on success, 1 otherwise.
7209@end deftypemethod
7210
7211@deftypemethod {parser} {std::ostream&} debug_stream ()
7212@deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
7213Get or set the stream used for tracing the parsing. It defaults to
7214@code{std::cerr}.
7215@end deftypemethod
7216
7217@deftypemethod {parser} {debug_level_type} debug_level ()
7218@deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
7219Get or set the tracing level. Currently its value is either 0, no trace,
9d9b8b70 7220or nonzero, full tracing.
12545799
AD
7221@end deftypemethod
7222
7223@deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
7224The definition for this member function must be supplied by the user:
7225the parser uses it to report a parser error occurring at @var{l},
7226described by @var{m}.
7227@end deftypemethod
7228
7229
7230@node C++ Scanner Interface
7231@subsection C++ Scanner Interface
7232@c - prefix for yylex.
7233@c - Pure interface to yylex
7234@c - %lex-param
7235
7236The parser invokes the scanner by calling @code{yylex}. Contrary to C
7237parsers, C++ parsers are always pure: there is no point in using the
7238@code{%pure-parser} directive. Therefore the interface is as follows.
7239
7240@deftypemethod {parser} {int} yylex (semantic_value_type& @var{yylval}, location_type& @var{yylloc}, @var{type1} @var{arg1}, ...)
7241Return the next token. Its type is the return value, its semantic
7242value and location being @var{yylval} and @var{yylloc}. Invocations of
7243@samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
7244@end deftypemethod
7245
7246
7247@node A Complete C++ Example
7248@section A Complete C++ Example
7249
7250This section demonstrates the use of a C++ parser with a simple but
7251complete example. This example should be available on your system,
7252ready to compile, in the directory @dfn{../bison/examples/calc++}. It
7253focuses on the use of Bison, therefore the design of the various C++
7254classes is very naive: no accessors, no encapsulation of members etc.
7255We will use a Lex scanner, and more precisely, a Flex scanner, to
7256demonstrate the various interaction. A hand written scanner is
7257actually easier to interface with.
7258
7259@menu
7260* Calc++ --- C++ Calculator:: The specifications
7261* Calc++ Parsing Driver:: An active parsing context
7262* Calc++ Parser:: A parser class
7263* Calc++ Scanner:: A pure C++ Flex scanner
7264* Calc++ Top Level:: Conducting the band
7265@end menu
7266
7267@node Calc++ --- C++ Calculator
7268@subsection Calc++ --- C++ Calculator
7269
7270Of course the grammar is dedicated to arithmetics, a single
9d9b8b70 7271expression, possibly preceded by variable assignments. An
12545799
AD
7272environment containing possibly predefined variables such as
7273@code{one} and @code{two}, is exchanged with the parser. An example
7274of valid input follows.
7275
7276@example
7277three := 3
7278seven := one + two * three
7279seven * seven
7280@end example
7281
7282@node Calc++ Parsing Driver
7283@subsection Calc++ Parsing Driver
7284@c - An env
7285@c - A place to store error messages
7286@c - A place for the result
7287
7288To support a pure interface with the parser (and the scanner) the
7289technique of the ``parsing context'' is convenient: a structure
7290containing all the data to exchange. Since, in addition to simply
7291launch the parsing, there are several auxiliary tasks to execute (open
7292the file for parsing, instantiate the parser etc.), we recommend
7293transforming the simple parsing context structure into a fully blown
7294@dfn{parsing driver} class.
7295
7296The declaration of this driver class, @file{calc++-driver.hh}, is as
7297follows. The first part includes the CPP guard and imports the
fb9712a9
AD
7298required standard library components, and the declaration of the parser
7299class.
12545799 7300
1c59e0a1 7301@comment file: calc++-driver.hh
12545799
AD
7302@example
7303#ifndef CALCXX_DRIVER_HH
7304# define CALCXX_DRIVER_HH
7305# include <string>
7306# include <map>
fb9712a9 7307# include "calc++-parser.hh"
12545799
AD
7308@end example
7309
12545799
AD
7310
7311@noindent
7312Then comes the declaration of the scanning function. Flex expects
7313the signature of @code{yylex} to be defined in the macro
7314@code{YY_DECL}, and the C++ parser expects it to be declared. We can
7315factor both as follows.
1c59e0a1
AD
7316
7317@comment file: calc++-driver.hh
12545799
AD
7318@example
7319// Announce to Flex the prototype we want for lexing function, ...
1c59e0a1 7320# define YY_DECL \
fb9712a9
AD
7321 int yylex (yy::calcxx_parser::semantic_type* yylval, \
7322 yy::calcxx_parser::location_type* yylloc, \
7323 calcxx_driver& driver)
12545799
AD
7324// ... and declare it for the parser's sake.
7325YY_DECL;
7326@end example
7327
7328@noindent
7329The @code{calcxx_driver} class is then declared with its most obvious
7330members.
7331
1c59e0a1 7332@comment file: calc++-driver.hh
12545799
AD
7333@example
7334// Conducting the whole scanning and parsing of Calc++.
7335class calcxx_driver
7336@{
7337public:
7338 calcxx_driver ();
7339 virtual ~calcxx_driver ();
7340
7341 std::map<std::string, int> variables;
7342
7343 int result;
7344@end example
7345
7346@noindent
7347To encapsulate the coordination with the Flex scanner, it is useful to
7348have two members function to open and close the scanning phase.
7349members.
7350
1c59e0a1 7351@comment file: calc++-driver.hh
12545799
AD
7352@example
7353 // Handling the scanner.
7354 void scan_begin ();
7355 void scan_end ();
7356 bool trace_scanning;
7357@end example
7358
7359@noindent
7360Similarly for the parser itself.
7361
1c59e0a1 7362@comment file: calc++-driver.hh
12545799
AD
7363@example
7364 // Handling the parser.
7365 void parse (const std::string& f);
7366 std::string file;
7367 bool trace_parsing;
7368@end example
7369
7370@noindent
7371To demonstrate pure handling of parse errors, instead of simply
7372dumping them on the standard error output, we will pass them to the
7373compiler driver using the following two member functions. Finally, we
7374close the class declaration and CPP guard.
7375
1c59e0a1 7376@comment file: calc++-driver.hh
12545799
AD
7377@example
7378 // Error handling.
7379 void error (const yy::location& l, const std::string& m);
7380 void error (const std::string& m);
7381@};
7382#endif // ! CALCXX_DRIVER_HH
7383@end example
7384
7385The implementation of the driver is straightforward. The @code{parse}
7386member function deserves some attention. The @code{error} functions
7387are simple stubs, they should actually register the located error
7388messages and set error state.
7389
1c59e0a1 7390@comment file: calc++-driver.cc
12545799
AD
7391@example
7392#include "calc++-driver.hh"
7393#include "calc++-parser.hh"
7394
7395calcxx_driver::calcxx_driver ()
7396 : trace_scanning (false), trace_parsing (false)
7397@{
7398 variables["one"] = 1;
7399 variables["two"] = 2;
7400@}
7401
7402calcxx_driver::~calcxx_driver ()
7403@{
7404@}
7405
7406void
7407calcxx_driver::parse (const std::string &f)
7408@{
7409 file = f;
7410 scan_begin ();
7411 yy::calcxx_parser parser (*this);
7412 parser.set_debug_level (trace_parsing);
7413 parser.parse ();
7414 scan_end ();
7415@}
7416
7417void
7418calcxx_driver::error (const yy::location& l, const std::string& m)
7419@{
7420 std::cerr << l << ": " << m << std::endl;
7421@}
7422
7423void
7424calcxx_driver::error (const std::string& m)
7425@{
7426 std::cerr << m << std::endl;
7427@}
7428@end example
7429
7430@node Calc++ Parser
7431@subsection Calc++ Parser
7432
b50d2359
AD
7433The parser definition file @file{calc++-parser.yy} starts by asking for
7434the C++ LALR(1) skeleton, the creation of the parser header file, and
7435specifies the name of the parser class. Because the C++ skeleton
7436changed several times, it is safer to require the version you designed
7437the grammar for.
1c59e0a1
AD
7438
7439@comment file: calc++-parser.yy
12545799
AD
7440@example
7441%skeleton "lalr1.cc" /* -*- C++ -*- */
b50d2359 7442%require "2.1a"
12545799 7443%defines
fb9712a9
AD
7444%define "parser_class_name" "calcxx_parser"
7445@end example
7446
7447@noindent
7448Then come the declarations/inclusions needed to define the
7449@code{%union}. Because the parser uses the parsing driver and
7450reciprocally, both cannot include the header of the other. Because the
7451driver's header needs detailed knowledge about the parser class (in
7452particular its inner types), it is the parser's header which will simply
7453use a forward declaration of the driver.
7454
7455@comment file: calc++-parser.yy
7456@example
12545799
AD
7457%@{
7458# include <string>
fb9712a9 7459class calcxx_driver;
12545799
AD
7460%@}
7461@end example
7462
7463@noindent
7464The driver is passed by reference to the parser and to the scanner.
7465This provides a simple but effective pure interface, not relying on
7466global variables.
7467
1c59e0a1 7468@comment file: calc++-parser.yy
12545799
AD
7469@example
7470// The parsing context.
7471%parse-param @{ calcxx_driver& driver @}
7472%lex-param @{ calcxx_driver& driver @}
7473@end example
7474
7475@noindent
7476Then we request the location tracking feature, and initialize the
7477first location's file name. Afterwards new locations are computed
7478relatively to the previous locations: the file name will be
7479automatically propagated.
7480
1c59e0a1 7481@comment file: calc++-parser.yy
12545799
AD
7482@example
7483%locations
7484%initial-action
7485@{
7486 // Initialize the initial location.
b47dbebe 7487 @@$.begin.filename = @@$.end.filename = &driver.file;
12545799
AD
7488@};
7489@end example
7490
7491@noindent
7492Use the two following directives to enable parser tracing and verbose
7493error messages.
7494
1c59e0a1 7495@comment file: calc++-parser.yy
12545799
AD
7496@example
7497%debug
7498%error-verbose
7499@end example
7500
7501@noindent
7502Semantic values cannot use ``real'' objects, but only pointers to
7503them.
7504
1c59e0a1 7505@comment file: calc++-parser.yy
12545799
AD
7506@example
7507// Symbols.
7508%union
7509@{
7510 int ival;
7511 std::string *sval;
7512@};
7513@end example
7514
fb9712a9
AD
7515@noindent
7516The code between @samp{%@{} and @samp{%@}} after the introduction of the
7517@samp{%union} is output in the @file{*.cc} file; it needs detailed
7518knowledge about the driver.
7519
7520@comment file: calc++-parser.yy
7521@example
7522%@{
7523# include "calc++-driver.hh"
7524%@}
7525@end example
7526
7527
12545799
AD
7528@noindent
7529The token numbered as 0 corresponds to end of file; the following line
7530allows for nicer error messages referring to ``end of file'' instead
7531of ``$end''. Similarly user friendly named are provided for each
7532symbol. Note that the tokens names are prefixed by @code{TOKEN_} to
7533avoid name clashes.
7534
1c59e0a1 7535@comment file: calc++-parser.yy
12545799 7536@example
fb9712a9
AD
7537%token END 0 "end of file"
7538%token ASSIGN ":="
7539%token <sval> IDENTIFIER "identifier"
7540%token <ival> NUMBER "number"
7541%type <ival> exp "expression"
12545799
AD
7542@end example
7543
7544@noindent
7545To enable memory deallocation during error recovery, use
7546@code{%destructor}.
7547
287c78f6 7548@c FIXME: Document %printer, and mention that it takes a braced-code operand.
1c59e0a1 7549@comment file: calc++-parser.yy
12545799
AD
7550@example
7551%printer @{ debug_stream () << *$$; @} "identifier"
7552%destructor @{ delete $$; @} "identifier"
7553
7554%printer @{ debug_stream () << $$; @} "number" "expression"
7555@end example
7556
7557@noindent
7558The grammar itself is straightforward.
7559
1c59e0a1 7560@comment file: calc++-parser.yy
12545799
AD
7561@example
7562%%
7563%start unit;
7564unit: assignments exp @{ driver.result = $2; @};
7565
7566assignments: assignments assignment @{@}
9d9b8b70 7567 | /* Nothing. */ @{@};
12545799 7568
fb9712a9 7569assignment: "identifier" ":=" exp @{ driver.variables[*$1] = $3; @};
12545799
AD
7570
7571%left '+' '-';
7572%left '*' '/';
7573exp: exp '+' exp @{ $$ = $1 + $3; @}
7574 | exp '-' exp @{ $$ = $1 - $3; @}
7575 | exp '*' exp @{ $$ = $1 * $3; @}
7576 | exp '/' exp @{ $$ = $1 / $3; @}
fb9712a9
AD
7577 | "identifier" @{ $$ = driver.variables[*$1]; @}
7578 | "number" @{ $$ = $1; @};
12545799
AD
7579%%
7580@end example
7581
7582@noindent
7583Finally the @code{error} member function registers the errors to the
7584driver.
7585
1c59e0a1 7586@comment file: calc++-parser.yy
12545799
AD
7587@example
7588void
1c59e0a1
AD
7589yy::calcxx_parser::error (const yy::calcxx_parser::location_type& l,
7590 const std::string& m)
12545799
AD
7591@{
7592 driver.error (l, m);
7593@}
7594@end example
7595
7596@node Calc++ Scanner
7597@subsection Calc++ Scanner
7598
7599The Flex scanner first includes the driver declaration, then the
7600parser's to get the set of defined tokens.
7601
1c59e0a1 7602@comment file: calc++-scanner.ll
12545799
AD
7603@example
7604%@{ /* -*- C++ -*- */
04098407
PE
7605# include <cstdlib>
7606# include <errno.h>
7607# include <limits.h>
12545799
AD
7608# include <string>
7609# include "calc++-driver.hh"
7610# include "calc++-parser.hh"
7870f699
PE
7611/* Work around a bug in flex 2.5.31. See Debian bug 333231
7612 <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */
7613# undef yywrap
7614# define yywrap() 1
12545799
AD
7615%@}
7616@end example
7617
7618@noindent
7619Because there is no @code{#include}-like feature we don't need
7620@code{yywrap}, we don't need @code{unput} either, and we parse an
7621actual file, this is not an interactive session with the user.
7622Finally we enable the scanner tracing features.
7623
1c59e0a1 7624@comment file: calc++-scanner.ll
12545799
AD
7625@example
7626%option noyywrap nounput batch debug
7627@end example
7628
7629@noindent
7630Abbreviations allow for more readable rules.
7631
1c59e0a1 7632@comment file: calc++-scanner.ll
12545799
AD
7633@example
7634id [a-zA-Z][a-zA-Z_0-9]*
7635int [0-9]+
7636blank [ \t]
7637@end example
7638
7639@noindent
9d9b8b70 7640The following paragraph suffices to track locations accurately. Each
12545799
AD
7641time @code{yylex} is invoked, the begin position is moved onto the end
7642position. Then when a pattern is matched, the end position is
7643advanced of its width. In case it matched ends of lines, the end
7644cursor is adjusted, and each time blanks are matched, the begin cursor
7645is moved onto the end cursor to effectively ignore the blanks
7646preceding tokens. Comments would be treated equally.
7647
1c59e0a1 7648@comment file: calc++-scanner.ll
12545799 7649@example
828c373b
AD
7650%@{
7651# define YY_USER_ACTION yylloc->columns (yyleng);
7652%@}
12545799
AD
7653%%
7654%@{
7655 yylloc->step ();
12545799
AD
7656%@}
7657@{blank@}+ yylloc->step ();
7658[\n]+ yylloc->lines (yyleng); yylloc->step ();
7659@end example
7660
7661@noindent
fb9712a9
AD
7662The rules are simple, just note the use of the driver to report errors.
7663It is convenient to use a typedef to shorten
7664@code{yy::calcxx_parser::token::identifier} into
9d9b8b70 7665@code{token::identifier} for instance.
12545799 7666
1c59e0a1 7667@comment file: calc++-scanner.ll
12545799 7668@example
fb9712a9
AD
7669%@{
7670 typedef yy::calcxx_parser::token token;
7671%@}
7672
12545799 7673[-+*/] return yytext[0];
fb9712a9 7674":=" return token::ASSIGN;
04098407
PE
7675@{int@} @{
7676 errno = 0;
7677 long n = strtol (yytext, NULL, 10);
7678 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
7679 driver.error (*yylloc, "integer is out of range");
7680 yylval->ival = n;
fb9712a9 7681 return token::NUMBER;
04098407 7682@}
fb9712a9 7683@{id@} yylval->sval = new std::string (yytext); return token::IDENTIFIER;
12545799
AD
7684. driver.error (*yylloc, "invalid character");
7685%%
7686@end example
7687
7688@noindent
7689Finally, because the scanner related driver's member function depend
7690on the scanner's data, it is simpler to implement them in this file.
7691
1c59e0a1 7692@comment file: calc++-scanner.ll
12545799
AD
7693@example
7694void
7695calcxx_driver::scan_begin ()
7696@{
7697 yy_flex_debug = trace_scanning;
7698 if (!(yyin = fopen (file.c_str (), "r")))
7699 error (std::string ("cannot open ") + file);
7700@}
7701
7702void
7703calcxx_driver::scan_end ()
7704@{
7705 fclose (yyin);
7706@}
7707@end example
7708
7709@node Calc++ Top Level
7710@subsection Calc++ Top Level
7711
7712The top level file, @file{calc++.cc}, poses no problem.
7713
1c59e0a1 7714@comment file: calc++.cc
12545799
AD
7715@example
7716#include <iostream>
7717#include "calc++-driver.hh"
7718
7719int
fa4d969f 7720main (int argc, char *argv[])
12545799
AD
7721@{
7722 calcxx_driver driver;
7723 for (++argv; argv[0]; ++argv)
7724 if (*argv == std::string ("-p"))
7725 driver.trace_parsing = true;
7726 else if (*argv == std::string ("-s"))
7727 driver.trace_scanning = true;
7728 else
7729 @{
7730 driver.parse (*argv);
7731 std::cout << driver.result << std::endl;
7732 @}
7733@}
7734@end example
7735
7736@c ================================================= FAQ
d1a1114f
AD
7737
7738@node FAQ
7739@chapter Frequently Asked Questions
7740@cindex frequently asked questions
7741@cindex questions
7742
7743Several questions about Bison come up occasionally. Here some of them
7744are addressed.
7745
7746@menu
1a059451 7747* Memory Exhausted:: Breaking the Stack Limits
e64fec0a 7748* How Can I Reset the Parser:: @code{yyparse} Keeps some State
fef4cb51 7749* Strings are Destroyed:: @code{yylval} Loses Track of Strings
2fa09258 7750* Implementing Gotos/Loops:: Control Flow in the Calculator
d1a1114f
AD
7751@end menu
7752
1a059451
PE
7753@node Memory Exhausted
7754@section Memory Exhausted
d1a1114f
AD
7755
7756@display
1a059451 7757My parser returns with error with a @samp{memory exhausted}
d1a1114f
AD
7758message. What can I do?
7759@end display
7760
7761This question is already addressed elsewhere, @xref{Recursion,
7762,Recursive Rules}.
7763
e64fec0a
PE
7764@node How Can I Reset the Parser
7765@section How Can I Reset the Parser
5b066063 7766
0e14ad77
PE
7767The following phenomenon has several symptoms, resulting in the
7768following typical questions:
5b066063
AD
7769
7770@display
7771I invoke @code{yyparse} several times, and on correct input it works
7772properly; but when a parse error is found, all the other calls fail
0e14ad77 7773too. How can I reset the error flag of @code{yyparse}?
5b066063
AD
7774@end display
7775
7776@noindent
7777or
7778
7779@display
0e14ad77 7780My parser includes support for an @samp{#include}-like feature, in
5b066063
AD
7781which case I run @code{yyparse} from @code{yyparse}. This fails
7782although I did specify I needed a @code{%pure-parser}.
7783@end display
7784
0e14ad77
PE
7785These problems typically come not from Bison itself, but from
7786Lex-generated scanners. Because these scanners use large buffers for
5b066063
AD
7787speed, they might not notice a change of input file. As a
7788demonstration, consider the following source file,
7789@file{first-line.l}:
7790
7791@verbatim
7792%{
7793#include <stdio.h>
7794#include <stdlib.h>
7795%}
7796%%
7797.*\n ECHO; return 1;
7798%%
7799int
0e14ad77 7800yyparse (char const *file)
5b066063
AD
7801{
7802 yyin = fopen (file, "r");
7803 if (!yyin)
7804 exit (2);
fa7e68c3 7805 /* One token only. */
5b066063 7806 yylex ();
0e14ad77 7807 if (fclose (yyin) != 0)
5b066063
AD
7808 exit (3);
7809 return 0;
7810}
7811
7812int
0e14ad77 7813main (void)
5b066063
AD
7814{
7815 yyparse ("input");
7816 yyparse ("input");
7817 return 0;
7818}
7819@end verbatim
7820
7821@noindent
7822If the file @file{input} contains
7823
7824@verbatim
7825input:1: Hello,
7826input:2: World!
7827@end verbatim
7828
7829@noindent
0e14ad77 7830then instead of getting the first line twice, you get:
5b066063
AD
7831
7832@example
7833$ @kbd{flex -ofirst-line.c first-line.l}
7834$ @kbd{gcc -ofirst-line first-line.c -ll}
7835$ @kbd{./first-line}
7836input:1: Hello,
7837input:2: World!
7838@end example
7839
0e14ad77
PE
7840Therefore, whenever you change @code{yyin}, you must tell the
7841Lex-generated scanner to discard its current buffer and switch to the
7842new one. This depends upon your implementation of Lex; see its
7843documentation for more. For Flex, it suffices to call
7844@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
7845Flex-generated scanner needs to read from several input streams to
7846handle features like include files, you might consider using Flex
7847functions like @samp{yy_switch_to_buffer} that manipulate multiple
7848input buffers.
5b066063 7849
b165c324
AD
7850If your Flex-generated scanner uses start conditions (@pxref{Start
7851conditions, , Start conditions, flex, The Flex Manual}), you might
7852also want to reset the scanner's state, i.e., go back to the initial
7853start condition, through a call to @samp{BEGIN (0)}.
7854
fef4cb51
AD
7855@node Strings are Destroyed
7856@section Strings are Destroyed
7857
7858@display
c7e441b4 7859My parser seems to destroy old strings, or maybe it loses track of
fef4cb51
AD
7860them. Instead of reporting @samp{"foo", "bar"}, it reports
7861@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
7862@end display
7863
7864This error is probably the single most frequent ``bug report'' sent to
7865Bison lists, but is only concerned with a misunderstanding of the role
7866of scanner. Consider the following Lex code:
7867
7868@verbatim
7869%{
7870#include <stdio.h>
7871char *yylval = NULL;
7872%}
7873%%
7874.* yylval = yytext; return 1;
7875\n /* IGNORE */
7876%%
7877int
7878main ()
7879{
fa7e68c3 7880 /* Similar to using $1, $2 in a Bison action. */
fef4cb51
AD
7881 char *fst = (yylex (), yylval);
7882 char *snd = (yylex (), yylval);
7883 printf ("\"%s\", \"%s\"\n", fst, snd);
7884 return 0;
7885}
7886@end verbatim
7887
7888If you compile and run this code, you get:
7889
7890@example
7891$ @kbd{flex -osplit-lines.c split-lines.l}
7892$ @kbd{gcc -osplit-lines split-lines.c -ll}
7893$ @kbd{printf 'one\ntwo\n' | ./split-lines}
7894"one
7895two", "two"
7896@end example
7897
7898@noindent
7899this is because @code{yytext} is a buffer provided for @emph{reading}
7900in the action, but if you want to keep it, you have to duplicate it
7901(e.g., using @code{strdup}). Note that the output may depend on how
7902your implementation of Lex handles @code{yytext}. For instance, when
7903given the Lex compatibility option @option{-l} (which triggers the
7904option @samp{%array}) Flex generates a different behavior:
7905
7906@example
7907$ @kbd{flex -l -osplit-lines.c split-lines.l}
7908$ @kbd{gcc -osplit-lines split-lines.c -ll}
7909$ @kbd{printf 'one\ntwo\n' | ./split-lines}
7910"two", "two"
7911@end example
7912
7913
2fa09258
AD
7914@node Implementing Gotos/Loops
7915@section Implementing Gotos/Loops
a06ea4aa
AD
7916
7917@display
7918My simple calculator supports variables, assignments, and functions,
2fa09258 7919but how can I implement gotos, or loops?
a06ea4aa
AD
7920@end display
7921
7922Although very pedagogical, the examples included in the document blur
a1c84f45 7923the distinction to make between the parser---whose job is to recover
a06ea4aa 7924the structure of a text and to transmit it to subsequent modules of
a1c84f45 7925the program---and the processing (such as the execution) of this
a06ea4aa
AD
7926structure. This works well with so called straight line programs,
7927i.e., precisely those that have a straightforward execution model:
7928execute simple instructions one after the others.
7929
7930@cindex abstract syntax tree
7931@cindex @acronym{AST}
7932If you want a richer model, you will probably need to use the parser
7933to construct a tree that does represent the structure it has
7934recovered; this tree is usually called the @dfn{abstract syntax tree},
7935or @dfn{@acronym{AST}} for short. Then, walking through this tree,
7936traversing it in various ways, will enable treatments such as its
7937execution or its translation, which will result in an interpreter or a
7938compiler.
7939
7940This topic is way beyond the scope of this manual, and the reader is
7941invited to consult the dedicated literature.
7942
7943
7944
d1a1114f
AD
7945@c ================================================= Table of Symbols
7946
342b8b6e 7947@node Table of Symbols
bfa74976
RS
7948@appendix Bison Symbols
7949@cindex Bison symbols, table of
7950@cindex symbols in Bison, table of
7951
18b519c0 7952@deffn {Variable} @@$
3ded9a63 7953In an action, the location of the left-hand side of the rule.
88bce5a2 7954@xref{Locations, , Locations Overview}.
18b519c0 7955@end deffn
3ded9a63 7956
18b519c0 7957@deffn {Variable} @@@var{n}
3ded9a63
AD
7958In an action, the location of the @var{n}-th symbol of the right-hand
7959side of the rule. @xref{Locations, , Locations Overview}.
18b519c0 7960@end deffn
3ded9a63 7961
18b519c0 7962@deffn {Variable} $$
3ded9a63
AD
7963In an action, the semantic value of the left-hand side of the rule.
7964@xref{Actions}.
18b519c0 7965@end deffn
3ded9a63 7966
18b519c0 7967@deffn {Variable} $@var{n}
3ded9a63
AD
7968In an action, the semantic value of the @var{n}-th symbol of the
7969right-hand side of the rule. @xref{Actions}.
18b519c0 7970@end deffn
3ded9a63 7971
dd8d9022
AD
7972@deffn {Delimiter} %%
7973Delimiter used to separate the grammar rule section from the
7974Bison declarations section or the epilogue.
7975@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
18b519c0 7976@end deffn
bfa74976 7977
dd8d9022
AD
7978@c Don't insert spaces, or check the DVI output.
7979@deffn {Delimiter} %@{@var{code}%@}
7980All code listed between @samp{%@{} and @samp{%@}} is copied directly to
7981the output file uninterpreted. Such code forms the prologue of the input
7982file. @xref{Grammar Outline, ,Outline of a Bison
7983Grammar}.
18b519c0 7984@end deffn
bfa74976 7985
dd8d9022
AD
7986@deffn {Construct} /*@dots{}*/
7987Comment delimiters, as in C.
18b519c0 7988@end deffn
bfa74976 7989
dd8d9022
AD
7990@deffn {Delimiter} :
7991Separates a rule's result from its components. @xref{Rules, ,Syntax of
7992Grammar Rules}.
18b519c0 7993@end deffn
bfa74976 7994
dd8d9022
AD
7995@deffn {Delimiter} ;
7996Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 7997@end deffn
bfa74976 7998
dd8d9022
AD
7999@deffn {Delimiter} |
8000Separates alternate rules for the same result nonterminal.
8001@xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 8002@end deffn
bfa74976 8003
dd8d9022
AD
8004@deffn {Symbol} $accept
8005The predefined nonterminal whose only rule is @samp{$accept: @var{start}
8006$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
8007Start-Symbol}. It cannot be used in the grammar.
18b519c0 8008@end deffn
bfa74976 8009
18b519c0 8010@deffn {Directive} %debug
6deb4447 8011Equip the parser for debugging. @xref{Decl Summary}.
18b519c0 8012@end deffn
6deb4447 8013
91d2c560 8014@ifset defaultprec
22fccf95
PE
8015@deffn {Directive} %default-prec
8016Assign a precedence to rules that lack an explicit @samp{%prec}
8017modifier. @xref{Contextual Precedence, ,Context-Dependent
8018Precedence}.
39a06c25 8019@end deffn
91d2c560 8020@end ifset
39a06c25 8021
18b519c0 8022@deffn {Directive} %defines
6deb4447
AD
8023Bison declaration to create a header file meant for the scanner.
8024@xref{Decl Summary}.
18b519c0 8025@end deffn
6deb4447 8026
18b519c0 8027@deffn {Directive} %destructor
258b75ca 8028Specify how the parser should reclaim the memory associated to
fa7e68c3 8029discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 8030@end deffn
72f889cc 8031
18b519c0 8032@deffn {Directive} %dprec
676385e2 8033Bison declaration to assign a precedence to a rule that is used at parse
c827f760
PE
8034time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
8035@acronym{GLR} Parsers}.
18b519c0 8036@end deffn
676385e2 8037
dd8d9022
AD
8038@deffn {Symbol} $end
8039The predefined token marking the end of the token stream. It cannot be
8040used in the grammar.
8041@end deffn
8042
8043@deffn {Symbol} error
8044A token name reserved for error recovery. This token may be used in
8045grammar rules so as to allow the Bison parser to recognize an error in
8046the grammar without halting the process. In effect, a sentence
8047containing an error may be recognized as valid. On a syntax error, the
8048token @code{error} becomes the current look-ahead token. Actions
8049corresponding to @code{error} are then executed, and the look-ahead
8050token is reset to the token that originally caused the violation.
8051@xref{Error Recovery}.
18d192f0
AD
8052@end deffn
8053
18b519c0 8054@deffn {Directive} %error-verbose
2a8d363a
AD
8055Bison declaration to request verbose, specific error message strings
8056when @code{yyerror} is called.
18b519c0 8057@end deffn
2a8d363a 8058
18b519c0 8059@deffn {Directive} %file-prefix="@var{prefix}"
72d2299c 8060Bison declaration to set the prefix of the output files. @xref{Decl
d8988b2f 8061Summary}.
18b519c0 8062@end deffn
d8988b2f 8063
18b519c0 8064@deffn {Directive} %glr-parser
c827f760
PE
8065Bison declaration to produce a @acronym{GLR} parser. @xref{GLR
8066Parsers, ,Writing @acronym{GLR} Parsers}.
18b519c0 8067@end deffn
676385e2 8068
dd8d9022
AD
8069@deffn {Directive} %initial-action
8070Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
8071@end deffn
8072
18b519c0 8073@deffn {Directive} %left
bfa74976
RS
8074Bison declaration to assign left associativity to token(s).
8075@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 8076@end deffn
bfa74976 8077
feeb0eda 8078@deffn {Directive} %lex-param @{@var{argument-declaration}@}
2a8d363a
AD
8079Bison declaration to specifying an additional parameter that
8080@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
8081for Pure Parsers}.
18b519c0 8082@end deffn
2a8d363a 8083
18b519c0 8084@deffn {Directive} %merge
676385e2 8085Bison declaration to assign a merging function to a rule. If there is a
fae437e8 8086reduce/reduce conflict with a rule having the same merging function, the
676385e2 8087function is applied to the two semantic values to get a single result.
c827f760 8088@xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}.
18b519c0 8089@end deffn
676385e2 8090
18b519c0 8091@deffn {Directive} %name-prefix="@var{prefix}"
72d2299c 8092Bison declaration to rename the external symbols. @xref{Decl Summary}.
18b519c0 8093@end deffn
d8988b2f 8094
91d2c560 8095@ifset defaultprec
22fccf95
PE
8096@deffn {Directive} %no-default-prec
8097Do not assign a precedence to rules that lack an explicit @samp{%prec}
8098modifier. @xref{Contextual Precedence, ,Context-Dependent
8099Precedence}.
8100@end deffn
91d2c560 8101@end ifset
22fccf95 8102
18b519c0 8103@deffn {Directive} %no-lines
931c7513
RS
8104Bison declaration to avoid generating @code{#line} directives in the
8105parser file. @xref{Decl Summary}.
18b519c0 8106@end deffn
931c7513 8107
18b519c0 8108@deffn {Directive} %nonassoc
9d9b8b70 8109Bison declaration to assign nonassociativity to token(s).
bfa74976 8110@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 8111@end deffn
bfa74976 8112
fa4d969f 8113@deffn {Directive} %output="@var{file}"
72d2299c 8114Bison declaration to set the name of the parser file. @xref{Decl
d8988b2f 8115Summary}.
18b519c0 8116@end deffn
d8988b2f 8117
feeb0eda 8118@deffn {Directive} %parse-param @{@var{argument-declaration}@}
2a8d363a
AD
8119Bison declaration to specifying an additional parameter that
8120@code{yyparse} should accept. @xref{Parser Function,, The Parser
8121Function @code{yyparse}}.
18b519c0 8122@end deffn
2a8d363a 8123
18b519c0 8124@deffn {Directive} %prec
bfa74976
RS
8125Bison declaration to assign a precedence to a specific rule.
8126@xref{Contextual Precedence, ,Context-Dependent Precedence}.
18b519c0 8127@end deffn
bfa74976 8128
18b519c0 8129@deffn {Directive} %pure-parser
bfa74976
RS
8130Bison declaration to request a pure (reentrant) parser.
8131@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
18b519c0 8132@end deffn
bfa74976 8133
b50d2359 8134@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
8135Require version @var{version} or higher of Bison. @xref{Require Decl, ,
8136Require a Version of Bison}.
b50d2359
AD
8137@end deffn
8138
18b519c0 8139@deffn {Directive} %right
bfa74976
RS
8140Bison declaration to assign right associativity to token(s).
8141@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 8142@end deffn
bfa74976 8143
18b519c0 8144@deffn {Directive} %start
704a47c4
AD
8145Bison declaration to specify the start symbol. @xref{Start Decl, ,The
8146Start-Symbol}.
18b519c0 8147@end deffn
bfa74976 8148
18b519c0 8149@deffn {Directive} %token
bfa74976
RS
8150Bison declaration to declare token(s) without specifying precedence.
8151@xref{Token Decl, ,Token Type Names}.
18b519c0 8152@end deffn
bfa74976 8153
18b519c0 8154@deffn {Directive} %token-table
931c7513
RS
8155Bison declaration to include a token name table in the parser file.
8156@xref{Decl Summary}.
18b519c0 8157@end deffn
931c7513 8158
18b519c0 8159@deffn {Directive} %type
704a47c4
AD
8160Bison declaration to declare nonterminals. @xref{Type Decl,
8161,Nonterminal Symbols}.
18b519c0 8162@end deffn
bfa74976 8163
dd8d9022
AD
8164@deffn {Symbol} $undefined
8165The predefined token onto which all undefined values returned by
8166@code{yylex} are mapped. It cannot be used in the grammar, rather, use
8167@code{error}.
8168@end deffn
8169
18b519c0 8170@deffn {Directive} %union
bfa74976
RS
8171Bison declaration to specify several possible data types for semantic
8172values. @xref{Union Decl, ,The Collection of Value Types}.
18b519c0 8173@end deffn
bfa74976 8174
dd8d9022
AD
8175@deffn {Macro} YYABORT
8176Macro to pretend that an unrecoverable syntax error has occurred, by
8177making @code{yyparse} return 1 immediately. The error reporting
8178function @code{yyerror} is not called. @xref{Parser Function, ,The
8179Parser Function @code{yyparse}}.
8180@end deffn
3ded9a63 8181
dd8d9022
AD
8182@deffn {Macro} YYACCEPT
8183Macro to pretend that a complete utterance of the language has been
8184read, by making @code{yyparse} return 0 immediately.
8185@xref{Parser Function, ,The Parser Function @code{yyparse}}.
8186@end deffn
bfa74976 8187
dd8d9022
AD
8188@deffn {Macro} YYBACKUP
8189Macro to discard a value from the parser stack and fake a look-ahead
8190token. @xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 8191@end deffn
bfa74976 8192
dd8d9022 8193@deffn {Variable} yychar
32c29292 8194External integer variable that contains the integer value of the
dd8d9022
AD
8195look-ahead token. (In a pure parser, it is a local variable within
8196@code{yyparse}.) Error-recovery rule actions may examine this variable.
8197@xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 8198@end deffn
bfa74976 8199
dd8d9022
AD
8200@deffn {Variable} yyclearin
8201Macro used in error-recovery rule actions. It clears the previous
8202look-ahead token. @xref{Error Recovery}.
18b519c0 8203@end deffn
bfa74976 8204
dd8d9022
AD
8205@deffn {Macro} YYDEBUG
8206Macro to define to equip the parser with tracing code. @xref{Tracing,
8207,Tracing Your Parser}.
18b519c0 8208@end deffn
bfa74976 8209
dd8d9022
AD
8210@deffn {Variable} yydebug
8211External integer variable set to zero by default. If @code{yydebug}
8212is given a nonzero value, the parser will output information on input
8213symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
18b519c0 8214@end deffn
bfa74976 8215
dd8d9022
AD
8216@deffn {Macro} yyerrok
8217Macro to cause parser to recover immediately to its normal mode
8218after a syntax error. @xref{Error Recovery}.
8219@end deffn
8220
8221@deffn {Macro} YYERROR
8222Macro to pretend that a syntax error has just been detected: call
8223@code{yyerror} and then perform normal error recovery if possible
8224(@pxref{Error Recovery}), or (if recovery is impossible) make
8225@code{yyparse} return 1. @xref{Error Recovery}.
8226@end deffn
8227
8228@deffn {Function} yyerror
8229User-supplied function to be called by @code{yyparse} on error.
8230@xref{Error Reporting, ,The Error
8231Reporting Function @code{yyerror}}.
8232@end deffn
8233
8234@deffn {Macro} YYERROR_VERBOSE
8235An obsolete macro that you define with @code{#define} in the prologue
8236to request verbose, specific error message strings
8237when @code{yyerror} is called. It doesn't matter what definition you
8238use for @code{YYERROR_VERBOSE}, just whether you define it. Using
8239@code{%error-verbose} is preferred.
8240@end deffn
8241
8242@deffn {Macro} YYINITDEPTH
8243Macro for specifying the initial size of the parser stack.
1a059451 8244@xref{Memory Management}.
dd8d9022
AD
8245@end deffn
8246
8247@deffn {Function} yylex
8248User-supplied lexical analyzer function, called with no arguments to get
8249the next token. @xref{Lexical, ,The Lexical Analyzer Function
8250@code{yylex}}.
8251@end deffn
8252
8253@deffn {Macro} YYLEX_PARAM
8254An obsolete macro for specifying an extra argument (or list of extra
32c29292 8255arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
dd8d9022
AD
8256macro is deprecated, and is supported only for Yacc like parsers.
8257@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
8258@end deffn
8259
8260@deffn {Variable} yylloc
8261External variable in which @code{yylex} should place the line and column
8262numbers associated with a token. (In a pure parser, it is a local
8263variable within @code{yyparse}, and its address is passed to
32c29292
JD
8264@code{yylex}.)
8265You can ignore this variable if you don't use the @samp{@@} feature in the
8266grammar actions.
8267@xref{Token Locations, ,Textual Locations of Tokens}.
8268In semantic actions, it stores the location of the look-ahead token.
8269@xref{Actions and Locations, ,Actions and Locations}.
dd8d9022
AD
8270@end deffn
8271
8272@deffn {Type} YYLTYPE
8273Data type of @code{yylloc}; by default, a structure with four
8274members. @xref{Location Type, , Data Types of Locations}.
8275@end deffn
8276
8277@deffn {Variable} yylval
8278External variable in which @code{yylex} should place the semantic
8279value associated with a token. (In a pure parser, it is a local
8280variable within @code{yyparse}, and its address is passed to
32c29292
JD
8281@code{yylex}.)
8282@xref{Token Values, ,Semantic Values of Tokens}.
8283In semantic actions, it stores the semantic value of the look-ahead token.
8284@xref{Actions, ,Actions}.
dd8d9022
AD
8285@end deffn
8286
8287@deffn {Macro} YYMAXDEPTH
1a059451
PE
8288Macro for specifying the maximum size of the parser stack. @xref{Memory
8289Management}.
dd8d9022
AD
8290@end deffn
8291
8292@deffn {Variable} yynerrs
8a2800e7 8293Global variable which Bison increments each time it reports a syntax error.
dd8d9022
AD
8294(In a pure parser, it is a local variable within @code{yyparse}.)
8295@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
8296@end deffn
8297
8298@deffn {Function} yyparse
8299The parser function produced by Bison; call this function to start
8300parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
8301@end deffn
8302
8303@deffn {Macro} YYPARSE_PARAM
8304An obsolete macro for specifying the name of a parameter that
8305@code{yyparse} should accept. The use of this macro is deprecated, and
8306is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
8307Conventions for Pure Parsers}.
8308@end deffn
8309
8310@deffn {Macro} YYRECOVERING
8311Macro whose value indicates whether the parser is recovering from a
8312syntax error. @xref{Action Features, ,Special Features for Use in Actions}.
8313@end deffn
8314
8315@deffn {Macro} YYSTACK_USE_ALLOCA
d7e14fc0
PE
8316Macro used to control the use of @code{alloca} when the C
8317@acronym{LALR}(1) parser needs to extend its stacks. If defined to 0,
8318the parser will use @code{malloc} to extend its stacks. If defined to
83191, the parser will use @code{alloca}. Values other than 0 and 1 are
8320reserved for future Bison extensions. If not defined,
8321@code{YYSTACK_USE_ALLOCA} defaults to 0.
8322
55289366 8323In the all-too-common case where your code may run on a host with a
d7e14fc0
PE
8324limited stack and with unreliable stack-overflow checking, you should
8325set @code{YYMAXDEPTH} to a value that cannot possibly result in
8326unchecked stack overflow on any of your target hosts when
8327@code{alloca} is called. You can inspect the code that Bison
8328generates in order to determine the proper numeric values. This will
8329require some expertise in low-level implementation details.
dd8d9022
AD
8330@end deffn
8331
8332@deffn {Type} YYSTYPE
8333Data type of semantic values; @code{int} by default.
8334@xref{Value Type, ,Data Types of Semantic Values}.
18b519c0 8335@end deffn
bfa74976 8336
342b8b6e 8337@node Glossary
bfa74976
RS
8338@appendix Glossary
8339@cindex glossary
8340
8341@table @asis
c827f760
PE
8342@item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'')
8343Formal method of specifying context-free grammars originally proposed
8344by John Backus, and slightly improved by Peter Naur in his 1960-01-02
8345committee document contributing to what became the Algol 60 report.
8346@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
8347
8348@item Context-free grammars
8349Grammars specified as rules that can be applied regardless of context.
8350Thus, if there is a rule which says that an integer can be used as an
8351expression, integers are allowed @emph{anywhere} an expression is
89cab50d
AD
8352permitted. @xref{Language and Grammar, ,Languages and Context-Free
8353Grammars}.
bfa74976
RS
8354
8355@item Dynamic allocation
8356Allocation of memory that occurs during execution, rather than at
8357compile time or on entry to a function.
8358
8359@item Empty string
8360Analogous to the empty set in set theory, the empty string is a
8361character string of length zero.
8362
8363@item Finite-state stack machine
8364A ``machine'' that has discrete states in which it is said to exist at
8365each instant in time. As input to the machine is processed, the
8366machine moves from state to state as specified by the logic of the
8367machine. In the case of the parser, the input is the language being
8368parsed, and the states correspond to various stages in the grammar
c827f760 8369rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976 8370
c827f760 8371@item Generalized @acronym{LR} (@acronym{GLR})
676385e2 8372A parsing algorithm that can handle all context-free grammars, including those
c827f760
PE
8373that are not @acronym{LALR}(1). It resolves situations that Bison's
8374usual @acronym{LALR}(1)
676385e2
PH
8375algorithm cannot by effectively splitting off multiple parsers, trying all
8376possible parsers, and discarding those that fail in the light of additional
c827f760
PE
8377right context. @xref{Generalized LR Parsing, ,Generalized
8378@acronym{LR} Parsing}.
676385e2 8379
bfa74976
RS
8380@item Grouping
8381A language construct that is (in general) grammatically divisible;
c827f760 8382for example, `expression' or `declaration' in C@.
bfa74976
RS
8383@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
8384
8385@item Infix operator
8386An arithmetic operator that is placed between the operands on which it
8387performs some operation.
8388
8389@item Input stream
8390A continuous flow of data between devices or programs.
8391
8392@item Language construct
8393One of the typical usage schemas of the language. For example, one of
8394the constructs of the C language is the @code{if} statement.
8395@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
8396
8397@item Left associativity
8398Operators having left associativity are analyzed from left to right:
8399@samp{a+b+c} first computes @samp{a+b} and then combines with
8400@samp{c}. @xref{Precedence, ,Operator Precedence}.
8401
8402@item Left recursion
89cab50d
AD
8403A rule whose result symbol is also its first component symbol; for
8404example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
8405Rules}.
bfa74976
RS
8406
8407@item Left-to-right parsing
8408Parsing a sentence of a language by analyzing it token by token from
c827f760 8409left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
8410
8411@item Lexical analyzer (scanner)
8412A function that reads an input stream and returns tokens one by one.
8413@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
8414
8415@item Lexical tie-in
8416A flag, set by actions in the grammar rules, which alters the way
8417tokens are parsed. @xref{Lexical Tie-ins}.
8418
931c7513 8419@item Literal string token
14ded682 8420A token which consists of two or more fixed characters. @xref{Symbols}.
931c7513 8421
bfa74976 8422@item Look-ahead token
89cab50d
AD
8423A token already read but not yet shifted. @xref{Look-Ahead, ,Look-Ahead
8424Tokens}.
bfa74976 8425
c827f760 8426@item @acronym{LALR}(1)
bfa74976 8427The class of context-free grammars that Bison (like most other parser
c827f760
PE
8428generators) can handle; a subset of @acronym{LR}(1). @xref{Mystery
8429Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
bfa74976 8430
c827f760 8431@item @acronym{LR}(1)
bfa74976
RS
8432The class of context-free grammars in which at most one token of
8433look-ahead is needed to disambiguate the parsing of any piece of input.
8434
8435@item Nonterminal symbol
8436A grammar symbol standing for a grammatical construct that can
8437be expressed through rules in terms of smaller constructs; in other
8438words, a construct that is not a token. @xref{Symbols}.
8439
bfa74976
RS
8440@item Parser
8441A function that recognizes valid sentences of a language by analyzing
8442the syntax structure of a set of tokens passed to it from a lexical
8443analyzer.
8444
8445@item Postfix operator
8446An arithmetic operator that is placed after the operands upon which it
8447performs some operation.
8448
8449@item Reduction
8450Replacing a string of nonterminals and/or terminals with a single
89cab50d 8451nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
c827f760 8452Parser Algorithm}.
bfa74976
RS
8453
8454@item Reentrant
8455A reentrant subprogram is a subprogram which can be in invoked any
8456number of times in parallel, without interference between the various
8457invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
8458
8459@item Reverse polish notation
8460A language in which all operators are postfix operators.
8461
8462@item Right recursion
89cab50d
AD
8463A rule whose result symbol is also its last component symbol; for
8464example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
8465Rules}.
bfa74976
RS
8466
8467@item Semantics
8468In computer languages, the semantics are specified by the actions
8469taken for each instance of the language, i.e., the meaning of
8470each statement. @xref{Semantics, ,Defining Language Semantics}.
8471
8472@item Shift
8473A parser is said to shift when it makes the choice of analyzing
8474further input from the stream rather than reducing immediately some
c827f760 8475already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
8476
8477@item Single-character literal
8478A single character that is recognized and interpreted as is.
8479@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
8480
8481@item Start symbol
8482The nonterminal symbol that stands for a complete valid utterance in
8483the language being parsed. The start symbol is usually listed as the
13863333 8484first nonterminal symbol in a language specification.
bfa74976
RS
8485@xref{Start Decl, ,The Start-Symbol}.
8486
8487@item Symbol table
8488A data structure where symbol names and associated data are stored
8489during parsing to allow for recognition and use of existing
8490information in repeated uses of a symbol. @xref{Multi-function Calc}.
8491
6e649e65
PE
8492@item Syntax error
8493An error encountered during parsing of an input stream due to invalid
8494syntax. @xref{Error Recovery}.
8495
bfa74976
RS
8496@item Token
8497A basic, grammatically indivisible unit of a language. The symbol
8498that describes a token in the grammar is a terminal symbol.
8499The input of the Bison parser is a stream of tokens which comes from
8500the lexical analyzer. @xref{Symbols}.
8501
8502@item Terminal symbol
89cab50d
AD
8503A grammar symbol that has no rules in the grammar and therefore is
8504grammatically indivisible. The piece of text it represents is a token.
8505@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
8506@end table
8507
342b8b6e 8508@node Copying This Manual
f2b5126e 8509@appendix Copying This Manual
f9a8293a 8510
f2b5126e
PB
8511@menu
8512* GNU Free Documentation License:: License for copying this manual.
8513@end menu
f9a8293a 8514
f2b5126e
PB
8515@include fdl.texi
8516
342b8b6e 8517@node Index
bfa74976
RS
8518@unnumbered Index
8519
8520@printindex cp
8521
bfa74976 8522@bye
a06ea4aa
AD
8523
8524@c LocalWords: texinfo setfilename settitle setchapternewpage finalout
8525@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex
8526@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry
8527@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa
8528@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc
8529@c LocalWords: rpcalc Lexer Gen Comp Expr ltcalc mfcalc Decl Symtab yylex
8530@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref
8531@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex
8532@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge
8533@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG
8534@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit
8535@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok
8536@c LocalWords: longjmp fprintf stderr preg yylloc YYLTYPE cos ln
8537@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym
8538@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof
8539@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum
8540@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype
8541@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless typefull yynerrs
8542@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES
8543@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param
8544@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP
32c29292 8545@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword
a06ea4aa
AD
8546@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH
8547@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm VCG notype
8548@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args
8549@c LocalWords: YYPRINTF infile ypp yxx outfile itemx vcg tex leaderfill
8550@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll
8551@c LocalWords: yyrestart nbar yytext fst snd osplit ntwo strdup AST
8552@c LocalWords: YYSTACK DVI fdl printindex