]> git.saurik.com Git - bison.git/blame - doc/bison.texinfo
YYACCEPT, YYERROR, and YYABORT, as user actions, should not
[bison.git] / doc / bison.texinfo
CommitLineData
bfa74976
RS
1\input texinfo @c -*-texinfo-*-
2@comment %**start of header
3@setfilename bison.info
df1af54c
JT
4@include version.texi
5@settitle Bison @value{VERSION}
bfa74976
RS
6@setchapternewpage odd
7
5378c3e7 8@finalout
5378c3e7 9
13863333 10@c SMALL BOOK version
bfa74976 11@c This edition has been formatted so that you can format and print it in
13863333 12@c the smallbook format.
bfa74976
RS
13@c @smallbook
14
bfa74976
RS
15@c Set following if you have the new `shorttitlepage' command
16@c @clear shorttitlepage-enabled
17@c @set shorttitlepage-enabled
18
91d2c560
PE
19@c Set following if you want to document %default-prec and %no-default-prec.
20@c This feature is experimental and may change in future Bison versions.
21@c @set defaultprec
22
bfa74976
RS
23@c ISPELL CHECK: done, 14 Jan 1993 --bob
24
25@c Check COPYRIGHT dates. should be updated in the titlepage, ifinfo
26@c titlepage; should NOT be changed in the GPL. --mew
27
ec3bc396 28@c FIXME: I don't understand this `iftex'. Obsolete? --akim.
bfa74976
RS
29@iftex
30@syncodeindex fn cp
31@syncodeindex vr cp
32@syncodeindex tp cp
33@end iftex
34@ifinfo
35@synindex fn cp
36@synindex vr cp
37@synindex tp cp
38@end ifinfo
39@comment %**end of header
40
fae437e8 41@copying
bd773d73 42
c827f760
PE
43This manual is for @acronym{GNU} Bison (version @value{VERSION},
44@value{UPDATED}), the @acronym{GNU} parser generator.
fae437e8 45
a06ea4aa 46Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
e62f1a89 471999, 2000, 2001, 2002, 2003, 2004, 2005 Free Software Foundation, Inc.
fae437e8
AD
48
49@quotation
50Permission is granted to copy, distribute and/or modify this document
c827f760 51under the terms of the @acronym{GNU} Free Documentation License,
592fde95 52Version 1.2 or any later version published by the Free Software
c827f760
PE
53Foundation; with no Invariant Sections, with the Front-Cover texts
54being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in
55(a) below. A copy of the license is included in the section entitled
56``@acronym{GNU} Free Documentation License.''
57
58(a) The @acronym{FSF}'s Back-Cover Text is: ``You have freedom to copy
59and modify this @acronym{GNU} Manual, like @acronym{GNU} software.
60Copies published by the Free Software Foundation raise funds for
61@acronym{GNU} development.''
fae437e8
AD
62@end quotation
63@end copying
64
e62f1a89 65@dircategory Software development
fae437e8 66@direntry
c827f760 67* bison: (bison). @acronym{GNU} parser generator (Yacc replacement).
fae437e8 68@end direntry
bfa74976
RS
69
70@ifset shorttitlepage-enabled
71@shorttitlepage Bison
72@end ifset
73@titlepage
74@title Bison
c827f760 75@subtitle The Yacc-compatible Parser Generator
df1af54c 76@subtitle @value{UPDATED}, Bison Version @value{VERSION}
bfa74976
RS
77
78@author by Charles Donnelly and Richard Stallman
79
80@page
81@vskip 0pt plus 1filll
fae437e8 82@insertcopying
bfa74976
RS
83@sp 2
84Published by the Free Software Foundation @*
0fb669f9
PE
8551 Franklin Street, Fifth Floor @*
86Boston, MA 02110-1301 USA @*
9ecbd125 87Printed copies are available from the Free Software Foundation.@*
c827f760 88@acronym{ISBN} 1-882114-44-2
bfa74976
RS
89@sp 2
90Cover art by Etienne Suvasa.
91@end titlepage
d5796688
JT
92
93@contents
bfa74976 94
342b8b6e
AD
95@ifnottex
96@node Top
97@top Bison
fae437e8 98@insertcopying
342b8b6e 99@end ifnottex
bfa74976
RS
100
101@menu
13863333
AD
102* Introduction::
103* Conditions::
c827f760 104* Copying:: The @acronym{GNU} General Public License says
bfa74976
RS
105 how you can copy and share Bison
106
107Tutorial sections:
108* Concepts:: Basic concepts for understanding Bison.
109* Examples:: Three simple explained examples of using Bison.
110
111Reference sections:
112* Grammar File:: Writing Bison declarations and rules.
113* Interface:: C-language interface to the parser function @code{yyparse}.
114* Algorithm:: How the Bison parser works at run-time.
115* Error Recovery:: Writing rules for error recovery.
116* Context Dependency:: What to do if your language syntax is too
117 messy for Bison to handle straightforwardly.
ec3bc396 118* Debugging:: Understanding or debugging Bison parsers.
bfa74976 119* Invocation:: How to run Bison (to produce the parser source file).
12545799
AD
120* C++ Language Interface:: Creating C++ parser objects.
121* FAQ:: Frequently Asked Questions
bfa74976
RS
122* Table of Symbols:: All the keywords of the Bison language are explained.
123* Glossary:: Basic concepts are explained.
f2b5126e 124* Copying This Manual:: License for copying this manual.
bfa74976
RS
125* Index:: Cross-references to the text.
126
93dd49ab
PE
127@detailmenu
128 --- The Detailed Node Listing ---
bfa74976
RS
129
130The Concepts of Bison
131
132* Language and Grammar:: Languages and context-free grammars,
133 as mathematical ideas.
134* Grammar in Bison:: How we represent grammars for Bison's sake.
135* Semantic Values:: Each token or syntactic grouping can have
136 a semantic value (the value of an integer,
137 the name of an identifier, etc.).
138* Semantic Actions:: Each rule can have an action containing C code.
99a9344e 139* GLR Parsers:: Writing parsers for general context-free languages.
93dd49ab 140* Locations Overview:: Tracking Locations.
bfa74976
RS
141* Bison Parser:: What are Bison's input and output,
142 how is the output used?
143* Stages:: Stages in writing and running Bison grammars.
144* Grammar Layout:: Overall structure of a Bison grammar file.
145
fa7e68c3
PE
146Writing @acronym{GLR} Parsers
147
b50d2359
AD
148* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars
149* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities
150* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler
fa7e68c3 151
bfa74976
RS
152Examples
153
154* RPN Calc:: Reverse polish notation calculator;
155 a first example with no operator precedence.
156* Infix Calc:: Infix (algebraic) notation calculator.
157 Operator precedence is introduced.
158* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 159* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
93dd49ab
PE
160* Multi-function Calc:: Calculator with memory and trig functions.
161 It uses multiple data-types for semantic values.
bfa74976
RS
162* Exercises:: Ideas for improving the multi-function calculator.
163
164Reverse Polish Notation Calculator
165
75f5aaea 166* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
bfa74976
RS
167* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
168* Lexer: Rpcalc Lexer. The lexical analyzer.
169* Main: Rpcalc Main. The controlling function.
170* Error: Rpcalc Error. The error reporting function.
171* Gen: Rpcalc Gen. Running Bison on the grammar file.
172* Comp: Rpcalc Compile. Run the C compiler on the output code.
173
174Grammar Rules for @code{rpcalc}
175
13863333
AD
176* Rpcalc Input::
177* Rpcalc Line::
178* Rpcalc Expr::
bfa74976 179
342b8b6e
AD
180Location Tracking Calculator: @code{ltcalc}
181
182* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
183* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
184* Lexer: Ltcalc Lexer. The lexical analyzer.
185
bfa74976
RS
186Multi-Function Calculator: @code{mfcalc}
187
188* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
189* Rules: Mfcalc Rules. Grammar rules for the calculator.
190* Symtab: Mfcalc Symtab. Symbol table management subroutines.
191
192Bison Grammar Files
193
194* Grammar Outline:: Overall layout of the grammar file.
195* Symbols:: Terminal and nonterminal symbols.
196* Rules:: How to write grammar rules.
197* Recursion:: Writing recursive rules.
198* Semantics:: Semantic values and actions.
93dd49ab 199* Locations:: Locations and actions.
bfa74976
RS
200* Declarations:: All kinds of Bison declarations are described here.
201* Multiple Parsers:: Putting more than one Bison parser in one program.
202
203Outline of a Bison Grammar
204
93dd49ab 205* Prologue:: Syntax and usage of the prologue.
bfa74976
RS
206* Bison Declarations:: Syntax and usage of the Bison declarations section.
207* Grammar Rules:: Syntax and usage of the grammar rules section.
93dd49ab 208* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
209
210Defining Language Semantics
211
212* Value Type:: Specifying one data type for all semantic values.
213* Multiple Types:: Specifying several alternative data types.
214* Actions:: An action is the semantic definition of a grammar rule.
215* Action Types:: Specifying data types for actions to operate on.
216* Mid-Rule Actions:: Most actions go at the end of a rule.
217 This says when, why and how to use the exceptional
218 action in the middle of a rule.
219
93dd49ab
PE
220Tracking Locations
221
222* Location Type:: Specifying a data type for locations.
223* Actions and Locations:: Using locations in actions.
224* Location Default Action:: Defining a general way to compute locations.
225
bfa74976
RS
226Bison Declarations
227
b50d2359 228* Require Decl:: Requiring a Bison version.
bfa74976
RS
229* Token Decl:: Declaring terminal symbols.
230* Precedence Decl:: Declaring terminals with precedence and associativity.
231* Union Decl:: Declaring the set of all semantic value types.
232* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 233* Initial Action Decl:: Code run before parsing starts.
72f889cc 234* Destructor Decl:: Declaring how symbols are freed.
d6328241 235* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
236* Start Decl:: Specifying the start symbol.
237* Pure Decl:: Requesting a reentrant parser.
238* Decl Summary:: Table of all Bison declarations.
239
240Parser C-Language Interface
241
242* Parser Function:: How to call @code{yyparse} and what it returns.
13863333 243* Lexical:: You must supply a function @code{yylex}
bfa74976
RS
244 which reads tokens.
245* Error Reporting:: You must supply a function @code{yyerror}.
246* Action Features:: Special features for use in actions.
f7ab6a50
PE
247* Internationalization:: How to let the parser speak in the user's
248 native language.
bfa74976
RS
249
250The Lexical Analyzer Function @code{yylex}
251
252* Calling Convention:: How @code{yyparse} calls @code{yylex}.
253* Token Values:: How @code{yylex} must return the semantic value
254 of the token it has read.
95923bd6 255* Token Locations:: How @code{yylex} must return the text location
bfa74976 256 (line number, etc.) of the token, if the
93dd49ab 257 actions want that.
bfa74976
RS
258* Pure Calling:: How the calling convention differs
259 in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
260
13863333 261The Bison Parser Algorithm
bfa74976
RS
262
263* Look-Ahead:: Parser looks one token ahead when deciding what to do.
264* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
265* Precedence:: Operator precedence works by resolving conflicts.
266* Contextual Precedence:: When an operator's precedence depends on context.
267* Parser States:: The parser is a finite-state-machine with stack.
268* Reduce/Reduce:: When two rules are applicable in the same situation.
269* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 270* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 271* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
272
273Operator Precedence
274
275* Why Precedence:: An example showing why precedence is needed.
276* Using Precedence:: How to specify precedence in Bison grammars.
277* Precedence Examples:: How these features are used in the previous example.
278* How Precedence:: How they work.
279
280Handling Context Dependencies
281
282* Semantic Tokens:: Token parsing can depend on the semantic context.
283* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
284* Tie-in Recovery:: Lexical tie-ins have implications for how
285 error recovery rules must be written.
286
93dd49ab 287Debugging Your Parser
ec3bc396
AD
288
289* Understanding:: Understanding the structure of your parser.
290* Tracing:: Tracing the execution of your parser.
291
bfa74976
RS
292Invoking Bison
293
13863333 294* Bison Options:: All the options described in detail,
c827f760 295 in alphabetical order by short options.
bfa74976 296* Option Cross Key:: Alphabetical list of long options.
93dd49ab 297* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
f2b5126e 298
12545799
AD
299C++ Language Interface
300
301* C++ Parsers:: The interface to generate C++ parser classes
302* A Complete C++ Example:: Demonstrating their use
303
304C++ Parsers
305
306* C++ Bison Interface:: Asking for C++ parser generation
307* C++ Semantic Values:: %union vs. C++
308* C++ Location Values:: The position and location classes
309* C++ Parser Interface:: Instantiating and running the parser
310* C++ Scanner Interface:: Exchanges between yylex and parse
311
312A Complete C++ Example
313
314* Calc++ --- C++ Calculator:: The specifications
315* Calc++ Parsing Driver:: An active parsing context
316* Calc++ Parser:: A parser class
317* Calc++ Scanner:: A pure C++ Flex scanner
318* Calc++ Top Level:: Conducting the band
319
d1a1114f
AD
320Frequently Asked Questions
321
1a059451 322* Memory Exhausted:: Breaking the Stack Limits
e64fec0a 323* How Can I Reset the Parser:: @code{yyparse} Keeps some State
fef4cb51 324* Strings are Destroyed:: @code{yylval} Loses Track of Strings
2fa09258 325* Implementing Gotos/Loops:: Control Flow in the Calculator
d1a1114f 326
f2b5126e
PB
327Copying This Manual
328
329* GNU Free Documentation License:: License for copying this manual.
330
342b8b6e 331@end detailmenu
bfa74976
RS
332@end menu
333
342b8b6e 334@node Introduction
bfa74976
RS
335@unnumbered Introduction
336@cindex introduction
337
338@dfn{Bison} is a general-purpose parser generator that converts a
c827f760 339grammar description for an @acronym{LALR}(1) context-free grammar into a C
bfa74976
RS
340program to parse that grammar. Once you are proficient with Bison,
341you may use it to develop a wide range of language parsers, from those
342used in simple desk calculators to complex programming languages.
343
344Bison is upward compatible with Yacc: all properly-written Yacc grammars
345ought to work with Bison with no change. Anyone familiar with Yacc
346should be able to use Bison with little trouble. You need to be fluent in
347C programming in order to use Bison or to understand this manual.
348
349We begin with tutorial chapters that explain the basic concepts of using
350Bison and show three explained examples, each building on the last. If you
351don't know Bison or Yacc, start by reading these chapters. Reference
352chapters follow which describe specific aspects of Bison in detail.
353
931c7513
RS
354Bison was written primarily by Robert Corbett; Richard Stallman made it
355Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
14ded682 356multi-character string literals and other features.
931c7513 357
df1af54c 358This edition corresponds to version @value{VERSION} of Bison.
bfa74976 359
342b8b6e 360@node Conditions
bfa74976
RS
361@unnumbered Conditions for Using Bison
362
a31239f1 363As of Bison version 1.24, we have changed the distribution terms for
262aa8dd 364@code{yyparse} to permit using Bison's output in nonfree programs when
c827f760 365Bison is generating C code for @acronym{LALR}(1) parsers. Formerly, these
262aa8dd 366parsers could be used only in programs that were free software.
a31239f1 367
c827f760
PE
368The other @acronym{GNU} programming tools, such as the @acronym{GNU} C
369compiler, have never
9ecbd125 370had such a requirement. They could always be used for nonfree
a31239f1
RS
371software. The reason Bison was different was not due to a special
372policy decision; it resulted from applying the usual General Public
373License to all of the Bison source code.
374
375The output of the Bison utility---the Bison parser file---contains a
376verbatim copy of a sizable piece of Bison, which is the code for the
377@code{yyparse} function. (The actions from your grammar are inserted
378into this function at one point, but the rest of the function is not
c827f760
PE
379changed.) When we applied the @acronym{GPL} terms to the code for
380@code{yyparse},
a31239f1
RS
381the effect was to restrict the use of Bison output to free software.
382
383We didn't change the terms because of sympathy for people who want to
384make software proprietary. @strong{Software should be free.} But we
385concluded that limiting Bison's use to free software was doing little to
386encourage people to make other software free. So we decided to make the
387practical conditions for using Bison match the practical conditions for
c827f760 388using the other @acronym{GNU} tools.
bfa74976 389
eda42934 390This exception applies only when Bison is generating C code for an
c827f760
PE
391@acronym{LALR}(1) parser; otherwise, the @acronym{GPL} terms operate
392as usual. You can
262aa8dd
PE
393tell whether the exception applies to your @samp{.c} output file by
394inspecting it to see whether it says ``As a special exception, when
395this file is copied by Bison into a Bison output file, you may use
396that output file without restriction.''
397
c67a198d 398@include gpl.texi
bfa74976 399
342b8b6e 400@node Concepts
bfa74976
RS
401@chapter The Concepts of Bison
402
403This chapter introduces many of the basic concepts without which the
404details of Bison will not make sense. If you do not already know how to
405use Bison or Yacc, we suggest you start by reading this chapter carefully.
406
407@menu
408* Language and Grammar:: Languages and context-free grammars,
409 as mathematical ideas.
410* Grammar in Bison:: How we represent grammars for Bison's sake.
411* Semantic Values:: Each token or syntactic grouping can have
412 a semantic value (the value of an integer,
413 the name of an identifier, etc.).
414* Semantic Actions:: Each rule can have an action containing C code.
99a9344e 415* GLR Parsers:: Writing parsers for general context-free languages.
847bf1f5 416* Locations Overview:: Tracking Locations.
bfa74976
RS
417* Bison Parser:: What are Bison's input and output,
418 how is the output used?
419* Stages:: Stages in writing and running Bison grammars.
420* Grammar Layout:: Overall structure of a Bison grammar file.
421@end menu
422
342b8b6e 423@node Language and Grammar
bfa74976
RS
424@section Languages and Context-Free Grammars
425
bfa74976
RS
426@cindex context-free grammar
427@cindex grammar, context-free
428In order for Bison to parse a language, it must be described by a
429@dfn{context-free grammar}. This means that you specify one or more
430@dfn{syntactic groupings} and give rules for constructing them from their
431parts. For example, in the C language, one kind of grouping is called an
432`expression'. One rule for making an expression might be, ``An expression
433can be made of a minus sign and another expression''. Another would be,
434``An expression can be an integer''. As you can see, rules are often
435recursive, but there must be at least one rule which leads out of the
436recursion.
437
c827f760 438@cindex @acronym{BNF}
bfa74976
RS
439@cindex Backus-Naur form
440The most common formal system for presenting such rules for humans to read
c827f760
PE
441is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in
442order to specify the language Algol 60. Any grammar expressed in
443@acronym{BNF} is a context-free grammar. The input to Bison is
444essentially machine-readable @acronym{BNF}.
bfa74976 445
c827f760
PE
446@cindex @acronym{LALR}(1) grammars
447@cindex @acronym{LR}(1) grammars
676385e2
PH
448There are various important subclasses of context-free grammar. Although it
449can handle almost all context-free grammars, Bison is optimized for what
c827f760 450are called @acronym{LALR}(1) grammars.
676385e2 451In brief, in these grammars, it must be possible to
bfa74976
RS
452tell how to parse any portion of an input string with just a single
453token of look-ahead. Strictly speaking, that is a description of an
c827f760
PE
454@acronym{LR}(1) grammar, and @acronym{LALR}(1) involves additional
455restrictions that are
bfa74976 456hard to explain simply; but it is rare in actual practice to find an
c827f760
PE
457@acronym{LR}(1) grammar that fails to be @acronym{LALR}(1).
458@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
459more information on this.
bfa74976 460
c827f760
PE
461@cindex @acronym{GLR} parsing
462@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2
PH
463@cindex ambiguous grammars
464@cindex non-deterministic parsing
9501dc6e
AD
465
466Parsers for @acronym{LALR}(1) grammars are @dfn{deterministic}, meaning
467roughly that the next grammar rule to apply at any point in the input is
468uniquely determined by the preceding input and a fixed, finite portion
469(called a @dfn{look-ahead}) of the remaining input. A context-free
470grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
e4f85c39 471apply the grammar rules to get the same inputs. Even unambiguous
9501dc6e
AD
472grammars can be @dfn{non-deterministic}, meaning that no fixed
473look-ahead always suffices to determine the next grammar rule to apply.
474With the proper declarations, Bison is also able to parse these more
475general context-free grammars, using a technique known as @acronym{GLR}
476parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers
477are able to handle any context-free grammar for which the number of
478possible parses of any given string is finite.
676385e2 479
bfa74976
RS
480@cindex symbols (abstract)
481@cindex token
482@cindex syntactic grouping
483@cindex grouping, syntactic
9501dc6e
AD
484In the formal grammatical rules for a language, each kind of syntactic
485unit or grouping is named by a @dfn{symbol}. Those which are built by
486grouping smaller constructs according to grammatical rules are called
bfa74976
RS
487@dfn{nonterminal symbols}; those which can't be subdivided are called
488@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
489corresponding to a single terminal symbol a @dfn{token}, and a piece
e0c471a9 490corresponding to a single nonterminal symbol a @dfn{grouping}.
bfa74976
RS
491
492We can use the C language as an example of what symbols, terminal and
9501dc6e
AD
493nonterminal, mean. The tokens of C are identifiers, constants (numeric
494and string), and the various keywords, arithmetic operators and
495punctuation marks. So the terminal symbols of a grammar for C include
496`identifier', `number', `string', plus one symbol for each keyword,
497operator or punctuation mark: `if', `return', `const', `static', `int',
498`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
499(These tokens can be subdivided into characters, but that is a matter of
bfa74976
RS
500lexicography, not grammar.)
501
502Here is a simple C function subdivided into tokens:
503
9edcd895
AD
504@ifinfo
505@example
506int /* @r{keyword `int'} */
14d4662b 507square (int x) /* @r{identifier, open-paren, keyword `int',}
9edcd895
AD
508 @r{identifier, close-paren} */
509@{ /* @r{open-brace} */
510 return x * x; /* @r{keyword `return', identifier, asterisk,
511 identifier, semicolon} */
512@} /* @r{close-brace} */
513@end example
514@end ifinfo
515@ifnotinfo
bfa74976
RS
516@example
517int /* @r{keyword `int'} */
14d4662b 518square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
bfa74976 519@{ /* @r{open-brace} */
9edcd895 520 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
bfa74976
RS
521@} /* @r{close-brace} */
522@end example
9edcd895 523@end ifnotinfo
bfa74976
RS
524
525The syntactic groupings of C include the expression, the statement, the
526declaration, and the function definition. These are represented in the
527grammar of C by nonterminal symbols `expression', `statement',
528`declaration' and `function definition'. The full grammar uses dozens of
529additional language constructs, each with its own nonterminal symbol, in
530order to express the meanings of these four. The example above is a
531function definition; it contains one declaration, and one statement. In
532the statement, each @samp{x} is an expression and so is @samp{x * x}.
533
534Each nonterminal symbol must have grammatical rules showing how it is made
535out of simpler constructs. For example, one kind of C statement is the
536@code{return} statement; this would be described with a grammar rule which
537reads informally as follows:
538
539@quotation
540A `statement' can be made of a `return' keyword, an `expression' and a
541`semicolon'.
542@end quotation
543
544@noindent
545There would be many other rules for `statement', one for each kind of
546statement in C.
547
548@cindex start symbol
549One nonterminal symbol must be distinguished as the special one which
550defines a complete utterance in the language. It is called the @dfn{start
551symbol}. In a compiler, this means a complete input program. In the C
552language, the nonterminal symbol `sequence of definitions and declarations'
553plays this role.
554
555For example, @samp{1 + 2} is a valid C expression---a valid part of a C
556program---but it is not valid as an @emph{entire} C program. In the
557context-free grammar of C, this follows from the fact that `expression' is
558not the start symbol.
559
560The Bison parser reads a sequence of tokens as its input, and groups the
561tokens using the grammar rules. If the input is valid, the end result is
562that the entire token sequence reduces to a single grouping whose symbol is
563the grammar's start symbol. If we use a grammar for C, the entire input
564must be a `sequence of definitions and declarations'. If not, the parser
565reports a syntax error.
566
342b8b6e 567@node Grammar in Bison
bfa74976
RS
568@section From Formal Rules to Bison Input
569@cindex Bison grammar
570@cindex grammar, Bison
571@cindex formal grammar
572
573A formal grammar is a mathematical construct. To define the language
574for Bison, you must write a file expressing the grammar in Bison syntax:
575a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
576
577A nonterminal symbol in the formal grammar is represented in Bison input
c827f760 578as an identifier, like an identifier in C@. By convention, it should be
bfa74976
RS
579in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
580
581The Bison representation for a terminal symbol is also called a @dfn{token
582type}. Token types as well can be represented as C-like identifiers. By
583convention, these identifiers should be upper case to distinguish them from
584nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
585@code{RETURN}. A terminal symbol that stands for a particular keyword in
586the language should be named after that keyword converted to upper case.
587The terminal symbol @code{error} is reserved for error recovery.
931c7513 588@xref{Symbols}.
bfa74976
RS
589
590A terminal symbol can also be represented as a character literal, just like
591a C character constant. You should do this whenever a token is just a
592single character (parenthesis, plus-sign, etc.): use that same character in
593a literal as the terminal symbol for that token.
594
931c7513
RS
595A third way to represent a terminal symbol is with a C string constant
596containing several characters. @xref{Symbols}, for more information.
597
bfa74976
RS
598The grammar rules also have an expression in Bison syntax. For example,
599here is the Bison rule for a C @code{return} statement. The semicolon in
600quotes is a literal character token, representing part of the C syntax for
601the statement; the naked semicolon, and the colon, are Bison punctuation
602used in every rule.
603
604@example
605stmt: RETURN expr ';'
606 ;
607@end example
608
609@noindent
610@xref{Rules, ,Syntax of Grammar Rules}.
611
342b8b6e 612@node Semantic Values
bfa74976
RS
613@section Semantic Values
614@cindex semantic value
615@cindex value, semantic
616
617A formal grammar selects tokens only by their classifications: for example,
618if a rule mentions the terminal symbol `integer constant', it means that
619@emph{any} integer constant is grammatically valid in that position. The
620precise value of the constant is irrelevant to how to parse the input: if
621@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
e0c471a9 622grammatical.
bfa74976
RS
623
624But the precise value is very important for what the input means once it is
625parsed. A compiler is useless if it fails to distinguish between 4, 1 and
6263989 as constants in the program! Therefore, each token in a Bison grammar
c827f760
PE
627has both a token type and a @dfn{semantic value}. @xref{Semantics,
628,Defining Language Semantics},
bfa74976
RS
629for details.
630
631The token type is a terminal symbol defined in the grammar, such as
632@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
633you need to know to decide where the token may validly appear and how to
634group it with other tokens. The grammar rules know nothing about tokens
e0c471a9 635except their types.
bfa74976
RS
636
637The semantic value has all the rest of the information about the
638meaning of the token, such as the value of an integer, or the name of an
639identifier. (A token such as @code{','} which is just punctuation doesn't
640need to have any semantic value.)
641
642For example, an input token might be classified as token type
643@code{INTEGER} and have the semantic value 4. Another input token might
644have the same token type @code{INTEGER} but value 3989. When a grammar
645rule says that @code{INTEGER} is allowed, either of these tokens is
646acceptable because each is an @code{INTEGER}. When the parser accepts the
647token, it keeps track of the token's semantic value.
648
649Each grouping can also have a semantic value as well as its nonterminal
650symbol. For example, in a calculator, an expression typically has a
651semantic value that is a number. In a compiler for a programming
652language, an expression typically has a semantic value that is a tree
653structure describing the meaning of the expression.
654
342b8b6e 655@node Semantic Actions
bfa74976
RS
656@section Semantic Actions
657@cindex semantic actions
658@cindex actions, semantic
659
660In order to be useful, a program must do more than parse input; it must
661also produce some output based on the input. In a Bison grammar, a grammar
662rule can have an @dfn{action} made up of C statements. Each time the
663parser recognizes a match for that rule, the action is executed.
664@xref{Actions}.
13863333 665
bfa74976
RS
666Most of the time, the purpose of an action is to compute the semantic value
667of the whole construct from the semantic values of its parts. For example,
668suppose we have a rule which says an expression can be the sum of two
669expressions. When the parser recognizes such a sum, each of the
670subexpressions has a semantic value which describes how it was built up.
671The action for this rule should create a similar sort of value for the
672newly recognized larger expression.
673
674For example, here is a rule that says an expression can be the sum of
675two subexpressions:
676
677@example
678expr: expr '+' expr @{ $$ = $1 + $3; @}
679 ;
680@end example
681
682@noindent
683The action says how to produce the semantic value of the sum expression
684from the values of the two subexpressions.
685
676385e2 686@node GLR Parsers
c827f760
PE
687@section Writing @acronym{GLR} Parsers
688@cindex @acronym{GLR} parsing
689@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2
PH
690@findex %glr-parser
691@cindex conflicts
692@cindex shift/reduce conflicts
fa7e68c3 693@cindex reduce/reduce conflicts
676385e2 694
fa7e68c3 695In some grammars, Bison's standard
9501dc6e
AD
696@acronym{LALR}(1) parsing algorithm cannot decide whether to apply a
697certain grammar rule at a given point. That is, it may not be able to
698decide (on the basis of the input read so far) which of two possible
699reductions (applications of a grammar rule) applies, or whether to apply
700a reduction or read more of the input and apply a reduction later in the
701input. These are known respectively as @dfn{reduce/reduce} conflicts
702(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
703(@pxref{Shift/Reduce}).
704
705To use a grammar that is not easily modified to be @acronym{LALR}(1), a
706more general parsing algorithm is sometimes necessary. If you include
676385e2 707@code{%glr-parser} among the Bison declarations in your file
fa7e68c3 708(@pxref{Grammar Outline}), the result is a Generalized @acronym{LR}
9501dc6e
AD
709(@acronym{GLR}) parser. These parsers handle Bison grammars that
710contain no unresolved conflicts (i.e., after applying precedence
711declarations) identically to @acronym{LALR}(1) parsers. However, when
712faced with unresolved shift/reduce and reduce/reduce conflicts,
713@acronym{GLR} parsers use the simple expedient of doing both,
714effectively cloning the parser to follow both possibilities. Each of
715the resulting parsers can again split, so that at any given time, there
716can be any number of possible parses being explored. The parsers
676385e2
PH
717proceed in lockstep; that is, all of them consume (shift) a given input
718symbol before any of them proceed to the next. Each of the cloned
719parsers eventually meets one of two possible fates: either it runs into
720a parsing error, in which case it simply vanishes, or it merges with
721another parser, because the two of them have reduced the input to an
722identical set of symbols.
723
724During the time that there are multiple parsers, semantic actions are
725recorded, but not performed. When a parser disappears, its recorded
726semantic actions disappear as well, and are never performed. When a
727reduction makes two parsers identical, causing them to merge, Bison
728records both sets of semantic actions. Whenever the last two parsers
729merge, reverting to the single-parser case, Bison resolves all the
730outstanding actions either by precedences given to the grammar rules
731involved, or by performing both actions, and then calling a designated
732user-defined function on the resulting values to produce an arbitrary
733merged result.
734
fa7e68c3 735@menu
b50d2359
AD
736* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars
737* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities
738* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler
fa7e68c3
PE
739@end menu
740
741@node Simple GLR Parsers
742@subsection Using @acronym{GLR} on Unambiguous Grammars
743@cindex @acronym{GLR} parsing, unambiguous grammars
744@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars
745@findex %glr-parser
746@findex %expect-rr
747@cindex conflicts
748@cindex reduce/reduce conflicts
749@cindex shift/reduce conflicts
750
751In the simplest cases, you can use the @acronym{GLR} algorithm
752to parse grammars that are unambiguous, but fail to be @acronym{LALR}(1).
753Such grammars typically require more than one symbol of look-ahead,
754or (in rare cases) fall into the category of grammars in which the
755@acronym{LALR}(1) algorithm throws away too much information (they are in
756@acronym{LR}(1), but not @acronym{LALR}(1), @ref{Mystery Conflicts}).
757
758Consider a problem that
759arises in the declaration of enumerated and subrange types in the
760programming language Pascal. Here are some examples:
761
762@example
763type subrange = lo .. hi;
764type enum = (a, b, c);
765@end example
766
767@noindent
768The original language standard allows only numeric
769literals and constant identifiers for the subrange bounds (@samp{lo}
770and @samp{hi}), but Extended Pascal (@acronym{ISO}/@acronym{IEC}
77110206) and many other
772Pascal implementations allow arbitrary expressions there. This gives
773rise to the following situation, containing a superfluous pair of
774parentheses:
775
776@example
777type subrange = (a) .. b;
778@end example
779
780@noindent
781Compare this to the following declaration of an enumerated
782type with only one value:
783
784@example
785type enum = (a);
786@end example
787
788@noindent
789(These declarations are contrived, but they are syntactically
790valid, and more-complicated cases can come up in practical programs.)
791
792These two declarations look identical until the @samp{..} token.
793With normal @acronym{LALR}(1) one-token look-ahead it is not
794possible to decide between the two forms when the identifier
795@samp{a} is parsed. It is, however, desirable
796for a parser to decide this, since in the latter case
797@samp{a} must become a new identifier to represent the enumeration
798value, while in the former case @samp{a} must be evaluated with its
799current meaning, which may be a constant or even a function call.
800
801You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
802to be resolved later, but this typically requires substantial
803contortions in both semantic actions and large parts of the
804grammar, where the parentheses are nested in the recursive rules for
805expressions.
806
807You might think of using the lexer to distinguish between the two
808forms by returning different tokens for currently defined and
809undefined identifiers. But if these declarations occur in a local
810scope, and @samp{a} is defined in an outer scope, then both forms
811are possible---either locally redefining @samp{a}, or using the
812value of @samp{a} from the outer scope. So this approach cannot
813work.
814
e757bb10 815A simple solution to this problem is to declare the parser to
fa7e68c3
PE
816use the @acronym{GLR} algorithm.
817When the @acronym{GLR} parser reaches the critical state, it
818merely splits into two branches and pursues both syntax rules
819simultaneously. Sooner or later, one of them runs into a parsing
820error. If there is a @samp{..} token before the next
821@samp{;}, the rule for enumerated types fails since it cannot
822accept @samp{..} anywhere; otherwise, the subrange type rule
823fails since it requires a @samp{..} token. So one of the branches
824fails silently, and the other one continues normally, performing
825all the intermediate actions that were postponed during the split.
826
827If the input is syntactically incorrect, both branches fail and the parser
828reports a syntax error as usual.
829
830The effect of all this is that the parser seems to ``guess'' the
831correct branch to take, or in other words, it seems to use more
832look-ahead than the underlying @acronym{LALR}(1) algorithm actually allows
833for. In this example, @acronym{LALR}(2) would suffice, but also some cases
834that are not @acronym{LALR}(@math{k}) for any @math{k} can be handled this way.
835
836In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time,
837and the current Bison parser even takes exponential time and space
838for some grammars. In practice, this rarely happens, and for many
839grammars it is possible to prove that it cannot happen.
840The present example contains only one conflict between two
841rules, and the type-declaration context containing the conflict
842cannot be nested. So the number of
843branches that can exist at any time is limited by the constant 2,
844and the parsing time is still linear.
845
846Here is a Bison grammar corresponding to the example above. It
847parses a vastly simplified form of Pascal type declarations.
848
849@example
850%token TYPE DOTDOT ID
851
852@group
853%left '+' '-'
854%left '*' '/'
855@end group
856
857%%
858
859@group
860type_decl : TYPE ID '=' type ';'
861 ;
862@end group
863
864@group
865type : '(' id_list ')'
866 | expr DOTDOT expr
867 ;
868@end group
869
870@group
871id_list : ID
872 | id_list ',' ID
873 ;
874@end group
875
876@group
877expr : '(' expr ')'
878 | expr '+' expr
879 | expr '-' expr
880 | expr '*' expr
881 | expr '/' expr
882 | ID
883 ;
884@end group
885@end example
886
887When used as a normal @acronym{LALR}(1) grammar, Bison correctly complains
888about one reduce/reduce conflict. In the conflicting situation the
889parser chooses one of the alternatives, arbitrarily the one
890declared first. Therefore the following correct input is not
891recognized:
892
893@example
894type t = (a) .. b;
895@end example
896
897The parser can be turned into a @acronym{GLR} parser, while also telling Bison
898to be silent about the one known reduce/reduce conflict, by
e757bb10 899adding these two declarations to the Bison input file (before the first
fa7e68c3
PE
900@samp{%%}):
901
902@example
903%glr-parser
904%expect-rr 1
905@end example
906
907@noindent
908No change in the grammar itself is required. Now the
909parser recognizes all valid declarations, according to the
910limited syntax above, transparently. In fact, the user does not even
911notice when the parser splits.
912
913So here we have a case where we can use the benefits of @acronym{GLR}, almost
914without disadvantages. Even in simple cases like this, however, there
915are at least two potential problems to beware.
916First, always analyze the conflicts reported by
917Bison to make sure that @acronym{GLR} splitting is only done where it is
918intended. A @acronym{GLR} parser splitting inadvertently may cause
919problems less obvious than an @acronym{LALR} parser statically choosing the
920wrong alternative in a conflict.
e757bb10 921Second, consider interactions with the lexer (@pxref{Semantic Tokens})
fa7e68c3
PE
922with great care. Since a split parser consumes tokens
923without performing any actions during the split, the lexer cannot
924obtain information via parser actions. Some cases of
925lexer interactions can be eliminated by using @acronym{GLR} to
926shift the complications from the lexer to the parser. You must check
927the remaining cases for correctness.
928
929In our example, it would be safe for the lexer to return tokens
930based on their current meanings in some symbol table, because no new
931symbols are defined in the middle of a type declaration. Though it
932is possible for a parser to define the enumeration
933constants as they are parsed, before the type declaration is
934completed, it actually makes no difference since they cannot be used
935within the same enumerated type declaration.
936
937@node Merging GLR Parses
938@subsection Using @acronym{GLR} to Resolve Ambiguities
939@cindex @acronym{GLR} parsing, ambiguous grammars
940@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, ambiguous grammars
941@findex %dprec
942@findex %merge
943@cindex conflicts
944@cindex reduce/reduce conflicts
945
2a8d363a 946Let's consider an example, vastly simplified from a C++ grammar.
676385e2
PH
947
948@example
949%@{
38a92d50
PE
950 #include <stdio.h>
951 #define YYSTYPE char const *
952 int yylex (void);
953 void yyerror (char const *);
676385e2
PH
954%@}
955
956%token TYPENAME ID
957
958%right '='
959%left '+'
960
961%glr-parser
962
963%%
964
fae437e8 965prog :
676385e2
PH
966 | prog stmt @{ printf ("\n"); @}
967 ;
968
969stmt : expr ';' %dprec 1
970 | decl %dprec 2
971 ;
972
2a8d363a 973expr : ID @{ printf ("%s ", $$); @}
fae437e8 974 | TYPENAME '(' expr ')'
2a8d363a
AD
975 @{ printf ("%s <cast> ", $1); @}
976 | expr '+' expr @{ printf ("+ "); @}
977 | expr '=' expr @{ printf ("= "); @}
676385e2
PH
978 ;
979
fae437e8 980decl : TYPENAME declarator ';'
2a8d363a 981 @{ printf ("%s <declare> ", $1); @}
676385e2 982 | TYPENAME declarator '=' expr ';'
2a8d363a 983 @{ printf ("%s <init-declare> ", $1); @}
676385e2
PH
984 ;
985
2a8d363a 986declarator : ID @{ printf ("\"%s\" ", $1); @}
676385e2
PH
987 | '(' declarator ')'
988 ;
989@end example
990
991@noindent
992This models a problematic part of the C++ grammar---the ambiguity between
993certain declarations and statements. For example,
994
995@example
996T (x) = y+z;
997@end example
998
999@noindent
1000parses as either an @code{expr} or a @code{stmt}
c827f760
PE
1001(assuming that @samp{T} is recognized as a @code{TYPENAME} and
1002@samp{x} as an @code{ID}).
676385e2 1003Bison detects this as a reduce/reduce conflict between the rules
fae437e8 1004@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
e757bb10
AD
1005time it encounters @code{x} in the example above. Since this is a
1006@acronym{GLR} parser, it therefore splits the problem into two parses, one for
fa7e68c3
PE
1007each choice of resolving the reduce/reduce conflict.
1008Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1009however, neither of these parses ``dies,'' because the grammar as it stands is
e757bb10
AD
1010ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1011the other reduces @code{stmt : decl}, after which both parsers are in an
1012identical state: they've seen @samp{prog stmt} and have the same unprocessed
1013input remaining. We say that these parses have @dfn{merged.}
fa7e68c3
PE
1014
1015At this point, the @acronym{GLR} parser requires a specification in the
1016grammar of how to choose between the competing parses.
1017In the example above, the two @code{%dprec}
e757bb10 1018declarations specify that Bison is to give precedence
fa7e68c3 1019to the parse that interprets the example as a
676385e2
PH
1020@code{decl}, which implies that @code{x} is a declarator.
1021The parser therefore prints
1022
1023@example
fae437e8 1024"x" y z + T <init-declare>
676385e2
PH
1025@end example
1026
fa7e68c3
PE
1027The @code{%dprec} declarations only come into play when more than one
1028parse survives. Consider a different input string for this parser:
676385e2
PH
1029
1030@example
1031T (x) + y;
1032@end example
1033
1034@noindent
e757bb10 1035This is another example of using @acronym{GLR} to parse an unambiguous
fa7e68c3 1036construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
676385e2
PH
1037Here, there is no ambiguity (this cannot be parsed as a declaration).
1038However, at the time the Bison parser encounters @code{x}, it does not
1039have enough information to resolve the reduce/reduce conflict (again,
1040between @code{x} as an @code{expr} or a @code{declarator}). In this
fa7e68c3 1041case, no precedence declaration is used. Again, the parser splits
676385e2
PH
1042into two, one assuming that @code{x} is an @code{expr}, and the other
1043assuming @code{x} is a @code{declarator}. The second of these parsers
1044then vanishes when it sees @code{+}, and the parser prints
1045
1046@example
fae437e8 1047x T <cast> y +
676385e2
PH
1048@end example
1049
1050Suppose that instead of resolving the ambiguity, you wanted to see all
fa7e68c3 1051the possibilities. For this purpose, you must merge the semantic
676385e2
PH
1052actions of the two possible parsers, rather than choosing one over the
1053other. To do so, you could change the declaration of @code{stmt} as
1054follows:
1055
1056@example
1057stmt : expr ';' %merge <stmtMerge>
1058 | decl %merge <stmtMerge>
1059 ;
1060@end example
1061
1062@noindent
676385e2
PH
1063and define the @code{stmtMerge} function as:
1064
1065@example
38a92d50
PE
1066static YYSTYPE
1067stmtMerge (YYSTYPE x0, YYSTYPE x1)
676385e2
PH
1068@{
1069 printf ("<OR> ");
1070 return "";
1071@}
1072@end example
1073
1074@noindent
1075with an accompanying forward declaration
1076in the C declarations at the beginning of the file:
1077
1078@example
1079%@{
38a92d50 1080 #define YYSTYPE char const *
676385e2
PH
1081 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1082%@}
1083@end example
1084
1085@noindent
fa7e68c3
PE
1086With these declarations, the resulting parser parses the first example
1087as both an @code{expr} and a @code{decl}, and prints
676385e2
PH
1088
1089@example
fae437e8 1090"x" y z + T <init-declare> x T <cast> y z + = <OR>
676385e2
PH
1091@end example
1092
fa7e68c3 1093Bison requires that all of the
e757bb10 1094productions that participate in any particular merge have identical
fa7e68c3
PE
1095@samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1096and the parser will report an error during any parse that results in
1097the offending merge.
9501dc6e 1098
fa7e68c3
PE
1099@node Compiler Requirements
1100@subsection Considerations when Compiling @acronym{GLR} Parsers
1101@cindex @code{inline}
9501dc6e 1102@cindex @acronym{GLR} parsers and @code{inline}
fa7e68c3 1103
38a92d50
PE
1104The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or
1105later. In addition, they use the @code{inline} keyword, which is not
1106C89, but is C99 and is a common extension in pre-C99 compilers. It is
1107up to the user of these parsers to handle
9501dc6e
AD
1108portability issues. For instance, if using Autoconf and the Autoconf
1109macro @code{AC_C_INLINE}, a mere
1110
1111@example
1112%@{
38a92d50 1113 #include <config.h>
9501dc6e
AD
1114%@}
1115@end example
1116
1117@noindent
1118will suffice. Otherwise, we suggest
1119
1120@example
1121%@{
38a92d50
PE
1122 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1123 #define inline
1124 #endif
9501dc6e
AD
1125%@}
1126@end example
676385e2 1127
342b8b6e 1128@node Locations Overview
847bf1f5
AD
1129@section Locations
1130@cindex location
95923bd6
AD
1131@cindex textual location
1132@cindex location, textual
847bf1f5
AD
1133
1134Many applications, like interpreters or compilers, have to produce verbose
72d2299c 1135and useful error messages. To achieve this, one must be able to keep track of
95923bd6 1136the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
847bf1f5
AD
1137Bison provides a mechanism for handling these locations.
1138
72d2299c 1139Each token has a semantic value. In a similar fashion, each token has an
847bf1f5 1140associated location, but the type of locations is the same for all tokens and
72d2299c 1141groupings. Moreover, the output parser is equipped with a default data
847bf1f5
AD
1142structure for storing locations (@pxref{Locations}, for more details).
1143
1144Like semantic values, locations can be reached in actions using a dedicated
72d2299c 1145set of constructs. In the example above, the location of the whole grouping
847bf1f5
AD
1146is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1147@code{@@3}.
1148
1149When a rule is matched, a default action is used to compute the semantic value
72d2299c
PE
1150of its left hand side (@pxref{Actions}). In the same way, another default
1151action is used for locations. However, the action for locations is general
847bf1f5 1152enough for most cases, meaning there is usually no need to describe for each
72d2299c 1153rule how @code{@@$} should be formed. When building a new location for a given
847bf1f5
AD
1154grouping, the default behavior of the output parser is to take the beginning
1155of the first symbol, and the end of the last symbol.
1156
342b8b6e 1157@node Bison Parser
bfa74976
RS
1158@section Bison Output: the Parser File
1159@cindex Bison parser
1160@cindex Bison utility
1161@cindex lexical analyzer, purpose
1162@cindex parser
1163
1164When you run Bison, you give it a Bison grammar file as input. The output
1165is a C source file that parses the language described by the grammar.
1166This file is called a @dfn{Bison parser}. Keep in mind that the Bison
1167utility and the Bison parser are two distinct programs: the Bison utility
1168is a program whose output is the Bison parser that becomes part of your
1169program.
1170
1171The job of the Bison parser is to group tokens into groupings according to
1172the grammar rules---for example, to build identifiers and operators into
1173expressions. As it does this, it runs the actions for the grammar rules it
1174uses.
1175
704a47c4
AD
1176The tokens come from a function called the @dfn{lexical analyzer} that
1177you must supply in some fashion (such as by writing it in C). The Bison
1178parser calls the lexical analyzer each time it wants a new token. It
1179doesn't know what is ``inside'' the tokens (though their semantic values
1180may reflect this). Typically the lexical analyzer makes the tokens by
1181parsing characters of text, but Bison does not depend on this.
1182@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
bfa74976
RS
1183
1184The Bison parser file is C code which defines a function named
1185@code{yyparse} which implements that grammar. This function does not make
1186a complete C program: you must supply some additional functions. One is
1187the lexical analyzer. Another is an error-reporting function which the
1188parser calls to report an error. In addition, a complete C program must
1189start with a function called @code{main}; you have to provide this, and
1190arrange for it to call @code{yyparse} or the parser will never run.
1191@xref{Interface, ,Parser C-Language Interface}.
1192
f7ab6a50 1193Aside from the token type names and the symbols in the actions you
7093d0f5 1194write, all symbols defined in the Bison parser file itself
bfa74976
RS
1195begin with @samp{yy} or @samp{YY}. This includes interface functions
1196such as the lexical analyzer function @code{yylex}, the error reporting
1197function @code{yyerror} and the parser function @code{yyparse} itself.
1198This also includes numerous identifiers used for internal purposes.
1199Therefore, you should avoid using C identifiers starting with @samp{yy}
1200or @samp{YY} in the Bison grammar file except for the ones defined in
55289366
PE
1201this manual. Also, you should avoid using the C identifiers
1202@samp{malloc} and @samp{free} for anything other than their usual
1203meanings.
bfa74976 1204
7093d0f5
AD
1205In some cases the Bison parser file includes system headers, and in
1206those cases your code should respect the identifiers reserved by those
55289366 1207headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>}, @code{<malloc.h>},
7093d0f5 1208@code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
30757c8c
PE
1209declare memory allocators and related types. @code{<libintl.h>} is
1210included if message translation is in use
1211(@pxref{Internationalization}). Other system headers may
ec3bc396
AD
1212be included if you define @code{YYDEBUG} to a nonzero value
1213(@pxref{Tracing, ,Tracing Your Parser}).
7093d0f5 1214
342b8b6e 1215@node Stages
bfa74976
RS
1216@section Stages in Using Bison
1217@cindex stages in using Bison
1218@cindex using Bison
1219
1220The actual language-design process using Bison, from grammar specification
1221to a working compiler or interpreter, has these parts:
1222
1223@enumerate
1224@item
1225Formally specify the grammar in a form recognized by Bison
704a47c4
AD
1226(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1227in the language, describe the action that is to be taken when an
1228instance of that rule is recognized. The action is described by a
1229sequence of C statements.
bfa74976
RS
1230
1231@item
704a47c4
AD
1232Write a lexical analyzer to process input and pass tokens to the parser.
1233The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1234Lexical Analyzer Function @code{yylex}}). It could also be produced
1235using Lex, but the use of Lex is not discussed in this manual.
bfa74976
RS
1236
1237@item
1238Write a controlling function that calls the Bison-produced parser.
1239
1240@item
1241Write error-reporting routines.
1242@end enumerate
1243
1244To turn this source code as written into a runnable program, you
1245must follow these steps:
1246
1247@enumerate
1248@item
1249Run Bison on the grammar to produce the parser.
1250
1251@item
1252Compile the code output by Bison, as well as any other source files.
1253
1254@item
1255Link the object files to produce the finished product.
1256@end enumerate
1257
342b8b6e 1258@node Grammar Layout
bfa74976
RS
1259@section The Overall Layout of a Bison Grammar
1260@cindex grammar file
1261@cindex file format
1262@cindex format of grammar file
1263@cindex layout of Bison grammar
1264
1265The input file for the Bison utility is a @dfn{Bison grammar file}. The
1266general form of a Bison grammar file is as follows:
1267
1268@example
1269%@{
08e49d20 1270@var{Prologue}
bfa74976
RS
1271%@}
1272
1273@var{Bison declarations}
1274
1275%%
1276@var{Grammar rules}
1277%%
08e49d20 1278@var{Epilogue}
bfa74976
RS
1279@end example
1280
1281@noindent
1282The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1283in every Bison grammar file to separate the sections.
1284
72d2299c 1285The prologue may define types and variables used in the actions. You can
342b8b6e 1286also use preprocessor commands to define macros used there, and use
bfa74976 1287@code{#include} to include header files that do any of these things.
38a92d50
PE
1288You need to declare the lexical analyzer @code{yylex} and the error
1289printer @code{yyerror} here, along with any other global identifiers
1290used by the actions in the grammar rules.
bfa74976
RS
1291
1292The Bison declarations declare the names of the terminal and nonterminal
1293symbols, and may also describe operator precedence and the data types of
1294semantic values of various symbols.
1295
1296The grammar rules define how to construct each nonterminal symbol from its
1297parts.
1298
38a92d50
PE
1299The epilogue can contain any code you want to use. Often the
1300definitions of functions declared in the prologue go here. In a
1301simple program, all the rest of the program can go here.
bfa74976 1302
342b8b6e 1303@node Examples
bfa74976
RS
1304@chapter Examples
1305@cindex simple examples
1306@cindex examples, simple
1307
1308Now we show and explain three sample programs written using Bison: a
1309reverse polish notation calculator, an algebraic (infix) notation
1310calculator, and a multi-function calculator. All three have been tested
1311under BSD Unix 4.3; each produces a usable, though limited, interactive
1312desk-top calculator.
1313
1314These examples are simple, but Bison grammars for real programming
1315languages are written the same way.
1316@ifinfo
1317You can copy these examples out of the Info file and into a source file
1318to try them.
1319@end ifinfo
1320
1321@menu
1322* RPN Calc:: Reverse polish notation calculator;
1323 a first example with no operator precedence.
1324* Infix Calc:: Infix (algebraic) notation calculator.
1325 Operator precedence is introduced.
1326* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 1327* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
bfa74976
RS
1328* Multi-function Calc:: Calculator with memory and trig functions.
1329 It uses multiple data-types for semantic values.
1330* Exercises:: Ideas for improving the multi-function calculator.
1331@end menu
1332
342b8b6e 1333@node RPN Calc
bfa74976
RS
1334@section Reverse Polish Notation Calculator
1335@cindex reverse polish notation
1336@cindex polish notation calculator
1337@cindex @code{rpcalc}
1338@cindex calculator, simple
1339
1340The first example is that of a simple double-precision @dfn{reverse polish
1341notation} calculator (a calculator using postfix operators). This example
1342provides a good starting point, since operator precedence is not an issue.
1343The second example will illustrate how operator precedence is handled.
1344
1345The source code for this calculator is named @file{rpcalc.y}. The
1346@samp{.y} extension is a convention used for Bison input files.
1347
1348@menu
75f5aaea 1349* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
bfa74976
RS
1350* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
1351* Lexer: Rpcalc Lexer. The lexical analyzer.
1352* Main: Rpcalc Main. The controlling function.
1353* Error: Rpcalc Error. The error reporting function.
1354* Gen: Rpcalc Gen. Running Bison on the grammar file.
1355* Comp: Rpcalc Compile. Run the C compiler on the output code.
1356@end menu
1357
342b8b6e 1358@node Rpcalc Decls
bfa74976
RS
1359@subsection Declarations for @code{rpcalc}
1360
1361Here are the C and Bison declarations for the reverse polish notation
1362calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1363
1364@example
72d2299c 1365/* Reverse polish notation calculator. */
bfa74976
RS
1366
1367%@{
38a92d50
PE
1368 #define YYSTYPE double
1369 #include <math.h>
1370 int yylex (void);
1371 void yyerror (char const *);
bfa74976
RS
1372%@}
1373
1374%token NUM
1375
72d2299c 1376%% /* Grammar rules and actions follow. */
bfa74976
RS
1377@end example
1378
75f5aaea 1379The declarations section (@pxref{Prologue, , The prologue}) contains two
38a92d50 1380preprocessor directives and two forward declarations.
bfa74976
RS
1381
1382The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1964ad8c
AD
1383specifying the C data type for semantic values of both tokens and
1384groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1385Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1386don't define it, @code{int} is the default. Because we specify
1387@code{double}, each token and each expression has an associated value,
1388which is a floating point number.
bfa74976
RS
1389
1390The @code{#include} directive is used to declare the exponentiation
1391function @code{pow}.
1392
38a92d50
PE
1393The forward declarations for @code{yylex} and @code{yyerror} are
1394needed because the C language requires that functions be declared
1395before they are used. These functions will be defined in the
1396epilogue, but the parser calls them so they must be declared in the
1397prologue.
1398
704a47c4
AD
1399The second section, Bison declarations, provides information to Bison
1400about the token types (@pxref{Bison Declarations, ,The Bison
1401Declarations Section}). Each terminal symbol that is not a
1402single-character literal must be declared here. (Single-character
bfa74976
RS
1403literals normally don't need to be declared.) In this example, all the
1404arithmetic operators are designated by single-character literals, so the
1405only terminal symbol that needs to be declared is @code{NUM}, the token
1406type for numeric constants.
1407
342b8b6e 1408@node Rpcalc Rules
bfa74976
RS
1409@subsection Grammar Rules for @code{rpcalc}
1410
1411Here are the grammar rules for the reverse polish notation calculator.
1412
1413@example
1414input: /* empty */
1415 | input line
1416;
1417
1418line: '\n'
18b519c0 1419 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
bfa74976
RS
1420;
1421
18b519c0
AD
1422exp: NUM @{ $$ = $1; @}
1423 | exp exp '+' @{ $$ = $1 + $2; @}
1424 | exp exp '-' @{ $$ = $1 - $2; @}
1425 | exp exp '*' @{ $$ = $1 * $2; @}
1426 | exp exp '/' @{ $$ = $1 / $2; @}
1427 /* Exponentiation */
1428 | exp exp '^' @{ $$ = pow ($1, $2); @}
1429 /* Unary minus */
1430 | exp 'n' @{ $$ = -$1; @}
bfa74976
RS
1431;
1432%%
1433@end example
1434
1435The groupings of the rpcalc ``language'' defined here are the expression
1436(given the name @code{exp}), the line of input (@code{line}), and the
1437complete input transcript (@code{input}). Each of these nonterminal
1438symbols has several alternate rules, joined by the @samp{|} punctuator
1439which is read as ``or''. The following sections explain what these rules
1440mean.
1441
1442The semantics of the language is determined by the actions taken when a
1443grouping is recognized. The actions are the C code that appears inside
1444braces. @xref{Actions}.
1445
1446You must specify these actions in C, but Bison provides the means for
1447passing semantic values between the rules. In each action, the
1448pseudo-variable @code{$$} stands for the semantic value for the grouping
1449that the rule is going to construct. Assigning a value to @code{$$} is the
1450main job of most actions. The semantic values of the components of the
1451rule are referred to as @code{$1}, @code{$2}, and so on.
1452
1453@menu
13863333
AD
1454* Rpcalc Input::
1455* Rpcalc Line::
1456* Rpcalc Expr::
bfa74976
RS
1457@end menu
1458
342b8b6e 1459@node Rpcalc Input
bfa74976
RS
1460@subsubsection Explanation of @code{input}
1461
1462Consider the definition of @code{input}:
1463
1464@example
1465input: /* empty */
1466 | input line
1467;
1468@end example
1469
1470This definition reads as follows: ``A complete input is either an empty
1471string, or a complete input followed by an input line''. Notice that
1472``complete input'' is defined in terms of itself. This definition is said
1473to be @dfn{left recursive} since @code{input} appears always as the
1474leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1475
1476The first alternative is empty because there are no symbols between the
1477colon and the first @samp{|}; this means that @code{input} can match an
1478empty string of input (no tokens). We write the rules this way because it
1479is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1480It's conventional to put an empty alternative first and write the comment
1481@samp{/* empty */} in it.
1482
1483The second alternate rule (@code{input line}) handles all nontrivial input.
1484It means, ``After reading any number of lines, read one more line if
1485possible.'' The left recursion makes this rule into a loop. Since the
1486first alternative matches empty input, the loop can be executed zero or
1487more times.
1488
1489The parser function @code{yyparse} continues to process input until a
1490grammatical error is seen or the lexical analyzer says there are no more
72d2299c 1491input tokens; we will arrange for the latter to happen at end-of-input.
bfa74976 1492
342b8b6e 1493@node Rpcalc Line
bfa74976
RS
1494@subsubsection Explanation of @code{line}
1495
1496Now consider the definition of @code{line}:
1497
1498@example
1499line: '\n'
1500 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1501;
1502@end example
1503
1504The first alternative is a token which is a newline character; this means
1505that rpcalc accepts a blank line (and ignores it, since there is no
1506action). The second alternative is an expression followed by a newline.
1507This is the alternative that makes rpcalc useful. The semantic value of
1508the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1509question is the first symbol in the alternative. The action prints this
1510value, which is the result of the computation the user asked for.
1511
1512This action is unusual because it does not assign a value to @code{$$}. As
1513a consequence, the semantic value associated with the @code{line} is
1514uninitialized (its value will be unpredictable). This would be a bug if
1515that value were ever used, but we don't use it: once rpcalc has printed the
1516value of the user's input line, that value is no longer needed.
1517
342b8b6e 1518@node Rpcalc Expr
bfa74976
RS
1519@subsubsection Explanation of @code{expr}
1520
1521The @code{exp} grouping has several rules, one for each kind of expression.
1522The first rule handles the simplest expressions: those that are just numbers.
1523The second handles an addition-expression, which looks like two expressions
1524followed by a plus-sign. The third handles subtraction, and so on.
1525
1526@example
1527exp: NUM
1528 | exp exp '+' @{ $$ = $1 + $2; @}
1529 | exp exp '-' @{ $$ = $1 - $2; @}
1530 @dots{}
1531 ;
1532@end example
1533
1534We have used @samp{|} to join all the rules for @code{exp}, but we could
1535equally well have written them separately:
1536
1537@example
1538exp: NUM ;
1539exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1540exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1541 @dots{}
1542@end example
1543
1544Most of the rules have actions that compute the value of the expression in
1545terms of the value of its parts. For example, in the rule for addition,
1546@code{$1} refers to the first component @code{exp} and @code{$2} refers to
1547the second one. The third component, @code{'+'}, has no meaningful
1548associated semantic value, but if it had one you could refer to it as
1549@code{$3}. When @code{yyparse} recognizes a sum expression using this
1550rule, the sum of the two subexpressions' values is produced as the value of
1551the entire expression. @xref{Actions}.
1552
1553You don't have to give an action for every rule. When a rule has no
1554action, Bison by default copies the value of @code{$1} into @code{$$}.
1555This is what happens in the first rule (the one that uses @code{NUM}).
1556
1557The formatting shown here is the recommended convention, but Bison does
72d2299c 1558not require it. You can add or change white space as much as you wish.
bfa74976
RS
1559For example, this:
1560
1561@example
99a9344e 1562exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
bfa74976
RS
1563@end example
1564
1565@noindent
1566means the same thing as this:
1567
1568@example
1569exp: NUM
1570 | exp exp '+' @{ $$ = $1 + $2; @}
1571 | @dots{}
99a9344e 1572;
bfa74976
RS
1573@end example
1574
1575@noindent
1576The latter, however, is much more readable.
1577
342b8b6e 1578@node Rpcalc Lexer
bfa74976
RS
1579@subsection The @code{rpcalc} Lexical Analyzer
1580@cindex writing a lexical analyzer
1581@cindex lexical analyzer, writing
1582
704a47c4
AD
1583The lexical analyzer's job is low-level parsing: converting characters
1584or sequences of characters into tokens. The Bison parser gets its
1585tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1586Analyzer Function @code{yylex}}.
bfa74976 1587
c827f760
PE
1588Only a simple lexical analyzer is needed for the @acronym{RPN}
1589calculator. This
bfa74976
RS
1590lexical analyzer skips blanks and tabs, then reads in numbers as
1591@code{double} and returns them as @code{NUM} tokens. Any other character
1592that isn't part of a number is a separate token. Note that the token-code
1593for such a single-character token is the character itself.
1594
1595The return value of the lexical analyzer function is a numeric code which
1596represents a token type. The same text used in Bison rules to stand for
1597this token type is also a C expression for the numeric code for the type.
1598This works in two ways. If the token type is a character literal, then its
e966383b 1599numeric code is that of the character; you can use the same
bfa74976
RS
1600character literal in the lexical analyzer to express the number. If the
1601token type is an identifier, that identifier is defined by Bison as a C
1602macro whose definition is the appropriate number. In this example,
1603therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1604
1964ad8c
AD
1605The semantic value of the token (if it has one) is stored into the
1606global variable @code{yylval}, which is where the Bison parser will look
1607for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1608defined at the beginning of the grammar; @pxref{Rpcalc Decls,
1609,Declarations for @code{rpcalc}}.)
bfa74976 1610
72d2299c
PE
1611A token type code of zero is returned if the end-of-input is encountered.
1612(Bison recognizes any nonpositive value as indicating end-of-input.)
bfa74976
RS
1613
1614Here is the code for the lexical analyzer:
1615
1616@example
1617@group
72d2299c 1618/* The lexical analyzer returns a double floating point
e966383b 1619 number on the stack and the token NUM, or the numeric code
72d2299c
PE
1620 of the character read if not a number. It skips all blanks
1621 and tabs, and returns 0 for end-of-input. */
bfa74976
RS
1622
1623#include <ctype.h>
1624@end group
1625
1626@group
13863333
AD
1627int
1628yylex (void)
bfa74976
RS
1629@{
1630 int c;
1631
72d2299c 1632 /* Skip white space. */
13863333 1633 while ((c = getchar ()) == ' ' || c == '\t')
bfa74976
RS
1634 ;
1635@end group
1636@group
72d2299c 1637 /* Process numbers. */
13863333 1638 if (c == '.' || isdigit (c))
bfa74976
RS
1639 @{
1640 ungetc (c, stdin);
1641 scanf ("%lf", &yylval);
1642 return NUM;
1643 @}
1644@end group
1645@group
72d2299c 1646 /* Return end-of-input. */
13863333 1647 if (c == EOF)
bfa74976 1648 return 0;
72d2299c 1649 /* Return a single char. */
13863333 1650 return c;
bfa74976
RS
1651@}
1652@end group
1653@end example
1654
342b8b6e 1655@node Rpcalc Main
bfa74976
RS
1656@subsection The Controlling Function
1657@cindex controlling function
1658@cindex main function in simple example
1659
1660In keeping with the spirit of this example, the controlling function is
1661kept to the bare minimum. The only requirement is that it call
1662@code{yyparse} to start the process of parsing.
1663
1664@example
1665@group
13863333
AD
1666int
1667main (void)
bfa74976 1668@{
13863333 1669 return yyparse ();
bfa74976
RS
1670@}
1671@end group
1672@end example
1673
342b8b6e 1674@node Rpcalc Error
bfa74976
RS
1675@subsection The Error Reporting Routine
1676@cindex error reporting routine
1677
1678When @code{yyparse} detects a syntax error, it calls the error reporting
13863333 1679function @code{yyerror} to print an error message (usually but not
6e649e65 1680always @code{"syntax error"}). It is up to the programmer to supply
13863333
AD
1681@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1682here is the definition we will use:
bfa74976
RS
1683
1684@example
1685@group
1686#include <stdio.h>
1687
38a92d50 1688/* Called by yyparse on error. */
13863333 1689void
38a92d50 1690yyerror (char const *s)
bfa74976 1691@{
4e03e201 1692 fprintf (stderr, "%s\n", s);
bfa74976
RS
1693@}
1694@end group
1695@end example
1696
1697After @code{yyerror} returns, the Bison parser may recover from the error
1698and continue parsing if the grammar contains a suitable error rule
1699(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1700have not written any error rules in this example, so any invalid input will
1701cause the calculator program to exit. This is not clean behavior for a
9ecbd125 1702real calculator, but it is adequate for the first example.
bfa74976 1703
342b8b6e 1704@node Rpcalc Gen
bfa74976
RS
1705@subsection Running Bison to Make the Parser
1706@cindex running Bison (introduction)
1707
ceed8467
AD
1708Before running Bison to produce a parser, we need to decide how to
1709arrange all the source code in one or more source files. For such a
1710simple example, the easiest thing is to put everything in one file. The
1711definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
342b8b6e 1712end, in the epilogue of the file
75f5aaea 1713(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
bfa74976
RS
1714
1715For a large project, you would probably have several source files, and use
1716@code{make} to arrange to recompile them.
1717
1718With all the source in a single file, you use the following command to
1719convert it into a parser file:
1720
1721@example
fa4d969f 1722bison @var{file}.y
bfa74976
RS
1723@end example
1724
1725@noindent
1726In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
fa4d969f 1727@sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c},
72d2299c 1728removing the @samp{.y} from the original file name. The file output by
bfa74976
RS
1729Bison contains the source code for @code{yyparse}. The additional
1730functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
1731are copied verbatim to the output.
1732
342b8b6e 1733@node Rpcalc Compile
bfa74976
RS
1734@subsection Compiling the Parser File
1735@cindex compiling the parser
1736
1737Here is how to compile and run the parser file:
1738
1739@example
1740@group
1741# @r{List files in current directory.}
9edcd895 1742$ @kbd{ls}
bfa74976
RS
1743rpcalc.tab.c rpcalc.y
1744@end group
1745
1746@group
1747# @r{Compile the Bison parser.}
1748# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
b56471a6 1749$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
bfa74976
RS
1750@end group
1751
1752@group
1753# @r{List files again.}
9edcd895 1754$ @kbd{ls}
bfa74976
RS
1755rpcalc rpcalc.tab.c rpcalc.y
1756@end group
1757@end example
1758
1759The file @file{rpcalc} now contains the executable code. Here is an
1760example session using @code{rpcalc}.
1761
1762@example
9edcd895
AD
1763$ @kbd{rpcalc}
1764@kbd{4 9 +}
bfa74976 176513
9edcd895 1766@kbd{3 7 + 3 4 5 *+-}
bfa74976 1767-13
9edcd895 1768@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
bfa74976 176913
9edcd895 1770@kbd{5 6 / 4 n +}
bfa74976 1771-3.166666667
9edcd895 1772@kbd{3 4 ^} @r{Exponentiation}
bfa74976 177381
9edcd895
AD
1774@kbd{^D} @r{End-of-file indicator}
1775$
bfa74976
RS
1776@end example
1777
342b8b6e 1778@node Infix Calc
bfa74976
RS
1779@section Infix Notation Calculator: @code{calc}
1780@cindex infix notation calculator
1781@cindex @code{calc}
1782@cindex calculator, infix notation
1783
1784We now modify rpcalc to handle infix operators instead of postfix. Infix
1785notation involves the concept of operator precedence and the need for
1786parentheses nested to arbitrary depth. Here is the Bison code for
1787@file{calc.y}, an infix desk-top calculator.
1788
1789@example
38a92d50 1790/* Infix notation calculator. */
bfa74976
RS
1791
1792%@{
38a92d50
PE
1793 #define YYSTYPE double
1794 #include <math.h>
1795 #include <stdio.h>
1796 int yylex (void);
1797 void yyerror (char const *);
bfa74976
RS
1798%@}
1799
38a92d50 1800/* Bison declarations. */
bfa74976
RS
1801%token NUM
1802%left '-' '+'
1803%left '*' '/'
1804%left NEG /* negation--unary minus */
38a92d50 1805%right '^' /* exponentiation */
bfa74976 1806
38a92d50
PE
1807%% /* The grammar follows. */
1808input: /* empty */
bfa74976
RS
1809 | input line
1810;
1811
1812line: '\n'
1813 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1814;
1815
1816exp: NUM @{ $$ = $1; @}
1817 | exp '+' exp @{ $$ = $1 + $3; @}
1818 | exp '-' exp @{ $$ = $1 - $3; @}
1819 | exp '*' exp @{ $$ = $1 * $3; @}
1820 | exp '/' exp @{ $$ = $1 / $3; @}
1821 | '-' exp %prec NEG @{ $$ = -$2; @}
1822 | exp '^' exp @{ $$ = pow ($1, $3); @}
1823 | '(' exp ')' @{ $$ = $2; @}
1824;
1825%%
1826@end example
1827
1828@noindent
ceed8467
AD
1829The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1830same as before.
bfa74976
RS
1831
1832There are two important new features shown in this code.
1833
1834In the second section (Bison declarations), @code{%left} declares token
1835types and says they are left-associative operators. The declarations
1836@code{%left} and @code{%right} (right associativity) take the place of
1837@code{%token} which is used to declare a token type name without
1838associativity. (These tokens are single-character literals, which
1839ordinarily don't need to be declared. We declare them here to specify
1840the associativity.)
1841
1842Operator precedence is determined by the line ordering of the
1843declarations; the higher the line number of the declaration (lower on
1844the page or screen), the higher the precedence. Hence, exponentiation
1845has the highest precedence, unary minus (@code{NEG}) is next, followed
704a47c4
AD
1846by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator
1847Precedence}.
bfa74976 1848
704a47c4
AD
1849The other important new feature is the @code{%prec} in the grammar
1850section for the unary minus operator. The @code{%prec} simply instructs
1851Bison that the rule @samp{| '-' exp} has the same precedence as
1852@code{NEG}---in this case the next-to-highest. @xref{Contextual
1853Precedence, ,Context-Dependent Precedence}.
bfa74976
RS
1854
1855Here is a sample run of @file{calc.y}:
1856
1857@need 500
1858@example
9edcd895
AD
1859$ @kbd{calc}
1860@kbd{4 + 4.5 - (34/(8*3+-3))}
bfa74976 18616.880952381
9edcd895 1862@kbd{-56 + 2}
bfa74976 1863-54
9edcd895 1864@kbd{3 ^ 2}
bfa74976
RS
18659
1866@end example
1867
342b8b6e 1868@node Simple Error Recovery
bfa74976
RS
1869@section Simple Error Recovery
1870@cindex error recovery, simple
1871
1872Up to this point, this manual has not addressed the issue of @dfn{error
1873recovery}---how to continue parsing after the parser detects a syntax
ceed8467
AD
1874error. All we have handled is error reporting with @code{yyerror}.
1875Recall that by default @code{yyparse} returns after calling
1876@code{yyerror}. This means that an erroneous input line causes the
1877calculator program to exit. Now we show how to rectify this deficiency.
bfa74976
RS
1878
1879The Bison language itself includes the reserved word @code{error}, which
1880may be included in the grammar rules. In the example below it has
1881been added to one of the alternatives for @code{line}:
1882
1883@example
1884@group
1885line: '\n'
1886 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1887 | error '\n' @{ yyerrok; @}
1888;
1889@end group
1890@end example
1891
ceed8467 1892This addition to the grammar allows for simple error recovery in the
6e649e65 1893event of a syntax error. If an expression that cannot be evaluated is
ceed8467
AD
1894read, the error will be recognized by the third rule for @code{line},
1895and parsing will continue. (The @code{yyerror} function is still called
1896upon to print its message as well.) The action executes the statement
1897@code{yyerrok}, a macro defined automatically by Bison; its meaning is
1898that error recovery is complete (@pxref{Error Recovery}). Note the
1899difference between @code{yyerrok} and @code{yyerror}; neither one is a
e0c471a9 1900misprint.
bfa74976
RS
1901
1902This form of error recovery deals with syntax errors. There are other
1903kinds of errors; for example, division by zero, which raises an exception
1904signal that is normally fatal. A real calculator program must handle this
1905signal and use @code{longjmp} to return to @code{main} and resume parsing
1906input lines; it would also have to discard the rest of the current line of
1907input. We won't discuss this issue further because it is not specific to
1908Bison programs.
1909
342b8b6e
AD
1910@node Location Tracking Calc
1911@section Location Tracking Calculator: @code{ltcalc}
1912@cindex location tracking calculator
1913@cindex @code{ltcalc}
1914@cindex calculator, location tracking
1915
9edcd895
AD
1916This example extends the infix notation calculator with location
1917tracking. This feature will be used to improve the error messages. For
1918the sake of clarity, this example is a simple integer calculator, since
1919most of the work needed to use locations will be done in the lexical
72d2299c 1920analyzer.
342b8b6e
AD
1921
1922@menu
1923* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
1924* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
1925* Lexer: Ltcalc Lexer. The lexical analyzer.
1926@end menu
1927
1928@node Ltcalc Decls
1929@subsection Declarations for @code{ltcalc}
1930
9edcd895
AD
1931The C and Bison declarations for the location tracking calculator are
1932the same as the declarations for the infix notation calculator.
342b8b6e
AD
1933
1934@example
1935/* Location tracking calculator. */
1936
1937%@{
38a92d50
PE
1938 #define YYSTYPE int
1939 #include <math.h>
1940 int yylex (void);
1941 void yyerror (char const *);
342b8b6e
AD
1942%@}
1943
1944/* Bison declarations. */
1945%token NUM
1946
1947%left '-' '+'
1948%left '*' '/'
1949%left NEG
1950%right '^'
1951
38a92d50 1952%% /* The grammar follows. */
342b8b6e
AD
1953@end example
1954
9edcd895
AD
1955@noindent
1956Note there are no declarations specific to locations. Defining a data
1957type for storing locations is not needed: we will use the type provided
1958by default (@pxref{Location Type, ,Data Types of Locations}), which is a
1959four member structure with the following integer fields:
1960@code{first_line}, @code{first_column}, @code{last_line} and
1961@code{last_column}.
342b8b6e
AD
1962
1963@node Ltcalc Rules
1964@subsection Grammar Rules for @code{ltcalc}
1965
9edcd895
AD
1966Whether handling locations or not has no effect on the syntax of your
1967language. Therefore, grammar rules for this example will be very close
1968to those of the previous example: we will only modify them to benefit
1969from the new information.
342b8b6e 1970
9edcd895
AD
1971Here, we will use locations to report divisions by zero, and locate the
1972wrong expressions or subexpressions.
342b8b6e
AD
1973
1974@example
1975@group
1976input : /* empty */
1977 | input line
1978;
1979@end group
1980
1981@group
1982line : '\n'
1983 | exp '\n' @{ printf ("%d\n", $1); @}
1984;
1985@end group
1986
1987@group
1988exp : NUM @{ $$ = $1; @}
1989 | exp '+' exp @{ $$ = $1 + $3; @}
1990 | exp '-' exp @{ $$ = $1 - $3; @}
1991 | exp '*' exp @{ $$ = $1 * $3; @}
1992@end group
342b8b6e 1993@group
9edcd895 1994 | exp '/' exp
342b8b6e
AD
1995 @{
1996 if ($3)
1997 $$ = $1 / $3;
1998 else
1999 @{
2000 $$ = 1;
9edcd895
AD
2001 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2002 @@3.first_line, @@3.first_column,
2003 @@3.last_line, @@3.last_column);
342b8b6e
AD
2004 @}
2005 @}
2006@end group
2007@group
2008 | '-' exp %preg NEG @{ $$ = -$2; @}
2009 | exp '^' exp @{ $$ = pow ($1, $3); @}
2010 | '(' exp ')' @{ $$ = $2; @}
2011@end group
2012@end example
2013
2014This code shows how to reach locations inside of semantic actions, by
2015using the pseudo-variables @code{@@@var{n}} for rule components, and the
2016pseudo-variable @code{@@$} for groupings.
2017
9edcd895
AD
2018We don't need to assign a value to @code{@@$}: the output parser does it
2019automatically. By default, before executing the C code of each action,
2020@code{@@$} is set to range from the beginning of @code{@@1} to the end
2021of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2022can be redefined (@pxref{Location Default Action, , Default Action for
2023Locations}), and for very specific rules, @code{@@$} can be computed by
2024hand.
342b8b6e
AD
2025
2026@node Ltcalc Lexer
2027@subsection The @code{ltcalc} Lexical Analyzer.
2028
9edcd895 2029Until now, we relied on Bison's defaults to enable location
72d2299c 2030tracking. The next step is to rewrite the lexical analyzer, and make it
9edcd895
AD
2031able to feed the parser with the token locations, as it already does for
2032semantic values.
342b8b6e 2033
9edcd895
AD
2034To this end, we must take into account every single character of the
2035input text, to avoid the computed locations of being fuzzy or wrong:
342b8b6e
AD
2036
2037@example
2038@group
2039int
2040yylex (void)
2041@{
2042 int c;
18b519c0 2043@end group
342b8b6e 2044
18b519c0 2045@group
72d2299c 2046 /* Skip white space. */
342b8b6e
AD
2047 while ((c = getchar ()) == ' ' || c == '\t')
2048 ++yylloc.last_column;
18b519c0 2049@end group
342b8b6e 2050
18b519c0 2051@group
72d2299c 2052 /* Step. */
342b8b6e
AD
2053 yylloc.first_line = yylloc.last_line;
2054 yylloc.first_column = yylloc.last_column;
2055@end group
2056
2057@group
72d2299c 2058 /* Process numbers. */
342b8b6e
AD
2059 if (isdigit (c))
2060 @{
2061 yylval = c - '0';
2062 ++yylloc.last_column;
2063 while (isdigit (c = getchar ()))
2064 @{
2065 ++yylloc.last_column;
2066 yylval = yylval * 10 + c - '0';
2067 @}
2068 ungetc (c, stdin);
2069 return NUM;
2070 @}
2071@end group
2072
72d2299c 2073 /* Return end-of-input. */
342b8b6e
AD
2074 if (c == EOF)
2075 return 0;
2076
72d2299c 2077 /* Return a single char, and update location. */
342b8b6e
AD
2078 if (c == '\n')
2079 @{
2080 ++yylloc.last_line;
2081 yylloc.last_column = 0;
2082 @}
2083 else
2084 ++yylloc.last_column;
2085 return c;
2086@}
2087@end example
2088
9edcd895
AD
2089Basically, the lexical analyzer performs the same processing as before:
2090it skips blanks and tabs, and reads numbers or single-character tokens.
2091In addition, it updates @code{yylloc}, the global variable (of type
2092@code{YYLTYPE}) containing the token's location.
342b8b6e 2093
9edcd895 2094Now, each time this function returns a token, the parser has its number
72d2299c 2095as well as its semantic value, and its location in the text. The last
9edcd895
AD
2096needed change is to initialize @code{yylloc}, for example in the
2097controlling function:
342b8b6e
AD
2098
2099@example
9edcd895 2100@group
342b8b6e
AD
2101int
2102main (void)
2103@{
2104 yylloc.first_line = yylloc.last_line = 1;
2105 yylloc.first_column = yylloc.last_column = 0;
2106 return yyparse ();
2107@}
9edcd895 2108@end group
342b8b6e
AD
2109@end example
2110
9edcd895
AD
2111Remember that computing locations is not a matter of syntax. Every
2112character must be associated to a location update, whether it is in
2113valid input, in comments, in literal strings, and so on.
342b8b6e
AD
2114
2115@node Multi-function Calc
bfa74976
RS
2116@section Multi-Function Calculator: @code{mfcalc}
2117@cindex multi-function calculator
2118@cindex @code{mfcalc}
2119@cindex calculator, multi-function
2120
2121Now that the basics of Bison have been discussed, it is time to move on to
2122a more advanced problem. The above calculators provided only five
2123functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2124be nice to have a calculator that provides other mathematical functions such
2125as @code{sin}, @code{cos}, etc.
2126
2127It is easy to add new operators to the infix calculator as long as they are
2128only single-character literals. The lexical analyzer @code{yylex} passes
9ecbd125 2129back all nonnumber characters as tokens, so new grammar rules suffice for
bfa74976
RS
2130adding a new operator. But we want something more flexible: built-in
2131functions whose syntax has this form:
2132
2133@example
2134@var{function_name} (@var{argument})
2135@end example
2136
2137@noindent
2138At the same time, we will add memory to the calculator, by allowing you
2139to create named variables, store values in them, and use them later.
2140Here is a sample session with the multi-function calculator:
2141
2142@example
9edcd895
AD
2143$ @kbd{mfcalc}
2144@kbd{pi = 3.141592653589}
bfa74976 21453.1415926536
9edcd895 2146@kbd{sin(pi)}
bfa74976 21470.0000000000
9edcd895 2148@kbd{alpha = beta1 = 2.3}
bfa74976 21492.3000000000
9edcd895 2150@kbd{alpha}
bfa74976 21512.3000000000
9edcd895 2152@kbd{ln(alpha)}
bfa74976 21530.8329091229
9edcd895 2154@kbd{exp(ln(beta1))}
bfa74976 21552.3000000000
9edcd895 2156$
bfa74976
RS
2157@end example
2158
2159Note that multiple assignment and nested function calls are permitted.
2160
2161@menu
2162* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
2163* Rules: Mfcalc Rules. Grammar rules for the calculator.
2164* Symtab: Mfcalc Symtab. Symbol table management subroutines.
2165@end menu
2166
342b8b6e 2167@node Mfcalc Decl
bfa74976
RS
2168@subsection Declarations for @code{mfcalc}
2169
2170Here are the C and Bison declarations for the multi-function calculator.
2171
2172@smallexample
18b519c0 2173@group
bfa74976 2174%@{
38a92d50
PE
2175 #include <math.h> /* For math functions, cos(), sin(), etc. */
2176 #include "calc.h" /* Contains definition of `symrec'. */
2177 int yylex (void);
2178 void yyerror (char const *);
bfa74976 2179%@}
18b519c0
AD
2180@end group
2181@group
bfa74976 2182%union @{
38a92d50
PE
2183 double val; /* For returning numbers. */
2184 symrec *tptr; /* For returning symbol-table pointers. */
bfa74976 2185@}
18b519c0 2186@end group
38a92d50
PE
2187%token <val> NUM /* Simple double precision number. */
2188%token <tptr> VAR FNCT /* Variable and Function. */
bfa74976
RS
2189%type <val> exp
2190
18b519c0 2191@group
bfa74976
RS
2192%right '='
2193%left '-' '+'
2194%left '*' '/'
38a92d50
PE
2195%left NEG /* negation--unary minus */
2196%right '^' /* exponentiation */
18b519c0 2197@end group
38a92d50 2198%% /* The grammar follows. */
bfa74976
RS
2199@end smallexample
2200
2201The above grammar introduces only two new features of the Bison language.
2202These features allow semantic values to have various data types
2203(@pxref{Multiple Types, ,More Than One Value Type}).
2204
2205The @code{%union} declaration specifies the entire list of possible types;
2206this is instead of defining @code{YYSTYPE}. The allowable types are now
2207double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2208the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2209
2210Since values can now have various types, it is necessary to associate a
2211type with each grammar symbol whose semantic value is used. These symbols
2212are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2213declarations are augmented with information about their data type (placed
2214between angle brackets).
2215
704a47c4
AD
2216The Bison construct @code{%type} is used for declaring nonterminal
2217symbols, just as @code{%token} is used for declaring token types. We
2218have not used @code{%type} before because nonterminal symbols are
2219normally declared implicitly by the rules that define them. But
2220@code{exp} must be declared explicitly so we can specify its value type.
2221@xref{Type Decl, ,Nonterminal Symbols}.
bfa74976 2222
342b8b6e 2223@node Mfcalc Rules
bfa74976
RS
2224@subsection Grammar Rules for @code{mfcalc}
2225
2226Here are the grammar rules for the multi-function calculator.
2227Most of them are copied directly from @code{calc}; three rules,
2228those which mention @code{VAR} or @code{FNCT}, are new.
2229
2230@smallexample
18b519c0 2231@group
bfa74976
RS
2232input: /* empty */
2233 | input line
2234;
18b519c0 2235@end group
bfa74976 2236
18b519c0 2237@group
bfa74976
RS
2238line:
2239 '\n'
2240 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2241 | error '\n' @{ yyerrok; @}
2242;
18b519c0 2243@end group
bfa74976 2244
18b519c0 2245@group
bfa74976
RS
2246exp: NUM @{ $$ = $1; @}
2247 | VAR @{ $$ = $1->value.var; @}
2248 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2249 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2250 | exp '+' exp @{ $$ = $1 + $3; @}
2251 | exp '-' exp @{ $$ = $1 - $3; @}
2252 | exp '*' exp @{ $$ = $1 * $3; @}
2253 | exp '/' exp @{ $$ = $1 / $3; @}
2254 | '-' exp %prec NEG @{ $$ = -$2; @}
2255 | exp '^' exp @{ $$ = pow ($1, $3); @}
2256 | '(' exp ')' @{ $$ = $2; @}
2257;
18b519c0 2258@end group
38a92d50 2259/* End of grammar. */
bfa74976
RS
2260%%
2261@end smallexample
2262
342b8b6e 2263@node Mfcalc Symtab
bfa74976
RS
2264@subsection The @code{mfcalc} Symbol Table
2265@cindex symbol table example
2266
2267The multi-function calculator requires a symbol table to keep track of the
2268names and meanings of variables and functions. This doesn't affect the
2269grammar rules (except for the actions) or the Bison declarations, but it
2270requires some additional C functions for support.
2271
2272The symbol table itself consists of a linked list of records. Its
2273definition, which is kept in the header @file{calc.h}, is as follows. It
2274provides for either functions or variables to be placed in the table.
2275
2276@smallexample
2277@group
38a92d50 2278/* Function type. */
32dfccf8 2279typedef double (*func_t) (double);
72f889cc 2280@end group
32dfccf8 2281
72f889cc 2282@group
38a92d50 2283/* Data type for links in the chain of symbols. */
bfa74976
RS
2284struct symrec
2285@{
38a92d50 2286 char *name; /* name of symbol */
bfa74976 2287 int type; /* type of symbol: either VAR or FNCT */
32dfccf8
AD
2288 union
2289 @{
38a92d50
PE
2290 double var; /* value of a VAR */
2291 func_t fnctptr; /* value of a FNCT */
bfa74976 2292 @} value;
38a92d50 2293 struct symrec *next; /* link field */
bfa74976
RS
2294@};
2295@end group
2296
2297@group
2298typedef struct symrec symrec;
2299
38a92d50 2300/* The symbol table: a chain of `struct symrec'. */
bfa74976
RS
2301extern symrec *sym_table;
2302
a730d142 2303symrec *putsym (char const *, int);
38a92d50 2304symrec *getsym (char const *);
bfa74976
RS
2305@end group
2306@end smallexample
2307
2308The new version of @code{main} includes a call to @code{init_table}, a
2309function that initializes the symbol table. Here it is, and
2310@code{init_table} as well:
2311
2312@smallexample
bfa74976
RS
2313#include <stdio.h>
2314
18b519c0 2315@group
38a92d50 2316/* Called by yyparse on error. */
13863333 2317void
38a92d50 2318yyerror (char const *s)
bfa74976
RS
2319@{
2320 printf ("%s\n", s);
2321@}
18b519c0 2322@end group
bfa74976 2323
18b519c0 2324@group
bfa74976
RS
2325struct init
2326@{
38a92d50
PE
2327 char const *fname;
2328 double (*fnct) (double);
bfa74976
RS
2329@};
2330@end group
2331
2332@group
38a92d50 2333struct init const arith_fncts[] =
13863333 2334@{
32dfccf8
AD
2335 "sin", sin,
2336 "cos", cos,
13863333 2337 "atan", atan,
32dfccf8
AD
2338 "ln", log,
2339 "exp", exp,
13863333
AD
2340 "sqrt", sqrt,
2341 0, 0
2342@};
18b519c0 2343@end group
bfa74976 2344
18b519c0 2345@group
bfa74976 2346/* The symbol table: a chain of `struct symrec'. */
38a92d50 2347symrec *sym_table;
bfa74976
RS
2348@end group
2349
2350@group
72d2299c 2351/* Put arithmetic functions in table. */
13863333
AD
2352void
2353init_table (void)
bfa74976
RS
2354@{
2355 int i;
2356 symrec *ptr;
2357 for (i = 0; arith_fncts[i].fname != 0; i++)
2358 @{
2359 ptr = putsym (arith_fncts[i].fname, FNCT);
2360 ptr->value.fnctptr = arith_fncts[i].fnct;
2361 @}
2362@}
2363@end group
38a92d50
PE
2364
2365@group
2366int
2367main (void)
2368@{
2369 init_table ();
2370 return yyparse ();
2371@}
2372@end group
bfa74976
RS
2373@end smallexample
2374
2375By simply editing the initialization list and adding the necessary include
2376files, you can add additional functions to the calculator.
2377
2378Two important functions allow look-up and installation of symbols in the
2379symbol table. The function @code{putsym} is passed a name and the type
2380(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2381linked to the front of the list, and a pointer to the object is returned.
2382The function @code{getsym} is passed the name of the symbol to look up. If
2383found, a pointer to that symbol is returned; otherwise zero is returned.
2384
2385@smallexample
2386symrec *
38a92d50 2387putsym (char const *sym_name, int sym_type)
bfa74976
RS
2388@{
2389 symrec *ptr;
2390 ptr = (symrec *) malloc (sizeof (symrec));
2391 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2392 strcpy (ptr->name,sym_name);
2393 ptr->type = sym_type;
72d2299c 2394 ptr->value.var = 0; /* Set value to 0 even if fctn. */
bfa74976
RS
2395 ptr->next = (struct symrec *)sym_table;
2396 sym_table = ptr;
2397 return ptr;
2398@}
2399
2400symrec *
38a92d50 2401getsym (char const *sym_name)
bfa74976
RS
2402@{
2403 symrec *ptr;
2404 for (ptr = sym_table; ptr != (symrec *) 0;
2405 ptr = (symrec *)ptr->next)
2406 if (strcmp (ptr->name,sym_name) == 0)
2407 return ptr;
2408 return 0;
2409@}
2410@end smallexample
2411
2412The function @code{yylex} must now recognize variables, numeric values, and
2413the single-character arithmetic operators. Strings of alphanumeric
14ded682 2414characters with a leading non-digit are recognized as either variables or
bfa74976
RS
2415functions depending on what the symbol table says about them.
2416
2417The string is passed to @code{getsym} for look up in the symbol table. If
2418the name appears in the table, a pointer to its location and its type
2419(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2420already in the table, then it is installed as a @code{VAR} using
2421@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
e0c471a9 2422returned to @code{yyparse}.
bfa74976
RS
2423
2424No change is needed in the handling of numeric values and arithmetic
2425operators in @code{yylex}.
2426
2427@smallexample
2428@group
2429#include <ctype.h>
18b519c0 2430@end group
13863333 2431
18b519c0 2432@group
13863333
AD
2433int
2434yylex (void)
bfa74976
RS
2435@{
2436 int c;
2437
72d2299c 2438 /* Ignore white space, get first nonwhite character. */
bfa74976
RS
2439 while ((c = getchar ()) == ' ' || c == '\t');
2440
2441 if (c == EOF)
2442 return 0;
2443@end group
2444
2445@group
2446 /* Char starts a number => parse the number. */
2447 if (c == '.' || isdigit (c))
2448 @{
2449 ungetc (c, stdin);
2450 scanf ("%lf", &yylval.val);
2451 return NUM;
2452 @}
2453@end group
2454
2455@group
2456 /* Char starts an identifier => read the name. */
2457 if (isalpha (c))
2458 @{
2459 symrec *s;
2460 static char *symbuf = 0;
2461 static int length = 0;
2462 int i;
2463@end group
2464
2465@group
2466 /* Initially make the buffer long enough
2467 for a 40-character symbol name. */
2468 if (length == 0)
2469 length = 40, symbuf = (char *)malloc (length + 1);
2470
2471 i = 0;
2472 do
2473@end group
2474@group
2475 @{
2476 /* If buffer is full, make it bigger. */
2477 if (i == length)
2478 @{
2479 length *= 2;
18b519c0 2480 symbuf = (char *) realloc (symbuf, length + 1);
bfa74976
RS
2481 @}
2482 /* Add this character to the buffer. */
2483 symbuf[i++] = c;
2484 /* Get another character. */
2485 c = getchar ();
2486 @}
2487@end group
2488@group
72d2299c 2489 while (isalnum (c));
bfa74976
RS
2490
2491 ungetc (c, stdin);
2492 symbuf[i] = '\0';
2493@end group
2494
2495@group
2496 s = getsym (symbuf);
2497 if (s == 0)
2498 s = putsym (symbuf, VAR);
2499 yylval.tptr = s;
2500 return s->type;
2501 @}
2502
2503 /* Any other character is a token by itself. */
2504 return c;
2505@}
2506@end group
2507@end smallexample
2508
72d2299c 2509This program is both powerful and flexible. You may easily add new
704a47c4
AD
2510functions, and it is a simple job to modify this code to install
2511predefined variables such as @code{pi} or @code{e} as well.
bfa74976 2512
342b8b6e 2513@node Exercises
bfa74976
RS
2514@section Exercises
2515@cindex exercises
2516
2517@enumerate
2518@item
2519Add some new functions from @file{math.h} to the initialization list.
2520
2521@item
2522Add another array that contains constants and their values. Then
2523modify @code{init_table} to add these constants to the symbol table.
2524It will be easiest to give the constants type @code{VAR}.
2525
2526@item
2527Make the program report an error if the user refers to an
2528uninitialized variable in any way except to store a value in it.
2529@end enumerate
2530
342b8b6e 2531@node Grammar File
bfa74976
RS
2532@chapter Bison Grammar Files
2533
2534Bison takes as input a context-free grammar specification and produces a
2535C-language function that recognizes correct instances of the grammar.
2536
2537The Bison grammar input file conventionally has a name ending in @samp{.y}.
234a3be3 2538@xref{Invocation, ,Invoking Bison}.
bfa74976
RS
2539
2540@menu
2541* Grammar Outline:: Overall layout of the grammar file.
2542* Symbols:: Terminal and nonterminal symbols.
2543* Rules:: How to write grammar rules.
2544* Recursion:: Writing recursive rules.
2545* Semantics:: Semantic values and actions.
847bf1f5 2546* Locations:: Locations and actions.
bfa74976
RS
2547* Declarations:: All kinds of Bison declarations are described here.
2548* Multiple Parsers:: Putting more than one Bison parser in one program.
2549@end menu
2550
342b8b6e 2551@node Grammar Outline
bfa74976
RS
2552@section Outline of a Bison Grammar
2553
2554A Bison grammar file has four main sections, shown here with the
2555appropriate delimiters:
2556
2557@example
2558%@{
38a92d50 2559 @var{Prologue}
bfa74976
RS
2560%@}
2561
2562@var{Bison declarations}
2563
2564%%
2565@var{Grammar rules}
2566%%
2567
75f5aaea 2568@var{Epilogue}
bfa74976
RS
2569@end example
2570
2571Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2bfc2e2a
PE
2572As a @acronym{GNU} extension, @samp{//} introduces a comment that
2573continues until end of line.
bfa74976
RS
2574
2575@menu
75f5aaea 2576* Prologue:: Syntax and usage of the prologue.
bfa74976
RS
2577* Bison Declarations:: Syntax and usage of the Bison declarations section.
2578* Grammar Rules:: Syntax and usage of the grammar rules section.
75f5aaea 2579* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
2580@end menu
2581
38a92d50 2582@node Prologue
75f5aaea
MA
2583@subsection The prologue
2584@cindex declarations section
2585@cindex Prologue
2586@cindex declarations
bfa74976 2587
08e49d20 2588The @var{Prologue} section contains macro definitions and
bfa74976
RS
2589declarations of functions and variables that are used in the actions in the
2590grammar rules. These are copied to the beginning of the parser file so
2591that they precede the definition of @code{yyparse}. You can use
2592@samp{#include} to get the declarations from a header file. If you don't
2593need any C declarations, you may omit the @samp{%@{} and @samp{%@}}
2594delimiters that bracket this section.
2595
c732d2c6
AD
2596You may have more than one @var{Prologue} section, intermixed with the
2597@var{Bison declarations}. This allows you to have C and Bison
2598declarations that refer to each other. For example, the @code{%union}
2599declaration may use types defined in a header file, and you may wish to
2600prototype functions that take arguments of type @code{YYSTYPE}. This
2601can be done with two @var{Prologue} blocks, one before and one after the
2602@code{%union} declaration.
2603
2604@smallexample
2605%@{
38a92d50
PE
2606 #include <stdio.h>
2607 #include "ptypes.h"
c732d2c6
AD
2608%@}
2609
2610%union @{
779e7ceb 2611 long int n;
c732d2c6
AD
2612 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2613@}
2614
2615%@{
38a92d50
PE
2616 static void print_token_value (FILE *, int, YYSTYPE);
2617 #define YYPRINT(F, N, L) print_token_value (F, N, L)
c732d2c6
AD
2618%@}
2619
2620@dots{}
2621@end smallexample
2622
342b8b6e 2623@node Bison Declarations
bfa74976
RS
2624@subsection The Bison Declarations Section
2625@cindex Bison declarations (introduction)
2626@cindex declarations, Bison (introduction)
2627
2628The @var{Bison declarations} section contains declarations that define
2629terminal and nonterminal symbols, specify precedence, and so on.
2630In some simple grammars you may not need any declarations.
2631@xref{Declarations, ,Bison Declarations}.
2632
342b8b6e 2633@node Grammar Rules
bfa74976
RS
2634@subsection The Grammar Rules Section
2635@cindex grammar rules section
2636@cindex rules section for grammar
2637
2638The @dfn{grammar rules} section contains one or more Bison grammar
2639rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
2640
2641There must always be at least one grammar rule, and the first
2642@samp{%%} (which precedes the grammar rules) may never be omitted even
2643if it is the first thing in the file.
2644
38a92d50 2645@node Epilogue
75f5aaea 2646@subsection The epilogue
bfa74976 2647@cindex additional C code section
75f5aaea 2648@cindex epilogue
bfa74976
RS
2649@cindex C code, section for additional
2650
08e49d20
PE
2651The @var{Epilogue} is copied verbatim to the end of the parser file, just as
2652the @var{Prologue} is copied to the beginning. This is the most convenient
342b8b6e
AD
2653place to put anything that you want to have in the parser file but which need
2654not come before the definition of @code{yyparse}. For example, the
38a92d50
PE
2655definitions of @code{yylex} and @code{yyerror} often go here. Because
2656C requires functions to be declared before being used, you often need
2657to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
e4f85c39 2658even if you define them in the Epilogue.
75f5aaea 2659@xref{Interface, ,Parser C-Language Interface}.
bfa74976
RS
2660
2661If the last section is empty, you may omit the @samp{%%} that separates it
2662from the grammar rules.
2663
38a92d50
PE
2664The Bison parser itself contains many macros and identifiers whose
2665names start with @samp{yy} or @samp{YY}, so it is a
bfa74976 2666good idea to avoid using any such names (except those documented in this
75f5aaea 2667manual) in the epilogue of the grammar file.
bfa74976 2668
342b8b6e 2669@node Symbols
bfa74976
RS
2670@section Symbols, Terminal and Nonterminal
2671@cindex nonterminal symbol
2672@cindex terminal symbol
2673@cindex token type
2674@cindex symbol
2675
2676@dfn{Symbols} in Bison grammars represent the grammatical classifications
2677of the language.
2678
2679A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
2680class of syntactically equivalent tokens. You use the symbol in grammar
2681rules to mean that a token in that class is allowed. The symbol is
2682represented in the Bison parser by a numeric code, and the @code{yylex}
2683function returns a token type code to indicate what kind of token has been
2684read. You don't need to know what the code value is; you can use the
2685symbol to stand for it.
2686
2687A @dfn{nonterminal symbol} stands for a class of syntactically equivalent
2688groupings. The symbol name is used in writing grammar rules. By convention,
2689it should be all lower case.
2690
2691Symbol names can contain letters, digits (not at the beginning),
2692underscores and periods. Periods make sense only in nonterminals.
2693
931c7513 2694There are three ways of writing terminal symbols in the grammar:
bfa74976
RS
2695
2696@itemize @bullet
2697@item
2698A @dfn{named token type} is written with an identifier, like an
c827f760 2699identifier in C@. By convention, it should be all upper case. Each
bfa74976
RS
2700such name must be defined with a Bison declaration such as
2701@code{%token}. @xref{Token Decl, ,Token Type Names}.
2702
2703@item
2704@cindex character token
2705@cindex literal token
2706@cindex single-character literal
931c7513
RS
2707A @dfn{character token type} (or @dfn{literal character token}) is
2708written in the grammar using the same syntax used in C for character
2709constants; for example, @code{'+'} is a character token type. A
2710character token type doesn't need to be declared unless you need to
2711specify its semantic value data type (@pxref{Value Type, ,Data Types of
2712Semantic Values}), associativity, or precedence (@pxref{Precedence,
2713,Operator Precedence}).
bfa74976
RS
2714
2715By convention, a character token type is used only to represent a
2716token that consists of that particular character. Thus, the token
2717type @code{'+'} is used to represent the character @samp{+} as a
2718token. Nothing enforces this convention, but if you depart from it,
2719your program will confuse other readers.
2720
2721All the usual escape sequences used in character literals in C can be
2722used in Bison as well, but you must not use the null character as a
72d2299c
PE
2723character literal because its numeric code, zero, signifies
2724end-of-input (@pxref{Calling Convention, ,Calling Convention
2bfc2e2a
PE
2725for @code{yylex}}). Also, unlike standard C, trigraphs have no
2726special meaning in Bison character literals, nor is backslash-newline
2727allowed.
931c7513
RS
2728
2729@item
2730@cindex string token
2731@cindex literal string token
9ecbd125 2732@cindex multicharacter literal
931c7513
RS
2733A @dfn{literal string token} is written like a C string constant; for
2734example, @code{"<="} is a literal string token. A literal string token
2735doesn't need to be declared unless you need to specify its semantic
14ded682 2736value data type (@pxref{Value Type}), associativity, or precedence
931c7513
RS
2737(@pxref{Precedence}).
2738
2739You can associate the literal string token with a symbolic name as an
2740alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
2741Declarations}). If you don't do that, the lexical analyzer has to
2742retrieve the token number for the literal string token from the
2743@code{yytname} table (@pxref{Calling Convention}).
2744
c827f760 2745@strong{Warning}: literal string tokens do not work in Yacc.
931c7513
RS
2746
2747By convention, a literal string token is used only to represent a token
2748that consists of that particular string. Thus, you should use the token
2749type @code{"<="} to represent the string @samp{<=} as a token. Bison
9ecbd125 2750does not enforce this convention, but if you depart from it, people who
931c7513
RS
2751read your program will be confused.
2752
2753All the escape sequences used in string literals in C can be used in
92ac3705
PE
2754Bison as well, except that you must not use a null character within a
2755string literal. Also, unlike Standard C, trigraphs have no special
2bfc2e2a
PE
2756meaning in Bison string literals, nor is backslash-newline allowed. A
2757literal string token must contain two or more characters; for a token
2758containing just one character, use a character token (see above).
bfa74976
RS
2759@end itemize
2760
2761How you choose to write a terminal symbol has no effect on its
2762grammatical meaning. That depends only on where it appears in rules and
2763on when the parser function returns that symbol.
2764
72d2299c
PE
2765The value returned by @code{yylex} is always one of the terminal
2766symbols, except that a zero or negative value signifies end-of-input.
2767Whichever way you write the token type in the grammar rules, you write
2768it the same way in the definition of @code{yylex}. The numeric code
2769for a character token type is simply the positive numeric code of the
2770character, so @code{yylex} can use the identical value to generate the
2771requisite code, though you may need to convert it to @code{unsigned
2772char} to avoid sign-extension on hosts where @code{char} is signed.
2773Each named token type becomes a C macro in
bfa74976 2774the parser file, so @code{yylex} can use the name to stand for the code.
13863333 2775(This is why periods don't make sense in terminal symbols.)
bfa74976
RS
2776@xref{Calling Convention, ,Calling Convention for @code{yylex}}.
2777
2778If @code{yylex} is defined in a separate file, you need to arrange for the
2779token-type macro definitions to be available there. Use the @samp{-d}
2780option when you run Bison, so that it will write these macro definitions
2781into a separate header file @file{@var{name}.tab.h} which you can include
2782in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
2783
72d2299c
PE
2784If you want to write a grammar that is portable to any Standard C
2785host, you must use only non-null character tokens taken from the basic
c827f760 2786execution character set of Standard C@. This set consists of the ten
72d2299c
PE
2787digits, the 52 lower- and upper-case English letters, and the
2788characters in the following C-language string:
2789
2790@example
2791"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
2792@end example
2793
2794The @code{yylex} function and Bison must use a consistent character
2795set and encoding for character tokens. For example, if you run Bison in an
c827f760 2796@acronym{ASCII} environment, but then compile and run the resulting program
e966383b 2797in an environment that uses an incompatible character set like
c827f760
PE
2798@acronym{EBCDIC}, the resulting program may not work because the
2799tables generated by Bison will assume @acronym{ASCII} numeric values for
72d2299c 2800character tokens. It is standard
e966383b 2801practice for software distributions to contain C source files that
c827f760
PE
2802were generated by Bison in an @acronym{ASCII} environment, so installers on
2803platforms that are incompatible with @acronym{ASCII} must rebuild those
e966383b
PE
2804files before compiling them.
2805
bfa74976
RS
2806The symbol @code{error} is a terminal symbol reserved for error recovery
2807(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
23c5a174
AD
2808In particular, @code{yylex} should never return this value. The default
2809value of the error token is 256, unless you explicitly assigned 256 to
2810one of your tokens with a @code{%token} declaration.
bfa74976 2811
342b8b6e 2812@node Rules
bfa74976
RS
2813@section Syntax of Grammar Rules
2814@cindex rule syntax
2815@cindex grammar rule syntax
2816@cindex syntax of grammar rules
2817
2818A Bison grammar rule has the following general form:
2819
2820@example
e425e872 2821@group
bfa74976
RS
2822@var{result}: @var{components}@dots{}
2823 ;
e425e872 2824@end group
bfa74976
RS
2825@end example
2826
2827@noindent
9ecbd125 2828where @var{result} is the nonterminal symbol that this rule describes,
bfa74976 2829and @var{components} are various terminal and nonterminal symbols that
13863333 2830are put together by this rule (@pxref{Symbols}).
bfa74976
RS
2831
2832For example,
2833
2834@example
2835@group
2836exp: exp '+' exp
2837 ;
2838@end group
2839@end example
2840
2841@noindent
2842says that two groupings of type @code{exp}, with a @samp{+} token in between,
2843can be combined into a larger grouping of type @code{exp}.
2844
72d2299c
PE
2845White space in rules is significant only to separate symbols. You can add
2846extra white space as you wish.
bfa74976
RS
2847
2848Scattered among the components can be @var{actions} that determine
2849the semantics of the rule. An action looks like this:
2850
2851@example
2852@{@var{C statements}@}
2853@end example
2854
2855@noindent
2856Usually there is only one action and it follows the components.
2857@xref{Actions}.
2858
2859@findex |
2860Multiple rules for the same @var{result} can be written separately or can
2861be joined with the vertical-bar character @samp{|} as follows:
2862
2863@ifinfo
2864@example
2865@var{result}: @var{rule1-components}@dots{}
2866 | @var{rule2-components}@dots{}
2867 @dots{}
2868 ;
2869@end example
2870@end ifinfo
2871@iftex
2872@example
2873@group
2874@var{result}: @var{rule1-components}@dots{}
2875 | @var{rule2-components}@dots{}
2876 @dots{}
2877 ;
2878@end group
2879@end example
2880@end iftex
2881
2882@noindent
2883They are still considered distinct rules even when joined in this way.
2884
2885If @var{components} in a rule is empty, it means that @var{result} can
2886match the empty string. For example, here is how to define a
2887comma-separated sequence of zero or more @code{exp} groupings:
2888
2889@example
2890@group
2891expseq: /* empty */
2892 | expseq1
2893 ;
2894@end group
2895
2896@group
2897expseq1: exp
2898 | expseq1 ',' exp
2899 ;
2900@end group
2901@end example
2902
2903@noindent
2904It is customary to write a comment @samp{/* empty */} in each rule
2905with no components.
2906
342b8b6e 2907@node Recursion
bfa74976
RS
2908@section Recursive Rules
2909@cindex recursive rule
2910
2911A rule is called @dfn{recursive} when its @var{result} nonterminal appears
2912also on its right hand side. Nearly all Bison grammars need to use
2913recursion, because that is the only way to define a sequence of any number
9ecbd125
JT
2914of a particular thing. Consider this recursive definition of a
2915comma-separated sequence of one or more expressions:
bfa74976
RS
2916
2917@example
2918@group
2919expseq1: exp
2920 | expseq1 ',' exp
2921 ;
2922@end group
2923@end example
2924
2925@cindex left recursion
2926@cindex right recursion
2927@noindent
2928Since the recursive use of @code{expseq1} is the leftmost symbol in the
2929right hand side, we call this @dfn{left recursion}. By contrast, here
2930the same construct is defined using @dfn{right recursion}:
2931
2932@example
2933@group
2934expseq1: exp
2935 | exp ',' expseq1
2936 ;
2937@end group
2938@end example
2939
2940@noindent
ec3bc396
AD
2941Any kind of sequence can be defined using either left recursion or right
2942recursion, but you should always use left recursion, because it can
2943parse a sequence of any number of elements with bounded stack space.
2944Right recursion uses up space on the Bison stack in proportion to the
2945number of elements in the sequence, because all the elements must be
2946shifted onto the stack before the rule can be applied even once.
2947@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
2948of this.
bfa74976
RS
2949
2950@cindex mutual recursion
2951@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
2952rule does not appear directly on its right hand side, but does appear
2953in rules for other nonterminals which do appear on its right hand
13863333 2954side.
bfa74976
RS
2955
2956For example:
2957
2958@example
2959@group
2960expr: primary
2961 | primary '+' primary
2962 ;
2963@end group
2964
2965@group
2966primary: constant
2967 | '(' expr ')'
2968 ;
2969@end group
2970@end example
2971
2972@noindent
2973defines two mutually-recursive nonterminals, since each refers to the
2974other.
2975
342b8b6e 2976@node Semantics
bfa74976
RS
2977@section Defining Language Semantics
2978@cindex defining language semantics
13863333 2979@cindex language semantics, defining
bfa74976
RS
2980
2981The grammar rules for a language determine only the syntax. The semantics
2982are determined by the semantic values associated with various tokens and
2983groupings, and by the actions taken when various groupings are recognized.
2984
2985For example, the calculator calculates properly because the value
2986associated with each expression is the proper number; it adds properly
2987because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
2988the numbers associated with @var{x} and @var{y}.
2989
2990@menu
2991* Value Type:: Specifying one data type for all semantic values.
2992* Multiple Types:: Specifying several alternative data types.
2993* Actions:: An action is the semantic definition of a grammar rule.
2994* Action Types:: Specifying data types for actions to operate on.
2995* Mid-Rule Actions:: Most actions go at the end of a rule.
2996 This says when, why and how to use the exceptional
2997 action in the middle of a rule.
2998@end menu
2999
342b8b6e 3000@node Value Type
bfa74976
RS
3001@subsection Data Types of Semantic Values
3002@cindex semantic value type
3003@cindex value type, semantic
3004@cindex data types of semantic values
3005@cindex default data type
3006
3007In a simple program it may be sufficient to use the same data type for
3008the semantic values of all language constructs. This was true in the
c827f760 3009@acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
1964ad8c 3010Notation Calculator}).
bfa74976
RS
3011
3012Bison's default is to use type @code{int} for all semantic values. To
3013specify some other type, define @code{YYSTYPE} as a macro, like this:
3014
3015@example
3016#define YYSTYPE double
3017@end example
3018
3019@noindent
342b8b6e 3020This macro definition must go in the prologue of the grammar file
75f5aaea 3021(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
bfa74976 3022
342b8b6e 3023@node Multiple Types
bfa74976
RS
3024@subsection More Than One Value Type
3025
3026In most programs, you will need different data types for different kinds
3027of tokens and groupings. For example, a numeric constant may need type
779e7ceb 3028@code{int} or @code{long int}, while a string constant needs type @code{char *},
bfa74976
RS
3029and an identifier might need a pointer to an entry in the symbol table.
3030
3031To use more than one data type for semantic values in one parser, Bison
3032requires you to do two things:
3033
3034@itemize @bullet
3035@item
3036Specify the entire collection of possible data types, with the
704a47c4
AD
3037@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
3038Value Types}).
bfa74976
RS
3039
3040@item
14ded682
AD
3041Choose one of those types for each symbol (terminal or nonterminal) for
3042which semantic values are used. This is done for tokens with the
3043@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3044and for groupings with the @code{%type} Bison declaration (@pxref{Type
3045Decl, ,Nonterminal Symbols}).
bfa74976
RS
3046@end itemize
3047
342b8b6e 3048@node Actions
bfa74976
RS
3049@subsection Actions
3050@cindex action
3051@vindex $$
3052@vindex $@var{n}
3053
3054An action accompanies a syntactic rule and contains C code to be executed
3055each time an instance of that rule is recognized. The task of most actions
3056is to compute a semantic value for the grouping built by the rule from the
3057semantic values associated with tokens or smaller groupings.
3058
3059An action consists of C statements surrounded by braces, much like a
2bfc2e2a
PE
3060compound statement in C@. An action can contain any sequence of C
3061statements. Bison does not look for trigraphs, though, so if your C
3062code uses trigraphs you should ensure that they do not affect the
3063nesting of braces or the boundaries of comments, strings, or character
3064literals.
3065
3066An action can be placed at any position in the rule;
704a47c4
AD
3067it is executed at that position. Most rules have just one action at the
3068end of the rule, following all the components. Actions in the middle of
3069a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3070Actions, ,Actions in Mid-Rule}).
bfa74976
RS
3071
3072The C code in an action can refer to the semantic values of the components
3073matched by the rule with the construct @code{$@var{n}}, which stands for
3074the value of the @var{n}th component. The semantic value for the grouping
0cc3da3a
PE
3075being constructed is @code{$$}. Bison translates both of these
3076constructs into expressions of the appropriate type when it copies the
3077actions into the parser file. @code{$$} is translated to a modifiable
3078lvalue, so it can be assigned to.
bfa74976
RS
3079
3080Here is a typical example:
3081
3082@example
3083@group
3084exp: @dots{}
3085 | exp '+' exp
3086 @{ $$ = $1 + $3; @}
3087@end group
3088@end example
3089
3090@noindent
3091This rule constructs an @code{exp} from two smaller @code{exp} groupings
3092connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3093refer to the semantic values of the two component @code{exp} groupings,
3094which are the first and third symbols on the right hand side of the rule.
3095The sum is stored into @code{$$} so that it becomes the semantic value of
3096the addition-expression just recognized by the rule. If there were a
3097useful semantic value associated with the @samp{+} token, it could be
e0c471a9 3098referred to as @code{$2}.
bfa74976 3099
3ded9a63
AD
3100Note that the vertical-bar character @samp{|} is really a rule
3101separator, and actions are attached to a single rule. This is a
3102difference with tools like Flex, for which @samp{|} stands for either
3103``or'', or ``the same action as that of the next rule''. In the
3104following example, the action is triggered only when @samp{b} is found:
3105
3106@example
3107@group
3108a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3109@end group
3110@end example
3111
bfa74976
RS
3112@cindex default action
3113If you don't specify an action for a rule, Bison supplies a default:
72f889cc
AD
3114@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3115becomes the value of the whole rule. Of course, the default action is
3116valid only if the two data types match. There is no meaningful default
3117action for an empty rule; every empty rule must have an explicit action
3118unless the rule's value does not matter.
bfa74976
RS
3119
3120@code{$@var{n}} with @var{n} zero or negative is allowed for reference
3121to tokens and groupings on the stack @emph{before} those that match the
3122current rule. This is a very risky practice, and to use it reliably
3123you must be certain of the context in which the rule is applied. Here
3124is a case in which you can use this reliably:
3125
3126@example
3127@group
3128foo: expr bar '+' expr @{ @dots{} @}
3129 | expr bar '-' expr @{ @dots{} @}
3130 ;
3131@end group
3132
3133@group
3134bar: /* empty */
3135 @{ previous_expr = $0; @}
3136 ;
3137@end group
3138@end example
3139
3140As long as @code{bar} is used only in the fashion shown here, @code{$0}
3141always refers to the @code{expr} which precedes @code{bar} in the
3142definition of @code{foo}.
3143
342b8b6e 3144@node Action Types
bfa74976
RS
3145@subsection Data Types of Values in Actions
3146@cindex action data types
3147@cindex data types in actions
3148
3149If you have chosen a single data type for semantic values, the @code{$$}
3150and @code{$@var{n}} constructs always have that data type.
3151
3152If you have used @code{%union} to specify a variety of data types, then you
3153must declare a choice among these types for each terminal or nonterminal
3154symbol that can have a semantic value. Then each time you use @code{$$} or
3155@code{$@var{n}}, its data type is determined by which symbol it refers to
e0c471a9 3156in the rule. In this example,
bfa74976
RS
3157
3158@example
3159@group
3160exp: @dots{}
3161 | exp '+' exp
3162 @{ $$ = $1 + $3; @}
3163@end group
3164@end example
3165
3166@noindent
3167@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3168have the data type declared for the nonterminal symbol @code{exp}. If
3169@code{$2} were used, it would have the data type declared for the
e0c471a9 3170terminal symbol @code{'+'}, whatever that might be.
bfa74976
RS
3171
3172Alternatively, you can specify the data type when you refer to the value,
3173by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3174reference. For example, if you have defined types as shown here:
3175
3176@example
3177@group
3178%union @{
3179 int itype;
3180 double dtype;
3181@}
3182@end group
3183@end example
3184
3185@noindent
3186then you can write @code{$<itype>1} to refer to the first subunit of the
3187rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3188
342b8b6e 3189@node Mid-Rule Actions
bfa74976
RS
3190@subsection Actions in Mid-Rule
3191@cindex actions in mid-rule
3192@cindex mid-rule actions
3193
3194Occasionally it is useful to put an action in the middle of a rule.
3195These actions are written just like usual end-of-rule actions, but they
3196are executed before the parser even recognizes the following components.
3197
3198A mid-rule action may refer to the components preceding it using
3199@code{$@var{n}}, but it may not refer to subsequent components because
3200it is run before they are parsed.
3201
3202The mid-rule action itself counts as one of the components of the rule.
3203This makes a difference when there is another action later in the same rule
3204(and usually there is another at the end): you have to count the actions
3205along with the symbols when working out which number @var{n} to use in
3206@code{$@var{n}}.
3207
3208The mid-rule action can also have a semantic value. The action can set
3209its value with an assignment to @code{$$}, and actions later in the rule
3210can refer to the value using @code{$@var{n}}. Since there is no symbol
3211to name the action, there is no way to declare a data type for the value
fdc6758b
MA
3212in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3213specify a data type each time you refer to this value.
bfa74976
RS
3214
3215There is no way to set the value of the entire rule with a mid-rule
3216action, because assignments to @code{$$} do not have that effect. The
3217only way to set the value for the entire rule is with an ordinary action
3218at the end of the rule.
3219
3220Here is an example from a hypothetical compiler, handling a @code{let}
3221statement that looks like @samp{let (@var{variable}) @var{statement}} and
3222serves to create a variable named @var{variable} temporarily for the
3223duration of @var{statement}. To parse this construct, we must put
3224@var{variable} into the symbol table while @var{statement} is parsed, then
3225remove it afterward. Here is how it is done:
3226
3227@example
3228@group
3229stmt: LET '(' var ')'
3230 @{ $<context>$ = push_context ();
3231 declare_variable ($3); @}
3232 stmt @{ $$ = $6;
3233 pop_context ($<context>5); @}
3234@end group
3235@end example
3236
3237@noindent
3238As soon as @samp{let (@var{variable})} has been recognized, the first
3239action is run. It saves a copy of the current semantic context (the
3240list of accessible variables) as its semantic value, using alternative
3241@code{context} in the data-type union. Then it calls
3242@code{declare_variable} to add the new variable to that list. Once the
3243first action is finished, the embedded statement @code{stmt} can be
3244parsed. Note that the mid-rule action is component number 5, so the
3245@samp{stmt} is component number 6.
3246
3247After the embedded statement is parsed, its semantic value becomes the
3248value of the entire @code{let}-statement. Then the semantic value from the
3249earlier action is used to restore the prior list of variables. This
3250removes the temporary @code{let}-variable from the list so that it won't
3251appear to exist while the rest of the program is parsed.
3252
3253Taking action before a rule is completely recognized often leads to
3254conflicts since the parser must commit to a parse in order to execute the
3255action. For example, the following two rules, without mid-rule actions,
3256can coexist in a working parser because the parser can shift the open-brace
3257token and look at what follows before deciding whether there is a
3258declaration or not:
3259
3260@example
3261@group
3262compound: '@{' declarations statements '@}'
3263 | '@{' statements '@}'
3264 ;
3265@end group
3266@end example
3267
3268@noindent
3269But when we add a mid-rule action as follows, the rules become nonfunctional:
3270
3271@example
3272@group
3273compound: @{ prepare_for_local_variables (); @}
3274 '@{' declarations statements '@}'
3275@end group
3276@group
3277 | '@{' statements '@}'
3278 ;
3279@end group
3280@end example
3281
3282@noindent
3283Now the parser is forced to decide whether to run the mid-rule action
3284when it has read no farther than the open-brace. In other words, it
3285must commit to using one rule or the other, without sufficient
3286information to do it correctly. (The open-brace token is what is called
3287the @dfn{look-ahead} token at this time, since the parser is still
3288deciding what to do about it. @xref{Look-Ahead, ,Look-Ahead Tokens}.)
3289
3290You might think that you could correct the problem by putting identical
3291actions into the two rules, like this:
3292
3293@example
3294@group
3295compound: @{ prepare_for_local_variables (); @}
3296 '@{' declarations statements '@}'
3297 | @{ prepare_for_local_variables (); @}
3298 '@{' statements '@}'
3299 ;
3300@end group
3301@end example
3302
3303@noindent
3304But this does not help, because Bison does not realize that the two actions
3305are identical. (Bison never tries to understand the C code in an action.)
3306
3307If the grammar is such that a declaration can be distinguished from a
3308statement by the first token (which is true in C), then one solution which
3309does work is to put the action after the open-brace, like this:
3310
3311@example
3312@group
3313compound: '@{' @{ prepare_for_local_variables (); @}
3314 declarations statements '@}'
3315 | '@{' statements '@}'
3316 ;
3317@end group
3318@end example
3319
3320@noindent
3321Now the first token of the following declaration or statement,
3322which would in any case tell Bison which rule to use, can still do so.
3323
3324Another solution is to bury the action inside a nonterminal symbol which
3325serves as a subroutine:
3326
3327@example
3328@group
3329subroutine: /* empty */
3330 @{ prepare_for_local_variables (); @}
3331 ;
3332
3333@end group
3334
3335@group
3336compound: subroutine
3337 '@{' declarations statements '@}'
3338 | subroutine
3339 '@{' statements '@}'
3340 ;
3341@end group
3342@end example
3343
3344@noindent
3345Now Bison can execute the action in the rule for @code{subroutine} without
3346deciding which rule for @code{compound} it will eventually use. Note that
3347the action is now at the end of its rule. Any mid-rule action can be
3348converted to an end-of-rule action in this way, and this is what Bison
3349actually does to implement mid-rule actions.
3350
342b8b6e 3351@node Locations
847bf1f5
AD
3352@section Tracking Locations
3353@cindex location
95923bd6
AD
3354@cindex textual location
3355@cindex location, textual
847bf1f5
AD
3356
3357Though grammar rules and semantic actions are enough to write a fully
72d2299c 3358functional parser, it can be useful to process some additional information,
3e259915
MA
3359especially symbol locations.
3360
704a47c4
AD
3361The way locations are handled is defined by providing a data type, and
3362actions to take when rules are matched.
847bf1f5
AD
3363
3364@menu
3365* Location Type:: Specifying a data type for locations.
3366* Actions and Locations:: Using locations in actions.
3367* Location Default Action:: Defining a general way to compute locations.
3368@end menu
3369
342b8b6e 3370@node Location Type
847bf1f5
AD
3371@subsection Data Type of Locations
3372@cindex data type of locations
3373@cindex default location type
3374
3375Defining a data type for locations is much simpler than for semantic values,
3376since all tokens and groupings always use the same type.
3377
3378The type of locations is specified by defining a macro called @code{YYLTYPE}.
3379When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3380four members:
3381
3382@example
6273355b 3383typedef struct YYLTYPE
847bf1f5
AD
3384@{
3385 int first_line;
3386 int first_column;
3387 int last_line;
3388 int last_column;
6273355b 3389@} YYLTYPE;
847bf1f5
AD
3390@end example
3391
342b8b6e 3392@node Actions and Locations
847bf1f5
AD
3393@subsection Actions and Locations
3394@cindex location actions
3395@cindex actions, location
3396@vindex @@$
3397@vindex @@@var{n}
3398
3399Actions are not only useful for defining language semantics, but also for
3400describing the behavior of the output parser with locations.
3401
3402The most obvious way for building locations of syntactic groupings is very
72d2299c 3403similar to the way semantic values are computed. In a given rule, several
847bf1f5
AD
3404constructs can be used to access the locations of the elements being matched.
3405The location of the @var{n}th component of the right hand side is
3406@code{@@@var{n}}, while the location of the left hand side grouping is
3407@code{@@$}.
3408
3e259915 3409Here is a basic example using the default data type for locations:
847bf1f5
AD
3410
3411@example
3412@group
3413exp: @dots{}
3e259915 3414 | exp '/' exp
847bf1f5 3415 @{
3e259915
MA
3416 @@$.first_column = @@1.first_column;
3417 @@$.first_line = @@1.first_line;
847bf1f5
AD
3418 @@$.last_column = @@3.last_column;
3419 @@$.last_line = @@3.last_line;
3e259915
MA
3420 if ($3)
3421 $$ = $1 / $3;
3422 else
3423 @{
3424 $$ = 1;
4e03e201
AD
3425 fprintf (stderr,
3426 "Division by zero, l%d,c%d-l%d,c%d",
3427 @@3.first_line, @@3.first_column,
3428 @@3.last_line, @@3.last_column);
3e259915 3429 @}
847bf1f5
AD
3430 @}
3431@end group
3432@end example
3433
3e259915 3434As for semantic values, there is a default action for locations that is
72d2299c 3435run each time a rule is matched. It sets the beginning of @code{@@$} to the
3e259915 3436beginning of the first symbol, and the end of @code{@@$} to the end of the
79282c6c 3437last symbol.
3e259915 3438
72d2299c 3439With this default action, the location tracking can be fully automatic. The
3e259915
MA
3440example above simply rewrites this way:
3441
3442@example
3443@group
3444exp: @dots{}
3445 | exp '/' exp
3446 @{
3447 if ($3)
3448 $$ = $1 / $3;
3449 else
3450 @{
3451 $$ = 1;
4e03e201
AD
3452 fprintf (stderr,
3453 "Division by zero, l%d,c%d-l%d,c%d",
3454 @@3.first_line, @@3.first_column,
3455 @@3.last_line, @@3.last_column);
3e259915
MA
3456 @}
3457 @}
3458@end group
3459@end example
847bf1f5 3460
342b8b6e 3461@node Location Default Action
847bf1f5
AD
3462@subsection Default Action for Locations
3463@vindex YYLLOC_DEFAULT
3464
72d2299c 3465Actually, actions are not the best place to compute locations. Since
704a47c4
AD
3466locations are much more general than semantic values, there is room in
3467the output parser to redefine the default action to take for each
72d2299c 3468rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
96b93a3d
PE
3469matched, before the associated action is run. It is also invoked
3470while processing a syntax error, to compute the error's location.
847bf1f5 3471
3e259915 3472Most of the time, this macro is general enough to suppress location
79282c6c 3473dedicated code from semantic actions.
847bf1f5 3474
72d2299c 3475The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
96b93a3d 3476the location of the grouping (the result of the computation). When a
766de5eb 3477rule is matched, the second parameter identifies locations of
96b93a3d
PE
3478all right hand side elements of the rule being matched, and the third
3479parameter is the size of the rule's right hand side. When processing
766de5eb 3480a syntax error, the second parameter identifies locations of
96b93a3d
PE
3481the symbols that were discarded during error processing, and the third
3482parameter is the number of discarded symbols.
847bf1f5 3483
766de5eb 3484By default, @code{YYLLOC_DEFAULT} is defined this way:
847bf1f5 3485
766de5eb 3486@smallexample
847bf1f5 3487@group
766de5eb
PE
3488# define YYLLOC_DEFAULT(Current, Rhs, N) \
3489 do \
3490 if (N) \
3491 @{ \
3492 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
3493 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
3494 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
3495 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
3496 @} \
3497 else \
3498 @{ \
3499 (Current).first_line = (Current).last_line = \
3500 YYRHSLOC(Rhs, 0).last_line; \
3501 (Current).first_column = (Current).last_column = \
3502 YYRHSLOC(Rhs, 0).last_column; \
3503 @} \
3504 while (0)
847bf1f5 3505@end group
766de5eb 3506@end smallexample
676385e2 3507
766de5eb
PE
3508where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
3509in @var{rhs} when @var{k} is positive, and the location of the symbol
f28ac696 3510just before the reduction when @var{k} and @var{n} are both zero.
676385e2 3511
3e259915 3512When defining @code{YYLLOC_DEFAULT}, you should consider that:
847bf1f5 3513
3e259915 3514@itemize @bullet
79282c6c 3515@item
72d2299c 3516All arguments are free of side-effects. However, only the first one (the
3e259915 3517result) should be modified by @code{YYLLOC_DEFAULT}.
847bf1f5 3518
3e259915 3519@item
766de5eb
PE
3520For consistency with semantic actions, valid indexes within the
3521right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
3522valid index, and it refers to the symbol just before the reduction.
3523During error processing @var{n} is always positive.
0ae99356
PE
3524
3525@item
3526Your macro should parenthesize its arguments, if need be, since the
3527actual arguments may not be surrounded by parentheses. Also, your
3528macro should expand to something that can be used as a single
3529statement when it is followed by a semicolon.
3e259915 3530@end itemize
847bf1f5 3531
342b8b6e 3532@node Declarations
bfa74976
RS
3533@section Bison Declarations
3534@cindex declarations, Bison
3535@cindex Bison declarations
3536
3537The @dfn{Bison declarations} section of a Bison grammar defines the symbols
3538used in formulating the grammar and the data types of semantic values.
3539@xref{Symbols}.
3540
3541All token type names (but not single-character literal tokens such as
3542@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
3543declared if you need to specify which data type to use for the semantic
3544value (@pxref{Multiple Types, ,More Than One Value Type}).
3545
3546The first rule in the file also specifies the start symbol, by default.
3547If you want some other symbol to be the start symbol, you must declare
704a47c4
AD
3548it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
3549Grammars}).
bfa74976
RS
3550
3551@menu
b50d2359 3552* Require Decl:: Requiring a Bison version.
bfa74976
RS
3553* Token Decl:: Declaring terminal symbols.
3554* Precedence Decl:: Declaring terminals with precedence and associativity.
3555* Union Decl:: Declaring the set of all semantic value types.
3556* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 3557* Initial Action Decl:: Code run before parsing starts.
72f889cc 3558* Destructor Decl:: Declaring how symbols are freed.
d6328241 3559* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
3560* Start Decl:: Specifying the start symbol.
3561* Pure Decl:: Requesting a reentrant parser.
3562* Decl Summary:: Table of all Bison declarations.
3563@end menu
3564
b50d2359
AD
3565@node Require Decl
3566@subsection Require a Version of Bison
3567@cindex version requirement
3568@cindex requiring a version of Bison
3569@findex %require
3570
3571You may require the minimum version of Bison to process the grammar. If
9b8a5ce0
AD
3572the requirement is not met, @command{bison} exits with an error (exit
3573status 63).
b50d2359
AD
3574
3575@example
3576%require "@var{version}"
3577@end example
3578
342b8b6e 3579@node Token Decl
bfa74976
RS
3580@subsection Token Type Names
3581@cindex declaring token type names
3582@cindex token type names, declaring
931c7513 3583@cindex declaring literal string tokens
bfa74976
RS
3584@findex %token
3585
3586The basic way to declare a token type name (terminal symbol) is as follows:
3587
3588@example
3589%token @var{name}
3590@end example
3591
3592Bison will convert this into a @code{#define} directive in
3593the parser, so that the function @code{yylex} (if it is in this file)
3594can use the name @var{name} to stand for this token type's code.
3595
14ded682
AD
3596Alternatively, you can use @code{%left}, @code{%right}, or
3597@code{%nonassoc} instead of @code{%token}, if you wish to specify
3598associativity and precedence. @xref{Precedence Decl, ,Operator
3599Precedence}.
bfa74976
RS
3600
3601You can explicitly specify the numeric code for a token type by appending
1452af69
PE
3602a decimal or hexadecimal integer value in the field immediately
3603following the token name:
bfa74976
RS
3604
3605@example
3606%token NUM 300
1452af69 3607%token XNUM 0x12d // a GNU extension
bfa74976
RS
3608@end example
3609
3610@noindent
3611It is generally best, however, to let Bison choose the numeric codes for
3612all token types. Bison will automatically select codes that don't conflict
e966383b 3613with each other or with normal characters.
bfa74976
RS
3614
3615In the event that the stack type is a union, you must augment the
3616@code{%token} or other token declaration to include the data type
704a47c4
AD
3617alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
3618Than One Value Type}).
bfa74976
RS
3619
3620For example:
3621
3622@example
3623@group
3624%union @{ /* define stack type */
3625 double val;
3626 symrec *tptr;
3627@}
3628%token <val> NUM /* define token NUM and its type */
3629@end group
3630@end example
3631
931c7513
RS
3632You can associate a literal string token with a token type name by
3633writing the literal string at the end of a @code{%token}
3634declaration which declares the name. For example:
3635
3636@example
3637%token arrow "=>"
3638@end example
3639
3640@noindent
3641For example, a grammar for the C language might specify these names with
3642equivalent literal string tokens:
3643
3644@example
3645%token <operator> OR "||"
3646%token <operator> LE 134 "<="
3647%left OR "<="
3648@end example
3649
3650@noindent
3651Once you equate the literal string and the token name, you can use them
3652interchangeably in further declarations or the grammar rules. The
3653@code{yylex} function can use the token name or the literal string to
3654obtain the token type code number (@pxref{Calling Convention}).
3655
342b8b6e 3656@node Precedence Decl
bfa74976
RS
3657@subsection Operator Precedence
3658@cindex precedence declarations
3659@cindex declaring operator precedence
3660@cindex operator precedence, declaring
3661
3662Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to
3663declare a token and specify its precedence and associativity, all at
3664once. These are called @dfn{precedence declarations}.
704a47c4
AD
3665@xref{Precedence, ,Operator Precedence}, for general information on
3666operator precedence.
bfa74976
RS
3667
3668The syntax of a precedence declaration is the same as that of
3669@code{%token}: either
3670
3671@example
3672%left @var{symbols}@dots{}
3673@end example
3674
3675@noindent
3676or
3677
3678@example
3679%left <@var{type}> @var{symbols}@dots{}
3680@end example
3681
3682And indeed any of these declarations serves the purposes of @code{%token}.
3683But in addition, they specify the associativity and relative precedence for
3684all the @var{symbols}:
3685
3686@itemize @bullet
3687@item
3688The associativity of an operator @var{op} determines how repeated uses
3689of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
3690@var{z}} is parsed by grouping @var{x} with @var{y} first or by
3691grouping @var{y} with @var{z} first. @code{%left} specifies
3692left-associativity (grouping @var{x} with @var{y} first) and
3693@code{%right} specifies right-associativity (grouping @var{y} with
3694@var{z} first). @code{%nonassoc} specifies no associativity, which
3695means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
3696considered a syntax error.
3697
3698@item
3699The precedence of an operator determines how it nests with other operators.
3700All the tokens declared in a single precedence declaration have equal
3701precedence and nest together according to their associativity.
3702When two tokens declared in different precedence declarations associate,
3703the one declared later has the higher precedence and is grouped first.
3704@end itemize
3705
342b8b6e 3706@node Union Decl
bfa74976
RS
3707@subsection The Collection of Value Types
3708@cindex declaring value types
3709@cindex value types, declaring
3710@findex %union
3711
3712The @code{%union} declaration specifies the entire collection of possible
3713data types for semantic values. The keyword @code{%union} is followed by a
3714pair of braces containing the same thing that goes inside a @code{union} in
13863333 3715C.
bfa74976
RS
3716
3717For example:
3718
3719@example
3720@group
3721%union @{
3722 double val;
3723 symrec *tptr;
3724@}
3725@end group
3726@end example
3727
3728@noindent
3729This says that the two alternative types are @code{double} and @code{symrec
3730*}. They are given names @code{val} and @code{tptr}; these names are used
3731in the @code{%token} and @code{%type} declarations to pick one of the types
3732for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
3733
6273355b
PE
3734As an extension to @acronym{POSIX}, a tag is allowed after the
3735@code{union}. For example:
3736
3737@example
3738@group
3739%union value @{
3740 double val;
3741 symrec *tptr;
3742@}
3743@end group
3744@end example
3745
3746specifies the union tag @code{value}, so the corresponding C type is
3747@code{union value}. If you do not specify a tag, it defaults to
3748@code{YYSTYPE}.
3749
3750Note that, unlike making a @code{union} declaration in C, you need not write
bfa74976
RS
3751a semicolon after the closing brace.
3752
342b8b6e 3753@node Type Decl
bfa74976
RS
3754@subsection Nonterminal Symbols
3755@cindex declaring value types, nonterminals
3756@cindex value types, nonterminals, declaring
3757@findex %type
3758
3759@noindent
3760When you use @code{%union} to specify multiple value types, you must
3761declare the value type of each nonterminal symbol for which values are
3762used. This is done with a @code{%type} declaration, like this:
3763
3764@example
3765%type <@var{type}> @var{nonterminal}@dots{}
3766@end example
3767
3768@noindent
704a47c4
AD
3769Here @var{nonterminal} is the name of a nonterminal symbol, and
3770@var{type} is the name given in the @code{%union} to the alternative
3771that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
3772can give any number of nonterminal symbols in the same @code{%type}
3773declaration, if they have the same value type. Use spaces to separate
3774the symbol names.
bfa74976 3775
931c7513
RS
3776You can also declare the value type of a terminal symbol. To do this,
3777use the same @code{<@var{type}>} construction in a declaration for the
3778terminal symbol. All kinds of token declarations allow
3779@code{<@var{type}>}.
3780
18d192f0
AD
3781@node Initial Action Decl
3782@subsection Performing Actions before Parsing
3783@findex %initial-action
3784
3785Sometimes your parser needs to perform some initializations before
3786parsing. The @code{%initial-action} directive allows for such arbitrary
3787code.
3788
3789@deffn {Directive} %initial-action @{ @var{code} @}
3790@findex %initial-action
3791Declare that the @var{code} must be invoked before parsing each time
451364ed
AD
3792@code{yyparse} is called. The @var{code} may use @code{$$} and
3793@code{@@$} --- initial value and location of the look-ahead --- and the
3794@code{%parse-param}.
18d192f0
AD
3795@end deffn
3796
451364ed
AD
3797For instance, if your locations use a file name, you may use
3798
3799@example
48b16bbc 3800%parse-param @{ char const *file_name @};
451364ed
AD
3801%initial-action
3802@{
4626a15d 3803 @@$.initialize (file_name);
451364ed
AD
3804@};
3805@end example
3806
18d192f0 3807
72f889cc
AD
3808@node Destructor Decl
3809@subsection Freeing Discarded Symbols
3810@cindex freeing discarded symbols
3811@findex %destructor
3812
a85284cf
AD
3813During error recovery (@pxref{Error Recovery}), symbols already pushed
3814on the stack and tokens coming from the rest of the file are discarded
3815until the parser falls on its feet. If the parser runs out of memory,
3816or if the parsing is cut by @code{YYACCEPT} or @code{YYABORT}, all the
3817symbols on the stack must be discarded. Even if the parser succeeds, it
3818must discard the start symbol.
258b75ca
PE
3819
3820When discarded symbols convey heap based information, this memory is
3821lost. While this behavior can be tolerable for batch parsers, such as
4b367315
AD
3822in traditional compilers, it is unacceptable for programs like shells or
3823protocol implementations that may parse and execute indefinitely.
258b75ca 3824
a85284cf
AD
3825The @code{%destructor} directive defines code that is called when a
3826symbol is automatically discarded.
72f889cc
AD
3827
3828@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
3829@findex %destructor
4b367315
AD
3830Invoke @var{code} whenever the parser discards one of the @var{symbols}.
3831Within @var{code}, @code{$$} designates the semantic value associated
3832with the discarded symbol. The additional parser parameters are also
3833available (@pxref{Parser Function, , The Parser Function
3834@code{yyparse}}).
72f889cc
AD
3835@end deffn
3836
3837For instance:
3838
3839@smallexample
3840%union
3841@{
3842 char *string;
3843@}
3844%token <string> STRING
3845%type <string> string
3846%destructor @{ free ($$); @} STRING string
3847@end smallexample
3848
3849@noindent
258b75ca 3850guarantees that when a @code{STRING} or a @code{string} is discarded,
72f889cc
AD
3851its associated memory will be freed.
3852
e757bb10
AD
3853@sp 1
3854
3855@cindex discarded symbols
3856@dfn{Discarded symbols} are the following:
3857
3858@itemize
3859@item
3860stacked symbols popped during the first phase of error recovery,
3861@item
3862incoming terminals during the second phase of error recovery,
3863@item
a85284cf
AD
3864the current look-ahead and the entire stack (except the current
3865right-hand side symbols) when the parser aborts (either via an explicit
3866call to @code{YYABORT} or @code{YYACCEPT}, or as a consequence of a
3867failed error recovery or of memory exhaustion), and
258b75ca
PE
3868@item
3869the start symbol, when the parser succeeds.
e757bb10
AD
3870@end itemize
3871
a85284cf
AD
3872Note that right-hand size symbols of a rule that explicitly triggers a
3873syntax error via @code{YYERROR} are not reclaimed. As a thumb rule,
3874destructors are invoked only when you do not have other means to manage
3875the memory.
e757bb10 3876
342b8b6e 3877@node Expect Decl
bfa74976
RS
3878@subsection Suppressing Conflict Warnings
3879@cindex suppressing conflict warnings
3880@cindex preventing warnings about conflicts
3881@cindex warnings, preventing
3882@cindex conflicts, suppressing warnings of
3883@findex %expect
d6328241 3884@findex %expect-rr
bfa74976
RS
3885
3886Bison normally warns if there are any conflicts in the grammar
7da99ede
AD
3887(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
3888have harmless shift/reduce conflicts which are resolved in a predictable
3889way and would be difficult to eliminate. It is desirable to suppress
3890the warning about these conflicts unless the number of conflicts
3891changes. You can do this with the @code{%expect} declaration.
bfa74976
RS
3892
3893The declaration looks like this:
3894
3895@example
3896%expect @var{n}
3897@end example
3898
035aa4a0
PE
3899Here @var{n} is a decimal integer. The declaration says there should
3900be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
3901Bison reports an error if the number of shift/reduce conflicts differs
3902from @var{n}, or if there are any reduce/reduce conflicts.
bfa74976 3903
035aa4a0
PE
3904For normal @acronym{LALR}(1) parsers, reduce/reduce conflicts are more
3905serious, and should be eliminated entirely. Bison will always report
3906reduce/reduce conflicts for these parsers. With @acronym{GLR}
3907parsers, however, both kinds of conflicts are routine; otherwise,
3908there would be no need to use @acronym{GLR} parsing. Therefore, it is
3909also possible to specify an expected number of reduce/reduce conflicts
3910in @acronym{GLR} parsers, using the declaration:
d6328241
PH
3911
3912@example
3913%expect-rr @var{n}
3914@end example
3915
bfa74976
RS
3916In general, using @code{%expect} involves these steps:
3917
3918@itemize @bullet
3919@item
3920Compile your grammar without @code{%expect}. Use the @samp{-v} option
3921to get a verbose list of where the conflicts occur. Bison will also
3922print the number of conflicts.
3923
3924@item
3925Check each of the conflicts to make sure that Bison's default
3926resolution is what you really want. If not, rewrite the grammar and
3927go back to the beginning.
3928
3929@item
3930Add an @code{%expect} declaration, copying the number @var{n} from the
035aa4a0
PE
3931number which Bison printed. With @acronym{GLR} parsers, add an
3932@code{%expect-rr} declaration as well.
bfa74976
RS
3933@end itemize
3934
035aa4a0
PE
3935Now Bison will warn you if you introduce an unexpected conflict, but
3936will keep silent otherwise.
bfa74976 3937
342b8b6e 3938@node Start Decl
bfa74976
RS
3939@subsection The Start-Symbol
3940@cindex declaring the start symbol
3941@cindex start symbol, declaring
3942@cindex default start symbol
3943@findex %start
3944
3945Bison assumes by default that the start symbol for the grammar is the first
3946nonterminal specified in the grammar specification section. The programmer
3947may override this restriction with the @code{%start} declaration as follows:
3948
3949@example
3950%start @var{symbol}
3951@end example
3952
342b8b6e 3953@node Pure Decl
bfa74976
RS
3954@subsection A Pure (Reentrant) Parser
3955@cindex reentrant parser
3956@cindex pure parser
8c9a50be 3957@findex %pure-parser
bfa74976
RS
3958
3959A @dfn{reentrant} program is one which does not alter in the course of
3960execution; in other words, it consists entirely of @dfn{pure} (read-only)
3961code. Reentrancy is important whenever asynchronous execution is possible;
14ded682
AD
3962for example, a non-reentrant program may not be safe to call from a signal
3963handler. In systems with multiple threads of control, a non-reentrant
bfa74976
RS
3964program must be called only within interlocks.
3965
70811b85 3966Normally, Bison generates a parser which is not reentrant. This is
c827f760
PE
3967suitable for most uses, and it permits compatibility with Yacc. (The
3968standard Yacc interfaces are inherently nonreentrant, because they use
70811b85
RS
3969statically allocated variables for communication with @code{yylex},
3970including @code{yylval} and @code{yylloc}.)
bfa74976 3971
70811b85 3972Alternatively, you can generate a pure, reentrant parser. The Bison
8c9a50be 3973declaration @code{%pure-parser} says that you want the parser to be
70811b85 3974reentrant. It looks like this:
bfa74976
RS
3975
3976@example
8c9a50be 3977%pure-parser
bfa74976
RS
3978@end example
3979
70811b85
RS
3980The result is that the communication variables @code{yylval} and
3981@code{yylloc} become local variables in @code{yyparse}, and a different
3982calling convention is used for the lexical analyzer function
3983@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
3984Parsers}, for the details of this. The variable @code{yynerrs} also
3985becomes local in @code{yyparse} (@pxref{Error Reporting, ,The Error
3986Reporting Function @code{yyerror}}). The convention for calling
3987@code{yyparse} itself is unchanged.
3988
3989Whether the parser is pure has nothing to do with the grammar rules.
3990You can generate either a pure parser or a nonreentrant parser from any
3991valid grammar.
bfa74976 3992
342b8b6e 3993@node Decl Summary
bfa74976
RS
3994@subsection Bison Declaration Summary
3995@cindex Bison declaration summary
3996@cindex declaration summary
3997@cindex summary, Bison declaration
3998
d8988b2f 3999Here is a summary of the declarations used to define a grammar:
bfa74976 4000
18b519c0 4001@deffn {Directive} %union
bfa74976
RS
4002Declare the collection of data types that semantic values may have
4003(@pxref{Union Decl, ,The Collection of Value Types}).
18b519c0 4004@end deffn
bfa74976 4005
18b519c0 4006@deffn {Directive} %token
bfa74976
RS
4007Declare a terminal symbol (token type name) with no precedence
4008or associativity specified (@pxref{Token Decl, ,Token Type Names}).
18b519c0 4009@end deffn
bfa74976 4010
18b519c0 4011@deffn {Directive} %right
bfa74976
RS
4012Declare a terminal symbol (token type name) that is right-associative
4013(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4014@end deffn
bfa74976 4015
18b519c0 4016@deffn {Directive} %left
bfa74976
RS
4017Declare a terminal symbol (token type name) that is left-associative
4018(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4019@end deffn
bfa74976 4020
18b519c0 4021@deffn {Directive} %nonassoc
bfa74976 4022Declare a terminal symbol (token type name) that is nonassociative
bfa74976 4023(@pxref{Precedence Decl, ,Operator Precedence}).
39a06c25
PE
4024Using it in a way that would be associative is a syntax error.
4025@end deffn
4026
91d2c560 4027@ifset defaultprec
39a06c25 4028@deffn {Directive} %default-prec
22fccf95 4029Assign a precedence to rules lacking an explicit @code{%prec} modifier
39a06c25
PE
4030(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4031@end deffn
91d2c560 4032@end ifset
bfa74976 4033
18b519c0 4034@deffn {Directive} %type
bfa74976
RS
4035Declare the type of semantic values for a nonterminal symbol
4036(@pxref{Type Decl, ,Nonterminal Symbols}).
18b519c0 4037@end deffn
bfa74976 4038
18b519c0 4039@deffn {Directive} %start
89cab50d
AD
4040Specify the grammar's start symbol (@pxref{Start Decl, ,The
4041Start-Symbol}).
18b519c0 4042@end deffn
bfa74976 4043
18b519c0 4044@deffn {Directive} %expect
bfa74976
RS
4045Declare the expected number of shift-reduce conflicts
4046(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
18b519c0
AD
4047@end deffn
4048
bfa74976 4049
d8988b2f
AD
4050@sp 1
4051@noindent
4052In order to change the behavior of @command{bison}, use the following
4053directives:
4054
18b519c0 4055@deffn {Directive} %debug
4947ebdb
PE
4056In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
4057already defined, so that the debugging facilities are compiled.
18b519c0 4058@end deffn
ec3bc396 4059@xref{Tracing, ,Tracing Your Parser}.
d8988b2f 4060
18b519c0 4061@deffn {Directive} %defines
4bfd5e4e
PE
4062Write a header file containing macro definitions for the token type
4063names defined in the grammar as well as a few other declarations.
d8988b2f 4064If the parser output file is named @file{@var{name}.c} then this file
e0c471a9 4065is named @file{@var{name}.h}.
d8988b2f 4066
4bfd5e4e 4067Unless @code{YYSTYPE} is already defined as a macro, the output header
5c9be03d
PE
4068declares @code{YYSTYPE}. Therefore, if you are using a @code{%union}
4069(@pxref{Multiple Types, ,More Than One Value Type}) with components
4070that require other definitions, or if you have defined a
4bfd5e4e 4071@code{YYSTYPE} macro (@pxref{Value Type, ,Data Types of Semantic
5c9be03d
PE
4072Values}), you need to arrange for these definitions to be propagated to
4073all modules, e.g., by putting them in a
4bfd5e4e
PE
4074prerequisite header that is included both by your parser and by any
4075other module that needs @code{YYSTYPE}.
4076
4077Unless your parser is pure, the output header declares @code{yylval}
4078as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
4079Parser}.
4080
4081If you have also used locations, the output header declares
4082@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
4083@code{YYSTYPE} and @code{yylval}. @xref{Locations, ,Tracking
4084Locations}.
4085
4086This output file is normally essential if you wish to put the
4087definition of @code{yylex} in a separate source file, because
4088@code{yylex} typically needs to be able to refer to the
4089above-mentioned declarations and to the token type codes.
4090@xref{Token Values, ,Semantic Values of Tokens}.
18b519c0 4091@end deffn
d8988b2f 4092
18b519c0 4093@deffn {Directive} %destructor
258b75ca 4094Specify how the parser should reclaim the memory associated to
fa7e68c3 4095discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 4096@end deffn
72f889cc 4097
18b519c0 4098@deffn {Directive} %file-prefix="@var{prefix}"
d8988b2f
AD
4099Specify a prefix to use for all Bison output file names. The names are
4100chosen as if the input file were named @file{@var{prefix}.y}.
18b519c0 4101@end deffn
d8988b2f 4102
18b519c0 4103@deffn {Directive} %locations
89cab50d
AD
4104Generate the code processing the locations (@pxref{Action Features,
4105,Special Features for Use in Actions}). This mode is enabled as soon as
4106the grammar uses the special @samp{@@@var{n}} tokens, but if your
4107grammar does not use it, using @samp{%locations} allows for more
6e649e65 4108accurate syntax error messages.
18b519c0 4109@end deffn
89cab50d 4110
18b519c0 4111@deffn {Directive} %name-prefix="@var{prefix}"
d8988b2f
AD
4112Rename the external symbols used in the parser so that they start with
4113@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
4114is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
2a8d363a
AD
4115@code{yylval}, @code{yylloc}, @code{yychar}, @code{yydebug}, and
4116possible @code{yylloc}. For example, if you use
4117@samp{%name-prefix="c_"}, the names become @code{c_parse}, @code{c_lex},
4118and so on. @xref{Multiple Parsers, ,Multiple Parsers in the Same
4119Program}.
18b519c0 4120@end deffn
931c7513 4121
91d2c560 4122@ifset defaultprec
22fccf95
PE
4123@deffn {Directive} %no-default-prec
4124Do not assign a precedence to rules lacking an explicit @code{%prec}
4125modifier (@pxref{Contextual Precedence, ,Context-Dependent
4126Precedence}).
4127@end deffn
91d2c560 4128@end ifset
22fccf95 4129
18b519c0 4130@deffn {Directive} %no-parser
6deb4447
AD
4131Do not include any C code in the parser file; generate tables only. The
4132parser file contains just @code{#define} directives and static variable
4133declarations.
4134
4135This option also tells Bison to write the C code for the grammar actions
fa4d969f 4136into a file named @file{@var{file}.act}, in the form of a
6deb4447 4137brace-surrounded body fit for a @code{switch} statement.
18b519c0 4138@end deffn
6deb4447 4139
18b519c0 4140@deffn {Directive} %no-lines
931c7513
RS
4141Don't generate any @code{#line} preprocessor commands in the parser
4142file. Ordinarily Bison writes these commands in the parser file so that
4143the C compiler and debuggers will associate errors and object code with
4144your source file (the grammar file). This directive causes them to
4145associate errors with the parser file, treating it an independent source
4146file in its own right.
18b519c0 4147@end deffn
931c7513 4148
fa4d969f
PE
4149@deffn {Directive} %output="@var{file}"
4150Specify @var{file} for the parser file.
18b519c0 4151@end deffn
6deb4447 4152
18b519c0 4153@deffn {Directive} %pure-parser
d8988b2f
AD
4154Request a pure (reentrant) parser program (@pxref{Pure Decl, ,A Pure
4155(Reentrant) Parser}).
18b519c0 4156@end deffn
6deb4447 4157
b50d2359 4158@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
4159Require version @var{version} or higher of Bison. @xref{Require Decl, ,
4160Require a Version of Bison}.
b50d2359
AD
4161@end deffn
4162
18b519c0 4163@deffn {Directive} %token-table
931c7513
RS
4164Generate an array of token names in the parser file. The name of the
4165array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
3650b4b8 4166token whose internal Bison token code number is @var{i}. The first
f67ad422
PE
4167three elements of @code{yytname} correspond to the predefined tokens
4168@code{"$end"},
88bce5a2
AD
4169@code{"error"}, and @code{"$undefined"}; after these come the symbols
4170defined in the grammar file.
931c7513 4171
9e0876fb
PE
4172The name in the table includes all the characters needed to represent
4173the token in Bison. For single-character literals and literal
4174strings, this includes the surrounding quoting characters and any
4175escape sequences. For example, the Bison single-character literal
4176@code{'+'} corresponds to a three-character name, represented in C as
4177@code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
4178corresponds to a five-character name, represented in C as
4179@code{"\"\\\\/\""}.
931c7513 4180
8c9a50be 4181When you specify @code{%token-table}, Bison also generates macro
931c7513
RS
4182definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
4183@code{YYNRULES}, and @code{YYNSTATES}:
4184
4185@table @code
4186@item YYNTOKENS
4187The highest token number, plus one.
4188@item YYNNTS
9ecbd125 4189The number of nonterminal symbols.
931c7513
RS
4190@item YYNRULES
4191The number of grammar rules,
4192@item YYNSTATES
4193The number of parser states (@pxref{Parser States}).
4194@end table
18b519c0 4195@end deffn
d8988b2f 4196
18b519c0 4197@deffn {Directive} %verbose
d8988b2f
AD
4198Write an extra output file containing verbose descriptions of the
4199parser states and what is done for each type of look-ahead token in
72d2299c 4200that state. @xref{Understanding, , Understanding Your Parser}, for more
ec3bc396 4201information.
18b519c0 4202@end deffn
d8988b2f 4203
18b519c0 4204@deffn {Directive} %yacc
d8988b2f
AD
4205Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
4206including its naming conventions. @xref{Bison Options}, for more.
18b519c0 4207@end deffn
d8988b2f
AD
4208
4209
342b8b6e 4210@node Multiple Parsers
bfa74976
RS
4211@section Multiple Parsers in the Same Program
4212
4213Most programs that use Bison parse only one language and therefore contain
4214only one Bison parser. But what if you want to parse more than one
4215language with the same program? Then you need to avoid a name conflict
4216between different definitions of @code{yyparse}, @code{yylval}, and so on.
4217
4218The easy way to do this is to use the option @samp{-p @var{prefix}}
704a47c4
AD
4219(@pxref{Invocation, ,Invoking Bison}). This renames the interface
4220functions and variables of the Bison parser to start with @var{prefix}
4221instead of @samp{yy}. You can use this to give each parser distinct
4222names that do not conflict.
bfa74976
RS
4223
4224The precise list of symbols renamed is @code{yyparse}, @code{yylex},
2a8d363a
AD
4225@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
4226@code{yychar} and @code{yydebug}. For example, if you use @samp{-p c},
4227the names become @code{cparse}, @code{clex}, and so on.
bfa74976
RS
4228
4229@strong{All the other variables and macros associated with Bison are not
4230renamed.} These others are not global; there is no conflict if the same
4231name is used in different parsers. For example, @code{YYSTYPE} is not
4232renamed, but defining this in different ways in different parsers causes
4233no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
4234
4235The @samp{-p} option works by adding macro definitions to the beginning
4236of the parser source file, defining @code{yyparse} as
4237@code{@var{prefix}parse}, and so on. This effectively substitutes one
4238name for the other in the entire parser file.
4239
342b8b6e 4240@node Interface
bfa74976
RS
4241@chapter Parser C-Language Interface
4242@cindex C-language interface
4243@cindex interface
4244
4245The Bison parser is actually a C function named @code{yyparse}. Here we
4246describe the interface conventions of @code{yyparse} and the other
4247functions that it needs to use.
4248
4249Keep in mind that the parser uses many C identifiers starting with
4250@samp{yy} and @samp{YY} for internal purposes. If you use such an
75f5aaea
MA
4251identifier (aside from those in this manual) in an action or in epilogue
4252in the grammar file, you are likely to run into trouble.
bfa74976
RS
4253
4254@menu
4255* Parser Function:: How to call @code{yyparse} and what it returns.
13863333 4256* Lexical:: You must supply a function @code{yylex}
bfa74976
RS
4257 which reads tokens.
4258* Error Reporting:: You must supply a function @code{yyerror}.
4259* Action Features:: Special features for use in actions.
f7ab6a50
PE
4260* Internationalization:: How to let the parser speak in the user's
4261 native language.
bfa74976
RS
4262@end menu
4263
342b8b6e 4264@node Parser Function
bfa74976
RS
4265@section The Parser Function @code{yyparse}
4266@findex yyparse
4267
4268You call the function @code{yyparse} to cause parsing to occur. This
4269function reads tokens, executes actions, and ultimately returns when it
4270encounters end-of-input or an unrecoverable syntax error. You can also
14ded682
AD
4271write an action which directs @code{yyparse} to return immediately
4272without reading further.
bfa74976 4273
2a8d363a
AD
4274
4275@deftypefun int yyparse (void)
bfa74976
RS
4276The value returned by @code{yyparse} is 0 if parsing was successful (return
4277is due to end-of-input).
4278
b47dbebe
PE
4279The value is 1 if parsing failed because of invalid input, i.e., input
4280that contains a syntax error or that causes @code{YYABORT} to be
4281invoked.
4282
4283The value is 2 if parsing failed due to memory exhaustion.
2a8d363a 4284@end deftypefun
bfa74976
RS
4285
4286In an action, you can cause immediate return from @code{yyparse} by using
4287these macros:
4288
2a8d363a 4289@defmac YYACCEPT
bfa74976
RS
4290@findex YYACCEPT
4291Return immediately with value 0 (to report success).
2a8d363a 4292@end defmac
bfa74976 4293
2a8d363a 4294@defmac YYABORT
bfa74976
RS
4295@findex YYABORT
4296Return immediately with value 1 (to report failure).
2a8d363a
AD
4297@end defmac
4298
4299If you use a reentrant parser, you can optionally pass additional
4300parameter information to it in a reentrant way. To do so, use the
4301declaration @code{%parse-param}:
4302
feeb0eda 4303@deffn {Directive} %parse-param @{@var{argument-declaration}@}
2a8d363a 4304@findex %parse-param
feeb0eda 4305Declare that an argument declared by @code{argument-declaration} is an
94175978
PE
4306additional @code{yyparse} argument.
4307The @var{argument-declaration} is used when declaring
feeb0eda
PE
4308functions or prototypes. The last identifier in
4309@var{argument-declaration} must be the argument name.
2a8d363a
AD
4310@end deffn
4311
4312Here's an example. Write this in the parser:
4313
4314@example
feeb0eda
PE
4315%parse-param @{int *nastiness@}
4316%parse-param @{int *randomness@}
2a8d363a
AD
4317@end example
4318
4319@noindent
4320Then call the parser like this:
4321
4322@example
4323@{
4324 int nastiness, randomness;
4325 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
4326 value = yyparse (&nastiness, &randomness);
4327 @dots{}
4328@}
4329@end example
4330
4331@noindent
4332In the grammar actions, use expressions like this to refer to the data:
4333
4334@example
4335exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
4336@end example
4337
bfa74976 4338
342b8b6e 4339@node Lexical
bfa74976
RS
4340@section The Lexical Analyzer Function @code{yylex}
4341@findex yylex
4342@cindex lexical analyzer
4343
4344The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
4345the input stream and returns them to the parser. Bison does not create
4346this function automatically; you must write it so that @code{yyparse} can
4347call it. The function is sometimes referred to as a lexical scanner.
4348
4349In simple programs, @code{yylex} is often defined at the end of the Bison
4350grammar file. If @code{yylex} is defined in a separate source file, you
4351need to arrange for the token-type macro definitions to be available there.
4352To do this, use the @samp{-d} option when you run Bison, so that it will
4353write these macro definitions into a separate header file
4354@file{@var{name}.tab.h} which you can include in the other source files
e0c471a9 4355that need it. @xref{Invocation, ,Invoking Bison}.
bfa74976
RS
4356
4357@menu
4358* Calling Convention:: How @code{yyparse} calls @code{yylex}.
4359* Token Values:: How @code{yylex} must return the semantic value
4360 of the token it has read.
95923bd6 4361* Token Locations:: How @code{yylex} must return the text location
bfa74976
RS
4362 (line number, etc.) of the token, if the
4363 actions want that.
4364* Pure Calling:: How the calling convention differs
4365 in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
4366@end menu
4367
342b8b6e 4368@node Calling Convention
bfa74976
RS
4369@subsection Calling Convention for @code{yylex}
4370
72d2299c
PE
4371The value that @code{yylex} returns must be the positive numeric code
4372for the type of token it has just found; a zero or negative value
4373signifies end-of-input.
bfa74976
RS
4374
4375When a token is referred to in the grammar rules by a name, that name
4376in the parser file becomes a C macro whose definition is the proper
4377numeric code for that token type. So @code{yylex} can use the name
4378to indicate that type. @xref{Symbols}.
4379
4380When a token is referred to in the grammar rules by a character literal,
4381the numeric code for that character is also the code for the token type.
72d2299c
PE
4382So @code{yylex} can simply return that character code, possibly converted
4383to @code{unsigned char} to avoid sign-extension. The null character
4384must not be used this way, because its code is zero and that
bfa74976
RS
4385signifies end-of-input.
4386
4387Here is an example showing these things:
4388
4389@example
13863333
AD
4390int
4391yylex (void)
bfa74976
RS
4392@{
4393 @dots{}
72d2299c 4394 if (c == EOF) /* Detect end-of-input. */
bfa74976
RS
4395 return 0;
4396 @dots{}
4397 if (c == '+' || c == '-')
72d2299c 4398 return c; /* Assume token type for `+' is '+'. */
bfa74976 4399 @dots{}
72d2299c 4400 return INT; /* Return the type of the token. */
bfa74976
RS
4401 @dots{}
4402@}
4403@end example
4404
4405@noindent
4406This interface has been designed so that the output from the @code{lex}
4407utility can be used without change as the definition of @code{yylex}.
4408
931c7513
RS
4409If the grammar uses literal string tokens, there are two ways that
4410@code{yylex} can determine the token type codes for them:
4411
4412@itemize @bullet
4413@item
4414If the grammar defines symbolic token names as aliases for the
4415literal string tokens, @code{yylex} can use these symbolic names like
4416all others. In this case, the use of the literal string tokens in
4417the grammar file has no effect on @code{yylex}.
4418
4419@item
9ecbd125 4420@code{yylex} can find the multicharacter token in the @code{yytname}
931c7513 4421table. The index of the token in the table is the token type's code.
9ecbd125 4422The name of a multicharacter token is recorded in @code{yytname} with a
931c7513 4423double-quote, the token's characters, and another double-quote. The
9e0876fb
PE
4424token's characters are escaped as necessary to be suitable as input
4425to Bison.
931c7513 4426
9e0876fb
PE
4427Here's code for looking up a multicharacter token in @code{yytname},
4428assuming that the characters of the token are stored in
4429@code{token_buffer}, and assuming that the token does not contain any
4430characters like @samp{"} that require escaping.
931c7513
RS
4431
4432@smallexample
4433for (i = 0; i < YYNTOKENS; i++)
4434 @{
4435 if (yytname[i] != 0
4436 && yytname[i][0] == '"'
68449b3a
PE
4437 && ! strncmp (yytname[i] + 1, token_buffer,
4438 strlen (token_buffer))
931c7513
RS
4439 && yytname[i][strlen (token_buffer) + 1] == '"'
4440 && yytname[i][strlen (token_buffer) + 2] == 0)
4441 break;
4442 @}
4443@end smallexample
4444
4445The @code{yytname} table is generated only if you use the
8c9a50be 4446@code{%token-table} declaration. @xref{Decl Summary}.
931c7513
RS
4447@end itemize
4448
342b8b6e 4449@node Token Values
bfa74976
RS
4450@subsection Semantic Values of Tokens
4451
4452@vindex yylval
14ded682 4453In an ordinary (non-reentrant) parser, the semantic value of the token must
bfa74976
RS
4454be stored into the global variable @code{yylval}. When you are using
4455just one data type for semantic values, @code{yylval} has that type.
4456Thus, if the type is @code{int} (the default), you might write this in
4457@code{yylex}:
4458
4459@example
4460@group
4461 @dots{}
72d2299c
PE
4462 yylval = value; /* Put value onto Bison stack. */
4463 return INT; /* Return the type of the token. */
bfa74976
RS
4464 @dots{}
4465@end group
4466@end example
4467
4468When you are using multiple data types, @code{yylval}'s type is a union
704a47c4
AD
4469made from the @code{%union} declaration (@pxref{Union Decl, ,The
4470Collection of Value Types}). So when you store a token's value, you
4471must use the proper member of the union. If the @code{%union}
4472declaration looks like this:
bfa74976
RS
4473
4474@example
4475@group
4476%union @{
4477 int intval;
4478 double val;
4479 symrec *tptr;
4480@}
4481@end group
4482@end example
4483
4484@noindent
4485then the code in @code{yylex} might look like this:
4486
4487@example
4488@group
4489 @dots{}
72d2299c
PE
4490 yylval.intval = value; /* Put value onto Bison stack. */
4491 return INT; /* Return the type of the token. */
bfa74976
RS
4492 @dots{}
4493@end group
4494@end example
4495
95923bd6
AD
4496@node Token Locations
4497@subsection Textual Locations of Tokens
bfa74976
RS
4498
4499@vindex yylloc
847bf1f5
AD
4500If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
4501Tracking Locations}) in actions to keep track of the
89cab50d
AD
4502textual locations of tokens and groupings, then you must provide this
4503information in @code{yylex}. The function @code{yyparse} expects to
4504find the textual location of a token just parsed in the global variable
4505@code{yylloc}. So @code{yylex} must store the proper data in that
847bf1f5
AD
4506variable.
4507
4508By default, the value of @code{yylloc} is a structure and you need only
89cab50d
AD
4509initialize the members that are going to be used by the actions. The
4510four members are called @code{first_line}, @code{first_column},
4511@code{last_line} and @code{last_column}. Note that the use of this
4512feature makes the parser noticeably slower.
bfa74976
RS
4513
4514@tindex YYLTYPE
4515The data type of @code{yylloc} has the name @code{YYLTYPE}.
4516
342b8b6e 4517@node Pure Calling
c656404a 4518@subsection Calling Conventions for Pure Parsers
bfa74976 4519
8c9a50be 4520When you use the Bison declaration @code{%pure-parser} to request a
e425e872
RS
4521pure, reentrant parser, the global communication variables @code{yylval}
4522and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
4523Parser}.) In such parsers the two global variables are replaced by
4524pointers passed as arguments to @code{yylex}. You must declare them as
4525shown here, and pass the information back by storing it through those
4526pointers.
bfa74976
RS
4527
4528@example
13863333
AD
4529int
4530yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
bfa74976
RS
4531@{
4532 @dots{}
4533 *lvalp = value; /* Put value onto Bison stack. */
4534 return INT; /* Return the type of the token. */
4535 @dots{}
4536@}
4537@end example
4538
4539If the grammar file does not use the @samp{@@} constructs to refer to
95923bd6 4540textual locations, then the type @code{YYLTYPE} will not be defined. In
bfa74976
RS
4541this case, omit the second argument; @code{yylex} will be called with
4542only one argument.
4543
e425e872 4544
2a8d363a
AD
4545If you wish to pass the additional parameter data to @code{yylex}, use
4546@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
4547Function}).
e425e872 4548
feeb0eda 4549@deffn {Directive} lex-param @{@var{argument-declaration}@}
2a8d363a 4550@findex %lex-param
feeb0eda
PE
4551Declare that @code{argument-declaration} is an additional @code{yylex}
4552argument declaration.
2a8d363a 4553@end deffn
e425e872 4554
2a8d363a 4555For instance:
e425e872
RS
4556
4557@example
feeb0eda
PE
4558%parse-param @{int *nastiness@}
4559%lex-param @{int *nastiness@}
4560%parse-param @{int *randomness@}
e425e872
RS
4561@end example
4562
4563@noindent
2a8d363a 4564results in the following signature:
e425e872
RS
4565
4566@example
2a8d363a
AD
4567int yylex (int *nastiness);
4568int yyparse (int *nastiness, int *randomness);
e425e872
RS
4569@end example
4570
2a8d363a 4571If @code{%pure-parser} is added:
c656404a
RS
4572
4573@example
2a8d363a
AD
4574int yylex (YYSTYPE *lvalp, int *nastiness);
4575int yyparse (int *nastiness, int *randomness);
c656404a
RS
4576@end example
4577
2a8d363a
AD
4578@noindent
4579and finally, if both @code{%pure-parser} and @code{%locations} are used:
c656404a 4580
2a8d363a
AD
4581@example
4582int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
4583int yyparse (int *nastiness, int *randomness);
4584@end example
931c7513 4585
342b8b6e 4586@node Error Reporting
bfa74976
RS
4587@section The Error Reporting Function @code{yyerror}
4588@cindex error reporting function
4589@findex yyerror
4590@cindex parse error
4591@cindex syntax error
4592
6e649e65 4593The Bison parser detects a @dfn{syntax error} or @dfn{parse error}
9ecbd125 4594whenever it reads a token which cannot satisfy any syntax rule. An
bfa74976 4595action in the grammar can also explicitly proclaim an error, using the
ceed8467
AD
4596macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
4597in Actions}).
bfa74976
RS
4598
4599The Bison parser expects to report the error by calling an error
4600reporting function named @code{yyerror}, which you must supply. It is
4601called by @code{yyparse} whenever a syntax error is found, and it
6e649e65
PE
4602receives one argument. For a syntax error, the string is normally
4603@w{@code{"syntax error"}}.
bfa74976 4604
2a8d363a
AD
4605@findex %error-verbose
4606If you invoke the directive @code{%error-verbose} in the Bison
4607declarations section (@pxref{Bison Declarations, ,The Bison Declarations
4608Section}), then Bison provides a more verbose and specific error message
6e649e65 4609string instead of just plain @w{@code{"syntax error"}}.
bfa74976 4610
1a059451
PE
4611The parser can detect one other kind of error: memory exhaustion. This
4612can happen when the input contains constructions that are very deeply
bfa74976 4613nested. It isn't likely you will encounter this, since the Bison
1a059451
PE
4614parser normally extends its stack automatically up to a very large limit. But
4615if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
4616fashion, except that the argument string is @w{@code{"memory exhausted"}}.
4617
4618In some cases diagnostics like @w{@code{"syntax error"}} are
4619translated automatically from English to some other language before
4620they are passed to @code{yyerror}. @xref{Internationalization}.
bfa74976
RS
4621
4622The following definition suffices in simple programs:
4623
4624@example
4625@group
13863333 4626void
38a92d50 4627yyerror (char const *s)
bfa74976
RS
4628@{
4629@end group
4630@group
4631 fprintf (stderr, "%s\n", s);
4632@}
4633@end group
4634@end example
4635
4636After @code{yyerror} returns to @code{yyparse}, the latter will attempt
4637error recovery if you have written suitable error recovery grammar rules
4638(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
4639immediately return 1.
4640
93724f13 4641Obviously, in location tracking pure parsers, @code{yyerror} should have
fa7e68c3
PE
4642an access to the current location.
4643This is indeed the case for the @acronym{GLR}
2a8d363a
AD
4644parsers, but not for the Yacc parser, for historical reasons. I.e., if
4645@samp{%locations %pure-parser} is passed then the prototypes for
4646@code{yyerror} are:
4647
4648@example
38a92d50
PE
4649void yyerror (char const *msg); /* Yacc parsers. */
4650void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
2a8d363a
AD
4651@end example
4652
feeb0eda 4653If @samp{%parse-param @{int *nastiness@}} is used, then:
2a8d363a
AD
4654
4655@example
b317297e
PE
4656void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
4657void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
2a8d363a
AD
4658@end example
4659
fa7e68c3 4660Finally, @acronym{GLR} and Yacc parsers share the same @code{yyerror} calling
2a8d363a
AD
4661convention for absolutely pure parsers, i.e., when the calling
4662convention of @code{yylex} @emph{and} the calling convention of
4663@code{%pure-parser} are pure. I.e.:
4664
4665@example
4666/* Location tracking. */
4667%locations
4668/* Pure yylex. */
4669%pure-parser
feeb0eda 4670%lex-param @{int *nastiness@}
2a8d363a 4671/* Pure yyparse. */
feeb0eda
PE
4672%parse-param @{int *nastiness@}
4673%parse-param @{int *randomness@}
2a8d363a
AD
4674@end example
4675
4676@noindent
4677results in the following signatures for all the parser kinds:
4678
4679@example
4680int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
4681int yyparse (int *nastiness, int *randomness);
93724f13
AD
4682void yyerror (YYLTYPE *locp,
4683 int *nastiness, int *randomness,
38a92d50 4684 char const *msg);
2a8d363a
AD
4685@end example
4686
1c0c3e95 4687@noindent
38a92d50
PE
4688The prototypes are only indications of how the code produced by Bison
4689uses @code{yyerror}. Bison-generated code always ignores the returned
4690value, so @code{yyerror} can return any type, including @code{void}.
4691Also, @code{yyerror} can be a variadic function; that is why the
4692message is always passed last.
4693
4694Traditionally @code{yyerror} returns an @code{int} that is always
4695ignored, but this is purely for historical reasons, and @code{void} is
4696preferable since it more accurately describes the return type for
4697@code{yyerror}.
93724f13 4698
bfa74976
RS
4699@vindex yynerrs
4700The variable @code{yynerrs} contains the number of syntax errors
8a2800e7 4701reported so far. Normally this variable is global; but if you
704a47c4
AD
4702request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
4703then it is a local variable which only the actions can access.
bfa74976 4704
342b8b6e 4705@node Action Features
bfa74976
RS
4706@section Special Features for Use in Actions
4707@cindex summary, action features
4708@cindex action features summary
4709
4710Here is a table of Bison constructs, variables and macros that
4711are useful in actions.
4712
18b519c0 4713@deffn {Variable} $$
bfa74976
RS
4714Acts like a variable that contains the semantic value for the
4715grouping made by the current rule. @xref{Actions}.
18b519c0 4716@end deffn
bfa74976 4717
18b519c0 4718@deffn {Variable} $@var{n}
bfa74976
RS
4719Acts like a variable that contains the semantic value for the
4720@var{n}th component of the current rule. @xref{Actions}.
18b519c0 4721@end deffn
bfa74976 4722
18b519c0 4723@deffn {Variable} $<@var{typealt}>$
bfa74976 4724Like @code{$$} but specifies alternative @var{typealt} in the union
704a47c4
AD
4725specified by the @code{%union} declaration. @xref{Action Types, ,Data
4726Types of Values in Actions}.
18b519c0 4727@end deffn
bfa74976 4728
18b519c0 4729@deffn {Variable} $<@var{typealt}>@var{n}
bfa74976 4730Like @code{$@var{n}} but specifies alternative @var{typealt} in the
13863333 4731union specified by the @code{%union} declaration.
e0c471a9 4732@xref{Action Types, ,Data Types of Values in Actions}.
18b519c0 4733@end deffn
bfa74976 4734
18b519c0 4735@deffn {Macro} YYABORT;
bfa74976
RS
4736Return immediately from @code{yyparse}, indicating failure.
4737@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 4738@end deffn
bfa74976 4739
18b519c0 4740@deffn {Macro} YYACCEPT;
bfa74976
RS
4741Return immediately from @code{yyparse}, indicating success.
4742@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 4743@end deffn
bfa74976 4744
18b519c0 4745@deffn {Macro} YYBACKUP (@var{token}, @var{value});
bfa74976
RS
4746@findex YYBACKUP
4747Unshift a token. This macro is allowed only for rules that reduce
4748a single value, and only when there is no look-ahead token.
c827f760 4749It is also disallowed in @acronym{GLR} parsers.
bfa74976
RS
4750It installs a look-ahead token with token type @var{token} and
4751semantic value @var{value}; then it discards the value that was
4752going to be reduced by this rule.
4753
4754If the macro is used when it is not valid, such as when there is
4755a look-ahead token already, then it reports a syntax error with
4756a message @samp{cannot back up} and performs ordinary error
4757recovery.
4758
4759In either case, the rest of the action is not executed.
18b519c0 4760@end deffn
bfa74976 4761
18b519c0 4762@deffn {Macro} YYEMPTY
bfa74976
RS
4763@vindex YYEMPTY
4764Value stored in @code{yychar} when there is no look-ahead token.
18b519c0 4765@end deffn
bfa74976 4766
18b519c0 4767@deffn {Macro} YYERROR;
bfa74976
RS
4768@findex YYERROR
4769Cause an immediate syntax error. This statement initiates error
4770recovery just as if the parser itself had detected an error; however, it
4771does not call @code{yyerror}, and does not print any message. If you
4772want to print an error message, call @code{yyerror} explicitly before
4773the @samp{YYERROR;} statement. @xref{Error Recovery}.
18b519c0 4774@end deffn
bfa74976 4775
18b519c0 4776@deffn {Macro} YYRECOVERING
bfa74976
RS
4777This macro stands for an expression that has the value 1 when the parser
4778is recovering from a syntax error, and 0 the rest of the time.
4779@xref{Error Recovery}.
18b519c0 4780@end deffn
bfa74976 4781
18b519c0 4782@deffn {Variable} yychar
bfa74976
RS
4783Variable containing the current look-ahead token. (In a pure parser,
4784this is actually a local variable within @code{yyparse}.) When there is
4785no look-ahead token, the value @code{YYEMPTY} is stored in the variable.
4786@xref{Look-Ahead, ,Look-Ahead Tokens}.
18b519c0 4787@end deffn
bfa74976 4788
18b519c0 4789@deffn {Macro} yyclearin;
bfa74976
RS
4790Discard the current look-ahead token. This is useful primarily in
4791error rules. @xref{Error Recovery}.
18b519c0 4792@end deffn
bfa74976 4793
18b519c0 4794@deffn {Macro} yyerrok;
bfa74976 4795Resume generating error messages immediately for subsequent syntax
13863333 4796errors. This is useful primarily in error rules.
bfa74976 4797@xref{Error Recovery}.
18b519c0 4798@end deffn
bfa74976 4799
18b519c0 4800@deffn {Value} @@$
847bf1f5 4801@findex @@$
95923bd6 4802Acts like a structure variable containing information on the textual location
847bf1f5
AD
4803of the grouping made by the current rule. @xref{Locations, ,
4804Tracking Locations}.
bfa74976 4805
847bf1f5
AD
4806@c Check if those paragraphs are still useful or not.
4807
4808@c @example
4809@c struct @{
4810@c int first_line, last_line;
4811@c int first_column, last_column;
4812@c @};
4813@c @end example
4814
4815@c Thus, to get the starting line number of the third component, you would
4816@c use @samp{@@3.first_line}.
bfa74976 4817
847bf1f5
AD
4818@c In order for the members of this structure to contain valid information,
4819@c you must make @code{yylex} supply this information about each token.
4820@c If you need only certain members, then @code{yylex} need only fill in
4821@c those members.
bfa74976 4822
847bf1f5 4823@c The use of this feature makes the parser noticeably slower.
18b519c0 4824@end deffn
847bf1f5 4825
18b519c0 4826@deffn {Value} @@@var{n}
847bf1f5 4827@findex @@@var{n}
95923bd6 4828Acts like a structure variable containing information on the textual location
847bf1f5
AD
4829of the @var{n}th component of the current rule. @xref{Locations, ,
4830Tracking Locations}.
18b519c0 4831@end deffn
bfa74976 4832
f7ab6a50
PE
4833@node Internationalization
4834@section Parser Internationalization
4835@cindex internationalization
4836@cindex i18n
4837@cindex NLS
4838@cindex gettext
4839@cindex bison-po
4840
4841A Bison-generated parser can print diagnostics, including error and
4842tracing messages. By default, they appear in English. However, Bison
4843also supports outputting diagnostics in the user's native language.
4844To make this work, the user should set the usual environment
30757c8c
PE
4845variables. @xref{Users, , The User's View, gettext, GNU
4846@code{gettext} utilities}. For
f7ab6a50
PE
4847example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might set
4848the user's locale to French Canadian using the @acronym{UTF}-8
4849encoding. The exact set of available locales depends on the user's
4850installation.
4851
4852The maintainer of a package that uses a Bison-generated parser enables
4853the internationalization of the parser's output through the following
4854steps. Here we assume a package that uses @acronym{GNU} Autoconf and
4855@acronym{GNU} Automake.
4856
4857@enumerate
4858@item
30757c8c 4859@cindex bison-i18n.m4
f7ab6a50
PE
4860Into the directory containing the @acronym{GNU} Autoconf macros used
4861by the package---often called @file{m4}---copy the
4862@file{bison-i18n.m4} file installed by Bison under
4863@samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
4864For example:
4865
4866@example
4867cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
4868@end example
4869
4870@item
30757c8c
PE
4871@findex BISON_I18N
4872@vindex BISON_LOCALEDIR
4873@vindex YYENABLE_NLS
f7ab6a50
PE
4874In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
4875invocation, add an invocation of @code{BISON_I18N}. This macro is
4876defined in the file @file{bison-i18n.m4} that you copied earlier. It
4877causes @samp{configure} to find the value of the
30757c8c
PE
4878@code{BISON_LOCALEDIR} variable, and it defines the source-language
4879symbol @code{YYENABLE_NLS} to enable translations in the
4880Bison-generated parser.
f7ab6a50
PE
4881
4882@item
4883In the @code{main} function of your program, designate the directory
4884containing Bison's runtime message catalog, through a call to
4885@samp{bindtextdomain} with domain name @samp{bison-runtime}.
4886For example:
4887
4888@example
4889bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
4890@end example
4891
4892Typically this appears after any other call @code{bindtextdomain
4893(PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
4894@samp{BISON_LOCALEDIR} to be defined as a string through the
4895@file{Makefile}.
4896
4897@item
4898In the @file{Makefile.am} that controls the compilation of the @code{main}
4899function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
4900either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
4901
4902@example
4903DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
4904@end example
4905
4906or:
4907
4908@example
4909AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
4910@end example
4911
4912@item
4913Finally, invoke the command @command{autoreconf} to generate the build
4914infrastructure.
4915@end enumerate
4916
bfa74976 4917
342b8b6e 4918@node Algorithm
13863333
AD
4919@chapter The Bison Parser Algorithm
4920@cindex Bison parser algorithm
bfa74976
RS
4921@cindex algorithm of parser
4922@cindex shifting
4923@cindex reduction
4924@cindex parser stack
4925@cindex stack, parser
4926
4927As Bison reads tokens, it pushes them onto a stack along with their
4928semantic values. The stack is called the @dfn{parser stack}. Pushing a
4929token is traditionally called @dfn{shifting}.
4930
4931For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
4932@samp{3} to come. The stack will have four elements, one for each token
4933that was shifted.
4934
4935But the stack does not always have an element for each token read. When
4936the last @var{n} tokens and groupings shifted match the components of a
4937grammar rule, they can be combined according to that rule. This is called
4938@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
4939single grouping whose symbol is the result (left hand side) of that rule.
4940Running the rule's action is part of the process of reduction, because this
4941is what computes the semantic value of the resulting grouping.
4942
4943For example, if the infix calculator's parser stack contains this:
4944
4945@example
49461 + 5 * 3
4947@end example
4948
4949@noindent
4950and the next input token is a newline character, then the last three
4951elements can be reduced to 15 via the rule:
4952
4953@example
4954expr: expr '*' expr;
4955@end example
4956
4957@noindent
4958Then the stack contains just these three elements:
4959
4960@example
49611 + 15
4962@end example
4963
4964@noindent
4965At this point, another reduction can be made, resulting in the single value
496616. Then the newline token can be shifted.
4967
4968The parser tries, by shifts and reductions, to reduce the entire input down
4969to a single grouping whose symbol is the grammar's start-symbol
4970(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
4971
4972This kind of parser is known in the literature as a bottom-up parser.
4973
4974@menu
4975* Look-Ahead:: Parser looks one token ahead when deciding what to do.
4976* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
4977* Precedence:: Operator precedence works by resolving conflicts.
4978* Contextual Precedence:: When an operator's precedence depends on context.
4979* Parser States:: The parser is a finite-state-machine with stack.
4980* Reduce/Reduce:: When two rules are applicable in the same situation.
4981* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 4982* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 4983* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
4984@end menu
4985
342b8b6e 4986@node Look-Ahead
bfa74976
RS
4987@section Look-Ahead Tokens
4988@cindex look-ahead token
4989
4990The Bison parser does @emph{not} always reduce immediately as soon as the
4991last @var{n} tokens and groupings match a rule. This is because such a
4992simple strategy is inadequate to handle most languages. Instead, when a
4993reduction is possible, the parser sometimes ``looks ahead'' at the next
4994token in order to decide what to do.
4995
4996When a token is read, it is not immediately shifted; first it becomes the
4997@dfn{look-ahead token}, which is not on the stack. Now the parser can
4998perform one or more reductions of tokens and groupings on the stack, while
4999the look-ahead token remains off to the side. When no more reductions
5000should take place, the look-ahead token is shifted onto the stack. This
5001does not mean that all possible reductions have been done; depending on the
5002token type of the look-ahead token, some rules may choose to delay their
5003application.
5004
5005Here is a simple case where look-ahead is needed. These three rules define
5006expressions which contain binary addition operators and postfix unary
5007factorial operators (@samp{!}), and allow parentheses for grouping.
5008
5009@example
5010@group
5011expr: term '+' expr
5012 | term
5013 ;
5014@end group
5015
5016@group
5017term: '(' expr ')'
5018 | term '!'
5019 | NUMBER
5020 ;
5021@end group
5022@end example
5023
5024Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
5025should be done? If the following token is @samp{)}, then the first three
5026tokens must be reduced to form an @code{expr}. This is the only valid
5027course, because shifting the @samp{)} would produce a sequence of symbols
5028@w{@code{term ')'}}, and no rule allows this.
5029
5030If the following token is @samp{!}, then it must be shifted immediately so
5031that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
5032parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
5033@code{expr}. It would then be impossible to shift the @samp{!} because
5034doing so would produce on the stack the sequence of symbols @code{expr
5035'!'}. No rule allows that sequence.
5036
5037@vindex yychar
5038The current look-ahead token is stored in the variable @code{yychar}.
5039@xref{Action Features, ,Special Features for Use in Actions}.
5040
342b8b6e 5041@node Shift/Reduce
bfa74976
RS
5042@section Shift/Reduce Conflicts
5043@cindex conflicts
5044@cindex shift/reduce conflicts
5045@cindex dangling @code{else}
5046@cindex @code{else}, dangling
5047
5048Suppose we are parsing a language which has if-then and if-then-else
5049statements, with a pair of rules like this:
5050
5051@example
5052@group
5053if_stmt:
5054 IF expr THEN stmt
5055 | IF expr THEN stmt ELSE stmt
5056 ;
5057@end group
5058@end example
5059
5060@noindent
5061Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
5062terminal symbols for specific keyword tokens.
5063
5064When the @code{ELSE} token is read and becomes the look-ahead token, the
5065contents of the stack (assuming the input is valid) are just right for
5066reduction by the first rule. But it is also legitimate to shift the
5067@code{ELSE}, because that would lead to eventual reduction by the second
5068rule.
5069
5070This situation, where either a shift or a reduction would be valid, is
5071called a @dfn{shift/reduce conflict}. Bison is designed to resolve
5072these conflicts by choosing to shift, unless otherwise directed by
5073operator precedence declarations. To see the reason for this, let's
5074contrast it with the other alternative.
5075
5076Since the parser prefers to shift the @code{ELSE}, the result is to attach
5077the else-clause to the innermost if-statement, making these two inputs
5078equivalent:
5079
5080@example
5081if x then if y then win (); else lose;
5082
5083if x then do; if y then win (); else lose; end;
5084@end example
5085
5086But if the parser chose to reduce when possible rather than shift, the
5087result would be to attach the else-clause to the outermost if-statement,
5088making these two inputs equivalent:
5089
5090@example
5091if x then if y then win (); else lose;
5092
5093if x then do; if y then win (); end; else lose;
5094@end example
5095
5096The conflict exists because the grammar as written is ambiguous: either
5097parsing of the simple nested if-statement is legitimate. The established
5098convention is that these ambiguities are resolved by attaching the
5099else-clause to the innermost if-statement; this is what Bison accomplishes
5100by choosing to shift rather than reduce. (It would ideally be cleaner to
5101write an unambiguous grammar, but that is very hard to do in this case.)
5102This particular ambiguity was first encountered in the specifications of
5103Algol 60 and is called the ``dangling @code{else}'' ambiguity.
5104
5105To avoid warnings from Bison about predictable, legitimate shift/reduce
5106conflicts, use the @code{%expect @var{n}} declaration. There will be no
5107warning as long as the number of shift/reduce conflicts is exactly @var{n}.
5108@xref{Expect Decl, ,Suppressing Conflict Warnings}.
5109
5110The definition of @code{if_stmt} above is solely to blame for the
5111conflict, but the conflict does not actually appear without additional
5112rules. Here is a complete Bison input file that actually manifests the
5113conflict:
5114
5115@example
5116@group
5117%token IF THEN ELSE variable
5118%%
5119@end group
5120@group
5121stmt: expr
5122 | if_stmt
5123 ;
5124@end group
5125
5126@group
5127if_stmt:
5128 IF expr THEN stmt
5129 | IF expr THEN stmt ELSE stmt
5130 ;
5131@end group
5132
5133expr: variable
5134 ;
5135@end example
5136
342b8b6e 5137@node Precedence
bfa74976
RS
5138@section Operator Precedence
5139@cindex operator precedence
5140@cindex precedence of operators
5141
5142Another situation where shift/reduce conflicts appear is in arithmetic
5143expressions. Here shifting is not always the preferred resolution; the
5144Bison declarations for operator precedence allow you to specify when to
5145shift and when to reduce.
5146
5147@menu
5148* Why Precedence:: An example showing why precedence is needed.
5149* Using Precedence:: How to specify precedence in Bison grammars.
5150* Precedence Examples:: How these features are used in the previous example.
5151* How Precedence:: How they work.
5152@end menu
5153
342b8b6e 5154@node Why Precedence
bfa74976
RS
5155@subsection When Precedence is Needed
5156
5157Consider the following ambiguous grammar fragment (ambiguous because the
5158input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
5159
5160@example
5161@group
5162expr: expr '-' expr
5163 | expr '*' expr
5164 | expr '<' expr
5165 | '(' expr ')'
5166 @dots{}
5167 ;
5168@end group
5169@end example
5170
5171@noindent
5172Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
14ded682
AD
5173should it reduce them via the rule for the subtraction operator? It
5174depends on the next token. Of course, if the next token is @samp{)}, we
5175must reduce; shifting is invalid because no single rule can reduce the
5176token sequence @w{@samp{- 2 )}} or anything starting with that. But if
5177the next token is @samp{*} or @samp{<}, we have a choice: either
5178shifting or reduction would allow the parse to complete, but with
5179different results.
5180
5181To decide which one Bison should do, we must consider the results. If
5182the next operator token @var{op} is shifted, then it must be reduced
5183first in order to permit another opportunity to reduce the difference.
5184The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
5185hand, if the subtraction is reduced before shifting @var{op}, the result
5186is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
5187reduce should depend on the relative precedence of the operators
5188@samp{-} and @var{op}: @samp{*} should be shifted first, but not
5189@samp{<}.
bfa74976
RS
5190
5191@cindex associativity
5192What about input such as @w{@samp{1 - 2 - 5}}; should this be
14ded682
AD
5193@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
5194operators we prefer the former, which is called @dfn{left association}.
5195The latter alternative, @dfn{right association}, is desirable for
5196assignment operators. The choice of left or right association is a
5197matter of whether the parser chooses to shift or reduce when the stack
5198contains @w{@samp{1 - 2}} and the look-ahead token is @samp{-}: shifting
5199makes right-associativity.
bfa74976 5200
342b8b6e 5201@node Using Precedence
bfa74976
RS
5202@subsection Specifying Operator Precedence
5203@findex %left
5204@findex %right
5205@findex %nonassoc
5206
5207Bison allows you to specify these choices with the operator precedence
5208declarations @code{%left} and @code{%right}. Each such declaration
5209contains a list of tokens, which are operators whose precedence and
5210associativity is being declared. The @code{%left} declaration makes all
5211those operators left-associative and the @code{%right} declaration makes
5212them right-associative. A third alternative is @code{%nonassoc}, which
5213declares that it is a syntax error to find the same operator twice ``in a
5214row''.
5215
5216The relative precedence of different operators is controlled by the
5217order in which they are declared. The first @code{%left} or
5218@code{%right} declaration in the file declares the operators whose
5219precedence is lowest, the next such declaration declares the operators
5220whose precedence is a little higher, and so on.
5221
342b8b6e 5222@node Precedence Examples
bfa74976
RS
5223@subsection Precedence Examples
5224
5225In our example, we would want the following declarations:
5226
5227@example
5228%left '<'
5229%left '-'
5230%left '*'
5231@end example
5232
5233In a more complete example, which supports other operators as well, we
5234would declare them in groups of equal precedence. For example, @code{'+'} is
5235declared with @code{'-'}:
5236
5237@example
5238%left '<' '>' '=' NE LE GE
5239%left '+' '-'
5240%left '*' '/'
5241@end example
5242
5243@noindent
5244(Here @code{NE} and so on stand for the operators for ``not equal''
5245and so on. We assume that these tokens are more than one character long
5246and therefore are represented by names, not character literals.)
5247
342b8b6e 5248@node How Precedence
bfa74976
RS
5249@subsection How Precedence Works
5250
5251The first effect of the precedence declarations is to assign precedence
5252levels to the terminal symbols declared. The second effect is to assign
704a47c4
AD
5253precedence levels to certain rules: each rule gets its precedence from
5254the last terminal symbol mentioned in the components. (You can also
5255specify explicitly the precedence of a rule. @xref{Contextual
5256Precedence, ,Context-Dependent Precedence}.)
5257
5258Finally, the resolution of conflicts works by comparing the precedence
5259of the rule being considered with that of the look-ahead token. If the
5260token's precedence is higher, the choice is to shift. If the rule's
5261precedence is higher, the choice is to reduce. If they have equal
5262precedence, the choice is made based on the associativity of that
5263precedence level. The verbose output file made by @samp{-v}
5264(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
5265resolved.
bfa74976
RS
5266
5267Not all rules and not all tokens have precedence. If either the rule or
5268the look-ahead token has no precedence, then the default is to shift.
5269
342b8b6e 5270@node Contextual Precedence
bfa74976
RS
5271@section Context-Dependent Precedence
5272@cindex context-dependent precedence
5273@cindex unary operator precedence
5274@cindex precedence, context-dependent
5275@cindex precedence, unary operator
5276@findex %prec
5277
5278Often the precedence of an operator depends on the context. This sounds
5279outlandish at first, but it is really very common. For example, a minus
5280sign typically has a very high precedence as a unary operator, and a
5281somewhat lower precedence (lower than multiplication) as a binary operator.
5282
5283The Bison precedence declarations, @code{%left}, @code{%right} and
5284@code{%nonassoc}, can only be used once for a given token; so a token has
5285only one precedence declared in this way. For context-dependent
5286precedence, you need to use an additional mechanism: the @code{%prec}
e0c471a9 5287modifier for rules.
bfa74976
RS
5288
5289The @code{%prec} modifier declares the precedence of a particular rule by
5290specifying a terminal symbol whose precedence should be used for that rule.
5291It's not necessary for that symbol to appear otherwise in the rule. The
5292modifier's syntax is:
5293
5294@example
5295%prec @var{terminal-symbol}
5296@end example
5297
5298@noindent
5299and it is written after the components of the rule. Its effect is to
5300assign the rule the precedence of @var{terminal-symbol}, overriding
5301the precedence that would be deduced for it in the ordinary way. The
5302altered rule precedence then affects how conflicts involving that rule
5303are resolved (@pxref{Precedence, ,Operator Precedence}).
5304
5305Here is how @code{%prec} solves the problem of unary minus. First, declare
5306a precedence for a fictitious terminal symbol named @code{UMINUS}. There
5307are no tokens of this type, but the symbol serves to stand for its
5308precedence:
5309
5310@example
5311@dots{}
5312%left '+' '-'
5313%left '*'
5314%left UMINUS
5315@end example
5316
5317Now the precedence of @code{UMINUS} can be used in specific rules:
5318
5319@example
5320@group
5321exp: @dots{}
5322 | exp '-' exp
5323 @dots{}
5324 | '-' exp %prec UMINUS
5325@end group
5326@end example
5327
91d2c560 5328@ifset defaultprec
39a06c25
PE
5329If you forget to append @code{%prec UMINUS} to the rule for unary
5330minus, Bison silently assumes that minus has its usual precedence.
5331This kind of problem can be tricky to debug, since one typically
5332discovers the mistake only by testing the code.
5333
22fccf95 5334The @code{%no-default-prec;} declaration makes it easier to discover
39a06c25
PE
5335this kind of problem systematically. It causes rules that lack a
5336@code{%prec} modifier to have no precedence, even if the last terminal
5337symbol mentioned in their components has a declared precedence.
5338
22fccf95 5339If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
39a06c25
PE
5340for all rules that participate in precedence conflict resolution.
5341Then you will see any shift/reduce conflict until you tell Bison how
5342to resolve it, either by changing your grammar or by adding an
5343explicit precedence. This will probably add declarations to the
5344grammar, but it helps to protect against incorrect rule precedences.
5345
22fccf95
PE
5346The effect of @code{%no-default-prec;} can be reversed by giving
5347@code{%default-prec;}, which is the default.
91d2c560 5348@end ifset
39a06c25 5349
342b8b6e 5350@node Parser States
bfa74976
RS
5351@section Parser States
5352@cindex finite-state machine
5353@cindex parser state
5354@cindex state (of parser)
5355
5356The function @code{yyparse} is implemented using a finite-state machine.
5357The values pushed on the parser stack are not simply token type codes; they
5358represent the entire sequence of terminal and nonterminal symbols at or
5359near the top of the stack. The current state collects all the information
5360about previous input which is relevant to deciding what to do next.
5361
5362Each time a look-ahead token is read, the current parser state together
5363with the type of look-ahead token are looked up in a table. This table
5364entry can say, ``Shift the look-ahead token.'' In this case, it also
5365specifies the new parser state, which is pushed onto the top of the
5366parser stack. Or it can say, ``Reduce using rule number @var{n}.''
5367This means that a certain number of tokens or groupings are taken off
5368the top of the stack, and replaced by one grouping. In other words,
5369that number of states are popped from the stack, and one new state is
5370pushed.
5371
5372There is one other alternative: the table can say that the look-ahead token
5373is erroneous in the current state. This causes error processing to begin
5374(@pxref{Error Recovery}).
5375
342b8b6e 5376@node Reduce/Reduce
bfa74976
RS
5377@section Reduce/Reduce Conflicts
5378@cindex reduce/reduce conflict
5379@cindex conflicts, reduce/reduce
5380
5381A reduce/reduce conflict occurs if there are two or more rules that apply
5382to the same sequence of input. This usually indicates a serious error
5383in the grammar.
5384
5385For example, here is an erroneous attempt to define a sequence
5386of zero or more @code{word} groupings.
5387
5388@example
5389sequence: /* empty */
5390 @{ printf ("empty sequence\n"); @}
5391 | maybeword
5392 | sequence word
5393 @{ printf ("added word %s\n", $2); @}
5394 ;
5395
5396maybeword: /* empty */
5397 @{ printf ("empty maybeword\n"); @}
5398 | word
5399 @{ printf ("single word %s\n", $1); @}
5400 ;
5401@end example
5402
5403@noindent
5404The error is an ambiguity: there is more than one way to parse a single
5405@code{word} into a @code{sequence}. It could be reduced to a
5406@code{maybeword} and then into a @code{sequence} via the second rule.
5407Alternatively, nothing-at-all could be reduced into a @code{sequence}
5408via the first rule, and this could be combined with the @code{word}
5409using the third rule for @code{sequence}.
5410
5411There is also more than one way to reduce nothing-at-all into a
5412@code{sequence}. This can be done directly via the first rule,
5413or indirectly via @code{maybeword} and then the second rule.
5414
5415You might think that this is a distinction without a difference, because it
5416does not change whether any particular input is valid or not. But it does
5417affect which actions are run. One parsing order runs the second rule's
5418action; the other runs the first rule's action and the third rule's action.
5419In this example, the output of the program changes.
5420
5421Bison resolves a reduce/reduce conflict by choosing to use the rule that
5422appears first in the grammar, but it is very risky to rely on this. Every
5423reduce/reduce conflict must be studied and usually eliminated. Here is the
5424proper way to define @code{sequence}:
5425
5426@example
5427sequence: /* empty */
5428 @{ printf ("empty sequence\n"); @}
5429 | sequence word
5430 @{ printf ("added word %s\n", $2); @}
5431 ;
5432@end example
5433
5434Here is another common error that yields a reduce/reduce conflict:
5435
5436@example
5437sequence: /* empty */
5438 | sequence words
5439 | sequence redirects
5440 ;
5441
5442words: /* empty */
5443 | words word
5444 ;
5445
5446redirects:/* empty */
5447 | redirects redirect
5448 ;
5449@end example
5450
5451@noindent
5452The intention here is to define a sequence which can contain either
5453@code{word} or @code{redirect} groupings. The individual definitions of
5454@code{sequence}, @code{words} and @code{redirects} are error-free, but the
5455three together make a subtle ambiguity: even an empty input can be parsed
5456in infinitely many ways!
5457
5458Consider: nothing-at-all could be a @code{words}. Or it could be two
5459@code{words} in a row, or three, or any number. It could equally well be a
5460@code{redirects}, or two, or any number. Or it could be a @code{words}
5461followed by three @code{redirects} and another @code{words}. And so on.
5462
5463Here are two ways to correct these rules. First, to make it a single level
5464of sequence:
5465
5466@example
5467sequence: /* empty */
5468 | sequence word
5469 | sequence redirect
5470 ;
5471@end example
5472
5473Second, to prevent either a @code{words} or a @code{redirects}
5474from being empty:
5475
5476@example
5477sequence: /* empty */
5478 | sequence words
5479 | sequence redirects
5480 ;
5481
5482words: word
5483 | words word
5484 ;
5485
5486redirects:redirect
5487 | redirects redirect
5488 ;
5489@end example
5490
342b8b6e 5491@node Mystery Conflicts
bfa74976
RS
5492@section Mysterious Reduce/Reduce Conflicts
5493
5494Sometimes reduce/reduce conflicts can occur that don't look warranted.
5495Here is an example:
5496
5497@example
5498@group
5499%token ID
5500
5501%%
5502def: param_spec return_spec ','
5503 ;
5504param_spec:
5505 type
5506 | name_list ':' type
5507 ;
5508@end group
5509@group
5510return_spec:
5511 type
5512 | name ':' type
5513 ;
5514@end group
5515@group
5516type: ID
5517 ;
5518@end group
5519@group
5520name: ID
5521 ;
5522name_list:
5523 name
5524 | name ',' name_list
5525 ;
5526@end group
5527@end example
5528
5529It would seem that this grammar can be parsed with only a single token
13863333 5530of look-ahead: when a @code{param_spec} is being read, an @code{ID} is
bfa74976 5531a @code{name} if a comma or colon follows, or a @code{type} if another
c827f760 5532@code{ID} follows. In other words, this grammar is @acronym{LR}(1).
bfa74976 5533
c827f760
PE
5534@cindex @acronym{LR}(1)
5535@cindex @acronym{LALR}(1)
bfa74976 5536However, Bison, like most parser generators, cannot actually handle all
c827f760
PE
5537@acronym{LR}(1) grammars. In this grammar, two contexts, that after
5538an @code{ID}
bfa74976
RS
5539at the beginning of a @code{param_spec} and likewise at the beginning of
5540a @code{return_spec}, are similar enough that Bison assumes they are the
5541same. They appear similar because the same set of rules would be
5542active---the rule for reducing to a @code{name} and that for reducing to
5543a @code{type}. Bison is unable to determine at that stage of processing
5544that the rules would require different look-ahead tokens in the two
5545contexts, so it makes a single parser state for them both. Combining
5546the two contexts causes a conflict later. In parser terminology, this
c827f760 5547occurrence means that the grammar is not @acronym{LALR}(1).
bfa74976
RS
5548
5549In general, it is better to fix deficiencies than to document them. But
5550this particular deficiency is intrinsically hard to fix; parser
c827f760
PE
5551generators that can handle @acronym{LR}(1) grammars are hard to write
5552and tend to
bfa74976
RS
5553produce parsers that are very large. In practice, Bison is more useful
5554as it is now.
5555
5556When the problem arises, you can often fix it by identifying the two
a220f555
MA
5557parser states that are being confused, and adding something to make them
5558look distinct. In the above example, adding one rule to
bfa74976
RS
5559@code{return_spec} as follows makes the problem go away:
5560
5561@example
5562@group
5563%token BOGUS
5564@dots{}
5565%%
5566@dots{}
5567return_spec:
5568 type
5569 | name ':' type
5570 /* This rule is never used. */
5571 | ID BOGUS
5572 ;
5573@end group
5574@end example
5575
5576This corrects the problem because it introduces the possibility of an
5577additional active rule in the context after the @code{ID} at the beginning of
5578@code{return_spec}. This rule is not active in the corresponding context
5579in a @code{param_spec}, so the two contexts receive distinct parser states.
5580As long as the token @code{BOGUS} is never generated by @code{yylex},
5581the added rule cannot alter the way actual input is parsed.
5582
5583In this particular example, there is another way to solve the problem:
5584rewrite the rule for @code{return_spec} to use @code{ID} directly
5585instead of via @code{name}. This also causes the two confusing
5586contexts to have different sets of active rules, because the one for
5587@code{return_spec} activates the altered rule for @code{return_spec}
5588rather than the one for @code{name}.
5589
5590@example
5591param_spec:
5592 type
5593 | name_list ':' type
5594 ;
5595return_spec:
5596 type
5597 | ID ':' type
5598 ;
5599@end example
5600
e054b190
PE
5601For a more detailed exposition of @acronym{LALR}(1) parsers and parser
5602generators, please see:
5603Frank DeRemer and Thomas Pennello, Efficient Computation of
5604@acronym{LALR}(1) Look-Ahead Sets, @cite{@acronym{ACM} Transactions on
5605Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
5606pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
5607
fae437e8 5608@node Generalized LR Parsing
c827f760
PE
5609@section Generalized @acronym{LR} (@acronym{GLR}) Parsing
5610@cindex @acronym{GLR} parsing
5611@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2
PH
5612@cindex ambiguous grammars
5613@cindex non-deterministic parsing
5614
fae437e8
AD
5615Bison produces @emph{deterministic} parsers that choose uniquely
5616when to reduce and which reduction to apply
8dd162d3 5617based on a summary of the preceding input and on one extra token of look-ahead.
676385e2
PH
5618As a result, normal Bison handles a proper subset of the family of
5619context-free languages.
fae437e8 5620Ambiguous grammars, since they have strings with more than one possible
676385e2
PH
5621sequence of reductions cannot have deterministic parsers in this sense.
5622The same is true of languages that require more than one symbol of
8dd162d3 5623look-ahead, since the parser lacks the information necessary to make a
676385e2 5624decision at the point it must be made in a shift-reduce parser.
fae437e8 5625Finally, as previously mentioned (@pxref{Mystery Conflicts}),
676385e2
PH
5626there are languages where Bison's particular choice of how to
5627summarize the input seen so far loses necessary information.
5628
5629When you use the @samp{%glr-parser} declaration in your grammar file,
5630Bison generates a parser that uses a different algorithm, called
c827f760
PE
5631Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR}
5632parser uses the same basic
676385e2
PH
5633algorithm for parsing as an ordinary Bison parser, but behaves
5634differently in cases where there is a shift-reduce conflict that has not
fae437e8 5635been resolved by precedence rules (@pxref{Precedence}) or a
c827f760
PE
5636reduce-reduce conflict. When a @acronym{GLR} parser encounters such a
5637situation, it
fae437e8 5638effectively @emph{splits} into a several parsers, one for each possible
676385e2
PH
5639shift or reduction. These parsers then proceed as usual, consuming
5640tokens in lock-step. Some of the stacks may encounter other conflicts
fae437e8 5641and split further, with the result that instead of a sequence of states,
c827f760 5642a Bison @acronym{GLR} parsing stack is what is in effect a tree of states.
676385e2
PH
5643
5644In effect, each stack represents a guess as to what the proper parse
5645is. Additional input may indicate that a guess was wrong, in which case
5646the appropriate stack silently disappears. Otherwise, the semantics
fae437e8 5647actions generated in each stack are saved, rather than being executed
676385e2 5648immediately. When a stack disappears, its saved semantic actions never
fae437e8 5649get executed. When a reduction causes two stacks to become equivalent,
676385e2
PH
5650their sets of semantic actions are both saved with the state that
5651results from the reduction. We say that two stacks are equivalent
fae437e8 5652when they both represent the same sequence of states,
676385e2
PH
5653and each pair of corresponding states represents a
5654grammar symbol that produces the same segment of the input token
5655stream.
5656
5657Whenever the parser makes a transition from having multiple
c827f760 5658states to having one, it reverts to the normal @acronym{LALR}(1) parsing
676385e2
PH
5659algorithm, after resolving and executing the saved-up actions.
5660At this transition, some of the states on the stack will have semantic
5661values that are sets (actually multisets) of possible actions. The
5662parser tries to pick one of the actions by first finding one whose rule
5663has the highest dynamic precedence, as set by the @samp{%dprec}
fae437e8 5664declaration. Otherwise, if the alternative actions are not ordered by
676385e2 5665precedence, but there the same merging function is declared for both
fae437e8 5666rules by the @samp{%merge} declaration,
676385e2
PH
5667Bison resolves and evaluates both and then calls the merge function on
5668the result. Otherwise, it reports an ambiguity.
5669
c827f760
PE
5670It is possible to use a data structure for the @acronym{GLR} parsing tree that
5671permits the processing of any @acronym{LALR}(1) grammar in linear time (in the
5672size of the input), any unambiguous (not necessarily
5673@acronym{LALR}(1)) grammar in
fae437e8 5674quadratic worst-case time, and any general (possibly ambiguous)
676385e2
PH
5675context-free grammar in cubic worst-case time. However, Bison currently
5676uses a simpler data structure that requires time proportional to the
5677length of the input times the maximum number of stacks required for any
5678prefix of the input. Thus, really ambiguous or non-deterministic
5679grammars can require exponential time and space to process. Such badly
5680behaving examples, however, are not generally of practical interest.
5681Usually, non-determinism in a grammar is local---the parser is ``in
5682doubt'' only for a few tokens at a time. Therefore, the current data
c827f760 5683structure should generally be adequate. On @acronym{LALR}(1) portions of a
676385e2
PH
5684grammar, in particular, it is only slightly slower than with the default
5685Bison parser.
5686
fa7e68c3 5687For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth
f6481e2f
PE
5688Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
5689Generalised @acronym{LR} Parsers, Royal Holloway, University of
5690London, Department of Computer Science, TR-00-12,
5691@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
5692(2000-12-24).
5693
1a059451
PE
5694@node Memory Management
5695@section Memory Management, and How to Avoid Memory Exhaustion
5696@cindex memory exhaustion
5697@cindex memory management
bfa74976
RS
5698@cindex stack overflow
5699@cindex parser stack overflow
5700@cindex overflow of parser stack
5701
1a059451 5702The Bison parser stack can run out of memory if too many tokens are shifted and
bfa74976 5703not reduced. When this happens, the parser function @code{yyparse}
1a059451 5704calls @code{yyerror} and then returns 2.
bfa74976 5705
c827f760 5706Because Bison parsers have growing stacks, hitting the upper limit
d1a1114f
AD
5707usually results from using a right recursion instead of a left
5708recursion, @xref{Recursion, ,Recursive Rules}.
5709
bfa74976
RS
5710@vindex YYMAXDEPTH
5711By defining the macro @code{YYMAXDEPTH}, you can control how deep the
1a059451 5712parser stack can become before memory is exhausted. Define the
bfa74976
RS
5713macro with a value that is an integer. This value is the maximum number
5714of tokens that can be shifted (and not reduced) before overflow.
bfa74976
RS
5715
5716The stack space allowed is not necessarily allocated. If you specify a
1a059451 5717large value for @code{YYMAXDEPTH}, the parser normally allocates a small
bfa74976
RS
5718stack at first, and then makes it bigger by stages as needed. This
5719increasing allocation happens automatically and silently. Therefore,
5720you do not need to make @code{YYMAXDEPTH} painfully small merely to save
5721space for ordinary inputs that do not need much stack.
5722
d7e14fc0
PE
5723However, do not allow @code{YYMAXDEPTH} to be a value so large that
5724arithmetic overflow could occur when calculating the size of the stack
5725space. Also, do not allow @code{YYMAXDEPTH} to be less than
5726@code{YYINITDEPTH}.
5727
bfa74976
RS
5728@cindex default stack limit
5729The default value of @code{YYMAXDEPTH}, if you do not define it, is
573010000.
5731
5732@vindex YYINITDEPTH
5733You can control how much stack is allocated initially by defining the
d7e14fc0
PE
5734macro @code{YYINITDEPTH} to a positive integer. For the C
5735@acronym{LALR}(1) parser, this value must be a compile-time constant
5736unless you are assuming C99 or some other target language or compiler
5737that allows variable-length arrays. The default is 200.
5738
1a059451 5739Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
bfa74976 5740
d1a1114f 5741@c FIXME: C++ output.
c827f760 5742Because of semantical differences between C and C++, the
1a059451
PE
5743@acronym{LALR}(1) parsers in C produced by Bison cannot grow when compiled
5744by C++ compilers. In this precise case (compiling a C parser as C++) you are
5745suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix
5746this deficiency in a future release.
d1a1114f 5747
342b8b6e 5748@node Error Recovery
bfa74976
RS
5749@chapter Error Recovery
5750@cindex error recovery
5751@cindex recovery from errors
5752
6e649e65 5753It is not usually acceptable to have a program terminate on a syntax
bfa74976
RS
5754error. For example, a compiler should recover sufficiently to parse the
5755rest of the input file and check it for errors; a calculator should accept
5756another expression.
5757
5758In a simple interactive command parser where each input is one line, it may
5759be sufficient to allow @code{yyparse} to return 1 on error and have the
5760caller ignore the rest of the input line when that happens (and then call
5761@code{yyparse} again). But this is inadequate for a compiler, because it
5762forgets all the syntactic context leading up to the error. A syntax error
5763deep within a function in the compiler input should not cause the compiler
5764to treat the following line like the beginning of a source file.
5765
5766@findex error
5767You can define how to recover from a syntax error by writing rules to
5768recognize the special token @code{error}. This is a terminal symbol that
5769is always defined (you need not declare it) and reserved for error
5770handling. The Bison parser generates an @code{error} token whenever a
5771syntax error happens; if you have provided a rule to recognize this token
13863333 5772in the current context, the parse can continue.
bfa74976
RS
5773
5774For example:
5775
5776@example
5777stmnts: /* empty string */
5778 | stmnts '\n'
5779 | stmnts exp '\n'
5780 | stmnts error '\n'
5781@end example
5782
5783The fourth rule in this example says that an error followed by a newline
5784makes a valid addition to any @code{stmnts}.
5785
5786What happens if a syntax error occurs in the middle of an @code{exp}? The
5787error recovery rule, interpreted strictly, applies to the precise sequence
5788of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
5789the middle of an @code{exp}, there will probably be some additional tokens
5790and subexpressions on the stack after the last @code{stmnts}, and there
5791will be tokens to read before the next newline. So the rule is not
5792applicable in the ordinary way.
5793
5794But Bison can force the situation to fit the rule, by discarding part of
72f889cc
AD
5795the semantic context and part of the input. First it discards states
5796and objects from the stack until it gets back to a state in which the
bfa74976 5797@code{error} token is acceptable. (This means that the subexpressions
72f889cc
AD
5798already parsed are discarded, back to the last complete @code{stmnts}.)
5799At this point the @code{error} token can be shifted. Then, if the old
bfa74976
RS
5800look-ahead token is not acceptable to be shifted next, the parser reads
5801tokens and discards them until it finds a token which is acceptable. In
72f889cc
AD
5802this example, Bison reads and discards input until the next newline so
5803that the fourth rule can apply. Note that discarded symbols are
5804possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
5805Discarded Symbols}, for a means to reclaim this memory.
bfa74976
RS
5806
5807The choice of error rules in the grammar is a choice of strategies for
5808error recovery. A simple and useful strategy is simply to skip the rest of
5809the current input line or current statement if an error is detected:
5810
5811@example
72d2299c 5812stmnt: error ';' /* On error, skip until ';' is read. */
bfa74976
RS
5813@end example
5814
5815It is also useful to recover to the matching close-delimiter of an
5816opening-delimiter that has already been parsed. Otherwise the
5817close-delimiter will probably appear to be unmatched, and generate another,
5818spurious error message:
5819
5820@example
5821primary: '(' expr ')'
5822 | '(' error ')'
5823 @dots{}
5824 ;
5825@end example
5826
5827Error recovery strategies are necessarily guesses. When they guess wrong,
5828one syntax error often leads to another. In the above example, the error
5829recovery rule guesses that an error is due to bad input within one
5830@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
5831middle of a valid @code{stmnt}. After the error recovery rule recovers
5832from the first error, another syntax error will be found straightaway,
5833since the text following the spurious semicolon is also an invalid
5834@code{stmnt}.
5835
5836To prevent an outpouring of error messages, the parser will output no error
5837message for another syntax error that happens shortly after the first; only
5838after three consecutive input tokens have been successfully shifted will
5839error messages resume.
5840
5841Note that rules which accept the @code{error} token may have actions, just
5842as any other rules can.
5843
5844@findex yyerrok
5845You can make error messages resume immediately by using the macro
5846@code{yyerrok} in an action. If you do this in the error rule's action, no
5847error messages will be suppressed. This macro requires no arguments;
5848@samp{yyerrok;} is a valid C statement.
5849
5850@findex yyclearin
5851The previous look-ahead token is reanalyzed immediately after an error. If
5852this is unacceptable, then the macro @code{yyclearin} may be used to clear
5853this token. Write the statement @samp{yyclearin;} in the error rule's
5854action.
5855
6e649e65 5856For example, suppose that on a syntax error, an error handling routine is
bfa74976
RS
5857called that advances the input stream to some point where parsing should
5858once again commence. The next symbol returned by the lexical scanner is
5859probably correct. The previous look-ahead token ought to be discarded
5860with @samp{yyclearin;}.
5861
5862@vindex YYRECOVERING
5863The macro @code{YYRECOVERING} stands for an expression that has the
5864value 1 when the parser is recovering from a syntax error, and 0 the
5865rest of the time. A value of 1 indicates that error messages are
5866currently suppressed for new syntax errors.
5867
342b8b6e 5868@node Context Dependency
bfa74976
RS
5869@chapter Handling Context Dependencies
5870
5871The Bison paradigm is to parse tokens first, then group them into larger
5872syntactic units. In many languages, the meaning of a token is affected by
5873its context. Although this violates the Bison paradigm, certain techniques
5874(known as @dfn{kludges}) may enable you to write Bison parsers for such
5875languages.
5876
5877@menu
5878* Semantic Tokens:: Token parsing can depend on the semantic context.
5879* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
5880* Tie-in Recovery:: Lexical tie-ins have implications for how
5881 error recovery rules must be written.
5882@end menu
5883
5884(Actually, ``kludge'' means any technique that gets its job done but is
5885neither clean nor robust.)
5886
342b8b6e 5887@node Semantic Tokens
bfa74976
RS
5888@section Semantic Info in Token Types
5889
5890The C language has a context dependency: the way an identifier is used
5891depends on what its current meaning is. For example, consider this:
5892
5893@example
5894foo (x);
5895@end example
5896
5897This looks like a function call statement, but if @code{foo} is a typedef
5898name, then this is actually a declaration of @code{x}. How can a Bison
5899parser for C decide how to parse this input?
5900
c827f760 5901The method used in @acronym{GNU} C is to have two different token types,
bfa74976
RS
5902@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
5903identifier, it looks up the current declaration of the identifier in order
5904to decide which token type to return: @code{TYPENAME} if the identifier is
5905declared as a typedef, @code{IDENTIFIER} otherwise.
5906
5907The grammar rules can then express the context dependency by the choice of
5908token type to recognize. @code{IDENTIFIER} is accepted as an expression,
5909but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
5910@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
5911is @emph{not} significant, such as in declarations that can shadow a
5912typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
5913accepted---there is one rule for each of the two token types.
5914
5915This technique is simple to use if the decision of which kinds of
5916identifiers to allow is made at a place close to where the identifier is
5917parsed. But in C this is not always so: C allows a declaration to
5918redeclare a typedef name provided an explicit type has been specified
5919earlier:
5920
5921@example
3a4f411f
PE
5922typedef int foo, bar;
5923int baz (void)
5924@{
5925 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
5926 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
5927 return foo (bar);
5928@}
bfa74976
RS
5929@end example
5930
5931Unfortunately, the name being declared is separated from the declaration
5932construct itself by a complicated syntactic structure---the ``declarator''.
5933
9ecbd125 5934As a result, part of the Bison parser for C needs to be duplicated, with
14ded682
AD
5935all the nonterminal names changed: once for parsing a declaration in
5936which a typedef name can be redefined, and once for parsing a
5937declaration in which that can't be done. Here is a part of the
5938duplication, with actions omitted for brevity:
bfa74976
RS
5939
5940@example
5941initdcl:
5942 declarator maybeasm '='
5943 init
5944 | declarator maybeasm
5945 ;
5946
5947notype_initdcl:
5948 notype_declarator maybeasm '='
5949 init
5950 | notype_declarator maybeasm
5951 ;
5952@end example
5953
5954@noindent
5955Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
5956cannot. The distinction between @code{declarator} and
5957@code{notype_declarator} is the same sort of thing.
5958
5959There is some similarity between this technique and a lexical tie-in
5960(described next), in that information which alters the lexical analysis is
5961changed during parsing by other parts of the program. The difference is
5962here the information is global, and is used for other purposes in the
5963program. A true lexical tie-in has a special-purpose flag controlled by
5964the syntactic context.
5965
342b8b6e 5966@node Lexical Tie-ins
bfa74976
RS
5967@section Lexical Tie-ins
5968@cindex lexical tie-in
5969
5970One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
5971which is set by Bison actions, whose purpose is to alter the way tokens are
5972parsed.
5973
5974For example, suppose we have a language vaguely like C, but with a special
5975construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
5976an expression in parentheses in which all integers are hexadecimal. In
5977particular, the token @samp{a1b} must be treated as an integer rather than
5978as an identifier if it appears in that context. Here is how you can do it:
5979
5980@example
5981@group
5982%@{
38a92d50
PE
5983 int hexflag;
5984 int yylex (void);
5985 void yyerror (char const *);
bfa74976
RS
5986%@}
5987%%
5988@dots{}
5989@end group
5990@group
5991expr: IDENTIFIER
5992 | constant
5993 | HEX '('
5994 @{ hexflag = 1; @}
5995 expr ')'
5996 @{ hexflag = 0;
5997 $$ = $4; @}
5998 | expr '+' expr
5999 @{ $$ = make_sum ($1, $3); @}
6000 @dots{}
6001 ;
6002@end group
6003
6004@group
6005constant:
6006 INTEGER
6007 | STRING
6008 ;
6009@end group
6010@end example
6011
6012@noindent
6013Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
6014it is nonzero, all integers are parsed in hexadecimal, and tokens starting
6015with letters are parsed as integers if possible.
6016
342b8b6e
AD
6017The declaration of @code{hexflag} shown in the prologue of the parser file
6018is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}).
75f5aaea 6019You must also write the code in @code{yylex} to obey the flag.
bfa74976 6020
342b8b6e 6021@node Tie-in Recovery
bfa74976
RS
6022@section Lexical Tie-ins and Error Recovery
6023
6024Lexical tie-ins make strict demands on any error recovery rules you have.
6025@xref{Error Recovery}.
6026
6027The reason for this is that the purpose of an error recovery rule is to
6028abort the parsing of one construct and resume in some larger construct.
6029For example, in C-like languages, a typical error recovery rule is to skip
6030tokens until the next semicolon, and then start a new statement, like this:
6031
6032@example
6033stmt: expr ';'
6034 | IF '(' expr ')' stmt @{ @dots{} @}
6035 @dots{}
6036 error ';'
6037 @{ hexflag = 0; @}
6038 ;
6039@end example
6040
6041If there is a syntax error in the middle of a @samp{hex (@var{expr})}
6042construct, this error rule will apply, and then the action for the
6043completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
6044remain set for the entire rest of the input, or until the next @code{hex}
6045keyword, causing identifiers to be misinterpreted as integers.
6046
6047To avoid this problem the error recovery rule itself clears @code{hexflag}.
6048
6049There may also be an error recovery rule that works within expressions.
6050For example, there could be a rule which applies within parentheses
6051and skips to the close-parenthesis:
6052
6053@example
6054@group
6055expr: @dots{}
6056 | '(' expr ')'
6057 @{ $$ = $2; @}
6058 | '(' error ')'
6059 @dots{}
6060@end group
6061@end example
6062
6063If this rule acts within the @code{hex} construct, it is not going to abort
6064that construct (since it applies to an inner level of parentheses within
6065the construct). Therefore, it should not clear the flag: the rest of
6066the @code{hex} construct should be parsed with the flag still in effect.
6067
6068What if there is an error recovery rule which might abort out of the
6069@code{hex} construct or might not, depending on circumstances? There is no
6070way you can write the action to determine whether a @code{hex} construct is
6071being aborted or not. So if you are using a lexical tie-in, you had better
6072make sure your error recovery rules are not of this kind. Each rule must
6073be such that you can be sure that it always will, or always won't, have to
6074clear the flag.
6075
ec3bc396
AD
6076@c ================================================== Debugging Your Parser
6077
342b8b6e 6078@node Debugging
bfa74976 6079@chapter Debugging Your Parser
ec3bc396
AD
6080
6081Developing a parser can be a challenge, especially if you don't
6082understand the algorithm (@pxref{Algorithm, ,The Bison Parser
6083Algorithm}). Even so, sometimes a detailed description of the automaton
6084can help (@pxref{Understanding, , Understanding Your Parser}), or
6085tracing the execution of the parser can give some insight on why it
6086behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
6087
6088@menu
6089* Understanding:: Understanding the structure of your parser.
6090* Tracing:: Tracing the execution of your parser.
6091@end menu
6092
6093@node Understanding
6094@section Understanding Your Parser
6095
6096As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
6097Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
6098frequent than one would hope), looking at this automaton is required to
6099tune or simply fix a parser. Bison provides two different
c827f760 6100representation of it, either textually or graphically (as a @acronym{VCG}
ec3bc396
AD
6101file).
6102
6103The textual file is generated when the options @option{--report} or
6104@option{--verbose} are specified, see @xref{Invocation, , Invoking
6105Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
6106the parser output file name, and adding @samp{.output} instead.
6107Therefore, if the input file is @file{foo.y}, then the parser file is
6108called @file{foo.tab.c} by default. As a consequence, the verbose
6109output file is called @file{foo.output}.
6110
6111The following grammar file, @file{calc.y}, will be used in the sequel:
6112
6113@example
6114%token NUM STR
6115%left '+' '-'
6116%left '*'
6117%%
6118exp: exp '+' exp
6119 | exp '-' exp
6120 | exp '*' exp
6121 | exp '/' exp
6122 | NUM
6123 ;
6124useless: STR;
6125%%
6126@end example
6127
88bce5a2
AD
6128@command{bison} reports:
6129
6130@example
6131calc.y: warning: 1 useless nonterminal and 1 useless rule
6132calc.y:11.1-7: warning: useless nonterminal: useless
5a99098d
PE
6133calc.y:11.10-12: warning: useless rule: useless: STR
6134calc.y: conflicts: 7 shift/reduce
88bce5a2
AD
6135@end example
6136
6137When given @option{--report=state}, in addition to @file{calc.tab.c}, it
6138creates a file @file{calc.output} with contents detailed below. The
6139order of the output and the exact presentation might vary, but the
6140interpretation is the same.
ec3bc396
AD
6141
6142The first section includes details on conflicts that were solved thanks
6143to precedence and/or associativity:
6144
6145@example
6146Conflict in state 8 between rule 2 and token '+' resolved as reduce.
6147Conflict in state 8 between rule 2 and token '-' resolved as reduce.
6148Conflict in state 8 between rule 2 and token '*' resolved as shift.
6149@exdent @dots{}
6150@end example
6151
6152@noindent
6153The next section lists states that still have conflicts.
6154
6155@example
5a99098d
PE
6156State 8 conflicts: 1 shift/reduce
6157State 9 conflicts: 1 shift/reduce
6158State 10 conflicts: 1 shift/reduce
6159State 11 conflicts: 4 shift/reduce
ec3bc396
AD
6160@end example
6161
6162@noindent
6163@cindex token, useless
6164@cindex useless token
6165@cindex nonterminal, useless
6166@cindex useless nonterminal
6167@cindex rule, useless
6168@cindex useless rule
6169The next section reports useless tokens, nonterminal and rules. Useless
6170nonterminals and rules are removed in order to produce a smaller parser,
6171but useless tokens are preserved, since they might be used by the
6172scanner (note the difference between ``useless'' and ``not used''
6173below):
6174
6175@example
6176Useless nonterminals:
6177 useless
6178
6179Terminals which are not used:
6180 STR
6181
6182Useless rules:
6183#6 useless: STR;
6184@end example
6185
6186@noindent
6187The next section reproduces the exact grammar that Bison used:
6188
6189@example
6190Grammar
6191
6192 Number, Line, Rule
88bce5a2 6193 0 5 $accept -> exp $end
ec3bc396
AD
6194 1 5 exp -> exp '+' exp
6195 2 6 exp -> exp '-' exp
6196 3 7 exp -> exp '*' exp
6197 4 8 exp -> exp '/' exp
6198 5 9 exp -> NUM
6199@end example
6200
6201@noindent
6202and reports the uses of the symbols:
6203
6204@example
6205Terminals, with rules where they appear
6206
88bce5a2 6207$end (0) 0
ec3bc396
AD
6208'*' (42) 3
6209'+' (43) 1
6210'-' (45) 2
6211'/' (47) 4
6212error (256)
6213NUM (258) 5
6214
6215Nonterminals, with rules where they appear
6216
88bce5a2 6217$accept (8)
ec3bc396
AD
6218 on left: 0
6219exp (9)
6220 on left: 1 2 3 4 5, on right: 0 1 2 3 4
6221@end example
6222
6223@noindent
6224@cindex item
6225@cindex pointed rule
6226@cindex rule, pointed
6227Bison then proceeds onto the automaton itself, describing each state
6228with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
6229item is a production rule together with a point (marked by @samp{.})
6230that the input cursor.
6231
6232@example
6233state 0
6234
88bce5a2 6235 $accept -> . exp $ (rule 0)
ec3bc396 6236
2a8d363a 6237 NUM shift, and go to state 1
ec3bc396 6238
2a8d363a 6239 exp go to state 2
ec3bc396
AD
6240@end example
6241
6242This reads as follows: ``state 0 corresponds to being at the very
6243beginning of the parsing, in the initial rule, right before the start
6244symbol (here, @code{exp}). When the parser returns to this state right
6245after having reduced a rule that produced an @code{exp}, the control
6246flow jumps to state 2. If there is no such transition on a nonterminal
8dd162d3 6247symbol, and the look-ahead is a @code{NUM}, then this token is shifted on
ec3bc396 6248the parse stack, and the control flow jumps to state 1. Any other
8dd162d3 6249look-ahead triggers a syntax error.''
ec3bc396
AD
6250
6251@cindex core, item set
6252@cindex item set core
6253@cindex kernel, item set
6254@cindex item set core
6255Even though the only active rule in state 0 seems to be rule 0, the
8dd162d3 6256report lists @code{NUM} as a look-ahead token because @code{NUM} can be
ec3bc396
AD
6257at the beginning of any rule deriving an @code{exp}. By default Bison
6258reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
6259you want to see more detail you can invoke @command{bison} with
6260@option{--report=itemset} to list all the items, include those that can
6261be derived:
6262
6263@example
6264state 0
6265
88bce5a2 6266 $accept -> . exp $ (rule 0)
ec3bc396
AD
6267 exp -> . exp '+' exp (rule 1)
6268 exp -> . exp '-' exp (rule 2)
6269 exp -> . exp '*' exp (rule 3)
6270 exp -> . exp '/' exp (rule 4)
6271 exp -> . NUM (rule 5)
6272
6273 NUM shift, and go to state 1
6274
6275 exp go to state 2
6276@end example
6277
6278@noindent
6279In the state 1...
6280
6281@example
6282state 1
6283
6284 exp -> NUM . (rule 5)
6285
2a8d363a 6286 $default reduce using rule 5 (exp)
ec3bc396
AD
6287@end example
6288
6289@noindent
8dd162d3 6290the rule 5, @samp{exp: NUM;}, is completed. Whatever the look-ahead token
ec3bc396
AD
6291(@samp{$default}), the parser will reduce it. If it was coming from
6292state 0, then, after this reduction it will return to state 0, and will
6293jump to state 2 (@samp{exp: go to state 2}).
6294
6295@example
6296state 2
6297
88bce5a2 6298 $accept -> exp . $ (rule 0)
ec3bc396
AD
6299 exp -> exp . '+' exp (rule 1)
6300 exp -> exp . '-' exp (rule 2)
6301 exp -> exp . '*' exp (rule 3)
6302 exp -> exp . '/' exp (rule 4)
6303
2a8d363a
AD
6304 $ shift, and go to state 3
6305 '+' shift, and go to state 4
6306 '-' shift, and go to state 5
6307 '*' shift, and go to state 6
6308 '/' shift, and go to state 7
ec3bc396
AD
6309@end example
6310
6311@noindent
6312In state 2, the automaton can only shift a symbol. For instance,
8dd162d3 6313because of the item @samp{exp -> exp . '+' exp}, if the look-ahead if
ec3bc396
AD
6314@samp{+}, it will be shifted on the parse stack, and the automaton
6315control will jump to state 4, corresponding to the item @samp{exp -> exp
6316'+' . exp}. Since there is no default action, any other token than
6e649e65 6317those listed above will trigger a syntax error.
ec3bc396
AD
6318
6319The state 3 is named the @dfn{final state}, or the @dfn{accepting
6320state}:
6321
6322@example
6323state 3
6324
88bce5a2 6325 $accept -> exp $ . (rule 0)
ec3bc396 6326
2a8d363a 6327 $default accept
ec3bc396
AD
6328@end example
6329
6330@noindent
6331the initial rule is completed (the start symbol and the end
6332of input were read), the parsing exits successfully.
6333
6334The interpretation of states 4 to 7 is straightforward, and is left to
6335the reader.
6336
6337@example
6338state 4
6339
6340 exp -> exp '+' . exp (rule 1)
6341
2a8d363a 6342 NUM shift, and go to state 1
ec3bc396 6343
2a8d363a 6344 exp go to state 8
ec3bc396
AD
6345
6346state 5
6347
6348 exp -> exp '-' . exp (rule 2)
6349
2a8d363a 6350 NUM shift, and go to state 1
ec3bc396 6351
2a8d363a 6352 exp go to state 9
ec3bc396
AD
6353
6354state 6
6355
6356 exp -> exp '*' . exp (rule 3)
6357
2a8d363a 6358 NUM shift, and go to state 1
ec3bc396 6359
2a8d363a 6360 exp go to state 10
ec3bc396
AD
6361
6362state 7
6363
6364 exp -> exp '/' . exp (rule 4)
6365
2a8d363a 6366 NUM shift, and go to state 1
ec3bc396 6367
2a8d363a 6368 exp go to state 11
ec3bc396
AD
6369@end example
6370
5a99098d
PE
6371As was announced in beginning of the report, @samp{State 8 conflicts:
63721 shift/reduce}:
ec3bc396
AD
6373
6374@example
6375state 8
6376
6377 exp -> exp . '+' exp (rule 1)
6378 exp -> exp '+' exp . (rule 1)
6379 exp -> exp . '-' exp (rule 2)
6380 exp -> exp . '*' exp (rule 3)
6381 exp -> exp . '/' exp (rule 4)
6382
2a8d363a
AD
6383 '*' shift, and go to state 6
6384 '/' shift, and go to state 7
ec3bc396 6385
2a8d363a
AD
6386 '/' [reduce using rule 1 (exp)]
6387 $default reduce using rule 1 (exp)
ec3bc396
AD
6388@end example
6389
8dd162d3 6390Indeed, there are two actions associated to the look-ahead @samp{/}:
ec3bc396
AD
6391either shifting (and going to state 7), or reducing rule 1. The
6392conflict means that either the grammar is ambiguous, or the parser lacks
6393information to make the right decision. Indeed the grammar is
6394ambiguous, as, since we did not specify the precedence of @samp{/}, the
6395sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
6396NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
6397NUM}, which corresponds to reducing rule 1.
6398
c827f760 6399Because in @acronym{LALR}(1) parsing a single decision can be made, Bison
ec3bc396
AD
6400arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
6401Shift/Reduce Conflicts}. Discarded actions are reported in between
6402square brackets.
6403
6404Note that all the previous states had a single possible action: either
6405shifting the next token and going to the corresponding state, or
6406reducing a single rule. In the other cases, i.e., when shifting
6407@emph{and} reducing is possible or when @emph{several} reductions are
8dd162d3
PE
6408possible, the look-ahead is required to select the action. State 8 is
6409one such state: if the look-ahead is @samp{*} or @samp{/} then the action
ec3bc396
AD
6410is shifting, otherwise the action is reducing rule 1. In other words,
6411the first two items, corresponding to rule 1, are not eligible when the
8dd162d3
PE
6412look-ahead token is @samp{*}, since we specified that @samp{*} has higher
6413precedence than @samp{+}. More generally, some items are eligible only
6414with some set of possible look-ahead tokens. When run with
6415@option{--report=look-ahead}, Bison specifies these look-ahead tokens:
ec3bc396
AD
6416
6417@example
6418state 8
6419
6420 exp -> exp . '+' exp [$, '+', '-', '/'] (rule 1)
6421 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
6422 exp -> exp . '-' exp (rule 2)
6423 exp -> exp . '*' exp (rule 3)
6424 exp -> exp . '/' exp (rule 4)
6425
6426 '*' shift, and go to state 6
6427 '/' shift, and go to state 7
6428
6429 '/' [reduce using rule 1 (exp)]
6430 $default reduce using rule 1 (exp)
6431@end example
6432
6433The remaining states are similar:
6434
6435@example
6436state 9
6437
6438 exp -> exp . '+' exp (rule 1)
6439 exp -> exp . '-' exp (rule 2)
6440 exp -> exp '-' exp . (rule 2)
6441 exp -> exp . '*' exp (rule 3)
6442 exp -> exp . '/' exp (rule 4)
6443
2a8d363a
AD
6444 '*' shift, and go to state 6
6445 '/' shift, and go to state 7
ec3bc396 6446
2a8d363a
AD
6447 '/' [reduce using rule 2 (exp)]
6448 $default reduce using rule 2 (exp)
ec3bc396
AD
6449
6450state 10
6451
6452 exp -> exp . '+' exp (rule 1)
6453 exp -> exp . '-' exp (rule 2)
6454 exp -> exp . '*' exp (rule 3)
6455 exp -> exp '*' exp . (rule 3)
6456 exp -> exp . '/' exp (rule 4)
6457
2a8d363a 6458 '/' shift, and go to state 7
ec3bc396 6459
2a8d363a
AD
6460 '/' [reduce using rule 3 (exp)]
6461 $default reduce using rule 3 (exp)
ec3bc396
AD
6462
6463state 11
6464
6465 exp -> exp . '+' exp (rule 1)
6466 exp -> exp . '-' exp (rule 2)
6467 exp -> exp . '*' exp (rule 3)
6468 exp -> exp . '/' exp (rule 4)
6469 exp -> exp '/' exp . (rule 4)
6470
2a8d363a
AD
6471 '+' shift, and go to state 4
6472 '-' shift, and go to state 5
6473 '*' shift, and go to state 6
6474 '/' shift, and go to state 7
ec3bc396 6475
2a8d363a
AD
6476 '+' [reduce using rule 4 (exp)]
6477 '-' [reduce using rule 4 (exp)]
6478 '*' [reduce using rule 4 (exp)]
6479 '/' [reduce using rule 4 (exp)]
6480 $default reduce using rule 4 (exp)
ec3bc396
AD
6481@end example
6482
6483@noindent
fa7e68c3
PE
6484Observe that state 11 contains conflicts not only due to the lack of
6485precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
6486@samp{*}, but also because the
ec3bc396
AD
6487associativity of @samp{/} is not specified.
6488
6489
6490@node Tracing
6491@section Tracing Your Parser
bfa74976
RS
6492@findex yydebug
6493@cindex debugging
6494@cindex tracing the parser
6495
6496If a Bison grammar compiles properly but doesn't do what you want when it
6497runs, the @code{yydebug} parser-trace feature can help you figure out why.
6498
3ded9a63
AD
6499There are several means to enable compilation of trace facilities:
6500
6501@table @asis
6502@item the macro @code{YYDEBUG}
6503@findex YYDEBUG
6504Define the macro @code{YYDEBUG} to a nonzero value when you compile the
c827f760 6505parser. This is compliant with @acronym{POSIX} Yacc. You could use
3ded9a63
AD
6506@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
6507YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
6508Prologue}).
6509
6510@item the option @option{-t}, @option{--debug}
6511Use the @samp{-t} option when you run Bison (@pxref{Invocation,
c827f760 6512,Invoking Bison}). This is @acronym{POSIX} compliant too.
3ded9a63
AD
6513
6514@item the directive @samp{%debug}
6515@findex %debug
6516Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison
6517Declaration Summary}). This is a Bison extension, which will prove
6518useful when Bison will output parsers for languages that don't use a
c827f760
PE
6519preprocessor. Unless @acronym{POSIX} and Yacc portability matter to
6520you, this is
3ded9a63
AD
6521the preferred solution.
6522@end table
6523
6524We suggest that you always enable the debug option so that debugging is
6525always possible.
bfa74976 6526
02a81e05 6527The trace facility outputs messages with macro calls of the form
e2742e46 6528@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
02a81e05 6529@var{format} and @var{args} are the usual @code{printf} format and
4947ebdb
PE
6530arguments. If you define @code{YYDEBUG} to a nonzero value but do not
6531define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
e4e1a4dc 6532and @code{YYPRINTF} is defined to @code{fprintf}.
bfa74976
RS
6533
6534Once you have compiled the program with trace facilities, the way to
6535request a trace is to store a nonzero value in the variable @code{yydebug}.
6536You can do this by making the C code do it (in @code{main}, perhaps), or
6537you can alter the value with a C debugger.
6538
6539Each step taken by the parser when @code{yydebug} is nonzero produces a
6540line or two of trace information, written on @code{stderr}. The trace
6541messages tell you these things:
6542
6543@itemize @bullet
6544@item
6545Each time the parser calls @code{yylex}, what kind of token was read.
6546
6547@item
6548Each time a token is shifted, the depth and complete contents of the
6549state stack (@pxref{Parser States}).
6550
6551@item
6552Each time a rule is reduced, which rule it is, and the complete contents
6553of the state stack afterward.
6554@end itemize
6555
6556To make sense of this information, it helps to refer to the listing file
704a47c4
AD
6557produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
6558Bison}). This file shows the meaning of each state in terms of
6559positions in various rules, and also what each state will do with each
6560possible input token. As you read the successive trace messages, you
6561can see that the parser is functioning according to its specification in
6562the listing file. Eventually you will arrive at the place where
6563something undesirable happens, and you will see which parts of the
6564grammar are to blame.
bfa74976
RS
6565
6566The parser file is a C program and you can use C debuggers on it, but it's
6567not easy to interpret what it is doing. The parser function is a
6568finite-state machine interpreter, and aside from the actions it executes
6569the same code over and over. Only the values of variables show where in
6570the grammar it is working.
6571
6572@findex YYPRINT
6573The debugging information normally gives the token type of each token
6574read, but not its semantic value. You can optionally define a macro
6575named @code{YYPRINT} to provide a way to print the value. If you define
6576@code{YYPRINT}, it should take three arguments. The parser will pass a
6577standard I/O stream, the numeric code for the token type, and the token
6578value (from @code{yylval}).
6579
6580Here is an example of @code{YYPRINT} suitable for the multi-function
6581calculator (@pxref{Mfcalc Decl, ,Declarations for @code{mfcalc}}):
6582
6583@smallexample
38a92d50
PE
6584%@{
6585 static void print_token_value (FILE *, int, YYSTYPE);
6586 #define YYPRINT(file, type, value) print_token_value (file, type, value)
6587%@}
6588
6589@dots{} %% @dots{} %% @dots{}
bfa74976
RS
6590
6591static void
831d3c99 6592print_token_value (FILE *file, int type, YYSTYPE value)
bfa74976
RS
6593@{
6594 if (type == VAR)
d3c4e709 6595 fprintf (file, "%s", value.tptr->name);
bfa74976 6596 else if (type == NUM)
d3c4e709 6597 fprintf (file, "%d", value.val);
bfa74976
RS
6598@}
6599@end smallexample
6600
ec3bc396
AD
6601@c ================================================= Invoking Bison
6602
342b8b6e 6603@node Invocation
bfa74976
RS
6604@chapter Invoking Bison
6605@cindex invoking Bison
6606@cindex Bison invocation
6607@cindex options for invoking Bison
6608
6609The usual way to invoke Bison is as follows:
6610
6611@example
6612bison @var{infile}
6613@end example
6614
6615Here @var{infile} is the grammar file name, which usually ends in
6616@samp{.y}. The parser file's name is made by replacing the @samp{.y}
fa4d969f
PE
6617with @samp{.tab.c} and removing any leading directory. Thus, the
6618@samp{bison foo.y} file name yields
6619@file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields
6620@file{foo.tab.c}. It's also possible, in case you are writing
79282c6c 6621C++ code instead of C in your grammar file, to name it @file{foo.ypp}
72d2299c
PE
6622or @file{foo.y++}. Then, the output files will take an extension like
6623the given one as input (respectively @file{foo.tab.cpp} and
6624@file{foo.tab.c++}).
fa4d969f 6625This feature takes effect with all options that manipulate file names like
234a3be3
AD
6626@samp{-o} or @samp{-d}.
6627
6628For example :
6629
6630@example
6631bison -d @var{infile.yxx}
6632@end example
84163231 6633@noindent
72d2299c 6634will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
234a3be3
AD
6635
6636@example
b56471a6 6637bison -d -o @var{output.c++} @var{infile.y}
234a3be3 6638@end example
84163231 6639@noindent
234a3be3
AD
6640will produce @file{output.c++} and @file{outfile.h++}.
6641
397ec073
PE
6642For compatibility with @acronym{POSIX}, the standard Bison
6643distribution also contains a shell script called @command{yacc} that
6644invokes Bison with the @option{-y} option.
6645
bfa74976 6646@menu
13863333 6647* Bison Options:: All the options described in detail,
c827f760 6648 in alphabetical order by short options.
bfa74976 6649* Option Cross Key:: Alphabetical list of long options.
93dd49ab 6650* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
bfa74976
RS
6651@end menu
6652
342b8b6e 6653@node Bison Options
bfa74976
RS
6654@section Bison Options
6655
6656Bison supports both traditional single-letter options and mnemonic long
6657option names. Long option names are indicated with @samp{--} instead of
6658@samp{-}. Abbreviations for option names are allowed as long as they
6659are unique. When a long option takes an argument, like
6660@samp{--file-prefix}, connect the option name and the argument with
6661@samp{=}.
6662
6663Here is a list of options that can be used with Bison, alphabetized by
6664short option. It is followed by a cross key alphabetized by long
6665option.
6666
89cab50d
AD
6667@c Please, keep this ordered as in `bison --help'.
6668@noindent
6669Operations modes:
6670@table @option
6671@item -h
6672@itemx --help
6673Print a summary of the command-line options to Bison and exit.
bfa74976 6674
89cab50d
AD
6675@item -V
6676@itemx --version
6677Print the version number of Bison and exit.
bfa74976 6678
f7ab6a50
PE
6679@item --print-localedir
6680Print the name of the directory containing locale-dependent data.
6681
89cab50d
AD
6682@need 1750
6683@item -y
6684@itemx --yacc
89cab50d
AD
6685Equivalent to @samp{-o y.tab.c}; the parser output file is called
6686@file{y.tab.c}, and the other outputs are called @file{y.output} and
6687@file{y.tab.h}. The purpose of this option is to imitate Yacc's output
6688file name conventions. Thus, the following shell script can substitute
397ec073
PE
6689for Yacc, and the Bison distribution contains such a script for
6690compatibility with @acronym{POSIX}:
bfa74976 6691
89cab50d 6692@example
397ec073 6693#! /bin/sh
26e06a21 6694bison -y "$@@"
89cab50d
AD
6695@end example
6696@end table
6697
6698@noindent
6699Tuning the parser:
6700
6701@table @option
cd5bd6ac
AD
6702@item -S @var{file}
6703@itemx --skeleton=@var{file}
6704Specify the skeleton to use. You probably don't need this option unless
6705you are developing Bison.
6706
89cab50d
AD
6707@item -t
6708@itemx --debug
4947ebdb
PE
6709In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
6710already defined, so that the debugging facilities are compiled.
ec3bc396 6711@xref{Tracing, ,Tracing Your Parser}.
89cab50d
AD
6712
6713@item --locations
d8988b2f 6714Pretend that @code{%locations} was specified. @xref{Decl Summary}.
89cab50d
AD
6715
6716@item -p @var{prefix}
6717@itemx --name-prefix=@var{prefix}
d8988b2f
AD
6718Pretend that @code{%name-prefix="@var{prefix}"} was specified.
6719@xref{Decl Summary}.
bfa74976
RS
6720
6721@item -l
6722@itemx --no-lines
6723Don't put any @code{#line} preprocessor commands in the parser file.
6724Ordinarily Bison puts them in the parser file so that the C compiler
6725and debuggers will associate errors with your source file, the
6726grammar file. This option causes them to associate errors with the
95e742f7 6727parser file, treating it as an independent source file in its own right.
bfa74976 6728
931c7513
RS
6729@item -n
6730@itemx --no-parser
d8988b2f 6731Pretend that @code{%no-parser} was specified. @xref{Decl Summary}.
931c7513 6732
89cab50d
AD
6733@item -k
6734@itemx --token-table
d8988b2f 6735Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
89cab50d 6736@end table
bfa74976 6737
89cab50d
AD
6738@noindent
6739Adjust the output:
bfa74976 6740
89cab50d
AD
6741@table @option
6742@item -d
d8988b2f
AD
6743@itemx --defines
6744Pretend that @code{%defines} was specified, i.e., write an extra output
6deb4447 6745file containing macro definitions for the token type names defined in
4bfd5e4e 6746the grammar, as well as a few other declarations. @xref{Decl Summary}.
931c7513 6747
342b8b6e 6748@item --defines=@var{defines-file}
d8988b2f 6749Same as above, but save in the file @var{defines-file}.
342b8b6e 6750
89cab50d
AD
6751@item -b @var{file-prefix}
6752@itemx --file-prefix=@var{prefix}
d8988b2f 6753Pretend that @code{%verbose} was specified, i.e, specify prefix to use
72d2299c 6754for all Bison output file names. @xref{Decl Summary}.
bfa74976 6755
ec3bc396
AD
6756@item -r @var{things}
6757@itemx --report=@var{things}
6758Write an extra output file containing verbose description of the comma
6759separated list of @var{things} among:
6760
6761@table @code
6762@item state
6763Description of the grammar, conflicts (resolved and unresolved), and
c827f760 6764@acronym{LALR} automaton.
ec3bc396 6765
8dd162d3 6766@item look-ahead
ec3bc396 6767Implies @code{state} and augments the description of the automaton with
8dd162d3 6768each rule's look-ahead set.
ec3bc396
AD
6769
6770@item itemset
6771Implies @code{state} and augments the description of the automaton with
6772the full set of items for each state, instead of its core only.
6773@end table
6774
6775For instance, on the following grammar
6776
bfa74976
RS
6777@item -v
6778@itemx --verbose
6deb4447
AD
6779Pretend that @code{%verbose} was specified, i.e, write an extra output
6780file containing verbose descriptions of the grammar and
72d2299c 6781parser. @xref{Decl Summary}.
bfa74976 6782
fa4d969f
PE
6783@item -o @var{file}
6784@itemx --output=@var{file}
6785Specify the @var{file} for the parser file.
bfa74976 6786
fa4d969f 6787The other output files' names are constructed from @var{file} as
d8988b2f 6788described under the @samp{-v} and @samp{-d} options.
342b8b6e
AD
6789
6790@item -g
c827f760
PE
6791Output a @acronym{VCG} definition of the @acronym{LALR}(1) grammar
6792automaton computed by Bison. If the grammar file is @file{foo.y}, the
6793@acronym{VCG} output file will
342b8b6e
AD
6794be @file{foo.vcg}.
6795
6796@item --graph=@var{graph-file}
72d2299c
PE
6797The behavior of @var{--graph} is the same than @samp{-g}. The only
6798difference is that it has an optional argument which is the name of
fa4d969f 6799the output graph file.
bfa74976
RS
6800@end table
6801
342b8b6e 6802@node Option Cross Key
bfa74976
RS
6803@section Option Cross Key
6804
6805Here is a list of options, alphabetized by long option, to help you find
6806the corresponding short option.
6807
6808@tex
6809\def\leaderfill{\leaders\hbox to 1em{\hss.\hss}\hfill}
6810
6811{\tt
6812\line{ --debug \leaderfill -t}
6813\line{ --defines \leaderfill -d}
6814\line{ --file-prefix \leaderfill -b}
342b8b6e 6815\line{ --graph \leaderfill -g}
ff51d159 6816\line{ --help \leaderfill -h}
bfa74976
RS
6817\line{ --name-prefix \leaderfill -p}
6818\line{ --no-lines \leaderfill -l}
931c7513 6819\line{ --no-parser \leaderfill -n}
d8988b2f 6820\line{ --output \leaderfill -o}
f7ab6a50 6821\line{ --print-localedir}
931c7513 6822\line{ --token-table \leaderfill -k}
bfa74976
RS
6823\line{ --verbose \leaderfill -v}
6824\line{ --version \leaderfill -V}
6825\line{ --yacc \leaderfill -y}
6826}
6827@end tex
6828
6829@ifinfo
6830@example
6831--debug -t
342b8b6e 6832--defines=@var{defines-file} -d
bfa74976 6833--file-prefix=@var{prefix} -b @var{file-prefix}
342b8b6e 6834--graph=@var{graph-file} -d
ff51d159 6835--help -h
931c7513 6836--name-prefix=@var{prefix} -p @var{name-prefix}
bfa74976 6837--no-lines -l
931c7513 6838--no-parser -n
d8988b2f 6839--output=@var{outfile} -o @var{outfile}
f7ab6a50 6840--print-localedir
931c7513 6841--token-table -k
bfa74976
RS
6842--verbose -v
6843--version -V
8c9a50be 6844--yacc -y
bfa74976
RS
6845@end example
6846@end ifinfo
6847
93dd49ab
PE
6848@node Yacc Library
6849@section Yacc Library
6850
6851The Yacc library contains default implementations of the
6852@code{yyerror} and @code{main} functions. These default
6853implementations are normally not useful, but @acronym{POSIX} requires
6854them. To use the Yacc library, link your program with the
6855@option{-ly} option. Note that Bison's implementation of the Yacc
6856library is distributed under the terms of the @acronym{GNU} General
6857Public License (@pxref{Copying}).
6858
6859If you use the Yacc library's @code{yyerror} function, you should
6860declare @code{yyerror} as follows:
6861
6862@example
6863int yyerror (char const *);
6864@end example
6865
6866Bison ignores the @code{int} value returned by this @code{yyerror}.
6867If you use the Yacc library's @code{main} function, your
6868@code{yyparse} function should have the following type signature:
6869
6870@example
6871int yyparse (void);
6872@end example
6873
12545799
AD
6874@c ================================================= C++ Bison
6875
6876@node C++ Language Interface
6877@chapter C++ Language Interface
6878
6879@menu
6880* C++ Parsers:: The interface to generate C++ parser classes
6881* A Complete C++ Example:: Demonstrating their use
6882@end menu
6883
6884@node C++ Parsers
6885@section C++ Parsers
6886
6887@menu
6888* C++ Bison Interface:: Asking for C++ parser generation
6889* C++ Semantic Values:: %union vs. C++
6890* C++ Location Values:: The position and location classes
6891* C++ Parser Interface:: Instantiating and running the parser
6892* C++ Scanner Interface:: Exchanges between yylex and parse
6893@end menu
6894
6895@node C++ Bison Interface
6896@subsection C++ Bison Interface
6897@c - %skeleton "lalr1.cc"
6898@c - Always pure
6899@c - initial action
6900
e054b190 6901The C++ parser @acronym{LALR}(1) skeleton is named @file{lalr1.cc}. To select
12545799
AD
6902it, you may either pass the option @option{--skeleton=lalr1.cc} to
6903Bison, or include the directive @samp{%skeleton "lalr1.cc"} in the
6904grammar preamble. When run, @command{bison} will create several
6905files:
6906@table @file
6907@item position.hh
6908@itemx location.hh
6909The definition of the classes @code{position} and @code{location},
6910used for location tracking. @xref{C++ Location Values}.
6911
6912@item stack.hh
6913An auxiliary class @code{stack} used by the parser.
6914
fa4d969f
PE
6915@item @var{file}.hh
6916@itemx @var{file}.cc
12545799 6917The declaration and implementation of the C++ parser class.
fa4d969f 6918@var{file} is the name of the output file. It follows the same
12545799
AD
6919rules as with regular C parsers.
6920
fa4d969f 6921Note that @file{@var{file}.hh} is @emph{mandatory}, the C++ cannot
12545799
AD
6922work without the parser class declaration. Therefore, you must either
6923pass @option{-d}/@option{--defines} to @command{bison}, or use the
6924@samp{%defines} directive.
6925@end table
6926
6927All these files are documented using Doxygen; run @command{doxygen}
6928for a complete and accurate documentation.
6929
6930@node C++ Semantic Values
6931@subsection C++ Semantic Values
6932@c - No objects in unions
6933@c - YSTYPE
6934@c - Printer and destructor
6935
6936The @code{%union} directive works as for C, see @ref{Union Decl, ,The
6937Collection of Value Types}. In particular it produces a genuine
6938@code{union}@footnote{In the future techniques to allow complex types
fb9712a9
AD
6939within pseudo-unions (similar to Boost variants) might be implemented to
6940alleviate these issues.}, which have a few specific features in C++.
12545799
AD
6941@itemize @minus
6942@item
fb9712a9
AD
6943The type @code{YYSTYPE} is defined but its use is discouraged: rather
6944you should refer to the parser's encapsulated type
6945@code{yy::parser::semantic_type}.
12545799
AD
6946@item
6947Non POD (Plain Old Data) types cannot be used. C++ forbids any
6948instance of classes with constructors in unions: only @emph{pointers}
6949to such objects are allowed.
6950@end itemize
6951
6952Because objects have to be stored via pointers, memory is not
6953reclaimed automatically: using the @code{%destructor} directive is the
6954only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
6955Symbols}.
6956
6957
6958@node C++ Location Values
6959@subsection C++ Location Values
6960@c - %locations
6961@c - class Position
6962@c - class Location
b47dbebe 6963@c - %define "filename_type" "const symbol::Symbol"
12545799
AD
6964
6965When the directive @code{%locations} is used, the C++ parser supports
6966location tracking, see @ref{Locations, , Locations Overview}. Two
6967auxiliary classes define a @code{position}, a single point in a file,
6968and a @code{location}, a range composed of a pair of
6969@code{position}s (possibly spanning several files).
6970
fa4d969f 6971@deftypemethod {position} {std::string*} file
12545799
AD
6972The name of the file. It will always be handled as a pointer, the
6973parser will never duplicate nor deallocate it. As an experimental
6974feature you may change it to @samp{@var{type}*} using @samp{%define
b47dbebe 6975"filename_type" "@var{type}"}.
12545799
AD
6976@end deftypemethod
6977
6978@deftypemethod {position} {unsigned int} line
6979The line, starting at 1.
6980@end deftypemethod
6981
6982@deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
6983Advance by @var{height} lines, resetting the column number.
6984@end deftypemethod
6985
6986@deftypemethod {position} {unsigned int} column
6987The column, starting at 0.
6988@end deftypemethod
6989
6990@deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
6991Advance by @var{width} columns, without changing the line number.
6992@end deftypemethod
6993
6994@deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
6995@deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
6996@deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
6997@deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
6998Various forms of syntactic sugar for @code{columns}.
6999@end deftypemethod
7000
7001@deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
7002Report @var{p} on @var{o} like this:
fa4d969f
PE
7003@samp{@var{file}:@var{line}.@var{column}}, or
7004@samp{@var{line}.@var{column}} if @var{file} is null.
12545799
AD
7005@end deftypemethod
7006
7007@deftypemethod {location} {position} begin
7008@deftypemethodx {location} {position} end
7009The first, inclusive, position of the range, and the first beyond.
7010@end deftypemethod
7011
7012@deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
7013@deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
7014Advance the @code{end} position.
7015@end deftypemethod
7016
7017@deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
7018@deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
7019@deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
7020Various forms of syntactic sugar.
7021@end deftypemethod
7022
7023@deftypemethod {location} {void} step ()
7024Move @code{begin} onto @code{end}.
7025@end deftypemethod
7026
7027
7028@node C++ Parser Interface
7029@subsection C++ Parser Interface
7030@c - define parser_class_name
7031@c - Ctor
7032@c - parse, error, set_debug_level, debug_level, set_debug_stream,
7033@c debug_stream.
7034@c - Reporting errors
7035
7036The output files @file{@var{output}.hh} and @file{@var{output}.cc}
7037declare and define the parser class in the namespace @code{yy}. The
7038class name defaults to @code{parser}, but may be changed using
7039@samp{%define "parser_class_name" "@var{name}"}. The interface of
7040this class is detailled below. It can be extended using the
7041@code{%parse-param} feature: its semantics is slightly changed since
7042it describes an additional member of the parser class, and an
7043additional argument for its constructor.
7044
8a0adb01
AD
7045@defcv {Type} {parser} {semantic_value_type}
7046@defcvx {Type} {parser} {location_value_type}
12545799 7047The types for semantics value and locations.
8a0adb01 7048@end defcv
12545799
AD
7049
7050@deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
7051Build a new parser object. There are no arguments by default, unless
7052@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
7053@end deftypemethod
7054
7055@deftypemethod {parser} {int} parse ()
7056Run the syntactic analysis, and return 0 on success, 1 otherwise.
7057@end deftypemethod
7058
7059@deftypemethod {parser} {std::ostream&} debug_stream ()
7060@deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
7061Get or set the stream used for tracing the parsing. It defaults to
7062@code{std::cerr}.
7063@end deftypemethod
7064
7065@deftypemethod {parser} {debug_level_type} debug_level ()
7066@deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
7067Get or set the tracing level. Currently its value is either 0, no trace,
7068or non-zero, full tracing.
7069@end deftypemethod
7070
7071@deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
7072The definition for this member function must be supplied by the user:
7073the parser uses it to report a parser error occurring at @var{l},
7074described by @var{m}.
7075@end deftypemethod
7076
7077
7078@node C++ Scanner Interface
7079@subsection C++ Scanner Interface
7080@c - prefix for yylex.
7081@c - Pure interface to yylex
7082@c - %lex-param
7083
7084The parser invokes the scanner by calling @code{yylex}. Contrary to C
7085parsers, C++ parsers are always pure: there is no point in using the
7086@code{%pure-parser} directive. Therefore the interface is as follows.
7087
7088@deftypemethod {parser} {int} yylex (semantic_value_type& @var{yylval}, location_type& @var{yylloc}, @var{type1} @var{arg1}, ...)
7089Return the next token. Its type is the return value, its semantic
7090value and location being @var{yylval} and @var{yylloc}. Invocations of
7091@samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
7092@end deftypemethod
7093
7094
7095@node A Complete C++ Example
7096@section A Complete C++ Example
7097
7098This section demonstrates the use of a C++ parser with a simple but
7099complete example. This example should be available on your system,
7100ready to compile, in the directory @dfn{../bison/examples/calc++}. It
7101focuses on the use of Bison, therefore the design of the various C++
7102classes is very naive: no accessors, no encapsulation of members etc.
7103We will use a Lex scanner, and more precisely, a Flex scanner, to
7104demonstrate the various interaction. A hand written scanner is
7105actually easier to interface with.
7106
7107@menu
7108* Calc++ --- C++ Calculator:: The specifications
7109* Calc++ Parsing Driver:: An active parsing context
7110* Calc++ Parser:: A parser class
7111* Calc++ Scanner:: A pure C++ Flex scanner
7112* Calc++ Top Level:: Conducting the band
7113@end menu
7114
7115@node Calc++ --- C++ Calculator
7116@subsection Calc++ --- C++ Calculator
7117
7118Of course the grammar is dedicated to arithmetics, a single
7119expression, possibily preceded by variable assignments. An
7120environment containing possibly predefined variables such as
7121@code{one} and @code{two}, is exchanged with the parser. An example
7122of valid input follows.
7123
7124@example
7125three := 3
7126seven := one + two * three
7127seven * seven
7128@end example
7129
7130@node Calc++ Parsing Driver
7131@subsection Calc++ Parsing Driver
7132@c - An env
7133@c - A place to store error messages
7134@c - A place for the result
7135
7136To support a pure interface with the parser (and the scanner) the
7137technique of the ``parsing context'' is convenient: a structure
7138containing all the data to exchange. Since, in addition to simply
7139launch the parsing, there are several auxiliary tasks to execute (open
7140the file for parsing, instantiate the parser etc.), we recommend
7141transforming the simple parsing context structure into a fully blown
7142@dfn{parsing driver} class.
7143
7144The declaration of this driver class, @file{calc++-driver.hh}, is as
7145follows. The first part includes the CPP guard and imports the
fb9712a9
AD
7146required standard library components, and the declaration of the parser
7147class.
12545799 7148
1c59e0a1 7149@comment file: calc++-driver.hh
12545799
AD
7150@example
7151#ifndef CALCXX_DRIVER_HH
7152# define CALCXX_DRIVER_HH
7153# include <string>
7154# include <map>
fb9712a9 7155# include "calc++-parser.hh"
12545799
AD
7156@end example
7157
12545799
AD
7158
7159@noindent
7160Then comes the declaration of the scanning function. Flex expects
7161the signature of @code{yylex} to be defined in the macro
7162@code{YY_DECL}, and the C++ parser expects it to be declared. We can
7163factor both as follows.
1c59e0a1
AD
7164
7165@comment file: calc++-driver.hh
12545799
AD
7166@example
7167// Announce to Flex the prototype we want for lexing function, ...
1c59e0a1 7168# define YY_DECL \
fb9712a9
AD
7169 int yylex (yy::calcxx_parser::semantic_type* yylval, \
7170 yy::calcxx_parser::location_type* yylloc, \
7171 calcxx_driver& driver)
12545799
AD
7172// ... and declare it for the parser's sake.
7173YY_DECL;
7174@end example
7175
7176@noindent
7177The @code{calcxx_driver} class is then declared with its most obvious
7178members.
7179
1c59e0a1 7180@comment file: calc++-driver.hh
12545799
AD
7181@example
7182// Conducting the whole scanning and parsing of Calc++.
7183class calcxx_driver
7184@{
7185public:
7186 calcxx_driver ();
7187 virtual ~calcxx_driver ();
7188
7189 std::map<std::string, int> variables;
7190
7191 int result;
7192@end example
7193
7194@noindent
7195To encapsulate the coordination with the Flex scanner, it is useful to
7196have two members function to open and close the scanning phase.
7197members.
7198
1c59e0a1 7199@comment file: calc++-driver.hh
12545799
AD
7200@example
7201 // Handling the scanner.
7202 void scan_begin ();
7203 void scan_end ();
7204 bool trace_scanning;
7205@end example
7206
7207@noindent
7208Similarly for the parser itself.
7209
1c59e0a1 7210@comment file: calc++-driver.hh
12545799
AD
7211@example
7212 // Handling the parser.
7213 void parse (const std::string& f);
7214 std::string file;
7215 bool trace_parsing;
7216@end example
7217
7218@noindent
7219To demonstrate pure handling of parse errors, instead of simply
7220dumping them on the standard error output, we will pass them to the
7221compiler driver using the following two member functions. Finally, we
7222close the class declaration and CPP guard.
7223
1c59e0a1 7224@comment file: calc++-driver.hh
12545799
AD
7225@example
7226 // Error handling.
7227 void error (const yy::location& l, const std::string& m);
7228 void error (const std::string& m);
7229@};
7230#endif // ! CALCXX_DRIVER_HH
7231@end example
7232
7233The implementation of the driver is straightforward. The @code{parse}
7234member function deserves some attention. The @code{error} functions
7235are simple stubs, they should actually register the located error
7236messages and set error state.
7237
1c59e0a1 7238@comment file: calc++-driver.cc
12545799
AD
7239@example
7240#include "calc++-driver.hh"
7241#include "calc++-parser.hh"
7242
7243calcxx_driver::calcxx_driver ()
7244 : trace_scanning (false), trace_parsing (false)
7245@{
7246 variables["one"] = 1;
7247 variables["two"] = 2;
7248@}
7249
7250calcxx_driver::~calcxx_driver ()
7251@{
7252@}
7253
7254void
7255calcxx_driver::parse (const std::string &f)
7256@{
7257 file = f;
7258 scan_begin ();
7259 yy::calcxx_parser parser (*this);
7260 parser.set_debug_level (trace_parsing);
7261 parser.parse ();
7262 scan_end ();
7263@}
7264
7265void
7266calcxx_driver::error (const yy::location& l, const std::string& m)
7267@{
7268 std::cerr << l << ": " << m << std::endl;
7269@}
7270
7271void
7272calcxx_driver::error (const std::string& m)
7273@{
7274 std::cerr << m << std::endl;
7275@}
7276@end example
7277
7278@node Calc++ Parser
7279@subsection Calc++ Parser
7280
b50d2359
AD
7281The parser definition file @file{calc++-parser.yy} starts by asking for
7282the C++ LALR(1) skeleton, the creation of the parser header file, and
7283specifies the name of the parser class. Because the C++ skeleton
7284changed several times, it is safer to require the version you designed
7285the grammar for.
1c59e0a1
AD
7286
7287@comment file: calc++-parser.yy
12545799
AD
7288@example
7289%skeleton "lalr1.cc" /* -*- C++ -*- */
b50d2359 7290%require "2.1a"
12545799 7291%defines
fb9712a9
AD
7292%define "parser_class_name" "calcxx_parser"
7293@end example
7294
7295@noindent
7296Then come the declarations/inclusions needed to define the
7297@code{%union}. Because the parser uses the parsing driver and
7298reciprocally, both cannot include the header of the other. Because the
7299driver's header needs detailed knowledge about the parser class (in
7300particular its inner types), it is the parser's header which will simply
7301use a forward declaration of the driver.
7302
7303@comment file: calc++-parser.yy
7304@example
12545799
AD
7305%@{
7306# include <string>
fb9712a9 7307class calcxx_driver;
12545799
AD
7308%@}
7309@end example
7310
7311@noindent
7312The driver is passed by reference to the parser and to the scanner.
7313This provides a simple but effective pure interface, not relying on
7314global variables.
7315
1c59e0a1 7316@comment file: calc++-parser.yy
12545799
AD
7317@example
7318// The parsing context.
7319%parse-param @{ calcxx_driver& driver @}
7320%lex-param @{ calcxx_driver& driver @}
7321@end example
7322
7323@noindent
7324Then we request the location tracking feature, and initialize the
7325first location's file name. Afterwards new locations are computed
7326relatively to the previous locations: the file name will be
7327automatically propagated.
7328
1c59e0a1 7329@comment file: calc++-parser.yy
12545799
AD
7330@example
7331%locations
7332%initial-action
7333@{
7334 // Initialize the initial location.
b47dbebe 7335 @@$.begin.filename = @@$.end.filename = &driver.file;
12545799
AD
7336@};
7337@end example
7338
7339@noindent
7340Use the two following directives to enable parser tracing and verbose
7341error messages.
7342
1c59e0a1 7343@comment file: calc++-parser.yy
12545799
AD
7344@example
7345%debug
7346%error-verbose
7347@end example
7348
7349@noindent
7350Semantic values cannot use ``real'' objects, but only pointers to
7351them.
7352
1c59e0a1 7353@comment file: calc++-parser.yy
12545799
AD
7354@example
7355// Symbols.
7356%union
7357@{
7358 int ival;
7359 std::string *sval;
7360@};
7361@end example
7362
fb9712a9
AD
7363@noindent
7364The code between @samp{%@{} and @samp{%@}} after the introduction of the
7365@samp{%union} is output in the @file{*.cc} file; it needs detailed
7366knowledge about the driver.
7367
7368@comment file: calc++-parser.yy
7369@example
7370%@{
7371# include "calc++-driver.hh"
7372%@}
7373@end example
7374
7375
12545799
AD
7376@noindent
7377The token numbered as 0 corresponds to end of file; the following line
7378allows for nicer error messages referring to ``end of file'' instead
7379of ``$end''. Similarly user friendly named are provided for each
7380symbol. Note that the tokens names are prefixed by @code{TOKEN_} to
7381avoid name clashes.
7382
1c59e0a1 7383@comment file: calc++-parser.yy
12545799 7384@example
fb9712a9
AD
7385%token END 0 "end of file"
7386%token ASSIGN ":="
7387%token <sval> IDENTIFIER "identifier"
7388%token <ival> NUMBER "number"
7389%type <ival> exp "expression"
12545799
AD
7390@end example
7391
7392@noindent
7393To enable memory deallocation during error recovery, use
7394@code{%destructor}.
7395
1c59e0a1 7396@comment file: calc++-parser.yy
12545799
AD
7397@example
7398%printer @{ debug_stream () << *$$; @} "identifier"
7399%destructor @{ delete $$; @} "identifier"
7400
7401%printer @{ debug_stream () << $$; @} "number" "expression"
7402@end example
7403
7404@noindent
7405The grammar itself is straightforward.
7406
1c59e0a1 7407@comment file: calc++-parser.yy
12545799
AD
7408@example
7409%%
7410%start unit;
7411unit: assignments exp @{ driver.result = $2; @};
7412
7413assignments: assignments assignment @{@}
7414 | /* Nothing. */ @{@};
7415
fb9712a9 7416assignment: "identifier" ":=" exp @{ driver.variables[*$1] = $3; @};
12545799
AD
7417
7418%left '+' '-';
7419%left '*' '/';
7420exp: exp '+' exp @{ $$ = $1 + $3; @}
7421 | exp '-' exp @{ $$ = $1 - $3; @}
7422 | exp '*' exp @{ $$ = $1 * $3; @}
7423 | exp '/' exp @{ $$ = $1 / $3; @}
fb9712a9
AD
7424 | "identifier" @{ $$ = driver.variables[*$1]; @}
7425 | "number" @{ $$ = $1; @};
12545799
AD
7426%%
7427@end example
7428
7429@noindent
7430Finally the @code{error} member function registers the errors to the
7431driver.
7432
1c59e0a1 7433@comment file: calc++-parser.yy
12545799
AD
7434@example
7435void
1c59e0a1
AD
7436yy::calcxx_parser::error (const yy::calcxx_parser::location_type& l,
7437 const std::string& m)
12545799
AD
7438@{
7439 driver.error (l, m);
7440@}
7441@end example
7442
7443@node Calc++ Scanner
7444@subsection Calc++ Scanner
7445
7446The Flex scanner first includes the driver declaration, then the
7447parser's to get the set of defined tokens.
7448
1c59e0a1 7449@comment file: calc++-scanner.ll
12545799
AD
7450@example
7451%@{ /* -*- C++ -*- */
04098407
PE
7452# include <cstdlib>
7453# include <errno.h>
7454# include <limits.h>
12545799
AD
7455# include <string>
7456# include "calc++-driver.hh"
7457# include "calc++-parser.hh"
7458%@}
7459@end example
7460
7461@noindent
7462Because there is no @code{#include}-like feature we don't need
7463@code{yywrap}, we don't need @code{unput} either, and we parse an
7464actual file, this is not an interactive session with the user.
7465Finally we enable the scanner tracing features.
7466
1c59e0a1 7467@comment file: calc++-scanner.ll
12545799
AD
7468@example
7469%option noyywrap nounput batch debug
7470@end example
7471
7472@noindent
7473Abbreviations allow for more readable rules.
7474
1c59e0a1 7475@comment file: calc++-scanner.ll
12545799
AD
7476@example
7477id [a-zA-Z][a-zA-Z_0-9]*
7478int [0-9]+
7479blank [ \t]
7480@end example
7481
7482@noindent
7483The following paragraph suffices to track locations acurately. Each
7484time @code{yylex} is invoked, the begin position is moved onto the end
7485position. Then when a pattern is matched, the end position is
7486advanced of its width. In case it matched ends of lines, the end
7487cursor is adjusted, and each time blanks are matched, the begin cursor
7488is moved onto the end cursor to effectively ignore the blanks
7489preceding tokens. Comments would be treated equally.
7490
1c59e0a1 7491@comment file: calc++-scanner.ll
12545799 7492@example
828c373b
AD
7493%@{
7494# define YY_USER_ACTION yylloc->columns (yyleng);
7495%@}
12545799
AD
7496%%
7497%@{
7498 yylloc->step ();
12545799
AD
7499%@}
7500@{blank@}+ yylloc->step ();
7501[\n]+ yylloc->lines (yyleng); yylloc->step ();
7502@end example
7503
7504@noindent
fb9712a9
AD
7505The rules are simple, just note the use of the driver to report errors.
7506It is convenient to use a typedef to shorten
7507@code{yy::calcxx_parser::token::identifier} into
7508@code{token::identifier} for isntance.
12545799 7509
1c59e0a1 7510@comment file: calc++-scanner.ll
12545799 7511@example
fb9712a9
AD
7512%@{
7513 typedef yy::calcxx_parser::token token;
7514%@}
7515
12545799 7516[-+*/] return yytext[0];
fb9712a9 7517":=" return token::ASSIGN;
04098407
PE
7518@{int@} @{
7519 errno = 0;
7520 long n = strtol (yytext, NULL, 10);
7521 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
7522 driver.error (*yylloc, "integer is out of range");
7523 yylval->ival = n;
fb9712a9 7524 return token::NUMBER;
04098407 7525@}
fb9712a9 7526@{id@} yylval->sval = new std::string (yytext); return token::IDENTIFIER;
12545799
AD
7527. driver.error (*yylloc, "invalid character");
7528%%
7529@end example
7530
7531@noindent
7532Finally, because the scanner related driver's member function depend
7533on the scanner's data, it is simpler to implement them in this file.
7534
1c59e0a1 7535@comment file: calc++-scanner.ll
12545799
AD
7536@example
7537void
7538calcxx_driver::scan_begin ()
7539@{
7540 yy_flex_debug = trace_scanning;
7541 if (!(yyin = fopen (file.c_str (), "r")))
7542 error (std::string ("cannot open ") + file);
7543@}
7544
7545void
7546calcxx_driver::scan_end ()
7547@{
7548 fclose (yyin);
7549@}
7550@end example
7551
7552@node Calc++ Top Level
7553@subsection Calc++ Top Level
7554
7555The top level file, @file{calc++.cc}, poses no problem.
7556
1c59e0a1 7557@comment file: calc++.cc
12545799
AD
7558@example
7559#include <iostream>
7560#include "calc++-driver.hh"
7561
7562int
fa4d969f 7563main (int argc, char *argv[])
12545799
AD
7564@{
7565 calcxx_driver driver;
7566 for (++argv; argv[0]; ++argv)
7567 if (*argv == std::string ("-p"))
7568 driver.trace_parsing = true;
7569 else if (*argv == std::string ("-s"))
7570 driver.trace_scanning = true;
7571 else
7572 @{
7573 driver.parse (*argv);
7574 std::cout << driver.result << std::endl;
7575 @}
7576@}
7577@end example
7578
7579@c ================================================= FAQ
d1a1114f
AD
7580
7581@node FAQ
7582@chapter Frequently Asked Questions
7583@cindex frequently asked questions
7584@cindex questions
7585
7586Several questions about Bison come up occasionally. Here some of them
7587are addressed.
7588
7589@menu
1a059451 7590* Memory Exhausted:: Breaking the Stack Limits
e64fec0a 7591* How Can I Reset the Parser:: @code{yyparse} Keeps some State
fef4cb51 7592* Strings are Destroyed:: @code{yylval} Loses Track of Strings
2fa09258 7593* Implementing Gotos/Loops:: Control Flow in the Calculator
d1a1114f
AD
7594@end menu
7595
1a059451
PE
7596@node Memory Exhausted
7597@section Memory Exhausted
d1a1114f
AD
7598
7599@display
1a059451 7600My parser returns with error with a @samp{memory exhausted}
d1a1114f
AD
7601message. What can I do?
7602@end display
7603
7604This question is already addressed elsewhere, @xref{Recursion,
7605,Recursive Rules}.
7606
e64fec0a
PE
7607@node How Can I Reset the Parser
7608@section How Can I Reset the Parser
5b066063 7609
0e14ad77
PE
7610The following phenomenon has several symptoms, resulting in the
7611following typical questions:
5b066063
AD
7612
7613@display
7614I invoke @code{yyparse} several times, and on correct input it works
7615properly; but when a parse error is found, all the other calls fail
0e14ad77 7616too. How can I reset the error flag of @code{yyparse}?
5b066063
AD
7617@end display
7618
7619@noindent
7620or
7621
7622@display
0e14ad77 7623My parser includes support for an @samp{#include}-like feature, in
5b066063
AD
7624which case I run @code{yyparse} from @code{yyparse}. This fails
7625although I did specify I needed a @code{%pure-parser}.
7626@end display
7627
0e14ad77
PE
7628These problems typically come not from Bison itself, but from
7629Lex-generated scanners. Because these scanners use large buffers for
5b066063
AD
7630speed, they might not notice a change of input file. As a
7631demonstration, consider the following source file,
7632@file{first-line.l}:
7633
7634@verbatim
7635%{
7636#include <stdio.h>
7637#include <stdlib.h>
7638%}
7639%%
7640.*\n ECHO; return 1;
7641%%
7642int
0e14ad77 7643yyparse (char const *file)
5b066063
AD
7644{
7645 yyin = fopen (file, "r");
7646 if (!yyin)
7647 exit (2);
fa7e68c3 7648 /* One token only. */
5b066063 7649 yylex ();
0e14ad77 7650 if (fclose (yyin) != 0)
5b066063
AD
7651 exit (3);
7652 return 0;
7653}
7654
7655int
0e14ad77 7656main (void)
5b066063
AD
7657{
7658 yyparse ("input");
7659 yyparse ("input");
7660 return 0;
7661}
7662@end verbatim
7663
7664@noindent
7665If the file @file{input} contains
7666
7667@verbatim
7668input:1: Hello,
7669input:2: World!
7670@end verbatim
7671
7672@noindent
0e14ad77 7673then instead of getting the first line twice, you get:
5b066063
AD
7674
7675@example
7676$ @kbd{flex -ofirst-line.c first-line.l}
7677$ @kbd{gcc -ofirst-line first-line.c -ll}
7678$ @kbd{./first-line}
7679input:1: Hello,
7680input:2: World!
7681@end example
7682
0e14ad77
PE
7683Therefore, whenever you change @code{yyin}, you must tell the
7684Lex-generated scanner to discard its current buffer and switch to the
7685new one. This depends upon your implementation of Lex; see its
7686documentation for more. For Flex, it suffices to call
7687@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
7688Flex-generated scanner needs to read from several input streams to
7689handle features like include files, you might consider using Flex
7690functions like @samp{yy_switch_to_buffer} that manipulate multiple
7691input buffers.
5b066063 7692
b165c324
AD
7693If your Flex-generated scanner uses start conditions (@pxref{Start
7694conditions, , Start conditions, flex, The Flex Manual}), you might
7695also want to reset the scanner's state, i.e., go back to the initial
7696start condition, through a call to @samp{BEGIN (0)}.
7697
fef4cb51
AD
7698@node Strings are Destroyed
7699@section Strings are Destroyed
7700
7701@display
c7e441b4 7702My parser seems to destroy old strings, or maybe it loses track of
fef4cb51
AD
7703them. Instead of reporting @samp{"foo", "bar"}, it reports
7704@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
7705@end display
7706
7707This error is probably the single most frequent ``bug report'' sent to
7708Bison lists, but is only concerned with a misunderstanding of the role
7709of scanner. Consider the following Lex code:
7710
7711@verbatim
7712%{
7713#include <stdio.h>
7714char *yylval = NULL;
7715%}
7716%%
7717.* yylval = yytext; return 1;
7718\n /* IGNORE */
7719%%
7720int
7721main ()
7722{
fa7e68c3 7723 /* Similar to using $1, $2 in a Bison action. */
fef4cb51
AD
7724 char *fst = (yylex (), yylval);
7725 char *snd = (yylex (), yylval);
7726 printf ("\"%s\", \"%s\"\n", fst, snd);
7727 return 0;
7728}
7729@end verbatim
7730
7731If you compile and run this code, you get:
7732
7733@example
7734$ @kbd{flex -osplit-lines.c split-lines.l}
7735$ @kbd{gcc -osplit-lines split-lines.c -ll}
7736$ @kbd{printf 'one\ntwo\n' | ./split-lines}
7737"one
7738two", "two"
7739@end example
7740
7741@noindent
7742this is because @code{yytext} is a buffer provided for @emph{reading}
7743in the action, but if you want to keep it, you have to duplicate it
7744(e.g., using @code{strdup}). Note that the output may depend on how
7745your implementation of Lex handles @code{yytext}. For instance, when
7746given the Lex compatibility option @option{-l} (which triggers the
7747option @samp{%array}) Flex generates a different behavior:
7748
7749@example
7750$ @kbd{flex -l -osplit-lines.c split-lines.l}
7751$ @kbd{gcc -osplit-lines split-lines.c -ll}
7752$ @kbd{printf 'one\ntwo\n' | ./split-lines}
7753"two", "two"
7754@end example
7755
7756
2fa09258
AD
7757@node Implementing Gotos/Loops
7758@section Implementing Gotos/Loops
a06ea4aa
AD
7759
7760@display
7761My simple calculator supports variables, assignments, and functions,
2fa09258 7762but how can I implement gotos, or loops?
a06ea4aa
AD
7763@end display
7764
7765Although very pedagogical, the examples included in the document blur
a1c84f45 7766the distinction to make between the parser---whose job is to recover
a06ea4aa 7767the structure of a text and to transmit it to subsequent modules of
a1c84f45 7768the program---and the processing (such as the execution) of this
a06ea4aa
AD
7769structure. This works well with so called straight line programs,
7770i.e., precisely those that have a straightforward execution model:
7771execute simple instructions one after the others.
7772
7773@cindex abstract syntax tree
7774@cindex @acronym{AST}
7775If you want a richer model, you will probably need to use the parser
7776to construct a tree that does represent the structure it has
7777recovered; this tree is usually called the @dfn{abstract syntax tree},
7778or @dfn{@acronym{AST}} for short. Then, walking through this tree,
7779traversing it in various ways, will enable treatments such as its
7780execution or its translation, which will result in an interpreter or a
7781compiler.
7782
7783This topic is way beyond the scope of this manual, and the reader is
7784invited to consult the dedicated literature.
7785
7786
7787
d1a1114f
AD
7788@c ================================================= Table of Symbols
7789
342b8b6e 7790@node Table of Symbols
bfa74976
RS
7791@appendix Bison Symbols
7792@cindex Bison symbols, table of
7793@cindex symbols in Bison, table of
7794
18b519c0 7795@deffn {Variable} @@$
3ded9a63 7796In an action, the location of the left-hand side of the rule.
88bce5a2 7797@xref{Locations, , Locations Overview}.
18b519c0 7798@end deffn
3ded9a63 7799
18b519c0 7800@deffn {Variable} @@@var{n}
3ded9a63
AD
7801In an action, the location of the @var{n}-th symbol of the right-hand
7802side of the rule. @xref{Locations, , Locations Overview}.
18b519c0 7803@end deffn
3ded9a63 7804
18b519c0 7805@deffn {Variable} $$
3ded9a63
AD
7806In an action, the semantic value of the left-hand side of the rule.
7807@xref{Actions}.
18b519c0 7808@end deffn
3ded9a63 7809
18b519c0 7810@deffn {Variable} $@var{n}
3ded9a63
AD
7811In an action, the semantic value of the @var{n}-th symbol of the
7812right-hand side of the rule. @xref{Actions}.
18b519c0 7813@end deffn
3ded9a63 7814
dd8d9022
AD
7815@deffn {Delimiter} %%
7816Delimiter used to separate the grammar rule section from the
7817Bison declarations section or the epilogue.
7818@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
18b519c0 7819@end deffn
bfa74976 7820
dd8d9022
AD
7821@c Don't insert spaces, or check the DVI output.
7822@deffn {Delimiter} %@{@var{code}%@}
7823All code listed between @samp{%@{} and @samp{%@}} is copied directly to
7824the output file uninterpreted. Such code forms the prologue of the input
7825file. @xref{Grammar Outline, ,Outline of a Bison
7826Grammar}.
18b519c0 7827@end deffn
bfa74976 7828
dd8d9022
AD
7829@deffn {Construct} /*@dots{}*/
7830Comment delimiters, as in C.
18b519c0 7831@end deffn
bfa74976 7832
dd8d9022
AD
7833@deffn {Delimiter} :
7834Separates a rule's result from its components. @xref{Rules, ,Syntax of
7835Grammar Rules}.
18b519c0 7836@end deffn
bfa74976 7837
dd8d9022
AD
7838@deffn {Delimiter} ;
7839Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 7840@end deffn
bfa74976 7841
dd8d9022
AD
7842@deffn {Delimiter} |
7843Separates alternate rules for the same result nonterminal.
7844@xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 7845@end deffn
bfa74976 7846
dd8d9022
AD
7847@deffn {Symbol} $accept
7848The predefined nonterminal whose only rule is @samp{$accept: @var{start}
7849$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
7850Start-Symbol}. It cannot be used in the grammar.
18b519c0 7851@end deffn
bfa74976 7852
18b519c0 7853@deffn {Directive} %debug
6deb4447 7854Equip the parser for debugging. @xref{Decl Summary}.
18b519c0 7855@end deffn
6deb4447 7856
91d2c560 7857@ifset defaultprec
22fccf95
PE
7858@deffn {Directive} %default-prec
7859Assign a precedence to rules that lack an explicit @samp{%prec}
7860modifier. @xref{Contextual Precedence, ,Context-Dependent
7861Precedence}.
39a06c25 7862@end deffn
91d2c560 7863@end ifset
39a06c25 7864
18b519c0 7865@deffn {Directive} %defines
6deb4447
AD
7866Bison declaration to create a header file meant for the scanner.
7867@xref{Decl Summary}.
18b519c0 7868@end deffn
6deb4447 7869
18b519c0 7870@deffn {Directive} %destructor
258b75ca 7871Specify how the parser should reclaim the memory associated to
fa7e68c3 7872discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 7873@end deffn
72f889cc 7874
18b519c0 7875@deffn {Directive} %dprec
676385e2 7876Bison declaration to assign a precedence to a rule that is used at parse
c827f760
PE
7877time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
7878@acronym{GLR} Parsers}.
18b519c0 7879@end deffn
676385e2 7880
dd8d9022
AD
7881@deffn {Symbol} $end
7882The predefined token marking the end of the token stream. It cannot be
7883used in the grammar.
7884@end deffn
7885
7886@deffn {Symbol} error
7887A token name reserved for error recovery. This token may be used in
7888grammar rules so as to allow the Bison parser to recognize an error in
7889the grammar without halting the process. In effect, a sentence
7890containing an error may be recognized as valid. On a syntax error, the
7891token @code{error} becomes the current look-ahead token. Actions
7892corresponding to @code{error} are then executed, and the look-ahead
7893token is reset to the token that originally caused the violation.
7894@xref{Error Recovery}.
18d192f0
AD
7895@end deffn
7896
18b519c0 7897@deffn {Directive} %error-verbose
2a8d363a
AD
7898Bison declaration to request verbose, specific error message strings
7899when @code{yyerror} is called.
18b519c0 7900@end deffn
2a8d363a 7901
18b519c0 7902@deffn {Directive} %file-prefix="@var{prefix}"
72d2299c 7903Bison declaration to set the prefix of the output files. @xref{Decl
d8988b2f 7904Summary}.
18b519c0 7905@end deffn
d8988b2f 7906
18b519c0 7907@deffn {Directive} %glr-parser
c827f760
PE
7908Bison declaration to produce a @acronym{GLR} parser. @xref{GLR
7909Parsers, ,Writing @acronym{GLR} Parsers}.
18b519c0 7910@end deffn
676385e2 7911
dd8d9022
AD
7912@deffn {Directive} %initial-action
7913Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
7914@end deffn
7915
18b519c0 7916@deffn {Directive} %left
bfa74976
RS
7917Bison declaration to assign left associativity to token(s).
7918@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 7919@end deffn
bfa74976 7920
feeb0eda 7921@deffn {Directive} %lex-param @{@var{argument-declaration}@}
2a8d363a
AD
7922Bison declaration to specifying an additional parameter that
7923@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
7924for Pure Parsers}.
18b519c0 7925@end deffn
2a8d363a 7926
18b519c0 7927@deffn {Directive} %merge
676385e2 7928Bison declaration to assign a merging function to a rule. If there is a
fae437e8 7929reduce/reduce conflict with a rule having the same merging function, the
676385e2 7930function is applied to the two semantic values to get a single result.
c827f760 7931@xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}.
18b519c0 7932@end deffn
676385e2 7933
18b519c0 7934@deffn {Directive} %name-prefix="@var{prefix}"
72d2299c 7935Bison declaration to rename the external symbols. @xref{Decl Summary}.
18b519c0 7936@end deffn
d8988b2f 7937
91d2c560 7938@ifset defaultprec
22fccf95
PE
7939@deffn {Directive} %no-default-prec
7940Do not assign a precedence to rules that lack an explicit @samp{%prec}
7941modifier. @xref{Contextual Precedence, ,Context-Dependent
7942Precedence}.
7943@end deffn
91d2c560 7944@end ifset
22fccf95 7945
18b519c0 7946@deffn {Directive} %no-lines
931c7513
RS
7947Bison declaration to avoid generating @code{#line} directives in the
7948parser file. @xref{Decl Summary}.
18b519c0 7949@end deffn
931c7513 7950
18b519c0 7951@deffn {Directive} %nonassoc
14ded682 7952Bison declaration to assign non-associativity to token(s).
bfa74976 7953@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 7954@end deffn
bfa74976 7955
fa4d969f 7956@deffn {Directive} %output="@var{file}"
72d2299c 7957Bison declaration to set the name of the parser file. @xref{Decl
d8988b2f 7958Summary}.
18b519c0 7959@end deffn
d8988b2f 7960
feeb0eda 7961@deffn {Directive} %parse-param @{@var{argument-declaration}@}
2a8d363a
AD
7962Bison declaration to specifying an additional parameter that
7963@code{yyparse} should accept. @xref{Parser Function,, The Parser
7964Function @code{yyparse}}.
18b519c0 7965@end deffn
2a8d363a 7966
18b519c0 7967@deffn {Directive} %prec
bfa74976
RS
7968Bison declaration to assign a precedence to a specific rule.
7969@xref{Contextual Precedence, ,Context-Dependent Precedence}.
18b519c0 7970@end deffn
bfa74976 7971
18b519c0 7972@deffn {Directive} %pure-parser
bfa74976
RS
7973Bison declaration to request a pure (reentrant) parser.
7974@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
18b519c0 7975@end deffn
bfa74976 7976
b50d2359 7977@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
7978Require version @var{version} or higher of Bison. @xref{Require Decl, ,
7979Require a Version of Bison}.
b50d2359
AD
7980@end deffn
7981
18b519c0 7982@deffn {Directive} %right
bfa74976
RS
7983Bison declaration to assign right associativity to token(s).
7984@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 7985@end deffn
bfa74976 7986
18b519c0 7987@deffn {Directive} %start
704a47c4
AD
7988Bison declaration to specify the start symbol. @xref{Start Decl, ,The
7989Start-Symbol}.
18b519c0 7990@end deffn
bfa74976 7991
18b519c0 7992@deffn {Directive} %token
bfa74976
RS
7993Bison declaration to declare token(s) without specifying precedence.
7994@xref{Token Decl, ,Token Type Names}.
18b519c0 7995@end deffn
bfa74976 7996
18b519c0 7997@deffn {Directive} %token-table
931c7513
RS
7998Bison declaration to include a token name table in the parser file.
7999@xref{Decl Summary}.
18b519c0 8000@end deffn
931c7513 8001
18b519c0 8002@deffn {Directive} %type
704a47c4
AD
8003Bison declaration to declare nonterminals. @xref{Type Decl,
8004,Nonterminal Symbols}.
18b519c0 8005@end deffn
bfa74976 8006
dd8d9022
AD
8007@deffn {Symbol} $undefined
8008The predefined token onto which all undefined values returned by
8009@code{yylex} are mapped. It cannot be used in the grammar, rather, use
8010@code{error}.
8011@end deffn
8012
18b519c0 8013@deffn {Directive} %union
bfa74976
RS
8014Bison declaration to specify several possible data types for semantic
8015values. @xref{Union Decl, ,The Collection of Value Types}.
18b519c0 8016@end deffn
bfa74976 8017
dd8d9022
AD
8018@deffn {Macro} YYABORT
8019Macro to pretend that an unrecoverable syntax error has occurred, by
8020making @code{yyparse} return 1 immediately. The error reporting
8021function @code{yyerror} is not called. @xref{Parser Function, ,The
8022Parser Function @code{yyparse}}.
8023@end deffn
3ded9a63 8024
dd8d9022
AD
8025@deffn {Macro} YYACCEPT
8026Macro to pretend that a complete utterance of the language has been
8027read, by making @code{yyparse} return 0 immediately.
8028@xref{Parser Function, ,The Parser Function @code{yyparse}}.
8029@end deffn
bfa74976 8030
dd8d9022
AD
8031@deffn {Macro} YYBACKUP
8032Macro to discard a value from the parser stack and fake a look-ahead
8033token. @xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 8034@end deffn
bfa74976 8035
dd8d9022
AD
8036@deffn {Variable} yychar
8037External integer variable that contains the integer value of the current
8038look-ahead token. (In a pure parser, it is a local variable within
8039@code{yyparse}.) Error-recovery rule actions may examine this variable.
8040@xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 8041@end deffn
bfa74976 8042
dd8d9022
AD
8043@deffn {Variable} yyclearin
8044Macro used in error-recovery rule actions. It clears the previous
8045look-ahead token. @xref{Error Recovery}.
18b519c0 8046@end deffn
bfa74976 8047
dd8d9022
AD
8048@deffn {Macro} YYDEBUG
8049Macro to define to equip the parser with tracing code. @xref{Tracing,
8050,Tracing Your Parser}.
18b519c0 8051@end deffn
bfa74976 8052
dd8d9022
AD
8053@deffn {Variable} yydebug
8054External integer variable set to zero by default. If @code{yydebug}
8055is given a nonzero value, the parser will output information on input
8056symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
18b519c0 8057@end deffn
bfa74976 8058
dd8d9022
AD
8059@deffn {Macro} yyerrok
8060Macro to cause parser to recover immediately to its normal mode
8061after a syntax error. @xref{Error Recovery}.
8062@end deffn
8063
8064@deffn {Macro} YYERROR
8065Macro to pretend that a syntax error has just been detected: call
8066@code{yyerror} and then perform normal error recovery if possible
8067(@pxref{Error Recovery}), or (if recovery is impossible) make
8068@code{yyparse} return 1. @xref{Error Recovery}.
8069@end deffn
8070
8071@deffn {Function} yyerror
8072User-supplied function to be called by @code{yyparse} on error.
8073@xref{Error Reporting, ,The Error
8074Reporting Function @code{yyerror}}.
8075@end deffn
8076
8077@deffn {Macro} YYERROR_VERBOSE
8078An obsolete macro that you define with @code{#define} in the prologue
8079to request verbose, specific error message strings
8080when @code{yyerror} is called. It doesn't matter what definition you
8081use for @code{YYERROR_VERBOSE}, just whether you define it. Using
8082@code{%error-verbose} is preferred.
8083@end deffn
8084
8085@deffn {Macro} YYINITDEPTH
8086Macro for specifying the initial size of the parser stack.
1a059451 8087@xref{Memory Management}.
dd8d9022
AD
8088@end deffn
8089
8090@deffn {Function} yylex
8091User-supplied lexical analyzer function, called with no arguments to get
8092the next token. @xref{Lexical, ,The Lexical Analyzer Function
8093@code{yylex}}.
8094@end deffn
8095
8096@deffn {Macro} YYLEX_PARAM
8097An obsolete macro for specifying an extra argument (or list of extra
8098arguments) for @code{yyparse} to pass to @code{yylex}. he use of this
8099macro is deprecated, and is supported only for Yacc like parsers.
8100@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
8101@end deffn
8102
8103@deffn {Variable} yylloc
8104External variable in which @code{yylex} should place the line and column
8105numbers associated with a token. (In a pure parser, it is a local
8106variable within @code{yyparse}, and its address is passed to
8107@code{yylex}.) You can ignore this variable if you don't use the
8108@samp{@@} feature in the grammar actions. @xref{Token Locations,
8109,Textual Locations of Tokens}.
8110@end deffn
8111
8112@deffn {Type} YYLTYPE
8113Data type of @code{yylloc}; by default, a structure with four
8114members. @xref{Location Type, , Data Types of Locations}.
8115@end deffn
8116
8117@deffn {Variable} yylval
8118External variable in which @code{yylex} should place the semantic
8119value associated with a token. (In a pure parser, it is a local
8120variable within @code{yyparse}, and its address is passed to
8121@code{yylex}.) @xref{Token Values, ,Semantic Values of Tokens}.
8122@end deffn
8123
8124@deffn {Macro} YYMAXDEPTH
1a059451
PE
8125Macro for specifying the maximum size of the parser stack. @xref{Memory
8126Management}.
dd8d9022
AD
8127@end deffn
8128
8129@deffn {Variable} yynerrs
8a2800e7 8130Global variable which Bison increments each time it reports a syntax error.
dd8d9022
AD
8131(In a pure parser, it is a local variable within @code{yyparse}.)
8132@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
8133@end deffn
8134
8135@deffn {Function} yyparse
8136The parser function produced by Bison; call this function to start
8137parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
8138@end deffn
8139
8140@deffn {Macro} YYPARSE_PARAM
8141An obsolete macro for specifying the name of a parameter that
8142@code{yyparse} should accept. The use of this macro is deprecated, and
8143is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
8144Conventions for Pure Parsers}.
8145@end deffn
8146
8147@deffn {Macro} YYRECOVERING
8148Macro whose value indicates whether the parser is recovering from a
8149syntax error. @xref{Action Features, ,Special Features for Use in Actions}.
8150@end deffn
8151
8152@deffn {Macro} YYSTACK_USE_ALLOCA
d7e14fc0
PE
8153Macro used to control the use of @code{alloca} when the C
8154@acronym{LALR}(1) parser needs to extend its stacks. If defined to 0,
8155the parser will use @code{malloc} to extend its stacks. If defined to
81561, the parser will use @code{alloca}. Values other than 0 and 1 are
8157reserved for future Bison extensions. If not defined,
8158@code{YYSTACK_USE_ALLOCA} defaults to 0.
8159
55289366 8160In the all-too-common case where your code may run on a host with a
d7e14fc0
PE
8161limited stack and with unreliable stack-overflow checking, you should
8162set @code{YYMAXDEPTH} to a value that cannot possibly result in
8163unchecked stack overflow on any of your target hosts when
8164@code{alloca} is called. You can inspect the code that Bison
8165generates in order to determine the proper numeric values. This will
8166require some expertise in low-level implementation details.
dd8d9022
AD
8167@end deffn
8168
8169@deffn {Type} YYSTYPE
8170Data type of semantic values; @code{int} by default.
8171@xref{Value Type, ,Data Types of Semantic Values}.
18b519c0 8172@end deffn
bfa74976 8173
342b8b6e 8174@node Glossary
bfa74976
RS
8175@appendix Glossary
8176@cindex glossary
8177
8178@table @asis
c827f760
PE
8179@item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'')
8180Formal method of specifying context-free grammars originally proposed
8181by John Backus, and slightly improved by Peter Naur in his 1960-01-02
8182committee document contributing to what became the Algol 60 report.
8183@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
8184
8185@item Context-free grammars
8186Grammars specified as rules that can be applied regardless of context.
8187Thus, if there is a rule which says that an integer can be used as an
8188expression, integers are allowed @emph{anywhere} an expression is
89cab50d
AD
8189permitted. @xref{Language and Grammar, ,Languages and Context-Free
8190Grammars}.
bfa74976
RS
8191
8192@item Dynamic allocation
8193Allocation of memory that occurs during execution, rather than at
8194compile time or on entry to a function.
8195
8196@item Empty string
8197Analogous to the empty set in set theory, the empty string is a
8198character string of length zero.
8199
8200@item Finite-state stack machine
8201A ``machine'' that has discrete states in which it is said to exist at
8202each instant in time. As input to the machine is processed, the
8203machine moves from state to state as specified by the logic of the
8204machine. In the case of the parser, the input is the language being
8205parsed, and the states correspond to various stages in the grammar
c827f760 8206rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976 8207
c827f760 8208@item Generalized @acronym{LR} (@acronym{GLR})
676385e2 8209A parsing algorithm that can handle all context-free grammars, including those
c827f760
PE
8210that are not @acronym{LALR}(1). It resolves situations that Bison's
8211usual @acronym{LALR}(1)
676385e2
PH
8212algorithm cannot by effectively splitting off multiple parsers, trying all
8213possible parsers, and discarding those that fail in the light of additional
c827f760
PE
8214right context. @xref{Generalized LR Parsing, ,Generalized
8215@acronym{LR} Parsing}.
676385e2 8216
bfa74976
RS
8217@item Grouping
8218A language construct that is (in general) grammatically divisible;
c827f760 8219for example, `expression' or `declaration' in C@.
bfa74976
RS
8220@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
8221
8222@item Infix operator
8223An arithmetic operator that is placed between the operands on which it
8224performs some operation.
8225
8226@item Input stream
8227A continuous flow of data between devices or programs.
8228
8229@item Language construct
8230One of the typical usage schemas of the language. For example, one of
8231the constructs of the C language is the @code{if} statement.
8232@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
8233
8234@item Left associativity
8235Operators having left associativity are analyzed from left to right:
8236@samp{a+b+c} first computes @samp{a+b} and then combines with
8237@samp{c}. @xref{Precedence, ,Operator Precedence}.
8238
8239@item Left recursion
89cab50d
AD
8240A rule whose result symbol is also its first component symbol; for
8241example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
8242Rules}.
bfa74976
RS
8243
8244@item Left-to-right parsing
8245Parsing a sentence of a language by analyzing it token by token from
c827f760 8246left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
8247
8248@item Lexical analyzer (scanner)
8249A function that reads an input stream and returns tokens one by one.
8250@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
8251
8252@item Lexical tie-in
8253A flag, set by actions in the grammar rules, which alters the way
8254tokens are parsed. @xref{Lexical Tie-ins}.
8255
931c7513 8256@item Literal string token
14ded682 8257A token which consists of two or more fixed characters. @xref{Symbols}.
931c7513 8258
bfa74976 8259@item Look-ahead token
89cab50d
AD
8260A token already read but not yet shifted. @xref{Look-Ahead, ,Look-Ahead
8261Tokens}.
bfa74976 8262
c827f760 8263@item @acronym{LALR}(1)
bfa74976 8264The class of context-free grammars that Bison (like most other parser
c827f760
PE
8265generators) can handle; a subset of @acronym{LR}(1). @xref{Mystery
8266Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
bfa74976 8267
c827f760 8268@item @acronym{LR}(1)
bfa74976
RS
8269The class of context-free grammars in which at most one token of
8270look-ahead is needed to disambiguate the parsing of any piece of input.
8271
8272@item Nonterminal symbol
8273A grammar symbol standing for a grammatical construct that can
8274be expressed through rules in terms of smaller constructs; in other
8275words, a construct that is not a token. @xref{Symbols}.
8276
bfa74976
RS
8277@item Parser
8278A function that recognizes valid sentences of a language by analyzing
8279the syntax structure of a set of tokens passed to it from a lexical
8280analyzer.
8281
8282@item Postfix operator
8283An arithmetic operator that is placed after the operands upon which it
8284performs some operation.
8285
8286@item Reduction
8287Replacing a string of nonterminals and/or terminals with a single
89cab50d 8288nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
c827f760 8289Parser Algorithm}.
bfa74976
RS
8290
8291@item Reentrant
8292A reentrant subprogram is a subprogram which can be in invoked any
8293number of times in parallel, without interference between the various
8294invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
8295
8296@item Reverse polish notation
8297A language in which all operators are postfix operators.
8298
8299@item Right recursion
89cab50d
AD
8300A rule whose result symbol is also its last component symbol; for
8301example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
8302Rules}.
bfa74976
RS
8303
8304@item Semantics
8305In computer languages, the semantics are specified by the actions
8306taken for each instance of the language, i.e., the meaning of
8307each statement. @xref{Semantics, ,Defining Language Semantics}.
8308
8309@item Shift
8310A parser is said to shift when it makes the choice of analyzing
8311further input from the stream rather than reducing immediately some
c827f760 8312already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
8313
8314@item Single-character literal
8315A single character that is recognized and interpreted as is.
8316@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
8317
8318@item Start symbol
8319The nonterminal symbol that stands for a complete valid utterance in
8320the language being parsed. The start symbol is usually listed as the
13863333 8321first nonterminal symbol in a language specification.
bfa74976
RS
8322@xref{Start Decl, ,The Start-Symbol}.
8323
8324@item Symbol table
8325A data structure where symbol names and associated data are stored
8326during parsing to allow for recognition and use of existing
8327information in repeated uses of a symbol. @xref{Multi-function Calc}.
8328
6e649e65
PE
8329@item Syntax error
8330An error encountered during parsing of an input stream due to invalid
8331syntax. @xref{Error Recovery}.
8332
bfa74976
RS
8333@item Token
8334A basic, grammatically indivisible unit of a language. The symbol
8335that describes a token in the grammar is a terminal symbol.
8336The input of the Bison parser is a stream of tokens which comes from
8337the lexical analyzer. @xref{Symbols}.
8338
8339@item Terminal symbol
89cab50d
AD
8340A grammar symbol that has no rules in the grammar and therefore is
8341grammatically indivisible. The piece of text it represents is a token.
8342@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
8343@end table
8344
342b8b6e 8345@node Copying This Manual
f2b5126e 8346@appendix Copying This Manual
f9a8293a 8347
f2b5126e
PB
8348@menu
8349* GNU Free Documentation License:: License for copying this manual.
8350@end menu
f9a8293a 8351
f2b5126e
PB
8352@include fdl.texi
8353
342b8b6e 8354@node Index
bfa74976
RS
8355@unnumbered Index
8356
8357@printindex cp
8358
bfa74976 8359@bye
a06ea4aa
AD
8360
8361@c LocalWords: texinfo setfilename settitle setchapternewpage finalout
8362@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex
8363@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry
8364@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa
8365@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc
8366@c LocalWords: rpcalc Lexer Gen Comp Expr ltcalc mfcalc Decl Symtab yylex
8367@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref
8368@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex
8369@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge
8370@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG
8371@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit
8372@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok
8373@c LocalWords: longjmp fprintf stderr preg yylloc YYLTYPE cos ln
8374@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym
8375@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof
8376@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum
8377@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype
8378@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless typefull yynerrs
8379@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES
8380@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param
8381@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP
8382@c LocalWords: YYEMPTY YYRECOVERING yyclearin GE def UMINUS maybeword
8383@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH
8384@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm VCG notype
8385@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args
8386@c LocalWords: YYPRINTF infile ypp yxx outfile itemx vcg tex leaderfill
8387@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll
8388@c LocalWords: yyrestart nbar yytext fst snd osplit ntwo strdup AST
8389@c LocalWords: YYSTACK DVI fdl printindex