Accept a token number in a %left, %right, or %nonassoc for POSIX
[bison.git] / doc / bison.texinfo
CommitLineData
bfa74976
RS
1\input texinfo @c -*-texinfo-*-
2@comment %**start of header
3@setfilename bison.info
df1af54c
JT
4@include version.texi
5@settitle Bison @value{VERSION}
bfa74976
RS
6@setchapternewpage odd
7
5378c3e7 8@finalout
5378c3e7 9
13863333 10@c SMALL BOOK version
bfa74976 11@c This edition has been formatted so that you can format and print it in
13863333 12@c the smallbook format.
bfa74976
RS
13@c @smallbook
14
91d2c560
PE
15@c Set following if you want to document %default-prec and %no-default-prec.
16@c This feature is experimental and may change in future Bison versions.
17@c @set defaultprec
18
8c5b881d 19@ifnotinfo
bfa74976
RS
20@syncodeindex fn cp
21@syncodeindex vr cp
22@syncodeindex tp cp
8c5b881d 23@end ifnotinfo
bfa74976
RS
24@ifinfo
25@synindex fn cp
26@synindex vr cp
27@synindex tp cp
28@end ifinfo
29@comment %**end of header
30
fae437e8 31@copying
bd773d73 32
c827f760
PE
33This manual is for @acronym{GNU} Bison (version @value{VERSION},
34@value{UPDATED}), the @acronym{GNU} parser generator.
fae437e8 35
a06ea4aa 36Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
ab7f29f8
JD
371999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008 Free Software
38Foundation, Inc.
fae437e8
AD
39
40@quotation
41Permission is granted to copy, distribute and/or modify this document
c827f760 42under the terms of the @acronym{GNU} Free Documentation License,
592fde95 43Version 1.2 or any later version published by the Free Software
c827f760
PE
44Foundation; with no Invariant Sections, with the Front-Cover texts
45being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in
46(a) below. A copy of the license is included in the section entitled
47``@acronym{GNU} Free Documentation License.''
48
389c8cfd
PE
49(a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
50modify this @acronym{GNU} manual. Buying copies from the @acronym{FSF}
51supports it in developing @acronym{GNU} and promoting software
52freedom.''
fae437e8
AD
53@end quotation
54@end copying
55
e62f1a89 56@dircategory Software development
fae437e8 57@direntry
c827f760 58* bison: (bison). @acronym{GNU} parser generator (Yacc replacement).
fae437e8 59@end direntry
bfa74976 60
bfa74976
RS
61@titlepage
62@title Bison
c827f760 63@subtitle The Yacc-compatible Parser Generator
df1af54c 64@subtitle @value{UPDATED}, Bison Version @value{VERSION}
bfa74976
RS
65
66@author by Charles Donnelly and Richard Stallman
67
68@page
69@vskip 0pt plus 1filll
fae437e8 70@insertcopying
bfa74976
RS
71@sp 2
72Published by the Free Software Foundation @*
0fb669f9
PE
7351 Franklin Street, Fifth Floor @*
74Boston, MA 02110-1301 USA @*
9ecbd125 75Printed copies are available from the Free Software Foundation.@*
c827f760 76@acronym{ISBN} 1-882114-44-2
bfa74976
RS
77@sp 2
78Cover art by Etienne Suvasa.
79@end titlepage
d5796688
JT
80
81@contents
bfa74976 82
342b8b6e
AD
83@ifnottex
84@node Top
85@top Bison
fae437e8 86@insertcopying
342b8b6e 87@end ifnottex
bfa74976
RS
88
89@menu
13863333
AD
90* Introduction::
91* Conditions::
c827f760 92* Copying:: The @acronym{GNU} General Public License says
bfa74976
RS
93 how you can copy and share Bison
94
95Tutorial sections:
96* Concepts:: Basic concepts for understanding Bison.
97* Examples:: Three simple explained examples of using Bison.
98
99Reference sections:
100* Grammar File:: Writing Bison declarations and rules.
101* Interface:: C-language interface to the parser function @code{yyparse}.
102* Algorithm:: How the Bison parser works at run-time.
103* Error Recovery:: Writing rules for error recovery.
104* Context Dependency:: What to do if your language syntax is too
105 messy for Bison to handle straightforwardly.
ec3bc396 106* Debugging:: Understanding or debugging Bison parsers.
bfa74976 107* Invocation:: How to run Bison (to produce the parser source file).
8405b70c 108* Other Languages:: Creating C++ and Java parsers.
12545799 109* FAQ:: Frequently Asked Questions
bfa74976
RS
110* Table of Symbols:: All the keywords of the Bison language are explained.
111* Glossary:: Basic concepts are explained.
f2b5126e 112* Copying This Manual:: License for copying this manual.
bfa74976
RS
113* Index:: Cross-references to the text.
114
93dd49ab
PE
115@detailmenu
116 --- The Detailed Node Listing ---
bfa74976
RS
117
118The Concepts of Bison
119
120* Language and Grammar:: Languages and context-free grammars,
121 as mathematical ideas.
122* Grammar in Bison:: How we represent grammars for Bison's sake.
123* Semantic Values:: Each token or syntactic grouping can have
124 a semantic value (the value of an integer,
125 the name of an identifier, etc.).
126* Semantic Actions:: Each rule can have an action containing C code.
99a9344e 127* GLR Parsers:: Writing parsers for general context-free languages.
93dd49ab 128* Locations Overview:: Tracking Locations.
bfa74976
RS
129* Bison Parser:: What are Bison's input and output,
130 how is the output used?
131* Stages:: Stages in writing and running Bison grammars.
132* Grammar Layout:: Overall structure of a Bison grammar file.
133
fa7e68c3
PE
134Writing @acronym{GLR} Parsers
135
32c29292
JD
136* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
137* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
138* GLR Semantic Actions:: Deferred semantic actions have special concerns.
139* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
fa7e68c3 140
bfa74976
RS
141Examples
142
143* RPN Calc:: Reverse polish notation calculator;
144 a first example with no operator precedence.
145* Infix Calc:: Infix (algebraic) notation calculator.
146 Operator precedence is introduced.
147* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 148* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
93dd49ab
PE
149* Multi-function Calc:: Calculator with memory and trig functions.
150 It uses multiple data-types for semantic values.
bfa74976
RS
151* Exercises:: Ideas for improving the multi-function calculator.
152
153Reverse Polish Notation Calculator
154
75f5aaea 155* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
bfa74976
RS
156* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
157* Lexer: Rpcalc Lexer. The lexical analyzer.
158* Main: Rpcalc Main. The controlling function.
159* Error: Rpcalc Error. The error reporting function.
160* Gen: Rpcalc Gen. Running Bison on the grammar file.
161* Comp: Rpcalc Compile. Run the C compiler on the output code.
162
163Grammar Rules for @code{rpcalc}
164
13863333
AD
165* Rpcalc Input::
166* Rpcalc Line::
167* Rpcalc Expr::
bfa74976 168
342b8b6e
AD
169Location Tracking Calculator: @code{ltcalc}
170
171* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
172* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
173* Lexer: Ltcalc Lexer. The lexical analyzer.
174
bfa74976
RS
175Multi-Function Calculator: @code{mfcalc}
176
177* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
178* Rules: Mfcalc Rules. Grammar rules for the calculator.
179* Symtab: Mfcalc Symtab. Symbol table management subroutines.
180
181Bison Grammar Files
182
183* Grammar Outline:: Overall layout of the grammar file.
184* Symbols:: Terminal and nonterminal symbols.
185* Rules:: How to write grammar rules.
186* Recursion:: Writing recursive rules.
187* Semantics:: Semantic values and actions.
93dd49ab 188* Locations:: Locations and actions.
bfa74976
RS
189* Declarations:: All kinds of Bison declarations are described here.
190* Multiple Parsers:: Putting more than one Bison parser in one program.
191
192Outline of a Bison Grammar
193
93dd49ab 194* Prologue:: Syntax and usage of the prologue.
2cbe6b7f 195* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
bfa74976
RS
196* Bison Declarations:: Syntax and usage of the Bison declarations section.
197* Grammar Rules:: Syntax and usage of the grammar rules section.
93dd49ab 198* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
199
200Defining Language Semantics
201
202* Value Type:: Specifying one data type for all semantic values.
203* Multiple Types:: Specifying several alternative data types.
204* Actions:: An action is the semantic definition of a grammar rule.
205* Action Types:: Specifying data types for actions to operate on.
206* Mid-Rule Actions:: Most actions go at the end of a rule.
207 This says when, why and how to use the exceptional
208 action in the middle of a rule.
209
93dd49ab
PE
210Tracking Locations
211
212* Location Type:: Specifying a data type for locations.
213* Actions and Locations:: Using locations in actions.
214* Location Default Action:: Defining a general way to compute locations.
215
bfa74976
RS
216Bison Declarations
217
b50d2359 218* Require Decl:: Requiring a Bison version.
bfa74976
RS
219* Token Decl:: Declaring terminal symbols.
220* Precedence Decl:: Declaring terminals with precedence and associativity.
221* Union Decl:: Declaring the set of all semantic value types.
222* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 223* Initial Action Decl:: Code run before parsing starts.
72f889cc 224* Destructor Decl:: Declaring how symbols are freed.
d6328241 225* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
226* Start Decl:: Specifying the start symbol.
227* Pure Decl:: Requesting a reentrant parser.
9987d1b3 228* Push Decl:: Requesting a push parser.
bfa74976
RS
229* Decl Summary:: Table of all Bison declarations.
230
231Parser C-Language Interface
232
233* Parser Function:: How to call @code{yyparse} and what it returns.
13863333 234* Lexical:: You must supply a function @code{yylex}
bfa74976
RS
235 which reads tokens.
236* Error Reporting:: You must supply a function @code{yyerror}.
237* Action Features:: Special features for use in actions.
f7ab6a50
PE
238* Internationalization:: How to let the parser speak in the user's
239 native language.
bfa74976
RS
240
241The Lexical Analyzer Function @code{yylex}
242
243* Calling Convention:: How @code{yyparse} calls @code{yylex}.
244* Token Values:: How @code{yylex} must return the semantic value
245 of the token it has read.
95923bd6 246* Token Locations:: How @code{yylex} must return the text location
bfa74976 247 (line number, etc.) of the token, if the
93dd49ab 248 actions want that.
bfa74976
RS
249* Pure Calling:: How the calling convention differs
250 in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
251
13863333 252The Bison Parser Algorithm
bfa74976 253
742e4900 254* Lookahead:: Parser looks one token ahead when deciding what to do.
bfa74976
RS
255* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
256* Precedence:: Operator precedence works by resolving conflicts.
257* Contextual Precedence:: When an operator's precedence depends on context.
258* Parser States:: The parser is a finite-state-machine with stack.
259* Reduce/Reduce:: When two rules are applicable in the same situation.
260* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 261* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 262* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
263
264Operator Precedence
265
266* Why Precedence:: An example showing why precedence is needed.
267* Using Precedence:: How to specify precedence in Bison grammars.
268* Precedence Examples:: How these features are used in the previous example.
269* How Precedence:: How they work.
270
271Handling Context Dependencies
272
273* Semantic Tokens:: Token parsing can depend on the semantic context.
274* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
275* Tie-in Recovery:: Lexical tie-ins have implications for how
276 error recovery rules must be written.
277
93dd49ab 278Debugging Your Parser
ec3bc396
AD
279
280* Understanding:: Understanding the structure of your parser.
281* Tracing:: Tracing the execution of your parser.
282
bfa74976
RS
283Invoking Bison
284
13863333 285* Bison Options:: All the options described in detail,
c827f760 286 in alphabetical order by short options.
bfa74976 287* Option Cross Key:: Alphabetical list of long options.
93dd49ab 288* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
f2b5126e 289
8405b70c 290Parsers Written In Other Languages
12545799
AD
291
292* C++ Parsers:: The interface to generate C++ parser classes
8405b70c 293* Java Parsers:: The interface to generate Java parser classes
12545799
AD
294
295C++ Parsers
296
297* C++ Bison Interface:: Asking for C++ parser generation
298* C++ Semantic Values:: %union vs. C++
299* C++ Location Values:: The position and location classes
300* C++ Parser Interface:: Instantiating and running the parser
301* C++ Scanner Interface:: Exchanges between yylex and parse
8405b70c 302* A Complete C++ Example:: Demonstrating their use
12545799
AD
303
304A Complete C++ Example
305
306* Calc++ --- C++ Calculator:: The specifications
307* Calc++ Parsing Driver:: An active parsing context
308* Calc++ Parser:: A parser class
309* Calc++ Scanner:: A pure C++ Flex scanner
310* Calc++ Top Level:: Conducting the band
311
8405b70c
PB
312Java Parsers
313
314* Java Bison Interface:: Asking for Java parser generation
315* Java Semantic Values:: %type and %token vs. Java
316* Java Location Values:: The position and location classes
317* Java Parser Interface:: Instantiating and running the parser
318* Java Scanner Interface:: Java scanners, and pure parsers
319* Java Differences:: Differences between C/C++ and Java Grammars
320
d1a1114f
AD
321Frequently Asked Questions
322
1a059451 323* Memory Exhausted:: Breaking the Stack Limits
e64fec0a 324* How Can I Reset the Parser:: @code{yyparse} Keeps some State
fef4cb51 325* Strings are Destroyed:: @code{yylval} Loses Track of Strings
2fa09258 326* Implementing Gotos/Loops:: Control Flow in the Calculator
ed2e6384 327* Multiple start-symbols:: Factoring closely related grammars
55ba27be
AD
328* Secure? Conform?:: Is Bison @acronym{POSIX} safe?
329* I can't build Bison:: Troubleshooting
330* Where can I find help?:: Troubleshouting
331* Bug Reports:: Troublereporting
332* Other Languages:: Parsers in Java and others
333* Beta Testing:: Experimenting development versions
334* Mailing Lists:: Meeting other Bison users
d1a1114f 335
f2b5126e
PB
336Copying This Manual
337
f16b0819 338* Copying This Manual:: License for copying this manual.
f2b5126e 339
342b8b6e 340@end detailmenu
bfa74976
RS
341@end menu
342
342b8b6e 343@node Introduction
bfa74976
RS
344@unnumbered Introduction
345@cindex introduction
346
6077da58
PE
347@dfn{Bison} is a general-purpose parser generator that converts an
348annotated context-free grammar into an @acronym{LALR}(1) or
349@acronym{GLR} parser for that grammar. Once you are proficient with
1e137b71 350Bison, you can use it to develop a wide range of language parsers, from those
bfa74976
RS
351used in simple desk calculators to complex programming languages.
352
353Bison is upward compatible with Yacc: all properly-written Yacc grammars
354ought to work with Bison with no change. Anyone familiar with Yacc
355should be able to use Bison with little trouble. You need to be fluent in
1e137b71 356C or C++ programming in order to use Bison or to understand this manual.
bfa74976
RS
357
358We begin with tutorial chapters that explain the basic concepts of using
359Bison and show three explained examples, each building on the last. If you
360don't know Bison or Yacc, start by reading these chapters. Reference
361chapters follow which describe specific aspects of Bison in detail.
362
931c7513
RS
363Bison was written primarily by Robert Corbett; Richard Stallman made it
364Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
14ded682 365multi-character string literals and other features.
931c7513 366
df1af54c 367This edition corresponds to version @value{VERSION} of Bison.
bfa74976 368
342b8b6e 369@node Conditions
bfa74976
RS
370@unnumbered Conditions for Using Bison
371
193d7c70
PE
372The distribution terms for Bison-generated parsers permit using the
373parsers in nonfree programs. Before Bison version 2.2, these extra
374permissions applied only when Bison was generating @acronym{LALR}(1)
375parsers in C@. And before Bison version 1.24, Bison-generated
262aa8dd 376parsers could be used only in programs that were free software.
a31239f1 377
c827f760
PE
378The other @acronym{GNU} programming tools, such as the @acronym{GNU} C
379compiler, have never
9ecbd125 380had such a requirement. They could always be used for nonfree
a31239f1
RS
381software. The reason Bison was different was not due to a special
382policy decision; it resulted from applying the usual General Public
383License to all of the Bison source code.
384
385The output of the Bison utility---the Bison parser file---contains a
386verbatim copy of a sizable piece of Bison, which is the code for the
193d7c70
PE
387parser's implementation. (The actions from your grammar are inserted
388into this implementation at one point, but most of the rest of the
389implementation is not changed.) When we applied the @acronym{GPL}
390terms to the skeleton code for the parser's implementation,
a31239f1
RS
391the effect was to restrict the use of Bison output to free software.
392
393We didn't change the terms because of sympathy for people who want to
394make software proprietary. @strong{Software should be free.} But we
395concluded that limiting Bison's use to free software was doing little to
396encourage people to make other software free. So we decided to make the
397practical conditions for using Bison match the practical conditions for
c827f760 398using the other @acronym{GNU} tools.
bfa74976 399
193d7c70
PE
400This exception applies when Bison is generating code for a parser.
401You can tell whether the exception applies to a Bison output file by
402inspecting the file for text beginning with ``As a special
403exception@dots{}''. The text spells out the exact terms of the
404exception.
262aa8dd 405
f16b0819
PE
406@node Copying
407@unnumbered GNU GENERAL PUBLIC LICENSE
408@include gpl-3.0.texi
bfa74976 409
342b8b6e 410@node Concepts
bfa74976
RS
411@chapter The Concepts of Bison
412
413This chapter introduces many of the basic concepts without which the
414details of Bison will not make sense. If you do not already know how to
415use Bison or Yacc, we suggest you start by reading this chapter carefully.
416
417@menu
418* Language and Grammar:: Languages and context-free grammars,
419 as mathematical ideas.
420* Grammar in Bison:: How we represent grammars for Bison's sake.
421* Semantic Values:: Each token or syntactic grouping can have
422 a semantic value (the value of an integer,
423 the name of an identifier, etc.).
424* Semantic Actions:: Each rule can have an action containing C code.
99a9344e 425* GLR Parsers:: Writing parsers for general context-free languages.
847bf1f5 426* Locations Overview:: Tracking Locations.
bfa74976
RS
427* Bison Parser:: What are Bison's input and output,
428 how is the output used?
429* Stages:: Stages in writing and running Bison grammars.
430* Grammar Layout:: Overall structure of a Bison grammar file.
431@end menu
432
342b8b6e 433@node Language and Grammar
bfa74976
RS
434@section Languages and Context-Free Grammars
435
bfa74976
RS
436@cindex context-free grammar
437@cindex grammar, context-free
438In order for Bison to parse a language, it must be described by a
439@dfn{context-free grammar}. This means that you specify one or more
440@dfn{syntactic groupings} and give rules for constructing them from their
441parts. For example, in the C language, one kind of grouping is called an
442`expression'. One rule for making an expression might be, ``An expression
443can be made of a minus sign and another expression''. Another would be,
444``An expression can be an integer''. As you can see, rules are often
445recursive, but there must be at least one rule which leads out of the
446recursion.
447
c827f760 448@cindex @acronym{BNF}
bfa74976
RS
449@cindex Backus-Naur form
450The most common formal system for presenting such rules for humans to read
c827f760
PE
451is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in
452order to specify the language Algol 60. Any grammar expressed in
453@acronym{BNF} is a context-free grammar. The input to Bison is
454essentially machine-readable @acronym{BNF}.
bfa74976 455
c827f760
PE
456@cindex @acronym{LALR}(1) grammars
457@cindex @acronym{LR}(1) grammars
676385e2
PH
458There are various important subclasses of context-free grammar. Although it
459can handle almost all context-free grammars, Bison is optimized for what
c827f760 460are called @acronym{LALR}(1) grammars.
676385e2 461In brief, in these grammars, it must be possible to
bfa74976 462tell how to parse any portion of an input string with just a single
742e4900 463token of lookahead. Strictly speaking, that is a description of an
c827f760
PE
464@acronym{LR}(1) grammar, and @acronym{LALR}(1) involves additional
465restrictions that are
bfa74976 466hard to explain simply; but it is rare in actual practice to find an
c827f760
PE
467@acronym{LR}(1) grammar that fails to be @acronym{LALR}(1).
468@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
469more information on this.
bfa74976 470
c827f760
PE
471@cindex @acronym{GLR} parsing
472@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2 473@cindex ambiguous grammars
9d9b8b70 474@cindex nondeterministic parsing
9501dc6e
AD
475
476Parsers for @acronym{LALR}(1) grammars are @dfn{deterministic}, meaning
477roughly that the next grammar rule to apply at any point in the input is
478uniquely determined by the preceding input and a fixed, finite portion
742e4900 479(called a @dfn{lookahead}) of the remaining input. A context-free
9501dc6e 480grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
e4f85c39 481apply the grammar rules to get the same inputs. Even unambiguous
9d9b8b70 482grammars can be @dfn{nondeterministic}, meaning that no fixed
742e4900 483lookahead always suffices to determine the next grammar rule to apply.
9501dc6e
AD
484With the proper declarations, Bison is also able to parse these more
485general context-free grammars, using a technique known as @acronym{GLR}
486parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers
487are able to handle any context-free grammar for which the number of
488possible parses of any given string is finite.
676385e2 489
bfa74976
RS
490@cindex symbols (abstract)
491@cindex token
492@cindex syntactic grouping
493@cindex grouping, syntactic
9501dc6e
AD
494In the formal grammatical rules for a language, each kind of syntactic
495unit or grouping is named by a @dfn{symbol}. Those which are built by
496grouping smaller constructs according to grammatical rules are called
bfa74976
RS
497@dfn{nonterminal symbols}; those which can't be subdivided are called
498@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
499corresponding to a single terminal symbol a @dfn{token}, and a piece
e0c471a9 500corresponding to a single nonterminal symbol a @dfn{grouping}.
bfa74976
RS
501
502We can use the C language as an example of what symbols, terminal and
9501dc6e
AD
503nonterminal, mean. The tokens of C are identifiers, constants (numeric
504and string), and the various keywords, arithmetic operators and
505punctuation marks. So the terminal symbols of a grammar for C include
506`identifier', `number', `string', plus one symbol for each keyword,
507operator or punctuation mark: `if', `return', `const', `static', `int',
508`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
509(These tokens can be subdivided into characters, but that is a matter of
bfa74976
RS
510lexicography, not grammar.)
511
512Here is a simple C function subdivided into tokens:
513
9edcd895
AD
514@ifinfo
515@example
516int /* @r{keyword `int'} */
14d4662b 517square (int x) /* @r{identifier, open-paren, keyword `int',}
9edcd895
AD
518 @r{identifier, close-paren} */
519@{ /* @r{open-brace} */
aa08666d
AD
520 return x * x; /* @r{keyword `return', identifier, asterisk,}
521 @r{identifier, semicolon} */
9edcd895
AD
522@} /* @r{close-brace} */
523@end example
524@end ifinfo
525@ifnotinfo
bfa74976
RS
526@example
527int /* @r{keyword `int'} */
14d4662b 528square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
bfa74976 529@{ /* @r{open-brace} */
9edcd895 530 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
bfa74976
RS
531@} /* @r{close-brace} */
532@end example
9edcd895 533@end ifnotinfo
bfa74976
RS
534
535The syntactic groupings of C include the expression, the statement, the
536declaration, and the function definition. These are represented in the
537grammar of C by nonterminal symbols `expression', `statement',
538`declaration' and `function definition'. The full grammar uses dozens of
539additional language constructs, each with its own nonterminal symbol, in
540order to express the meanings of these four. The example above is a
541function definition; it contains one declaration, and one statement. In
542the statement, each @samp{x} is an expression and so is @samp{x * x}.
543
544Each nonterminal symbol must have grammatical rules showing how it is made
545out of simpler constructs. For example, one kind of C statement is the
546@code{return} statement; this would be described with a grammar rule which
547reads informally as follows:
548
549@quotation
550A `statement' can be made of a `return' keyword, an `expression' and a
551`semicolon'.
552@end quotation
553
554@noindent
555There would be many other rules for `statement', one for each kind of
556statement in C.
557
558@cindex start symbol
559One nonterminal symbol must be distinguished as the special one which
560defines a complete utterance in the language. It is called the @dfn{start
561symbol}. In a compiler, this means a complete input program. In the C
562language, the nonterminal symbol `sequence of definitions and declarations'
563plays this role.
564
565For example, @samp{1 + 2} is a valid C expression---a valid part of a C
566program---but it is not valid as an @emph{entire} C program. In the
567context-free grammar of C, this follows from the fact that `expression' is
568not the start symbol.
569
570The Bison parser reads a sequence of tokens as its input, and groups the
571tokens using the grammar rules. If the input is valid, the end result is
572that the entire token sequence reduces to a single grouping whose symbol is
573the grammar's start symbol. If we use a grammar for C, the entire input
574must be a `sequence of definitions and declarations'. If not, the parser
575reports a syntax error.
576
342b8b6e 577@node Grammar in Bison
bfa74976
RS
578@section From Formal Rules to Bison Input
579@cindex Bison grammar
580@cindex grammar, Bison
581@cindex formal grammar
582
583A formal grammar is a mathematical construct. To define the language
584for Bison, you must write a file expressing the grammar in Bison syntax:
585a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
586
587A nonterminal symbol in the formal grammar is represented in Bison input
c827f760 588as an identifier, like an identifier in C@. By convention, it should be
bfa74976
RS
589in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
590
591The Bison representation for a terminal symbol is also called a @dfn{token
592type}. Token types as well can be represented as C-like identifiers. By
593convention, these identifiers should be upper case to distinguish them from
594nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
595@code{RETURN}. A terminal symbol that stands for a particular keyword in
596the language should be named after that keyword converted to upper case.
597The terminal symbol @code{error} is reserved for error recovery.
931c7513 598@xref{Symbols}.
bfa74976
RS
599
600A terminal symbol can also be represented as a character literal, just like
601a C character constant. You should do this whenever a token is just a
602single character (parenthesis, plus-sign, etc.): use that same character in
603a literal as the terminal symbol for that token.
604
931c7513
RS
605A third way to represent a terminal symbol is with a C string constant
606containing several characters. @xref{Symbols}, for more information.
607
bfa74976
RS
608The grammar rules also have an expression in Bison syntax. For example,
609here is the Bison rule for a C @code{return} statement. The semicolon in
610quotes is a literal character token, representing part of the C syntax for
611the statement; the naked semicolon, and the colon, are Bison punctuation
612used in every rule.
613
614@example
615stmt: RETURN expr ';'
616 ;
617@end example
618
619@noindent
620@xref{Rules, ,Syntax of Grammar Rules}.
621
342b8b6e 622@node Semantic Values
bfa74976
RS
623@section Semantic Values
624@cindex semantic value
625@cindex value, semantic
626
627A formal grammar selects tokens only by their classifications: for example,
628if a rule mentions the terminal symbol `integer constant', it means that
629@emph{any} integer constant is grammatically valid in that position. The
630precise value of the constant is irrelevant to how to parse the input: if
631@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
e0c471a9 632grammatical.
bfa74976
RS
633
634But the precise value is very important for what the input means once it is
635parsed. A compiler is useless if it fails to distinguish between 4, 1 and
6363989 as constants in the program! Therefore, each token in a Bison grammar
c827f760
PE
637has both a token type and a @dfn{semantic value}. @xref{Semantics,
638,Defining Language Semantics},
bfa74976
RS
639for details.
640
641The token type is a terminal symbol defined in the grammar, such as
642@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
643you need to know to decide where the token may validly appear and how to
644group it with other tokens. The grammar rules know nothing about tokens
e0c471a9 645except their types.
bfa74976
RS
646
647The semantic value has all the rest of the information about the
648meaning of the token, such as the value of an integer, or the name of an
649identifier. (A token such as @code{','} which is just punctuation doesn't
650need to have any semantic value.)
651
652For example, an input token might be classified as token type
653@code{INTEGER} and have the semantic value 4. Another input token might
654have the same token type @code{INTEGER} but value 3989. When a grammar
655rule says that @code{INTEGER} is allowed, either of these tokens is
656acceptable because each is an @code{INTEGER}. When the parser accepts the
657token, it keeps track of the token's semantic value.
658
659Each grouping can also have a semantic value as well as its nonterminal
660symbol. For example, in a calculator, an expression typically has a
661semantic value that is a number. In a compiler for a programming
662language, an expression typically has a semantic value that is a tree
663structure describing the meaning of the expression.
664
342b8b6e 665@node Semantic Actions
bfa74976
RS
666@section Semantic Actions
667@cindex semantic actions
668@cindex actions, semantic
669
670In order to be useful, a program must do more than parse input; it must
671also produce some output based on the input. In a Bison grammar, a grammar
672rule can have an @dfn{action} made up of C statements. Each time the
673parser recognizes a match for that rule, the action is executed.
674@xref{Actions}.
13863333 675
bfa74976
RS
676Most of the time, the purpose of an action is to compute the semantic value
677of the whole construct from the semantic values of its parts. For example,
678suppose we have a rule which says an expression can be the sum of two
679expressions. When the parser recognizes such a sum, each of the
680subexpressions has a semantic value which describes how it was built up.
681The action for this rule should create a similar sort of value for the
682newly recognized larger expression.
683
684For example, here is a rule that says an expression can be the sum of
685two subexpressions:
686
687@example
688expr: expr '+' expr @{ $$ = $1 + $3; @}
689 ;
690@end example
691
692@noindent
693The action says how to produce the semantic value of the sum expression
694from the values of the two subexpressions.
695
676385e2 696@node GLR Parsers
c827f760
PE
697@section Writing @acronym{GLR} Parsers
698@cindex @acronym{GLR} parsing
699@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2
PH
700@findex %glr-parser
701@cindex conflicts
702@cindex shift/reduce conflicts
fa7e68c3 703@cindex reduce/reduce conflicts
676385e2 704
fa7e68c3 705In some grammars, Bison's standard
9501dc6e
AD
706@acronym{LALR}(1) parsing algorithm cannot decide whether to apply a
707certain grammar rule at a given point. That is, it may not be able to
708decide (on the basis of the input read so far) which of two possible
709reductions (applications of a grammar rule) applies, or whether to apply
710a reduction or read more of the input and apply a reduction later in the
711input. These are known respectively as @dfn{reduce/reduce} conflicts
712(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
713(@pxref{Shift/Reduce}).
714
715To use a grammar that is not easily modified to be @acronym{LALR}(1), a
716more general parsing algorithm is sometimes necessary. If you include
676385e2 717@code{%glr-parser} among the Bison declarations in your file
fa7e68c3 718(@pxref{Grammar Outline}), the result is a Generalized @acronym{LR}
9501dc6e
AD
719(@acronym{GLR}) parser. These parsers handle Bison grammars that
720contain no unresolved conflicts (i.e., after applying precedence
721declarations) identically to @acronym{LALR}(1) parsers. However, when
722faced with unresolved shift/reduce and reduce/reduce conflicts,
723@acronym{GLR} parsers use the simple expedient of doing both,
724effectively cloning the parser to follow both possibilities. Each of
725the resulting parsers can again split, so that at any given time, there
726can be any number of possible parses being explored. The parsers
676385e2
PH
727proceed in lockstep; that is, all of them consume (shift) a given input
728symbol before any of them proceed to the next. Each of the cloned
729parsers eventually meets one of two possible fates: either it runs into
730a parsing error, in which case it simply vanishes, or it merges with
731another parser, because the two of them have reduced the input to an
732identical set of symbols.
733
734During the time that there are multiple parsers, semantic actions are
735recorded, but not performed. When a parser disappears, its recorded
736semantic actions disappear as well, and are never performed. When a
737reduction makes two parsers identical, causing them to merge, Bison
738records both sets of semantic actions. Whenever the last two parsers
739merge, reverting to the single-parser case, Bison resolves all the
740outstanding actions either by precedences given to the grammar rules
741involved, or by performing both actions, and then calling a designated
742user-defined function on the resulting values to produce an arbitrary
743merged result.
744
fa7e68c3 745@menu
32c29292
JD
746* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
747* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
748* GLR Semantic Actions:: Deferred semantic actions have special concerns.
749* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
fa7e68c3
PE
750@end menu
751
752@node Simple GLR Parsers
753@subsection Using @acronym{GLR} on Unambiguous Grammars
754@cindex @acronym{GLR} parsing, unambiguous grammars
755@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars
756@findex %glr-parser
757@findex %expect-rr
758@cindex conflicts
759@cindex reduce/reduce conflicts
760@cindex shift/reduce conflicts
761
762In the simplest cases, you can use the @acronym{GLR} algorithm
763to parse grammars that are unambiguous, but fail to be @acronym{LALR}(1).
742e4900 764Such grammars typically require more than one symbol of lookahead,
fa7e68c3
PE
765or (in rare cases) fall into the category of grammars in which the
766@acronym{LALR}(1) algorithm throws away too much information (they are in
767@acronym{LR}(1), but not @acronym{LALR}(1), @ref{Mystery Conflicts}).
768
769Consider a problem that
770arises in the declaration of enumerated and subrange types in the
771programming language Pascal. Here are some examples:
772
773@example
774type subrange = lo .. hi;
775type enum = (a, b, c);
776@end example
777
778@noindent
779The original language standard allows only numeric
780literals and constant identifiers for the subrange bounds (@samp{lo}
781and @samp{hi}), but Extended Pascal (@acronym{ISO}/@acronym{IEC}
78210206) and many other
783Pascal implementations allow arbitrary expressions there. This gives
784rise to the following situation, containing a superfluous pair of
785parentheses:
786
787@example
788type subrange = (a) .. b;
789@end example
790
791@noindent
792Compare this to the following declaration of an enumerated
793type with only one value:
794
795@example
796type enum = (a);
797@end example
798
799@noindent
800(These declarations are contrived, but they are syntactically
801valid, and more-complicated cases can come up in practical programs.)
802
803These two declarations look identical until the @samp{..} token.
742e4900 804With normal @acronym{LALR}(1) one-token lookahead it is not
fa7e68c3
PE
805possible to decide between the two forms when the identifier
806@samp{a} is parsed. It is, however, desirable
807for a parser to decide this, since in the latter case
808@samp{a} must become a new identifier to represent the enumeration
809value, while in the former case @samp{a} must be evaluated with its
810current meaning, which may be a constant or even a function call.
811
812You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
813to be resolved later, but this typically requires substantial
814contortions in both semantic actions and large parts of the
815grammar, where the parentheses are nested in the recursive rules for
816expressions.
817
818You might think of using the lexer to distinguish between the two
819forms by returning different tokens for currently defined and
820undefined identifiers. But if these declarations occur in a local
821scope, and @samp{a} is defined in an outer scope, then both forms
822are possible---either locally redefining @samp{a}, or using the
823value of @samp{a} from the outer scope. So this approach cannot
824work.
825
e757bb10 826A simple solution to this problem is to declare the parser to
fa7e68c3
PE
827use the @acronym{GLR} algorithm.
828When the @acronym{GLR} parser reaches the critical state, it
829merely splits into two branches and pursues both syntax rules
830simultaneously. Sooner or later, one of them runs into a parsing
831error. If there is a @samp{..} token before the next
832@samp{;}, the rule for enumerated types fails since it cannot
833accept @samp{..} anywhere; otherwise, the subrange type rule
834fails since it requires a @samp{..} token. So one of the branches
835fails silently, and the other one continues normally, performing
836all the intermediate actions that were postponed during the split.
837
838If the input is syntactically incorrect, both branches fail and the parser
839reports a syntax error as usual.
840
841The effect of all this is that the parser seems to ``guess'' the
842correct branch to take, or in other words, it seems to use more
742e4900 843lookahead than the underlying @acronym{LALR}(1) algorithm actually allows
fa7e68c3
PE
844for. In this example, @acronym{LALR}(2) would suffice, but also some cases
845that are not @acronym{LALR}(@math{k}) for any @math{k} can be handled this way.
846
847In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time,
848and the current Bison parser even takes exponential time and space
849for some grammars. In practice, this rarely happens, and for many
850grammars it is possible to prove that it cannot happen.
851The present example contains only one conflict between two
852rules, and the type-declaration context containing the conflict
853cannot be nested. So the number of
854branches that can exist at any time is limited by the constant 2,
855and the parsing time is still linear.
856
857Here is a Bison grammar corresponding to the example above. It
858parses a vastly simplified form of Pascal type declarations.
859
860@example
861%token TYPE DOTDOT ID
862
863@group
864%left '+' '-'
865%left '*' '/'
866@end group
867
868%%
869
870@group
871type_decl : TYPE ID '=' type ';'
872 ;
873@end group
874
875@group
876type : '(' id_list ')'
877 | expr DOTDOT expr
878 ;
879@end group
880
881@group
882id_list : ID
883 | id_list ',' ID
884 ;
885@end group
886
887@group
888expr : '(' expr ')'
889 | expr '+' expr
890 | expr '-' expr
891 | expr '*' expr
892 | expr '/' expr
893 | ID
894 ;
895@end group
896@end example
897
898When used as a normal @acronym{LALR}(1) grammar, Bison correctly complains
899about one reduce/reduce conflict. In the conflicting situation the
900parser chooses one of the alternatives, arbitrarily the one
901declared first. Therefore the following correct input is not
902recognized:
903
904@example
905type t = (a) .. b;
906@end example
907
908The parser can be turned into a @acronym{GLR} parser, while also telling Bison
909to be silent about the one known reduce/reduce conflict, by
e757bb10 910adding these two declarations to the Bison input file (before the first
fa7e68c3
PE
911@samp{%%}):
912
913@example
914%glr-parser
915%expect-rr 1
916@end example
917
918@noindent
919No change in the grammar itself is required. Now the
920parser recognizes all valid declarations, according to the
921limited syntax above, transparently. In fact, the user does not even
922notice when the parser splits.
923
f8e1c9e5
AD
924So here we have a case where we can use the benefits of @acronym{GLR},
925almost without disadvantages. Even in simple cases like this, however,
926there are at least two potential problems to beware. First, always
927analyze the conflicts reported by Bison to make sure that @acronym{GLR}
928splitting is only done where it is intended. A @acronym{GLR} parser
929splitting inadvertently may cause problems less obvious than an
930@acronym{LALR} parser statically choosing the wrong alternative in a
931conflict. Second, consider interactions with the lexer (@pxref{Semantic
932Tokens}) with great care. Since a split parser consumes tokens without
933performing any actions during the split, the lexer cannot obtain
934information via parser actions. Some cases of lexer interactions can be
935eliminated by using @acronym{GLR} to shift the complications from the
936lexer to the parser. You must check the remaining cases for
937correctness.
938
939In our example, it would be safe for the lexer to return tokens based on
940their current meanings in some symbol table, because no new symbols are
941defined in the middle of a type declaration. Though it is possible for
942a parser to define the enumeration constants as they are parsed, before
943the type declaration is completed, it actually makes no difference since
944they cannot be used within the same enumerated type declaration.
fa7e68c3
PE
945
946@node Merging GLR Parses
947@subsection Using @acronym{GLR} to Resolve Ambiguities
948@cindex @acronym{GLR} parsing, ambiguous grammars
949@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, ambiguous grammars
950@findex %dprec
951@findex %merge
952@cindex conflicts
953@cindex reduce/reduce conflicts
954
2a8d363a 955Let's consider an example, vastly simplified from a C++ grammar.
676385e2
PH
956
957@example
958%@{
38a92d50
PE
959 #include <stdio.h>
960 #define YYSTYPE char const *
961 int yylex (void);
962 void yyerror (char const *);
676385e2
PH
963%@}
964
965%token TYPENAME ID
966
967%right '='
968%left '+'
969
970%glr-parser
971
972%%
973
fae437e8 974prog :
676385e2
PH
975 | prog stmt @{ printf ("\n"); @}
976 ;
977
978stmt : expr ';' %dprec 1
979 | decl %dprec 2
980 ;
981
2a8d363a 982expr : ID @{ printf ("%s ", $$); @}
fae437e8 983 | TYPENAME '(' expr ')'
2a8d363a
AD
984 @{ printf ("%s <cast> ", $1); @}
985 | expr '+' expr @{ printf ("+ "); @}
986 | expr '=' expr @{ printf ("= "); @}
676385e2
PH
987 ;
988
fae437e8 989decl : TYPENAME declarator ';'
2a8d363a 990 @{ printf ("%s <declare> ", $1); @}
676385e2 991 | TYPENAME declarator '=' expr ';'
2a8d363a 992 @{ printf ("%s <init-declare> ", $1); @}
676385e2
PH
993 ;
994
2a8d363a 995declarator : ID @{ printf ("\"%s\" ", $1); @}
676385e2
PH
996 | '(' declarator ')'
997 ;
998@end example
999
1000@noindent
1001This models a problematic part of the C++ grammar---the ambiguity between
1002certain declarations and statements. For example,
1003
1004@example
1005T (x) = y+z;
1006@end example
1007
1008@noindent
1009parses as either an @code{expr} or a @code{stmt}
c827f760
PE
1010(assuming that @samp{T} is recognized as a @code{TYPENAME} and
1011@samp{x} as an @code{ID}).
676385e2 1012Bison detects this as a reduce/reduce conflict between the rules
fae437e8 1013@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
e757bb10
AD
1014time it encounters @code{x} in the example above. Since this is a
1015@acronym{GLR} parser, it therefore splits the problem into two parses, one for
fa7e68c3
PE
1016each choice of resolving the reduce/reduce conflict.
1017Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1018however, neither of these parses ``dies,'' because the grammar as it stands is
e757bb10
AD
1019ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1020the other reduces @code{stmt : decl}, after which both parsers are in an
1021identical state: they've seen @samp{prog stmt} and have the same unprocessed
1022input remaining. We say that these parses have @dfn{merged.}
fa7e68c3
PE
1023
1024At this point, the @acronym{GLR} parser requires a specification in the
1025grammar of how to choose between the competing parses.
1026In the example above, the two @code{%dprec}
e757bb10 1027declarations specify that Bison is to give precedence
fa7e68c3 1028to the parse that interprets the example as a
676385e2
PH
1029@code{decl}, which implies that @code{x} is a declarator.
1030The parser therefore prints
1031
1032@example
fae437e8 1033"x" y z + T <init-declare>
676385e2
PH
1034@end example
1035
fa7e68c3
PE
1036The @code{%dprec} declarations only come into play when more than one
1037parse survives. Consider a different input string for this parser:
676385e2
PH
1038
1039@example
1040T (x) + y;
1041@end example
1042
1043@noindent
e757bb10 1044This is another example of using @acronym{GLR} to parse an unambiguous
fa7e68c3 1045construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
676385e2
PH
1046Here, there is no ambiguity (this cannot be parsed as a declaration).
1047However, at the time the Bison parser encounters @code{x}, it does not
1048have enough information to resolve the reduce/reduce conflict (again,
1049between @code{x} as an @code{expr} or a @code{declarator}). In this
fa7e68c3 1050case, no precedence declaration is used. Again, the parser splits
676385e2
PH
1051into two, one assuming that @code{x} is an @code{expr}, and the other
1052assuming @code{x} is a @code{declarator}. The second of these parsers
1053then vanishes when it sees @code{+}, and the parser prints
1054
1055@example
fae437e8 1056x T <cast> y +
676385e2
PH
1057@end example
1058
1059Suppose that instead of resolving the ambiguity, you wanted to see all
fa7e68c3 1060the possibilities. For this purpose, you must merge the semantic
676385e2
PH
1061actions of the two possible parsers, rather than choosing one over the
1062other. To do so, you could change the declaration of @code{stmt} as
1063follows:
1064
1065@example
1066stmt : expr ';' %merge <stmtMerge>
1067 | decl %merge <stmtMerge>
1068 ;
1069@end example
1070
1071@noindent
676385e2
PH
1072and define the @code{stmtMerge} function as:
1073
1074@example
38a92d50
PE
1075static YYSTYPE
1076stmtMerge (YYSTYPE x0, YYSTYPE x1)
676385e2
PH
1077@{
1078 printf ("<OR> ");
1079 return "";
1080@}
1081@end example
1082
1083@noindent
1084with an accompanying forward declaration
1085in the C declarations at the beginning of the file:
1086
1087@example
1088%@{
38a92d50 1089 #define YYSTYPE char const *
676385e2
PH
1090 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1091%@}
1092@end example
1093
1094@noindent
fa7e68c3
PE
1095With these declarations, the resulting parser parses the first example
1096as both an @code{expr} and a @code{decl}, and prints
676385e2
PH
1097
1098@example
fae437e8 1099"x" y z + T <init-declare> x T <cast> y z + = <OR>
676385e2
PH
1100@end example
1101
fa7e68c3 1102Bison requires that all of the
e757bb10 1103productions that participate in any particular merge have identical
fa7e68c3
PE
1104@samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1105and the parser will report an error during any parse that results in
1106the offending merge.
9501dc6e 1107
32c29292
JD
1108@node GLR Semantic Actions
1109@subsection GLR Semantic Actions
1110
1111@cindex deferred semantic actions
1112By definition, a deferred semantic action is not performed at the same time as
1113the associated reduction.
1114This raises caveats for several Bison features you might use in a semantic
1115action in a @acronym{GLR} parser.
1116
1117@vindex yychar
1118@cindex @acronym{GLR} parsers and @code{yychar}
1119@vindex yylval
1120@cindex @acronym{GLR} parsers and @code{yylval}
1121@vindex yylloc
1122@cindex @acronym{GLR} parsers and @code{yylloc}
1123In any semantic action, you can examine @code{yychar} to determine the type of
742e4900 1124the lookahead token present at the time of the associated reduction.
32c29292
JD
1125After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1126you can then examine @code{yylval} and @code{yylloc} to determine the
742e4900 1127lookahead token's semantic value and location, if any.
32c29292
JD
1128In a nondeferred semantic action, you can also modify any of these variables to
1129influence syntax analysis.
742e4900 1130@xref{Lookahead, ,Lookahead Tokens}.
32c29292
JD
1131
1132@findex yyclearin
1133@cindex @acronym{GLR} parsers and @code{yyclearin}
1134In a deferred semantic action, it's too late to influence syntax analysis.
1135In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1136shallow copies of the values they had at the time of the associated reduction.
1137For this reason alone, modifying them is dangerous.
1138Moreover, the result of modifying them is undefined and subject to change with
1139future versions of Bison.
1140For example, if a semantic action might be deferred, you should never write it
1141to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1142memory referenced by @code{yylval}.
1143
1144@findex YYERROR
1145@cindex @acronym{GLR} parsers and @code{YYERROR}
1146Another Bison feature requiring special consideration is @code{YYERROR}
8710fc41 1147(@pxref{Action Features}), which you can invoke in a semantic action to
32c29292
JD
1148initiate error recovery.
1149During deterministic @acronym{GLR} operation, the effect of @code{YYERROR} is
1150the same as its effect in an @acronym{LALR}(1) parser.
1151In a deferred semantic action, its effect is undefined.
1152@c The effect is probably a syntax error at the split point.
1153
8710fc41
JD
1154Also, see @ref{Location Default Action, ,Default Action for Locations}, which
1155describes a special usage of @code{YYLLOC_DEFAULT} in @acronym{GLR} parsers.
1156
fa7e68c3
PE
1157@node Compiler Requirements
1158@subsection Considerations when Compiling @acronym{GLR} Parsers
1159@cindex @code{inline}
9501dc6e 1160@cindex @acronym{GLR} parsers and @code{inline}
fa7e68c3 1161
38a92d50
PE
1162The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or
1163later. In addition, they use the @code{inline} keyword, which is not
1164C89, but is C99 and is a common extension in pre-C99 compilers. It is
1165up to the user of these parsers to handle
9501dc6e
AD
1166portability issues. For instance, if using Autoconf and the Autoconf
1167macro @code{AC_C_INLINE}, a mere
1168
1169@example
1170%@{
38a92d50 1171 #include <config.h>
9501dc6e
AD
1172%@}
1173@end example
1174
1175@noindent
1176will suffice. Otherwise, we suggest
1177
1178@example
1179%@{
38a92d50
PE
1180 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1181 #define inline
1182 #endif
9501dc6e
AD
1183%@}
1184@end example
676385e2 1185
342b8b6e 1186@node Locations Overview
847bf1f5
AD
1187@section Locations
1188@cindex location
95923bd6
AD
1189@cindex textual location
1190@cindex location, textual
847bf1f5
AD
1191
1192Many applications, like interpreters or compilers, have to produce verbose
72d2299c 1193and useful error messages. To achieve this, one must be able to keep track of
95923bd6 1194the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
847bf1f5
AD
1195Bison provides a mechanism for handling these locations.
1196
72d2299c 1197Each token has a semantic value. In a similar fashion, each token has an
847bf1f5 1198associated location, but the type of locations is the same for all tokens and
72d2299c 1199groupings. Moreover, the output parser is equipped with a default data
847bf1f5
AD
1200structure for storing locations (@pxref{Locations}, for more details).
1201
1202Like semantic values, locations can be reached in actions using a dedicated
72d2299c 1203set of constructs. In the example above, the location of the whole grouping
847bf1f5
AD
1204is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1205@code{@@3}.
1206
1207When a rule is matched, a default action is used to compute the semantic value
72d2299c
PE
1208of its left hand side (@pxref{Actions}). In the same way, another default
1209action is used for locations. However, the action for locations is general
847bf1f5 1210enough for most cases, meaning there is usually no need to describe for each
72d2299c 1211rule how @code{@@$} should be formed. When building a new location for a given
847bf1f5
AD
1212grouping, the default behavior of the output parser is to take the beginning
1213of the first symbol, and the end of the last symbol.
1214
342b8b6e 1215@node Bison Parser
bfa74976
RS
1216@section Bison Output: the Parser File
1217@cindex Bison parser
1218@cindex Bison utility
1219@cindex lexical analyzer, purpose
1220@cindex parser
1221
1222When you run Bison, you give it a Bison grammar file as input. The output
1223is a C source file that parses the language described by the grammar.
1224This file is called a @dfn{Bison parser}. Keep in mind that the Bison
1225utility and the Bison parser are two distinct programs: the Bison utility
1226is a program whose output is the Bison parser that becomes part of your
1227program.
1228
1229The job of the Bison parser is to group tokens into groupings according to
1230the grammar rules---for example, to build identifiers and operators into
1231expressions. As it does this, it runs the actions for the grammar rules it
1232uses.
1233
704a47c4
AD
1234The tokens come from a function called the @dfn{lexical analyzer} that
1235you must supply in some fashion (such as by writing it in C). The Bison
1236parser calls the lexical analyzer each time it wants a new token. It
1237doesn't know what is ``inside'' the tokens (though their semantic values
1238may reflect this). Typically the lexical analyzer makes the tokens by
1239parsing characters of text, but Bison does not depend on this.
1240@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
bfa74976
RS
1241
1242The Bison parser file is C code which defines a function named
1243@code{yyparse} which implements that grammar. This function does not make
1244a complete C program: you must supply some additional functions. One is
1245the lexical analyzer. Another is an error-reporting function which the
1246parser calls to report an error. In addition, a complete C program must
1247start with a function called @code{main}; you have to provide this, and
1248arrange for it to call @code{yyparse} or the parser will never run.
1249@xref{Interface, ,Parser C-Language Interface}.
1250
f7ab6a50 1251Aside from the token type names and the symbols in the actions you
7093d0f5 1252write, all symbols defined in the Bison parser file itself
bfa74976
RS
1253begin with @samp{yy} or @samp{YY}. This includes interface functions
1254such as the lexical analyzer function @code{yylex}, the error reporting
1255function @code{yyerror} and the parser function @code{yyparse} itself.
1256This also includes numerous identifiers used for internal purposes.
1257Therefore, you should avoid using C identifiers starting with @samp{yy}
1258or @samp{YY} in the Bison grammar file except for the ones defined in
55289366
PE
1259this manual. Also, you should avoid using the C identifiers
1260@samp{malloc} and @samp{free} for anything other than their usual
1261meanings.
bfa74976 1262
7093d0f5
AD
1263In some cases the Bison parser file includes system headers, and in
1264those cases your code should respect the identifiers reserved by those
55289366 1265headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>}, @code{<malloc.h>},
7093d0f5 1266@code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
30757c8c
PE
1267declare memory allocators and related types. @code{<libintl.h>} is
1268included if message translation is in use
1269(@pxref{Internationalization}). Other system headers may
ec3bc396
AD
1270be included if you define @code{YYDEBUG} to a nonzero value
1271(@pxref{Tracing, ,Tracing Your Parser}).
7093d0f5 1272
342b8b6e 1273@node Stages
bfa74976
RS
1274@section Stages in Using Bison
1275@cindex stages in using Bison
1276@cindex using Bison
1277
1278The actual language-design process using Bison, from grammar specification
1279to a working compiler or interpreter, has these parts:
1280
1281@enumerate
1282@item
1283Formally specify the grammar in a form recognized by Bison
704a47c4
AD
1284(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1285in the language, describe the action that is to be taken when an
1286instance of that rule is recognized. The action is described by a
1287sequence of C statements.
bfa74976
RS
1288
1289@item
704a47c4
AD
1290Write a lexical analyzer to process input and pass tokens to the parser.
1291The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1292Lexical Analyzer Function @code{yylex}}). It could also be produced
1293using Lex, but the use of Lex is not discussed in this manual.
bfa74976
RS
1294
1295@item
1296Write a controlling function that calls the Bison-produced parser.
1297
1298@item
1299Write error-reporting routines.
1300@end enumerate
1301
1302To turn this source code as written into a runnable program, you
1303must follow these steps:
1304
1305@enumerate
1306@item
1307Run Bison on the grammar to produce the parser.
1308
1309@item
1310Compile the code output by Bison, as well as any other source files.
1311
1312@item
1313Link the object files to produce the finished product.
1314@end enumerate
1315
342b8b6e 1316@node Grammar Layout
bfa74976
RS
1317@section The Overall Layout of a Bison Grammar
1318@cindex grammar file
1319@cindex file format
1320@cindex format of grammar file
1321@cindex layout of Bison grammar
1322
1323The input file for the Bison utility is a @dfn{Bison grammar file}. The
1324general form of a Bison grammar file is as follows:
1325
1326@example
1327%@{
08e49d20 1328@var{Prologue}
bfa74976
RS
1329%@}
1330
1331@var{Bison declarations}
1332
1333%%
1334@var{Grammar rules}
1335%%
08e49d20 1336@var{Epilogue}
bfa74976
RS
1337@end example
1338
1339@noindent
1340The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1341in every Bison grammar file to separate the sections.
1342
72d2299c 1343The prologue may define types and variables used in the actions. You can
342b8b6e 1344also use preprocessor commands to define macros used there, and use
bfa74976 1345@code{#include} to include header files that do any of these things.
38a92d50
PE
1346You need to declare the lexical analyzer @code{yylex} and the error
1347printer @code{yyerror} here, along with any other global identifiers
1348used by the actions in the grammar rules.
bfa74976
RS
1349
1350The Bison declarations declare the names of the terminal and nonterminal
1351symbols, and may also describe operator precedence and the data types of
1352semantic values of various symbols.
1353
1354The grammar rules define how to construct each nonterminal symbol from its
1355parts.
1356
38a92d50
PE
1357The epilogue can contain any code you want to use. Often the
1358definitions of functions declared in the prologue go here. In a
1359simple program, all the rest of the program can go here.
bfa74976 1360
342b8b6e 1361@node Examples
bfa74976
RS
1362@chapter Examples
1363@cindex simple examples
1364@cindex examples, simple
1365
1366Now we show and explain three sample programs written using Bison: a
1367reverse polish notation calculator, an algebraic (infix) notation
1368calculator, and a multi-function calculator. All three have been tested
1369under BSD Unix 4.3; each produces a usable, though limited, interactive
1370desk-top calculator.
1371
1372These examples are simple, but Bison grammars for real programming
aa08666d
AD
1373languages are written the same way. You can copy these examples into a
1374source file to try them.
bfa74976
RS
1375
1376@menu
1377* RPN Calc:: Reverse polish notation calculator;
1378 a first example with no operator precedence.
1379* Infix Calc:: Infix (algebraic) notation calculator.
1380 Operator precedence is introduced.
1381* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 1382* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
bfa74976
RS
1383* Multi-function Calc:: Calculator with memory and trig functions.
1384 It uses multiple data-types for semantic values.
1385* Exercises:: Ideas for improving the multi-function calculator.
1386@end menu
1387
342b8b6e 1388@node RPN Calc
bfa74976
RS
1389@section Reverse Polish Notation Calculator
1390@cindex reverse polish notation
1391@cindex polish notation calculator
1392@cindex @code{rpcalc}
1393@cindex calculator, simple
1394
1395The first example is that of a simple double-precision @dfn{reverse polish
1396notation} calculator (a calculator using postfix operators). This example
1397provides a good starting point, since operator precedence is not an issue.
1398The second example will illustrate how operator precedence is handled.
1399
1400The source code for this calculator is named @file{rpcalc.y}. The
1401@samp{.y} extension is a convention used for Bison input files.
1402
1403@menu
75f5aaea 1404* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
bfa74976
RS
1405* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
1406* Lexer: Rpcalc Lexer. The lexical analyzer.
1407* Main: Rpcalc Main. The controlling function.
1408* Error: Rpcalc Error. The error reporting function.
1409* Gen: Rpcalc Gen. Running Bison on the grammar file.
1410* Comp: Rpcalc Compile. Run the C compiler on the output code.
1411@end menu
1412
342b8b6e 1413@node Rpcalc Decls
bfa74976
RS
1414@subsection Declarations for @code{rpcalc}
1415
1416Here are the C and Bison declarations for the reverse polish notation
1417calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1418
1419@example
72d2299c 1420/* Reverse polish notation calculator. */
bfa74976
RS
1421
1422%@{
38a92d50
PE
1423 #define YYSTYPE double
1424 #include <math.h>
1425 int yylex (void);
1426 void yyerror (char const *);
bfa74976
RS
1427%@}
1428
1429%token NUM
1430
72d2299c 1431%% /* Grammar rules and actions follow. */
bfa74976
RS
1432@end example
1433
75f5aaea 1434The declarations section (@pxref{Prologue, , The prologue}) contains two
38a92d50 1435preprocessor directives and two forward declarations.
bfa74976
RS
1436
1437The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1964ad8c
AD
1438specifying the C data type for semantic values of both tokens and
1439groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1440Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1441don't define it, @code{int} is the default. Because we specify
1442@code{double}, each token and each expression has an associated value,
1443which is a floating point number.
bfa74976
RS
1444
1445The @code{#include} directive is used to declare the exponentiation
1446function @code{pow}.
1447
38a92d50
PE
1448The forward declarations for @code{yylex} and @code{yyerror} are
1449needed because the C language requires that functions be declared
1450before they are used. These functions will be defined in the
1451epilogue, but the parser calls them so they must be declared in the
1452prologue.
1453
704a47c4
AD
1454The second section, Bison declarations, provides information to Bison
1455about the token types (@pxref{Bison Declarations, ,The Bison
1456Declarations Section}). Each terminal symbol that is not a
1457single-character literal must be declared here. (Single-character
bfa74976
RS
1458literals normally don't need to be declared.) In this example, all the
1459arithmetic operators are designated by single-character literals, so the
1460only terminal symbol that needs to be declared is @code{NUM}, the token
1461type for numeric constants.
1462
342b8b6e 1463@node Rpcalc Rules
bfa74976
RS
1464@subsection Grammar Rules for @code{rpcalc}
1465
1466Here are the grammar rules for the reverse polish notation calculator.
1467
1468@example
1469input: /* empty */
1470 | input line
1471;
1472
1473line: '\n'
18b519c0 1474 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
bfa74976
RS
1475;
1476
18b519c0
AD
1477exp: NUM @{ $$ = $1; @}
1478 | exp exp '+' @{ $$ = $1 + $2; @}
1479 | exp exp '-' @{ $$ = $1 - $2; @}
1480 | exp exp '*' @{ $$ = $1 * $2; @}
1481 | exp exp '/' @{ $$ = $1 / $2; @}
1482 /* Exponentiation */
1483 | exp exp '^' @{ $$ = pow ($1, $2); @}
1484 /* Unary minus */
1485 | exp 'n' @{ $$ = -$1; @}
bfa74976
RS
1486;
1487%%
1488@end example
1489
1490The groupings of the rpcalc ``language'' defined here are the expression
1491(given the name @code{exp}), the line of input (@code{line}), and the
1492complete input transcript (@code{input}). Each of these nonterminal
8c5b881d 1493symbols has several alternate rules, joined by the vertical bar @samp{|}
bfa74976
RS
1494which is read as ``or''. The following sections explain what these rules
1495mean.
1496
1497The semantics of the language is determined by the actions taken when a
1498grouping is recognized. The actions are the C code that appears inside
1499braces. @xref{Actions}.
1500
1501You must specify these actions in C, but Bison provides the means for
1502passing semantic values between the rules. In each action, the
1503pseudo-variable @code{$$} stands for the semantic value for the grouping
1504that the rule is going to construct. Assigning a value to @code{$$} is the
1505main job of most actions. The semantic values of the components of the
1506rule are referred to as @code{$1}, @code{$2}, and so on.
1507
1508@menu
13863333
AD
1509* Rpcalc Input::
1510* Rpcalc Line::
1511* Rpcalc Expr::
bfa74976
RS
1512@end menu
1513
342b8b6e 1514@node Rpcalc Input
bfa74976
RS
1515@subsubsection Explanation of @code{input}
1516
1517Consider the definition of @code{input}:
1518
1519@example
1520input: /* empty */
1521 | input line
1522;
1523@end example
1524
1525This definition reads as follows: ``A complete input is either an empty
1526string, or a complete input followed by an input line''. Notice that
1527``complete input'' is defined in terms of itself. This definition is said
1528to be @dfn{left recursive} since @code{input} appears always as the
1529leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1530
1531The first alternative is empty because there are no symbols between the
1532colon and the first @samp{|}; this means that @code{input} can match an
1533empty string of input (no tokens). We write the rules this way because it
1534is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1535It's conventional to put an empty alternative first and write the comment
1536@samp{/* empty */} in it.
1537
1538The second alternate rule (@code{input line}) handles all nontrivial input.
1539It means, ``After reading any number of lines, read one more line if
1540possible.'' The left recursion makes this rule into a loop. Since the
1541first alternative matches empty input, the loop can be executed zero or
1542more times.
1543
1544The parser function @code{yyparse} continues to process input until a
1545grammatical error is seen or the lexical analyzer says there are no more
72d2299c 1546input tokens; we will arrange for the latter to happen at end-of-input.
bfa74976 1547
342b8b6e 1548@node Rpcalc Line
bfa74976
RS
1549@subsubsection Explanation of @code{line}
1550
1551Now consider the definition of @code{line}:
1552
1553@example
1554line: '\n'
1555 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1556;
1557@end example
1558
1559The first alternative is a token which is a newline character; this means
1560that rpcalc accepts a blank line (and ignores it, since there is no
1561action). The second alternative is an expression followed by a newline.
1562This is the alternative that makes rpcalc useful. The semantic value of
1563the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1564question is the first symbol in the alternative. The action prints this
1565value, which is the result of the computation the user asked for.
1566
1567This action is unusual because it does not assign a value to @code{$$}. As
1568a consequence, the semantic value associated with the @code{line} is
1569uninitialized (its value will be unpredictable). This would be a bug if
1570that value were ever used, but we don't use it: once rpcalc has printed the
1571value of the user's input line, that value is no longer needed.
1572
342b8b6e 1573@node Rpcalc Expr
bfa74976
RS
1574@subsubsection Explanation of @code{expr}
1575
1576The @code{exp} grouping has several rules, one for each kind of expression.
1577The first rule handles the simplest expressions: those that are just numbers.
1578The second handles an addition-expression, which looks like two expressions
1579followed by a plus-sign. The third handles subtraction, and so on.
1580
1581@example
1582exp: NUM
1583 | exp exp '+' @{ $$ = $1 + $2; @}
1584 | exp exp '-' @{ $$ = $1 - $2; @}
1585 @dots{}
1586 ;
1587@end example
1588
1589We have used @samp{|} to join all the rules for @code{exp}, but we could
1590equally well have written them separately:
1591
1592@example
1593exp: NUM ;
1594exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1595exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1596 @dots{}
1597@end example
1598
1599Most of the rules have actions that compute the value of the expression in
1600terms of the value of its parts. For example, in the rule for addition,
1601@code{$1} refers to the first component @code{exp} and @code{$2} refers to
1602the second one. The third component, @code{'+'}, has no meaningful
1603associated semantic value, but if it had one you could refer to it as
1604@code{$3}. When @code{yyparse} recognizes a sum expression using this
1605rule, the sum of the two subexpressions' values is produced as the value of
1606the entire expression. @xref{Actions}.
1607
1608You don't have to give an action for every rule. When a rule has no
1609action, Bison by default copies the value of @code{$1} into @code{$$}.
1610This is what happens in the first rule (the one that uses @code{NUM}).
1611
1612The formatting shown here is the recommended convention, but Bison does
72d2299c 1613not require it. You can add or change white space as much as you wish.
bfa74976
RS
1614For example, this:
1615
1616@example
99a9344e 1617exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
bfa74976
RS
1618@end example
1619
1620@noindent
1621means the same thing as this:
1622
1623@example
1624exp: NUM
1625 | exp exp '+' @{ $$ = $1 + $2; @}
1626 | @dots{}
99a9344e 1627;
bfa74976
RS
1628@end example
1629
1630@noindent
1631The latter, however, is much more readable.
1632
342b8b6e 1633@node Rpcalc Lexer
bfa74976
RS
1634@subsection The @code{rpcalc} Lexical Analyzer
1635@cindex writing a lexical analyzer
1636@cindex lexical analyzer, writing
1637
704a47c4
AD
1638The lexical analyzer's job is low-level parsing: converting characters
1639or sequences of characters into tokens. The Bison parser gets its
1640tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1641Analyzer Function @code{yylex}}.
bfa74976 1642
c827f760
PE
1643Only a simple lexical analyzer is needed for the @acronym{RPN}
1644calculator. This
bfa74976
RS
1645lexical analyzer skips blanks and tabs, then reads in numbers as
1646@code{double} and returns them as @code{NUM} tokens. Any other character
1647that isn't part of a number is a separate token. Note that the token-code
1648for such a single-character token is the character itself.
1649
1650The return value of the lexical analyzer function is a numeric code which
1651represents a token type. The same text used in Bison rules to stand for
1652this token type is also a C expression for the numeric code for the type.
1653This works in two ways. If the token type is a character literal, then its
e966383b 1654numeric code is that of the character; you can use the same
bfa74976
RS
1655character literal in the lexical analyzer to express the number. If the
1656token type is an identifier, that identifier is defined by Bison as a C
1657macro whose definition is the appropriate number. In this example,
1658therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1659
1964ad8c
AD
1660The semantic value of the token (if it has one) is stored into the
1661global variable @code{yylval}, which is where the Bison parser will look
1662for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1663defined at the beginning of the grammar; @pxref{Rpcalc Decls,
1664,Declarations for @code{rpcalc}}.)
bfa74976 1665
72d2299c
PE
1666A token type code of zero is returned if the end-of-input is encountered.
1667(Bison recognizes any nonpositive value as indicating end-of-input.)
bfa74976
RS
1668
1669Here is the code for the lexical analyzer:
1670
1671@example
1672@group
72d2299c 1673/* The lexical analyzer returns a double floating point
e966383b 1674 number on the stack and the token NUM, or the numeric code
72d2299c
PE
1675 of the character read if not a number. It skips all blanks
1676 and tabs, and returns 0 for end-of-input. */
bfa74976
RS
1677
1678#include <ctype.h>
1679@end group
1680
1681@group
13863333
AD
1682int
1683yylex (void)
bfa74976
RS
1684@{
1685 int c;
1686
72d2299c 1687 /* Skip white space. */
13863333 1688 while ((c = getchar ()) == ' ' || c == '\t')
bfa74976
RS
1689 ;
1690@end group
1691@group
72d2299c 1692 /* Process numbers. */
13863333 1693 if (c == '.' || isdigit (c))
bfa74976
RS
1694 @{
1695 ungetc (c, stdin);
1696 scanf ("%lf", &yylval);
1697 return NUM;
1698 @}
1699@end group
1700@group
72d2299c 1701 /* Return end-of-input. */
13863333 1702 if (c == EOF)
bfa74976 1703 return 0;
72d2299c 1704 /* Return a single char. */
13863333 1705 return c;
bfa74976
RS
1706@}
1707@end group
1708@end example
1709
342b8b6e 1710@node Rpcalc Main
bfa74976
RS
1711@subsection The Controlling Function
1712@cindex controlling function
1713@cindex main function in simple example
1714
1715In keeping with the spirit of this example, the controlling function is
1716kept to the bare minimum. The only requirement is that it call
1717@code{yyparse} to start the process of parsing.
1718
1719@example
1720@group
13863333
AD
1721int
1722main (void)
bfa74976 1723@{
13863333 1724 return yyparse ();
bfa74976
RS
1725@}
1726@end group
1727@end example
1728
342b8b6e 1729@node Rpcalc Error
bfa74976
RS
1730@subsection The Error Reporting Routine
1731@cindex error reporting routine
1732
1733When @code{yyparse} detects a syntax error, it calls the error reporting
13863333 1734function @code{yyerror} to print an error message (usually but not
6e649e65 1735always @code{"syntax error"}). It is up to the programmer to supply
13863333
AD
1736@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1737here is the definition we will use:
bfa74976
RS
1738
1739@example
1740@group
1741#include <stdio.h>
1742
38a92d50 1743/* Called by yyparse on error. */
13863333 1744void
38a92d50 1745yyerror (char const *s)
bfa74976 1746@{
4e03e201 1747 fprintf (stderr, "%s\n", s);
bfa74976
RS
1748@}
1749@end group
1750@end example
1751
1752After @code{yyerror} returns, the Bison parser may recover from the error
1753and continue parsing if the grammar contains a suitable error rule
1754(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1755have not written any error rules in this example, so any invalid input will
1756cause the calculator program to exit. This is not clean behavior for a
9ecbd125 1757real calculator, but it is adequate for the first example.
bfa74976 1758
342b8b6e 1759@node Rpcalc Gen
bfa74976
RS
1760@subsection Running Bison to Make the Parser
1761@cindex running Bison (introduction)
1762
ceed8467
AD
1763Before running Bison to produce a parser, we need to decide how to
1764arrange all the source code in one or more source files. For such a
1765simple example, the easiest thing is to put everything in one file. The
1766definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
342b8b6e 1767end, in the epilogue of the file
75f5aaea 1768(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
bfa74976
RS
1769
1770For a large project, you would probably have several source files, and use
1771@code{make} to arrange to recompile them.
1772
1773With all the source in a single file, you use the following command to
1774convert it into a parser file:
1775
1776@example
fa4d969f 1777bison @var{file}.y
bfa74976
RS
1778@end example
1779
1780@noindent
1781In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
fa4d969f 1782@sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c},
72d2299c 1783removing the @samp{.y} from the original file name. The file output by
bfa74976
RS
1784Bison contains the source code for @code{yyparse}. The additional
1785functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
1786are copied verbatim to the output.
1787
342b8b6e 1788@node Rpcalc Compile
bfa74976
RS
1789@subsection Compiling the Parser File
1790@cindex compiling the parser
1791
1792Here is how to compile and run the parser file:
1793
1794@example
1795@group
1796# @r{List files in current directory.}
9edcd895 1797$ @kbd{ls}
bfa74976
RS
1798rpcalc.tab.c rpcalc.y
1799@end group
1800
1801@group
1802# @r{Compile the Bison parser.}
1803# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
b56471a6 1804$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
bfa74976
RS
1805@end group
1806
1807@group
1808# @r{List files again.}
9edcd895 1809$ @kbd{ls}
bfa74976
RS
1810rpcalc rpcalc.tab.c rpcalc.y
1811@end group
1812@end example
1813
1814The file @file{rpcalc} now contains the executable code. Here is an
1815example session using @code{rpcalc}.
1816
1817@example
9edcd895
AD
1818$ @kbd{rpcalc}
1819@kbd{4 9 +}
bfa74976 182013
9edcd895 1821@kbd{3 7 + 3 4 5 *+-}
bfa74976 1822-13
9edcd895 1823@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
bfa74976 182413
9edcd895 1825@kbd{5 6 / 4 n +}
bfa74976 1826-3.166666667
9edcd895 1827@kbd{3 4 ^} @r{Exponentiation}
bfa74976 182881
9edcd895
AD
1829@kbd{^D} @r{End-of-file indicator}
1830$
bfa74976
RS
1831@end example
1832
342b8b6e 1833@node Infix Calc
bfa74976
RS
1834@section Infix Notation Calculator: @code{calc}
1835@cindex infix notation calculator
1836@cindex @code{calc}
1837@cindex calculator, infix notation
1838
1839We now modify rpcalc to handle infix operators instead of postfix. Infix
1840notation involves the concept of operator precedence and the need for
1841parentheses nested to arbitrary depth. Here is the Bison code for
1842@file{calc.y}, an infix desk-top calculator.
1843
1844@example
38a92d50 1845/* Infix notation calculator. */
bfa74976
RS
1846
1847%@{
38a92d50
PE
1848 #define YYSTYPE double
1849 #include <math.h>
1850 #include <stdio.h>
1851 int yylex (void);
1852 void yyerror (char const *);
bfa74976
RS
1853%@}
1854
38a92d50 1855/* Bison declarations. */
bfa74976
RS
1856%token NUM
1857%left '-' '+'
1858%left '*' '/'
1859%left NEG /* negation--unary minus */
38a92d50 1860%right '^' /* exponentiation */
bfa74976 1861
38a92d50
PE
1862%% /* The grammar follows. */
1863input: /* empty */
bfa74976
RS
1864 | input line
1865;
1866
1867line: '\n'
1868 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1869;
1870
1871exp: NUM @{ $$ = $1; @}
1872 | exp '+' exp @{ $$ = $1 + $3; @}
1873 | exp '-' exp @{ $$ = $1 - $3; @}
1874 | exp '*' exp @{ $$ = $1 * $3; @}
1875 | exp '/' exp @{ $$ = $1 / $3; @}
1876 | '-' exp %prec NEG @{ $$ = -$2; @}
1877 | exp '^' exp @{ $$ = pow ($1, $3); @}
1878 | '(' exp ')' @{ $$ = $2; @}
1879;
1880%%
1881@end example
1882
1883@noindent
ceed8467
AD
1884The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1885same as before.
bfa74976
RS
1886
1887There are two important new features shown in this code.
1888
1889In the second section (Bison declarations), @code{%left} declares token
1890types and says they are left-associative operators. The declarations
1891@code{%left} and @code{%right} (right associativity) take the place of
1892@code{%token} which is used to declare a token type name without
1893associativity. (These tokens are single-character literals, which
1894ordinarily don't need to be declared. We declare them here to specify
1895the associativity.)
1896
1897Operator precedence is determined by the line ordering of the
1898declarations; the higher the line number of the declaration (lower on
1899the page or screen), the higher the precedence. Hence, exponentiation
1900has the highest precedence, unary minus (@code{NEG}) is next, followed
704a47c4
AD
1901by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator
1902Precedence}.
bfa74976 1903
704a47c4
AD
1904The other important new feature is the @code{%prec} in the grammar
1905section for the unary minus operator. The @code{%prec} simply instructs
1906Bison that the rule @samp{| '-' exp} has the same precedence as
1907@code{NEG}---in this case the next-to-highest. @xref{Contextual
1908Precedence, ,Context-Dependent Precedence}.
bfa74976
RS
1909
1910Here is a sample run of @file{calc.y}:
1911
1912@need 500
1913@example
9edcd895
AD
1914$ @kbd{calc}
1915@kbd{4 + 4.5 - (34/(8*3+-3))}
bfa74976 19166.880952381
9edcd895 1917@kbd{-56 + 2}
bfa74976 1918-54
9edcd895 1919@kbd{3 ^ 2}
bfa74976
RS
19209
1921@end example
1922
342b8b6e 1923@node Simple Error Recovery
bfa74976
RS
1924@section Simple Error Recovery
1925@cindex error recovery, simple
1926
1927Up to this point, this manual has not addressed the issue of @dfn{error
1928recovery}---how to continue parsing after the parser detects a syntax
ceed8467
AD
1929error. All we have handled is error reporting with @code{yyerror}.
1930Recall that by default @code{yyparse} returns after calling
1931@code{yyerror}. This means that an erroneous input line causes the
1932calculator program to exit. Now we show how to rectify this deficiency.
bfa74976
RS
1933
1934The Bison language itself includes the reserved word @code{error}, which
1935may be included in the grammar rules. In the example below it has
1936been added to one of the alternatives for @code{line}:
1937
1938@example
1939@group
1940line: '\n'
1941 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1942 | error '\n' @{ yyerrok; @}
1943;
1944@end group
1945@end example
1946
ceed8467 1947This addition to the grammar allows for simple error recovery in the
6e649e65 1948event of a syntax error. If an expression that cannot be evaluated is
ceed8467
AD
1949read, the error will be recognized by the third rule for @code{line},
1950and parsing will continue. (The @code{yyerror} function is still called
1951upon to print its message as well.) The action executes the statement
1952@code{yyerrok}, a macro defined automatically by Bison; its meaning is
1953that error recovery is complete (@pxref{Error Recovery}). Note the
1954difference between @code{yyerrok} and @code{yyerror}; neither one is a
e0c471a9 1955misprint.
bfa74976
RS
1956
1957This form of error recovery deals with syntax errors. There are other
1958kinds of errors; for example, division by zero, which raises an exception
1959signal that is normally fatal. A real calculator program must handle this
1960signal and use @code{longjmp} to return to @code{main} and resume parsing
1961input lines; it would also have to discard the rest of the current line of
1962input. We won't discuss this issue further because it is not specific to
1963Bison programs.
1964
342b8b6e
AD
1965@node Location Tracking Calc
1966@section Location Tracking Calculator: @code{ltcalc}
1967@cindex location tracking calculator
1968@cindex @code{ltcalc}
1969@cindex calculator, location tracking
1970
9edcd895
AD
1971This example extends the infix notation calculator with location
1972tracking. This feature will be used to improve the error messages. For
1973the sake of clarity, this example is a simple integer calculator, since
1974most of the work needed to use locations will be done in the lexical
72d2299c 1975analyzer.
342b8b6e
AD
1976
1977@menu
1978* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
1979* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
1980* Lexer: Ltcalc Lexer. The lexical analyzer.
1981@end menu
1982
1983@node Ltcalc Decls
1984@subsection Declarations for @code{ltcalc}
1985
9edcd895
AD
1986The C and Bison declarations for the location tracking calculator are
1987the same as the declarations for the infix notation calculator.
342b8b6e
AD
1988
1989@example
1990/* Location tracking calculator. */
1991
1992%@{
38a92d50
PE
1993 #define YYSTYPE int
1994 #include <math.h>
1995 int yylex (void);
1996 void yyerror (char const *);
342b8b6e
AD
1997%@}
1998
1999/* Bison declarations. */
2000%token NUM
2001
2002%left '-' '+'
2003%left '*' '/'
2004%left NEG
2005%right '^'
2006
38a92d50 2007%% /* The grammar follows. */
342b8b6e
AD
2008@end example
2009
9edcd895
AD
2010@noindent
2011Note there are no declarations specific to locations. Defining a data
2012type for storing locations is not needed: we will use the type provided
2013by default (@pxref{Location Type, ,Data Types of Locations}), which is a
2014four member structure with the following integer fields:
2015@code{first_line}, @code{first_column}, @code{last_line} and
cd48d21d
AD
2016@code{last_column}. By conventions, and in accordance with the GNU
2017Coding Standards and common practice, the line and column count both
2018start at 1.
342b8b6e
AD
2019
2020@node Ltcalc Rules
2021@subsection Grammar Rules for @code{ltcalc}
2022
9edcd895
AD
2023Whether handling locations or not has no effect on the syntax of your
2024language. Therefore, grammar rules for this example will be very close
2025to those of the previous example: we will only modify them to benefit
2026from the new information.
342b8b6e 2027
9edcd895
AD
2028Here, we will use locations to report divisions by zero, and locate the
2029wrong expressions or subexpressions.
342b8b6e
AD
2030
2031@example
2032@group
2033input : /* empty */
2034 | input line
2035;
2036@end group
2037
2038@group
2039line : '\n'
2040 | exp '\n' @{ printf ("%d\n", $1); @}
2041;
2042@end group
2043
2044@group
2045exp : NUM @{ $$ = $1; @}
2046 | exp '+' exp @{ $$ = $1 + $3; @}
2047 | exp '-' exp @{ $$ = $1 - $3; @}
2048 | exp '*' exp @{ $$ = $1 * $3; @}
2049@end group
342b8b6e 2050@group
9edcd895 2051 | exp '/' exp
342b8b6e
AD
2052 @{
2053 if ($3)
2054 $$ = $1 / $3;
2055 else
2056 @{
2057 $$ = 1;
9edcd895
AD
2058 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2059 @@3.first_line, @@3.first_column,
2060 @@3.last_line, @@3.last_column);
342b8b6e
AD
2061 @}
2062 @}
2063@end group
2064@group
178e123e 2065 | '-' exp %prec NEG @{ $$ = -$2; @}
342b8b6e
AD
2066 | exp '^' exp @{ $$ = pow ($1, $3); @}
2067 | '(' exp ')' @{ $$ = $2; @}
2068@end group
2069@end example
2070
2071This code shows how to reach locations inside of semantic actions, by
2072using the pseudo-variables @code{@@@var{n}} for rule components, and the
2073pseudo-variable @code{@@$} for groupings.
2074
9edcd895
AD
2075We don't need to assign a value to @code{@@$}: the output parser does it
2076automatically. By default, before executing the C code of each action,
2077@code{@@$} is set to range from the beginning of @code{@@1} to the end
2078of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2079can be redefined (@pxref{Location Default Action, , Default Action for
2080Locations}), and for very specific rules, @code{@@$} can be computed by
2081hand.
342b8b6e
AD
2082
2083@node Ltcalc Lexer
2084@subsection The @code{ltcalc} Lexical Analyzer.
2085
9edcd895 2086Until now, we relied on Bison's defaults to enable location
72d2299c 2087tracking. The next step is to rewrite the lexical analyzer, and make it
9edcd895
AD
2088able to feed the parser with the token locations, as it already does for
2089semantic values.
342b8b6e 2090
9edcd895
AD
2091To this end, we must take into account every single character of the
2092input text, to avoid the computed locations of being fuzzy or wrong:
342b8b6e
AD
2093
2094@example
2095@group
2096int
2097yylex (void)
2098@{
2099 int c;
18b519c0 2100@end group
342b8b6e 2101
18b519c0 2102@group
72d2299c 2103 /* Skip white space. */
342b8b6e
AD
2104 while ((c = getchar ()) == ' ' || c == '\t')
2105 ++yylloc.last_column;
18b519c0 2106@end group
342b8b6e 2107
18b519c0 2108@group
72d2299c 2109 /* Step. */
342b8b6e
AD
2110 yylloc.first_line = yylloc.last_line;
2111 yylloc.first_column = yylloc.last_column;
2112@end group
2113
2114@group
72d2299c 2115 /* Process numbers. */
342b8b6e
AD
2116 if (isdigit (c))
2117 @{
2118 yylval = c - '0';
2119 ++yylloc.last_column;
2120 while (isdigit (c = getchar ()))
2121 @{
2122 ++yylloc.last_column;
2123 yylval = yylval * 10 + c - '0';
2124 @}
2125 ungetc (c, stdin);
2126 return NUM;
2127 @}
2128@end group
2129
72d2299c 2130 /* Return end-of-input. */
342b8b6e
AD
2131 if (c == EOF)
2132 return 0;
2133
72d2299c 2134 /* Return a single char, and update location. */
342b8b6e
AD
2135 if (c == '\n')
2136 @{
2137 ++yylloc.last_line;
2138 yylloc.last_column = 0;
2139 @}
2140 else
2141 ++yylloc.last_column;
2142 return c;
2143@}
2144@end example
2145
9edcd895
AD
2146Basically, the lexical analyzer performs the same processing as before:
2147it skips blanks and tabs, and reads numbers or single-character tokens.
2148In addition, it updates @code{yylloc}, the global variable (of type
2149@code{YYLTYPE}) containing the token's location.
342b8b6e 2150
9edcd895 2151Now, each time this function returns a token, the parser has its number
72d2299c 2152as well as its semantic value, and its location in the text. The last
9edcd895
AD
2153needed change is to initialize @code{yylloc}, for example in the
2154controlling function:
342b8b6e
AD
2155
2156@example
9edcd895 2157@group
342b8b6e
AD
2158int
2159main (void)
2160@{
2161 yylloc.first_line = yylloc.last_line = 1;
2162 yylloc.first_column = yylloc.last_column = 0;
2163 return yyparse ();
2164@}
9edcd895 2165@end group
342b8b6e
AD
2166@end example
2167
9edcd895
AD
2168Remember that computing locations is not a matter of syntax. Every
2169character must be associated to a location update, whether it is in
2170valid input, in comments, in literal strings, and so on.
342b8b6e
AD
2171
2172@node Multi-function Calc
bfa74976
RS
2173@section Multi-Function Calculator: @code{mfcalc}
2174@cindex multi-function calculator
2175@cindex @code{mfcalc}
2176@cindex calculator, multi-function
2177
2178Now that the basics of Bison have been discussed, it is time to move on to
2179a more advanced problem. The above calculators provided only five
2180functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2181be nice to have a calculator that provides other mathematical functions such
2182as @code{sin}, @code{cos}, etc.
2183
2184It is easy to add new operators to the infix calculator as long as they are
2185only single-character literals. The lexical analyzer @code{yylex} passes
9d9b8b70 2186back all nonnumeric characters as tokens, so new grammar rules suffice for
bfa74976
RS
2187adding a new operator. But we want something more flexible: built-in
2188functions whose syntax has this form:
2189
2190@example
2191@var{function_name} (@var{argument})
2192@end example
2193
2194@noindent
2195At the same time, we will add memory to the calculator, by allowing you
2196to create named variables, store values in them, and use them later.
2197Here is a sample session with the multi-function calculator:
2198
2199@example
9edcd895
AD
2200$ @kbd{mfcalc}
2201@kbd{pi = 3.141592653589}
bfa74976 22023.1415926536
9edcd895 2203@kbd{sin(pi)}
bfa74976 22040.0000000000
9edcd895 2205@kbd{alpha = beta1 = 2.3}
bfa74976 22062.3000000000
9edcd895 2207@kbd{alpha}
bfa74976 22082.3000000000
9edcd895 2209@kbd{ln(alpha)}
bfa74976 22100.8329091229
9edcd895 2211@kbd{exp(ln(beta1))}
bfa74976 22122.3000000000
9edcd895 2213$
bfa74976
RS
2214@end example
2215
2216Note that multiple assignment and nested function calls are permitted.
2217
2218@menu
2219* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
2220* Rules: Mfcalc Rules. Grammar rules for the calculator.
2221* Symtab: Mfcalc Symtab. Symbol table management subroutines.
2222@end menu
2223
342b8b6e 2224@node Mfcalc Decl
bfa74976
RS
2225@subsection Declarations for @code{mfcalc}
2226
2227Here are the C and Bison declarations for the multi-function calculator.
2228
2229@smallexample
18b519c0 2230@group
bfa74976 2231%@{
38a92d50
PE
2232 #include <math.h> /* For math functions, cos(), sin(), etc. */
2233 #include "calc.h" /* Contains definition of `symrec'. */
2234 int yylex (void);
2235 void yyerror (char const *);
bfa74976 2236%@}
18b519c0
AD
2237@end group
2238@group
bfa74976 2239%union @{
38a92d50
PE
2240 double val; /* For returning numbers. */
2241 symrec *tptr; /* For returning symbol-table pointers. */
bfa74976 2242@}
18b519c0 2243@end group
38a92d50
PE
2244%token <val> NUM /* Simple double precision number. */
2245%token <tptr> VAR FNCT /* Variable and Function. */
bfa74976
RS
2246%type <val> exp
2247
18b519c0 2248@group
bfa74976
RS
2249%right '='
2250%left '-' '+'
2251%left '*' '/'
38a92d50
PE
2252%left NEG /* negation--unary minus */
2253%right '^' /* exponentiation */
18b519c0 2254@end group
38a92d50 2255%% /* The grammar follows. */
bfa74976
RS
2256@end smallexample
2257
2258The above grammar introduces only two new features of the Bison language.
2259These features allow semantic values to have various data types
2260(@pxref{Multiple Types, ,More Than One Value Type}).
2261
2262The @code{%union} declaration specifies the entire list of possible types;
2263this is instead of defining @code{YYSTYPE}. The allowable types are now
2264double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2265the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2266
2267Since values can now have various types, it is necessary to associate a
2268type with each grammar symbol whose semantic value is used. These symbols
2269are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2270declarations are augmented with information about their data type (placed
2271between angle brackets).
2272
704a47c4
AD
2273The Bison construct @code{%type} is used for declaring nonterminal
2274symbols, just as @code{%token} is used for declaring token types. We
2275have not used @code{%type} before because nonterminal symbols are
2276normally declared implicitly by the rules that define them. But
2277@code{exp} must be declared explicitly so we can specify its value type.
2278@xref{Type Decl, ,Nonterminal Symbols}.
bfa74976 2279
342b8b6e 2280@node Mfcalc Rules
bfa74976
RS
2281@subsection Grammar Rules for @code{mfcalc}
2282
2283Here are the grammar rules for the multi-function calculator.
2284Most of them are copied directly from @code{calc}; three rules,
2285those which mention @code{VAR} or @code{FNCT}, are new.
2286
2287@smallexample
18b519c0 2288@group
bfa74976
RS
2289input: /* empty */
2290 | input line
2291;
18b519c0 2292@end group
bfa74976 2293
18b519c0 2294@group
bfa74976
RS
2295line:
2296 '\n'
2297 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2298 | error '\n' @{ yyerrok; @}
2299;
18b519c0 2300@end group
bfa74976 2301
18b519c0 2302@group
bfa74976
RS
2303exp: NUM @{ $$ = $1; @}
2304 | VAR @{ $$ = $1->value.var; @}
2305 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2306 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2307 | exp '+' exp @{ $$ = $1 + $3; @}
2308 | exp '-' exp @{ $$ = $1 - $3; @}
2309 | exp '*' exp @{ $$ = $1 * $3; @}
2310 | exp '/' exp @{ $$ = $1 / $3; @}
2311 | '-' exp %prec NEG @{ $$ = -$2; @}
2312 | exp '^' exp @{ $$ = pow ($1, $3); @}
2313 | '(' exp ')' @{ $$ = $2; @}
2314;
18b519c0 2315@end group
38a92d50 2316/* End of grammar. */
bfa74976
RS
2317%%
2318@end smallexample
2319
342b8b6e 2320@node Mfcalc Symtab
bfa74976
RS
2321@subsection The @code{mfcalc} Symbol Table
2322@cindex symbol table example
2323
2324The multi-function calculator requires a symbol table to keep track of the
2325names and meanings of variables and functions. This doesn't affect the
2326grammar rules (except for the actions) or the Bison declarations, but it
2327requires some additional C functions for support.
2328
2329The symbol table itself consists of a linked list of records. Its
2330definition, which is kept in the header @file{calc.h}, is as follows. It
2331provides for either functions or variables to be placed in the table.
2332
2333@smallexample
2334@group
38a92d50 2335/* Function type. */
32dfccf8 2336typedef double (*func_t) (double);
72f889cc 2337@end group
32dfccf8 2338
72f889cc 2339@group
38a92d50 2340/* Data type for links in the chain of symbols. */
bfa74976
RS
2341struct symrec
2342@{
38a92d50 2343 char *name; /* name of symbol */
bfa74976 2344 int type; /* type of symbol: either VAR or FNCT */
32dfccf8
AD
2345 union
2346 @{
38a92d50
PE
2347 double var; /* value of a VAR */
2348 func_t fnctptr; /* value of a FNCT */
bfa74976 2349 @} value;
38a92d50 2350 struct symrec *next; /* link field */
bfa74976
RS
2351@};
2352@end group
2353
2354@group
2355typedef struct symrec symrec;
2356
38a92d50 2357/* The symbol table: a chain of `struct symrec'. */
bfa74976
RS
2358extern symrec *sym_table;
2359
a730d142 2360symrec *putsym (char const *, int);
38a92d50 2361symrec *getsym (char const *);
bfa74976
RS
2362@end group
2363@end smallexample
2364
2365The new version of @code{main} includes a call to @code{init_table}, a
2366function that initializes the symbol table. Here it is, and
2367@code{init_table} as well:
2368
2369@smallexample
bfa74976
RS
2370#include <stdio.h>
2371
18b519c0 2372@group
38a92d50 2373/* Called by yyparse on error. */
13863333 2374void
38a92d50 2375yyerror (char const *s)
bfa74976
RS
2376@{
2377 printf ("%s\n", s);
2378@}
18b519c0 2379@end group
bfa74976 2380
18b519c0 2381@group
bfa74976
RS
2382struct init
2383@{
38a92d50
PE
2384 char const *fname;
2385 double (*fnct) (double);
bfa74976
RS
2386@};
2387@end group
2388
2389@group
38a92d50 2390struct init const arith_fncts[] =
13863333 2391@{
32dfccf8
AD
2392 "sin", sin,
2393 "cos", cos,
13863333 2394 "atan", atan,
32dfccf8
AD
2395 "ln", log,
2396 "exp", exp,
13863333
AD
2397 "sqrt", sqrt,
2398 0, 0
2399@};
18b519c0 2400@end group
bfa74976 2401
18b519c0 2402@group
bfa74976 2403/* The symbol table: a chain of `struct symrec'. */
38a92d50 2404symrec *sym_table;
bfa74976
RS
2405@end group
2406
2407@group
72d2299c 2408/* Put arithmetic functions in table. */
13863333
AD
2409void
2410init_table (void)
bfa74976
RS
2411@{
2412 int i;
2413 symrec *ptr;
2414 for (i = 0; arith_fncts[i].fname != 0; i++)
2415 @{
2416 ptr = putsym (arith_fncts[i].fname, FNCT);
2417 ptr->value.fnctptr = arith_fncts[i].fnct;
2418 @}
2419@}
2420@end group
38a92d50
PE
2421
2422@group
2423int
2424main (void)
2425@{
2426 init_table ();
2427 return yyparse ();
2428@}
2429@end group
bfa74976
RS
2430@end smallexample
2431
2432By simply editing the initialization list and adding the necessary include
2433files, you can add additional functions to the calculator.
2434
2435Two important functions allow look-up and installation of symbols in the
2436symbol table. The function @code{putsym} is passed a name and the type
2437(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2438linked to the front of the list, and a pointer to the object is returned.
2439The function @code{getsym} is passed the name of the symbol to look up. If
2440found, a pointer to that symbol is returned; otherwise zero is returned.
2441
2442@smallexample
2443symrec *
38a92d50 2444putsym (char const *sym_name, int sym_type)
bfa74976
RS
2445@{
2446 symrec *ptr;
2447 ptr = (symrec *) malloc (sizeof (symrec));
2448 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2449 strcpy (ptr->name,sym_name);
2450 ptr->type = sym_type;
72d2299c 2451 ptr->value.var = 0; /* Set value to 0 even if fctn. */
bfa74976
RS
2452 ptr->next = (struct symrec *)sym_table;
2453 sym_table = ptr;
2454 return ptr;
2455@}
2456
2457symrec *
38a92d50 2458getsym (char const *sym_name)
bfa74976
RS
2459@{
2460 symrec *ptr;
2461 for (ptr = sym_table; ptr != (symrec *) 0;
2462 ptr = (symrec *)ptr->next)
2463 if (strcmp (ptr->name,sym_name) == 0)
2464 return ptr;
2465 return 0;
2466@}
2467@end smallexample
2468
2469The function @code{yylex} must now recognize variables, numeric values, and
2470the single-character arithmetic operators. Strings of alphanumeric
9d9b8b70 2471characters with a leading letter are recognized as either variables or
bfa74976
RS
2472functions depending on what the symbol table says about them.
2473
2474The string is passed to @code{getsym} for look up in the symbol table. If
2475the name appears in the table, a pointer to its location and its type
2476(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2477already in the table, then it is installed as a @code{VAR} using
2478@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
e0c471a9 2479returned to @code{yyparse}.
bfa74976
RS
2480
2481No change is needed in the handling of numeric values and arithmetic
2482operators in @code{yylex}.
2483
2484@smallexample
2485@group
2486#include <ctype.h>
18b519c0 2487@end group
13863333 2488
18b519c0 2489@group
13863333
AD
2490int
2491yylex (void)
bfa74976
RS
2492@{
2493 int c;
2494
72d2299c 2495 /* Ignore white space, get first nonwhite character. */
bfa74976
RS
2496 while ((c = getchar ()) == ' ' || c == '\t');
2497
2498 if (c == EOF)
2499 return 0;
2500@end group
2501
2502@group
2503 /* Char starts a number => parse the number. */
2504 if (c == '.' || isdigit (c))
2505 @{
2506 ungetc (c, stdin);
2507 scanf ("%lf", &yylval.val);
2508 return NUM;
2509 @}
2510@end group
2511
2512@group
2513 /* Char starts an identifier => read the name. */
2514 if (isalpha (c))
2515 @{
2516 symrec *s;
2517 static char *symbuf = 0;
2518 static int length = 0;
2519 int i;
2520@end group
2521
2522@group
2523 /* Initially make the buffer long enough
2524 for a 40-character symbol name. */
2525 if (length == 0)
2526 length = 40, symbuf = (char *)malloc (length + 1);
2527
2528 i = 0;
2529 do
2530@end group
2531@group
2532 @{
2533 /* If buffer is full, make it bigger. */
2534 if (i == length)
2535 @{
2536 length *= 2;
18b519c0 2537 symbuf = (char *) realloc (symbuf, length + 1);
bfa74976
RS
2538 @}
2539 /* Add this character to the buffer. */
2540 symbuf[i++] = c;
2541 /* Get another character. */
2542 c = getchar ();
2543 @}
2544@end group
2545@group
72d2299c 2546 while (isalnum (c));
bfa74976
RS
2547
2548 ungetc (c, stdin);
2549 symbuf[i] = '\0';
2550@end group
2551
2552@group
2553 s = getsym (symbuf);
2554 if (s == 0)
2555 s = putsym (symbuf, VAR);
2556 yylval.tptr = s;
2557 return s->type;
2558 @}
2559
2560 /* Any other character is a token by itself. */
2561 return c;
2562@}
2563@end group
2564@end smallexample
2565
72d2299c 2566This program is both powerful and flexible. You may easily add new
704a47c4
AD
2567functions, and it is a simple job to modify this code to install
2568predefined variables such as @code{pi} or @code{e} as well.
bfa74976 2569
342b8b6e 2570@node Exercises
bfa74976
RS
2571@section Exercises
2572@cindex exercises
2573
2574@enumerate
2575@item
2576Add some new functions from @file{math.h} to the initialization list.
2577
2578@item
2579Add another array that contains constants and their values. Then
2580modify @code{init_table} to add these constants to the symbol table.
2581It will be easiest to give the constants type @code{VAR}.
2582
2583@item
2584Make the program report an error if the user refers to an
2585uninitialized variable in any way except to store a value in it.
2586@end enumerate
2587
342b8b6e 2588@node Grammar File
bfa74976
RS
2589@chapter Bison Grammar Files
2590
2591Bison takes as input a context-free grammar specification and produces a
2592C-language function that recognizes correct instances of the grammar.
2593
2594The Bison grammar input file conventionally has a name ending in @samp{.y}.
234a3be3 2595@xref{Invocation, ,Invoking Bison}.
bfa74976
RS
2596
2597@menu
2598* Grammar Outline:: Overall layout of the grammar file.
2599* Symbols:: Terminal and nonterminal symbols.
2600* Rules:: How to write grammar rules.
2601* Recursion:: Writing recursive rules.
2602* Semantics:: Semantic values and actions.
847bf1f5 2603* Locations:: Locations and actions.
bfa74976
RS
2604* Declarations:: All kinds of Bison declarations are described here.
2605* Multiple Parsers:: Putting more than one Bison parser in one program.
2606@end menu
2607
342b8b6e 2608@node Grammar Outline
bfa74976
RS
2609@section Outline of a Bison Grammar
2610
2611A Bison grammar file has four main sections, shown here with the
2612appropriate delimiters:
2613
2614@example
2615%@{
38a92d50 2616 @var{Prologue}
bfa74976
RS
2617%@}
2618
2619@var{Bison declarations}
2620
2621%%
2622@var{Grammar rules}
2623%%
2624
75f5aaea 2625@var{Epilogue}
bfa74976
RS
2626@end example
2627
2628Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2bfc2e2a
PE
2629As a @acronym{GNU} extension, @samp{//} introduces a comment that
2630continues until end of line.
bfa74976
RS
2631
2632@menu
75f5aaea 2633* Prologue:: Syntax and usage of the prologue.
2cbe6b7f 2634* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
bfa74976
RS
2635* Bison Declarations:: Syntax and usage of the Bison declarations section.
2636* Grammar Rules:: Syntax and usage of the grammar rules section.
75f5aaea 2637* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
2638@end menu
2639
38a92d50 2640@node Prologue
75f5aaea
MA
2641@subsection The prologue
2642@cindex declarations section
2643@cindex Prologue
2644@cindex declarations
bfa74976 2645
f8e1c9e5
AD
2646The @var{Prologue} section contains macro definitions and declarations
2647of functions and variables that are used in the actions in the grammar
2648rules. These are copied to the beginning of the parser file so that
2649they precede the definition of @code{yyparse}. You can use
2650@samp{#include} to get the declarations from a header file. If you
2651don't need any C declarations, you may omit the @samp{%@{} and
2652@samp{%@}} delimiters that bracket this section.
bfa74976 2653
9c437126 2654The @var{Prologue} section is terminated by the first occurrence
287c78f6
PE
2655of @samp{%@}} that is outside a comment, a string literal, or a
2656character constant.
2657
c732d2c6
AD
2658You may have more than one @var{Prologue} section, intermixed with the
2659@var{Bison declarations}. This allows you to have C and Bison
2660declarations that refer to each other. For example, the @code{%union}
2661declaration may use types defined in a header file, and you may wish to
2662prototype functions that take arguments of type @code{YYSTYPE}. This
2663can be done with two @var{Prologue} blocks, one before and one after the
2664@code{%union} declaration.
2665
2666@smallexample
2667%@{
aef3da86 2668 #define _GNU_SOURCE
38a92d50
PE
2669 #include <stdio.h>
2670 #include "ptypes.h"
c732d2c6
AD
2671%@}
2672
2673%union @{
779e7ceb 2674 long int n;
c732d2c6
AD
2675 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2676@}
2677
2678%@{
38a92d50
PE
2679 static void print_token_value (FILE *, int, YYSTYPE);
2680 #define YYPRINT(F, N, L) print_token_value (F, N, L)
c732d2c6
AD
2681%@}
2682
2683@dots{}
2684@end smallexample
2685
aef3da86
PE
2686When in doubt, it is usually safer to put prologue code before all
2687Bison declarations, rather than after. For example, any definitions
2688of feature test macros like @code{_GNU_SOURCE} or
2689@code{_POSIX_C_SOURCE} should appear before all Bison declarations, as
2690feature test macros can affect the behavior of Bison-generated
2691@code{#include} directives.
2692
2cbe6b7f
JD
2693@node Prologue Alternatives
2694@subsection Prologue Alternatives
2695@cindex Prologue Alternatives
2696
136a0f76 2697@findex %code
16dc6a9e
JD
2698@findex %code requires
2699@findex %code provides
2700@findex %code top
85894313
JD
2701(The prologue alternatives described here are experimental.
2702More user feedback will help to determine whether they should become permanent
2703features.)
2704
2cbe6b7f
JD
2705The functionality of @var{Prologue} sections can often be subtle and
2706inflexible.
8e0a5e9e
JD
2707As an alternative, Bison provides a %code directive with an explicit qualifier
2708field, which identifies the purpose of the code and thus the location(s) where
2709Bison should generate it.
2710For C/C++, the qualifier can be omitted for the default location, or it can be
8405b70c 2711one of @code{requires}, @code{provides}, @code{top}.
148d66d8 2712@xref{Decl Summary,,%code}.
2cbe6b7f
JD
2713
2714Look again at the example of the previous section:
2715
2716@smallexample
2717%@{
2718 #define _GNU_SOURCE
2719 #include <stdio.h>
2720 #include "ptypes.h"
2721%@}
2722
2723%union @{
2724 long int n;
2725 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2726@}
2727
2728%@{
2729 static void print_token_value (FILE *, int, YYSTYPE);
2730 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2731%@}
2732
2733@dots{}
2734@end smallexample
2735
2736@noindent
2737Notice that there are two @var{Prologue} sections here, but there's a subtle
2738distinction between their functionality.
2739For example, if you decide to override Bison's default definition for
2740@code{YYLTYPE}, in which @var{Prologue} section should you write your new
2741definition?
2742You should write it in the first since Bison will insert that code into the
8e0a5e9e 2743parser source code file @emph{before} the default @code{YYLTYPE} definition.
2cbe6b7f
JD
2744In which @var{Prologue} section should you prototype an internal function,
2745@code{trace_token}, that accepts @code{YYLTYPE} and @code{yytokentype} as
2746arguments?
2747You should prototype it in the second since Bison will insert that code
2748@emph{after} the @code{YYLTYPE} and @code{yytokentype} definitions.
2749
2750This distinction in functionality between the two @var{Prologue} sections is
2751established by the appearance of the @code{%union} between them.
a501eca9 2752This behavior raises a few questions.
2cbe6b7f
JD
2753First, why should the position of a @code{%union} affect definitions related to
2754@code{YYLTYPE} and @code{yytokentype}?
2755Second, what if there is no @code{%union}?
2756In that case, the second kind of @var{Prologue} section is not available.
2757This behavior is not intuitive.
2758
8e0a5e9e 2759To avoid this subtle @code{%union} dependency, rewrite the example using a
16dc6a9e 2760@code{%code top} and an unqualified @code{%code}.
2cbe6b7f
JD
2761Let's go ahead and add the new @code{YYLTYPE} definition and the
2762@code{trace_token} prototype at the same time:
2763
2764@smallexample
16dc6a9e 2765%code top @{
2cbe6b7f
JD
2766 #define _GNU_SOURCE
2767 #include <stdio.h>
8e0a5e9e
JD
2768
2769 /* WARNING: The following code really belongs
16dc6a9e 2770 * in a `%code requires'; see below. */
8e0a5e9e 2771
2cbe6b7f
JD
2772 #include "ptypes.h"
2773 #define YYLTYPE YYLTYPE
2774 typedef struct YYLTYPE
2775 @{
2776 int first_line;
2777 int first_column;
2778 int last_line;
2779 int last_column;
2780 char *filename;
2781 @} YYLTYPE;
2782@}
2783
2784%union @{
2785 long int n;
2786 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2787@}
2788
2789%code @{
2790 static void print_token_value (FILE *, int, YYSTYPE);
2791 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2792 static void trace_token (enum yytokentype token, YYLTYPE loc);
2793@}
2794
2795@dots{}
2796@end smallexample
2797
2798@noindent
16dc6a9e
JD
2799In this way, @code{%code top} and the unqualified @code{%code} achieve the same
2800functionality as the two kinds of @var{Prologue} sections, but it's always
8e0a5e9e 2801explicit which kind you intend.
2cbe6b7f
JD
2802Moreover, both kinds are always available even in the absence of @code{%union}.
2803
16dc6a9e 2804The @code{%code top} block above logically contains two parts.
8e0a5e9e
JD
2805The first two lines before the warning need to appear near the top of the
2806parser source code file.
2807The first line after the warning is required by @code{YYSTYPE} and thus also
2808needs to appear in the parser source code file.
2cbe6b7f 2809However, if you've instructed Bison to generate a parser header file
148d66d8
JD
2810(@pxref{Decl Summary, ,%defines}), you probably want that line to appear before
2811the @code{YYSTYPE} definition in that header file as well.
8e0a5e9e 2812The @code{YYLTYPE} definition should also appear in the parser header file to
2cbe6b7f
JD
2813override the default @code{YYLTYPE} definition there.
2814
16dc6a9e 2815In other words, in the @code{%code top} block above, all but the first two
8e0a5e9e
JD
2816lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
2817definitions.
16dc6a9e 2818Thus, they belong in one or more @code{%code requires}:
9bc0dd67
JD
2819
2820@smallexample
16dc6a9e 2821%code top @{
2cbe6b7f
JD
2822 #define _GNU_SOURCE
2823 #include <stdio.h>
2824@}
2825
16dc6a9e 2826%code requires @{
9bc0dd67
JD
2827 #include "ptypes.h"
2828@}
2829%union @{
2830 long int n;
2831 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2832@}
2833
16dc6a9e 2834%code requires @{
2cbe6b7f
JD
2835 #define YYLTYPE YYLTYPE
2836 typedef struct YYLTYPE
2837 @{
2838 int first_line;
2839 int first_column;
2840 int last_line;
2841 int last_column;
2842 char *filename;
2843 @} YYLTYPE;
2844@}
2845
136a0f76 2846%code @{
2cbe6b7f
JD
2847 static void print_token_value (FILE *, int, YYSTYPE);
2848 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2849 static void trace_token (enum yytokentype token, YYLTYPE loc);
2850@}
2851
2852@dots{}
2853@end smallexample
2854
2855@noindent
2856Now Bison will insert @code{#include "ptypes.h"} and the new @code{YYLTYPE}
2857definition before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
8e0a5e9e 2858definitions in both the parser source code file and the parser header file.
16dc6a9e 2859(By the same reasoning, @code{%code requires} would also be the appropriate
8e0a5e9e 2860place to write your own definition for @code{YYSTYPE}.)
2cbe6b7f 2861
a501eca9 2862When you are writing dependency code for @code{YYSTYPE} and @code{YYLTYPE}, you
16dc6a9e
JD
2863should prefer @code{%code requires} over @code{%code top} regardless of whether
2864you instruct Bison to generate a parser header file.
a501eca9 2865When you are writing code that you need Bison to insert only into the parser
8e0a5e9e 2866source code file and that has no special need to appear at the top of that
16dc6a9e 2867file, you should prefer the unqualified @code{%code} over @code{%code top}.
a501eca9
JD
2868These practices will make the purpose of each block of your code explicit to
2869Bison and to other developers reading your grammar file.
8e0a5e9e 2870Following these practices, we expect the unqualified @code{%code} and
16dc6a9e
JD
2871@code{%code requires} to be the most important of the four @var{Prologue}
2872alternatives.
a501eca9 2873
2cbe6b7f
JD
2874At some point while developing your parser, you might decide to provide
2875@code{trace_token} to modules that are external to your parser.
2876Thus, you might wish for Bison to insert the prototype into both the parser
8e0a5e9e
JD
2877header file and the parser source code file.
2878Since this function is not a dependency required by @code{YYSTYPE} or
2879@code{YYLTYPE}, it doesn't make sense to move its prototype to a
16dc6a9e 2880@code{%code requires}.
2cbe6b7f 2881More importantly, since it depends upon @code{YYLTYPE} and @code{yytokentype},
16dc6a9e 2882@code{%code requires} is not sufficient.
8e0a5e9e 2883Instead, move its prototype from the unqualified @code{%code} to a
16dc6a9e 2884@code{%code provides}:
2cbe6b7f
JD
2885
2886@smallexample
16dc6a9e 2887%code top @{
2cbe6b7f 2888 #define _GNU_SOURCE
136a0f76 2889 #include <stdio.h>
2cbe6b7f 2890@}
136a0f76 2891
16dc6a9e 2892%code requires @{
2cbe6b7f
JD
2893 #include "ptypes.h"
2894@}
2895%union @{
2896 long int n;
2897 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2898@}
2899
16dc6a9e 2900%code requires @{
2cbe6b7f
JD
2901 #define YYLTYPE YYLTYPE
2902 typedef struct YYLTYPE
2903 @{
2904 int first_line;
2905 int first_column;
2906 int last_line;
2907 int last_column;
2908 char *filename;
2909 @} YYLTYPE;
2910@}
2911
16dc6a9e 2912%code provides @{
2cbe6b7f
JD
2913 void trace_token (enum yytokentype token, YYLTYPE loc);
2914@}
2915
2916%code @{
9bc0dd67
JD
2917 static void print_token_value (FILE *, int, YYSTYPE);
2918 #define YYPRINT(F, N, L) print_token_value (F, N, L)
34f98f46 2919@}
9bc0dd67
JD
2920
2921@dots{}
2922@end smallexample
2923
2cbe6b7f
JD
2924@noindent
2925Bison will insert the @code{trace_token} prototype into both the parser header
8e0a5e9e
JD
2926file and the parser source code file after the definitions for
2927@code{yytokentype}, @code{YYLTYPE}, and @code{YYSTYPE}.
2cbe6b7f
JD
2928
2929The above examples are careful to write directives in an order that reflects
8e0a5e9e 2930the layout of the generated parser source code and header files:
16dc6a9e 2931@code{%code top}, @code{%code requires}, @code{%code provides}, and then
8e0a5e9e 2932@code{%code}.
a501eca9 2933While your grammar files may generally be easier to read if you also follow
2cbe6b7f
JD
2934this order, Bison does not require it.
2935Instead, Bison lets you choose an organization that makes sense to you.
2936
a501eca9 2937You may declare any of these directives multiple times in the grammar file.
2cbe6b7f
JD
2938In that case, Bison concatenates the contained code in declaration order.
2939This is the only way in which the position of one of these directives within
2940the grammar file affects its functionality.
2941
2942The result of the previous two properties is greater flexibility in how you may
2943organize your grammar file.
2944For example, you may organize semantic-type-related directives by semantic
2945type:
2946
2947@smallexample
16dc6a9e 2948%code requires @{ #include "type1.h" @}
2cbe6b7f
JD
2949%union @{ type1 field1; @}
2950%destructor @{ type1_free ($$); @} <field1>
2951%printer @{ type1_print ($$); @} <field1>
2952
16dc6a9e 2953%code requires @{ #include "type2.h" @}
2cbe6b7f
JD
2954%union @{ type2 field2; @}
2955%destructor @{ type2_free ($$); @} <field2>
2956%printer @{ type2_print ($$); @} <field2>
2957@end smallexample
2958
2959@noindent
2960You could even place each of the above directive groups in the rules section of
2961the grammar file next to the set of rules that uses the associated semantic
2962type.
61fee93e
JD
2963(In the rules section, you must terminate each of those directives with a
2964semicolon.)
2cbe6b7f
JD
2965And you don't have to worry that some directive (like a @code{%union}) in the
2966definitions section is going to adversely affect their functionality in some
2967counter-intuitive manner just because it comes first.
2968Such an organization is not possible using @var{Prologue} sections.
2969
a501eca9 2970This section has been concerned with explaining the advantages of the four
8e0a5e9e 2971@var{Prologue} alternatives over the original Yacc @var{Prologue}.
a501eca9
JD
2972However, in most cases when using these directives, you shouldn't need to
2973think about all the low-level ordering issues discussed here.
2974Instead, you should simply use these directives to label each block of your
2975code according to its purpose and let Bison handle the ordering.
2976@code{%code} is the most generic label.
16dc6a9e
JD
2977Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
2978as needed.
a501eca9 2979
342b8b6e 2980@node Bison Declarations
bfa74976
RS
2981@subsection The Bison Declarations Section
2982@cindex Bison declarations (introduction)
2983@cindex declarations, Bison (introduction)
2984
2985The @var{Bison declarations} section contains declarations that define
2986terminal and nonterminal symbols, specify precedence, and so on.
2987In some simple grammars you may not need any declarations.
2988@xref{Declarations, ,Bison Declarations}.
2989
342b8b6e 2990@node Grammar Rules
bfa74976
RS
2991@subsection The Grammar Rules Section
2992@cindex grammar rules section
2993@cindex rules section for grammar
2994
2995The @dfn{grammar rules} section contains one or more Bison grammar
2996rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
2997
2998There must always be at least one grammar rule, and the first
2999@samp{%%} (which precedes the grammar rules) may never be omitted even
3000if it is the first thing in the file.
3001
38a92d50 3002@node Epilogue
75f5aaea 3003@subsection The epilogue
bfa74976 3004@cindex additional C code section
75f5aaea 3005@cindex epilogue
bfa74976
RS
3006@cindex C code, section for additional
3007
08e49d20
PE
3008The @var{Epilogue} is copied verbatim to the end of the parser file, just as
3009the @var{Prologue} is copied to the beginning. This is the most convenient
342b8b6e
AD
3010place to put anything that you want to have in the parser file but which need
3011not come before the definition of @code{yyparse}. For example, the
38a92d50
PE
3012definitions of @code{yylex} and @code{yyerror} often go here. Because
3013C requires functions to be declared before being used, you often need
3014to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
e4f85c39 3015even if you define them in the Epilogue.
75f5aaea 3016@xref{Interface, ,Parser C-Language Interface}.
bfa74976
RS
3017
3018If the last section is empty, you may omit the @samp{%%} that separates it
3019from the grammar rules.
3020
f8e1c9e5
AD
3021The Bison parser itself contains many macros and identifiers whose names
3022start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3023any such names (except those documented in this manual) in the epilogue
3024of the grammar file.
bfa74976 3025
342b8b6e 3026@node Symbols
bfa74976
RS
3027@section Symbols, Terminal and Nonterminal
3028@cindex nonterminal symbol
3029@cindex terminal symbol
3030@cindex token type
3031@cindex symbol
3032
3033@dfn{Symbols} in Bison grammars represent the grammatical classifications
3034of the language.
3035
3036A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
3037class of syntactically equivalent tokens. You use the symbol in grammar
3038rules to mean that a token in that class is allowed. The symbol is
3039represented in the Bison parser by a numeric code, and the @code{yylex}
f8e1c9e5
AD
3040function returns a token type code to indicate what kind of token has
3041been read. You don't need to know what the code value is; you can use
3042the symbol to stand for it.
bfa74976 3043
f8e1c9e5
AD
3044A @dfn{nonterminal symbol} stands for a class of syntactically
3045equivalent groupings. The symbol name is used in writing grammar rules.
3046By convention, it should be all lower case.
bfa74976
RS
3047
3048Symbol names can contain letters, digits (not at the beginning),
3049underscores and periods. Periods make sense only in nonterminals.
3050
931c7513 3051There are three ways of writing terminal symbols in the grammar:
bfa74976
RS
3052
3053@itemize @bullet
3054@item
3055A @dfn{named token type} is written with an identifier, like an
c827f760 3056identifier in C@. By convention, it should be all upper case. Each
bfa74976
RS
3057such name must be defined with a Bison declaration such as
3058@code{%token}. @xref{Token Decl, ,Token Type Names}.
3059
3060@item
3061@cindex character token
3062@cindex literal token
3063@cindex single-character literal
931c7513
RS
3064A @dfn{character token type} (or @dfn{literal character token}) is
3065written in the grammar using the same syntax used in C for character
3066constants; for example, @code{'+'} is a character token type. A
3067character token type doesn't need to be declared unless you need to
3068specify its semantic value data type (@pxref{Value Type, ,Data Types of
3069Semantic Values}), associativity, or precedence (@pxref{Precedence,
3070,Operator Precedence}).
bfa74976
RS
3071
3072By convention, a character token type is used only to represent a
3073token that consists of that particular character. Thus, the token
3074type @code{'+'} is used to represent the character @samp{+} as a
3075token. Nothing enforces this convention, but if you depart from it,
3076your program will confuse other readers.
3077
3078All the usual escape sequences used in character literals in C can be
3079used in Bison as well, but you must not use the null character as a
72d2299c
PE
3080character literal because its numeric code, zero, signifies
3081end-of-input (@pxref{Calling Convention, ,Calling Convention
2bfc2e2a
PE
3082for @code{yylex}}). Also, unlike standard C, trigraphs have no
3083special meaning in Bison character literals, nor is backslash-newline
3084allowed.
931c7513
RS
3085
3086@item
3087@cindex string token
3088@cindex literal string token
9ecbd125 3089@cindex multicharacter literal
931c7513
RS
3090A @dfn{literal string token} is written like a C string constant; for
3091example, @code{"<="} is a literal string token. A literal string token
3092doesn't need to be declared unless you need to specify its semantic
14ded682 3093value data type (@pxref{Value Type}), associativity, or precedence
931c7513
RS
3094(@pxref{Precedence}).
3095
3096You can associate the literal string token with a symbolic name as an
3097alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
3098Declarations}). If you don't do that, the lexical analyzer has to
3099retrieve the token number for the literal string token from the
3100@code{yytname} table (@pxref{Calling Convention}).
3101
c827f760 3102@strong{Warning}: literal string tokens do not work in Yacc.
931c7513
RS
3103
3104By convention, a literal string token is used only to represent a token
3105that consists of that particular string. Thus, you should use the token
3106type @code{"<="} to represent the string @samp{<=} as a token. Bison
9ecbd125 3107does not enforce this convention, but if you depart from it, people who
931c7513
RS
3108read your program will be confused.
3109
3110All the escape sequences used in string literals in C can be used in
92ac3705
PE
3111Bison as well, except that you must not use a null character within a
3112string literal. Also, unlike Standard C, trigraphs have no special
2bfc2e2a
PE
3113meaning in Bison string literals, nor is backslash-newline allowed. A
3114literal string token must contain two or more characters; for a token
3115containing just one character, use a character token (see above).
bfa74976
RS
3116@end itemize
3117
3118How you choose to write a terminal symbol has no effect on its
3119grammatical meaning. That depends only on where it appears in rules and
3120on when the parser function returns that symbol.
3121
72d2299c
PE
3122The value returned by @code{yylex} is always one of the terminal
3123symbols, except that a zero or negative value signifies end-of-input.
3124Whichever way you write the token type in the grammar rules, you write
3125it the same way in the definition of @code{yylex}. The numeric code
3126for a character token type is simply the positive numeric code of the
3127character, so @code{yylex} can use the identical value to generate the
3128requisite code, though you may need to convert it to @code{unsigned
3129char} to avoid sign-extension on hosts where @code{char} is signed.
3130Each named token type becomes a C macro in
bfa74976 3131the parser file, so @code{yylex} can use the name to stand for the code.
13863333 3132(This is why periods don't make sense in terminal symbols.)
bfa74976
RS
3133@xref{Calling Convention, ,Calling Convention for @code{yylex}}.
3134
3135If @code{yylex} is defined in a separate file, you need to arrange for the
3136token-type macro definitions to be available there. Use the @samp{-d}
3137option when you run Bison, so that it will write these macro definitions
3138into a separate header file @file{@var{name}.tab.h} which you can include
3139in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
3140
72d2299c 3141If you want to write a grammar that is portable to any Standard C
9d9b8b70 3142host, you must use only nonnull character tokens taken from the basic
c827f760 3143execution character set of Standard C@. This set consists of the ten
72d2299c
PE
3144digits, the 52 lower- and upper-case English letters, and the
3145characters in the following C-language string:
3146
3147@example
3148"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3149@end example
3150
f8e1c9e5
AD
3151The @code{yylex} function and Bison must use a consistent character set
3152and encoding for character tokens. For example, if you run Bison in an
3153@acronym{ASCII} environment, but then compile and run the resulting
3154program in an environment that uses an incompatible character set like
3155@acronym{EBCDIC}, the resulting program may not work because the tables
3156generated by Bison will assume @acronym{ASCII} numeric values for
3157character tokens. It is standard practice for software distributions to
3158contain C source files that were generated by Bison in an
3159@acronym{ASCII} environment, so installers on platforms that are
3160incompatible with @acronym{ASCII} must rebuild those files before
3161compiling them.
e966383b 3162
bfa74976
RS
3163The symbol @code{error} is a terminal symbol reserved for error recovery
3164(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
23c5a174
AD
3165In particular, @code{yylex} should never return this value. The default
3166value of the error token is 256, unless you explicitly assigned 256 to
3167one of your tokens with a @code{%token} declaration.
bfa74976 3168
342b8b6e 3169@node Rules
bfa74976
RS
3170@section Syntax of Grammar Rules
3171@cindex rule syntax
3172@cindex grammar rule syntax
3173@cindex syntax of grammar rules
3174
3175A Bison grammar rule has the following general form:
3176
3177@example
e425e872 3178@group
bfa74976
RS
3179@var{result}: @var{components}@dots{}
3180 ;
e425e872 3181@end group
bfa74976
RS
3182@end example
3183
3184@noindent
9ecbd125 3185where @var{result} is the nonterminal symbol that this rule describes,
bfa74976 3186and @var{components} are various terminal and nonterminal symbols that
13863333 3187are put together by this rule (@pxref{Symbols}).
bfa74976
RS
3188
3189For example,
3190
3191@example
3192@group
3193exp: exp '+' exp
3194 ;
3195@end group
3196@end example
3197
3198@noindent
3199says that two groupings of type @code{exp}, with a @samp{+} token in between,
3200can be combined into a larger grouping of type @code{exp}.
3201
72d2299c
PE
3202White space in rules is significant only to separate symbols. You can add
3203extra white space as you wish.
bfa74976
RS
3204
3205Scattered among the components can be @var{actions} that determine
3206the semantics of the rule. An action looks like this:
3207
3208@example
3209@{@var{C statements}@}
3210@end example
3211
3212@noindent
287c78f6
PE
3213@cindex braced code
3214This is an example of @dfn{braced code}, that is, C code surrounded by
3215braces, much like a compound statement in C@. Braced code can contain
3216any sequence of C tokens, so long as its braces are balanced. Bison
3217does not check the braced code for correctness directly; it merely
3218copies the code to the output file, where the C compiler can check it.
3219
3220Within braced code, the balanced-brace count is not affected by braces
3221within comments, string literals, or character constants, but it is
3222affected by the C digraphs @samp{<%} and @samp{%>} that represent
3223braces. At the top level braced code must be terminated by @samp{@}}
3224and not by a digraph. Bison does not look for trigraphs, so if braced
3225code uses trigraphs you should ensure that they do not affect the
3226nesting of braces or the boundaries of comments, string literals, or
3227character constants.
3228
bfa74976
RS
3229Usually there is only one action and it follows the components.
3230@xref{Actions}.
3231
3232@findex |
3233Multiple rules for the same @var{result} can be written separately or can
3234be joined with the vertical-bar character @samp{|} as follows:
3235
bfa74976
RS
3236@example
3237@group
3238@var{result}: @var{rule1-components}@dots{}
3239 | @var{rule2-components}@dots{}
3240 @dots{}
3241 ;
3242@end group
3243@end example
bfa74976
RS
3244
3245@noindent
3246They are still considered distinct rules even when joined in this way.
3247
3248If @var{components} in a rule is empty, it means that @var{result} can
3249match the empty string. For example, here is how to define a
3250comma-separated sequence of zero or more @code{exp} groupings:
3251
3252@example
3253@group
3254expseq: /* empty */
3255 | expseq1
3256 ;
3257@end group
3258
3259@group
3260expseq1: exp
3261 | expseq1 ',' exp
3262 ;
3263@end group
3264@end example
3265
3266@noindent
3267It is customary to write a comment @samp{/* empty */} in each rule
3268with no components.
3269
342b8b6e 3270@node Recursion
bfa74976
RS
3271@section Recursive Rules
3272@cindex recursive rule
3273
f8e1c9e5
AD
3274A rule is called @dfn{recursive} when its @var{result} nonterminal
3275appears also on its right hand side. Nearly all Bison grammars need to
3276use recursion, because that is the only way to define a sequence of any
3277number of a particular thing. Consider this recursive definition of a
9ecbd125 3278comma-separated sequence of one or more expressions:
bfa74976
RS
3279
3280@example
3281@group
3282expseq1: exp
3283 | expseq1 ',' exp
3284 ;
3285@end group
3286@end example
3287
3288@cindex left recursion
3289@cindex right recursion
3290@noindent
3291Since the recursive use of @code{expseq1} is the leftmost symbol in the
3292right hand side, we call this @dfn{left recursion}. By contrast, here
3293the same construct is defined using @dfn{right recursion}:
3294
3295@example
3296@group
3297expseq1: exp
3298 | exp ',' expseq1
3299 ;
3300@end group
3301@end example
3302
3303@noindent
ec3bc396
AD
3304Any kind of sequence can be defined using either left recursion or right
3305recursion, but you should always use left recursion, because it can
3306parse a sequence of any number of elements with bounded stack space.
3307Right recursion uses up space on the Bison stack in proportion to the
3308number of elements in the sequence, because all the elements must be
3309shifted onto the stack before the rule can be applied even once.
3310@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3311of this.
bfa74976
RS
3312
3313@cindex mutual recursion
3314@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3315rule does not appear directly on its right hand side, but does appear
3316in rules for other nonterminals which do appear on its right hand
13863333 3317side.
bfa74976
RS
3318
3319For example:
3320
3321@example
3322@group
3323expr: primary
3324 | primary '+' primary
3325 ;
3326@end group
3327
3328@group
3329primary: constant
3330 | '(' expr ')'
3331 ;
3332@end group
3333@end example
3334
3335@noindent
3336defines two mutually-recursive nonterminals, since each refers to the
3337other.
3338
342b8b6e 3339@node Semantics
bfa74976
RS
3340@section Defining Language Semantics
3341@cindex defining language semantics
13863333 3342@cindex language semantics, defining
bfa74976
RS
3343
3344The grammar rules for a language determine only the syntax. The semantics
3345are determined by the semantic values associated with various tokens and
3346groupings, and by the actions taken when various groupings are recognized.
3347
3348For example, the calculator calculates properly because the value
3349associated with each expression is the proper number; it adds properly
3350because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3351the numbers associated with @var{x} and @var{y}.
3352
3353@menu
3354* Value Type:: Specifying one data type for all semantic values.
3355* Multiple Types:: Specifying several alternative data types.
3356* Actions:: An action is the semantic definition of a grammar rule.
3357* Action Types:: Specifying data types for actions to operate on.
3358* Mid-Rule Actions:: Most actions go at the end of a rule.
3359 This says when, why and how to use the exceptional
3360 action in the middle of a rule.
3361@end menu
3362
342b8b6e 3363@node Value Type
bfa74976
RS
3364@subsection Data Types of Semantic Values
3365@cindex semantic value type
3366@cindex value type, semantic
3367@cindex data types of semantic values
3368@cindex default data type
3369
3370In a simple program it may be sufficient to use the same data type for
3371the semantic values of all language constructs. This was true in the
c827f760 3372@acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
1964ad8c 3373Notation Calculator}).
bfa74976 3374
ddc8ede1
PE
3375Bison normally uses the type @code{int} for semantic values if your
3376program uses the same data type for all language constructs. To
bfa74976
RS
3377specify some other type, define @code{YYSTYPE} as a macro, like this:
3378
3379@example
3380#define YYSTYPE double
3381@end example
3382
3383@noindent
50cce58e
PE
3384@code{YYSTYPE}'s replacement list should be a type name
3385that does not contain parentheses or square brackets.
342b8b6e 3386This macro definition must go in the prologue of the grammar file
75f5aaea 3387(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
bfa74976 3388
342b8b6e 3389@node Multiple Types
bfa74976
RS
3390@subsection More Than One Value Type
3391
3392In most programs, you will need different data types for different kinds
3393of tokens and groupings. For example, a numeric constant may need type
f8e1c9e5
AD
3394@code{int} or @code{long int}, while a string constant needs type
3395@code{char *}, and an identifier might need a pointer to an entry in the
3396symbol table.
bfa74976
RS
3397
3398To use more than one data type for semantic values in one parser, Bison
3399requires you to do two things:
3400
3401@itemize @bullet
3402@item
ddc8ede1 3403Specify the entire collection of possible data types, either by using the
704a47c4 3404@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
ddc8ede1
PE
3405Value Types}), or by using a @code{typedef} or a @code{#define} to
3406define @code{YYSTYPE} to be a union type whose member names are
3407the type tags.
bfa74976
RS
3408
3409@item
14ded682
AD
3410Choose one of those types for each symbol (terminal or nonterminal) for
3411which semantic values are used. This is done for tokens with the
3412@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3413and for groupings with the @code{%type} Bison declaration (@pxref{Type
3414Decl, ,Nonterminal Symbols}).
bfa74976
RS
3415@end itemize
3416
342b8b6e 3417@node Actions
bfa74976
RS
3418@subsection Actions
3419@cindex action
3420@vindex $$
3421@vindex $@var{n}
3422
3423An action accompanies a syntactic rule and contains C code to be executed
3424each time an instance of that rule is recognized. The task of most actions
3425is to compute a semantic value for the grouping built by the rule from the
3426semantic values associated with tokens or smaller groupings.
3427
287c78f6
PE
3428An action consists of braced code containing C statements, and can be
3429placed at any position in the rule;
704a47c4
AD
3430it is executed at that position. Most rules have just one action at the
3431end of the rule, following all the components. Actions in the middle of
3432a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3433Actions, ,Actions in Mid-Rule}).
bfa74976
RS
3434
3435The C code in an action can refer to the semantic values of the components
3436matched by the rule with the construct @code{$@var{n}}, which stands for
3437the value of the @var{n}th component. The semantic value for the grouping
0cc3da3a
PE
3438being constructed is @code{$$}. Bison translates both of these
3439constructs into expressions of the appropriate type when it copies the
3440actions into the parser file. @code{$$} is translated to a modifiable
3441lvalue, so it can be assigned to.
bfa74976
RS
3442
3443Here is a typical example:
3444
3445@example
3446@group
3447exp: @dots{}
3448 | exp '+' exp
3449 @{ $$ = $1 + $3; @}
3450@end group
3451@end example
3452
3453@noindent
3454This rule constructs an @code{exp} from two smaller @code{exp} groupings
3455connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3456refer to the semantic values of the two component @code{exp} groupings,
3457which are the first and third symbols on the right hand side of the rule.
3458The sum is stored into @code{$$} so that it becomes the semantic value of
3459the addition-expression just recognized by the rule. If there were a
3460useful semantic value associated with the @samp{+} token, it could be
e0c471a9 3461referred to as @code{$2}.
bfa74976 3462
3ded9a63
AD
3463Note that the vertical-bar character @samp{|} is really a rule
3464separator, and actions are attached to a single rule. This is a
3465difference with tools like Flex, for which @samp{|} stands for either
3466``or'', or ``the same action as that of the next rule''. In the
3467following example, the action is triggered only when @samp{b} is found:
3468
3469@example
3470@group
3471a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3472@end group
3473@end example
3474
bfa74976
RS
3475@cindex default action
3476If you don't specify an action for a rule, Bison supplies a default:
72f889cc
AD
3477@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3478becomes the value of the whole rule. Of course, the default action is
3479valid only if the two data types match. There is no meaningful default
3480action for an empty rule; every empty rule must have an explicit action
3481unless the rule's value does not matter.
bfa74976
RS
3482
3483@code{$@var{n}} with @var{n} zero or negative is allowed for reference
3484to tokens and groupings on the stack @emph{before} those that match the
3485current rule. This is a very risky practice, and to use it reliably
3486you must be certain of the context in which the rule is applied. Here
3487is a case in which you can use this reliably:
3488
3489@example
3490@group
3491foo: expr bar '+' expr @{ @dots{} @}
3492 | expr bar '-' expr @{ @dots{} @}
3493 ;
3494@end group
3495
3496@group
3497bar: /* empty */
3498 @{ previous_expr = $0; @}
3499 ;
3500@end group
3501@end example
3502
3503As long as @code{bar} is used only in the fashion shown here, @code{$0}
3504always refers to the @code{expr} which precedes @code{bar} in the
3505definition of @code{foo}.
3506
32c29292 3507@vindex yylval
742e4900 3508It is also possible to access the semantic value of the lookahead token, if
32c29292
JD
3509any, from a semantic action.
3510This semantic value is stored in @code{yylval}.
3511@xref{Action Features, ,Special Features for Use in Actions}.
3512
342b8b6e 3513@node Action Types
bfa74976
RS
3514@subsection Data Types of Values in Actions
3515@cindex action data types
3516@cindex data types in actions
3517
3518If you have chosen a single data type for semantic values, the @code{$$}
3519and @code{$@var{n}} constructs always have that data type.
3520
3521If you have used @code{%union} to specify a variety of data types, then you
3522must declare a choice among these types for each terminal or nonterminal
3523symbol that can have a semantic value. Then each time you use @code{$$} or
3524@code{$@var{n}}, its data type is determined by which symbol it refers to
e0c471a9 3525in the rule. In this example,
bfa74976
RS
3526
3527@example
3528@group
3529exp: @dots{}
3530 | exp '+' exp
3531 @{ $$ = $1 + $3; @}
3532@end group
3533@end example
3534
3535@noindent
3536@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3537have the data type declared for the nonterminal symbol @code{exp}. If
3538@code{$2} were used, it would have the data type declared for the
e0c471a9 3539terminal symbol @code{'+'}, whatever that might be.
bfa74976
RS
3540
3541Alternatively, you can specify the data type when you refer to the value,
3542by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3543reference. For example, if you have defined types as shown here:
3544
3545@example
3546@group
3547%union @{
3548 int itype;
3549 double dtype;
3550@}
3551@end group
3552@end example
3553
3554@noindent
3555then you can write @code{$<itype>1} to refer to the first subunit of the
3556rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3557
342b8b6e 3558@node Mid-Rule Actions
bfa74976
RS
3559@subsection Actions in Mid-Rule
3560@cindex actions in mid-rule
3561@cindex mid-rule actions
3562
3563Occasionally it is useful to put an action in the middle of a rule.
3564These actions are written just like usual end-of-rule actions, but they
3565are executed before the parser even recognizes the following components.
3566
3567A mid-rule action may refer to the components preceding it using
3568@code{$@var{n}}, but it may not refer to subsequent components because
3569it is run before they are parsed.
3570
3571The mid-rule action itself counts as one of the components of the rule.
3572This makes a difference when there is another action later in the same rule
3573(and usually there is another at the end): you have to count the actions
3574along with the symbols when working out which number @var{n} to use in
3575@code{$@var{n}}.
3576
3577The mid-rule action can also have a semantic value. The action can set
3578its value with an assignment to @code{$$}, and actions later in the rule
3579can refer to the value using @code{$@var{n}}. Since there is no symbol
3580to name the action, there is no way to declare a data type for the value
fdc6758b
MA
3581in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3582specify a data type each time you refer to this value.
bfa74976
RS
3583
3584There is no way to set the value of the entire rule with a mid-rule
3585action, because assignments to @code{$$} do not have that effect. The
3586only way to set the value for the entire rule is with an ordinary action
3587at the end of the rule.
3588
3589Here is an example from a hypothetical compiler, handling a @code{let}
3590statement that looks like @samp{let (@var{variable}) @var{statement}} and
3591serves to create a variable named @var{variable} temporarily for the
3592duration of @var{statement}. To parse this construct, we must put
3593@var{variable} into the symbol table while @var{statement} is parsed, then
3594remove it afterward. Here is how it is done:
3595
3596@example
3597@group
3598stmt: LET '(' var ')'
3599 @{ $<context>$ = push_context ();
3600 declare_variable ($3); @}
3601 stmt @{ $$ = $6;
3602 pop_context ($<context>5); @}
3603@end group
3604@end example
3605
3606@noindent
3607As soon as @samp{let (@var{variable})} has been recognized, the first
3608action is run. It saves a copy of the current semantic context (the
3609list of accessible variables) as its semantic value, using alternative
3610@code{context} in the data-type union. Then it calls
3611@code{declare_variable} to add the new variable to that list. Once the
3612first action is finished, the embedded statement @code{stmt} can be
3613parsed. Note that the mid-rule action is component number 5, so the
3614@samp{stmt} is component number 6.
3615
3616After the embedded statement is parsed, its semantic value becomes the
3617value of the entire @code{let}-statement. Then the semantic value from the
3618earlier action is used to restore the prior list of variables. This
3619removes the temporary @code{let}-variable from the list so that it won't
3620appear to exist while the rest of the program is parsed.
3621
841a7737
JD
3622@findex %destructor
3623@cindex discarded symbols, mid-rule actions
3624@cindex error recovery, mid-rule actions
3625In the above example, if the parser initiates error recovery (@pxref{Error
3626Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3627it might discard the previous semantic context @code{$<context>5} without
3628restoring it.
3629Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3630Discarded Symbols}).
ec5479ce
JD
3631However, Bison currently provides no means to declare a destructor specific to
3632a particular mid-rule action's semantic value.
841a7737
JD
3633
3634One solution is to bury the mid-rule action inside a nonterminal symbol and to
3635declare a destructor for that symbol:
3636
3637@example
3638@group
3639%type <context> let
3640%destructor @{ pop_context ($$); @} let
3641
3642%%
3643
3644stmt: let stmt
3645 @{ $$ = $2;
3646 pop_context ($1); @}
3647 ;
3648
3649let: LET '(' var ')'
3650 @{ $$ = push_context ();
3651 declare_variable ($3); @}
3652 ;
3653
3654@end group
3655@end example
3656
3657@noindent
3658Note that the action is now at the end of its rule.
3659Any mid-rule action can be converted to an end-of-rule action in this way, and
3660this is what Bison actually does to implement mid-rule actions.
3661
bfa74976
RS
3662Taking action before a rule is completely recognized often leads to
3663conflicts since the parser must commit to a parse in order to execute the
3664action. For example, the following two rules, without mid-rule actions,
3665can coexist in a working parser because the parser can shift the open-brace
3666token and look at what follows before deciding whether there is a
3667declaration or not:
3668
3669@example
3670@group
3671compound: '@{' declarations statements '@}'
3672 | '@{' statements '@}'
3673 ;
3674@end group
3675@end example
3676
3677@noindent
3678But when we add a mid-rule action as follows, the rules become nonfunctional:
3679
3680@example
3681@group
3682compound: @{ prepare_for_local_variables (); @}
3683 '@{' declarations statements '@}'
3684@end group
3685@group
3686 | '@{' statements '@}'
3687 ;
3688@end group
3689@end example
3690
3691@noindent
3692Now the parser is forced to decide whether to run the mid-rule action
3693when it has read no farther than the open-brace. In other words, it
3694must commit to using one rule or the other, without sufficient
3695information to do it correctly. (The open-brace token is what is called
742e4900
JD
3696the @dfn{lookahead} token at this time, since the parser is still
3697deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.)
bfa74976
RS
3698
3699You might think that you could correct the problem by putting identical
3700actions into the two rules, like this:
3701
3702@example
3703@group
3704compound: @{ prepare_for_local_variables (); @}
3705 '@{' declarations statements '@}'
3706 | @{ prepare_for_local_variables (); @}
3707 '@{' statements '@}'
3708 ;
3709@end group
3710@end example
3711
3712@noindent
3713But this does not help, because Bison does not realize that the two actions
3714are identical. (Bison never tries to understand the C code in an action.)
3715
3716If the grammar is such that a declaration can be distinguished from a
3717statement by the first token (which is true in C), then one solution which
3718does work is to put the action after the open-brace, like this:
3719
3720@example
3721@group
3722compound: '@{' @{ prepare_for_local_variables (); @}
3723 declarations statements '@}'
3724 | '@{' statements '@}'
3725 ;
3726@end group
3727@end example
3728
3729@noindent
3730Now the first token of the following declaration or statement,
3731which would in any case tell Bison which rule to use, can still do so.
3732
3733Another solution is to bury the action inside a nonterminal symbol which
3734serves as a subroutine:
3735
3736@example
3737@group
3738subroutine: /* empty */
3739 @{ prepare_for_local_variables (); @}
3740 ;
3741
3742@end group
3743
3744@group
3745compound: subroutine
3746 '@{' declarations statements '@}'
3747 | subroutine
3748 '@{' statements '@}'
3749 ;
3750@end group
3751@end example
3752
3753@noindent
3754Now Bison can execute the action in the rule for @code{subroutine} without
841a7737 3755deciding which rule for @code{compound} it will eventually use.
bfa74976 3756
342b8b6e 3757@node Locations
847bf1f5
AD
3758@section Tracking Locations
3759@cindex location
95923bd6
AD
3760@cindex textual location
3761@cindex location, textual
847bf1f5
AD
3762
3763Though grammar rules and semantic actions are enough to write a fully
72d2299c 3764functional parser, it can be useful to process some additional information,
3e259915
MA
3765especially symbol locations.
3766
704a47c4
AD
3767The way locations are handled is defined by providing a data type, and
3768actions to take when rules are matched.
847bf1f5
AD
3769
3770@menu
3771* Location Type:: Specifying a data type for locations.
3772* Actions and Locations:: Using locations in actions.
3773* Location Default Action:: Defining a general way to compute locations.
3774@end menu
3775
342b8b6e 3776@node Location Type
847bf1f5
AD
3777@subsection Data Type of Locations
3778@cindex data type of locations
3779@cindex default location type
3780
3781Defining a data type for locations is much simpler than for semantic values,
3782since all tokens and groupings always use the same type.
3783
50cce58e
PE
3784You can specify the type of locations by defining a macro called
3785@code{YYLTYPE}, just as you can specify the semantic value type by
ddc8ede1 3786defining a @code{YYSTYPE} macro (@pxref{Value Type}).
847bf1f5
AD
3787When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3788four members:
3789
3790@example
6273355b 3791typedef struct YYLTYPE
847bf1f5
AD
3792@{
3793 int first_line;
3794 int first_column;
3795 int last_line;
3796 int last_column;
6273355b 3797@} YYLTYPE;
847bf1f5
AD
3798@end example
3799
cd48d21d
AD
3800At the beginning of the parsing, Bison initializes all these fields to 1
3801for @code{yylloc}.
3802
342b8b6e 3803@node Actions and Locations
847bf1f5
AD
3804@subsection Actions and Locations
3805@cindex location actions
3806@cindex actions, location
3807@vindex @@$
3808@vindex @@@var{n}
3809
3810Actions are not only useful for defining language semantics, but also for
3811describing the behavior of the output parser with locations.
3812
3813The most obvious way for building locations of syntactic groupings is very
72d2299c 3814similar to the way semantic values are computed. In a given rule, several
847bf1f5
AD
3815constructs can be used to access the locations of the elements being matched.
3816The location of the @var{n}th component of the right hand side is
3817@code{@@@var{n}}, while the location of the left hand side grouping is
3818@code{@@$}.
3819
3e259915 3820Here is a basic example using the default data type for locations:
847bf1f5
AD
3821
3822@example
3823@group
3824exp: @dots{}
3e259915 3825 | exp '/' exp
847bf1f5 3826 @{
3e259915
MA
3827 @@$.first_column = @@1.first_column;
3828 @@$.first_line = @@1.first_line;
847bf1f5
AD
3829 @@$.last_column = @@3.last_column;
3830 @@$.last_line = @@3.last_line;
3e259915
MA
3831 if ($3)
3832 $$ = $1 / $3;
3833 else
3834 @{
3835 $$ = 1;
4e03e201
AD
3836 fprintf (stderr,
3837 "Division by zero, l%d,c%d-l%d,c%d",
3838 @@3.first_line, @@3.first_column,
3839 @@3.last_line, @@3.last_column);
3e259915 3840 @}
847bf1f5
AD
3841 @}
3842@end group
3843@end example
3844
3e259915 3845As for semantic values, there is a default action for locations that is
72d2299c 3846run each time a rule is matched. It sets the beginning of @code{@@$} to the
3e259915 3847beginning of the first symbol, and the end of @code{@@$} to the end of the
79282c6c 3848last symbol.
3e259915 3849
72d2299c 3850With this default action, the location tracking can be fully automatic. The
3e259915
MA
3851example above simply rewrites this way:
3852
3853@example
3854@group
3855exp: @dots{}
3856 | exp '/' exp
3857 @{
3858 if ($3)
3859 $$ = $1 / $3;
3860 else
3861 @{
3862 $$ = 1;
4e03e201
AD
3863 fprintf (stderr,
3864 "Division by zero, l%d,c%d-l%d,c%d",
3865 @@3.first_line, @@3.first_column,
3866 @@3.last_line, @@3.last_column);
3e259915
MA
3867 @}
3868 @}
3869@end group
3870@end example
847bf1f5 3871
32c29292 3872@vindex yylloc
742e4900 3873It is also possible to access the location of the lookahead token, if any,
32c29292
JD
3874from a semantic action.
3875This location is stored in @code{yylloc}.
3876@xref{Action Features, ,Special Features for Use in Actions}.
3877
342b8b6e 3878@node Location Default Action
847bf1f5
AD
3879@subsection Default Action for Locations
3880@vindex YYLLOC_DEFAULT
8710fc41 3881@cindex @acronym{GLR} parsers and @code{YYLLOC_DEFAULT}
847bf1f5 3882
72d2299c 3883Actually, actions are not the best place to compute locations. Since
704a47c4
AD
3884locations are much more general than semantic values, there is room in
3885the output parser to redefine the default action to take for each
72d2299c 3886rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
96b93a3d
PE
3887matched, before the associated action is run. It is also invoked
3888while processing a syntax error, to compute the error's location.
8710fc41
JD
3889Before reporting an unresolvable syntactic ambiguity, a @acronym{GLR}
3890parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
3891of that ambiguity.
847bf1f5 3892
3e259915 3893Most of the time, this macro is general enough to suppress location
79282c6c 3894dedicated code from semantic actions.
847bf1f5 3895
72d2299c 3896The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
96b93a3d 3897the location of the grouping (the result of the computation). When a
766de5eb 3898rule is matched, the second parameter identifies locations of
96b93a3d 3899all right hand side elements of the rule being matched, and the third
8710fc41
JD
3900parameter is the size of the rule's right hand side.
3901When a @acronym{GLR} parser reports an ambiguity, which of multiple candidate
3902right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
3903When processing a syntax error, the second parameter identifies locations
3904of the symbols that were discarded during error processing, and the third
96b93a3d 3905parameter is the number of discarded symbols.
847bf1f5 3906
766de5eb 3907By default, @code{YYLLOC_DEFAULT} is defined this way:
847bf1f5 3908
766de5eb 3909@smallexample
847bf1f5 3910@group
766de5eb
PE
3911# define YYLLOC_DEFAULT(Current, Rhs, N) \
3912 do \
3913 if (N) \
3914 @{ \
3915 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
3916 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
3917 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
3918 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
3919 @} \
3920 else \
3921 @{ \
3922 (Current).first_line = (Current).last_line = \
3923 YYRHSLOC(Rhs, 0).last_line; \
3924 (Current).first_column = (Current).last_column = \
3925 YYRHSLOC(Rhs, 0).last_column; \
3926 @} \
3927 while (0)
847bf1f5 3928@end group
766de5eb 3929@end smallexample
676385e2 3930
766de5eb
PE
3931where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
3932in @var{rhs} when @var{k} is positive, and the location of the symbol
f28ac696 3933just before the reduction when @var{k} and @var{n} are both zero.
676385e2 3934
3e259915 3935When defining @code{YYLLOC_DEFAULT}, you should consider that:
847bf1f5 3936
3e259915 3937@itemize @bullet
79282c6c 3938@item
72d2299c 3939All arguments are free of side-effects. However, only the first one (the
3e259915 3940result) should be modified by @code{YYLLOC_DEFAULT}.
847bf1f5 3941
3e259915 3942@item
766de5eb
PE
3943For consistency with semantic actions, valid indexes within the
3944right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
3945valid index, and it refers to the symbol just before the reduction.
3946During error processing @var{n} is always positive.
0ae99356
PE
3947
3948@item
3949Your macro should parenthesize its arguments, if need be, since the
3950actual arguments may not be surrounded by parentheses. Also, your
3951macro should expand to something that can be used as a single
3952statement when it is followed by a semicolon.
3e259915 3953@end itemize
847bf1f5 3954
342b8b6e 3955@node Declarations
bfa74976
RS
3956@section Bison Declarations
3957@cindex declarations, Bison
3958@cindex Bison declarations
3959
3960The @dfn{Bison declarations} section of a Bison grammar defines the symbols
3961used in formulating the grammar and the data types of semantic values.
3962@xref{Symbols}.
3963
3964All token type names (but not single-character literal tokens such as
3965@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
3966declared if you need to specify which data type to use for the semantic
3967value (@pxref{Multiple Types, ,More Than One Value Type}).
3968
3969The first rule in the file also specifies the start symbol, by default.
3970If you want some other symbol to be the start symbol, you must declare
704a47c4
AD
3971it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
3972Grammars}).
bfa74976
RS
3973
3974@menu
b50d2359 3975* Require Decl:: Requiring a Bison version.
bfa74976
RS
3976* Token Decl:: Declaring terminal symbols.
3977* Precedence Decl:: Declaring terminals with precedence and associativity.
3978* Union Decl:: Declaring the set of all semantic value types.
3979* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 3980* Initial Action Decl:: Code run before parsing starts.
72f889cc 3981* Destructor Decl:: Declaring how symbols are freed.
d6328241 3982* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
3983* Start Decl:: Specifying the start symbol.
3984* Pure Decl:: Requesting a reentrant parser.
9987d1b3 3985* Push Decl:: Requesting a push parser.
bfa74976
RS
3986* Decl Summary:: Table of all Bison declarations.
3987@end menu
3988
b50d2359
AD
3989@node Require Decl
3990@subsection Require a Version of Bison
3991@cindex version requirement
3992@cindex requiring a version of Bison
3993@findex %require
3994
3995You may require the minimum version of Bison to process the grammar. If
9b8a5ce0
AD
3996the requirement is not met, @command{bison} exits with an error (exit
3997status 63).
b50d2359
AD
3998
3999@example
4000%require "@var{version}"
4001@end example
4002
342b8b6e 4003@node Token Decl
bfa74976
RS
4004@subsection Token Type Names
4005@cindex declaring token type names
4006@cindex token type names, declaring
931c7513 4007@cindex declaring literal string tokens
bfa74976
RS
4008@findex %token
4009
4010The basic way to declare a token type name (terminal symbol) is as follows:
4011
4012@example
4013%token @var{name}
4014@end example
4015
4016Bison will convert this into a @code{#define} directive in
4017the parser, so that the function @code{yylex} (if it is in this file)
4018can use the name @var{name} to stand for this token type's code.
4019
14ded682
AD
4020Alternatively, you can use @code{%left}, @code{%right}, or
4021@code{%nonassoc} instead of @code{%token}, if you wish to specify
4022associativity and precedence. @xref{Precedence Decl, ,Operator
4023Precedence}.
bfa74976
RS
4024
4025You can explicitly specify the numeric code for a token type by appending
1452af69
PE
4026a decimal or hexadecimal integer value in the field immediately
4027following the token name:
bfa74976
RS
4028
4029@example
4030%token NUM 300
1452af69 4031%token XNUM 0x12d // a GNU extension
bfa74976
RS
4032@end example
4033
4034@noindent
4035It is generally best, however, to let Bison choose the numeric codes for
4036all token types. Bison will automatically select codes that don't conflict
e966383b 4037with each other or with normal characters.
bfa74976
RS
4038
4039In the event that the stack type is a union, you must augment the
4040@code{%token} or other token declaration to include the data type
704a47c4
AD
4041alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
4042Than One Value Type}).
bfa74976
RS
4043
4044For example:
4045
4046@example
4047@group
4048%union @{ /* define stack type */
4049 double val;
4050 symrec *tptr;
4051@}
4052%token <val> NUM /* define token NUM and its type */
4053@end group
4054@end example
4055
931c7513
RS
4056You can associate a literal string token with a token type name by
4057writing the literal string at the end of a @code{%token}
4058declaration which declares the name. For example:
4059
4060@example
4061%token arrow "=>"
4062@end example
4063
4064@noindent
4065For example, a grammar for the C language might specify these names with
4066equivalent literal string tokens:
4067
4068@example
4069%token <operator> OR "||"
4070%token <operator> LE 134 "<="
4071%left OR "<="
4072@end example
4073
4074@noindent
4075Once you equate the literal string and the token name, you can use them
4076interchangeably in further declarations or the grammar rules. The
4077@code{yylex} function can use the token name or the literal string to
4078obtain the token type code number (@pxref{Calling Convention}).
4079
342b8b6e 4080@node Precedence Decl
bfa74976
RS
4081@subsection Operator Precedence
4082@cindex precedence declarations
4083@cindex declaring operator precedence
4084@cindex operator precedence, declaring
4085
4086Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to
4087declare a token and specify its precedence and associativity, all at
4088once. These are called @dfn{precedence declarations}.
704a47c4
AD
4089@xref{Precedence, ,Operator Precedence}, for general information on
4090operator precedence.
bfa74976 4091
ab7f29f8 4092The syntax of a precedence declaration is nearly the same as that of
bfa74976
RS
4093@code{%token}: either
4094
4095@example
4096%left @var{symbols}@dots{}
4097@end example
4098
4099@noindent
4100or
4101
4102@example
4103%left <@var{type}> @var{symbols}@dots{}
4104@end example
4105
4106And indeed any of these declarations serves the purposes of @code{%token}.
4107But in addition, they specify the associativity and relative precedence for
4108all the @var{symbols}:
4109
4110@itemize @bullet
4111@item
4112The associativity of an operator @var{op} determines how repeated uses
4113of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
4114@var{z}} is parsed by grouping @var{x} with @var{y} first or by
4115grouping @var{y} with @var{z} first. @code{%left} specifies
4116left-associativity (grouping @var{x} with @var{y} first) and
4117@code{%right} specifies right-associativity (grouping @var{y} with
4118@var{z} first). @code{%nonassoc} specifies no associativity, which
4119means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
4120considered a syntax error.
4121
4122@item
4123The precedence of an operator determines how it nests with other operators.
4124All the tokens declared in a single precedence declaration have equal
4125precedence and nest together according to their associativity.
4126When two tokens declared in different precedence declarations associate,
4127the one declared later has the higher precedence and is grouped first.
4128@end itemize
4129
ab7f29f8
JD
4130For backward compatibility, there is a confusing difference between the
4131argument lists of @code{%token} and precedence declarations.
4132Only a @code{%token} can associate a literal string with a token type name.
4133A precedence declaration always interprets a literal string as a reference to a
4134separate token.
4135For example:
4136
4137@example
4138%left OR "<=" // Does not declare an alias.
4139%left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
4140@end example
4141
342b8b6e 4142@node Union Decl
bfa74976
RS
4143@subsection The Collection of Value Types
4144@cindex declaring value types
4145@cindex value types, declaring
4146@findex %union
4147
287c78f6
PE
4148The @code{%union} declaration specifies the entire collection of
4149possible data types for semantic values. The keyword @code{%union} is
4150followed by braced code containing the same thing that goes inside a
4151@code{union} in C@.
bfa74976
RS
4152
4153For example:
4154
4155@example
4156@group
4157%union @{
4158 double val;
4159 symrec *tptr;
4160@}
4161@end group
4162@end example
4163
4164@noindent
4165This says that the two alternative types are @code{double} and @code{symrec
4166*}. They are given names @code{val} and @code{tptr}; these names are used
4167in the @code{%token} and @code{%type} declarations to pick one of the types
4168for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
4169
6273355b
PE
4170As an extension to @acronym{POSIX}, a tag is allowed after the
4171@code{union}. For example:
4172
4173@example
4174@group
4175%union value @{
4176 double val;
4177 symrec *tptr;
4178@}
4179@end group
4180@end example
4181
d6ca7905 4182@noindent
6273355b
PE
4183specifies the union tag @code{value}, so the corresponding C type is
4184@code{union value}. If you do not specify a tag, it defaults to
4185@code{YYSTYPE}.
4186
d6ca7905
PE
4187As another extension to @acronym{POSIX}, you may specify multiple
4188@code{%union} declarations; their contents are concatenated. However,
4189only the first @code{%union} declaration can specify a tag.
4190
6273355b 4191Note that, unlike making a @code{union} declaration in C, you need not write
bfa74976
RS
4192a semicolon after the closing brace.
4193
ddc8ede1
PE
4194Instead of @code{%union}, you can define and use your own union type
4195@code{YYSTYPE} if your grammar contains at least one
4196@samp{<@var{type}>} tag. For example, you can put the following into
4197a header file @file{parser.h}:
4198
4199@example
4200@group
4201union YYSTYPE @{
4202 double val;
4203 symrec *tptr;
4204@};
4205typedef union YYSTYPE YYSTYPE;
4206@end group
4207@end example
4208
4209@noindent
4210and then your grammar can use the following
4211instead of @code{%union}:
4212
4213@example
4214@group
4215%@{
4216#include "parser.h"
4217%@}
4218%type <val> expr
4219%token <tptr> ID
4220@end group
4221@end example
4222
342b8b6e 4223@node Type Decl
bfa74976
RS
4224@subsection Nonterminal Symbols
4225@cindex declaring value types, nonterminals
4226@cindex value types, nonterminals, declaring
4227@findex %type
4228
4229@noindent
4230When you use @code{%union} to specify multiple value types, you must
4231declare the value type of each nonterminal symbol for which values are
4232used. This is done with a @code{%type} declaration, like this:
4233
4234@example
4235%type <@var{type}> @var{nonterminal}@dots{}
4236@end example
4237
4238@noindent
704a47c4
AD
4239Here @var{nonterminal} is the name of a nonterminal symbol, and
4240@var{type} is the name given in the @code{%union} to the alternative
4241that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
4242can give any number of nonterminal symbols in the same @code{%type}
4243declaration, if they have the same value type. Use spaces to separate
4244the symbol names.
bfa74976 4245
931c7513
RS
4246You can also declare the value type of a terminal symbol. To do this,
4247use the same @code{<@var{type}>} construction in a declaration for the
4248terminal symbol. All kinds of token declarations allow
4249@code{<@var{type}>}.
4250
18d192f0
AD
4251@node Initial Action Decl
4252@subsection Performing Actions before Parsing
4253@findex %initial-action
4254
4255Sometimes your parser needs to perform some initializations before
4256parsing. The @code{%initial-action} directive allows for such arbitrary
4257code.
4258
4259@deffn {Directive} %initial-action @{ @var{code} @}
4260@findex %initial-action
287c78f6 4261Declare that the braced @var{code} must be invoked before parsing each time
451364ed 4262@code{yyparse} is called. The @var{code} may use @code{$$} and
742e4900 4263@code{@@$} --- initial value and location of the lookahead --- and the
451364ed 4264@code{%parse-param}.
18d192f0
AD
4265@end deffn
4266
451364ed
AD
4267For instance, if your locations use a file name, you may use
4268
4269@example
48b16bbc 4270%parse-param @{ char const *file_name @};
451364ed
AD
4271%initial-action
4272@{
4626a15d 4273 @@$.initialize (file_name);
451364ed
AD
4274@};
4275@end example
4276
18d192f0 4277
72f889cc
AD
4278@node Destructor Decl
4279@subsection Freeing Discarded Symbols
4280@cindex freeing discarded symbols
4281@findex %destructor
12e35840 4282@findex <*>
3ebecc24 4283@findex <>
a85284cf
AD
4284During error recovery (@pxref{Error Recovery}), symbols already pushed
4285on the stack and tokens coming from the rest of the file are discarded
4286until the parser falls on its feet. If the parser runs out of memory,
9d9b8b70 4287or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
a85284cf
AD
4288symbols on the stack must be discarded. Even if the parser succeeds, it
4289must discard the start symbol.
258b75ca
PE
4290
4291When discarded symbols convey heap based information, this memory is
4292lost. While this behavior can be tolerable for batch parsers, such as
4b367315
AD
4293in traditional compilers, it is unacceptable for programs like shells or
4294protocol implementations that may parse and execute indefinitely.
258b75ca 4295
a85284cf
AD
4296The @code{%destructor} directive defines code that is called when a
4297symbol is automatically discarded.
72f889cc
AD
4298
4299@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
4300@findex %destructor
287c78f6
PE
4301Invoke the braced @var{code} whenever the parser discards one of the
4302@var{symbols}.
4b367315 4303Within @var{code}, @code{$$} designates the semantic value associated
ec5479ce
JD
4304with the discarded symbol, and @code{@@$} designates its location.
4305The additional parser parameters are also available (@pxref{Parser Function, ,
4306The Parser Function @code{yyparse}}).
ec5479ce 4307
b2a0b7ca
JD
4308When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
4309per-symbol @code{%destructor}.
4310You may also define a per-type @code{%destructor} by listing a semantic type
12e35840 4311tag among @var{symbols}.
b2a0b7ca 4312In that case, the parser will invoke this @var{code} whenever it discards any
12e35840 4313grammar symbol that has that semantic type tag unless that symbol has its own
b2a0b7ca
JD
4314per-symbol @code{%destructor}.
4315
12e35840 4316Finally, you can define two different kinds of default @code{%destructor}s.
85894313
JD
4317(These default forms are experimental.
4318More user feedback will help to determine whether they should become permanent
4319features.)
3ebecc24 4320You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
12e35840
JD
4321exactly one @code{%destructor} declaration in your grammar file.
4322The parser will invoke the @var{code} associated with one of these whenever it
4323discards any user-defined grammar symbol that has no per-symbol and no per-type
4324@code{%destructor}.
4325The parser uses the @var{code} for @code{<*>} in the case of such a grammar
4326symbol for which you have formally declared a semantic type tag (@code{%type}
4327counts as such a declaration, but @code{$<tag>$} does not).
3ebecc24 4328The parser uses the @var{code} for @code{<>} in the case of such a grammar
12e35840 4329symbol that has no declared semantic type tag.
72f889cc
AD
4330@end deffn
4331
b2a0b7ca 4332@noindent
12e35840 4333For example:
72f889cc
AD
4334
4335@smallexample
ec5479ce
JD
4336%union @{ char *string; @}
4337%token <string> STRING1
4338%token <string> STRING2
4339%type <string> string1
4340%type <string> string2
b2a0b7ca
JD
4341%union @{ char character; @}
4342%token <character> CHR
4343%type <character> chr
12e35840
JD
4344%token TAGLESS
4345
b2a0b7ca 4346%destructor @{ @} <character>
12e35840
JD
4347%destructor @{ free ($$); @} <*>
4348%destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
3ebecc24 4349%destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
72f889cc
AD
4350@end smallexample
4351
4352@noindent
b2a0b7ca
JD
4353guarantees that, when the parser discards any user-defined symbol that has a
4354semantic type tag other than @code{<character>}, it passes its semantic value
12e35840 4355to @code{free} by default.
ec5479ce
JD
4356However, when the parser discards a @code{STRING1} or a @code{string1}, it also
4357prints its line number to @code{stdout}.
4358It performs only the second @code{%destructor} in this case, so it invokes
4359@code{free} only once.
12e35840
JD
4360Finally, the parser merely prints a message whenever it discards any symbol,
4361such as @code{TAGLESS}, that has no semantic type tag.
4362
4363A Bison-generated parser invokes the default @code{%destructor}s only for
4364user-defined as opposed to Bison-defined symbols.
4365For example, the parser will not invoke either kind of default
4366@code{%destructor} for the special Bison-defined symbols @code{$accept},
4367@code{$undefined}, or @code{$end} (@pxref{Table of Symbols, ,Bison Symbols}),
4368none of which you can reference in your grammar.
4369It also will not invoke either for the @code{error} token (@pxref{Table of
4370Symbols, ,error}), which is always defined by Bison regardless of whether you
4371reference it in your grammar.
4372However, it may invoke one of them for the end token (token 0) if you
4373redefine it from @code{$end} to, for example, @code{END}:
3508ce36
JD
4374
4375@smallexample
4376%token END 0
4377@end smallexample
4378
12e35840
JD
4379@cindex actions in mid-rule
4380@cindex mid-rule actions
4381Finally, Bison will never invoke a @code{%destructor} for an unreferenced
4382mid-rule semantic value (@pxref{Mid-Rule Actions,,Actions in Mid-Rule}).
4383That is, Bison does not consider a mid-rule to have a semantic value if you do
4384not reference @code{$$} in the mid-rule's action or @code{$@var{n}} (where
4385@var{n} is the RHS symbol position of the mid-rule) in any later action in that
4386rule.
4387However, if you do reference either, the Bison-generated parser will invoke the
3ebecc24 4388@code{<>} @code{%destructor} whenever it discards the mid-rule symbol.
12e35840 4389
3508ce36
JD
4390@ignore
4391@noindent
4392In the future, it may be possible to redefine the @code{error} token as a
4393nonterminal that captures the discarded symbols.
4394In that case, the parser will invoke the default destructor for it as well.
4395@end ignore
4396
e757bb10
AD
4397@sp 1
4398
4399@cindex discarded symbols
4400@dfn{Discarded symbols} are the following:
4401
4402@itemize
4403@item
4404stacked symbols popped during the first phase of error recovery,
4405@item
4406incoming terminals during the second phase of error recovery,
4407@item
742e4900 4408the current lookahead and the entire stack (except the current
9d9b8b70 4409right-hand side symbols) when the parser returns immediately, and
258b75ca
PE
4410@item
4411the start symbol, when the parser succeeds.
e757bb10
AD
4412@end itemize
4413
9d9b8b70
PE
4414The parser can @dfn{return immediately} because of an explicit call to
4415@code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
4416exhaustion.
4417
29553547 4418Right-hand side symbols of a rule that explicitly triggers a syntax
9d9b8b70
PE
4419error via @code{YYERROR} are not discarded automatically. As a rule
4420of thumb, destructors are invoked only when user actions cannot manage
a85284cf 4421the memory.
e757bb10 4422
342b8b6e 4423@node Expect Decl
bfa74976
RS
4424@subsection Suppressing Conflict Warnings
4425@cindex suppressing conflict warnings
4426@cindex preventing warnings about conflicts
4427@cindex warnings, preventing
4428@cindex conflicts, suppressing warnings of
4429@findex %expect
d6328241 4430@findex %expect-rr
bfa74976
RS
4431
4432Bison normally warns if there are any conflicts in the grammar
7da99ede
AD
4433(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4434have harmless shift/reduce conflicts which are resolved in a predictable
4435way and would be difficult to eliminate. It is desirable to suppress
4436the warning about these conflicts unless the number of conflicts
4437changes. You can do this with the @code{%expect} declaration.
bfa74976
RS
4438
4439The declaration looks like this:
4440
4441@example
4442%expect @var{n}
4443@end example
4444
035aa4a0
PE
4445Here @var{n} is a decimal integer. The declaration says there should
4446be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4447Bison reports an error if the number of shift/reduce conflicts differs
4448from @var{n}, or if there are any reduce/reduce conflicts.
bfa74976 4449
035aa4a0
PE
4450For normal @acronym{LALR}(1) parsers, reduce/reduce conflicts are more
4451serious, and should be eliminated entirely. Bison will always report
4452reduce/reduce conflicts for these parsers. With @acronym{GLR}
4453parsers, however, both kinds of conflicts are routine; otherwise,
4454there would be no need to use @acronym{GLR} parsing. Therefore, it is
4455also possible to specify an expected number of reduce/reduce conflicts
4456in @acronym{GLR} parsers, using the declaration:
d6328241
PH
4457
4458@example
4459%expect-rr @var{n}
4460@end example
4461
bfa74976
RS
4462In general, using @code{%expect} involves these steps:
4463
4464@itemize @bullet
4465@item
4466Compile your grammar without @code{%expect}. Use the @samp{-v} option
4467to get a verbose list of where the conflicts occur. Bison will also
4468print the number of conflicts.
4469
4470@item
4471Check each of the conflicts to make sure that Bison's default
4472resolution is what you really want. If not, rewrite the grammar and
4473go back to the beginning.
4474
4475@item
4476Add an @code{%expect} declaration, copying the number @var{n} from the
035aa4a0
PE
4477number which Bison printed. With @acronym{GLR} parsers, add an
4478@code{%expect-rr} declaration as well.
bfa74976
RS
4479@end itemize
4480
035aa4a0
PE
4481Now Bison will warn you if you introduce an unexpected conflict, but
4482will keep silent otherwise.
bfa74976 4483
342b8b6e 4484@node Start Decl
bfa74976
RS
4485@subsection The Start-Symbol
4486@cindex declaring the start symbol
4487@cindex start symbol, declaring
4488@cindex default start symbol
4489@findex %start
4490
4491Bison assumes by default that the start symbol for the grammar is the first
4492nonterminal specified in the grammar specification section. The programmer
4493may override this restriction with the @code{%start} declaration as follows:
4494
4495@example
4496%start @var{symbol}
4497@end example
4498
342b8b6e 4499@node Pure Decl
bfa74976
RS
4500@subsection A Pure (Reentrant) Parser
4501@cindex reentrant parser
4502@cindex pure parser
d9df47b6 4503@findex %define api.pure
bfa74976
RS
4504
4505A @dfn{reentrant} program is one which does not alter in the course of
4506execution; in other words, it consists entirely of @dfn{pure} (read-only)
4507code. Reentrancy is important whenever asynchronous execution is possible;
9d9b8b70
PE
4508for example, a nonreentrant program may not be safe to call from a signal
4509handler. In systems with multiple threads of control, a nonreentrant
bfa74976
RS
4510program must be called only within interlocks.
4511
70811b85 4512Normally, Bison generates a parser which is not reentrant. This is
c827f760
PE
4513suitable for most uses, and it permits compatibility with Yacc. (The
4514standard Yacc interfaces are inherently nonreentrant, because they use
70811b85
RS
4515statically allocated variables for communication with @code{yylex},
4516including @code{yylval} and @code{yylloc}.)
bfa74976 4517
70811b85 4518Alternatively, you can generate a pure, reentrant parser. The Bison
d9df47b6 4519declaration @code{%define api.pure} says that you want the parser to be
70811b85 4520reentrant. It looks like this:
bfa74976
RS
4521
4522@example
d9df47b6 4523%define api.pure
bfa74976
RS
4524@end example
4525
70811b85
RS
4526The result is that the communication variables @code{yylval} and
4527@code{yylloc} become local variables in @code{yyparse}, and a different
4528calling convention is used for the lexical analyzer function
4529@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
f4101aa6
AD
4530Parsers}, for the details of this. The variable @code{yynerrs}
4531becomes local in @code{yyparse} in pull mode but it becomes a member
9987d1b3 4532of yypstate in push mode. (@pxref{Error Reporting, ,The Error
70811b85
RS
4533Reporting Function @code{yyerror}}). The convention for calling
4534@code{yyparse} itself is unchanged.
4535
4536Whether the parser is pure has nothing to do with the grammar rules.
4537You can generate either a pure parser or a nonreentrant parser from any
4538valid grammar.
bfa74976 4539
9987d1b3
JD
4540@node Push Decl
4541@subsection A Push Parser
4542@cindex push parser
4543@cindex push parser
c373bf8b 4544@findex %define api.push_pull
9987d1b3 4545
f4101aa6
AD
4546A pull parser is called once and it takes control until all its input
4547is completely parsed. A push parser, on the other hand, is called
9987d1b3
JD
4548each time a new token is made available.
4549
f4101aa6 4550A push parser is typically useful when the parser is part of a
9987d1b3 4551main event loop in the client's application. This is typically
f4101aa6
AD
4552a requirement of a GUI, when the main event loop needs to be triggered
4553within a certain time period.
9987d1b3 4554
d782395d
JD
4555Normally, Bison generates a pull parser.
4556The following Bison declaration says that you want the parser to be a push
c373bf8b 4557parser (@pxref{Decl Summary,,%define api.push_pull}):
9987d1b3
JD
4558
4559@example
c373bf8b 4560%define api.push_pull "push"
9987d1b3
JD
4561@end example
4562
4563In almost all cases, you want to ensure that your push parser is also
4564a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). The only
f4101aa6 4565time you should create an impure push parser is to have backwards
9987d1b3
JD
4566compatibility with the impure Yacc pull mode interface. Unless you know
4567what you are doing, your declarations should look like this:
4568
4569@example
d9df47b6 4570%define api.pure
c373bf8b 4571%define api.push_pull "push"
9987d1b3
JD
4572@end example
4573
f4101aa6
AD
4574There is a major notable functional difference between the pure push parser
4575and the impure push parser. It is acceptable for a pure push parser to have
9987d1b3
JD
4576many parser instances, of the same type of parser, in memory at the same time.
4577An impure push parser should only use one parser at a time.
4578
4579When a push parser is selected, Bison will generate some new symbols in
f4101aa6
AD
4580the generated parser. @code{yypstate} is a structure that the generated
4581parser uses to store the parser's state. @code{yypstate_new} is the
9987d1b3
JD
4582function that will create a new parser instance. @code{yypstate_delete}
4583will free the resources associated with the corresponding parser instance.
f4101aa6 4584Finally, @code{yypush_parse} is the function that should be called whenever a
9987d1b3
JD
4585token is available to provide the parser. A trivial example
4586of using a pure push parser would look like this:
4587
4588@example
4589int status;
4590yypstate *ps = yypstate_new ();
4591do @{
4592 status = yypush_parse (ps, yylex (), NULL);
4593@} while (status == YYPUSH_MORE);
4594yypstate_delete (ps);
4595@end example
4596
4597If the user decided to use an impure push parser, a few things about
f4101aa6 4598the generated parser will change. The @code{yychar} variable becomes
9987d1b3
JD
4599a global variable instead of a variable in the @code{yypush_parse} function.
4600For this reason, the signature of the @code{yypush_parse} function is
f4101aa6 4601changed to remove the token as a parameter. A nonreentrant push parser
9987d1b3
JD
4602example would thus look like this:
4603
4604@example
4605extern int yychar;
4606int status;
4607yypstate *ps = yypstate_new ();
4608do @{
4609 yychar = yylex ();
4610 status = yypush_parse (ps);
4611@} while (status == YYPUSH_MORE);
4612yypstate_delete (ps);
4613@end example
4614
f4101aa6 4615That's it. Notice the next token is put into the global variable @code{yychar}
9987d1b3
JD
4616for use by the next invocation of the @code{yypush_parse} function.
4617
f4101aa6 4618Bison also supports both the push parser interface along with the pull parser
9987d1b3 4619interface in the same generated parser. In order to get this functionality,
f4101aa6 4620you should replace the @code{%define api.push_pull "push"} declaration with the
c373bf8b
JD
4621@code{%define api.push_pull "both"} declaration. Doing this will create all of
4622the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
f4101aa6
AD
4623and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
4624would be used. However, the user should note that it is implemented in the
d782395d
JD
4625generated parser by calling @code{yypull_parse}.
4626This makes the @code{yyparse} function that is generated with the
c373bf8b 4627@code{%define api.push_pull "both"} declaration slower than the normal
d782395d
JD
4628@code{yyparse} function. If the user
4629calls the @code{yypull_parse} function it will parse the rest of the input
f4101aa6
AD
4630stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
4631and then @code{yypull_parse} the rest of the input stream. If you would like
4632to switch back and forth between between parsing styles, you would have to
4633write your own @code{yypull_parse} function that knows when to quit looking
4634for input. An example of using the @code{yypull_parse} function would look
9987d1b3
JD
4635like this:
4636
4637@example
4638yypstate *ps = yypstate_new ();
4639yypull_parse (ps); /* Will call the lexer */
4640yypstate_delete (ps);
4641@end example
4642
d9df47b6 4643Adding the @code{%define api.pure} declaration does exactly the same thing to
f4101aa6 4644the generated parser with @code{%define api.push_pull "both"} as it did for
c373bf8b 4645@code{%define api.push_pull "push"}.
9987d1b3 4646
342b8b6e 4647@node Decl Summary
bfa74976
RS
4648@subsection Bison Declaration Summary
4649@cindex Bison declaration summary
4650@cindex declaration summary
4651@cindex summary, Bison declaration
4652
d8988b2f 4653Here is a summary of the declarations used to define a grammar:
bfa74976 4654
18b519c0 4655@deffn {Directive} %union
bfa74976
RS
4656Declare the collection of data types that semantic values may have
4657(@pxref{Union Decl, ,The Collection of Value Types}).
18b519c0 4658@end deffn
bfa74976 4659
18b519c0 4660@deffn {Directive} %token
bfa74976
RS
4661Declare a terminal symbol (token type name) with no precedence
4662or associativity specified (@pxref{Token Decl, ,Token Type Names}).
18b519c0 4663@end deffn
bfa74976 4664
18b519c0 4665@deffn {Directive} %right
bfa74976
RS
4666Declare a terminal symbol (token type name) that is right-associative
4667(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4668@end deffn
bfa74976 4669
18b519c0 4670@deffn {Directive} %left
bfa74976
RS
4671Declare a terminal symbol (token type name) that is left-associative
4672(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4673@end deffn
bfa74976 4674
18b519c0 4675@deffn {Directive} %nonassoc
bfa74976 4676Declare a terminal symbol (token type name) that is nonassociative
bfa74976 4677(@pxref{Precedence Decl, ,Operator Precedence}).
39a06c25
PE
4678Using it in a way that would be associative is a syntax error.
4679@end deffn
4680
91d2c560 4681@ifset defaultprec
39a06c25 4682@deffn {Directive} %default-prec
22fccf95 4683Assign a precedence to rules lacking an explicit @code{%prec} modifier
39a06c25
PE
4684(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4685@end deffn
91d2c560 4686@end ifset
bfa74976 4687
18b519c0 4688@deffn {Directive} %type
bfa74976
RS
4689Declare the type of semantic values for a nonterminal symbol
4690(@pxref{Type Decl, ,Nonterminal Symbols}).
18b519c0 4691@end deffn
bfa74976 4692
18b519c0 4693@deffn {Directive} %start
89cab50d
AD
4694Specify the grammar's start symbol (@pxref{Start Decl, ,The
4695Start-Symbol}).
18b519c0 4696@end deffn
bfa74976 4697
18b519c0 4698@deffn {Directive} %expect
bfa74976
RS
4699Declare the expected number of shift-reduce conflicts
4700(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
18b519c0
AD
4701@end deffn
4702
bfa74976 4703
d8988b2f
AD
4704@sp 1
4705@noindent
4706In order to change the behavior of @command{bison}, use the following
4707directives:
4708
148d66d8
JD
4709@deffn {Directive} %code @{@var{code}@}
4710@findex %code
4711This is the unqualified form of the @code{%code} directive.
8405b70c
PB
4712It inserts @var{code} verbatim at a language-dependent default location in the
4713output@footnote{The default location is actually skeleton-dependent;
4714 writers of non-standard skeletons however should choose the default location
4715 consistently with the behavior of the standard Bison skeletons.}.
148d66d8
JD
4716
4717@cindex Prologue
8405b70c 4718For C/C++, the default location is the parser source code
148d66d8
JD
4719file after the usual contents of the parser header file.
4720Thus, @code{%code} replaces the traditional Yacc prologue,
4721@code{%@{@var{code}%@}}, for most purposes.
4722For a detailed discussion, see @ref{Prologue Alternatives}.
4723
8405b70c 4724For Java, the default location is inside the parser class.
148d66d8
JD
4725
4726(Like all the Yacc prologue alternatives, this directive is experimental.
4727More user feedback will help to determine whether it should become a permanent
4728feature.)
4729@end deffn
4730
4731@deffn {Directive} %code @var{qualifier} @{@var{code}@}
4732This is the qualified form of the @code{%code} directive.
4733If you need to specify location-sensitive verbatim @var{code} that does not
4734belong at the default location selected by the unqualified @code{%code} form,
4735use this form instead.
4736
4737@var{qualifier} identifies the purpose of @var{code} and thus the location(s)
4738where Bison should generate it.
4739Not all values of @var{qualifier} are available for all target languages:
4740
4741@itemize @bullet
148d66d8 4742@item requires
793fbca5 4743@findex %code requires
148d66d8
JD
4744
4745@itemize @bullet
4746@item Language(s): C, C++
4747
4748@item Purpose: This is the best place to write dependency code required for
4749@code{YYSTYPE} and @code{YYLTYPE}.
4750In other words, it's the best place to define types referenced in @code{%union}
4751directives, and it's the best place to override Bison's default @code{YYSTYPE}
4752and @code{YYLTYPE} definitions.
4753
4754@item Location(s): The parser header file and the parser source code file
4755before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE} definitions.
4756@end itemize
4757
4758@item provides
4759@findex %code provides
4760
4761@itemize @bullet
4762@item Language(s): C, C++
4763
4764@item Purpose: This is the best place to write additional definitions and
4765declarations that should be provided to other modules.
4766
4767@item Location(s): The parser header file and the parser source code file after
4768the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and token definitions.
4769@end itemize
4770
4771@item top
4772@findex %code top
4773
4774@itemize @bullet
4775@item Language(s): C, C++
4776
4777@item Purpose: The unqualified @code{%code} or @code{%code requires} should
4778usually be more appropriate than @code{%code top}.
4779However, occasionally it is necessary to insert code much nearer the top of the
4780parser source code file.
4781For example:
4782
4783@smallexample
4784%code top @{
4785 #define _GNU_SOURCE
4786 #include <stdio.h>
4787@}
4788@end smallexample
4789
4790@item Location(s): Near the top of the parser source code file.
4791@end itemize
8405b70c 4792
148d66d8
JD
4793@item imports
4794@findex %code imports
4795
4796@itemize @bullet
4797@item Language(s): Java
4798
4799@item Purpose: This is the best place to write Java import directives.
4800
4801@item Location(s): The parser Java file after any Java package directive and
4802before any class definitions.
4803@end itemize
148d66d8
JD
4804@end itemize
4805
4806(Like all the Yacc prologue alternatives, this directive is experimental.
4807More user feedback will help to determine whether it should become a permanent
4808feature.)
4809
4810@cindex Prologue
4811For a detailed discussion of how to use @code{%code} in place of the
4812traditional Yacc prologue for C/C++, see @ref{Prologue Alternatives}.
4813@end deffn
4814
18b519c0 4815@deffn {Directive} %debug
4947ebdb
PE
4816In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
4817already defined, so that the debugging facilities are compiled.
18b519c0 4818@end deffn
ec3bc396 4819@xref{Tracing, ,Tracing Your Parser}.
d8988b2f 4820
c1d19e10
PB
4821@deffn {Directive} %define @var{variable}
4822@deffnx {Directive} %define @var{variable} "@var{value}"
9611cfa2
JD
4823Define a variable to adjust Bison's behavior.
4824The possible choices for @var{variable}, as well as their meanings, depend on
4825the selected target language and/or the parser skeleton (@pxref{Decl
4826Summary,,%language}).
4827
4828Bison will warn if a @var{variable} is defined multiple times.
4829
4830Omitting @code{"@var{value}"} is always equivalent to specifying it as
4831@code{""}.
4832
922bdd7f 4833Some @var{variable}s may be used as Booleans.
9611cfa2
JD
4834In this case, Bison will complain if the variable definition does not meet one
4835of the following four conditions:
4836
4837@enumerate
4838@item @code{"@var{value}"} is @code{"true"}
4839
4840@item @code{"@var{value}"} is omitted (or is @code{""}).
4841This is equivalent to @code{"true"}.
4842
4843@item @code{"@var{value}"} is @code{"false"}.
4844
4845@item @var{variable} is never defined.
4846In this case, Bison selects a default value, which may depend on the selected
4847target language and/or parser skeleton.
4848@end enumerate
148d66d8 4849
793fbca5
JD
4850Some of the accepted @var{variable}s are:
4851
4852@itemize @bullet
d9df47b6
JD
4853@item api.pure
4854@findex %define api.pure
4855
4856@itemize @bullet
4857@item Language(s): C
4858
4859@item Purpose: Request a pure (reentrant) parser program.
4860@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
4861
4862@item Accepted Values: Boolean
4863
4864@item Default Value: @code{"false"}
4865@end itemize
4866
c373bf8b
JD
4867@item api.push_pull
4868@findex %define api.push_pull
793fbca5
JD
4869
4870@itemize @bullet
4871@item Language(s): C (LALR(1) only)
4872
4873@item Purpose: Requests a pull parser, a push parser, or both.
d782395d 4874@xref{Push Decl, ,A Push Parser}.
793fbca5
JD
4875
4876@item Accepted Values: @code{"pull"}, @code{"push"}, @code{"both"}
4877
4878@item Default Value: @code{"pull"}
4879@end itemize
4880
31984206
JD
4881@item lr.keep_unreachable_states
4882@findex %define lr.keep_unreachable_states
4883
4884@itemize @bullet
4885@item Language(s): all
4886
4887@item Purpose: Requests that Bison allow unreachable parser states to remain in
4888the parser tables.
4889Bison considers a state to be unreachable if there exists no sequence of
4890transitions from the start state to that state.
4891A state can become unreachable during conflict resolution if Bison disables a
4892shift action leading to it from a predecessor state.
4893Keeping unreachable states is sometimes useful for analysis purposes, but they
4894are useless in the generated parser.
4895
4896@item Accepted Values: Boolean
4897
4898@item Default Value: @code{"false"}
4899
4900@item Caveats:
4901
4902@itemize @bullet
cff03fb2
JD
4903
4904@item Unreachable states may contain conflicts and may use rules not used in
4905any other state.
31984206
JD
4906Thus, keeping unreachable states may induce warnings that are irrelevant to
4907your parser's behavior, and it may eliminate warnings that are relevant.
4908Of course, the change in warnings may actually be relevant to a parser table
4909analysis that wants to keep unreachable states, so this behavior will likely
4910remain in future Bison releases.
4911
4912@item While Bison is able to remove unreachable states, it is not guaranteed to
4913remove other kinds of useless states.
4914Specifically, when Bison disables reduce actions during conflict resolution,
4915some goto actions may become useless, and thus some additional states may
4916become useless.
4917If Bison were to compute which goto actions were useless and then disable those
4918actions, it could identify such states as unreachable and then remove those
4919states.
4920However, Bison does not compute which goto actions are useless.
4921@end itemize
4922@end itemize
4923
793fbca5
JD
4924@item namespace
4925@findex %define namespace
4926
4927@itemize
4928@item Languages(s): C++
4929
4930@item Purpose: Specifies the namespace for the parser class.
4931For example, if you specify:
4932
4933@smallexample
4934%define namespace "foo::bar"
4935@end smallexample
4936
4937Bison uses @code{foo::bar} verbatim in references such as:
4938
4939@smallexample
4940foo::bar::parser::semantic_type
4941@end smallexample
4942
4943However, to open a namespace, Bison removes any leading @code{::} and then
4944splits on any remaining occurrences:
4945
4946@smallexample
4947namespace foo @{ namespace bar @{
4948 class position;
4949 class location;
4950@} @}
4951@end smallexample
4952
4953@item Accepted Values: Any absolute or relative C++ namespace reference without
4954a trailing @code{"::"}.
4955For example, @code{"foo"} or @code{"::foo::bar"}.
4956
4957@item Default Value: The value specified by @code{%name-prefix}, which defaults
4958to @code{yy}.
4959This usage of @code{%name-prefix} is for backward compatibility and can be
4960confusing since @code{%name-prefix} also specifies the textual prefix for the
4961lexical analyzer function.
4962Thus, if you specify @code{%name-prefix}, it is best to also specify
4963@code{%define namespace} so that @code{%name-prefix} @emph{only} affects the
4964lexical analyzer function.
4965For example, if you specify:
4966
4967@smallexample
4968%define namespace "foo"
4969%name-prefix "bar::"
4970@end smallexample
4971
4972The parser namespace is @code{foo} and @code{yylex} is referenced as
4973@code{bar::lex}.
4974@end itemize
4975@end itemize
4976
d782395d
JD
4977@end deffn
4978
18b519c0 4979@deffn {Directive} %defines
4bfd5e4e
PE
4980Write a header file containing macro definitions for the token type
4981names defined in the grammar as well as a few other declarations.
d8988b2f 4982If the parser output file is named @file{@var{name}.c} then this file
e0c471a9 4983is named @file{@var{name}.h}.
d8988b2f 4984
b321737f 4985For C parsers, the output header declares @code{YYSTYPE} unless
ddc8ede1
PE
4986@code{YYSTYPE} is already defined as a macro or you have used a
4987@code{<@var{type}>} tag without using @code{%union}.
4988Therefore, if you are using a @code{%union}
f8e1c9e5
AD
4989(@pxref{Multiple Types, ,More Than One Value Type}) with components that
4990require other definitions, or if you have defined a @code{YYSTYPE} macro
ddc8ede1 4991or type definition
f8e1c9e5
AD
4992(@pxref{Value Type, ,Data Types of Semantic Values}), you need to
4993arrange for these definitions to be propagated to all modules, e.g., by
4994putting them in a prerequisite header that is included both by your
4995parser and by any other module that needs @code{YYSTYPE}.
4bfd5e4e
PE
4996
4997Unless your parser is pure, the output header declares @code{yylval}
4998as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
4999Parser}.
5000
5001If you have also used locations, the output header declares
5002@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
ddc8ede1 5003the @code{YYSTYPE} macro and @code{yylval}. @xref{Locations, ,Tracking
4bfd5e4e
PE
5004Locations}.
5005
f8e1c9e5
AD
5006This output file is normally essential if you wish to put the definition
5007of @code{yylex} in a separate source file, because @code{yylex}
5008typically needs to be able to refer to the above-mentioned declarations
5009and to the token type codes. @xref{Token Values, ,Semantic Values of
5010Tokens}.
9bc0dd67 5011
16dc6a9e
JD
5012@findex %code requires
5013@findex %code provides
5014If you have declared @code{%code requires} or @code{%code provides}, the output
5015header also contains their code.
148d66d8 5016@xref{Decl Summary, ,%code}.
592d0b1e
PB
5017@end deffn
5018
02975b9a
JD
5019@deffn {Directive} %defines @var{defines-file}
5020Same as above, but save in the file @var{defines-file}.
5021@end deffn
5022
18b519c0 5023@deffn {Directive} %destructor
258b75ca 5024Specify how the parser should reclaim the memory associated to
fa7e68c3 5025discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 5026@end deffn
72f889cc 5027
02975b9a 5028@deffn {Directive} %file-prefix "@var{prefix}"
d8988b2f
AD
5029Specify a prefix to use for all Bison output file names. The names are
5030chosen as if the input file were named @file{@var{prefix}.y}.
18b519c0 5031@end deffn
d8988b2f 5032
e6e704dc 5033@deffn {Directive} %language "@var{language}"
0e021770
PE
5034Specify the programming language for the generated parser. Currently
5035supported languages include C and C++.
e6e704dc 5036@var{language} is case-insensitive.
0e021770
PE
5037@end deffn
5038
18b519c0 5039@deffn {Directive} %locations
89cab50d
AD
5040Generate the code processing the locations (@pxref{Action Features,
5041,Special Features for Use in Actions}). This mode is enabled as soon as
5042the grammar uses the special @samp{@@@var{n}} tokens, but if your
5043grammar does not use it, using @samp{%locations} allows for more
6e649e65 5044accurate syntax error messages.
18b519c0 5045@end deffn
89cab50d 5046
02975b9a 5047@deffn {Directive} %name-prefix "@var{prefix}"
d8988b2f
AD
5048Rename the external symbols used in the parser so that they start with
5049@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
aa08666d 5050in C parsers
d8988b2f 5051is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
91e3ac9a 5052@code{yylval}, @code{yychar}, @code{yydebug}, and
f4101aa6
AD
5053(if locations are used) @code{yylloc}. If you use a push parser,
5054@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5055@code{yypstate_new} and @code{yypstate_delete} will
5056also be renamed. For example, if you use @samp{%name-prefix "c_"}, the
793fbca5
JD
5057names become @code{c_parse}, @code{c_lex}, and so on.
5058For C++ parsers, see the @code{%define namespace} documentation in this
5059section.
aa08666d 5060@xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
18b519c0 5061@end deffn
931c7513 5062
91d2c560 5063@ifset defaultprec
22fccf95
PE
5064@deffn {Directive} %no-default-prec
5065Do not assign a precedence to rules lacking an explicit @code{%prec}
5066modifier (@pxref{Contextual Precedence, ,Context-Dependent
5067Precedence}).
5068@end deffn
91d2c560 5069@end ifset
22fccf95 5070
18b519c0 5071@deffn {Directive} %no-lines
931c7513
RS
5072Don't generate any @code{#line} preprocessor commands in the parser
5073file. Ordinarily Bison writes these commands in the parser file so that
5074the C compiler and debuggers will associate errors and object code with
5075your source file (the grammar file). This directive causes them to
5076associate errors with the parser file, treating it an independent source
5077file in its own right.
18b519c0 5078@end deffn
931c7513 5079
02975b9a 5080@deffn {Directive} %output "@var{file}"
fa4d969f 5081Specify @var{file} for the parser file.
18b519c0 5082@end deffn
6deb4447 5083
18b519c0 5084@deffn {Directive} %pure-parser
d9df47b6
JD
5085Deprecated version of @code{%define api.pure} (@pxref{Decl Summary, ,%define}),
5086for which Bison is more careful to warn about unreasonable usage.
18b519c0 5087@end deffn
6deb4447 5088
b50d2359 5089@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
5090Require version @var{version} or higher of Bison. @xref{Require Decl, ,
5091Require a Version of Bison}.
b50d2359
AD
5092@end deffn
5093
0e021770 5094@deffn {Directive} %skeleton "@var{file}"
a7867f53
JD
5095Specify the skeleton to use.
5096
5097You probably don't need this option unless you are developing Bison.
5098You should use @code{%language} if you want to specify the skeleton for a
5099different language, because it is clearer and because it will always choose the
5100correct skeleton for non-deterministic or push parsers.
5101
5102If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5103file in the Bison installation directory.
5104If it does, @var{file} is an absolute file name or a file name relative to the
5105directory of the grammar file.
5106This is similar to how most shells resolve commands.
0e021770
PE
5107@end deffn
5108
18b519c0 5109@deffn {Directive} %token-table
931c7513
RS
5110Generate an array of token names in the parser file. The name of the
5111array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
3650b4b8 5112token whose internal Bison token code number is @var{i}. The first
f67ad422
PE
5113three elements of @code{yytname} correspond to the predefined tokens
5114@code{"$end"},
88bce5a2
AD
5115@code{"error"}, and @code{"$undefined"}; after these come the symbols
5116defined in the grammar file.
931c7513 5117
9e0876fb
PE
5118The name in the table includes all the characters needed to represent
5119the token in Bison. For single-character literals and literal
5120strings, this includes the surrounding quoting characters and any
5121escape sequences. For example, the Bison single-character literal
5122@code{'+'} corresponds to a three-character name, represented in C as
5123@code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
5124corresponds to a five-character name, represented in C as
5125@code{"\"\\\\/\""}.
931c7513 5126
8c9a50be 5127When you specify @code{%token-table}, Bison also generates macro
931c7513
RS
5128definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
5129@code{YYNRULES}, and @code{YYNSTATES}:
5130
5131@table @code
5132@item YYNTOKENS
5133The highest token number, plus one.
5134@item YYNNTS
9ecbd125 5135The number of nonterminal symbols.
931c7513
RS
5136@item YYNRULES
5137The number of grammar rules,
5138@item YYNSTATES
5139The number of parser states (@pxref{Parser States}).
5140@end table
18b519c0 5141@end deffn
d8988b2f 5142
18b519c0 5143@deffn {Directive} %verbose
d8988b2f 5144Write an extra output file containing verbose descriptions of the
742e4900 5145parser states and what is done for each type of lookahead token in
72d2299c 5146that state. @xref{Understanding, , Understanding Your Parser}, for more
ec3bc396 5147information.
18b519c0 5148@end deffn
d8988b2f 5149
18b519c0 5150@deffn {Directive} %yacc
d8988b2f
AD
5151Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
5152including its naming conventions. @xref{Bison Options}, for more.
18b519c0 5153@end deffn
d8988b2f
AD
5154
5155
342b8b6e 5156@node Multiple Parsers
bfa74976
RS
5157@section Multiple Parsers in the Same Program
5158
5159Most programs that use Bison parse only one language and therefore contain
5160only one Bison parser. But what if you want to parse more than one
5161language with the same program? Then you need to avoid a name conflict
5162between different definitions of @code{yyparse}, @code{yylval}, and so on.
5163
5164The easy way to do this is to use the option @samp{-p @var{prefix}}
704a47c4
AD
5165(@pxref{Invocation, ,Invoking Bison}). This renames the interface
5166functions and variables of the Bison parser to start with @var{prefix}
5167instead of @samp{yy}. You can use this to give each parser distinct
5168names that do not conflict.
bfa74976
RS
5169
5170The precise list of symbols renamed is @code{yyparse}, @code{yylex},
2a8d363a 5171@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
f4101aa6
AD
5172@code{yychar} and @code{yydebug}. If you use a push parser,
5173@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
9987d1b3 5174@code{yypstate_new} and @code{yypstate_delete} will also be renamed.
f4101aa6 5175For example, if you use @samp{-p c}, the names become @code{cparse},
9987d1b3 5176@code{clex}, and so on.
bfa74976
RS
5177
5178@strong{All the other variables and macros associated with Bison are not
5179renamed.} These others are not global; there is no conflict if the same
5180name is used in different parsers. For example, @code{YYSTYPE} is not
5181renamed, but defining this in different ways in different parsers causes
5182no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
5183
5184The @samp{-p} option works by adding macro definitions to the beginning
5185of the parser source file, defining @code{yyparse} as
5186@code{@var{prefix}parse}, and so on. This effectively substitutes one
5187name for the other in the entire parser file.
5188
342b8b6e 5189@node Interface
bfa74976
RS
5190@chapter Parser C-Language Interface
5191@cindex C-language interface
5192@cindex interface
5193
5194The Bison parser is actually a C function named @code{yyparse}. Here we
5195describe the interface conventions of @code{yyparse} and the other
5196functions that it needs to use.
5197
5198Keep in mind that the parser uses many C identifiers starting with
5199@samp{yy} and @samp{YY} for internal purposes. If you use such an
75f5aaea
MA
5200identifier (aside from those in this manual) in an action or in epilogue
5201in the grammar file, you are likely to run into trouble.
bfa74976
RS
5202
5203@menu
5204* Parser Function:: How to call @code{yyparse} and what it returns.
9987d1b3
JD
5205* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
5206* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
f4101aa6 5207* Parser Create Function:: How to call @code{yypstate_new} and what it
9987d1b3 5208 returns.
f4101aa6 5209* Parser Delete Function:: How to call @code{yypstate_delete} and what it
9987d1b3 5210 returns.
13863333 5211* Lexical:: You must supply a function @code{yylex}
bfa74976
RS
5212 which reads tokens.
5213* Error Reporting:: You must supply a function @code{yyerror}.
5214* Action Features:: Special features for use in actions.
f7ab6a50
PE
5215* Internationalization:: How to let the parser speak in the user's
5216 native language.
bfa74976
RS
5217@end menu
5218
342b8b6e 5219@node Parser Function
bfa74976
RS
5220@section The Parser Function @code{yyparse}
5221@findex yyparse
5222
5223You call the function @code{yyparse} to cause parsing to occur. This
5224function reads tokens, executes actions, and ultimately returns when it
5225encounters end-of-input or an unrecoverable syntax error. You can also
14ded682
AD
5226write an action which directs @code{yyparse} to return immediately
5227without reading further.
bfa74976 5228
2a8d363a
AD
5229
5230@deftypefun int yyparse (void)
bfa74976
RS
5231The value returned by @code{yyparse} is 0 if parsing was successful (return
5232is due to end-of-input).
5233
b47dbebe
PE
5234The value is 1 if parsing failed because of invalid input, i.e., input
5235that contains a syntax error or that causes @code{YYABORT} to be
5236invoked.
5237
5238The value is 2 if parsing failed due to memory exhaustion.
2a8d363a 5239@end deftypefun
bfa74976
RS
5240
5241In an action, you can cause immediate return from @code{yyparse} by using
5242these macros:
5243
2a8d363a 5244@defmac YYACCEPT
bfa74976
RS
5245@findex YYACCEPT
5246Return immediately with value 0 (to report success).
2a8d363a 5247@end defmac
bfa74976 5248
2a8d363a 5249@defmac YYABORT
bfa74976
RS
5250@findex YYABORT
5251Return immediately with value 1 (to report failure).
2a8d363a
AD
5252@end defmac
5253
5254If you use a reentrant parser, you can optionally pass additional
5255parameter information to it in a reentrant way. To do so, use the
5256declaration @code{%parse-param}:
5257
feeb0eda 5258@deffn {Directive} %parse-param @{@var{argument-declaration}@}
2a8d363a 5259@findex %parse-param
287c78f6
PE
5260Declare that an argument declared by the braced-code
5261@var{argument-declaration} is an additional @code{yyparse} argument.
94175978 5262The @var{argument-declaration} is used when declaring
feeb0eda
PE
5263functions or prototypes. The last identifier in
5264@var{argument-declaration} must be the argument name.
2a8d363a
AD
5265@end deffn
5266
5267Here's an example. Write this in the parser:
5268
5269@example
feeb0eda
PE
5270%parse-param @{int *nastiness@}
5271%parse-param @{int *randomness@}
2a8d363a
AD
5272@end example
5273
5274@noindent
5275Then call the parser like this:
5276
5277@example
5278@{
5279 int nastiness, randomness;
5280 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
5281 value = yyparse (&nastiness, &randomness);
5282 @dots{}
5283@}
5284@end example
5285
5286@noindent
5287In the grammar actions, use expressions like this to refer to the data:
5288
5289@example
5290exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
5291@end example
5292
9987d1b3
JD
5293@node Push Parser Function
5294@section The Push Parser Function @code{yypush_parse}
5295@findex yypush_parse
5296
f4101aa6
AD
5297You call the function @code{yypush_parse} to parse a single token. This
5298function is available if either the @code{%define api.push_pull "push"} or
5299@code{%define api.push_pull "both"} declaration is used.
9987d1b3
JD
5300@xref{Push Decl, ,A Push Parser}.
5301
5302@deftypefun int yypush_parse (yypstate *yyps)
f4101aa6 5303The value returned by @code{yypush_parse} is the same as for yyparse with the
9987d1b3
JD
5304following exception. @code{yypush_parse} will return YYPUSH_MORE if more input
5305is required to finish parsing the grammar.
5306@end deftypefun
5307
5308@node Pull Parser Function
5309@section The Pull Parser Function @code{yypull_parse}
5310@findex yypull_parse
5311
f4101aa6
AD
5312You call the function @code{yypull_parse} to parse the rest of the input
5313stream. This function is available if the @code{%define api.push_pull "both"}
5314declaration is used.
9987d1b3
JD
5315@xref{Push Decl, ,A Push Parser}.
5316
5317@deftypefun int yypull_parse (yypstate *yyps)
5318The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
5319@end deftypefun
5320
5321@node Parser Create Function
5322@section The Parser Create Function @code{yystate_new}
5323@findex yypstate_new
5324
f4101aa6
AD
5325You call the function @code{yypstate_new} to create a new parser instance.
5326This function is available if either the @code{%define api.push_pull "push"} or
5327@code{%define api.push_pull "both"} declaration is used.
9987d1b3
JD
5328@xref{Push Decl, ,A Push Parser}.
5329
5330@deftypefun yypstate *yypstate_new (void)
5331The fuction will return a valid parser instance if there was memory available
5332or NULL if no memory was available.
5333@end deftypefun
5334
5335@node Parser Delete Function
5336@section The Parser Delete Function @code{yystate_delete}
5337@findex yypstate_delete
5338
5339You call the function @code{yypstate_delete} to delete a parser instance.
f4101aa6
AD
5340function is available if either the @code{%define api.push_pull "push"} or
5341@code{%define api.push_pull "both"} declaration is used.
9987d1b3
JD
5342@xref{Push Decl, ,A Push Parser}.
5343
5344@deftypefun void yypstate_delete (yypstate *yyps)
5345This function will reclaim the memory associated with a parser instance.
5346After this call, you should no longer attempt to use the parser instance.
5347@end deftypefun
bfa74976 5348
342b8b6e 5349@node Lexical
bfa74976
RS
5350@section The Lexical Analyzer Function @code{yylex}
5351@findex yylex
5352@cindex lexical analyzer
5353
5354The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
5355the input stream and returns them to the parser. Bison does not create
5356this function automatically; you must write it so that @code{yyparse} can
5357call it. The function is sometimes referred to as a lexical scanner.
5358
5359In simple programs, @code{yylex} is often defined at the end of the Bison
5360grammar file. If @code{yylex} is defined in a separate source file, you
5361need to arrange for the token-type macro definitions to be available there.
5362To do this, use the @samp{-d} option when you run Bison, so that it will
5363write these macro definitions into a separate header file
5364@file{@var{name}.tab.h} which you can include in the other source files
e0c471a9 5365that need it. @xref{Invocation, ,Invoking Bison}.
bfa74976
RS
5366
5367@menu
5368* Calling Convention:: How @code{yyparse} calls @code{yylex}.
5369* Token Values:: How @code{yylex} must return the semantic value
5370 of the token it has read.
95923bd6 5371* Token Locations:: How @code{yylex} must return the text location
bfa74976
RS
5372 (line number, etc.) of the token, if the
5373 actions want that.
5374* Pure Calling:: How the calling convention differs
5375 in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
5376@end menu
5377
342b8b6e 5378@node Calling Convention
bfa74976
RS
5379@subsection Calling Convention for @code{yylex}
5380
72d2299c
PE
5381The value that @code{yylex} returns must be the positive numeric code
5382for the type of token it has just found; a zero or negative value
5383signifies end-of-input.
bfa74976
RS
5384
5385When a token is referred to in the grammar rules by a name, that name
5386in the parser file becomes a C macro whose definition is the proper
5387numeric code for that token type. So @code{yylex} can use the name
5388to indicate that type. @xref{Symbols}.
5389
5390When a token is referred to in the grammar rules by a character literal,
5391the numeric code for that character is also the code for the token type.
72d2299c
PE
5392So @code{yylex} can simply return that character code, possibly converted
5393to @code{unsigned char} to avoid sign-extension. The null character
5394must not be used this way, because its code is zero and that
bfa74976
RS
5395signifies end-of-input.
5396
5397Here is an example showing these things:
5398
5399@example
13863333
AD
5400int
5401yylex (void)
bfa74976
RS
5402@{
5403 @dots{}
72d2299c 5404 if (c == EOF) /* Detect end-of-input. */
bfa74976
RS
5405 return 0;
5406 @dots{}
5407 if (c == '+' || c == '-')
72d2299c 5408 return c; /* Assume token type for `+' is '+'. */
bfa74976 5409 @dots{}
72d2299c 5410 return INT; /* Return the type of the token. */
bfa74976
RS
5411 @dots{}
5412@}
5413@end example
5414
5415@noindent
5416This interface has been designed so that the output from the @code{lex}
5417utility can be used without change as the definition of @code{yylex}.
5418
931c7513
RS
5419If the grammar uses literal string tokens, there are two ways that
5420@code{yylex} can determine the token type codes for them:
5421
5422@itemize @bullet
5423@item
5424If the grammar defines symbolic token names as aliases for the
5425literal string tokens, @code{yylex} can use these symbolic names like
5426all others. In this case, the use of the literal string tokens in
5427the grammar file has no effect on @code{yylex}.
5428
5429@item
9ecbd125 5430@code{yylex} can find the multicharacter token in the @code{yytname}
931c7513 5431table. The index of the token in the table is the token type's code.
9ecbd125 5432The name of a multicharacter token is recorded in @code{yytname} with a
931c7513 5433double-quote, the token's characters, and another double-quote. The
9e0876fb
PE
5434token's characters are escaped as necessary to be suitable as input
5435to Bison.
931c7513 5436
9e0876fb
PE
5437Here's code for looking up a multicharacter token in @code{yytname},
5438assuming that the characters of the token are stored in
5439@code{token_buffer}, and assuming that the token does not contain any
5440characters like @samp{"} that require escaping.
931c7513
RS
5441
5442@smallexample
5443for (i = 0; i < YYNTOKENS; i++)
5444 @{
5445 if (yytname[i] != 0
5446 && yytname[i][0] == '"'
68449b3a
PE
5447 && ! strncmp (yytname[i] + 1, token_buffer,
5448 strlen (token_buffer))
931c7513
RS
5449 && yytname[i][strlen (token_buffer) + 1] == '"'
5450 && yytname[i][strlen (token_buffer) + 2] == 0)
5451 break;
5452 @}
5453@end smallexample
5454
5455The @code{yytname} table is generated only if you use the
8c9a50be 5456@code{%token-table} declaration. @xref{Decl Summary}.
931c7513
RS
5457@end itemize
5458
342b8b6e 5459@node Token Values
bfa74976
RS
5460@subsection Semantic Values of Tokens
5461
5462@vindex yylval
9d9b8b70 5463In an ordinary (nonreentrant) parser, the semantic value of the token must
bfa74976
RS
5464be stored into the global variable @code{yylval}. When you are using
5465just one data type for semantic values, @code{yylval} has that type.
5466Thus, if the type is @code{int} (the default), you might write this in
5467@code{yylex}:
5468
5469@example
5470@group
5471 @dots{}
72d2299c
PE
5472 yylval = value; /* Put value onto Bison stack. */
5473 return INT; /* Return the type of the token. */
bfa74976
RS
5474 @dots{}
5475@end group
5476@end example
5477
5478When you are using multiple data types, @code{yylval}'s type is a union
704a47c4
AD
5479made from the @code{%union} declaration (@pxref{Union Decl, ,The
5480Collection of Value Types}). So when you store a token's value, you
5481must use the proper member of the union. If the @code{%union}
5482declaration looks like this:
bfa74976
RS
5483
5484@example
5485@group
5486%union @{
5487 int intval;
5488 double val;
5489 symrec *tptr;
5490@}
5491@end group
5492@end example
5493
5494@noindent
5495then the code in @code{yylex} might look like this:
5496
5497@example
5498@group
5499 @dots{}
72d2299c
PE
5500 yylval.intval = value; /* Put value onto Bison stack. */
5501 return INT; /* Return the type of the token. */
bfa74976
RS
5502 @dots{}
5503@end group
5504@end example
5505
95923bd6
AD
5506@node Token Locations
5507@subsection Textual Locations of Tokens
bfa74976
RS
5508
5509@vindex yylloc
847bf1f5 5510If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
f8e1c9e5
AD
5511Tracking Locations}) in actions to keep track of the textual locations
5512of tokens and groupings, then you must provide this information in
5513@code{yylex}. The function @code{yyparse} expects to find the textual
5514location of a token just parsed in the global variable @code{yylloc}.
5515So @code{yylex} must store the proper data in that variable.
847bf1f5
AD
5516
5517By default, the value of @code{yylloc} is a structure and you need only
89cab50d
AD
5518initialize the members that are going to be used by the actions. The
5519four members are called @code{first_line}, @code{first_column},
5520@code{last_line} and @code{last_column}. Note that the use of this
5521feature makes the parser noticeably slower.
bfa74976
RS
5522
5523@tindex YYLTYPE
5524The data type of @code{yylloc} has the name @code{YYLTYPE}.
5525
342b8b6e 5526@node Pure Calling
c656404a 5527@subsection Calling Conventions for Pure Parsers
bfa74976 5528
d9df47b6 5529When you use the Bison declaration @code{%define api.pure} to request a
e425e872
RS
5530pure, reentrant parser, the global communication variables @code{yylval}
5531and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
5532Parser}.) In such parsers the two global variables are replaced by
5533pointers passed as arguments to @code{yylex}. You must declare them as
5534shown here, and pass the information back by storing it through those
5535pointers.
bfa74976
RS
5536
5537@example
13863333
AD
5538int
5539yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
bfa74976
RS
5540@{
5541 @dots{}
5542 *lvalp = value; /* Put value onto Bison stack. */
5543 return INT; /* Return the type of the token. */
5544 @dots{}
5545@}
5546@end example
5547
5548If the grammar file does not use the @samp{@@} constructs to refer to
95923bd6 5549textual locations, then the type @code{YYLTYPE} will not be defined. In
bfa74976
RS
5550this case, omit the second argument; @code{yylex} will be called with
5551only one argument.
5552
e425e872 5553
2a8d363a
AD
5554If you wish to pass the additional parameter data to @code{yylex}, use
5555@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
5556Function}).
e425e872 5557
feeb0eda 5558@deffn {Directive} lex-param @{@var{argument-declaration}@}
2a8d363a 5559@findex %lex-param
287c78f6
PE
5560Declare that the braced-code @var{argument-declaration} is an
5561additional @code{yylex} argument declaration.
2a8d363a 5562@end deffn
e425e872 5563
2a8d363a 5564For instance:
e425e872
RS
5565
5566@example
feeb0eda
PE
5567%parse-param @{int *nastiness@}
5568%lex-param @{int *nastiness@}
5569%parse-param @{int *randomness@}
e425e872
RS
5570@end example
5571
5572@noindent
2a8d363a 5573results in the following signature:
e425e872
RS
5574
5575@example
2a8d363a
AD
5576int yylex (int *nastiness);
5577int yyparse (int *nastiness, int *randomness);
e425e872
RS
5578@end example
5579
d9df47b6 5580If @code{%define api.pure} is added:
c656404a
RS
5581
5582@example
2a8d363a
AD
5583int yylex (YYSTYPE *lvalp, int *nastiness);
5584int yyparse (int *nastiness, int *randomness);
c656404a
RS
5585@end example
5586
2a8d363a 5587@noindent
d9df47b6 5588and finally, if both @code{%define api.pure} and @code{%locations} are used:
c656404a 5589
2a8d363a
AD
5590@example
5591int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
5592int yyparse (int *nastiness, int *randomness);
5593@end example
931c7513 5594
342b8b6e 5595@node Error Reporting
bfa74976
RS
5596@section The Error Reporting Function @code{yyerror}
5597@cindex error reporting function
5598@findex yyerror
5599@cindex parse error
5600@cindex syntax error
5601
6e649e65 5602The Bison parser detects a @dfn{syntax error} or @dfn{parse error}
9ecbd125 5603whenever it reads a token which cannot satisfy any syntax rule. An
bfa74976 5604action in the grammar can also explicitly proclaim an error, using the
ceed8467
AD
5605macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
5606in Actions}).
bfa74976
RS
5607
5608The Bison parser expects to report the error by calling an error
5609reporting function named @code{yyerror}, which you must supply. It is
5610called by @code{yyparse} whenever a syntax error is found, and it
6e649e65
PE
5611receives one argument. For a syntax error, the string is normally
5612@w{@code{"syntax error"}}.
bfa74976 5613
2a8d363a
AD
5614@findex %error-verbose
5615If you invoke the directive @code{%error-verbose} in the Bison
5616declarations section (@pxref{Bison Declarations, ,The Bison Declarations
5617Section}), then Bison provides a more verbose and specific error message
6e649e65 5618string instead of just plain @w{@code{"syntax error"}}.
bfa74976 5619
1a059451
PE
5620The parser can detect one other kind of error: memory exhaustion. This
5621can happen when the input contains constructions that are very deeply
bfa74976 5622nested. It isn't likely you will encounter this, since the Bison
1a059451
PE
5623parser normally extends its stack automatically up to a very large limit. But
5624if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
5625fashion, except that the argument string is @w{@code{"memory exhausted"}}.
5626
5627In some cases diagnostics like @w{@code{"syntax error"}} are
5628translated automatically from English to some other language before
5629they are passed to @code{yyerror}. @xref{Internationalization}.
bfa74976
RS
5630
5631The following definition suffices in simple programs:
5632
5633@example
5634@group
13863333 5635void
38a92d50 5636yyerror (char const *s)
bfa74976
RS
5637@{
5638@end group
5639@group
5640 fprintf (stderr, "%s\n", s);
5641@}
5642@end group
5643@end example
5644
5645After @code{yyerror} returns to @code{yyparse}, the latter will attempt
5646error recovery if you have written suitable error recovery grammar rules
5647(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
5648immediately return 1.
5649
93724f13 5650Obviously, in location tracking pure parsers, @code{yyerror} should have
fa7e68c3
PE
5651an access to the current location.
5652This is indeed the case for the @acronym{GLR}
2a8d363a 5653parsers, but not for the Yacc parser, for historical reasons. I.e., if
d9df47b6 5654@samp{%locations %define api.pure} is passed then the prototypes for
2a8d363a
AD
5655@code{yyerror} are:
5656
5657@example
38a92d50
PE
5658void yyerror (char const *msg); /* Yacc parsers. */
5659void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
2a8d363a
AD
5660@end example
5661
feeb0eda 5662If @samp{%parse-param @{int *nastiness@}} is used, then:
2a8d363a
AD
5663
5664@example
b317297e
PE
5665void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
5666void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
2a8d363a
AD
5667@end example
5668
fa7e68c3 5669Finally, @acronym{GLR} and Yacc parsers share the same @code{yyerror} calling
2a8d363a
AD
5670convention for absolutely pure parsers, i.e., when the calling
5671convention of @code{yylex} @emph{and} the calling convention of
d9df47b6
JD
5672@code{%define api.pure} are pure.
5673I.e.:
2a8d363a
AD
5674
5675@example
5676/* Location tracking. */
5677%locations
5678/* Pure yylex. */
d9df47b6 5679%define api.pure
feeb0eda 5680%lex-param @{int *nastiness@}
2a8d363a 5681/* Pure yyparse. */
feeb0eda
PE
5682%parse-param @{int *nastiness@}
5683%parse-param @{int *randomness@}
2a8d363a
AD
5684@end example
5685
5686@noindent
5687results in the following signatures for all the parser kinds:
5688
5689@example
5690int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
5691int yyparse (int *nastiness, int *randomness);
93724f13
AD
5692void yyerror (YYLTYPE *locp,
5693 int *nastiness, int *randomness,
38a92d50 5694 char const *msg);
2a8d363a
AD
5695@end example
5696
1c0c3e95 5697@noindent
38a92d50
PE
5698The prototypes are only indications of how the code produced by Bison
5699uses @code{yyerror}. Bison-generated code always ignores the returned
5700value, so @code{yyerror} can return any type, including @code{void}.
5701Also, @code{yyerror} can be a variadic function; that is why the
5702message is always passed last.
5703
5704Traditionally @code{yyerror} returns an @code{int} that is always
5705ignored, but this is purely for historical reasons, and @code{void} is
5706preferable since it more accurately describes the return type for
5707@code{yyerror}.
93724f13 5708
bfa74976
RS
5709@vindex yynerrs
5710The variable @code{yynerrs} contains the number of syntax errors
8a2800e7 5711reported so far. Normally this variable is global; but if you
704a47c4
AD
5712request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
5713then it is a local variable which only the actions can access.
bfa74976 5714
342b8b6e 5715@node Action Features
bfa74976
RS
5716@section Special Features for Use in Actions
5717@cindex summary, action features
5718@cindex action features summary
5719
5720Here is a table of Bison constructs, variables and macros that
5721are useful in actions.
5722
18b519c0 5723@deffn {Variable} $$
bfa74976
RS
5724Acts like a variable that contains the semantic value for the
5725grouping made by the current rule. @xref{Actions}.
18b519c0 5726@end deffn
bfa74976 5727
18b519c0 5728@deffn {Variable} $@var{n}
bfa74976
RS
5729Acts like a variable that contains the semantic value for the
5730@var{n}th component of the current rule. @xref{Actions}.
18b519c0 5731@end deffn
bfa74976 5732
18b519c0 5733@deffn {Variable} $<@var{typealt}>$
bfa74976 5734Like @code{$$} but specifies alternative @var{typealt} in the union
704a47c4
AD
5735specified by the @code{%union} declaration. @xref{Action Types, ,Data
5736Types of Values in Actions}.
18b519c0 5737@end deffn
bfa74976 5738
18b519c0 5739@deffn {Variable} $<@var{typealt}>@var{n}
bfa74976 5740Like @code{$@var{n}} but specifies alternative @var{typealt} in the
13863333 5741union specified by the @code{%union} declaration.
e0c471a9 5742@xref{Action Types, ,Data Types of Values in Actions}.
18b519c0 5743@end deffn
bfa74976 5744
18b519c0 5745@deffn {Macro} YYABORT;
bfa74976
RS
5746Return immediately from @code{yyparse}, indicating failure.
5747@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 5748@end deffn
bfa74976 5749
18b519c0 5750@deffn {Macro} YYACCEPT;
bfa74976
RS
5751Return immediately from @code{yyparse}, indicating success.
5752@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 5753@end deffn
bfa74976 5754
18b519c0 5755@deffn {Macro} YYBACKUP (@var{token}, @var{value});
bfa74976
RS
5756@findex YYBACKUP
5757Unshift a token. This macro is allowed only for rules that reduce
742e4900 5758a single value, and only when there is no lookahead token.
c827f760 5759It is also disallowed in @acronym{GLR} parsers.
742e4900 5760It installs a lookahead token with token type @var{token} and
bfa74976
RS
5761semantic value @var{value}; then it discards the value that was
5762going to be reduced by this rule.
5763
5764If the macro is used when it is not valid, such as when there is
742e4900 5765a lookahead token already, then it reports a syntax error with
bfa74976
RS
5766a message @samp{cannot back up} and performs ordinary error
5767recovery.
5768
5769In either case, the rest of the action is not executed.
18b519c0 5770@end deffn
bfa74976 5771
18b519c0 5772@deffn {Macro} YYEMPTY
bfa74976 5773@vindex YYEMPTY
742e4900 5774Value stored in @code{yychar} when there is no lookahead token.
18b519c0 5775@end deffn
bfa74976 5776
32c29292
JD
5777@deffn {Macro} YYEOF
5778@vindex YYEOF
742e4900 5779Value stored in @code{yychar} when the lookahead is the end of the input
32c29292
JD
5780stream.
5781@end deffn
5782
18b519c0 5783@deffn {Macro} YYERROR;
bfa74976
RS
5784@findex YYERROR
5785Cause an immediate syntax error. This statement initiates error
5786recovery just as if the parser itself had detected an error; however, it
5787does not call @code{yyerror}, and does not print any message. If you
5788want to print an error message, call @code{yyerror} explicitly before
5789the @samp{YYERROR;} statement. @xref{Error Recovery}.
18b519c0 5790@end deffn
bfa74976 5791
18b519c0 5792@deffn {Macro} YYRECOVERING
02103984
PE
5793@findex YYRECOVERING
5794The expression @code{YYRECOVERING ()} yields 1 when the parser
5795is recovering from a syntax error, and 0 otherwise.
bfa74976 5796@xref{Error Recovery}.
18b519c0 5797@end deffn
bfa74976 5798
18b519c0 5799@deffn {Variable} yychar
742e4900
JD
5800Variable containing either the lookahead token, or @code{YYEOF} when the
5801lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
32c29292
JD
5802has been performed so the next token is not yet known.
5803Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
5804Actions}).
742e4900 5805@xref{Lookahead, ,Lookahead Tokens}.
18b519c0 5806@end deffn
bfa74976 5807
18b519c0 5808@deffn {Macro} yyclearin;
742e4900 5809Discard the current lookahead token. This is useful primarily in
32c29292
JD
5810error rules.
5811Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
5812Semantic Actions}).
5813@xref{Error Recovery}.
18b519c0 5814@end deffn
bfa74976 5815
18b519c0 5816@deffn {Macro} yyerrok;
bfa74976 5817Resume generating error messages immediately for subsequent syntax
13863333 5818errors. This is useful primarily in error rules.
bfa74976 5819@xref{Error Recovery}.
18b519c0 5820@end deffn
bfa74976 5821
32c29292 5822@deffn {Variable} yylloc
742e4900 5823Variable containing the lookahead token location when @code{yychar} is not set
32c29292
JD
5824to @code{YYEMPTY} or @code{YYEOF}.
5825Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
5826Actions}).
5827@xref{Actions and Locations, ,Actions and Locations}.
5828@end deffn
5829
5830@deffn {Variable} yylval
742e4900 5831Variable containing the lookahead token semantic value when @code{yychar} is
32c29292
JD
5832not set to @code{YYEMPTY} or @code{YYEOF}.
5833Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
5834Actions}).
5835@xref{Actions, ,Actions}.
5836@end deffn
5837
18b519c0 5838@deffn {Value} @@$
847bf1f5 5839@findex @@$
95923bd6 5840Acts like a structure variable containing information on the textual location
847bf1f5
AD
5841of the grouping made by the current rule. @xref{Locations, ,
5842Tracking Locations}.
bfa74976 5843
847bf1f5
AD
5844@c Check if those paragraphs are still useful or not.
5845
5846@c @example
5847@c struct @{
5848@c int first_line, last_line;
5849@c int first_column, last_column;
5850@c @};
5851@c @end example
5852
5853@c Thus, to get the starting line number of the third component, you would
5854@c use @samp{@@3.first_line}.
bfa74976 5855
847bf1f5
AD
5856@c In order for the members of this structure to contain valid information,
5857@c you must make @code{yylex} supply this information about each token.
5858@c If you need only certain members, then @code{yylex} need only fill in
5859@c those members.
bfa74976 5860
847bf1f5 5861@c The use of this feature makes the parser noticeably slower.
18b519c0 5862@end deffn
847bf1f5 5863
18b519c0 5864@deffn {Value} @@@var{n}
847bf1f5 5865@findex @@@var{n}
95923bd6 5866Acts like a structure variable containing information on the textual location
847bf1f5
AD
5867of the @var{n}th component of the current rule. @xref{Locations, ,
5868Tracking Locations}.
18b519c0 5869@end deffn
bfa74976 5870
f7ab6a50
PE
5871@node Internationalization
5872@section Parser Internationalization
5873@cindex internationalization
5874@cindex i18n
5875@cindex NLS
5876@cindex gettext
5877@cindex bison-po
5878
5879A Bison-generated parser can print diagnostics, including error and
5880tracing messages. By default, they appear in English. However, Bison
f8e1c9e5
AD
5881also supports outputting diagnostics in the user's native language. To
5882make this work, the user should set the usual environment variables.
5883@xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
5884For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
5885set the user's locale to French Canadian using the @acronym{UTF}-8
f7ab6a50
PE
5886encoding. The exact set of available locales depends on the user's
5887installation.
5888
5889The maintainer of a package that uses a Bison-generated parser enables
5890the internationalization of the parser's output through the following
5891steps. Here we assume a package that uses @acronym{GNU} Autoconf and
5892@acronym{GNU} Automake.
5893
5894@enumerate
5895@item
30757c8c 5896@cindex bison-i18n.m4
f7ab6a50
PE
5897Into the directory containing the @acronym{GNU} Autoconf macros used
5898by the package---often called @file{m4}---copy the
5899@file{bison-i18n.m4} file installed by Bison under
5900@samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
5901For example:
5902
5903@example
5904cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
5905@end example
5906
5907@item
30757c8c
PE
5908@findex BISON_I18N
5909@vindex BISON_LOCALEDIR
5910@vindex YYENABLE_NLS
f7ab6a50
PE
5911In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
5912invocation, add an invocation of @code{BISON_I18N}. This macro is
5913defined in the file @file{bison-i18n.m4} that you copied earlier. It
5914causes @samp{configure} to find the value of the
30757c8c
PE
5915@code{BISON_LOCALEDIR} variable, and it defines the source-language
5916symbol @code{YYENABLE_NLS} to enable translations in the
5917Bison-generated parser.
f7ab6a50
PE
5918
5919@item
5920In the @code{main} function of your program, designate the directory
5921containing Bison's runtime message catalog, through a call to
5922@samp{bindtextdomain} with domain name @samp{bison-runtime}.
5923For example:
5924
5925@example
5926bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
5927@end example
5928
5929Typically this appears after any other call @code{bindtextdomain
5930(PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
5931@samp{BISON_LOCALEDIR} to be defined as a string through the
5932@file{Makefile}.
5933
5934@item
5935In the @file{Makefile.am} that controls the compilation of the @code{main}
5936function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
5937either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
5938
5939@example
5940DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5941@end example
5942
5943or:
5944
5945@example
5946AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5947@end example
5948
5949@item
5950Finally, invoke the command @command{autoreconf} to generate the build
5951infrastructure.
5952@end enumerate
5953
bfa74976 5954
342b8b6e 5955@node Algorithm
13863333
AD
5956@chapter The Bison Parser Algorithm
5957@cindex Bison parser algorithm
bfa74976
RS
5958@cindex algorithm of parser
5959@cindex shifting
5960@cindex reduction
5961@cindex parser stack
5962@cindex stack, parser
5963
5964As Bison reads tokens, it pushes them onto a stack along with their
5965semantic values. The stack is called the @dfn{parser stack}. Pushing a
5966token is traditionally called @dfn{shifting}.
5967
5968For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
5969@samp{3} to come. The stack will have four elements, one for each token
5970that was shifted.
5971
5972But the stack does not always have an element for each token read. When
5973the last @var{n} tokens and groupings shifted match the components of a
5974grammar rule, they can be combined according to that rule. This is called
5975@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
5976single grouping whose symbol is the result (left hand side) of that rule.
5977Running the rule's action is part of the process of reduction, because this
5978is what computes the semantic value of the resulting grouping.
5979
5980For example, if the infix calculator's parser stack contains this:
5981
5982@example
59831 + 5 * 3
5984@end example
5985
5986@noindent
5987and the next input token is a newline character, then the last three
5988elements can be reduced to 15 via the rule:
5989
5990@example
5991expr: expr '*' expr;
5992@end example
5993
5994@noindent
5995Then the stack contains just these three elements:
5996
5997@example
59981 + 15
5999@end example
6000
6001@noindent
6002At this point, another reduction can be made, resulting in the single value
600316. Then the newline token can be shifted.
6004
6005The parser tries, by shifts and reductions, to reduce the entire input down
6006to a single grouping whose symbol is the grammar's start-symbol
6007(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
6008
6009This kind of parser is known in the literature as a bottom-up parser.
6010
6011@menu
742e4900 6012* Lookahead:: Parser looks one token ahead when deciding what to do.
bfa74976
RS
6013* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
6014* Precedence:: Operator precedence works by resolving conflicts.
6015* Contextual Precedence:: When an operator's precedence depends on context.
6016* Parser States:: The parser is a finite-state-machine with stack.
6017* Reduce/Reduce:: When two rules are applicable in the same situation.
6018* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 6019* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 6020* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
6021@end menu
6022
742e4900
JD
6023@node Lookahead
6024@section Lookahead Tokens
6025@cindex lookahead token
bfa74976
RS
6026
6027The Bison parser does @emph{not} always reduce immediately as soon as the
6028last @var{n} tokens and groupings match a rule. This is because such a
6029simple strategy is inadequate to handle most languages. Instead, when a
6030reduction is possible, the parser sometimes ``looks ahead'' at the next
6031token in order to decide what to do.
6032
6033When a token is read, it is not immediately shifted; first it becomes the
742e4900 6034@dfn{lookahead token}, which is not on the stack. Now the parser can
bfa74976 6035perform one or more reductions of tokens and groupings on the stack, while
742e4900
JD
6036the lookahead token remains off to the side. When no more reductions
6037should take place, the lookahead token is shifted onto the stack. This
bfa74976 6038does not mean that all possible reductions have been done; depending on the
742e4900 6039token type of the lookahead token, some rules may choose to delay their
bfa74976
RS
6040application.
6041
742e4900 6042Here is a simple case where lookahead is needed. These three rules define
bfa74976
RS
6043expressions which contain binary addition operators and postfix unary
6044factorial operators (@samp{!}), and allow parentheses for grouping.
6045
6046@example
6047@group
6048expr: term '+' expr
6049 | term
6050 ;
6051@end group
6052
6053@group
6054term: '(' expr ')'
6055 | term '!'
6056 | NUMBER
6057 ;
6058@end group
6059@end example
6060
6061Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
6062should be done? If the following token is @samp{)}, then the first three
6063tokens must be reduced to form an @code{expr}. This is the only valid
6064course, because shifting the @samp{)} would produce a sequence of symbols
6065@w{@code{term ')'}}, and no rule allows this.
6066
6067If the following token is @samp{!}, then it must be shifted immediately so
6068that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
6069parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
6070@code{expr}. It would then be impossible to shift the @samp{!} because
6071doing so would produce on the stack the sequence of symbols @code{expr
6072'!'}. No rule allows that sequence.
6073
6074@vindex yychar
32c29292
JD
6075@vindex yylval
6076@vindex yylloc
742e4900 6077The lookahead token is stored in the variable @code{yychar}.
32c29292
JD
6078Its semantic value and location, if any, are stored in the variables
6079@code{yylval} and @code{yylloc}.
bfa74976
RS
6080@xref{Action Features, ,Special Features for Use in Actions}.
6081
342b8b6e 6082@node Shift/Reduce
bfa74976
RS
6083@section Shift/Reduce Conflicts
6084@cindex conflicts
6085@cindex shift/reduce conflicts
6086@cindex dangling @code{else}
6087@cindex @code{else}, dangling
6088
6089Suppose we are parsing a language which has if-then and if-then-else
6090statements, with a pair of rules like this:
6091
6092@example
6093@group
6094if_stmt:
6095 IF expr THEN stmt
6096 | IF expr THEN stmt ELSE stmt
6097 ;
6098@end group
6099@end example
6100
6101@noindent
6102Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
6103terminal symbols for specific keyword tokens.
6104
742e4900 6105When the @code{ELSE} token is read and becomes the lookahead token, the
bfa74976
RS
6106contents of the stack (assuming the input is valid) are just right for
6107reduction by the first rule. But it is also legitimate to shift the
6108@code{ELSE}, because that would lead to eventual reduction by the second
6109rule.
6110
6111This situation, where either a shift or a reduction would be valid, is
6112called a @dfn{shift/reduce conflict}. Bison is designed to resolve
6113these conflicts by choosing to shift, unless otherwise directed by
6114operator precedence declarations. To see the reason for this, let's
6115contrast it with the other alternative.
6116
6117Since the parser prefers to shift the @code{ELSE}, the result is to attach
6118the else-clause to the innermost if-statement, making these two inputs
6119equivalent:
6120
6121@example
6122if x then if y then win (); else lose;
6123
6124if x then do; if y then win (); else lose; end;
6125@end example
6126
6127But if the parser chose to reduce when possible rather than shift, the
6128result would be to attach the else-clause to the outermost if-statement,
6129making these two inputs equivalent:
6130
6131@example
6132if x then if y then win (); else lose;
6133
6134if x then do; if y then win (); end; else lose;
6135@end example
6136
6137The conflict exists because the grammar as written is ambiguous: either
6138parsing of the simple nested if-statement is legitimate. The established
6139convention is that these ambiguities are resolved by attaching the
6140else-clause to the innermost if-statement; this is what Bison accomplishes
6141by choosing to shift rather than reduce. (It would ideally be cleaner to
6142write an unambiguous grammar, but that is very hard to do in this case.)
6143This particular ambiguity was first encountered in the specifications of
6144Algol 60 and is called the ``dangling @code{else}'' ambiguity.
6145
6146To avoid warnings from Bison about predictable, legitimate shift/reduce
6147conflicts, use the @code{%expect @var{n}} declaration. There will be no
6148warning as long as the number of shift/reduce conflicts is exactly @var{n}.
6149@xref{Expect Decl, ,Suppressing Conflict Warnings}.
6150
6151The definition of @code{if_stmt} above is solely to blame for the
6152conflict, but the conflict does not actually appear without additional
6153rules. Here is a complete Bison input file that actually manifests the
6154conflict:
6155
6156@example
6157@group
6158%token IF THEN ELSE variable
6159%%
6160@end group
6161@group
6162stmt: expr
6163 | if_stmt
6164 ;
6165@end group
6166
6167@group
6168if_stmt:
6169 IF expr THEN stmt
6170 | IF expr THEN stmt ELSE stmt
6171 ;
6172@end group
6173
6174expr: variable
6175 ;
6176@end example
6177
342b8b6e 6178@node Precedence
bfa74976
RS
6179@section Operator Precedence
6180@cindex operator precedence
6181@cindex precedence of operators
6182
6183Another situation where shift/reduce conflicts appear is in arithmetic
6184expressions. Here shifting is not always the preferred resolution; the
6185Bison declarations for operator precedence allow you to specify when to
6186shift and when to reduce.
6187
6188@menu
6189* Why Precedence:: An example showing why precedence is needed.
6190* Using Precedence:: How to specify precedence in Bison grammars.
6191* Precedence Examples:: How these features are used in the previous example.
6192* How Precedence:: How they work.
6193@end menu
6194
342b8b6e 6195@node Why Precedence
bfa74976
RS
6196@subsection When Precedence is Needed
6197
6198Consider the following ambiguous grammar fragment (ambiguous because the
6199input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
6200
6201@example
6202@group
6203expr: expr '-' expr
6204 | expr '*' expr
6205 | expr '<' expr
6206 | '(' expr ')'
6207 @dots{}
6208 ;
6209@end group
6210@end example
6211
6212@noindent
6213Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
14ded682
AD
6214should it reduce them via the rule for the subtraction operator? It
6215depends on the next token. Of course, if the next token is @samp{)}, we
6216must reduce; shifting is invalid because no single rule can reduce the
6217token sequence @w{@samp{- 2 )}} or anything starting with that. But if
6218the next token is @samp{*} or @samp{<}, we have a choice: either
6219shifting or reduction would allow the parse to complete, but with
6220different results.
6221
6222To decide which one Bison should do, we must consider the results. If
6223the next operator token @var{op} is shifted, then it must be reduced
6224first in order to permit another opportunity to reduce the difference.
6225The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
6226hand, if the subtraction is reduced before shifting @var{op}, the result
6227is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
6228reduce should depend on the relative precedence of the operators
6229@samp{-} and @var{op}: @samp{*} should be shifted first, but not
6230@samp{<}.
bfa74976
RS
6231
6232@cindex associativity
6233What about input such as @w{@samp{1 - 2 - 5}}; should this be
14ded682
AD
6234@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
6235operators we prefer the former, which is called @dfn{left association}.
6236The latter alternative, @dfn{right association}, is desirable for
6237assignment operators. The choice of left or right association is a
6238matter of whether the parser chooses to shift or reduce when the stack
742e4900 6239contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
14ded682 6240makes right-associativity.
bfa74976 6241
342b8b6e 6242@node Using Precedence
bfa74976
RS
6243@subsection Specifying Operator Precedence
6244@findex %left
6245@findex %right
6246@findex %nonassoc
6247
6248Bison allows you to specify these choices with the operator precedence
6249declarations @code{%left} and @code{%right}. Each such declaration
6250contains a list of tokens, which are operators whose precedence and
6251associativity is being declared. The @code{%left} declaration makes all
6252those operators left-associative and the @code{%right} declaration makes
6253them right-associative. A third alternative is @code{%nonassoc}, which
6254declares that it is a syntax error to find the same operator twice ``in a
6255row''.
6256
6257The relative precedence of different operators is controlled by the
6258order in which they are declared. The first @code{%left} or
6259@code{%right} declaration in the file declares the operators whose
6260precedence is lowest, the next such declaration declares the operators
6261whose precedence is a little higher, and so on.
6262
342b8b6e 6263@node Precedence Examples
bfa74976
RS
6264@subsection Precedence Examples
6265
6266In our example, we would want the following declarations:
6267
6268@example
6269%left '<'
6270%left '-'
6271%left '*'
6272@end example
6273
6274In a more complete example, which supports other operators as well, we
6275would declare them in groups of equal precedence. For example, @code{'+'} is
6276declared with @code{'-'}:
6277
6278@example
6279%left '<' '>' '=' NE LE GE
6280%left '+' '-'
6281%left '*' '/'
6282@end example
6283
6284@noindent
6285(Here @code{NE} and so on stand for the operators for ``not equal''
6286and so on. We assume that these tokens are more than one character long
6287and therefore are represented by names, not character literals.)
6288
342b8b6e 6289@node How Precedence
bfa74976
RS
6290@subsection How Precedence Works
6291
6292The first effect of the precedence declarations is to assign precedence
6293levels to the terminal symbols declared. The second effect is to assign
704a47c4
AD
6294precedence levels to certain rules: each rule gets its precedence from
6295the last terminal symbol mentioned in the components. (You can also
6296specify explicitly the precedence of a rule. @xref{Contextual
6297Precedence, ,Context-Dependent Precedence}.)
6298
6299Finally, the resolution of conflicts works by comparing the precedence
742e4900 6300of the rule being considered with that of the lookahead token. If the
704a47c4
AD
6301token's precedence is higher, the choice is to shift. If the rule's
6302precedence is higher, the choice is to reduce. If they have equal
6303precedence, the choice is made based on the associativity of that
6304precedence level. The verbose output file made by @samp{-v}
6305(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
6306resolved.
bfa74976
RS
6307
6308Not all rules and not all tokens have precedence. If either the rule or
742e4900 6309the lookahead token has no precedence, then the default is to shift.
bfa74976 6310
342b8b6e 6311@node Contextual Precedence
bfa74976
RS
6312@section Context-Dependent Precedence
6313@cindex context-dependent precedence
6314@cindex unary operator precedence
6315@cindex precedence, context-dependent
6316@cindex precedence, unary operator
6317@findex %prec
6318
6319Often the precedence of an operator depends on the context. This sounds
6320outlandish at first, but it is really very common. For example, a minus
6321sign typically has a very high precedence as a unary operator, and a
6322somewhat lower precedence (lower than multiplication) as a binary operator.
6323
6324The Bison precedence declarations, @code{%left}, @code{%right} and
6325@code{%nonassoc}, can only be used once for a given token; so a token has
6326only one precedence declared in this way. For context-dependent
6327precedence, you need to use an additional mechanism: the @code{%prec}
e0c471a9 6328modifier for rules.
bfa74976
RS
6329
6330The @code{%prec} modifier declares the precedence of a particular rule by
6331specifying a terminal symbol whose precedence should be used for that rule.
6332It's not necessary for that symbol to appear otherwise in the rule. The
6333modifier's syntax is:
6334
6335@example
6336%prec @var{terminal-symbol}
6337@end example
6338
6339@noindent
6340and it is written after the components of the rule. Its effect is to
6341assign the rule the precedence of @var{terminal-symbol}, overriding
6342the precedence that would be deduced for it in the ordinary way. The
6343altered rule precedence then affects how conflicts involving that rule
6344are resolved (@pxref{Precedence, ,Operator Precedence}).
6345
6346Here is how @code{%prec} solves the problem of unary minus. First, declare
6347a precedence for a fictitious terminal symbol named @code{UMINUS}. There
6348are no tokens of this type, but the symbol serves to stand for its
6349precedence:
6350
6351@example
6352@dots{}
6353%left '+' '-'
6354%left '*'
6355%left UMINUS
6356@end example
6357
6358Now the precedence of @code{UMINUS} can be used in specific rules:
6359
6360@example
6361@group
6362exp: @dots{}
6363 | exp '-' exp
6364 @dots{}
6365 | '-' exp %prec UMINUS
6366@end group
6367@end example
6368
91d2c560 6369@ifset defaultprec
39a06c25
PE
6370If you forget to append @code{%prec UMINUS} to the rule for unary
6371minus, Bison silently assumes that minus has its usual precedence.
6372This kind of problem can be tricky to debug, since one typically
6373discovers the mistake only by testing the code.
6374
22fccf95 6375The @code{%no-default-prec;} declaration makes it easier to discover
39a06c25
PE
6376this kind of problem systematically. It causes rules that lack a
6377@code{%prec} modifier to have no precedence, even if the last terminal
6378symbol mentioned in their components has a declared precedence.
6379
22fccf95 6380If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
39a06c25
PE
6381for all rules that participate in precedence conflict resolution.
6382Then you will see any shift/reduce conflict until you tell Bison how
6383to resolve it, either by changing your grammar or by adding an
6384explicit precedence. This will probably add declarations to the
6385grammar, but it helps to protect against incorrect rule precedences.
6386
22fccf95
PE
6387The effect of @code{%no-default-prec;} can be reversed by giving
6388@code{%default-prec;}, which is the default.
91d2c560 6389@end ifset
39a06c25 6390
342b8b6e 6391@node Parser States
bfa74976
RS
6392@section Parser States
6393@cindex finite-state machine
6394@cindex parser state
6395@cindex state (of parser)
6396
6397The function @code{yyparse} is implemented using a finite-state machine.
6398The values pushed on the parser stack are not simply token type codes; they
6399represent the entire sequence of terminal and nonterminal symbols at or
6400near the top of the stack. The current state collects all the information
6401about previous input which is relevant to deciding what to do next.
6402
742e4900
JD
6403Each time a lookahead token is read, the current parser state together
6404with the type of lookahead token are looked up in a table. This table
6405entry can say, ``Shift the lookahead token.'' In this case, it also
bfa74976
RS
6406specifies the new parser state, which is pushed onto the top of the
6407parser stack. Or it can say, ``Reduce using rule number @var{n}.''
6408This means that a certain number of tokens or groupings are taken off
6409the top of the stack, and replaced by one grouping. In other words,
6410that number of states are popped from the stack, and one new state is
6411pushed.
6412
742e4900 6413There is one other alternative: the table can say that the lookahead token
bfa74976
RS
6414is erroneous in the current state. This causes error processing to begin
6415(@pxref{Error Recovery}).
6416
342b8b6e 6417@node Reduce/Reduce
bfa74976
RS
6418@section Reduce/Reduce Conflicts
6419@cindex reduce/reduce conflict
6420@cindex conflicts, reduce/reduce
6421
6422A reduce/reduce conflict occurs if there are two or more rules that apply
6423to the same sequence of input. This usually indicates a serious error
6424in the grammar.
6425
6426For example, here is an erroneous attempt to define a sequence
6427of zero or more @code{word} groupings.
6428
6429@example
6430sequence: /* empty */
6431 @{ printf ("empty sequence\n"); @}
6432 | maybeword
6433 | sequence word
6434 @{ printf ("added word %s\n", $2); @}
6435 ;
6436
6437maybeword: /* empty */
6438 @{ printf ("empty maybeword\n"); @}
6439 | word
6440 @{ printf ("single word %s\n", $1); @}
6441 ;
6442@end example
6443
6444@noindent
6445The error is an ambiguity: there is more than one way to parse a single
6446@code{word} into a @code{sequence}. It could be reduced to a
6447@code{maybeword} and then into a @code{sequence} via the second rule.
6448Alternatively, nothing-at-all could be reduced into a @code{sequence}
6449via the first rule, and this could be combined with the @code{word}
6450using the third rule for @code{sequence}.
6451
6452There is also more than one way to reduce nothing-at-all into a
6453@code{sequence}. This can be done directly via the first rule,
6454or indirectly via @code{maybeword} and then the second rule.
6455
6456You might think that this is a distinction without a difference, because it
6457does not change whether any particular input is valid or not. But it does
6458affect which actions are run. One parsing order runs the second rule's
6459action; the other runs the first rule's action and the third rule's action.
6460In this example, the output of the program changes.
6461
6462Bison resolves a reduce/reduce conflict by choosing to use the rule that
6463appears first in the grammar, but it is very risky to rely on this. Every
6464reduce/reduce conflict must be studied and usually eliminated. Here is the
6465proper way to define @code{sequence}:
6466
6467@example
6468sequence: /* empty */
6469 @{ printf ("empty sequence\n"); @}
6470 | sequence word
6471 @{ printf ("added word %s\n", $2); @}
6472 ;
6473@end example
6474
6475Here is another common error that yields a reduce/reduce conflict:
6476
6477@example
6478sequence: /* empty */
6479 | sequence words
6480 | sequence redirects
6481 ;
6482
6483words: /* empty */
6484 | words word
6485 ;
6486
6487redirects:/* empty */
6488 | redirects redirect
6489 ;
6490@end example
6491
6492@noindent
6493The intention here is to define a sequence which can contain either
6494@code{word} or @code{redirect} groupings. The individual definitions of
6495@code{sequence}, @code{words} and @code{redirects} are error-free, but the
6496three together make a subtle ambiguity: even an empty input can be parsed
6497in infinitely many ways!
6498
6499Consider: nothing-at-all could be a @code{words}. Or it could be two
6500@code{words} in a row, or three, or any number. It could equally well be a
6501@code{redirects}, or two, or any number. Or it could be a @code{words}
6502followed by three @code{redirects} and another @code{words}. And so on.
6503
6504Here are two ways to correct these rules. First, to make it a single level
6505of sequence:
6506
6507@example
6508sequence: /* empty */
6509 | sequence word
6510 | sequence redirect
6511 ;
6512@end example
6513
6514Second, to prevent either a @code{words} or a @code{redirects}
6515from being empty:
6516
6517@example
6518sequence: /* empty */
6519 | sequence words
6520 | sequence redirects
6521 ;
6522
6523words: word
6524 | words word
6525 ;
6526
6527redirects:redirect
6528 | redirects redirect
6529 ;
6530@end example
6531
342b8b6e 6532@node Mystery Conflicts
bfa74976
RS
6533@section Mysterious Reduce/Reduce Conflicts
6534
6535Sometimes reduce/reduce conflicts can occur that don't look warranted.
6536Here is an example:
6537
6538@example
6539@group
6540%token ID
6541
6542%%
6543def: param_spec return_spec ','
6544 ;
6545param_spec:
6546 type
6547 | name_list ':' type
6548 ;
6549@end group
6550@group
6551return_spec:
6552 type
6553 | name ':' type
6554 ;
6555@end group
6556@group
6557type: ID
6558 ;
6559@end group
6560@group
6561name: ID
6562 ;
6563name_list:
6564 name
6565 | name ',' name_list
6566 ;
6567@end group
6568@end example
6569
6570It would seem that this grammar can be parsed with only a single token
742e4900 6571of lookahead: when a @code{param_spec} is being read, an @code{ID} is
bfa74976 6572a @code{name} if a comma or colon follows, or a @code{type} if another
c827f760 6573@code{ID} follows. In other words, this grammar is @acronym{LR}(1).
bfa74976 6574
c827f760
PE
6575@cindex @acronym{LR}(1)
6576@cindex @acronym{LALR}(1)
bfa74976 6577However, Bison, like most parser generators, cannot actually handle all
c827f760
PE
6578@acronym{LR}(1) grammars. In this grammar, two contexts, that after
6579an @code{ID}
bfa74976
RS
6580at the beginning of a @code{param_spec} and likewise at the beginning of
6581a @code{return_spec}, are similar enough that Bison assumes they are the
6582same. They appear similar because the same set of rules would be
6583active---the rule for reducing to a @code{name} and that for reducing to
6584a @code{type}. Bison is unable to determine at that stage of processing
742e4900 6585that the rules would require different lookahead tokens in the two
bfa74976
RS
6586contexts, so it makes a single parser state for them both. Combining
6587the two contexts causes a conflict later. In parser terminology, this
c827f760 6588occurrence means that the grammar is not @acronym{LALR}(1).
bfa74976
RS
6589
6590In general, it is better to fix deficiencies than to document them. But
6591this particular deficiency is intrinsically hard to fix; parser
c827f760
PE
6592generators that can handle @acronym{LR}(1) grammars are hard to write
6593and tend to
bfa74976
RS
6594produce parsers that are very large. In practice, Bison is more useful
6595as it is now.
6596
6597When the problem arises, you can often fix it by identifying the two
a220f555
MA
6598parser states that are being confused, and adding something to make them
6599look distinct. In the above example, adding one rule to
bfa74976
RS
6600@code{return_spec} as follows makes the problem go away:
6601
6602@example
6603@group
6604%token BOGUS
6605@dots{}
6606%%
6607@dots{}
6608return_spec:
6609 type
6610 | name ':' type
6611 /* This rule is never used. */
6612 | ID BOGUS
6613 ;
6614@end group
6615@end example
6616
6617This corrects the problem because it introduces the possibility of an
6618additional active rule in the context after the @code{ID} at the beginning of
6619@code{return_spec}. This rule is not active in the corresponding context
6620in a @code{param_spec}, so the two contexts receive distinct parser states.
6621As long as the token @code{BOGUS} is never generated by @code{yylex},
6622the added rule cannot alter the way actual input is parsed.
6623
6624In this particular example, there is another way to solve the problem:
6625rewrite the rule for @code{return_spec} to use @code{ID} directly
6626instead of via @code{name}. This also causes the two confusing
6627contexts to have different sets of active rules, because the one for
6628@code{return_spec} activates the altered rule for @code{return_spec}
6629rather than the one for @code{name}.
6630
6631@example
6632param_spec:
6633 type
6634 | name_list ':' type
6635 ;
6636return_spec:
6637 type
6638 | ID ':' type
6639 ;
6640@end example
6641
e054b190
PE
6642For a more detailed exposition of @acronym{LALR}(1) parsers and parser
6643generators, please see:
6644Frank DeRemer and Thomas Pennello, Efficient Computation of
6645@acronym{LALR}(1) Look-Ahead Sets, @cite{@acronym{ACM} Transactions on
6646Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
6647pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
6648
fae437e8 6649@node Generalized LR Parsing
c827f760
PE
6650@section Generalized @acronym{LR} (@acronym{GLR}) Parsing
6651@cindex @acronym{GLR} parsing
6652@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2 6653@cindex ambiguous grammars
9d9b8b70 6654@cindex nondeterministic parsing
676385e2 6655
fae437e8
AD
6656Bison produces @emph{deterministic} parsers that choose uniquely
6657when to reduce and which reduction to apply
742e4900 6658based on a summary of the preceding input and on one extra token of lookahead.
676385e2
PH
6659As a result, normal Bison handles a proper subset of the family of
6660context-free languages.
fae437e8 6661Ambiguous grammars, since they have strings with more than one possible
676385e2
PH
6662sequence of reductions cannot have deterministic parsers in this sense.
6663The same is true of languages that require more than one symbol of
742e4900 6664lookahead, since the parser lacks the information necessary to make a
676385e2 6665decision at the point it must be made in a shift-reduce parser.
fae437e8 6666Finally, as previously mentioned (@pxref{Mystery Conflicts}),
676385e2
PH
6667there are languages where Bison's particular choice of how to
6668summarize the input seen so far loses necessary information.
6669
6670When you use the @samp{%glr-parser} declaration in your grammar file,
6671Bison generates a parser that uses a different algorithm, called
c827f760
PE
6672Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR}
6673parser uses the same basic
676385e2
PH
6674algorithm for parsing as an ordinary Bison parser, but behaves
6675differently in cases where there is a shift-reduce conflict that has not
fae437e8 6676been resolved by precedence rules (@pxref{Precedence}) or a
c827f760
PE
6677reduce-reduce conflict. When a @acronym{GLR} parser encounters such a
6678situation, it
fae437e8 6679effectively @emph{splits} into a several parsers, one for each possible
676385e2
PH
6680shift or reduction. These parsers then proceed as usual, consuming
6681tokens in lock-step. Some of the stacks may encounter other conflicts
fae437e8 6682and split further, with the result that instead of a sequence of states,
c827f760 6683a Bison @acronym{GLR} parsing stack is what is in effect a tree of states.
676385e2
PH
6684
6685In effect, each stack represents a guess as to what the proper parse
6686is. Additional input may indicate that a guess was wrong, in which case
6687the appropriate stack silently disappears. Otherwise, the semantics
fae437e8 6688actions generated in each stack are saved, rather than being executed
676385e2 6689immediately. When a stack disappears, its saved semantic actions never
fae437e8 6690get executed. When a reduction causes two stacks to become equivalent,
676385e2
PH
6691their sets of semantic actions are both saved with the state that
6692results from the reduction. We say that two stacks are equivalent
fae437e8 6693when they both represent the same sequence of states,
676385e2
PH
6694and each pair of corresponding states represents a
6695grammar symbol that produces the same segment of the input token
6696stream.
6697
6698Whenever the parser makes a transition from having multiple
c827f760 6699states to having one, it reverts to the normal @acronym{LALR}(1) parsing
676385e2
PH
6700algorithm, after resolving and executing the saved-up actions.
6701At this transition, some of the states on the stack will have semantic
6702values that are sets (actually multisets) of possible actions. The
6703parser tries to pick one of the actions by first finding one whose rule
6704has the highest dynamic precedence, as set by the @samp{%dprec}
fae437e8 6705declaration. Otherwise, if the alternative actions are not ordered by
676385e2 6706precedence, but there the same merging function is declared for both
fae437e8 6707rules by the @samp{%merge} declaration,
676385e2
PH
6708Bison resolves and evaluates both and then calls the merge function on
6709the result. Otherwise, it reports an ambiguity.
6710
c827f760
PE
6711It is possible to use a data structure for the @acronym{GLR} parsing tree that
6712permits the processing of any @acronym{LALR}(1) grammar in linear time (in the
6713size of the input), any unambiguous (not necessarily
6714@acronym{LALR}(1)) grammar in
fae437e8 6715quadratic worst-case time, and any general (possibly ambiguous)
676385e2
PH
6716context-free grammar in cubic worst-case time. However, Bison currently
6717uses a simpler data structure that requires time proportional to the
6718length of the input times the maximum number of stacks required for any
9d9b8b70 6719prefix of the input. Thus, really ambiguous or nondeterministic
676385e2
PH
6720grammars can require exponential time and space to process. Such badly
6721behaving examples, however, are not generally of practical interest.
9d9b8b70 6722Usually, nondeterminism in a grammar is local---the parser is ``in
676385e2 6723doubt'' only for a few tokens at a time. Therefore, the current data
c827f760 6724structure should generally be adequate. On @acronym{LALR}(1) portions of a
676385e2
PH
6725grammar, in particular, it is only slightly slower than with the default
6726Bison parser.
6727
fa7e68c3 6728For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth
f6481e2f
PE
6729Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
6730Generalised @acronym{LR} Parsers, Royal Holloway, University of
6731London, Department of Computer Science, TR-00-12,
6732@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
6733(2000-12-24).
6734
1a059451
PE
6735@node Memory Management
6736@section Memory Management, and How to Avoid Memory Exhaustion
6737@cindex memory exhaustion
6738@cindex memory management
bfa74976
RS
6739@cindex stack overflow
6740@cindex parser stack overflow
6741@cindex overflow of parser stack
6742
1a059451 6743The Bison parser stack can run out of memory if too many tokens are shifted and
bfa74976 6744not reduced. When this happens, the parser function @code{yyparse}
1a059451 6745calls @code{yyerror} and then returns 2.
bfa74976 6746
c827f760 6747Because Bison parsers have growing stacks, hitting the upper limit
d1a1114f
AD
6748usually results from using a right recursion instead of a left
6749recursion, @xref{Recursion, ,Recursive Rules}.
6750
bfa74976
RS
6751@vindex YYMAXDEPTH
6752By defining the macro @code{YYMAXDEPTH}, you can control how deep the
1a059451 6753parser stack can become before memory is exhausted. Define the
bfa74976
RS
6754macro with a value that is an integer. This value is the maximum number
6755of tokens that can be shifted (and not reduced) before overflow.
bfa74976
RS
6756
6757The stack space allowed is not necessarily allocated. If you specify a
1a059451 6758large value for @code{YYMAXDEPTH}, the parser normally allocates a small
bfa74976
RS
6759stack at first, and then makes it bigger by stages as needed. This
6760increasing allocation happens automatically and silently. Therefore,
6761you do not need to make @code{YYMAXDEPTH} painfully small merely to save
6762space for ordinary inputs that do not need much stack.
6763
d7e14fc0
PE
6764However, do not allow @code{YYMAXDEPTH} to be a value so large that
6765arithmetic overflow could occur when calculating the size of the stack
6766space. Also, do not allow @code{YYMAXDEPTH} to be less than
6767@code{YYINITDEPTH}.
6768
bfa74976
RS
6769@cindex default stack limit
6770The default value of @code{YYMAXDEPTH}, if you do not define it, is
677110000.
6772
6773@vindex YYINITDEPTH
6774You can control how much stack is allocated initially by defining the
d7e14fc0
PE
6775macro @code{YYINITDEPTH} to a positive integer. For the C
6776@acronym{LALR}(1) parser, this value must be a compile-time constant
6777unless you are assuming C99 or some other target language or compiler
6778that allows variable-length arrays. The default is 200.
6779
1a059451 6780Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
bfa74976 6781
d1a1114f 6782@c FIXME: C++ output.
c827f760 6783Because of semantical differences between C and C++, the
1a059451
PE
6784@acronym{LALR}(1) parsers in C produced by Bison cannot grow when compiled
6785by C++ compilers. In this precise case (compiling a C parser as C++) you are
6786suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix
6787this deficiency in a future release.
d1a1114f 6788
342b8b6e 6789@node Error Recovery
bfa74976
RS
6790@chapter Error Recovery
6791@cindex error recovery
6792@cindex recovery from errors
6793
6e649e65 6794It is not usually acceptable to have a program terminate on a syntax
bfa74976
RS
6795error. For example, a compiler should recover sufficiently to parse the
6796rest of the input file and check it for errors; a calculator should accept
6797another expression.
6798
6799In a simple interactive command parser where each input is one line, it may
6800be sufficient to allow @code{yyparse} to return 1 on error and have the
6801caller ignore the rest of the input line when that happens (and then call
6802@code{yyparse} again). But this is inadequate for a compiler, because it
6803forgets all the syntactic context leading up to the error. A syntax error
6804deep within a function in the compiler input should not cause the compiler
6805to treat the following line like the beginning of a source file.
6806
6807@findex error
6808You can define how to recover from a syntax error by writing rules to
6809recognize the special token @code{error}. This is a terminal symbol that
6810is always defined (you need not declare it) and reserved for error
6811handling. The Bison parser generates an @code{error} token whenever a
6812syntax error happens; if you have provided a rule to recognize this token
13863333 6813in the current context, the parse can continue.
bfa74976
RS
6814
6815For example:
6816
6817@example
6818stmnts: /* empty string */
6819 | stmnts '\n'
6820 | stmnts exp '\n'
6821 | stmnts error '\n'
6822@end example
6823
6824The fourth rule in this example says that an error followed by a newline
6825makes a valid addition to any @code{stmnts}.
6826
6827What happens if a syntax error occurs in the middle of an @code{exp}? The
6828error recovery rule, interpreted strictly, applies to the precise sequence
6829of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
6830the middle of an @code{exp}, there will probably be some additional tokens
6831and subexpressions on the stack after the last @code{stmnts}, and there
6832will be tokens to read before the next newline. So the rule is not
6833applicable in the ordinary way.
6834
6835But Bison can force the situation to fit the rule, by discarding part of
72f889cc
AD
6836the semantic context and part of the input. First it discards states
6837and objects from the stack until it gets back to a state in which the
bfa74976 6838@code{error} token is acceptable. (This means that the subexpressions
72f889cc
AD
6839already parsed are discarded, back to the last complete @code{stmnts}.)
6840At this point the @code{error} token can be shifted. Then, if the old
742e4900 6841lookahead token is not acceptable to be shifted next, the parser reads
bfa74976 6842tokens and discards them until it finds a token which is acceptable. In
72f889cc
AD
6843this example, Bison reads and discards input until the next newline so
6844that the fourth rule can apply. Note that discarded symbols are
6845possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
6846Discarded Symbols}, for a means to reclaim this memory.
bfa74976
RS
6847
6848The choice of error rules in the grammar is a choice of strategies for
6849error recovery. A simple and useful strategy is simply to skip the rest of
6850the current input line or current statement if an error is detected:
6851
6852@example
72d2299c 6853stmnt: error ';' /* On error, skip until ';' is read. */
bfa74976
RS
6854@end example
6855
6856It is also useful to recover to the matching close-delimiter of an
6857opening-delimiter that has already been parsed. Otherwise the
6858close-delimiter will probably appear to be unmatched, and generate another,
6859spurious error message:
6860
6861@example
6862primary: '(' expr ')'
6863 | '(' error ')'
6864 @dots{}
6865 ;
6866@end example
6867
6868Error recovery strategies are necessarily guesses. When they guess wrong,
6869one syntax error often leads to another. In the above example, the error
6870recovery rule guesses that an error is due to bad input within one
6871@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
6872middle of a valid @code{stmnt}. After the error recovery rule recovers
6873from the first error, another syntax error will be found straightaway,
6874since the text following the spurious semicolon is also an invalid
6875@code{stmnt}.
6876
6877To prevent an outpouring of error messages, the parser will output no error
6878message for another syntax error that happens shortly after the first; only
6879after three consecutive input tokens have been successfully shifted will
6880error messages resume.
6881
6882Note that rules which accept the @code{error} token may have actions, just
6883as any other rules can.
6884
6885@findex yyerrok
6886You can make error messages resume immediately by using the macro
6887@code{yyerrok} in an action. If you do this in the error rule's action, no
6888error messages will be suppressed. This macro requires no arguments;
6889@samp{yyerrok;} is a valid C statement.
6890
6891@findex yyclearin
742e4900 6892The previous lookahead token is reanalyzed immediately after an error. If
bfa74976
RS
6893this is unacceptable, then the macro @code{yyclearin} may be used to clear
6894this token. Write the statement @samp{yyclearin;} in the error rule's
6895action.
32c29292 6896@xref{Action Features, ,Special Features for Use in Actions}.
bfa74976 6897
6e649e65 6898For example, suppose that on a syntax error, an error handling routine is
bfa74976
RS
6899called that advances the input stream to some point where parsing should
6900once again commence. The next symbol returned by the lexical scanner is
742e4900 6901probably correct. The previous lookahead token ought to be discarded
bfa74976
RS
6902with @samp{yyclearin;}.
6903
6904@vindex YYRECOVERING
02103984
PE
6905The expression @code{YYRECOVERING ()} yields 1 when the parser
6906is recovering from a syntax error, and 0 otherwise.
6907Syntax error diagnostics are suppressed while recovering from a syntax
6908error.
bfa74976 6909
342b8b6e 6910@node Context Dependency
bfa74976
RS
6911@chapter Handling Context Dependencies
6912
6913The Bison paradigm is to parse tokens first, then group them into larger
6914syntactic units. In many languages, the meaning of a token is affected by
6915its context. Although this violates the Bison paradigm, certain techniques
6916(known as @dfn{kludges}) may enable you to write Bison parsers for such
6917languages.
6918
6919@menu
6920* Semantic Tokens:: Token parsing can depend on the semantic context.
6921* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
6922* Tie-in Recovery:: Lexical tie-ins have implications for how
6923 error recovery rules must be written.
6924@end menu
6925
6926(Actually, ``kludge'' means any technique that gets its job done but is
6927neither clean nor robust.)
6928
342b8b6e 6929@node Semantic Tokens
bfa74976
RS
6930@section Semantic Info in Token Types
6931
6932The C language has a context dependency: the way an identifier is used
6933depends on what its current meaning is. For example, consider this:
6934
6935@example
6936foo (x);
6937@end example
6938
6939This looks like a function call statement, but if @code{foo} is a typedef
6940name, then this is actually a declaration of @code{x}. How can a Bison
6941parser for C decide how to parse this input?
6942
c827f760 6943The method used in @acronym{GNU} C is to have two different token types,
bfa74976
RS
6944@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
6945identifier, it looks up the current declaration of the identifier in order
6946to decide which token type to return: @code{TYPENAME} if the identifier is
6947declared as a typedef, @code{IDENTIFIER} otherwise.
6948
6949The grammar rules can then express the context dependency by the choice of
6950token type to recognize. @code{IDENTIFIER} is accepted as an expression,
6951but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
6952@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
6953is @emph{not} significant, such as in declarations that can shadow a
6954typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
6955accepted---there is one rule for each of the two token types.
6956
6957This technique is simple to use if the decision of which kinds of
6958identifiers to allow is made at a place close to where the identifier is
6959parsed. But in C this is not always so: C allows a declaration to
6960redeclare a typedef name provided an explicit type has been specified
6961earlier:
6962
6963@example
3a4f411f
PE
6964typedef int foo, bar;
6965int baz (void)
6966@{
6967 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
6968 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
6969 return foo (bar);
6970@}
bfa74976
RS
6971@end example
6972
6973Unfortunately, the name being declared is separated from the declaration
6974construct itself by a complicated syntactic structure---the ``declarator''.
6975
9ecbd125 6976As a result, part of the Bison parser for C needs to be duplicated, with
14ded682
AD
6977all the nonterminal names changed: once for parsing a declaration in
6978which a typedef name can be redefined, and once for parsing a
6979declaration in which that can't be done. Here is a part of the
6980duplication, with actions omitted for brevity:
bfa74976
RS
6981
6982@example
6983initdcl:
6984 declarator maybeasm '='
6985 init
6986 | declarator maybeasm
6987 ;
6988
6989notype_initdcl:
6990 notype_declarator maybeasm '='
6991 init
6992 | notype_declarator maybeasm
6993 ;
6994@end example
6995
6996@noindent
6997Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
6998cannot. The distinction between @code{declarator} and
6999@code{notype_declarator} is the same sort of thing.
7000
7001There is some similarity between this technique and a lexical tie-in
7002(described next), in that information which alters the lexical analysis is
7003changed during parsing by other parts of the program. The difference is
7004here the information is global, and is used for other purposes in the
7005program. A true lexical tie-in has a special-purpose flag controlled by
7006the syntactic context.
7007
342b8b6e 7008@node Lexical Tie-ins
bfa74976
RS
7009@section Lexical Tie-ins
7010@cindex lexical tie-in
7011
7012One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
7013which is set by Bison actions, whose purpose is to alter the way tokens are
7014parsed.
7015
7016For example, suppose we have a language vaguely like C, but with a special
7017construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
7018an expression in parentheses in which all integers are hexadecimal. In
7019particular, the token @samp{a1b} must be treated as an integer rather than
7020as an identifier if it appears in that context. Here is how you can do it:
7021
7022@example
7023@group
7024%@{
38a92d50
PE
7025 int hexflag;
7026 int yylex (void);
7027 void yyerror (char const *);
bfa74976
RS
7028%@}
7029%%
7030@dots{}
7031@end group
7032@group
7033expr: IDENTIFIER
7034 | constant
7035 | HEX '('
7036 @{ hexflag = 1; @}
7037 expr ')'
7038 @{ hexflag = 0;
7039 $$ = $4; @}
7040 | expr '+' expr
7041 @{ $$ = make_sum ($1, $3); @}
7042 @dots{}
7043 ;
7044@end group
7045
7046@group
7047constant:
7048 INTEGER
7049 | STRING
7050 ;
7051@end group
7052@end example
7053
7054@noindent
7055Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
7056it is nonzero, all integers are parsed in hexadecimal, and tokens starting
7057with letters are parsed as integers if possible.
7058
342b8b6e
AD
7059The declaration of @code{hexflag} shown in the prologue of the parser file
7060is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}).
75f5aaea 7061You must also write the code in @code{yylex} to obey the flag.
bfa74976 7062
342b8b6e 7063@node Tie-in Recovery
bfa74976
RS
7064@section Lexical Tie-ins and Error Recovery
7065
7066Lexical tie-ins make strict demands on any error recovery rules you have.
7067@xref{Error Recovery}.
7068
7069The reason for this is that the purpose of an error recovery rule is to
7070abort the parsing of one construct and resume in some larger construct.
7071For example, in C-like languages, a typical error recovery rule is to skip
7072tokens until the next semicolon, and then start a new statement, like this:
7073
7074@example
7075stmt: expr ';'
7076 | IF '(' expr ')' stmt @{ @dots{} @}
7077 @dots{}
7078 error ';'
7079 @{ hexflag = 0; @}
7080 ;
7081@end example
7082
7083If there is a syntax error in the middle of a @samp{hex (@var{expr})}
7084construct, this error rule will apply, and then the action for the
7085completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
7086remain set for the entire rest of the input, or until the next @code{hex}
7087keyword, causing identifiers to be misinterpreted as integers.
7088
7089To avoid this problem the error recovery rule itself clears @code{hexflag}.
7090
7091There may also be an error recovery rule that works within expressions.
7092For example, there could be a rule which applies within parentheses
7093and skips to the close-parenthesis:
7094
7095@example
7096@group
7097expr: @dots{}
7098 | '(' expr ')'
7099 @{ $$ = $2; @}
7100 | '(' error ')'
7101 @dots{}
7102@end group
7103@end example
7104
7105If this rule acts within the @code{hex} construct, it is not going to abort
7106that construct (since it applies to an inner level of parentheses within
7107the construct). Therefore, it should not clear the flag: the rest of
7108the @code{hex} construct should be parsed with the flag still in effect.
7109
7110What if there is an error recovery rule which might abort out of the
7111@code{hex} construct or might not, depending on circumstances? There is no
7112way you can write the action to determine whether a @code{hex} construct is
7113being aborted or not. So if you are using a lexical tie-in, you had better
7114make sure your error recovery rules are not of this kind. Each rule must
7115be such that you can be sure that it always will, or always won't, have to
7116clear the flag.
7117
ec3bc396
AD
7118@c ================================================== Debugging Your Parser
7119
342b8b6e 7120@node Debugging
bfa74976 7121@chapter Debugging Your Parser
ec3bc396
AD
7122
7123Developing a parser can be a challenge, especially if you don't
7124understand the algorithm (@pxref{Algorithm, ,The Bison Parser
7125Algorithm}). Even so, sometimes a detailed description of the automaton
7126can help (@pxref{Understanding, , Understanding Your Parser}), or
7127tracing the execution of the parser can give some insight on why it
7128behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
7129
7130@menu
7131* Understanding:: Understanding the structure of your parser.
7132* Tracing:: Tracing the execution of your parser.
7133@end menu
7134
7135@node Understanding
7136@section Understanding Your Parser
7137
7138As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
7139Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
7140frequent than one would hope), looking at this automaton is required to
7141tune or simply fix a parser. Bison provides two different
35fe0834 7142representation of it, either textually or graphically (as a DOT file).
ec3bc396
AD
7143
7144The textual file is generated when the options @option{--report} or
7145@option{--verbose} are specified, see @xref{Invocation, , Invoking
7146Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
7147the parser output file name, and adding @samp{.output} instead.
7148Therefore, if the input file is @file{foo.y}, then the parser file is
7149called @file{foo.tab.c} by default. As a consequence, the verbose
7150output file is called @file{foo.output}.
7151
7152The following grammar file, @file{calc.y}, will be used in the sequel:
7153
7154@example
7155%token NUM STR
7156%left '+' '-'
7157%left '*'
7158%%
7159exp: exp '+' exp
7160 | exp '-' exp
7161 | exp '*' exp
7162 | exp '/' exp
7163 | NUM
7164 ;
7165useless: STR;
7166%%
7167@end example
7168
88bce5a2
AD
7169@command{bison} reports:
7170
7171@example
cff03fb2
JD
7172calc.y: warning: 1 nonterminal and 1 rule useless in grammar
7173calc.y:11.1-7: warning: nonterminal useless in grammar: useless
7174calc.y:11.10-12: warning: rule useless in grammar: useless: STR
5a99098d 7175calc.y: conflicts: 7 shift/reduce
88bce5a2
AD
7176@end example
7177
7178When given @option{--report=state}, in addition to @file{calc.tab.c}, it
7179creates a file @file{calc.output} with contents detailed below. The
7180order of the output and the exact presentation might vary, but the
7181interpretation is the same.
ec3bc396
AD
7182
7183The first section includes details on conflicts that were solved thanks
7184to precedence and/or associativity:
7185
7186@example
7187Conflict in state 8 between rule 2 and token '+' resolved as reduce.
7188Conflict in state 8 between rule 2 and token '-' resolved as reduce.
7189Conflict in state 8 between rule 2 and token '*' resolved as shift.
7190@exdent @dots{}
7191@end example
7192
7193@noindent
7194The next section lists states that still have conflicts.
7195
7196@example
5a99098d
PE
7197State 8 conflicts: 1 shift/reduce
7198State 9 conflicts: 1 shift/reduce
7199State 10 conflicts: 1 shift/reduce
7200State 11 conflicts: 4 shift/reduce
ec3bc396
AD
7201@end example
7202
7203@noindent
7204@cindex token, useless
7205@cindex useless token
7206@cindex nonterminal, useless
7207@cindex useless nonterminal
7208@cindex rule, useless
7209@cindex useless rule
7210The next section reports useless tokens, nonterminal and rules. Useless
7211nonterminals and rules are removed in order to produce a smaller parser,
7212but useless tokens are preserved, since they might be used by the
d80fb37a 7213scanner (note the difference between ``useless'' and ``unused''
ec3bc396
AD
7214below):
7215
7216@example
d80fb37a 7217Nonterminals useless in grammar:
ec3bc396
AD
7218 useless
7219
d80fb37a 7220Terminals unused in grammar:
ec3bc396
AD
7221 STR
7222
cff03fb2 7223Rules useless in grammar:
ec3bc396
AD
7224#6 useless: STR;
7225@end example
7226
7227@noindent
7228The next section reproduces the exact grammar that Bison used:
7229
7230@example
7231Grammar
7232
7233 Number, Line, Rule
88bce5a2 7234 0 5 $accept -> exp $end
ec3bc396
AD
7235 1 5 exp -> exp '+' exp
7236 2 6 exp -> exp '-' exp
7237 3 7 exp -> exp '*' exp
7238 4 8 exp -> exp '/' exp
7239 5 9 exp -> NUM
7240@end example
7241
7242@noindent
7243and reports the uses of the symbols:
7244
7245@example
7246Terminals, with rules where they appear
7247
88bce5a2 7248$end (0) 0
ec3bc396
AD
7249'*' (42) 3
7250'+' (43) 1
7251'-' (45) 2
7252'/' (47) 4
7253error (256)
7254NUM (258) 5
7255
7256Nonterminals, with rules where they appear
7257
88bce5a2 7258$accept (8)
ec3bc396
AD
7259 on left: 0
7260exp (9)
7261 on left: 1 2 3 4 5, on right: 0 1 2 3 4
7262@end example
7263
7264@noindent
7265@cindex item
7266@cindex pointed rule
7267@cindex rule, pointed
7268Bison then proceeds onto the automaton itself, describing each state
7269with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
7270item is a production rule together with a point (marked by @samp{.})
7271that the input cursor.
7272
7273@example
7274state 0
7275
88bce5a2 7276 $accept -> . exp $ (rule 0)
ec3bc396 7277
2a8d363a 7278 NUM shift, and go to state 1
ec3bc396 7279
2a8d363a 7280 exp go to state 2
ec3bc396
AD
7281@end example
7282
7283This reads as follows: ``state 0 corresponds to being at the very
7284beginning of the parsing, in the initial rule, right before the start
7285symbol (here, @code{exp}). When the parser returns to this state right
7286after having reduced a rule that produced an @code{exp}, the control
7287flow jumps to state 2. If there is no such transition on a nonterminal
742e4900 7288symbol, and the lookahead is a @code{NUM}, then this token is shifted on
ec3bc396 7289the parse stack, and the control flow jumps to state 1. Any other
742e4900 7290lookahead triggers a syntax error.''
ec3bc396
AD
7291
7292@cindex core, item set
7293@cindex item set core
7294@cindex kernel, item set
7295@cindex item set core
7296Even though the only active rule in state 0 seems to be rule 0, the
742e4900 7297report lists @code{NUM} as a lookahead token because @code{NUM} can be
ec3bc396
AD
7298at the beginning of any rule deriving an @code{exp}. By default Bison
7299reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
7300you want to see more detail you can invoke @command{bison} with
7301@option{--report=itemset} to list all the items, include those that can
7302be derived:
7303
7304@example
7305state 0
7306
88bce5a2 7307 $accept -> . exp $ (rule 0)
ec3bc396
AD
7308 exp -> . exp '+' exp (rule 1)
7309 exp -> . exp '-' exp (rule 2)
7310 exp -> . exp '*' exp (rule 3)
7311 exp -> . exp '/' exp (rule 4)
7312 exp -> . NUM (rule 5)
7313
7314 NUM shift, and go to state 1
7315
7316 exp go to state 2
7317@end example
7318
7319@noindent
7320In the state 1...
7321
7322@example
7323state 1
7324
7325 exp -> NUM . (rule 5)
7326
2a8d363a 7327 $default reduce using rule 5 (exp)
ec3bc396
AD
7328@end example
7329
7330@noindent
742e4900 7331the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
ec3bc396
AD
7332(@samp{$default}), the parser will reduce it. If it was coming from
7333state 0, then, after this reduction it will return to state 0, and will
7334jump to state 2 (@samp{exp: go to state 2}).
7335
7336@example
7337state 2
7338
88bce5a2 7339 $accept -> exp . $ (rule 0)
ec3bc396
AD
7340 exp -> exp . '+' exp (rule 1)
7341 exp -> exp . '-' exp (rule 2)
7342 exp -> exp . '*' exp (rule 3)
7343 exp -> exp . '/' exp (rule 4)
7344
2a8d363a
AD
7345 $ shift, and go to state 3
7346 '+' shift, and go to state 4
7347 '-' shift, and go to state 5
7348 '*' shift, and go to state 6
7349 '/' shift, and go to state 7
ec3bc396
AD
7350@end example
7351
7352@noindent
7353In state 2, the automaton can only shift a symbol. For instance,
742e4900 7354because of the item @samp{exp -> exp . '+' exp}, if the lookahead if
ec3bc396
AD
7355@samp{+}, it will be shifted on the parse stack, and the automaton
7356control will jump to state 4, corresponding to the item @samp{exp -> exp
7357'+' . exp}. Since there is no default action, any other token than
6e649e65 7358those listed above will trigger a syntax error.
ec3bc396
AD
7359
7360The state 3 is named the @dfn{final state}, or the @dfn{accepting
7361state}:
7362
7363@example
7364state 3
7365
88bce5a2 7366 $accept -> exp $ . (rule 0)
ec3bc396 7367
2a8d363a 7368 $default accept
ec3bc396
AD
7369@end example
7370
7371@noindent
7372the initial rule is completed (the start symbol and the end
7373of input were read), the parsing exits successfully.
7374
7375The interpretation of states 4 to 7 is straightforward, and is left to
7376the reader.
7377
7378@example
7379state 4
7380
7381 exp -> exp '+' . exp (rule 1)
7382
2a8d363a 7383 NUM shift, and go to state 1
ec3bc396 7384
2a8d363a 7385 exp go to state 8
ec3bc396
AD
7386
7387state 5
7388
7389 exp -> exp '-' . exp (rule 2)
7390
2a8d363a 7391 NUM shift, and go to state 1
ec3bc396 7392
2a8d363a 7393 exp go to state 9
ec3bc396
AD
7394
7395state 6
7396
7397 exp -> exp '*' . exp (rule 3)
7398
2a8d363a 7399 NUM shift, and go to state 1
ec3bc396 7400
2a8d363a 7401 exp go to state 10
ec3bc396
AD
7402
7403state 7
7404
7405 exp -> exp '/' . exp (rule 4)
7406
2a8d363a 7407 NUM shift, and go to state 1
ec3bc396 7408
2a8d363a 7409 exp go to state 11
ec3bc396
AD
7410@end example
7411
5a99098d
PE
7412As was announced in beginning of the report, @samp{State 8 conflicts:
74131 shift/reduce}:
ec3bc396
AD
7414
7415@example
7416state 8
7417
7418 exp -> exp . '+' exp (rule 1)
7419 exp -> exp '+' exp . (rule 1)
7420 exp -> exp . '-' exp (rule 2)
7421 exp -> exp . '*' exp (rule 3)
7422 exp -> exp . '/' exp (rule 4)
7423
2a8d363a
AD
7424 '*' shift, and go to state 6
7425 '/' shift, and go to state 7
ec3bc396 7426
2a8d363a
AD
7427 '/' [reduce using rule 1 (exp)]
7428 $default reduce using rule 1 (exp)
ec3bc396
AD
7429@end example
7430
742e4900 7431Indeed, there are two actions associated to the lookahead @samp{/}:
ec3bc396
AD
7432either shifting (and going to state 7), or reducing rule 1. The
7433conflict means that either the grammar is ambiguous, or the parser lacks
7434information to make the right decision. Indeed the grammar is
7435ambiguous, as, since we did not specify the precedence of @samp{/}, the
7436sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
7437NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
7438NUM}, which corresponds to reducing rule 1.
7439
c827f760 7440Because in @acronym{LALR}(1) parsing a single decision can be made, Bison
ec3bc396
AD
7441arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
7442Shift/Reduce Conflicts}. Discarded actions are reported in between
7443square brackets.
7444
7445Note that all the previous states had a single possible action: either
7446shifting the next token and going to the corresponding state, or
7447reducing a single rule. In the other cases, i.e., when shifting
7448@emph{and} reducing is possible or when @emph{several} reductions are
742e4900
JD
7449possible, the lookahead is required to select the action. State 8 is
7450one such state: if the lookahead is @samp{*} or @samp{/} then the action
ec3bc396
AD
7451is shifting, otherwise the action is reducing rule 1. In other words,
7452the first two items, corresponding to rule 1, are not eligible when the
742e4900 7453lookahead token is @samp{*}, since we specified that @samp{*} has higher
8dd162d3 7454precedence than @samp{+}. More generally, some items are eligible only
742e4900
JD
7455with some set of possible lookahead tokens. When run with
7456@option{--report=lookahead}, Bison specifies these lookahead tokens:
ec3bc396
AD
7457
7458@example
7459state 8
7460
88c78747 7461 exp -> exp . '+' exp (rule 1)
ec3bc396
AD
7462 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
7463 exp -> exp . '-' exp (rule 2)
7464 exp -> exp . '*' exp (rule 3)
7465 exp -> exp . '/' exp (rule 4)
7466
7467 '*' shift, and go to state 6
7468 '/' shift, and go to state 7
7469
7470 '/' [reduce using rule 1 (exp)]
7471 $default reduce using rule 1 (exp)
7472@end example
7473
7474The remaining states are similar:
7475
7476@example
7477state 9
7478
7479 exp -> exp . '+' exp (rule 1)
7480 exp -> exp . '-' exp (rule 2)
7481 exp -> exp '-' exp . (rule 2)
7482 exp -> exp . '*' exp (rule 3)
7483 exp -> exp . '/' exp (rule 4)
7484
2a8d363a
AD
7485 '*' shift, and go to state 6
7486 '/' shift, and go to state 7
ec3bc396 7487
2a8d363a
AD
7488 '/' [reduce using rule 2 (exp)]
7489 $default reduce using rule 2 (exp)
ec3bc396
AD
7490
7491state 10
7492
7493 exp -> exp . '+' exp (rule 1)
7494 exp -> exp . '-' exp (rule 2)
7495 exp -> exp . '*' exp (rule 3)
7496 exp -> exp '*' exp . (rule 3)
7497 exp -> exp . '/' exp (rule 4)
7498
2a8d363a 7499 '/' shift, and go to state 7
ec3bc396 7500
2a8d363a
AD
7501 '/' [reduce using rule 3 (exp)]
7502 $default reduce using rule 3 (exp)
ec3bc396
AD
7503
7504state 11
7505
7506 exp -> exp . '+' exp (rule 1)
7507 exp -> exp . '-' exp (rule 2)
7508 exp -> exp . '*' exp (rule 3)
7509 exp -> exp . '/' exp (rule 4)
7510 exp -> exp '/' exp . (rule 4)
7511
2a8d363a
AD
7512 '+' shift, and go to state 4
7513 '-' shift, and go to state 5
7514 '*' shift, and go to state 6
7515 '/' shift, and go to state 7
ec3bc396 7516
2a8d363a
AD
7517 '+' [reduce using rule 4 (exp)]
7518 '-' [reduce using rule 4 (exp)]
7519 '*' [reduce using rule 4 (exp)]
7520 '/' [reduce using rule 4 (exp)]
7521 $default reduce using rule 4 (exp)
ec3bc396
AD
7522@end example
7523
7524@noindent
fa7e68c3
PE
7525Observe that state 11 contains conflicts not only due to the lack of
7526precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
7527@samp{*}, but also because the
ec3bc396
AD
7528associativity of @samp{/} is not specified.
7529
7530
7531@node Tracing
7532@section Tracing Your Parser
bfa74976
RS
7533@findex yydebug
7534@cindex debugging
7535@cindex tracing the parser
7536
7537If a Bison grammar compiles properly but doesn't do what you want when it
7538runs, the @code{yydebug} parser-trace feature can help you figure out why.
7539
3ded9a63
AD
7540There are several means to enable compilation of trace facilities:
7541
7542@table @asis
7543@item the macro @code{YYDEBUG}
7544@findex YYDEBUG
7545Define the macro @code{YYDEBUG} to a nonzero value when you compile the
c827f760 7546parser. This is compliant with @acronym{POSIX} Yacc. You could use
3ded9a63
AD
7547@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
7548YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
7549Prologue}).
7550
7551@item the option @option{-t}, @option{--debug}
7552Use the @samp{-t} option when you run Bison (@pxref{Invocation,
c827f760 7553,Invoking Bison}). This is @acronym{POSIX} compliant too.
3ded9a63
AD
7554
7555@item the directive @samp{%debug}
7556@findex %debug
7557Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison
7558Declaration Summary}). This is a Bison extension, which will prove
7559useful when Bison will output parsers for languages that don't use a
c827f760
PE
7560preprocessor. Unless @acronym{POSIX} and Yacc portability matter to
7561you, this is
3ded9a63
AD
7562the preferred solution.
7563@end table
7564
7565We suggest that you always enable the debug option so that debugging is
7566always possible.
bfa74976 7567
02a81e05 7568The trace facility outputs messages with macro calls of the form
e2742e46 7569@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
f57a7536 7570@var{format} and @var{args} are the usual @code{printf} format and variadic
4947ebdb
PE
7571arguments. If you define @code{YYDEBUG} to a nonzero value but do not
7572define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
9c437126 7573and @code{YYFPRINTF} is defined to @code{fprintf}.
bfa74976
RS
7574
7575Once you have compiled the program with trace facilities, the way to
7576request a trace is to store a nonzero value in the variable @code{yydebug}.
7577You can do this by making the C code do it (in @code{main}, perhaps), or
7578you can alter the value with a C debugger.
7579
7580Each step taken by the parser when @code{yydebug} is nonzero produces a
7581line or two of trace information, written on @code{stderr}. The trace
7582messages tell you these things:
7583
7584@itemize @bullet
7585@item
7586Each time the parser calls @code{yylex}, what kind of token was read.
7587
7588@item
7589Each time a token is shifted, the depth and complete contents of the
7590state stack (@pxref{Parser States}).
7591
7592@item
7593Each time a rule is reduced, which rule it is, and the complete contents
7594of the state stack afterward.
7595@end itemize
7596
7597To make sense of this information, it helps to refer to the listing file
704a47c4
AD
7598produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
7599Bison}). This file shows the meaning of each state in terms of
7600positions in various rules, and also what each state will do with each
7601possible input token. As you read the successive trace messages, you
7602can see that the parser is functioning according to its specification in
7603the listing file. Eventually you will arrive at the place where
7604something undesirable happens, and you will see which parts of the
7605grammar are to blame.
bfa74976
RS
7606
7607The parser file is a C program and you can use C debuggers on it, but it's
7608not easy to interpret what it is doing. The parser function is a
7609finite-state machine interpreter, and aside from the actions it executes
7610the same code over and over. Only the values of variables show where in
7611the grammar it is working.
7612
7613@findex YYPRINT
7614The debugging information normally gives the token type of each token
7615read, but not its semantic value. You can optionally define a macro
7616named @code{YYPRINT} to provide a way to print the value. If you define
7617@code{YYPRINT}, it should take three arguments. The parser will pass a
7618standard I/O stream, the numeric code for the token type, and the token
7619value (from @code{yylval}).
7620
7621Here is an example of @code{YYPRINT} suitable for the multi-function
7622calculator (@pxref{Mfcalc Decl, ,Declarations for @code{mfcalc}}):
7623
7624@smallexample
38a92d50
PE
7625%@{
7626 static void print_token_value (FILE *, int, YYSTYPE);
7627 #define YYPRINT(file, type, value) print_token_value (file, type, value)
7628%@}
7629
7630@dots{} %% @dots{} %% @dots{}
bfa74976
RS
7631
7632static void
831d3c99 7633print_token_value (FILE *file, int type, YYSTYPE value)
bfa74976
RS
7634@{
7635 if (type == VAR)
d3c4e709 7636 fprintf (file, "%s", value.tptr->name);
bfa74976 7637 else if (type == NUM)
d3c4e709 7638 fprintf (file, "%d", value.val);
bfa74976
RS
7639@}
7640@end smallexample
7641
ec3bc396
AD
7642@c ================================================= Invoking Bison
7643
342b8b6e 7644@node Invocation
bfa74976
RS
7645@chapter Invoking Bison
7646@cindex invoking Bison
7647@cindex Bison invocation
7648@cindex options for invoking Bison
7649
7650The usual way to invoke Bison is as follows:
7651
7652@example
7653bison @var{infile}
7654@end example
7655
7656Here @var{infile} is the grammar file name, which usually ends in
7657@samp{.y}. The parser file's name is made by replacing the @samp{.y}
fa4d969f
PE
7658with @samp{.tab.c} and removing any leading directory. Thus, the
7659@samp{bison foo.y} file name yields
7660@file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields
7661@file{foo.tab.c}. It's also possible, in case you are writing
79282c6c 7662C++ code instead of C in your grammar file, to name it @file{foo.ypp}
72d2299c
PE
7663or @file{foo.y++}. Then, the output files will take an extension like
7664the given one as input (respectively @file{foo.tab.cpp} and
7665@file{foo.tab.c++}).
fa4d969f 7666This feature takes effect with all options that manipulate file names like
234a3be3
AD
7667@samp{-o} or @samp{-d}.
7668
7669For example :
7670
7671@example
7672bison -d @var{infile.yxx}
7673@end example
84163231 7674@noindent
72d2299c 7675will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
234a3be3
AD
7676
7677@example
b56471a6 7678bison -d -o @var{output.c++} @var{infile.y}
234a3be3 7679@end example
84163231 7680@noindent
234a3be3
AD
7681will produce @file{output.c++} and @file{outfile.h++}.
7682
397ec073
PE
7683For compatibility with @acronym{POSIX}, the standard Bison
7684distribution also contains a shell script called @command{yacc} that
7685invokes Bison with the @option{-y} option.
7686
bfa74976 7687@menu
13863333 7688* Bison Options:: All the options described in detail,
c827f760 7689 in alphabetical order by short options.
bfa74976 7690* Option Cross Key:: Alphabetical list of long options.
93dd49ab 7691* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
bfa74976
RS
7692@end menu
7693
342b8b6e 7694@node Bison Options
bfa74976
RS
7695@section Bison Options
7696
7697Bison supports both traditional single-letter options and mnemonic long
7698option names. Long option names are indicated with @samp{--} instead of
7699@samp{-}. Abbreviations for option names are allowed as long as they
7700are unique. When a long option takes an argument, like
7701@samp{--file-prefix}, connect the option name and the argument with
7702@samp{=}.
7703
7704Here is a list of options that can be used with Bison, alphabetized by
7705short option. It is followed by a cross key alphabetized by long
7706option.
7707
89cab50d
AD
7708@c Please, keep this ordered as in `bison --help'.
7709@noindent
7710Operations modes:
7711@table @option
7712@item -h
7713@itemx --help
7714Print a summary of the command-line options to Bison and exit.
bfa74976 7715
89cab50d
AD
7716@item -V
7717@itemx --version
7718Print the version number of Bison and exit.
bfa74976 7719
f7ab6a50
PE
7720@item --print-localedir
7721Print the name of the directory containing locale-dependent data.
7722
a0de5091
JD
7723@item --print-datadir
7724Print the name of the directory containing skeletons and XSLT.
7725
89cab50d
AD
7726@item -y
7727@itemx --yacc
54662697
PE
7728Act more like the traditional Yacc command. This can cause
7729different diagnostics to be generated, and may change behavior in
7730other minor ways. Most importantly, imitate Yacc's output
7731file name conventions, so that the parser output file is called
89cab50d 7732@file{y.tab.c}, and the other outputs are called @file{y.output} and
b931235e
JD
7733@file{y.tab.h}.
7734Also, if generating an @acronym{LALR}(1) parser in C, generate @code{#define}
7735statements in addition to an @code{enum} to associate token numbers with token
7736names.
7737Thus, the following shell script can substitute for Yacc, and the Bison
7738distribution contains such a script for compatibility with @acronym{POSIX}:
bfa74976 7739
89cab50d 7740@example
397ec073 7741#! /bin/sh
26e06a21 7742bison -y "$@@"
89cab50d 7743@end example
54662697
PE
7744
7745The @option{-y}/@option{--yacc} option is intended for use with
7746traditional Yacc grammars. If your grammar uses a Bison extension
7747like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
7748this option is specified.
7749
89cab50d
AD
7750@end table
7751
7752@noindent
7753Tuning the parser:
7754
7755@table @option
7756@item -t
7757@itemx --debug
4947ebdb
PE
7758In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
7759already defined, so that the debugging facilities are compiled.
ec3bc396 7760@xref{Tracing, ,Tracing Your Parser}.
89cab50d 7761
0e021770
PE
7762@item -L @var{language}
7763@itemx --language=@var{language}
7764Specify the programming language for the generated parser, as if
7765@code{%language} was specified (@pxref{Decl Summary, , Bison Declaration
7766Summary}). Currently supported languages include C and C++.
e6e704dc 7767@var{language} is case-insensitive.
0e021770 7768
89cab50d 7769@item --locations
d8988b2f 7770Pretend that @code{%locations} was specified. @xref{Decl Summary}.
89cab50d
AD
7771
7772@item -p @var{prefix}
7773@itemx --name-prefix=@var{prefix}
02975b9a 7774Pretend that @code{%name-prefix "@var{prefix}"} was specified.
d8988b2f 7775@xref{Decl Summary}.
bfa74976
RS
7776
7777@item -l
7778@itemx --no-lines
7779Don't put any @code{#line} preprocessor commands in the parser file.
7780Ordinarily Bison puts them in the parser file so that the C compiler
7781and debuggers will associate errors with your source file, the
7782grammar file. This option causes them to associate errors with the
95e742f7 7783parser file, treating it as an independent source file in its own right.
bfa74976 7784
e6e704dc
JD
7785@item -S @var{file}
7786@itemx --skeleton=@var{file}
a7867f53 7787Specify the skeleton to use, similar to @code{%skeleton}
e6e704dc
JD
7788(@pxref{Decl Summary, , Bison Declaration Summary}).
7789
a7867f53
JD
7790You probably don't need this option unless you are developing Bison.
7791You should use @option{--language} if you want to specify the skeleton for a
e6e704dc
JD
7792different language, because it is clearer and because it will always
7793choose the correct skeleton for non-deterministic or push parsers.
7794
a7867f53
JD
7795If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
7796file in the Bison installation directory.
7797If it does, @var{file} is an absolute file name or a file name relative to the
7798current working directory.
7799This is similar to how most shells resolve commands.
7800
89cab50d
AD
7801@item -k
7802@itemx --token-table
d8988b2f 7803Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
89cab50d 7804@end table
bfa74976 7805
89cab50d
AD
7806@noindent
7807Adjust the output:
bfa74976 7808
89cab50d
AD
7809@table @option
7810@item -d
d8988b2f
AD
7811@itemx --defines
7812Pretend that @code{%defines} was specified, i.e., write an extra output
6deb4447 7813file containing macro definitions for the token type names defined in
4bfd5e4e 7814the grammar, as well as a few other declarations. @xref{Decl Summary}.
931c7513 7815
342b8b6e 7816@item --defines=@var{defines-file}
d8988b2f 7817Same as above, but save in the file @var{defines-file}.
342b8b6e 7818
89cab50d
AD
7819@item -b @var{file-prefix}
7820@itemx --file-prefix=@var{prefix}
9c437126 7821Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
72d2299c 7822for all Bison output file names. @xref{Decl Summary}.
bfa74976 7823
ec3bc396
AD
7824@item -r @var{things}
7825@itemx --report=@var{things}
7826Write an extra output file containing verbose description of the comma
7827separated list of @var{things} among:
7828
7829@table @code
7830@item state
7831Description of the grammar, conflicts (resolved and unresolved), and
c827f760 7832@acronym{LALR} automaton.
ec3bc396 7833
742e4900 7834@item lookahead
ec3bc396 7835Implies @code{state} and augments the description of the automaton with
742e4900 7836each rule's lookahead set.
ec3bc396
AD
7837
7838@item itemset
7839Implies @code{state} and augments the description of the automaton with
7840the full set of items for each state, instead of its core only.
7841@end table
7842
1bb2bd75
JD
7843@item --report-file=@var{file}
7844Specify the @var{file} for the verbose description.
7845
bfa74976
RS
7846@item -v
7847@itemx --verbose
9c437126 7848Pretend that @code{%verbose} was specified, i.e., write an extra output
6deb4447 7849file containing verbose descriptions of the grammar and
72d2299c 7850parser. @xref{Decl Summary}.
bfa74976 7851
fa4d969f
PE
7852@item -o @var{file}
7853@itemx --output=@var{file}
7854Specify the @var{file} for the parser file.
bfa74976 7855
fa4d969f 7856The other output files' names are constructed from @var{file} as
d8988b2f 7857described under the @samp{-v} and @samp{-d} options.
342b8b6e
AD
7858
7859@item -g
35fe0834
PE
7860Output a graphical representation of the @acronym{LALR}(1) grammar
7861automaton computed by Bison, in @uref{http://www.graphviz.org/, Graphviz}
7862@uref{http://www.graphviz.org/doc/info/lang.html, @acronym{DOT}} format.
7863If the grammar file is @file{foo.y}, the output file will
7864be @file{foo.dot}.
342b8b6e
AD
7865
7866@item --graph=@var{graph-file}
72d2299c
PE
7867The behavior of @var{--graph} is the same than @samp{-g}. The only
7868difference is that it has an optional argument which is the name of
fa4d969f 7869the output graph file.
bfa74976
RS
7870@end table
7871
342b8b6e 7872@node Option Cross Key
bfa74976
RS
7873@section Option Cross Key
7874
aa08666d 7875@c FIXME: How about putting the directives too?
bfa74976
RS
7876Here is a list of options, alphabetized by long option, to help you find
7877the corresponding short option.
7878
aa08666d
AD
7879@multitable {@option{--defines=@var{defines-file}}} {@option{-b @var{file-prefix}XXX}}
7880@headitem Long Option @tab Short Option
f4101aa6 7881@include cross-options.texi
aa08666d 7882@end multitable
bfa74976 7883
93dd49ab
PE
7884@node Yacc Library
7885@section Yacc Library
7886
7887The Yacc library contains default implementations of the
7888@code{yyerror} and @code{main} functions. These default
7889implementations are normally not useful, but @acronym{POSIX} requires
7890them. To use the Yacc library, link your program with the
7891@option{-ly} option. Note that Bison's implementation of the Yacc
7892library is distributed under the terms of the @acronym{GNU} General
7893Public License (@pxref{Copying}).
7894
7895If you use the Yacc library's @code{yyerror} function, you should
7896declare @code{yyerror} as follows:
7897
7898@example
7899int yyerror (char const *);
7900@end example
7901
7902Bison ignores the @code{int} value returned by this @code{yyerror}.
7903If you use the Yacc library's @code{main} function, your
7904@code{yyparse} function should have the following type signature:
7905
7906@example
7907int yyparse (void);
7908@end example
7909
12545799
AD
7910@c ================================================= C++ Bison
7911
8405b70c
PB
7912@node Other Languages
7913@chapter Parsers Written In Other Languages
12545799
AD
7914
7915@menu
7916* C++ Parsers:: The interface to generate C++ parser classes
8405b70c 7917* Java Parsers:: The interface to generate Java parser classes
12545799
AD
7918@end menu
7919
7920@node C++ Parsers
7921@section C++ Parsers
7922
7923@menu
7924* C++ Bison Interface:: Asking for C++ parser generation
7925* C++ Semantic Values:: %union vs. C++
7926* C++ Location Values:: The position and location classes
7927* C++ Parser Interface:: Instantiating and running the parser
7928* C++ Scanner Interface:: Exchanges between yylex and parse
8405b70c 7929* A Complete C++ Example:: Demonstrating their use
12545799
AD
7930@end menu
7931
7932@node C++ Bison Interface
7933@subsection C++ Bison Interface
0e021770 7934@c - %language "C++"
12545799
AD
7935@c - Always pure
7936@c - initial action
7937
e6e704dc
JD
7938The C++ @acronym{LALR}(1) parser is selected using the language directive,
7939@samp{%language "C++"}, or the synonymous command-line option
7940@option{--language=c++}.
7941@xref{Decl Summary}.
0e021770 7942
793fbca5
JD
7943When run, @command{bison} will create several entities in the @samp{yy}
7944namespace.
7945@findex %define namespace
7946Use the @samp{%define namespace} directive to change the namespace name, see
7947@ref{Decl Summary}.
7948The various classes are generated in the following files:
aa08666d 7949
12545799
AD
7950@table @file
7951@item position.hh
7952@itemx location.hh
7953The definition of the classes @code{position} and @code{location},
7954used for location tracking. @xref{C++ Location Values}.
7955
7956@item stack.hh
7957An auxiliary class @code{stack} used by the parser.
7958
fa4d969f
PE
7959@item @var{file}.hh
7960@itemx @var{file}.cc
cd8b5791
AD
7961(Assuming the extension of the input file was @samp{.yy}.) The
7962declaration and implementation of the C++ parser class. The basename
7963and extension of these two files follow the same rules as with regular C
7964parsers (@pxref{Invocation}).
12545799 7965
cd8b5791
AD
7966The header is @emph{mandatory}; you must either pass
7967@option{-d}/@option{--defines} to @command{bison}, or use the
12545799
AD
7968@samp{%defines} directive.
7969@end table
7970
7971All these files are documented using Doxygen; run @command{doxygen}
7972for a complete and accurate documentation.
7973
7974@node C++ Semantic Values
7975@subsection C++ Semantic Values
7976@c - No objects in unions
178e123e 7977@c - YYSTYPE
12545799
AD
7978@c - Printer and destructor
7979
7980The @code{%union} directive works as for C, see @ref{Union Decl, ,The
7981Collection of Value Types}. In particular it produces a genuine
7982@code{union}@footnote{In the future techniques to allow complex types
fb9712a9
AD
7983within pseudo-unions (similar to Boost variants) might be implemented to
7984alleviate these issues.}, which have a few specific features in C++.
12545799
AD
7985@itemize @minus
7986@item
fb9712a9
AD
7987The type @code{YYSTYPE} is defined but its use is discouraged: rather
7988you should refer to the parser's encapsulated type
7989@code{yy::parser::semantic_type}.
12545799
AD
7990@item
7991Non POD (Plain Old Data) types cannot be used. C++ forbids any
7992instance of classes with constructors in unions: only @emph{pointers}
7993to such objects are allowed.
7994@end itemize
7995
7996Because objects have to be stored via pointers, memory is not
7997reclaimed automatically: using the @code{%destructor} directive is the
7998only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
7999Symbols}.
8000
8001
8002@node C++ Location Values
8003@subsection C++ Location Values
8004@c - %locations
8005@c - class Position
8006@c - class Location
16dc6a9e 8007@c - %define filename_type "const symbol::Symbol"
12545799
AD
8008
8009When the directive @code{%locations} is used, the C++ parser supports
8010location tracking, see @ref{Locations, , Locations Overview}. Two
8011auxiliary classes define a @code{position}, a single point in a file,
8012and a @code{location}, a range composed of a pair of
8013@code{position}s (possibly spanning several files).
8014
fa4d969f 8015@deftypemethod {position} {std::string*} file
12545799
AD
8016The name of the file. It will always be handled as a pointer, the
8017parser will never duplicate nor deallocate it. As an experimental
8018feature you may change it to @samp{@var{type}*} using @samp{%define
16dc6a9e 8019filename_type "@var{type}"}.
12545799
AD
8020@end deftypemethod
8021
8022@deftypemethod {position} {unsigned int} line
8023The line, starting at 1.
8024@end deftypemethod
8025
8026@deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
8027Advance by @var{height} lines, resetting the column number.
8028@end deftypemethod
8029
8030@deftypemethod {position} {unsigned int} column
8031The column, starting at 0.
8032@end deftypemethod
8033
8034@deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
8035Advance by @var{width} columns, without changing the line number.
8036@end deftypemethod
8037
8038@deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
8039@deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
8040@deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
8041@deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
8042Various forms of syntactic sugar for @code{columns}.
8043@end deftypemethod
8044
8045@deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
8046Report @var{p} on @var{o} like this:
fa4d969f
PE
8047@samp{@var{file}:@var{line}.@var{column}}, or
8048@samp{@var{line}.@var{column}} if @var{file} is null.
12545799
AD
8049@end deftypemethod
8050
8051@deftypemethod {location} {position} begin
8052@deftypemethodx {location} {position} end
8053The first, inclusive, position of the range, and the first beyond.
8054@end deftypemethod
8055
8056@deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
8057@deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
8058Advance the @code{end} position.
8059@end deftypemethod
8060
8061@deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
8062@deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
8063@deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
8064Various forms of syntactic sugar.
8065@end deftypemethod
8066
8067@deftypemethod {location} {void} step ()
8068Move @code{begin} onto @code{end}.
8069@end deftypemethod
8070
8071
8072@node C++ Parser Interface
8073@subsection C++ Parser Interface
8074@c - define parser_class_name
8075@c - Ctor
8076@c - parse, error, set_debug_level, debug_level, set_debug_stream,
8077@c debug_stream.
8078@c - Reporting errors
8079
8080The output files @file{@var{output}.hh} and @file{@var{output}.cc}
8081declare and define the parser class in the namespace @code{yy}. The
8082class name defaults to @code{parser}, but may be changed using
16dc6a9e 8083@samp{%define parser_class_name "@var{name}"}. The interface of
9d9b8b70 8084this class is detailed below. It can be extended using the
12545799
AD
8085@code{%parse-param} feature: its semantics is slightly changed since
8086it describes an additional member of the parser class, and an
8087additional argument for its constructor.
8088
8a0adb01
AD
8089@defcv {Type} {parser} {semantic_value_type}
8090@defcvx {Type} {parser} {location_value_type}
12545799 8091The types for semantics value and locations.
8a0adb01 8092@end defcv
12545799
AD
8093
8094@deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
8095Build a new parser object. There are no arguments by default, unless
8096@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
8097@end deftypemethod
8098
8099@deftypemethod {parser} {int} parse ()
8100Run the syntactic analysis, and return 0 on success, 1 otherwise.
8101@end deftypemethod
8102
8103@deftypemethod {parser} {std::ostream&} debug_stream ()
8104@deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
8105Get or set the stream used for tracing the parsing. It defaults to
8106@code{std::cerr}.
8107@end deftypemethod
8108
8109@deftypemethod {parser} {debug_level_type} debug_level ()
8110@deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
8111Get or set the tracing level. Currently its value is either 0, no trace,
9d9b8b70 8112or nonzero, full tracing.
12545799
AD
8113@end deftypemethod
8114
8115@deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
8116The definition for this member function must be supplied by the user:
8117the parser uses it to report a parser error occurring at @var{l},
8118described by @var{m}.
8119@end deftypemethod
8120
8121
8122@node C++ Scanner Interface
8123@subsection C++ Scanner Interface
8124@c - prefix for yylex.
8125@c - Pure interface to yylex
8126@c - %lex-param
8127
8128The parser invokes the scanner by calling @code{yylex}. Contrary to C
8129parsers, C++ parsers are always pure: there is no point in using the
d9df47b6 8130@code{%define api.pure} directive. Therefore the interface is as follows.
12545799
AD
8131
8132@deftypemethod {parser} {int} yylex (semantic_value_type& @var{yylval}, location_type& @var{yylloc}, @var{type1} @var{arg1}, ...)
8133Return the next token. Its type is the return value, its semantic
8134value and location being @var{yylval} and @var{yylloc}. Invocations of
8135@samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
8136@end deftypemethod
8137
8138
8139@node A Complete C++ Example
8405b70c 8140@subsection A Complete C++ Example
12545799
AD
8141
8142This section demonstrates the use of a C++ parser with a simple but
8143complete example. This example should be available on your system,
8144ready to compile, in the directory @dfn{../bison/examples/calc++}. It
8145focuses on the use of Bison, therefore the design of the various C++
8146classes is very naive: no accessors, no encapsulation of members etc.
8147We will use a Lex scanner, and more precisely, a Flex scanner, to
8148demonstrate the various interaction. A hand written scanner is
8149actually easier to interface with.
8150
8151@menu
8152* Calc++ --- C++ Calculator:: The specifications
8153* Calc++ Parsing Driver:: An active parsing context
8154* Calc++ Parser:: A parser class
8155* Calc++ Scanner:: A pure C++ Flex scanner
8156* Calc++ Top Level:: Conducting the band
8157@end menu
8158
8159@node Calc++ --- C++ Calculator
8405b70c 8160@subsubsection Calc++ --- C++ Calculator
12545799
AD
8161
8162Of course the grammar is dedicated to arithmetics, a single
9d9b8b70 8163expression, possibly preceded by variable assignments. An
12545799
AD
8164environment containing possibly predefined variables such as
8165@code{one} and @code{two}, is exchanged with the parser. An example
8166of valid input follows.
8167
8168@example
8169three := 3
8170seven := one + two * three
8171seven * seven
8172@end example
8173
8174@node Calc++ Parsing Driver
8405b70c 8175@subsubsection Calc++ Parsing Driver
12545799
AD
8176@c - An env
8177@c - A place to store error messages
8178@c - A place for the result
8179
8180To support a pure interface with the parser (and the scanner) the
8181technique of the ``parsing context'' is convenient: a structure
8182containing all the data to exchange. Since, in addition to simply
8183launch the parsing, there are several auxiliary tasks to execute (open
8184the file for parsing, instantiate the parser etc.), we recommend
8185transforming the simple parsing context structure into a fully blown
8186@dfn{parsing driver} class.
8187
8188The declaration of this driver class, @file{calc++-driver.hh}, is as
8189follows. The first part includes the CPP guard and imports the
fb9712a9
AD
8190required standard library components, and the declaration of the parser
8191class.
12545799 8192
1c59e0a1 8193@comment file: calc++-driver.hh
12545799
AD
8194@example
8195#ifndef CALCXX_DRIVER_HH
8196# define CALCXX_DRIVER_HH
8197# include <string>
8198# include <map>
fb9712a9 8199# include "calc++-parser.hh"
12545799
AD
8200@end example
8201
12545799
AD
8202
8203@noindent
8204Then comes the declaration of the scanning function. Flex expects
8205the signature of @code{yylex} to be defined in the macro
8206@code{YY_DECL}, and the C++ parser expects it to be declared. We can
8207factor both as follows.
1c59e0a1
AD
8208
8209@comment file: calc++-driver.hh
12545799 8210@example
3dc5e96b
PE
8211// Tell Flex the lexer's prototype ...
8212# define YY_DECL \
c095d689
AD
8213 yy::calcxx_parser::token_type \
8214 yylex (yy::calcxx_parser::semantic_type* yylval, \
8215 yy::calcxx_parser::location_type* yylloc, \
8216 calcxx_driver& driver)
12545799
AD
8217// ... and declare it for the parser's sake.
8218YY_DECL;
8219@end example
8220
8221@noindent
8222The @code{calcxx_driver} class is then declared with its most obvious
8223members.
8224
1c59e0a1 8225@comment file: calc++-driver.hh
12545799
AD
8226@example
8227// Conducting the whole scanning and parsing of Calc++.
8228class calcxx_driver
8229@{
8230public:
8231 calcxx_driver ();
8232 virtual ~calcxx_driver ();
8233
8234 std::map<std::string, int> variables;
8235
8236 int result;
8237@end example
8238
8239@noindent
8240To encapsulate the coordination with the Flex scanner, it is useful to
8241have two members function to open and close the scanning phase.
12545799 8242
1c59e0a1 8243@comment file: calc++-driver.hh
12545799
AD
8244@example
8245 // Handling the scanner.
8246 void scan_begin ();
8247 void scan_end ();
8248 bool trace_scanning;
8249@end example
8250
8251@noindent
8252Similarly for the parser itself.
8253
1c59e0a1 8254@comment file: calc++-driver.hh
12545799 8255@example
bb32f4f2
AD
8256 // Run the parser. Return 0 on success.
8257 int parse (const std::string& f);
12545799
AD
8258 std::string file;
8259 bool trace_parsing;
8260@end example
8261
8262@noindent
8263To demonstrate pure handling of parse errors, instead of simply
8264dumping them on the standard error output, we will pass them to the
8265compiler driver using the following two member functions. Finally, we
8266close the class declaration and CPP guard.
8267
1c59e0a1 8268@comment file: calc++-driver.hh
12545799
AD
8269@example
8270 // Error handling.
8271 void error (const yy::location& l, const std::string& m);
8272 void error (const std::string& m);
8273@};
8274#endif // ! CALCXX_DRIVER_HH
8275@end example
8276
8277The implementation of the driver is straightforward. The @code{parse}
8278member function deserves some attention. The @code{error} functions
8279are simple stubs, they should actually register the located error
8280messages and set error state.
8281
1c59e0a1 8282@comment file: calc++-driver.cc
12545799
AD
8283@example
8284#include "calc++-driver.hh"
8285#include "calc++-parser.hh"
8286
8287calcxx_driver::calcxx_driver ()
8288 : trace_scanning (false), trace_parsing (false)
8289@{
8290 variables["one"] = 1;
8291 variables["two"] = 2;
8292@}
8293
8294calcxx_driver::~calcxx_driver ()
8295@{
8296@}
8297
bb32f4f2 8298int
12545799
AD
8299calcxx_driver::parse (const std::string &f)
8300@{
8301 file = f;
8302 scan_begin ();
8303 yy::calcxx_parser parser (*this);
8304 parser.set_debug_level (trace_parsing);
bb32f4f2 8305 int res = parser.parse ();
12545799 8306 scan_end ();
bb32f4f2 8307 return res;
12545799
AD
8308@}
8309
8310void
8311calcxx_driver::error (const yy::location& l, const std::string& m)
8312@{
8313 std::cerr << l << ": " << m << std::endl;
8314@}
8315
8316void
8317calcxx_driver::error (const std::string& m)
8318@{
8319 std::cerr << m << std::endl;
8320@}
8321@end example
8322
8323@node Calc++ Parser
8405b70c 8324@subsubsection Calc++ Parser
12545799 8325
b50d2359
AD
8326The parser definition file @file{calc++-parser.yy} starts by asking for
8327the C++ LALR(1) skeleton, the creation of the parser header file, and
8328specifies the name of the parser class. Because the C++ skeleton
8329changed several times, it is safer to require the version you designed
8330the grammar for.
1c59e0a1
AD
8331
8332@comment file: calc++-parser.yy
12545799 8333@example
0e021770 8334%language "C++" /* -*- C++ -*- */
e6e704dc 8335%require "@value{VERSION}"
12545799 8336%defines
16dc6a9e 8337%define parser_class_name "calcxx_parser"
fb9712a9
AD
8338@end example
8339
8340@noindent
16dc6a9e 8341@findex %code requires
fb9712a9
AD
8342Then come the declarations/inclusions needed to define the
8343@code{%union}. Because the parser uses the parsing driver and
8344reciprocally, both cannot include the header of the other. Because the
8345driver's header needs detailed knowledge about the parser class (in
8346particular its inner types), it is the parser's header which will simply
8347use a forward declaration of the driver.
148d66d8 8348@xref{Decl Summary, ,%code}.
fb9712a9
AD
8349
8350@comment file: calc++-parser.yy
8351@example
16dc6a9e 8352%code requires @{
12545799 8353# include <string>
fb9712a9 8354class calcxx_driver;
9bc0dd67 8355@}
12545799
AD
8356@end example
8357
8358@noindent
8359The driver is passed by reference to the parser and to the scanner.
8360This provides a simple but effective pure interface, not relying on
8361global variables.
8362
1c59e0a1 8363@comment file: calc++-parser.yy
12545799
AD
8364@example
8365// The parsing context.
8366%parse-param @{ calcxx_driver& driver @}
8367%lex-param @{ calcxx_driver& driver @}
8368@end example
8369
8370@noindent
8371Then we request the location tracking feature, and initialize the
8372first location's file name. Afterwards new locations are computed
8373relatively to the previous locations: the file name will be
8374automatically propagated.
8375
1c59e0a1 8376@comment file: calc++-parser.yy
12545799
AD
8377@example
8378%locations
8379%initial-action
8380@{
8381 // Initialize the initial location.
b47dbebe 8382 @@$.begin.filename = @@$.end.filename = &driver.file;
12545799
AD
8383@};
8384@end example
8385
8386@noindent
8387Use the two following directives to enable parser tracing and verbose
8388error messages.
8389
1c59e0a1 8390@comment file: calc++-parser.yy
12545799
AD
8391@example
8392%debug
8393%error-verbose
8394@end example
8395
8396@noindent
8397Semantic values cannot use ``real'' objects, but only pointers to
8398them.
8399
1c59e0a1 8400@comment file: calc++-parser.yy
12545799
AD
8401@example
8402// Symbols.
8403%union
8404@{
8405 int ival;
8406 std::string *sval;
8407@};
8408@end example
8409
fb9712a9 8410@noindent
136a0f76
PB
8411@findex %code
8412The code between @samp{%code @{} and @samp{@}} is output in the
34f98f46 8413@file{*.cc} file; it needs detailed knowledge about the driver.
fb9712a9
AD
8414
8415@comment file: calc++-parser.yy
8416@example
136a0f76 8417%code @{
fb9712a9 8418# include "calc++-driver.hh"
34f98f46 8419@}
fb9712a9
AD
8420@end example
8421
8422
12545799
AD
8423@noindent
8424The token numbered as 0 corresponds to end of file; the following line
8425allows for nicer error messages referring to ``end of file'' instead
8426of ``$end''. Similarly user friendly named are provided for each
8427symbol. Note that the tokens names are prefixed by @code{TOKEN_} to
8428avoid name clashes.
8429
1c59e0a1 8430@comment file: calc++-parser.yy
12545799 8431@example
fb9712a9
AD
8432%token END 0 "end of file"
8433%token ASSIGN ":="
8434%token <sval> IDENTIFIER "identifier"
8435%token <ival> NUMBER "number"
a8c2e813 8436%type <ival> exp
12545799
AD
8437@end example
8438
8439@noindent
8440To enable memory deallocation during error recovery, use
8441@code{%destructor}.
8442
287c78f6 8443@c FIXME: Document %printer, and mention that it takes a braced-code operand.
1c59e0a1 8444@comment file: calc++-parser.yy
12545799
AD
8445@example
8446%printer @{ debug_stream () << *$$; @} "identifier"
8447%destructor @{ delete $$; @} "identifier"
8448
a8c2e813 8449%printer @{ debug_stream () << $$; @} <ival>
12545799
AD
8450@end example
8451
8452@noindent
8453The grammar itself is straightforward.
8454
1c59e0a1 8455@comment file: calc++-parser.yy
12545799
AD
8456@example
8457%%
8458%start unit;
8459unit: assignments exp @{ driver.result = $2; @};
8460
8461assignments: assignments assignment @{@}
9d9b8b70 8462 | /* Nothing. */ @{@};
12545799 8463
3dc5e96b
PE
8464assignment:
8465 "identifier" ":=" exp
8466 @{ driver.variables[*$1] = $3; delete $1; @};
12545799
AD
8467
8468%left '+' '-';
8469%left '*' '/';
8470exp: exp '+' exp @{ $$ = $1 + $3; @}
8471 | exp '-' exp @{ $$ = $1 - $3; @}
8472 | exp '*' exp @{ $$ = $1 * $3; @}
8473 | exp '/' exp @{ $$ = $1 / $3; @}
3dc5e96b 8474 | "identifier" @{ $$ = driver.variables[*$1]; delete $1; @}
fb9712a9 8475 | "number" @{ $$ = $1; @};
12545799
AD
8476%%
8477@end example
8478
8479@noindent
8480Finally the @code{error} member function registers the errors to the
8481driver.
8482
1c59e0a1 8483@comment file: calc++-parser.yy
12545799
AD
8484@example
8485void
1c59e0a1
AD
8486yy::calcxx_parser::error (const yy::calcxx_parser::location_type& l,
8487 const std::string& m)
12545799
AD
8488@{
8489 driver.error (l, m);
8490@}
8491@end example
8492
8493@node Calc++ Scanner
8405b70c 8494@subsubsection Calc++ Scanner
12545799
AD
8495
8496The Flex scanner first includes the driver declaration, then the
8497parser's to get the set of defined tokens.
8498
1c59e0a1 8499@comment file: calc++-scanner.ll
12545799
AD
8500@example
8501%@{ /* -*- C++ -*- */
04098407
PE
8502# include <cstdlib>
8503# include <errno.h>
8504# include <limits.h>
12545799
AD
8505# include <string>
8506# include "calc++-driver.hh"
8507# include "calc++-parser.hh"
eaea13f5
PE
8508
8509/* Work around an incompatibility in flex (at least versions
8510 2.5.31 through 2.5.33): it generates code that does
8511 not conform to C89. See Debian bug 333231
8512 <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */
7870f699
PE
8513# undef yywrap
8514# define yywrap() 1
eaea13f5 8515
c095d689
AD
8516/* By default yylex returns int, we use token_type.
8517 Unfortunately yyterminate by default returns 0, which is
8518 not of token_type. */
8c5b881d 8519#define yyterminate() return token::END
12545799
AD
8520%@}
8521@end example
8522
8523@noindent
8524Because there is no @code{#include}-like feature we don't need
8525@code{yywrap}, we don't need @code{unput} either, and we parse an
8526actual file, this is not an interactive session with the user.
8527Finally we enable the scanner tracing features.
8528
1c59e0a1 8529@comment file: calc++-scanner.ll
12545799
AD
8530@example
8531%option noyywrap nounput batch debug
8532@end example
8533
8534@noindent
8535Abbreviations allow for more readable rules.
8536
1c59e0a1 8537@comment file: calc++-scanner.ll
12545799
AD
8538@example
8539id [a-zA-Z][a-zA-Z_0-9]*
8540int [0-9]+
8541blank [ \t]
8542@end example
8543
8544@noindent
9d9b8b70 8545The following paragraph suffices to track locations accurately. Each
12545799
AD
8546time @code{yylex} is invoked, the begin position is moved onto the end
8547position. Then when a pattern is matched, the end position is
8548advanced of its width. In case it matched ends of lines, the end
8549cursor is adjusted, and each time blanks are matched, the begin cursor
8550is moved onto the end cursor to effectively ignore the blanks
8551preceding tokens. Comments would be treated equally.
8552
1c59e0a1 8553@comment file: calc++-scanner.ll
12545799 8554@example
828c373b
AD
8555%@{
8556# define YY_USER_ACTION yylloc->columns (yyleng);
8557%@}
12545799
AD
8558%%
8559%@{
8560 yylloc->step ();
12545799
AD
8561%@}
8562@{blank@}+ yylloc->step ();
8563[\n]+ yylloc->lines (yyleng); yylloc->step ();
8564@end example
8565
8566@noindent
fb9712a9
AD
8567The rules are simple, just note the use of the driver to report errors.
8568It is convenient to use a typedef to shorten
8569@code{yy::calcxx_parser::token::identifier} into
9d9b8b70 8570@code{token::identifier} for instance.
12545799 8571
1c59e0a1 8572@comment file: calc++-scanner.ll
12545799 8573@example
fb9712a9
AD
8574%@{
8575 typedef yy::calcxx_parser::token token;
8576%@}
8c5b881d 8577 /* Convert ints to the actual type of tokens. */
c095d689 8578[-+*/] return yy::calcxx_parser::token_type (yytext[0]);
fb9712a9 8579":=" return token::ASSIGN;
04098407
PE
8580@{int@} @{
8581 errno = 0;
8582 long n = strtol (yytext, NULL, 10);
8583 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
8584 driver.error (*yylloc, "integer is out of range");
8585 yylval->ival = n;
fb9712a9 8586 return token::NUMBER;
04098407 8587@}
fb9712a9 8588@{id@} yylval->sval = new std::string (yytext); return token::IDENTIFIER;
12545799
AD
8589. driver.error (*yylloc, "invalid character");
8590%%
8591@end example
8592
8593@noindent
8594Finally, because the scanner related driver's member function depend
8595on the scanner's data, it is simpler to implement them in this file.
8596
1c59e0a1 8597@comment file: calc++-scanner.ll
12545799
AD
8598@example
8599void
8600calcxx_driver::scan_begin ()
8601@{
8602 yy_flex_debug = trace_scanning;
bb32f4f2
AD
8603 if (file == "-")
8604 yyin = stdin;
8605 else if (!(yyin = fopen (file.c_str (), "r")))
8606 @{
8607 error (std::string ("cannot open ") + file);
8608 exit (1);
8609 @}
12545799
AD
8610@}
8611
8612void
8613calcxx_driver::scan_end ()
8614@{
8615 fclose (yyin);
8616@}
8617@end example
8618
8619@node Calc++ Top Level
8405b70c 8620@subsubsection Calc++ Top Level
12545799
AD
8621
8622The top level file, @file{calc++.cc}, poses no problem.
8623
1c59e0a1 8624@comment file: calc++.cc
12545799
AD
8625@example
8626#include <iostream>
8627#include "calc++-driver.hh"
8628
8629int
fa4d969f 8630main (int argc, char *argv[])
12545799
AD
8631@{
8632 calcxx_driver driver;
8633 for (++argv; argv[0]; ++argv)
8634 if (*argv == std::string ("-p"))
8635 driver.trace_parsing = true;
8636 else if (*argv == std::string ("-s"))
8637 driver.trace_scanning = true;
bb32f4f2
AD
8638 else if (!driver.parse (*argv))
8639 std::cout << driver.result << std::endl;
12545799
AD
8640@}
8641@end example
8642
8405b70c
PB
8643@node Java Parsers
8644@section Java Parsers
8645
8646@menu
8647* Java Bison Interface:: Asking for Java parser generation
8648* Java Semantic Values:: %type and %token vs. Java
8649* Java Location Values:: The position and location classes
8650* Java Parser Interface:: Instantiating and running the parser
8651* Java Scanner Interface:: Java scanners, and pure parsers
8652* Java Differences:: Differences between C/C++ and Java Grammars
8653@end menu
8654
8655@node Java Bison Interface
8656@subsection Java Bison Interface
8657@c - %language "Java"
8658@c - initial action
8659
8660The Java parser skeletons are selected using a language directive,
8661@samp{%language "Java"}, or the synonymous command-line option
8662@option{--language=java}.
8663
8664When run, @command{bison} will create several entities whose name
8665starts with @samp{YY}. Use the @samp{%name-prefix} directive to
8666change the prefix, see @ref{Decl Summary}; classes can be placed
8667in an arbitrary Java package using a @samp{%define package} section.
8668
8669The parser class defines an inner class, @code{Location}, that is used
8670for location tracking. If the parser is pure, it also defines an
8671inner interface, @code{Lexer}; see~@ref{Java Scanner Interface} for the
8672meaning of pure parsers when the Java language is chosen. Other than
8673these inner class/interface, and the members described in~@ref{Java
8674Parser Interface}, all the other members and fields are preceded
8675with a @code{yy} prefix to avoid clashes with user code.
8676
8677No header file can be generated for Java parsers; you must not pass
8678@option{-d}/@option{--defines} to @command{bison}, nor use the
8679@samp{%defines} directive.
8680
8681By default, the @samp{YYParser} class has package visibility. A
8682declaration @samp{%define "public"} will change to public visibility.
8683Remember that, according to the Java language specification, the name
8684of the @file{.java} file should match the name of the class in this
8685case.
8686
01b477c6
PB
8687Similarly, a declaration @samp{%define "abstract"} will make your
8688class abstract.
8689
8690You can create documentation for generated parsers using Javadoc.
8405b70c
PB
8691
8692@node Java Semantic Values
8693@subsection Java Semantic Values
8694@c - No %union, specify type in %type/%token.
8695@c - YYSTYPE
8696@c - Printer and destructor
8697
8698There is no @code{%union} directive in Java parsers. Instead, the
8699semantic values' types (class names) should be specified in the
8700@code{%type} or @code{%token} directive:
8701
8702@example
8703%type <Expression> expr assignment_expr term factor
8704%type <Integer> number
8705@end example
8706
8707By default, the semantic stack is declared to have @code{Object} members,
8708which means that the class types you specify can be of any class.
8709To improve the type safety of the parser, you can declare the common
8710superclass of all the semantic values using the @samp{%define} directive.
8711For example, after the following declaration:
8712
8713@example
01b477c6 8714%define "stype" "ASTNode"
8405b70c
PB
8715@end example
8716
8717@noindent
8718any @code{%type} or @code{%token} specifying a semantic type which
8719is not a subclass of ASTNode, will cause a compile-time error.
8720
8721Types used in the directives may be qualified with a package name.
8722Primitive data types are accepted for Java version 1.5 or later. Note
8723that in this case the autoboxing feature of Java 1.5 will be used.
8724
8725Java parsers do not support @code{%destructor}, since the language
8726adopts garbage collection. The parser will try to hold references
8727to semantic values for as little time as needed.
8728
8729Java parsers do not support @code{%printer}, as @code{toString()}
8730can be used to print the semantic values. This however may change
8731(in a backwards-compatible way) in future versions of Bison.
8732
8733
8734@node Java Location Values
8735@subsection Java Location Values
8736@c - %locations
8737@c - class Position
8738@c - class Location
8739
8740When the directive @code{%locations} is used, the Java parser
8741supports location tracking, see @ref{Locations, , Locations Overview}.
8742An auxiliary user-defined class defines a @dfn{position}, a single point
8743in a file; Bison itself defines a class representing a @dfn{location},
8744a range composed of a pair of positions (possibly spanning several
8745files). The location class is an inner class of the parser; the name
8746is @code{Location} by default, may also be renamed using @code{%define
8747"location_type" "@var{class-name}}.
8748
8749The location class treats the position as a completely opaque value.
8750By default, the class name is @code{Position}, but this can be changed
8751with @code{%define "position_type" "@var{class-name}"}.
8752
8753
8754@deftypemethod {Location} {Position} begin
8755@deftypemethodx {Location} {Position} end
8756The first, inclusive, position of the range, and the first beyond.
8757@end deftypemethod
8758
8759@deftypemethod {Location} {void} toString ()
8760Prints the range represented by the location. For this to work
8761properly, the position class should override the @code{equals} and
8762@code{toString} methods appropriately.
8763@end deftypemethod
8764
8765
8766@node Java Parser Interface
8767@subsection Java Parser Interface
8768@c - define parser_class_name
8769@c - Ctor
8770@c - parse, error, set_debug_level, debug_level, set_debug_stream,
8771@c debug_stream.
8772@c - Reporting errors
8773
8774The output file defines the parser class in the package optionally
8775indicated in the @code{%define package} section. The class name defaults
8776to @code{YYParser}. The @code{YY} prefix may be changed using
8777@samp{%name-prefix}; alternatively, you can use @samp{%define
8778"parser_class_name" "@var{name}"} to give a custom name to the class.
8779The interface of this class is detailed below. It can be extended using
8780the @code{%parse-param} directive; each occurrence of the directive will
8781add a field to the parser class, and an argument to its constructor.
8782
8783@deftypemethod {YYParser} {} YYParser (@var{type1} @var{arg1}, ...)
8784Build a new parser object. There are no arguments by default, unless
8785@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
8786@end deftypemethod
8787
8788@deftypemethod {YYParser} {boolean} parse ()
8789Run the syntactic analysis, and return @code{true} on success,
8790@code{false} otherwise.
8791@end deftypemethod
8792
01b477c6 8793@deftypemethod {YYParser} {boolean} recovering ()
8405b70c
PB
8794During the syntactic analysis, return @code{true} if recovering
8795from a syntax error. @xref{Error Recovery}.
8796@end deftypemethod
8797
8798@deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
8799@deftypemethodx {YYParser} {void} setDebugStream (java.io.printStream @var{o})
8800Get or set the stream used for tracing the parsing. It defaults to
8801@code{System.err}.
8802@end deftypemethod
8803
8804@deftypemethod {YYParser} {int} getDebugLevel ()
8805@deftypemethodx {YYParser} {void} setDebugLevel (int @var{l})
8806Get or set the tracing level. Currently its value is either 0, no trace,
8807or nonzero, full tracing.
8808@end deftypemethod
8809
8810@deftypemethod {YYParser} {void} error (Location @var{l}, String @var{m})
8811The definition for this member function must be supplied by the user
8812in the same way as the scanner interface (@pxref{Java Scanner
8813Interface}); the parser uses it to report a parser error occurring at
8814@var{l}, described by @var{m}.
8815@end deftypemethod
8816
8817
8818@node Java Scanner Interface
8819@subsection Java Scanner Interface
01b477c6 8820@c - %code lexer
8405b70c 8821@c - %lex-param
01b477c6 8822@c - Lexer interface
8405b70c 8823
8405b70c
PB
8824Contrary to C parsers, Java parsers do not use global variables; the
8825state of the parser is always local to an instance of the parser class.
01b477c6
PB
8826Therefore, all Java parsers are ``pure'', and the @code{%pure-parser}
8827directive does not do anything when used in Java.
8828
8829The scanner always resides in a separate class than the parser.
8830Still, Java also two possible ways to interface a Bison-generated Java
8831parser with a scanner, that is, the scanner may reside in a separate file
8832than the Bison grammar, or in the same file. The interface
8833to the scanner is similar in the two cases.
8834
8835In the first case, where the scanner in the same file as the grammar, the
8836scanner code has to be placed in @code{%code lexer} blocks. If you want
8837to pass parameters from the parser constructor to the scanner constructor,
8838specify them with @code{%lex-param}; they are passed before
8839@code{%parse-param}s to the constructor.
8840
8841In the second case, the scanner has to implement interface @code{Lexer},
8842which is defined within the parser class (e.g., @code{YYParser.Lexer}).
8843The constructor of the parser object will then accept an object
8844implementing the interface; @code{%lex-param} is not used in this
8845case.
8846
8847In both cases, the scanner has to implement the following methods.
8848
8849@deftypemethod {Lexer} {void} yyerror (Location @var{l}, String @var{m})
8405b70c 8850As explained in @pxref{Java Parser Interface}, this method is defined
01b477c6
PB
8851by the user to emit an error message. The first parameter is omitted
8852if location tracking is not active. Its type can be changed using
8405b70c
PB
8853@samp{%define "location_type" "@var{class-name}".}
8854@end deftypemethod
8855
8856@deftypemethod {Lexer} {int} yylex (@var{type1} @var{arg1}, ...)
8857Return the next token. Its type is the return value, its semantic
8858value and location are saved and returned by the ther methods in the
8859interface. Invocations of @samp{%lex-param @{@var{type1}
8860@var{arg1}@}} yield additional arguments.
8861@end deftypemethod
8862
8863@deftypemethod {Lexer} {Position} getStartPos ()
8864@deftypemethodx {Lexer} {Position} getEndPos ()
01b477c6
PB
8865Return respectively the first position of the last token that
8866@code{yylex} returned, and the first position beyond it. These
8867methods are not needed unless location tracking is active.
8405b70c
PB
8868
8869The return type can be changed using @samp{%define "position_type"
8870"@var{class-name}".}
8871@end deftypemethod
8872
8873@deftypemethod {Lexer} {Object} getLVal ()
8874Return respectively the first position of the last token that yylex
8875returned, and the first position beyond it.
8876
01b477c6 8877The return type can be changed using @samp{%define "stype"
8405b70c
PB
8878"@var{class-name}".}
8879@end deftypemethod
8880
8881
d9df47b6
JD
8882The lexer interface resides in the same class (@code{YYParser}) as the
8883Bison-generated parser.
8884The fields and methods that are provided to this end are as follows.
8405b70c
PB
8885
8886@deftypemethod {YYParser} {void} error (Location @var{l}, String @var{m})
8887As explained in @pxref{Java Parser Interface}, this method is defined
8888by the user to emit an error message. The first parameter is not used
8889unless location tracking is active. Its type can be changed using
8890@samp{%define "location_type" "@var{class-name}".}
8891@end deftypemethod
8892
8893@deftypemethod {YYParser} {int} yylex (@var{type1} @var{arg1}, ...)
8894Return the next token. Its type is the return value, its semantic
8895value and location are saved into @code{yylval}, @code{yystartpos},
8896@code{yyendpos}. Invocations of @samp{%lex-param @{@var{type1}
8897@var{arg1}@}} yield additional arguments.
8898@end deftypemethod
8899
8900@deftypecv {Field} {YYParser} Position yystartpos
8901@deftypecvx {Field} {YYParser} Position yyendpos
8902Contain respectively the first position of the last token that yylex
8903returned, and the first position beyond it. These methods are not
8904needed unless location tracking is active.
8905
8906The field's type can be changed using @samp{%define "position_type"
8907"@var{class-name}".}
8908@end deftypecv
8909
8910@deftypecv {Field} {YYParser} Object yylval
8911Return respectively the first position of the last token that yylex
8912returned, and the first position beyond it.
8913
01b477c6 8914The field's type can be changed using @samp{%define "stype"
8405b70c
PB
8915"@var{class-name}".}
8916@end deftypecv
8917
8405b70c
PB
8918@node Java Differences
8919@subsection Differences between C/C++ and Java Grammars
8920
8921The different structure of the Java language forces several differences
8922between C/C++ grammars, and grammars designed for Java parsers. This
29553547 8923section summarizes these differences.
8405b70c
PB
8924
8925@itemize
8926@item
01b477c6 8927Java lacks a preprocessor, so the @code{YYERROR}, @code{YYACCEPT},
8405b70c 8928@code{YYABORT} symbols (@pxref{Table of Symbols}) cannot obviously be
01b477c6
PB
8929macros. Instead, they should be preceded by @code{return} when they
8930appear in an action. The actual definition of these symbols is
8405b70c
PB
8931opaque to the Bison grammar, and it might change in the future. The
8932only meaningful operation that you can do, is to return them.
8933
8934Note that of these three symbols, only @code{YYACCEPT} and
8935@code{YYABORT} will cause a return from the @code{yyparse}
8936method@footnote{Java parsers include the actions in a separate
8937method than @code{yyparse} in order to have an intuitive syntax that
8938corresponds to these C macros.}.
8939
8940@item
8941The prolog declarations have a different meaning than in C/C++ code.
01b477c6
PB
8942@table @asis
8943@item @code{%code imports}
8944blocks are placed at the beginning of the Java source code. They may
8945include copyright notices. For a @code{package} declarations, it is
8946suggested to use @code{%define package} instead.
8405b70c 8947
01b477c6
PB
8948@item unqualified @code{%code}
8949blocks are placed inside the parser class.
8950
8951@item @code{%code lexer}
8952blocks, if specified, should include the implementation of the
8953scanner. If there is no such block, the scanner can be any class
8954that implements the appropriate interface (see @pxref{Java Scanner
8955Interface}).
29553547 8956@end table
8405b70c
PB
8957
8958Other @code{%code} blocks are not supported in Java parsers.
01b477c6
PB
8959The epilogue has the same meaning as in C/C++ code and it can
8960be used to define other classes used by the parser.
8405b70c
PB
8961@end itemize
8962
12545799 8963@c ================================================= FAQ
d1a1114f
AD
8964
8965@node FAQ
8966@chapter Frequently Asked Questions
8967@cindex frequently asked questions
8968@cindex questions
8969
8970Several questions about Bison come up occasionally. Here some of them
8971are addressed.
8972
8973@menu
55ba27be
AD
8974* Memory Exhausted:: Breaking the Stack Limits
8975* How Can I Reset the Parser:: @code{yyparse} Keeps some State
8976* Strings are Destroyed:: @code{yylval} Loses Track of Strings
8977* Implementing Gotos/Loops:: Control Flow in the Calculator
ed2e6384 8978* Multiple start-symbols:: Factoring closely related grammars
55ba27be
AD
8979* Secure? Conform?:: Is Bison @acronym{POSIX} safe?
8980* I can't build Bison:: Troubleshooting
8981* Where can I find help?:: Troubleshouting
8982* Bug Reports:: Troublereporting
8405b70c 8983* More Languages:: Parsers in C++, Java, and so on
55ba27be
AD
8984* Beta Testing:: Experimenting development versions
8985* Mailing Lists:: Meeting other Bison users
d1a1114f
AD
8986@end menu
8987
1a059451
PE
8988@node Memory Exhausted
8989@section Memory Exhausted
d1a1114f
AD
8990
8991@display
1a059451 8992My parser returns with error with a @samp{memory exhausted}
d1a1114f
AD
8993message. What can I do?
8994@end display
8995
8996This question is already addressed elsewhere, @xref{Recursion,
8997,Recursive Rules}.
8998
e64fec0a
PE
8999@node How Can I Reset the Parser
9000@section How Can I Reset the Parser
5b066063 9001
0e14ad77
PE
9002The following phenomenon has several symptoms, resulting in the
9003following typical questions:
5b066063
AD
9004
9005@display
9006I invoke @code{yyparse} several times, and on correct input it works
9007properly; but when a parse error is found, all the other calls fail
0e14ad77 9008too. How can I reset the error flag of @code{yyparse}?
5b066063
AD
9009@end display
9010
9011@noindent
9012or
9013
9014@display
0e14ad77 9015My parser includes support for an @samp{#include}-like feature, in
5b066063 9016which case I run @code{yyparse} from @code{yyparse}. This fails
d9df47b6 9017although I did specify @code{%define api.pure}.
5b066063
AD
9018@end display
9019
0e14ad77
PE
9020These problems typically come not from Bison itself, but from
9021Lex-generated scanners. Because these scanners use large buffers for
5b066063
AD
9022speed, they might not notice a change of input file. As a
9023demonstration, consider the following source file,
9024@file{first-line.l}:
9025
9026@verbatim
9027%{
9028#include <stdio.h>
9029#include <stdlib.h>
9030%}
9031%%
9032.*\n ECHO; return 1;
9033%%
9034int
0e14ad77 9035yyparse (char const *file)
5b066063
AD
9036{
9037 yyin = fopen (file, "r");
9038 if (!yyin)
9039 exit (2);
fa7e68c3 9040 /* One token only. */
5b066063 9041 yylex ();
0e14ad77 9042 if (fclose (yyin) != 0)
5b066063
AD
9043 exit (3);
9044 return 0;
9045}
9046
9047int
0e14ad77 9048main (void)
5b066063
AD
9049{
9050 yyparse ("input");
9051 yyparse ("input");
9052 return 0;
9053}
9054@end verbatim
9055
9056@noindent
9057If the file @file{input} contains
9058
9059@verbatim
9060input:1: Hello,
9061input:2: World!
9062@end verbatim
9063
9064@noindent
0e14ad77 9065then instead of getting the first line twice, you get:
5b066063
AD
9066
9067@example
9068$ @kbd{flex -ofirst-line.c first-line.l}
9069$ @kbd{gcc -ofirst-line first-line.c -ll}
9070$ @kbd{./first-line}
9071input:1: Hello,
9072input:2: World!
9073@end example
9074
0e14ad77
PE
9075Therefore, whenever you change @code{yyin}, you must tell the
9076Lex-generated scanner to discard its current buffer and switch to the
9077new one. This depends upon your implementation of Lex; see its
9078documentation for more. For Flex, it suffices to call
9079@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
9080Flex-generated scanner needs to read from several input streams to
9081handle features like include files, you might consider using Flex
9082functions like @samp{yy_switch_to_buffer} that manipulate multiple
9083input buffers.
5b066063 9084
b165c324
AD
9085If your Flex-generated scanner uses start conditions (@pxref{Start
9086conditions, , Start conditions, flex, The Flex Manual}), you might
9087also want to reset the scanner's state, i.e., go back to the initial
9088start condition, through a call to @samp{BEGIN (0)}.
9089
fef4cb51
AD
9090@node Strings are Destroyed
9091@section Strings are Destroyed
9092
9093@display
c7e441b4 9094My parser seems to destroy old strings, or maybe it loses track of
fef4cb51
AD
9095them. Instead of reporting @samp{"foo", "bar"}, it reports
9096@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
9097@end display
9098
9099This error is probably the single most frequent ``bug report'' sent to
9100Bison lists, but is only concerned with a misunderstanding of the role
8c5b881d 9101of the scanner. Consider the following Lex code:
fef4cb51
AD
9102
9103@verbatim
9104%{
9105#include <stdio.h>
9106char *yylval = NULL;
9107%}
9108%%
9109.* yylval = yytext; return 1;
9110\n /* IGNORE */
9111%%
9112int
9113main ()
9114{
fa7e68c3 9115 /* Similar to using $1, $2 in a Bison action. */
fef4cb51
AD
9116 char *fst = (yylex (), yylval);
9117 char *snd = (yylex (), yylval);
9118 printf ("\"%s\", \"%s\"\n", fst, snd);
9119 return 0;
9120}
9121@end verbatim
9122
9123If you compile and run this code, you get:
9124
9125@example
9126$ @kbd{flex -osplit-lines.c split-lines.l}
9127$ @kbd{gcc -osplit-lines split-lines.c -ll}
9128$ @kbd{printf 'one\ntwo\n' | ./split-lines}
9129"one
9130two", "two"
9131@end example
9132
9133@noindent
9134this is because @code{yytext} is a buffer provided for @emph{reading}
9135in the action, but if you want to keep it, you have to duplicate it
9136(e.g., using @code{strdup}). Note that the output may depend on how
9137your implementation of Lex handles @code{yytext}. For instance, when
9138given the Lex compatibility option @option{-l} (which triggers the
9139option @samp{%array}) Flex generates a different behavior:
9140
9141@example
9142$ @kbd{flex -l -osplit-lines.c split-lines.l}
9143$ @kbd{gcc -osplit-lines split-lines.c -ll}
9144$ @kbd{printf 'one\ntwo\n' | ./split-lines}
9145"two", "two"
9146@end example
9147
9148
2fa09258
AD
9149@node Implementing Gotos/Loops
9150@section Implementing Gotos/Loops
a06ea4aa
AD
9151
9152@display
9153My simple calculator supports variables, assignments, and functions,
2fa09258 9154but how can I implement gotos, or loops?
a06ea4aa
AD
9155@end display
9156
9157Although very pedagogical, the examples included in the document blur
a1c84f45 9158the distinction to make between the parser---whose job is to recover
a06ea4aa 9159the structure of a text and to transmit it to subsequent modules of
a1c84f45 9160the program---and the processing (such as the execution) of this
a06ea4aa
AD
9161structure. This works well with so called straight line programs,
9162i.e., precisely those that have a straightforward execution model:
9163execute simple instructions one after the others.
9164
9165@cindex abstract syntax tree
9166@cindex @acronym{AST}
9167If you want a richer model, you will probably need to use the parser
9168to construct a tree that does represent the structure it has
9169recovered; this tree is usually called the @dfn{abstract syntax tree},
9170or @dfn{@acronym{AST}} for short. Then, walking through this tree,
9171traversing it in various ways, will enable treatments such as its
9172execution or its translation, which will result in an interpreter or a
9173compiler.
9174
9175This topic is way beyond the scope of this manual, and the reader is
9176invited to consult the dedicated literature.
9177
9178
ed2e6384
AD
9179@node Multiple start-symbols
9180@section Multiple start-symbols
9181
9182@display
9183I have several closely related grammars, and I would like to share their
9184implementations. In fact, I could use a single grammar but with
9185multiple entry points.
9186@end display
9187
9188Bison does not support multiple start-symbols, but there is a very
9189simple means to simulate them. If @code{foo} and @code{bar} are the two
9190pseudo start-symbols, then introduce two new tokens, say
9191@code{START_FOO} and @code{START_BAR}, and use them as switches from the
9192real start-symbol:
9193
9194@example
9195%token START_FOO START_BAR;
9196%start start;
9197start: START_FOO foo
9198 | START_BAR bar;
9199@end example
9200
9201These tokens prevents the introduction of new conflicts. As far as the
9202parser goes, that is all that is needed.
9203
9204Now the difficult part is ensuring that the scanner will send these
9205tokens first. If your scanner is hand-written, that should be
9206straightforward. If your scanner is generated by Lex, them there is
9207simple means to do it: recall that anything between @samp{%@{ ... %@}}
9208after the first @code{%%} is copied verbatim in the top of the generated
9209@code{yylex} function. Make sure a variable @code{start_token} is
9210available in the scanner (e.g., a global variable or using
9211@code{%lex-param} etc.), and use the following:
9212
9213@example
9214 /* @r{Prologue.} */
9215%%
9216%@{
9217 if (start_token)
9218 @{
9219 int t = start_token;
9220 start_token = 0;
9221 return t;
9222 @}
9223%@}
9224 /* @r{The rules.} */
9225@end example
9226
9227
55ba27be
AD
9228@node Secure? Conform?
9229@section Secure? Conform?
9230
9231@display
9232Is Bison secure? Does it conform to POSIX?
9233@end display
9234
9235If you're looking for a guarantee or certification, we don't provide it.
9236However, Bison is intended to be a reliable program that conforms to the
9237@acronym{POSIX} specification for Yacc. If you run into problems,
9238please send us a bug report.
9239
9240@node I can't build Bison
9241@section I can't build Bison
9242
9243@display
8c5b881d
PE
9244I can't build Bison because @command{make} complains that
9245@code{msgfmt} is not found.
55ba27be
AD
9246What should I do?
9247@end display
9248
9249Like most GNU packages with internationalization support, that feature
9250is turned on by default. If you have problems building in the @file{po}
9251subdirectory, it indicates that your system's internationalization
9252support is lacking. You can re-configure Bison with
9253@option{--disable-nls} to turn off this support, or you can install GNU
9254gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
9255Bison. See the file @file{ABOUT-NLS} for more information.
9256
9257
9258@node Where can I find help?
9259@section Where can I find help?
9260
9261@display
9262I'm having trouble using Bison. Where can I find help?
9263@end display
9264
9265First, read this fine manual. Beyond that, you can send mail to
9266@email{help-bison@@gnu.org}. This mailing list is intended to be
9267populated with people who are willing to answer questions about using
9268and installing Bison. Please keep in mind that (most of) the people on
9269the list have aspects of their lives which are not related to Bison (!),
9270so you may not receive an answer to your question right away. This can
9271be frustrating, but please try not to honk them off; remember that any
9272help they provide is purely voluntary and out of the kindness of their
9273hearts.
9274
9275@node Bug Reports
9276@section Bug Reports
9277
9278@display
9279I found a bug. What should I include in the bug report?
9280@end display
9281
9282Before you send a bug report, make sure you are using the latest
9283version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
9284mirrors. Be sure to include the version number in your bug report. If
9285the bug is present in the latest version but not in a previous version,
9286try to determine the most recent version which did not contain the bug.
9287
9288If the bug is parser-related, you should include the smallest grammar
9289you can which demonstrates the bug. The grammar file should also be
9290complete (i.e., I should be able to run it through Bison without having
9291to edit or add anything). The smaller and simpler the grammar, the
9292easier it will be to fix the bug.
9293
9294Include information about your compilation environment, including your
9295operating system's name and version and your compiler's name and
9296version. If you have trouble compiling, you should also include a
9297transcript of the build session, starting with the invocation of
9298`configure'. Depending on the nature of the bug, you may be asked to
9299send additional files as well (such as `config.h' or `config.cache').
9300
9301Patches are most welcome, but not required. That is, do not hesitate to
9302send a bug report just because you can not provide a fix.
9303
9304Send bug reports to @email{bug-bison@@gnu.org}.
9305
8405b70c
PB
9306@node More Languages
9307@section More Languages
55ba27be
AD
9308
9309@display
8405b70c 9310Will Bison ever have C++ and Java support? How about @var{insert your
55ba27be
AD
9311favorite language here}?
9312@end display
9313
8405b70c 9314C++ and Java support is there now, and is documented. We'd love to add other
55ba27be
AD
9315languages; contributions are welcome.
9316
9317@node Beta Testing
9318@section Beta Testing
9319
9320@display
9321What is involved in being a beta tester?
9322@end display
9323
9324It's not terribly involved. Basically, you would download a test
9325release, compile it, and use it to build and run a parser or two. After
9326that, you would submit either a bug report or a message saying that
9327everything is okay. It is important to report successes as well as
9328failures because test releases eventually become mainstream releases,
9329but only if they are adequately tested. If no one tests, development is
9330essentially halted.
9331
9332Beta testers are particularly needed for operating systems to which the
9333developers do not have easy access. They currently have easy access to
9334recent GNU/Linux and Solaris versions. Reports about other operating
9335systems are especially welcome.
9336
9337@node Mailing Lists
9338@section Mailing Lists
9339
9340@display
9341How do I join the help-bison and bug-bison mailing lists?
9342@end display
9343
9344See @url{http://lists.gnu.org/}.
a06ea4aa 9345
d1a1114f
AD
9346@c ================================================= Table of Symbols
9347
342b8b6e 9348@node Table of Symbols
bfa74976
RS
9349@appendix Bison Symbols
9350@cindex Bison symbols, table of
9351@cindex symbols in Bison, table of
9352
18b519c0 9353@deffn {Variable} @@$
3ded9a63 9354In an action, the location of the left-hand side of the rule.
88bce5a2 9355@xref{Locations, , Locations Overview}.
18b519c0 9356@end deffn
3ded9a63 9357
18b519c0 9358@deffn {Variable} @@@var{n}
3ded9a63
AD
9359In an action, the location of the @var{n}-th symbol of the right-hand
9360side of the rule. @xref{Locations, , Locations Overview}.
18b519c0 9361@end deffn
3ded9a63 9362
18b519c0 9363@deffn {Variable} $$
3ded9a63
AD
9364In an action, the semantic value of the left-hand side of the rule.
9365@xref{Actions}.
18b519c0 9366@end deffn
3ded9a63 9367
18b519c0 9368@deffn {Variable} $@var{n}
3ded9a63
AD
9369In an action, the semantic value of the @var{n}-th symbol of the
9370right-hand side of the rule. @xref{Actions}.
18b519c0 9371@end deffn
3ded9a63 9372
dd8d9022
AD
9373@deffn {Delimiter} %%
9374Delimiter used to separate the grammar rule section from the
9375Bison declarations section or the epilogue.
9376@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
18b519c0 9377@end deffn
bfa74976 9378
dd8d9022
AD
9379@c Don't insert spaces, or check the DVI output.
9380@deffn {Delimiter} %@{@var{code}%@}
9381All code listed between @samp{%@{} and @samp{%@}} is copied directly to
9382the output file uninterpreted. Such code forms the prologue of the input
9383file. @xref{Grammar Outline, ,Outline of a Bison
9384Grammar}.
18b519c0 9385@end deffn
bfa74976 9386
dd8d9022
AD
9387@deffn {Construct} /*@dots{}*/
9388Comment delimiters, as in C.
18b519c0 9389@end deffn
bfa74976 9390
dd8d9022
AD
9391@deffn {Delimiter} :
9392Separates a rule's result from its components. @xref{Rules, ,Syntax of
9393Grammar Rules}.
18b519c0 9394@end deffn
bfa74976 9395
dd8d9022
AD
9396@deffn {Delimiter} ;
9397Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 9398@end deffn
bfa74976 9399
dd8d9022
AD
9400@deffn {Delimiter} |
9401Separates alternate rules for the same result nonterminal.
9402@xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 9403@end deffn
bfa74976 9404
12e35840
JD
9405@deffn {Directive} <*>
9406Used to define a default tagged @code{%destructor} or default tagged
9407@code{%printer}.
85894313
JD
9408
9409This feature is experimental.
9410More user feedback will help to determine whether it should become a permanent
9411feature.
9412
12e35840
JD
9413@xref{Destructor Decl, , Freeing Discarded Symbols}.
9414@end deffn
9415
3ebecc24 9416@deffn {Directive} <>
12e35840
JD
9417Used to define a default tagless @code{%destructor} or default tagless
9418@code{%printer}.
85894313
JD
9419
9420This feature is experimental.
9421More user feedback will help to determine whether it should become a permanent
9422feature.
9423
12e35840
JD
9424@xref{Destructor Decl, , Freeing Discarded Symbols}.
9425@end deffn
9426
dd8d9022
AD
9427@deffn {Symbol} $accept
9428The predefined nonterminal whose only rule is @samp{$accept: @var{start}
9429$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
9430Start-Symbol}. It cannot be used in the grammar.
18b519c0 9431@end deffn
bfa74976 9432
136a0f76 9433@deffn {Directive} %code @{@var{code}@}
148d66d8
JD
9434@deffnx {Directive} %code @var{qualifier} @{@var{code}@}
9435Insert @var{code} verbatim into output parser source.
9436@xref{Decl Summary,,%code}.
9bc0dd67
JD
9437@end deffn
9438
9439@deffn {Directive} %debug
9440Equip the parser for debugging. @xref{Decl Summary}.
9441@end deffn
9442
18b519c0 9443@deffn {Directive} %debug
6deb4447 9444Equip the parser for debugging. @xref{Decl Summary}.
18b519c0 9445@end deffn
6deb4447 9446
91d2c560 9447@ifset defaultprec
22fccf95
PE
9448@deffn {Directive} %default-prec
9449Assign a precedence to rules that lack an explicit @samp{%prec}
9450modifier. @xref{Contextual Precedence, ,Context-Dependent
9451Precedence}.
39a06c25 9452@end deffn
91d2c560 9453@end ifset
39a06c25 9454
148d66d8
JD
9455@deffn {Directive} %define @var{define-variable}
9456@deffnx {Directive} %define @var{define-variable} @var{value}
9457Define a variable to adjust Bison's behavior.
9458@xref{Decl Summary,,%define}.
9459@end deffn
9460
18b519c0 9461@deffn {Directive} %defines
6deb4447
AD
9462Bison declaration to create a header file meant for the scanner.
9463@xref{Decl Summary}.
18b519c0 9464@end deffn
6deb4447 9465
02975b9a
JD
9466@deffn {Directive} %defines @var{defines-file}
9467Same as above, but save in the file @var{defines-file}.
9468@xref{Decl Summary}.
9469@end deffn
9470
18b519c0 9471@deffn {Directive} %destructor
258b75ca 9472Specify how the parser should reclaim the memory associated to
fa7e68c3 9473discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 9474@end deffn
72f889cc 9475
18b519c0 9476@deffn {Directive} %dprec
676385e2 9477Bison declaration to assign a precedence to a rule that is used at parse
c827f760
PE
9478time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
9479@acronym{GLR} Parsers}.
18b519c0 9480@end deffn
676385e2 9481
dd8d9022
AD
9482@deffn {Symbol} $end
9483The predefined token marking the end of the token stream. It cannot be
9484used in the grammar.
9485@end deffn
9486
9487@deffn {Symbol} error
9488A token name reserved for error recovery. This token may be used in
9489grammar rules so as to allow the Bison parser to recognize an error in
9490the grammar without halting the process. In effect, a sentence
9491containing an error may be recognized as valid. On a syntax error, the
742e4900
JD
9492token @code{error} becomes the current lookahead token. Actions
9493corresponding to @code{error} are then executed, and the lookahead
dd8d9022
AD
9494token is reset to the token that originally caused the violation.
9495@xref{Error Recovery}.
18d192f0
AD
9496@end deffn
9497
18b519c0 9498@deffn {Directive} %error-verbose
2a8d363a
AD
9499Bison declaration to request verbose, specific error message strings
9500when @code{yyerror} is called.
18b519c0 9501@end deffn
2a8d363a 9502
02975b9a 9503@deffn {Directive} %file-prefix "@var{prefix}"
72d2299c 9504Bison declaration to set the prefix of the output files. @xref{Decl
d8988b2f 9505Summary}.
18b519c0 9506@end deffn
d8988b2f 9507
18b519c0 9508@deffn {Directive} %glr-parser
c827f760
PE
9509Bison declaration to produce a @acronym{GLR} parser. @xref{GLR
9510Parsers, ,Writing @acronym{GLR} Parsers}.
18b519c0 9511@end deffn
676385e2 9512
dd8d9022
AD
9513@deffn {Directive} %initial-action
9514Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
9515@end deffn
9516
e6e704dc
JD
9517@deffn {Directive} %language
9518Specify the programming language for the generated parser.
9519@xref{Decl Summary}.
9520@end deffn
9521
18b519c0 9522@deffn {Directive} %left
bfa74976
RS
9523Bison declaration to assign left associativity to token(s).
9524@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 9525@end deffn
bfa74976 9526
feeb0eda 9527@deffn {Directive} %lex-param @{@var{argument-declaration}@}
2a8d363a
AD
9528Bison declaration to specifying an additional parameter that
9529@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
9530for Pure Parsers}.
18b519c0 9531@end deffn
2a8d363a 9532
18b519c0 9533@deffn {Directive} %merge
676385e2 9534Bison declaration to assign a merging function to a rule. If there is a
fae437e8 9535reduce/reduce conflict with a rule having the same merging function, the
676385e2 9536function is applied to the two semantic values to get a single result.
c827f760 9537@xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}.
18b519c0 9538@end deffn
676385e2 9539
02975b9a 9540@deffn {Directive} %name-prefix "@var{prefix}"
72d2299c 9541Bison declaration to rename the external symbols. @xref{Decl Summary}.
18b519c0 9542@end deffn
d8988b2f 9543
91d2c560 9544@ifset defaultprec
22fccf95
PE
9545@deffn {Directive} %no-default-prec
9546Do not assign a precedence to rules that lack an explicit @samp{%prec}
9547modifier. @xref{Contextual Precedence, ,Context-Dependent
9548Precedence}.
9549@end deffn
91d2c560 9550@end ifset
22fccf95 9551
18b519c0 9552@deffn {Directive} %no-lines
931c7513
RS
9553Bison declaration to avoid generating @code{#line} directives in the
9554parser file. @xref{Decl Summary}.
18b519c0 9555@end deffn
931c7513 9556
18b519c0 9557@deffn {Directive} %nonassoc
9d9b8b70 9558Bison declaration to assign nonassociativity to token(s).
bfa74976 9559@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 9560@end deffn
bfa74976 9561
02975b9a 9562@deffn {Directive} %output "@var{file}"
72d2299c 9563Bison declaration to set the name of the parser file. @xref{Decl
d8988b2f 9564Summary}.
18b519c0 9565@end deffn
d8988b2f 9566
feeb0eda 9567@deffn {Directive} %parse-param @{@var{argument-declaration}@}
2a8d363a
AD
9568Bison declaration to specifying an additional parameter that
9569@code{yyparse} should accept. @xref{Parser Function,, The Parser
9570Function @code{yyparse}}.
18b519c0 9571@end deffn
2a8d363a 9572
18b519c0 9573@deffn {Directive} %prec
bfa74976
RS
9574Bison declaration to assign a precedence to a specific rule.
9575@xref{Contextual Precedence, ,Context-Dependent Precedence}.
18b519c0 9576@end deffn
bfa74976 9577
18b519c0 9578@deffn {Directive} %pure-parser
d9df47b6
JD
9579Deprecated version of @code{%define api.pure} (@pxref{Decl Summary, ,%define}),
9580for which Bison is more careful to warn about unreasonable usage.
18b519c0 9581@end deffn
bfa74976 9582
b50d2359 9583@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
9584Require version @var{version} or higher of Bison. @xref{Require Decl, ,
9585Require a Version of Bison}.
b50d2359
AD
9586@end deffn
9587
18b519c0 9588@deffn {Directive} %right
bfa74976
RS
9589Bison declaration to assign right associativity to token(s).
9590@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 9591@end deffn
bfa74976 9592
e6e704dc
JD
9593@deffn {Directive} %skeleton
9594Specify the skeleton to use; usually for development.
9595@xref{Decl Summary}.
9596@end deffn
9597
18b519c0 9598@deffn {Directive} %start
704a47c4
AD
9599Bison declaration to specify the start symbol. @xref{Start Decl, ,The
9600Start-Symbol}.
18b519c0 9601@end deffn
bfa74976 9602
18b519c0 9603@deffn {Directive} %token
bfa74976
RS
9604Bison declaration to declare token(s) without specifying precedence.
9605@xref{Token Decl, ,Token Type Names}.
18b519c0 9606@end deffn
bfa74976 9607
18b519c0 9608@deffn {Directive} %token-table
931c7513
RS
9609Bison declaration to include a token name table in the parser file.
9610@xref{Decl Summary}.
18b519c0 9611@end deffn
931c7513 9612
18b519c0 9613@deffn {Directive} %type
704a47c4
AD
9614Bison declaration to declare nonterminals. @xref{Type Decl,
9615,Nonterminal Symbols}.
18b519c0 9616@end deffn
bfa74976 9617
dd8d9022
AD
9618@deffn {Symbol} $undefined
9619The predefined token onto which all undefined values returned by
9620@code{yylex} are mapped. It cannot be used in the grammar, rather, use
9621@code{error}.
9622@end deffn
9623
18b519c0 9624@deffn {Directive} %union
bfa74976
RS
9625Bison declaration to specify several possible data types for semantic
9626values. @xref{Union Decl, ,The Collection of Value Types}.
18b519c0 9627@end deffn
bfa74976 9628
dd8d9022
AD
9629@deffn {Macro} YYABORT
9630Macro to pretend that an unrecoverable syntax error has occurred, by
9631making @code{yyparse} return 1 immediately. The error reporting
9632function @code{yyerror} is not called. @xref{Parser Function, ,The
9633Parser Function @code{yyparse}}.
8405b70c
PB
9634
9635For Java parsers, this functionality is invoked using @code{return YYABORT;}
9636instead.
dd8d9022 9637@end deffn
3ded9a63 9638
dd8d9022
AD
9639@deffn {Macro} YYACCEPT
9640Macro to pretend that a complete utterance of the language has been
9641read, by making @code{yyparse} return 0 immediately.
9642@xref{Parser Function, ,The Parser Function @code{yyparse}}.
8405b70c
PB
9643
9644For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
9645instead.
dd8d9022 9646@end deffn
bfa74976 9647
dd8d9022 9648@deffn {Macro} YYBACKUP
742e4900 9649Macro to discard a value from the parser stack and fake a lookahead
dd8d9022 9650token. @xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 9651@end deffn
bfa74976 9652
dd8d9022 9653@deffn {Variable} yychar
32c29292 9654External integer variable that contains the integer value of the
742e4900 9655lookahead token. (In a pure parser, it is a local variable within
dd8d9022
AD
9656@code{yyparse}.) Error-recovery rule actions may examine this variable.
9657@xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 9658@end deffn
bfa74976 9659
dd8d9022
AD
9660@deffn {Variable} yyclearin
9661Macro used in error-recovery rule actions. It clears the previous
742e4900 9662lookahead token. @xref{Error Recovery}.
18b519c0 9663@end deffn
bfa74976 9664
dd8d9022
AD
9665@deffn {Macro} YYDEBUG
9666Macro to define to equip the parser with tracing code. @xref{Tracing,
9667,Tracing Your Parser}.
18b519c0 9668@end deffn
bfa74976 9669
dd8d9022
AD
9670@deffn {Variable} yydebug
9671External integer variable set to zero by default. If @code{yydebug}
9672is given a nonzero value, the parser will output information on input
9673symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
18b519c0 9674@end deffn
bfa74976 9675
dd8d9022
AD
9676@deffn {Macro} yyerrok
9677Macro to cause parser to recover immediately to its normal mode
9678after a syntax error. @xref{Error Recovery}.
9679@end deffn
9680
9681@deffn {Macro} YYERROR
9682Macro to pretend that a syntax error has just been detected: call
9683@code{yyerror} and then perform normal error recovery if possible
9684(@pxref{Error Recovery}), or (if recovery is impossible) make
9685@code{yyparse} return 1. @xref{Error Recovery}.
8405b70c
PB
9686
9687For Java parsers, this functionality is invoked using @code{return YYERROR;}
9688instead.
dd8d9022
AD
9689@end deffn
9690
9691@deffn {Function} yyerror
9692User-supplied function to be called by @code{yyparse} on error.
9693@xref{Error Reporting, ,The Error
9694Reporting Function @code{yyerror}}.
9695@end deffn
9696
9697@deffn {Macro} YYERROR_VERBOSE
9698An obsolete macro that you define with @code{#define} in the prologue
9699to request verbose, specific error message strings
9700when @code{yyerror} is called. It doesn't matter what definition you
9701use for @code{YYERROR_VERBOSE}, just whether you define it. Using
9702@code{%error-verbose} is preferred.
9703@end deffn
9704
9705@deffn {Macro} YYINITDEPTH
9706Macro for specifying the initial size of the parser stack.
1a059451 9707@xref{Memory Management}.
dd8d9022
AD
9708@end deffn
9709
9710@deffn {Function} yylex
9711User-supplied lexical analyzer function, called with no arguments to get
9712the next token. @xref{Lexical, ,The Lexical Analyzer Function
9713@code{yylex}}.
9714@end deffn
9715
9716@deffn {Macro} YYLEX_PARAM
9717An obsolete macro for specifying an extra argument (or list of extra
32c29292 9718arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
dd8d9022
AD
9719macro is deprecated, and is supported only for Yacc like parsers.
9720@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
9721@end deffn
9722
9723@deffn {Variable} yylloc
9724External variable in which @code{yylex} should place the line and column
9725numbers associated with a token. (In a pure parser, it is a local
9726variable within @code{yyparse}, and its address is passed to
32c29292
JD
9727@code{yylex}.)
9728You can ignore this variable if you don't use the @samp{@@} feature in the
9729grammar actions.
9730@xref{Token Locations, ,Textual Locations of Tokens}.
742e4900 9731In semantic actions, it stores the location of the lookahead token.
32c29292 9732@xref{Actions and Locations, ,Actions and Locations}.
dd8d9022
AD
9733@end deffn
9734
9735@deffn {Type} YYLTYPE
9736Data type of @code{yylloc}; by default, a structure with four
9737members. @xref{Location Type, , Data Types of Locations}.
9738@end deffn
9739
9740@deffn {Variable} yylval
9741External variable in which @code{yylex} should place the semantic
9742value associated with a token. (In a pure parser, it is a local
9743variable within @code{yyparse}, and its address is passed to
32c29292
JD
9744@code{yylex}.)
9745@xref{Token Values, ,Semantic Values of Tokens}.
742e4900 9746In semantic actions, it stores the semantic value of the lookahead token.
32c29292 9747@xref{Actions, ,Actions}.
dd8d9022
AD
9748@end deffn
9749
9750@deffn {Macro} YYMAXDEPTH
1a059451
PE
9751Macro for specifying the maximum size of the parser stack. @xref{Memory
9752Management}.
dd8d9022
AD
9753@end deffn
9754
9755@deffn {Variable} yynerrs
8a2800e7 9756Global variable which Bison increments each time it reports a syntax error.
f4101aa6 9757(In a pure parser, it is a local variable within @code{yyparse}. In a
9987d1b3 9758pure push parser, it is a member of yypstate.)
dd8d9022
AD
9759@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
9760@end deffn
9761
9762@deffn {Function} yyparse
9763The parser function produced by Bison; call this function to start
9764parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
9765@end deffn
9766
9987d1b3 9767@deffn {Function} yypstate_delete
f4101aa6 9768The function to delete a parser instance, produced by Bison in push mode;
9987d1b3 9769call this function to delete the memory associated with a parser.
f4101aa6 9770@xref{Parser Delete Function, ,The Parser Delete Function
9987d1b3
JD
9771@code{yypstate_delete}}.
9772@end deffn
9773
9774@deffn {Function} yypstate_new
f4101aa6 9775The function to create a parser instance, produced by Bison in push mode;
9987d1b3 9776call this function to create a new parser.
f4101aa6 9777@xref{Parser Create Function, ,The Parser Create Function
9987d1b3
JD
9778@code{yypstate_new}}.
9779@end deffn
9780
9781@deffn {Function} yypull_parse
f4101aa6
AD
9782The parser function produced by Bison in push mode; call this function to
9783parse the rest of the input stream.
9784@xref{Pull Parser Function, ,The Pull Parser Function
9987d1b3
JD
9785@code{yypull_parse}}.
9786@end deffn
9787
9788@deffn {Function} yypush_parse
f4101aa6
AD
9789The parser function produced by Bison in push mode; call this function to
9790parse a single token. @xref{Push Parser Function, ,The Push Parser Function
9987d1b3
JD
9791@code{yypush_parse}}.
9792@end deffn
9793
dd8d9022
AD
9794@deffn {Macro} YYPARSE_PARAM
9795An obsolete macro for specifying the name of a parameter that
9796@code{yyparse} should accept. The use of this macro is deprecated, and
9797is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
9798Conventions for Pure Parsers}.
9799@end deffn
9800
9801@deffn {Macro} YYRECOVERING
02103984
PE
9802The expression @code{YYRECOVERING ()} yields 1 when the parser
9803is recovering from a syntax error, and 0 otherwise.
9804@xref{Action Features, ,Special Features for Use in Actions}.
dd8d9022
AD
9805@end deffn
9806
9807@deffn {Macro} YYSTACK_USE_ALLOCA
d7e14fc0
PE
9808Macro used to control the use of @code{alloca} when the C
9809@acronym{LALR}(1) parser needs to extend its stacks. If defined to 0,
9810the parser will use @code{malloc} to extend its stacks. If defined to
98111, the parser will use @code{alloca}. Values other than 0 and 1 are
9812reserved for future Bison extensions. If not defined,
9813@code{YYSTACK_USE_ALLOCA} defaults to 0.
9814
55289366 9815In the all-too-common case where your code may run on a host with a
d7e14fc0
PE
9816limited stack and with unreliable stack-overflow checking, you should
9817set @code{YYMAXDEPTH} to a value that cannot possibly result in
9818unchecked stack overflow on any of your target hosts when
9819@code{alloca} is called. You can inspect the code that Bison
9820generates in order to determine the proper numeric values. This will
9821require some expertise in low-level implementation details.
dd8d9022
AD
9822@end deffn
9823
9824@deffn {Type} YYSTYPE
9825Data type of semantic values; @code{int} by default.
9826@xref{Value Type, ,Data Types of Semantic Values}.
18b519c0 9827@end deffn
bfa74976 9828
342b8b6e 9829@node Glossary
bfa74976
RS
9830@appendix Glossary
9831@cindex glossary
9832
9833@table @asis
c827f760
PE
9834@item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'')
9835Formal method of specifying context-free grammars originally proposed
9836by John Backus, and slightly improved by Peter Naur in his 1960-01-02
9837committee document contributing to what became the Algol 60 report.
9838@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
9839
9840@item Context-free grammars
9841Grammars specified as rules that can be applied regardless of context.
9842Thus, if there is a rule which says that an integer can be used as an
9843expression, integers are allowed @emph{anywhere} an expression is
89cab50d
AD
9844permitted. @xref{Language and Grammar, ,Languages and Context-Free
9845Grammars}.
bfa74976
RS
9846
9847@item Dynamic allocation
9848Allocation of memory that occurs during execution, rather than at
9849compile time or on entry to a function.
9850
9851@item Empty string
9852Analogous to the empty set in set theory, the empty string is a
9853character string of length zero.
9854
9855@item Finite-state stack machine
9856A ``machine'' that has discrete states in which it is said to exist at
9857each instant in time. As input to the machine is processed, the
9858machine moves from state to state as specified by the logic of the
9859machine. In the case of the parser, the input is the language being
9860parsed, and the states correspond to various stages in the grammar
c827f760 9861rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976 9862
c827f760 9863@item Generalized @acronym{LR} (@acronym{GLR})
676385e2 9864A parsing algorithm that can handle all context-free grammars, including those
c827f760
PE
9865that are not @acronym{LALR}(1). It resolves situations that Bison's
9866usual @acronym{LALR}(1)
676385e2
PH
9867algorithm cannot by effectively splitting off multiple parsers, trying all
9868possible parsers, and discarding those that fail in the light of additional
c827f760
PE
9869right context. @xref{Generalized LR Parsing, ,Generalized
9870@acronym{LR} Parsing}.
676385e2 9871
bfa74976
RS
9872@item Grouping
9873A language construct that is (in general) grammatically divisible;
c827f760 9874for example, `expression' or `declaration' in C@.
bfa74976
RS
9875@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
9876
9877@item Infix operator
9878An arithmetic operator that is placed between the operands on which it
9879performs some operation.
9880
9881@item Input stream
9882A continuous flow of data between devices or programs.
9883
9884@item Language construct
9885One of the typical usage schemas of the language. For example, one of
9886the constructs of the C language is the @code{if} statement.
9887@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
9888
9889@item Left associativity
9890Operators having left associativity are analyzed from left to right:
9891@samp{a+b+c} first computes @samp{a+b} and then combines with
9892@samp{c}. @xref{Precedence, ,Operator Precedence}.
9893
9894@item Left recursion
89cab50d
AD
9895A rule whose result symbol is also its first component symbol; for
9896example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
9897Rules}.
bfa74976
RS
9898
9899@item Left-to-right parsing
9900Parsing a sentence of a language by analyzing it token by token from
c827f760 9901left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
9902
9903@item Lexical analyzer (scanner)
9904A function that reads an input stream and returns tokens one by one.
9905@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
9906
9907@item Lexical tie-in
9908A flag, set by actions in the grammar rules, which alters the way
9909tokens are parsed. @xref{Lexical Tie-ins}.
9910
931c7513 9911@item Literal string token
14ded682 9912A token which consists of two or more fixed characters. @xref{Symbols}.
931c7513 9913
742e4900
JD
9914@item Lookahead token
9915A token already read but not yet shifted. @xref{Lookahead, ,Lookahead
89cab50d 9916Tokens}.
bfa74976 9917
c827f760 9918@item @acronym{LALR}(1)
bfa74976 9919The class of context-free grammars that Bison (like most other parser
c827f760
PE
9920generators) can handle; a subset of @acronym{LR}(1). @xref{Mystery
9921Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
bfa74976 9922
c827f760 9923@item @acronym{LR}(1)
bfa74976 9924The class of context-free grammars in which at most one token of
742e4900 9925lookahead is needed to disambiguate the parsing of any piece of input.
bfa74976
RS
9926
9927@item Nonterminal symbol
9928A grammar symbol standing for a grammatical construct that can
9929be expressed through rules in terms of smaller constructs; in other
9930words, a construct that is not a token. @xref{Symbols}.
9931
bfa74976
RS
9932@item Parser
9933A function that recognizes valid sentences of a language by analyzing
9934the syntax structure of a set of tokens passed to it from a lexical
9935analyzer.
9936
9937@item Postfix operator
9938An arithmetic operator that is placed after the operands upon which it
9939performs some operation.
9940
9941@item Reduction
9942Replacing a string of nonterminals and/or terminals with a single
89cab50d 9943nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
c827f760 9944Parser Algorithm}.
bfa74976
RS
9945
9946@item Reentrant
9947A reentrant subprogram is a subprogram which can be in invoked any
9948number of times in parallel, without interference between the various
9949invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
9950
9951@item Reverse polish notation
9952A language in which all operators are postfix operators.
9953
9954@item Right recursion
89cab50d
AD
9955A rule whose result symbol is also its last component symbol; for
9956example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
9957Rules}.
bfa74976
RS
9958
9959@item Semantics
9960In computer languages, the semantics are specified by the actions
9961taken for each instance of the language, i.e., the meaning of
9962each statement. @xref{Semantics, ,Defining Language Semantics}.
9963
9964@item Shift
9965A parser is said to shift when it makes the choice of analyzing
9966further input from the stream rather than reducing immediately some
c827f760 9967already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
9968
9969@item Single-character literal
9970A single character that is recognized and interpreted as is.
9971@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
9972
9973@item Start symbol
9974The nonterminal symbol that stands for a complete valid utterance in
9975the language being parsed. The start symbol is usually listed as the
13863333 9976first nonterminal symbol in a language specification.
bfa74976
RS
9977@xref{Start Decl, ,The Start-Symbol}.
9978
9979@item Symbol table
9980A data structure where symbol names and associated data are stored
9981during parsing to allow for recognition and use of existing
9982information in repeated uses of a symbol. @xref{Multi-function Calc}.
9983
6e649e65
PE
9984@item Syntax error
9985An error encountered during parsing of an input stream due to invalid
9986syntax. @xref{Error Recovery}.
9987
bfa74976
RS
9988@item Token
9989A basic, grammatically indivisible unit of a language. The symbol
9990that describes a token in the grammar is a terminal symbol.
9991The input of the Bison parser is a stream of tokens which comes from
9992the lexical analyzer. @xref{Symbols}.
9993
9994@item Terminal symbol
89cab50d
AD
9995A grammar symbol that has no rules in the grammar and therefore is
9996grammatically indivisible. The piece of text it represents is a token.
9997@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
9998@end table
9999
342b8b6e 10000@node Copying This Manual
f2b5126e 10001@appendix Copying This Manual
f2b5126e
PB
10002@include fdl.texi
10003
342b8b6e 10004@node Index
bfa74976
RS
10005@unnumbered Index
10006
10007@printindex cp
10008
bfa74976 10009@bye
a06ea4aa
AD
10010
10011@c LocalWords: texinfo setfilename settitle setchapternewpage finalout
10012@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex
10013@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry
10014@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa
10015@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc
10016@c LocalWords: rpcalc Lexer Gen Comp Expr ltcalc mfcalc Decl Symtab yylex
10017@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref
10018@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex
10019@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge
10020@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG
10021@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit
10022@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok
178e123e 10023@c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln
a06ea4aa
AD
10024@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym
10025@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof
10026@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum
10027@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype
178e123e 10028@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs
a06ea4aa
AD
10029@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES
10030@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param
10031@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP
32c29292 10032@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword
a06ea4aa 10033@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH
35fe0834 10034@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype
a06ea4aa 10035@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args
35fe0834 10036@c LocalWords: infile ypp yxx outfile itemx tex leaderfill
a06ea4aa 10037@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll
178e123e 10038@c LocalWords: nbar yytext fst snd osplit ntwo strdup AST
a06ea4aa 10039@c LocalWords: YYSTACK DVI fdl printindex