]> git.saurik.com Git - bison.git/blame - doc/bison.texinfo
* data/c.m4 (b4_location_initial_column, b4_location_initial_line):
[bison.git] / doc / bison.texinfo
CommitLineData
bfa74976
RS
1\input texinfo @c -*-texinfo-*-
2@comment %**start of header
3@setfilename bison.info
df1af54c
JT
4@include version.texi
5@settitle Bison @value{VERSION}
bfa74976
RS
6@setchapternewpage odd
7
5378c3e7 8@finalout
5378c3e7 9
13863333 10@c SMALL BOOK version
bfa74976 11@c This edition has been formatted so that you can format and print it in
13863333 12@c the smallbook format.
bfa74976
RS
13@c @smallbook
14
91d2c560
PE
15@c Set following if you want to document %default-prec and %no-default-prec.
16@c This feature is experimental and may change in future Bison versions.
17@c @set defaultprec
18
8c5b881d 19@ifnotinfo
bfa74976
RS
20@syncodeindex fn cp
21@syncodeindex vr cp
22@syncodeindex tp cp
8c5b881d 23@end ifnotinfo
bfa74976
RS
24@ifinfo
25@synindex fn cp
26@synindex vr cp
27@synindex tp cp
28@end ifinfo
29@comment %**end of header
30
fae437e8 31@copying
bd773d73 32
c827f760
PE
33This manual is for @acronym{GNU} Bison (version @value{VERSION},
34@value{UPDATED}), the @acronym{GNU} parser generator.
fae437e8 35
a06ea4aa 36Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
32c29292 371999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation, Inc.
fae437e8
AD
38
39@quotation
40Permission is granted to copy, distribute and/or modify this document
c827f760 41under the terms of the @acronym{GNU} Free Documentation License,
592fde95 42Version 1.2 or any later version published by the Free Software
c827f760
PE
43Foundation; with no Invariant Sections, with the Front-Cover texts
44being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in
45(a) below. A copy of the license is included in the section entitled
46``@acronym{GNU} Free Documentation License.''
47
48(a) The @acronym{FSF}'s Back-Cover Text is: ``You have freedom to copy
49and modify this @acronym{GNU} Manual, like @acronym{GNU} software.
50Copies published by the Free Software Foundation raise funds for
51@acronym{GNU} development.''
fae437e8
AD
52@end quotation
53@end copying
54
e62f1a89 55@dircategory Software development
fae437e8 56@direntry
c827f760 57* bison: (bison). @acronym{GNU} parser generator (Yacc replacement).
fae437e8 58@end direntry
bfa74976 59
bfa74976
RS
60@titlepage
61@title Bison
c827f760 62@subtitle The Yacc-compatible Parser Generator
df1af54c 63@subtitle @value{UPDATED}, Bison Version @value{VERSION}
bfa74976
RS
64
65@author by Charles Donnelly and Richard Stallman
66
67@page
68@vskip 0pt plus 1filll
fae437e8 69@insertcopying
bfa74976
RS
70@sp 2
71Published by the Free Software Foundation @*
0fb669f9
PE
7251 Franklin Street, Fifth Floor @*
73Boston, MA 02110-1301 USA @*
9ecbd125 74Printed copies are available from the Free Software Foundation.@*
c827f760 75@acronym{ISBN} 1-882114-44-2
bfa74976
RS
76@sp 2
77Cover art by Etienne Suvasa.
78@end titlepage
d5796688
JT
79
80@contents
bfa74976 81
342b8b6e
AD
82@ifnottex
83@node Top
84@top Bison
fae437e8 85@insertcopying
342b8b6e 86@end ifnottex
bfa74976
RS
87
88@menu
13863333
AD
89* Introduction::
90* Conditions::
c827f760 91* Copying:: The @acronym{GNU} General Public License says
bfa74976
RS
92 how you can copy and share Bison
93
94Tutorial sections:
95* Concepts:: Basic concepts for understanding Bison.
96* Examples:: Three simple explained examples of using Bison.
97
98Reference sections:
99* Grammar File:: Writing Bison declarations and rules.
100* Interface:: C-language interface to the parser function @code{yyparse}.
101* Algorithm:: How the Bison parser works at run-time.
102* Error Recovery:: Writing rules for error recovery.
103* Context Dependency:: What to do if your language syntax is too
104 messy for Bison to handle straightforwardly.
ec3bc396 105* Debugging:: Understanding or debugging Bison parsers.
bfa74976 106* Invocation:: How to run Bison (to produce the parser source file).
12545799
AD
107* C++ Language Interface:: Creating C++ parser objects.
108* FAQ:: Frequently Asked Questions
bfa74976
RS
109* Table of Symbols:: All the keywords of the Bison language are explained.
110* Glossary:: Basic concepts are explained.
f2b5126e 111* Copying This Manual:: License for copying this manual.
bfa74976
RS
112* Index:: Cross-references to the text.
113
93dd49ab
PE
114@detailmenu
115 --- The Detailed Node Listing ---
bfa74976
RS
116
117The Concepts of Bison
118
119* Language and Grammar:: Languages and context-free grammars,
120 as mathematical ideas.
121* Grammar in Bison:: How we represent grammars for Bison's sake.
122* Semantic Values:: Each token or syntactic grouping can have
123 a semantic value (the value of an integer,
124 the name of an identifier, etc.).
125* Semantic Actions:: Each rule can have an action containing C code.
99a9344e 126* GLR Parsers:: Writing parsers for general context-free languages.
93dd49ab 127* Locations Overview:: Tracking Locations.
bfa74976
RS
128* Bison Parser:: What are Bison's input and output,
129 how is the output used?
130* Stages:: Stages in writing and running Bison grammars.
131* Grammar Layout:: Overall structure of a Bison grammar file.
132
fa7e68c3
PE
133Writing @acronym{GLR} Parsers
134
32c29292
JD
135* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
136* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
137* GLR Semantic Actions:: Deferred semantic actions have special concerns.
138* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
fa7e68c3 139
bfa74976
RS
140Examples
141
142* RPN Calc:: Reverse polish notation calculator;
143 a first example with no operator precedence.
144* Infix Calc:: Infix (algebraic) notation calculator.
145 Operator precedence is introduced.
146* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 147* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
93dd49ab
PE
148* Multi-function Calc:: Calculator with memory and trig functions.
149 It uses multiple data-types for semantic values.
bfa74976
RS
150* Exercises:: Ideas for improving the multi-function calculator.
151
152Reverse Polish Notation Calculator
153
75f5aaea 154* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
bfa74976
RS
155* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
156* Lexer: Rpcalc Lexer. The lexical analyzer.
157* Main: Rpcalc Main. The controlling function.
158* Error: Rpcalc Error. The error reporting function.
159* Gen: Rpcalc Gen. Running Bison on the grammar file.
160* Comp: Rpcalc Compile. Run the C compiler on the output code.
161
162Grammar Rules for @code{rpcalc}
163
13863333
AD
164* Rpcalc Input::
165* Rpcalc Line::
166* Rpcalc Expr::
bfa74976 167
342b8b6e
AD
168Location Tracking Calculator: @code{ltcalc}
169
170* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
171* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
172* Lexer: Ltcalc Lexer. The lexical analyzer.
173
bfa74976
RS
174Multi-Function Calculator: @code{mfcalc}
175
176* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
177* Rules: Mfcalc Rules. Grammar rules for the calculator.
178* Symtab: Mfcalc Symtab. Symbol table management subroutines.
179
180Bison Grammar Files
181
182* Grammar Outline:: Overall layout of the grammar file.
183* Symbols:: Terminal and nonterminal symbols.
184* Rules:: How to write grammar rules.
185* Recursion:: Writing recursive rules.
186* Semantics:: Semantic values and actions.
93dd49ab 187* Locations:: Locations and actions.
bfa74976
RS
188* Declarations:: All kinds of Bison declarations are described here.
189* Multiple Parsers:: Putting more than one Bison parser in one program.
190
191Outline of a Bison Grammar
192
93dd49ab 193* Prologue:: Syntax and usage of the prologue.
bfa74976
RS
194* Bison Declarations:: Syntax and usage of the Bison declarations section.
195* Grammar Rules:: Syntax and usage of the grammar rules section.
93dd49ab 196* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
197
198Defining Language Semantics
199
200* Value Type:: Specifying one data type for all semantic values.
201* Multiple Types:: Specifying several alternative data types.
202* Actions:: An action is the semantic definition of a grammar rule.
203* Action Types:: Specifying data types for actions to operate on.
204* Mid-Rule Actions:: Most actions go at the end of a rule.
205 This says when, why and how to use the exceptional
206 action in the middle of a rule.
207
93dd49ab
PE
208Tracking Locations
209
210* Location Type:: Specifying a data type for locations.
211* Actions and Locations:: Using locations in actions.
212* Location Default Action:: Defining a general way to compute locations.
213
bfa74976
RS
214Bison Declarations
215
b50d2359 216* Require Decl:: Requiring a Bison version.
bfa74976
RS
217* Token Decl:: Declaring terminal symbols.
218* Precedence Decl:: Declaring terminals with precedence and associativity.
219* Union Decl:: Declaring the set of all semantic value types.
220* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 221* Initial Action Decl:: Code run before parsing starts.
72f889cc 222* Destructor Decl:: Declaring how symbols are freed.
d6328241 223* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
224* Start Decl:: Specifying the start symbol.
225* Pure Decl:: Requesting a reentrant parser.
226* Decl Summary:: Table of all Bison declarations.
227
228Parser C-Language Interface
229
230* Parser Function:: How to call @code{yyparse} and what it returns.
13863333 231* Lexical:: You must supply a function @code{yylex}
bfa74976
RS
232 which reads tokens.
233* Error Reporting:: You must supply a function @code{yyerror}.
234* Action Features:: Special features for use in actions.
f7ab6a50
PE
235* Internationalization:: How to let the parser speak in the user's
236 native language.
bfa74976
RS
237
238The Lexical Analyzer Function @code{yylex}
239
240* Calling Convention:: How @code{yyparse} calls @code{yylex}.
241* Token Values:: How @code{yylex} must return the semantic value
242 of the token it has read.
95923bd6 243* Token Locations:: How @code{yylex} must return the text location
bfa74976 244 (line number, etc.) of the token, if the
93dd49ab 245 actions want that.
bfa74976
RS
246* Pure Calling:: How the calling convention differs
247 in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
248
13863333 249The Bison Parser Algorithm
bfa74976 250
742e4900 251* Lookahead:: Parser looks one token ahead when deciding what to do.
bfa74976
RS
252* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
253* Precedence:: Operator precedence works by resolving conflicts.
254* Contextual Precedence:: When an operator's precedence depends on context.
255* Parser States:: The parser is a finite-state-machine with stack.
256* Reduce/Reduce:: When two rules are applicable in the same situation.
257* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 258* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 259* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
260
261Operator Precedence
262
263* Why Precedence:: An example showing why precedence is needed.
264* Using Precedence:: How to specify precedence in Bison grammars.
265* Precedence Examples:: How these features are used in the previous example.
266* How Precedence:: How they work.
267
268Handling Context Dependencies
269
270* Semantic Tokens:: Token parsing can depend on the semantic context.
271* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
272* Tie-in Recovery:: Lexical tie-ins have implications for how
273 error recovery rules must be written.
274
93dd49ab 275Debugging Your Parser
ec3bc396
AD
276
277* Understanding:: Understanding the structure of your parser.
278* Tracing:: Tracing the execution of your parser.
279
bfa74976
RS
280Invoking Bison
281
13863333 282* Bison Options:: All the options described in detail,
c827f760 283 in alphabetical order by short options.
bfa74976 284* Option Cross Key:: Alphabetical list of long options.
93dd49ab 285* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
f2b5126e 286
12545799
AD
287C++ Language Interface
288
289* C++ Parsers:: The interface to generate C++ parser classes
290* A Complete C++ Example:: Demonstrating their use
291
292C++ Parsers
293
294* C++ Bison Interface:: Asking for C++ parser generation
295* C++ Semantic Values:: %union vs. C++
296* C++ Location Values:: The position and location classes
297* C++ Parser Interface:: Instantiating and running the parser
298* C++ Scanner Interface:: Exchanges between yylex and parse
299
300A Complete C++ Example
301
302* Calc++ --- C++ Calculator:: The specifications
303* Calc++ Parsing Driver:: An active parsing context
304* Calc++ Parser:: A parser class
305* Calc++ Scanner:: A pure C++ Flex scanner
306* Calc++ Top Level:: Conducting the band
307
d1a1114f
AD
308Frequently Asked Questions
309
1a059451 310* Memory Exhausted:: Breaking the Stack Limits
e64fec0a 311* How Can I Reset the Parser:: @code{yyparse} Keeps some State
fef4cb51 312* Strings are Destroyed:: @code{yylval} Loses Track of Strings
2fa09258 313* Implementing Gotos/Loops:: Control Flow in the Calculator
ed2e6384 314* Multiple start-symbols:: Factoring closely related grammars
55ba27be
AD
315* Secure? Conform?:: Is Bison @acronym{POSIX} safe?
316* I can't build Bison:: Troubleshooting
317* Where can I find help?:: Troubleshouting
318* Bug Reports:: Troublereporting
319* Other Languages:: Parsers in Java and others
320* Beta Testing:: Experimenting development versions
321* Mailing Lists:: Meeting other Bison users
d1a1114f 322
f2b5126e
PB
323Copying This Manual
324
325* GNU Free Documentation License:: License for copying this manual.
326
342b8b6e 327@end detailmenu
bfa74976
RS
328@end menu
329
342b8b6e 330@node Introduction
bfa74976
RS
331@unnumbered Introduction
332@cindex introduction
333
6077da58
PE
334@dfn{Bison} is a general-purpose parser generator that converts an
335annotated context-free grammar into an @acronym{LALR}(1) or
336@acronym{GLR} parser for that grammar. Once you are proficient with
1e137b71 337Bison, you can use it to develop a wide range of language parsers, from those
bfa74976
RS
338used in simple desk calculators to complex programming languages.
339
340Bison is upward compatible with Yacc: all properly-written Yacc grammars
341ought to work with Bison with no change. Anyone familiar with Yacc
342should be able to use Bison with little trouble. You need to be fluent in
1e137b71 343C or C++ programming in order to use Bison or to understand this manual.
bfa74976
RS
344
345We begin with tutorial chapters that explain the basic concepts of using
346Bison and show three explained examples, each building on the last. If you
347don't know Bison or Yacc, start by reading these chapters. Reference
348chapters follow which describe specific aspects of Bison in detail.
349
931c7513
RS
350Bison was written primarily by Robert Corbett; Richard Stallman made it
351Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
14ded682 352multi-character string literals and other features.
931c7513 353
df1af54c 354This edition corresponds to version @value{VERSION} of Bison.
bfa74976 355
342b8b6e 356@node Conditions
bfa74976
RS
357@unnumbered Conditions for Using Bison
358
193d7c70
PE
359The distribution terms for Bison-generated parsers permit using the
360parsers in nonfree programs. Before Bison version 2.2, these extra
361permissions applied only when Bison was generating @acronym{LALR}(1)
362parsers in C@. And before Bison version 1.24, Bison-generated
262aa8dd 363parsers could be used only in programs that were free software.
a31239f1 364
c827f760
PE
365The other @acronym{GNU} programming tools, such as the @acronym{GNU} C
366compiler, have never
9ecbd125 367had such a requirement. They could always be used for nonfree
a31239f1
RS
368software. The reason Bison was different was not due to a special
369policy decision; it resulted from applying the usual General Public
370License to all of the Bison source code.
371
372The output of the Bison utility---the Bison parser file---contains a
373verbatim copy of a sizable piece of Bison, which is the code for the
193d7c70
PE
374parser's implementation. (The actions from your grammar are inserted
375into this implementation at one point, but most of the rest of the
376implementation is not changed.) When we applied the @acronym{GPL}
377terms to the skeleton code for the parser's implementation,
a31239f1
RS
378the effect was to restrict the use of Bison output to free software.
379
380We didn't change the terms because of sympathy for people who want to
381make software proprietary. @strong{Software should be free.} But we
382concluded that limiting Bison's use to free software was doing little to
383encourage people to make other software free. So we decided to make the
384practical conditions for using Bison match the practical conditions for
c827f760 385using the other @acronym{GNU} tools.
bfa74976 386
193d7c70
PE
387This exception applies when Bison is generating code for a parser.
388You can tell whether the exception applies to a Bison output file by
389inspecting the file for text beginning with ``As a special
390exception@dots{}''. The text spells out the exact terms of the
391exception.
262aa8dd 392
c67a198d 393@include gpl.texi
bfa74976 394
342b8b6e 395@node Concepts
bfa74976
RS
396@chapter The Concepts of Bison
397
398This chapter introduces many of the basic concepts without which the
399details of Bison will not make sense. If you do not already know how to
400use Bison or Yacc, we suggest you start by reading this chapter carefully.
401
402@menu
403* Language and Grammar:: Languages and context-free grammars,
404 as mathematical ideas.
405* Grammar in Bison:: How we represent grammars for Bison's sake.
406* Semantic Values:: Each token or syntactic grouping can have
407 a semantic value (the value of an integer,
408 the name of an identifier, etc.).
409* Semantic Actions:: Each rule can have an action containing C code.
99a9344e 410* GLR Parsers:: Writing parsers for general context-free languages.
847bf1f5 411* Locations Overview:: Tracking Locations.
bfa74976
RS
412* Bison Parser:: What are Bison's input and output,
413 how is the output used?
414* Stages:: Stages in writing and running Bison grammars.
415* Grammar Layout:: Overall structure of a Bison grammar file.
416@end menu
417
342b8b6e 418@node Language and Grammar
bfa74976
RS
419@section Languages and Context-Free Grammars
420
bfa74976
RS
421@cindex context-free grammar
422@cindex grammar, context-free
423In order for Bison to parse a language, it must be described by a
424@dfn{context-free grammar}. This means that you specify one or more
425@dfn{syntactic groupings} and give rules for constructing them from their
426parts. For example, in the C language, one kind of grouping is called an
427`expression'. One rule for making an expression might be, ``An expression
428can be made of a minus sign and another expression''. Another would be,
429``An expression can be an integer''. As you can see, rules are often
430recursive, but there must be at least one rule which leads out of the
431recursion.
432
c827f760 433@cindex @acronym{BNF}
bfa74976
RS
434@cindex Backus-Naur form
435The most common formal system for presenting such rules for humans to read
c827f760
PE
436is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in
437order to specify the language Algol 60. Any grammar expressed in
438@acronym{BNF} is a context-free grammar. The input to Bison is
439essentially machine-readable @acronym{BNF}.
bfa74976 440
c827f760
PE
441@cindex @acronym{LALR}(1) grammars
442@cindex @acronym{LR}(1) grammars
676385e2
PH
443There are various important subclasses of context-free grammar. Although it
444can handle almost all context-free grammars, Bison is optimized for what
c827f760 445are called @acronym{LALR}(1) grammars.
676385e2 446In brief, in these grammars, it must be possible to
bfa74976 447tell how to parse any portion of an input string with just a single
742e4900 448token of lookahead. Strictly speaking, that is a description of an
c827f760
PE
449@acronym{LR}(1) grammar, and @acronym{LALR}(1) involves additional
450restrictions that are
bfa74976 451hard to explain simply; but it is rare in actual practice to find an
c827f760
PE
452@acronym{LR}(1) grammar that fails to be @acronym{LALR}(1).
453@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
454more information on this.
bfa74976 455
c827f760
PE
456@cindex @acronym{GLR} parsing
457@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2 458@cindex ambiguous grammars
9d9b8b70 459@cindex nondeterministic parsing
9501dc6e
AD
460
461Parsers for @acronym{LALR}(1) grammars are @dfn{deterministic}, meaning
462roughly that the next grammar rule to apply at any point in the input is
463uniquely determined by the preceding input and a fixed, finite portion
742e4900 464(called a @dfn{lookahead}) of the remaining input. A context-free
9501dc6e 465grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
e4f85c39 466apply the grammar rules to get the same inputs. Even unambiguous
9d9b8b70 467grammars can be @dfn{nondeterministic}, meaning that no fixed
742e4900 468lookahead always suffices to determine the next grammar rule to apply.
9501dc6e
AD
469With the proper declarations, Bison is also able to parse these more
470general context-free grammars, using a technique known as @acronym{GLR}
471parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers
472are able to handle any context-free grammar for which the number of
473possible parses of any given string is finite.
676385e2 474
bfa74976
RS
475@cindex symbols (abstract)
476@cindex token
477@cindex syntactic grouping
478@cindex grouping, syntactic
9501dc6e
AD
479In the formal grammatical rules for a language, each kind of syntactic
480unit or grouping is named by a @dfn{symbol}. Those which are built by
481grouping smaller constructs according to grammatical rules are called
bfa74976
RS
482@dfn{nonterminal symbols}; those which can't be subdivided are called
483@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
484corresponding to a single terminal symbol a @dfn{token}, and a piece
e0c471a9 485corresponding to a single nonterminal symbol a @dfn{grouping}.
bfa74976
RS
486
487We can use the C language as an example of what symbols, terminal and
9501dc6e
AD
488nonterminal, mean. The tokens of C are identifiers, constants (numeric
489and string), and the various keywords, arithmetic operators and
490punctuation marks. So the terminal symbols of a grammar for C include
491`identifier', `number', `string', plus one symbol for each keyword,
492operator or punctuation mark: `if', `return', `const', `static', `int',
493`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
494(These tokens can be subdivided into characters, but that is a matter of
bfa74976
RS
495lexicography, not grammar.)
496
497Here is a simple C function subdivided into tokens:
498
9edcd895
AD
499@ifinfo
500@example
501int /* @r{keyword `int'} */
14d4662b 502square (int x) /* @r{identifier, open-paren, keyword `int',}
9edcd895
AD
503 @r{identifier, close-paren} */
504@{ /* @r{open-brace} */
aa08666d
AD
505 return x * x; /* @r{keyword `return', identifier, asterisk,}
506 @r{identifier, semicolon} */
9edcd895
AD
507@} /* @r{close-brace} */
508@end example
509@end ifinfo
510@ifnotinfo
bfa74976
RS
511@example
512int /* @r{keyword `int'} */
14d4662b 513square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
bfa74976 514@{ /* @r{open-brace} */
9edcd895 515 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
bfa74976
RS
516@} /* @r{close-brace} */
517@end example
9edcd895 518@end ifnotinfo
bfa74976
RS
519
520The syntactic groupings of C include the expression, the statement, the
521declaration, and the function definition. These are represented in the
522grammar of C by nonterminal symbols `expression', `statement',
523`declaration' and `function definition'. The full grammar uses dozens of
524additional language constructs, each with its own nonterminal symbol, in
525order to express the meanings of these four. The example above is a
526function definition; it contains one declaration, and one statement. In
527the statement, each @samp{x} is an expression and so is @samp{x * x}.
528
529Each nonterminal symbol must have grammatical rules showing how it is made
530out of simpler constructs. For example, one kind of C statement is the
531@code{return} statement; this would be described with a grammar rule which
532reads informally as follows:
533
534@quotation
535A `statement' can be made of a `return' keyword, an `expression' and a
536`semicolon'.
537@end quotation
538
539@noindent
540There would be many other rules for `statement', one for each kind of
541statement in C.
542
543@cindex start symbol
544One nonterminal symbol must be distinguished as the special one which
545defines a complete utterance in the language. It is called the @dfn{start
546symbol}. In a compiler, this means a complete input program. In the C
547language, the nonterminal symbol `sequence of definitions and declarations'
548plays this role.
549
550For example, @samp{1 + 2} is a valid C expression---a valid part of a C
551program---but it is not valid as an @emph{entire} C program. In the
552context-free grammar of C, this follows from the fact that `expression' is
553not the start symbol.
554
555The Bison parser reads a sequence of tokens as its input, and groups the
556tokens using the grammar rules. If the input is valid, the end result is
557that the entire token sequence reduces to a single grouping whose symbol is
558the grammar's start symbol. If we use a grammar for C, the entire input
559must be a `sequence of definitions and declarations'. If not, the parser
560reports a syntax error.
561
342b8b6e 562@node Grammar in Bison
bfa74976
RS
563@section From Formal Rules to Bison Input
564@cindex Bison grammar
565@cindex grammar, Bison
566@cindex formal grammar
567
568A formal grammar is a mathematical construct. To define the language
569for Bison, you must write a file expressing the grammar in Bison syntax:
570a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
571
572A nonterminal symbol in the formal grammar is represented in Bison input
c827f760 573as an identifier, like an identifier in C@. By convention, it should be
bfa74976
RS
574in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
575
576The Bison representation for a terminal symbol is also called a @dfn{token
577type}. Token types as well can be represented as C-like identifiers. By
578convention, these identifiers should be upper case to distinguish them from
579nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
580@code{RETURN}. A terminal symbol that stands for a particular keyword in
581the language should be named after that keyword converted to upper case.
582The terminal symbol @code{error} is reserved for error recovery.
931c7513 583@xref{Symbols}.
bfa74976
RS
584
585A terminal symbol can also be represented as a character literal, just like
586a C character constant. You should do this whenever a token is just a
587single character (parenthesis, plus-sign, etc.): use that same character in
588a literal as the terminal symbol for that token.
589
931c7513
RS
590A third way to represent a terminal symbol is with a C string constant
591containing several characters. @xref{Symbols}, for more information.
592
bfa74976
RS
593The grammar rules also have an expression in Bison syntax. For example,
594here is the Bison rule for a C @code{return} statement. The semicolon in
595quotes is a literal character token, representing part of the C syntax for
596the statement; the naked semicolon, and the colon, are Bison punctuation
597used in every rule.
598
599@example
600stmt: RETURN expr ';'
601 ;
602@end example
603
604@noindent
605@xref{Rules, ,Syntax of Grammar Rules}.
606
342b8b6e 607@node Semantic Values
bfa74976
RS
608@section Semantic Values
609@cindex semantic value
610@cindex value, semantic
611
612A formal grammar selects tokens only by their classifications: for example,
613if a rule mentions the terminal symbol `integer constant', it means that
614@emph{any} integer constant is grammatically valid in that position. The
615precise value of the constant is irrelevant to how to parse the input: if
616@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
e0c471a9 617grammatical.
bfa74976
RS
618
619But the precise value is very important for what the input means once it is
620parsed. A compiler is useless if it fails to distinguish between 4, 1 and
6213989 as constants in the program! Therefore, each token in a Bison grammar
c827f760
PE
622has both a token type and a @dfn{semantic value}. @xref{Semantics,
623,Defining Language Semantics},
bfa74976
RS
624for details.
625
626The token type is a terminal symbol defined in the grammar, such as
627@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
628you need to know to decide where the token may validly appear and how to
629group it with other tokens. The grammar rules know nothing about tokens
e0c471a9 630except their types.
bfa74976
RS
631
632The semantic value has all the rest of the information about the
633meaning of the token, such as the value of an integer, or the name of an
634identifier. (A token such as @code{','} which is just punctuation doesn't
635need to have any semantic value.)
636
637For example, an input token might be classified as token type
638@code{INTEGER} and have the semantic value 4. Another input token might
639have the same token type @code{INTEGER} but value 3989. When a grammar
640rule says that @code{INTEGER} is allowed, either of these tokens is
641acceptable because each is an @code{INTEGER}. When the parser accepts the
642token, it keeps track of the token's semantic value.
643
644Each grouping can also have a semantic value as well as its nonterminal
645symbol. For example, in a calculator, an expression typically has a
646semantic value that is a number. In a compiler for a programming
647language, an expression typically has a semantic value that is a tree
648structure describing the meaning of the expression.
649
342b8b6e 650@node Semantic Actions
bfa74976
RS
651@section Semantic Actions
652@cindex semantic actions
653@cindex actions, semantic
654
655In order to be useful, a program must do more than parse input; it must
656also produce some output based on the input. In a Bison grammar, a grammar
657rule can have an @dfn{action} made up of C statements. Each time the
658parser recognizes a match for that rule, the action is executed.
659@xref{Actions}.
13863333 660
bfa74976
RS
661Most of the time, the purpose of an action is to compute the semantic value
662of the whole construct from the semantic values of its parts. For example,
663suppose we have a rule which says an expression can be the sum of two
664expressions. When the parser recognizes such a sum, each of the
665subexpressions has a semantic value which describes how it was built up.
666The action for this rule should create a similar sort of value for the
667newly recognized larger expression.
668
669For example, here is a rule that says an expression can be the sum of
670two subexpressions:
671
672@example
673expr: expr '+' expr @{ $$ = $1 + $3; @}
674 ;
675@end example
676
677@noindent
678The action says how to produce the semantic value of the sum expression
679from the values of the two subexpressions.
680
676385e2 681@node GLR Parsers
c827f760
PE
682@section Writing @acronym{GLR} Parsers
683@cindex @acronym{GLR} parsing
684@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2
PH
685@findex %glr-parser
686@cindex conflicts
687@cindex shift/reduce conflicts
fa7e68c3 688@cindex reduce/reduce conflicts
676385e2 689
fa7e68c3 690In some grammars, Bison's standard
9501dc6e
AD
691@acronym{LALR}(1) parsing algorithm cannot decide whether to apply a
692certain grammar rule at a given point. That is, it may not be able to
693decide (on the basis of the input read so far) which of two possible
694reductions (applications of a grammar rule) applies, or whether to apply
695a reduction or read more of the input and apply a reduction later in the
696input. These are known respectively as @dfn{reduce/reduce} conflicts
697(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
698(@pxref{Shift/Reduce}).
699
700To use a grammar that is not easily modified to be @acronym{LALR}(1), a
701more general parsing algorithm is sometimes necessary. If you include
676385e2 702@code{%glr-parser} among the Bison declarations in your file
fa7e68c3 703(@pxref{Grammar Outline}), the result is a Generalized @acronym{LR}
9501dc6e
AD
704(@acronym{GLR}) parser. These parsers handle Bison grammars that
705contain no unresolved conflicts (i.e., after applying precedence
706declarations) identically to @acronym{LALR}(1) parsers. However, when
707faced with unresolved shift/reduce and reduce/reduce conflicts,
708@acronym{GLR} parsers use the simple expedient of doing both,
709effectively cloning the parser to follow both possibilities. Each of
710the resulting parsers can again split, so that at any given time, there
711can be any number of possible parses being explored. The parsers
676385e2
PH
712proceed in lockstep; that is, all of them consume (shift) a given input
713symbol before any of them proceed to the next. Each of the cloned
714parsers eventually meets one of two possible fates: either it runs into
715a parsing error, in which case it simply vanishes, or it merges with
716another parser, because the two of them have reduced the input to an
717identical set of symbols.
718
719During the time that there are multiple parsers, semantic actions are
720recorded, but not performed. When a parser disappears, its recorded
721semantic actions disappear as well, and are never performed. When a
722reduction makes two parsers identical, causing them to merge, Bison
723records both sets of semantic actions. Whenever the last two parsers
724merge, reverting to the single-parser case, Bison resolves all the
725outstanding actions either by precedences given to the grammar rules
726involved, or by performing both actions, and then calling a designated
727user-defined function on the resulting values to produce an arbitrary
728merged result.
729
fa7e68c3 730@menu
32c29292
JD
731* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
732* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
733* GLR Semantic Actions:: Deferred semantic actions have special concerns.
734* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
fa7e68c3
PE
735@end menu
736
737@node Simple GLR Parsers
738@subsection Using @acronym{GLR} on Unambiguous Grammars
739@cindex @acronym{GLR} parsing, unambiguous grammars
740@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars
741@findex %glr-parser
742@findex %expect-rr
743@cindex conflicts
744@cindex reduce/reduce conflicts
745@cindex shift/reduce conflicts
746
747In the simplest cases, you can use the @acronym{GLR} algorithm
748to parse grammars that are unambiguous, but fail to be @acronym{LALR}(1).
742e4900 749Such grammars typically require more than one symbol of lookahead,
fa7e68c3
PE
750or (in rare cases) fall into the category of grammars in which the
751@acronym{LALR}(1) algorithm throws away too much information (they are in
752@acronym{LR}(1), but not @acronym{LALR}(1), @ref{Mystery Conflicts}).
753
754Consider a problem that
755arises in the declaration of enumerated and subrange types in the
756programming language Pascal. Here are some examples:
757
758@example
759type subrange = lo .. hi;
760type enum = (a, b, c);
761@end example
762
763@noindent
764The original language standard allows only numeric
765literals and constant identifiers for the subrange bounds (@samp{lo}
766and @samp{hi}), but Extended Pascal (@acronym{ISO}/@acronym{IEC}
76710206) and many other
768Pascal implementations allow arbitrary expressions there. This gives
769rise to the following situation, containing a superfluous pair of
770parentheses:
771
772@example
773type subrange = (a) .. b;
774@end example
775
776@noindent
777Compare this to the following declaration of an enumerated
778type with only one value:
779
780@example
781type enum = (a);
782@end example
783
784@noindent
785(These declarations are contrived, but they are syntactically
786valid, and more-complicated cases can come up in practical programs.)
787
788These two declarations look identical until the @samp{..} token.
742e4900 789With normal @acronym{LALR}(1) one-token lookahead it is not
fa7e68c3
PE
790possible to decide between the two forms when the identifier
791@samp{a} is parsed. It is, however, desirable
792for a parser to decide this, since in the latter case
793@samp{a} must become a new identifier to represent the enumeration
794value, while in the former case @samp{a} must be evaluated with its
795current meaning, which may be a constant or even a function call.
796
797You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
798to be resolved later, but this typically requires substantial
799contortions in both semantic actions and large parts of the
800grammar, where the parentheses are nested in the recursive rules for
801expressions.
802
803You might think of using the lexer to distinguish between the two
804forms by returning different tokens for currently defined and
805undefined identifiers. But if these declarations occur in a local
806scope, and @samp{a} is defined in an outer scope, then both forms
807are possible---either locally redefining @samp{a}, or using the
808value of @samp{a} from the outer scope. So this approach cannot
809work.
810
e757bb10 811A simple solution to this problem is to declare the parser to
fa7e68c3
PE
812use the @acronym{GLR} algorithm.
813When the @acronym{GLR} parser reaches the critical state, it
814merely splits into two branches and pursues both syntax rules
815simultaneously. Sooner or later, one of them runs into a parsing
816error. If there is a @samp{..} token before the next
817@samp{;}, the rule for enumerated types fails since it cannot
818accept @samp{..} anywhere; otherwise, the subrange type rule
819fails since it requires a @samp{..} token. So one of the branches
820fails silently, and the other one continues normally, performing
821all the intermediate actions that were postponed during the split.
822
823If the input is syntactically incorrect, both branches fail and the parser
824reports a syntax error as usual.
825
826The effect of all this is that the parser seems to ``guess'' the
827correct branch to take, or in other words, it seems to use more
742e4900 828lookahead than the underlying @acronym{LALR}(1) algorithm actually allows
fa7e68c3
PE
829for. In this example, @acronym{LALR}(2) would suffice, but also some cases
830that are not @acronym{LALR}(@math{k}) for any @math{k} can be handled this way.
831
832In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time,
833and the current Bison parser even takes exponential time and space
834for some grammars. In practice, this rarely happens, and for many
835grammars it is possible to prove that it cannot happen.
836The present example contains only one conflict between two
837rules, and the type-declaration context containing the conflict
838cannot be nested. So the number of
839branches that can exist at any time is limited by the constant 2,
840and the parsing time is still linear.
841
842Here is a Bison grammar corresponding to the example above. It
843parses a vastly simplified form of Pascal type declarations.
844
845@example
846%token TYPE DOTDOT ID
847
848@group
849%left '+' '-'
850%left '*' '/'
851@end group
852
853%%
854
855@group
856type_decl : TYPE ID '=' type ';'
857 ;
858@end group
859
860@group
861type : '(' id_list ')'
862 | expr DOTDOT expr
863 ;
864@end group
865
866@group
867id_list : ID
868 | id_list ',' ID
869 ;
870@end group
871
872@group
873expr : '(' expr ')'
874 | expr '+' expr
875 | expr '-' expr
876 | expr '*' expr
877 | expr '/' expr
878 | ID
879 ;
880@end group
881@end example
882
883When used as a normal @acronym{LALR}(1) grammar, Bison correctly complains
884about one reduce/reduce conflict. In the conflicting situation the
885parser chooses one of the alternatives, arbitrarily the one
886declared first. Therefore the following correct input is not
887recognized:
888
889@example
890type t = (a) .. b;
891@end example
892
893The parser can be turned into a @acronym{GLR} parser, while also telling Bison
894to be silent about the one known reduce/reduce conflict, by
e757bb10 895adding these two declarations to the Bison input file (before the first
fa7e68c3
PE
896@samp{%%}):
897
898@example
899%glr-parser
900%expect-rr 1
901@end example
902
903@noindent
904No change in the grammar itself is required. Now the
905parser recognizes all valid declarations, according to the
906limited syntax above, transparently. In fact, the user does not even
907notice when the parser splits.
908
f8e1c9e5
AD
909So here we have a case where we can use the benefits of @acronym{GLR},
910almost without disadvantages. Even in simple cases like this, however,
911there are at least two potential problems to beware. First, always
912analyze the conflicts reported by Bison to make sure that @acronym{GLR}
913splitting is only done where it is intended. A @acronym{GLR} parser
914splitting inadvertently may cause problems less obvious than an
915@acronym{LALR} parser statically choosing the wrong alternative in a
916conflict. Second, consider interactions with the lexer (@pxref{Semantic
917Tokens}) with great care. Since a split parser consumes tokens without
918performing any actions during the split, the lexer cannot obtain
919information via parser actions. Some cases of lexer interactions can be
920eliminated by using @acronym{GLR} to shift the complications from the
921lexer to the parser. You must check the remaining cases for
922correctness.
923
924In our example, it would be safe for the lexer to return tokens based on
925their current meanings in some symbol table, because no new symbols are
926defined in the middle of a type declaration. Though it is possible for
927a parser to define the enumeration constants as they are parsed, before
928the type declaration is completed, it actually makes no difference since
929they cannot be used within the same enumerated type declaration.
fa7e68c3
PE
930
931@node Merging GLR Parses
932@subsection Using @acronym{GLR} to Resolve Ambiguities
933@cindex @acronym{GLR} parsing, ambiguous grammars
934@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, ambiguous grammars
935@findex %dprec
936@findex %merge
937@cindex conflicts
938@cindex reduce/reduce conflicts
939
2a8d363a 940Let's consider an example, vastly simplified from a C++ grammar.
676385e2
PH
941
942@example
943%@{
38a92d50
PE
944 #include <stdio.h>
945 #define YYSTYPE char const *
946 int yylex (void);
947 void yyerror (char const *);
676385e2
PH
948%@}
949
950%token TYPENAME ID
951
952%right '='
953%left '+'
954
955%glr-parser
956
957%%
958
fae437e8 959prog :
676385e2
PH
960 | prog stmt @{ printf ("\n"); @}
961 ;
962
963stmt : expr ';' %dprec 1
964 | decl %dprec 2
965 ;
966
2a8d363a 967expr : ID @{ printf ("%s ", $$); @}
fae437e8 968 | TYPENAME '(' expr ')'
2a8d363a
AD
969 @{ printf ("%s <cast> ", $1); @}
970 | expr '+' expr @{ printf ("+ "); @}
971 | expr '=' expr @{ printf ("= "); @}
676385e2
PH
972 ;
973
fae437e8 974decl : TYPENAME declarator ';'
2a8d363a 975 @{ printf ("%s <declare> ", $1); @}
676385e2 976 | TYPENAME declarator '=' expr ';'
2a8d363a 977 @{ printf ("%s <init-declare> ", $1); @}
676385e2
PH
978 ;
979
2a8d363a 980declarator : ID @{ printf ("\"%s\" ", $1); @}
676385e2
PH
981 | '(' declarator ')'
982 ;
983@end example
984
985@noindent
986This models a problematic part of the C++ grammar---the ambiguity between
987certain declarations and statements. For example,
988
989@example
990T (x) = y+z;
991@end example
992
993@noindent
994parses as either an @code{expr} or a @code{stmt}
c827f760
PE
995(assuming that @samp{T} is recognized as a @code{TYPENAME} and
996@samp{x} as an @code{ID}).
676385e2 997Bison detects this as a reduce/reduce conflict between the rules
fae437e8 998@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
e757bb10
AD
999time it encounters @code{x} in the example above. Since this is a
1000@acronym{GLR} parser, it therefore splits the problem into two parses, one for
fa7e68c3
PE
1001each choice of resolving the reduce/reduce conflict.
1002Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1003however, neither of these parses ``dies,'' because the grammar as it stands is
e757bb10
AD
1004ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1005the other reduces @code{stmt : decl}, after which both parsers are in an
1006identical state: they've seen @samp{prog stmt} and have the same unprocessed
1007input remaining. We say that these parses have @dfn{merged.}
fa7e68c3
PE
1008
1009At this point, the @acronym{GLR} parser requires a specification in the
1010grammar of how to choose between the competing parses.
1011In the example above, the two @code{%dprec}
e757bb10 1012declarations specify that Bison is to give precedence
fa7e68c3 1013to the parse that interprets the example as a
676385e2
PH
1014@code{decl}, which implies that @code{x} is a declarator.
1015The parser therefore prints
1016
1017@example
fae437e8 1018"x" y z + T <init-declare>
676385e2
PH
1019@end example
1020
fa7e68c3
PE
1021The @code{%dprec} declarations only come into play when more than one
1022parse survives. Consider a different input string for this parser:
676385e2
PH
1023
1024@example
1025T (x) + y;
1026@end example
1027
1028@noindent
e757bb10 1029This is another example of using @acronym{GLR} to parse an unambiguous
fa7e68c3 1030construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
676385e2
PH
1031Here, there is no ambiguity (this cannot be parsed as a declaration).
1032However, at the time the Bison parser encounters @code{x}, it does not
1033have enough information to resolve the reduce/reduce conflict (again,
1034between @code{x} as an @code{expr} or a @code{declarator}). In this
fa7e68c3 1035case, no precedence declaration is used. Again, the parser splits
676385e2
PH
1036into two, one assuming that @code{x} is an @code{expr}, and the other
1037assuming @code{x} is a @code{declarator}. The second of these parsers
1038then vanishes when it sees @code{+}, and the parser prints
1039
1040@example
fae437e8 1041x T <cast> y +
676385e2
PH
1042@end example
1043
1044Suppose that instead of resolving the ambiguity, you wanted to see all
fa7e68c3 1045the possibilities. For this purpose, you must merge the semantic
676385e2
PH
1046actions of the two possible parsers, rather than choosing one over the
1047other. To do so, you could change the declaration of @code{stmt} as
1048follows:
1049
1050@example
1051stmt : expr ';' %merge <stmtMerge>
1052 | decl %merge <stmtMerge>
1053 ;
1054@end example
1055
1056@noindent
676385e2
PH
1057and define the @code{stmtMerge} function as:
1058
1059@example
38a92d50
PE
1060static YYSTYPE
1061stmtMerge (YYSTYPE x0, YYSTYPE x1)
676385e2
PH
1062@{
1063 printf ("<OR> ");
1064 return "";
1065@}
1066@end example
1067
1068@noindent
1069with an accompanying forward declaration
1070in the C declarations at the beginning of the file:
1071
1072@example
1073%@{
38a92d50 1074 #define YYSTYPE char const *
676385e2
PH
1075 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1076%@}
1077@end example
1078
1079@noindent
fa7e68c3
PE
1080With these declarations, the resulting parser parses the first example
1081as both an @code{expr} and a @code{decl}, and prints
676385e2
PH
1082
1083@example
fae437e8 1084"x" y z + T <init-declare> x T <cast> y z + = <OR>
676385e2
PH
1085@end example
1086
fa7e68c3 1087Bison requires that all of the
e757bb10 1088productions that participate in any particular merge have identical
fa7e68c3
PE
1089@samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1090and the parser will report an error during any parse that results in
1091the offending merge.
9501dc6e 1092
32c29292
JD
1093@node GLR Semantic Actions
1094@subsection GLR Semantic Actions
1095
1096@cindex deferred semantic actions
1097By definition, a deferred semantic action is not performed at the same time as
1098the associated reduction.
1099This raises caveats for several Bison features you might use in a semantic
1100action in a @acronym{GLR} parser.
1101
1102@vindex yychar
1103@cindex @acronym{GLR} parsers and @code{yychar}
1104@vindex yylval
1105@cindex @acronym{GLR} parsers and @code{yylval}
1106@vindex yylloc
1107@cindex @acronym{GLR} parsers and @code{yylloc}
1108In any semantic action, you can examine @code{yychar} to determine the type of
742e4900 1109the lookahead token present at the time of the associated reduction.
32c29292
JD
1110After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1111you can then examine @code{yylval} and @code{yylloc} to determine the
742e4900 1112lookahead token's semantic value and location, if any.
32c29292
JD
1113In a nondeferred semantic action, you can also modify any of these variables to
1114influence syntax analysis.
742e4900 1115@xref{Lookahead, ,Lookahead Tokens}.
32c29292
JD
1116
1117@findex yyclearin
1118@cindex @acronym{GLR} parsers and @code{yyclearin}
1119In a deferred semantic action, it's too late to influence syntax analysis.
1120In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1121shallow copies of the values they had at the time of the associated reduction.
1122For this reason alone, modifying them is dangerous.
1123Moreover, the result of modifying them is undefined and subject to change with
1124future versions of Bison.
1125For example, if a semantic action might be deferred, you should never write it
1126to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1127memory referenced by @code{yylval}.
1128
1129@findex YYERROR
1130@cindex @acronym{GLR} parsers and @code{YYERROR}
1131Another Bison feature requiring special consideration is @code{YYERROR}
8710fc41 1132(@pxref{Action Features}), which you can invoke in a semantic action to
32c29292
JD
1133initiate error recovery.
1134During deterministic @acronym{GLR} operation, the effect of @code{YYERROR} is
1135the same as its effect in an @acronym{LALR}(1) parser.
1136In a deferred semantic action, its effect is undefined.
1137@c The effect is probably a syntax error at the split point.
1138
8710fc41
JD
1139Also, see @ref{Location Default Action, ,Default Action for Locations}, which
1140describes a special usage of @code{YYLLOC_DEFAULT} in @acronym{GLR} parsers.
1141
fa7e68c3
PE
1142@node Compiler Requirements
1143@subsection Considerations when Compiling @acronym{GLR} Parsers
1144@cindex @code{inline}
9501dc6e 1145@cindex @acronym{GLR} parsers and @code{inline}
fa7e68c3 1146
38a92d50
PE
1147The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or
1148later. In addition, they use the @code{inline} keyword, which is not
1149C89, but is C99 and is a common extension in pre-C99 compilers. It is
1150up to the user of these parsers to handle
9501dc6e
AD
1151portability issues. For instance, if using Autoconf and the Autoconf
1152macro @code{AC_C_INLINE}, a mere
1153
1154@example
1155%@{
38a92d50 1156 #include <config.h>
9501dc6e
AD
1157%@}
1158@end example
1159
1160@noindent
1161will suffice. Otherwise, we suggest
1162
1163@example
1164%@{
38a92d50
PE
1165 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1166 #define inline
1167 #endif
9501dc6e
AD
1168%@}
1169@end example
676385e2 1170
342b8b6e 1171@node Locations Overview
847bf1f5
AD
1172@section Locations
1173@cindex location
95923bd6
AD
1174@cindex textual location
1175@cindex location, textual
847bf1f5
AD
1176
1177Many applications, like interpreters or compilers, have to produce verbose
72d2299c 1178and useful error messages. To achieve this, one must be able to keep track of
95923bd6 1179the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
847bf1f5
AD
1180Bison provides a mechanism for handling these locations.
1181
72d2299c 1182Each token has a semantic value. In a similar fashion, each token has an
847bf1f5 1183associated location, but the type of locations is the same for all tokens and
72d2299c 1184groupings. Moreover, the output parser is equipped with a default data
847bf1f5
AD
1185structure for storing locations (@pxref{Locations}, for more details).
1186
1187Like semantic values, locations can be reached in actions using a dedicated
72d2299c 1188set of constructs. In the example above, the location of the whole grouping
847bf1f5
AD
1189is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1190@code{@@3}.
1191
1192When a rule is matched, a default action is used to compute the semantic value
72d2299c
PE
1193of its left hand side (@pxref{Actions}). In the same way, another default
1194action is used for locations. However, the action for locations is general
847bf1f5 1195enough for most cases, meaning there is usually no need to describe for each
72d2299c 1196rule how @code{@@$} should be formed. When building a new location for a given
847bf1f5
AD
1197grouping, the default behavior of the output parser is to take the beginning
1198of the first symbol, and the end of the last symbol.
1199
342b8b6e 1200@node Bison Parser
bfa74976
RS
1201@section Bison Output: the Parser File
1202@cindex Bison parser
1203@cindex Bison utility
1204@cindex lexical analyzer, purpose
1205@cindex parser
1206
1207When you run Bison, you give it a Bison grammar file as input. The output
1208is a C source file that parses the language described by the grammar.
1209This file is called a @dfn{Bison parser}. Keep in mind that the Bison
1210utility and the Bison parser are two distinct programs: the Bison utility
1211is a program whose output is the Bison parser that becomes part of your
1212program.
1213
1214The job of the Bison parser is to group tokens into groupings according to
1215the grammar rules---for example, to build identifiers and operators into
1216expressions. As it does this, it runs the actions for the grammar rules it
1217uses.
1218
704a47c4
AD
1219The tokens come from a function called the @dfn{lexical analyzer} that
1220you must supply in some fashion (such as by writing it in C). The Bison
1221parser calls the lexical analyzer each time it wants a new token. It
1222doesn't know what is ``inside'' the tokens (though their semantic values
1223may reflect this). Typically the lexical analyzer makes the tokens by
1224parsing characters of text, but Bison does not depend on this.
1225@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
bfa74976
RS
1226
1227The Bison parser file is C code which defines a function named
1228@code{yyparse} which implements that grammar. This function does not make
1229a complete C program: you must supply some additional functions. One is
1230the lexical analyzer. Another is an error-reporting function which the
1231parser calls to report an error. In addition, a complete C program must
1232start with a function called @code{main}; you have to provide this, and
1233arrange for it to call @code{yyparse} or the parser will never run.
1234@xref{Interface, ,Parser C-Language Interface}.
1235
f7ab6a50 1236Aside from the token type names and the symbols in the actions you
7093d0f5 1237write, all symbols defined in the Bison parser file itself
bfa74976
RS
1238begin with @samp{yy} or @samp{YY}. This includes interface functions
1239such as the lexical analyzer function @code{yylex}, the error reporting
1240function @code{yyerror} and the parser function @code{yyparse} itself.
1241This also includes numerous identifiers used for internal purposes.
1242Therefore, you should avoid using C identifiers starting with @samp{yy}
1243or @samp{YY} in the Bison grammar file except for the ones defined in
55289366
PE
1244this manual. Also, you should avoid using the C identifiers
1245@samp{malloc} and @samp{free} for anything other than their usual
1246meanings.
bfa74976 1247
7093d0f5
AD
1248In some cases the Bison parser file includes system headers, and in
1249those cases your code should respect the identifiers reserved by those
55289366 1250headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>}, @code{<malloc.h>},
7093d0f5 1251@code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
30757c8c
PE
1252declare memory allocators and related types. @code{<libintl.h>} is
1253included if message translation is in use
1254(@pxref{Internationalization}). Other system headers may
ec3bc396
AD
1255be included if you define @code{YYDEBUG} to a nonzero value
1256(@pxref{Tracing, ,Tracing Your Parser}).
7093d0f5 1257
342b8b6e 1258@node Stages
bfa74976
RS
1259@section Stages in Using Bison
1260@cindex stages in using Bison
1261@cindex using Bison
1262
1263The actual language-design process using Bison, from grammar specification
1264to a working compiler or interpreter, has these parts:
1265
1266@enumerate
1267@item
1268Formally specify the grammar in a form recognized by Bison
704a47c4
AD
1269(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1270in the language, describe the action that is to be taken when an
1271instance of that rule is recognized. The action is described by a
1272sequence of C statements.
bfa74976
RS
1273
1274@item
704a47c4
AD
1275Write a lexical analyzer to process input and pass tokens to the parser.
1276The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1277Lexical Analyzer Function @code{yylex}}). It could also be produced
1278using Lex, but the use of Lex is not discussed in this manual.
bfa74976
RS
1279
1280@item
1281Write a controlling function that calls the Bison-produced parser.
1282
1283@item
1284Write error-reporting routines.
1285@end enumerate
1286
1287To turn this source code as written into a runnable program, you
1288must follow these steps:
1289
1290@enumerate
1291@item
1292Run Bison on the grammar to produce the parser.
1293
1294@item
1295Compile the code output by Bison, as well as any other source files.
1296
1297@item
1298Link the object files to produce the finished product.
1299@end enumerate
1300
342b8b6e 1301@node Grammar Layout
bfa74976
RS
1302@section The Overall Layout of a Bison Grammar
1303@cindex grammar file
1304@cindex file format
1305@cindex format of grammar file
1306@cindex layout of Bison grammar
1307
1308The input file for the Bison utility is a @dfn{Bison grammar file}. The
1309general form of a Bison grammar file is as follows:
1310
1311@example
1312%@{
08e49d20 1313@var{Prologue}
bfa74976
RS
1314%@}
1315
1316@var{Bison declarations}
1317
1318%%
1319@var{Grammar rules}
1320%%
08e49d20 1321@var{Epilogue}
bfa74976
RS
1322@end example
1323
1324@noindent
1325The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1326in every Bison grammar file to separate the sections.
1327
72d2299c 1328The prologue may define types and variables used in the actions. You can
342b8b6e 1329also use preprocessor commands to define macros used there, and use
bfa74976 1330@code{#include} to include header files that do any of these things.
38a92d50
PE
1331You need to declare the lexical analyzer @code{yylex} and the error
1332printer @code{yyerror} here, along with any other global identifiers
1333used by the actions in the grammar rules.
bfa74976
RS
1334
1335The Bison declarations declare the names of the terminal and nonterminal
1336symbols, and may also describe operator precedence and the data types of
1337semantic values of various symbols.
1338
1339The grammar rules define how to construct each nonterminal symbol from its
1340parts.
1341
38a92d50
PE
1342The epilogue can contain any code you want to use. Often the
1343definitions of functions declared in the prologue go here. In a
1344simple program, all the rest of the program can go here.
bfa74976 1345
342b8b6e 1346@node Examples
bfa74976
RS
1347@chapter Examples
1348@cindex simple examples
1349@cindex examples, simple
1350
1351Now we show and explain three sample programs written using Bison: a
1352reverse polish notation calculator, an algebraic (infix) notation
1353calculator, and a multi-function calculator. All three have been tested
1354under BSD Unix 4.3; each produces a usable, though limited, interactive
1355desk-top calculator.
1356
1357These examples are simple, but Bison grammars for real programming
aa08666d
AD
1358languages are written the same way. You can copy these examples into a
1359source file to try them.
bfa74976
RS
1360
1361@menu
1362* RPN Calc:: Reverse polish notation calculator;
1363 a first example with no operator precedence.
1364* Infix Calc:: Infix (algebraic) notation calculator.
1365 Operator precedence is introduced.
1366* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 1367* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
bfa74976
RS
1368* Multi-function Calc:: Calculator with memory and trig functions.
1369 It uses multiple data-types for semantic values.
1370* Exercises:: Ideas for improving the multi-function calculator.
1371@end menu
1372
342b8b6e 1373@node RPN Calc
bfa74976
RS
1374@section Reverse Polish Notation Calculator
1375@cindex reverse polish notation
1376@cindex polish notation calculator
1377@cindex @code{rpcalc}
1378@cindex calculator, simple
1379
1380The first example is that of a simple double-precision @dfn{reverse polish
1381notation} calculator (a calculator using postfix operators). This example
1382provides a good starting point, since operator precedence is not an issue.
1383The second example will illustrate how operator precedence is handled.
1384
1385The source code for this calculator is named @file{rpcalc.y}. The
1386@samp{.y} extension is a convention used for Bison input files.
1387
1388@menu
75f5aaea 1389* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
bfa74976
RS
1390* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
1391* Lexer: Rpcalc Lexer. The lexical analyzer.
1392* Main: Rpcalc Main. The controlling function.
1393* Error: Rpcalc Error. The error reporting function.
1394* Gen: Rpcalc Gen. Running Bison on the grammar file.
1395* Comp: Rpcalc Compile. Run the C compiler on the output code.
1396@end menu
1397
342b8b6e 1398@node Rpcalc Decls
bfa74976
RS
1399@subsection Declarations for @code{rpcalc}
1400
1401Here are the C and Bison declarations for the reverse polish notation
1402calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1403
1404@example
72d2299c 1405/* Reverse polish notation calculator. */
bfa74976
RS
1406
1407%@{
38a92d50
PE
1408 #define YYSTYPE double
1409 #include <math.h>
1410 int yylex (void);
1411 void yyerror (char const *);
bfa74976
RS
1412%@}
1413
1414%token NUM
1415
72d2299c 1416%% /* Grammar rules and actions follow. */
bfa74976
RS
1417@end example
1418
75f5aaea 1419The declarations section (@pxref{Prologue, , The prologue}) contains two
38a92d50 1420preprocessor directives and two forward declarations.
bfa74976
RS
1421
1422The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1964ad8c
AD
1423specifying the C data type for semantic values of both tokens and
1424groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1425Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1426don't define it, @code{int} is the default. Because we specify
1427@code{double}, each token and each expression has an associated value,
1428which is a floating point number.
bfa74976
RS
1429
1430The @code{#include} directive is used to declare the exponentiation
1431function @code{pow}.
1432
38a92d50
PE
1433The forward declarations for @code{yylex} and @code{yyerror} are
1434needed because the C language requires that functions be declared
1435before they are used. These functions will be defined in the
1436epilogue, but the parser calls them so they must be declared in the
1437prologue.
1438
704a47c4
AD
1439The second section, Bison declarations, provides information to Bison
1440about the token types (@pxref{Bison Declarations, ,The Bison
1441Declarations Section}). Each terminal symbol that is not a
1442single-character literal must be declared here. (Single-character
bfa74976
RS
1443literals normally don't need to be declared.) In this example, all the
1444arithmetic operators are designated by single-character literals, so the
1445only terminal symbol that needs to be declared is @code{NUM}, the token
1446type for numeric constants.
1447
342b8b6e 1448@node Rpcalc Rules
bfa74976
RS
1449@subsection Grammar Rules for @code{rpcalc}
1450
1451Here are the grammar rules for the reverse polish notation calculator.
1452
1453@example
1454input: /* empty */
1455 | input line
1456;
1457
1458line: '\n'
18b519c0 1459 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
bfa74976
RS
1460;
1461
18b519c0
AD
1462exp: NUM @{ $$ = $1; @}
1463 | exp exp '+' @{ $$ = $1 + $2; @}
1464 | exp exp '-' @{ $$ = $1 - $2; @}
1465 | exp exp '*' @{ $$ = $1 * $2; @}
1466 | exp exp '/' @{ $$ = $1 / $2; @}
1467 /* Exponentiation */
1468 | exp exp '^' @{ $$ = pow ($1, $2); @}
1469 /* Unary minus */
1470 | exp 'n' @{ $$ = -$1; @}
bfa74976
RS
1471;
1472%%
1473@end example
1474
1475The groupings of the rpcalc ``language'' defined here are the expression
1476(given the name @code{exp}), the line of input (@code{line}), and the
1477complete input transcript (@code{input}). Each of these nonterminal
8c5b881d 1478symbols has several alternate rules, joined by the vertical bar @samp{|}
bfa74976
RS
1479which is read as ``or''. The following sections explain what these rules
1480mean.
1481
1482The semantics of the language is determined by the actions taken when a
1483grouping is recognized. The actions are the C code that appears inside
1484braces. @xref{Actions}.
1485
1486You must specify these actions in C, but Bison provides the means for
1487passing semantic values between the rules. In each action, the
1488pseudo-variable @code{$$} stands for the semantic value for the grouping
1489that the rule is going to construct. Assigning a value to @code{$$} is the
1490main job of most actions. The semantic values of the components of the
1491rule are referred to as @code{$1}, @code{$2}, and so on.
1492
1493@menu
13863333
AD
1494* Rpcalc Input::
1495* Rpcalc Line::
1496* Rpcalc Expr::
bfa74976
RS
1497@end menu
1498
342b8b6e 1499@node Rpcalc Input
bfa74976
RS
1500@subsubsection Explanation of @code{input}
1501
1502Consider the definition of @code{input}:
1503
1504@example
1505input: /* empty */
1506 | input line
1507;
1508@end example
1509
1510This definition reads as follows: ``A complete input is either an empty
1511string, or a complete input followed by an input line''. Notice that
1512``complete input'' is defined in terms of itself. This definition is said
1513to be @dfn{left recursive} since @code{input} appears always as the
1514leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1515
1516The first alternative is empty because there are no symbols between the
1517colon and the first @samp{|}; this means that @code{input} can match an
1518empty string of input (no tokens). We write the rules this way because it
1519is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1520It's conventional to put an empty alternative first and write the comment
1521@samp{/* empty */} in it.
1522
1523The second alternate rule (@code{input line}) handles all nontrivial input.
1524It means, ``After reading any number of lines, read one more line if
1525possible.'' The left recursion makes this rule into a loop. Since the
1526first alternative matches empty input, the loop can be executed zero or
1527more times.
1528
1529The parser function @code{yyparse} continues to process input until a
1530grammatical error is seen or the lexical analyzer says there are no more
72d2299c 1531input tokens; we will arrange for the latter to happen at end-of-input.
bfa74976 1532
342b8b6e 1533@node Rpcalc Line
bfa74976
RS
1534@subsubsection Explanation of @code{line}
1535
1536Now consider the definition of @code{line}:
1537
1538@example
1539line: '\n'
1540 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1541;
1542@end example
1543
1544The first alternative is a token which is a newline character; this means
1545that rpcalc accepts a blank line (and ignores it, since there is no
1546action). The second alternative is an expression followed by a newline.
1547This is the alternative that makes rpcalc useful. The semantic value of
1548the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1549question is the first symbol in the alternative. The action prints this
1550value, which is the result of the computation the user asked for.
1551
1552This action is unusual because it does not assign a value to @code{$$}. As
1553a consequence, the semantic value associated with the @code{line} is
1554uninitialized (its value will be unpredictable). This would be a bug if
1555that value were ever used, but we don't use it: once rpcalc has printed the
1556value of the user's input line, that value is no longer needed.
1557
342b8b6e 1558@node Rpcalc Expr
bfa74976
RS
1559@subsubsection Explanation of @code{expr}
1560
1561The @code{exp} grouping has several rules, one for each kind of expression.
1562The first rule handles the simplest expressions: those that are just numbers.
1563The second handles an addition-expression, which looks like two expressions
1564followed by a plus-sign. The third handles subtraction, and so on.
1565
1566@example
1567exp: NUM
1568 | exp exp '+' @{ $$ = $1 + $2; @}
1569 | exp exp '-' @{ $$ = $1 - $2; @}
1570 @dots{}
1571 ;
1572@end example
1573
1574We have used @samp{|} to join all the rules for @code{exp}, but we could
1575equally well have written them separately:
1576
1577@example
1578exp: NUM ;
1579exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1580exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1581 @dots{}
1582@end example
1583
1584Most of the rules have actions that compute the value of the expression in
1585terms of the value of its parts. For example, in the rule for addition,
1586@code{$1} refers to the first component @code{exp} and @code{$2} refers to
1587the second one. The third component, @code{'+'}, has no meaningful
1588associated semantic value, but if it had one you could refer to it as
1589@code{$3}. When @code{yyparse} recognizes a sum expression using this
1590rule, the sum of the two subexpressions' values is produced as the value of
1591the entire expression. @xref{Actions}.
1592
1593You don't have to give an action for every rule. When a rule has no
1594action, Bison by default copies the value of @code{$1} into @code{$$}.
1595This is what happens in the first rule (the one that uses @code{NUM}).
1596
1597The formatting shown here is the recommended convention, but Bison does
72d2299c 1598not require it. You can add or change white space as much as you wish.
bfa74976
RS
1599For example, this:
1600
1601@example
99a9344e 1602exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
bfa74976
RS
1603@end example
1604
1605@noindent
1606means the same thing as this:
1607
1608@example
1609exp: NUM
1610 | exp exp '+' @{ $$ = $1 + $2; @}
1611 | @dots{}
99a9344e 1612;
bfa74976
RS
1613@end example
1614
1615@noindent
1616The latter, however, is much more readable.
1617
342b8b6e 1618@node Rpcalc Lexer
bfa74976
RS
1619@subsection The @code{rpcalc} Lexical Analyzer
1620@cindex writing a lexical analyzer
1621@cindex lexical analyzer, writing
1622
704a47c4
AD
1623The lexical analyzer's job is low-level parsing: converting characters
1624or sequences of characters into tokens. The Bison parser gets its
1625tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1626Analyzer Function @code{yylex}}.
bfa74976 1627
c827f760
PE
1628Only a simple lexical analyzer is needed for the @acronym{RPN}
1629calculator. This
bfa74976
RS
1630lexical analyzer skips blanks and tabs, then reads in numbers as
1631@code{double} and returns them as @code{NUM} tokens. Any other character
1632that isn't part of a number is a separate token. Note that the token-code
1633for such a single-character token is the character itself.
1634
1635The return value of the lexical analyzer function is a numeric code which
1636represents a token type. The same text used in Bison rules to stand for
1637this token type is also a C expression for the numeric code for the type.
1638This works in two ways. If the token type is a character literal, then its
e966383b 1639numeric code is that of the character; you can use the same
bfa74976
RS
1640character literal in the lexical analyzer to express the number. If the
1641token type is an identifier, that identifier is defined by Bison as a C
1642macro whose definition is the appropriate number. In this example,
1643therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1644
1964ad8c
AD
1645The semantic value of the token (if it has one) is stored into the
1646global variable @code{yylval}, which is where the Bison parser will look
1647for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1648defined at the beginning of the grammar; @pxref{Rpcalc Decls,
1649,Declarations for @code{rpcalc}}.)
bfa74976 1650
72d2299c
PE
1651A token type code of zero is returned if the end-of-input is encountered.
1652(Bison recognizes any nonpositive value as indicating end-of-input.)
bfa74976
RS
1653
1654Here is the code for the lexical analyzer:
1655
1656@example
1657@group
72d2299c 1658/* The lexical analyzer returns a double floating point
e966383b 1659 number on the stack and the token NUM, or the numeric code
72d2299c
PE
1660 of the character read if not a number. It skips all blanks
1661 and tabs, and returns 0 for end-of-input. */
bfa74976
RS
1662
1663#include <ctype.h>
1664@end group
1665
1666@group
13863333
AD
1667int
1668yylex (void)
bfa74976
RS
1669@{
1670 int c;
1671
72d2299c 1672 /* Skip white space. */
13863333 1673 while ((c = getchar ()) == ' ' || c == '\t')
bfa74976
RS
1674 ;
1675@end group
1676@group
72d2299c 1677 /* Process numbers. */
13863333 1678 if (c == '.' || isdigit (c))
bfa74976
RS
1679 @{
1680 ungetc (c, stdin);
1681 scanf ("%lf", &yylval);
1682 return NUM;
1683 @}
1684@end group
1685@group
72d2299c 1686 /* Return end-of-input. */
13863333 1687 if (c == EOF)
bfa74976 1688 return 0;
72d2299c 1689 /* Return a single char. */
13863333 1690 return c;
bfa74976
RS
1691@}
1692@end group
1693@end example
1694
342b8b6e 1695@node Rpcalc Main
bfa74976
RS
1696@subsection The Controlling Function
1697@cindex controlling function
1698@cindex main function in simple example
1699
1700In keeping with the spirit of this example, the controlling function is
1701kept to the bare minimum. The only requirement is that it call
1702@code{yyparse} to start the process of parsing.
1703
1704@example
1705@group
13863333
AD
1706int
1707main (void)
bfa74976 1708@{
13863333 1709 return yyparse ();
bfa74976
RS
1710@}
1711@end group
1712@end example
1713
342b8b6e 1714@node Rpcalc Error
bfa74976
RS
1715@subsection The Error Reporting Routine
1716@cindex error reporting routine
1717
1718When @code{yyparse} detects a syntax error, it calls the error reporting
13863333 1719function @code{yyerror} to print an error message (usually but not
6e649e65 1720always @code{"syntax error"}). It is up to the programmer to supply
13863333
AD
1721@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1722here is the definition we will use:
bfa74976
RS
1723
1724@example
1725@group
1726#include <stdio.h>
1727
38a92d50 1728/* Called by yyparse on error. */
13863333 1729void
38a92d50 1730yyerror (char const *s)
bfa74976 1731@{
4e03e201 1732 fprintf (stderr, "%s\n", s);
bfa74976
RS
1733@}
1734@end group
1735@end example
1736
1737After @code{yyerror} returns, the Bison parser may recover from the error
1738and continue parsing if the grammar contains a suitable error rule
1739(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1740have not written any error rules in this example, so any invalid input will
1741cause the calculator program to exit. This is not clean behavior for a
9ecbd125 1742real calculator, but it is adequate for the first example.
bfa74976 1743
342b8b6e 1744@node Rpcalc Gen
bfa74976
RS
1745@subsection Running Bison to Make the Parser
1746@cindex running Bison (introduction)
1747
ceed8467
AD
1748Before running Bison to produce a parser, we need to decide how to
1749arrange all the source code in one or more source files. For such a
1750simple example, the easiest thing is to put everything in one file. The
1751definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
342b8b6e 1752end, in the epilogue of the file
75f5aaea 1753(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
bfa74976
RS
1754
1755For a large project, you would probably have several source files, and use
1756@code{make} to arrange to recompile them.
1757
1758With all the source in a single file, you use the following command to
1759convert it into a parser file:
1760
1761@example
fa4d969f 1762bison @var{file}.y
bfa74976
RS
1763@end example
1764
1765@noindent
1766In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
fa4d969f 1767@sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c},
72d2299c 1768removing the @samp{.y} from the original file name. The file output by
bfa74976
RS
1769Bison contains the source code for @code{yyparse}. The additional
1770functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
1771are copied verbatim to the output.
1772
342b8b6e 1773@node Rpcalc Compile
bfa74976
RS
1774@subsection Compiling the Parser File
1775@cindex compiling the parser
1776
1777Here is how to compile and run the parser file:
1778
1779@example
1780@group
1781# @r{List files in current directory.}
9edcd895 1782$ @kbd{ls}
bfa74976
RS
1783rpcalc.tab.c rpcalc.y
1784@end group
1785
1786@group
1787# @r{Compile the Bison parser.}
1788# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
b56471a6 1789$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
bfa74976
RS
1790@end group
1791
1792@group
1793# @r{List files again.}
9edcd895 1794$ @kbd{ls}
bfa74976
RS
1795rpcalc rpcalc.tab.c rpcalc.y
1796@end group
1797@end example
1798
1799The file @file{rpcalc} now contains the executable code. Here is an
1800example session using @code{rpcalc}.
1801
1802@example
9edcd895
AD
1803$ @kbd{rpcalc}
1804@kbd{4 9 +}
bfa74976 180513
9edcd895 1806@kbd{3 7 + 3 4 5 *+-}
bfa74976 1807-13
9edcd895 1808@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
bfa74976 180913
9edcd895 1810@kbd{5 6 / 4 n +}
bfa74976 1811-3.166666667
9edcd895 1812@kbd{3 4 ^} @r{Exponentiation}
bfa74976 181381
9edcd895
AD
1814@kbd{^D} @r{End-of-file indicator}
1815$
bfa74976
RS
1816@end example
1817
342b8b6e 1818@node Infix Calc
bfa74976
RS
1819@section Infix Notation Calculator: @code{calc}
1820@cindex infix notation calculator
1821@cindex @code{calc}
1822@cindex calculator, infix notation
1823
1824We now modify rpcalc to handle infix operators instead of postfix. Infix
1825notation involves the concept of operator precedence and the need for
1826parentheses nested to arbitrary depth. Here is the Bison code for
1827@file{calc.y}, an infix desk-top calculator.
1828
1829@example
38a92d50 1830/* Infix notation calculator. */
bfa74976
RS
1831
1832%@{
38a92d50
PE
1833 #define YYSTYPE double
1834 #include <math.h>
1835 #include <stdio.h>
1836 int yylex (void);
1837 void yyerror (char const *);
bfa74976
RS
1838%@}
1839
38a92d50 1840/* Bison declarations. */
bfa74976
RS
1841%token NUM
1842%left '-' '+'
1843%left '*' '/'
1844%left NEG /* negation--unary minus */
38a92d50 1845%right '^' /* exponentiation */
bfa74976 1846
38a92d50
PE
1847%% /* The grammar follows. */
1848input: /* empty */
bfa74976
RS
1849 | input line
1850;
1851
1852line: '\n'
1853 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1854;
1855
1856exp: NUM @{ $$ = $1; @}
1857 | exp '+' exp @{ $$ = $1 + $3; @}
1858 | exp '-' exp @{ $$ = $1 - $3; @}
1859 | exp '*' exp @{ $$ = $1 * $3; @}
1860 | exp '/' exp @{ $$ = $1 / $3; @}
1861 | '-' exp %prec NEG @{ $$ = -$2; @}
1862 | exp '^' exp @{ $$ = pow ($1, $3); @}
1863 | '(' exp ')' @{ $$ = $2; @}
1864;
1865%%
1866@end example
1867
1868@noindent
ceed8467
AD
1869The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1870same as before.
bfa74976
RS
1871
1872There are two important new features shown in this code.
1873
1874In the second section (Bison declarations), @code{%left} declares token
1875types and says they are left-associative operators. The declarations
1876@code{%left} and @code{%right} (right associativity) take the place of
1877@code{%token} which is used to declare a token type name without
1878associativity. (These tokens are single-character literals, which
1879ordinarily don't need to be declared. We declare them here to specify
1880the associativity.)
1881
1882Operator precedence is determined by the line ordering of the
1883declarations; the higher the line number of the declaration (lower on
1884the page or screen), the higher the precedence. Hence, exponentiation
1885has the highest precedence, unary minus (@code{NEG}) is next, followed
704a47c4
AD
1886by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator
1887Precedence}.
bfa74976 1888
704a47c4
AD
1889The other important new feature is the @code{%prec} in the grammar
1890section for the unary minus operator. The @code{%prec} simply instructs
1891Bison that the rule @samp{| '-' exp} has the same precedence as
1892@code{NEG}---in this case the next-to-highest. @xref{Contextual
1893Precedence, ,Context-Dependent Precedence}.
bfa74976
RS
1894
1895Here is a sample run of @file{calc.y}:
1896
1897@need 500
1898@example
9edcd895
AD
1899$ @kbd{calc}
1900@kbd{4 + 4.5 - (34/(8*3+-3))}
bfa74976 19016.880952381
9edcd895 1902@kbd{-56 + 2}
bfa74976 1903-54
9edcd895 1904@kbd{3 ^ 2}
bfa74976
RS
19059
1906@end example
1907
342b8b6e 1908@node Simple Error Recovery
bfa74976
RS
1909@section Simple Error Recovery
1910@cindex error recovery, simple
1911
1912Up to this point, this manual has not addressed the issue of @dfn{error
1913recovery}---how to continue parsing after the parser detects a syntax
ceed8467
AD
1914error. All we have handled is error reporting with @code{yyerror}.
1915Recall that by default @code{yyparse} returns after calling
1916@code{yyerror}. This means that an erroneous input line causes the
1917calculator program to exit. Now we show how to rectify this deficiency.
bfa74976
RS
1918
1919The Bison language itself includes the reserved word @code{error}, which
1920may be included in the grammar rules. In the example below it has
1921been added to one of the alternatives for @code{line}:
1922
1923@example
1924@group
1925line: '\n'
1926 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1927 | error '\n' @{ yyerrok; @}
1928;
1929@end group
1930@end example
1931
ceed8467 1932This addition to the grammar allows for simple error recovery in the
6e649e65 1933event of a syntax error. If an expression that cannot be evaluated is
ceed8467
AD
1934read, the error will be recognized by the third rule for @code{line},
1935and parsing will continue. (The @code{yyerror} function is still called
1936upon to print its message as well.) The action executes the statement
1937@code{yyerrok}, a macro defined automatically by Bison; its meaning is
1938that error recovery is complete (@pxref{Error Recovery}). Note the
1939difference between @code{yyerrok} and @code{yyerror}; neither one is a
e0c471a9 1940misprint.
bfa74976
RS
1941
1942This form of error recovery deals with syntax errors. There are other
1943kinds of errors; for example, division by zero, which raises an exception
1944signal that is normally fatal. A real calculator program must handle this
1945signal and use @code{longjmp} to return to @code{main} and resume parsing
1946input lines; it would also have to discard the rest of the current line of
1947input. We won't discuss this issue further because it is not specific to
1948Bison programs.
1949
342b8b6e
AD
1950@node Location Tracking Calc
1951@section Location Tracking Calculator: @code{ltcalc}
1952@cindex location tracking calculator
1953@cindex @code{ltcalc}
1954@cindex calculator, location tracking
1955
9edcd895
AD
1956This example extends the infix notation calculator with location
1957tracking. This feature will be used to improve the error messages. For
1958the sake of clarity, this example is a simple integer calculator, since
1959most of the work needed to use locations will be done in the lexical
72d2299c 1960analyzer.
342b8b6e
AD
1961
1962@menu
1963* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
1964* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
1965* Lexer: Ltcalc Lexer. The lexical analyzer.
1966@end menu
1967
1968@node Ltcalc Decls
1969@subsection Declarations for @code{ltcalc}
1970
9edcd895
AD
1971The C and Bison declarations for the location tracking calculator are
1972the same as the declarations for the infix notation calculator.
342b8b6e
AD
1973
1974@example
1975/* Location tracking calculator. */
1976
1977%@{
38a92d50
PE
1978 #define YYSTYPE int
1979 #include <math.h>
1980 int yylex (void);
1981 void yyerror (char const *);
342b8b6e
AD
1982%@}
1983
1984/* Bison declarations. */
1985%token NUM
1986
1987%left '-' '+'
1988%left '*' '/'
1989%left NEG
1990%right '^'
1991
38a92d50 1992%% /* The grammar follows. */
342b8b6e
AD
1993@end example
1994
9edcd895
AD
1995@noindent
1996Note there are no declarations specific to locations. Defining a data
1997type for storing locations is not needed: we will use the type provided
1998by default (@pxref{Location Type, ,Data Types of Locations}), which is a
1999four member structure with the following integer fields:
2000@code{first_line}, @code{first_column}, @code{last_line} and
cd48d21d
AD
2001@code{last_column}. By conventions, and in accordance with the GNU
2002Coding Standards and common practice, the line and column count both
2003start at 1.
342b8b6e
AD
2004
2005@node Ltcalc Rules
2006@subsection Grammar Rules for @code{ltcalc}
2007
9edcd895
AD
2008Whether handling locations or not has no effect on the syntax of your
2009language. Therefore, grammar rules for this example will be very close
2010to those of the previous example: we will only modify them to benefit
2011from the new information.
342b8b6e 2012
9edcd895
AD
2013Here, we will use locations to report divisions by zero, and locate the
2014wrong expressions or subexpressions.
342b8b6e
AD
2015
2016@example
2017@group
2018input : /* empty */
2019 | input line
2020;
2021@end group
2022
2023@group
2024line : '\n'
2025 | exp '\n' @{ printf ("%d\n", $1); @}
2026;
2027@end group
2028
2029@group
2030exp : NUM @{ $$ = $1; @}
2031 | exp '+' exp @{ $$ = $1 + $3; @}
2032 | exp '-' exp @{ $$ = $1 - $3; @}
2033 | exp '*' exp @{ $$ = $1 * $3; @}
2034@end group
342b8b6e 2035@group
9edcd895 2036 | exp '/' exp
342b8b6e
AD
2037 @{
2038 if ($3)
2039 $$ = $1 / $3;
2040 else
2041 @{
2042 $$ = 1;
9edcd895
AD
2043 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2044 @@3.first_line, @@3.first_column,
2045 @@3.last_line, @@3.last_column);
342b8b6e
AD
2046 @}
2047 @}
2048@end group
2049@group
2050 | '-' exp %preg NEG @{ $$ = -$2; @}
2051 | exp '^' exp @{ $$ = pow ($1, $3); @}
2052 | '(' exp ')' @{ $$ = $2; @}
2053@end group
2054@end example
2055
2056This code shows how to reach locations inside of semantic actions, by
2057using the pseudo-variables @code{@@@var{n}} for rule components, and the
2058pseudo-variable @code{@@$} for groupings.
2059
9edcd895
AD
2060We don't need to assign a value to @code{@@$}: the output parser does it
2061automatically. By default, before executing the C code of each action,
2062@code{@@$} is set to range from the beginning of @code{@@1} to the end
2063of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2064can be redefined (@pxref{Location Default Action, , Default Action for
2065Locations}), and for very specific rules, @code{@@$} can be computed by
2066hand.
342b8b6e
AD
2067
2068@node Ltcalc Lexer
2069@subsection The @code{ltcalc} Lexical Analyzer.
2070
9edcd895 2071Until now, we relied on Bison's defaults to enable location
72d2299c 2072tracking. The next step is to rewrite the lexical analyzer, and make it
9edcd895
AD
2073able to feed the parser with the token locations, as it already does for
2074semantic values.
342b8b6e 2075
9edcd895
AD
2076To this end, we must take into account every single character of the
2077input text, to avoid the computed locations of being fuzzy or wrong:
342b8b6e
AD
2078
2079@example
2080@group
2081int
2082yylex (void)
2083@{
2084 int c;
18b519c0 2085@end group
342b8b6e 2086
18b519c0 2087@group
72d2299c 2088 /* Skip white space. */
342b8b6e
AD
2089 while ((c = getchar ()) == ' ' || c == '\t')
2090 ++yylloc.last_column;
18b519c0 2091@end group
342b8b6e 2092
18b519c0 2093@group
72d2299c 2094 /* Step. */
342b8b6e
AD
2095 yylloc.first_line = yylloc.last_line;
2096 yylloc.first_column = yylloc.last_column;
2097@end group
2098
2099@group
72d2299c 2100 /* Process numbers. */
342b8b6e
AD
2101 if (isdigit (c))
2102 @{
2103 yylval = c - '0';
2104 ++yylloc.last_column;
2105 while (isdigit (c = getchar ()))
2106 @{
2107 ++yylloc.last_column;
2108 yylval = yylval * 10 + c - '0';
2109 @}
2110 ungetc (c, stdin);
2111 return NUM;
2112 @}
2113@end group
2114
72d2299c 2115 /* Return end-of-input. */
342b8b6e
AD
2116 if (c == EOF)
2117 return 0;
2118
72d2299c 2119 /* Return a single char, and update location. */
342b8b6e
AD
2120 if (c == '\n')
2121 @{
2122 ++yylloc.last_line;
2123 yylloc.last_column = 0;
2124 @}
2125 else
2126 ++yylloc.last_column;
2127 return c;
2128@}
2129@end example
2130
9edcd895
AD
2131Basically, the lexical analyzer performs the same processing as before:
2132it skips blanks and tabs, and reads numbers or single-character tokens.
2133In addition, it updates @code{yylloc}, the global variable (of type
2134@code{YYLTYPE}) containing the token's location.
342b8b6e 2135
9edcd895 2136Now, each time this function returns a token, the parser has its number
72d2299c 2137as well as its semantic value, and its location in the text. The last
9edcd895
AD
2138needed change is to initialize @code{yylloc}, for example in the
2139controlling function:
342b8b6e
AD
2140
2141@example
9edcd895 2142@group
342b8b6e
AD
2143int
2144main (void)
2145@{
2146 yylloc.first_line = yylloc.last_line = 1;
2147 yylloc.first_column = yylloc.last_column = 0;
2148 return yyparse ();
2149@}
9edcd895 2150@end group
342b8b6e
AD
2151@end example
2152
9edcd895
AD
2153Remember that computing locations is not a matter of syntax. Every
2154character must be associated to a location update, whether it is in
2155valid input, in comments, in literal strings, and so on.
342b8b6e
AD
2156
2157@node Multi-function Calc
bfa74976
RS
2158@section Multi-Function Calculator: @code{mfcalc}
2159@cindex multi-function calculator
2160@cindex @code{mfcalc}
2161@cindex calculator, multi-function
2162
2163Now that the basics of Bison have been discussed, it is time to move on to
2164a more advanced problem. The above calculators provided only five
2165functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2166be nice to have a calculator that provides other mathematical functions such
2167as @code{sin}, @code{cos}, etc.
2168
2169It is easy to add new operators to the infix calculator as long as they are
2170only single-character literals. The lexical analyzer @code{yylex} passes
9d9b8b70 2171back all nonnumeric characters as tokens, so new grammar rules suffice for
bfa74976
RS
2172adding a new operator. But we want something more flexible: built-in
2173functions whose syntax has this form:
2174
2175@example
2176@var{function_name} (@var{argument})
2177@end example
2178
2179@noindent
2180At the same time, we will add memory to the calculator, by allowing you
2181to create named variables, store values in them, and use them later.
2182Here is a sample session with the multi-function calculator:
2183
2184@example
9edcd895
AD
2185$ @kbd{mfcalc}
2186@kbd{pi = 3.141592653589}
bfa74976 21873.1415926536
9edcd895 2188@kbd{sin(pi)}
bfa74976 21890.0000000000
9edcd895 2190@kbd{alpha = beta1 = 2.3}
bfa74976 21912.3000000000
9edcd895 2192@kbd{alpha}
bfa74976 21932.3000000000
9edcd895 2194@kbd{ln(alpha)}
bfa74976 21950.8329091229
9edcd895 2196@kbd{exp(ln(beta1))}
bfa74976 21972.3000000000
9edcd895 2198$
bfa74976
RS
2199@end example
2200
2201Note that multiple assignment and nested function calls are permitted.
2202
2203@menu
2204* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
2205* Rules: Mfcalc Rules. Grammar rules for the calculator.
2206* Symtab: Mfcalc Symtab. Symbol table management subroutines.
2207@end menu
2208
342b8b6e 2209@node Mfcalc Decl
bfa74976
RS
2210@subsection Declarations for @code{mfcalc}
2211
2212Here are the C and Bison declarations for the multi-function calculator.
2213
2214@smallexample
18b519c0 2215@group
bfa74976 2216%@{
38a92d50
PE
2217 #include <math.h> /* For math functions, cos(), sin(), etc. */
2218 #include "calc.h" /* Contains definition of `symrec'. */
2219 int yylex (void);
2220 void yyerror (char const *);
bfa74976 2221%@}
18b519c0
AD
2222@end group
2223@group
bfa74976 2224%union @{
38a92d50
PE
2225 double val; /* For returning numbers. */
2226 symrec *tptr; /* For returning symbol-table pointers. */
bfa74976 2227@}
18b519c0 2228@end group
38a92d50
PE
2229%token <val> NUM /* Simple double precision number. */
2230%token <tptr> VAR FNCT /* Variable and Function. */
bfa74976
RS
2231%type <val> exp
2232
18b519c0 2233@group
bfa74976
RS
2234%right '='
2235%left '-' '+'
2236%left '*' '/'
38a92d50
PE
2237%left NEG /* negation--unary minus */
2238%right '^' /* exponentiation */
18b519c0 2239@end group
38a92d50 2240%% /* The grammar follows. */
bfa74976
RS
2241@end smallexample
2242
2243The above grammar introduces only two new features of the Bison language.
2244These features allow semantic values to have various data types
2245(@pxref{Multiple Types, ,More Than One Value Type}).
2246
2247The @code{%union} declaration specifies the entire list of possible types;
2248this is instead of defining @code{YYSTYPE}. The allowable types are now
2249double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2250the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2251
2252Since values can now have various types, it is necessary to associate a
2253type with each grammar symbol whose semantic value is used. These symbols
2254are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2255declarations are augmented with information about their data type (placed
2256between angle brackets).
2257
704a47c4
AD
2258The Bison construct @code{%type} is used for declaring nonterminal
2259symbols, just as @code{%token} is used for declaring token types. We
2260have not used @code{%type} before because nonterminal symbols are
2261normally declared implicitly by the rules that define them. But
2262@code{exp} must be declared explicitly so we can specify its value type.
2263@xref{Type Decl, ,Nonterminal Symbols}.
bfa74976 2264
342b8b6e 2265@node Mfcalc Rules
bfa74976
RS
2266@subsection Grammar Rules for @code{mfcalc}
2267
2268Here are the grammar rules for the multi-function calculator.
2269Most of them are copied directly from @code{calc}; three rules,
2270those which mention @code{VAR} or @code{FNCT}, are new.
2271
2272@smallexample
18b519c0 2273@group
bfa74976
RS
2274input: /* empty */
2275 | input line
2276;
18b519c0 2277@end group
bfa74976 2278
18b519c0 2279@group
bfa74976
RS
2280line:
2281 '\n'
2282 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2283 | error '\n' @{ yyerrok; @}
2284;
18b519c0 2285@end group
bfa74976 2286
18b519c0 2287@group
bfa74976
RS
2288exp: NUM @{ $$ = $1; @}
2289 | VAR @{ $$ = $1->value.var; @}
2290 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2291 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2292 | exp '+' exp @{ $$ = $1 + $3; @}
2293 | exp '-' exp @{ $$ = $1 - $3; @}
2294 | exp '*' exp @{ $$ = $1 * $3; @}
2295 | exp '/' exp @{ $$ = $1 / $3; @}
2296 | '-' exp %prec NEG @{ $$ = -$2; @}
2297 | exp '^' exp @{ $$ = pow ($1, $3); @}
2298 | '(' exp ')' @{ $$ = $2; @}
2299;
18b519c0 2300@end group
38a92d50 2301/* End of grammar. */
bfa74976
RS
2302%%
2303@end smallexample
2304
342b8b6e 2305@node Mfcalc Symtab
bfa74976
RS
2306@subsection The @code{mfcalc} Symbol Table
2307@cindex symbol table example
2308
2309The multi-function calculator requires a symbol table to keep track of the
2310names and meanings of variables and functions. This doesn't affect the
2311grammar rules (except for the actions) or the Bison declarations, but it
2312requires some additional C functions for support.
2313
2314The symbol table itself consists of a linked list of records. Its
2315definition, which is kept in the header @file{calc.h}, is as follows. It
2316provides for either functions or variables to be placed in the table.
2317
2318@smallexample
2319@group
38a92d50 2320/* Function type. */
32dfccf8 2321typedef double (*func_t) (double);
72f889cc 2322@end group
32dfccf8 2323
72f889cc 2324@group
38a92d50 2325/* Data type for links in the chain of symbols. */
bfa74976
RS
2326struct symrec
2327@{
38a92d50 2328 char *name; /* name of symbol */
bfa74976 2329 int type; /* type of symbol: either VAR or FNCT */
32dfccf8
AD
2330 union
2331 @{
38a92d50
PE
2332 double var; /* value of a VAR */
2333 func_t fnctptr; /* value of a FNCT */
bfa74976 2334 @} value;
38a92d50 2335 struct symrec *next; /* link field */
bfa74976
RS
2336@};
2337@end group
2338
2339@group
2340typedef struct symrec symrec;
2341
38a92d50 2342/* The symbol table: a chain of `struct symrec'. */
bfa74976
RS
2343extern symrec *sym_table;
2344
a730d142 2345symrec *putsym (char const *, int);
38a92d50 2346symrec *getsym (char const *);
bfa74976
RS
2347@end group
2348@end smallexample
2349
2350The new version of @code{main} includes a call to @code{init_table}, a
2351function that initializes the symbol table. Here it is, and
2352@code{init_table} as well:
2353
2354@smallexample
bfa74976
RS
2355#include <stdio.h>
2356
18b519c0 2357@group
38a92d50 2358/* Called by yyparse on error. */
13863333 2359void
38a92d50 2360yyerror (char const *s)
bfa74976
RS
2361@{
2362 printf ("%s\n", s);
2363@}
18b519c0 2364@end group
bfa74976 2365
18b519c0 2366@group
bfa74976
RS
2367struct init
2368@{
38a92d50
PE
2369 char const *fname;
2370 double (*fnct) (double);
bfa74976
RS
2371@};
2372@end group
2373
2374@group
38a92d50 2375struct init const arith_fncts[] =
13863333 2376@{
32dfccf8
AD
2377 "sin", sin,
2378 "cos", cos,
13863333 2379 "atan", atan,
32dfccf8
AD
2380 "ln", log,
2381 "exp", exp,
13863333
AD
2382 "sqrt", sqrt,
2383 0, 0
2384@};
18b519c0 2385@end group
bfa74976 2386
18b519c0 2387@group
bfa74976 2388/* The symbol table: a chain of `struct symrec'. */
38a92d50 2389symrec *sym_table;
bfa74976
RS
2390@end group
2391
2392@group
72d2299c 2393/* Put arithmetic functions in table. */
13863333
AD
2394void
2395init_table (void)
bfa74976
RS
2396@{
2397 int i;
2398 symrec *ptr;
2399 for (i = 0; arith_fncts[i].fname != 0; i++)
2400 @{
2401 ptr = putsym (arith_fncts[i].fname, FNCT);
2402 ptr->value.fnctptr = arith_fncts[i].fnct;
2403 @}
2404@}
2405@end group
38a92d50
PE
2406
2407@group
2408int
2409main (void)
2410@{
2411 init_table ();
2412 return yyparse ();
2413@}
2414@end group
bfa74976
RS
2415@end smallexample
2416
2417By simply editing the initialization list and adding the necessary include
2418files, you can add additional functions to the calculator.
2419
2420Two important functions allow look-up and installation of symbols in the
2421symbol table. The function @code{putsym} is passed a name and the type
2422(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2423linked to the front of the list, and a pointer to the object is returned.
2424The function @code{getsym} is passed the name of the symbol to look up. If
2425found, a pointer to that symbol is returned; otherwise zero is returned.
2426
2427@smallexample
2428symrec *
38a92d50 2429putsym (char const *sym_name, int sym_type)
bfa74976
RS
2430@{
2431 symrec *ptr;
2432 ptr = (symrec *) malloc (sizeof (symrec));
2433 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2434 strcpy (ptr->name,sym_name);
2435 ptr->type = sym_type;
72d2299c 2436 ptr->value.var = 0; /* Set value to 0 even if fctn. */
bfa74976
RS
2437 ptr->next = (struct symrec *)sym_table;
2438 sym_table = ptr;
2439 return ptr;
2440@}
2441
2442symrec *
38a92d50 2443getsym (char const *sym_name)
bfa74976
RS
2444@{
2445 symrec *ptr;
2446 for (ptr = sym_table; ptr != (symrec *) 0;
2447 ptr = (symrec *)ptr->next)
2448 if (strcmp (ptr->name,sym_name) == 0)
2449 return ptr;
2450 return 0;
2451@}
2452@end smallexample
2453
2454The function @code{yylex} must now recognize variables, numeric values, and
2455the single-character arithmetic operators. Strings of alphanumeric
9d9b8b70 2456characters with a leading letter are recognized as either variables or
bfa74976
RS
2457functions depending on what the symbol table says about them.
2458
2459The string is passed to @code{getsym} for look up in the symbol table. If
2460the name appears in the table, a pointer to its location and its type
2461(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2462already in the table, then it is installed as a @code{VAR} using
2463@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
e0c471a9 2464returned to @code{yyparse}.
bfa74976
RS
2465
2466No change is needed in the handling of numeric values and arithmetic
2467operators in @code{yylex}.
2468
2469@smallexample
2470@group
2471#include <ctype.h>
18b519c0 2472@end group
13863333 2473
18b519c0 2474@group
13863333
AD
2475int
2476yylex (void)
bfa74976
RS
2477@{
2478 int c;
2479
72d2299c 2480 /* Ignore white space, get first nonwhite character. */
bfa74976
RS
2481 while ((c = getchar ()) == ' ' || c == '\t');
2482
2483 if (c == EOF)
2484 return 0;
2485@end group
2486
2487@group
2488 /* Char starts a number => parse the number. */
2489 if (c == '.' || isdigit (c))
2490 @{
2491 ungetc (c, stdin);
2492 scanf ("%lf", &yylval.val);
2493 return NUM;
2494 @}
2495@end group
2496
2497@group
2498 /* Char starts an identifier => read the name. */
2499 if (isalpha (c))
2500 @{
2501 symrec *s;
2502 static char *symbuf = 0;
2503 static int length = 0;
2504 int i;
2505@end group
2506
2507@group
2508 /* Initially make the buffer long enough
2509 for a 40-character symbol name. */
2510 if (length == 0)
2511 length = 40, symbuf = (char *)malloc (length + 1);
2512
2513 i = 0;
2514 do
2515@end group
2516@group
2517 @{
2518 /* If buffer is full, make it bigger. */
2519 if (i == length)
2520 @{
2521 length *= 2;
18b519c0 2522 symbuf = (char *) realloc (symbuf, length + 1);
bfa74976
RS
2523 @}
2524 /* Add this character to the buffer. */
2525 symbuf[i++] = c;
2526 /* Get another character. */
2527 c = getchar ();
2528 @}
2529@end group
2530@group
72d2299c 2531 while (isalnum (c));
bfa74976
RS
2532
2533 ungetc (c, stdin);
2534 symbuf[i] = '\0';
2535@end group
2536
2537@group
2538 s = getsym (symbuf);
2539 if (s == 0)
2540 s = putsym (symbuf, VAR);
2541 yylval.tptr = s;
2542 return s->type;
2543 @}
2544
2545 /* Any other character is a token by itself. */
2546 return c;
2547@}
2548@end group
2549@end smallexample
2550
72d2299c 2551This program is both powerful and flexible. You may easily add new
704a47c4
AD
2552functions, and it is a simple job to modify this code to install
2553predefined variables such as @code{pi} or @code{e} as well.
bfa74976 2554
342b8b6e 2555@node Exercises
bfa74976
RS
2556@section Exercises
2557@cindex exercises
2558
2559@enumerate
2560@item
2561Add some new functions from @file{math.h} to the initialization list.
2562
2563@item
2564Add another array that contains constants and their values. Then
2565modify @code{init_table} to add these constants to the symbol table.
2566It will be easiest to give the constants type @code{VAR}.
2567
2568@item
2569Make the program report an error if the user refers to an
2570uninitialized variable in any way except to store a value in it.
2571@end enumerate
2572
342b8b6e 2573@node Grammar File
bfa74976
RS
2574@chapter Bison Grammar Files
2575
2576Bison takes as input a context-free grammar specification and produces a
2577C-language function that recognizes correct instances of the grammar.
2578
2579The Bison grammar input file conventionally has a name ending in @samp{.y}.
234a3be3 2580@xref{Invocation, ,Invoking Bison}.
bfa74976
RS
2581
2582@menu
2583* Grammar Outline:: Overall layout of the grammar file.
2584* Symbols:: Terminal and nonterminal symbols.
2585* Rules:: How to write grammar rules.
2586* Recursion:: Writing recursive rules.
2587* Semantics:: Semantic values and actions.
847bf1f5 2588* Locations:: Locations and actions.
bfa74976
RS
2589* Declarations:: All kinds of Bison declarations are described here.
2590* Multiple Parsers:: Putting more than one Bison parser in one program.
2591@end menu
2592
342b8b6e 2593@node Grammar Outline
bfa74976
RS
2594@section Outline of a Bison Grammar
2595
2596A Bison grammar file has four main sections, shown here with the
2597appropriate delimiters:
2598
2599@example
2600%@{
38a92d50 2601 @var{Prologue}
bfa74976
RS
2602%@}
2603
2604@var{Bison declarations}
2605
2606%%
2607@var{Grammar rules}
2608%%
2609
75f5aaea 2610@var{Epilogue}
bfa74976
RS
2611@end example
2612
2613Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2bfc2e2a
PE
2614As a @acronym{GNU} extension, @samp{//} introduces a comment that
2615continues until end of line.
bfa74976
RS
2616
2617@menu
75f5aaea 2618* Prologue:: Syntax and usage of the prologue.
bfa74976
RS
2619* Bison Declarations:: Syntax and usage of the Bison declarations section.
2620* Grammar Rules:: Syntax and usage of the grammar rules section.
75f5aaea 2621* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
2622@end menu
2623
38a92d50 2624@node Prologue
75f5aaea
MA
2625@subsection The prologue
2626@cindex declarations section
2627@cindex Prologue
2628@cindex declarations
bfa74976 2629
f8e1c9e5
AD
2630The @var{Prologue} section contains macro definitions and declarations
2631of functions and variables that are used in the actions in the grammar
2632rules. These are copied to the beginning of the parser file so that
2633they precede the definition of @code{yyparse}. You can use
2634@samp{#include} to get the declarations from a header file. If you
2635don't need any C declarations, you may omit the @samp{%@{} and
2636@samp{%@}} delimiters that bracket this section.
bfa74976 2637
287c78f6
PE
2638The @var{Prologue} section is terminated by the the first occurrence
2639of @samp{%@}} that is outside a comment, a string literal, or a
2640character constant.
2641
c732d2c6
AD
2642You may have more than one @var{Prologue} section, intermixed with the
2643@var{Bison declarations}. This allows you to have C and Bison
2644declarations that refer to each other. For example, the @code{%union}
2645declaration may use types defined in a header file, and you may wish to
2646prototype functions that take arguments of type @code{YYSTYPE}. This
2647can be done with two @var{Prologue} blocks, one before and one after the
2648@code{%union} declaration.
2649
2650@smallexample
2651%@{
38a92d50
PE
2652 #include <stdio.h>
2653 #include "ptypes.h"
c732d2c6
AD
2654%@}
2655
2656%union @{
779e7ceb 2657 long int n;
c732d2c6
AD
2658 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2659@}
2660
2661%@{
38a92d50
PE
2662 static void print_token_value (FILE *, int, YYSTYPE);
2663 #define YYPRINT(F, N, L) print_token_value (F, N, L)
c732d2c6
AD
2664%@}
2665
2666@dots{}
2667@end smallexample
2668
34f98f46
JD
2669@findex %before-header
2670@findex %start-header
2671@findex %after-header
9bc0dd67
JD
2672If you've instructed Bison to generate a header file (@pxref{Table of Symbols,
2673,%defines}), you probably want @code{#include "ptypes.h"} to appear
2674in that header file as well.
34f98f46
JD
2675In that case, use @code{%before-header}, @code{%start-header}, and
2676@code{%after-header} instead of @var{Prologue} sections
2677(@pxref{Table of Symbols, ,%start-header}):
9bc0dd67
JD
2678
2679@smallexample
34f98f46 2680%before-header @{
9bc0dd67 2681 #include <stdio.h>
34f98f46 2682@}
9bc0dd67 2683
34f98f46 2684%start-header @{
9bc0dd67
JD
2685 #include "ptypes.h"
2686@}
2687%union @{
2688 long int n;
2689 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2690@}
2691
34f98f46 2692%after-header @{
9bc0dd67
JD
2693 static void print_token_value (FILE *, int, YYSTYPE);
2694 #define YYPRINT(F, N, L) print_token_value (F, N, L)
34f98f46 2695@}
9bc0dd67
JD
2696
2697@dots{}
2698@end smallexample
2699
342b8b6e 2700@node Bison Declarations
bfa74976
RS
2701@subsection The Bison Declarations Section
2702@cindex Bison declarations (introduction)
2703@cindex declarations, Bison (introduction)
2704
2705The @var{Bison declarations} section contains declarations that define
2706terminal and nonterminal symbols, specify precedence, and so on.
2707In some simple grammars you may not need any declarations.
2708@xref{Declarations, ,Bison Declarations}.
2709
342b8b6e 2710@node Grammar Rules
bfa74976
RS
2711@subsection The Grammar Rules Section
2712@cindex grammar rules section
2713@cindex rules section for grammar
2714
2715The @dfn{grammar rules} section contains one or more Bison grammar
2716rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
2717
2718There must always be at least one grammar rule, and the first
2719@samp{%%} (which precedes the grammar rules) may never be omitted even
2720if it is the first thing in the file.
2721
38a92d50 2722@node Epilogue
75f5aaea 2723@subsection The epilogue
bfa74976 2724@cindex additional C code section
75f5aaea 2725@cindex epilogue
bfa74976
RS
2726@cindex C code, section for additional
2727
08e49d20
PE
2728The @var{Epilogue} is copied verbatim to the end of the parser file, just as
2729the @var{Prologue} is copied to the beginning. This is the most convenient
342b8b6e
AD
2730place to put anything that you want to have in the parser file but which need
2731not come before the definition of @code{yyparse}. For example, the
38a92d50
PE
2732definitions of @code{yylex} and @code{yyerror} often go here. Because
2733C requires functions to be declared before being used, you often need
2734to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
e4f85c39 2735even if you define them in the Epilogue.
75f5aaea 2736@xref{Interface, ,Parser C-Language Interface}.
bfa74976
RS
2737
2738If the last section is empty, you may omit the @samp{%%} that separates it
2739from the grammar rules.
2740
f8e1c9e5
AD
2741The Bison parser itself contains many macros and identifiers whose names
2742start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
2743any such names (except those documented in this manual) in the epilogue
2744of the grammar file.
bfa74976 2745
342b8b6e 2746@node Symbols
bfa74976
RS
2747@section Symbols, Terminal and Nonterminal
2748@cindex nonterminal symbol
2749@cindex terminal symbol
2750@cindex token type
2751@cindex symbol
2752
2753@dfn{Symbols} in Bison grammars represent the grammatical classifications
2754of the language.
2755
2756A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
2757class of syntactically equivalent tokens. You use the symbol in grammar
2758rules to mean that a token in that class is allowed. The symbol is
2759represented in the Bison parser by a numeric code, and the @code{yylex}
f8e1c9e5
AD
2760function returns a token type code to indicate what kind of token has
2761been read. You don't need to know what the code value is; you can use
2762the symbol to stand for it.
bfa74976 2763
f8e1c9e5
AD
2764A @dfn{nonterminal symbol} stands for a class of syntactically
2765equivalent groupings. The symbol name is used in writing grammar rules.
2766By convention, it should be all lower case.
bfa74976
RS
2767
2768Symbol names can contain letters, digits (not at the beginning),
2769underscores and periods. Periods make sense only in nonterminals.
2770
931c7513 2771There are three ways of writing terminal symbols in the grammar:
bfa74976
RS
2772
2773@itemize @bullet
2774@item
2775A @dfn{named token type} is written with an identifier, like an
c827f760 2776identifier in C@. By convention, it should be all upper case. Each
bfa74976
RS
2777such name must be defined with a Bison declaration such as
2778@code{%token}. @xref{Token Decl, ,Token Type Names}.
2779
2780@item
2781@cindex character token
2782@cindex literal token
2783@cindex single-character literal
931c7513
RS
2784A @dfn{character token type} (or @dfn{literal character token}) is
2785written in the grammar using the same syntax used in C for character
2786constants; for example, @code{'+'} is a character token type. A
2787character token type doesn't need to be declared unless you need to
2788specify its semantic value data type (@pxref{Value Type, ,Data Types of
2789Semantic Values}), associativity, or precedence (@pxref{Precedence,
2790,Operator Precedence}).
bfa74976
RS
2791
2792By convention, a character token type is used only to represent a
2793token that consists of that particular character. Thus, the token
2794type @code{'+'} is used to represent the character @samp{+} as a
2795token. Nothing enforces this convention, but if you depart from it,
2796your program will confuse other readers.
2797
2798All the usual escape sequences used in character literals in C can be
2799used in Bison as well, but you must not use the null character as a
72d2299c
PE
2800character literal because its numeric code, zero, signifies
2801end-of-input (@pxref{Calling Convention, ,Calling Convention
2bfc2e2a
PE
2802for @code{yylex}}). Also, unlike standard C, trigraphs have no
2803special meaning in Bison character literals, nor is backslash-newline
2804allowed.
931c7513
RS
2805
2806@item
2807@cindex string token
2808@cindex literal string token
9ecbd125 2809@cindex multicharacter literal
931c7513
RS
2810A @dfn{literal string token} is written like a C string constant; for
2811example, @code{"<="} is a literal string token. A literal string token
2812doesn't need to be declared unless you need to specify its semantic
14ded682 2813value data type (@pxref{Value Type}), associativity, or precedence
931c7513
RS
2814(@pxref{Precedence}).
2815
2816You can associate the literal string token with a symbolic name as an
2817alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
2818Declarations}). If you don't do that, the lexical analyzer has to
2819retrieve the token number for the literal string token from the
2820@code{yytname} table (@pxref{Calling Convention}).
2821
c827f760 2822@strong{Warning}: literal string tokens do not work in Yacc.
931c7513
RS
2823
2824By convention, a literal string token is used only to represent a token
2825that consists of that particular string. Thus, you should use the token
2826type @code{"<="} to represent the string @samp{<=} as a token. Bison
9ecbd125 2827does not enforce this convention, but if you depart from it, people who
931c7513
RS
2828read your program will be confused.
2829
2830All the escape sequences used in string literals in C can be used in
92ac3705
PE
2831Bison as well, except that you must not use a null character within a
2832string literal. Also, unlike Standard C, trigraphs have no special
2bfc2e2a
PE
2833meaning in Bison string literals, nor is backslash-newline allowed. A
2834literal string token must contain two or more characters; for a token
2835containing just one character, use a character token (see above).
bfa74976
RS
2836@end itemize
2837
2838How you choose to write a terminal symbol has no effect on its
2839grammatical meaning. That depends only on where it appears in rules and
2840on when the parser function returns that symbol.
2841
72d2299c
PE
2842The value returned by @code{yylex} is always one of the terminal
2843symbols, except that a zero or negative value signifies end-of-input.
2844Whichever way you write the token type in the grammar rules, you write
2845it the same way in the definition of @code{yylex}. The numeric code
2846for a character token type is simply the positive numeric code of the
2847character, so @code{yylex} can use the identical value to generate the
2848requisite code, though you may need to convert it to @code{unsigned
2849char} to avoid sign-extension on hosts where @code{char} is signed.
2850Each named token type becomes a C macro in
bfa74976 2851the parser file, so @code{yylex} can use the name to stand for the code.
13863333 2852(This is why periods don't make sense in terminal symbols.)
bfa74976
RS
2853@xref{Calling Convention, ,Calling Convention for @code{yylex}}.
2854
2855If @code{yylex} is defined in a separate file, you need to arrange for the
2856token-type macro definitions to be available there. Use the @samp{-d}
2857option when you run Bison, so that it will write these macro definitions
2858into a separate header file @file{@var{name}.tab.h} which you can include
2859in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
2860
72d2299c 2861If you want to write a grammar that is portable to any Standard C
9d9b8b70 2862host, you must use only nonnull character tokens taken from the basic
c827f760 2863execution character set of Standard C@. This set consists of the ten
72d2299c
PE
2864digits, the 52 lower- and upper-case English letters, and the
2865characters in the following C-language string:
2866
2867@example
2868"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
2869@end example
2870
f8e1c9e5
AD
2871The @code{yylex} function and Bison must use a consistent character set
2872and encoding for character tokens. For example, if you run Bison in an
2873@acronym{ASCII} environment, but then compile and run the resulting
2874program in an environment that uses an incompatible character set like
2875@acronym{EBCDIC}, the resulting program may not work because the tables
2876generated by Bison will assume @acronym{ASCII} numeric values for
2877character tokens. It is standard practice for software distributions to
2878contain C source files that were generated by Bison in an
2879@acronym{ASCII} environment, so installers on platforms that are
2880incompatible with @acronym{ASCII} must rebuild those files before
2881compiling them.
e966383b 2882
bfa74976
RS
2883The symbol @code{error} is a terminal symbol reserved for error recovery
2884(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
23c5a174
AD
2885In particular, @code{yylex} should never return this value. The default
2886value of the error token is 256, unless you explicitly assigned 256 to
2887one of your tokens with a @code{%token} declaration.
bfa74976 2888
342b8b6e 2889@node Rules
bfa74976
RS
2890@section Syntax of Grammar Rules
2891@cindex rule syntax
2892@cindex grammar rule syntax
2893@cindex syntax of grammar rules
2894
2895A Bison grammar rule has the following general form:
2896
2897@example
e425e872 2898@group
bfa74976
RS
2899@var{result}: @var{components}@dots{}
2900 ;
e425e872 2901@end group
bfa74976
RS
2902@end example
2903
2904@noindent
9ecbd125 2905where @var{result} is the nonterminal symbol that this rule describes,
bfa74976 2906and @var{components} are various terminal and nonterminal symbols that
13863333 2907are put together by this rule (@pxref{Symbols}).
bfa74976
RS
2908
2909For example,
2910
2911@example
2912@group
2913exp: exp '+' exp
2914 ;
2915@end group
2916@end example
2917
2918@noindent
2919says that two groupings of type @code{exp}, with a @samp{+} token in between,
2920can be combined into a larger grouping of type @code{exp}.
2921
72d2299c
PE
2922White space in rules is significant only to separate symbols. You can add
2923extra white space as you wish.
bfa74976
RS
2924
2925Scattered among the components can be @var{actions} that determine
2926the semantics of the rule. An action looks like this:
2927
2928@example
2929@{@var{C statements}@}
2930@end example
2931
2932@noindent
287c78f6
PE
2933@cindex braced code
2934This is an example of @dfn{braced code}, that is, C code surrounded by
2935braces, much like a compound statement in C@. Braced code can contain
2936any sequence of C tokens, so long as its braces are balanced. Bison
2937does not check the braced code for correctness directly; it merely
2938copies the code to the output file, where the C compiler can check it.
2939
2940Within braced code, the balanced-brace count is not affected by braces
2941within comments, string literals, or character constants, but it is
2942affected by the C digraphs @samp{<%} and @samp{%>} that represent
2943braces. At the top level braced code must be terminated by @samp{@}}
2944and not by a digraph. Bison does not look for trigraphs, so if braced
2945code uses trigraphs you should ensure that they do not affect the
2946nesting of braces or the boundaries of comments, string literals, or
2947character constants.
2948
bfa74976
RS
2949Usually there is only one action and it follows the components.
2950@xref{Actions}.
2951
2952@findex |
2953Multiple rules for the same @var{result} can be written separately or can
2954be joined with the vertical-bar character @samp{|} as follows:
2955
bfa74976
RS
2956@example
2957@group
2958@var{result}: @var{rule1-components}@dots{}
2959 | @var{rule2-components}@dots{}
2960 @dots{}
2961 ;
2962@end group
2963@end example
bfa74976
RS
2964
2965@noindent
2966They are still considered distinct rules even when joined in this way.
2967
2968If @var{components} in a rule is empty, it means that @var{result} can
2969match the empty string. For example, here is how to define a
2970comma-separated sequence of zero or more @code{exp} groupings:
2971
2972@example
2973@group
2974expseq: /* empty */
2975 | expseq1
2976 ;
2977@end group
2978
2979@group
2980expseq1: exp
2981 | expseq1 ',' exp
2982 ;
2983@end group
2984@end example
2985
2986@noindent
2987It is customary to write a comment @samp{/* empty */} in each rule
2988with no components.
2989
342b8b6e 2990@node Recursion
bfa74976
RS
2991@section Recursive Rules
2992@cindex recursive rule
2993
f8e1c9e5
AD
2994A rule is called @dfn{recursive} when its @var{result} nonterminal
2995appears also on its right hand side. Nearly all Bison grammars need to
2996use recursion, because that is the only way to define a sequence of any
2997number of a particular thing. Consider this recursive definition of a
9ecbd125 2998comma-separated sequence of one or more expressions:
bfa74976
RS
2999
3000@example
3001@group
3002expseq1: exp
3003 | expseq1 ',' exp
3004 ;
3005@end group
3006@end example
3007
3008@cindex left recursion
3009@cindex right recursion
3010@noindent
3011Since the recursive use of @code{expseq1} is the leftmost symbol in the
3012right hand side, we call this @dfn{left recursion}. By contrast, here
3013the same construct is defined using @dfn{right recursion}:
3014
3015@example
3016@group
3017expseq1: exp
3018 | exp ',' expseq1
3019 ;
3020@end group
3021@end example
3022
3023@noindent
ec3bc396
AD
3024Any kind of sequence can be defined using either left recursion or right
3025recursion, but you should always use left recursion, because it can
3026parse a sequence of any number of elements with bounded stack space.
3027Right recursion uses up space on the Bison stack in proportion to the
3028number of elements in the sequence, because all the elements must be
3029shifted onto the stack before the rule can be applied even once.
3030@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3031of this.
bfa74976
RS
3032
3033@cindex mutual recursion
3034@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3035rule does not appear directly on its right hand side, but does appear
3036in rules for other nonterminals which do appear on its right hand
13863333 3037side.
bfa74976
RS
3038
3039For example:
3040
3041@example
3042@group
3043expr: primary
3044 | primary '+' primary
3045 ;
3046@end group
3047
3048@group
3049primary: constant
3050 | '(' expr ')'
3051 ;
3052@end group
3053@end example
3054
3055@noindent
3056defines two mutually-recursive nonterminals, since each refers to the
3057other.
3058
342b8b6e 3059@node Semantics
bfa74976
RS
3060@section Defining Language Semantics
3061@cindex defining language semantics
13863333 3062@cindex language semantics, defining
bfa74976
RS
3063
3064The grammar rules for a language determine only the syntax. The semantics
3065are determined by the semantic values associated with various tokens and
3066groupings, and by the actions taken when various groupings are recognized.
3067
3068For example, the calculator calculates properly because the value
3069associated with each expression is the proper number; it adds properly
3070because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3071the numbers associated with @var{x} and @var{y}.
3072
3073@menu
3074* Value Type:: Specifying one data type for all semantic values.
3075* Multiple Types:: Specifying several alternative data types.
3076* Actions:: An action is the semantic definition of a grammar rule.
3077* Action Types:: Specifying data types for actions to operate on.
3078* Mid-Rule Actions:: Most actions go at the end of a rule.
3079 This says when, why and how to use the exceptional
3080 action in the middle of a rule.
3081@end menu
3082
342b8b6e 3083@node Value Type
bfa74976
RS
3084@subsection Data Types of Semantic Values
3085@cindex semantic value type
3086@cindex value type, semantic
3087@cindex data types of semantic values
3088@cindex default data type
3089
3090In a simple program it may be sufficient to use the same data type for
3091the semantic values of all language constructs. This was true in the
c827f760 3092@acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
1964ad8c 3093Notation Calculator}).
bfa74976
RS
3094
3095Bison's default is to use type @code{int} for all semantic values. To
3096specify some other type, define @code{YYSTYPE} as a macro, like this:
3097
3098@example
3099#define YYSTYPE double
3100@end example
3101
3102@noindent
50cce58e
PE
3103@code{YYSTYPE}'s replacement list should be a type name
3104that does not contain parentheses or square brackets.
342b8b6e 3105This macro definition must go in the prologue of the grammar file
75f5aaea 3106(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
bfa74976 3107
342b8b6e 3108@node Multiple Types
bfa74976
RS
3109@subsection More Than One Value Type
3110
3111In most programs, you will need different data types for different kinds
3112of tokens and groupings. For example, a numeric constant may need type
f8e1c9e5
AD
3113@code{int} or @code{long int}, while a string constant needs type
3114@code{char *}, and an identifier might need a pointer to an entry in the
3115symbol table.
bfa74976
RS
3116
3117To use more than one data type for semantic values in one parser, Bison
3118requires you to do two things:
3119
3120@itemize @bullet
3121@item
3122Specify the entire collection of possible data types, with the
704a47c4
AD
3123@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
3124Value Types}).
bfa74976
RS
3125
3126@item
14ded682
AD
3127Choose one of those types for each symbol (terminal or nonterminal) for
3128which semantic values are used. This is done for tokens with the
3129@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3130and for groupings with the @code{%type} Bison declaration (@pxref{Type
3131Decl, ,Nonterminal Symbols}).
bfa74976
RS
3132@end itemize
3133
342b8b6e 3134@node Actions
bfa74976
RS
3135@subsection Actions
3136@cindex action
3137@vindex $$
3138@vindex $@var{n}
3139
3140An action accompanies a syntactic rule and contains C code to be executed
3141each time an instance of that rule is recognized. The task of most actions
3142is to compute a semantic value for the grouping built by the rule from the
3143semantic values associated with tokens or smaller groupings.
3144
287c78f6
PE
3145An action consists of braced code containing C statements, and can be
3146placed at any position in the rule;
704a47c4
AD
3147it is executed at that position. Most rules have just one action at the
3148end of the rule, following all the components. Actions in the middle of
3149a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3150Actions, ,Actions in Mid-Rule}).
bfa74976
RS
3151
3152The C code in an action can refer to the semantic values of the components
3153matched by the rule with the construct @code{$@var{n}}, which stands for
3154the value of the @var{n}th component. The semantic value for the grouping
0cc3da3a
PE
3155being constructed is @code{$$}. Bison translates both of these
3156constructs into expressions of the appropriate type when it copies the
3157actions into the parser file. @code{$$} is translated to a modifiable
3158lvalue, so it can be assigned to.
bfa74976
RS
3159
3160Here is a typical example:
3161
3162@example
3163@group
3164exp: @dots{}
3165 | exp '+' exp
3166 @{ $$ = $1 + $3; @}
3167@end group
3168@end example
3169
3170@noindent
3171This rule constructs an @code{exp} from two smaller @code{exp} groupings
3172connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3173refer to the semantic values of the two component @code{exp} groupings,
3174which are the first and third symbols on the right hand side of the rule.
3175The sum is stored into @code{$$} so that it becomes the semantic value of
3176the addition-expression just recognized by the rule. If there were a
3177useful semantic value associated with the @samp{+} token, it could be
e0c471a9 3178referred to as @code{$2}.
bfa74976 3179
3ded9a63
AD
3180Note that the vertical-bar character @samp{|} is really a rule
3181separator, and actions are attached to a single rule. This is a
3182difference with tools like Flex, for which @samp{|} stands for either
3183``or'', or ``the same action as that of the next rule''. In the
3184following example, the action is triggered only when @samp{b} is found:
3185
3186@example
3187@group
3188a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3189@end group
3190@end example
3191
bfa74976
RS
3192@cindex default action
3193If you don't specify an action for a rule, Bison supplies a default:
72f889cc
AD
3194@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3195becomes the value of the whole rule. Of course, the default action is
3196valid only if the two data types match. There is no meaningful default
3197action for an empty rule; every empty rule must have an explicit action
3198unless the rule's value does not matter.
bfa74976
RS
3199
3200@code{$@var{n}} with @var{n} zero or negative is allowed for reference
3201to tokens and groupings on the stack @emph{before} those that match the
3202current rule. This is a very risky practice, and to use it reliably
3203you must be certain of the context in which the rule is applied. Here
3204is a case in which you can use this reliably:
3205
3206@example
3207@group
3208foo: expr bar '+' expr @{ @dots{} @}
3209 | expr bar '-' expr @{ @dots{} @}
3210 ;
3211@end group
3212
3213@group
3214bar: /* empty */
3215 @{ previous_expr = $0; @}
3216 ;
3217@end group
3218@end example
3219
3220As long as @code{bar} is used only in the fashion shown here, @code{$0}
3221always refers to the @code{expr} which precedes @code{bar} in the
3222definition of @code{foo}.
3223
32c29292 3224@vindex yylval
742e4900 3225It is also possible to access the semantic value of the lookahead token, if
32c29292
JD
3226any, from a semantic action.
3227This semantic value is stored in @code{yylval}.
3228@xref{Action Features, ,Special Features for Use in Actions}.
3229
342b8b6e 3230@node Action Types
bfa74976
RS
3231@subsection Data Types of Values in Actions
3232@cindex action data types
3233@cindex data types in actions
3234
3235If you have chosen a single data type for semantic values, the @code{$$}
3236and @code{$@var{n}} constructs always have that data type.
3237
3238If you have used @code{%union} to specify a variety of data types, then you
3239must declare a choice among these types for each terminal or nonterminal
3240symbol that can have a semantic value. Then each time you use @code{$$} or
3241@code{$@var{n}}, its data type is determined by which symbol it refers to
e0c471a9 3242in the rule. In this example,
bfa74976
RS
3243
3244@example
3245@group
3246exp: @dots{}
3247 | exp '+' exp
3248 @{ $$ = $1 + $3; @}
3249@end group
3250@end example
3251
3252@noindent
3253@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3254have the data type declared for the nonterminal symbol @code{exp}. If
3255@code{$2} were used, it would have the data type declared for the
e0c471a9 3256terminal symbol @code{'+'}, whatever that might be.
bfa74976
RS
3257
3258Alternatively, you can specify the data type when you refer to the value,
3259by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3260reference. For example, if you have defined types as shown here:
3261
3262@example
3263@group
3264%union @{
3265 int itype;
3266 double dtype;
3267@}
3268@end group
3269@end example
3270
3271@noindent
3272then you can write @code{$<itype>1} to refer to the first subunit of the
3273rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3274
342b8b6e 3275@node Mid-Rule Actions
bfa74976
RS
3276@subsection Actions in Mid-Rule
3277@cindex actions in mid-rule
3278@cindex mid-rule actions
3279
3280Occasionally it is useful to put an action in the middle of a rule.
3281These actions are written just like usual end-of-rule actions, but they
3282are executed before the parser even recognizes the following components.
3283
3284A mid-rule action may refer to the components preceding it using
3285@code{$@var{n}}, but it may not refer to subsequent components because
3286it is run before they are parsed.
3287
3288The mid-rule action itself counts as one of the components of the rule.
3289This makes a difference when there is another action later in the same rule
3290(and usually there is another at the end): you have to count the actions
3291along with the symbols when working out which number @var{n} to use in
3292@code{$@var{n}}.
3293
3294The mid-rule action can also have a semantic value. The action can set
3295its value with an assignment to @code{$$}, and actions later in the rule
3296can refer to the value using @code{$@var{n}}. Since there is no symbol
3297to name the action, there is no way to declare a data type for the value
fdc6758b
MA
3298in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3299specify a data type each time you refer to this value.
bfa74976
RS
3300
3301There is no way to set the value of the entire rule with a mid-rule
3302action, because assignments to @code{$$} do not have that effect. The
3303only way to set the value for the entire rule is with an ordinary action
3304at the end of the rule.
3305
3306Here is an example from a hypothetical compiler, handling a @code{let}
3307statement that looks like @samp{let (@var{variable}) @var{statement}} and
3308serves to create a variable named @var{variable} temporarily for the
3309duration of @var{statement}. To parse this construct, we must put
3310@var{variable} into the symbol table while @var{statement} is parsed, then
3311remove it afterward. Here is how it is done:
3312
3313@example
3314@group
3315stmt: LET '(' var ')'
3316 @{ $<context>$ = push_context ();
3317 declare_variable ($3); @}
3318 stmt @{ $$ = $6;
3319 pop_context ($<context>5); @}
3320@end group
3321@end example
3322
3323@noindent
3324As soon as @samp{let (@var{variable})} has been recognized, the first
3325action is run. It saves a copy of the current semantic context (the
3326list of accessible variables) as its semantic value, using alternative
3327@code{context} in the data-type union. Then it calls
3328@code{declare_variable} to add the new variable to that list. Once the
3329first action is finished, the embedded statement @code{stmt} can be
3330parsed. Note that the mid-rule action is component number 5, so the
3331@samp{stmt} is component number 6.
3332
3333After the embedded statement is parsed, its semantic value becomes the
3334value of the entire @code{let}-statement. Then the semantic value from the
3335earlier action is used to restore the prior list of variables. This
3336removes the temporary @code{let}-variable from the list so that it won't
3337appear to exist while the rest of the program is parsed.
3338
841a7737
JD
3339@findex %destructor
3340@cindex discarded symbols, mid-rule actions
3341@cindex error recovery, mid-rule actions
3342In the above example, if the parser initiates error recovery (@pxref{Error
3343Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3344it might discard the previous semantic context @code{$<context>5} without
3345restoring it.
3346Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3347Discarded Symbols}).
3348However, Bison currently provides no means to declare a destructor for a
3349mid-rule action's semantic value.
3350
3351One solution is to bury the mid-rule action inside a nonterminal symbol and to
3352declare a destructor for that symbol:
3353
3354@example
3355@group
3356%type <context> let
3357%destructor @{ pop_context ($$); @} let
3358
3359%%
3360
3361stmt: let stmt
3362 @{ $$ = $2;
3363 pop_context ($1); @}
3364 ;
3365
3366let: LET '(' var ')'
3367 @{ $$ = push_context ();
3368 declare_variable ($3); @}
3369 ;
3370
3371@end group
3372@end example
3373
3374@noindent
3375Note that the action is now at the end of its rule.
3376Any mid-rule action can be converted to an end-of-rule action in this way, and
3377this is what Bison actually does to implement mid-rule actions.
3378
bfa74976
RS
3379Taking action before a rule is completely recognized often leads to
3380conflicts since the parser must commit to a parse in order to execute the
3381action. For example, the following two rules, without mid-rule actions,
3382can coexist in a working parser because the parser can shift the open-brace
3383token and look at what follows before deciding whether there is a
3384declaration or not:
3385
3386@example
3387@group
3388compound: '@{' declarations statements '@}'
3389 | '@{' statements '@}'
3390 ;
3391@end group
3392@end example
3393
3394@noindent
3395But when we add a mid-rule action as follows, the rules become nonfunctional:
3396
3397@example
3398@group
3399compound: @{ prepare_for_local_variables (); @}
3400 '@{' declarations statements '@}'
3401@end group
3402@group
3403 | '@{' statements '@}'
3404 ;
3405@end group
3406@end example
3407
3408@noindent
3409Now the parser is forced to decide whether to run the mid-rule action
3410when it has read no farther than the open-brace. In other words, it
3411must commit to using one rule or the other, without sufficient
3412information to do it correctly. (The open-brace token is what is called
742e4900
JD
3413the @dfn{lookahead} token at this time, since the parser is still
3414deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.)
bfa74976
RS
3415
3416You might think that you could correct the problem by putting identical
3417actions into the two rules, like this:
3418
3419@example
3420@group
3421compound: @{ prepare_for_local_variables (); @}
3422 '@{' declarations statements '@}'
3423 | @{ prepare_for_local_variables (); @}
3424 '@{' statements '@}'
3425 ;
3426@end group
3427@end example
3428
3429@noindent
3430But this does not help, because Bison does not realize that the two actions
3431are identical. (Bison never tries to understand the C code in an action.)
3432
3433If the grammar is such that a declaration can be distinguished from a
3434statement by the first token (which is true in C), then one solution which
3435does work is to put the action after the open-brace, like this:
3436
3437@example
3438@group
3439compound: '@{' @{ prepare_for_local_variables (); @}
3440 declarations statements '@}'
3441 | '@{' statements '@}'
3442 ;
3443@end group
3444@end example
3445
3446@noindent
3447Now the first token of the following declaration or statement,
3448which would in any case tell Bison which rule to use, can still do so.
3449
3450Another solution is to bury the action inside a nonterminal symbol which
3451serves as a subroutine:
3452
3453@example
3454@group
3455subroutine: /* empty */
3456 @{ prepare_for_local_variables (); @}
3457 ;
3458
3459@end group
3460
3461@group
3462compound: subroutine
3463 '@{' declarations statements '@}'
3464 | subroutine
3465 '@{' statements '@}'
3466 ;
3467@end group
3468@end example
3469
3470@noindent
3471Now Bison can execute the action in the rule for @code{subroutine} without
841a7737 3472deciding which rule for @code{compound} it will eventually use.
bfa74976 3473
342b8b6e 3474@node Locations
847bf1f5
AD
3475@section Tracking Locations
3476@cindex location
95923bd6
AD
3477@cindex textual location
3478@cindex location, textual
847bf1f5
AD
3479
3480Though grammar rules and semantic actions are enough to write a fully
72d2299c 3481functional parser, it can be useful to process some additional information,
3e259915
MA
3482especially symbol locations.
3483
704a47c4
AD
3484The way locations are handled is defined by providing a data type, and
3485actions to take when rules are matched.
847bf1f5
AD
3486
3487@menu
3488* Location Type:: Specifying a data type for locations.
3489* Actions and Locations:: Using locations in actions.
3490* Location Default Action:: Defining a general way to compute locations.
3491@end menu
3492
342b8b6e 3493@node Location Type
847bf1f5
AD
3494@subsection Data Type of Locations
3495@cindex data type of locations
3496@cindex default location type
3497
3498Defining a data type for locations is much simpler than for semantic values,
3499since all tokens and groupings always use the same type.
3500
50cce58e
PE
3501You can specify the type of locations by defining a macro called
3502@code{YYLTYPE}, just as you can specify the semantic value type by
3503defining @code{YYSTYPE} (@pxref{Value Type}).
847bf1f5
AD
3504When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3505four members:
3506
3507@example
6273355b 3508typedef struct YYLTYPE
847bf1f5
AD
3509@{
3510 int first_line;
3511 int first_column;
3512 int last_line;
3513 int last_column;
6273355b 3514@} YYLTYPE;
847bf1f5
AD
3515@end example
3516
cd48d21d
AD
3517At the beginning of the parsing, Bison initializes all these fields to 1
3518for @code{yylloc}.
3519
342b8b6e 3520@node Actions and Locations
847bf1f5
AD
3521@subsection Actions and Locations
3522@cindex location actions
3523@cindex actions, location
3524@vindex @@$
3525@vindex @@@var{n}
3526
3527Actions are not only useful for defining language semantics, but also for
3528describing the behavior of the output parser with locations.
3529
3530The most obvious way for building locations of syntactic groupings is very
72d2299c 3531similar to the way semantic values are computed. In a given rule, several
847bf1f5
AD
3532constructs can be used to access the locations of the elements being matched.
3533The location of the @var{n}th component of the right hand side is
3534@code{@@@var{n}}, while the location of the left hand side grouping is
3535@code{@@$}.
3536
3e259915 3537Here is a basic example using the default data type for locations:
847bf1f5
AD
3538
3539@example
3540@group
3541exp: @dots{}
3e259915 3542 | exp '/' exp
847bf1f5 3543 @{
3e259915
MA
3544 @@$.first_column = @@1.first_column;
3545 @@$.first_line = @@1.first_line;
847bf1f5
AD
3546 @@$.last_column = @@3.last_column;
3547 @@$.last_line = @@3.last_line;
3e259915
MA
3548 if ($3)
3549 $$ = $1 / $3;
3550 else
3551 @{
3552 $$ = 1;
4e03e201
AD
3553 fprintf (stderr,
3554 "Division by zero, l%d,c%d-l%d,c%d",
3555 @@3.first_line, @@3.first_column,
3556 @@3.last_line, @@3.last_column);
3e259915 3557 @}
847bf1f5
AD
3558 @}
3559@end group
3560@end example
3561
3e259915 3562As for semantic values, there is a default action for locations that is
72d2299c 3563run each time a rule is matched. It sets the beginning of @code{@@$} to the
3e259915 3564beginning of the first symbol, and the end of @code{@@$} to the end of the
79282c6c 3565last symbol.
3e259915 3566
72d2299c 3567With this default action, the location tracking can be fully automatic. The
3e259915
MA
3568example above simply rewrites this way:
3569
3570@example
3571@group
3572exp: @dots{}
3573 | exp '/' exp
3574 @{
3575 if ($3)
3576 $$ = $1 / $3;
3577 else
3578 @{
3579 $$ = 1;
4e03e201
AD
3580 fprintf (stderr,
3581 "Division by zero, l%d,c%d-l%d,c%d",
3582 @@3.first_line, @@3.first_column,
3583 @@3.last_line, @@3.last_column);
3e259915
MA
3584 @}
3585 @}
3586@end group
3587@end example
847bf1f5 3588
32c29292 3589@vindex yylloc
742e4900 3590It is also possible to access the location of the lookahead token, if any,
32c29292
JD
3591from a semantic action.
3592This location is stored in @code{yylloc}.
3593@xref{Action Features, ,Special Features for Use in Actions}.
3594
342b8b6e 3595@node Location Default Action
847bf1f5
AD
3596@subsection Default Action for Locations
3597@vindex YYLLOC_DEFAULT
8710fc41 3598@cindex @acronym{GLR} parsers and @code{YYLLOC_DEFAULT}
847bf1f5 3599
72d2299c 3600Actually, actions are not the best place to compute locations. Since
704a47c4
AD
3601locations are much more general than semantic values, there is room in
3602the output parser to redefine the default action to take for each
72d2299c 3603rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
96b93a3d
PE
3604matched, before the associated action is run. It is also invoked
3605while processing a syntax error, to compute the error's location.
8710fc41
JD
3606Before reporting an unresolvable syntactic ambiguity, a @acronym{GLR}
3607parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
3608of that ambiguity.
847bf1f5 3609
3e259915 3610Most of the time, this macro is general enough to suppress location
79282c6c 3611dedicated code from semantic actions.
847bf1f5 3612
72d2299c 3613The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
96b93a3d 3614the location of the grouping (the result of the computation). When a
766de5eb 3615rule is matched, the second parameter identifies locations of
96b93a3d 3616all right hand side elements of the rule being matched, and the third
8710fc41
JD
3617parameter is the size of the rule's right hand side.
3618When a @acronym{GLR} parser reports an ambiguity, which of multiple candidate
3619right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
3620When processing a syntax error, the second parameter identifies locations
3621of the symbols that were discarded during error processing, and the third
96b93a3d 3622parameter is the number of discarded symbols.
847bf1f5 3623
766de5eb 3624By default, @code{YYLLOC_DEFAULT} is defined this way:
847bf1f5 3625
766de5eb 3626@smallexample
847bf1f5 3627@group
766de5eb
PE
3628# define YYLLOC_DEFAULT(Current, Rhs, N) \
3629 do \
3630 if (N) \
3631 @{ \
3632 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
3633 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
3634 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
3635 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
3636 @} \
3637 else \
3638 @{ \
3639 (Current).first_line = (Current).last_line = \
3640 YYRHSLOC(Rhs, 0).last_line; \
3641 (Current).first_column = (Current).last_column = \
3642 YYRHSLOC(Rhs, 0).last_column; \
3643 @} \
3644 while (0)
847bf1f5 3645@end group
766de5eb 3646@end smallexample
676385e2 3647
766de5eb
PE
3648where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
3649in @var{rhs} when @var{k} is positive, and the location of the symbol
f28ac696 3650just before the reduction when @var{k} and @var{n} are both zero.
676385e2 3651
3e259915 3652When defining @code{YYLLOC_DEFAULT}, you should consider that:
847bf1f5 3653
3e259915 3654@itemize @bullet
79282c6c 3655@item
72d2299c 3656All arguments are free of side-effects. However, only the first one (the
3e259915 3657result) should be modified by @code{YYLLOC_DEFAULT}.
847bf1f5 3658
3e259915 3659@item
766de5eb
PE
3660For consistency with semantic actions, valid indexes within the
3661right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
3662valid index, and it refers to the symbol just before the reduction.
3663During error processing @var{n} is always positive.
0ae99356
PE
3664
3665@item
3666Your macro should parenthesize its arguments, if need be, since the
3667actual arguments may not be surrounded by parentheses. Also, your
3668macro should expand to something that can be used as a single
3669statement when it is followed by a semicolon.
3e259915 3670@end itemize
847bf1f5 3671
342b8b6e 3672@node Declarations
bfa74976
RS
3673@section Bison Declarations
3674@cindex declarations, Bison
3675@cindex Bison declarations
3676
3677The @dfn{Bison declarations} section of a Bison grammar defines the symbols
3678used in formulating the grammar and the data types of semantic values.
3679@xref{Symbols}.
3680
3681All token type names (but not single-character literal tokens such as
3682@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
3683declared if you need to specify which data type to use for the semantic
3684value (@pxref{Multiple Types, ,More Than One Value Type}).
3685
3686The first rule in the file also specifies the start symbol, by default.
3687If you want some other symbol to be the start symbol, you must declare
704a47c4
AD
3688it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
3689Grammars}).
bfa74976
RS
3690
3691@menu
b50d2359 3692* Require Decl:: Requiring a Bison version.
bfa74976
RS
3693* Token Decl:: Declaring terminal symbols.
3694* Precedence Decl:: Declaring terminals with precedence and associativity.
3695* Union Decl:: Declaring the set of all semantic value types.
3696* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 3697* Initial Action Decl:: Code run before parsing starts.
72f889cc 3698* Destructor Decl:: Declaring how symbols are freed.
d6328241 3699* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
3700* Start Decl:: Specifying the start symbol.
3701* Pure Decl:: Requesting a reentrant parser.
3702* Decl Summary:: Table of all Bison declarations.
3703@end menu
3704
b50d2359
AD
3705@node Require Decl
3706@subsection Require a Version of Bison
3707@cindex version requirement
3708@cindex requiring a version of Bison
3709@findex %require
3710
3711You may require the minimum version of Bison to process the grammar. If
9b8a5ce0
AD
3712the requirement is not met, @command{bison} exits with an error (exit
3713status 63).
b50d2359
AD
3714
3715@example
3716%require "@var{version}"
3717@end example
3718
342b8b6e 3719@node Token Decl
bfa74976
RS
3720@subsection Token Type Names
3721@cindex declaring token type names
3722@cindex token type names, declaring
931c7513 3723@cindex declaring literal string tokens
bfa74976
RS
3724@findex %token
3725
3726The basic way to declare a token type name (terminal symbol) is as follows:
3727
3728@example
3729%token @var{name}
3730@end example
3731
3732Bison will convert this into a @code{#define} directive in
3733the parser, so that the function @code{yylex} (if it is in this file)
3734can use the name @var{name} to stand for this token type's code.
3735
14ded682
AD
3736Alternatively, you can use @code{%left}, @code{%right}, or
3737@code{%nonassoc} instead of @code{%token}, if you wish to specify
3738associativity and precedence. @xref{Precedence Decl, ,Operator
3739Precedence}.
bfa74976
RS
3740
3741You can explicitly specify the numeric code for a token type by appending
1452af69
PE
3742a decimal or hexadecimal integer value in the field immediately
3743following the token name:
bfa74976
RS
3744
3745@example
3746%token NUM 300
1452af69 3747%token XNUM 0x12d // a GNU extension
bfa74976
RS
3748@end example
3749
3750@noindent
3751It is generally best, however, to let Bison choose the numeric codes for
3752all token types. Bison will automatically select codes that don't conflict
e966383b 3753with each other or with normal characters.
bfa74976
RS
3754
3755In the event that the stack type is a union, you must augment the
3756@code{%token} or other token declaration to include the data type
704a47c4
AD
3757alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
3758Than One Value Type}).
bfa74976
RS
3759
3760For example:
3761
3762@example
3763@group
3764%union @{ /* define stack type */
3765 double val;
3766 symrec *tptr;
3767@}
3768%token <val> NUM /* define token NUM and its type */
3769@end group
3770@end example
3771
931c7513
RS
3772You can associate a literal string token with a token type name by
3773writing the literal string at the end of a @code{%token}
3774declaration which declares the name. For example:
3775
3776@example
3777%token arrow "=>"
3778@end example
3779
3780@noindent
3781For example, a grammar for the C language might specify these names with
3782equivalent literal string tokens:
3783
3784@example
3785%token <operator> OR "||"
3786%token <operator> LE 134 "<="
3787%left OR "<="
3788@end example
3789
3790@noindent
3791Once you equate the literal string and the token name, you can use them
3792interchangeably in further declarations or the grammar rules. The
3793@code{yylex} function can use the token name or the literal string to
3794obtain the token type code number (@pxref{Calling Convention}).
3795
342b8b6e 3796@node Precedence Decl
bfa74976
RS
3797@subsection Operator Precedence
3798@cindex precedence declarations
3799@cindex declaring operator precedence
3800@cindex operator precedence, declaring
3801
3802Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to
3803declare a token and specify its precedence and associativity, all at
3804once. These are called @dfn{precedence declarations}.
704a47c4
AD
3805@xref{Precedence, ,Operator Precedence}, for general information on
3806operator precedence.
bfa74976
RS
3807
3808The syntax of a precedence declaration is the same as that of
3809@code{%token}: either
3810
3811@example
3812%left @var{symbols}@dots{}
3813@end example
3814
3815@noindent
3816or
3817
3818@example
3819%left <@var{type}> @var{symbols}@dots{}
3820@end example
3821
3822And indeed any of these declarations serves the purposes of @code{%token}.
3823But in addition, they specify the associativity and relative precedence for
3824all the @var{symbols}:
3825
3826@itemize @bullet
3827@item
3828The associativity of an operator @var{op} determines how repeated uses
3829of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
3830@var{z}} is parsed by grouping @var{x} with @var{y} first or by
3831grouping @var{y} with @var{z} first. @code{%left} specifies
3832left-associativity (grouping @var{x} with @var{y} first) and
3833@code{%right} specifies right-associativity (grouping @var{y} with
3834@var{z} first). @code{%nonassoc} specifies no associativity, which
3835means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
3836considered a syntax error.
3837
3838@item
3839The precedence of an operator determines how it nests with other operators.
3840All the tokens declared in a single precedence declaration have equal
3841precedence and nest together according to their associativity.
3842When two tokens declared in different precedence declarations associate,
3843the one declared later has the higher precedence and is grouped first.
3844@end itemize
3845
342b8b6e 3846@node Union Decl
bfa74976
RS
3847@subsection The Collection of Value Types
3848@cindex declaring value types
3849@cindex value types, declaring
3850@findex %union
3851
287c78f6
PE
3852The @code{%union} declaration specifies the entire collection of
3853possible data types for semantic values. The keyword @code{%union} is
3854followed by braced code containing the same thing that goes inside a
3855@code{union} in C@.
bfa74976
RS
3856
3857For example:
3858
3859@example
3860@group
3861%union @{
3862 double val;
3863 symrec *tptr;
3864@}
3865@end group
3866@end example
3867
3868@noindent
3869This says that the two alternative types are @code{double} and @code{symrec
3870*}. They are given names @code{val} and @code{tptr}; these names are used
3871in the @code{%token} and @code{%type} declarations to pick one of the types
3872for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
3873
6273355b
PE
3874As an extension to @acronym{POSIX}, a tag is allowed after the
3875@code{union}. For example:
3876
3877@example
3878@group
3879%union value @{
3880 double val;
3881 symrec *tptr;
3882@}
3883@end group
3884@end example
3885
d6ca7905 3886@noindent
6273355b
PE
3887specifies the union tag @code{value}, so the corresponding C type is
3888@code{union value}. If you do not specify a tag, it defaults to
3889@code{YYSTYPE}.
3890
d6ca7905
PE
3891As another extension to @acronym{POSIX}, you may specify multiple
3892@code{%union} declarations; their contents are concatenated. However,
3893only the first @code{%union} declaration can specify a tag.
3894
6273355b 3895Note that, unlike making a @code{union} declaration in C, you need not write
bfa74976
RS
3896a semicolon after the closing brace.
3897
342b8b6e 3898@node Type Decl
bfa74976
RS
3899@subsection Nonterminal Symbols
3900@cindex declaring value types, nonterminals
3901@cindex value types, nonterminals, declaring
3902@findex %type
3903
3904@noindent
3905When you use @code{%union} to specify multiple value types, you must
3906declare the value type of each nonterminal symbol for which values are
3907used. This is done with a @code{%type} declaration, like this:
3908
3909@example
3910%type <@var{type}> @var{nonterminal}@dots{}
3911@end example
3912
3913@noindent
704a47c4
AD
3914Here @var{nonterminal} is the name of a nonterminal symbol, and
3915@var{type} is the name given in the @code{%union} to the alternative
3916that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
3917can give any number of nonterminal symbols in the same @code{%type}
3918declaration, if they have the same value type. Use spaces to separate
3919the symbol names.
bfa74976 3920
931c7513
RS
3921You can also declare the value type of a terminal symbol. To do this,
3922use the same @code{<@var{type}>} construction in a declaration for the
3923terminal symbol. All kinds of token declarations allow
3924@code{<@var{type}>}.
3925
18d192f0
AD
3926@node Initial Action Decl
3927@subsection Performing Actions before Parsing
3928@findex %initial-action
3929
3930Sometimes your parser needs to perform some initializations before
3931parsing. The @code{%initial-action} directive allows for such arbitrary
3932code.
3933
3934@deffn {Directive} %initial-action @{ @var{code} @}
3935@findex %initial-action
287c78f6 3936Declare that the braced @var{code} must be invoked before parsing each time
451364ed 3937@code{yyparse} is called. The @var{code} may use @code{$$} and
742e4900 3938@code{@@$} --- initial value and location of the lookahead --- and the
451364ed 3939@code{%parse-param}.
18d192f0
AD
3940@end deffn
3941
451364ed
AD
3942For instance, if your locations use a file name, you may use
3943
3944@example
48b16bbc 3945%parse-param @{ char const *file_name @};
451364ed
AD
3946%initial-action
3947@{
4626a15d 3948 @@$.initialize (file_name);
451364ed
AD
3949@};
3950@end example
3951
18d192f0 3952
72f889cc
AD
3953@node Destructor Decl
3954@subsection Freeing Discarded Symbols
3955@cindex freeing discarded symbols
3956@findex %destructor
3957
a85284cf
AD
3958During error recovery (@pxref{Error Recovery}), symbols already pushed
3959on the stack and tokens coming from the rest of the file are discarded
3960until the parser falls on its feet. If the parser runs out of memory,
9d9b8b70 3961or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
a85284cf
AD
3962symbols on the stack must be discarded. Even if the parser succeeds, it
3963must discard the start symbol.
258b75ca
PE
3964
3965When discarded symbols convey heap based information, this memory is
3966lost. While this behavior can be tolerable for batch parsers, such as
4b367315
AD
3967in traditional compilers, it is unacceptable for programs like shells or
3968protocol implementations that may parse and execute indefinitely.
258b75ca 3969
a85284cf
AD
3970The @code{%destructor} directive defines code that is called when a
3971symbol is automatically discarded.
72f889cc
AD
3972
3973@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
3974@findex %destructor
287c78f6
PE
3975Invoke the braced @var{code} whenever the parser discards one of the
3976@var{symbols}.
4b367315
AD
3977Within @var{code}, @code{$$} designates the semantic value associated
3978with the discarded symbol. The additional parser parameters are also
3979available (@pxref{Parser Function, , The Parser Function
3980@code{yyparse}}).
72f889cc
AD
3981@end deffn
3982
3983For instance:
3984
3985@smallexample
3986%union
3987@{
3988 char *string;
3989@}
3990%token <string> STRING
3991%type <string> string
3992%destructor @{ free ($$); @} STRING string
3993@end smallexample
3994
3995@noindent
258b75ca 3996guarantees that when a @code{STRING} or a @code{string} is discarded,
72f889cc
AD
3997its associated memory will be freed.
3998
e757bb10
AD
3999@sp 1
4000
4001@cindex discarded symbols
4002@dfn{Discarded symbols} are the following:
4003
4004@itemize
4005@item
4006stacked symbols popped during the first phase of error recovery,
4007@item
4008incoming terminals during the second phase of error recovery,
4009@item
742e4900 4010the current lookahead and the entire stack (except the current
9d9b8b70 4011right-hand side symbols) when the parser returns immediately, and
258b75ca
PE
4012@item
4013the start symbol, when the parser succeeds.
e757bb10
AD
4014@end itemize
4015
9d9b8b70
PE
4016The parser can @dfn{return immediately} because of an explicit call to
4017@code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
4018exhaustion.
4019
4020Right-hand size symbols of a rule that explicitly triggers a syntax
4021error via @code{YYERROR} are not discarded automatically. As a rule
4022of thumb, destructors are invoked only when user actions cannot manage
a85284cf 4023the memory.
e757bb10 4024
342b8b6e 4025@node Expect Decl
bfa74976
RS
4026@subsection Suppressing Conflict Warnings
4027@cindex suppressing conflict warnings
4028@cindex preventing warnings about conflicts
4029@cindex warnings, preventing
4030@cindex conflicts, suppressing warnings of
4031@findex %expect
d6328241 4032@findex %expect-rr
bfa74976
RS
4033
4034Bison normally warns if there are any conflicts in the grammar
7da99ede
AD
4035(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4036have harmless shift/reduce conflicts which are resolved in a predictable
4037way and would be difficult to eliminate. It is desirable to suppress
4038the warning about these conflicts unless the number of conflicts
4039changes. You can do this with the @code{%expect} declaration.
bfa74976
RS
4040
4041The declaration looks like this:
4042
4043@example
4044%expect @var{n}
4045@end example
4046
035aa4a0
PE
4047Here @var{n} is a decimal integer. The declaration says there should
4048be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4049Bison reports an error if the number of shift/reduce conflicts differs
4050from @var{n}, or if there are any reduce/reduce conflicts.
bfa74976 4051
035aa4a0
PE
4052For normal @acronym{LALR}(1) parsers, reduce/reduce conflicts are more
4053serious, and should be eliminated entirely. Bison will always report
4054reduce/reduce conflicts for these parsers. With @acronym{GLR}
4055parsers, however, both kinds of conflicts are routine; otherwise,
4056there would be no need to use @acronym{GLR} parsing. Therefore, it is
4057also possible to specify an expected number of reduce/reduce conflicts
4058in @acronym{GLR} parsers, using the declaration:
d6328241
PH
4059
4060@example
4061%expect-rr @var{n}
4062@end example
4063
bfa74976
RS
4064In general, using @code{%expect} involves these steps:
4065
4066@itemize @bullet
4067@item
4068Compile your grammar without @code{%expect}. Use the @samp{-v} option
4069to get a verbose list of where the conflicts occur. Bison will also
4070print the number of conflicts.
4071
4072@item
4073Check each of the conflicts to make sure that Bison's default
4074resolution is what you really want. If not, rewrite the grammar and
4075go back to the beginning.
4076
4077@item
4078Add an @code{%expect} declaration, copying the number @var{n} from the
035aa4a0
PE
4079number which Bison printed. With @acronym{GLR} parsers, add an
4080@code{%expect-rr} declaration as well.
bfa74976
RS
4081@end itemize
4082
035aa4a0
PE
4083Now Bison will warn you if you introduce an unexpected conflict, but
4084will keep silent otherwise.
bfa74976 4085
342b8b6e 4086@node Start Decl
bfa74976
RS
4087@subsection The Start-Symbol
4088@cindex declaring the start symbol
4089@cindex start symbol, declaring
4090@cindex default start symbol
4091@findex %start
4092
4093Bison assumes by default that the start symbol for the grammar is the first
4094nonterminal specified in the grammar specification section. The programmer
4095may override this restriction with the @code{%start} declaration as follows:
4096
4097@example
4098%start @var{symbol}
4099@end example
4100
342b8b6e 4101@node Pure Decl
bfa74976
RS
4102@subsection A Pure (Reentrant) Parser
4103@cindex reentrant parser
4104@cindex pure parser
8c9a50be 4105@findex %pure-parser
bfa74976
RS
4106
4107A @dfn{reentrant} program is one which does not alter in the course of
4108execution; in other words, it consists entirely of @dfn{pure} (read-only)
4109code. Reentrancy is important whenever asynchronous execution is possible;
9d9b8b70
PE
4110for example, a nonreentrant program may not be safe to call from a signal
4111handler. In systems with multiple threads of control, a nonreentrant
bfa74976
RS
4112program must be called only within interlocks.
4113
70811b85 4114Normally, Bison generates a parser which is not reentrant. This is
c827f760
PE
4115suitable for most uses, and it permits compatibility with Yacc. (The
4116standard Yacc interfaces are inherently nonreentrant, because they use
70811b85
RS
4117statically allocated variables for communication with @code{yylex},
4118including @code{yylval} and @code{yylloc}.)
bfa74976 4119
70811b85 4120Alternatively, you can generate a pure, reentrant parser. The Bison
8c9a50be 4121declaration @code{%pure-parser} says that you want the parser to be
70811b85 4122reentrant. It looks like this:
bfa74976
RS
4123
4124@example
8c9a50be 4125%pure-parser
bfa74976
RS
4126@end example
4127
70811b85
RS
4128The result is that the communication variables @code{yylval} and
4129@code{yylloc} become local variables in @code{yyparse}, and a different
4130calling convention is used for the lexical analyzer function
4131@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
4132Parsers}, for the details of this. The variable @code{yynerrs} also
4133becomes local in @code{yyparse} (@pxref{Error Reporting, ,The Error
4134Reporting Function @code{yyerror}}). The convention for calling
4135@code{yyparse} itself is unchanged.
4136
4137Whether the parser is pure has nothing to do with the grammar rules.
4138You can generate either a pure parser or a nonreentrant parser from any
4139valid grammar.
bfa74976 4140
342b8b6e 4141@node Decl Summary
bfa74976
RS
4142@subsection Bison Declaration Summary
4143@cindex Bison declaration summary
4144@cindex declaration summary
4145@cindex summary, Bison declaration
4146
d8988b2f 4147Here is a summary of the declarations used to define a grammar:
bfa74976 4148
18b519c0 4149@deffn {Directive} %union
bfa74976
RS
4150Declare the collection of data types that semantic values may have
4151(@pxref{Union Decl, ,The Collection of Value Types}).
18b519c0 4152@end deffn
bfa74976 4153
18b519c0 4154@deffn {Directive} %token
bfa74976
RS
4155Declare a terminal symbol (token type name) with no precedence
4156or associativity specified (@pxref{Token Decl, ,Token Type Names}).
18b519c0 4157@end deffn
bfa74976 4158
18b519c0 4159@deffn {Directive} %right
bfa74976
RS
4160Declare a terminal symbol (token type name) that is right-associative
4161(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4162@end deffn
bfa74976 4163
18b519c0 4164@deffn {Directive} %left
bfa74976
RS
4165Declare a terminal symbol (token type name) that is left-associative
4166(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4167@end deffn
bfa74976 4168
18b519c0 4169@deffn {Directive} %nonassoc
bfa74976 4170Declare a terminal symbol (token type name) that is nonassociative
bfa74976 4171(@pxref{Precedence Decl, ,Operator Precedence}).
39a06c25
PE
4172Using it in a way that would be associative is a syntax error.
4173@end deffn
4174
91d2c560 4175@ifset defaultprec
39a06c25 4176@deffn {Directive} %default-prec
22fccf95 4177Assign a precedence to rules lacking an explicit @code{%prec} modifier
39a06c25
PE
4178(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4179@end deffn
91d2c560 4180@end ifset
bfa74976 4181
18b519c0 4182@deffn {Directive} %type
bfa74976
RS
4183Declare the type of semantic values for a nonterminal symbol
4184(@pxref{Type Decl, ,Nonterminal Symbols}).
18b519c0 4185@end deffn
bfa74976 4186
18b519c0 4187@deffn {Directive} %start
89cab50d
AD
4188Specify the grammar's start symbol (@pxref{Start Decl, ,The
4189Start-Symbol}).
18b519c0 4190@end deffn
bfa74976 4191
18b519c0 4192@deffn {Directive} %expect
bfa74976
RS
4193Declare the expected number of shift-reduce conflicts
4194(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
18b519c0
AD
4195@end deffn
4196
bfa74976 4197
d8988b2f
AD
4198@sp 1
4199@noindent
4200In order to change the behavior of @command{bison}, use the following
4201directives:
4202
18b519c0 4203@deffn {Directive} %debug
4947ebdb
PE
4204In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
4205already defined, so that the debugging facilities are compiled.
18b519c0 4206@end deffn
ec3bc396 4207@xref{Tracing, ,Tracing Your Parser}.
d8988b2f 4208
18b519c0 4209@deffn {Directive} %defines
4bfd5e4e
PE
4210Write a header file containing macro definitions for the token type
4211names defined in the grammar as well as a few other declarations.
d8988b2f 4212If the parser output file is named @file{@var{name}.c} then this file
e0c471a9 4213is named @file{@var{name}.h}.
d8988b2f 4214
4bfd5e4e 4215Unless @code{YYSTYPE} is already defined as a macro, the output header
5c9be03d 4216declares @code{YYSTYPE}. Therefore, if you are using a @code{%union}
f8e1c9e5
AD
4217(@pxref{Multiple Types, ,More Than One Value Type}) with components that
4218require other definitions, or if you have defined a @code{YYSTYPE} macro
4219(@pxref{Value Type, ,Data Types of Semantic Values}), you need to
4220arrange for these definitions to be propagated to all modules, e.g., by
4221putting them in a prerequisite header that is included both by your
4222parser and by any other module that needs @code{YYSTYPE}.
4bfd5e4e
PE
4223
4224Unless your parser is pure, the output header declares @code{yylval}
4225as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
4226Parser}.
4227
4228If you have also used locations, the output header declares
4229@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
4230@code{YYSTYPE} and @code{yylval}. @xref{Locations, ,Tracking
4231Locations}.
4232
f8e1c9e5
AD
4233This output file is normally essential if you wish to put the definition
4234of @code{yylex} in a separate source file, because @code{yylex}
4235typically needs to be able to refer to the above-mentioned declarations
4236and to the token type codes. @xref{Token Values, ,Semantic Values of
4237Tokens}.
9bc0dd67 4238
34f98f46
JD
4239@findex %start-header
4240@findex %end-header
4241If you have declared @code{%start-header} or @code{%end-header}, the output
4242header also contains their code.
4243@xref{Table of Symbols, ,%start-header}.
18b519c0 4244@end deffn
d8988b2f 4245
18b519c0 4246@deffn {Directive} %destructor
258b75ca 4247Specify how the parser should reclaim the memory associated to
fa7e68c3 4248discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 4249@end deffn
72f889cc 4250
18b519c0 4251@deffn {Directive} %file-prefix="@var{prefix}"
d8988b2f
AD
4252Specify a prefix to use for all Bison output file names. The names are
4253chosen as if the input file were named @file{@var{prefix}.y}.
18b519c0 4254@end deffn
d8988b2f 4255
18b519c0 4256@deffn {Directive} %locations
89cab50d
AD
4257Generate the code processing the locations (@pxref{Action Features,
4258,Special Features for Use in Actions}). This mode is enabled as soon as
4259the grammar uses the special @samp{@@@var{n}} tokens, but if your
4260grammar does not use it, using @samp{%locations} allows for more
6e649e65 4261accurate syntax error messages.
18b519c0 4262@end deffn
89cab50d 4263
18b519c0 4264@deffn {Directive} %name-prefix="@var{prefix}"
d8988b2f
AD
4265Rename the external symbols used in the parser so that they start with
4266@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
aa08666d 4267in C parsers
d8988b2f 4268is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
91e3ac9a
PE
4269@code{yylval}, @code{yychar}, @code{yydebug}, and
4270(if locations are used) @code{yylloc}. For example, if you use
2a8d363a 4271@samp{%name-prefix="c_"}, the names become @code{c_parse}, @code{c_lex},
aa08666d
AD
4272and so on. In C++ parsers, it is only the surrounding namespace which is
4273named @var{prefix} instead of @samp{yy}.
4274@xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
18b519c0 4275@end deffn
931c7513 4276
91d2c560 4277@ifset defaultprec
22fccf95
PE
4278@deffn {Directive} %no-default-prec
4279Do not assign a precedence to rules lacking an explicit @code{%prec}
4280modifier (@pxref{Contextual Precedence, ,Context-Dependent
4281Precedence}).
4282@end deffn
91d2c560 4283@end ifset
22fccf95 4284
18b519c0 4285@deffn {Directive} %no-parser
6deb4447
AD
4286Do not include any C code in the parser file; generate tables only. The
4287parser file contains just @code{#define} directives and static variable
4288declarations.
4289
4290This option also tells Bison to write the C code for the grammar actions
fa4d969f 4291into a file named @file{@var{file}.act}, in the form of a
6deb4447 4292brace-surrounded body fit for a @code{switch} statement.
18b519c0 4293@end deffn
6deb4447 4294
18b519c0 4295@deffn {Directive} %no-lines
931c7513
RS
4296Don't generate any @code{#line} preprocessor commands in the parser
4297file. Ordinarily Bison writes these commands in the parser file so that
4298the C compiler and debuggers will associate errors and object code with
4299your source file (the grammar file). This directive causes them to
4300associate errors with the parser file, treating it an independent source
4301file in its own right.
18b519c0 4302@end deffn
931c7513 4303
fa4d969f
PE
4304@deffn {Directive} %output="@var{file}"
4305Specify @var{file} for the parser file.
18b519c0 4306@end deffn
6deb4447 4307
18b519c0 4308@deffn {Directive} %pure-parser
d8988b2f
AD
4309Request a pure (reentrant) parser program (@pxref{Pure Decl, ,A Pure
4310(Reentrant) Parser}).
18b519c0 4311@end deffn
6deb4447 4312
b50d2359 4313@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
4314Require version @var{version} or higher of Bison. @xref{Require Decl, ,
4315Require a Version of Bison}.
b50d2359
AD
4316@end deffn
4317
18b519c0 4318@deffn {Directive} %token-table
931c7513
RS
4319Generate an array of token names in the parser file. The name of the
4320array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
3650b4b8 4321token whose internal Bison token code number is @var{i}. The first
f67ad422
PE
4322three elements of @code{yytname} correspond to the predefined tokens
4323@code{"$end"},
88bce5a2
AD
4324@code{"error"}, and @code{"$undefined"}; after these come the symbols
4325defined in the grammar file.
931c7513 4326
9e0876fb
PE
4327The name in the table includes all the characters needed to represent
4328the token in Bison. For single-character literals and literal
4329strings, this includes the surrounding quoting characters and any
4330escape sequences. For example, the Bison single-character literal
4331@code{'+'} corresponds to a three-character name, represented in C as
4332@code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
4333corresponds to a five-character name, represented in C as
4334@code{"\"\\\\/\""}.
931c7513 4335
8c9a50be 4336When you specify @code{%token-table}, Bison also generates macro
931c7513
RS
4337definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
4338@code{YYNRULES}, and @code{YYNSTATES}:
4339
4340@table @code
4341@item YYNTOKENS
4342The highest token number, plus one.
4343@item YYNNTS
9ecbd125 4344The number of nonterminal symbols.
931c7513
RS
4345@item YYNRULES
4346The number of grammar rules,
4347@item YYNSTATES
4348The number of parser states (@pxref{Parser States}).
4349@end table
18b519c0 4350@end deffn
d8988b2f 4351
18b519c0 4352@deffn {Directive} %verbose
d8988b2f 4353Write an extra output file containing verbose descriptions of the
742e4900 4354parser states and what is done for each type of lookahead token in
72d2299c 4355that state. @xref{Understanding, , Understanding Your Parser}, for more
ec3bc396 4356information.
18b519c0 4357@end deffn
d8988b2f 4358
18b519c0 4359@deffn {Directive} %yacc
d8988b2f
AD
4360Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
4361including its naming conventions. @xref{Bison Options}, for more.
18b519c0 4362@end deffn
d8988b2f
AD
4363
4364
342b8b6e 4365@node Multiple Parsers
bfa74976
RS
4366@section Multiple Parsers in the Same Program
4367
4368Most programs that use Bison parse only one language and therefore contain
4369only one Bison parser. But what if you want to parse more than one
4370language with the same program? Then you need to avoid a name conflict
4371between different definitions of @code{yyparse}, @code{yylval}, and so on.
4372
4373The easy way to do this is to use the option @samp{-p @var{prefix}}
704a47c4
AD
4374(@pxref{Invocation, ,Invoking Bison}). This renames the interface
4375functions and variables of the Bison parser to start with @var{prefix}
4376instead of @samp{yy}. You can use this to give each parser distinct
4377names that do not conflict.
bfa74976
RS
4378
4379The precise list of symbols renamed is @code{yyparse}, @code{yylex},
2a8d363a
AD
4380@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
4381@code{yychar} and @code{yydebug}. For example, if you use @samp{-p c},
4382the names become @code{cparse}, @code{clex}, and so on.
bfa74976
RS
4383
4384@strong{All the other variables and macros associated with Bison are not
4385renamed.} These others are not global; there is no conflict if the same
4386name is used in different parsers. For example, @code{YYSTYPE} is not
4387renamed, but defining this in different ways in different parsers causes
4388no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
4389
4390The @samp{-p} option works by adding macro definitions to the beginning
4391of the parser source file, defining @code{yyparse} as
4392@code{@var{prefix}parse}, and so on. This effectively substitutes one
4393name for the other in the entire parser file.
4394
342b8b6e 4395@node Interface
bfa74976
RS
4396@chapter Parser C-Language Interface
4397@cindex C-language interface
4398@cindex interface
4399
4400The Bison parser is actually a C function named @code{yyparse}. Here we
4401describe the interface conventions of @code{yyparse} and the other
4402functions that it needs to use.
4403
4404Keep in mind that the parser uses many C identifiers starting with
4405@samp{yy} and @samp{YY} for internal purposes. If you use such an
75f5aaea
MA
4406identifier (aside from those in this manual) in an action or in epilogue
4407in the grammar file, you are likely to run into trouble.
bfa74976
RS
4408
4409@menu
4410* Parser Function:: How to call @code{yyparse} and what it returns.
13863333 4411* Lexical:: You must supply a function @code{yylex}
bfa74976
RS
4412 which reads tokens.
4413* Error Reporting:: You must supply a function @code{yyerror}.
4414* Action Features:: Special features for use in actions.
f7ab6a50
PE
4415* Internationalization:: How to let the parser speak in the user's
4416 native language.
bfa74976
RS
4417@end menu
4418
342b8b6e 4419@node Parser Function
bfa74976
RS
4420@section The Parser Function @code{yyparse}
4421@findex yyparse
4422
4423You call the function @code{yyparse} to cause parsing to occur. This
4424function reads tokens, executes actions, and ultimately returns when it
4425encounters end-of-input or an unrecoverable syntax error. You can also
14ded682
AD
4426write an action which directs @code{yyparse} to return immediately
4427without reading further.
bfa74976 4428
2a8d363a
AD
4429
4430@deftypefun int yyparse (void)
bfa74976
RS
4431The value returned by @code{yyparse} is 0 if parsing was successful (return
4432is due to end-of-input).
4433
b47dbebe
PE
4434The value is 1 if parsing failed because of invalid input, i.e., input
4435that contains a syntax error or that causes @code{YYABORT} to be
4436invoked.
4437
4438The value is 2 if parsing failed due to memory exhaustion.
2a8d363a 4439@end deftypefun
bfa74976
RS
4440
4441In an action, you can cause immediate return from @code{yyparse} by using
4442these macros:
4443
2a8d363a 4444@defmac YYACCEPT
bfa74976
RS
4445@findex YYACCEPT
4446Return immediately with value 0 (to report success).
2a8d363a 4447@end defmac
bfa74976 4448
2a8d363a 4449@defmac YYABORT
bfa74976
RS
4450@findex YYABORT
4451Return immediately with value 1 (to report failure).
2a8d363a
AD
4452@end defmac
4453
4454If you use a reentrant parser, you can optionally pass additional
4455parameter information to it in a reentrant way. To do so, use the
4456declaration @code{%parse-param}:
4457
feeb0eda 4458@deffn {Directive} %parse-param @{@var{argument-declaration}@}
2a8d363a 4459@findex %parse-param
287c78f6
PE
4460Declare that an argument declared by the braced-code
4461@var{argument-declaration} is an additional @code{yyparse} argument.
94175978 4462The @var{argument-declaration} is used when declaring
feeb0eda
PE
4463functions or prototypes. The last identifier in
4464@var{argument-declaration} must be the argument name.
2a8d363a
AD
4465@end deffn
4466
4467Here's an example. Write this in the parser:
4468
4469@example
feeb0eda
PE
4470%parse-param @{int *nastiness@}
4471%parse-param @{int *randomness@}
2a8d363a
AD
4472@end example
4473
4474@noindent
4475Then call the parser like this:
4476
4477@example
4478@{
4479 int nastiness, randomness;
4480 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
4481 value = yyparse (&nastiness, &randomness);
4482 @dots{}
4483@}
4484@end example
4485
4486@noindent
4487In the grammar actions, use expressions like this to refer to the data:
4488
4489@example
4490exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
4491@end example
4492
bfa74976 4493
342b8b6e 4494@node Lexical
bfa74976
RS
4495@section The Lexical Analyzer Function @code{yylex}
4496@findex yylex
4497@cindex lexical analyzer
4498
4499The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
4500the input stream and returns them to the parser. Bison does not create
4501this function automatically; you must write it so that @code{yyparse} can
4502call it. The function is sometimes referred to as a lexical scanner.
4503
4504In simple programs, @code{yylex} is often defined at the end of the Bison
4505grammar file. If @code{yylex} is defined in a separate source file, you
4506need to arrange for the token-type macro definitions to be available there.
4507To do this, use the @samp{-d} option when you run Bison, so that it will
4508write these macro definitions into a separate header file
4509@file{@var{name}.tab.h} which you can include in the other source files
e0c471a9 4510that need it. @xref{Invocation, ,Invoking Bison}.
bfa74976
RS
4511
4512@menu
4513* Calling Convention:: How @code{yyparse} calls @code{yylex}.
4514* Token Values:: How @code{yylex} must return the semantic value
4515 of the token it has read.
95923bd6 4516* Token Locations:: How @code{yylex} must return the text location
bfa74976
RS
4517 (line number, etc.) of the token, if the
4518 actions want that.
4519* Pure Calling:: How the calling convention differs
4520 in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
4521@end menu
4522
342b8b6e 4523@node Calling Convention
bfa74976
RS
4524@subsection Calling Convention for @code{yylex}
4525
72d2299c
PE
4526The value that @code{yylex} returns must be the positive numeric code
4527for the type of token it has just found; a zero or negative value
4528signifies end-of-input.
bfa74976
RS
4529
4530When a token is referred to in the grammar rules by a name, that name
4531in the parser file becomes a C macro whose definition is the proper
4532numeric code for that token type. So @code{yylex} can use the name
4533to indicate that type. @xref{Symbols}.
4534
4535When a token is referred to in the grammar rules by a character literal,
4536the numeric code for that character is also the code for the token type.
72d2299c
PE
4537So @code{yylex} can simply return that character code, possibly converted
4538to @code{unsigned char} to avoid sign-extension. The null character
4539must not be used this way, because its code is zero and that
bfa74976
RS
4540signifies end-of-input.
4541
4542Here is an example showing these things:
4543
4544@example
13863333
AD
4545int
4546yylex (void)
bfa74976
RS
4547@{
4548 @dots{}
72d2299c 4549 if (c == EOF) /* Detect end-of-input. */
bfa74976
RS
4550 return 0;
4551 @dots{}
4552 if (c == '+' || c == '-')
72d2299c 4553 return c; /* Assume token type for `+' is '+'. */
bfa74976 4554 @dots{}
72d2299c 4555 return INT; /* Return the type of the token. */
bfa74976
RS
4556 @dots{}
4557@}
4558@end example
4559
4560@noindent
4561This interface has been designed so that the output from the @code{lex}
4562utility can be used without change as the definition of @code{yylex}.
4563
931c7513
RS
4564If the grammar uses literal string tokens, there are two ways that
4565@code{yylex} can determine the token type codes for them:
4566
4567@itemize @bullet
4568@item
4569If the grammar defines symbolic token names as aliases for the
4570literal string tokens, @code{yylex} can use these symbolic names like
4571all others. In this case, the use of the literal string tokens in
4572the grammar file has no effect on @code{yylex}.
4573
4574@item
9ecbd125 4575@code{yylex} can find the multicharacter token in the @code{yytname}
931c7513 4576table. The index of the token in the table is the token type's code.
9ecbd125 4577The name of a multicharacter token is recorded in @code{yytname} with a
931c7513 4578double-quote, the token's characters, and another double-quote. The
9e0876fb
PE
4579token's characters are escaped as necessary to be suitable as input
4580to Bison.
931c7513 4581
9e0876fb
PE
4582Here's code for looking up a multicharacter token in @code{yytname},
4583assuming that the characters of the token are stored in
4584@code{token_buffer}, and assuming that the token does not contain any
4585characters like @samp{"} that require escaping.
931c7513
RS
4586
4587@smallexample
4588for (i = 0; i < YYNTOKENS; i++)
4589 @{
4590 if (yytname[i] != 0
4591 && yytname[i][0] == '"'
68449b3a
PE
4592 && ! strncmp (yytname[i] + 1, token_buffer,
4593 strlen (token_buffer))
931c7513
RS
4594 && yytname[i][strlen (token_buffer) + 1] == '"'
4595 && yytname[i][strlen (token_buffer) + 2] == 0)
4596 break;
4597 @}
4598@end smallexample
4599
4600The @code{yytname} table is generated only if you use the
8c9a50be 4601@code{%token-table} declaration. @xref{Decl Summary}.
931c7513
RS
4602@end itemize
4603
342b8b6e 4604@node Token Values
bfa74976
RS
4605@subsection Semantic Values of Tokens
4606
4607@vindex yylval
9d9b8b70 4608In an ordinary (nonreentrant) parser, the semantic value of the token must
bfa74976
RS
4609be stored into the global variable @code{yylval}. When you are using
4610just one data type for semantic values, @code{yylval} has that type.
4611Thus, if the type is @code{int} (the default), you might write this in
4612@code{yylex}:
4613
4614@example
4615@group
4616 @dots{}
72d2299c
PE
4617 yylval = value; /* Put value onto Bison stack. */
4618 return INT; /* Return the type of the token. */
bfa74976
RS
4619 @dots{}
4620@end group
4621@end example
4622
4623When you are using multiple data types, @code{yylval}'s type is a union
704a47c4
AD
4624made from the @code{%union} declaration (@pxref{Union Decl, ,The
4625Collection of Value Types}). So when you store a token's value, you
4626must use the proper member of the union. If the @code{%union}
4627declaration looks like this:
bfa74976
RS
4628
4629@example
4630@group
4631%union @{
4632 int intval;
4633 double val;
4634 symrec *tptr;
4635@}
4636@end group
4637@end example
4638
4639@noindent
4640then the code in @code{yylex} might look like this:
4641
4642@example
4643@group
4644 @dots{}
72d2299c
PE
4645 yylval.intval = value; /* Put value onto Bison stack. */
4646 return INT; /* Return the type of the token. */
bfa74976
RS
4647 @dots{}
4648@end group
4649@end example
4650
95923bd6
AD
4651@node Token Locations
4652@subsection Textual Locations of Tokens
bfa74976
RS
4653
4654@vindex yylloc
847bf1f5 4655If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
f8e1c9e5
AD
4656Tracking Locations}) in actions to keep track of the textual locations
4657of tokens and groupings, then you must provide this information in
4658@code{yylex}. The function @code{yyparse} expects to find the textual
4659location of a token just parsed in the global variable @code{yylloc}.
4660So @code{yylex} must store the proper data in that variable.
847bf1f5
AD
4661
4662By default, the value of @code{yylloc} is a structure and you need only
89cab50d
AD
4663initialize the members that are going to be used by the actions. The
4664four members are called @code{first_line}, @code{first_column},
4665@code{last_line} and @code{last_column}. Note that the use of this
4666feature makes the parser noticeably slower.
bfa74976
RS
4667
4668@tindex YYLTYPE
4669The data type of @code{yylloc} has the name @code{YYLTYPE}.
4670
342b8b6e 4671@node Pure Calling
c656404a 4672@subsection Calling Conventions for Pure Parsers
bfa74976 4673
8c9a50be 4674When you use the Bison declaration @code{%pure-parser} to request a
e425e872
RS
4675pure, reentrant parser, the global communication variables @code{yylval}
4676and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
4677Parser}.) In such parsers the two global variables are replaced by
4678pointers passed as arguments to @code{yylex}. You must declare them as
4679shown here, and pass the information back by storing it through those
4680pointers.
bfa74976
RS
4681
4682@example
13863333
AD
4683int
4684yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
bfa74976
RS
4685@{
4686 @dots{}
4687 *lvalp = value; /* Put value onto Bison stack. */
4688 return INT; /* Return the type of the token. */
4689 @dots{}
4690@}
4691@end example
4692
4693If the grammar file does not use the @samp{@@} constructs to refer to
95923bd6 4694textual locations, then the type @code{YYLTYPE} will not be defined. In
bfa74976
RS
4695this case, omit the second argument; @code{yylex} will be called with
4696only one argument.
4697
e425e872 4698
2a8d363a
AD
4699If you wish to pass the additional parameter data to @code{yylex}, use
4700@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
4701Function}).
e425e872 4702
feeb0eda 4703@deffn {Directive} lex-param @{@var{argument-declaration}@}
2a8d363a 4704@findex %lex-param
287c78f6
PE
4705Declare that the braced-code @var{argument-declaration} is an
4706additional @code{yylex} argument declaration.
2a8d363a 4707@end deffn
e425e872 4708
2a8d363a 4709For instance:
e425e872
RS
4710
4711@example
feeb0eda
PE
4712%parse-param @{int *nastiness@}
4713%lex-param @{int *nastiness@}
4714%parse-param @{int *randomness@}
e425e872
RS
4715@end example
4716
4717@noindent
2a8d363a 4718results in the following signature:
e425e872
RS
4719
4720@example
2a8d363a
AD
4721int yylex (int *nastiness);
4722int yyparse (int *nastiness, int *randomness);
e425e872
RS
4723@end example
4724
2a8d363a 4725If @code{%pure-parser} is added:
c656404a
RS
4726
4727@example
2a8d363a
AD
4728int yylex (YYSTYPE *lvalp, int *nastiness);
4729int yyparse (int *nastiness, int *randomness);
c656404a
RS
4730@end example
4731
2a8d363a
AD
4732@noindent
4733and finally, if both @code{%pure-parser} and @code{%locations} are used:
c656404a 4734
2a8d363a
AD
4735@example
4736int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
4737int yyparse (int *nastiness, int *randomness);
4738@end example
931c7513 4739
342b8b6e 4740@node Error Reporting
bfa74976
RS
4741@section The Error Reporting Function @code{yyerror}
4742@cindex error reporting function
4743@findex yyerror
4744@cindex parse error
4745@cindex syntax error
4746
6e649e65 4747The Bison parser detects a @dfn{syntax error} or @dfn{parse error}
9ecbd125 4748whenever it reads a token which cannot satisfy any syntax rule. An
bfa74976 4749action in the grammar can also explicitly proclaim an error, using the
ceed8467
AD
4750macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
4751in Actions}).
bfa74976
RS
4752
4753The Bison parser expects to report the error by calling an error
4754reporting function named @code{yyerror}, which you must supply. It is
4755called by @code{yyparse} whenever a syntax error is found, and it
6e649e65
PE
4756receives one argument. For a syntax error, the string is normally
4757@w{@code{"syntax error"}}.
bfa74976 4758
2a8d363a
AD
4759@findex %error-verbose
4760If you invoke the directive @code{%error-verbose} in the Bison
4761declarations section (@pxref{Bison Declarations, ,The Bison Declarations
4762Section}), then Bison provides a more verbose and specific error message
6e649e65 4763string instead of just plain @w{@code{"syntax error"}}.
bfa74976 4764
1a059451
PE
4765The parser can detect one other kind of error: memory exhaustion. This
4766can happen when the input contains constructions that are very deeply
bfa74976 4767nested. It isn't likely you will encounter this, since the Bison
1a059451
PE
4768parser normally extends its stack automatically up to a very large limit. But
4769if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
4770fashion, except that the argument string is @w{@code{"memory exhausted"}}.
4771
4772In some cases diagnostics like @w{@code{"syntax error"}} are
4773translated automatically from English to some other language before
4774they are passed to @code{yyerror}. @xref{Internationalization}.
bfa74976
RS
4775
4776The following definition suffices in simple programs:
4777
4778@example
4779@group
13863333 4780void
38a92d50 4781yyerror (char const *s)
bfa74976
RS
4782@{
4783@end group
4784@group
4785 fprintf (stderr, "%s\n", s);
4786@}
4787@end group
4788@end example
4789
4790After @code{yyerror} returns to @code{yyparse}, the latter will attempt
4791error recovery if you have written suitable error recovery grammar rules
4792(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
4793immediately return 1.
4794
93724f13 4795Obviously, in location tracking pure parsers, @code{yyerror} should have
fa7e68c3
PE
4796an access to the current location.
4797This is indeed the case for the @acronym{GLR}
2a8d363a
AD
4798parsers, but not for the Yacc parser, for historical reasons. I.e., if
4799@samp{%locations %pure-parser} is passed then the prototypes for
4800@code{yyerror} are:
4801
4802@example
38a92d50
PE
4803void yyerror (char const *msg); /* Yacc parsers. */
4804void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
2a8d363a
AD
4805@end example
4806
feeb0eda 4807If @samp{%parse-param @{int *nastiness@}} is used, then:
2a8d363a
AD
4808
4809@example
b317297e
PE
4810void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
4811void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
2a8d363a
AD
4812@end example
4813
fa7e68c3 4814Finally, @acronym{GLR} and Yacc parsers share the same @code{yyerror} calling
2a8d363a
AD
4815convention for absolutely pure parsers, i.e., when the calling
4816convention of @code{yylex} @emph{and} the calling convention of
4817@code{%pure-parser} are pure. I.e.:
4818
4819@example
4820/* Location tracking. */
4821%locations
4822/* Pure yylex. */
4823%pure-parser
feeb0eda 4824%lex-param @{int *nastiness@}
2a8d363a 4825/* Pure yyparse. */
feeb0eda
PE
4826%parse-param @{int *nastiness@}
4827%parse-param @{int *randomness@}
2a8d363a
AD
4828@end example
4829
4830@noindent
4831results in the following signatures for all the parser kinds:
4832
4833@example
4834int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
4835int yyparse (int *nastiness, int *randomness);
93724f13
AD
4836void yyerror (YYLTYPE *locp,
4837 int *nastiness, int *randomness,
38a92d50 4838 char const *msg);
2a8d363a
AD
4839@end example
4840
1c0c3e95 4841@noindent
38a92d50
PE
4842The prototypes are only indications of how the code produced by Bison
4843uses @code{yyerror}. Bison-generated code always ignores the returned
4844value, so @code{yyerror} can return any type, including @code{void}.
4845Also, @code{yyerror} can be a variadic function; that is why the
4846message is always passed last.
4847
4848Traditionally @code{yyerror} returns an @code{int} that is always
4849ignored, but this is purely for historical reasons, and @code{void} is
4850preferable since it more accurately describes the return type for
4851@code{yyerror}.
93724f13 4852
bfa74976
RS
4853@vindex yynerrs
4854The variable @code{yynerrs} contains the number of syntax errors
8a2800e7 4855reported so far. Normally this variable is global; but if you
704a47c4
AD
4856request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
4857then it is a local variable which only the actions can access.
bfa74976 4858
342b8b6e 4859@node Action Features
bfa74976
RS
4860@section Special Features for Use in Actions
4861@cindex summary, action features
4862@cindex action features summary
4863
4864Here is a table of Bison constructs, variables and macros that
4865are useful in actions.
4866
18b519c0 4867@deffn {Variable} $$
bfa74976
RS
4868Acts like a variable that contains the semantic value for the
4869grouping made by the current rule. @xref{Actions}.
18b519c0 4870@end deffn
bfa74976 4871
18b519c0 4872@deffn {Variable} $@var{n}
bfa74976
RS
4873Acts like a variable that contains the semantic value for the
4874@var{n}th component of the current rule. @xref{Actions}.
18b519c0 4875@end deffn
bfa74976 4876
18b519c0 4877@deffn {Variable} $<@var{typealt}>$
bfa74976 4878Like @code{$$} but specifies alternative @var{typealt} in the union
704a47c4
AD
4879specified by the @code{%union} declaration. @xref{Action Types, ,Data
4880Types of Values in Actions}.
18b519c0 4881@end deffn
bfa74976 4882
18b519c0 4883@deffn {Variable} $<@var{typealt}>@var{n}
bfa74976 4884Like @code{$@var{n}} but specifies alternative @var{typealt} in the
13863333 4885union specified by the @code{%union} declaration.
e0c471a9 4886@xref{Action Types, ,Data Types of Values in Actions}.
18b519c0 4887@end deffn
bfa74976 4888
18b519c0 4889@deffn {Macro} YYABORT;
bfa74976
RS
4890Return immediately from @code{yyparse}, indicating failure.
4891@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 4892@end deffn
bfa74976 4893
18b519c0 4894@deffn {Macro} YYACCEPT;
bfa74976
RS
4895Return immediately from @code{yyparse}, indicating success.
4896@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 4897@end deffn
bfa74976 4898
18b519c0 4899@deffn {Macro} YYBACKUP (@var{token}, @var{value});
bfa74976
RS
4900@findex YYBACKUP
4901Unshift a token. This macro is allowed only for rules that reduce
742e4900 4902a single value, and only when there is no lookahead token.
c827f760 4903It is also disallowed in @acronym{GLR} parsers.
742e4900 4904It installs a lookahead token with token type @var{token} and
bfa74976
RS
4905semantic value @var{value}; then it discards the value that was
4906going to be reduced by this rule.
4907
4908If the macro is used when it is not valid, such as when there is
742e4900 4909a lookahead token already, then it reports a syntax error with
bfa74976
RS
4910a message @samp{cannot back up} and performs ordinary error
4911recovery.
4912
4913In either case, the rest of the action is not executed.
18b519c0 4914@end deffn
bfa74976 4915
18b519c0 4916@deffn {Macro} YYEMPTY
bfa74976 4917@vindex YYEMPTY
742e4900 4918Value stored in @code{yychar} when there is no lookahead token.
18b519c0 4919@end deffn
bfa74976 4920
32c29292
JD
4921@deffn {Macro} YYEOF
4922@vindex YYEOF
742e4900 4923Value stored in @code{yychar} when the lookahead is the end of the input
32c29292
JD
4924stream.
4925@end deffn
4926
18b519c0 4927@deffn {Macro} YYERROR;
bfa74976
RS
4928@findex YYERROR
4929Cause an immediate syntax error. This statement initiates error
4930recovery just as if the parser itself had detected an error; however, it
4931does not call @code{yyerror}, and does not print any message. If you
4932want to print an error message, call @code{yyerror} explicitly before
4933the @samp{YYERROR;} statement. @xref{Error Recovery}.
18b519c0 4934@end deffn
bfa74976 4935
18b519c0 4936@deffn {Macro} YYRECOVERING
02103984
PE
4937@findex YYRECOVERING
4938The expression @code{YYRECOVERING ()} yields 1 when the parser
4939is recovering from a syntax error, and 0 otherwise.
bfa74976 4940@xref{Error Recovery}.
18b519c0 4941@end deffn
bfa74976 4942
18b519c0 4943@deffn {Variable} yychar
742e4900
JD
4944Variable containing either the lookahead token, or @code{YYEOF} when the
4945lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
32c29292
JD
4946has been performed so the next token is not yet known.
4947Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
4948Actions}).
742e4900 4949@xref{Lookahead, ,Lookahead Tokens}.
18b519c0 4950@end deffn
bfa74976 4951
18b519c0 4952@deffn {Macro} yyclearin;
742e4900 4953Discard the current lookahead token. This is useful primarily in
32c29292
JD
4954error rules.
4955Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
4956Semantic Actions}).
4957@xref{Error Recovery}.
18b519c0 4958@end deffn
bfa74976 4959
18b519c0 4960@deffn {Macro} yyerrok;
bfa74976 4961Resume generating error messages immediately for subsequent syntax
13863333 4962errors. This is useful primarily in error rules.
bfa74976 4963@xref{Error Recovery}.
18b519c0 4964@end deffn
bfa74976 4965
32c29292 4966@deffn {Variable} yylloc
742e4900 4967Variable containing the lookahead token location when @code{yychar} is not set
32c29292
JD
4968to @code{YYEMPTY} or @code{YYEOF}.
4969Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
4970Actions}).
4971@xref{Actions and Locations, ,Actions and Locations}.
4972@end deffn
4973
4974@deffn {Variable} yylval
742e4900 4975Variable containing the lookahead token semantic value when @code{yychar} is
32c29292
JD
4976not set to @code{YYEMPTY} or @code{YYEOF}.
4977Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
4978Actions}).
4979@xref{Actions, ,Actions}.
4980@end deffn
4981
18b519c0 4982@deffn {Value} @@$
847bf1f5 4983@findex @@$
95923bd6 4984Acts like a structure variable containing information on the textual location
847bf1f5
AD
4985of the grouping made by the current rule. @xref{Locations, ,
4986Tracking Locations}.
bfa74976 4987
847bf1f5
AD
4988@c Check if those paragraphs are still useful or not.
4989
4990@c @example
4991@c struct @{
4992@c int first_line, last_line;
4993@c int first_column, last_column;
4994@c @};
4995@c @end example
4996
4997@c Thus, to get the starting line number of the third component, you would
4998@c use @samp{@@3.first_line}.
bfa74976 4999
847bf1f5
AD
5000@c In order for the members of this structure to contain valid information,
5001@c you must make @code{yylex} supply this information about each token.
5002@c If you need only certain members, then @code{yylex} need only fill in
5003@c those members.
bfa74976 5004
847bf1f5 5005@c The use of this feature makes the parser noticeably slower.
18b519c0 5006@end deffn
847bf1f5 5007
18b519c0 5008@deffn {Value} @@@var{n}
847bf1f5 5009@findex @@@var{n}
95923bd6 5010Acts like a structure variable containing information on the textual location
847bf1f5
AD
5011of the @var{n}th component of the current rule. @xref{Locations, ,
5012Tracking Locations}.
18b519c0 5013@end deffn
bfa74976 5014
f7ab6a50
PE
5015@node Internationalization
5016@section Parser Internationalization
5017@cindex internationalization
5018@cindex i18n
5019@cindex NLS
5020@cindex gettext
5021@cindex bison-po
5022
5023A Bison-generated parser can print diagnostics, including error and
5024tracing messages. By default, they appear in English. However, Bison
f8e1c9e5
AD
5025also supports outputting diagnostics in the user's native language. To
5026make this work, the user should set the usual environment variables.
5027@xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
5028For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
5029set the user's locale to French Canadian using the @acronym{UTF}-8
f7ab6a50
PE
5030encoding. The exact set of available locales depends on the user's
5031installation.
5032
5033The maintainer of a package that uses a Bison-generated parser enables
5034the internationalization of the parser's output through the following
5035steps. Here we assume a package that uses @acronym{GNU} Autoconf and
5036@acronym{GNU} Automake.
5037
5038@enumerate
5039@item
30757c8c 5040@cindex bison-i18n.m4
f7ab6a50
PE
5041Into the directory containing the @acronym{GNU} Autoconf macros used
5042by the package---often called @file{m4}---copy the
5043@file{bison-i18n.m4} file installed by Bison under
5044@samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
5045For example:
5046
5047@example
5048cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
5049@end example
5050
5051@item
30757c8c
PE
5052@findex BISON_I18N
5053@vindex BISON_LOCALEDIR
5054@vindex YYENABLE_NLS
f7ab6a50
PE
5055In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
5056invocation, add an invocation of @code{BISON_I18N}. This macro is
5057defined in the file @file{bison-i18n.m4} that you copied earlier. It
5058causes @samp{configure} to find the value of the
30757c8c
PE
5059@code{BISON_LOCALEDIR} variable, and it defines the source-language
5060symbol @code{YYENABLE_NLS} to enable translations in the
5061Bison-generated parser.
f7ab6a50
PE
5062
5063@item
5064In the @code{main} function of your program, designate the directory
5065containing Bison's runtime message catalog, through a call to
5066@samp{bindtextdomain} with domain name @samp{bison-runtime}.
5067For example:
5068
5069@example
5070bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
5071@end example
5072
5073Typically this appears after any other call @code{bindtextdomain
5074(PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
5075@samp{BISON_LOCALEDIR} to be defined as a string through the
5076@file{Makefile}.
5077
5078@item
5079In the @file{Makefile.am} that controls the compilation of the @code{main}
5080function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
5081either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
5082
5083@example
5084DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5085@end example
5086
5087or:
5088
5089@example
5090AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5091@end example
5092
5093@item
5094Finally, invoke the command @command{autoreconf} to generate the build
5095infrastructure.
5096@end enumerate
5097
bfa74976 5098
342b8b6e 5099@node Algorithm
13863333
AD
5100@chapter The Bison Parser Algorithm
5101@cindex Bison parser algorithm
bfa74976
RS
5102@cindex algorithm of parser
5103@cindex shifting
5104@cindex reduction
5105@cindex parser stack
5106@cindex stack, parser
5107
5108As Bison reads tokens, it pushes them onto a stack along with their
5109semantic values. The stack is called the @dfn{parser stack}. Pushing a
5110token is traditionally called @dfn{shifting}.
5111
5112For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
5113@samp{3} to come. The stack will have four elements, one for each token
5114that was shifted.
5115
5116But the stack does not always have an element for each token read. When
5117the last @var{n} tokens and groupings shifted match the components of a
5118grammar rule, they can be combined according to that rule. This is called
5119@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
5120single grouping whose symbol is the result (left hand side) of that rule.
5121Running the rule's action is part of the process of reduction, because this
5122is what computes the semantic value of the resulting grouping.
5123
5124For example, if the infix calculator's parser stack contains this:
5125
5126@example
51271 + 5 * 3
5128@end example
5129
5130@noindent
5131and the next input token is a newline character, then the last three
5132elements can be reduced to 15 via the rule:
5133
5134@example
5135expr: expr '*' expr;
5136@end example
5137
5138@noindent
5139Then the stack contains just these three elements:
5140
5141@example
51421 + 15
5143@end example
5144
5145@noindent
5146At this point, another reduction can be made, resulting in the single value
514716. Then the newline token can be shifted.
5148
5149The parser tries, by shifts and reductions, to reduce the entire input down
5150to a single grouping whose symbol is the grammar's start-symbol
5151(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
5152
5153This kind of parser is known in the literature as a bottom-up parser.
5154
5155@menu
742e4900 5156* Lookahead:: Parser looks one token ahead when deciding what to do.
bfa74976
RS
5157* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
5158* Precedence:: Operator precedence works by resolving conflicts.
5159* Contextual Precedence:: When an operator's precedence depends on context.
5160* Parser States:: The parser is a finite-state-machine with stack.
5161* Reduce/Reduce:: When two rules are applicable in the same situation.
5162* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 5163* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 5164* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
5165@end menu
5166
742e4900
JD
5167@node Lookahead
5168@section Lookahead Tokens
5169@cindex lookahead token
bfa74976
RS
5170
5171The Bison parser does @emph{not} always reduce immediately as soon as the
5172last @var{n} tokens and groupings match a rule. This is because such a
5173simple strategy is inadequate to handle most languages. Instead, when a
5174reduction is possible, the parser sometimes ``looks ahead'' at the next
5175token in order to decide what to do.
5176
5177When a token is read, it is not immediately shifted; first it becomes the
742e4900 5178@dfn{lookahead token}, which is not on the stack. Now the parser can
bfa74976 5179perform one or more reductions of tokens and groupings on the stack, while
742e4900
JD
5180the lookahead token remains off to the side. When no more reductions
5181should take place, the lookahead token is shifted onto the stack. This
bfa74976 5182does not mean that all possible reductions have been done; depending on the
742e4900 5183token type of the lookahead token, some rules may choose to delay their
bfa74976
RS
5184application.
5185
742e4900 5186Here is a simple case where lookahead is needed. These three rules define
bfa74976
RS
5187expressions which contain binary addition operators and postfix unary
5188factorial operators (@samp{!}), and allow parentheses for grouping.
5189
5190@example
5191@group
5192expr: term '+' expr
5193 | term
5194 ;
5195@end group
5196
5197@group
5198term: '(' expr ')'
5199 | term '!'
5200 | NUMBER
5201 ;
5202@end group
5203@end example
5204
5205Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
5206should be done? If the following token is @samp{)}, then the first three
5207tokens must be reduced to form an @code{expr}. This is the only valid
5208course, because shifting the @samp{)} would produce a sequence of symbols
5209@w{@code{term ')'}}, and no rule allows this.
5210
5211If the following token is @samp{!}, then it must be shifted immediately so
5212that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
5213parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
5214@code{expr}. It would then be impossible to shift the @samp{!} because
5215doing so would produce on the stack the sequence of symbols @code{expr
5216'!'}. No rule allows that sequence.
5217
5218@vindex yychar
32c29292
JD
5219@vindex yylval
5220@vindex yylloc
742e4900 5221The lookahead token is stored in the variable @code{yychar}.
32c29292
JD
5222Its semantic value and location, if any, are stored in the variables
5223@code{yylval} and @code{yylloc}.
bfa74976
RS
5224@xref{Action Features, ,Special Features for Use in Actions}.
5225
342b8b6e 5226@node Shift/Reduce
bfa74976
RS
5227@section Shift/Reduce Conflicts
5228@cindex conflicts
5229@cindex shift/reduce conflicts
5230@cindex dangling @code{else}
5231@cindex @code{else}, dangling
5232
5233Suppose we are parsing a language which has if-then and if-then-else
5234statements, with a pair of rules like this:
5235
5236@example
5237@group
5238if_stmt:
5239 IF expr THEN stmt
5240 | IF expr THEN stmt ELSE stmt
5241 ;
5242@end group
5243@end example
5244
5245@noindent
5246Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
5247terminal symbols for specific keyword tokens.
5248
742e4900 5249When the @code{ELSE} token is read and becomes the lookahead token, the
bfa74976
RS
5250contents of the stack (assuming the input is valid) are just right for
5251reduction by the first rule. But it is also legitimate to shift the
5252@code{ELSE}, because that would lead to eventual reduction by the second
5253rule.
5254
5255This situation, where either a shift or a reduction would be valid, is
5256called a @dfn{shift/reduce conflict}. Bison is designed to resolve
5257these conflicts by choosing to shift, unless otherwise directed by
5258operator precedence declarations. To see the reason for this, let's
5259contrast it with the other alternative.
5260
5261Since the parser prefers to shift the @code{ELSE}, the result is to attach
5262the else-clause to the innermost if-statement, making these two inputs
5263equivalent:
5264
5265@example
5266if x then if y then win (); else lose;
5267
5268if x then do; if y then win (); else lose; end;
5269@end example
5270
5271But if the parser chose to reduce when possible rather than shift, the
5272result would be to attach the else-clause to the outermost if-statement,
5273making these two inputs equivalent:
5274
5275@example
5276if x then if y then win (); else lose;
5277
5278if x then do; if y then win (); end; else lose;
5279@end example
5280
5281The conflict exists because the grammar as written is ambiguous: either
5282parsing of the simple nested if-statement is legitimate. The established
5283convention is that these ambiguities are resolved by attaching the
5284else-clause to the innermost if-statement; this is what Bison accomplishes
5285by choosing to shift rather than reduce. (It would ideally be cleaner to
5286write an unambiguous grammar, but that is very hard to do in this case.)
5287This particular ambiguity was first encountered in the specifications of
5288Algol 60 and is called the ``dangling @code{else}'' ambiguity.
5289
5290To avoid warnings from Bison about predictable, legitimate shift/reduce
5291conflicts, use the @code{%expect @var{n}} declaration. There will be no
5292warning as long as the number of shift/reduce conflicts is exactly @var{n}.
5293@xref{Expect Decl, ,Suppressing Conflict Warnings}.
5294
5295The definition of @code{if_stmt} above is solely to blame for the
5296conflict, but the conflict does not actually appear without additional
5297rules. Here is a complete Bison input file that actually manifests the
5298conflict:
5299
5300@example
5301@group
5302%token IF THEN ELSE variable
5303%%
5304@end group
5305@group
5306stmt: expr
5307 | if_stmt
5308 ;
5309@end group
5310
5311@group
5312if_stmt:
5313 IF expr THEN stmt
5314 | IF expr THEN stmt ELSE stmt
5315 ;
5316@end group
5317
5318expr: variable
5319 ;
5320@end example
5321
342b8b6e 5322@node Precedence
bfa74976
RS
5323@section Operator Precedence
5324@cindex operator precedence
5325@cindex precedence of operators
5326
5327Another situation where shift/reduce conflicts appear is in arithmetic
5328expressions. Here shifting is not always the preferred resolution; the
5329Bison declarations for operator precedence allow you to specify when to
5330shift and when to reduce.
5331
5332@menu
5333* Why Precedence:: An example showing why precedence is needed.
5334* Using Precedence:: How to specify precedence in Bison grammars.
5335* Precedence Examples:: How these features are used in the previous example.
5336* How Precedence:: How they work.
5337@end menu
5338
342b8b6e 5339@node Why Precedence
bfa74976
RS
5340@subsection When Precedence is Needed
5341
5342Consider the following ambiguous grammar fragment (ambiguous because the
5343input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
5344
5345@example
5346@group
5347expr: expr '-' expr
5348 | expr '*' expr
5349 | expr '<' expr
5350 | '(' expr ')'
5351 @dots{}
5352 ;
5353@end group
5354@end example
5355
5356@noindent
5357Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
14ded682
AD
5358should it reduce them via the rule for the subtraction operator? It
5359depends on the next token. Of course, if the next token is @samp{)}, we
5360must reduce; shifting is invalid because no single rule can reduce the
5361token sequence @w{@samp{- 2 )}} or anything starting with that. But if
5362the next token is @samp{*} or @samp{<}, we have a choice: either
5363shifting or reduction would allow the parse to complete, but with
5364different results.
5365
5366To decide which one Bison should do, we must consider the results. If
5367the next operator token @var{op} is shifted, then it must be reduced
5368first in order to permit another opportunity to reduce the difference.
5369The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
5370hand, if the subtraction is reduced before shifting @var{op}, the result
5371is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
5372reduce should depend on the relative precedence of the operators
5373@samp{-} and @var{op}: @samp{*} should be shifted first, but not
5374@samp{<}.
bfa74976
RS
5375
5376@cindex associativity
5377What about input such as @w{@samp{1 - 2 - 5}}; should this be
14ded682
AD
5378@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
5379operators we prefer the former, which is called @dfn{left association}.
5380The latter alternative, @dfn{right association}, is desirable for
5381assignment operators. The choice of left or right association is a
5382matter of whether the parser chooses to shift or reduce when the stack
742e4900 5383contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
14ded682 5384makes right-associativity.
bfa74976 5385
342b8b6e 5386@node Using Precedence
bfa74976
RS
5387@subsection Specifying Operator Precedence
5388@findex %left
5389@findex %right
5390@findex %nonassoc
5391
5392Bison allows you to specify these choices with the operator precedence
5393declarations @code{%left} and @code{%right}. Each such declaration
5394contains a list of tokens, which are operators whose precedence and
5395associativity is being declared. The @code{%left} declaration makes all
5396those operators left-associative and the @code{%right} declaration makes
5397them right-associative. A third alternative is @code{%nonassoc}, which
5398declares that it is a syntax error to find the same operator twice ``in a
5399row''.
5400
5401The relative precedence of different operators is controlled by the
5402order in which they are declared. The first @code{%left} or
5403@code{%right} declaration in the file declares the operators whose
5404precedence is lowest, the next such declaration declares the operators
5405whose precedence is a little higher, and so on.
5406
342b8b6e 5407@node Precedence Examples
bfa74976
RS
5408@subsection Precedence Examples
5409
5410In our example, we would want the following declarations:
5411
5412@example
5413%left '<'
5414%left '-'
5415%left '*'
5416@end example
5417
5418In a more complete example, which supports other operators as well, we
5419would declare them in groups of equal precedence. For example, @code{'+'} is
5420declared with @code{'-'}:
5421
5422@example
5423%left '<' '>' '=' NE LE GE
5424%left '+' '-'
5425%left '*' '/'
5426@end example
5427
5428@noindent
5429(Here @code{NE} and so on stand for the operators for ``not equal''
5430and so on. We assume that these tokens are more than one character long
5431and therefore are represented by names, not character literals.)
5432
342b8b6e 5433@node How Precedence
bfa74976
RS
5434@subsection How Precedence Works
5435
5436The first effect of the precedence declarations is to assign precedence
5437levels to the terminal symbols declared. The second effect is to assign
704a47c4
AD
5438precedence levels to certain rules: each rule gets its precedence from
5439the last terminal symbol mentioned in the components. (You can also
5440specify explicitly the precedence of a rule. @xref{Contextual
5441Precedence, ,Context-Dependent Precedence}.)
5442
5443Finally, the resolution of conflicts works by comparing the precedence
742e4900 5444of the rule being considered with that of the lookahead token. If the
704a47c4
AD
5445token's precedence is higher, the choice is to shift. If the rule's
5446precedence is higher, the choice is to reduce. If they have equal
5447precedence, the choice is made based on the associativity of that
5448precedence level. The verbose output file made by @samp{-v}
5449(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
5450resolved.
bfa74976
RS
5451
5452Not all rules and not all tokens have precedence. If either the rule or
742e4900 5453the lookahead token has no precedence, then the default is to shift.
bfa74976 5454
342b8b6e 5455@node Contextual Precedence
bfa74976
RS
5456@section Context-Dependent Precedence
5457@cindex context-dependent precedence
5458@cindex unary operator precedence
5459@cindex precedence, context-dependent
5460@cindex precedence, unary operator
5461@findex %prec
5462
5463Often the precedence of an operator depends on the context. This sounds
5464outlandish at first, but it is really very common. For example, a minus
5465sign typically has a very high precedence as a unary operator, and a
5466somewhat lower precedence (lower than multiplication) as a binary operator.
5467
5468The Bison precedence declarations, @code{%left}, @code{%right} and
5469@code{%nonassoc}, can only be used once for a given token; so a token has
5470only one precedence declared in this way. For context-dependent
5471precedence, you need to use an additional mechanism: the @code{%prec}
e0c471a9 5472modifier for rules.
bfa74976
RS
5473
5474The @code{%prec} modifier declares the precedence of a particular rule by
5475specifying a terminal symbol whose precedence should be used for that rule.
5476It's not necessary for that symbol to appear otherwise in the rule. The
5477modifier's syntax is:
5478
5479@example
5480%prec @var{terminal-symbol}
5481@end example
5482
5483@noindent
5484and it is written after the components of the rule. Its effect is to
5485assign the rule the precedence of @var{terminal-symbol}, overriding
5486the precedence that would be deduced for it in the ordinary way. The
5487altered rule precedence then affects how conflicts involving that rule
5488are resolved (@pxref{Precedence, ,Operator Precedence}).
5489
5490Here is how @code{%prec} solves the problem of unary minus. First, declare
5491a precedence for a fictitious terminal symbol named @code{UMINUS}. There
5492are no tokens of this type, but the symbol serves to stand for its
5493precedence:
5494
5495@example
5496@dots{}
5497%left '+' '-'
5498%left '*'
5499%left UMINUS
5500@end example
5501
5502Now the precedence of @code{UMINUS} can be used in specific rules:
5503
5504@example
5505@group
5506exp: @dots{}
5507 | exp '-' exp
5508 @dots{}
5509 | '-' exp %prec UMINUS
5510@end group
5511@end example
5512
91d2c560 5513@ifset defaultprec
39a06c25
PE
5514If you forget to append @code{%prec UMINUS} to the rule for unary
5515minus, Bison silently assumes that minus has its usual precedence.
5516This kind of problem can be tricky to debug, since one typically
5517discovers the mistake only by testing the code.
5518
22fccf95 5519The @code{%no-default-prec;} declaration makes it easier to discover
39a06c25
PE
5520this kind of problem systematically. It causes rules that lack a
5521@code{%prec} modifier to have no precedence, even if the last terminal
5522symbol mentioned in their components has a declared precedence.
5523
22fccf95 5524If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
39a06c25
PE
5525for all rules that participate in precedence conflict resolution.
5526Then you will see any shift/reduce conflict until you tell Bison how
5527to resolve it, either by changing your grammar or by adding an
5528explicit precedence. This will probably add declarations to the
5529grammar, but it helps to protect against incorrect rule precedences.
5530
22fccf95
PE
5531The effect of @code{%no-default-prec;} can be reversed by giving
5532@code{%default-prec;}, which is the default.
91d2c560 5533@end ifset
39a06c25 5534
342b8b6e 5535@node Parser States
bfa74976
RS
5536@section Parser States
5537@cindex finite-state machine
5538@cindex parser state
5539@cindex state (of parser)
5540
5541The function @code{yyparse} is implemented using a finite-state machine.
5542The values pushed on the parser stack are not simply token type codes; they
5543represent the entire sequence of terminal and nonterminal symbols at or
5544near the top of the stack. The current state collects all the information
5545about previous input which is relevant to deciding what to do next.
5546
742e4900
JD
5547Each time a lookahead token is read, the current parser state together
5548with the type of lookahead token are looked up in a table. This table
5549entry can say, ``Shift the lookahead token.'' In this case, it also
bfa74976
RS
5550specifies the new parser state, which is pushed onto the top of the
5551parser stack. Or it can say, ``Reduce using rule number @var{n}.''
5552This means that a certain number of tokens or groupings are taken off
5553the top of the stack, and replaced by one grouping. In other words,
5554that number of states are popped from the stack, and one new state is
5555pushed.
5556
742e4900 5557There is one other alternative: the table can say that the lookahead token
bfa74976
RS
5558is erroneous in the current state. This causes error processing to begin
5559(@pxref{Error Recovery}).
5560
342b8b6e 5561@node Reduce/Reduce
bfa74976
RS
5562@section Reduce/Reduce Conflicts
5563@cindex reduce/reduce conflict
5564@cindex conflicts, reduce/reduce
5565
5566A reduce/reduce conflict occurs if there are two or more rules that apply
5567to the same sequence of input. This usually indicates a serious error
5568in the grammar.
5569
5570For example, here is an erroneous attempt to define a sequence
5571of zero or more @code{word} groupings.
5572
5573@example
5574sequence: /* empty */
5575 @{ printf ("empty sequence\n"); @}
5576 | maybeword
5577 | sequence word
5578 @{ printf ("added word %s\n", $2); @}
5579 ;
5580
5581maybeword: /* empty */
5582 @{ printf ("empty maybeword\n"); @}
5583 | word
5584 @{ printf ("single word %s\n", $1); @}
5585 ;
5586@end example
5587
5588@noindent
5589The error is an ambiguity: there is more than one way to parse a single
5590@code{word} into a @code{sequence}. It could be reduced to a
5591@code{maybeword} and then into a @code{sequence} via the second rule.
5592Alternatively, nothing-at-all could be reduced into a @code{sequence}
5593via the first rule, and this could be combined with the @code{word}
5594using the third rule for @code{sequence}.
5595
5596There is also more than one way to reduce nothing-at-all into a
5597@code{sequence}. This can be done directly via the first rule,
5598or indirectly via @code{maybeword} and then the second rule.
5599
5600You might think that this is a distinction without a difference, because it
5601does not change whether any particular input is valid or not. But it does
5602affect which actions are run. One parsing order runs the second rule's
5603action; the other runs the first rule's action and the third rule's action.
5604In this example, the output of the program changes.
5605
5606Bison resolves a reduce/reduce conflict by choosing to use the rule that
5607appears first in the grammar, but it is very risky to rely on this. Every
5608reduce/reduce conflict must be studied and usually eliminated. Here is the
5609proper way to define @code{sequence}:
5610
5611@example
5612sequence: /* empty */
5613 @{ printf ("empty sequence\n"); @}
5614 | sequence word
5615 @{ printf ("added word %s\n", $2); @}
5616 ;
5617@end example
5618
5619Here is another common error that yields a reduce/reduce conflict:
5620
5621@example
5622sequence: /* empty */
5623 | sequence words
5624 | sequence redirects
5625 ;
5626
5627words: /* empty */
5628 | words word
5629 ;
5630
5631redirects:/* empty */
5632 | redirects redirect
5633 ;
5634@end example
5635
5636@noindent
5637The intention here is to define a sequence which can contain either
5638@code{word} or @code{redirect} groupings. The individual definitions of
5639@code{sequence}, @code{words} and @code{redirects} are error-free, but the
5640three together make a subtle ambiguity: even an empty input can be parsed
5641in infinitely many ways!
5642
5643Consider: nothing-at-all could be a @code{words}. Or it could be two
5644@code{words} in a row, or three, or any number. It could equally well be a
5645@code{redirects}, or two, or any number. Or it could be a @code{words}
5646followed by three @code{redirects} and another @code{words}. And so on.
5647
5648Here are two ways to correct these rules. First, to make it a single level
5649of sequence:
5650
5651@example
5652sequence: /* empty */
5653 | sequence word
5654 | sequence redirect
5655 ;
5656@end example
5657
5658Second, to prevent either a @code{words} or a @code{redirects}
5659from being empty:
5660
5661@example
5662sequence: /* empty */
5663 | sequence words
5664 | sequence redirects
5665 ;
5666
5667words: word
5668 | words word
5669 ;
5670
5671redirects:redirect
5672 | redirects redirect
5673 ;
5674@end example
5675
342b8b6e 5676@node Mystery Conflicts
bfa74976
RS
5677@section Mysterious Reduce/Reduce Conflicts
5678
5679Sometimes reduce/reduce conflicts can occur that don't look warranted.
5680Here is an example:
5681
5682@example
5683@group
5684%token ID
5685
5686%%
5687def: param_spec return_spec ','
5688 ;
5689param_spec:
5690 type
5691 | name_list ':' type
5692 ;
5693@end group
5694@group
5695return_spec:
5696 type
5697 | name ':' type
5698 ;
5699@end group
5700@group
5701type: ID
5702 ;
5703@end group
5704@group
5705name: ID
5706 ;
5707name_list:
5708 name
5709 | name ',' name_list
5710 ;
5711@end group
5712@end example
5713
5714It would seem that this grammar can be parsed with only a single token
742e4900 5715of lookahead: when a @code{param_spec} is being read, an @code{ID} is
bfa74976 5716a @code{name} if a comma or colon follows, or a @code{type} if another
c827f760 5717@code{ID} follows. In other words, this grammar is @acronym{LR}(1).
bfa74976 5718
c827f760
PE
5719@cindex @acronym{LR}(1)
5720@cindex @acronym{LALR}(1)
bfa74976 5721However, Bison, like most parser generators, cannot actually handle all
c827f760
PE
5722@acronym{LR}(1) grammars. In this grammar, two contexts, that after
5723an @code{ID}
bfa74976
RS
5724at the beginning of a @code{param_spec} and likewise at the beginning of
5725a @code{return_spec}, are similar enough that Bison assumes they are the
5726same. They appear similar because the same set of rules would be
5727active---the rule for reducing to a @code{name} and that for reducing to
5728a @code{type}. Bison is unable to determine at that stage of processing
742e4900 5729that the rules would require different lookahead tokens in the two
bfa74976
RS
5730contexts, so it makes a single parser state for them both. Combining
5731the two contexts causes a conflict later. In parser terminology, this
c827f760 5732occurrence means that the grammar is not @acronym{LALR}(1).
bfa74976
RS
5733
5734In general, it is better to fix deficiencies than to document them. But
5735this particular deficiency is intrinsically hard to fix; parser
c827f760
PE
5736generators that can handle @acronym{LR}(1) grammars are hard to write
5737and tend to
bfa74976
RS
5738produce parsers that are very large. In practice, Bison is more useful
5739as it is now.
5740
5741When the problem arises, you can often fix it by identifying the two
a220f555
MA
5742parser states that are being confused, and adding something to make them
5743look distinct. In the above example, adding one rule to
bfa74976
RS
5744@code{return_spec} as follows makes the problem go away:
5745
5746@example
5747@group
5748%token BOGUS
5749@dots{}
5750%%
5751@dots{}
5752return_spec:
5753 type
5754 | name ':' type
5755 /* This rule is never used. */
5756 | ID BOGUS
5757 ;
5758@end group
5759@end example
5760
5761This corrects the problem because it introduces the possibility of an
5762additional active rule in the context after the @code{ID} at the beginning of
5763@code{return_spec}. This rule is not active in the corresponding context
5764in a @code{param_spec}, so the two contexts receive distinct parser states.
5765As long as the token @code{BOGUS} is never generated by @code{yylex},
5766the added rule cannot alter the way actual input is parsed.
5767
5768In this particular example, there is another way to solve the problem:
5769rewrite the rule for @code{return_spec} to use @code{ID} directly
5770instead of via @code{name}. This also causes the two confusing
5771contexts to have different sets of active rules, because the one for
5772@code{return_spec} activates the altered rule for @code{return_spec}
5773rather than the one for @code{name}.
5774
5775@example
5776param_spec:
5777 type
5778 | name_list ':' type
5779 ;
5780return_spec:
5781 type
5782 | ID ':' type
5783 ;
5784@end example
5785
e054b190
PE
5786For a more detailed exposition of @acronym{LALR}(1) parsers and parser
5787generators, please see:
5788Frank DeRemer and Thomas Pennello, Efficient Computation of
5789@acronym{LALR}(1) Look-Ahead Sets, @cite{@acronym{ACM} Transactions on
5790Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
5791pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
5792
fae437e8 5793@node Generalized LR Parsing
c827f760
PE
5794@section Generalized @acronym{LR} (@acronym{GLR}) Parsing
5795@cindex @acronym{GLR} parsing
5796@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2 5797@cindex ambiguous grammars
9d9b8b70 5798@cindex nondeterministic parsing
676385e2 5799
fae437e8
AD
5800Bison produces @emph{deterministic} parsers that choose uniquely
5801when to reduce and which reduction to apply
742e4900 5802based on a summary of the preceding input and on one extra token of lookahead.
676385e2
PH
5803As a result, normal Bison handles a proper subset of the family of
5804context-free languages.
fae437e8 5805Ambiguous grammars, since they have strings with more than one possible
676385e2
PH
5806sequence of reductions cannot have deterministic parsers in this sense.
5807The same is true of languages that require more than one symbol of
742e4900 5808lookahead, since the parser lacks the information necessary to make a
676385e2 5809decision at the point it must be made in a shift-reduce parser.
fae437e8 5810Finally, as previously mentioned (@pxref{Mystery Conflicts}),
676385e2
PH
5811there are languages where Bison's particular choice of how to
5812summarize the input seen so far loses necessary information.
5813
5814When you use the @samp{%glr-parser} declaration in your grammar file,
5815Bison generates a parser that uses a different algorithm, called
c827f760
PE
5816Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR}
5817parser uses the same basic
676385e2
PH
5818algorithm for parsing as an ordinary Bison parser, but behaves
5819differently in cases where there is a shift-reduce conflict that has not
fae437e8 5820been resolved by precedence rules (@pxref{Precedence}) or a
c827f760
PE
5821reduce-reduce conflict. When a @acronym{GLR} parser encounters such a
5822situation, it
fae437e8 5823effectively @emph{splits} into a several parsers, one for each possible
676385e2
PH
5824shift or reduction. These parsers then proceed as usual, consuming
5825tokens in lock-step. Some of the stacks may encounter other conflicts
fae437e8 5826and split further, with the result that instead of a sequence of states,
c827f760 5827a Bison @acronym{GLR} parsing stack is what is in effect a tree of states.
676385e2
PH
5828
5829In effect, each stack represents a guess as to what the proper parse
5830is. Additional input may indicate that a guess was wrong, in which case
5831the appropriate stack silently disappears. Otherwise, the semantics
fae437e8 5832actions generated in each stack are saved, rather than being executed
676385e2 5833immediately. When a stack disappears, its saved semantic actions never
fae437e8 5834get executed. When a reduction causes two stacks to become equivalent,
676385e2
PH
5835their sets of semantic actions are both saved with the state that
5836results from the reduction. We say that two stacks are equivalent
fae437e8 5837when they both represent the same sequence of states,
676385e2
PH
5838and each pair of corresponding states represents a
5839grammar symbol that produces the same segment of the input token
5840stream.
5841
5842Whenever the parser makes a transition from having multiple
c827f760 5843states to having one, it reverts to the normal @acronym{LALR}(1) parsing
676385e2
PH
5844algorithm, after resolving and executing the saved-up actions.
5845At this transition, some of the states on the stack will have semantic
5846values that are sets (actually multisets) of possible actions. The
5847parser tries to pick one of the actions by first finding one whose rule
5848has the highest dynamic precedence, as set by the @samp{%dprec}
fae437e8 5849declaration. Otherwise, if the alternative actions are not ordered by
676385e2 5850precedence, but there the same merging function is declared for both
fae437e8 5851rules by the @samp{%merge} declaration,
676385e2
PH
5852Bison resolves and evaluates both and then calls the merge function on
5853the result. Otherwise, it reports an ambiguity.
5854
c827f760
PE
5855It is possible to use a data structure for the @acronym{GLR} parsing tree that
5856permits the processing of any @acronym{LALR}(1) grammar in linear time (in the
5857size of the input), any unambiguous (not necessarily
5858@acronym{LALR}(1)) grammar in
fae437e8 5859quadratic worst-case time, and any general (possibly ambiguous)
676385e2
PH
5860context-free grammar in cubic worst-case time. However, Bison currently
5861uses a simpler data structure that requires time proportional to the
5862length of the input times the maximum number of stacks required for any
9d9b8b70 5863prefix of the input. Thus, really ambiguous or nondeterministic
676385e2
PH
5864grammars can require exponential time and space to process. Such badly
5865behaving examples, however, are not generally of practical interest.
9d9b8b70 5866Usually, nondeterminism in a grammar is local---the parser is ``in
676385e2 5867doubt'' only for a few tokens at a time. Therefore, the current data
c827f760 5868structure should generally be adequate. On @acronym{LALR}(1) portions of a
676385e2
PH
5869grammar, in particular, it is only slightly slower than with the default
5870Bison parser.
5871
fa7e68c3 5872For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth
f6481e2f
PE
5873Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
5874Generalised @acronym{LR} Parsers, Royal Holloway, University of
5875London, Department of Computer Science, TR-00-12,
5876@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
5877(2000-12-24).
5878
1a059451
PE
5879@node Memory Management
5880@section Memory Management, and How to Avoid Memory Exhaustion
5881@cindex memory exhaustion
5882@cindex memory management
bfa74976
RS
5883@cindex stack overflow
5884@cindex parser stack overflow
5885@cindex overflow of parser stack
5886
1a059451 5887The Bison parser stack can run out of memory if too many tokens are shifted and
bfa74976 5888not reduced. When this happens, the parser function @code{yyparse}
1a059451 5889calls @code{yyerror} and then returns 2.
bfa74976 5890
c827f760 5891Because Bison parsers have growing stacks, hitting the upper limit
d1a1114f
AD
5892usually results from using a right recursion instead of a left
5893recursion, @xref{Recursion, ,Recursive Rules}.
5894
bfa74976
RS
5895@vindex YYMAXDEPTH
5896By defining the macro @code{YYMAXDEPTH}, you can control how deep the
1a059451 5897parser stack can become before memory is exhausted. Define the
bfa74976
RS
5898macro with a value that is an integer. This value is the maximum number
5899of tokens that can be shifted (and not reduced) before overflow.
bfa74976
RS
5900
5901The stack space allowed is not necessarily allocated. If you specify a
1a059451 5902large value for @code{YYMAXDEPTH}, the parser normally allocates a small
bfa74976
RS
5903stack at first, and then makes it bigger by stages as needed. This
5904increasing allocation happens automatically and silently. Therefore,
5905you do not need to make @code{YYMAXDEPTH} painfully small merely to save
5906space for ordinary inputs that do not need much stack.
5907
d7e14fc0
PE
5908However, do not allow @code{YYMAXDEPTH} to be a value so large that
5909arithmetic overflow could occur when calculating the size of the stack
5910space. Also, do not allow @code{YYMAXDEPTH} to be less than
5911@code{YYINITDEPTH}.
5912
bfa74976
RS
5913@cindex default stack limit
5914The default value of @code{YYMAXDEPTH}, if you do not define it, is
591510000.
5916
5917@vindex YYINITDEPTH
5918You can control how much stack is allocated initially by defining the
d7e14fc0
PE
5919macro @code{YYINITDEPTH} to a positive integer. For the C
5920@acronym{LALR}(1) parser, this value must be a compile-time constant
5921unless you are assuming C99 or some other target language or compiler
5922that allows variable-length arrays. The default is 200.
5923
1a059451 5924Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
bfa74976 5925
d1a1114f 5926@c FIXME: C++ output.
c827f760 5927Because of semantical differences between C and C++, the
1a059451
PE
5928@acronym{LALR}(1) parsers in C produced by Bison cannot grow when compiled
5929by C++ compilers. In this precise case (compiling a C parser as C++) you are
5930suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix
5931this deficiency in a future release.
d1a1114f 5932
342b8b6e 5933@node Error Recovery
bfa74976
RS
5934@chapter Error Recovery
5935@cindex error recovery
5936@cindex recovery from errors
5937
6e649e65 5938It is not usually acceptable to have a program terminate on a syntax
bfa74976
RS
5939error. For example, a compiler should recover sufficiently to parse the
5940rest of the input file and check it for errors; a calculator should accept
5941another expression.
5942
5943In a simple interactive command parser where each input is one line, it may
5944be sufficient to allow @code{yyparse} to return 1 on error and have the
5945caller ignore the rest of the input line when that happens (and then call
5946@code{yyparse} again). But this is inadequate for a compiler, because it
5947forgets all the syntactic context leading up to the error. A syntax error
5948deep within a function in the compiler input should not cause the compiler
5949to treat the following line like the beginning of a source file.
5950
5951@findex error
5952You can define how to recover from a syntax error by writing rules to
5953recognize the special token @code{error}. This is a terminal symbol that
5954is always defined (you need not declare it) and reserved for error
5955handling. The Bison parser generates an @code{error} token whenever a
5956syntax error happens; if you have provided a rule to recognize this token
13863333 5957in the current context, the parse can continue.
bfa74976
RS
5958
5959For example:
5960
5961@example
5962stmnts: /* empty string */
5963 | stmnts '\n'
5964 | stmnts exp '\n'
5965 | stmnts error '\n'
5966@end example
5967
5968The fourth rule in this example says that an error followed by a newline
5969makes a valid addition to any @code{stmnts}.
5970
5971What happens if a syntax error occurs in the middle of an @code{exp}? The
5972error recovery rule, interpreted strictly, applies to the precise sequence
5973of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
5974the middle of an @code{exp}, there will probably be some additional tokens
5975and subexpressions on the stack after the last @code{stmnts}, and there
5976will be tokens to read before the next newline. So the rule is not
5977applicable in the ordinary way.
5978
5979But Bison can force the situation to fit the rule, by discarding part of
72f889cc
AD
5980the semantic context and part of the input. First it discards states
5981and objects from the stack until it gets back to a state in which the
bfa74976 5982@code{error} token is acceptable. (This means that the subexpressions
72f889cc
AD
5983already parsed are discarded, back to the last complete @code{stmnts}.)
5984At this point the @code{error} token can be shifted. Then, if the old
742e4900 5985lookahead token is not acceptable to be shifted next, the parser reads
bfa74976 5986tokens and discards them until it finds a token which is acceptable. In
72f889cc
AD
5987this example, Bison reads and discards input until the next newline so
5988that the fourth rule can apply. Note that discarded symbols are
5989possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
5990Discarded Symbols}, for a means to reclaim this memory.
bfa74976
RS
5991
5992The choice of error rules in the grammar is a choice of strategies for
5993error recovery. A simple and useful strategy is simply to skip the rest of
5994the current input line or current statement if an error is detected:
5995
5996@example
72d2299c 5997stmnt: error ';' /* On error, skip until ';' is read. */
bfa74976
RS
5998@end example
5999
6000It is also useful to recover to the matching close-delimiter of an
6001opening-delimiter that has already been parsed. Otherwise the
6002close-delimiter will probably appear to be unmatched, and generate another,
6003spurious error message:
6004
6005@example
6006primary: '(' expr ')'
6007 | '(' error ')'
6008 @dots{}
6009 ;
6010@end example
6011
6012Error recovery strategies are necessarily guesses. When they guess wrong,
6013one syntax error often leads to another. In the above example, the error
6014recovery rule guesses that an error is due to bad input within one
6015@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
6016middle of a valid @code{stmnt}. After the error recovery rule recovers
6017from the first error, another syntax error will be found straightaway,
6018since the text following the spurious semicolon is also an invalid
6019@code{stmnt}.
6020
6021To prevent an outpouring of error messages, the parser will output no error
6022message for another syntax error that happens shortly after the first; only
6023after three consecutive input tokens have been successfully shifted will
6024error messages resume.
6025
6026Note that rules which accept the @code{error} token may have actions, just
6027as any other rules can.
6028
6029@findex yyerrok
6030You can make error messages resume immediately by using the macro
6031@code{yyerrok} in an action. If you do this in the error rule's action, no
6032error messages will be suppressed. This macro requires no arguments;
6033@samp{yyerrok;} is a valid C statement.
6034
6035@findex yyclearin
742e4900 6036The previous lookahead token is reanalyzed immediately after an error. If
bfa74976
RS
6037this is unacceptable, then the macro @code{yyclearin} may be used to clear
6038this token. Write the statement @samp{yyclearin;} in the error rule's
6039action.
32c29292 6040@xref{Action Features, ,Special Features for Use in Actions}.
bfa74976 6041
6e649e65 6042For example, suppose that on a syntax error, an error handling routine is
bfa74976
RS
6043called that advances the input stream to some point where parsing should
6044once again commence. The next symbol returned by the lexical scanner is
742e4900 6045probably correct. The previous lookahead token ought to be discarded
bfa74976
RS
6046with @samp{yyclearin;}.
6047
6048@vindex YYRECOVERING
02103984
PE
6049The expression @code{YYRECOVERING ()} yields 1 when the parser
6050is recovering from a syntax error, and 0 otherwise.
6051Syntax error diagnostics are suppressed while recovering from a syntax
6052error.
bfa74976 6053
342b8b6e 6054@node Context Dependency
bfa74976
RS
6055@chapter Handling Context Dependencies
6056
6057The Bison paradigm is to parse tokens first, then group them into larger
6058syntactic units. In many languages, the meaning of a token is affected by
6059its context. Although this violates the Bison paradigm, certain techniques
6060(known as @dfn{kludges}) may enable you to write Bison parsers for such
6061languages.
6062
6063@menu
6064* Semantic Tokens:: Token parsing can depend on the semantic context.
6065* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
6066* Tie-in Recovery:: Lexical tie-ins have implications for how
6067 error recovery rules must be written.
6068@end menu
6069
6070(Actually, ``kludge'' means any technique that gets its job done but is
6071neither clean nor robust.)
6072
342b8b6e 6073@node Semantic Tokens
bfa74976
RS
6074@section Semantic Info in Token Types
6075
6076The C language has a context dependency: the way an identifier is used
6077depends on what its current meaning is. For example, consider this:
6078
6079@example
6080foo (x);
6081@end example
6082
6083This looks like a function call statement, but if @code{foo} is a typedef
6084name, then this is actually a declaration of @code{x}. How can a Bison
6085parser for C decide how to parse this input?
6086
c827f760 6087The method used in @acronym{GNU} C is to have two different token types,
bfa74976
RS
6088@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
6089identifier, it looks up the current declaration of the identifier in order
6090to decide which token type to return: @code{TYPENAME} if the identifier is
6091declared as a typedef, @code{IDENTIFIER} otherwise.
6092
6093The grammar rules can then express the context dependency by the choice of
6094token type to recognize. @code{IDENTIFIER} is accepted as an expression,
6095but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
6096@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
6097is @emph{not} significant, such as in declarations that can shadow a
6098typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
6099accepted---there is one rule for each of the two token types.
6100
6101This technique is simple to use if the decision of which kinds of
6102identifiers to allow is made at a place close to where the identifier is
6103parsed. But in C this is not always so: C allows a declaration to
6104redeclare a typedef name provided an explicit type has been specified
6105earlier:
6106
6107@example
3a4f411f
PE
6108typedef int foo, bar;
6109int baz (void)
6110@{
6111 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
6112 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
6113 return foo (bar);
6114@}
bfa74976
RS
6115@end example
6116
6117Unfortunately, the name being declared is separated from the declaration
6118construct itself by a complicated syntactic structure---the ``declarator''.
6119
9ecbd125 6120As a result, part of the Bison parser for C needs to be duplicated, with
14ded682
AD
6121all the nonterminal names changed: once for parsing a declaration in
6122which a typedef name can be redefined, and once for parsing a
6123declaration in which that can't be done. Here is a part of the
6124duplication, with actions omitted for brevity:
bfa74976
RS
6125
6126@example
6127initdcl:
6128 declarator maybeasm '='
6129 init
6130 | declarator maybeasm
6131 ;
6132
6133notype_initdcl:
6134 notype_declarator maybeasm '='
6135 init
6136 | notype_declarator maybeasm
6137 ;
6138@end example
6139
6140@noindent
6141Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
6142cannot. The distinction between @code{declarator} and
6143@code{notype_declarator} is the same sort of thing.
6144
6145There is some similarity between this technique and a lexical tie-in
6146(described next), in that information which alters the lexical analysis is
6147changed during parsing by other parts of the program. The difference is
6148here the information is global, and is used for other purposes in the
6149program. A true lexical tie-in has a special-purpose flag controlled by
6150the syntactic context.
6151
342b8b6e 6152@node Lexical Tie-ins
bfa74976
RS
6153@section Lexical Tie-ins
6154@cindex lexical tie-in
6155
6156One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
6157which is set by Bison actions, whose purpose is to alter the way tokens are
6158parsed.
6159
6160For example, suppose we have a language vaguely like C, but with a special
6161construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
6162an expression in parentheses in which all integers are hexadecimal. In
6163particular, the token @samp{a1b} must be treated as an integer rather than
6164as an identifier if it appears in that context. Here is how you can do it:
6165
6166@example
6167@group
6168%@{
38a92d50
PE
6169 int hexflag;
6170 int yylex (void);
6171 void yyerror (char const *);
bfa74976
RS
6172%@}
6173%%
6174@dots{}
6175@end group
6176@group
6177expr: IDENTIFIER
6178 | constant
6179 | HEX '('
6180 @{ hexflag = 1; @}
6181 expr ')'
6182 @{ hexflag = 0;
6183 $$ = $4; @}
6184 | expr '+' expr
6185 @{ $$ = make_sum ($1, $3); @}
6186 @dots{}
6187 ;
6188@end group
6189
6190@group
6191constant:
6192 INTEGER
6193 | STRING
6194 ;
6195@end group
6196@end example
6197
6198@noindent
6199Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
6200it is nonzero, all integers are parsed in hexadecimal, and tokens starting
6201with letters are parsed as integers if possible.
6202
342b8b6e
AD
6203The declaration of @code{hexflag} shown in the prologue of the parser file
6204is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}).
75f5aaea 6205You must also write the code in @code{yylex} to obey the flag.
bfa74976 6206
342b8b6e 6207@node Tie-in Recovery
bfa74976
RS
6208@section Lexical Tie-ins and Error Recovery
6209
6210Lexical tie-ins make strict demands on any error recovery rules you have.
6211@xref{Error Recovery}.
6212
6213The reason for this is that the purpose of an error recovery rule is to
6214abort the parsing of one construct and resume in some larger construct.
6215For example, in C-like languages, a typical error recovery rule is to skip
6216tokens until the next semicolon, and then start a new statement, like this:
6217
6218@example
6219stmt: expr ';'
6220 | IF '(' expr ')' stmt @{ @dots{} @}
6221 @dots{}
6222 error ';'
6223 @{ hexflag = 0; @}
6224 ;
6225@end example
6226
6227If there is a syntax error in the middle of a @samp{hex (@var{expr})}
6228construct, this error rule will apply, and then the action for the
6229completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
6230remain set for the entire rest of the input, or until the next @code{hex}
6231keyword, causing identifiers to be misinterpreted as integers.
6232
6233To avoid this problem the error recovery rule itself clears @code{hexflag}.
6234
6235There may also be an error recovery rule that works within expressions.
6236For example, there could be a rule which applies within parentheses
6237and skips to the close-parenthesis:
6238
6239@example
6240@group
6241expr: @dots{}
6242 | '(' expr ')'
6243 @{ $$ = $2; @}
6244 | '(' error ')'
6245 @dots{}
6246@end group
6247@end example
6248
6249If this rule acts within the @code{hex} construct, it is not going to abort
6250that construct (since it applies to an inner level of parentheses within
6251the construct). Therefore, it should not clear the flag: the rest of
6252the @code{hex} construct should be parsed with the flag still in effect.
6253
6254What if there is an error recovery rule which might abort out of the
6255@code{hex} construct or might not, depending on circumstances? There is no
6256way you can write the action to determine whether a @code{hex} construct is
6257being aborted or not. So if you are using a lexical tie-in, you had better
6258make sure your error recovery rules are not of this kind. Each rule must
6259be such that you can be sure that it always will, or always won't, have to
6260clear the flag.
6261
ec3bc396
AD
6262@c ================================================== Debugging Your Parser
6263
342b8b6e 6264@node Debugging
bfa74976 6265@chapter Debugging Your Parser
ec3bc396
AD
6266
6267Developing a parser can be a challenge, especially if you don't
6268understand the algorithm (@pxref{Algorithm, ,The Bison Parser
6269Algorithm}). Even so, sometimes a detailed description of the automaton
6270can help (@pxref{Understanding, , Understanding Your Parser}), or
6271tracing the execution of the parser can give some insight on why it
6272behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
6273
6274@menu
6275* Understanding:: Understanding the structure of your parser.
6276* Tracing:: Tracing the execution of your parser.
6277@end menu
6278
6279@node Understanding
6280@section Understanding Your Parser
6281
6282As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
6283Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
6284frequent than one would hope), looking at this automaton is required to
6285tune or simply fix a parser. Bison provides two different
c827f760 6286representation of it, either textually or graphically (as a @acronym{VCG}
ec3bc396
AD
6287file).
6288
6289The textual file is generated when the options @option{--report} or
6290@option{--verbose} are specified, see @xref{Invocation, , Invoking
6291Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
6292the parser output file name, and adding @samp{.output} instead.
6293Therefore, if the input file is @file{foo.y}, then the parser file is
6294called @file{foo.tab.c} by default. As a consequence, the verbose
6295output file is called @file{foo.output}.
6296
6297The following grammar file, @file{calc.y}, will be used in the sequel:
6298
6299@example
6300%token NUM STR
6301%left '+' '-'
6302%left '*'
6303%%
6304exp: exp '+' exp
6305 | exp '-' exp
6306 | exp '*' exp
6307 | exp '/' exp
6308 | NUM
6309 ;
6310useless: STR;
6311%%
6312@end example
6313
88bce5a2
AD
6314@command{bison} reports:
6315
6316@example
6317calc.y: warning: 1 useless nonterminal and 1 useless rule
6318calc.y:11.1-7: warning: useless nonterminal: useless
5a99098d
PE
6319calc.y:11.10-12: warning: useless rule: useless: STR
6320calc.y: conflicts: 7 shift/reduce
88bce5a2
AD
6321@end example
6322
6323When given @option{--report=state}, in addition to @file{calc.tab.c}, it
6324creates a file @file{calc.output} with contents detailed below. The
6325order of the output and the exact presentation might vary, but the
6326interpretation is the same.
ec3bc396
AD
6327
6328The first section includes details on conflicts that were solved thanks
6329to precedence and/or associativity:
6330
6331@example
6332Conflict in state 8 between rule 2 and token '+' resolved as reduce.
6333Conflict in state 8 between rule 2 and token '-' resolved as reduce.
6334Conflict in state 8 between rule 2 and token '*' resolved as shift.
6335@exdent @dots{}
6336@end example
6337
6338@noindent
6339The next section lists states that still have conflicts.
6340
6341@example
5a99098d
PE
6342State 8 conflicts: 1 shift/reduce
6343State 9 conflicts: 1 shift/reduce
6344State 10 conflicts: 1 shift/reduce
6345State 11 conflicts: 4 shift/reduce
ec3bc396
AD
6346@end example
6347
6348@noindent
6349@cindex token, useless
6350@cindex useless token
6351@cindex nonterminal, useless
6352@cindex useless nonterminal
6353@cindex rule, useless
6354@cindex useless rule
6355The next section reports useless tokens, nonterminal and rules. Useless
6356nonterminals and rules are removed in order to produce a smaller parser,
6357but useless tokens are preserved, since they might be used by the
6358scanner (note the difference between ``useless'' and ``not used''
6359below):
6360
6361@example
6362Useless nonterminals:
6363 useless
6364
6365Terminals which are not used:
6366 STR
6367
6368Useless rules:
6369#6 useless: STR;
6370@end example
6371
6372@noindent
6373The next section reproduces the exact grammar that Bison used:
6374
6375@example
6376Grammar
6377
6378 Number, Line, Rule
88bce5a2 6379 0 5 $accept -> exp $end
ec3bc396
AD
6380 1 5 exp -> exp '+' exp
6381 2 6 exp -> exp '-' exp
6382 3 7 exp -> exp '*' exp
6383 4 8 exp -> exp '/' exp
6384 5 9 exp -> NUM
6385@end example
6386
6387@noindent
6388and reports the uses of the symbols:
6389
6390@example
6391Terminals, with rules where they appear
6392
88bce5a2 6393$end (0) 0
ec3bc396
AD
6394'*' (42) 3
6395'+' (43) 1
6396'-' (45) 2
6397'/' (47) 4
6398error (256)
6399NUM (258) 5
6400
6401Nonterminals, with rules where they appear
6402
88bce5a2 6403$accept (8)
ec3bc396
AD
6404 on left: 0
6405exp (9)
6406 on left: 1 2 3 4 5, on right: 0 1 2 3 4
6407@end example
6408
6409@noindent
6410@cindex item
6411@cindex pointed rule
6412@cindex rule, pointed
6413Bison then proceeds onto the automaton itself, describing each state
6414with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
6415item is a production rule together with a point (marked by @samp{.})
6416that the input cursor.
6417
6418@example
6419state 0
6420
88bce5a2 6421 $accept -> . exp $ (rule 0)
ec3bc396 6422
2a8d363a 6423 NUM shift, and go to state 1
ec3bc396 6424
2a8d363a 6425 exp go to state 2
ec3bc396
AD
6426@end example
6427
6428This reads as follows: ``state 0 corresponds to being at the very
6429beginning of the parsing, in the initial rule, right before the start
6430symbol (here, @code{exp}). When the parser returns to this state right
6431after having reduced a rule that produced an @code{exp}, the control
6432flow jumps to state 2. If there is no such transition on a nonterminal
742e4900 6433symbol, and the lookahead is a @code{NUM}, then this token is shifted on
ec3bc396 6434the parse stack, and the control flow jumps to state 1. Any other
742e4900 6435lookahead triggers a syntax error.''
ec3bc396
AD
6436
6437@cindex core, item set
6438@cindex item set core
6439@cindex kernel, item set
6440@cindex item set core
6441Even though the only active rule in state 0 seems to be rule 0, the
742e4900 6442report lists @code{NUM} as a lookahead token because @code{NUM} can be
ec3bc396
AD
6443at the beginning of any rule deriving an @code{exp}. By default Bison
6444reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
6445you want to see more detail you can invoke @command{bison} with
6446@option{--report=itemset} to list all the items, include those that can
6447be derived:
6448
6449@example
6450state 0
6451
88bce5a2 6452 $accept -> . exp $ (rule 0)
ec3bc396
AD
6453 exp -> . exp '+' exp (rule 1)
6454 exp -> . exp '-' exp (rule 2)
6455 exp -> . exp '*' exp (rule 3)
6456 exp -> . exp '/' exp (rule 4)
6457 exp -> . NUM (rule 5)
6458
6459 NUM shift, and go to state 1
6460
6461 exp go to state 2
6462@end example
6463
6464@noindent
6465In the state 1...
6466
6467@example
6468state 1
6469
6470 exp -> NUM . (rule 5)
6471
2a8d363a 6472 $default reduce using rule 5 (exp)
ec3bc396
AD
6473@end example
6474
6475@noindent
742e4900 6476the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
ec3bc396
AD
6477(@samp{$default}), the parser will reduce it. If it was coming from
6478state 0, then, after this reduction it will return to state 0, and will
6479jump to state 2 (@samp{exp: go to state 2}).
6480
6481@example
6482state 2
6483
88bce5a2 6484 $accept -> exp . $ (rule 0)
ec3bc396
AD
6485 exp -> exp . '+' exp (rule 1)
6486 exp -> exp . '-' exp (rule 2)
6487 exp -> exp . '*' exp (rule 3)
6488 exp -> exp . '/' exp (rule 4)
6489
2a8d363a
AD
6490 $ shift, and go to state 3
6491 '+' shift, and go to state 4
6492 '-' shift, and go to state 5
6493 '*' shift, and go to state 6
6494 '/' shift, and go to state 7
ec3bc396
AD
6495@end example
6496
6497@noindent
6498In state 2, the automaton can only shift a symbol. For instance,
742e4900 6499because of the item @samp{exp -> exp . '+' exp}, if the lookahead if
ec3bc396
AD
6500@samp{+}, it will be shifted on the parse stack, and the automaton
6501control will jump to state 4, corresponding to the item @samp{exp -> exp
6502'+' . exp}. Since there is no default action, any other token than
6e649e65 6503those listed above will trigger a syntax error.
ec3bc396
AD
6504
6505The state 3 is named the @dfn{final state}, or the @dfn{accepting
6506state}:
6507
6508@example
6509state 3
6510
88bce5a2 6511 $accept -> exp $ . (rule 0)
ec3bc396 6512
2a8d363a 6513 $default accept
ec3bc396
AD
6514@end example
6515
6516@noindent
6517the initial rule is completed (the start symbol and the end
6518of input were read), the parsing exits successfully.
6519
6520The interpretation of states 4 to 7 is straightforward, and is left to
6521the reader.
6522
6523@example
6524state 4
6525
6526 exp -> exp '+' . exp (rule 1)
6527
2a8d363a 6528 NUM shift, and go to state 1
ec3bc396 6529
2a8d363a 6530 exp go to state 8
ec3bc396
AD
6531
6532state 5
6533
6534 exp -> exp '-' . exp (rule 2)
6535
2a8d363a 6536 NUM shift, and go to state 1
ec3bc396 6537
2a8d363a 6538 exp go to state 9
ec3bc396
AD
6539
6540state 6
6541
6542 exp -> exp '*' . exp (rule 3)
6543
2a8d363a 6544 NUM shift, and go to state 1
ec3bc396 6545
2a8d363a 6546 exp go to state 10
ec3bc396
AD
6547
6548state 7
6549
6550 exp -> exp '/' . exp (rule 4)
6551
2a8d363a 6552 NUM shift, and go to state 1
ec3bc396 6553
2a8d363a 6554 exp go to state 11
ec3bc396
AD
6555@end example
6556
5a99098d
PE
6557As was announced in beginning of the report, @samp{State 8 conflicts:
65581 shift/reduce}:
ec3bc396
AD
6559
6560@example
6561state 8
6562
6563 exp -> exp . '+' exp (rule 1)
6564 exp -> exp '+' exp . (rule 1)
6565 exp -> exp . '-' exp (rule 2)
6566 exp -> exp . '*' exp (rule 3)
6567 exp -> exp . '/' exp (rule 4)
6568
2a8d363a
AD
6569 '*' shift, and go to state 6
6570 '/' shift, and go to state 7
ec3bc396 6571
2a8d363a
AD
6572 '/' [reduce using rule 1 (exp)]
6573 $default reduce using rule 1 (exp)
ec3bc396
AD
6574@end example
6575
742e4900 6576Indeed, there are two actions associated to the lookahead @samp{/}:
ec3bc396
AD
6577either shifting (and going to state 7), or reducing rule 1. The
6578conflict means that either the grammar is ambiguous, or the parser lacks
6579information to make the right decision. Indeed the grammar is
6580ambiguous, as, since we did not specify the precedence of @samp{/}, the
6581sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
6582NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
6583NUM}, which corresponds to reducing rule 1.
6584
c827f760 6585Because in @acronym{LALR}(1) parsing a single decision can be made, Bison
ec3bc396
AD
6586arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
6587Shift/Reduce Conflicts}. Discarded actions are reported in between
6588square brackets.
6589
6590Note that all the previous states had a single possible action: either
6591shifting the next token and going to the corresponding state, or
6592reducing a single rule. In the other cases, i.e., when shifting
6593@emph{and} reducing is possible or when @emph{several} reductions are
742e4900
JD
6594possible, the lookahead is required to select the action. State 8 is
6595one such state: if the lookahead is @samp{*} or @samp{/} then the action
ec3bc396
AD
6596is shifting, otherwise the action is reducing rule 1. In other words,
6597the first two items, corresponding to rule 1, are not eligible when the
742e4900 6598lookahead token is @samp{*}, since we specified that @samp{*} has higher
8dd162d3 6599precedence than @samp{+}. More generally, some items are eligible only
742e4900
JD
6600with some set of possible lookahead tokens. When run with
6601@option{--report=lookahead}, Bison specifies these lookahead tokens:
ec3bc396
AD
6602
6603@example
6604state 8
6605
6606 exp -> exp . '+' exp [$, '+', '-', '/'] (rule 1)
6607 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
6608 exp -> exp . '-' exp (rule 2)
6609 exp -> exp . '*' exp (rule 3)
6610 exp -> exp . '/' exp (rule 4)
6611
6612 '*' shift, and go to state 6
6613 '/' shift, and go to state 7
6614
6615 '/' [reduce using rule 1 (exp)]
6616 $default reduce using rule 1 (exp)
6617@end example
6618
6619The remaining states are similar:
6620
6621@example
6622state 9
6623
6624 exp -> exp . '+' exp (rule 1)
6625 exp -> exp . '-' exp (rule 2)
6626 exp -> exp '-' exp . (rule 2)
6627 exp -> exp . '*' exp (rule 3)
6628 exp -> exp . '/' exp (rule 4)
6629
2a8d363a
AD
6630 '*' shift, and go to state 6
6631 '/' shift, and go to state 7
ec3bc396 6632
2a8d363a
AD
6633 '/' [reduce using rule 2 (exp)]
6634 $default reduce using rule 2 (exp)
ec3bc396
AD
6635
6636state 10
6637
6638 exp -> exp . '+' exp (rule 1)
6639 exp -> exp . '-' exp (rule 2)
6640 exp -> exp . '*' exp (rule 3)
6641 exp -> exp '*' exp . (rule 3)
6642 exp -> exp . '/' exp (rule 4)
6643
2a8d363a 6644 '/' shift, and go to state 7
ec3bc396 6645
2a8d363a
AD
6646 '/' [reduce using rule 3 (exp)]
6647 $default reduce using rule 3 (exp)
ec3bc396
AD
6648
6649state 11
6650
6651 exp -> exp . '+' exp (rule 1)
6652 exp -> exp . '-' exp (rule 2)
6653 exp -> exp . '*' exp (rule 3)
6654 exp -> exp . '/' exp (rule 4)
6655 exp -> exp '/' exp . (rule 4)
6656
2a8d363a
AD
6657 '+' shift, and go to state 4
6658 '-' shift, and go to state 5
6659 '*' shift, and go to state 6
6660 '/' shift, and go to state 7
ec3bc396 6661
2a8d363a
AD
6662 '+' [reduce using rule 4 (exp)]
6663 '-' [reduce using rule 4 (exp)]
6664 '*' [reduce using rule 4 (exp)]
6665 '/' [reduce using rule 4 (exp)]
6666 $default reduce using rule 4 (exp)
ec3bc396
AD
6667@end example
6668
6669@noindent
fa7e68c3
PE
6670Observe that state 11 contains conflicts not only due to the lack of
6671precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
6672@samp{*}, but also because the
ec3bc396
AD
6673associativity of @samp{/} is not specified.
6674
6675
6676@node Tracing
6677@section Tracing Your Parser
bfa74976
RS
6678@findex yydebug
6679@cindex debugging
6680@cindex tracing the parser
6681
6682If a Bison grammar compiles properly but doesn't do what you want when it
6683runs, the @code{yydebug} parser-trace feature can help you figure out why.
6684
3ded9a63
AD
6685There are several means to enable compilation of trace facilities:
6686
6687@table @asis
6688@item the macro @code{YYDEBUG}
6689@findex YYDEBUG
6690Define the macro @code{YYDEBUG} to a nonzero value when you compile the
c827f760 6691parser. This is compliant with @acronym{POSIX} Yacc. You could use
3ded9a63
AD
6692@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
6693YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
6694Prologue}).
6695
6696@item the option @option{-t}, @option{--debug}
6697Use the @samp{-t} option when you run Bison (@pxref{Invocation,
c827f760 6698,Invoking Bison}). This is @acronym{POSIX} compliant too.
3ded9a63
AD
6699
6700@item the directive @samp{%debug}
6701@findex %debug
6702Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison
6703Declaration Summary}). This is a Bison extension, which will prove
6704useful when Bison will output parsers for languages that don't use a
c827f760
PE
6705preprocessor. Unless @acronym{POSIX} and Yacc portability matter to
6706you, this is
3ded9a63
AD
6707the preferred solution.
6708@end table
6709
6710We suggest that you always enable the debug option so that debugging is
6711always possible.
bfa74976 6712
02a81e05 6713The trace facility outputs messages with macro calls of the form
e2742e46 6714@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
02a81e05 6715@var{format} and @var{args} are the usual @code{printf} format and
4947ebdb
PE
6716arguments. If you define @code{YYDEBUG} to a nonzero value but do not
6717define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
e4e1a4dc 6718and @code{YYPRINTF} is defined to @code{fprintf}.
bfa74976
RS
6719
6720Once you have compiled the program with trace facilities, the way to
6721request a trace is to store a nonzero value in the variable @code{yydebug}.
6722You can do this by making the C code do it (in @code{main}, perhaps), or
6723you can alter the value with a C debugger.
6724
6725Each step taken by the parser when @code{yydebug} is nonzero produces a
6726line or two of trace information, written on @code{stderr}. The trace
6727messages tell you these things:
6728
6729@itemize @bullet
6730@item
6731Each time the parser calls @code{yylex}, what kind of token was read.
6732
6733@item
6734Each time a token is shifted, the depth and complete contents of the
6735state stack (@pxref{Parser States}).
6736
6737@item
6738Each time a rule is reduced, which rule it is, and the complete contents
6739of the state stack afterward.
6740@end itemize
6741
6742To make sense of this information, it helps to refer to the listing file
704a47c4
AD
6743produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
6744Bison}). This file shows the meaning of each state in terms of
6745positions in various rules, and also what each state will do with each
6746possible input token. As you read the successive trace messages, you
6747can see that the parser is functioning according to its specification in
6748the listing file. Eventually you will arrive at the place where
6749something undesirable happens, and you will see which parts of the
6750grammar are to blame.
bfa74976
RS
6751
6752The parser file is a C program and you can use C debuggers on it, but it's
6753not easy to interpret what it is doing. The parser function is a
6754finite-state machine interpreter, and aside from the actions it executes
6755the same code over and over. Only the values of variables show where in
6756the grammar it is working.
6757
6758@findex YYPRINT
6759The debugging information normally gives the token type of each token
6760read, but not its semantic value. You can optionally define a macro
6761named @code{YYPRINT} to provide a way to print the value. If you define
6762@code{YYPRINT}, it should take three arguments. The parser will pass a
6763standard I/O stream, the numeric code for the token type, and the token
6764value (from @code{yylval}).
6765
6766Here is an example of @code{YYPRINT} suitable for the multi-function
6767calculator (@pxref{Mfcalc Decl, ,Declarations for @code{mfcalc}}):
6768
6769@smallexample
38a92d50
PE
6770%@{
6771 static void print_token_value (FILE *, int, YYSTYPE);
6772 #define YYPRINT(file, type, value) print_token_value (file, type, value)
6773%@}
6774
6775@dots{} %% @dots{} %% @dots{}
bfa74976
RS
6776
6777static void
831d3c99 6778print_token_value (FILE *file, int type, YYSTYPE value)
bfa74976
RS
6779@{
6780 if (type == VAR)
d3c4e709 6781 fprintf (file, "%s", value.tptr->name);
bfa74976 6782 else if (type == NUM)
d3c4e709 6783 fprintf (file, "%d", value.val);
bfa74976
RS
6784@}
6785@end smallexample
6786
ec3bc396
AD
6787@c ================================================= Invoking Bison
6788
342b8b6e 6789@node Invocation
bfa74976
RS
6790@chapter Invoking Bison
6791@cindex invoking Bison
6792@cindex Bison invocation
6793@cindex options for invoking Bison
6794
6795The usual way to invoke Bison is as follows:
6796
6797@example
6798bison @var{infile}
6799@end example
6800
6801Here @var{infile} is the grammar file name, which usually ends in
6802@samp{.y}. The parser file's name is made by replacing the @samp{.y}
fa4d969f
PE
6803with @samp{.tab.c} and removing any leading directory. Thus, the
6804@samp{bison foo.y} file name yields
6805@file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields
6806@file{foo.tab.c}. It's also possible, in case you are writing
79282c6c 6807C++ code instead of C in your grammar file, to name it @file{foo.ypp}
72d2299c
PE
6808or @file{foo.y++}. Then, the output files will take an extension like
6809the given one as input (respectively @file{foo.tab.cpp} and
6810@file{foo.tab.c++}).
fa4d969f 6811This feature takes effect with all options that manipulate file names like
234a3be3
AD
6812@samp{-o} or @samp{-d}.
6813
6814For example :
6815
6816@example
6817bison -d @var{infile.yxx}
6818@end example
84163231 6819@noindent
72d2299c 6820will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
234a3be3
AD
6821
6822@example
b56471a6 6823bison -d -o @var{output.c++} @var{infile.y}
234a3be3 6824@end example
84163231 6825@noindent
234a3be3
AD
6826will produce @file{output.c++} and @file{outfile.h++}.
6827
397ec073
PE
6828For compatibility with @acronym{POSIX}, the standard Bison
6829distribution also contains a shell script called @command{yacc} that
6830invokes Bison with the @option{-y} option.
6831
bfa74976 6832@menu
13863333 6833* Bison Options:: All the options described in detail,
c827f760 6834 in alphabetical order by short options.
bfa74976 6835* Option Cross Key:: Alphabetical list of long options.
93dd49ab 6836* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
bfa74976
RS
6837@end menu
6838
342b8b6e 6839@node Bison Options
bfa74976
RS
6840@section Bison Options
6841
6842Bison supports both traditional single-letter options and mnemonic long
6843option names. Long option names are indicated with @samp{--} instead of
6844@samp{-}. Abbreviations for option names are allowed as long as they
6845are unique. When a long option takes an argument, like
6846@samp{--file-prefix}, connect the option name and the argument with
6847@samp{=}.
6848
6849Here is a list of options that can be used with Bison, alphabetized by
6850short option. It is followed by a cross key alphabetized by long
6851option.
6852
89cab50d
AD
6853@c Please, keep this ordered as in `bison --help'.
6854@noindent
6855Operations modes:
6856@table @option
6857@item -h
6858@itemx --help
6859Print a summary of the command-line options to Bison and exit.
bfa74976 6860
89cab50d
AD
6861@item -V
6862@itemx --version
6863Print the version number of Bison and exit.
bfa74976 6864
f7ab6a50
PE
6865@item --print-localedir
6866Print the name of the directory containing locale-dependent data.
6867
89cab50d
AD
6868@item -y
6869@itemx --yacc
54662697
PE
6870Act more like the traditional Yacc command. This can cause
6871different diagnostics to be generated, and may change behavior in
6872other minor ways. Most importantly, imitate Yacc's output
6873file name conventions, so that the parser output file is called
89cab50d 6874@file{y.tab.c}, and the other outputs are called @file{y.output} and
b931235e
JD
6875@file{y.tab.h}.
6876Also, if generating an @acronym{LALR}(1) parser in C, generate @code{#define}
6877statements in addition to an @code{enum} to associate token numbers with token
6878names.
6879Thus, the following shell script can substitute for Yacc, and the Bison
6880distribution contains such a script for compatibility with @acronym{POSIX}:
bfa74976 6881
89cab50d 6882@example
397ec073 6883#! /bin/sh
26e06a21 6884bison -y "$@@"
89cab50d 6885@end example
54662697
PE
6886
6887The @option{-y}/@option{--yacc} option is intended for use with
6888traditional Yacc grammars. If your grammar uses a Bison extension
6889like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
6890this option is specified.
6891
89cab50d
AD
6892@end table
6893
6894@noindent
6895Tuning the parser:
6896
6897@table @option
cd5bd6ac
AD
6898@item -S @var{file}
6899@itemx --skeleton=@var{file}
6900Specify the skeleton to use. You probably don't need this option unless
6901you are developing Bison.
6902
89cab50d
AD
6903@item -t
6904@itemx --debug
4947ebdb
PE
6905In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
6906already defined, so that the debugging facilities are compiled.
ec3bc396 6907@xref{Tracing, ,Tracing Your Parser}.
89cab50d
AD
6908
6909@item --locations
d8988b2f 6910Pretend that @code{%locations} was specified. @xref{Decl Summary}.
89cab50d
AD
6911
6912@item -p @var{prefix}
6913@itemx --name-prefix=@var{prefix}
d8988b2f
AD
6914Pretend that @code{%name-prefix="@var{prefix}"} was specified.
6915@xref{Decl Summary}.
bfa74976
RS
6916
6917@item -l
6918@itemx --no-lines
6919Don't put any @code{#line} preprocessor commands in the parser file.
6920Ordinarily Bison puts them in the parser file so that the C compiler
6921and debuggers will associate errors with your source file, the
6922grammar file. This option causes them to associate errors with the
95e742f7 6923parser file, treating it as an independent source file in its own right.
bfa74976 6924
931c7513
RS
6925@item -n
6926@itemx --no-parser
d8988b2f 6927Pretend that @code{%no-parser} was specified. @xref{Decl Summary}.
931c7513 6928
89cab50d
AD
6929@item -k
6930@itemx --token-table
d8988b2f 6931Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
89cab50d 6932@end table
bfa74976 6933
89cab50d
AD
6934@noindent
6935Adjust the output:
bfa74976 6936
89cab50d
AD
6937@table @option
6938@item -d
d8988b2f
AD
6939@itemx --defines
6940Pretend that @code{%defines} was specified, i.e., write an extra output
6deb4447 6941file containing macro definitions for the token type names defined in
4bfd5e4e 6942the grammar, as well as a few other declarations. @xref{Decl Summary}.
931c7513 6943
342b8b6e 6944@item --defines=@var{defines-file}
d8988b2f 6945Same as above, but save in the file @var{defines-file}.
342b8b6e 6946
89cab50d
AD
6947@item -b @var{file-prefix}
6948@itemx --file-prefix=@var{prefix}
aa08666d 6949Pretend that @code{%file-prefix} was specified, i.e, specify prefix to use
72d2299c 6950for all Bison output file names. @xref{Decl Summary}.
bfa74976 6951
ec3bc396
AD
6952@item -r @var{things}
6953@itemx --report=@var{things}
6954Write an extra output file containing verbose description of the comma
6955separated list of @var{things} among:
6956
6957@table @code
6958@item state
6959Description of the grammar, conflicts (resolved and unresolved), and
c827f760 6960@acronym{LALR} automaton.
ec3bc396 6961
742e4900 6962@item lookahead
ec3bc396 6963Implies @code{state} and augments the description of the automaton with
742e4900 6964each rule's lookahead set.
ec3bc396
AD
6965
6966@item itemset
6967Implies @code{state} and augments the description of the automaton with
6968the full set of items for each state, instead of its core only.
6969@end table
6970
bfa74976
RS
6971@item -v
6972@itemx --verbose
6deb4447
AD
6973Pretend that @code{%verbose} was specified, i.e, write an extra output
6974file containing verbose descriptions of the grammar and
72d2299c 6975parser. @xref{Decl Summary}.
bfa74976 6976
fa4d969f
PE
6977@item -o @var{file}
6978@itemx --output=@var{file}
6979Specify the @var{file} for the parser file.
bfa74976 6980
fa4d969f 6981The other output files' names are constructed from @var{file} as
d8988b2f 6982described under the @samp{-v} and @samp{-d} options.
342b8b6e
AD
6983
6984@item -g
c827f760
PE
6985Output a @acronym{VCG} definition of the @acronym{LALR}(1) grammar
6986automaton computed by Bison. If the grammar file is @file{foo.y}, the
6987@acronym{VCG} output file will
342b8b6e
AD
6988be @file{foo.vcg}.
6989
6990@item --graph=@var{graph-file}
72d2299c
PE
6991The behavior of @var{--graph} is the same than @samp{-g}. The only
6992difference is that it has an optional argument which is the name of
fa4d969f 6993the output graph file.
bfa74976
RS
6994@end table
6995
342b8b6e 6996@node Option Cross Key
bfa74976
RS
6997@section Option Cross Key
6998
aa08666d 6999@c FIXME: How about putting the directives too?
bfa74976
RS
7000Here is a list of options, alphabetized by long option, to help you find
7001the corresponding short option.
7002
aa08666d
AD
7003@multitable {@option{--defines=@var{defines-file}}} {@option{-b @var{file-prefix}XXX}}
7004@headitem Long Option @tab Short Option
7005@item @option{--debug} @tab @option{-t}
7006@item @option{--defines=@var{defines-file}} @tab @option{-d}
7007@item @option{--file-prefix=@var{prefix}} @tab @option{-b @var{file-prefix}}
7008@item @option{--graph=@var{graph-file}} @tab @option{-d}
7009@item @option{--help} @tab @option{-h}
7010@item @option{--name-prefix=@var{prefix}} @tab @option{-p @var{name-prefix}}
7011@item @option{--no-lines} @tab @option{-l}
7012@item @option{--no-parser} @tab @option{-n}
7013@item @option{--output=@var{outfile}} @tab @option{-o @var{outfile}}
7014@item @option{--print-localedir} @tab
7015@item @option{--token-table} @tab @option{-k}
7016@item @option{--verbose} @tab @option{-v}
7017@item @option{--version} @tab @option{-V}
7018@item @option{--yacc} @tab @option{-y}
7019@end multitable
bfa74976 7020
93dd49ab
PE
7021@node Yacc Library
7022@section Yacc Library
7023
7024The Yacc library contains default implementations of the
7025@code{yyerror} and @code{main} functions. These default
7026implementations are normally not useful, but @acronym{POSIX} requires
7027them. To use the Yacc library, link your program with the
7028@option{-ly} option. Note that Bison's implementation of the Yacc
7029library is distributed under the terms of the @acronym{GNU} General
7030Public License (@pxref{Copying}).
7031
7032If you use the Yacc library's @code{yyerror} function, you should
7033declare @code{yyerror} as follows:
7034
7035@example
7036int yyerror (char const *);
7037@end example
7038
7039Bison ignores the @code{int} value returned by this @code{yyerror}.
7040If you use the Yacc library's @code{main} function, your
7041@code{yyparse} function should have the following type signature:
7042
7043@example
7044int yyparse (void);
7045@end example
7046
12545799
AD
7047@c ================================================= C++ Bison
7048
7049@node C++ Language Interface
7050@chapter C++ Language Interface
7051
7052@menu
7053* C++ Parsers:: The interface to generate C++ parser classes
7054* A Complete C++ Example:: Demonstrating their use
7055@end menu
7056
7057@node C++ Parsers
7058@section C++ Parsers
7059
7060@menu
7061* C++ Bison Interface:: Asking for C++ parser generation
7062* C++ Semantic Values:: %union vs. C++
7063* C++ Location Values:: The position and location classes
7064* C++ Parser Interface:: Instantiating and running the parser
7065* C++ Scanner Interface:: Exchanges between yylex and parse
7066@end menu
7067
7068@node C++ Bison Interface
7069@subsection C++ Bison Interface
7070@c - %skeleton "lalr1.cc"
7071@c - Always pure
7072@c - initial action
7073
aa08666d
AD
7074The C++ parser @acronym{LALR}(1) skeleton is named @file{lalr1.cc}. To
7075select it, you may either pass the option @option{--skeleton=lalr1.cc}
7076to Bison, or include the directive @samp{%skeleton "lalr1.cc"} in the
12545799 7077grammar preamble. When run, @command{bison} will create several
aa08666d
AD
7078entities in the @samp{yy} namespace. Use the @samp{%name-prefix}
7079directive to change the namespace name, see @ref{Decl Summary}. The
7080various classes are generated in the following files:
7081
12545799
AD
7082@table @file
7083@item position.hh
7084@itemx location.hh
7085The definition of the classes @code{position} and @code{location},
7086used for location tracking. @xref{C++ Location Values}.
7087
7088@item stack.hh
7089An auxiliary class @code{stack} used by the parser.
7090
fa4d969f
PE
7091@item @var{file}.hh
7092@itemx @var{file}.cc
cd8b5791
AD
7093(Assuming the extension of the input file was @samp{.yy}.) The
7094declaration and implementation of the C++ parser class. The basename
7095and extension of these two files follow the same rules as with regular C
7096parsers (@pxref{Invocation}).
12545799 7097
cd8b5791
AD
7098The header is @emph{mandatory}; you must either pass
7099@option{-d}/@option{--defines} to @command{bison}, or use the
12545799
AD
7100@samp{%defines} directive.
7101@end table
7102
7103All these files are documented using Doxygen; run @command{doxygen}
7104for a complete and accurate documentation.
7105
7106@node C++ Semantic Values
7107@subsection C++ Semantic Values
7108@c - No objects in unions
7109@c - YSTYPE
7110@c - Printer and destructor
7111
7112The @code{%union} directive works as for C, see @ref{Union Decl, ,The
7113Collection of Value Types}. In particular it produces a genuine
7114@code{union}@footnote{In the future techniques to allow complex types
fb9712a9
AD
7115within pseudo-unions (similar to Boost variants) might be implemented to
7116alleviate these issues.}, which have a few specific features in C++.
12545799
AD
7117@itemize @minus
7118@item
fb9712a9
AD
7119The type @code{YYSTYPE} is defined but its use is discouraged: rather
7120you should refer to the parser's encapsulated type
7121@code{yy::parser::semantic_type}.
12545799
AD
7122@item
7123Non POD (Plain Old Data) types cannot be used. C++ forbids any
7124instance of classes with constructors in unions: only @emph{pointers}
7125to such objects are allowed.
7126@end itemize
7127
7128Because objects have to be stored via pointers, memory is not
7129reclaimed automatically: using the @code{%destructor} directive is the
7130only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
7131Symbols}.
7132
7133
7134@node C++ Location Values
7135@subsection C++ Location Values
7136@c - %locations
7137@c - class Position
7138@c - class Location
b47dbebe 7139@c - %define "filename_type" "const symbol::Symbol"
12545799
AD
7140
7141When the directive @code{%locations} is used, the C++ parser supports
7142location tracking, see @ref{Locations, , Locations Overview}. Two
7143auxiliary classes define a @code{position}, a single point in a file,
7144and a @code{location}, a range composed of a pair of
7145@code{position}s (possibly spanning several files).
7146
fa4d969f 7147@deftypemethod {position} {std::string*} file
12545799
AD
7148The name of the file. It will always be handled as a pointer, the
7149parser will never duplicate nor deallocate it. As an experimental
7150feature you may change it to @samp{@var{type}*} using @samp{%define
b47dbebe 7151"filename_type" "@var{type}"}.
12545799
AD
7152@end deftypemethod
7153
7154@deftypemethod {position} {unsigned int} line
7155The line, starting at 1.
7156@end deftypemethod
7157
7158@deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
7159Advance by @var{height} lines, resetting the column number.
7160@end deftypemethod
7161
7162@deftypemethod {position} {unsigned int} column
7163The column, starting at 0.
7164@end deftypemethod
7165
7166@deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
7167Advance by @var{width} columns, without changing the line number.
7168@end deftypemethod
7169
7170@deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
7171@deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
7172@deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
7173@deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
7174Various forms of syntactic sugar for @code{columns}.
7175@end deftypemethod
7176
7177@deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
7178Report @var{p} on @var{o} like this:
fa4d969f
PE
7179@samp{@var{file}:@var{line}.@var{column}}, or
7180@samp{@var{line}.@var{column}} if @var{file} is null.
12545799
AD
7181@end deftypemethod
7182
7183@deftypemethod {location} {position} begin
7184@deftypemethodx {location} {position} end
7185The first, inclusive, position of the range, and the first beyond.
7186@end deftypemethod
7187
7188@deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
7189@deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
7190Advance the @code{end} position.
7191@end deftypemethod
7192
7193@deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
7194@deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
7195@deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
7196Various forms of syntactic sugar.
7197@end deftypemethod
7198
7199@deftypemethod {location} {void} step ()
7200Move @code{begin} onto @code{end}.
7201@end deftypemethod
7202
7203
7204@node C++ Parser Interface
7205@subsection C++ Parser Interface
7206@c - define parser_class_name
7207@c - Ctor
7208@c - parse, error, set_debug_level, debug_level, set_debug_stream,
7209@c debug_stream.
7210@c - Reporting errors
7211
7212The output files @file{@var{output}.hh} and @file{@var{output}.cc}
7213declare and define the parser class in the namespace @code{yy}. The
7214class name defaults to @code{parser}, but may be changed using
7215@samp{%define "parser_class_name" "@var{name}"}. The interface of
9d9b8b70 7216this class is detailed below. It can be extended using the
12545799
AD
7217@code{%parse-param} feature: its semantics is slightly changed since
7218it describes an additional member of the parser class, and an
7219additional argument for its constructor.
7220
8a0adb01
AD
7221@defcv {Type} {parser} {semantic_value_type}
7222@defcvx {Type} {parser} {location_value_type}
12545799 7223The types for semantics value and locations.
8a0adb01 7224@end defcv
12545799
AD
7225
7226@deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
7227Build a new parser object. There are no arguments by default, unless
7228@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
7229@end deftypemethod
7230
7231@deftypemethod {parser} {int} parse ()
7232Run the syntactic analysis, and return 0 on success, 1 otherwise.
7233@end deftypemethod
7234
7235@deftypemethod {parser} {std::ostream&} debug_stream ()
7236@deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
7237Get or set the stream used for tracing the parsing. It defaults to
7238@code{std::cerr}.
7239@end deftypemethod
7240
7241@deftypemethod {parser} {debug_level_type} debug_level ()
7242@deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
7243Get or set the tracing level. Currently its value is either 0, no trace,
9d9b8b70 7244or nonzero, full tracing.
12545799
AD
7245@end deftypemethod
7246
7247@deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
7248The definition for this member function must be supplied by the user:
7249the parser uses it to report a parser error occurring at @var{l},
7250described by @var{m}.
7251@end deftypemethod
7252
7253
7254@node C++ Scanner Interface
7255@subsection C++ Scanner Interface
7256@c - prefix for yylex.
7257@c - Pure interface to yylex
7258@c - %lex-param
7259
7260The parser invokes the scanner by calling @code{yylex}. Contrary to C
7261parsers, C++ parsers are always pure: there is no point in using the
7262@code{%pure-parser} directive. Therefore the interface is as follows.
7263
7264@deftypemethod {parser} {int} yylex (semantic_value_type& @var{yylval}, location_type& @var{yylloc}, @var{type1} @var{arg1}, ...)
7265Return the next token. Its type is the return value, its semantic
7266value and location being @var{yylval} and @var{yylloc}. Invocations of
7267@samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
7268@end deftypemethod
7269
7270
7271@node A Complete C++ Example
7272@section A Complete C++ Example
7273
7274This section demonstrates the use of a C++ parser with a simple but
7275complete example. This example should be available on your system,
7276ready to compile, in the directory @dfn{../bison/examples/calc++}. It
7277focuses on the use of Bison, therefore the design of the various C++
7278classes is very naive: no accessors, no encapsulation of members etc.
7279We will use a Lex scanner, and more precisely, a Flex scanner, to
7280demonstrate the various interaction. A hand written scanner is
7281actually easier to interface with.
7282
7283@menu
7284* Calc++ --- C++ Calculator:: The specifications
7285* Calc++ Parsing Driver:: An active parsing context
7286* Calc++ Parser:: A parser class
7287* Calc++ Scanner:: A pure C++ Flex scanner
7288* Calc++ Top Level:: Conducting the band
7289@end menu
7290
7291@node Calc++ --- C++ Calculator
7292@subsection Calc++ --- C++ Calculator
7293
7294Of course the grammar is dedicated to arithmetics, a single
9d9b8b70 7295expression, possibly preceded by variable assignments. An
12545799
AD
7296environment containing possibly predefined variables such as
7297@code{one} and @code{two}, is exchanged with the parser. An example
7298of valid input follows.
7299
7300@example
7301three := 3
7302seven := one + two * three
7303seven * seven
7304@end example
7305
7306@node Calc++ Parsing Driver
7307@subsection Calc++ Parsing Driver
7308@c - An env
7309@c - A place to store error messages
7310@c - A place for the result
7311
7312To support a pure interface with the parser (and the scanner) the
7313technique of the ``parsing context'' is convenient: a structure
7314containing all the data to exchange. Since, in addition to simply
7315launch the parsing, there are several auxiliary tasks to execute (open
7316the file for parsing, instantiate the parser etc.), we recommend
7317transforming the simple parsing context structure into a fully blown
7318@dfn{parsing driver} class.
7319
7320The declaration of this driver class, @file{calc++-driver.hh}, is as
7321follows. The first part includes the CPP guard and imports the
fb9712a9
AD
7322required standard library components, and the declaration of the parser
7323class.
12545799 7324
1c59e0a1 7325@comment file: calc++-driver.hh
12545799
AD
7326@example
7327#ifndef CALCXX_DRIVER_HH
7328# define CALCXX_DRIVER_HH
7329# include <string>
7330# include <map>
fb9712a9 7331# include "calc++-parser.hh"
12545799
AD
7332@end example
7333
12545799
AD
7334
7335@noindent
7336Then comes the declaration of the scanning function. Flex expects
7337the signature of @code{yylex} to be defined in the macro
7338@code{YY_DECL}, and the C++ parser expects it to be declared. We can
7339factor both as follows.
1c59e0a1
AD
7340
7341@comment file: calc++-driver.hh
12545799
AD
7342@example
7343// Announce to Flex the prototype we want for lexing function, ...
c095d689
AD
7344# define YY_DECL \
7345 yy::calcxx_parser::token_type \
7346 yylex (yy::calcxx_parser::semantic_type* yylval, \
7347 yy::calcxx_parser::location_type* yylloc, \
7348 calcxx_driver& driver)
12545799
AD
7349// ... and declare it for the parser's sake.
7350YY_DECL;
7351@end example
7352
7353@noindent
7354The @code{calcxx_driver} class is then declared with its most obvious
7355members.
7356
1c59e0a1 7357@comment file: calc++-driver.hh
12545799
AD
7358@example
7359// Conducting the whole scanning and parsing of Calc++.
7360class calcxx_driver
7361@{
7362public:
7363 calcxx_driver ();
7364 virtual ~calcxx_driver ();
7365
7366 std::map<std::string, int> variables;
7367
7368 int result;
7369@end example
7370
7371@noindent
7372To encapsulate the coordination with the Flex scanner, it is useful to
7373have two members function to open and close the scanning phase.
7374members.
7375
1c59e0a1 7376@comment file: calc++-driver.hh
12545799
AD
7377@example
7378 // Handling the scanner.
7379 void scan_begin ();
7380 void scan_end ();
7381 bool trace_scanning;
7382@end example
7383
7384@noindent
7385Similarly for the parser itself.
7386
1c59e0a1 7387@comment file: calc++-driver.hh
12545799
AD
7388@example
7389 // Handling the parser.
7390 void parse (const std::string& f);
7391 std::string file;
7392 bool trace_parsing;
7393@end example
7394
7395@noindent
7396To demonstrate pure handling of parse errors, instead of simply
7397dumping them on the standard error output, we will pass them to the
7398compiler driver using the following two member functions. Finally, we
7399close the class declaration and CPP guard.
7400
1c59e0a1 7401@comment file: calc++-driver.hh
12545799
AD
7402@example
7403 // Error handling.
7404 void error (const yy::location& l, const std::string& m);
7405 void error (const std::string& m);
7406@};
7407#endif // ! CALCXX_DRIVER_HH
7408@end example
7409
7410The implementation of the driver is straightforward. The @code{parse}
7411member function deserves some attention. The @code{error} functions
7412are simple stubs, they should actually register the located error
7413messages and set error state.
7414
1c59e0a1 7415@comment file: calc++-driver.cc
12545799
AD
7416@example
7417#include "calc++-driver.hh"
7418#include "calc++-parser.hh"
7419
7420calcxx_driver::calcxx_driver ()
7421 : trace_scanning (false), trace_parsing (false)
7422@{
7423 variables["one"] = 1;
7424 variables["two"] = 2;
7425@}
7426
7427calcxx_driver::~calcxx_driver ()
7428@{
7429@}
7430
7431void
7432calcxx_driver::parse (const std::string &f)
7433@{
7434 file = f;
7435 scan_begin ();
7436 yy::calcxx_parser parser (*this);
7437 parser.set_debug_level (trace_parsing);
7438 parser.parse ();
7439 scan_end ();
7440@}
7441
7442void
7443calcxx_driver::error (const yy::location& l, const std::string& m)
7444@{
7445 std::cerr << l << ": " << m << std::endl;
7446@}
7447
7448void
7449calcxx_driver::error (const std::string& m)
7450@{
7451 std::cerr << m << std::endl;
7452@}
7453@end example
7454
7455@node Calc++ Parser
7456@subsection Calc++ Parser
7457
b50d2359
AD
7458The parser definition file @file{calc++-parser.yy} starts by asking for
7459the C++ LALR(1) skeleton, the creation of the parser header file, and
7460specifies the name of the parser class. Because the C++ skeleton
7461changed several times, it is safer to require the version you designed
7462the grammar for.
1c59e0a1
AD
7463
7464@comment file: calc++-parser.yy
12545799
AD
7465@example
7466%skeleton "lalr1.cc" /* -*- C++ -*- */
b50d2359 7467%require "2.1a"
12545799 7468%defines
fb9712a9
AD
7469%define "parser_class_name" "calcxx_parser"
7470@end example
7471
7472@noindent
34f98f46 7473@findex %start-header
fb9712a9
AD
7474Then come the declarations/inclusions needed to define the
7475@code{%union}. Because the parser uses the parsing driver and
7476reciprocally, both cannot include the header of the other. Because the
7477driver's header needs detailed knowledge about the parser class (in
7478particular its inner types), it is the parser's header which will simply
7479use a forward declaration of the driver.
34f98f46 7480@xref{Table of Symbols, ,%start-header}.
fb9712a9
AD
7481
7482@comment file: calc++-parser.yy
7483@example
34f98f46 7484%start-header @{
12545799 7485# include <string>
fb9712a9 7486class calcxx_driver;
9bc0dd67 7487@}
12545799
AD
7488@end example
7489
7490@noindent
7491The driver is passed by reference to the parser and to the scanner.
7492This provides a simple but effective pure interface, not relying on
7493global variables.
7494
1c59e0a1 7495@comment file: calc++-parser.yy
12545799
AD
7496@example
7497// The parsing context.
7498%parse-param @{ calcxx_driver& driver @}
7499%lex-param @{ calcxx_driver& driver @}
7500@end example
7501
7502@noindent
7503Then we request the location tracking feature, and initialize the
7504first location's file name. Afterwards new locations are computed
7505relatively to the previous locations: the file name will be
7506automatically propagated.
7507
1c59e0a1 7508@comment file: calc++-parser.yy
12545799
AD
7509@example
7510%locations
7511%initial-action
7512@{
7513 // Initialize the initial location.
b47dbebe 7514 @@$.begin.filename = @@$.end.filename = &driver.file;
12545799
AD
7515@};
7516@end example
7517
7518@noindent
7519Use the two following directives to enable parser tracing and verbose
7520error messages.
7521
1c59e0a1 7522@comment file: calc++-parser.yy
12545799
AD
7523@example
7524%debug
7525%error-verbose
7526@end example
7527
7528@noindent
7529Semantic values cannot use ``real'' objects, but only pointers to
7530them.
7531
1c59e0a1 7532@comment file: calc++-parser.yy
12545799
AD
7533@example
7534// Symbols.
7535%union
7536@{
7537 int ival;
7538 std::string *sval;
7539@};
7540@end example
7541
fb9712a9 7542@noindent
34f98f46
JD
7543@findex %after-header
7544The code between @samp{%after-header @{} and @samp{@}} is output in the
7545@file{*.cc} file; it needs detailed knowledge about the driver.
fb9712a9
AD
7546
7547@comment file: calc++-parser.yy
7548@example
34f98f46 7549%after-header @{
fb9712a9 7550# include "calc++-driver.hh"
34f98f46 7551@}
fb9712a9
AD
7552@end example
7553
7554
12545799
AD
7555@noindent
7556The token numbered as 0 corresponds to end of file; the following line
7557allows for nicer error messages referring to ``end of file'' instead
7558of ``$end''. Similarly user friendly named are provided for each
7559symbol. Note that the tokens names are prefixed by @code{TOKEN_} to
7560avoid name clashes.
7561
1c59e0a1 7562@comment file: calc++-parser.yy
12545799 7563@example
fb9712a9
AD
7564%token END 0 "end of file"
7565%token ASSIGN ":="
7566%token <sval> IDENTIFIER "identifier"
7567%token <ival> NUMBER "number"
7568%type <ival> exp "expression"
12545799
AD
7569@end example
7570
7571@noindent
7572To enable memory deallocation during error recovery, use
7573@code{%destructor}.
7574
287c78f6 7575@c FIXME: Document %printer, and mention that it takes a braced-code operand.
1c59e0a1 7576@comment file: calc++-parser.yy
12545799
AD
7577@example
7578%printer @{ debug_stream () << *$$; @} "identifier"
7579%destructor @{ delete $$; @} "identifier"
7580
7581%printer @{ debug_stream () << $$; @} "number" "expression"
7582@end example
7583
7584@noindent
7585The grammar itself is straightforward.
7586
1c59e0a1 7587@comment file: calc++-parser.yy
12545799
AD
7588@example
7589%%
7590%start unit;
7591unit: assignments exp @{ driver.result = $2; @};
7592
7593assignments: assignments assignment @{@}
9d9b8b70 7594 | /* Nothing. */ @{@};
12545799 7595
fb9712a9 7596assignment: "identifier" ":=" exp @{ driver.variables[*$1] = $3; @};
12545799
AD
7597
7598%left '+' '-';
7599%left '*' '/';
7600exp: exp '+' exp @{ $$ = $1 + $3; @}
7601 | exp '-' exp @{ $$ = $1 - $3; @}
7602 | exp '*' exp @{ $$ = $1 * $3; @}
7603 | exp '/' exp @{ $$ = $1 / $3; @}
fb9712a9
AD
7604 | "identifier" @{ $$ = driver.variables[*$1]; @}
7605 | "number" @{ $$ = $1; @};
12545799
AD
7606%%
7607@end example
7608
7609@noindent
7610Finally the @code{error} member function registers the errors to the
7611driver.
7612
1c59e0a1 7613@comment file: calc++-parser.yy
12545799
AD
7614@example
7615void
1c59e0a1
AD
7616yy::calcxx_parser::error (const yy::calcxx_parser::location_type& l,
7617 const std::string& m)
12545799
AD
7618@{
7619 driver.error (l, m);
7620@}
7621@end example
7622
7623@node Calc++ Scanner
7624@subsection Calc++ Scanner
7625
7626The Flex scanner first includes the driver declaration, then the
7627parser's to get the set of defined tokens.
7628
1c59e0a1 7629@comment file: calc++-scanner.ll
12545799
AD
7630@example
7631%@{ /* -*- C++ -*- */
04098407
PE
7632# include <cstdlib>
7633# include <errno.h>
7634# include <limits.h>
12545799
AD
7635# include <string>
7636# include "calc++-driver.hh"
7637# include "calc++-parser.hh"
eaea13f5
PE
7638
7639/* Work around an incompatibility in flex (at least versions
7640 2.5.31 through 2.5.33): it generates code that does
7641 not conform to C89. See Debian bug 333231
7642 <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */
7870f699
PE
7643# undef yywrap
7644# define yywrap() 1
eaea13f5 7645
c095d689
AD
7646/* By default yylex returns int, we use token_type.
7647 Unfortunately yyterminate by default returns 0, which is
7648 not of token_type. */
8c5b881d 7649#define yyterminate() return token::END
12545799
AD
7650%@}
7651@end example
7652
7653@noindent
7654Because there is no @code{#include}-like feature we don't need
7655@code{yywrap}, we don't need @code{unput} either, and we parse an
7656actual file, this is not an interactive session with the user.
7657Finally we enable the scanner tracing features.
7658
1c59e0a1 7659@comment file: calc++-scanner.ll
12545799
AD
7660@example
7661%option noyywrap nounput batch debug
7662@end example
7663
7664@noindent
7665Abbreviations allow for more readable rules.
7666
1c59e0a1 7667@comment file: calc++-scanner.ll
12545799
AD
7668@example
7669id [a-zA-Z][a-zA-Z_0-9]*
7670int [0-9]+
7671blank [ \t]
7672@end example
7673
7674@noindent
9d9b8b70 7675The following paragraph suffices to track locations accurately. Each
12545799
AD
7676time @code{yylex} is invoked, the begin position is moved onto the end
7677position. Then when a pattern is matched, the end position is
7678advanced of its width. In case it matched ends of lines, the end
7679cursor is adjusted, and each time blanks are matched, the begin cursor
7680is moved onto the end cursor to effectively ignore the blanks
7681preceding tokens. Comments would be treated equally.
7682
1c59e0a1 7683@comment file: calc++-scanner.ll
12545799 7684@example
828c373b
AD
7685%@{
7686# define YY_USER_ACTION yylloc->columns (yyleng);
7687%@}
12545799
AD
7688%%
7689%@{
7690 yylloc->step ();
12545799
AD
7691%@}
7692@{blank@}+ yylloc->step ();
7693[\n]+ yylloc->lines (yyleng); yylloc->step ();
7694@end example
7695
7696@noindent
fb9712a9
AD
7697The rules are simple, just note the use of the driver to report errors.
7698It is convenient to use a typedef to shorten
7699@code{yy::calcxx_parser::token::identifier} into
9d9b8b70 7700@code{token::identifier} for instance.
12545799 7701
1c59e0a1 7702@comment file: calc++-scanner.ll
12545799 7703@example
fb9712a9
AD
7704%@{
7705 typedef yy::calcxx_parser::token token;
7706%@}
8c5b881d 7707 /* Convert ints to the actual type of tokens. */
c095d689 7708[-+*/] return yy::calcxx_parser::token_type (yytext[0]);
fb9712a9 7709":=" return token::ASSIGN;
04098407
PE
7710@{int@} @{
7711 errno = 0;
7712 long n = strtol (yytext, NULL, 10);
7713 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
7714 driver.error (*yylloc, "integer is out of range");
7715 yylval->ival = n;
fb9712a9 7716 return token::NUMBER;
04098407 7717@}
fb9712a9 7718@{id@} yylval->sval = new std::string (yytext); return token::IDENTIFIER;
12545799
AD
7719. driver.error (*yylloc, "invalid character");
7720%%
7721@end example
7722
7723@noindent
7724Finally, because the scanner related driver's member function depend
7725on the scanner's data, it is simpler to implement them in this file.
7726
1c59e0a1 7727@comment file: calc++-scanner.ll
12545799
AD
7728@example
7729void
7730calcxx_driver::scan_begin ()
7731@{
7732 yy_flex_debug = trace_scanning;
7733 if (!(yyin = fopen (file.c_str (), "r")))
7734 error (std::string ("cannot open ") + file);
7735@}
7736
7737void
7738calcxx_driver::scan_end ()
7739@{
7740 fclose (yyin);
7741@}
7742@end example
7743
7744@node Calc++ Top Level
7745@subsection Calc++ Top Level
7746
7747The top level file, @file{calc++.cc}, poses no problem.
7748
1c59e0a1 7749@comment file: calc++.cc
12545799
AD
7750@example
7751#include <iostream>
7752#include "calc++-driver.hh"
7753
7754int
fa4d969f 7755main (int argc, char *argv[])
12545799
AD
7756@{
7757 calcxx_driver driver;
7758 for (++argv; argv[0]; ++argv)
7759 if (*argv == std::string ("-p"))
7760 driver.trace_parsing = true;
7761 else if (*argv == std::string ("-s"))
7762 driver.trace_scanning = true;
7763 else
7764 @{
7765 driver.parse (*argv);
7766 std::cout << driver.result << std::endl;
7767 @}
7768@}
7769@end example
7770
7771@c ================================================= FAQ
d1a1114f
AD
7772
7773@node FAQ
7774@chapter Frequently Asked Questions
7775@cindex frequently asked questions
7776@cindex questions
7777
7778Several questions about Bison come up occasionally. Here some of them
7779are addressed.
7780
7781@menu
55ba27be
AD
7782* Memory Exhausted:: Breaking the Stack Limits
7783* How Can I Reset the Parser:: @code{yyparse} Keeps some State
7784* Strings are Destroyed:: @code{yylval} Loses Track of Strings
7785* Implementing Gotos/Loops:: Control Flow in the Calculator
ed2e6384 7786* Multiple start-symbols:: Factoring closely related grammars
55ba27be
AD
7787* Secure? Conform?:: Is Bison @acronym{POSIX} safe?
7788* I can't build Bison:: Troubleshooting
7789* Where can I find help?:: Troubleshouting
7790* Bug Reports:: Troublereporting
7791* Other Languages:: Parsers in Java and others
7792* Beta Testing:: Experimenting development versions
7793* Mailing Lists:: Meeting other Bison users
d1a1114f
AD
7794@end menu
7795
1a059451
PE
7796@node Memory Exhausted
7797@section Memory Exhausted
d1a1114f
AD
7798
7799@display
1a059451 7800My parser returns with error with a @samp{memory exhausted}
d1a1114f
AD
7801message. What can I do?
7802@end display
7803
7804This question is already addressed elsewhere, @xref{Recursion,
7805,Recursive Rules}.
7806
e64fec0a
PE
7807@node How Can I Reset the Parser
7808@section How Can I Reset the Parser
5b066063 7809
0e14ad77
PE
7810The following phenomenon has several symptoms, resulting in the
7811following typical questions:
5b066063
AD
7812
7813@display
7814I invoke @code{yyparse} several times, and on correct input it works
7815properly; but when a parse error is found, all the other calls fail
0e14ad77 7816too. How can I reset the error flag of @code{yyparse}?
5b066063
AD
7817@end display
7818
7819@noindent
7820or
7821
7822@display
0e14ad77 7823My parser includes support for an @samp{#include}-like feature, in
5b066063
AD
7824which case I run @code{yyparse} from @code{yyparse}. This fails
7825although I did specify I needed a @code{%pure-parser}.
7826@end display
7827
0e14ad77
PE
7828These problems typically come not from Bison itself, but from
7829Lex-generated scanners. Because these scanners use large buffers for
5b066063
AD
7830speed, they might not notice a change of input file. As a
7831demonstration, consider the following source file,
7832@file{first-line.l}:
7833
7834@verbatim
7835%{
7836#include <stdio.h>
7837#include <stdlib.h>
7838%}
7839%%
7840.*\n ECHO; return 1;
7841%%
7842int
0e14ad77 7843yyparse (char const *file)
5b066063
AD
7844{
7845 yyin = fopen (file, "r");
7846 if (!yyin)
7847 exit (2);
fa7e68c3 7848 /* One token only. */
5b066063 7849 yylex ();
0e14ad77 7850 if (fclose (yyin) != 0)
5b066063
AD
7851 exit (3);
7852 return 0;
7853}
7854
7855int
0e14ad77 7856main (void)
5b066063
AD
7857{
7858 yyparse ("input");
7859 yyparse ("input");
7860 return 0;
7861}
7862@end verbatim
7863
7864@noindent
7865If the file @file{input} contains
7866
7867@verbatim
7868input:1: Hello,
7869input:2: World!
7870@end verbatim
7871
7872@noindent
0e14ad77 7873then instead of getting the first line twice, you get:
5b066063
AD
7874
7875@example
7876$ @kbd{flex -ofirst-line.c first-line.l}
7877$ @kbd{gcc -ofirst-line first-line.c -ll}
7878$ @kbd{./first-line}
7879input:1: Hello,
7880input:2: World!
7881@end example
7882
0e14ad77
PE
7883Therefore, whenever you change @code{yyin}, you must tell the
7884Lex-generated scanner to discard its current buffer and switch to the
7885new one. This depends upon your implementation of Lex; see its
7886documentation for more. For Flex, it suffices to call
7887@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
7888Flex-generated scanner needs to read from several input streams to
7889handle features like include files, you might consider using Flex
7890functions like @samp{yy_switch_to_buffer} that manipulate multiple
7891input buffers.
5b066063 7892
b165c324
AD
7893If your Flex-generated scanner uses start conditions (@pxref{Start
7894conditions, , Start conditions, flex, The Flex Manual}), you might
7895also want to reset the scanner's state, i.e., go back to the initial
7896start condition, through a call to @samp{BEGIN (0)}.
7897
fef4cb51
AD
7898@node Strings are Destroyed
7899@section Strings are Destroyed
7900
7901@display
c7e441b4 7902My parser seems to destroy old strings, or maybe it loses track of
fef4cb51
AD
7903them. Instead of reporting @samp{"foo", "bar"}, it reports
7904@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
7905@end display
7906
7907This error is probably the single most frequent ``bug report'' sent to
7908Bison lists, but is only concerned with a misunderstanding of the role
8c5b881d 7909of the scanner. Consider the following Lex code:
fef4cb51
AD
7910
7911@verbatim
7912%{
7913#include <stdio.h>
7914char *yylval = NULL;
7915%}
7916%%
7917.* yylval = yytext; return 1;
7918\n /* IGNORE */
7919%%
7920int
7921main ()
7922{
fa7e68c3 7923 /* Similar to using $1, $2 in a Bison action. */
fef4cb51
AD
7924 char *fst = (yylex (), yylval);
7925 char *snd = (yylex (), yylval);
7926 printf ("\"%s\", \"%s\"\n", fst, snd);
7927 return 0;
7928}
7929@end verbatim
7930
7931If you compile and run this code, you get:
7932
7933@example
7934$ @kbd{flex -osplit-lines.c split-lines.l}
7935$ @kbd{gcc -osplit-lines split-lines.c -ll}
7936$ @kbd{printf 'one\ntwo\n' | ./split-lines}
7937"one
7938two", "two"
7939@end example
7940
7941@noindent
7942this is because @code{yytext} is a buffer provided for @emph{reading}
7943in the action, but if you want to keep it, you have to duplicate it
7944(e.g., using @code{strdup}). Note that the output may depend on how
7945your implementation of Lex handles @code{yytext}. For instance, when
7946given the Lex compatibility option @option{-l} (which triggers the
7947option @samp{%array}) Flex generates a different behavior:
7948
7949@example
7950$ @kbd{flex -l -osplit-lines.c split-lines.l}
7951$ @kbd{gcc -osplit-lines split-lines.c -ll}
7952$ @kbd{printf 'one\ntwo\n' | ./split-lines}
7953"two", "two"
7954@end example
7955
7956
2fa09258
AD
7957@node Implementing Gotos/Loops
7958@section Implementing Gotos/Loops
a06ea4aa
AD
7959
7960@display
7961My simple calculator supports variables, assignments, and functions,
2fa09258 7962but how can I implement gotos, or loops?
a06ea4aa
AD
7963@end display
7964
7965Although very pedagogical, the examples included in the document blur
a1c84f45 7966the distinction to make between the parser---whose job is to recover
a06ea4aa 7967the structure of a text and to transmit it to subsequent modules of
a1c84f45 7968the program---and the processing (such as the execution) of this
a06ea4aa
AD
7969structure. This works well with so called straight line programs,
7970i.e., precisely those that have a straightforward execution model:
7971execute simple instructions one after the others.
7972
7973@cindex abstract syntax tree
7974@cindex @acronym{AST}
7975If you want a richer model, you will probably need to use the parser
7976to construct a tree that does represent the structure it has
7977recovered; this tree is usually called the @dfn{abstract syntax tree},
7978or @dfn{@acronym{AST}} for short. Then, walking through this tree,
7979traversing it in various ways, will enable treatments such as its
7980execution or its translation, which will result in an interpreter or a
7981compiler.
7982
7983This topic is way beyond the scope of this manual, and the reader is
7984invited to consult the dedicated literature.
7985
7986
ed2e6384
AD
7987@node Multiple start-symbols
7988@section Multiple start-symbols
7989
7990@display
7991I have several closely related grammars, and I would like to share their
7992implementations. In fact, I could use a single grammar but with
7993multiple entry points.
7994@end display
7995
7996Bison does not support multiple start-symbols, but there is a very
7997simple means to simulate them. If @code{foo} and @code{bar} are the two
7998pseudo start-symbols, then introduce two new tokens, say
7999@code{START_FOO} and @code{START_BAR}, and use them as switches from the
8000real start-symbol:
8001
8002@example
8003%token START_FOO START_BAR;
8004%start start;
8005start: START_FOO foo
8006 | START_BAR bar;
8007@end example
8008
8009These tokens prevents the introduction of new conflicts. As far as the
8010parser goes, that is all that is needed.
8011
8012Now the difficult part is ensuring that the scanner will send these
8013tokens first. If your scanner is hand-written, that should be
8014straightforward. If your scanner is generated by Lex, them there is
8015simple means to do it: recall that anything between @samp{%@{ ... %@}}
8016after the first @code{%%} is copied verbatim in the top of the generated
8017@code{yylex} function. Make sure a variable @code{start_token} is
8018available in the scanner (e.g., a global variable or using
8019@code{%lex-param} etc.), and use the following:
8020
8021@example
8022 /* @r{Prologue.} */
8023%%
8024%@{
8025 if (start_token)
8026 @{
8027 int t = start_token;
8028 start_token = 0;
8029 return t;
8030 @}
8031%@}
8032 /* @r{The rules.} */
8033@end example
8034
8035
55ba27be
AD
8036@node Secure? Conform?
8037@section Secure? Conform?
8038
8039@display
8040Is Bison secure? Does it conform to POSIX?
8041@end display
8042
8043If you're looking for a guarantee or certification, we don't provide it.
8044However, Bison is intended to be a reliable program that conforms to the
8045@acronym{POSIX} specification for Yacc. If you run into problems,
8046please send us a bug report.
8047
8048@node I can't build Bison
8049@section I can't build Bison
8050
8051@display
8c5b881d
PE
8052I can't build Bison because @command{make} complains that
8053@code{msgfmt} is not found.
55ba27be
AD
8054What should I do?
8055@end display
8056
8057Like most GNU packages with internationalization support, that feature
8058is turned on by default. If you have problems building in the @file{po}
8059subdirectory, it indicates that your system's internationalization
8060support is lacking. You can re-configure Bison with
8061@option{--disable-nls} to turn off this support, or you can install GNU
8062gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
8063Bison. See the file @file{ABOUT-NLS} for more information.
8064
8065
8066@node Where can I find help?
8067@section Where can I find help?
8068
8069@display
8070I'm having trouble using Bison. Where can I find help?
8071@end display
8072
8073First, read this fine manual. Beyond that, you can send mail to
8074@email{help-bison@@gnu.org}. This mailing list is intended to be
8075populated with people who are willing to answer questions about using
8076and installing Bison. Please keep in mind that (most of) the people on
8077the list have aspects of their lives which are not related to Bison (!),
8078so you may not receive an answer to your question right away. This can
8079be frustrating, but please try not to honk them off; remember that any
8080help they provide is purely voluntary and out of the kindness of their
8081hearts.
8082
8083@node Bug Reports
8084@section Bug Reports
8085
8086@display
8087I found a bug. What should I include in the bug report?
8088@end display
8089
8090Before you send a bug report, make sure you are using the latest
8091version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
8092mirrors. Be sure to include the version number in your bug report. If
8093the bug is present in the latest version but not in a previous version,
8094try to determine the most recent version which did not contain the bug.
8095
8096If the bug is parser-related, you should include the smallest grammar
8097you can which demonstrates the bug. The grammar file should also be
8098complete (i.e., I should be able to run it through Bison without having
8099to edit or add anything). The smaller and simpler the grammar, the
8100easier it will be to fix the bug.
8101
8102Include information about your compilation environment, including your
8103operating system's name and version and your compiler's name and
8104version. If you have trouble compiling, you should also include a
8105transcript of the build session, starting with the invocation of
8106`configure'. Depending on the nature of the bug, you may be asked to
8107send additional files as well (such as `config.h' or `config.cache').
8108
8109Patches are most welcome, but not required. That is, do not hesitate to
8110send a bug report just because you can not provide a fix.
8111
8112Send bug reports to @email{bug-bison@@gnu.org}.
8113
8114@node Other Languages
8115@section Other Languages
8116
8117@display
8118Will Bison ever have C++ support? How about Java or @var{insert your
8119favorite language here}?
8120@end display
8121
8122C++ support is there now, and is documented. We'd love to add other
8123languages; contributions are welcome.
8124
8125@node Beta Testing
8126@section Beta Testing
8127
8128@display
8129What is involved in being a beta tester?
8130@end display
8131
8132It's not terribly involved. Basically, you would download a test
8133release, compile it, and use it to build and run a parser or two. After
8134that, you would submit either a bug report or a message saying that
8135everything is okay. It is important to report successes as well as
8136failures because test releases eventually become mainstream releases,
8137but only if they are adequately tested. If no one tests, development is
8138essentially halted.
8139
8140Beta testers are particularly needed for operating systems to which the
8141developers do not have easy access. They currently have easy access to
8142recent GNU/Linux and Solaris versions. Reports about other operating
8143systems are especially welcome.
8144
8145@node Mailing Lists
8146@section Mailing Lists
8147
8148@display
8149How do I join the help-bison and bug-bison mailing lists?
8150@end display
8151
8152See @url{http://lists.gnu.org/}.
a06ea4aa 8153
d1a1114f
AD
8154@c ================================================= Table of Symbols
8155
342b8b6e 8156@node Table of Symbols
bfa74976
RS
8157@appendix Bison Symbols
8158@cindex Bison symbols, table of
8159@cindex symbols in Bison, table of
8160
18b519c0 8161@deffn {Variable} @@$
3ded9a63 8162In an action, the location of the left-hand side of the rule.
88bce5a2 8163@xref{Locations, , Locations Overview}.
18b519c0 8164@end deffn
3ded9a63 8165
18b519c0 8166@deffn {Variable} @@@var{n}
3ded9a63
AD
8167In an action, the location of the @var{n}-th symbol of the right-hand
8168side of the rule. @xref{Locations, , Locations Overview}.
18b519c0 8169@end deffn
3ded9a63 8170
18b519c0 8171@deffn {Variable} $$
3ded9a63
AD
8172In an action, the semantic value of the left-hand side of the rule.
8173@xref{Actions}.
18b519c0 8174@end deffn
3ded9a63 8175
18b519c0 8176@deffn {Variable} $@var{n}
3ded9a63
AD
8177In an action, the semantic value of the @var{n}-th symbol of the
8178right-hand side of the rule. @xref{Actions}.
18b519c0 8179@end deffn
3ded9a63 8180
dd8d9022
AD
8181@deffn {Delimiter} %%
8182Delimiter used to separate the grammar rule section from the
8183Bison declarations section or the epilogue.
8184@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
18b519c0 8185@end deffn
bfa74976 8186
dd8d9022
AD
8187@c Don't insert spaces, or check the DVI output.
8188@deffn {Delimiter} %@{@var{code}%@}
8189All code listed between @samp{%@{} and @samp{%@}} is copied directly to
8190the output file uninterpreted. Such code forms the prologue of the input
8191file. @xref{Grammar Outline, ,Outline of a Bison
8192Grammar}.
18b519c0 8193@end deffn
bfa74976 8194
dd8d9022
AD
8195@deffn {Construct} /*@dots{}*/
8196Comment delimiters, as in C.
18b519c0 8197@end deffn
bfa74976 8198
dd8d9022
AD
8199@deffn {Delimiter} :
8200Separates a rule's result from its components. @xref{Rules, ,Syntax of
8201Grammar Rules}.
18b519c0 8202@end deffn
bfa74976 8203
dd8d9022
AD
8204@deffn {Delimiter} ;
8205Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 8206@end deffn
bfa74976 8207
dd8d9022
AD
8208@deffn {Delimiter} |
8209Separates alternate rules for the same result nonterminal.
8210@xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 8211@end deffn
bfa74976 8212
dd8d9022
AD
8213@deffn {Symbol} $accept
8214The predefined nonterminal whose only rule is @samp{$accept: @var{start}
8215$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
8216Start-Symbol}. It cannot be used in the grammar.
18b519c0 8217@end deffn
bfa74976 8218
34f98f46
JD
8219@deffn {Directive} %after-header @{@var{code}@}
8220Specifies code to be inserted into the code file after the contents of the
8221header file.
8222@xref{Table of Symbols, ,%start-header}.
8223@end deffn
8224
8225@deffn {Directive} %before-header @{@var{code}@}
8226Specifies code to be inserted into the code file before the contents of the
8227header file.
8228@xref{Table of Symbols, ,%start-header}.
8229@end deffn
8230
8231@deffn {Directive} %end-header @{@var{code}@}
9bc0dd67
JD
8232Specifies code to be inserted both into the header file (if generated;
8233@pxref{Table of Symbols, ,%defines}) and into the code file after any
8234Bison-generated definitions.
34f98f46 8235@xref{Table of Symbols, ,%start-header}.
9bc0dd67
JD
8236@end deffn
8237
34f98f46 8238@deffn {Directive} %start-header @{@var{code}@}
9bc0dd67
JD
8239Specifies code to be inserted both into the header file (if generated;
8240@pxref{Table of Symbols, ,%defines}) and into the code file before any
8241Bison-generated definitions.
8242
8243@cindex Prologue
34f98f46 8244@findex %before-header
9bc0dd67 8245@findex %union
34f98f46
JD
8246@findex %end-header
8247@findex %after-header
8248For example, the following declaration order in the grammar file reflects the
8249order in which Bison will output these code blocks. However, you are free to
8250declare these code blocks in your grammar file in whatever order is most
8251convenient for you:
9bc0dd67
JD
8252
8253@smallexample
34f98f46
JD
8254%before-header @{
8255 /* Bison treats this block like a pre-prologue block: it inserts it
8256 * into the code file before the contents of the header file. It
8257 * does *not* insert it into the header file. This is a good place
8258 * to put #include's that you want at the top of your code file. A
8259 * common example is `#include "system.h"'. */
8260@}
8261%start-header @{
8262 /* Bison inserts this block into both the header file and the code
8263 * file. In both files, the point of insertion is before any
8264 * Bison-generated token, semantic type, location type, and class
8265 * definitions. This is a good place to define %union
8266 * dependencies, for example. */
9bc0dd67
JD
8267@}
8268%union @{
34f98f46
JD
8269 /* Unlike the traditional Yacc prologue blocks, the output order
8270 * for the %*-header blocks is not affected by their declaration
8271 * position relative to any %union in the grammar file. */
9bc0dd67 8272@}
34f98f46
JD
8273%end-header @{
8274 /* Bison inserts this block into both the header file and the code
8275 * file. In both files, the point of insertion is after the
8276 * Bison-generated definitions. This is a good place to declare or
8277 * define public functions or data structures that depend on the
8278 * Bison-generated definitions. */
8279@}
8280%after-header @{
8281 /* Bison treats this block like a post-prologue block: it inserts
8282 * it into the code file after the contents of the header file. It
8283 * does *not* insert it into the header file. This is a good place
8284 * to declare or define internal functions or data structures that
8285 * depend on the Bison-generated definitions. */
9bc0dd67 8286@}
9bc0dd67
JD
8287@end smallexample
8288
34f98f46
JD
8289If you have multiple occurrences of any one of the above declarations, Bison
8290will concatenate the contents in declaration order.
8291
9bc0dd67
JD
8292@xref{Prologue, ,The Prologue}.
8293@end deffn
8294
8295@deffn {Directive} %debug
8296Equip the parser for debugging. @xref{Decl Summary}.
8297@end deffn
8298
18b519c0 8299@deffn {Directive} %debug
6deb4447 8300Equip the parser for debugging. @xref{Decl Summary}.
18b519c0 8301@end deffn
6deb4447 8302
91d2c560 8303@ifset defaultprec
22fccf95
PE
8304@deffn {Directive} %default-prec
8305Assign a precedence to rules that lack an explicit @samp{%prec}
8306modifier. @xref{Contextual Precedence, ,Context-Dependent
8307Precedence}.
39a06c25 8308@end deffn
91d2c560 8309@end ifset
39a06c25 8310
18b519c0 8311@deffn {Directive} %defines
6deb4447
AD
8312Bison declaration to create a header file meant for the scanner.
8313@xref{Decl Summary}.
18b519c0 8314@end deffn
6deb4447 8315
18b519c0 8316@deffn {Directive} %destructor
258b75ca 8317Specify how the parser should reclaim the memory associated to
fa7e68c3 8318discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 8319@end deffn
72f889cc 8320
18b519c0 8321@deffn {Directive} %dprec
676385e2 8322Bison declaration to assign a precedence to a rule that is used at parse
c827f760
PE
8323time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
8324@acronym{GLR} Parsers}.
18b519c0 8325@end deffn
676385e2 8326
dd8d9022
AD
8327@deffn {Symbol} $end
8328The predefined token marking the end of the token stream. It cannot be
8329used in the grammar.
8330@end deffn
8331
8332@deffn {Symbol} error
8333A token name reserved for error recovery. This token may be used in
8334grammar rules so as to allow the Bison parser to recognize an error in
8335the grammar without halting the process. In effect, a sentence
8336containing an error may be recognized as valid. On a syntax error, the
742e4900
JD
8337token @code{error} becomes the current lookahead token. Actions
8338corresponding to @code{error} are then executed, and the lookahead
dd8d9022
AD
8339token is reset to the token that originally caused the violation.
8340@xref{Error Recovery}.
18d192f0
AD
8341@end deffn
8342
18b519c0 8343@deffn {Directive} %error-verbose
2a8d363a
AD
8344Bison declaration to request verbose, specific error message strings
8345when @code{yyerror} is called.
18b519c0 8346@end deffn
2a8d363a 8347
18b519c0 8348@deffn {Directive} %file-prefix="@var{prefix}"
72d2299c 8349Bison declaration to set the prefix of the output files. @xref{Decl
d8988b2f 8350Summary}.
18b519c0 8351@end deffn
d8988b2f 8352
18b519c0 8353@deffn {Directive} %glr-parser
c827f760
PE
8354Bison declaration to produce a @acronym{GLR} parser. @xref{GLR
8355Parsers, ,Writing @acronym{GLR} Parsers}.
18b519c0 8356@end deffn
676385e2 8357
dd8d9022
AD
8358@deffn {Directive} %initial-action
8359Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
8360@end deffn
8361
18b519c0 8362@deffn {Directive} %left
bfa74976
RS
8363Bison declaration to assign left associativity to token(s).
8364@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 8365@end deffn
bfa74976 8366
feeb0eda 8367@deffn {Directive} %lex-param @{@var{argument-declaration}@}
2a8d363a
AD
8368Bison declaration to specifying an additional parameter that
8369@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
8370for Pure Parsers}.
18b519c0 8371@end deffn
2a8d363a 8372
18b519c0 8373@deffn {Directive} %merge
676385e2 8374Bison declaration to assign a merging function to a rule. If there is a
fae437e8 8375reduce/reduce conflict with a rule having the same merging function, the
676385e2 8376function is applied to the two semantic values to get a single result.
c827f760 8377@xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}.
18b519c0 8378@end deffn
676385e2 8379
18b519c0 8380@deffn {Directive} %name-prefix="@var{prefix}"
72d2299c 8381Bison declaration to rename the external symbols. @xref{Decl Summary}.
18b519c0 8382@end deffn
d8988b2f 8383
91d2c560 8384@ifset defaultprec
22fccf95
PE
8385@deffn {Directive} %no-default-prec
8386Do not assign a precedence to rules that lack an explicit @samp{%prec}
8387modifier. @xref{Contextual Precedence, ,Context-Dependent
8388Precedence}.
8389@end deffn
91d2c560 8390@end ifset
22fccf95 8391
18b519c0 8392@deffn {Directive} %no-lines
931c7513
RS
8393Bison declaration to avoid generating @code{#line} directives in the
8394parser file. @xref{Decl Summary}.
18b519c0 8395@end deffn
931c7513 8396
18b519c0 8397@deffn {Directive} %nonassoc
9d9b8b70 8398Bison declaration to assign nonassociativity to token(s).
bfa74976 8399@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 8400@end deffn
bfa74976 8401
fa4d969f 8402@deffn {Directive} %output="@var{file}"
72d2299c 8403Bison declaration to set the name of the parser file. @xref{Decl
d8988b2f 8404Summary}.
18b519c0 8405@end deffn
d8988b2f 8406
feeb0eda 8407@deffn {Directive} %parse-param @{@var{argument-declaration}@}
2a8d363a
AD
8408Bison declaration to specifying an additional parameter that
8409@code{yyparse} should accept. @xref{Parser Function,, The Parser
8410Function @code{yyparse}}.
18b519c0 8411@end deffn
2a8d363a 8412
18b519c0 8413@deffn {Directive} %prec
bfa74976
RS
8414Bison declaration to assign a precedence to a specific rule.
8415@xref{Contextual Precedence, ,Context-Dependent Precedence}.
18b519c0 8416@end deffn
bfa74976 8417
18b519c0 8418@deffn {Directive} %pure-parser
bfa74976
RS
8419Bison declaration to request a pure (reentrant) parser.
8420@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
18b519c0 8421@end deffn
bfa74976 8422
b50d2359 8423@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
8424Require version @var{version} or higher of Bison. @xref{Require Decl, ,
8425Require a Version of Bison}.
b50d2359
AD
8426@end deffn
8427
18b519c0 8428@deffn {Directive} %right
bfa74976
RS
8429Bison declaration to assign right associativity to token(s).
8430@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 8431@end deffn
bfa74976 8432
18b519c0 8433@deffn {Directive} %start
704a47c4
AD
8434Bison declaration to specify the start symbol. @xref{Start Decl, ,The
8435Start-Symbol}.
18b519c0 8436@end deffn
bfa74976 8437
18b519c0 8438@deffn {Directive} %token
bfa74976
RS
8439Bison declaration to declare token(s) without specifying precedence.
8440@xref{Token Decl, ,Token Type Names}.
18b519c0 8441@end deffn
bfa74976 8442
18b519c0 8443@deffn {Directive} %token-table
931c7513
RS
8444Bison declaration to include a token name table in the parser file.
8445@xref{Decl Summary}.
18b519c0 8446@end deffn
931c7513 8447
18b519c0 8448@deffn {Directive} %type
704a47c4
AD
8449Bison declaration to declare nonterminals. @xref{Type Decl,
8450,Nonterminal Symbols}.
18b519c0 8451@end deffn
bfa74976 8452
dd8d9022
AD
8453@deffn {Symbol} $undefined
8454The predefined token onto which all undefined values returned by
8455@code{yylex} are mapped. It cannot be used in the grammar, rather, use
8456@code{error}.
8457@end deffn
8458
18b519c0 8459@deffn {Directive} %union
bfa74976
RS
8460Bison declaration to specify several possible data types for semantic
8461values. @xref{Union Decl, ,The Collection of Value Types}.
18b519c0 8462@end deffn
bfa74976 8463
dd8d9022
AD
8464@deffn {Macro} YYABORT
8465Macro to pretend that an unrecoverable syntax error has occurred, by
8466making @code{yyparse} return 1 immediately. The error reporting
8467function @code{yyerror} is not called. @xref{Parser Function, ,The
8468Parser Function @code{yyparse}}.
8469@end deffn
3ded9a63 8470
dd8d9022
AD
8471@deffn {Macro} YYACCEPT
8472Macro to pretend that a complete utterance of the language has been
8473read, by making @code{yyparse} return 0 immediately.
8474@xref{Parser Function, ,The Parser Function @code{yyparse}}.
8475@end deffn
bfa74976 8476
dd8d9022 8477@deffn {Macro} YYBACKUP
742e4900 8478Macro to discard a value from the parser stack and fake a lookahead
dd8d9022 8479token. @xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 8480@end deffn
bfa74976 8481
dd8d9022 8482@deffn {Variable} yychar
32c29292 8483External integer variable that contains the integer value of the
742e4900 8484lookahead token. (In a pure parser, it is a local variable within
dd8d9022
AD
8485@code{yyparse}.) Error-recovery rule actions may examine this variable.
8486@xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 8487@end deffn
bfa74976 8488
dd8d9022
AD
8489@deffn {Variable} yyclearin
8490Macro used in error-recovery rule actions. It clears the previous
742e4900 8491lookahead token. @xref{Error Recovery}.
18b519c0 8492@end deffn
bfa74976 8493
dd8d9022
AD
8494@deffn {Macro} YYDEBUG
8495Macro to define to equip the parser with tracing code. @xref{Tracing,
8496,Tracing Your Parser}.
18b519c0 8497@end deffn
bfa74976 8498
dd8d9022
AD
8499@deffn {Variable} yydebug
8500External integer variable set to zero by default. If @code{yydebug}
8501is given a nonzero value, the parser will output information on input
8502symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
18b519c0 8503@end deffn
bfa74976 8504
dd8d9022
AD
8505@deffn {Macro} yyerrok
8506Macro to cause parser to recover immediately to its normal mode
8507after a syntax error. @xref{Error Recovery}.
8508@end deffn
8509
8510@deffn {Macro} YYERROR
8511Macro to pretend that a syntax error has just been detected: call
8512@code{yyerror} and then perform normal error recovery if possible
8513(@pxref{Error Recovery}), or (if recovery is impossible) make
8514@code{yyparse} return 1. @xref{Error Recovery}.
8515@end deffn
8516
8517@deffn {Function} yyerror
8518User-supplied function to be called by @code{yyparse} on error.
8519@xref{Error Reporting, ,The Error
8520Reporting Function @code{yyerror}}.
8521@end deffn
8522
8523@deffn {Macro} YYERROR_VERBOSE
8524An obsolete macro that you define with @code{#define} in the prologue
8525to request verbose, specific error message strings
8526when @code{yyerror} is called. It doesn't matter what definition you
8527use for @code{YYERROR_VERBOSE}, just whether you define it. Using
8528@code{%error-verbose} is preferred.
8529@end deffn
8530
8531@deffn {Macro} YYINITDEPTH
8532Macro for specifying the initial size of the parser stack.
1a059451 8533@xref{Memory Management}.
dd8d9022
AD
8534@end deffn
8535
8536@deffn {Function} yylex
8537User-supplied lexical analyzer function, called with no arguments to get
8538the next token. @xref{Lexical, ,The Lexical Analyzer Function
8539@code{yylex}}.
8540@end deffn
8541
8542@deffn {Macro} YYLEX_PARAM
8543An obsolete macro for specifying an extra argument (or list of extra
32c29292 8544arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
dd8d9022
AD
8545macro is deprecated, and is supported only for Yacc like parsers.
8546@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
8547@end deffn
8548
8549@deffn {Variable} yylloc
8550External variable in which @code{yylex} should place the line and column
8551numbers associated with a token. (In a pure parser, it is a local
8552variable within @code{yyparse}, and its address is passed to
32c29292
JD
8553@code{yylex}.)
8554You can ignore this variable if you don't use the @samp{@@} feature in the
8555grammar actions.
8556@xref{Token Locations, ,Textual Locations of Tokens}.
742e4900 8557In semantic actions, it stores the location of the lookahead token.
32c29292 8558@xref{Actions and Locations, ,Actions and Locations}.
dd8d9022
AD
8559@end deffn
8560
8561@deffn {Type} YYLTYPE
8562Data type of @code{yylloc}; by default, a structure with four
8563members. @xref{Location Type, , Data Types of Locations}.
8564@end deffn
8565
8566@deffn {Variable} yylval
8567External variable in which @code{yylex} should place the semantic
8568value associated with a token. (In a pure parser, it is a local
8569variable within @code{yyparse}, and its address is passed to
32c29292
JD
8570@code{yylex}.)
8571@xref{Token Values, ,Semantic Values of Tokens}.
742e4900 8572In semantic actions, it stores the semantic value of the lookahead token.
32c29292 8573@xref{Actions, ,Actions}.
dd8d9022
AD
8574@end deffn
8575
8576@deffn {Macro} YYMAXDEPTH
1a059451
PE
8577Macro for specifying the maximum size of the parser stack. @xref{Memory
8578Management}.
dd8d9022
AD
8579@end deffn
8580
8581@deffn {Variable} yynerrs
8a2800e7 8582Global variable which Bison increments each time it reports a syntax error.
dd8d9022
AD
8583(In a pure parser, it is a local variable within @code{yyparse}.)
8584@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
8585@end deffn
8586
8587@deffn {Function} yyparse
8588The parser function produced by Bison; call this function to start
8589parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
8590@end deffn
8591
8592@deffn {Macro} YYPARSE_PARAM
8593An obsolete macro for specifying the name of a parameter that
8594@code{yyparse} should accept. The use of this macro is deprecated, and
8595is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
8596Conventions for Pure Parsers}.
8597@end deffn
8598
8599@deffn {Macro} YYRECOVERING
02103984
PE
8600The expression @code{YYRECOVERING ()} yields 1 when the parser
8601is recovering from a syntax error, and 0 otherwise.
8602@xref{Action Features, ,Special Features for Use in Actions}.
dd8d9022
AD
8603@end deffn
8604
8605@deffn {Macro} YYSTACK_USE_ALLOCA
d7e14fc0
PE
8606Macro used to control the use of @code{alloca} when the C
8607@acronym{LALR}(1) parser needs to extend its stacks. If defined to 0,
8608the parser will use @code{malloc} to extend its stacks. If defined to
86091, the parser will use @code{alloca}. Values other than 0 and 1 are
8610reserved for future Bison extensions. If not defined,
8611@code{YYSTACK_USE_ALLOCA} defaults to 0.
8612
55289366 8613In the all-too-common case where your code may run on a host with a
d7e14fc0
PE
8614limited stack and with unreliable stack-overflow checking, you should
8615set @code{YYMAXDEPTH} to a value that cannot possibly result in
8616unchecked stack overflow on any of your target hosts when
8617@code{alloca} is called. You can inspect the code that Bison
8618generates in order to determine the proper numeric values. This will
8619require some expertise in low-level implementation details.
dd8d9022
AD
8620@end deffn
8621
8622@deffn {Type} YYSTYPE
8623Data type of semantic values; @code{int} by default.
8624@xref{Value Type, ,Data Types of Semantic Values}.
18b519c0 8625@end deffn
bfa74976 8626
342b8b6e 8627@node Glossary
bfa74976
RS
8628@appendix Glossary
8629@cindex glossary
8630
8631@table @asis
c827f760
PE
8632@item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'')
8633Formal method of specifying context-free grammars originally proposed
8634by John Backus, and slightly improved by Peter Naur in his 1960-01-02
8635committee document contributing to what became the Algol 60 report.
8636@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
8637
8638@item Context-free grammars
8639Grammars specified as rules that can be applied regardless of context.
8640Thus, if there is a rule which says that an integer can be used as an
8641expression, integers are allowed @emph{anywhere} an expression is
89cab50d
AD
8642permitted. @xref{Language and Grammar, ,Languages and Context-Free
8643Grammars}.
bfa74976
RS
8644
8645@item Dynamic allocation
8646Allocation of memory that occurs during execution, rather than at
8647compile time or on entry to a function.
8648
8649@item Empty string
8650Analogous to the empty set in set theory, the empty string is a
8651character string of length zero.
8652
8653@item Finite-state stack machine
8654A ``machine'' that has discrete states in which it is said to exist at
8655each instant in time. As input to the machine is processed, the
8656machine moves from state to state as specified by the logic of the
8657machine. In the case of the parser, the input is the language being
8658parsed, and the states correspond to various stages in the grammar
c827f760 8659rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976 8660
c827f760 8661@item Generalized @acronym{LR} (@acronym{GLR})
676385e2 8662A parsing algorithm that can handle all context-free grammars, including those
c827f760
PE
8663that are not @acronym{LALR}(1). It resolves situations that Bison's
8664usual @acronym{LALR}(1)
676385e2
PH
8665algorithm cannot by effectively splitting off multiple parsers, trying all
8666possible parsers, and discarding those that fail in the light of additional
c827f760
PE
8667right context. @xref{Generalized LR Parsing, ,Generalized
8668@acronym{LR} Parsing}.
676385e2 8669
bfa74976
RS
8670@item Grouping
8671A language construct that is (in general) grammatically divisible;
c827f760 8672for example, `expression' or `declaration' in C@.
bfa74976
RS
8673@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
8674
8675@item Infix operator
8676An arithmetic operator that is placed between the operands on which it
8677performs some operation.
8678
8679@item Input stream
8680A continuous flow of data between devices or programs.
8681
8682@item Language construct
8683One of the typical usage schemas of the language. For example, one of
8684the constructs of the C language is the @code{if} statement.
8685@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
8686
8687@item Left associativity
8688Operators having left associativity are analyzed from left to right:
8689@samp{a+b+c} first computes @samp{a+b} and then combines with
8690@samp{c}. @xref{Precedence, ,Operator Precedence}.
8691
8692@item Left recursion
89cab50d
AD
8693A rule whose result symbol is also its first component symbol; for
8694example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
8695Rules}.
bfa74976
RS
8696
8697@item Left-to-right parsing
8698Parsing a sentence of a language by analyzing it token by token from
c827f760 8699left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
8700
8701@item Lexical analyzer (scanner)
8702A function that reads an input stream and returns tokens one by one.
8703@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
8704
8705@item Lexical tie-in
8706A flag, set by actions in the grammar rules, which alters the way
8707tokens are parsed. @xref{Lexical Tie-ins}.
8708
931c7513 8709@item Literal string token
14ded682 8710A token which consists of two or more fixed characters. @xref{Symbols}.
931c7513 8711
742e4900
JD
8712@item Lookahead token
8713A token already read but not yet shifted. @xref{Lookahead, ,Lookahead
89cab50d 8714Tokens}.
bfa74976 8715
c827f760 8716@item @acronym{LALR}(1)
bfa74976 8717The class of context-free grammars that Bison (like most other parser
c827f760
PE
8718generators) can handle; a subset of @acronym{LR}(1). @xref{Mystery
8719Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
bfa74976 8720
c827f760 8721@item @acronym{LR}(1)
bfa74976 8722The class of context-free grammars in which at most one token of
742e4900 8723lookahead is needed to disambiguate the parsing of any piece of input.
bfa74976
RS
8724
8725@item Nonterminal symbol
8726A grammar symbol standing for a grammatical construct that can
8727be expressed through rules in terms of smaller constructs; in other
8728words, a construct that is not a token. @xref{Symbols}.
8729
bfa74976
RS
8730@item Parser
8731A function that recognizes valid sentences of a language by analyzing
8732the syntax structure of a set of tokens passed to it from a lexical
8733analyzer.
8734
8735@item Postfix operator
8736An arithmetic operator that is placed after the operands upon which it
8737performs some operation.
8738
8739@item Reduction
8740Replacing a string of nonterminals and/or terminals with a single
89cab50d 8741nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
c827f760 8742Parser Algorithm}.
bfa74976
RS
8743
8744@item Reentrant
8745A reentrant subprogram is a subprogram which can be in invoked any
8746number of times in parallel, without interference between the various
8747invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
8748
8749@item Reverse polish notation
8750A language in which all operators are postfix operators.
8751
8752@item Right recursion
89cab50d
AD
8753A rule whose result symbol is also its last component symbol; for
8754example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
8755Rules}.
bfa74976
RS
8756
8757@item Semantics
8758In computer languages, the semantics are specified by the actions
8759taken for each instance of the language, i.e., the meaning of
8760each statement. @xref{Semantics, ,Defining Language Semantics}.
8761
8762@item Shift
8763A parser is said to shift when it makes the choice of analyzing
8764further input from the stream rather than reducing immediately some
c827f760 8765already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
8766
8767@item Single-character literal
8768A single character that is recognized and interpreted as is.
8769@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
8770
8771@item Start symbol
8772The nonterminal symbol that stands for a complete valid utterance in
8773the language being parsed. The start symbol is usually listed as the
13863333 8774first nonterminal symbol in a language specification.
bfa74976
RS
8775@xref{Start Decl, ,The Start-Symbol}.
8776
8777@item Symbol table
8778A data structure where symbol names and associated data are stored
8779during parsing to allow for recognition and use of existing
8780information in repeated uses of a symbol. @xref{Multi-function Calc}.
8781
6e649e65
PE
8782@item Syntax error
8783An error encountered during parsing of an input stream due to invalid
8784syntax. @xref{Error Recovery}.
8785
bfa74976
RS
8786@item Token
8787A basic, grammatically indivisible unit of a language. The symbol
8788that describes a token in the grammar is a terminal symbol.
8789The input of the Bison parser is a stream of tokens which comes from
8790the lexical analyzer. @xref{Symbols}.
8791
8792@item Terminal symbol
89cab50d
AD
8793A grammar symbol that has no rules in the grammar and therefore is
8794grammatically indivisible. The piece of text it represents is a token.
8795@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
8796@end table
8797
342b8b6e 8798@node Copying This Manual
f2b5126e 8799@appendix Copying This Manual
f9a8293a 8800
f2b5126e
PB
8801@menu
8802* GNU Free Documentation License:: License for copying this manual.
8803@end menu
f9a8293a 8804
f2b5126e
PB
8805@include fdl.texi
8806
342b8b6e 8807@node Index
bfa74976
RS
8808@unnumbered Index
8809
8810@printindex cp
8811
bfa74976 8812@bye
a06ea4aa
AD
8813
8814@c LocalWords: texinfo setfilename settitle setchapternewpage finalout
8815@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex
8816@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry
8817@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa
8818@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc
8819@c LocalWords: rpcalc Lexer Gen Comp Expr ltcalc mfcalc Decl Symtab yylex
8820@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref
8821@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex
8822@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge
8823@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG
8824@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit
8825@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok
8826@c LocalWords: longjmp fprintf stderr preg yylloc YYLTYPE cos ln
8827@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym
8828@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof
8829@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum
8830@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype
8831@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless typefull yynerrs
8832@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES
8833@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param
8834@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP
32c29292 8835@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword
a06ea4aa
AD
8836@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH
8837@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm VCG notype
8838@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args
8839@c LocalWords: YYPRINTF infile ypp yxx outfile itemx vcg tex leaderfill
8840@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll
8841@c LocalWords: yyrestart nbar yytext fst snd osplit ntwo strdup AST
8842@c LocalWords: YYSTACK DVI fdl printindex