]> git.saurik.com Git - bison.git/blame - doc/bison.texinfo
* doc/bison.texinfo (Introduction): Don't say "GLR grammar".
[bison.git] / doc / bison.texinfo
CommitLineData
bfa74976
RS
1\input texinfo @c -*-texinfo-*-
2@comment %**start of header
3@setfilename bison.info
df1af54c
JT
4@include version.texi
5@settitle Bison @value{VERSION}
bfa74976
RS
6@setchapternewpage odd
7
5378c3e7 8@finalout
5378c3e7 9
13863333 10@c SMALL BOOK version
bfa74976 11@c This edition has been formatted so that you can format and print it in
13863333 12@c the smallbook format.
bfa74976
RS
13@c @smallbook
14
91d2c560
PE
15@c Set following if you want to document %default-prec and %no-default-prec.
16@c This feature is experimental and may change in future Bison versions.
17@c @set defaultprec
18
8c5b881d 19@ifnotinfo
bfa74976
RS
20@syncodeindex fn cp
21@syncodeindex vr cp
22@syncodeindex tp cp
8c5b881d 23@end ifnotinfo
bfa74976
RS
24@ifinfo
25@synindex fn cp
26@synindex vr cp
27@synindex tp cp
28@end ifinfo
29@comment %**end of header
30
fae437e8 31@copying
bd773d73 32
c827f760
PE
33This manual is for @acronym{GNU} Bison (version @value{VERSION},
34@value{UPDATED}), the @acronym{GNU} parser generator.
fae437e8 35
a06ea4aa 36Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
32c29292 371999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation, Inc.
fae437e8
AD
38
39@quotation
40Permission is granted to copy, distribute and/or modify this document
c827f760 41under the terms of the @acronym{GNU} Free Documentation License,
592fde95 42Version 1.2 or any later version published by the Free Software
c827f760
PE
43Foundation; with no Invariant Sections, with the Front-Cover texts
44being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in
45(a) below. A copy of the license is included in the section entitled
46``@acronym{GNU} Free Documentation License.''
47
48(a) The @acronym{FSF}'s Back-Cover Text is: ``You have freedom to copy
49and modify this @acronym{GNU} Manual, like @acronym{GNU} software.
50Copies published by the Free Software Foundation raise funds for
51@acronym{GNU} development.''
fae437e8
AD
52@end quotation
53@end copying
54
e62f1a89 55@dircategory Software development
fae437e8 56@direntry
c827f760 57* bison: (bison). @acronym{GNU} parser generator (Yacc replacement).
fae437e8 58@end direntry
bfa74976 59
bfa74976
RS
60@titlepage
61@title Bison
c827f760 62@subtitle The Yacc-compatible Parser Generator
df1af54c 63@subtitle @value{UPDATED}, Bison Version @value{VERSION}
bfa74976
RS
64
65@author by Charles Donnelly and Richard Stallman
66
67@page
68@vskip 0pt plus 1filll
fae437e8 69@insertcopying
bfa74976
RS
70@sp 2
71Published by the Free Software Foundation @*
0fb669f9
PE
7251 Franklin Street, Fifth Floor @*
73Boston, MA 02110-1301 USA @*
9ecbd125 74Printed copies are available from the Free Software Foundation.@*
c827f760 75@acronym{ISBN} 1-882114-44-2
bfa74976
RS
76@sp 2
77Cover art by Etienne Suvasa.
78@end titlepage
d5796688
JT
79
80@contents
bfa74976 81
342b8b6e
AD
82@ifnottex
83@node Top
84@top Bison
fae437e8 85@insertcopying
342b8b6e 86@end ifnottex
bfa74976
RS
87
88@menu
13863333
AD
89* Introduction::
90* Conditions::
c827f760 91* Copying:: The @acronym{GNU} General Public License says
bfa74976
RS
92 how you can copy and share Bison
93
94Tutorial sections:
95* Concepts:: Basic concepts for understanding Bison.
96* Examples:: Three simple explained examples of using Bison.
97
98Reference sections:
99* Grammar File:: Writing Bison declarations and rules.
100* Interface:: C-language interface to the parser function @code{yyparse}.
101* Algorithm:: How the Bison parser works at run-time.
102* Error Recovery:: Writing rules for error recovery.
103* Context Dependency:: What to do if your language syntax is too
104 messy for Bison to handle straightforwardly.
ec3bc396 105* Debugging:: Understanding or debugging Bison parsers.
bfa74976 106* Invocation:: How to run Bison (to produce the parser source file).
12545799
AD
107* C++ Language Interface:: Creating C++ parser objects.
108* FAQ:: Frequently Asked Questions
bfa74976
RS
109* Table of Symbols:: All the keywords of the Bison language are explained.
110* Glossary:: Basic concepts are explained.
f2b5126e 111* Copying This Manual:: License for copying this manual.
bfa74976
RS
112* Index:: Cross-references to the text.
113
93dd49ab
PE
114@detailmenu
115 --- The Detailed Node Listing ---
bfa74976
RS
116
117The Concepts of Bison
118
119* Language and Grammar:: Languages and context-free grammars,
120 as mathematical ideas.
121* Grammar in Bison:: How we represent grammars for Bison's sake.
122* Semantic Values:: Each token or syntactic grouping can have
123 a semantic value (the value of an integer,
124 the name of an identifier, etc.).
125* Semantic Actions:: Each rule can have an action containing C code.
99a9344e 126* GLR Parsers:: Writing parsers for general context-free languages.
93dd49ab 127* Locations Overview:: Tracking Locations.
bfa74976
RS
128* Bison Parser:: What are Bison's input and output,
129 how is the output used?
130* Stages:: Stages in writing and running Bison grammars.
131* Grammar Layout:: Overall structure of a Bison grammar file.
132
fa7e68c3
PE
133Writing @acronym{GLR} Parsers
134
32c29292
JD
135* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
136* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
137* GLR Semantic Actions:: Deferred semantic actions have special concerns.
138* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
fa7e68c3 139
bfa74976
RS
140Examples
141
142* RPN Calc:: Reverse polish notation calculator;
143 a first example with no operator precedence.
144* Infix Calc:: Infix (algebraic) notation calculator.
145 Operator precedence is introduced.
146* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 147* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
93dd49ab
PE
148* Multi-function Calc:: Calculator with memory and trig functions.
149 It uses multiple data-types for semantic values.
bfa74976
RS
150* Exercises:: Ideas for improving the multi-function calculator.
151
152Reverse Polish Notation Calculator
153
75f5aaea 154* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
bfa74976
RS
155* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
156* Lexer: Rpcalc Lexer. The lexical analyzer.
157* Main: Rpcalc Main. The controlling function.
158* Error: Rpcalc Error. The error reporting function.
159* Gen: Rpcalc Gen. Running Bison on the grammar file.
160* Comp: Rpcalc Compile. Run the C compiler on the output code.
161
162Grammar Rules for @code{rpcalc}
163
13863333
AD
164* Rpcalc Input::
165* Rpcalc Line::
166* Rpcalc Expr::
bfa74976 167
342b8b6e
AD
168Location Tracking Calculator: @code{ltcalc}
169
170* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
171* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
172* Lexer: Ltcalc Lexer. The lexical analyzer.
173
bfa74976
RS
174Multi-Function Calculator: @code{mfcalc}
175
176* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
177* Rules: Mfcalc Rules. Grammar rules for the calculator.
178* Symtab: Mfcalc Symtab. Symbol table management subroutines.
179
180Bison Grammar Files
181
182* Grammar Outline:: Overall layout of the grammar file.
183* Symbols:: Terminal and nonterminal symbols.
184* Rules:: How to write grammar rules.
185* Recursion:: Writing recursive rules.
186* Semantics:: Semantic values and actions.
93dd49ab 187* Locations:: Locations and actions.
bfa74976
RS
188* Declarations:: All kinds of Bison declarations are described here.
189* Multiple Parsers:: Putting more than one Bison parser in one program.
190
191Outline of a Bison Grammar
192
93dd49ab 193* Prologue:: Syntax and usage of the prologue.
bfa74976
RS
194* Bison Declarations:: Syntax and usage of the Bison declarations section.
195* Grammar Rules:: Syntax and usage of the grammar rules section.
93dd49ab 196* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
197
198Defining Language Semantics
199
200* Value Type:: Specifying one data type for all semantic values.
201* Multiple Types:: Specifying several alternative data types.
202* Actions:: An action is the semantic definition of a grammar rule.
203* Action Types:: Specifying data types for actions to operate on.
204* Mid-Rule Actions:: Most actions go at the end of a rule.
205 This says when, why and how to use the exceptional
206 action in the middle of a rule.
207
93dd49ab
PE
208Tracking Locations
209
210* Location Type:: Specifying a data type for locations.
211* Actions and Locations:: Using locations in actions.
212* Location Default Action:: Defining a general way to compute locations.
213
bfa74976
RS
214Bison Declarations
215
b50d2359 216* Require Decl:: Requiring a Bison version.
bfa74976
RS
217* Token Decl:: Declaring terminal symbols.
218* Precedence Decl:: Declaring terminals with precedence and associativity.
219* Union Decl:: Declaring the set of all semantic value types.
220* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 221* Initial Action Decl:: Code run before parsing starts.
72f889cc 222* Destructor Decl:: Declaring how symbols are freed.
d6328241 223* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
224* Start Decl:: Specifying the start symbol.
225* Pure Decl:: Requesting a reentrant parser.
226* Decl Summary:: Table of all Bison declarations.
227
228Parser C-Language Interface
229
230* Parser Function:: How to call @code{yyparse} and what it returns.
13863333 231* Lexical:: You must supply a function @code{yylex}
bfa74976
RS
232 which reads tokens.
233* Error Reporting:: You must supply a function @code{yyerror}.
234* Action Features:: Special features for use in actions.
f7ab6a50
PE
235* Internationalization:: How to let the parser speak in the user's
236 native language.
bfa74976
RS
237
238The Lexical Analyzer Function @code{yylex}
239
240* Calling Convention:: How @code{yyparse} calls @code{yylex}.
241* Token Values:: How @code{yylex} must return the semantic value
242 of the token it has read.
95923bd6 243* Token Locations:: How @code{yylex} must return the text location
bfa74976 244 (line number, etc.) of the token, if the
93dd49ab 245 actions want that.
bfa74976
RS
246* Pure Calling:: How the calling convention differs
247 in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
248
13863333 249The Bison Parser Algorithm
bfa74976
RS
250
251* Look-Ahead:: Parser looks one token ahead when deciding what to do.
252* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
253* Precedence:: Operator precedence works by resolving conflicts.
254* Contextual Precedence:: When an operator's precedence depends on context.
255* Parser States:: The parser is a finite-state-machine with stack.
256* Reduce/Reduce:: When two rules are applicable in the same situation.
257* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 258* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 259* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
260
261Operator Precedence
262
263* Why Precedence:: An example showing why precedence is needed.
264* Using Precedence:: How to specify precedence in Bison grammars.
265* Precedence Examples:: How these features are used in the previous example.
266* How Precedence:: How they work.
267
268Handling Context Dependencies
269
270* Semantic Tokens:: Token parsing can depend on the semantic context.
271* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
272* Tie-in Recovery:: Lexical tie-ins have implications for how
273 error recovery rules must be written.
274
93dd49ab 275Debugging Your Parser
ec3bc396
AD
276
277* Understanding:: Understanding the structure of your parser.
278* Tracing:: Tracing the execution of your parser.
279
bfa74976
RS
280Invoking Bison
281
13863333 282* Bison Options:: All the options described in detail,
c827f760 283 in alphabetical order by short options.
bfa74976 284* Option Cross Key:: Alphabetical list of long options.
93dd49ab 285* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
f2b5126e 286
12545799
AD
287C++ Language Interface
288
289* C++ Parsers:: The interface to generate C++ parser classes
290* A Complete C++ Example:: Demonstrating their use
291
292C++ Parsers
293
294* C++ Bison Interface:: Asking for C++ parser generation
295* C++ Semantic Values:: %union vs. C++
296* C++ Location Values:: The position and location classes
297* C++ Parser Interface:: Instantiating and running the parser
298* C++ Scanner Interface:: Exchanges between yylex and parse
299
300A Complete C++ Example
301
302* Calc++ --- C++ Calculator:: The specifications
303* Calc++ Parsing Driver:: An active parsing context
304* Calc++ Parser:: A parser class
305* Calc++ Scanner:: A pure C++ Flex scanner
306* Calc++ Top Level:: Conducting the band
307
d1a1114f
AD
308Frequently Asked Questions
309
1a059451 310* Memory Exhausted:: Breaking the Stack Limits
e64fec0a 311* How Can I Reset the Parser:: @code{yyparse} Keeps some State
fef4cb51 312* Strings are Destroyed:: @code{yylval} Loses Track of Strings
2fa09258 313* Implementing Gotos/Loops:: Control Flow in the Calculator
ed2e6384 314* Multiple start-symbols:: Factoring closely related grammars
55ba27be
AD
315* Secure? Conform?:: Is Bison @acronym{POSIX} safe?
316* I can't build Bison:: Troubleshooting
317* Where can I find help?:: Troubleshouting
318* Bug Reports:: Troublereporting
319* Other Languages:: Parsers in Java and others
320* Beta Testing:: Experimenting development versions
321* Mailing Lists:: Meeting other Bison users
d1a1114f 322
f2b5126e
PB
323Copying This Manual
324
325* GNU Free Documentation License:: License for copying this manual.
326
342b8b6e 327@end detailmenu
bfa74976
RS
328@end menu
329
342b8b6e 330@node Introduction
bfa74976
RS
331@unnumbered Introduction
332@cindex introduction
333
6077da58
PE
334@dfn{Bison} is a general-purpose parser generator that converts an
335annotated context-free grammar into an @acronym{LALR}(1) or
336@acronym{GLR} parser for that grammar. Once you are proficient with
1e137b71 337Bison, you can use it to develop a wide range of language parsers, from those
bfa74976
RS
338used in simple desk calculators to complex programming languages.
339
340Bison is upward compatible with Yacc: all properly-written Yacc grammars
341ought to work with Bison with no change. Anyone familiar with Yacc
342should be able to use Bison with little trouble. You need to be fluent in
1e137b71 343C or C++ programming in order to use Bison or to understand this manual.
bfa74976
RS
344
345We begin with tutorial chapters that explain the basic concepts of using
346Bison and show three explained examples, each building on the last. If you
347don't know Bison or Yacc, start by reading these chapters. Reference
348chapters follow which describe specific aspects of Bison in detail.
349
931c7513
RS
350Bison was written primarily by Robert Corbett; Richard Stallman made it
351Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
14ded682 352multi-character string literals and other features.
931c7513 353
df1af54c 354This edition corresponds to version @value{VERSION} of Bison.
bfa74976 355
342b8b6e 356@node Conditions
bfa74976
RS
357@unnumbered Conditions for Using Bison
358
193d7c70
PE
359The distribution terms for Bison-generated parsers permit using the
360parsers in nonfree programs. Before Bison version 2.2, these extra
361permissions applied only when Bison was generating @acronym{LALR}(1)
362parsers in C@. And before Bison version 1.24, Bison-generated
262aa8dd 363parsers could be used only in programs that were free software.
a31239f1 364
c827f760
PE
365The other @acronym{GNU} programming tools, such as the @acronym{GNU} C
366compiler, have never
9ecbd125 367had such a requirement. They could always be used for nonfree
a31239f1
RS
368software. The reason Bison was different was not due to a special
369policy decision; it resulted from applying the usual General Public
370License to all of the Bison source code.
371
372The output of the Bison utility---the Bison parser file---contains a
373verbatim copy of a sizable piece of Bison, which is the code for the
193d7c70
PE
374parser's implementation. (The actions from your grammar are inserted
375into this implementation at one point, but most of the rest of the
376implementation is not changed.) When we applied the @acronym{GPL}
377terms to the skeleton code for the parser's implementation,
a31239f1
RS
378the effect was to restrict the use of Bison output to free software.
379
380We didn't change the terms because of sympathy for people who want to
381make software proprietary. @strong{Software should be free.} But we
382concluded that limiting Bison's use to free software was doing little to
383encourage people to make other software free. So we decided to make the
384practical conditions for using Bison match the practical conditions for
c827f760 385using the other @acronym{GNU} tools.
bfa74976 386
193d7c70
PE
387This exception applies when Bison is generating code for a parser.
388You can tell whether the exception applies to a Bison output file by
389inspecting the file for text beginning with ``As a special
390exception@dots{}''. The text spells out the exact terms of the
391exception.
262aa8dd 392
c67a198d 393@include gpl.texi
bfa74976 394
342b8b6e 395@node Concepts
bfa74976
RS
396@chapter The Concepts of Bison
397
398This chapter introduces many of the basic concepts without which the
399details of Bison will not make sense. If you do not already know how to
400use Bison or Yacc, we suggest you start by reading this chapter carefully.
401
402@menu
403* Language and Grammar:: Languages and context-free grammars,
404 as mathematical ideas.
405* Grammar in Bison:: How we represent grammars for Bison's sake.
406* Semantic Values:: Each token or syntactic grouping can have
407 a semantic value (the value of an integer,
408 the name of an identifier, etc.).
409* Semantic Actions:: Each rule can have an action containing C code.
99a9344e 410* GLR Parsers:: Writing parsers for general context-free languages.
847bf1f5 411* Locations Overview:: Tracking Locations.
bfa74976
RS
412* Bison Parser:: What are Bison's input and output,
413 how is the output used?
414* Stages:: Stages in writing and running Bison grammars.
415* Grammar Layout:: Overall structure of a Bison grammar file.
416@end menu
417
342b8b6e 418@node Language and Grammar
bfa74976
RS
419@section Languages and Context-Free Grammars
420
bfa74976
RS
421@cindex context-free grammar
422@cindex grammar, context-free
423In order for Bison to parse a language, it must be described by a
424@dfn{context-free grammar}. This means that you specify one or more
425@dfn{syntactic groupings} and give rules for constructing them from their
426parts. For example, in the C language, one kind of grouping is called an
427`expression'. One rule for making an expression might be, ``An expression
428can be made of a minus sign and another expression''. Another would be,
429``An expression can be an integer''. As you can see, rules are often
430recursive, but there must be at least one rule which leads out of the
431recursion.
432
c827f760 433@cindex @acronym{BNF}
bfa74976
RS
434@cindex Backus-Naur form
435The most common formal system for presenting such rules for humans to read
c827f760
PE
436is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in
437order to specify the language Algol 60. Any grammar expressed in
438@acronym{BNF} is a context-free grammar. The input to Bison is
439essentially machine-readable @acronym{BNF}.
bfa74976 440
c827f760
PE
441@cindex @acronym{LALR}(1) grammars
442@cindex @acronym{LR}(1) grammars
676385e2
PH
443There are various important subclasses of context-free grammar. Although it
444can handle almost all context-free grammars, Bison is optimized for what
c827f760 445are called @acronym{LALR}(1) grammars.
676385e2 446In brief, in these grammars, it must be possible to
bfa74976
RS
447tell how to parse any portion of an input string with just a single
448token of look-ahead. Strictly speaking, that is a description of an
c827f760
PE
449@acronym{LR}(1) grammar, and @acronym{LALR}(1) involves additional
450restrictions that are
bfa74976 451hard to explain simply; but it is rare in actual practice to find an
c827f760
PE
452@acronym{LR}(1) grammar that fails to be @acronym{LALR}(1).
453@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
454more information on this.
bfa74976 455
c827f760
PE
456@cindex @acronym{GLR} parsing
457@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2 458@cindex ambiguous grammars
9d9b8b70 459@cindex nondeterministic parsing
9501dc6e
AD
460
461Parsers for @acronym{LALR}(1) grammars are @dfn{deterministic}, meaning
462roughly that the next grammar rule to apply at any point in the input is
463uniquely determined by the preceding input and a fixed, finite portion
464(called a @dfn{look-ahead}) of the remaining input. A context-free
465grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
e4f85c39 466apply the grammar rules to get the same inputs. Even unambiguous
9d9b8b70 467grammars can be @dfn{nondeterministic}, meaning that no fixed
9501dc6e
AD
468look-ahead always suffices to determine the next grammar rule to apply.
469With the proper declarations, Bison is also able to parse these more
470general context-free grammars, using a technique known as @acronym{GLR}
471parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers
472are able to handle any context-free grammar for which the number of
473possible parses of any given string is finite.
676385e2 474
bfa74976
RS
475@cindex symbols (abstract)
476@cindex token
477@cindex syntactic grouping
478@cindex grouping, syntactic
9501dc6e
AD
479In the formal grammatical rules for a language, each kind of syntactic
480unit or grouping is named by a @dfn{symbol}. Those which are built by
481grouping smaller constructs according to grammatical rules are called
bfa74976
RS
482@dfn{nonterminal symbols}; those which can't be subdivided are called
483@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
484corresponding to a single terminal symbol a @dfn{token}, and a piece
e0c471a9 485corresponding to a single nonterminal symbol a @dfn{grouping}.
bfa74976
RS
486
487We can use the C language as an example of what symbols, terminal and
9501dc6e
AD
488nonterminal, mean. The tokens of C are identifiers, constants (numeric
489and string), and the various keywords, arithmetic operators and
490punctuation marks. So the terminal symbols of a grammar for C include
491`identifier', `number', `string', plus one symbol for each keyword,
492operator or punctuation mark: `if', `return', `const', `static', `int',
493`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
494(These tokens can be subdivided into characters, but that is a matter of
bfa74976
RS
495lexicography, not grammar.)
496
497Here is a simple C function subdivided into tokens:
498
9edcd895
AD
499@ifinfo
500@example
501int /* @r{keyword `int'} */
14d4662b 502square (int x) /* @r{identifier, open-paren, keyword `int',}
9edcd895
AD
503 @r{identifier, close-paren} */
504@{ /* @r{open-brace} */
aa08666d
AD
505 return x * x; /* @r{keyword `return', identifier, asterisk,}
506 @r{identifier, semicolon} */
9edcd895
AD
507@} /* @r{close-brace} */
508@end example
509@end ifinfo
510@ifnotinfo
bfa74976
RS
511@example
512int /* @r{keyword `int'} */
14d4662b 513square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
bfa74976 514@{ /* @r{open-brace} */
9edcd895 515 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
bfa74976
RS
516@} /* @r{close-brace} */
517@end example
9edcd895 518@end ifnotinfo
bfa74976
RS
519
520The syntactic groupings of C include the expression, the statement, the
521declaration, and the function definition. These are represented in the
522grammar of C by nonterminal symbols `expression', `statement',
523`declaration' and `function definition'. The full grammar uses dozens of
524additional language constructs, each with its own nonterminal symbol, in
525order to express the meanings of these four. The example above is a
526function definition; it contains one declaration, and one statement. In
527the statement, each @samp{x} is an expression and so is @samp{x * x}.
528
529Each nonterminal symbol must have grammatical rules showing how it is made
530out of simpler constructs. For example, one kind of C statement is the
531@code{return} statement; this would be described with a grammar rule which
532reads informally as follows:
533
534@quotation
535A `statement' can be made of a `return' keyword, an `expression' and a
536`semicolon'.
537@end quotation
538
539@noindent
540There would be many other rules for `statement', one for each kind of
541statement in C.
542
543@cindex start symbol
544One nonterminal symbol must be distinguished as the special one which
545defines a complete utterance in the language. It is called the @dfn{start
546symbol}. In a compiler, this means a complete input program. In the C
547language, the nonterminal symbol `sequence of definitions and declarations'
548plays this role.
549
550For example, @samp{1 + 2} is a valid C expression---a valid part of a C
551program---but it is not valid as an @emph{entire} C program. In the
552context-free grammar of C, this follows from the fact that `expression' is
553not the start symbol.
554
555The Bison parser reads a sequence of tokens as its input, and groups the
556tokens using the grammar rules. If the input is valid, the end result is
557that the entire token sequence reduces to a single grouping whose symbol is
558the grammar's start symbol. If we use a grammar for C, the entire input
559must be a `sequence of definitions and declarations'. If not, the parser
560reports a syntax error.
561
342b8b6e 562@node Grammar in Bison
bfa74976
RS
563@section From Formal Rules to Bison Input
564@cindex Bison grammar
565@cindex grammar, Bison
566@cindex formal grammar
567
568A formal grammar is a mathematical construct. To define the language
569for Bison, you must write a file expressing the grammar in Bison syntax:
570a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
571
572A nonterminal symbol in the formal grammar is represented in Bison input
c827f760 573as an identifier, like an identifier in C@. By convention, it should be
bfa74976
RS
574in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
575
576The Bison representation for a terminal symbol is also called a @dfn{token
577type}. Token types as well can be represented as C-like identifiers. By
578convention, these identifiers should be upper case to distinguish them from
579nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
580@code{RETURN}. A terminal symbol that stands for a particular keyword in
581the language should be named after that keyword converted to upper case.
582The terminal symbol @code{error} is reserved for error recovery.
931c7513 583@xref{Symbols}.
bfa74976
RS
584
585A terminal symbol can also be represented as a character literal, just like
586a C character constant. You should do this whenever a token is just a
587single character (parenthesis, plus-sign, etc.): use that same character in
588a literal as the terminal symbol for that token.
589
931c7513
RS
590A third way to represent a terminal symbol is with a C string constant
591containing several characters. @xref{Symbols}, for more information.
592
bfa74976
RS
593The grammar rules also have an expression in Bison syntax. For example,
594here is the Bison rule for a C @code{return} statement. The semicolon in
595quotes is a literal character token, representing part of the C syntax for
596the statement; the naked semicolon, and the colon, are Bison punctuation
597used in every rule.
598
599@example
600stmt: RETURN expr ';'
601 ;
602@end example
603
604@noindent
605@xref{Rules, ,Syntax of Grammar Rules}.
606
342b8b6e 607@node Semantic Values
bfa74976
RS
608@section Semantic Values
609@cindex semantic value
610@cindex value, semantic
611
612A formal grammar selects tokens only by their classifications: for example,
613if a rule mentions the terminal symbol `integer constant', it means that
614@emph{any} integer constant is grammatically valid in that position. The
615precise value of the constant is irrelevant to how to parse the input: if
616@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
e0c471a9 617grammatical.
bfa74976
RS
618
619But the precise value is very important for what the input means once it is
620parsed. A compiler is useless if it fails to distinguish between 4, 1 and
6213989 as constants in the program! Therefore, each token in a Bison grammar
c827f760
PE
622has both a token type and a @dfn{semantic value}. @xref{Semantics,
623,Defining Language Semantics},
bfa74976
RS
624for details.
625
626The token type is a terminal symbol defined in the grammar, such as
627@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
628you need to know to decide where the token may validly appear and how to
629group it with other tokens. The grammar rules know nothing about tokens
e0c471a9 630except their types.
bfa74976
RS
631
632The semantic value has all the rest of the information about the
633meaning of the token, such as the value of an integer, or the name of an
634identifier. (A token such as @code{','} which is just punctuation doesn't
635need to have any semantic value.)
636
637For example, an input token might be classified as token type
638@code{INTEGER} and have the semantic value 4. Another input token might
639have the same token type @code{INTEGER} but value 3989. When a grammar
640rule says that @code{INTEGER} is allowed, either of these tokens is
641acceptable because each is an @code{INTEGER}. When the parser accepts the
642token, it keeps track of the token's semantic value.
643
644Each grouping can also have a semantic value as well as its nonterminal
645symbol. For example, in a calculator, an expression typically has a
646semantic value that is a number. In a compiler for a programming
647language, an expression typically has a semantic value that is a tree
648structure describing the meaning of the expression.
649
342b8b6e 650@node Semantic Actions
bfa74976
RS
651@section Semantic Actions
652@cindex semantic actions
653@cindex actions, semantic
654
655In order to be useful, a program must do more than parse input; it must
656also produce some output based on the input. In a Bison grammar, a grammar
657rule can have an @dfn{action} made up of C statements. Each time the
658parser recognizes a match for that rule, the action is executed.
659@xref{Actions}.
13863333 660
bfa74976
RS
661Most of the time, the purpose of an action is to compute the semantic value
662of the whole construct from the semantic values of its parts. For example,
663suppose we have a rule which says an expression can be the sum of two
664expressions. When the parser recognizes such a sum, each of the
665subexpressions has a semantic value which describes how it was built up.
666The action for this rule should create a similar sort of value for the
667newly recognized larger expression.
668
669For example, here is a rule that says an expression can be the sum of
670two subexpressions:
671
672@example
673expr: expr '+' expr @{ $$ = $1 + $3; @}
674 ;
675@end example
676
677@noindent
678The action says how to produce the semantic value of the sum expression
679from the values of the two subexpressions.
680
676385e2 681@node GLR Parsers
c827f760
PE
682@section Writing @acronym{GLR} Parsers
683@cindex @acronym{GLR} parsing
684@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2
PH
685@findex %glr-parser
686@cindex conflicts
687@cindex shift/reduce conflicts
fa7e68c3 688@cindex reduce/reduce conflicts
676385e2 689
fa7e68c3 690In some grammars, Bison's standard
9501dc6e
AD
691@acronym{LALR}(1) parsing algorithm cannot decide whether to apply a
692certain grammar rule at a given point. That is, it may not be able to
693decide (on the basis of the input read so far) which of two possible
694reductions (applications of a grammar rule) applies, or whether to apply
695a reduction or read more of the input and apply a reduction later in the
696input. These are known respectively as @dfn{reduce/reduce} conflicts
697(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
698(@pxref{Shift/Reduce}).
699
700To use a grammar that is not easily modified to be @acronym{LALR}(1), a
701more general parsing algorithm is sometimes necessary. If you include
676385e2 702@code{%glr-parser} among the Bison declarations in your file
fa7e68c3 703(@pxref{Grammar Outline}), the result is a Generalized @acronym{LR}
9501dc6e
AD
704(@acronym{GLR}) parser. These parsers handle Bison grammars that
705contain no unresolved conflicts (i.e., after applying precedence
706declarations) identically to @acronym{LALR}(1) parsers. However, when
707faced with unresolved shift/reduce and reduce/reduce conflicts,
708@acronym{GLR} parsers use the simple expedient of doing both,
709effectively cloning the parser to follow both possibilities. Each of
710the resulting parsers can again split, so that at any given time, there
711can be any number of possible parses being explored. The parsers
676385e2
PH
712proceed in lockstep; that is, all of them consume (shift) a given input
713symbol before any of them proceed to the next. Each of the cloned
714parsers eventually meets one of two possible fates: either it runs into
715a parsing error, in which case it simply vanishes, or it merges with
716another parser, because the two of them have reduced the input to an
717identical set of symbols.
718
719During the time that there are multiple parsers, semantic actions are
720recorded, but not performed. When a parser disappears, its recorded
721semantic actions disappear as well, and are never performed. When a
722reduction makes two parsers identical, causing them to merge, Bison
723records both sets of semantic actions. Whenever the last two parsers
724merge, reverting to the single-parser case, Bison resolves all the
725outstanding actions either by precedences given to the grammar rules
726involved, or by performing both actions, and then calling a designated
727user-defined function on the resulting values to produce an arbitrary
728merged result.
729
fa7e68c3 730@menu
32c29292
JD
731* Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
732* Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
733* GLR Semantic Actions:: Deferred semantic actions have special concerns.
734* Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
fa7e68c3
PE
735@end menu
736
737@node Simple GLR Parsers
738@subsection Using @acronym{GLR} on Unambiguous Grammars
739@cindex @acronym{GLR} parsing, unambiguous grammars
740@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars
741@findex %glr-parser
742@findex %expect-rr
743@cindex conflicts
744@cindex reduce/reduce conflicts
745@cindex shift/reduce conflicts
746
747In the simplest cases, you can use the @acronym{GLR} algorithm
748to parse grammars that are unambiguous, but fail to be @acronym{LALR}(1).
749Such grammars typically require more than one symbol of look-ahead,
750or (in rare cases) fall into the category of grammars in which the
751@acronym{LALR}(1) algorithm throws away too much information (they are in
752@acronym{LR}(1), but not @acronym{LALR}(1), @ref{Mystery Conflicts}).
753
754Consider a problem that
755arises in the declaration of enumerated and subrange types in the
756programming language Pascal. Here are some examples:
757
758@example
759type subrange = lo .. hi;
760type enum = (a, b, c);
761@end example
762
763@noindent
764The original language standard allows only numeric
765literals and constant identifiers for the subrange bounds (@samp{lo}
766and @samp{hi}), but Extended Pascal (@acronym{ISO}/@acronym{IEC}
76710206) and many other
768Pascal implementations allow arbitrary expressions there. This gives
769rise to the following situation, containing a superfluous pair of
770parentheses:
771
772@example
773type subrange = (a) .. b;
774@end example
775
776@noindent
777Compare this to the following declaration of an enumerated
778type with only one value:
779
780@example
781type enum = (a);
782@end example
783
784@noindent
785(These declarations are contrived, but they are syntactically
786valid, and more-complicated cases can come up in practical programs.)
787
788These two declarations look identical until the @samp{..} token.
789With normal @acronym{LALR}(1) one-token look-ahead it is not
790possible to decide between the two forms when the identifier
791@samp{a} is parsed. It is, however, desirable
792for a parser to decide this, since in the latter case
793@samp{a} must become a new identifier to represent the enumeration
794value, while in the former case @samp{a} must be evaluated with its
795current meaning, which may be a constant or even a function call.
796
797You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
798to be resolved later, but this typically requires substantial
799contortions in both semantic actions and large parts of the
800grammar, where the parentheses are nested in the recursive rules for
801expressions.
802
803You might think of using the lexer to distinguish between the two
804forms by returning different tokens for currently defined and
805undefined identifiers. But if these declarations occur in a local
806scope, and @samp{a} is defined in an outer scope, then both forms
807are possible---either locally redefining @samp{a}, or using the
808value of @samp{a} from the outer scope. So this approach cannot
809work.
810
e757bb10 811A simple solution to this problem is to declare the parser to
fa7e68c3
PE
812use the @acronym{GLR} algorithm.
813When the @acronym{GLR} parser reaches the critical state, it
814merely splits into two branches and pursues both syntax rules
815simultaneously. Sooner or later, one of them runs into a parsing
816error. If there is a @samp{..} token before the next
817@samp{;}, the rule for enumerated types fails since it cannot
818accept @samp{..} anywhere; otherwise, the subrange type rule
819fails since it requires a @samp{..} token. So one of the branches
820fails silently, and the other one continues normally, performing
821all the intermediate actions that were postponed during the split.
822
823If the input is syntactically incorrect, both branches fail and the parser
824reports a syntax error as usual.
825
826The effect of all this is that the parser seems to ``guess'' the
827correct branch to take, or in other words, it seems to use more
828look-ahead than the underlying @acronym{LALR}(1) algorithm actually allows
829for. In this example, @acronym{LALR}(2) would suffice, but also some cases
830that are not @acronym{LALR}(@math{k}) for any @math{k} can be handled this way.
831
832In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time,
833and the current Bison parser even takes exponential time and space
834for some grammars. In practice, this rarely happens, and for many
835grammars it is possible to prove that it cannot happen.
836The present example contains only one conflict between two
837rules, and the type-declaration context containing the conflict
838cannot be nested. So the number of
839branches that can exist at any time is limited by the constant 2,
840and the parsing time is still linear.
841
842Here is a Bison grammar corresponding to the example above. It
843parses a vastly simplified form of Pascal type declarations.
844
845@example
846%token TYPE DOTDOT ID
847
848@group
849%left '+' '-'
850%left '*' '/'
851@end group
852
853%%
854
855@group
856type_decl : TYPE ID '=' type ';'
857 ;
858@end group
859
860@group
861type : '(' id_list ')'
862 | expr DOTDOT expr
863 ;
864@end group
865
866@group
867id_list : ID
868 | id_list ',' ID
869 ;
870@end group
871
872@group
873expr : '(' expr ')'
874 | expr '+' expr
875 | expr '-' expr
876 | expr '*' expr
877 | expr '/' expr
878 | ID
879 ;
880@end group
881@end example
882
883When used as a normal @acronym{LALR}(1) grammar, Bison correctly complains
884about one reduce/reduce conflict. In the conflicting situation the
885parser chooses one of the alternatives, arbitrarily the one
886declared first. Therefore the following correct input is not
887recognized:
888
889@example
890type t = (a) .. b;
891@end example
892
893The parser can be turned into a @acronym{GLR} parser, while also telling Bison
894to be silent about the one known reduce/reduce conflict, by
e757bb10 895adding these two declarations to the Bison input file (before the first
fa7e68c3
PE
896@samp{%%}):
897
898@example
899%glr-parser
900%expect-rr 1
901@end example
902
903@noindent
904No change in the grammar itself is required. Now the
905parser recognizes all valid declarations, according to the
906limited syntax above, transparently. In fact, the user does not even
907notice when the parser splits.
908
f8e1c9e5
AD
909So here we have a case where we can use the benefits of @acronym{GLR},
910almost without disadvantages. Even in simple cases like this, however,
911there are at least two potential problems to beware. First, always
912analyze the conflicts reported by Bison to make sure that @acronym{GLR}
913splitting is only done where it is intended. A @acronym{GLR} parser
914splitting inadvertently may cause problems less obvious than an
915@acronym{LALR} parser statically choosing the wrong alternative in a
916conflict. Second, consider interactions with the lexer (@pxref{Semantic
917Tokens}) with great care. Since a split parser consumes tokens without
918performing any actions during the split, the lexer cannot obtain
919information via parser actions. Some cases of lexer interactions can be
920eliminated by using @acronym{GLR} to shift the complications from the
921lexer to the parser. You must check the remaining cases for
922correctness.
923
924In our example, it would be safe for the lexer to return tokens based on
925their current meanings in some symbol table, because no new symbols are
926defined in the middle of a type declaration. Though it is possible for
927a parser to define the enumeration constants as they are parsed, before
928the type declaration is completed, it actually makes no difference since
929they cannot be used within the same enumerated type declaration.
fa7e68c3
PE
930
931@node Merging GLR Parses
932@subsection Using @acronym{GLR} to Resolve Ambiguities
933@cindex @acronym{GLR} parsing, ambiguous grammars
934@cindex generalized @acronym{LR} (@acronym{GLR}) parsing, ambiguous grammars
935@findex %dprec
936@findex %merge
937@cindex conflicts
938@cindex reduce/reduce conflicts
939
2a8d363a 940Let's consider an example, vastly simplified from a C++ grammar.
676385e2
PH
941
942@example
943%@{
38a92d50
PE
944 #include <stdio.h>
945 #define YYSTYPE char const *
946 int yylex (void);
947 void yyerror (char const *);
676385e2
PH
948%@}
949
950%token TYPENAME ID
951
952%right '='
953%left '+'
954
955%glr-parser
956
957%%
958
fae437e8 959prog :
676385e2
PH
960 | prog stmt @{ printf ("\n"); @}
961 ;
962
963stmt : expr ';' %dprec 1
964 | decl %dprec 2
965 ;
966
2a8d363a 967expr : ID @{ printf ("%s ", $$); @}
fae437e8 968 | TYPENAME '(' expr ')'
2a8d363a
AD
969 @{ printf ("%s <cast> ", $1); @}
970 | expr '+' expr @{ printf ("+ "); @}
971 | expr '=' expr @{ printf ("= "); @}
676385e2
PH
972 ;
973
fae437e8 974decl : TYPENAME declarator ';'
2a8d363a 975 @{ printf ("%s <declare> ", $1); @}
676385e2 976 | TYPENAME declarator '=' expr ';'
2a8d363a 977 @{ printf ("%s <init-declare> ", $1); @}
676385e2
PH
978 ;
979
2a8d363a 980declarator : ID @{ printf ("\"%s\" ", $1); @}
676385e2
PH
981 | '(' declarator ')'
982 ;
983@end example
984
985@noindent
986This models a problematic part of the C++ grammar---the ambiguity between
987certain declarations and statements. For example,
988
989@example
990T (x) = y+z;
991@end example
992
993@noindent
994parses as either an @code{expr} or a @code{stmt}
c827f760
PE
995(assuming that @samp{T} is recognized as a @code{TYPENAME} and
996@samp{x} as an @code{ID}).
676385e2 997Bison detects this as a reduce/reduce conflict between the rules
fae437e8 998@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
e757bb10
AD
999time it encounters @code{x} in the example above. Since this is a
1000@acronym{GLR} parser, it therefore splits the problem into two parses, one for
fa7e68c3
PE
1001each choice of resolving the reduce/reduce conflict.
1002Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1003however, neither of these parses ``dies,'' because the grammar as it stands is
e757bb10
AD
1004ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1005the other reduces @code{stmt : decl}, after which both parsers are in an
1006identical state: they've seen @samp{prog stmt} and have the same unprocessed
1007input remaining. We say that these parses have @dfn{merged.}
fa7e68c3
PE
1008
1009At this point, the @acronym{GLR} parser requires a specification in the
1010grammar of how to choose between the competing parses.
1011In the example above, the two @code{%dprec}
e757bb10 1012declarations specify that Bison is to give precedence
fa7e68c3 1013to the parse that interprets the example as a
676385e2
PH
1014@code{decl}, which implies that @code{x} is a declarator.
1015The parser therefore prints
1016
1017@example
fae437e8 1018"x" y z + T <init-declare>
676385e2
PH
1019@end example
1020
fa7e68c3
PE
1021The @code{%dprec} declarations only come into play when more than one
1022parse survives. Consider a different input string for this parser:
676385e2
PH
1023
1024@example
1025T (x) + y;
1026@end example
1027
1028@noindent
e757bb10 1029This is another example of using @acronym{GLR} to parse an unambiguous
fa7e68c3 1030construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
676385e2
PH
1031Here, there is no ambiguity (this cannot be parsed as a declaration).
1032However, at the time the Bison parser encounters @code{x}, it does not
1033have enough information to resolve the reduce/reduce conflict (again,
1034between @code{x} as an @code{expr} or a @code{declarator}). In this
fa7e68c3 1035case, no precedence declaration is used. Again, the parser splits
676385e2
PH
1036into two, one assuming that @code{x} is an @code{expr}, and the other
1037assuming @code{x} is a @code{declarator}. The second of these parsers
1038then vanishes when it sees @code{+}, and the parser prints
1039
1040@example
fae437e8 1041x T <cast> y +
676385e2
PH
1042@end example
1043
1044Suppose that instead of resolving the ambiguity, you wanted to see all
fa7e68c3 1045the possibilities. For this purpose, you must merge the semantic
676385e2
PH
1046actions of the two possible parsers, rather than choosing one over the
1047other. To do so, you could change the declaration of @code{stmt} as
1048follows:
1049
1050@example
1051stmt : expr ';' %merge <stmtMerge>
1052 | decl %merge <stmtMerge>
1053 ;
1054@end example
1055
1056@noindent
676385e2
PH
1057and define the @code{stmtMerge} function as:
1058
1059@example
38a92d50
PE
1060static YYSTYPE
1061stmtMerge (YYSTYPE x0, YYSTYPE x1)
676385e2
PH
1062@{
1063 printf ("<OR> ");
1064 return "";
1065@}
1066@end example
1067
1068@noindent
1069with an accompanying forward declaration
1070in the C declarations at the beginning of the file:
1071
1072@example
1073%@{
38a92d50 1074 #define YYSTYPE char const *
676385e2
PH
1075 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1076%@}
1077@end example
1078
1079@noindent
fa7e68c3
PE
1080With these declarations, the resulting parser parses the first example
1081as both an @code{expr} and a @code{decl}, and prints
676385e2
PH
1082
1083@example
fae437e8 1084"x" y z + T <init-declare> x T <cast> y z + = <OR>
676385e2
PH
1085@end example
1086
fa7e68c3 1087Bison requires that all of the
e757bb10 1088productions that participate in any particular merge have identical
fa7e68c3
PE
1089@samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1090and the parser will report an error during any parse that results in
1091the offending merge.
9501dc6e 1092
32c29292
JD
1093@node GLR Semantic Actions
1094@subsection GLR Semantic Actions
1095
1096@cindex deferred semantic actions
1097By definition, a deferred semantic action is not performed at the same time as
1098the associated reduction.
1099This raises caveats for several Bison features you might use in a semantic
1100action in a @acronym{GLR} parser.
1101
1102@vindex yychar
1103@cindex @acronym{GLR} parsers and @code{yychar}
1104@vindex yylval
1105@cindex @acronym{GLR} parsers and @code{yylval}
1106@vindex yylloc
1107@cindex @acronym{GLR} parsers and @code{yylloc}
1108In any semantic action, you can examine @code{yychar} to determine the type of
1109the look-ahead token present at the time of the associated reduction.
1110After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1111you can then examine @code{yylval} and @code{yylloc} to determine the
1112look-ahead token's semantic value and location, if any.
1113In a nondeferred semantic action, you can also modify any of these variables to
1114influence syntax analysis.
1115@xref{Look-Ahead, ,Look-Ahead Tokens}.
1116
1117@findex yyclearin
1118@cindex @acronym{GLR} parsers and @code{yyclearin}
1119In a deferred semantic action, it's too late to influence syntax analysis.
1120In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1121shallow copies of the values they had at the time of the associated reduction.
1122For this reason alone, modifying them is dangerous.
1123Moreover, the result of modifying them is undefined and subject to change with
1124future versions of Bison.
1125For example, if a semantic action might be deferred, you should never write it
1126to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1127memory referenced by @code{yylval}.
1128
1129@findex YYERROR
1130@cindex @acronym{GLR} parsers and @code{YYERROR}
1131Another Bison feature requiring special consideration is @code{YYERROR}
8710fc41 1132(@pxref{Action Features}), which you can invoke in a semantic action to
32c29292
JD
1133initiate error recovery.
1134During deterministic @acronym{GLR} operation, the effect of @code{YYERROR} is
1135the same as its effect in an @acronym{LALR}(1) parser.
1136In a deferred semantic action, its effect is undefined.
1137@c The effect is probably a syntax error at the split point.
1138
8710fc41
JD
1139Also, see @ref{Location Default Action, ,Default Action for Locations}, which
1140describes a special usage of @code{YYLLOC_DEFAULT} in @acronym{GLR} parsers.
1141
fa7e68c3
PE
1142@node Compiler Requirements
1143@subsection Considerations when Compiling @acronym{GLR} Parsers
1144@cindex @code{inline}
9501dc6e 1145@cindex @acronym{GLR} parsers and @code{inline}
fa7e68c3 1146
38a92d50
PE
1147The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or
1148later. In addition, they use the @code{inline} keyword, which is not
1149C89, but is C99 and is a common extension in pre-C99 compilers. It is
1150up to the user of these parsers to handle
9501dc6e
AD
1151portability issues. For instance, if using Autoconf and the Autoconf
1152macro @code{AC_C_INLINE}, a mere
1153
1154@example
1155%@{
38a92d50 1156 #include <config.h>
9501dc6e
AD
1157%@}
1158@end example
1159
1160@noindent
1161will suffice. Otherwise, we suggest
1162
1163@example
1164%@{
38a92d50
PE
1165 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1166 #define inline
1167 #endif
9501dc6e
AD
1168%@}
1169@end example
676385e2 1170
342b8b6e 1171@node Locations Overview
847bf1f5
AD
1172@section Locations
1173@cindex location
95923bd6
AD
1174@cindex textual location
1175@cindex location, textual
847bf1f5
AD
1176
1177Many applications, like interpreters or compilers, have to produce verbose
72d2299c 1178and useful error messages. To achieve this, one must be able to keep track of
95923bd6 1179the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
847bf1f5
AD
1180Bison provides a mechanism for handling these locations.
1181
72d2299c 1182Each token has a semantic value. In a similar fashion, each token has an
847bf1f5 1183associated location, but the type of locations is the same for all tokens and
72d2299c 1184groupings. Moreover, the output parser is equipped with a default data
847bf1f5
AD
1185structure for storing locations (@pxref{Locations}, for more details).
1186
1187Like semantic values, locations can be reached in actions using a dedicated
72d2299c 1188set of constructs. In the example above, the location of the whole grouping
847bf1f5
AD
1189is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1190@code{@@3}.
1191
1192When a rule is matched, a default action is used to compute the semantic value
72d2299c
PE
1193of its left hand side (@pxref{Actions}). In the same way, another default
1194action is used for locations. However, the action for locations is general
847bf1f5 1195enough for most cases, meaning there is usually no need to describe for each
72d2299c 1196rule how @code{@@$} should be formed. When building a new location for a given
847bf1f5
AD
1197grouping, the default behavior of the output parser is to take the beginning
1198of the first symbol, and the end of the last symbol.
1199
342b8b6e 1200@node Bison Parser
bfa74976
RS
1201@section Bison Output: the Parser File
1202@cindex Bison parser
1203@cindex Bison utility
1204@cindex lexical analyzer, purpose
1205@cindex parser
1206
1207When you run Bison, you give it a Bison grammar file as input. The output
1208is a C source file that parses the language described by the grammar.
1209This file is called a @dfn{Bison parser}. Keep in mind that the Bison
1210utility and the Bison parser are two distinct programs: the Bison utility
1211is a program whose output is the Bison parser that becomes part of your
1212program.
1213
1214The job of the Bison parser is to group tokens into groupings according to
1215the grammar rules---for example, to build identifiers and operators into
1216expressions. As it does this, it runs the actions for the grammar rules it
1217uses.
1218
704a47c4
AD
1219The tokens come from a function called the @dfn{lexical analyzer} that
1220you must supply in some fashion (such as by writing it in C). The Bison
1221parser calls the lexical analyzer each time it wants a new token. It
1222doesn't know what is ``inside'' the tokens (though their semantic values
1223may reflect this). Typically the lexical analyzer makes the tokens by
1224parsing characters of text, but Bison does not depend on this.
1225@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
bfa74976
RS
1226
1227The Bison parser file is C code which defines a function named
1228@code{yyparse} which implements that grammar. This function does not make
1229a complete C program: you must supply some additional functions. One is
1230the lexical analyzer. Another is an error-reporting function which the
1231parser calls to report an error. In addition, a complete C program must
1232start with a function called @code{main}; you have to provide this, and
1233arrange for it to call @code{yyparse} or the parser will never run.
1234@xref{Interface, ,Parser C-Language Interface}.
1235
f7ab6a50 1236Aside from the token type names and the symbols in the actions you
7093d0f5 1237write, all symbols defined in the Bison parser file itself
bfa74976
RS
1238begin with @samp{yy} or @samp{YY}. This includes interface functions
1239such as the lexical analyzer function @code{yylex}, the error reporting
1240function @code{yyerror} and the parser function @code{yyparse} itself.
1241This also includes numerous identifiers used for internal purposes.
1242Therefore, you should avoid using C identifiers starting with @samp{yy}
1243or @samp{YY} in the Bison grammar file except for the ones defined in
55289366
PE
1244this manual. Also, you should avoid using the C identifiers
1245@samp{malloc} and @samp{free} for anything other than their usual
1246meanings.
bfa74976 1247
7093d0f5
AD
1248In some cases the Bison parser file includes system headers, and in
1249those cases your code should respect the identifiers reserved by those
55289366 1250headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>}, @code{<malloc.h>},
7093d0f5 1251@code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
30757c8c
PE
1252declare memory allocators and related types. @code{<libintl.h>} is
1253included if message translation is in use
1254(@pxref{Internationalization}). Other system headers may
ec3bc396
AD
1255be included if you define @code{YYDEBUG} to a nonzero value
1256(@pxref{Tracing, ,Tracing Your Parser}).
7093d0f5 1257
342b8b6e 1258@node Stages
bfa74976
RS
1259@section Stages in Using Bison
1260@cindex stages in using Bison
1261@cindex using Bison
1262
1263The actual language-design process using Bison, from grammar specification
1264to a working compiler or interpreter, has these parts:
1265
1266@enumerate
1267@item
1268Formally specify the grammar in a form recognized by Bison
704a47c4
AD
1269(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1270in the language, describe the action that is to be taken when an
1271instance of that rule is recognized. The action is described by a
1272sequence of C statements.
bfa74976
RS
1273
1274@item
704a47c4
AD
1275Write a lexical analyzer to process input and pass tokens to the parser.
1276The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1277Lexical Analyzer Function @code{yylex}}). It could also be produced
1278using Lex, but the use of Lex is not discussed in this manual.
bfa74976
RS
1279
1280@item
1281Write a controlling function that calls the Bison-produced parser.
1282
1283@item
1284Write error-reporting routines.
1285@end enumerate
1286
1287To turn this source code as written into a runnable program, you
1288must follow these steps:
1289
1290@enumerate
1291@item
1292Run Bison on the grammar to produce the parser.
1293
1294@item
1295Compile the code output by Bison, as well as any other source files.
1296
1297@item
1298Link the object files to produce the finished product.
1299@end enumerate
1300
342b8b6e 1301@node Grammar Layout
bfa74976
RS
1302@section The Overall Layout of a Bison Grammar
1303@cindex grammar file
1304@cindex file format
1305@cindex format of grammar file
1306@cindex layout of Bison grammar
1307
1308The input file for the Bison utility is a @dfn{Bison grammar file}. The
1309general form of a Bison grammar file is as follows:
1310
1311@example
1312%@{
08e49d20 1313@var{Prologue}
bfa74976
RS
1314%@}
1315
1316@var{Bison declarations}
1317
1318%%
1319@var{Grammar rules}
1320%%
08e49d20 1321@var{Epilogue}
bfa74976
RS
1322@end example
1323
1324@noindent
1325The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1326in every Bison grammar file to separate the sections.
1327
72d2299c 1328The prologue may define types and variables used in the actions. You can
342b8b6e 1329also use preprocessor commands to define macros used there, and use
bfa74976 1330@code{#include} to include header files that do any of these things.
38a92d50
PE
1331You need to declare the lexical analyzer @code{yylex} and the error
1332printer @code{yyerror} here, along with any other global identifiers
1333used by the actions in the grammar rules.
bfa74976
RS
1334
1335The Bison declarations declare the names of the terminal and nonterminal
1336symbols, and may also describe operator precedence and the data types of
1337semantic values of various symbols.
1338
1339The grammar rules define how to construct each nonterminal symbol from its
1340parts.
1341
38a92d50
PE
1342The epilogue can contain any code you want to use. Often the
1343definitions of functions declared in the prologue go here. In a
1344simple program, all the rest of the program can go here.
bfa74976 1345
342b8b6e 1346@node Examples
bfa74976
RS
1347@chapter Examples
1348@cindex simple examples
1349@cindex examples, simple
1350
1351Now we show and explain three sample programs written using Bison: a
1352reverse polish notation calculator, an algebraic (infix) notation
1353calculator, and a multi-function calculator. All three have been tested
1354under BSD Unix 4.3; each produces a usable, though limited, interactive
1355desk-top calculator.
1356
1357These examples are simple, but Bison grammars for real programming
aa08666d
AD
1358languages are written the same way. You can copy these examples into a
1359source file to try them.
bfa74976
RS
1360
1361@menu
1362* RPN Calc:: Reverse polish notation calculator;
1363 a first example with no operator precedence.
1364* Infix Calc:: Infix (algebraic) notation calculator.
1365 Operator precedence is introduced.
1366* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 1367* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
bfa74976
RS
1368* Multi-function Calc:: Calculator with memory and trig functions.
1369 It uses multiple data-types for semantic values.
1370* Exercises:: Ideas for improving the multi-function calculator.
1371@end menu
1372
342b8b6e 1373@node RPN Calc
bfa74976
RS
1374@section Reverse Polish Notation Calculator
1375@cindex reverse polish notation
1376@cindex polish notation calculator
1377@cindex @code{rpcalc}
1378@cindex calculator, simple
1379
1380The first example is that of a simple double-precision @dfn{reverse polish
1381notation} calculator (a calculator using postfix operators). This example
1382provides a good starting point, since operator precedence is not an issue.
1383The second example will illustrate how operator precedence is handled.
1384
1385The source code for this calculator is named @file{rpcalc.y}. The
1386@samp{.y} extension is a convention used for Bison input files.
1387
1388@menu
75f5aaea 1389* Decls: Rpcalc Decls. Prologue (declarations) for rpcalc.
bfa74976
RS
1390* Rules: Rpcalc Rules. Grammar Rules for rpcalc, with explanation.
1391* Lexer: Rpcalc Lexer. The lexical analyzer.
1392* Main: Rpcalc Main. The controlling function.
1393* Error: Rpcalc Error. The error reporting function.
1394* Gen: Rpcalc Gen. Running Bison on the grammar file.
1395* Comp: Rpcalc Compile. Run the C compiler on the output code.
1396@end menu
1397
342b8b6e 1398@node Rpcalc Decls
bfa74976
RS
1399@subsection Declarations for @code{rpcalc}
1400
1401Here are the C and Bison declarations for the reverse polish notation
1402calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1403
1404@example
72d2299c 1405/* Reverse polish notation calculator. */
bfa74976
RS
1406
1407%@{
38a92d50
PE
1408 #define YYSTYPE double
1409 #include <math.h>
1410 int yylex (void);
1411 void yyerror (char const *);
bfa74976
RS
1412%@}
1413
1414%token NUM
1415
72d2299c 1416%% /* Grammar rules and actions follow. */
bfa74976
RS
1417@end example
1418
75f5aaea 1419The declarations section (@pxref{Prologue, , The prologue}) contains two
38a92d50 1420preprocessor directives and two forward declarations.
bfa74976
RS
1421
1422The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1964ad8c
AD
1423specifying the C data type for semantic values of both tokens and
1424groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1425Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1426don't define it, @code{int} is the default. Because we specify
1427@code{double}, each token and each expression has an associated value,
1428which is a floating point number.
bfa74976
RS
1429
1430The @code{#include} directive is used to declare the exponentiation
1431function @code{pow}.
1432
38a92d50
PE
1433The forward declarations for @code{yylex} and @code{yyerror} are
1434needed because the C language requires that functions be declared
1435before they are used. These functions will be defined in the
1436epilogue, but the parser calls them so they must be declared in the
1437prologue.
1438
704a47c4
AD
1439The second section, Bison declarations, provides information to Bison
1440about the token types (@pxref{Bison Declarations, ,The Bison
1441Declarations Section}). Each terminal symbol that is not a
1442single-character literal must be declared here. (Single-character
bfa74976
RS
1443literals normally don't need to be declared.) In this example, all the
1444arithmetic operators are designated by single-character literals, so the
1445only terminal symbol that needs to be declared is @code{NUM}, the token
1446type for numeric constants.
1447
342b8b6e 1448@node Rpcalc Rules
bfa74976
RS
1449@subsection Grammar Rules for @code{rpcalc}
1450
1451Here are the grammar rules for the reverse polish notation calculator.
1452
1453@example
1454input: /* empty */
1455 | input line
1456;
1457
1458line: '\n'
18b519c0 1459 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
bfa74976
RS
1460;
1461
18b519c0
AD
1462exp: NUM @{ $$ = $1; @}
1463 | exp exp '+' @{ $$ = $1 + $2; @}
1464 | exp exp '-' @{ $$ = $1 - $2; @}
1465 | exp exp '*' @{ $$ = $1 * $2; @}
1466 | exp exp '/' @{ $$ = $1 / $2; @}
1467 /* Exponentiation */
1468 | exp exp '^' @{ $$ = pow ($1, $2); @}
1469 /* Unary minus */
1470 | exp 'n' @{ $$ = -$1; @}
bfa74976
RS
1471;
1472%%
1473@end example
1474
1475The groupings of the rpcalc ``language'' defined here are the expression
1476(given the name @code{exp}), the line of input (@code{line}), and the
1477complete input transcript (@code{input}). Each of these nonterminal
8c5b881d 1478symbols has several alternate rules, joined by the vertical bar @samp{|}
bfa74976
RS
1479which is read as ``or''. The following sections explain what these rules
1480mean.
1481
1482The semantics of the language is determined by the actions taken when a
1483grouping is recognized. The actions are the C code that appears inside
1484braces. @xref{Actions}.
1485
1486You must specify these actions in C, but Bison provides the means for
1487passing semantic values between the rules. In each action, the
1488pseudo-variable @code{$$} stands for the semantic value for the grouping
1489that the rule is going to construct. Assigning a value to @code{$$} is the
1490main job of most actions. The semantic values of the components of the
1491rule are referred to as @code{$1}, @code{$2}, and so on.
1492
1493@menu
13863333
AD
1494* Rpcalc Input::
1495* Rpcalc Line::
1496* Rpcalc Expr::
bfa74976
RS
1497@end menu
1498
342b8b6e 1499@node Rpcalc Input
bfa74976
RS
1500@subsubsection Explanation of @code{input}
1501
1502Consider the definition of @code{input}:
1503
1504@example
1505input: /* empty */
1506 | input line
1507;
1508@end example
1509
1510This definition reads as follows: ``A complete input is either an empty
1511string, or a complete input followed by an input line''. Notice that
1512``complete input'' is defined in terms of itself. This definition is said
1513to be @dfn{left recursive} since @code{input} appears always as the
1514leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1515
1516The first alternative is empty because there are no symbols between the
1517colon and the first @samp{|}; this means that @code{input} can match an
1518empty string of input (no tokens). We write the rules this way because it
1519is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1520It's conventional to put an empty alternative first and write the comment
1521@samp{/* empty */} in it.
1522
1523The second alternate rule (@code{input line}) handles all nontrivial input.
1524It means, ``After reading any number of lines, read one more line if
1525possible.'' The left recursion makes this rule into a loop. Since the
1526first alternative matches empty input, the loop can be executed zero or
1527more times.
1528
1529The parser function @code{yyparse} continues to process input until a
1530grammatical error is seen or the lexical analyzer says there are no more
72d2299c 1531input tokens; we will arrange for the latter to happen at end-of-input.
bfa74976 1532
342b8b6e 1533@node Rpcalc Line
bfa74976
RS
1534@subsubsection Explanation of @code{line}
1535
1536Now consider the definition of @code{line}:
1537
1538@example
1539line: '\n'
1540 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1541;
1542@end example
1543
1544The first alternative is a token which is a newline character; this means
1545that rpcalc accepts a blank line (and ignores it, since there is no
1546action). The second alternative is an expression followed by a newline.
1547This is the alternative that makes rpcalc useful. The semantic value of
1548the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1549question is the first symbol in the alternative. The action prints this
1550value, which is the result of the computation the user asked for.
1551
1552This action is unusual because it does not assign a value to @code{$$}. As
1553a consequence, the semantic value associated with the @code{line} is
1554uninitialized (its value will be unpredictable). This would be a bug if
1555that value were ever used, but we don't use it: once rpcalc has printed the
1556value of the user's input line, that value is no longer needed.
1557
342b8b6e 1558@node Rpcalc Expr
bfa74976
RS
1559@subsubsection Explanation of @code{expr}
1560
1561The @code{exp} grouping has several rules, one for each kind of expression.
1562The first rule handles the simplest expressions: those that are just numbers.
1563The second handles an addition-expression, which looks like two expressions
1564followed by a plus-sign. The third handles subtraction, and so on.
1565
1566@example
1567exp: NUM
1568 | exp exp '+' @{ $$ = $1 + $2; @}
1569 | exp exp '-' @{ $$ = $1 - $2; @}
1570 @dots{}
1571 ;
1572@end example
1573
1574We have used @samp{|} to join all the rules for @code{exp}, but we could
1575equally well have written them separately:
1576
1577@example
1578exp: NUM ;
1579exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1580exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1581 @dots{}
1582@end example
1583
1584Most of the rules have actions that compute the value of the expression in
1585terms of the value of its parts. For example, in the rule for addition,
1586@code{$1} refers to the first component @code{exp} and @code{$2} refers to
1587the second one. The third component, @code{'+'}, has no meaningful
1588associated semantic value, but if it had one you could refer to it as
1589@code{$3}. When @code{yyparse} recognizes a sum expression using this
1590rule, the sum of the two subexpressions' values is produced as the value of
1591the entire expression. @xref{Actions}.
1592
1593You don't have to give an action for every rule. When a rule has no
1594action, Bison by default copies the value of @code{$1} into @code{$$}.
1595This is what happens in the first rule (the one that uses @code{NUM}).
1596
1597The formatting shown here is the recommended convention, but Bison does
72d2299c 1598not require it. You can add or change white space as much as you wish.
bfa74976
RS
1599For example, this:
1600
1601@example
99a9344e 1602exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
bfa74976
RS
1603@end example
1604
1605@noindent
1606means the same thing as this:
1607
1608@example
1609exp: NUM
1610 | exp exp '+' @{ $$ = $1 + $2; @}
1611 | @dots{}
99a9344e 1612;
bfa74976
RS
1613@end example
1614
1615@noindent
1616The latter, however, is much more readable.
1617
342b8b6e 1618@node Rpcalc Lexer
bfa74976
RS
1619@subsection The @code{rpcalc} Lexical Analyzer
1620@cindex writing a lexical analyzer
1621@cindex lexical analyzer, writing
1622
704a47c4
AD
1623The lexical analyzer's job is low-level parsing: converting characters
1624or sequences of characters into tokens. The Bison parser gets its
1625tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1626Analyzer Function @code{yylex}}.
bfa74976 1627
c827f760
PE
1628Only a simple lexical analyzer is needed for the @acronym{RPN}
1629calculator. This
bfa74976
RS
1630lexical analyzer skips blanks and tabs, then reads in numbers as
1631@code{double} and returns them as @code{NUM} tokens. Any other character
1632that isn't part of a number is a separate token. Note that the token-code
1633for such a single-character token is the character itself.
1634
1635The return value of the lexical analyzer function is a numeric code which
1636represents a token type. The same text used in Bison rules to stand for
1637this token type is also a C expression for the numeric code for the type.
1638This works in two ways. If the token type is a character literal, then its
e966383b 1639numeric code is that of the character; you can use the same
bfa74976
RS
1640character literal in the lexical analyzer to express the number. If the
1641token type is an identifier, that identifier is defined by Bison as a C
1642macro whose definition is the appropriate number. In this example,
1643therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1644
1964ad8c
AD
1645The semantic value of the token (if it has one) is stored into the
1646global variable @code{yylval}, which is where the Bison parser will look
1647for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1648defined at the beginning of the grammar; @pxref{Rpcalc Decls,
1649,Declarations for @code{rpcalc}}.)
bfa74976 1650
72d2299c
PE
1651A token type code of zero is returned if the end-of-input is encountered.
1652(Bison recognizes any nonpositive value as indicating end-of-input.)
bfa74976
RS
1653
1654Here is the code for the lexical analyzer:
1655
1656@example
1657@group
72d2299c 1658/* The lexical analyzer returns a double floating point
e966383b 1659 number on the stack and the token NUM, or the numeric code
72d2299c
PE
1660 of the character read if not a number. It skips all blanks
1661 and tabs, and returns 0 for end-of-input. */
bfa74976
RS
1662
1663#include <ctype.h>
1664@end group
1665
1666@group
13863333
AD
1667int
1668yylex (void)
bfa74976
RS
1669@{
1670 int c;
1671
72d2299c 1672 /* Skip white space. */
13863333 1673 while ((c = getchar ()) == ' ' || c == '\t')
bfa74976
RS
1674 ;
1675@end group
1676@group
72d2299c 1677 /* Process numbers. */
13863333 1678 if (c == '.' || isdigit (c))
bfa74976
RS
1679 @{
1680 ungetc (c, stdin);
1681 scanf ("%lf", &yylval);
1682 return NUM;
1683 @}
1684@end group
1685@group
72d2299c 1686 /* Return end-of-input. */
13863333 1687 if (c == EOF)
bfa74976 1688 return 0;
72d2299c 1689 /* Return a single char. */
13863333 1690 return c;
bfa74976
RS
1691@}
1692@end group
1693@end example
1694
342b8b6e 1695@node Rpcalc Main
bfa74976
RS
1696@subsection The Controlling Function
1697@cindex controlling function
1698@cindex main function in simple example
1699
1700In keeping with the spirit of this example, the controlling function is
1701kept to the bare minimum. The only requirement is that it call
1702@code{yyparse} to start the process of parsing.
1703
1704@example
1705@group
13863333
AD
1706int
1707main (void)
bfa74976 1708@{
13863333 1709 return yyparse ();
bfa74976
RS
1710@}
1711@end group
1712@end example
1713
342b8b6e 1714@node Rpcalc Error
bfa74976
RS
1715@subsection The Error Reporting Routine
1716@cindex error reporting routine
1717
1718When @code{yyparse} detects a syntax error, it calls the error reporting
13863333 1719function @code{yyerror} to print an error message (usually but not
6e649e65 1720always @code{"syntax error"}). It is up to the programmer to supply
13863333
AD
1721@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1722here is the definition we will use:
bfa74976
RS
1723
1724@example
1725@group
1726#include <stdio.h>
1727
38a92d50 1728/* Called by yyparse on error. */
13863333 1729void
38a92d50 1730yyerror (char const *s)
bfa74976 1731@{
4e03e201 1732 fprintf (stderr, "%s\n", s);
bfa74976
RS
1733@}
1734@end group
1735@end example
1736
1737After @code{yyerror} returns, the Bison parser may recover from the error
1738and continue parsing if the grammar contains a suitable error rule
1739(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1740have not written any error rules in this example, so any invalid input will
1741cause the calculator program to exit. This is not clean behavior for a
9ecbd125 1742real calculator, but it is adequate for the first example.
bfa74976 1743
342b8b6e 1744@node Rpcalc Gen
bfa74976
RS
1745@subsection Running Bison to Make the Parser
1746@cindex running Bison (introduction)
1747
ceed8467
AD
1748Before running Bison to produce a parser, we need to decide how to
1749arrange all the source code in one or more source files. For such a
1750simple example, the easiest thing is to put everything in one file. The
1751definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
342b8b6e 1752end, in the epilogue of the file
75f5aaea 1753(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
bfa74976
RS
1754
1755For a large project, you would probably have several source files, and use
1756@code{make} to arrange to recompile them.
1757
1758With all the source in a single file, you use the following command to
1759convert it into a parser file:
1760
1761@example
fa4d969f 1762bison @var{file}.y
bfa74976
RS
1763@end example
1764
1765@noindent
1766In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
fa4d969f 1767@sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c},
72d2299c 1768removing the @samp{.y} from the original file name. The file output by
bfa74976
RS
1769Bison contains the source code for @code{yyparse}. The additional
1770functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
1771are copied verbatim to the output.
1772
342b8b6e 1773@node Rpcalc Compile
bfa74976
RS
1774@subsection Compiling the Parser File
1775@cindex compiling the parser
1776
1777Here is how to compile and run the parser file:
1778
1779@example
1780@group
1781# @r{List files in current directory.}
9edcd895 1782$ @kbd{ls}
bfa74976
RS
1783rpcalc.tab.c rpcalc.y
1784@end group
1785
1786@group
1787# @r{Compile the Bison parser.}
1788# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
b56471a6 1789$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
bfa74976
RS
1790@end group
1791
1792@group
1793# @r{List files again.}
9edcd895 1794$ @kbd{ls}
bfa74976
RS
1795rpcalc rpcalc.tab.c rpcalc.y
1796@end group
1797@end example
1798
1799The file @file{rpcalc} now contains the executable code. Here is an
1800example session using @code{rpcalc}.
1801
1802@example
9edcd895
AD
1803$ @kbd{rpcalc}
1804@kbd{4 9 +}
bfa74976 180513
9edcd895 1806@kbd{3 7 + 3 4 5 *+-}
bfa74976 1807-13
9edcd895 1808@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
bfa74976 180913
9edcd895 1810@kbd{5 6 / 4 n +}
bfa74976 1811-3.166666667
9edcd895 1812@kbd{3 4 ^} @r{Exponentiation}
bfa74976 181381
9edcd895
AD
1814@kbd{^D} @r{End-of-file indicator}
1815$
bfa74976
RS
1816@end example
1817
342b8b6e 1818@node Infix Calc
bfa74976
RS
1819@section Infix Notation Calculator: @code{calc}
1820@cindex infix notation calculator
1821@cindex @code{calc}
1822@cindex calculator, infix notation
1823
1824We now modify rpcalc to handle infix operators instead of postfix. Infix
1825notation involves the concept of operator precedence and the need for
1826parentheses nested to arbitrary depth. Here is the Bison code for
1827@file{calc.y}, an infix desk-top calculator.
1828
1829@example
38a92d50 1830/* Infix notation calculator. */
bfa74976
RS
1831
1832%@{
38a92d50
PE
1833 #define YYSTYPE double
1834 #include <math.h>
1835 #include <stdio.h>
1836 int yylex (void);
1837 void yyerror (char const *);
bfa74976
RS
1838%@}
1839
38a92d50 1840/* Bison declarations. */
bfa74976
RS
1841%token NUM
1842%left '-' '+'
1843%left '*' '/'
1844%left NEG /* negation--unary minus */
38a92d50 1845%right '^' /* exponentiation */
bfa74976 1846
38a92d50
PE
1847%% /* The grammar follows. */
1848input: /* empty */
bfa74976
RS
1849 | input line
1850;
1851
1852line: '\n'
1853 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1854;
1855
1856exp: NUM @{ $$ = $1; @}
1857 | exp '+' exp @{ $$ = $1 + $3; @}
1858 | exp '-' exp @{ $$ = $1 - $3; @}
1859 | exp '*' exp @{ $$ = $1 * $3; @}
1860 | exp '/' exp @{ $$ = $1 / $3; @}
1861 | '-' exp %prec NEG @{ $$ = -$2; @}
1862 | exp '^' exp @{ $$ = pow ($1, $3); @}
1863 | '(' exp ')' @{ $$ = $2; @}
1864;
1865%%
1866@end example
1867
1868@noindent
ceed8467
AD
1869The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1870same as before.
bfa74976
RS
1871
1872There are two important new features shown in this code.
1873
1874In the second section (Bison declarations), @code{%left} declares token
1875types and says they are left-associative operators. The declarations
1876@code{%left} and @code{%right} (right associativity) take the place of
1877@code{%token} which is used to declare a token type name without
1878associativity. (These tokens are single-character literals, which
1879ordinarily don't need to be declared. We declare them here to specify
1880the associativity.)
1881
1882Operator precedence is determined by the line ordering of the
1883declarations; the higher the line number of the declaration (lower on
1884the page or screen), the higher the precedence. Hence, exponentiation
1885has the highest precedence, unary minus (@code{NEG}) is next, followed
704a47c4
AD
1886by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator
1887Precedence}.
bfa74976 1888
704a47c4
AD
1889The other important new feature is the @code{%prec} in the grammar
1890section for the unary minus operator. The @code{%prec} simply instructs
1891Bison that the rule @samp{| '-' exp} has the same precedence as
1892@code{NEG}---in this case the next-to-highest. @xref{Contextual
1893Precedence, ,Context-Dependent Precedence}.
bfa74976
RS
1894
1895Here is a sample run of @file{calc.y}:
1896
1897@need 500
1898@example
9edcd895
AD
1899$ @kbd{calc}
1900@kbd{4 + 4.5 - (34/(8*3+-3))}
bfa74976 19016.880952381
9edcd895 1902@kbd{-56 + 2}
bfa74976 1903-54
9edcd895 1904@kbd{3 ^ 2}
bfa74976
RS
19059
1906@end example
1907
342b8b6e 1908@node Simple Error Recovery
bfa74976
RS
1909@section Simple Error Recovery
1910@cindex error recovery, simple
1911
1912Up to this point, this manual has not addressed the issue of @dfn{error
1913recovery}---how to continue parsing after the parser detects a syntax
ceed8467
AD
1914error. All we have handled is error reporting with @code{yyerror}.
1915Recall that by default @code{yyparse} returns after calling
1916@code{yyerror}. This means that an erroneous input line causes the
1917calculator program to exit. Now we show how to rectify this deficiency.
bfa74976
RS
1918
1919The Bison language itself includes the reserved word @code{error}, which
1920may be included in the grammar rules. In the example below it has
1921been added to one of the alternatives for @code{line}:
1922
1923@example
1924@group
1925line: '\n'
1926 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1927 | error '\n' @{ yyerrok; @}
1928;
1929@end group
1930@end example
1931
ceed8467 1932This addition to the grammar allows for simple error recovery in the
6e649e65 1933event of a syntax error. If an expression that cannot be evaluated is
ceed8467
AD
1934read, the error will be recognized by the third rule for @code{line},
1935and parsing will continue. (The @code{yyerror} function is still called
1936upon to print its message as well.) The action executes the statement
1937@code{yyerrok}, a macro defined automatically by Bison; its meaning is
1938that error recovery is complete (@pxref{Error Recovery}). Note the
1939difference between @code{yyerrok} and @code{yyerror}; neither one is a
e0c471a9 1940misprint.
bfa74976
RS
1941
1942This form of error recovery deals with syntax errors. There are other
1943kinds of errors; for example, division by zero, which raises an exception
1944signal that is normally fatal. A real calculator program must handle this
1945signal and use @code{longjmp} to return to @code{main} and resume parsing
1946input lines; it would also have to discard the rest of the current line of
1947input. We won't discuss this issue further because it is not specific to
1948Bison programs.
1949
342b8b6e
AD
1950@node Location Tracking Calc
1951@section Location Tracking Calculator: @code{ltcalc}
1952@cindex location tracking calculator
1953@cindex @code{ltcalc}
1954@cindex calculator, location tracking
1955
9edcd895
AD
1956This example extends the infix notation calculator with location
1957tracking. This feature will be used to improve the error messages. For
1958the sake of clarity, this example is a simple integer calculator, since
1959most of the work needed to use locations will be done in the lexical
72d2299c 1960analyzer.
342b8b6e
AD
1961
1962@menu
1963* Decls: Ltcalc Decls. Bison and C declarations for ltcalc.
1964* Rules: Ltcalc Rules. Grammar rules for ltcalc, with explanations.
1965* Lexer: Ltcalc Lexer. The lexical analyzer.
1966@end menu
1967
1968@node Ltcalc Decls
1969@subsection Declarations for @code{ltcalc}
1970
9edcd895
AD
1971The C and Bison declarations for the location tracking calculator are
1972the same as the declarations for the infix notation calculator.
342b8b6e
AD
1973
1974@example
1975/* Location tracking calculator. */
1976
1977%@{
38a92d50
PE
1978 #define YYSTYPE int
1979 #include <math.h>
1980 int yylex (void);
1981 void yyerror (char const *);
342b8b6e
AD
1982%@}
1983
1984/* Bison declarations. */
1985%token NUM
1986
1987%left '-' '+'
1988%left '*' '/'
1989%left NEG
1990%right '^'
1991
38a92d50 1992%% /* The grammar follows. */
342b8b6e
AD
1993@end example
1994
9edcd895
AD
1995@noindent
1996Note there are no declarations specific to locations. Defining a data
1997type for storing locations is not needed: we will use the type provided
1998by default (@pxref{Location Type, ,Data Types of Locations}), which is a
1999four member structure with the following integer fields:
2000@code{first_line}, @code{first_column}, @code{last_line} and
2001@code{last_column}.
342b8b6e
AD
2002
2003@node Ltcalc Rules
2004@subsection Grammar Rules for @code{ltcalc}
2005
9edcd895
AD
2006Whether handling locations or not has no effect on the syntax of your
2007language. Therefore, grammar rules for this example will be very close
2008to those of the previous example: we will only modify them to benefit
2009from the new information.
342b8b6e 2010
9edcd895
AD
2011Here, we will use locations to report divisions by zero, and locate the
2012wrong expressions or subexpressions.
342b8b6e
AD
2013
2014@example
2015@group
2016input : /* empty */
2017 | input line
2018;
2019@end group
2020
2021@group
2022line : '\n'
2023 | exp '\n' @{ printf ("%d\n", $1); @}
2024;
2025@end group
2026
2027@group
2028exp : NUM @{ $$ = $1; @}
2029 | exp '+' exp @{ $$ = $1 + $3; @}
2030 | exp '-' exp @{ $$ = $1 - $3; @}
2031 | exp '*' exp @{ $$ = $1 * $3; @}
2032@end group
342b8b6e 2033@group
9edcd895 2034 | exp '/' exp
342b8b6e
AD
2035 @{
2036 if ($3)
2037 $$ = $1 / $3;
2038 else
2039 @{
2040 $$ = 1;
9edcd895
AD
2041 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2042 @@3.first_line, @@3.first_column,
2043 @@3.last_line, @@3.last_column);
342b8b6e
AD
2044 @}
2045 @}
2046@end group
2047@group
2048 | '-' exp %preg NEG @{ $$ = -$2; @}
2049 | exp '^' exp @{ $$ = pow ($1, $3); @}
2050 | '(' exp ')' @{ $$ = $2; @}
2051@end group
2052@end example
2053
2054This code shows how to reach locations inside of semantic actions, by
2055using the pseudo-variables @code{@@@var{n}} for rule components, and the
2056pseudo-variable @code{@@$} for groupings.
2057
9edcd895
AD
2058We don't need to assign a value to @code{@@$}: the output parser does it
2059automatically. By default, before executing the C code of each action,
2060@code{@@$} is set to range from the beginning of @code{@@1} to the end
2061of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2062can be redefined (@pxref{Location Default Action, , Default Action for
2063Locations}), and for very specific rules, @code{@@$} can be computed by
2064hand.
342b8b6e
AD
2065
2066@node Ltcalc Lexer
2067@subsection The @code{ltcalc} Lexical Analyzer.
2068
9edcd895 2069Until now, we relied on Bison's defaults to enable location
72d2299c 2070tracking. The next step is to rewrite the lexical analyzer, and make it
9edcd895
AD
2071able to feed the parser with the token locations, as it already does for
2072semantic values.
342b8b6e 2073
9edcd895
AD
2074To this end, we must take into account every single character of the
2075input text, to avoid the computed locations of being fuzzy or wrong:
342b8b6e
AD
2076
2077@example
2078@group
2079int
2080yylex (void)
2081@{
2082 int c;
18b519c0 2083@end group
342b8b6e 2084
18b519c0 2085@group
72d2299c 2086 /* Skip white space. */
342b8b6e
AD
2087 while ((c = getchar ()) == ' ' || c == '\t')
2088 ++yylloc.last_column;
18b519c0 2089@end group
342b8b6e 2090
18b519c0 2091@group
72d2299c 2092 /* Step. */
342b8b6e
AD
2093 yylloc.first_line = yylloc.last_line;
2094 yylloc.first_column = yylloc.last_column;
2095@end group
2096
2097@group
72d2299c 2098 /* Process numbers. */
342b8b6e
AD
2099 if (isdigit (c))
2100 @{
2101 yylval = c - '0';
2102 ++yylloc.last_column;
2103 while (isdigit (c = getchar ()))
2104 @{
2105 ++yylloc.last_column;
2106 yylval = yylval * 10 + c - '0';
2107 @}
2108 ungetc (c, stdin);
2109 return NUM;
2110 @}
2111@end group
2112
72d2299c 2113 /* Return end-of-input. */
342b8b6e
AD
2114 if (c == EOF)
2115 return 0;
2116
72d2299c 2117 /* Return a single char, and update location. */
342b8b6e
AD
2118 if (c == '\n')
2119 @{
2120 ++yylloc.last_line;
2121 yylloc.last_column = 0;
2122 @}
2123 else
2124 ++yylloc.last_column;
2125 return c;
2126@}
2127@end example
2128
9edcd895
AD
2129Basically, the lexical analyzer performs the same processing as before:
2130it skips blanks and tabs, and reads numbers or single-character tokens.
2131In addition, it updates @code{yylloc}, the global variable (of type
2132@code{YYLTYPE}) containing the token's location.
342b8b6e 2133
9edcd895 2134Now, each time this function returns a token, the parser has its number
72d2299c 2135as well as its semantic value, and its location in the text. The last
9edcd895
AD
2136needed change is to initialize @code{yylloc}, for example in the
2137controlling function:
342b8b6e
AD
2138
2139@example
9edcd895 2140@group
342b8b6e
AD
2141int
2142main (void)
2143@{
2144 yylloc.first_line = yylloc.last_line = 1;
2145 yylloc.first_column = yylloc.last_column = 0;
2146 return yyparse ();
2147@}
9edcd895 2148@end group
342b8b6e
AD
2149@end example
2150
9edcd895
AD
2151Remember that computing locations is not a matter of syntax. Every
2152character must be associated to a location update, whether it is in
2153valid input, in comments, in literal strings, and so on.
342b8b6e
AD
2154
2155@node Multi-function Calc
bfa74976
RS
2156@section Multi-Function Calculator: @code{mfcalc}
2157@cindex multi-function calculator
2158@cindex @code{mfcalc}
2159@cindex calculator, multi-function
2160
2161Now that the basics of Bison have been discussed, it is time to move on to
2162a more advanced problem. The above calculators provided only five
2163functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2164be nice to have a calculator that provides other mathematical functions such
2165as @code{sin}, @code{cos}, etc.
2166
2167It is easy to add new operators to the infix calculator as long as they are
2168only single-character literals. The lexical analyzer @code{yylex} passes
9d9b8b70 2169back all nonnumeric characters as tokens, so new grammar rules suffice for
bfa74976
RS
2170adding a new operator. But we want something more flexible: built-in
2171functions whose syntax has this form:
2172
2173@example
2174@var{function_name} (@var{argument})
2175@end example
2176
2177@noindent
2178At the same time, we will add memory to the calculator, by allowing you
2179to create named variables, store values in them, and use them later.
2180Here is a sample session with the multi-function calculator:
2181
2182@example
9edcd895
AD
2183$ @kbd{mfcalc}
2184@kbd{pi = 3.141592653589}
bfa74976 21853.1415926536
9edcd895 2186@kbd{sin(pi)}
bfa74976 21870.0000000000
9edcd895 2188@kbd{alpha = beta1 = 2.3}
bfa74976 21892.3000000000
9edcd895 2190@kbd{alpha}
bfa74976 21912.3000000000
9edcd895 2192@kbd{ln(alpha)}
bfa74976 21930.8329091229
9edcd895 2194@kbd{exp(ln(beta1))}
bfa74976 21952.3000000000
9edcd895 2196$
bfa74976
RS
2197@end example
2198
2199Note that multiple assignment and nested function calls are permitted.
2200
2201@menu
2202* Decl: Mfcalc Decl. Bison declarations for multi-function calculator.
2203* Rules: Mfcalc Rules. Grammar rules for the calculator.
2204* Symtab: Mfcalc Symtab. Symbol table management subroutines.
2205@end menu
2206
342b8b6e 2207@node Mfcalc Decl
bfa74976
RS
2208@subsection Declarations for @code{mfcalc}
2209
2210Here are the C and Bison declarations for the multi-function calculator.
2211
2212@smallexample
18b519c0 2213@group
bfa74976 2214%@{
38a92d50
PE
2215 #include <math.h> /* For math functions, cos(), sin(), etc. */
2216 #include "calc.h" /* Contains definition of `symrec'. */
2217 int yylex (void);
2218 void yyerror (char const *);
bfa74976 2219%@}
18b519c0
AD
2220@end group
2221@group
bfa74976 2222%union @{
38a92d50
PE
2223 double val; /* For returning numbers. */
2224 symrec *tptr; /* For returning symbol-table pointers. */
bfa74976 2225@}
18b519c0 2226@end group
38a92d50
PE
2227%token <val> NUM /* Simple double precision number. */
2228%token <tptr> VAR FNCT /* Variable and Function. */
bfa74976
RS
2229%type <val> exp
2230
18b519c0 2231@group
bfa74976
RS
2232%right '='
2233%left '-' '+'
2234%left '*' '/'
38a92d50
PE
2235%left NEG /* negation--unary minus */
2236%right '^' /* exponentiation */
18b519c0 2237@end group
38a92d50 2238%% /* The grammar follows. */
bfa74976
RS
2239@end smallexample
2240
2241The above grammar introduces only two new features of the Bison language.
2242These features allow semantic values to have various data types
2243(@pxref{Multiple Types, ,More Than One Value Type}).
2244
2245The @code{%union} declaration specifies the entire list of possible types;
2246this is instead of defining @code{YYSTYPE}. The allowable types are now
2247double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2248the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2249
2250Since values can now have various types, it is necessary to associate a
2251type with each grammar symbol whose semantic value is used. These symbols
2252are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2253declarations are augmented with information about their data type (placed
2254between angle brackets).
2255
704a47c4
AD
2256The Bison construct @code{%type} is used for declaring nonterminal
2257symbols, just as @code{%token} is used for declaring token types. We
2258have not used @code{%type} before because nonterminal symbols are
2259normally declared implicitly by the rules that define them. But
2260@code{exp} must be declared explicitly so we can specify its value type.
2261@xref{Type Decl, ,Nonterminal Symbols}.
bfa74976 2262
342b8b6e 2263@node Mfcalc Rules
bfa74976
RS
2264@subsection Grammar Rules for @code{mfcalc}
2265
2266Here are the grammar rules for the multi-function calculator.
2267Most of them are copied directly from @code{calc}; three rules,
2268those which mention @code{VAR} or @code{FNCT}, are new.
2269
2270@smallexample
18b519c0 2271@group
bfa74976
RS
2272input: /* empty */
2273 | input line
2274;
18b519c0 2275@end group
bfa74976 2276
18b519c0 2277@group
bfa74976
RS
2278line:
2279 '\n'
2280 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2281 | error '\n' @{ yyerrok; @}
2282;
18b519c0 2283@end group
bfa74976 2284
18b519c0 2285@group
bfa74976
RS
2286exp: NUM @{ $$ = $1; @}
2287 | VAR @{ $$ = $1->value.var; @}
2288 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2289 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2290 | exp '+' exp @{ $$ = $1 + $3; @}
2291 | exp '-' exp @{ $$ = $1 - $3; @}
2292 | exp '*' exp @{ $$ = $1 * $3; @}
2293 | exp '/' exp @{ $$ = $1 / $3; @}
2294 | '-' exp %prec NEG @{ $$ = -$2; @}
2295 | exp '^' exp @{ $$ = pow ($1, $3); @}
2296 | '(' exp ')' @{ $$ = $2; @}
2297;
18b519c0 2298@end group
38a92d50 2299/* End of grammar. */
bfa74976
RS
2300%%
2301@end smallexample
2302
342b8b6e 2303@node Mfcalc Symtab
bfa74976
RS
2304@subsection The @code{mfcalc} Symbol Table
2305@cindex symbol table example
2306
2307The multi-function calculator requires a symbol table to keep track of the
2308names and meanings of variables and functions. This doesn't affect the
2309grammar rules (except for the actions) or the Bison declarations, but it
2310requires some additional C functions for support.
2311
2312The symbol table itself consists of a linked list of records. Its
2313definition, which is kept in the header @file{calc.h}, is as follows. It
2314provides for either functions or variables to be placed in the table.
2315
2316@smallexample
2317@group
38a92d50 2318/* Function type. */
32dfccf8 2319typedef double (*func_t) (double);
72f889cc 2320@end group
32dfccf8 2321
72f889cc 2322@group
38a92d50 2323/* Data type for links in the chain of symbols. */
bfa74976
RS
2324struct symrec
2325@{
38a92d50 2326 char *name; /* name of symbol */
bfa74976 2327 int type; /* type of symbol: either VAR or FNCT */
32dfccf8
AD
2328 union
2329 @{
38a92d50
PE
2330 double var; /* value of a VAR */
2331 func_t fnctptr; /* value of a FNCT */
bfa74976 2332 @} value;
38a92d50 2333 struct symrec *next; /* link field */
bfa74976
RS
2334@};
2335@end group
2336
2337@group
2338typedef struct symrec symrec;
2339
38a92d50 2340/* The symbol table: a chain of `struct symrec'. */
bfa74976
RS
2341extern symrec *sym_table;
2342
a730d142 2343symrec *putsym (char const *, int);
38a92d50 2344symrec *getsym (char const *);
bfa74976
RS
2345@end group
2346@end smallexample
2347
2348The new version of @code{main} includes a call to @code{init_table}, a
2349function that initializes the symbol table. Here it is, and
2350@code{init_table} as well:
2351
2352@smallexample
bfa74976
RS
2353#include <stdio.h>
2354
18b519c0 2355@group
38a92d50 2356/* Called by yyparse on error. */
13863333 2357void
38a92d50 2358yyerror (char const *s)
bfa74976
RS
2359@{
2360 printf ("%s\n", s);
2361@}
18b519c0 2362@end group
bfa74976 2363
18b519c0 2364@group
bfa74976
RS
2365struct init
2366@{
38a92d50
PE
2367 char const *fname;
2368 double (*fnct) (double);
bfa74976
RS
2369@};
2370@end group
2371
2372@group
38a92d50 2373struct init const arith_fncts[] =
13863333 2374@{
32dfccf8
AD
2375 "sin", sin,
2376 "cos", cos,
13863333 2377 "atan", atan,
32dfccf8
AD
2378 "ln", log,
2379 "exp", exp,
13863333
AD
2380 "sqrt", sqrt,
2381 0, 0
2382@};
18b519c0 2383@end group
bfa74976 2384
18b519c0 2385@group
bfa74976 2386/* The symbol table: a chain of `struct symrec'. */
38a92d50 2387symrec *sym_table;
bfa74976
RS
2388@end group
2389
2390@group
72d2299c 2391/* Put arithmetic functions in table. */
13863333
AD
2392void
2393init_table (void)
bfa74976
RS
2394@{
2395 int i;
2396 symrec *ptr;
2397 for (i = 0; arith_fncts[i].fname != 0; i++)
2398 @{
2399 ptr = putsym (arith_fncts[i].fname, FNCT);
2400 ptr->value.fnctptr = arith_fncts[i].fnct;
2401 @}
2402@}
2403@end group
38a92d50
PE
2404
2405@group
2406int
2407main (void)
2408@{
2409 init_table ();
2410 return yyparse ();
2411@}
2412@end group
bfa74976
RS
2413@end smallexample
2414
2415By simply editing the initialization list and adding the necessary include
2416files, you can add additional functions to the calculator.
2417
2418Two important functions allow look-up and installation of symbols in the
2419symbol table. The function @code{putsym} is passed a name and the type
2420(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2421linked to the front of the list, and a pointer to the object is returned.
2422The function @code{getsym} is passed the name of the symbol to look up. If
2423found, a pointer to that symbol is returned; otherwise zero is returned.
2424
2425@smallexample
2426symrec *
38a92d50 2427putsym (char const *sym_name, int sym_type)
bfa74976
RS
2428@{
2429 symrec *ptr;
2430 ptr = (symrec *) malloc (sizeof (symrec));
2431 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2432 strcpy (ptr->name,sym_name);
2433 ptr->type = sym_type;
72d2299c 2434 ptr->value.var = 0; /* Set value to 0 even if fctn. */
bfa74976
RS
2435 ptr->next = (struct symrec *)sym_table;
2436 sym_table = ptr;
2437 return ptr;
2438@}
2439
2440symrec *
38a92d50 2441getsym (char const *sym_name)
bfa74976
RS
2442@{
2443 symrec *ptr;
2444 for (ptr = sym_table; ptr != (symrec *) 0;
2445 ptr = (symrec *)ptr->next)
2446 if (strcmp (ptr->name,sym_name) == 0)
2447 return ptr;
2448 return 0;
2449@}
2450@end smallexample
2451
2452The function @code{yylex} must now recognize variables, numeric values, and
2453the single-character arithmetic operators. Strings of alphanumeric
9d9b8b70 2454characters with a leading letter are recognized as either variables or
bfa74976
RS
2455functions depending on what the symbol table says about them.
2456
2457The string is passed to @code{getsym} for look up in the symbol table. If
2458the name appears in the table, a pointer to its location and its type
2459(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2460already in the table, then it is installed as a @code{VAR} using
2461@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
e0c471a9 2462returned to @code{yyparse}.
bfa74976
RS
2463
2464No change is needed in the handling of numeric values and arithmetic
2465operators in @code{yylex}.
2466
2467@smallexample
2468@group
2469#include <ctype.h>
18b519c0 2470@end group
13863333 2471
18b519c0 2472@group
13863333
AD
2473int
2474yylex (void)
bfa74976
RS
2475@{
2476 int c;
2477
72d2299c 2478 /* Ignore white space, get first nonwhite character. */
bfa74976
RS
2479 while ((c = getchar ()) == ' ' || c == '\t');
2480
2481 if (c == EOF)
2482 return 0;
2483@end group
2484
2485@group
2486 /* Char starts a number => parse the number. */
2487 if (c == '.' || isdigit (c))
2488 @{
2489 ungetc (c, stdin);
2490 scanf ("%lf", &yylval.val);
2491 return NUM;
2492 @}
2493@end group
2494
2495@group
2496 /* Char starts an identifier => read the name. */
2497 if (isalpha (c))
2498 @{
2499 symrec *s;
2500 static char *symbuf = 0;
2501 static int length = 0;
2502 int i;
2503@end group
2504
2505@group
2506 /* Initially make the buffer long enough
2507 for a 40-character symbol name. */
2508 if (length == 0)
2509 length = 40, symbuf = (char *)malloc (length + 1);
2510
2511 i = 0;
2512 do
2513@end group
2514@group
2515 @{
2516 /* If buffer is full, make it bigger. */
2517 if (i == length)
2518 @{
2519 length *= 2;
18b519c0 2520 symbuf = (char *) realloc (symbuf, length + 1);
bfa74976
RS
2521 @}
2522 /* Add this character to the buffer. */
2523 symbuf[i++] = c;
2524 /* Get another character. */
2525 c = getchar ();
2526 @}
2527@end group
2528@group
72d2299c 2529 while (isalnum (c));
bfa74976
RS
2530
2531 ungetc (c, stdin);
2532 symbuf[i] = '\0';
2533@end group
2534
2535@group
2536 s = getsym (symbuf);
2537 if (s == 0)
2538 s = putsym (symbuf, VAR);
2539 yylval.tptr = s;
2540 return s->type;
2541 @}
2542
2543 /* Any other character is a token by itself. */
2544 return c;
2545@}
2546@end group
2547@end smallexample
2548
72d2299c 2549This program is both powerful and flexible. You may easily add new
704a47c4
AD
2550functions, and it is a simple job to modify this code to install
2551predefined variables such as @code{pi} or @code{e} as well.
bfa74976 2552
342b8b6e 2553@node Exercises
bfa74976
RS
2554@section Exercises
2555@cindex exercises
2556
2557@enumerate
2558@item
2559Add some new functions from @file{math.h} to the initialization list.
2560
2561@item
2562Add another array that contains constants and their values. Then
2563modify @code{init_table} to add these constants to the symbol table.
2564It will be easiest to give the constants type @code{VAR}.
2565
2566@item
2567Make the program report an error if the user refers to an
2568uninitialized variable in any way except to store a value in it.
2569@end enumerate
2570
342b8b6e 2571@node Grammar File
bfa74976
RS
2572@chapter Bison Grammar Files
2573
2574Bison takes as input a context-free grammar specification and produces a
2575C-language function that recognizes correct instances of the grammar.
2576
2577The Bison grammar input file conventionally has a name ending in @samp{.y}.
234a3be3 2578@xref{Invocation, ,Invoking Bison}.
bfa74976
RS
2579
2580@menu
2581* Grammar Outline:: Overall layout of the grammar file.
2582* Symbols:: Terminal and nonterminal symbols.
2583* Rules:: How to write grammar rules.
2584* Recursion:: Writing recursive rules.
2585* Semantics:: Semantic values and actions.
847bf1f5 2586* Locations:: Locations and actions.
bfa74976
RS
2587* Declarations:: All kinds of Bison declarations are described here.
2588* Multiple Parsers:: Putting more than one Bison parser in one program.
2589@end menu
2590
342b8b6e 2591@node Grammar Outline
bfa74976
RS
2592@section Outline of a Bison Grammar
2593
2594A Bison grammar file has four main sections, shown here with the
2595appropriate delimiters:
2596
2597@example
2598%@{
38a92d50 2599 @var{Prologue}
bfa74976
RS
2600%@}
2601
2602@var{Bison declarations}
2603
2604%%
2605@var{Grammar rules}
2606%%
2607
75f5aaea 2608@var{Epilogue}
bfa74976
RS
2609@end example
2610
2611Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2bfc2e2a
PE
2612As a @acronym{GNU} extension, @samp{//} introduces a comment that
2613continues until end of line.
bfa74976
RS
2614
2615@menu
75f5aaea 2616* Prologue:: Syntax and usage of the prologue.
bfa74976
RS
2617* Bison Declarations:: Syntax and usage of the Bison declarations section.
2618* Grammar Rules:: Syntax and usage of the grammar rules section.
75f5aaea 2619* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
2620@end menu
2621
38a92d50 2622@node Prologue
75f5aaea
MA
2623@subsection The prologue
2624@cindex declarations section
2625@cindex Prologue
2626@cindex declarations
bfa74976 2627
f8e1c9e5
AD
2628The @var{Prologue} section contains macro definitions and declarations
2629of functions and variables that are used in the actions in the grammar
2630rules. These are copied to the beginning of the parser file so that
2631they precede the definition of @code{yyparse}. You can use
2632@samp{#include} to get the declarations from a header file. If you
2633don't need any C declarations, you may omit the @samp{%@{} and
2634@samp{%@}} delimiters that bracket this section.
bfa74976 2635
287c78f6
PE
2636The @var{Prologue} section is terminated by the the first occurrence
2637of @samp{%@}} that is outside a comment, a string literal, or a
2638character constant.
2639
c732d2c6
AD
2640You may have more than one @var{Prologue} section, intermixed with the
2641@var{Bison declarations}. This allows you to have C and Bison
2642declarations that refer to each other. For example, the @code{%union}
2643declaration may use types defined in a header file, and you may wish to
2644prototype functions that take arguments of type @code{YYSTYPE}. This
2645can be done with two @var{Prologue} blocks, one before and one after the
2646@code{%union} declaration.
2647
2648@smallexample
2649%@{
38a92d50
PE
2650 #include <stdio.h>
2651 #include "ptypes.h"
c732d2c6
AD
2652%@}
2653
2654%union @{
779e7ceb 2655 long int n;
c732d2c6
AD
2656 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2657@}
2658
2659%@{
38a92d50
PE
2660 static void print_token_value (FILE *, int, YYSTYPE);
2661 #define YYPRINT(F, N, L) print_token_value (F, N, L)
c732d2c6
AD
2662%@}
2663
2664@dots{}
2665@end smallexample
2666
342b8b6e 2667@node Bison Declarations
bfa74976
RS
2668@subsection The Bison Declarations Section
2669@cindex Bison declarations (introduction)
2670@cindex declarations, Bison (introduction)
2671
2672The @var{Bison declarations} section contains declarations that define
2673terminal and nonterminal symbols, specify precedence, and so on.
2674In some simple grammars you may not need any declarations.
2675@xref{Declarations, ,Bison Declarations}.
2676
342b8b6e 2677@node Grammar Rules
bfa74976
RS
2678@subsection The Grammar Rules Section
2679@cindex grammar rules section
2680@cindex rules section for grammar
2681
2682The @dfn{grammar rules} section contains one or more Bison grammar
2683rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
2684
2685There must always be at least one grammar rule, and the first
2686@samp{%%} (which precedes the grammar rules) may never be omitted even
2687if it is the first thing in the file.
2688
38a92d50 2689@node Epilogue
75f5aaea 2690@subsection The epilogue
bfa74976 2691@cindex additional C code section
75f5aaea 2692@cindex epilogue
bfa74976
RS
2693@cindex C code, section for additional
2694
08e49d20
PE
2695The @var{Epilogue} is copied verbatim to the end of the parser file, just as
2696the @var{Prologue} is copied to the beginning. This is the most convenient
342b8b6e
AD
2697place to put anything that you want to have in the parser file but which need
2698not come before the definition of @code{yyparse}. For example, the
38a92d50
PE
2699definitions of @code{yylex} and @code{yyerror} often go here. Because
2700C requires functions to be declared before being used, you often need
2701to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
e4f85c39 2702even if you define them in the Epilogue.
75f5aaea 2703@xref{Interface, ,Parser C-Language Interface}.
bfa74976
RS
2704
2705If the last section is empty, you may omit the @samp{%%} that separates it
2706from the grammar rules.
2707
f8e1c9e5
AD
2708The Bison parser itself contains many macros and identifiers whose names
2709start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
2710any such names (except those documented in this manual) in the epilogue
2711of the grammar file.
bfa74976 2712
342b8b6e 2713@node Symbols
bfa74976
RS
2714@section Symbols, Terminal and Nonterminal
2715@cindex nonterminal symbol
2716@cindex terminal symbol
2717@cindex token type
2718@cindex symbol
2719
2720@dfn{Symbols} in Bison grammars represent the grammatical classifications
2721of the language.
2722
2723A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
2724class of syntactically equivalent tokens. You use the symbol in grammar
2725rules to mean that a token in that class is allowed. The symbol is
2726represented in the Bison parser by a numeric code, and the @code{yylex}
f8e1c9e5
AD
2727function returns a token type code to indicate what kind of token has
2728been read. You don't need to know what the code value is; you can use
2729the symbol to stand for it.
bfa74976 2730
f8e1c9e5
AD
2731A @dfn{nonterminal symbol} stands for a class of syntactically
2732equivalent groupings. The symbol name is used in writing grammar rules.
2733By convention, it should be all lower case.
bfa74976
RS
2734
2735Symbol names can contain letters, digits (not at the beginning),
2736underscores and periods. Periods make sense only in nonterminals.
2737
931c7513 2738There are three ways of writing terminal symbols in the grammar:
bfa74976
RS
2739
2740@itemize @bullet
2741@item
2742A @dfn{named token type} is written with an identifier, like an
c827f760 2743identifier in C@. By convention, it should be all upper case. Each
bfa74976
RS
2744such name must be defined with a Bison declaration such as
2745@code{%token}. @xref{Token Decl, ,Token Type Names}.
2746
2747@item
2748@cindex character token
2749@cindex literal token
2750@cindex single-character literal
931c7513
RS
2751A @dfn{character token type} (or @dfn{literal character token}) is
2752written in the grammar using the same syntax used in C for character
2753constants; for example, @code{'+'} is a character token type. A
2754character token type doesn't need to be declared unless you need to
2755specify its semantic value data type (@pxref{Value Type, ,Data Types of
2756Semantic Values}), associativity, or precedence (@pxref{Precedence,
2757,Operator Precedence}).
bfa74976
RS
2758
2759By convention, a character token type is used only to represent a
2760token that consists of that particular character. Thus, the token
2761type @code{'+'} is used to represent the character @samp{+} as a
2762token. Nothing enforces this convention, but if you depart from it,
2763your program will confuse other readers.
2764
2765All the usual escape sequences used in character literals in C can be
2766used in Bison as well, but you must not use the null character as a
72d2299c
PE
2767character literal because its numeric code, zero, signifies
2768end-of-input (@pxref{Calling Convention, ,Calling Convention
2bfc2e2a
PE
2769for @code{yylex}}). Also, unlike standard C, trigraphs have no
2770special meaning in Bison character literals, nor is backslash-newline
2771allowed.
931c7513
RS
2772
2773@item
2774@cindex string token
2775@cindex literal string token
9ecbd125 2776@cindex multicharacter literal
931c7513
RS
2777A @dfn{literal string token} is written like a C string constant; for
2778example, @code{"<="} is a literal string token. A literal string token
2779doesn't need to be declared unless you need to specify its semantic
14ded682 2780value data type (@pxref{Value Type}), associativity, or precedence
931c7513
RS
2781(@pxref{Precedence}).
2782
2783You can associate the literal string token with a symbolic name as an
2784alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
2785Declarations}). If you don't do that, the lexical analyzer has to
2786retrieve the token number for the literal string token from the
2787@code{yytname} table (@pxref{Calling Convention}).
2788
c827f760 2789@strong{Warning}: literal string tokens do not work in Yacc.
931c7513
RS
2790
2791By convention, a literal string token is used only to represent a token
2792that consists of that particular string. Thus, you should use the token
2793type @code{"<="} to represent the string @samp{<=} as a token. Bison
9ecbd125 2794does not enforce this convention, but if you depart from it, people who
931c7513
RS
2795read your program will be confused.
2796
2797All the escape sequences used in string literals in C can be used in
92ac3705
PE
2798Bison as well, except that you must not use a null character within a
2799string literal. Also, unlike Standard C, trigraphs have no special
2bfc2e2a
PE
2800meaning in Bison string literals, nor is backslash-newline allowed. A
2801literal string token must contain two or more characters; for a token
2802containing just one character, use a character token (see above).
bfa74976
RS
2803@end itemize
2804
2805How you choose to write a terminal symbol has no effect on its
2806grammatical meaning. That depends only on where it appears in rules and
2807on when the parser function returns that symbol.
2808
72d2299c
PE
2809The value returned by @code{yylex} is always one of the terminal
2810symbols, except that a zero or negative value signifies end-of-input.
2811Whichever way you write the token type in the grammar rules, you write
2812it the same way in the definition of @code{yylex}. The numeric code
2813for a character token type is simply the positive numeric code of the
2814character, so @code{yylex} can use the identical value to generate the
2815requisite code, though you may need to convert it to @code{unsigned
2816char} to avoid sign-extension on hosts where @code{char} is signed.
2817Each named token type becomes a C macro in
bfa74976 2818the parser file, so @code{yylex} can use the name to stand for the code.
13863333 2819(This is why periods don't make sense in terminal symbols.)
bfa74976
RS
2820@xref{Calling Convention, ,Calling Convention for @code{yylex}}.
2821
2822If @code{yylex} is defined in a separate file, you need to arrange for the
2823token-type macro definitions to be available there. Use the @samp{-d}
2824option when you run Bison, so that it will write these macro definitions
2825into a separate header file @file{@var{name}.tab.h} which you can include
2826in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
2827
72d2299c 2828If you want to write a grammar that is portable to any Standard C
9d9b8b70 2829host, you must use only nonnull character tokens taken from the basic
c827f760 2830execution character set of Standard C@. This set consists of the ten
72d2299c
PE
2831digits, the 52 lower- and upper-case English letters, and the
2832characters in the following C-language string:
2833
2834@example
2835"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
2836@end example
2837
f8e1c9e5
AD
2838The @code{yylex} function and Bison must use a consistent character set
2839and encoding for character tokens. For example, if you run Bison in an
2840@acronym{ASCII} environment, but then compile and run the resulting
2841program in an environment that uses an incompatible character set like
2842@acronym{EBCDIC}, the resulting program may not work because the tables
2843generated by Bison will assume @acronym{ASCII} numeric values for
2844character tokens. It is standard practice for software distributions to
2845contain C source files that were generated by Bison in an
2846@acronym{ASCII} environment, so installers on platforms that are
2847incompatible with @acronym{ASCII} must rebuild those files before
2848compiling them.
e966383b 2849
bfa74976
RS
2850The symbol @code{error} is a terminal symbol reserved for error recovery
2851(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
23c5a174
AD
2852In particular, @code{yylex} should never return this value. The default
2853value of the error token is 256, unless you explicitly assigned 256 to
2854one of your tokens with a @code{%token} declaration.
bfa74976 2855
342b8b6e 2856@node Rules
bfa74976
RS
2857@section Syntax of Grammar Rules
2858@cindex rule syntax
2859@cindex grammar rule syntax
2860@cindex syntax of grammar rules
2861
2862A Bison grammar rule has the following general form:
2863
2864@example
e425e872 2865@group
bfa74976
RS
2866@var{result}: @var{components}@dots{}
2867 ;
e425e872 2868@end group
bfa74976
RS
2869@end example
2870
2871@noindent
9ecbd125 2872where @var{result} is the nonterminal symbol that this rule describes,
bfa74976 2873and @var{components} are various terminal and nonterminal symbols that
13863333 2874are put together by this rule (@pxref{Symbols}).
bfa74976
RS
2875
2876For example,
2877
2878@example
2879@group
2880exp: exp '+' exp
2881 ;
2882@end group
2883@end example
2884
2885@noindent
2886says that two groupings of type @code{exp}, with a @samp{+} token in between,
2887can be combined into a larger grouping of type @code{exp}.
2888
72d2299c
PE
2889White space in rules is significant only to separate symbols. You can add
2890extra white space as you wish.
bfa74976
RS
2891
2892Scattered among the components can be @var{actions} that determine
2893the semantics of the rule. An action looks like this:
2894
2895@example
2896@{@var{C statements}@}
2897@end example
2898
2899@noindent
287c78f6
PE
2900@cindex braced code
2901This is an example of @dfn{braced code}, that is, C code surrounded by
2902braces, much like a compound statement in C@. Braced code can contain
2903any sequence of C tokens, so long as its braces are balanced. Bison
2904does not check the braced code for correctness directly; it merely
2905copies the code to the output file, where the C compiler can check it.
2906
2907Within braced code, the balanced-brace count is not affected by braces
2908within comments, string literals, or character constants, but it is
2909affected by the C digraphs @samp{<%} and @samp{%>} that represent
2910braces. At the top level braced code must be terminated by @samp{@}}
2911and not by a digraph. Bison does not look for trigraphs, so if braced
2912code uses trigraphs you should ensure that they do not affect the
2913nesting of braces or the boundaries of comments, string literals, or
2914character constants.
2915
bfa74976
RS
2916Usually there is only one action and it follows the components.
2917@xref{Actions}.
2918
2919@findex |
2920Multiple rules for the same @var{result} can be written separately or can
2921be joined with the vertical-bar character @samp{|} as follows:
2922
bfa74976
RS
2923@example
2924@group
2925@var{result}: @var{rule1-components}@dots{}
2926 | @var{rule2-components}@dots{}
2927 @dots{}
2928 ;
2929@end group
2930@end example
bfa74976
RS
2931
2932@noindent
2933They are still considered distinct rules even when joined in this way.
2934
2935If @var{components} in a rule is empty, it means that @var{result} can
2936match the empty string. For example, here is how to define a
2937comma-separated sequence of zero or more @code{exp} groupings:
2938
2939@example
2940@group
2941expseq: /* empty */
2942 | expseq1
2943 ;
2944@end group
2945
2946@group
2947expseq1: exp
2948 | expseq1 ',' exp
2949 ;
2950@end group
2951@end example
2952
2953@noindent
2954It is customary to write a comment @samp{/* empty */} in each rule
2955with no components.
2956
342b8b6e 2957@node Recursion
bfa74976
RS
2958@section Recursive Rules
2959@cindex recursive rule
2960
f8e1c9e5
AD
2961A rule is called @dfn{recursive} when its @var{result} nonterminal
2962appears also on its right hand side. Nearly all Bison grammars need to
2963use recursion, because that is the only way to define a sequence of any
2964number of a particular thing. Consider this recursive definition of a
9ecbd125 2965comma-separated sequence of one or more expressions:
bfa74976
RS
2966
2967@example
2968@group
2969expseq1: exp
2970 | expseq1 ',' exp
2971 ;
2972@end group
2973@end example
2974
2975@cindex left recursion
2976@cindex right recursion
2977@noindent
2978Since the recursive use of @code{expseq1} is the leftmost symbol in the
2979right hand side, we call this @dfn{left recursion}. By contrast, here
2980the same construct is defined using @dfn{right recursion}:
2981
2982@example
2983@group
2984expseq1: exp
2985 | exp ',' expseq1
2986 ;
2987@end group
2988@end example
2989
2990@noindent
ec3bc396
AD
2991Any kind of sequence can be defined using either left recursion or right
2992recursion, but you should always use left recursion, because it can
2993parse a sequence of any number of elements with bounded stack space.
2994Right recursion uses up space on the Bison stack in proportion to the
2995number of elements in the sequence, because all the elements must be
2996shifted onto the stack before the rule can be applied even once.
2997@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
2998of this.
bfa74976
RS
2999
3000@cindex mutual recursion
3001@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3002rule does not appear directly on its right hand side, but does appear
3003in rules for other nonterminals which do appear on its right hand
13863333 3004side.
bfa74976
RS
3005
3006For example:
3007
3008@example
3009@group
3010expr: primary
3011 | primary '+' primary
3012 ;
3013@end group
3014
3015@group
3016primary: constant
3017 | '(' expr ')'
3018 ;
3019@end group
3020@end example
3021
3022@noindent
3023defines two mutually-recursive nonterminals, since each refers to the
3024other.
3025
342b8b6e 3026@node Semantics
bfa74976
RS
3027@section Defining Language Semantics
3028@cindex defining language semantics
13863333 3029@cindex language semantics, defining
bfa74976
RS
3030
3031The grammar rules for a language determine only the syntax. The semantics
3032are determined by the semantic values associated with various tokens and
3033groupings, and by the actions taken when various groupings are recognized.
3034
3035For example, the calculator calculates properly because the value
3036associated with each expression is the proper number; it adds properly
3037because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3038the numbers associated with @var{x} and @var{y}.
3039
3040@menu
3041* Value Type:: Specifying one data type for all semantic values.
3042* Multiple Types:: Specifying several alternative data types.
3043* Actions:: An action is the semantic definition of a grammar rule.
3044* Action Types:: Specifying data types for actions to operate on.
3045* Mid-Rule Actions:: Most actions go at the end of a rule.
3046 This says when, why and how to use the exceptional
3047 action in the middle of a rule.
3048@end menu
3049
342b8b6e 3050@node Value Type
bfa74976
RS
3051@subsection Data Types of Semantic Values
3052@cindex semantic value type
3053@cindex value type, semantic
3054@cindex data types of semantic values
3055@cindex default data type
3056
3057In a simple program it may be sufficient to use the same data type for
3058the semantic values of all language constructs. This was true in the
c827f760 3059@acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
1964ad8c 3060Notation Calculator}).
bfa74976
RS
3061
3062Bison's default is to use type @code{int} for all semantic values. To
3063specify some other type, define @code{YYSTYPE} as a macro, like this:
3064
3065@example
3066#define YYSTYPE double
3067@end example
3068
3069@noindent
50cce58e
PE
3070@code{YYSTYPE}'s replacement list should be a type name
3071that does not contain parentheses or square brackets.
342b8b6e 3072This macro definition must go in the prologue of the grammar file
75f5aaea 3073(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
bfa74976 3074
342b8b6e 3075@node Multiple Types
bfa74976
RS
3076@subsection More Than One Value Type
3077
3078In most programs, you will need different data types for different kinds
3079of tokens and groupings. For example, a numeric constant may need type
f8e1c9e5
AD
3080@code{int} or @code{long int}, while a string constant needs type
3081@code{char *}, and an identifier might need a pointer to an entry in the
3082symbol table.
bfa74976
RS
3083
3084To use more than one data type for semantic values in one parser, Bison
3085requires you to do two things:
3086
3087@itemize @bullet
3088@item
3089Specify the entire collection of possible data types, with the
704a47c4
AD
3090@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
3091Value Types}).
bfa74976
RS
3092
3093@item
14ded682
AD
3094Choose one of those types for each symbol (terminal or nonterminal) for
3095which semantic values are used. This is done for tokens with the
3096@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3097and for groupings with the @code{%type} Bison declaration (@pxref{Type
3098Decl, ,Nonterminal Symbols}).
bfa74976
RS
3099@end itemize
3100
342b8b6e 3101@node Actions
bfa74976
RS
3102@subsection Actions
3103@cindex action
3104@vindex $$
3105@vindex $@var{n}
3106
3107An action accompanies a syntactic rule and contains C code to be executed
3108each time an instance of that rule is recognized. The task of most actions
3109is to compute a semantic value for the grouping built by the rule from the
3110semantic values associated with tokens or smaller groupings.
3111
287c78f6
PE
3112An action consists of braced code containing C statements, and can be
3113placed at any position in the rule;
704a47c4
AD
3114it is executed at that position. Most rules have just one action at the
3115end of the rule, following all the components. Actions in the middle of
3116a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3117Actions, ,Actions in Mid-Rule}).
bfa74976
RS
3118
3119The C code in an action can refer to the semantic values of the components
3120matched by the rule with the construct @code{$@var{n}}, which stands for
3121the value of the @var{n}th component. The semantic value for the grouping
0cc3da3a
PE
3122being constructed is @code{$$}. Bison translates both of these
3123constructs into expressions of the appropriate type when it copies the
3124actions into the parser file. @code{$$} is translated to a modifiable
3125lvalue, so it can be assigned to.
bfa74976
RS
3126
3127Here is a typical example:
3128
3129@example
3130@group
3131exp: @dots{}
3132 | exp '+' exp
3133 @{ $$ = $1 + $3; @}
3134@end group
3135@end example
3136
3137@noindent
3138This rule constructs an @code{exp} from two smaller @code{exp} groupings
3139connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3140refer to the semantic values of the two component @code{exp} groupings,
3141which are the first and third symbols on the right hand side of the rule.
3142The sum is stored into @code{$$} so that it becomes the semantic value of
3143the addition-expression just recognized by the rule. If there were a
3144useful semantic value associated with the @samp{+} token, it could be
e0c471a9 3145referred to as @code{$2}.
bfa74976 3146
3ded9a63
AD
3147Note that the vertical-bar character @samp{|} is really a rule
3148separator, and actions are attached to a single rule. This is a
3149difference with tools like Flex, for which @samp{|} stands for either
3150``or'', or ``the same action as that of the next rule''. In the
3151following example, the action is triggered only when @samp{b} is found:
3152
3153@example
3154@group
3155a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3156@end group
3157@end example
3158
bfa74976
RS
3159@cindex default action
3160If you don't specify an action for a rule, Bison supplies a default:
72f889cc
AD
3161@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3162becomes the value of the whole rule. Of course, the default action is
3163valid only if the two data types match. There is no meaningful default
3164action for an empty rule; every empty rule must have an explicit action
3165unless the rule's value does not matter.
bfa74976
RS
3166
3167@code{$@var{n}} with @var{n} zero or negative is allowed for reference
3168to tokens and groupings on the stack @emph{before} those that match the
3169current rule. This is a very risky practice, and to use it reliably
3170you must be certain of the context in which the rule is applied. Here
3171is a case in which you can use this reliably:
3172
3173@example
3174@group
3175foo: expr bar '+' expr @{ @dots{} @}
3176 | expr bar '-' expr @{ @dots{} @}
3177 ;
3178@end group
3179
3180@group
3181bar: /* empty */
3182 @{ previous_expr = $0; @}
3183 ;
3184@end group
3185@end example
3186
3187As long as @code{bar} is used only in the fashion shown here, @code{$0}
3188always refers to the @code{expr} which precedes @code{bar} in the
3189definition of @code{foo}.
3190
32c29292
JD
3191@vindex yylval
3192It is also possible to access the semantic value of the look-ahead token, if
3193any, from a semantic action.
3194This semantic value is stored in @code{yylval}.
3195@xref{Action Features, ,Special Features for Use in Actions}.
3196
342b8b6e 3197@node Action Types
bfa74976
RS
3198@subsection Data Types of Values in Actions
3199@cindex action data types
3200@cindex data types in actions
3201
3202If you have chosen a single data type for semantic values, the @code{$$}
3203and @code{$@var{n}} constructs always have that data type.
3204
3205If you have used @code{%union} to specify a variety of data types, then you
3206must declare a choice among these types for each terminal or nonterminal
3207symbol that can have a semantic value. Then each time you use @code{$$} or
3208@code{$@var{n}}, its data type is determined by which symbol it refers to
e0c471a9 3209in the rule. In this example,
bfa74976
RS
3210
3211@example
3212@group
3213exp: @dots{}
3214 | exp '+' exp
3215 @{ $$ = $1 + $3; @}
3216@end group
3217@end example
3218
3219@noindent
3220@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3221have the data type declared for the nonterminal symbol @code{exp}. If
3222@code{$2} were used, it would have the data type declared for the
e0c471a9 3223terminal symbol @code{'+'}, whatever that might be.
bfa74976
RS
3224
3225Alternatively, you can specify the data type when you refer to the value,
3226by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3227reference. For example, if you have defined types as shown here:
3228
3229@example
3230@group
3231%union @{
3232 int itype;
3233 double dtype;
3234@}
3235@end group
3236@end example
3237
3238@noindent
3239then you can write @code{$<itype>1} to refer to the first subunit of the
3240rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3241
342b8b6e 3242@node Mid-Rule Actions
bfa74976
RS
3243@subsection Actions in Mid-Rule
3244@cindex actions in mid-rule
3245@cindex mid-rule actions
3246
3247Occasionally it is useful to put an action in the middle of a rule.
3248These actions are written just like usual end-of-rule actions, but they
3249are executed before the parser even recognizes the following components.
3250
3251A mid-rule action may refer to the components preceding it using
3252@code{$@var{n}}, but it may not refer to subsequent components because
3253it is run before they are parsed.
3254
3255The mid-rule action itself counts as one of the components of the rule.
3256This makes a difference when there is another action later in the same rule
3257(and usually there is another at the end): you have to count the actions
3258along with the symbols when working out which number @var{n} to use in
3259@code{$@var{n}}.
3260
3261The mid-rule action can also have a semantic value. The action can set
3262its value with an assignment to @code{$$}, and actions later in the rule
3263can refer to the value using @code{$@var{n}}. Since there is no symbol
3264to name the action, there is no way to declare a data type for the value
fdc6758b
MA
3265in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3266specify a data type each time you refer to this value.
bfa74976
RS
3267
3268There is no way to set the value of the entire rule with a mid-rule
3269action, because assignments to @code{$$} do not have that effect. The
3270only way to set the value for the entire rule is with an ordinary action
3271at the end of the rule.
3272
3273Here is an example from a hypothetical compiler, handling a @code{let}
3274statement that looks like @samp{let (@var{variable}) @var{statement}} and
3275serves to create a variable named @var{variable} temporarily for the
3276duration of @var{statement}. To parse this construct, we must put
3277@var{variable} into the symbol table while @var{statement} is parsed, then
3278remove it afterward. Here is how it is done:
3279
3280@example
3281@group
3282stmt: LET '(' var ')'
3283 @{ $<context>$ = push_context ();
3284 declare_variable ($3); @}
3285 stmt @{ $$ = $6;
3286 pop_context ($<context>5); @}
3287@end group
3288@end example
3289
3290@noindent
3291As soon as @samp{let (@var{variable})} has been recognized, the first
3292action is run. It saves a copy of the current semantic context (the
3293list of accessible variables) as its semantic value, using alternative
3294@code{context} in the data-type union. Then it calls
3295@code{declare_variable} to add the new variable to that list. Once the
3296first action is finished, the embedded statement @code{stmt} can be
3297parsed. Note that the mid-rule action is component number 5, so the
3298@samp{stmt} is component number 6.
3299
3300After the embedded statement is parsed, its semantic value becomes the
3301value of the entire @code{let}-statement. Then the semantic value from the
3302earlier action is used to restore the prior list of variables. This
3303removes the temporary @code{let}-variable from the list so that it won't
3304appear to exist while the rest of the program is parsed.
3305
841a7737
JD
3306@findex %destructor
3307@cindex discarded symbols, mid-rule actions
3308@cindex error recovery, mid-rule actions
3309In the above example, if the parser initiates error recovery (@pxref{Error
3310Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3311it might discard the previous semantic context @code{$<context>5} without
3312restoring it.
3313Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3314Discarded Symbols}).
3315However, Bison currently provides no means to declare a destructor for a
3316mid-rule action's semantic value.
3317
3318One solution is to bury the mid-rule action inside a nonterminal symbol and to
3319declare a destructor for that symbol:
3320
3321@example
3322@group
3323%type <context> let
3324%destructor @{ pop_context ($$); @} let
3325
3326%%
3327
3328stmt: let stmt
3329 @{ $$ = $2;
3330 pop_context ($1); @}
3331 ;
3332
3333let: LET '(' var ')'
3334 @{ $$ = push_context ();
3335 declare_variable ($3); @}
3336 ;
3337
3338@end group
3339@end example
3340
3341@noindent
3342Note that the action is now at the end of its rule.
3343Any mid-rule action can be converted to an end-of-rule action in this way, and
3344this is what Bison actually does to implement mid-rule actions.
3345
bfa74976
RS
3346Taking action before a rule is completely recognized often leads to
3347conflicts since the parser must commit to a parse in order to execute the
3348action. For example, the following two rules, without mid-rule actions,
3349can coexist in a working parser because the parser can shift the open-brace
3350token and look at what follows before deciding whether there is a
3351declaration or not:
3352
3353@example
3354@group
3355compound: '@{' declarations statements '@}'
3356 | '@{' statements '@}'
3357 ;
3358@end group
3359@end example
3360
3361@noindent
3362But when we add a mid-rule action as follows, the rules become nonfunctional:
3363
3364@example
3365@group
3366compound: @{ prepare_for_local_variables (); @}
3367 '@{' declarations statements '@}'
3368@end group
3369@group
3370 | '@{' statements '@}'
3371 ;
3372@end group
3373@end example
3374
3375@noindent
3376Now the parser is forced to decide whether to run the mid-rule action
3377when it has read no farther than the open-brace. In other words, it
3378must commit to using one rule or the other, without sufficient
3379information to do it correctly. (The open-brace token is what is called
3380the @dfn{look-ahead} token at this time, since the parser is still
3381deciding what to do about it. @xref{Look-Ahead, ,Look-Ahead Tokens}.)
3382
3383You might think that you could correct the problem by putting identical
3384actions into the two rules, like this:
3385
3386@example
3387@group
3388compound: @{ prepare_for_local_variables (); @}
3389 '@{' declarations statements '@}'
3390 | @{ prepare_for_local_variables (); @}
3391 '@{' statements '@}'
3392 ;
3393@end group
3394@end example
3395
3396@noindent
3397But this does not help, because Bison does not realize that the two actions
3398are identical. (Bison never tries to understand the C code in an action.)
3399
3400If the grammar is such that a declaration can be distinguished from a
3401statement by the first token (which is true in C), then one solution which
3402does work is to put the action after the open-brace, like this:
3403
3404@example
3405@group
3406compound: '@{' @{ prepare_for_local_variables (); @}
3407 declarations statements '@}'
3408 | '@{' statements '@}'
3409 ;
3410@end group
3411@end example
3412
3413@noindent
3414Now the first token of the following declaration or statement,
3415which would in any case tell Bison which rule to use, can still do so.
3416
3417Another solution is to bury the action inside a nonterminal symbol which
3418serves as a subroutine:
3419
3420@example
3421@group
3422subroutine: /* empty */
3423 @{ prepare_for_local_variables (); @}
3424 ;
3425
3426@end group
3427
3428@group
3429compound: subroutine
3430 '@{' declarations statements '@}'
3431 | subroutine
3432 '@{' statements '@}'
3433 ;
3434@end group
3435@end example
3436
3437@noindent
3438Now Bison can execute the action in the rule for @code{subroutine} without
841a7737 3439deciding which rule for @code{compound} it will eventually use.
bfa74976 3440
342b8b6e 3441@node Locations
847bf1f5
AD
3442@section Tracking Locations
3443@cindex location
95923bd6
AD
3444@cindex textual location
3445@cindex location, textual
847bf1f5
AD
3446
3447Though grammar rules and semantic actions are enough to write a fully
72d2299c 3448functional parser, it can be useful to process some additional information,
3e259915
MA
3449especially symbol locations.
3450
704a47c4
AD
3451The way locations are handled is defined by providing a data type, and
3452actions to take when rules are matched.
847bf1f5
AD
3453
3454@menu
3455* Location Type:: Specifying a data type for locations.
3456* Actions and Locations:: Using locations in actions.
3457* Location Default Action:: Defining a general way to compute locations.
3458@end menu
3459
342b8b6e 3460@node Location Type
847bf1f5
AD
3461@subsection Data Type of Locations
3462@cindex data type of locations
3463@cindex default location type
3464
3465Defining a data type for locations is much simpler than for semantic values,
3466since all tokens and groupings always use the same type.
3467
50cce58e
PE
3468You can specify the type of locations by defining a macro called
3469@code{YYLTYPE}, just as you can specify the semantic value type by
3470defining @code{YYSTYPE} (@pxref{Value Type}).
847bf1f5
AD
3471When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3472four members:
3473
3474@example
6273355b 3475typedef struct YYLTYPE
847bf1f5
AD
3476@{
3477 int first_line;
3478 int first_column;
3479 int last_line;
3480 int last_column;
6273355b 3481@} YYLTYPE;
847bf1f5
AD
3482@end example
3483
342b8b6e 3484@node Actions and Locations
847bf1f5
AD
3485@subsection Actions and Locations
3486@cindex location actions
3487@cindex actions, location
3488@vindex @@$
3489@vindex @@@var{n}
3490
3491Actions are not only useful for defining language semantics, but also for
3492describing the behavior of the output parser with locations.
3493
3494The most obvious way for building locations of syntactic groupings is very
72d2299c 3495similar to the way semantic values are computed. In a given rule, several
847bf1f5
AD
3496constructs can be used to access the locations of the elements being matched.
3497The location of the @var{n}th component of the right hand side is
3498@code{@@@var{n}}, while the location of the left hand side grouping is
3499@code{@@$}.
3500
3e259915 3501Here is a basic example using the default data type for locations:
847bf1f5
AD
3502
3503@example
3504@group
3505exp: @dots{}
3e259915 3506 | exp '/' exp
847bf1f5 3507 @{
3e259915
MA
3508 @@$.first_column = @@1.first_column;
3509 @@$.first_line = @@1.first_line;
847bf1f5
AD
3510 @@$.last_column = @@3.last_column;
3511 @@$.last_line = @@3.last_line;
3e259915
MA
3512 if ($3)
3513 $$ = $1 / $3;
3514 else
3515 @{
3516 $$ = 1;
4e03e201
AD
3517 fprintf (stderr,
3518 "Division by zero, l%d,c%d-l%d,c%d",
3519 @@3.first_line, @@3.first_column,
3520 @@3.last_line, @@3.last_column);
3e259915 3521 @}
847bf1f5
AD
3522 @}
3523@end group
3524@end example
3525
3e259915 3526As for semantic values, there is a default action for locations that is
72d2299c 3527run each time a rule is matched. It sets the beginning of @code{@@$} to the
3e259915 3528beginning of the first symbol, and the end of @code{@@$} to the end of the
79282c6c 3529last symbol.
3e259915 3530
72d2299c 3531With this default action, the location tracking can be fully automatic. The
3e259915
MA
3532example above simply rewrites this way:
3533
3534@example
3535@group
3536exp: @dots{}
3537 | exp '/' exp
3538 @{
3539 if ($3)
3540 $$ = $1 / $3;
3541 else
3542 @{
3543 $$ = 1;
4e03e201
AD
3544 fprintf (stderr,
3545 "Division by zero, l%d,c%d-l%d,c%d",
3546 @@3.first_line, @@3.first_column,
3547 @@3.last_line, @@3.last_column);
3e259915
MA
3548 @}
3549 @}
3550@end group
3551@end example
847bf1f5 3552
32c29292
JD
3553@vindex yylloc
3554It is also possible to access the location of the look-ahead token, if any,
3555from a semantic action.
3556This location is stored in @code{yylloc}.
3557@xref{Action Features, ,Special Features for Use in Actions}.
3558
342b8b6e 3559@node Location Default Action
847bf1f5
AD
3560@subsection Default Action for Locations
3561@vindex YYLLOC_DEFAULT
8710fc41 3562@cindex @acronym{GLR} parsers and @code{YYLLOC_DEFAULT}
847bf1f5 3563
72d2299c 3564Actually, actions are not the best place to compute locations. Since
704a47c4
AD
3565locations are much more general than semantic values, there is room in
3566the output parser to redefine the default action to take for each
72d2299c 3567rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
96b93a3d
PE
3568matched, before the associated action is run. It is also invoked
3569while processing a syntax error, to compute the error's location.
8710fc41
JD
3570Before reporting an unresolvable syntactic ambiguity, a @acronym{GLR}
3571parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
3572of that ambiguity.
847bf1f5 3573
3e259915 3574Most of the time, this macro is general enough to suppress location
79282c6c 3575dedicated code from semantic actions.
847bf1f5 3576
72d2299c 3577The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
96b93a3d 3578the location of the grouping (the result of the computation). When a
766de5eb 3579rule is matched, the second parameter identifies locations of
96b93a3d 3580all right hand side elements of the rule being matched, and the third
8710fc41
JD
3581parameter is the size of the rule's right hand side.
3582When a @acronym{GLR} parser reports an ambiguity, which of multiple candidate
3583right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
3584When processing a syntax error, the second parameter identifies locations
3585of the symbols that were discarded during error processing, and the third
96b93a3d 3586parameter is the number of discarded symbols.
847bf1f5 3587
766de5eb 3588By default, @code{YYLLOC_DEFAULT} is defined this way:
847bf1f5 3589
766de5eb 3590@smallexample
847bf1f5 3591@group
766de5eb
PE
3592# define YYLLOC_DEFAULT(Current, Rhs, N) \
3593 do \
3594 if (N) \
3595 @{ \
3596 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
3597 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
3598 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
3599 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
3600 @} \
3601 else \
3602 @{ \
3603 (Current).first_line = (Current).last_line = \
3604 YYRHSLOC(Rhs, 0).last_line; \
3605 (Current).first_column = (Current).last_column = \
3606 YYRHSLOC(Rhs, 0).last_column; \
3607 @} \
3608 while (0)
847bf1f5 3609@end group
766de5eb 3610@end smallexample
676385e2 3611
766de5eb
PE
3612where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
3613in @var{rhs} when @var{k} is positive, and the location of the symbol
f28ac696 3614just before the reduction when @var{k} and @var{n} are both zero.
676385e2 3615
3e259915 3616When defining @code{YYLLOC_DEFAULT}, you should consider that:
847bf1f5 3617
3e259915 3618@itemize @bullet
79282c6c 3619@item
72d2299c 3620All arguments are free of side-effects. However, only the first one (the
3e259915 3621result) should be modified by @code{YYLLOC_DEFAULT}.
847bf1f5 3622
3e259915 3623@item
766de5eb
PE
3624For consistency with semantic actions, valid indexes within the
3625right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
3626valid index, and it refers to the symbol just before the reduction.
3627During error processing @var{n} is always positive.
0ae99356
PE
3628
3629@item
3630Your macro should parenthesize its arguments, if need be, since the
3631actual arguments may not be surrounded by parentheses. Also, your
3632macro should expand to something that can be used as a single
3633statement when it is followed by a semicolon.
3e259915 3634@end itemize
847bf1f5 3635
342b8b6e 3636@node Declarations
bfa74976
RS
3637@section Bison Declarations
3638@cindex declarations, Bison
3639@cindex Bison declarations
3640
3641The @dfn{Bison declarations} section of a Bison grammar defines the symbols
3642used in formulating the grammar and the data types of semantic values.
3643@xref{Symbols}.
3644
3645All token type names (but not single-character literal tokens such as
3646@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
3647declared if you need to specify which data type to use for the semantic
3648value (@pxref{Multiple Types, ,More Than One Value Type}).
3649
3650The first rule in the file also specifies the start symbol, by default.
3651If you want some other symbol to be the start symbol, you must declare
704a47c4
AD
3652it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
3653Grammars}).
bfa74976
RS
3654
3655@menu
b50d2359 3656* Require Decl:: Requiring a Bison version.
bfa74976
RS
3657* Token Decl:: Declaring terminal symbols.
3658* Precedence Decl:: Declaring terminals with precedence and associativity.
3659* Union Decl:: Declaring the set of all semantic value types.
3660* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 3661* Initial Action Decl:: Code run before parsing starts.
72f889cc 3662* Destructor Decl:: Declaring how symbols are freed.
d6328241 3663* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
3664* Start Decl:: Specifying the start symbol.
3665* Pure Decl:: Requesting a reentrant parser.
3666* Decl Summary:: Table of all Bison declarations.
3667@end menu
3668
b50d2359
AD
3669@node Require Decl
3670@subsection Require a Version of Bison
3671@cindex version requirement
3672@cindex requiring a version of Bison
3673@findex %require
3674
3675You may require the minimum version of Bison to process the grammar. If
9b8a5ce0
AD
3676the requirement is not met, @command{bison} exits with an error (exit
3677status 63).
b50d2359
AD
3678
3679@example
3680%require "@var{version}"
3681@end example
3682
342b8b6e 3683@node Token Decl
bfa74976
RS
3684@subsection Token Type Names
3685@cindex declaring token type names
3686@cindex token type names, declaring
931c7513 3687@cindex declaring literal string tokens
bfa74976
RS
3688@findex %token
3689
3690The basic way to declare a token type name (terminal symbol) is as follows:
3691
3692@example
3693%token @var{name}
3694@end example
3695
3696Bison will convert this into a @code{#define} directive in
3697the parser, so that the function @code{yylex} (if it is in this file)
3698can use the name @var{name} to stand for this token type's code.
3699
14ded682
AD
3700Alternatively, you can use @code{%left}, @code{%right}, or
3701@code{%nonassoc} instead of @code{%token}, if you wish to specify
3702associativity and precedence. @xref{Precedence Decl, ,Operator
3703Precedence}.
bfa74976
RS
3704
3705You can explicitly specify the numeric code for a token type by appending
1452af69
PE
3706a decimal or hexadecimal integer value in the field immediately
3707following the token name:
bfa74976
RS
3708
3709@example
3710%token NUM 300
1452af69 3711%token XNUM 0x12d // a GNU extension
bfa74976
RS
3712@end example
3713
3714@noindent
3715It is generally best, however, to let Bison choose the numeric codes for
3716all token types. Bison will automatically select codes that don't conflict
e966383b 3717with each other or with normal characters.
bfa74976
RS
3718
3719In the event that the stack type is a union, you must augment the
3720@code{%token} or other token declaration to include the data type
704a47c4
AD
3721alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
3722Than One Value Type}).
bfa74976
RS
3723
3724For example:
3725
3726@example
3727@group
3728%union @{ /* define stack type */
3729 double val;
3730 symrec *tptr;
3731@}
3732%token <val> NUM /* define token NUM and its type */
3733@end group
3734@end example
3735
931c7513
RS
3736You can associate a literal string token with a token type name by
3737writing the literal string at the end of a @code{%token}
3738declaration which declares the name. For example:
3739
3740@example
3741%token arrow "=>"
3742@end example
3743
3744@noindent
3745For example, a grammar for the C language might specify these names with
3746equivalent literal string tokens:
3747
3748@example
3749%token <operator> OR "||"
3750%token <operator> LE 134 "<="
3751%left OR "<="
3752@end example
3753
3754@noindent
3755Once you equate the literal string and the token name, you can use them
3756interchangeably in further declarations or the grammar rules. The
3757@code{yylex} function can use the token name or the literal string to
3758obtain the token type code number (@pxref{Calling Convention}).
3759
342b8b6e 3760@node Precedence Decl
bfa74976
RS
3761@subsection Operator Precedence
3762@cindex precedence declarations
3763@cindex declaring operator precedence
3764@cindex operator precedence, declaring
3765
3766Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to
3767declare a token and specify its precedence and associativity, all at
3768once. These are called @dfn{precedence declarations}.
704a47c4
AD
3769@xref{Precedence, ,Operator Precedence}, for general information on
3770operator precedence.
bfa74976
RS
3771
3772The syntax of a precedence declaration is the same as that of
3773@code{%token}: either
3774
3775@example
3776%left @var{symbols}@dots{}
3777@end example
3778
3779@noindent
3780or
3781
3782@example
3783%left <@var{type}> @var{symbols}@dots{}
3784@end example
3785
3786And indeed any of these declarations serves the purposes of @code{%token}.
3787But in addition, they specify the associativity and relative precedence for
3788all the @var{symbols}:
3789
3790@itemize @bullet
3791@item
3792The associativity of an operator @var{op} determines how repeated uses
3793of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
3794@var{z}} is parsed by grouping @var{x} with @var{y} first or by
3795grouping @var{y} with @var{z} first. @code{%left} specifies
3796left-associativity (grouping @var{x} with @var{y} first) and
3797@code{%right} specifies right-associativity (grouping @var{y} with
3798@var{z} first). @code{%nonassoc} specifies no associativity, which
3799means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
3800considered a syntax error.
3801
3802@item
3803The precedence of an operator determines how it nests with other operators.
3804All the tokens declared in a single precedence declaration have equal
3805precedence and nest together according to their associativity.
3806When two tokens declared in different precedence declarations associate,
3807the one declared later has the higher precedence and is grouped first.
3808@end itemize
3809
342b8b6e 3810@node Union Decl
bfa74976
RS
3811@subsection The Collection of Value Types
3812@cindex declaring value types
3813@cindex value types, declaring
3814@findex %union
3815
287c78f6
PE
3816The @code{%union} declaration specifies the entire collection of
3817possible data types for semantic values. The keyword @code{%union} is
3818followed by braced code containing the same thing that goes inside a
3819@code{union} in C@.
bfa74976
RS
3820
3821For example:
3822
3823@example
3824@group
3825%union @{
3826 double val;
3827 symrec *tptr;
3828@}
3829@end group
3830@end example
3831
3832@noindent
3833This says that the two alternative types are @code{double} and @code{symrec
3834*}. They are given names @code{val} and @code{tptr}; these names are used
3835in the @code{%token} and @code{%type} declarations to pick one of the types
3836for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
3837
6273355b
PE
3838As an extension to @acronym{POSIX}, a tag is allowed after the
3839@code{union}. For example:
3840
3841@example
3842@group
3843%union value @{
3844 double val;
3845 symrec *tptr;
3846@}
3847@end group
3848@end example
3849
d6ca7905 3850@noindent
6273355b
PE
3851specifies the union tag @code{value}, so the corresponding C type is
3852@code{union value}. If you do not specify a tag, it defaults to
3853@code{YYSTYPE}.
3854
d6ca7905
PE
3855As another extension to @acronym{POSIX}, you may specify multiple
3856@code{%union} declarations; their contents are concatenated. However,
3857only the first @code{%union} declaration can specify a tag.
3858
6273355b 3859Note that, unlike making a @code{union} declaration in C, you need not write
bfa74976
RS
3860a semicolon after the closing brace.
3861
342b8b6e 3862@node Type Decl
bfa74976
RS
3863@subsection Nonterminal Symbols
3864@cindex declaring value types, nonterminals
3865@cindex value types, nonterminals, declaring
3866@findex %type
3867
3868@noindent
3869When you use @code{%union} to specify multiple value types, you must
3870declare the value type of each nonterminal symbol for which values are
3871used. This is done with a @code{%type} declaration, like this:
3872
3873@example
3874%type <@var{type}> @var{nonterminal}@dots{}
3875@end example
3876
3877@noindent
704a47c4
AD
3878Here @var{nonterminal} is the name of a nonterminal symbol, and
3879@var{type} is the name given in the @code{%union} to the alternative
3880that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
3881can give any number of nonterminal symbols in the same @code{%type}
3882declaration, if they have the same value type. Use spaces to separate
3883the symbol names.
bfa74976 3884
931c7513
RS
3885You can also declare the value type of a terminal symbol. To do this,
3886use the same @code{<@var{type}>} construction in a declaration for the
3887terminal symbol. All kinds of token declarations allow
3888@code{<@var{type}>}.
3889
18d192f0
AD
3890@node Initial Action Decl
3891@subsection Performing Actions before Parsing
3892@findex %initial-action
3893
3894Sometimes your parser needs to perform some initializations before
3895parsing. The @code{%initial-action} directive allows for such arbitrary
3896code.
3897
3898@deffn {Directive} %initial-action @{ @var{code} @}
3899@findex %initial-action
287c78f6 3900Declare that the braced @var{code} must be invoked before parsing each time
451364ed
AD
3901@code{yyparse} is called. The @var{code} may use @code{$$} and
3902@code{@@$} --- initial value and location of the look-ahead --- and the
3903@code{%parse-param}.
18d192f0
AD
3904@end deffn
3905
451364ed
AD
3906For instance, if your locations use a file name, you may use
3907
3908@example
48b16bbc 3909%parse-param @{ char const *file_name @};
451364ed
AD
3910%initial-action
3911@{
4626a15d 3912 @@$.initialize (file_name);
451364ed
AD
3913@};
3914@end example
3915
18d192f0 3916
72f889cc
AD
3917@node Destructor Decl
3918@subsection Freeing Discarded Symbols
3919@cindex freeing discarded symbols
3920@findex %destructor
3921
a85284cf
AD
3922During error recovery (@pxref{Error Recovery}), symbols already pushed
3923on the stack and tokens coming from the rest of the file are discarded
3924until the parser falls on its feet. If the parser runs out of memory,
9d9b8b70 3925or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
a85284cf
AD
3926symbols on the stack must be discarded. Even if the parser succeeds, it
3927must discard the start symbol.
258b75ca
PE
3928
3929When discarded symbols convey heap based information, this memory is
3930lost. While this behavior can be tolerable for batch parsers, such as
4b367315
AD
3931in traditional compilers, it is unacceptable for programs like shells or
3932protocol implementations that may parse and execute indefinitely.
258b75ca 3933
a85284cf
AD
3934The @code{%destructor} directive defines code that is called when a
3935symbol is automatically discarded.
72f889cc
AD
3936
3937@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
3938@findex %destructor
287c78f6
PE
3939Invoke the braced @var{code} whenever the parser discards one of the
3940@var{symbols}.
4b367315
AD
3941Within @var{code}, @code{$$} designates the semantic value associated
3942with the discarded symbol. The additional parser parameters are also
3943available (@pxref{Parser Function, , The Parser Function
3944@code{yyparse}}).
72f889cc
AD
3945@end deffn
3946
3947For instance:
3948
3949@smallexample
3950%union
3951@{
3952 char *string;
3953@}
3954%token <string> STRING
3955%type <string> string
3956%destructor @{ free ($$); @} STRING string
3957@end smallexample
3958
3959@noindent
258b75ca 3960guarantees that when a @code{STRING} or a @code{string} is discarded,
72f889cc
AD
3961its associated memory will be freed.
3962
e757bb10
AD
3963@sp 1
3964
3965@cindex discarded symbols
3966@dfn{Discarded symbols} are the following:
3967
3968@itemize
3969@item
3970stacked symbols popped during the first phase of error recovery,
3971@item
3972incoming terminals during the second phase of error recovery,
3973@item
a85284cf 3974the current look-ahead and the entire stack (except the current
9d9b8b70 3975right-hand side symbols) when the parser returns immediately, and
258b75ca
PE
3976@item
3977the start symbol, when the parser succeeds.
e757bb10
AD
3978@end itemize
3979
9d9b8b70
PE
3980The parser can @dfn{return immediately} because of an explicit call to
3981@code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
3982exhaustion.
3983
3984Right-hand size symbols of a rule that explicitly triggers a syntax
3985error via @code{YYERROR} are not discarded automatically. As a rule
3986of thumb, destructors are invoked only when user actions cannot manage
a85284cf 3987the memory.
e757bb10 3988
342b8b6e 3989@node Expect Decl
bfa74976
RS
3990@subsection Suppressing Conflict Warnings
3991@cindex suppressing conflict warnings
3992@cindex preventing warnings about conflicts
3993@cindex warnings, preventing
3994@cindex conflicts, suppressing warnings of
3995@findex %expect
d6328241 3996@findex %expect-rr
bfa74976
RS
3997
3998Bison normally warns if there are any conflicts in the grammar
7da99ede
AD
3999(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4000have harmless shift/reduce conflicts which are resolved in a predictable
4001way and would be difficult to eliminate. It is desirable to suppress
4002the warning about these conflicts unless the number of conflicts
4003changes. You can do this with the @code{%expect} declaration.
bfa74976
RS
4004
4005The declaration looks like this:
4006
4007@example
4008%expect @var{n}
4009@end example
4010
035aa4a0
PE
4011Here @var{n} is a decimal integer. The declaration says there should
4012be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4013Bison reports an error if the number of shift/reduce conflicts differs
4014from @var{n}, or if there are any reduce/reduce conflicts.
bfa74976 4015
035aa4a0
PE
4016For normal @acronym{LALR}(1) parsers, reduce/reduce conflicts are more
4017serious, and should be eliminated entirely. Bison will always report
4018reduce/reduce conflicts for these parsers. With @acronym{GLR}
4019parsers, however, both kinds of conflicts are routine; otherwise,
4020there would be no need to use @acronym{GLR} parsing. Therefore, it is
4021also possible to specify an expected number of reduce/reduce conflicts
4022in @acronym{GLR} parsers, using the declaration:
d6328241
PH
4023
4024@example
4025%expect-rr @var{n}
4026@end example
4027
bfa74976
RS
4028In general, using @code{%expect} involves these steps:
4029
4030@itemize @bullet
4031@item
4032Compile your grammar without @code{%expect}. Use the @samp{-v} option
4033to get a verbose list of where the conflicts occur. Bison will also
4034print the number of conflicts.
4035
4036@item
4037Check each of the conflicts to make sure that Bison's default
4038resolution is what you really want. If not, rewrite the grammar and
4039go back to the beginning.
4040
4041@item
4042Add an @code{%expect} declaration, copying the number @var{n} from the
035aa4a0
PE
4043number which Bison printed. With @acronym{GLR} parsers, add an
4044@code{%expect-rr} declaration as well.
bfa74976
RS
4045@end itemize
4046
035aa4a0
PE
4047Now Bison will warn you if you introduce an unexpected conflict, but
4048will keep silent otherwise.
bfa74976 4049
342b8b6e 4050@node Start Decl
bfa74976
RS
4051@subsection The Start-Symbol
4052@cindex declaring the start symbol
4053@cindex start symbol, declaring
4054@cindex default start symbol
4055@findex %start
4056
4057Bison assumes by default that the start symbol for the grammar is the first
4058nonterminal specified in the grammar specification section. The programmer
4059may override this restriction with the @code{%start} declaration as follows:
4060
4061@example
4062%start @var{symbol}
4063@end example
4064
342b8b6e 4065@node Pure Decl
bfa74976
RS
4066@subsection A Pure (Reentrant) Parser
4067@cindex reentrant parser
4068@cindex pure parser
8c9a50be 4069@findex %pure-parser
bfa74976
RS
4070
4071A @dfn{reentrant} program is one which does not alter in the course of
4072execution; in other words, it consists entirely of @dfn{pure} (read-only)
4073code. Reentrancy is important whenever asynchronous execution is possible;
9d9b8b70
PE
4074for example, a nonreentrant program may not be safe to call from a signal
4075handler. In systems with multiple threads of control, a nonreentrant
bfa74976
RS
4076program must be called only within interlocks.
4077
70811b85 4078Normally, Bison generates a parser which is not reentrant. This is
c827f760
PE
4079suitable for most uses, and it permits compatibility with Yacc. (The
4080standard Yacc interfaces are inherently nonreentrant, because they use
70811b85
RS
4081statically allocated variables for communication with @code{yylex},
4082including @code{yylval} and @code{yylloc}.)
bfa74976 4083
70811b85 4084Alternatively, you can generate a pure, reentrant parser. The Bison
8c9a50be 4085declaration @code{%pure-parser} says that you want the parser to be
70811b85 4086reentrant. It looks like this:
bfa74976
RS
4087
4088@example
8c9a50be 4089%pure-parser
bfa74976
RS
4090@end example
4091
70811b85
RS
4092The result is that the communication variables @code{yylval} and
4093@code{yylloc} become local variables in @code{yyparse}, and a different
4094calling convention is used for the lexical analyzer function
4095@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
4096Parsers}, for the details of this. The variable @code{yynerrs} also
4097becomes local in @code{yyparse} (@pxref{Error Reporting, ,The Error
4098Reporting Function @code{yyerror}}). The convention for calling
4099@code{yyparse} itself is unchanged.
4100
4101Whether the parser is pure has nothing to do with the grammar rules.
4102You can generate either a pure parser or a nonreentrant parser from any
4103valid grammar.
bfa74976 4104
342b8b6e 4105@node Decl Summary
bfa74976
RS
4106@subsection Bison Declaration Summary
4107@cindex Bison declaration summary
4108@cindex declaration summary
4109@cindex summary, Bison declaration
4110
d8988b2f 4111Here is a summary of the declarations used to define a grammar:
bfa74976 4112
18b519c0 4113@deffn {Directive} %union
bfa74976
RS
4114Declare the collection of data types that semantic values may have
4115(@pxref{Union Decl, ,The Collection of Value Types}).
18b519c0 4116@end deffn
bfa74976 4117
18b519c0 4118@deffn {Directive} %token
bfa74976
RS
4119Declare a terminal symbol (token type name) with no precedence
4120or associativity specified (@pxref{Token Decl, ,Token Type Names}).
18b519c0 4121@end deffn
bfa74976 4122
18b519c0 4123@deffn {Directive} %right
bfa74976
RS
4124Declare a terminal symbol (token type name) that is right-associative
4125(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4126@end deffn
bfa74976 4127
18b519c0 4128@deffn {Directive} %left
bfa74976
RS
4129Declare a terminal symbol (token type name) that is left-associative
4130(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4131@end deffn
bfa74976 4132
18b519c0 4133@deffn {Directive} %nonassoc
bfa74976 4134Declare a terminal symbol (token type name) that is nonassociative
bfa74976 4135(@pxref{Precedence Decl, ,Operator Precedence}).
39a06c25
PE
4136Using it in a way that would be associative is a syntax error.
4137@end deffn
4138
91d2c560 4139@ifset defaultprec
39a06c25 4140@deffn {Directive} %default-prec
22fccf95 4141Assign a precedence to rules lacking an explicit @code{%prec} modifier
39a06c25
PE
4142(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4143@end deffn
91d2c560 4144@end ifset
bfa74976 4145
18b519c0 4146@deffn {Directive} %type
bfa74976
RS
4147Declare the type of semantic values for a nonterminal symbol
4148(@pxref{Type Decl, ,Nonterminal Symbols}).
18b519c0 4149@end deffn
bfa74976 4150
18b519c0 4151@deffn {Directive} %start
89cab50d
AD
4152Specify the grammar's start symbol (@pxref{Start Decl, ,The
4153Start-Symbol}).
18b519c0 4154@end deffn
bfa74976 4155
18b519c0 4156@deffn {Directive} %expect
bfa74976
RS
4157Declare the expected number of shift-reduce conflicts
4158(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
18b519c0
AD
4159@end deffn
4160
bfa74976 4161
d8988b2f
AD
4162@sp 1
4163@noindent
4164In order to change the behavior of @command{bison}, use the following
4165directives:
4166
18b519c0 4167@deffn {Directive} %debug
4947ebdb
PE
4168In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
4169already defined, so that the debugging facilities are compiled.
18b519c0 4170@end deffn
ec3bc396 4171@xref{Tracing, ,Tracing Your Parser}.
d8988b2f 4172
18b519c0 4173@deffn {Directive} %defines
4bfd5e4e
PE
4174Write a header file containing macro definitions for the token type
4175names defined in the grammar as well as a few other declarations.
d8988b2f 4176If the parser output file is named @file{@var{name}.c} then this file
e0c471a9 4177is named @file{@var{name}.h}.
d8988b2f 4178
4bfd5e4e 4179Unless @code{YYSTYPE} is already defined as a macro, the output header
5c9be03d 4180declares @code{YYSTYPE}. Therefore, if you are using a @code{%union}
f8e1c9e5
AD
4181(@pxref{Multiple Types, ,More Than One Value Type}) with components that
4182require other definitions, or if you have defined a @code{YYSTYPE} macro
4183(@pxref{Value Type, ,Data Types of Semantic Values}), you need to
4184arrange for these definitions to be propagated to all modules, e.g., by
4185putting them in a prerequisite header that is included both by your
4186parser and by any other module that needs @code{YYSTYPE}.
4bfd5e4e
PE
4187
4188Unless your parser is pure, the output header declares @code{yylval}
4189as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
4190Parser}.
4191
4192If you have also used locations, the output header declares
4193@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
4194@code{YYSTYPE} and @code{yylval}. @xref{Locations, ,Tracking
4195Locations}.
4196
f8e1c9e5
AD
4197This output file is normally essential if you wish to put the definition
4198of @code{yylex} in a separate source file, because @code{yylex}
4199typically needs to be able to refer to the above-mentioned declarations
4200and to the token type codes. @xref{Token Values, ,Semantic Values of
4201Tokens}.
18b519c0 4202@end deffn
d8988b2f 4203
18b519c0 4204@deffn {Directive} %destructor
258b75ca 4205Specify how the parser should reclaim the memory associated to
fa7e68c3 4206discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 4207@end deffn
72f889cc 4208
18b519c0 4209@deffn {Directive} %file-prefix="@var{prefix}"
d8988b2f
AD
4210Specify a prefix to use for all Bison output file names. The names are
4211chosen as if the input file were named @file{@var{prefix}.y}.
18b519c0 4212@end deffn
d8988b2f 4213
18b519c0 4214@deffn {Directive} %locations
89cab50d
AD
4215Generate the code processing the locations (@pxref{Action Features,
4216,Special Features for Use in Actions}). This mode is enabled as soon as
4217the grammar uses the special @samp{@@@var{n}} tokens, but if your
4218grammar does not use it, using @samp{%locations} allows for more
6e649e65 4219accurate syntax error messages.
18b519c0 4220@end deffn
89cab50d 4221
18b519c0 4222@deffn {Directive} %name-prefix="@var{prefix}"
d8988b2f
AD
4223Rename the external symbols used in the parser so that they start with
4224@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
aa08666d 4225in C parsers
d8988b2f 4226is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
91e3ac9a
PE
4227@code{yylval}, @code{yychar}, @code{yydebug}, and
4228(if locations are used) @code{yylloc}. For example, if you use
2a8d363a 4229@samp{%name-prefix="c_"}, the names become @code{c_parse}, @code{c_lex},
aa08666d
AD
4230and so on. In C++ parsers, it is only the surrounding namespace which is
4231named @var{prefix} instead of @samp{yy}.
4232@xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
18b519c0 4233@end deffn
931c7513 4234
91d2c560 4235@ifset defaultprec
22fccf95
PE
4236@deffn {Directive} %no-default-prec
4237Do not assign a precedence to rules lacking an explicit @code{%prec}
4238modifier (@pxref{Contextual Precedence, ,Context-Dependent
4239Precedence}).
4240@end deffn
91d2c560 4241@end ifset
22fccf95 4242
18b519c0 4243@deffn {Directive} %no-parser
6deb4447
AD
4244Do not include any C code in the parser file; generate tables only. The
4245parser file contains just @code{#define} directives and static variable
4246declarations.
4247
4248This option also tells Bison to write the C code for the grammar actions
fa4d969f 4249into a file named @file{@var{file}.act}, in the form of a
6deb4447 4250brace-surrounded body fit for a @code{switch} statement.
18b519c0 4251@end deffn
6deb4447 4252
18b519c0 4253@deffn {Directive} %no-lines
931c7513
RS
4254Don't generate any @code{#line} preprocessor commands in the parser
4255file. Ordinarily Bison writes these commands in the parser file so that
4256the C compiler and debuggers will associate errors and object code with
4257your source file (the grammar file). This directive causes them to
4258associate errors with the parser file, treating it an independent source
4259file in its own right.
18b519c0 4260@end deffn
931c7513 4261
fa4d969f
PE
4262@deffn {Directive} %output="@var{file}"
4263Specify @var{file} for the parser file.
18b519c0 4264@end deffn
6deb4447 4265
18b519c0 4266@deffn {Directive} %pure-parser
d8988b2f
AD
4267Request a pure (reentrant) parser program (@pxref{Pure Decl, ,A Pure
4268(Reentrant) Parser}).
18b519c0 4269@end deffn
6deb4447 4270
b50d2359 4271@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
4272Require version @var{version} or higher of Bison. @xref{Require Decl, ,
4273Require a Version of Bison}.
b50d2359
AD
4274@end deffn
4275
18b519c0 4276@deffn {Directive} %token-table
931c7513
RS
4277Generate an array of token names in the parser file. The name of the
4278array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
3650b4b8 4279token whose internal Bison token code number is @var{i}. The first
f67ad422
PE
4280three elements of @code{yytname} correspond to the predefined tokens
4281@code{"$end"},
88bce5a2
AD
4282@code{"error"}, and @code{"$undefined"}; after these come the symbols
4283defined in the grammar file.
931c7513 4284
9e0876fb
PE
4285The name in the table includes all the characters needed to represent
4286the token in Bison. For single-character literals and literal
4287strings, this includes the surrounding quoting characters and any
4288escape sequences. For example, the Bison single-character literal
4289@code{'+'} corresponds to a three-character name, represented in C as
4290@code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
4291corresponds to a five-character name, represented in C as
4292@code{"\"\\\\/\""}.
931c7513 4293
8c9a50be 4294When you specify @code{%token-table}, Bison also generates macro
931c7513
RS
4295definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
4296@code{YYNRULES}, and @code{YYNSTATES}:
4297
4298@table @code
4299@item YYNTOKENS
4300The highest token number, plus one.
4301@item YYNNTS
9ecbd125 4302The number of nonterminal symbols.
931c7513
RS
4303@item YYNRULES
4304The number of grammar rules,
4305@item YYNSTATES
4306The number of parser states (@pxref{Parser States}).
4307@end table
18b519c0 4308@end deffn
d8988b2f 4309
18b519c0 4310@deffn {Directive} %verbose
d8988b2f
AD
4311Write an extra output file containing verbose descriptions of the
4312parser states and what is done for each type of look-ahead token in
72d2299c 4313that state. @xref{Understanding, , Understanding Your Parser}, for more
ec3bc396 4314information.
18b519c0 4315@end deffn
d8988b2f 4316
18b519c0 4317@deffn {Directive} %yacc
d8988b2f
AD
4318Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
4319including its naming conventions. @xref{Bison Options}, for more.
18b519c0 4320@end deffn
d8988b2f
AD
4321
4322
342b8b6e 4323@node Multiple Parsers
bfa74976
RS
4324@section Multiple Parsers in the Same Program
4325
4326Most programs that use Bison parse only one language and therefore contain
4327only one Bison parser. But what if you want to parse more than one
4328language with the same program? Then you need to avoid a name conflict
4329between different definitions of @code{yyparse}, @code{yylval}, and so on.
4330
4331The easy way to do this is to use the option @samp{-p @var{prefix}}
704a47c4
AD
4332(@pxref{Invocation, ,Invoking Bison}). This renames the interface
4333functions and variables of the Bison parser to start with @var{prefix}
4334instead of @samp{yy}. You can use this to give each parser distinct
4335names that do not conflict.
bfa74976
RS
4336
4337The precise list of symbols renamed is @code{yyparse}, @code{yylex},
2a8d363a
AD
4338@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
4339@code{yychar} and @code{yydebug}. For example, if you use @samp{-p c},
4340the names become @code{cparse}, @code{clex}, and so on.
bfa74976
RS
4341
4342@strong{All the other variables and macros associated with Bison are not
4343renamed.} These others are not global; there is no conflict if the same
4344name is used in different parsers. For example, @code{YYSTYPE} is not
4345renamed, but defining this in different ways in different parsers causes
4346no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
4347
4348The @samp{-p} option works by adding macro definitions to the beginning
4349of the parser source file, defining @code{yyparse} as
4350@code{@var{prefix}parse}, and so on. This effectively substitutes one
4351name for the other in the entire parser file.
4352
342b8b6e 4353@node Interface
bfa74976
RS
4354@chapter Parser C-Language Interface
4355@cindex C-language interface
4356@cindex interface
4357
4358The Bison parser is actually a C function named @code{yyparse}. Here we
4359describe the interface conventions of @code{yyparse} and the other
4360functions that it needs to use.
4361
4362Keep in mind that the parser uses many C identifiers starting with
4363@samp{yy} and @samp{YY} for internal purposes. If you use such an
75f5aaea
MA
4364identifier (aside from those in this manual) in an action or in epilogue
4365in the grammar file, you are likely to run into trouble.
bfa74976
RS
4366
4367@menu
4368* Parser Function:: How to call @code{yyparse} and what it returns.
13863333 4369* Lexical:: You must supply a function @code{yylex}
bfa74976
RS
4370 which reads tokens.
4371* Error Reporting:: You must supply a function @code{yyerror}.
4372* Action Features:: Special features for use in actions.
f7ab6a50
PE
4373* Internationalization:: How to let the parser speak in the user's
4374 native language.
bfa74976
RS
4375@end menu
4376
342b8b6e 4377@node Parser Function
bfa74976
RS
4378@section The Parser Function @code{yyparse}
4379@findex yyparse
4380
4381You call the function @code{yyparse} to cause parsing to occur. This
4382function reads tokens, executes actions, and ultimately returns when it
4383encounters end-of-input or an unrecoverable syntax error. You can also
14ded682
AD
4384write an action which directs @code{yyparse} to return immediately
4385without reading further.
bfa74976 4386
2a8d363a
AD
4387
4388@deftypefun int yyparse (void)
bfa74976
RS
4389The value returned by @code{yyparse} is 0 if parsing was successful (return
4390is due to end-of-input).
4391
b47dbebe
PE
4392The value is 1 if parsing failed because of invalid input, i.e., input
4393that contains a syntax error or that causes @code{YYABORT} to be
4394invoked.
4395
4396The value is 2 if parsing failed due to memory exhaustion.
2a8d363a 4397@end deftypefun
bfa74976
RS
4398
4399In an action, you can cause immediate return from @code{yyparse} by using
4400these macros:
4401
2a8d363a 4402@defmac YYACCEPT
bfa74976
RS
4403@findex YYACCEPT
4404Return immediately with value 0 (to report success).
2a8d363a 4405@end defmac
bfa74976 4406
2a8d363a 4407@defmac YYABORT
bfa74976
RS
4408@findex YYABORT
4409Return immediately with value 1 (to report failure).
2a8d363a
AD
4410@end defmac
4411
4412If you use a reentrant parser, you can optionally pass additional
4413parameter information to it in a reentrant way. To do so, use the
4414declaration @code{%parse-param}:
4415
feeb0eda 4416@deffn {Directive} %parse-param @{@var{argument-declaration}@}
2a8d363a 4417@findex %parse-param
287c78f6
PE
4418Declare that an argument declared by the braced-code
4419@var{argument-declaration} is an additional @code{yyparse} argument.
94175978 4420The @var{argument-declaration} is used when declaring
feeb0eda
PE
4421functions or prototypes. The last identifier in
4422@var{argument-declaration} must be the argument name.
2a8d363a
AD
4423@end deffn
4424
4425Here's an example. Write this in the parser:
4426
4427@example
feeb0eda
PE
4428%parse-param @{int *nastiness@}
4429%parse-param @{int *randomness@}
2a8d363a
AD
4430@end example
4431
4432@noindent
4433Then call the parser like this:
4434
4435@example
4436@{
4437 int nastiness, randomness;
4438 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
4439 value = yyparse (&nastiness, &randomness);
4440 @dots{}
4441@}
4442@end example
4443
4444@noindent
4445In the grammar actions, use expressions like this to refer to the data:
4446
4447@example
4448exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
4449@end example
4450
bfa74976 4451
342b8b6e 4452@node Lexical
bfa74976
RS
4453@section The Lexical Analyzer Function @code{yylex}
4454@findex yylex
4455@cindex lexical analyzer
4456
4457The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
4458the input stream and returns them to the parser. Bison does not create
4459this function automatically; you must write it so that @code{yyparse} can
4460call it. The function is sometimes referred to as a lexical scanner.
4461
4462In simple programs, @code{yylex} is often defined at the end of the Bison
4463grammar file. If @code{yylex} is defined in a separate source file, you
4464need to arrange for the token-type macro definitions to be available there.
4465To do this, use the @samp{-d} option when you run Bison, so that it will
4466write these macro definitions into a separate header file
4467@file{@var{name}.tab.h} which you can include in the other source files
e0c471a9 4468that need it. @xref{Invocation, ,Invoking Bison}.
bfa74976
RS
4469
4470@menu
4471* Calling Convention:: How @code{yyparse} calls @code{yylex}.
4472* Token Values:: How @code{yylex} must return the semantic value
4473 of the token it has read.
95923bd6 4474* Token Locations:: How @code{yylex} must return the text location
bfa74976
RS
4475 (line number, etc.) of the token, if the
4476 actions want that.
4477* Pure Calling:: How the calling convention differs
4478 in a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
4479@end menu
4480
342b8b6e 4481@node Calling Convention
bfa74976
RS
4482@subsection Calling Convention for @code{yylex}
4483
72d2299c
PE
4484The value that @code{yylex} returns must be the positive numeric code
4485for the type of token it has just found; a zero or negative value
4486signifies end-of-input.
bfa74976
RS
4487
4488When a token is referred to in the grammar rules by a name, that name
4489in the parser file becomes a C macro whose definition is the proper
4490numeric code for that token type. So @code{yylex} can use the name
4491to indicate that type. @xref{Symbols}.
4492
4493When a token is referred to in the grammar rules by a character literal,
4494the numeric code for that character is also the code for the token type.
72d2299c
PE
4495So @code{yylex} can simply return that character code, possibly converted
4496to @code{unsigned char} to avoid sign-extension. The null character
4497must not be used this way, because its code is zero and that
bfa74976
RS
4498signifies end-of-input.
4499
4500Here is an example showing these things:
4501
4502@example
13863333
AD
4503int
4504yylex (void)
bfa74976
RS
4505@{
4506 @dots{}
72d2299c 4507 if (c == EOF) /* Detect end-of-input. */
bfa74976
RS
4508 return 0;
4509 @dots{}
4510 if (c == '+' || c == '-')
72d2299c 4511 return c; /* Assume token type for `+' is '+'. */
bfa74976 4512 @dots{}
72d2299c 4513 return INT; /* Return the type of the token. */
bfa74976
RS
4514 @dots{}
4515@}
4516@end example
4517
4518@noindent
4519This interface has been designed so that the output from the @code{lex}
4520utility can be used without change as the definition of @code{yylex}.
4521
931c7513
RS
4522If the grammar uses literal string tokens, there are two ways that
4523@code{yylex} can determine the token type codes for them:
4524
4525@itemize @bullet
4526@item
4527If the grammar defines symbolic token names as aliases for the
4528literal string tokens, @code{yylex} can use these symbolic names like
4529all others. In this case, the use of the literal string tokens in
4530the grammar file has no effect on @code{yylex}.
4531
4532@item
9ecbd125 4533@code{yylex} can find the multicharacter token in the @code{yytname}
931c7513 4534table. The index of the token in the table is the token type's code.
9ecbd125 4535The name of a multicharacter token is recorded in @code{yytname} with a
931c7513 4536double-quote, the token's characters, and another double-quote. The
9e0876fb
PE
4537token's characters are escaped as necessary to be suitable as input
4538to Bison.
931c7513 4539
9e0876fb
PE
4540Here's code for looking up a multicharacter token in @code{yytname},
4541assuming that the characters of the token are stored in
4542@code{token_buffer}, and assuming that the token does not contain any
4543characters like @samp{"} that require escaping.
931c7513
RS
4544
4545@smallexample
4546for (i = 0; i < YYNTOKENS; i++)
4547 @{
4548 if (yytname[i] != 0
4549 && yytname[i][0] == '"'
68449b3a
PE
4550 && ! strncmp (yytname[i] + 1, token_buffer,
4551 strlen (token_buffer))
931c7513
RS
4552 && yytname[i][strlen (token_buffer) + 1] == '"'
4553 && yytname[i][strlen (token_buffer) + 2] == 0)
4554 break;
4555 @}
4556@end smallexample
4557
4558The @code{yytname} table is generated only if you use the
8c9a50be 4559@code{%token-table} declaration. @xref{Decl Summary}.
931c7513
RS
4560@end itemize
4561
342b8b6e 4562@node Token Values
bfa74976
RS
4563@subsection Semantic Values of Tokens
4564
4565@vindex yylval
9d9b8b70 4566In an ordinary (nonreentrant) parser, the semantic value of the token must
bfa74976
RS
4567be stored into the global variable @code{yylval}. When you are using
4568just one data type for semantic values, @code{yylval} has that type.
4569Thus, if the type is @code{int} (the default), you might write this in
4570@code{yylex}:
4571
4572@example
4573@group
4574 @dots{}
72d2299c
PE
4575 yylval = value; /* Put value onto Bison stack. */
4576 return INT; /* Return the type of the token. */
bfa74976
RS
4577 @dots{}
4578@end group
4579@end example
4580
4581When you are using multiple data types, @code{yylval}'s type is a union
704a47c4
AD
4582made from the @code{%union} declaration (@pxref{Union Decl, ,The
4583Collection of Value Types}). So when you store a token's value, you
4584must use the proper member of the union. If the @code{%union}
4585declaration looks like this:
bfa74976
RS
4586
4587@example
4588@group
4589%union @{
4590 int intval;
4591 double val;
4592 symrec *tptr;
4593@}
4594@end group
4595@end example
4596
4597@noindent
4598then the code in @code{yylex} might look like this:
4599
4600@example
4601@group
4602 @dots{}
72d2299c
PE
4603 yylval.intval = value; /* Put value onto Bison stack. */
4604 return INT; /* Return the type of the token. */
bfa74976
RS
4605 @dots{}
4606@end group
4607@end example
4608
95923bd6
AD
4609@node Token Locations
4610@subsection Textual Locations of Tokens
bfa74976
RS
4611
4612@vindex yylloc
847bf1f5 4613If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
f8e1c9e5
AD
4614Tracking Locations}) in actions to keep track of the textual locations
4615of tokens and groupings, then you must provide this information in
4616@code{yylex}. The function @code{yyparse} expects to find the textual
4617location of a token just parsed in the global variable @code{yylloc}.
4618So @code{yylex} must store the proper data in that variable.
847bf1f5
AD
4619
4620By default, the value of @code{yylloc} is a structure and you need only
89cab50d
AD
4621initialize the members that are going to be used by the actions. The
4622four members are called @code{first_line}, @code{first_column},
4623@code{last_line} and @code{last_column}. Note that the use of this
4624feature makes the parser noticeably slower.
bfa74976
RS
4625
4626@tindex YYLTYPE
4627The data type of @code{yylloc} has the name @code{YYLTYPE}.
4628
342b8b6e 4629@node Pure Calling
c656404a 4630@subsection Calling Conventions for Pure Parsers
bfa74976 4631
8c9a50be 4632When you use the Bison declaration @code{%pure-parser} to request a
e425e872
RS
4633pure, reentrant parser, the global communication variables @code{yylval}
4634and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
4635Parser}.) In such parsers the two global variables are replaced by
4636pointers passed as arguments to @code{yylex}. You must declare them as
4637shown here, and pass the information back by storing it through those
4638pointers.
bfa74976
RS
4639
4640@example
13863333
AD
4641int
4642yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
bfa74976
RS
4643@{
4644 @dots{}
4645 *lvalp = value; /* Put value onto Bison stack. */
4646 return INT; /* Return the type of the token. */
4647 @dots{}
4648@}
4649@end example
4650
4651If the grammar file does not use the @samp{@@} constructs to refer to
95923bd6 4652textual locations, then the type @code{YYLTYPE} will not be defined. In
bfa74976
RS
4653this case, omit the second argument; @code{yylex} will be called with
4654only one argument.
4655
e425e872 4656
2a8d363a
AD
4657If you wish to pass the additional parameter data to @code{yylex}, use
4658@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
4659Function}).
e425e872 4660
feeb0eda 4661@deffn {Directive} lex-param @{@var{argument-declaration}@}
2a8d363a 4662@findex %lex-param
287c78f6
PE
4663Declare that the braced-code @var{argument-declaration} is an
4664additional @code{yylex} argument declaration.
2a8d363a 4665@end deffn
e425e872 4666
2a8d363a 4667For instance:
e425e872
RS
4668
4669@example
feeb0eda
PE
4670%parse-param @{int *nastiness@}
4671%lex-param @{int *nastiness@}
4672%parse-param @{int *randomness@}
e425e872
RS
4673@end example
4674
4675@noindent
2a8d363a 4676results in the following signature:
e425e872
RS
4677
4678@example
2a8d363a
AD
4679int yylex (int *nastiness);
4680int yyparse (int *nastiness, int *randomness);
e425e872
RS
4681@end example
4682
2a8d363a 4683If @code{%pure-parser} is added:
c656404a
RS
4684
4685@example
2a8d363a
AD
4686int yylex (YYSTYPE *lvalp, int *nastiness);
4687int yyparse (int *nastiness, int *randomness);
c656404a
RS
4688@end example
4689
2a8d363a
AD
4690@noindent
4691and finally, if both @code{%pure-parser} and @code{%locations} are used:
c656404a 4692
2a8d363a
AD
4693@example
4694int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
4695int yyparse (int *nastiness, int *randomness);
4696@end example
931c7513 4697
342b8b6e 4698@node Error Reporting
bfa74976
RS
4699@section The Error Reporting Function @code{yyerror}
4700@cindex error reporting function
4701@findex yyerror
4702@cindex parse error
4703@cindex syntax error
4704
6e649e65 4705The Bison parser detects a @dfn{syntax error} or @dfn{parse error}
9ecbd125 4706whenever it reads a token which cannot satisfy any syntax rule. An
bfa74976 4707action in the grammar can also explicitly proclaim an error, using the
ceed8467
AD
4708macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
4709in Actions}).
bfa74976
RS
4710
4711The Bison parser expects to report the error by calling an error
4712reporting function named @code{yyerror}, which you must supply. It is
4713called by @code{yyparse} whenever a syntax error is found, and it
6e649e65
PE
4714receives one argument. For a syntax error, the string is normally
4715@w{@code{"syntax error"}}.
bfa74976 4716
2a8d363a
AD
4717@findex %error-verbose
4718If you invoke the directive @code{%error-verbose} in the Bison
4719declarations section (@pxref{Bison Declarations, ,The Bison Declarations
4720Section}), then Bison provides a more verbose and specific error message
6e649e65 4721string instead of just plain @w{@code{"syntax error"}}.
bfa74976 4722
1a059451
PE
4723The parser can detect one other kind of error: memory exhaustion. This
4724can happen when the input contains constructions that are very deeply
bfa74976 4725nested. It isn't likely you will encounter this, since the Bison
1a059451
PE
4726parser normally extends its stack automatically up to a very large limit. But
4727if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
4728fashion, except that the argument string is @w{@code{"memory exhausted"}}.
4729
4730In some cases diagnostics like @w{@code{"syntax error"}} are
4731translated automatically from English to some other language before
4732they are passed to @code{yyerror}. @xref{Internationalization}.
bfa74976
RS
4733
4734The following definition suffices in simple programs:
4735
4736@example
4737@group
13863333 4738void
38a92d50 4739yyerror (char const *s)
bfa74976
RS
4740@{
4741@end group
4742@group
4743 fprintf (stderr, "%s\n", s);
4744@}
4745@end group
4746@end example
4747
4748After @code{yyerror} returns to @code{yyparse}, the latter will attempt
4749error recovery if you have written suitable error recovery grammar rules
4750(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
4751immediately return 1.
4752
93724f13 4753Obviously, in location tracking pure parsers, @code{yyerror} should have
fa7e68c3
PE
4754an access to the current location.
4755This is indeed the case for the @acronym{GLR}
2a8d363a
AD
4756parsers, but not for the Yacc parser, for historical reasons. I.e., if
4757@samp{%locations %pure-parser} is passed then the prototypes for
4758@code{yyerror} are:
4759
4760@example
38a92d50
PE
4761void yyerror (char const *msg); /* Yacc parsers. */
4762void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
2a8d363a
AD
4763@end example
4764
feeb0eda 4765If @samp{%parse-param @{int *nastiness@}} is used, then:
2a8d363a
AD
4766
4767@example
b317297e
PE
4768void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
4769void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
2a8d363a
AD
4770@end example
4771
fa7e68c3 4772Finally, @acronym{GLR} and Yacc parsers share the same @code{yyerror} calling
2a8d363a
AD
4773convention for absolutely pure parsers, i.e., when the calling
4774convention of @code{yylex} @emph{and} the calling convention of
4775@code{%pure-parser} are pure. I.e.:
4776
4777@example
4778/* Location tracking. */
4779%locations
4780/* Pure yylex. */
4781%pure-parser
feeb0eda 4782%lex-param @{int *nastiness@}
2a8d363a 4783/* Pure yyparse. */
feeb0eda
PE
4784%parse-param @{int *nastiness@}
4785%parse-param @{int *randomness@}
2a8d363a
AD
4786@end example
4787
4788@noindent
4789results in the following signatures for all the parser kinds:
4790
4791@example
4792int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
4793int yyparse (int *nastiness, int *randomness);
93724f13
AD
4794void yyerror (YYLTYPE *locp,
4795 int *nastiness, int *randomness,
38a92d50 4796 char const *msg);
2a8d363a
AD
4797@end example
4798
1c0c3e95 4799@noindent
38a92d50
PE
4800The prototypes are only indications of how the code produced by Bison
4801uses @code{yyerror}. Bison-generated code always ignores the returned
4802value, so @code{yyerror} can return any type, including @code{void}.
4803Also, @code{yyerror} can be a variadic function; that is why the
4804message is always passed last.
4805
4806Traditionally @code{yyerror} returns an @code{int} that is always
4807ignored, but this is purely for historical reasons, and @code{void} is
4808preferable since it more accurately describes the return type for
4809@code{yyerror}.
93724f13 4810
bfa74976
RS
4811@vindex yynerrs
4812The variable @code{yynerrs} contains the number of syntax errors
8a2800e7 4813reported so far. Normally this variable is global; but if you
704a47c4
AD
4814request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
4815then it is a local variable which only the actions can access.
bfa74976 4816
342b8b6e 4817@node Action Features
bfa74976
RS
4818@section Special Features for Use in Actions
4819@cindex summary, action features
4820@cindex action features summary
4821
4822Here is a table of Bison constructs, variables and macros that
4823are useful in actions.
4824
18b519c0 4825@deffn {Variable} $$
bfa74976
RS
4826Acts like a variable that contains the semantic value for the
4827grouping made by the current rule. @xref{Actions}.
18b519c0 4828@end deffn
bfa74976 4829
18b519c0 4830@deffn {Variable} $@var{n}
bfa74976
RS
4831Acts like a variable that contains the semantic value for the
4832@var{n}th component of the current rule. @xref{Actions}.
18b519c0 4833@end deffn
bfa74976 4834
18b519c0 4835@deffn {Variable} $<@var{typealt}>$
bfa74976 4836Like @code{$$} but specifies alternative @var{typealt} in the union
704a47c4
AD
4837specified by the @code{%union} declaration. @xref{Action Types, ,Data
4838Types of Values in Actions}.
18b519c0 4839@end deffn
bfa74976 4840
18b519c0 4841@deffn {Variable} $<@var{typealt}>@var{n}
bfa74976 4842Like @code{$@var{n}} but specifies alternative @var{typealt} in the
13863333 4843union specified by the @code{%union} declaration.
e0c471a9 4844@xref{Action Types, ,Data Types of Values in Actions}.
18b519c0 4845@end deffn
bfa74976 4846
18b519c0 4847@deffn {Macro} YYABORT;
bfa74976
RS
4848Return immediately from @code{yyparse}, indicating failure.
4849@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 4850@end deffn
bfa74976 4851
18b519c0 4852@deffn {Macro} YYACCEPT;
bfa74976
RS
4853Return immediately from @code{yyparse}, indicating success.
4854@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 4855@end deffn
bfa74976 4856
18b519c0 4857@deffn {Macro} YYBACKUP (@var{token}, @var{value});
bfa74976
RS
4858@findex YYBACKUP
4859Unshift a token. This macro is allowed only for rules that reduce
4860a single value, and only when there is no look-ahead token.
c827f760 4861It is also disallowed in @acronym{GLR} parsers.
bfa74976
RS
4862It installs a look-ahead token with token type @var{token} and
4863semantic value @var{value}; then it discards the value that was
4864going to be reduced by this rule.
4865
4866If the macro is used when it is not valid, such as when there is
4867a look-ahead token already, then it reports a syntax error with
4868a message @samp{cannot back up} and performs ordinary error
4869recovery.
4870
4871In either case, the rest of the action is not executed.
18b519c0 4872@end deffn
bfa74976 4873
18b519c0 4874@deffn {Macro} YYEMPTY
bfa74976
RS
4875@vindex YYEMPTY
4876Value stored in @code{yychar} when there is no look-ahead token.
18b519c0 4877@end deffn
bfa74976 4878
32c29292
JD
4879@deffn {Macro} YYEOF
4880@vindex YYEOF
4881Value stored in @code{yychar} when the look-ahead is the end of the input
4882stream.
4883@end deffn
4884
18b519c0 4885@deffn {Macro} YYERROR;
bfa74976
RS
4886@findex YYERROR
4887Cause an immediate syntax error. This statement initiates error
4888recovery just as if the parser itself had detected an error; however, it
4889does not call @code{yyerror}, and does not print any message. If you
4890want to print an error message, call @code{yyerror} explicitly before
4891the @samp{YYERROR;} statement. @xref{Error Recovery}.
18b519c0 4892@end deffn
bfa74976 4893
18b519c0 4894@deffn {Macro} YYRECOVERING
bfa74976
RS
4895This macro stands for an expression that has the value 1 when the parser
4896is recovering from a syntax error, and 0 the rest of the time.
4897@xref{Error Recovery}.
18b519c0 4898@end deffn
bfa74976 4899
18b519c0 4900@deffn {Variable} yychar
32c29292
JD
4901Variable containing either the look-ahead token, or @code{YYEOF} when the
4902look-ahead is the end of the input stream, or @code{YYEMPTY} when no look-ahead
4903has been performed so the next token is not yet known.
4904Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
4905Actions}).
bfa74976 4906@xref{Look-Ahead, ,Look-Ahead Tokens}.
18b519c0 4907@end deffn
bfa74976 4908
18b519c0 4909@deffn {Macro} yyclearin;
bfa74976 4910Discard the current look-ahead token. This is useful primarily in
32c29292
JD
4911error rules.
4912Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
4913Semantic Actions}).
4914@xref{Error Recovery}.
18b519c0 4915@end deffn
bfa74976 4916
18b519c0 4917@deffn {Macro} yyerrok;
bfa74976 4918Resume generating error messages immediately for subsequent syntax
13863333 4919errors. This is useful primarily in error rules.
bfa74976 4920@xref{Error Recovery}.
18b519c0 4921@end deffn
bfa74976 4922
32c29292
JD
4923@deffn {Variable} yylloc
4924Variable containing the look-ahead token location when @code{yychar} is not set
4925to @code{YYEMPTY} or @code{YYEOF}.
4926Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
4927Actions}).
4928@xref{Actions and Locations, ,Actions and Locations}.
4929@end deffn
4930
4931@deffn {Variable} yylval
4932Variable containing the look-ahead token semantic value when @code{yychar} is
4933not set to @code{YYEMPTY} or @code{YYEOF}.
4934Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
4935Actions}).
4936@xref{Actions, ,Actions}.
4937@end deffn
4938
18b519c0 4939@deffn {Value} @@$
847bf1f5 4940@findex @@$
95923bd6 4941Acts like a structure variable containing information on the textual location
847bf1f5
AD
4942of the grouping made by the current rule. @xref{Locations, ,
4943Tracking Locations}.
bfa74976 4944
847bf1f5
AD
4945@c Check if those paragraphs are still useful or not.
4946
4947@c @example
4948@c struct @{
4949@c int first_line, last_line;
4950@c int first_column, last_column;
4951@c @};
4952@c @end example
4953
4954@c Thus, to get the starting line number of the third component, you would
4955@c use @samp{@@3.first_line}.
bfa74976 4956
847bf1f5
AD
4957@c In order for the members of this structure to contain valid information,
4958@c you must make @code{yylex} supply this information about each token.
4959@c If you need only certain members, then @code{yylex} need only fill in
4960@c those members.
bfa74976 4961
847bf1f5 4962@c The use of this feature makes the parser noticeably slower.
18b519c0 4963@end deffn
847bf1f5 4964
18b519c0 4965@deffn {Value} @@@var{n}
847bf1f5 4966@findex @@@var{n}
95923bd6 4967Acts like a structure variable containing information on the textual location
847bf1f5
AD
4968of the @var{n}th component of the current rule. @xref{Locations, ,
4969Tracking Locations}.
18b519c0 4970@end deffn
bfa74976 4971
f7ab6a50
PE
4972@node Internationalization
4973@section Parser Internationalization
4974@cindex internationalization
4975@cindex i18n
4976@cindex NLS
4977@cindex gettext
4978@cindex bison-po
4979
4980A Bison-generated parser can print diagnostics, including error and
4981tracing messages. By default, they appear in English. However, Bison
f8e1c9e5
AD
4982also supports outputting diagnostics in the user's native language. To
4983make this work, the user should set the usual environment variables.
4984@xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
4985For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
4986set the user's locale to French Canadian using the @acronym{UTF}-8
f7ab6a50
PE
4987encoding. The exact set of available locales depends on the user's
4988installation.
4989
4990The maintainer of a package that uses a Bison-generated parser enables
4991the internationalization of the parser's output through the following
4992steps. Here we assume a package that uses @acronym{GNU} Autoconf and
4993@acronym{GNU} Automake.
4994
4995@enumerate
4996@item
30757c8c 4997@cindex bison-i18n.m4
f7ab6a50
PE
4998Into the directory containing the @acronym{GNU} Autoconf macros used
4999by the package---often called @file{m4}---copy the
5000@file{bison-i18n.m4} file installed by Bison under
5001@samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
5002For example:
5003
5004@example
5005cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
5006@end example
5007
5008@item
30757c8c
PE
5009@findex BISON_I18N
5010@vindex BISON_LOCALEDIR
5011@vindex YYENABLE_NLS
f7ab6a50
PE
5012In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
5013invocation, add an invocation of @code{BISON_I18N}. This macro is
5014defined in the file @file{bison-i18n.m4} that you copied earlier. It
5015causes @samp{configure} to find the value of the
30757c8c
PE
5016@code{BISON_LOCALEDIR} variable, and it defines the source-language
5017symbol @code{YYENABLE_NLS} to enable translations in the
5018Bison-generated parser.
f7ab6a50
PE
5019
5020@item
5021In the @code{main} function of your program, designate the directory
5022containing Bison's runtime message catalog, through a call to
5023@samp{bindtextdomain} with domain name @samp{bison-runtime}.
5024For example:
5025
5026@example
5027bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
5028@end example
5029
5030Typically this appears after any other call @code{bindtextdomain
5031(PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
5032@samp{BISON_LOCALEDIR} to be defined as a string through the
5033@file{Makefile}.
5034
5035@item
5036In the @file{Makefile.am} that controls the compilation of the @code{main}
5037function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
5038either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
5039
5040@example
5041DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5042@end example
5043
5044or:
5045
5046@example
5047AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5048@end example
5049
5050@item
5051Finally, invoke the command @command{autoreconf} to generate the build
5052infrastructure.
5053@end enumerate
5054
bfa74976 5055
342b8b6e 5056@node Algorithm
13863333
AD
5057@chapter The Bison Parser Algorithm
5058@cindex Bison parser algorithm
bfa74976
RS
5059@cindex algorithm of parser
5060@cindex shifting
5061@cindex reduction
5062@cindex parser stack
5063@cindex stack, parser
5064
5065As Bison reads tokens, it pushes them onto a stack along with their
5066semantic values. The stack is called the @dfn{parser stack}. Pushing a
5067token is traditionally called @dfn{shifting}.
5068
5069For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
5070@samp{3} to come. The stack will have four elements, one for each token
5071that was shifted.
5072
5073But the stack does not always have an element for each token read. When
5074the last @var{n} tokens and groupings shifted match the components of a
5075grammar rule, they can be combined according to that rule. This is called
5076@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
5077single grouping whose symbol is the result (left hand side) of that rule.
5078Running the rule's action is part of the process of reduction, because this
5079is what computes the semantic value of the resulting grouping.
5080
5081For example, if the infix calculator's parser stack contains this:
5082
5083@example
50841 + 5 * 3
5085@end example
5086
5087@noindent
5088and the next input token is a newline character, then the last three
5089elements can be reduced to 15 via the rule:
5090
5091@example
5092expr: expr '*' expr;
5093@end example
5094
5095@noindent
5096Then the stack contains just these three elements:
5097
5098@example
50991 + 15
5100@end example
5101
5102@noindent
5103At this point, another reduction can be made, resulting in the single value
510416. Then the newline token can be shifted.
5105
5106The parser tries, by shifts and reductions, to reduce the entire input down
5107to a single grouping whose symbol is the grammar's start-symbol
5108(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
5109
5110This kind of parser is known in the literature as a bottom-up parser.
5111
5112@menu
5113* Look-Ahead:: Parser looks one token ahead when deciding what to do.
5114* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
5115* Precedence:: Operator precedence works by resolving conflicts.
5116* Contextual Precedence:: When an operator's precedence depends on context.
5117* Parser States:: The parser is a finite-state-machine with stack.
5118* Reduce/Reduce:: When two rules are applicable in the same situation.
5119* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 5120* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 5121* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
5122@end menu
5123
342b8b6e 5124@node Look-Ahead
bfa74976
RS
5125@section Look-Ahead Tokens
5126@cindex look-ahead token
5127
5128The Bison parser does @emph{not} always reduce immediately as soon as the
5129last @var{n} tokens and groupings match a rule. This is because such a
5130simple strategy is inadequate to handle most languages. Instead, when a
5131reduction is possible, the parser sometimes ``looks ahead'' at the next
5132token in order to decide what to do.
5133
5134When a token is read, it is not immediately shifted; first it becomes the
5135@dfn{look-ahead token}, which is not on the stack. Now the parser can
5136perform one or more reductions of tokens and groupings on the stack, while
5137the look-ahead token remains off to the side. When no more reductions
5138should take place, the look-ahead token is shifted onto the stack. This
5139does not mean that all possible reductions have been done; depending on the
5140token type of the look-ahead token, some rules may choose to delay their
5141application.
5142
5143Here is a simple case where look-ahead is needed. These three rules define
5144expressions which contain binary addition operators and postfix unary
5145factorial operators (@samp{!}), and allow parentheses for grouping.
5146
5147@example
5148@group
5149expr: term '+' expr
5150 | term
5151 ;
5152@end group
5153
5154@group
5155term: '(' expr ')'
5156 | term '!'
5157 | NUMBER
5158 ;
5159@end group
5160@end example
5161
5162Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
5163should be done? If the following token is @samp{)}, then the first three
5164tokens must be reduced to form an @code{expr}. This is the only valid
5165course, because shifting the @samp{)} would produce a sequence of symbols
5166@w{@code{term ')'}}, and no rule allows this.
5167
5168If the following token is @samp{!}, then it must be shifted immediately so
5169that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
5170parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
5171@code{expr}. It would then be impossible to shift the @samp{!} because
5172doing so would produce on the stack the sequence of symbols @code{expr
5173'!'}. No rule allows that sequence.
5174
5175@vindex yychar
32c29292
JD
5176@vindex yylval
5177@vindex yylloc
5178The look-ahead token is stored in the variable @code{yychar}.
5179Its semantic value and location, if any, are stored in the variables
5180@code{yylval} and @code{yylloc}.
bfa74976
RS
5181@xref{Action Features, ,Special Features for Use in Actions}.
5182
342b8b6e 5183@node Shift/Reduce
bfa74976
RS
5184@section Shift/Reduce Conflicts
5185@cindex conflicts
5186@cindex shift/reduce conflicts
5187@cindex dangling @code{else}
5188@cindex @code{else}, dangling
5189
5190Suppose we are parsing a language which has if-then and if-then-else
5191statements, with a pair of rules like this:
5192
5193@example
5194@group
5195if_stmt:
5196 IF expr THEN stmt
5197 | IF expr THEN stmt ELSE stmt
5198 ;
5199@end group
5200@end example
5201
5202@noindent
5203Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
5204terminal symbols for specific keyword tokens.
5205
5206When the @code{ELSE} token is read and becomes the look-ahead token, the
5207contents of the stack (assuming the input is valid) are just right for
5208reduction by the first rule. But it is also legitimate to shift the
5209@code{ELSE}, because that would lead to eventual reduction by the second
5210rule.
5211
5212This situation, where either a shift or a reduction would be valid, is
5213called a @dfn{shift/reduce conflict}. Bison is designed to resolve
5214these conflicts by choosing to shift, unless otherwise directed by
5215operator precedence declarations. To see the reason for this, let's
5216contrast it with the other alternative.
5217
5218Since the parser prefers to shift the @code{ELSE}, the result is to attach
5219the else-clause to the innermost if-statement, making these two inputs
5220equivalent:
5221
5222@example
5223if x then if y then win (); else lose;
5224
5225if x then do; if y then win (); else lose; end;
5226@end example
5227
5228But if the parser chose to reduce when possible rather than shift, the
5229result would be to attach the else-clause to the outermost if-statement,
5230making these two inputs equivalent:
5231
5232@example
5233if x then if y then win (); else lose;
5234
5235if x then do; if y then win (); end; else lose;
5236@end example
5237
5238The conflict exists because the grammar as written is ambiguous: either
5239parsing of the simple nested if-statement is legitimate. The established
5240convention is that these ambiguities are resolved by attaching the
5241else-clause to the innermost if-statement; this is what Bison accomplishes
5242by choosing to shift rather than reduce. (It would ideally be cleaner to
5243write an unambiguous grammar, but that is very hard to do in this case.)
5244This particular ambiguity was first encountered in the specifications of
5245Algol 60 and is called the ``dangling @code{else}'' ambiguity.
5246
5247To avoid warnings from Bison about predictable, legitimate shift/reduce
5248conflicts, use the @code{%expect @var{n}} declaration. There will be no
5249warning as long as the number of shift/reduce conflicts is exactly @var{n}.
5250@xref{Expect Decl, ,Suppressing Conflict Warnings}.
5251
5252The definition of @code{if_stmt} above is solely to blame for the
5253conflict, but the conflict does not actually appear without additional
5254rules. Here is a complete Bison input file that actually manifests the
5255conflict:
5256
5257@example
5258@group
5259%token IF THEN ELSE variable
5260%%
5261@end group
5262@group
5263stmt: expr
5264 | if_stmt
5265 ;
5266@end group
5267
5268@group
5269if_stmt:
5270 IF expr THEN stmt
5271 | IF expr THEN stmt ELSE stmt
5272 ;
5273@end group
5274
5275expr: variable
5276 ;
5277@end example
5278
342b8b6e 5279@node Precedence
bfa74976
RS
5280@section Operator Precedence
5281@cindex operator precedence
5282@cindex precedence of operators
5283
5284Another situation where shift/reduce conflicts appear is in arithmetic
5285expressions. Here shifting is not always the preferred resolution; the
5286Bison declarations for operator precedence allow you to specify when to
5287shift and when to reduce.
5288
5289@menu
5290* Why Precedence:: An example showing why precedence is needed.
5291* Using Precedence:: How to specify precedence in Bison grammars.
5292* Precedence Examples:: How these features are used in the previous example.
5293* How Precedence:: How they work.
5294@end menu
5295
342b8b6e 5296@node Why Precedence
bfa74976
RS
5297@subsection When Precedence is Needed
5298
5299Consider the following ambiguous grammar fragment (ambiguous because the
5300input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
5301
5302@example
5303@group
5304expr: expr '-' expr
5305 | expr '*' expr
5306 | expr '<' expr
5307 | '(' expr ')'
5308 @dots{}
5309 ;
5310@end group
5311@end example
5312
5313@noindent
5314Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
14ded682
AD
5315should it reduce them via the rule for the subtraction operator? It
5316depends on the next token. Of course, if the next token is @samp{)}, we
5317must reduce; shifting is invalid because no single rule can reduce the
5318token sequence @w{@samp{- 2 )}} or anything starting with that. But if
5319the next token is @samp{*} or @samp{<}, we have a choice: either
5320shifting or reduction would allow the parse to complete, but with
5321different results.
5322
5323To decide which one Bison should do, we must consider the results. If
5324the next operator token @var{op} is shifted, then it must be reduced
5325first in order to permit another opportunity to reduce the difference.
5326The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
5327hand, if the subtraction is reduced before shifting @var{op}, the result
5328is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
5329reduce should depend on the relative precedence of the operators
5330@samp{-} and @var{op}: @samp{*} should be shifted first, but not
5331@samp{<}.
bfa74976
RS
5332
5333@cindex associativity
5334What about input such as @w{@samp{1 - 2 - 5}}; should this be
14ded682
AD
5335@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
5336operators we prefer the former, which is called @dfn{left association}.
5337The latter alternative, @dfn{right association}, is desirable for
5338assignment operators. The choice of left or right association is a
5339matter of whether the parser chooses to shift or reduce when the stack
5340contains @w{@samp{1 - 2}} and the look-ahead token is @samp{-}: shifting
5341makes right-associativity.
bfa74976 5342
342b8b6e 5343@node Using Precedence
bfa74976
RS
5344@subsection Specifying Operator Precedence
5345@findex %left
5346@findex %right
5347@findex %nonassoc
5348
5349Bison allows you to specify these choices with the operator precedence
5350declarations @code{%left} and @code{%right}. Each such declaration
5351contains a list of tokens, which are operators whose precedence and
5352associativity is being declared. The @code{%left} declaration makes all
5353those operators left-associative and the @code{%right} declaration makes
5354them right-associative. A third alternative is @code{%nonassoc}, which
5355declares that it is a syntax error to find the same operator twice ``in a
5356row''.
5357
5358The relative precedence of different operators is controlled by the
5359order in which they are declared. The first @code{%left} or
5360@code{%right} declaration in the file declares the operators whose
5361precedence is lowest, the next such declaration declares the operators
5362whose precedence is a little higher, and so on.
5363
342b8b6e 5364@node Precedence Examples
bfa74976
RS
5365@subsection Precedence Examples
5366
5367In our example, we would want the following declarations:
5368
5369@example
5370%left '<'
5371%left '-'
5372%left '*'
5373@end example
5374
5375In a more complete example, which supports other operators as well, we
5376would declare them in groups of equal precedence. For example, @code{'+'} is
5377declared with @code{'-'}:
5378
5379@example
5380%left '<' '>' '=' NE LE GE
5381%left '+' '-'
5382%left '*' '/'
5383@end example
5384
5385@noindent
5386(Here @code{NE} and so on stand for the operators for ``not equal''
5387and so on. We assume that these tokens are more than one character long
5388and therefore are represented by names, not character literals.)
5389
342b8b6e 5390@node How Precedence
bfa74976
RS
5391@subsection How Precedence Works
5392
5393The first effect of the precedence declarations is to assign precedence
5394levels to the terminal symbols declared. The second effect is to assign
704a47c4
AD
5395precedence levels to certain rules: each rule gets its precedence from
5396the last terminal symbol mentioned in the components. (You can also
5397specify explicitly the precedence of a rule. @xref{Contextual
5398Precedence, ,Context-Dependent Precedence}.)
5399
5400Finally, the resolution of conflicts works by comparing the precedence
5401of the rule being considered with that of the look-ahead token. If the
5402token's precedence is higher, the choice is to shift. If the rule's
5403precedence is higher, the choice is to reduce. If they have equal
5404precedence, the choice is made based on the associativity of that
5405precedence level. The verbose output file made by @samp{-v}
5406(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
5407resolved.
bfa74976
RS
5408
5409Not all rules and not all tokens have precedence. If either the rule or
5410the look-ahead token has no precedence, then the default is to shift.
5411
342b8b6e 5412@node Contextual Precedence
bfa74976
RS
5413@section Context-Dependent Precedence
5414@cindex context-dependent precedence
5415@cindex unary operator precedence
5416@cindex precedence, context-dependent
5417@cindex precedence, unary operator
5418@findex %prec
5419
5420Often the precedence of an operator depends on the context. This sounds
5421outlandish at first, but it is really very common. For example, a minus
5422sign typically has a very high precedence as a unary operator, and a
5423somewhat lower precedence (lower than multiplication) as a binary operator.
5424
5425The Bison precedence declarations, @code{%left}, @code{%right} and
5426@code{%nonassoc}, can only be used once for a given token; so a token has
5427only one precedence declared in this way. For context-dependent
5428precedence, you need to use an additional mechanism: the @code{%prec}
e0c471a9 5429modifier for rules.
bfa74976
RS
5430
5431The @code{%prec} modifier declares the precedence of a particular rule by
5432specifying a terminal symbol whose precedence should be used for that rule.
5433It's not necessary for that symbol to appear otherwise in the rule. The
5434modifier's syntax is:
5435
5436@example
5437%prec @var{terminal-symbol}
5438@end example
5439
5440@noindent
5441and it is written after the components of the rule. Its effect is to
5442assign the rule the precedence of @var{terminal-symbol}, overriding
5443the precedence that would be deduced for it in the ordinary way. The
5444altered rule precedence then affects how conflicts involving that rule
5445are resolved (@pxref{Precedence, ,Operator Precedence}).
5446
5447Here is how @code{%prec} solves the problem of unary minus. First, declare
5448a precedence for a fictitious terminal symbol named @code{UMINUS}. There
5449are no tokens of this type, but the symbol serves to stand for its
5450precedence:
5451
5452@example
5453@dots{}
5454%left '+' '-'
5455%left '*'
5456%left UMINUS
5457@end example
5458
5459Now the precedence of @code{UMINUS} can be used in specific rules:
5460
5461@example
5462@group
5463exp: @dots{}
5464 | exp '-' exp
5465 @dots{}
5466 | '-' exp %prec UMINUS
5467@end group
5468@end example
5469
91d2c560 5470@ifset defaultprec
39a06c25
PE
5471If you forget to append @code{%prec UMINUS} to the rule for unary
5472minus, Bison silently assumes that minus has its usual precedence.
5473This kind of problem can be tricky to debug, since one typically
5474discovers the mistake only by testing the code.
5475
22fccf95 5476The @code{%no-default-prec;} declaration makes it easier to discover
39a06c25
PE
5477this kind of problem systematically. It causes rules that lack a
5478@code{%prec} modifier to have no precedence, even if the last terminal
5479symbol mentioned in their components has a declared precedence.
5480
22fccf95 5481If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
39a06c25
PE
5482for all rules that participate in precedence conflict resolution.
5483Then you will see any shift/reduce conflict until you tell Bison how
5484to resolve it, either by changing your grammar or by adding an
5485explicit precedence. This will probably add declarations to the
5486grammar, but it helps to protect against incorrect rule precedences.
5487
22fccf95
PE
5488The effect of @code{%no-default-prec;} can be reversed by giving
5489@code{%default-prec;}, which is the default.
91d2c560 5490@end ifset
39a06c25 5491
342b8b6e 5492@node Parser States
bfa74976
RS
5493@section Parser States
5494@cindex finite-state machine
5495@cindex parser state
5496@cindex state (of parser)
5497
5498The function @code{yyparse} is implemented using a finite-state machine.
5499The values pushed on the parser stack are not simply token type codes; they
5500represent the entire sequence of terminal and nonterminal symbols at or
5501near the top of the stack. The current state collects all the information
5502about previous input which is relevant to deciding what to do next.
5503
5504Each time a look-ahead token is read, the current parser state together
5505with the type of look-ahead token are looked up in a table. This table
5506entry can say, ``Shift the look-ahead token.'' In this case, it also
5507specifies the new parser state, which is pushed onto the top of the
5508parser stack. Or it can say, ``Reduce using rule number @var{n}.''
5509This means that a certain number of tokens or groupings are taken off
5510the top of the stack, and replaced by one grouping. In other words,
5511that number of states are popped from the stack, and one new state is
5512pushed.
5513
5514There is one other alternative: the table can say that the look-ahead token
5515is erroneous in the current state. This causes error processing to begin
5516(@pxref{Error Recovery}).
5517
342b8b6e 5518@node Reduce/Reduce
bfa74976
RS
5519@section Reduce/Reduce Conflicts
5520@cindex reduce/reduce conflict
5521@cindex conflicts, reduce/reduce
5522
5523A reduce/reduce conflict occurs if there are two or more rules that apply
5524to the same sequence of input. This usually indicates a serious error
5525in the grammar.
5526
5527For example, here is an erroneous attempt to define a sequence
5528of zero or more @code{word} groupings.
5529
5530@example
5531sequence: /* empty */
5532 @{ printf ("empty sequence\n"); @}
5533 | maybeword
5534 | sequence word
5535 @{ printf ("added word %s\n", $2); @}
5536 ;
5537
5538maybeword: /* empty */
5539 @{ printf ("empty maybeword\n"); @}
5540 | word
5541 @{ printf ("single word %s\n", $1); @}
5542 ;
5543@end example
5544
5545@noindent
5546The error is an ambiguity: there is more than one way to parse a single
5547@code{word} into a @code{sequence}. It could be reduced to a
5548@code{maybeword} and then into a @code{sequence} via the second rule.
5549Alternatively, nothing-at-all could be reduced into a @code{sequence}
5550via the first rule, and this could be combined with the @code{word}
5551using the third rule for @code{sequence}.
5552
5553There is also more than one way to reduce nothing-at-all into a
5554@code{sequence}. This can be done directly via the first rule,
5555or indirectly via @code{maybeword} and then the second rule.
5556
5557You might think that this is a distinction without a difference, because it
5558does not change whether any particular input is valid or not. But it does
5559affect which actions are run. One parsing order runs the second rule's
5560action; the other runs the first rule's action and the third rule's action.
5561In this example, the output of the program changes.
5562
5563Bison resolves a reduce/reduce conflict by choosing to use the rule that
5564appears first in the grammar, but it is very risky to rely on this. Every
5565reduce/reduce conflict must be studied and usually eliminated. Here is the
5566proper way to define @code{sequence}:
5567
5568@example
5569sequence: /* empty */
5570 @{ printf ("empty sequence\n"); @}
5571 | sequence word
5572 @{ printf ("added word %s\n", $2); @}
5573 ;
5574@end example
5575
5576Here is another common error that yields a reduce/reduce conflict:
5577
5578@example
5579sequence: /* empty */
5580 | sequence words
5581 | sequence redirects
5582 ;
5583
5584words: /* empty */
5585 | words word
5586 ;
5587
5588redirects:/* empty */
5589 | redirects redirect
5590 ;
5591@end example
5592
5593@noindent
5594The intention here is to define a sequence which can contain either
5595@code{word} or @code{redirect} groupings. The individual definitions of
5596@code{sequence}, @code{words} and @code{redirects} are error-free, but the
5597three together make a subtle ambiguity: even an empty input can be parsed
5598in infinitely many ways!
5599
5600Consider: nothing-at-all could be a @code{words}. Or it could be two
5601@code{words} in a row, or three, or any number. It could equally well be a
5602@code{redirects}, or two, or any number. Or it could be a @code{words}
5603followed by three @code{redirects} and another @code{words}. And so on.
5604
5605Here are two ways to correct these rules. First, to make it a single level
5606of sequence:
5607
5608@example
5609sequence: /* empty */
5610 | sequence word
5611 | sequence redirect
5612 ;
5613@end example
5614
5615Second, to prevent either a @code{words} or a @code{redirects}
5616from being empty:
5617
5618@example
5619sequence: /* empty */
5620 | sequence words
5621 | sequence redirects
5622 ;
5623
5624words: word
5625 | words word
5626 ;
5627
5628redirects:redirect
5629 | redirects redirect
5630 ;
5631@end example
5632
342b8b6e 5633@node Mystery Conflicts
bfa74976
RS
5634@section Mysterious Reduce/Reduce Conflicts
5635
5636Sometimes reduce/reduce conflicts can occur that don't look warranted.
5637Here is an example:
5638
5639@example
5640@group
5641%token ID
5642
5643%%
5644def: param_spec return_spec ','
5645 ;
5646param_spec:
5647 type
5648 | name_list ':' type
5649 ;
5650@end group
5651@group
5652return_spec:
5653 type
5654 | name ':' type
5655 ;
5656@end group
5657@group
5658type: ID
5659 ;
5660@end group
5661@group
5662name: ID
5663 ;
5664name_list:
5665 name
5666 | name ',' name_list
5667 ;
5668@end group
5669@end example
5670
5671It would seem that this grammar can be parsed with only a single token
13863333 5672of look-ahead: when a @code{param_spec} is being read, an @code{ID} is
bfa74976 5673a @code{name} if a comma or colon follows, or a @code{type} if another
c827f760 5674@code{ID} follows. In other words, this grammar is @acronym{LR}(1).
bfa74976 5675
c827f760
PE
5676@cindex @acronym{LR}(1)
5677@cindex @acronym{LALR}(1)
bfa74976 5678However, Bison, like most parser generators, cannot actually handle all
c827f760
PE
5679@acronym{LR}(1) grammars. In this grammar, two contexts, that after
5680an @code{ID}
bfa74976
RS
5681at the beginning of a @code{param_spec} and likewise at the beginning of
5682a @code{return_spec}, are similar enough that Bison assumes they are the
5683same. They appear similar because the same set of rules would be
5684active---the rule for reducing to a @code{name} and that for reducing to
5685a @code{type}. Bison is unable to determine at that stage of processing
5686that the rules would require different look-ahead tokens in the two
5687contexts, so it makes a single parser state for them both. Combining
5688the two contexts causes a conflict later. In parser terminology, this
c827f760 5689occurrence means that the grammar is not @acronym{LALR}(1).
bfa74976
RS
5690
5691In general, it is better to fix deficiencies than to document them. But
5692this particular deficiency is intrinsically hard to fix; parser
c827f760
PE
5693generators that can handle @acronym{LR}(1) grammars are hard to write
5694and tend to
bfa74976
RS
5695produce parsers that are very large. In practice, Bison is more useful
5696as it is now.
5697
5698When the problem arises, you can often fix it by identifying the two
a220f555
MA
5699parser states that are being confused, and adding something to make them
5700look distinct. In the above example, adding one rule to
bfa74976
RS
5701@code{return_spec} as follows makes the problem go away:
5702
5703@example
5704@group
5705%token BOGUS
5706@dots{}
5707%%
5708@dots{}
5709return_spec:
5710 type
5711 | name ':' type
5712 /* This rule is never used. */
5713 | ID BOGUS
5714 ;
5715@end group
5716@end example
5717
5718This corrects the problem because it introduces the possibility of an
5719additional active rule in the context after the @code{ID} at the beginning of
5720@code{return_spec}. This rule is not active in the corresponding context
5721in a @code{param_spec}, so the two contexts receive distinct parser states.
5722As long as the token @code{BOGUS} is never generated by @code{yylex},
5723the added rule cannot alter the way actual input is parsed.
5724
5725In this particular example, there is another way to solve the problem:
5726rewrite the rule for @code{return_spec} to use @code{ID} directly
5727instead of via @code{name}. This also causes the two confusing
5728contexts to have different sets of active rules, because the one for
5729@code{return_spec} activates the altered rule for @code{return_spec}
5730rather than the one for @code{name}.
5731
5732@example
5733param_spec:
5734 type
5735 | name_list ':' type
5736 ;
5737return_spec:
5738 type
5739 | ID ':' type
5740 ;
5741@end example
5742
e054b190
PE
5743For a more detailed exposition of @acronym{LALR}(1) parsers and parser
5744generators, please see:
5745Frank DeRemer and Thomas Pennello, Efficient Computation of
5746@acronym{LALR}(1) Look-Ahead Sets, @cite{@acronym{ACM} Transactions on
5747Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
5748pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
5749
fae437e8 5750@node Generalized LR Parsing
c827f760
PE
5751@section Generalized @acronym{LR} (@acronym{GLR}) Parsing
5752@cindex @acronym{GLR} parsing
5753@cindex generalized @acronym{LR} (@acronym{GLR}) parsing
676385e2 5754@cindex ambiguous grammars
9d9b8b70 5755@cindex nondeterministic parsing
676385e2 5756
fae437e8
AD
5757Bison produces @emph{deterministic} parsers that choose uniquely
5758when to reduce and which reduction to apply
8dd162d3 5759based on a summary of the preceding input and on one extra token of look-ahead.
676385e2
PH
5760As a result, normal Bison handles a proper subset of the family of
5761context-free languages.
fae437e8 5762Ambiguous grammars, since they have strings with more than one possible
676385e2
PH
5763sequence of reductions cannot have deterministic parsers in this sense.
5764The same is true of languages that require more than one symbol of
8dd162d3 5765look-ahead, since the parser lacks the information necessary to make a
676385e2 5766decision at the point it must be made in a shift-reduce parser.
fae437e8 5767Finally, as previously mentioned (@pxref{Mystery Conflicts}),
676385e2
PH
5768there are languages where Bison's particular choice of how to
5769summarize the input seen so far loses necessary information.
5770
5771When you use the @samp{%glr-parser} declaration in your grammar file,
5772Bison generates a parser that uses a different algorithm, called
c827f760
PE
5773Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR}
5774parser uses the same basic
676385e2
PH
5775algorithm for parsing as an ordinary Bison parser, but behaves
5776differently in cases where there is a shift-reduce conflict that has not
fae437e8 5777been resolved by precedence rules (@pxref{Precedence}) or a
c827f760
PE
5778reduce-reduce conflict. When a @acronym{GLR} parser encounters such a
5779situation, it
fae437e8 5780effectively @emph{splits} into a several parsers, one for each possible
676385e2
PH
5781shift or reduction. These parsers then proceed as usual, consuming
5782tokens in lock-step. Some of the stacks may encounter other conflicts
fae437e8 5783and split further, with the result that instead of a sequence of states,
c827f760 5784a Bison @acronym{GLR} parsing stack is what is in effect a tree of states.
676385e2
PH
5785
5786In effect, each stack represents a guess as to what the proper parse
5787is. Additional input may indicate that a guess was wrong, in which case
5788the appropriate stack silently disappears. Otherwise, the semantics
fae437e8 5789actions generated in each stack are saved, rather than being executed
676385e2 5790immediately. When a stack disappears, its saved semantic actions never
fae437e8 5791get executed. When a reduction causes two stacks to become equivalent,
676385e2
PH
5792their sets of semantic actions are both saved with the state that
5793results from the reduction. We say that two stacks are equivalent
fae437e8 5794when they both represent the same sequence of states,
676385e2
PH
5795and each pair of corresponding states represents a
5796grammar symbol that produces the same segment of the input token
5797stream.
5798
5799Whenever the parser makes a transition from having multiple
c827f760 5800states to having one, it reverts to the normal @acronym{LALR}(1) parsing
676385e2
PH
5801algorithm, after resolving and executing the saved-up actions.
5802At this transition, some of the states on the stack will have semantic
5803values that are sets (actually multisets) of possible actions. The
5804parser tries to pick one of the actions by first finding one whose rule
5805has the highest dynamic precedence, as set by the @samp{%dprec}
fae437e8 5806declaration. Otherwise, if the alternative actions are not ordered by
676385e2 5807precedence, but there the same merging function is declared for both
fae437e8 5808rules by the @samp{%merge} declaration,
676385e2
PH
5809Bison resolves and evaluates both and then calls the merge function on
5810the result. Otherwise, it reports an ambiguity.
5811
c827f760
PE
5812It is possible to use a data structure for the @acronym{GLR} parsing tree that
5813permits the processing of any @acronym{LALR}(1) grammar in linear time (in the
5814size of the input), any unambiguous (not necessarily
5815@acronym{LALR}(1)) grammar in
fae437e8 5816quadratic worst-case time, and any general (possibly ambiguous)
676385e2
PH
5817context-free grammar in cubic worst-case time. However, Bison currently
5818uses a simpler data structure that requires time proportional to the
5819length of the input times the maximum number of stacks required for any
9d9b8b70 5820prefix of the input. Thus, really ambiguous or nondeterministic
676385e2
PH
5821grammars can require exponential time and space to process. Such badly
5822behaving examples, however, are not generally of practical interest.
9d9b8b70 5823Usually, nondeterminism in a grammar is local---the parser is ``in
676385e2 5824doubt'' only for a few tokens at a time. Therefore, the current data
c827f760 5825structure should generally be adequate. On @acronym{LALR}(1) portions of a
676385e2
PH
5826grammar, in particular, it is only slightly slower than with the default
5827Bison parser.
5828
fa7e68c3 5829For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth
f6481e2f
PE
5830Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
5831Generalised @acronym{LR} Parsers, Royal Holloway, University of
5832London, Department of Computer Science, TR-00-12,
5833@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
5834(2000-12-24).
5835
1a059451
PE
5836@node Memory Management
5837@section Memory Management, and How to Avoid Memory Exhaustion
5838@cindex memory exhaustion
5839@cindex memory management
bfa74976
RS
5840@cindex stack overflow
5841@cindex parser stack overflow
5842@cindex overflow of parser stack
5843
1a059451 5844The Bison parser stack can run out of memory if too many tokens are shifted and
bfa74976 5845not reduced. When this happens, the parser function @code{yyparse}
1a059451 5846calls @code{yyerror} and then returns 2.
bfa74976 5847
c827f760 5848Because Bison parsers have growing stacks, hitting the upper limit
d1a1114f
AD
5849usually results from using a right recursion instead of a left
5850recursion, @xref{Recursion, ,Recursive Rules}.
5851
bfa74976
RS
5852@vindex YYMAXDEPTH
5853By defining the macro @code{YYMAXDEPTH}, you can control how deep the
1a059451 5854parser stack can become before memory is exhausted. Define the
bfa74976
RS
5855macro with a value that is an integer. This value is the maximum number
5856of tokens that can be shifted (and not reduced) before overflow.
bfa74976
RS
5857
5858The stack space allowed is not necessarily allocated. If you specify a
1a059451 5859large value for @code{YYMAXDEPTH}, the parser normally allocates a small
bfa74976
RS
5860stack at first, and then makes it bigger by stages as needed. This
5861increasing allocation happens automatically and silently. Therefore,
5862you do not need to make @code{YYMAXDEPTH} painfully small merely to save
5863space for ordinary inputs that do not need much stack.
5864
d7e14fc0
PE
5865However, do not allow @code{YYMAXDEPTH} to be a value so large that
5866arithmetic overflow could occur when calculating the size of the stack
5867space. Also, do not allow @code{YYMAXDEPTH} to be less than
5868@code{YYINITDEPTH}.
5869
bfa74976
RS
5870@cindex default stack limit
5871The default value of @code{YYMAXDEPTH}, if you do not define it, is
587210000.
5873
5874@vindex YYINITDEPTH
5875You can control how much stack is allocated initially by defining the
d7e14fc0
PE
5876macro @code{YYINITDEPTH} to a positive integer. For the C
5877@acronym{LALR}(1) parser, this value must be a compile-time constant
5878unless you are assuming C99 or some other target language or compiler
5879that allows variable-length arrays. The default is 200.
5880
1a059451 5881Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
bfa74976 5882
d1a1114f 5883@c FIXME: C++ output.
c827f760 5884Because of semantical differences between C and C++, the
1a059451
PE
5885@acronym{LALR}(1) parsers in C produced by Bison cannot grow when compiled
5886by C++ compilers. In this precise case (compiling a C parser as C++) you are
5887suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix
5888this deficiency in a future release.
d1a1114f 5889
342b8b6e 5890@node Error Recovery
bfa74976
RS
5891@chapter Error Recovery
5892@cindex error recovery
5893@cindex recovery from errors
5894
6e649e65 5895It is not usually acceptable to have a program terminate on a syntax
bfa74976
RS
5896error. For example, a compiler should recover sufficiently to parse the
5897rest of the input file and check it for errors; a calculator should accept
5898another expression.
5899
5900In a simple interactive command parser where each input is one line, it may
5901be sufficient to allow @code{yyparse} to return 1 on error and have the
5902caller ignore the rest of the input line when that happens (and then call
5903@code{yyparse} again). But this is inadequate for a compiler, because it
5904forgets all the syntactic context leading up to the error. A syntax error
5905deep within a function in the compiler input should not cause the compiler
5906to treat the following line like the beginning of a source file.
5907
5908@findex error
5909You can define how to recover from a syntax error by writing rules to
5910recognize the special token @code{error}. This is a terminal symbol that
5911is always defined (you need not declare it) and reserved for error
5912handling. The Bison parser generates an @code{error} token whenever a
5913syntax error happens; if you have provided a rule to recognize this token
13863333 5914in the current context, the parse can continue.
bfa74976
RS
5915
5916For example:
5917
5918@example
5919stmnts: /* empty string */
5920 | stmnts '\n'
5921 | stmnts exp '\n'
5922 | stmnts error '\n'
5923@end example
5924
5925The fourth rule in this example says that an error followed by a newline
5926makes a valid addition to any @code{stmnts}.
5927
5928What happens if a syntax error occurs in the middle of an @code{exp}? The
5929error recovery rule, interpreted strictly, applies to the precise sequence
5930of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
5931the middle of an @code{exp}, there will probably be some additional tokens
5932and subexpressions on the stack after the last @code{stmnts}, and there
5933will be tokens to read before the next newline. So the rule is not
5934applicable in the ordinary way.
5935
5936But Bison can force the situation to fit the rule, by discarding part of
72f889cc
AD
5937the semantic context and part of the input. First it discards states
5938and objects from the stack until it gets back to a state in which the
bfa74976 5939@code{error} token is acceptable. (This means that the subexpressions
72f889cc
AD
5940already parsed are discarded, back to the last complete @code{stmnts}.)
5941At this point the @code{error} token can be shifted. Then, if the old
bfa74976
RS
5942look-ahead token is not acceptable to be shifted next, the parser reads
5943tokens and discards them until it finds a token which is acceptable. In
72f889cc
AD
5944this example, Bison reads and discards input until the next newline so
5945that the fourth rule can apply. Note that discarded symbols are
5946possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
5947Discarded Symbols}, for a means to reclaim this memory.
bfa74976
RS
5948
5949The choice of error rules in the grammar is a choice of strategies for
5950error recovery. A simple and useful strategy is simply to skip the rest of
5951the current input line or current statement if an error is detected:
5952
5953@example
72d2299c 5954stmnt: error ';' /* On error, skip until ';' is read. */
bfa74976
RS
5955@end example
5956
5957It is also useful to recover to the matching close-delimiter of an
5958opening-delimiter that has already been parsed. Otherwise the
5959close-delimiter will probably appear to be unmatched, and generate another,
5960spurious error message:
5961
5962@example
5963primary: '(' expr ')'
5964 | '(' error ')'
5965 @dots{}
5966 ;
5967@end example
5968
5969Error recovery strategies are necessarily guesses. When they guess wrong,
5970one syntax error often leads to another. In the above example, the error
5971recovery rule guesses that an error is due to bad input within one
5972@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
5973middle of a valid @code{stmnt}. After the error recovery rule recovers
5974from the first error, another syntax error will be found straightaway,
5975since the text following the spurious semicolon is also an invalid
5976@code{stmnt}.
5977
5978To prevent an outpouring of error messages, the parser will output no error
5979message for another syntax error that happens shortly after the first; only
5980after three consecutive input tokens have been successfully shifted will
5981error messages resume.
5982
5983Note that rules which accept the @code{error} token may have actions, just
5984as any other rules can.
5985
5986@findex yyerrok
5987You can make error messages resume immediately by using the macro
5988@code{yyerrok} in an action. If you do this in the error rule's action, no
5989error messages will be suppressed. This macro requires no arguments;
5990@samp{yyerrok;} is a valid C statement.
5991
5992@findex yyclearin
5993The previous look-ahead token is reanalyzed immediately after an error. If
5994this is unacceptable, then the macro @code{yyclearin} may be used to clear
5995this token. Write the statement @samp{yyclearin;} in the error rule's
5996action.
32c29292 5997@xref{Action Features, ,Special Features for Use in Actions}.
bfa74976 5998
6e649e65 5999For example, suppose that on a syntax error, an error handling routine is
bfa74976
RS
6000called that advances the input stream to some point where parsing should
6001once again commence. The next symbol returned by the lexical scanner is
6002probably correct. The previous look-ahead token ought to be discarded
6003with @samp{yyclearin;}.
6004
6005@vindex YYRECOVERING
6006The macro @code{YYRECOVERING} stands for an expression that has the
6007value 1 when the parser is recovering from a syntax error, and 0 the
6008rest of the time. A value of 1 indicates that error messages are
6009currently suppressed for new syntax errors.
6010
342b8b6e 6011@node Context Dependency
bfa74976
RS
6012@chapter Handling Context Dependencies
6013
6014The Bison paradigm is to parse tokens first, then group them into larger
6015syntactic units. In many languages, the meaning of a token is affected by
6016its context. Although this violates the Bison paradigm, certain techniques
6017(known as @dfn{kludges}) may enable you to write Bison parsers for such
6018languages.
6019
6020@menu
6021* Semantic Tokens:: Token parsing can depend on the semantic context.
6022* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
6023* Tie-in Recovery:: Lexical tie-ins have implications for how
6024 error recovery rules must be written.
6025@end menu
6026
6027(Actually, ``kludge'' means any technique that gets its job done but is
6028neither clean nor robust.)
6029
342b8b6e 6030@node Semantic Tokens
bfa74976
RS
6031@section Semantic Info in Token Types
6032
6033The C language has a context dependency: the way an identifier is used
6034depends on what its current meaning is. For example, consider this:
6035
6036@example
6037foo (x);
6038@end example
6039
6040This looks like a function call statement, but if @code{foo} is a typedef
6041name, then this is actually a declaration of @code{x}. How can a Bison
6042parser for C decide how to parse this input?
6043
c827f760 6044The method used in @acronym{GNU} C is to have two different token types,
bfa74976
RS
6045@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
6046identifier, it looks up the current declaration of the identifier in order
6047to decide which token type to return: @code{TYPENAME} if the identifier is
6048declared as a typedef, @code{IDENTIFIER} otherwise.
6049
6050The grammar rules can then express the context dependency by the choice of
6051token type to recognize. @code{IDENTIFIER} is accepted as an expression,
6052but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
6053@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
6054is @emph{not} significant, such as in declarations that can shadow a
6055typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
6056accepted---there is one rule for each of the two token types.
6057
6058This technique is simple to use if the decision of which kinds of
6059identifiers to allow is made at a place close to where the identifier is
6060parsed. But in C this is not always so: C allows a declaration to
6061redeclare a typedef name provided an explicit type has been specified
6062earlier:
6063
6064@example
3a4f411f
PE
6065typedef int foo, bar;
6066int baz (void)
6067@{
6068 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
6069 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
6070 return foo (bar);
6071@}
bfa74976
RS
6072@end example
6073
6074Unfortunately, the name being declared is separated from the declaration
6075construct itself by a complicated syntactic structure---the ``declarator''.
6076
9ecbd125 6077As a result, part of the Bison parser for C needs to be duplicated, with
14ded682
AD
6078all the nonterminal names changed: once for parsing a declaration in
6079which a typedef name can be redefined, and once for parsing a
6080declaration in which that can't be done. Here is a part of the
6081duplication, with actions omitted for brevity:
bfa74976
RS
6082
6083@example
6084initdcl:
6085 declarator maybeasm '='
6086 init
6087 | declarator maybeasm
6088 ;
6089
6090notype_initdcl:
6091 notype_declarator maybeasm '='
6092 init
6093 | notype_declarator maybeasm
6094 ;
6095@end example
6096
6097@noindent
6098Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
6099cannot. The distinction between @code{declarator} and
6100@code{notype_declarator} is the same sort of thing.
6101
6102There is some similarity between this technique and a lexical tie-in
6103(described next), in that information which alters the lexical analysis is
6104changed during parsing by other parts of the program. The difference is
6105here the information is global, and is used for other purposes in the
6106program. A true lexical tie-in has a special-purpose flag controlled by
6107the syntactic context.
6108
342b8b6e 6109@node Lexical Tie-ins
bfa74976
RS
6110@section Lexical Tie-ins
6111@cindex lexical tie-in
6112
6113One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
6114which is set by Bison actions, whose purpose is to alter the way tokens are
6115parsed.
6116
6117For example, suppose we have a language vaguely like C, but with a special
6118construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
6119an expression in parentheses in which all integers are hexadecimal. In
6120particular, the token @samp{a1b} must be treated as an integer rather than
6121as an identifier if it appears in that context. Here is how you can do it:
6122
6123@example
6124@group
6125%@{
38a92d50
PE
6126 int hexflag;
6127 int yylex (void);
6128 void yyerror (char const *);
bfa74976
RS
6129%@}
6130%%
6131@dots{}
6132@end group
6133@group
6134expr: IDENTIFIER
6135 | constant
6136 | HEX '('
6137 @{ hexflag = 1; @}
6138 expr ')'
6139 @{ hexflag = 0;
6140 $$ = $4; @}
6141 | expr '+' expr
6142 @{ $$ = make_sum ($1, $3); @}
6143 @dots{}
6144 ;
6145@end group
6146
6147@group
6148constant:
6149 INTEGER
6150 | STRING
6151 ;
6152@end group
6153@end example
6154
6155@noindent
6156Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
6157it is nonzero, all integers are parsed in hexadecimal, and tokens starting
6158with letters are parsed as integers if possible.
6159
342b8b6e
AD
6160The declaration of @code{hexflag} shown in the prologue of the parser file
6161is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}).
75f5aaea 6162You must also write the code in @code{yylex} to obey the flag.
bfa74976 6163
342b8b6e 6164@node Tie-in Recovery
bfa74976
RS
6165@section Lexical Tie-ins and Error Recovery
6166
6167Lexical tie-ins make strict demands on any error recovery rules you have.
6168@xref{Error Recovery}.
6169
6170The reason for this is that the purpose of an error recovery rule is to
6171abort the parsing of one construct and resume in some larger construct.
6172For example, in C-like languages, a typical error recovery rule is to skip
6173tokens until the next semicolon, and then start a new statement, like this:
6174
6175@example
6176stmt: expr ';'
6177 | IF '(' expr ')' stmt @{ @dots{} @}
6178 @dots{}
6179 error ';'
6180 @{ hexflag = 0; @}
6181 ;
6182@end example
6183
6184If there is a syntax error in the middle of a @samp{hex (@var{expr})}
6185construct, this error rule will apply, and then the action for the
6186completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
6187remain set for the entire rest of the input, or until the next @code{hex}
6188keyword, causing identifiers to be misinterpreted as integers.
6189
6190To avoid this problem the error recovery rule itself clears @code{hexflag}.
6191
6192There may also be an error recovery rule that works within expressions.
6193For example, there could be a rule which applies within parentheses
6194and skips to the close-parenthesis:
6195
6196@example
6197@group
6198expr: @dots{}
6199 | '(' expr ')'
6200 @{ $$ = $2; @}
6201 | '(' error ')'
6202 @dots{}
6203@end group
6204@end example
6205
6206If this rule acts within the @code{hex} construct, it is not going to abort
6207that construct (since it applies to an inner level of parentheses within
6208the construct). Therefore, it should not clear the flag: the rest of
6209the @code{hex} construct should be parsed with the flag still in effect.
6210
6211What if there is an error recovery rule which might abort out of the
6212@code{hex} construct or might not, depending on circumstances? There is no
6213way you can write the action to determine whether a @code{hex} construct is
6214being aborted or not. So if you are using a lexical tie-in, you had better
6215make sure your error recovery rules are not of this kind. Each rule must
6216be such that you can be sure that it always will, or always won't, have to
6217clear the flag.
6218
ec3bc396
AD
6219@c ================================================== Debugging Your Parser
6220
342b8b6e 6221@node Debugging
bfa74976 6222@chapter Debugging Your Parser
ec3bc396
AD
6223
6224Developing a parser can be a challenge, especially if you don't
6225understand the algorithm (@pxref{Algorithm, ,The Bison Parser
6226Algorithm}). Even so, sometimes a detailed description of the automaton
6227can help (@pxref{Understanding, , Understanding Your Parser}), or
6228tracing the execution of the parser can give some insight on why it
6229behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
6230
6231@menu
6232* Understanding:: Understanding the structure of your parser.
6233* Tracing:: Tracing the execution of your parser.
6234@end menu
6235
6236@node Understanding
6237@section Understanding Your Parser
6238
6239As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
6240Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
6241frequent than one would hope), looking at this automaton is required to
6242tune or simply fix a parser. Bison provides two different
c827f760 6243representation of it, either textually or graphically (as a @acronym{VCG}
ec3bc396
AD
6244file).
6245
6246The textual file is generated when the options @option{--report} or
6247@option{--verbose} are specified, see @xref{Invocation, , Invoking
6248Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
6249the parser output file name, and adding @samp{.output} instead.
6250Therefore, if the input file is @file{foo.y}, then the parser file is
6251called @file{foo.tab.c} by default. As a consequence, the verbose
6252output file is called @file{foo.output}.
6253
6254The following grammar file, @file{calc.y}, will be used in the sequel:
6255
6256@example
6257%token NUM STR
6258%left '+' '-'
6259%left '*'
6260%%
6261exp: exp '+' exp
6262 | exp '-' exp
6263 | exp '*' exp
6264 | exp '/' exp
6265 | NUM
6266 ;
6267useless: STR;
6268%%
6269@end example
6270
88bce5a2
AD
6271@command{bison} reports:
6272
6273@example
6274calc.y: warning: 1 useless nonterminal and 1 useless rule
6275calc.y:11.1-7: warning: useless nonterminal: useless
5a99098d
PE
6276calc.y:11.10-12: warning: useless rule: useless: STR
6277calc.y: conflicts: 7 shift/reduce
88bce5a2
AD
6278@end example
6279
6280When given @option{--report=state}, in addition to @file{calc.tab.c}, it
6281creates a file @file{calc.output} with contents detailed below. The
6282order of the output and the exact presentation might vary, but the
6283interpretation is the same.
ec3bc396
AD
6284
6285The first section includes details on conflicts that were solved thanks
6286to precedence and/or associativity:
6287
6288@example
6289Conflict in state 8 between rule 2 and token '+' resolved as reduce.
6290Conflict in state 8 between rule 2 and token '-' resolved as reduce.
6291Conflict in state 8 between rule 2 and token '*' resolved as shift.
6292@exdent @dots{}
6293@end example
6294
6295@noindent
6296The next section lists states that still have conflicts.
6297
6298@example
5a99098d
PE
6299State 8 conflicts: 1 shift/reduce
6300State 9 conflicts: 1 shift/reduce
6301State 10 conflicts: 1 shift/reduce
6302State 11 conflicts: 4 shift/reduce
ec3bc396
AD
6303@end example
6304
6305@noindent
6306@cindex token, useless
6307@cindex useless token
6308@cindex nonterminal, useless
6309@cindex useless nonterminal
6310@cindex rule, useless
6311@cindex useless rule
6312The next section reports useless tokens, nonterminal and rules. Useless
6313nonterminals and rules are removed in order to produce a smaller parser,
6314but useless tokens are preserved, since they might be used by the
6315scanner (note the difference between ``useless'' and ``not used''
6316below):
6317
6318@example
6319Useless nonterminals:
6320 useless
6321
6322Terminals which are not used:
6323 STR
6324
6325Useless rules:
6326#6 useless: STR;
6327@end example
6328
6329@noindent
6330The next section reproduces the exact grammar that Bison used:
6331
6332@example
6333Grammar
6334
6335 Number, Line, Rule
88bce5a2 6336 0 5 $accept -> exp $end
ec3bc396
AD
6337 1 5 exp -> exp '+' exp
6338 2 6 exp -> exp '-' exp
6339 3 7 exp -> exp '*' exp
6340 4 8 exp -> exp '/' exp
6341 5 9 exp -> NUM
6342@end example
6343
6344@noindent
6345and reports the uses of the symbols:
6346
6347@example
6348Terminals, with rules where they appear
6349
88bce5a2 6350$end (0) 0
ec3bc396
AD
6351'*' (42) 3
6352'+' (43) 1
6353'-' (45) 2
6354'/' (47) 4
6355error (256)
6356NUM (258) 5
6357
6358Nonterminals, with rules where they appear
6359
88bce5a2 6360$accept (8)
ec3bc396
AD
6361 on left: 0
6362exp (9)
6363 on left: 1 2 3 4 5, on right: 0 1 2 3 4
6364@end example
6365
6366@noindent
6367@cindex item
6368@cindex pointed rule
6369@cindex rule, pointed
6370Bison then proceeds onto the automaton itself, describing each state
6371with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
6372item is a production rule together with a point (marked by @samp{.})
6373that the input cursor.
6374
6375@example
6376state 0
6377
88bce5a2 6378 $accept -> . exp $ (rule 0)
ec3bc396 6379
2a8d363a 6380 NUM shift, and go to state 1
ec3bc396 6381
2a8d363a 6382 exp go to state 2
ec3bc396
AD
6383@end example
6384
6385This reads as follows: ``state 0 corresponds to being at the very
6386beginning of the parsing, in the initial rule, right before the start
6387symbol (here, @code{exp}). When the parser returns to this state right
6388after having reduced a rule that produced an @code{exp}, the control
6389flow jumps to state 2. If there is no such transition on a nonterminal
8dd162d3 6390symbol, and the look-ahead is a @code{NUM}, then this token is shifted on
ec3bc396 6391the parse stack, and the control flow jumps to state 1. Any other
8dd162d3 6392look-ahead triggers a syntax error.''
ec3bc396
AD
6393
6394@cindex core, item set
6395@cindex item set core
6396@cindex kernel, item set
6397@cindex item set core
6398Even though the only active rule in state 0 seems to be rule 0, the
8dd162d3 6399report lists @code{NUM} as a look-ahead token because @code{NUM} can be
ec3bc396
AD
6400at the beginning of any rule deriving an @code{exp}. By default Bison
6401reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
6402you want to see more detail you can invoke @command{bison} with
6403@option{--report=itemset} to list all the items, include those that can
6404be derived:
6405
6406@example
6407state 0
6408
88bce5a2 6409 $accept -> . exp $ (rule 0)
ec3bc396
AD
6410 exp -> . exp '+' exp (rule 1)
6411 exp -> . exp '-' exp (rule 2)
6412 exp -> . exp '*' exp (rule 3)
6413 exp -> . exp '/' exp (rule 4)
6414 exp -> . NUM (rule 5)
6415
6416 NUM shift, and go to state 1
6417
6418 exp go to state 2
6419@end example
6420
6421@noindent
6422In the state 1...
6423
6424@example
6425state 1
6426
6427 exp -> NUM . (rule 5)
6428
2a8d363a 6429 $default reduce using rule 5 (exp)
ec3bc396
AD
6430@end example
6431
6432@noindent
8dd162d3 6433the rule 5, @samp{exp: NUM;}, is completed. Whatever the look-ahead token
ec3bc396
AD
6434(@samp{$default}), the parser will reduce it. If it was coming from
6435state 0, then, after this reduction it will return to state 0, and will
6436jump to state 2 (@samp{exp: go to state 2}).
6437
6438@example
6439state 2
6440
88bce5a2 6441 $accept -> exp . $ (rule 0)
ec3bc396
AD
6442 exp -> exp . '+' exp (rule 1)
6443 exp -> exp . '-' exp (rule 2)
6444 exp -> exp . '*' exp (rule 3)
6445 exp -> exp . '/' exp (rule 4)
6446
2a8d363a
AD
6447 $ shift, and go to state 3
6448 '+' shift, and go to state 4
6449 '-' shift, and go to state 5
6450 '*' shift, and go to state 6
6451 '/' shift, and go to state 7
ec3bc396
AD
6452@end example
6453
6454@noindent
6455In state 2, the automaton can only shift a symbol. For instance,
8dd162d3 6456because of the item @samp{exp -> exp . '+' exp}, if the look-ahead if
ec3bc396
AD
6457@samp{+}, it will be shifted on the parse stack, and the automaton
6458control will jump to state 4, corresponding to the item @samp{exp -> exp
6459'+' . exp}. Since there is no default action, any other token than
6e649e65 6460those listed above will trigger a syntax error.
ec3bc396
AD
6461
6462The state 3 is named the @dfn{final state}, or the @dfn{accepting
6463state}:
6464
6465@example
6466state 3
6467
88bce5a2 6468 $accept -> exp $ . (rule 0)
ec3bc396 6469
2a8d363a 6470 $default accept
ec3bc396
AD
6471@end example
6472
6473@noindent
6474the initial rule is completed (the start symbol and the end
6475of input were read), the parsing exits successfully.
6476
6477The interpretation of states 4 to 7 is straightforward, and is left to
6478the reader.
6479
6480@example
6481state 4
6482
6483 exp -> exp '+' . exp (rule 1)
6484
2a8d363a 6485 NUM shift, and go to state 1
ec3bc396 6486
2a8d363a 6487 exp go to state 8
ec3bc396
AD
6488
6489state 5
6490
6491 exp -> exp '-' . exp (rule 2)
6492
2a8d363a 6493 NUM shift, and go to state 1
ec3bc396 6494
2a8d363a 6495 exp go to state 9
ec3bc396
AD
6496
6497state 6
6498
6499 exp -> exp '*' . exp (rule 3)
6500
2a8d363a 6501 NUM shift, and go to state 1
ec3bc396 6502
2a8d363a 6503 exp go to state 10
ec3bc396
AD
6504
6505state 7
6506
6507 exp -> exp '/' . exp (rule 4)
6508
2a8d363a 6509 NUM shift, and go to state 1
ec3bc396 6510
2a8d363a 6511 exp go to state 11
ec3bc396
AD
6512@end example
6513
5a99098d
PE
6514As was announced in beginning of the report, @samp{State 8 conflicts:
65151 shift/reduce}:
ec3bc396
AD
6516
6517@example
6518state 8
6519
6520 exp -> exp . '+' exp (rule 1)
6521 exp -> exp '+' exp . (rule 1)
6522 exp -> exp . '-' exp (rule 2)
6523 exp -> exp . '*' exp (rule 3)
6524 exp -> exp . '/' exp (rule 4)
6525
2a8d363a
AD
6526 '*' shift, and go to state 6
6527 '/' shift, and go to state 7
ec3bc396 6528
2a8d363a
AD
6529 '/' [reduce using rule 1 (exp)]
6530 $default reduce using rule 1 (exp)
ec3bc396
AD
6531@end example
6532
8dd162d3 6533Indeed, there are two actions associated to the look-ahead @samp{/}:
ec3bc396
AD
6534either shifting (and going to state 7), or reducing rule 1. The
6535conflict means that either the grammar is ambiguous, or the parser lacks
6536information to make the right decision. Indeed the grammar is
6537ambiguous, as, since we did not specify the precedence of @samp{/}, the
6538sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
6539NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
6540NUM}, which corresponds to reducing rule 1.
6541
c827f760 6542Because in @acronym{LALR}(1) parsing a single decision can be made, Bison
ec3bc396
AD
6543arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
6544Shift/Reduce Conflicts}. Discarded actions are reported in between
6545square brackets.
6546
6547Note that all the previous states had a single possible action: either
6548shifting the next token and going to the corresponding state, or
6549reducing a single rule. In the other cases, i.e., when shifting
6550@emph{and} reducing is possible or when @emph{several} reductions are
8dd162d3
PE
6551possible, the look-ahead is required to select the action. State 8 is
6552one such state: if the look-ahead is @samp{*} or @samp{/} then the action
ec3bc396
AD
6553is shifting, otherwise the action is reducing rule 1. In other words,
6554the first two items, corresponding to rule 1, are not eligible when the
8dd162d3
PE
6555look-ahead token is @samp{*}, since we specified that @samp{*} has higher
6556precedence than @samp{+}. More generally, some items are eligible only
6557with some set of possible look-ahead tokens. When run with
6558@option{--report=look-ahead}, Bison specifies these look-ahead tokens:
ec3bc396
AD
6559
6560@example
6561state 8
6562
6563 exp -> exp . '+' exp [$, '+', '-', '/'] (rule 1)
6564 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
6565 exp -> exp . '-' exp (rule 2)
6566 exp -> exp . '*' exp (rule 3)
6567 exp -> exp . '/' exp (rule 4)
6568
6569 '*' shift, and go to state 6
6570 '/' shift, and go to state 7
6571
6572 '/' [reduce using rule 1 (exp)]
6573 $default reduce using rule 1 (exp)
6574@end example
6575
6576The remaining states are similar:
6577
6578@example
6579state 9
6580
6581 exp -> exp . '+' exp (rule 1)
6582 exp -> exp . '-' exp (rule 2)
6583 exp -> exp '-' exp . (rule 2)
6584 exp -> exp . '*' exp (rule 3)
6585 exp -> exp . '/' exp (rule 4)
6586
2a8d363a
AD
6587 '*' shift, and go to state 6
6588 '/' shift, and go to state 7
ec3bc396 6589
2a8d363a
AD
6590 '/' [reduce using rule 2 (exp)]
6591 $default reduce using rule 2 (exp)
ec3bc396
AD
6592
6593state 10
6594
6595 exp -> exp . '+' exp (rule 1)
6596 exp -> exp . '-' exp (rule 2)
6597 exp -> exp . '*' exp (rule 3)
6598 exp -> exp '*' exp . (rule 3)
6599 exp -> exp . '/' exp (rule 4)
6600
2a8d363a 6601 '/' shift, and go to state 7
ec3bc396 6602
2a8d363a
AD
6603 '/' [reduce using rule 3 (exp)]
6604 $default reduce using rule 3 (exp)
ec3bc396
AD
6605
6606state 11
6607
6608 exp -> exp . '+' exp (rule 1)
6609 exp -> exp . '-' exp (rule 2)
6610 exp -> exp . '*' exp (rule 3)
6611 exp -> exp . '/' exp (rule 4)
6612 exp -> exp '/' exp . (rule 4)
6613
2a8d363a
AD
6614 '+' shift, and go to state 4
6615 '-' shift, and go to state 5
6616 '*' shift, and go to state 6
6617 '/' shift, and go to state 7
ec3bc396 6618
2a8d363a
AD
6619 '+' [reduce using rule 4 (exp)]
6620 '-' [reduce using rule 4 (exp)]
6621 '*' [reduce using rule 4 (exp)]
6622 '/' [reduce using rule 4 (exp)]
6623 $default reduce using rule 4 (exp)
ec3bc396
AD
6624@end example
6625
6626@noindent
fa7e68c3
PE
6627Observe that state 11 contains conflicts not only due to the lack of
6628precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
6629@samp{*}, but also because the
ec3bc396
AD
6630associativity of @samp{/} is not specified.
6631
6632
6633@node Tracing
6634@section Tracing Your Parser
bfa74976
RS
6635@findex yydebug
6636@cindex debugging
6637@cindex tracing the parser
6638
6639If a Bison grammar compiles properly but doesn't do what you want when it
6640runs, the @code{yydebug} parser-trace feature can help you figure out why.
6641
3ded9a63
AD
6642There are several means to enable compilation of trace facilities:
6643
6644@table @asis
6645@item the macro @code{YYDEBUG}
6646@findex YYDEBUG
6647Define the macro @code{YYDEBUG} to a nonzero value when you compile the
c827f760 6648parser. This is compliant with @acronym{POSIX} Yacc. You could use
3ded9a63
AD
6649@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
6650YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
6651Prologue}).
6652
6653@item the option @option{-t}, @option{--debug}
6654Use the @samp{-t} option when you run Bison (@pxref{Invocation,
c827f760 6655,Invoking Bison}). This is @acronym{POSIX} compliant too.
3ded9a63
AD
6656
6657@item the directive @samp{%debug}
6658@findex %debug
6659Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison
6660Declaration Summary}). This is a Bison extension, which will prove
6661useful when Bison will output parsers for languages that don't use a
c827f760
PE
6662preprocessor. Unless @acronym{POSIX} and Yacc portability matter to
6663you, this is
3ded9a63
AD
6664the preferred solution.
6665@end table
6666
6667We suggest that you always enable the debug option so that debugging is
6668always possible.
bfa74976 6669
02a81e05 6670The trace facility outputs messages with macro calls of the form
e2742e46 6671@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
02a81e05 6672@var{format} and @var{args} are the usual @code{printf} format and
4947ebdb
PE
6673arguments. If you define @code{YYDEBUG} to a nonzero value but do not
6674define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
e4e1a4dc 6675and @code{YYPRINTF} is defined to @code{fprintf}.
bfa74976
RS
6676
6677Once you have compiled the program with trace facilities, the way to
6678request a trace is to store a nonzero value in the variable @code{yydebug}.
6679You can do this by making the C code do it (in @code{main}, perhaps), or
6680you can alter the value with a C debugger.
6681
6682Each step taken by the parser when @code{yydebug} is nonzero produces a
6683line or two of trace information, written on @code{stderr}. The trace
6684messages tell you these things:
6685
6686@itemize @bullet
6687@item
6688Each time the parser calls @code{yylex}, what kind of token was read.
6689
6690@item
6691Each time a token is shifted, the depth and complete contents of the
6692state stack (@pxref{Parser States}).
6693
6694@item
6695Each time a rule is reduced, which rule it is, and the complete contents
6696of the state stack afterward.
6697@end itemize
6698
6699To make sense of this information, it helps to refer to the listing file
704a47c4
AD
6700produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
6701Bison}). This file shows the meaning of each state in terms of
6702positions in various rules, and also what each state will do with each
6703possible input token. As you read the successive trace messages, you
6704can see that the parser is functioning according to its specification in
6705the listing file. Eventually you will arrive at the place where
6706something undesirable happens, and you will see which parts of the
6707grammar are to blame.
bfa74976
RS
6708
6709The parser file is a C program and you can use C debuggers on it, but it's
6710not easy to interpret what it is doing. The parser function is a
6711finite-state machine interpreter, and aside from the actions it executes
6712the same code over and over. Only the values of variables show where in
6713the grammar it is working.
6714
6715@findex YYPRINT
6716The debugging information normally gives the token type of each token
6717read, but not its semantic value. You can optionally define a macro
6718named @code{YYPRINT} to provide a way to print the value. If you define
6719@code{YYPRINT}, it should take three arguments. The parser will pass a
6720standard I/O stream, the numeric code for the token type, and the token
6721value (from @code{yylval}).
6722
6723Here is an example of @code{YYPRINT} suitable for the multi-function
6724calculator (@pxref{Mfcalc Decl, ,Declarations for @code{mfcalc}}):
6725
6726@smallexample
38a92d50
PE
6727%@{
6728 static void print_token_value (FILE *, int, YYSTYPE);
6729 #define YYPRINT(file, type, value) print_token_value (file, type, value)
6730%@}
6731
6732@dots{} %% @dots{} %% @dots{}
bfa74976
RS
6733
6734static void
831d3c99 6735print_token_value (FILE *file, int type, YYSTYPE value)
bfa74976
RS
6736@{
6737 if (type == VAR)
d3c4e709 6738 fprintf (file, "%s", value.tptr->name);
bfa74976 6739 else if (type == NUM)
d3c4e709 6740 fprintf (file, "%d", value.val);
bfa74976
RS
6741@}
6742@end smallexample
6743
ec3bc396
AD
6744@c ================================================= Invoking Bison
6745
342b8b6e 6746@node Invocation
bfa74976
RS
6747@chapter Invoking Bison
6748@cindex invoking Bison
6749@cindex Bison invocation
6750@cindex options for invoking Bison
6751
6752The usual way to invoke Bison is as follows:
6753
6754@example
6755bison @var{infile}
6756@end example
6757
6758Here @var{infile} is the grammar file name, which usually ends in
6759@samp{.y}. The parser file's name is made by replacing the @samp{.y}
fa4d969f
PE
6760with @samp{.tab.c} and removing any leading directory. Thus, the
6761@samp{bison foo.y} file name yields
6762@file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields
6763@file{foo.tab.c}. It's also possible, in case you are writing
79282c6c 6764C++ code instead of C in your grammar file, to name it @file{foo.ypp}
72d2299c
PE
6765or @file{foo.y++}. Then, the output files will take an extension like
6766the given one as input (respectively @file{foo.tab.cpp} and
6767@file{foo.tab.c++}).
fa4d969f 6768This feature takes effect with all options that manipulate file names like
234a3be3
AD
6769@samp{-o} or @samp{-d}.
6770
6771For example :
6772
6773@example
6774bison -d @var{infile.yxx}
6775@end example
84163231 6776@noindent
72d2299c 6777will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
234a3be3
AD
6778
6779@example
b56471a6 6780bison -d -o @var{output.c++} @var{infile.y}
234a3be3 6781@end example
84163231 6782@noindent
234a3be3
AD
6783will produce @file{output.c++} and @file{outfile.h++}.
6784
397ec073
PE
6785For compatibility with @acronym{POSIX}, the standard Bison
6786distribution also contains a shell script called @command{yacc} that
6787invokes Bison with the @option{-y} option.
6788
bfa74976 6789@menu
13863333 6790* Bison Options:: All the options described in detail,
c827f760 6791 in alphabetical order by short options.
bfa74976 6792* Option Cross Key:: Alphabetical list of long options.
93dd49ab 6793* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
bfa74976
RS
6794@end menu
6795
342b8b6e 6796@node Bison Options
bfa74976
RS
6797@section Bison Options
6798
6799Bison supports both traditional single-letter options and mnemonic long
6800option names. Long option names are indicated with @samp{--} instead of
6801@samp{-}. Abbreviations for option names are allowed as long as they
6802are unique. When a long option takes an argument, like
6803@samp{--file-prefix}, connect the option name and the argument with
6804@samp{=}.
6805
6806Here is a list of options that can be used with Bison, alphabetized by
6807short option. It is followed by a cross key alphabetized by long
6808option.
6809
89cab50d
AD
6810@c Please, keep this ordered as in `bison --help'.
6811@noindent
6812Operations modes:
6813@table @option
6814@item -h
6815@itemx --help
6816Print a summary of the command-line options to Bison and exit.
bfa74976 6817
89cab50d
AD
6818@item -V
6819@itemx --version
6820Print the version number of Bison and exit.
bfa74976 6821
f7ab6a50
PE
6822@item --print-localedir
6823Print the name of the directory containing locale-dependent data.
6824
89cab50d
AD
6825@item -y
6826@itemx --yacc
54662697
PE
6827Act more like the traditional Yacc command. This can cause
6828different diagnostics to be generated, and may change behavior in
6829other minor ways. Most importantly, imitate Yacc's output
6830file name conventions, so that the parser output file is called
89cab50d 6831@file{y.tab.c}, and the other outputs are called @file{y.output} and
54662697 6832@file{y.tab.h}. Thus, the following shell script can substitute
397ec073
PE
6833for Yacc, and the Bison distribution contains such a script for
6834compatibility with @acronym{POSIX}:
bfa74976 6835
89cab50d 6836@example
397ec073 6837#! /bin/sh
26e06a21 6838bison -y "$@@"
89cab50d 6839@end example
54662697
PE
6840
6841The @option{-y}/@option{--yacc} option is intended for use with
6842traditional Yacc grammars. If your grammar uses a Bison extension
6843like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
6844this option is specified.
6845
89cab50d
AD
6846@end table
6847
6848@noindent
6849Tuning the parser:
6850
6851@table @option
cd5bd6ac
AD
6852@item -S @var{file}
6853@itemx --skeleton=@var{file}
6854Specify the skeleton to use. You probably don't need this option unless
6855you are developing Bison.
6856
89cab50d
AD
6857@item -t
6858@itemx --debug
4947ebdb
PE
6859In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
6860already defined, so that the debugging facilities are compiled.
ec3bc396 6861@xref{Tracing, ,Tracing Your Parser}.
89cab50d
AD
6862
6863@item --locations
d8988b2f 6864Pretend that @code{%locations} was specified. @xref{Decl Summary}.
89cab50d
AD
6865
6866@item -p @var{prefix}
6867@itemx --name-prefix=@var{prefix}
d8988b2f
AD
6868Pretend that @code{%name-prefix="@var{prefix}"} was specified.
6869@xref{Decl Summary}.
bfa74976
RS
6870
6871@item -l
6872@itemx --no-lines
6873Don't put any @code{#line} preprocessor commands in the parser file.
6874Ordinarily Bison puts them in the parser file so that the C compiler
6875and debuggers will associate errors with your source file, the
6876grammar file. This option causes them to associate errors with the
95e742f7 6877parser file, treating it as an independent source file in its own right.
bfa74976 6878
931c7513
RS
6879@item -n
6880@itemx --no-parser
d8988b2f 6881Pretend that @code{%no-parser} was specified. @xref{Decl Summary}.
931c7513 6882
89cab50d
AD
6883@item -k
6884@itemx --token-table
d8988b2f 6885Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
89cab50d 6886@end table
bfa74976 6887
89cab50d
AD
6888@noindent
6889Adjust the output:
bfa74976 6890
89cab50d
AD
6891@table @option
6892@item -d
d8988b2f
AD
6893@itemx --defines
6894Pretend that @code{%defines} was specified, i.e., write an extra output
6deb4447 6895file containing macro definitions for the token type names defined in
4bfd5e4e 6896the grammar, as well as a few other declarations. @xref{Decl Summary}.
931c7513 6897
342b8b6e 6898@item --defines=@var{defines-file}
d8988b2f 6899Same as above, but save in the file @var{defines-file}.
342b8b6e 6900
89cab50d
AD
6901@item -b @var{file-prefix}
6902@itemx --file-prefix=@var{prefix}
aa08666d 6903Pretend that @code{%file-prefix} was specified, i.e, specify prefix to use
72d2299c 6904for all Bison output file names. @xref{Decl Summary}.
bfa74976 6905
ec3bc396
AD
6906@item -r @var{things}
6907@itemx --report=@var{things}
6908Write an extra output file containing verbose description of the comma
6909separated list of @var{things} among:
6910
6911@table @code
6912@item state
6913Description of the grammar, conflicts (resolved and unresolved), and
c827f760 6914@acronym{LALR} automaton.
ec3bc396 6915
8dd162d3 6916@item look-ahead
ec3bc396 6917Implies @code{state} and augments the description of the automaton with
8dd162d3 6918each rule's look-ahead set.
ec3bc396
AD
6919
6920@item itemset
6921Implies @code{state} and augments the description of the automaton with
6922the full set of items for each state, instead of its core only.
6923@end table
6924
bfa74976
RS
6925@item -v
6926@itemx --verbose
6deb4447
AD
6927Pretend that @code{%verbose} was specified, i.e, write an extra output
6928file containing verbose descriptions of the grammar and
72d2299c 6929parser. @xref{Decl Summary}.
bfa74976 6930
fa4d969f
PE
6931@item -o @var{file}
6932@itemx --output=@var{file}
6933Specify the @var{file} for the parser file.
bfa74976 6934
fa4d969f 6935The other output files' names are constructed from @var{file} as
d8988b2f 6936described under the @samp{-v} and @samp{-d} options.
342b8b6e
AD
6937
6938@item -g
c827f760
PE
6939Output a @acronym{VCG} definition of the @acronym{LALR}(1) grammar
6940automaton computed by Bison. If the grammar file is @file{foo.y}, the
6941@acronym{VCG} output file will
342b8b6e
AD
6942be @file{foo.vcg}.
6943
6944@item --graph=@var{graph-file}
72d2299c
PE
6945The behavior of @var{--graph} is the same than @samp{-g}. The only
6946difference is that it has an optional argument which is the name of
fa4d969f 6947the output graph file.
bfa74976
RS
6948@end table
6949
342b8b6e 6950@node Option Cross Key
bfa74976
RS
6951@section Option Cross Key
6952
aa08666d 6953@c FIXME: How about putting the directives too?
bfa74976
RS
6954Here is a list of options, alphabetized by long option, to help you find
6955the corresponding short option.
6956
aa08666d
AD
6957@multitable {@option{--defines=@var{defines-file}}} {@option{-b @var{file-prefix}XXX}}
6958@headitem Long Option @tab Short Option
6959@item @option{--debug} @tab @option{-t}
6960@item @option{--defines=@var{defines-file}} @tab @option{-d}
6961@item @option{--file-prefix=@var{prefix}} @tab @option{-b @var{file-prefix}}
6962@item @option{--graph=@var{graph-file}} @tab @option{-d}
6963@item @option{--help} @tab @option{-h}
6964@item @option{--name-prefix=@var{prefix}} @tab @option{-p @var{name-prefix}}
6965@item @option{--no-lines} @tab @option{-l}
6966@item @option{--no-parser} @tab @option{-n}
6967@item @option{--output=@var{outfile}} @tab @option{-o @var{outfile}}
6968@item @option{--print-localedir} @tab
6969@item @option{--token-table} @tab @option{-k}
6970@item @option{--verbose} @tab @option{-v}
6971@item @option{--version} @tab @option{-V}
6972@item @option{--yacc} @tab @option{-y}
6973@end multitable
bfa74976 6974
93dd49ab
PE
6975@node Yacc Library
6976@section Yacc Library
6977
6978The Yacc library contains default implementations of the
6979@code{yyerror} and @code{main} functions. These default
6980implementations are normally not useful, but @acronym{POSIX} requires
6981them. To use the Yacc library, link your program with the
6982@option{-ly} option. Note that Bison's implementation of the Yacc
6983library is distributed under the terms of the @acronym{GNU} General
6984Public License (@pxref{Copying}).
6985
6986If you use the Yacc library's @code{yyerror} function, you should
6987declare @code{yyerror} as follows:
6988
6989@example
6990int yyerror (char const *);
6991@end example
6992
6993Bison ignores the @code{int} value returned by this @code{yyerror}.
6994If you use the Yacc library's @code{main} function, your
6995@code{yyparse} function should have the following type signature:
6996
6997@example
6998int yyparse (void);
6999@end example
7000
12545799
AD
7001@c ================================================= C++ Bison
7002
7003@node C++ Language Interface
7004@chapter C++ Language Interface
7005
7006@menu
7007* C++ Parsers:: The interface to generate C++ parser classes
7008* A Complete C++ Example:: Demonstrating their use
7009@end menu
7010
7011@node C++ Parsers
7012@section C++ Parsers
7013
7014@menu
7015* C++ Bison Interface:: Asking for C++ parser generation
7016* C++ Semantic Values:: %union vs. C++
7017* C++ Location Values:: The position and location classes
7018* C++ Parser Interface:: Instantiating and running the parser
7019* C++ Scanner Interface:: Exchanges between yylex and parse
7020@end menu
7021
7022@node C++ Bison Interface
7023@subsection C++ Bison Interface
7024@c - %skeleton "lalr1.cc"
7025@c - Always pure
7026@c - initial action
7027
aa08666d
AD
7028The C++ parser @acronym{LALR}(1) skeleton is named @file{lalr1.cc}. To
7029select it, you may either pass the option @option{--skeleton=lalr1.cc}
7030to Bison, or include the directive @samp{%skeleton "lalr1.cc"} in the
12545799 7031grammar preamble. When run, @command{bison} will create several
aa08666d
AD
7032entities in the @samp{yy} namespace. Use the @samp{%name-prefix}
7033directive to change the namespace name, see @ref{Decl Summary}. The
7034various classes are generated in the following files:
7035
12545799
AD
7036@table @file
7037@item position.hh
7038@itemx location.hh
7039The definition of the classes @code{position} and @code{location},
7040used for location tracking. @xref{C++ Location Values}.
7041
7042@item stack.hh
7043An auxiliary class @code{stack} used by the parser.
7044
fa4d969f
PE
7045@item @var{file}.hh
7046@itemx @var{file}.cc
cd8b5791
AD
7047(Assuming the extension of the input file was @samp{.yy}.) The
7048declaration and implementation of the C++ parser class. The basename
7049and extension of these two files follow the same rules as with regular C
7050parsers (@pxref{Invocation}).
12545799 7051
cd8b5791
AD
7052The header is @emph{mandatory}; you must either pass
7053@option{-d}/@option{--defines} to @command{bison}, or use the
12545799
AD
7054@samp{%defines} directive.
7055@end table
7056
7057All these files are documented using Doxygen; run @command{doxygen}
7058for a complete and accurate documentation.
7059
7060@node C++ Semantic Values
7061@subsection C++ Semantic Values
7062@c - No objects in unions
7063@c - YSTYPE
7064@c - Printer and destructor
7065
7066The @code{%union} directive works as for C, see @ref{Union Decl, ,The
7067Collection of Value Types}. In particular it produces a genuine
7068@code{union}@footnote{In the future techniques to allow complex types
fb9712a9
AD
7069within pseudo-unions (similar to Boost variants) might be implemented to
7070alleviate these issues.}, which have a few specific features in C++.
12545799
AD
7071@itemize @minus
7072@item
fb9712a9
AD
7073The type @code{YYSTYPE} is defined but its use is discouraged: rather
7074you should refer to the parser's encapsulated type
7075@code{yy::parser::semantic_type}.
12545799
AD
7076@item
7077Non POD (Plain Old Data) types cannot be used. C++ forbids any
7078instance of classes with constructors in unions: only @emph{pointers}
7079to such objects are allowed.
7080@end itemize
7081
7082Because objects have to be stored via pointers, memory is not
7083reclaimed automatically: using the @code{%destructor} directive is the
7084only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
7085Symbols}.
7086
7087
7088@node C++ Location Values
7089@subsection C++ Location Values
7090@c - %locations
7091@c - class Position
7092@c - class Location
b47dbebe 7093@c - %define "filename_type" "const symbol::Symbol"
12545799
AD
7094
7095When the directive @code{%locations} is used, the C++ parser supports
7096location tracking, see @ref{Locations, , Locations Overview}. Two
7097auxiliary classes define a @code{position}, a single point in a file,
7098and a @code{location}, a range composed of a pair of
7099@code{position}s (possibly spanning several files).
7100
fa4d969f 7101@deftypemethod {position} {std::string*} file
12545799
AD
7102The name of the file. It will always be handled as a pointer, the
7103parser will never duplicate nor deallocate it. As an experimental
7104feature you may change it to @samp{@var{type}*} using @samp{%define
b47dbebe 7105"filename_type" "@var{type}"}.
12545799
AD
7106@end deftypemethod
7107
7108@deftypemethod {position} {unsigned int} line
7109The line, starting at 1.
7110@end deftypemethod
7111
7112@deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
7113Advance by @var{height} lines, resetting the column number.
7114@end deftypemethod
7115
7116@deftypemethod {position} {unsigned int} column
7117The column, starting at 0.
7118@end deftypemethod
7119
7120@deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
7121Advance by @var{width} columns, without changing the line number.
7122@end deftypemethod
7123
7124@deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
7125@deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
7126@deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
7127@deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
7128Various forms of syntactic sugar for @code{columns}.
7129@end deftypemethod
7130
7131@deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
7132Report @var{p} on @var{o} like this:
fa4d969f
PE
7133@samp{@var{file}:@var{line}.@var{column}}, or
7134@samp{@var{line}.@var{column}} if @var{file} is null.
12545799
AD
7135@end deftypemethod
7136
7137@deftypemethod {location} {position} begin
7138@deftypemethodx {location} {position} end
7139The first, inclusive, position of the range, and the first beyond.
7140@end deftypemethod
7141
7142@deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
7143@deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
7144Advance the @code{end} position.
7145@end deftypemethod
7146
7147@deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
7148@deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
7149@deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
7150Various forms of syntactic sugar.
7151@end deftypemethod
7152
7153@deftypemethod {location} {void} step ()
7154Move @code{begin} onto @code{end}.
7155@end deftypemethod
7156
7157
7158@node C++ Parser Interface
7159@subsection C++ Parser Interface
7160@c - define parser_class_name
7161@c - Ctor
7162@c - parse, error, set_debug_level, debug_level, set_debug_stream,
7163@c debug_stream.
7164@c - Reporting errors
7165
7166The output files @file{@var{output}.hh} and @file{@var{output}.cc}
7167declare and define the parser class in the namespace @code{yy}. The
7168class name defaults to @code{parser}, but may be changed using
7169@samp{%define "parser_class_name" "@var{name}"}. The interface of
9d9b8b70 7170this class is detailed below. It can be extended using the
12545799
AD
7171@code{%parse-param} feature: its semantics is slightly changed since
7172it describes an additional member of the parser class, and an
7173additional argument for its constructor.
7174
8a0adb01
AD
7175@defcv {Type} {parser} {semantic_value_type}
7176@defcvx {Type} {parser} {location_value_type}
12545799 7177The types for semantics value and locations.
8a0adb01 7178@end defcv
12545799
AD
7179
7180@deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
7181Build a new parser object. There are no arguments by default, unless
7182@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
7183@end deftypemethod
7184
7185@deftypemethod {parser} {int} parse ()
7186Run the syntactic analysis, and return 0 on success, 1 otherwise.
7187@end deftypemethod
7188
7189@deftypemethod {parser} {std::ostream&} debug_stream ()
7190@deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
7191Get or set the stream used for tracing the parsing. It defaults to
7192@code{std::cerr}.
7193@end deftypemethod
7194
7195@deftypemethod {parser} {debug_level_type} debug_level ()
7196@deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
7197Get or set the tracing level. Currently its value is either 0, no trace,
9d9b8b70 7198or nonzero, full tracing.
12545799
AD
7199@end deftypemethod
7200
7201@deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
7202The definition for this member function must be supplied by the user:
7203the parser uses it to report a parser error occurring at @var{l},
7204described by @var{m}.
7205@end deftypemethod
7206
7207
7208@node C++ Scanner Interface
7209@subsection C++ Scanner Interface
7210@c - prefix for yylex.
7211@c - Pure interface to yylex
7212@c - %lex-param
7213
7214The parser invokes the scanner by calling @code{yylex}. Contrary to C
7215parsers, C++ parsers are always pure: there is no point in using the
7216@code{%pure-parser} directive. Therefore the interface is as follows.
7217
7218@deftypemethod {parser} {int} yylex (semantic_value_type& @var{yylval}, location_type& @var{yylloc}, @var{type1} @var{arg1}, ...)
7219Return the next token. Its type is the return value, its semantic
7220value and location being @var{yylval} and @var{yylloc}. Invocations of
7221@samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
7222@end deftypemethod
7223
7224
7225@node A Complete C++ Example
7226@section A Complete C++ Example
7227
7228This section demonstrates the use of a C++ parser with a simple but
7229complete example. This example should be available on your system,
7230ready to compile, in the directory @dfn{../bison/examples/calc++}. It
7231focuses on the use of Bison, therefore the design of the various C++
7232classes is very naive: no accessors, no encapsulation of members etc.
7233We will use a Lex scanner, and more precisely, a Flex scanner, to
7234demonstrate the various interaction. A hand written scanner is
7235actually easier to interface with.
7236
7237@menu
7238* Calc++ --- C++ Calculator:: The specifications
7239* Calc++ Parsing Driver:: An active parsing context
7240* Calc++ Parser:: A parser class
7241* Calc++ Scanner:: A pure C++ Flex scanner
7242* Calc++ Top Level:: Conducting the band
7243@end menu
7244
7245@node Calc++ --- C++ Calculator
7246@subsection Calc++ --- C++ Calculator
7247
7248Of course the grammar is dedicated to arithmetics, a single
9d9b8b70 7249expression, possibly preceded by variable assignments. An
12545799
AD
7250environment containing possibly predefined variables such as
7251@code{one} and @code{two}, is exchanged with the parser. An example
7252of valid input follows.
7253
7254@example
7255three := 3
7256seven := one + two * three
7257seven * seven
7258@end example
7259
7260@node Calc++ Parsing Driver
7261@subsection Calc++ Parsing Driver
7262@c - An env
7263@c - A place to store error messages
7264@c - A place for the result
7265
7266To support a pure interface with the parser (and the scanner) the
7267technique of the ``parsing context'' is convenient: a structure
7268containing all the data to exchange. Since, in addition to simply
7269launch the parsing, there are several auxiliary tasks to execute (open
7270the file for parsing, instantiate the parser etc.), we recommend
7271transforming the simple parsing context structure into a fully blown
7272@dfn{parsing driver} class.
7273
7274The declaration of this driver class, @file{calc++-driver.hh}, is as
7275follows. The first part includes the CPP guard and imports the
fb9712a9
AD
7276required standard library components, and the declaration of the parser
7277class.
12545799 7278
1c59e0a1 7279@comment file: calc++-driver.hh
12545799
AD
7280@example
7281#ifndef CALCXX_DRIVER_HH
7282# define CALCXX_DRIVER_HH
7283# include <string>
7284# include <map>
fb9712a9 7285# include "calc++-parser.hh"
12545799
AD
7286@end example
7287
12545799
AD
7288
7289@noindent
7290Then comes the declaration of the scanning function. Flex expects
7291the signature of @code{yylex} to be defined in the macro
7292@code{YY_DECL}, and the C++ parser expects it to be declared. We can
7293factor both as follows.
1c59e0a1
AD
7294
7295@comment file: calc++-driver.hh
12545799
AD
7296@example
7297// Announce to Flex the prototype we want for lexing function, ...
c095d689
AD
7298# define YY_DECL \
7299 yy::calcxx_parser::token_type \
7300 yylex (yy::calcxx_parser::semantic_type* yylval, \
7301 yy::calcxx_parser::location_type* yylloc, \
7302 calcxx_driver& driver)
12545799
AD
7303// ... and declare it for the parser's sake.
7304YY_DECL;
7305@end example
7306
7307@noindent
7308The @code{calcxx_driver} class is then declared with its most obvious
7309members.
7310
1c59e0a1 7311@comment file: calc++-driver.hh
12545799
AD
7312@example
7313// Conducting the whole scanning and parsing of Calc++.
7314class calcxx_driver
7315@{
7316public:
7317 calcxx_driver ();
7318 virtual ~calcxx_driver ();
7319
7320 std::map<std::string, int> variables;
7321
7322 int result;
7323@end example
7324
7325@noindent
7326To encapsulate the coordination with the Flex scanner, it is useful to
7327have two members function to open and close the scanning phase.
7328members.
7329
1c59e0a1 7330@comment file: calc++-driver.hh
12545799
AD
7331@example
7332 // Handling the scanner.
7333 void scan_begin ();
7334 void scan_end ();
7335 bool trace_scanning;
7336@end example
7337
7338@noindent
7339Similarly for the parser itself.
7340
1c59e0a1 7341@comment file: calc++-driver.hh
12545799
AD
7342@example
7343 // Handling the parser.
7344 void parse (const std::string& f);
7345 std::string file;
7346 bool trace_parsing;
7347@end example
7348
7349@noindent
7350To demonstrate pure handling of parse errors, instead of simply
7351dumping them on the standard error output, we will pass them to the
7352compiler driver using the following two member functions. Finally, we
7353close the class declaration and CPP guard.
7354
1c59e0a1 7355@comment file: calc++-driver.hh
12545799
AD
7356@example
7357 // Error handling.
7358 void error (const yy::location& l, const std::string& m);
7359 void error (const std::string& m);
7360@};
7361#endif // ! CALCXX_DRIVER_HH
7362@end example
7363
7364The implementation of the driver is straightforward. The @code{parse}
7365member function deserves some attention. The @code{error} functions
7366are simple stubs, they should actually register the located error
7367messages and set error state.
7368
1c59e0a1 7369@comment file: calc++-driver.cc
12545799
AD
7370@example
7371#include "calc++-driver.hh"
7372#include "calc++-parser.hh"
7373
7374calcxx_driver::calcxx_driver ()
7375 : trace_scanning (false), trace_parsing (false)
7376@{
7377 variables["one"] = 1;
7378 variables["two"] = 2;
7379@}
7380
7381calcxx_driver::~calcxx_driver ()
7382@{
7383@}
7384
7385void
7386calcxx_driver::parse (const std::string &f)
7387@{
7388 file = f;
7389 scan_begin ();
7390 yy::calcxx_parser parser (*this);
7391 parser.set_debug_level (trace_parsing);
7392 parser.parse ();
7393 scan_end ();
7394@}
7395
7396void
7397calcxx_driver::error (const yy::location& l, const std::string& m)
7398@{
7399 std::cerr << l << ": " << m << std::endl;
7400@}
7401
7402void
7403calcxx_driver::error (const std::string& m)
7404@{
7405 std::cerr << m << std::endl;
7406@}
7407@end example
7408
7409@node Calc++ Parser
7410@subsection Calc++ Parser
7411
b50d2359
AD
7412The parser definition file @file{calc++-parser.yy} starts by asking for
7413the C++ LALR(1) skeleton, the creation of the parser header file, and
7414specifies the name of the parser class. Because the C++ skeleton
7415changed several times, it is safer to require the version you designed
7416the grammar for.
1c59e0a1
AD
7417
7418@comment file: calc++-parser.yy
12545799
AD
7419@example
7420%skeleton "lalr1.cc" /* -*- C++ -*- */
b50d2359 7421%require "2.1a"
12545799 7422%defines
fb9712a9
AD
7423%define "parser_class_name" "calcxx_parser"
7424@end example
7425
7426@noindent
7427Then come the declarations/inclusions needed to define the
7428@code{%union}. Because the parser uses the parsing driver and
7429reciprocally, both cannot include the header of the other. Because the
7430driver's header needs detailed knowledge about the parser class (in
7431particular its inner types), it is the parser's header which will simply
7432use a forward declaration of the driver.
7433
7434@comment file: calc++-parser.yy
7435@example
12545799
AD
7436%@{
7437# include <string>
fb9712a9 7438class calcxx_driver;
12545799
AD
7439%@}
7440@end example
7441
7442@noindent
7443The driver is passed by reference to the parser and to the scanner.
7444This provides a simple but effective pure interface, not relying on
7445global variables.
7446
1c59e0a1 7447@comment file: calc++-parser.yy
12545799
AD
7448@example
7449// The parsing context.
7450%parse-param @{ calcxx_driver& driver @}
7451%lex-param @{ calcxx_driver& driver @}
7452@end example
7453
7454@noindent
7455Then we request the location tracking feature, and initialize the
7456first location's file name. Afterwards new locations are computed
7457relatively to the previous locations: the file name will be
7458automatically propagated.
7459
1c59e0a1 7460@comment file: calc++-parser.yy
12545799
AD
7461@example
7462%locations
7463%initial-action
7464@{
7465 // Initialize the initial location.
b47dbebe 7466 @@$.begin.filename = @@$.end.filename = &driver.file;
12545799
AD
7467@};
7468@end example
7469
7470@noindent
7471Use the two following directives to enable parser tracing and verbose
7472error messages.
7473
1c59e0a1 7474@comment file: calc++-parser.yy
12545799
AD
7475@example
7476%debug
7477%error-verbose
7478@end example
7479
7480@noindent
7481Semantic values cannot use ``real'' objects, but only pointers to
7482them.
7483
1c59e0a1 7484@comment file: calc++-parser.yy
12545799
AD
7485@example
7486// Symbols.
7487%union
7488@{
7489 int ival;
7490 std::string *sval;
7491@};
7492@end example
7493
fb9712a9
AD
7494@noindent
7495The code between @samp{%@{} and @samp{%@}} after the introduction of the
7496@samp{%union} is output in the @file{*.cc} file; it needs detailed
7497knowledge about the driver.
7498
7499@comment file: calc++-parser.yy
7500@example
7501%@{
7502# include "calc++-driver.hh"
7503%@}
7504@end example
7505
7506
12545799
AD
7507@noindent
7508The token numbered as 0 corresponds to end of file; the following line
7509allows for nicer error messages referring to ``end of file'' instead
7510of ``$end''. Similarly user friendly named are provided for each
7511symbol. Note that the tokens names are prefixed by @code{TOKEN_} to
7512avoid name clashes.
7513
1c59e0a1 7514@comment file: calc++-parser.yy
12545799 7515@example
fb9712a9
AD
7516%token END 0 "end of file"
7517%token ASSIGN ":="
7518%token <sval> IDENTIFIER "identifier"
7519%token <ival> NUMBER "number"
7520%type <ival> exp "expression"
12545799
AD
7521@end example
7522
7523@noindent
7524To enable memory deallocation during error recovery, use
7525@code{%destructor}.
7526
287c78f6 7527@c FIXME: Document %printer, and mention that it takes a braced-code operand.
1c59e0a1 7528@comment file: calc++-parser.yy
12545799
AD
7529@example
7530%printer @{ debug_stream () << *$$; @} "identifier"
7531%destructor @{ delete $$; @} "identifier"
7532
7533%printer @{ debug_stream () << $$; @} "number" "expression"
7534@end example
7535
7536@noindent
7537The grammar itself is straightforward.
7538
1c59e0a1 7539@comment file: calc++-parser.yy
12545799
AD
7540@example
7541%%
7542%start unit;
7543unit: assignments exp @{ driver.result = $2; @};
7544
7545assignments: assignments assignment @{@}
9d9b8b70 7546 | /* Nothing. */ @{@};
12545799 7547
fb9712a9 7548assignment: "identifier" ":=" exp @{ driver.variables[*$1] = $3; @};
12545799
AD
7549
7550%left '+' '-';
7551%left '*' '/';
7552exp: exp '+' exp @{ $$ = $1 + $3; @}
7553 | exp '-' exp @{ $$ = $1 - $3; @}
7554 | exp '*' exp @{ $$ = $1 * $3; @}
7555 | exp '/' exp @{ $$ = $1 / $3; @}
fb9712a9
AD
7556 | "identifier" @{ $$ = driver.variables[*$1]; @}
7557 | "number" @{ $$ = $1; @};
12545799
AD
7558%%
7559@end example
7560
7561@noindent
7562Finally the @code{error} member function registers the errors to the
7563driver.
7564
1c59e0a1 7565@comment file: calc++-parser.yy
12545799
AD
7566@example
7567void
1c59e0a1
AD
7568yy::calcxx_parser::error (const yy::calcxx_parser::location_type& l,
7569 const std::string& m)
12545799
AD
7570@{
7571 driver.error (l, m);
7572@}
7573@end example
7574
7575@node Calc++ Scanner
7576@subsection Calc++ Scanner
7577
7578The Flex scanner first includes the driver declaration, then the
7579parser's to get the set of defined tokens.
7580
1c59e0a1 7581@comment file: calc++-scanner.ll
12545799
AD
7582@example
7583%@{ /* -*- C++ -*- */
04098407
PE
7584# include <cstdlib>
7585# include <errno.h>
7586# include <limits.h>
12545799
AD
7587# include <string>
7588# include "calc++-driver.hh"
7589# include "calc++-parser.hh"
eaea13f5
PE
7590
7591/* Work around an incompatibility in flex (at least versions
7592 2.5.31 through 2.5.33): it generates code that does
7593 not conform to C89. See Debian bug 333231
7594 <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */
7870f699
PE
7595# undef yywrap
7596# define yywrap() 1
eaea13f5 7597
c095d689
AD
7598/* By default yylex returns int, we use token_type.
7599 Unfortunately yyterminate by default returns 0, which is
7600 not of token_type. */
8c5b881d 7601#define yyterminate() return token::END
12545799
AD
7602%@}
7603@end example
7604
7605@noindent
7606Because there is no @code{#include}-like feature we don't need
7607@code{yywrap}, we don't need @code{unput} either, and we parse an
7608actual file, this is not an interactive session with the user.
7609Finally we enable the scanner tracing features.
7610
1c59e0a1 7611@comment file: calc++-scanner.ll
12545799
AD
7612@example
7613%option noyywrap nounput batch debug
7614@end example
7615
7616@noindent
7617Abbreviations allow for more readable rules.
7618
1c59e0a1 7619@comment file: calc++-scanner.ll
12545799
AD
7620@example
7621id [a-zA-Z][a-zA-Z_0-9]*
7622int [0-9]+
7623blank [ \t]
7624@end example
7625
7626@noindent
9d9b8b70 7627The following paragraph suffices to track locations accurately. Each
12545799
AD
7628time @code{yylex} is invoked, the begin position is moved onto the end
7629position. Then when a pattern is matched, the end position is
7630advanced of its width. In case it matched ends of lines, the end
7631cursor is adjusted, and each time blanks are matched, the begin cursor
7632is moved onto the end cursor to effectively ignore the blanks
7633preceding tokens. Comments would be treated equally.
7634
1c59e0a1 7635@comment file: calc++-scanner.ll
12545799 7636@example
828c373b
AD
7637%@{
7638# define YY_USER_ACTION yylloc->columns (yyleng);
7639%@}
12545799
AD
7640%%
7641%@{
7642 yylloc->step ();
12545799
AD
7643%@}
7644@{blank@}+ yylloc->step ();
7645[\n]+ yylloc->lines (yyleng); yylloc->step ();
7646@end example
7647
7648@noindent
fb9712a9
AD
7649The rules are simple, just note the use of the driver to report errors.
7650It is convenient to use a typedef to shorten
7651@code{yy::calcxx_parser::token::identifier} into
9d9b8b70 7652@code{token::identifier} for instance.
12545799 7653
1c59e0a1 7654@comment file: calc++-scanner.ll
12545799 7655@example
fb9712a9
AD
7656%@{
7657 typedef yy::calcxx_parser::token token;
7658%@}
8c5b881d 7659 /* Convert ints to the actual type of tokens. */
c095d689 7660[-+*/] return yy::calcxx_parser::token_type (yytext[0]);
fb9712a9 7661":=" return token::ASSIGN;
04098407
PE
7662@{int@} @{
7663 errno = 0;
7664 long n = strtol (yytext, NULL, 10);
7665 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
7666 driver.error (*yylloc, "integer is out of range");
7667 yylval->ival = n;
fb9712a9 7668 return token::NUMBER;
04098407 7669@}
fb9712a9 7670@{id@} yylval->sval = new std::string (yytext); return token::IDENTIFIER;
12545799
AD
7671. driver.error (*yylloc, "invalid character");
7672%%
7673@end example
7674
7675@noindent
7676Finally, because the scanner related driver's member function depend
7677on the scanner's data, it is simpler to implement them in this file.
7678
1c59e0a1 7679@comment file: calc++-scanner.ll
12545799
AD
7680@example
7681void
7682calcxx_driver::scan_begin ()
7683@{
7684 yy_flex_debug = trace_scanning;
7685 if (!(yyin = fopen (file.c_str (), "r")))
7686 error (std::string ("cannot open ") + file);
7687@}
7688
7689void
7690calcxx_driver::scan_end ()
7691@{
7692 fclose (yyin);
7693@}
7694@end example
7695
7696@node Calc++ Top Level
7697@subsection Calc++ Top Level
7698
7699The top level file, @file{calc++.cc}, poses no problem.
7700
1c59e0a1 7701@comment file: calc++.cc
12545799
AD
7702@example
7703#include <iostream>
7704#include "calc++-driver.hh"
7705
7706int
fa4d969f 7707main (int argc, char *argv[])
12545799
AD
7708@{
7709 calcxx_driver driver;
7710 for (++argv; argv[0]; ++argv)
7711 if (*argv == std::string ("-p"))
7712 driver.trace_parsing = true;
7713 else if (*argv == std::string ("-s"))
7714 driver.trace_scanning = true;
7715 else
7716 @{
7717 driver.parse (*argv);
7718 std::cout << driver.result << std::endl;
7719 @}
7720@}
7721@end example
7722
7723@c ================================================= FAQ
d1a1114f
AD
7724
7725@node FAQ
7726@chapter Frequently Asked Questions
7727@cindex frequently asked questions
7728@cindex questions
7729
7730Several questions about Bison come up occasionally. Here some of them
7731are addressed.
7732
7733@menu
55ba27be
AD
7734* Memory Exhausted:: Breaking the Stack Limits
7735* How Can I Reset the Parser:: @code{yyparse} Keeps some State
7736* Strings are Destroyed:: @code{yylval} Loses Track of Strings
7737* Implementing Gotos/Loops:: Control Flow in the Calculator
ed2e6384 7738* Multiple start-symbols:: Factoring closely related grammars
55ba27be
AD
7739* Secure? Conform?:: Is Bison @acronym{POSIX} safe?
7740* I can't build Bison:: Troubleshooting
7741* Where can I find help?:: Troubleshouting
7742* Bug Reports:: Troublereporting
7743* Other Languages:: Parsers in Java and others
7744* Beta Testing:: Experimenting development versions
7745* Mailing Lists:: Meeting other Bison users
d1a1114f
AD
7746@end menu
7747
1a059451
PE
7748@node Memory Exhausted
7749@section Memory Exhausted
d1a1114f
AD
7750
7751@display
1a059451 7752My parser returns with error with a @samp{memory exhausted}
d1a1114f
AD
7753message. What can I do?
7754@end display
7755
7756This question is already addressed elsewhere, @xref{Recursion,
7757,Recursive Rules}.
7758
e64fec0a
PE
7759@node How Can I Reset the Parser
7760@section How Can I Reset the Parser
5b066063 7761
0e14ad77
PE
7762The following phenomenon has several symptoms, resulting in the
7763following typical questions:
5b066063
AD
7764
7765@display
7766I invoke @code{yyparse} several times, and on correct input it works
7767properly; but when a parse error is found, all the other calls fail
0e14ad77 7768too. How can I reset the error flag of @code{yyparse}?
5b066063
AD
7769@end display
7770
7771@noindent
7772or
7773
7774@display
0e14ad77 7775My parser includes support for an @samp{#include}-like feature, in
5b066063
AD
7776which case I run @code{yyparse} from @code{yyparse}. This fails
7777although I did specify I needed a @code{%pure-parser}.
7778@end display
7779
0e14ad77
PE
7780These problems typically come not from Bison itself, but from
7781Lex-generated scanners. Because these scanners use large buffers for
5b066063
AD
7782speed, they might not notice a change of input file. As a
7783demonstration, consider the following source file,
7784@file{first-line.l}:
7785
7786@verbatim
7787%{
7788#include <stdio.h>
7789#include <stdlib.h>
7790%}
7791%%
7792.*\n ECHO; return 1;
7793%%
7794int
0e14ad77 7795yyparse (char const *file)
5b066063
AD
7796{
7797 yyin = fopen (file, "r");
7798 if (!yyin)
7799 exit (2);
fa7e68c3 7800 /* One token only. */
5b066063 7801 yylex ();
0e14ad77 7802 if (fclose (yyin) != 0)
5b066063
AD
7803 exit (3);
7804 return 0;
7805}
7806
7807int
0e14ad77 7808main (void)
5b066063
AD
7809{
7810 yyparse ("input");
7811 yyparse ("input");
7812 return 0;
7813}
7814@end verbatim
7815
7816@noindent
7817If the file @file{input} contains
7818
7819@verbatim
7820input:1: Hello,
7821input:2: World!
7822@end verbatim
7823
7824@noindent
0e14ad77 7825then instead of getting the first line twice, you get:
5b066063
AD
7826
7827@example
7828$ @kbd{flex -ofirst-line.c first-line.l}
7829$ @kbd{gcc -ofirst-line first-line.c -ll}
7830$ @kbd{./first-line}
7831input:1: Hello,
7832input:2: World!
7833@end example
7834
0e14ad77
PE
7835Therefore, whenever you change @code{yyin}, you must tell the
7836Lex-generated scanner to discard its current buffer and switch to the
7837new one. This depends upon your implementation of Lex; see its
7838documentation for more. For Flex, it suffices to call
7839@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
7840Flex-generated scanner needs to read from several input streams to
7841handle features like include files, you might consider using Flex
7842functions like @samp{yy_switch_to_buffer} that manipulate multiple
7843input buffers.
5b066063 7844
b165c324
AD
7845If your Flex-generated scanner uses start conditions (@pxref{Start
7846conditions, , Start conditions, flex, The Flex Manual}), you might
7847also want to reset the scanner's state, i.e., go back to the initial
7848start condition, through a call to @samp{BEGIN (0)}.
7849
fef4cb51
AD
7850@node Strings are Destroyed
7851@section Strings are Destroyed
7852
7853@display
c7e441b4 7854My parser seems to destroy old strings, or maybe it loses track of
fef4cb51
AD
7855them. Instead of reporting @samp{"foo", "bar"}, it reports
7856@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
7857@end display
7858
7859This error is probably the single most frequent ``bug report'' sent to
7860Bison lists, but is only concerned with a misunderstanding of the role
8c5b881d 7861of the scanner. Consider the following Lex code:
fef4cb51
AD
7862
7863@verbatim
7864%{
7865#include <stdio.h>
7866char *yylval = NULL;
7867%}
7868%%
7869.* yylval = yytext; return 1;
7870\n /* IGNORE */
7871%%
7872int
7873main ()
7874{
fa7e68c3 7875 /* Similar to using $1, $2 in a Bison action. */
fef4cb51
AD
7876 char *fst = (yylex (), yylval);
7877 char *snd = (yylex (), yylval);
7878 printf ("\"%s\", \"%s\"\n", fst, snd);
7879 return 0;
7880}
7881@end verbatim
7882
7883If you compile and run this code, you get:
7884
7885@example
7886$ @kbd{flex -osplit-lines.c split-lines.l}
7887$ @kbd{gcc -osplit-lines split-lines.c -ll}
7888$ @kbd{printf 'one\ntwo\n' | ./split-lines}
7889"one
7890two", "two"
7891@end example
7892
7893@noindent
7894this is because @code{yytext} is a buffer provided for @emph{reading}
7895in the action, but if you want to keep it, you have to duplicate it
7896(e.g., using @code{strdup}). Note that the output may depend on how
7897your implementation of Lex handles @code{yytext}. For instance, when
7898given the Lex compatibility option @option{-l} (which triggers the
7899option @samp{%array}) Flex generates a different behavior:
7900
7901@example
7902$ @kbd{flex -l -osplit-lines.c split-lines.l}
7903$ @kbd{gcc -osplit-lines split-lines.c -ll}
7904$ @kbd{printf 'one\ntwo\n' | ./split-lines}
7905"two", "two"
7906@end example
7907
7908
2fa09258
AD
7909@node Implementing Gotos/Loops
7910@section Implementing Gotos/Loops
a06ea4aa
AD
7911
7912@display
7913My simple calculator supports variables, assignments, and functions,
2fa09258 7914but how can I implement gotos, or loops?
a06ea4aa
AD
7915@end display
7916
7917Although very pedagogical, the examples included in the document blur
a1c84f45 7918the distinction to make between the parser---whose job is to recover
a06ea4aa 7919the structure of a text and to transmit it to subsequent modules of
a1c84f45 7920the program---and the processing (such as the execution) of this
a06ea4aa
AD
7921structure. This works well with so called straight line programs,
7922i.e., precisely those that have a straightforward execution model:
7923execute simple instructions one after the others.
7924
7925@cindex abstract syntax tree
7926@cindex @acronym{AST}
7927If you want a richer model, you will probably need to use the parser
7928to construct a tree that does represent the structure it has
7929recovered; this tree is usually called the @dfn{abstract syntax tree},
7930or @dfn{@acronym{AST}} for short. Then, walking through this tree,
7931traversing it in various ways, will enable treatments such as its
7932execution or its translation, which will result in an interpreter or a
7933compiler.
7934
7935This topic is way beyond the scope of this manual, and the reader is
7936invited to consult the dedicated literature.
7937
7938
ed2e6384
AD
7939@node Multiple start-symbols
7940@section Multiple start-symbols
7941
7942@display
7943I have several closely related grammars, and I would like to share their
7944implementations. In fact, I could use a single grammar but with
7945multiple entry points.
7946@end display
7947
7948Bison does not support multiple start-symbols, but there is a very
7949simple means to simulate them. If @code{foo} and @code{bar} are the two
7950pseudo start-symbols, then introduce two new tokens, say
7951@code{START_FOO} and @code{START_BAR}, and use them as switches from the
7952real start-symbol:
7953
7954@example
7955%token START_FOO START_BAR;
7956%start start;
7957start: START_FOO foo
7958 | START_BAR bar;
7959@end example
7960
7961These tokens prevents the introduction of new conflicts. As far as the
7962parser goes, that is all that is needed.
7963
7964Now the difficult part is ensuring that the scanner will send these
7965tokens first. If your scanner is hand-written, that should be
7966straightforward. If your scanner is generated by Lex, them there is
7967simple means to do it: recall that anything between @samp{%@{ ... %@}}
7968after the first @code{%%} is copied verbatim in the top of the generated
7969@code{yylex} function. Make sure a variable @code{start_token} is
7970available in the scanner (e.g., a global variable or using
7971@code{%lex-param} etc.), and use the following:
7972
7973@example
7974 /* @r{Prologue.} */
7975%%
7976%@{
7977 if (start_token)
7978 @{
7979 int t = start_token;
7980 start_token = 0;
7981 return t;
7982 @}
7983%@}
7984 /* @r{The rules.} */
7985@end example
7986
7987
55ba27be
AD
7988@node Secure? Conform?
7989@section Secure? Conform?
7990
7991@display
7992Is Bison secure? Does it conform to POSIX?
7993@end display
7994
7995If you're looking for a guarantee or certification, we don't provide it.
7996However, Bison is intended to be a reliable program that conforms to the
7997@acronym{POSIX} specification for Yacc. If you run into problems,
7998please send us a bug report.
7999
8000@node I can't build Bison
8001@section I can't build Bison
8002
8003@display
8c5b881d
PE
8004I can't build Bison because @command{make} complains that
8005@code{msgfmt} is not found.
55ba27be
AD
8006What should I do?
8007@end display
8008
8009Like most GNU packages with internationalization support, that feature
8010is turned on by default. If you have problems building in the @file{po}
8011subdirectory, it indicates that your system's internationalization
8012support is lacking. You can re-configure Bison with
8013@option{--disable-nls} to turn off this support, or you can install GNU
8014gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
8015Bison. See the file @file{ABOUT-NLS} for more information.
8016
8017
8018@node Where can I find help?
8019@section Where can I find help?
8020
8021@display
8022I'm having trouble using Bison. Where can I find help?
8023@end display
8024
8025First, read this fine manual. Beyond that, you can send mail to
8026@email{help-bison@@gnu.org}. This mailing list is intended to be
8027populated with people who are willing to answer questions about using
8028and installing Bison. Please keep in mind that (most of) the people on
8029the list have aspects of their lives which are not related to Bison (!),
8030so you may not receive an answer to your question right away. This can
8031be frustrating, but please try not to honk them off; remember that any
8032help they provide is purely voluntary and out of the kindness of their
8033hearts.
8034
8035@node Bug Reports
8036@section Bug Reports
8037
8038@display
8039I found a bug. What should I include in the bug report?
8040@end display
8041
8042Before you send a bug report, make sure you are using the latest
8043version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
8044mirrors. Be sure to include the version number in your bug report. If
8045the bug is present in the latest version but not in a previous version,
8046try to determine the most recent version which did not contain the bug.
8047
8048If the bug is parser-related, you should include the smallest grammar
8049you can which demonstrates the bug. The grammar file should also be
8050complete (i.e., I should be able to run it through Bison without having
8051to edit or add anything). The smaller and simpler the grammar, the
8052easier it will be to fix the bug.
8053
8054Include information about your compilation environment, including your
8055operating system's name and version and your compiler's name and
8056version. If you have trouble compiling, you should also include a
8057transcript of the build session, starting with the invocation of
8058`configure'. Depending on the nature of the bug, you may be asked to
8059send additional files as well (such as `config.h' or `config.cache').
8060
8061Patches are most welcome, but not required. That is, do not hesitate to
8062send a bug report just because you can not provide a fix.
8063
8064Send bug reports to @email{bug-bison@@gnu.org}.
8065
8066@node Other Languages
8067@section Other Languages
8068
8069@display
8070Will Bison ever have C++ support? How about Java or @var{insert your
8071favorite language here}?
8072@end display
8073
8074C++ support is there now, and is documented. We'd love to add other
8075languages; contributions are welcome.
8076
8077@node Beta Testing
8078@section Beta Testing
8079
8080@display
8081What is involved in being a beta tester?
8082@end display
8083
8084It's not terribly involved. Basically, you would download a test
8085release, compile it, and use it to build and run a parser or two. After
8086that, you would submit either a bug report or a message saying that
8087everything is okay. It is important to report successes as well as
8088failures because test releases eventually become mainstream releases,
8089but only if they are adequately tested. If no one tests, development is
8090essentially halted.
8091
8092Beta testers are particularly needed for operating systems to which the
8093developers do not have easy access. They currently have easy access to
8094recent GNU/Linux and Solaris versions. Reports about other operating
8095systems are especially welcome.
8096
8097@node Mailing Lists
8098@section Mailing Lists
8099
8100@display
8101How do I join the help-bison and bug-bison mailing lists?
8102@end display
8103
8104See @url{http://lists.gnu.org/}.
a06ea4aa 8105
d1a1114f
AD
8106@c ================================================= Table of Symbols
8107
342b8b6e 8108@node Table of Symbols
bfa74976
RS
8109@appendix Bison Symbols
8110@cindex Bison symbols, table of
8111@cindex symbols in Bison, table of
8112
18b519c0 8113@deffn {Variable} @@$
3ded9a63 8114In an action, the location of the left-hand side of the rule.
88bce5a2 8115@xref{Locations, , Locations Overview}.
18b519c0 8116@end deffn
3ded9a63 8117
18b519c0 8118@deffn {Variable} @@@var{n}
3ded9a63
AD
8119In an action, the location of the @var{n}-th symbol of the right-hand
8120side of the rule. @xref{Locations, , Locations Overview}.
18b519c0 8121@end deffn
3ded9a63 8122
18b519c0 8123@deffn {Variable} $$
3ded9a63
AD
8124In an action, the semantic value of the left-hand side of the rule.
8125@xref{Actions}.
18b519c0 8126@end deffn
3ded9a63 8127
18b519c0 8128@deffn {Variable} $@var{n}
3ded9a63
AD
8129In an action, the semantic value of the @var{n}-th symbol of the
8130right-hand side of the rule. @xref{Actions}.
18b519c0 8131@end deffn
3ded9a63 8132
dd8d9022
AD
8133@deffn {Delimiter} %%
8134Delimiter used to separate the grammar rule section from the
8135Bison declarations section or the epilogue.
8136@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
18b519c0 8137@end deffn
bfa74976 8138
dd8d9022
AD
8139@c Don't insert spaces, or check the DVI output.
8140@deffn {Delimiter} %@{@var{code}%@}
8141All code listed between @samp{%@{} and @samp{%@}} is copied directly to
8142the output file uninterpreted. Such code forms the prologue of the input
8143file. @xref{Grammar Outline, ,Outline of a Bison
8144Grammar}.
18b519c0 8145@end deffn
bfa74976 8146
dd8d9022
AD
8147@deffn {Construct} /*@dots{}*/
8148Comment delimiters, as in C.
18b519c0 8149@end deffn
bfa74976 8150
dd8d9022
AD
8151@deffn {Delimiter} :
8152Separates a rule's result from its components. @xref{Rules, ,Syntax of
8153Grammar Rules}.
18b519c0 8154@end deffn
bfa74976 8155
dd8d9022
AD
8156@deffn {Delimiter} ;
8157Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 8158@end deffn
bfa74976 8159
dd8d9022
AD
8160@deffn {Delimiter} |
8161Separates alternate rules for the same result nonterminal.
8162@xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 8163@end deffn
bfa74976 8164
dd8d9022
AD
8165@deffn {Symbol} $accept
8166The predefined nonterminal whose only rule is @samp{$accept: @var{start}
8167$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
8168Start-Symbol}. It cannot be used in the grammar.
18b519c0 8169@end deffn
bfa74976 8170
18b519c0 8171@deffn {Directive} %debug
6deb4447 8172Equip the parser for debugging. @xref{Decl Summary}.
18b519c0 8173@end deffn
6deb4447 8174
91d2c560 8175@ifset defaultprec
22fccf95
PE
8176@deffn {Directive} %default-prec
8177Assign a precedence to rules that lack an explicit @samp{%prec}
8178modifier. @xref{Contextual Precedence, ,Context-Dependent
8179Precedence}.
39a06c25 8180@end deffn
91d2c560 8181@end ifset
39a06c25 8182
18b519c0 8183@deffn {Directive} %defines
6deb4447
AD
8184Bison declaration to create a header file meant for the scanner.
8185@xref{Decl Summary}.
18b519c0 8186@end deffn
6deb4447 8187
18b519c0 8188@deffn {Directive} %destructor
258b75ca 8189Specify how the parser should reclaim the memory associated to
fa7e68c3 8190discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 8191@end deffn
72f889cc 8192
18b519c0 8193@deffn {Directive} %dprec
676385e2 8194Bison declaration to assign a precedence to a rule that is used at parse
c827f760
PE
8195time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
8196@acronym{GLR} Parsers}.
18b519c0 8197@end deffn
676385e2 8198
dd8d9022
AD
8199@deffn {Symbol} $end
8200The predefined token marking the end of the token stream. It cannot be
8201used in the grammar.
8202@end deffn
8203
8204@deffn {Symbol} error
8205A token name reserved for error recovery. This token may be used in
8206grammar rules so as to allow the Bison parser to recognize an error in
8207the grammar without halting the process. In effect, a sentence
8208containing an error may be recognized as valid. On a syntax error, the
8209token @code{error} becomes the current look-ahead token. Actions
8210corresponding to @code{error} are then executed, and the look-ahead
8211token is reset to the token that originally caused the violation.
8212@xref{Error Recovery}.
18d192f0
AD
8213@end deffn
8214
18b519c0 8215@deffn {Directive} %error-verbose
2a8d363a
AD
8216Bison declaration to request verbose, specific error message strings
8217when @code{yyerror} is called.
18b519c0 8218@end deffn
2a8d363a 8219
18b519c0 8220@deffn {Directive} %file-prefix="@var{prefix}"
72d2299c 8221Bison declaration to set the prefix of the output files. @xref{Decl
d8988b2f 8222Summary}.
18b519c0 8223@end deffn
d8988b2f 8224
18b519c0 8225@deffn {Directive} %glr-parser
c827f760
PE
8226Bison declaration to produce a @acronym{GLR} parser. @xref{GLR
8227Parsers, ,Writing @acronym{GLR} Parsers}.
18b519c0 8228@end deffn
676385e2 8229
dd8d9022
AD
8230@deffn {Directive} %initial-action
8231Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
8232@end deffn
8233
18b519c0 8234@deffn {Directive} %left
bfa74976
RS
8235Bison declaration to assign left associativity to token(s).
8236@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 8237@end deffn
bfa74976 8238
feeb0eda 8239@deffn {Directive} %lex-param @{@var{argument-declaration}@}
2a8d363a
AD
8240Bison declaration to specifying an additional parameter that
8241@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
8242for Pure Parsers}.
18b519c0 8243@end deffn
2a8d363a 8244
18b519c0 8245@deffn {Directive} %merge
676385e2 8246Bison declaration to assign a merging function to a rule. If there is a
fae437e8 8247reduce/reduce conflict with a rule having the same merging function, the
676385e2 8248function is applied to the two semantic values to get a single result.
c827f760 8249@xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}.
18b519c0 8250@end deffn
676385e2 8251
18b519c0 8252@deffn {Directive} %name-prefix="@var{prefix}"
72d2299c 8253Bison declaration to rename the external symbols. @xref{Decl Summary}.
18b519c0 8254@end deffn
d8988b2f 8255
91d2c560 8256@ifset defaultprec
22fccf95
PE
8257@deffn {Directive} %no-default-prec
8258Do not assign a precedence to rules that lack an explicit @samp{%prec}
8259modifier. @xref{Contextual Precedence, ,Context-Dependent
8260Precedence}.
8261@end deffn
91d2c560 8262@end ifset
22fccf95 8263
18b519c0 8264@deffn {Directive} %no-lines
931c7513
RS
8265Bison declaration to avoid generating @code{#line} directives in the
8266parser file. @xref{Decl Summary}.
18b519c0 8267@end deffn
931c7513 8268
18b519c0 8269@deffn {Directive} %nonassoc
9d9b8b70 8270Bison declaration to assign nonassociativity to token(s).
bfa74976 8271@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 8272@end deffn
bfa74976 8273
fa4d969f 8274@deffn {Directive} %output="@var{file}"
72d2299c 8275Bison declaration to set the name of the parser file. @xref{Decl
d8988b2f 8276Summary}.
18b519c0 8277@end deffn
d8988b2f 8278
feeb0eda 8279@deffn {Directive} %parse-param @{@var{argument-declaration}@}
2a8d363a
AD
8280Bison declaration to specifying an additional parameter that
8281@code{yyparse} should accept. @xref{Parser Function,, The Parser
8282Function @code{yyparse}}.
18b519c0 8283@end deffn
2a8d363a 8284
18b519c0 8285@deffn {Directive} %prec
bfa74976
RS
8286Bison declaration to assign a precedence to a specific rule.
8287@xref{Contextual Precedence, ,Context-Dependent Precedence}.
18b519c0 8288@end deffn
bfa74976 8289
18b519c0 8290@deffn {Directive} %pure-parser
bfa74976
RS
8291Bison declaration to request a pure (reentrant) parser.
8292@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
18b519c0 8293@end deffn
bfa74976 8294
b50d2359 8295@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
8296Require version @var{version} or higher of Bison. @xref{Require Decl, ,
8297Require a Version of Bison}.
b50d2359
AD
8298@end deffn
8299
18b519c0 8300@deffn {Directive} %right
bfa74976
RS
8301Bison declaration to assign right associativity to token(s).
8302@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 8303@end deffn
bfa74976 8304
18b519c0 8305@deffn {Directive} %start
704a47c4
AD
8306Bison declaration to specify the start symbol. @xref{Start Decl, ,The
8307Start-Symbol}.
18b519c0 8308@end deffn
bfa74976 8309
18b519c0 8310@deffn {Directive} %token
bfa74976
RS
8311Bison declaration to declare token(s) without specifying precedence.
8312@xref{Token Decl, ,Token Type Names}.
18b519c0 8313@end deffn
bfa74976 8314
18b519c0 8315@deffn {Directive} %token-table
931c7513
RS
8316Bison declaration to include a token name table in the parser file.
8317@xref{Decl Summary}.
18b519c0 8318@end deffn
931c7513 8319
18b519c0 8320@deffn {Directive} %type
704a47c4
AD
8321Bison declaration to declare nonterminals. @xref{Type Decl,
8322,Nonterminal Symbols}.
18b519c0 8323@end deffn
bfa74976 8324
dd8d9022
AD
8325@deffn {Symbol} $undefined
8326The predefined token onto which all undefined values returned by
8327@code{yylex} are mapped. It cannot be used in the grammar, rather, use
8328@code{error}.
8329@end deffn
8330
18b519c0 8331@deffn {Directive} %union
bfa74976
RS
8332Bison declaration to specify several possible data types for semantic
8333values. @xref{Union Decl, ,The Collection of Value Types}.
18b519c0 8334@end deffn
bfa74976 8335
dd8d9022
AD
8336@deffn {Macro} YYABORT
8337Macro to pretend that an unrecoverable syntax error has occurred, by
8338making @code{yyparse} return 1 immediately. The error reporting
8339function @code{yyerror} is not called. @xref{Parser Function, ,The
8340Parser Function @code{yyparse}}.
8341@end deffn
3ded9a63 8342
dd8d9022
AD
8343@deffn {Macro} YYACCEPT
8344Macro to pretend that a complete utterance of the language has been
8345read, by making @code{yyparse} return 0 immediately.
8346@xref{Parser Function, ,The Parser Function @code{yyparse}}.
8347@end deffn
bfa74976 8348
dd8d9022
AD
8349@deffn {Macro} YYBACKUP
8350Macro to discard a value from the parser stack and fake a look-ahead
8351token. @xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 8352@end deffn
bfa74976 8353
dd8d9022 8354@deffn {Variable} yychar
32c29292 8355External integer variable that contains the integer value of the
dd8d9022
AD
8356look-ahead token. (In a pure parser, it is a local variable within
8357@code{yyparse}.) Error-recovery rule actions may examine this variable.
8358@xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 8359@end deffn
bfa74976 8360
dd8d9022
AD
8361@deffn {Variable} yyclearin
8362Macro used in error-recovery rule actions. It clears the previous
8363look-ahead token. @xref{Error Recovery}.
18b519c0 8364@end deffn
bfa74976 8365
dd8d9022
AD
8366@deffn {Macro} YYDEBUG
8367Macro to define to equip the parser with tracing code. @xref{Tracing,
8368,Tracing Your Parser}.
18b519c0 8369@end deffn
bfa74976 8370
dd8d9022
AD
8371@deffn {Variable} yydebug
8372External integer variable set to zero by default. If @code{yydebug}
8373is given a nonzero value, the parser will output information on input
8374symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
18b519c0 8375@end deffn
bfa74976 8376
dd8d9022
AD
8377@deffn {Macro} yyerrok
8378Macro to cause parser to recover immediately to its normal mode
8379after a syntax error. @xref{Error Recovery}.
8380@end deffn
8381
8382@deffn {Macro} YYERROR
8383Macro to pretend that a syntax error has just been detected: call
8384@code{yyerror} and then perform normal error recovery if possible
8385(@pxref{Error Recovery}), or (if recovery is impossible) make
8386@code{yyparse} return 1. @xref{Error Recovery}.
8387@end deffn
8388
8389@deffn {Function} yyerror
8390User-supplied function to be called by @code{yyparse} on error.
8391@xref{Error Reporting, ,The Error
8392Reporting Function @code{yyerror}}.
8393@end deffn
8394
8395@deffn {Macro} YYERROR_VERBOSE
8396An obsolete macro that you define with @code{#define} in the prologue
8397to request verbose, specific error message strings
8398when @code{yyerror} is called. It doesn't matter what definition you
8399use for @code{YYERROR_VERBOSE}, just whether you define it. Using
8400@code{%error-verbose} is preferred.
8401@end deffn
8402
8403@deffn {Macro} YYINITDEPTH
8404Macro for specifying the initial size of the parser stack.
1a059451 8405@xref{Memory Management}.
dd8d9022
AD
8406@end deffn
8407
8408@deffn {Function} yylex
8409User-supplied lexical analyzer function, called with no arguments to get
8410the next token. @xref{Lexical, ,The Lexical Analyzer Function
8411@code{yylex}}.
8412@end deffn
8413
8414@deffn {Macro} YYLEX_PARAM
8415An obsolete macro for specifying an extra argument (or list of extra
32c29292 8416arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
dd8d9022
AD
8417macro is deprecated, and is supported only for Yacc like parsers.
8418@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
8419@end deffn
8420
8421@deffn {Variable} yylloc
8422External variable in which @code{yylex} should place the line and column
8423numbers associated with a token. (In a pure parser, it is a local
8424variable within @code{yyparse}, and its address is passed to
32c29292
JD
8425@code{yylex}.)
8426You can ignore this variable if you don't use the @samp{@@} feature in the
8427grammar actions.
8428@xref{Token Locations, ,Textual Locations of Tokens}.
8429In semantic actions, it stores the location of the look-ahead token.
8430@xref{Actions and Locations, ,Actions and Locations}.
dd8d9022
AD
8431@end deffn
8432
8433@deffn {Type} YYLTYPE
8434Data type of @code{yylloc}; by default, a structure with four
8435members. @xref{Location Type, , Data Types of Locations}.
8436@end deffn
8437
8438@deffn {Variable} yylval
8439External variable in which @code{yylex} should place the semantic
8440value associated with a token. (In a pure parser, it is a local
8441variable within @code{yyparse}, and its address is passed to
32c29292
JD
8442@code{yylex}.)
8443@xref{Token Values, ,Semantic Values of Tokens}.
8444In semantic actions, it stores the semantic value of the look-ahead token.
8445@xref{Actions, ,Actions}.
dd8d9022
AD
8446@end deffn
8447
8448@deffn {Macro} YYMAXDEPTH
1a059451
PE
8449Macro for specifying the maximum size of the parser stack. @xref{Memory
8450Management}.
dd8d9022
AD
8451@end deffn
8452
8453@deffn {Variable} yynerrs
8a2800e7 8454Global variable which Bison increments each time it reports a syntax error.
dd8d9022
AD
8455(In a pure parser, it is a local variable within @code{yyparse}.)
8456@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
8457@end deffn
8458
8459@deffn {Function} yyparse
8460The parser function produced by Bison; call this function to start
8461parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
8462@end deffn
8463
8464@deffn {Macro} YYPARSE_PARAM
8465An obsolete macro for specifying the name of a parameter that
8466@code{yyparse} should accept. The use of this macro is deprecated, and
8467is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
8468Conventions for Pure Parsers}.
8469@end deffn
8470
8471@deffn {Macro} YYRECOVERING
8472Macro whose value indicates whether the parser is recovering from a
8473syntax error. @xref{Action Features, ,Special Features for Use in Actions}.
8474@end deffn
8475
8476@deffn {Macro} YYSTACK_USE_ALLOCA
d7e14fc0
PE
8477Macro used to control the use of @code{alloca} when the C
8478@acronym{LALR}(1) parser needs to extend its stacks. If defined to 0,
8479the parser will use @code{malloc} to extend its stacks. If defined to
84801, the parser will use @code{alloca}. Values other than 0 and 1 are
8481reserved for future Bison extensions. If not defined,
8482@code{YYSTACK_USE_ALLOCA} defaults to 0.
8483
55289366 8484In the all-too-common case where your code may run on a host with a
d7e14fc0
PE
8485limited stack and with unreliable stack-overflow checking, you should
8486set @code{YYMAXDEPTH} to a value that cannot possibly result in
8487unchecked stack overflow on any of your target hosts when
8488@code{alloca} is called. You can inspect the code that Bison
8489generates in order to determine the proper numeric values. This will
8490require some expertise in low-level implementation details.
dd8d9022
AD
8491@end deffn
8492
8493@deffn {Type} YYSTYPE
8494Data type of semantic values; @code{int} by default.
8495@xref{Value Type, ,Data Types of Semantic Values}.
18b519c0 8496@end deffn
bfa74976 8497
342b8b6e 8498@node Glossary
bfa74976
RS
8499@appendix Glossary
8500@cindex glossary
8501
8502@table @asis
c827f760
PE
8503@item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'')
8504Formal method of specifying context-free grammars originally proposed
8505by John Backus, and slightly improved by Peter Naur in his 1960-01-02
8506committee document contributing to what became the Algol 60 report.
8507@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
8508
8509@item Context-free grammars
8510Grammars specified as rules that can be applied regardless of context.
8511Thus, if there is a rule which says that an integer can be used as an
8512expression, integers are allowed @emph{anywhere} an expression is
89cab50d
AD
8513permitted. @xref{Language and Grammar, ,Languages and Context-Free
8514Grammars}.
bfa74976
RS
8515
8516@item Dynamic allocation
8517Allocation of memory that occurs during execution, rather than at
8518compile time or on entry to a function.
8519
8520@item Empty string
8521Analogous to the empty set in set theory, the empty string is a
8522character string of length zero.
8523
8524@item Finite-state stack machine
8525A ``machine'' that has discrete states in which it is said to exist at
8526each instant in time. As input to the machine is processed, the
8527machine moves from state to state as specified by the logic of the
8528machine. In the case of the parser, the input is the language being
8529parsed, and the states correspond to various stages in the grammar
c827f760 8530rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976 8531
c827f760 8532@item Generalized @acronym{LR} (@acronym{GLR})
676385e2 8533A parsing algorithm that can handle all context-free grammars, including those
c827f760
PE
8534that are not @acronym{LALR}(1). It resolves situations that Bison's
8535usual @acronym{LALR}(1)
676385e2
PH
8536algorithm cannot by effectively splitting off multiple parsers, trying all
8537possible parsers, and discarding those that fail in the light of additional
c827f760
PE
8538right context. @xref{Generalized LR Parsing, ,Generalized
8539@acronym{LR} Parsing}.
676385e2 8540
bfa74976
RS
8541@item Grouping
8542A language construct that is (in general) grammatically divisible;
c827f760 8543for example, `expression' or `declaration' in C@.
bfa74976
RS
8544@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
8545
8546@item Infix operator
8547An arithmetic operator that is placed between the operands on which it
8548performs some operation.
8549
8550@item Input stream
8551A continuous flow of data between devices or programs.
8552
8553@item Language construct
8554One of the typical usage schemas of the language. For example, one of
8555the constructs of the C language is the @code{if} statement.
8556@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
8557
8558@item Left associativity
8559Operators having left associativity are analyzed from left to right:
8560@samp{a+b+c} first computes @samp{a+b} and then combines with
8561@samp{c}. @xref{Precedence, ,Operator Precedence}.
8562
8563@item Left recursion
89cab50d
AD
8564A rule whose result symbol is also its first component symbol; for
8565example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
8566Rules}.
bfa74976
RS
8567
8568@item Left-to-right parsing
8569Parsing a sentence of a language by analyzing it token by token from
c827f760 8570left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
8571
8572@item Lexical analyzer (scanner)
8573A function that reads an input stream and returns tokens one by one.
8574@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
8575
8576@item Lexical tie-in
8577A flag, set by actions in the grammar rules, which alters the way
8578tokens are parsed. @xref{Lexical Tie-ins}.
8579
931c7513 8580@item Literal string token
14ded682 8581A token which consists of two or more fixed characters. @xref{Symbols}.
931c7513 8582
bfa74976 8583@item Look-ahead token
89cab50d
AD
8584A token already read but not yet shifted. @xref{Look-Ahead, ,Look-Ahead
8585Tokens}.
bfa74976 8586
c827f760 8587@item @acronym{LALR}(1)
bfa74976 8588The class of context-free grammars that Bison (like most other parser
c827f760
PE
8589generators) can handle; a subset of @acronym{LR}(1). @xref{Mystery
8590Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
bfa74976 8591
c827f760 8592@item @acronym{LR}(1)
bfa74976
RS
8593The class of context-free grammars in which at most one token of
8594look-ahead is needed to disambiguate the parsing of any piece of input.
8595
8596@item Nonterminal symbol
8597A grammar symbol standing for a grammatical construct that can
8598be expressed through rules in terms of smaller constructs; in other
8599words, a construct that is not a token. @xref{Symbols}.
8600
bfa74976
RS
8601@item Parser
8602A function that recognizes valid sentences of a language by analyzing
8603the syntax structure of a set of tokens passed to it from a lexical
8604analyzer.
8605
8606@item Postfix operator
8607An arithmetic operator that is placed after the operands upon which it
8608performs some operation.
8609
8610@item Reduction
8611Replacing a string of nonterminals and/or terminals with a single
89cab50d 8612nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
c827f760 8613Parser Algorithm}.
bfa74976
RS
8614
8615@item Reentrant
8616A reentrant subprogram is a subprogram which can be in invoked any
8617number of times in parallel, without interference between the various
8618invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
8619
8620@item Reverse polish notation
8621A language in which all operators are postfix operators.
8622
8623@item Right recursion
89cab50d
AD
8624A rule whose result symbol is also its last component symbol; for
8625example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
8626Rules}.
bfa74976
RS
8627
8628@item Semantics
8629In computer languages, the semantics are specified by the actions
8630taken for each instance of the language, i.e., the meaning of
8631each statement. @xref{Semantics, ,Defining Language Semantics}.
8632
8633@item Shift
8634A parser is said to shift when it makes the choice of analyzing
8635further input from the stream rather than reducing immediately some
c827f760 8636already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
8637
8638@item Single-character literal
8639A single character that is recognized and interpreted as is.
8640@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
8641
8642@item Start symbol
8643The nonterminal symbol that stands for a complete valid utterance in
8644the language being parsed. The start symbol is usually listed as the
13863333 8645first nonterminal symbol in a language specification.
bfa74976
RS
8646@xref{Start Decl, ,The Start-Symbol}.
8647
8648@item Symbol table
8649A data structure where symbol names and associated data are stored
8650during parsing to allow for recognition and use of existing
8651information in repeated uses of a symbol. @xref{Multi-function Calc}.
8652
6e649e65
PE
8653@item Syntax error
8654An error encountered during parsing of an input stream due to invalid
8655syntax. @xref{Error Recovery}.
8656
bfa74976
RS
8657@item Token
8658A basic, grammatically indivisible unit of a language. The symbol
8659that describes a token in the grammar is a terminal symbol.
8660The input of the Bison parser is a stream of tokens which comes from
8661the lexical analyzer. @xref{Symbols}.
8662
8663@item Terminal symbol
89cab50d
AD
8664A grammar symbol that has no rules in the grammar and therefore is
8665grammatically indivisible. The piece of text it represents is a token.
8666@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
8667@end table
8668
342b8b6e 8669@node Copying This Manual
f2b5126e 8670@appendix Copying This Manual
f9a8293a 8671
f2b5126e
PB
8672@menu
8673* GNU Free Documentation License:: License for copying this manual.
8674@end menu
f9a8293a 8675
f2b5126e
PB
8676@include fdl.texi
8677
342b8b6e 8678@node Index
bfa74976
RS
8679@unnumbered Index
8680
8681@printindex cp
8682
bfa74976 8683@bye
a06ea4aa
AD
8684
8685@c LocalWords: texinfo setfilename settitle setchapternewpage finalout
8686@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex
8687@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry
8688@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa
8689@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc
8690@c LocalWords: rpcalc Lexer Gen Comp Expr ltcalc mfcalc Decl Symtab yylex
8691@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref
8692@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex
8693@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge
8694@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG
8695@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit
8696@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok
8697@c LocalWords: longjmp fprintf stderr preg yylloc YYLTYPE cos ln
8698@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym
8699@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof
8700@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum
8701@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype
8702@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless typefull yynerrs
8703@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES
8704@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param
8705@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP
32c29292 8706@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword
a06ea4aa
AD
8707@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH
8708@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm VCG notype
8709@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args
8710@c LocalWords: YYPRINTF infile ypp yxx outfile itemx vcg tex leaderfill
8711@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll
8712@c LocalWords: yyrestart nbar yytext fst snd osplit ntwo strdup AST
8713@c LocalWords: YYSTACK DVI fdl printindex