]> git.saurik.com Git - bison.git/blame - doc/bison.texinfo
java: test and document previous bug fix.
[bison.git] / doc / bison.texinfo
CommitLineData
bfa74976
RS
1\input texinfo @c -*-texinfo-*-
2@comment %**start of header
3@setfilename bison.info
df1af54c
JT
4@include version.texi
5@settitle Bison @value{VERSION}
bfa74976
RS
6@setchapternewpage odd
7
5378c3e7 8@finalout
5378c3e7 9
13863333 10@c SMALL BOOK version
bfa74976 11@c This edition has been formatted so that you can format and print it in
13863333 12@c the smallbook format.
bfa74976
RS
13@c @smallbook
14
91d2c560
PE
15@c Set following if you want to document %default-prec and %no-default-prec.
16@c This feature is experimental and may change in future Bison versions.
17@c @set defaultprec
18
8c5b881d 19@ifnotinfo
bfa74976
RS
20@syncodeindex fn cp
21@syncodeindex vr cp
22@syncodeindex tp cp
8c5b881d 23@end ifnotinfo
bfa74976
RS
24@ifinfo
25@synindex fn cp
26@synindex vr cp
27@synindex tp cp
28@end ifinfo
29@comment %**end of header
30
fae437e8 31@copying
bd773d73 32
8a4281b9
JD
33This manual (@value{UPDATED}) is for GNU Bison (version
34@value{VERSION}), the GNU parser generator.
fae437e8 35
575619af
JD
36Copyright @copyright{} 1988-1993, 1995, 1998-2011 Free Software
37Foundation, Inc.
fae437e8
AD
38
39@quotation
40Permission is granted to copy, distribute and/or modify this document
8a4281b9 41under the terms of the GNU Free Documentation License,
804e83b2 42Version 1.3 or any later version published by the Free Software
c827f760 43Foundation; with no Invariant Sections, with the Front-Cover texts
8a4281b9 44being ``A GNU Manual,'' and with the Back-Cover Texts as in
c827f760 45(a) below. A copy of the license is included in the section entitled
8a4281b9 46``GNU Free Documentation License.''
c827f760 47
389c8cfd 48(a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
8a4281b9
JD
49modify this GNU manual. Buying copies from the FSF
50supports it in developing GNU and promoting software
389c8cfd 51freedom.''
fae437e8
AD
52@end quotation
53@end copying
54
e62f1a89 55@dircategory Software development
fae437e8 56@direntry
8a4281b9 57* bison: (bison). GNU parser generator (Yacc replacement).
fae437e8 58@end direntry
bfa74976 59
bfa74976
RS
60@titlepage
61@title Bison
c827f760 62@subtitle The Yacc-compatible Parser Generator
df1af54c 63@subtitle @value{UPDATED}, Bison Version @value{VERSION}
bfa74976
RS
64
65@author by Charles Donnelly and Richard Stallman
66
67@page
68@vskip 0pt plus 1filll
fae437e8 69@insertcopying
bfa74976
RS
70@sp 2
71Published by the Free Software Foundation @*
0fb669f9
PE
7251 Franklin Street, Fifth Floor @*
73Boston, MA 02110-1301 USA @*
9ecbd125 74Printed copies are available from the Free Software Foundation.@*
8a4281b9 75ISBN 1-882114-44-2
bfa74976
RS
76@sp 2
77Cover art by Etienne Suvasa.
78@end titlepage
d5796688
JT
79
80@contents
bfa74976 81
342b8b6e
AD
82@ifnottex
83@node Top
84@top Bison
fae437e8 85@insertcopying
342b8b6e 86@end ifnottex
bfa74976
RS
87
88@menu
13863333
AD
89* Introduction::
90* Conditions::
8a4281b9 91* Copying:: The GNU General Public License says
f5f419de 92 how you can copy and share Bison.
bfa74976
RS
93
94Tutorial sections:
f5f419de
DJ
95* Concepts:: Basic concepts for understanding Bison.
96* Examples:: Three simple explained examples of using Bison.
bfa74976
RS
97
98Reference sections:
f5f419de
DJ
99* Grammar File:: Writing Bison declarations and rules.
100* Interface:: C-language interface to the parser function @code{yyparse}.
101* Algorithm:: How the Bison parser works at run-time.
102* Error Recovery:: Writing rules for error recovery.
bfa74976 103* Context Dependency:: What to do if your language syntax is too
f5f419de
DJ
104 messy for Bison to handle straightforwardly.
105* Debugging:: Understanding or debugging Bison parsers.
ff7571c0 106* Invocation:: How to run Bison (to produce the parser implementation).
f5f419de
DJ
107* Other Languages:: Creating C++ and Java parsers.
108* FAQ:: Frequently Asked Questions
109* Table of Symbols:: All the keywords of the Bison language are explained.
110* Glossary:: Basic concepts are explained.
111* Copying This Manual:: License for copying this manual.
112* Index:: Cross-references to the text.
bfa74976 113
93dd49ab
PE
114@detailmenu
115 --- The Detailed Node Listing ---
bfa74976
RS
116
117The Concepts of Bison
118
f5f419de
DJ
119* Language and Grammar:: Languages and context-free grammars,
120 as mathematical ideas.
121* Grammar in Bison:: How we represent grammars for Bison's sake.
122* Semantic Values:: Each token or syntactic grouping can have
123 a semantic value (the value of an integer,
124 the name of an identifier, etc.).
125* Semantic Actions:: Each rule can have an action containing C code.
126* GLR Parsers:: Writing parsers for general context-free languages.
127* Locations Overview:: Tracking Locations.
128* Bison Parser:: What are Bison's input and output,
129 how is the output used?
130* Stages:: Stages in writing and running Bison grammars.
131* Grammar Layout:: Overall structure of a Bison grammar file.
bfa74976 132
8a4281b9 133Writing GLR Parsers
fa7e68c3 134
8a4281b9
JD
135* Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
136* Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
20be2f92 137* GLR Semantic Actions:: Considerations for semantic values and deferred actions.
ca2a6d15 138* Semantic Predicates:: Controlling a parse with arbitrary computations.
8a4281b9 139* Compiler Requirements:: GLR parsers require a modern C compiler.
fa7e68c3 140
bfa74976
RS
141Examples
142
f5f419de
DJ
143* RPN Calc:: Reverse polish notation calculator;
144 a first example with no operator precedence.
145* Infix Calc:: Infix (algebraic) notation calculator.
146 Operator precedence is introduced.
bfa74976 147* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 148* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
f5f419de
DJ
149* Multi-function Calc:: Calculator with memory and trig functions.
150 It uses multiple data-types for semantic values.
151* Exercises:: Ideas for improving the multi-function calculator.
bfa74976
RS
152
153Reverse Polish Notation Calculator
154
f5f419de
DJ
155* Rpcalc Declarations:: Prologue (declarations) for rpcalc.
156* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
157* Rpcalc Lexer:: The lexical analyzer.
158* Rpcalc Main:: The controlling function.
159* Rpcalc Error:: The error reporting function.
160* Rpcalc Generate:: Running Bison on the grammar file.
161* Rpcalc Compile:: Run the C compiler on the output code.
bfa74976
RS
162
163Grammar Rules for @code{rpcalc}
164
13863333
AD
165* Rpcalc Input::
166* Rpcalc Line::
167* Rpcalc Expr::
bfa74976 168
342b8b6e
AD
169Location Tracking Calculator: @code{ltcalc}
170
f5f419de
DJ
171* Ltcalc Declarations:: Bison and C declarations for ltcalc.
172* Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
173* Ltcalc Lexer:: The lexical analyzer.
342b8b6e 174
bfa74976
RS
175Multi-Function Calculator: @code{mfcalc}
176
f5f419de
DJ
177* Mfcalc Declarations:: Bison declarations for multi-function calculator.
178* Mfcalc Rules:: Grammar rules for the calculator.
179* Mfcalc Symbol Table:: Symbol table management subroutines.
bfa74976
RS
180
181Bison Grammar Files
182
183* Grammar Outline:: Overall layout of the grammar file.
184* Symbols:: Terminal and nonterminal symbols.
185* Rules:: How to write grammar rules.
186* Recursion:: Writing recursive rules.
187* Semantics:: Semantic values and actions.
93dd49ab 188* Locations:: Locations and actions.
bfa74976
RS
189* Declarations:: All kinds of Bison declarations are described here.
190* Multiple Parsers:: Putting more than one Bison parser in one program.
191
192Outline of a Bison Grammar
193
f5f419de 194* Prologue:: Syntax and usage of the prologue.
2cbe6b7f 195* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
f5f419de
DJ
196* Bison Declarations:: Syntax and usage of the Bison declarations section.
197* Grammar Rules:: Syntax and usage of the grammar rules section.
198* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
199
200Defining Language Semantics
201
202* Value Type:: Specifying one data type for all semantic values.
203* Multiple Types:: Specifying several alternative data types.
204* Actions:: An action is the semantic definition of a grammar rule.
205* Action Types:: Specifying data types for actions to operate on.
206* Mid-Rule Actions:: Most actions go at the end of a rule.
207 This says when, why and how to use the exceptional
208 action in the middle of a rule.
d013372c 209* Named References:: Using named references in actions.
bfa74976 210
93dd49ab
PE
211Tracking Locations
212
213* Location Type:: Specifying a data type for locations.
214* Actions and Locations:: Using locations in actions.
215* Location Default Action:: Defining a general way to compute locations.
216
bfa74976
RS
217Bison Declarations
218
b50d2359 219* Require Decl:: Requiring a Bison version.
bfa74976
RS
220* Token Decl:: Declaring terminal symbols.
221* Precedence Decl:: Declaring terminals with precedence and associativity.
222* Union Decl:: Declaring the set of all semantic value types.
223* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 224* Initial Action Decl:: Code run before parsing starts.
72f889cc 225* Destructor Decl:: Declaring how symbols are freed.
d6328241 226* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
227* Start Decl:: Specifying the start symbol.
228* Pure Decl:: Requesting a reentrant parser.
9987d1b3 229* Push Decl:: Requesting a push parser.
bfa74976 230* Decl Summary:: Table of all Bison declarations.
35c1e5f0 231* %define Summary:: Defining variables to adjust Bison's behavior.
e0c07222 232* %code Summary:: Inserting code into the parser source.
bfa74976
RS
233
234Parser C-Language Interface
235
f5f419de
DJ
236* Parser Function:: How to call @code{yyparse} and what it returns.
237* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
238* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
239* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
240* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
241* Lexical:: You must supply a function @code{yylex}
242 which reads tokens.
243* Error Reporting:: You must supply a function @code{yyerror}.
244* Action Features:: Special features for use in actions.
245* Internationalization:: How to let the parser speak in the user's
246 native language.
bfa74976
RS
247
248The Lexical Analyzer Function @code{yylex}
249
250* Calling Convention:: How @code{yyparse} calls @code{yylex}.
f5f419de
DJ
251* Token Values:: How @code{yylex} must return the semantic value
252 of the token it has read.
253* Token Locations:: How @code{yylex} must return the text location
254 (line number, etc.) of the token, if the
255 actions want that.
256* Pure Calling:: How the calling convention differs in a pure parser
257 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
bfa74976 258
13863333 259The Bison Parser Algorithm
bfa74976 260
742e4900 261* Lookahead:: Parser looks one token ahead when deciding what to do.
bfa74976
RS
262* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
263* Precedence:: Operator precedence works by resolving conflicts.
264* Contextual Precedence:: When an operator's precedence depends on context.
265* Parser States:: The parser is a finite-state-machine with stack.
266* Reduce/Reduce:: When two rules are applicable in the same situation.
f5f419de 267* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 268* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 269* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
270
271Operator Precedence
272
273* Why Precedence:: An example showing why precedence is needed.
d78f0ac9
AD
274* Using Precedence:: How to specify precedence and associativity.
275* Precedence Only:: How to specify precedence only.
bfa74976
RS
276* Precedence Examples:: How these features are used in the previous example.
277* How Precedence:: How they work.
278
279Handling Context Dependencies
280
281* Semantic Tokens:: Token parsing can depend on the semantic context.
282* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
283* Tie-in Recovery:: Lexical tie-ins have implications for how
284 error recovery rules must be written.
285
93dd49ab 286Debugging Your Parser
ec3bc396
AD
287
288* Understanding:: Understanding the structure of your parser.
289* Tracing:: Tracing the execution of your parser.
290
bfa74976
RS
291Invoking Bison
292
13863333 293* Bison Options:: All the options described in detail,
c827f760 294 in alphabetical order by short options.
bfa74976 295* Option Cross Key:: Alphabetical list of long options.
93dd49ab 296* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
f2b5126e 297
8405b70c 298Parsers Written In Other Languages
12545799
AD
299
300* C++ Parsers:: The interface to generate C++ parser classes
8405b70c 301* Java Parsers:: The interface to generate Java parser classes
12545799
AD
302
303C++ Parsers
304
305* C++ Bison Interface:: Asking for C++ parser generation
306* C++ Semantic Values:: %union vs. C++
307* C++ Location Values:: The position and location classes
308* C++ Parser Interface:: Instantiating and running the parser
309* C++ Scanner Interface:: Exchanges between yylex and parse
8405b70c 310* A Complete C++ Example:: Demonstrating their use
12545799
AD
311
312A Complete C++ Example
313
314* Calc++ --- C++ Calculator:: The specifications
315* Calc++ Parsing Driver:: An active parsing context
316* Calc++ Parser:: A parser class
317* Calc++ Scanner:: A pure C++ Flex scanner
318* Calc++ Top Level:: Conducting the band
319
8405b70c
PB
320Java Parsers
321
f5f419de
DJ
322* Java Bison Interface:: Asking for Java parser generation
323* Java Semantic Values:: %type and %token vs. Java
324* Java Location Values:: The position and location classes
325* Java Parser Interface:: Instantiating and running the parser
326* Java Scanner Interface:: Specifying the scanner for the parser
327* Java Action Features:: Special features for use in actions
328* Java Differences:: Differences between C/C++ and Java Grammars
329* Java Declarations Summary:: List of Bison declarations used with Java
8405b70c 330
d1a1114f
AD
331Frequently Asked Questions
332
f5f419de
DJ
333* Memory Exhausted:: Breaking the Stack Limits
334* How Can I Reset the Parser:: @code{yyparse} Keeps some State
335* Strings are Destroyed:: @code{yylval} Loses Track of Strings
336* Implementing Gotos/Loops:: Control Flow in the Calculator
337* Multiple start-symbols:: Factoring closely related grammars
8a4281b9 338* Secure? Conform?:: Is Bison POSIX safe?
f5f419de
DJ
339* I can't build Bison:: Troubleshooting
340* Where can I find help?:: Troubleshouting
341* Bug Reports:: Troublereporting
342* More Languages:: Parsers in C++, Java, and so on
343* Beta Testing:: Experimenting development versions
344* Mailing Lists:: Meeting other Bison users
d1a1114f 345
f2b5126e
PB
346Copying This Manual
347
f5f419de 348* Copying This Manual:: License for copying this manual.
f2b5126e 349
342b8b6e 350@end detailmenu
bfa74976
RS
351@end menu
352
342b8b6e 353@node Introduction
bfa74976
RS
354@unnumbered Introduction
355@cindex introduction
356
6077da58 357@dfn{Bison} is a general-purpose parser generator that converts an
af28d414
JD
358annotated context-free grammar into a deterministic LR or generalized
359LR (GLR) parser employing LALR(1) parser tables. As an experimental
360feature, Bison can also generate IELR(1) or canonical LR(1) parser
361tables. Once you are proficient with Bison, you can use it to develop
362a wide range of language parsers, from those used in simple desk
363calculators to complex programming languages.
364
365Bison is upward compatible with Yacc: all properly-written Yacc
366grammars ought to work with Bison with no change. Anyone familiar
367with Yacc should be able to use Bison with little trouble. You need
368to be fluent in C or C++ programming in order to use Bison or to
369understand this manual. Java is also supported as an experimental
370feature.
371
372We begin with tutorial chapters that explain the basic concepts of
373using Bison and show three explained examples, each building on the
374last. If you don't know Bison or Yacc, start by reading these
375chapters. Reference chapters follow, which describe specific aspects
376of Bison in detail.
bfa74976 377
679e9935
JD
378Bison was written originally by Robert Corbett. Richard Stallman made
379it Yacc-compatible. Wilfred Hansen of Carnegie Mellon University
380added multi-character string literals and other features. Since then,
381Bison has grown more robust and evolved many other new features thanks
382to the hard work of a long list of volunteers. For details, see the
383@file{THANKS} and @file{ChangeLog} files included in the Bison
384distribution.
931c7513 385
df1af54c 386This edition corresponds to version @value{VERSION} of Bison.
bfa74976 387
342b8b6e 388@node Conditions
bfa74976
RS
389@unnumbered Conditions for Using Bison
390
193d7c70
PE
391The distribution terms for Bison-generated parsers permit using the
392parsers in nonfree programs. Before Bison version 2.2, these extra
8a4281b9 393permissions applied only when Bison was generating LALR(1)
193d7c70 394parsers in C@. And before Bison version 1.24, Bison-generated
262aa8dd 395parsers could be used only in programs that were free software.
a31239f1 396
8a4281b9 397The other GNU programming tools, such as the GNU C
c827f760 398compiler, have never
9ecbd125 399had such a requirement. They could always be used for nonfree
a31239f1
RS
400software. The reason Bison was different was not due to a special
401policy decision; it resulted from applying the usual General Public
402License to all of the Bison source code.
403
ff7571c0
JD
404The main output of the Bison utility---the Bison parser implementation
405file---contains a verbatim copy of a sizable piece of Bison, which is
406the code for the parser's implementation. (The actions from your
407grammar are inserted into this implementation at one point, but most
408of the rest of the implementation is not changed.) When we applied
409the GPL terms to the skeleton code for the parser's implementation,
a31239f1
RS
410the effect was to restrict the use of Bison output to free software.
411
412We didn't change the terms because of sympathy for people who want to
413make software proprietary. @strong{Software should be free.} But we
414concluded that limiting Bison's use to free software was doing little to
415encourage people to make other software free. So we decided to make the
416practical conditions for using Bison match the practical conditions for
8a4281b9 417using the other GNU tools.
bfa74976 418
193d7c70
PE
419This exception applies when Bison is generating code for a parser.
420You can tell whether the exception applies to a Bison output file by
421inspecting the file for text beginning with ``As a special
422exception@dots{}''. The text spells out the exact terms of the
423exception.
262aa8dd 424
f16b0819
PE
425@node Copying
426@unnumbered GNU GENERAL PUBLIC LICENSE
427@include gpl-3.0.texi
bfa74976 428
342b8b6e 429@node Concepts
bfa74976
RS
430@chapter The Concepts of Bison
431
432This chapter introduces many of the basic concepts without which the
433details of Bison will not make sense. If you do not already know how to
434use Bison or Yacc, we suggest you start by reading this chapter carefully.
435
436@menu
f5f419de
DJ
437* Language and Grammar:: Languages and context-free grammars,
438 as mathematical ideas.
439* Grammar in Bison:: How we represent grammars for Bison's sake.
440* Semantic Values:: Each token or syntactic grouping can have
441 a semantic value (the value of an integer,
442 the name of an identifier, etc.).
443* Semantic Actions:: Each rule can have an action containing C code.
444* GLR Parsers:: Writing parsers for general context-free languages.
445* Locations Overview:: Tracking Locations.
446* Bison Parser:: What are Bison's input and output,
447 how is the output used?
448* Stages:: Stages in writing and running Bison grammars.
449* Grammar Layout:: Overall structure of a Bison grammar file.
bfa74976
RS
450@end menu
451
342b8b6e 452@node Language and Grammar
bfa74976
RS
453@section Languages and Context-Free Grammars
454
bfa74976
RS
455@cindex context-free grammar
456@cindex grammar, context-free
457In order for Bison to parse a language, it must be described by a
458@dfn{context-free grammar}. This means that you specify one or more
459@dfn{syntactic groupings} and give rules for constructing them from their
460parts. For example, in the C language, one kind of grouping is called an
461`expression'. One rule for making an expression might be, ``An expression
462can be made of a minus sign and another expression''. Another would be,
463``An expression can be an integer''. As you can see, rules are often
464recursive, but there must be at least one rule which leads out of the
465recursion.
466
8a4281b9 467@cindex BNF
bfa74976
RS
468@cindex Backus-Naur form
469The most common formal system for presenting such rules for humans to read
8a4281b9 470is @dfn{Backus-Naur Form} or ``BNF'', which was developed in
c827f760 471order to specify the language Algol 60. Any grammar expressed in
8a4281b9
JD
472BNF is a context-free grammar. The input to Bison is
473essentially machine-readable BNF.
bfa74976 474
8a4281b9
JD
475@cindex LALR(1) grammars
476@cindex IELR(1) grammars
477@cindex LR(1) grammars
eb45ef3b
JD
478There are various important subclasses of context-free grammars.
479Although it can handle almost all context-free grammars, Bison is
8a4281b9 480optimized for what are called LR(1) grammars.
eb45ef3b
JD
481In brief, in these grammars, it must be possible to tell how to parse
482any portion of an input string with just a single token of lookahead.
483For historical reasons, Bison by default is limited by the additional
8a4281b9 484restrictions of LALR(1), which is hard to explain simply.
c827f760
PE
485@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
486more information on this.
f1b238df 487As an experimental feature, you can escape these additional restrictions by
8a4281b9 488requesting IELR(1) or canonical LR(1) parser tables.
35c1e5f0 489@xref{%define Summary,,lr.type}, to learn how.
bfa74976 490
8a4281b9
JD
491@cindex GLR parsing
492@cindex generalized LR (GLR) parsing
676385e2 493@cindex ambiguous grammars
9d9b8b70 494@cindex nondeterministic parsing
9501dc6e 495
8a4281b9 496Parsers for LR(1) grammars are @dfn{deterministic}, meaning
9501dc6e
AD
497roughly that the next grammar rule to apply at any point in the input is
498uniquely determined by the preceding input and a fixed, finite portion
742e4900 499(called a @dfn{lookahead}) of the remaining input. A context-free
9501dc6e 500grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
e4f85c39 501apply the grammar rules to get the same inputs. Even unambiguous
9d9b8b70 502grammars can be @dfn{nondeterministic}, meaning that no fixed
742e4900 503lookahead always suffices to determine the next grammar rule to apply.
9501dc6e 504With the proper declarations, Bison is also able to parse these more
8a4281b9
JD
505general context-free grammars, using a technique known as GLR
506parsing (for Generalized LR). Bison's GLR parsers
9501dc6e
AD
507are able to handle any context-free grammar for which the number of
508possible parses of any given string is finite.
676385e2 509
bfa74976
RS
510@cindex symbols (abstract)
511@cindex token
512@cindex syntactic grouping
513@cindex grouping, syntactic
9501dc6e
AD
514In the formal grammatical rules for a language, each kind of syntactic
515unit or grouping is named by a @dfn{symbol}. Those which are built by
516grouping smaller constructs according to grammatical rules are called
bfa74976
RS
517@dfn{nonterminal symbols}; those which can't be subdivided are called
518@dfn{terminal symbols} or @dfn{token types}. We call a piece of input
519corresponding to a single terminal symbol a @dfn{token}, and a piece
e0c471a9 520corresponding to a single nonterminal symbol a @dfn{grouping}.
bfa74976
RS
521
522We can use the C language as an example of what symbols, terminal and
9501dc6e
AD
523nonterminal, mean. The tokens of C are identifiers, constants (numeric
524and string), and the various keywords, arithmetic operators and
525punctuation marks. So the terminal symbols of a grammar for C include
526`identifier', `number', `string', plus one symbol for each keyword,
527operator or punctuation mark: `if', `return', `const', `static', `int',
528`char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
529(These tokens can be subdivided into characters, but that is a matter of
bfa74976
RS
530lexicography, not grammar.)
531
532Here is a simple C function subdivided into tokens:
533
9edcd895
AD
534@ifinfo
535@example
536int /* @r{keyword `int'} */
14d4662b 537square (int x) /* @r{identifier, open-paren, keyword `int',}
9edcd895
AD
538 @r{identifier, close-paren} */
539@{ /* @r{open-brace} */
aa08666d
AD
540 return x * x; /* @r{keyword `return', identifier, asterisk,}
541 @r{identifier, semicolon} */
9edcd895
AD
542@} /* @r{close-brace} */
543@end example
544@end ifinfo
545@ifnotinfo
bfa74976
RS
546@example
547int /* @r{keyword `int'} */
14d4662b 548square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-paren} */
bfa74976 549@{ /* @r{open-brace} */
9edcd895 550 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semicolon} */
bfa74976
RS
551@} /* @r{close-brace} */
552@end example
9edcd895 553@end ifnotinfo
bfa74976
RS
554
555The syntactic groupings of C include the expression, the statement, the
556declaration, and the function definition. These are represented in the
557grammar of C by nonterminal symbols `expression', `statement',
558`declaration' and `function definition'. The full grammar uses dozens of
559additional language constructs, each with its own nonterminal symbol, in
560order to express the meanings of these four. The example above is a
561function definition; it contains one declaration, and one statement. In
562the statement, each @samp{x} is an expression and so is @samp{x * x}.
563
564Each nonterminal symbol must have grammatical rules showing how it is made
565out of simpler constructs. For example, one kind of C statement is the
566@code{return} statement; this would be described with a grammar rule which
567reads informally as follows:
568
569@quotation
570A `statement' can be made of a `return' keyword, an `expression' and a
571`semicolon'.
572@end quotation
573
574@noindent
575There would be many other rules for `statement', one for each kind of
576statement in C.
577
578@cindex start symbol
579One nonterminal symbol must be distinguished as the special one which
580defines a complete utterance in the language. It is called the @dfn{start
581symbol}. In a compiler, this means a complete input program. In the C
582language, the nonterminal symbol `sequence of definitions and declarations'
583plays this role.
584
585For example, @samp{1 + 2} is a valid C expression---a valid part of a C
586program---but it is not valid as an @emph{entire} C program. In the
587context-free grammar of C, this follows from the fact that `expression' is
588not the start symbol.
589
590The Bison parser reads a sequence of tokens as its input, and groups the
591tokens using the grammar rules. If the input is valid, the end result is
592that the entire token sequence reduces to a single grouping whose symbol is
593the grammar's start symbol. If we use a grammar for C, the entire input
594must be a `sequence of definitions and declarations'. If not, the parser
595reports a syntax error.
596
342b8b6e 597@node Grammar in Bison
bfa74976
RS
598@section From Formal Rules to Bison Input
599@cindex Bison grammar
600@cindex grammar, Bison
601@cindex formal grammar
602
603A formal grammar is a mathematical construct. To define the language
604for Bison, you must write a file expressing the grammar in Bison syntax:
605a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
606
607A nonterminal symbol in the formal grammar is represented in Bison input
c827f760 608as an identifier, like an identifier in C@. By convention, it should be
bfa74976
RS
609in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
610
611The Bison representation for a terminal symbol is also called a @dfn{token
612type}. Token types as well can be represented as C-like identifiers. By
613convention, these identifiers should be upper case to distinguish them from
614nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
615@code{RETURN}. A terminal symbol that stands for a particular keyword in
616the language should be named after that keyword converted to upper case.
617The terminal symbol @code{error} is reserved for error recovery.
931c7513 618@xref{Symbols}.
bfa74976
RS
619
620A terminal symbol can also be represented as a character literal, just like
621a C character constant. You should do this whenever a token is just a
622single character (parenthesis, plus-sign, etc.): use that same character in
623a literal as the terminal symbol for that token.
624
931c7513
RS
625A third way to represent a terminal symbol is with a C string constant
626containing several characters. @xref{Symbols}, for more information.
627
bfa74976
RS
628The grammar rules also have an expression in Bison syntax. For example,
629here is the Bison rule for a C @code{return} statement. The semicolon in
630quotes is a literal character token, representing part of the C syntax for
631the statement; the naked semicolon, and the colon, are Bison punctuation
632used in every rule.
633
634@example
635stmt: RETURN expr ';'
636 ;
637@end example
638
639@noindent
640@xref{Rules, ,Syntax of Grammar Rules}.
641
342b8b6e 642@node Semantic Values
bfa74976
RS
643@section Semantic Values
644@cindex semantic value
645@cindex value, semantic
646
647A formal grammar selects tokens only by their classifications: for example,
648if a rule mentions the terminal symbol `integer constant', it means that
649@emph{any} integer constant is grammatically valid in that position. The
650precise value of the constant is irrelevant to how to parse the input: if
651@samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
e0c471a9 652grammatical.
bfa74976
RS
653
654But the precise value is very important for what the input means once it is
655parsed. A compiler is useless if it fails to distinguish between 4, 1 and
6563989 as constants in the program! Therefore, each token in a Bison grammar
c827f760
PE
657has both a token type and a @dfn{semantic value}. @xref{Semantics,
658,Defining Language Semantics},
bfa74976
RS
659for details.
660
661The token type is a terminal symbol defined in the grammar, such as
662@code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
663you need to know to decide where the token may validly appear and how to
664group it with other tokens. The grammar rules know nothing about tokens
e0c471a9 665except their types.
bfa74976
RS
666
667The semantic value has all the rest of the information about the
668meaning of the token, such as the value of an integer, or the name of an
669identifier. (A token such as @code{','} which is just punctuation doesn't
670need to have any semantic value.)
671
672For example, an input token might be classified as token type
673@code{INTEGER} and have the semantic value 4. Another input token might
674have the same token type @code{INTEGER} but value 3989. When a grammar
675rule says that @code{INTEGER} is allowed, either of these tokens is
676acceptable because each is an @code{INTEGER}. When the parser accepts the
677token, it keeps track of the token's semantic value.
678
679Each grouping can also have a semantic value as well as its nonterminal
680symbol. For example, in a calculator, an expression typically has a
681semantic value that is a number. In a compiler for a programming
682language, an expression typically has a semantic value that is a tree
683structure describing the meaning of the expression.
684
342b8b6e 685@node Semantic Actions
bfa74976
RS
686@section Semantic Actions
687@cindex semantic actions
688@cindex actions, semantic
689
690In order to be useful, a program must do more than parse input; it must
691also produce some output based on the input. In a Bison grammar, a grammar
692rule can have an @dfn{action} made up of C statements. Each time the
693parser recognizes a match for that rule, the action is executed.
694@xref{Actions}.
13863333 695
bfa74976
RS
696Most of the time, the purpose of an action is to compute the semantic value
697of the whole construct from the semantic values of its parts. For example,
698suppose we have a rule which says an expression can be the sum of two
699expressions. When the parser recognizes such a sum, each of the
700subexpressions has a semantic value which describes how it was built up.
701The action for this rule should create a similar sort of value for the
702newly recognized larger expression.
703
704For example, here is a rule that says an expression can be the sum of
705two subexpressions:
706
707@example
708expr: expr '+' expr @{ $$ = $1 + $3; @}
709 ;
710@end example
711
712@noindent
713The action says how to produce the semantic value of the sum expression
714from the values of the two subexpressions.
715
676385e2 716@node GLR Parsers
8a4281b9
JD
717@section Writing GLR Parsers
718@cindex GLR parsing
719@cindex generalized LR (GLR) parsing
676385e2
PH
720@findex %glr-parser
721@cindex conflicts
722@cindex shift/reduce conflicts
fa7e68c3 723@cindex reduce/reduce conflicts
676385e2 724
eb45ef3b 725In some grammars, Bison's deterministic
8a4281b9 726LR(1) parsing algorithm cannot decide whether to apply a
9501dc6e
AD
727certain grammar rule at a given point. That is, it may not be able to
728decide (on the basis of the input read so far) which of two possible
729reductions (applications of a grammar rule) applies, or whether to apply
730a reduction or read more of the input and apply a reduction later in the
731input. These are known respectively as @dfn{reduce/reduce} conflicts
732(@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
733(@pxref{Shift/Reduce}).
734
8a4281b9 735To use a grammar that is not easily modified to be LR(1), a
9501dc6e 736more general parsing algorithm is sometimes necessary. If you include
676385e2 737@code{%glr-parser} among the Bison declarations in your file
8a4281b9
JD
738(@pxref{Grammar Outline}), the result is a Generalized LR
739(GLR) parser. These parsers handle Bison grammars that
9501dc6e 740contain no unresolved conflicts (i.e., after applying precedence
eb45ef3b 741declarations) identically to deterministic parsers. However, when
9501dc6e 742faced with unresolved shift/reduce and reduce/reduce conflicts,
8a4281b9 743GLR parsers use the simple expedient of doing both,
9501dc6e
AD
744effectively cloning the parser to follow both possibilities. Each of
745the resulting parsers can again split, so that at any given time, there
746can be any number of possible parses being explored. The parsers
676385e2
PH
747proceed in lockstep; that is, all of them consume (shift) a given input
748symbol before any of them proceed to the next. Each of the cloned
749parsers eventually meets one of two possible fates: either it runs into
750a parsing error, in which case it simply vanishes, or it merges with
751another parser, because the two of them have reduced the input to an
752identical set of symbols.
753
754During the time that there are multiple parsers, semantic actions are
755recorded, but not performed. When a parser disappears, its recorded
756semantic actions disappear as well, and are never performed. When a
757reduction makes two parsers identical, causing them to merge, Bison
758records both sets of semantic actions. Whenever the last two parsers
759merge, reverting to the single-parser case, Bison resolves all the
760outstanding actions either by precedences given to the grammar rules
761involved, or by performing both actions, and then calling a designated
762user-defined function on the resulting values to produce an arbitrary
763merged result.
764
fa7e68c3 765@menu
8a4281b9
JD
766* Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
767* Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
20be2f92 768* GLR Semantic Actions:: Considerations for semantic values and deferred actions.
ca2a6d15 769* Semantic Predicates:: Controlling a parse with arbitrary computations.
8a4281b9 770* Compiler Requirements:: GLR parsers require a modern C compiler.
fa7e68c3
PE
771@end menu
772
773@node Simple GLR Parsers
8a4281b9
JD
774@subsection Using GLR on Unambiguous Grammars
775@cindex GLR parsing, unambiguous grammars
776@cindex generalized LR (GLR) parsing, unambiguous grammars
fa7e68c3
PE
777@findex %glr-parser
778@findex %expect-rr
779@cindex conflicts
780@cindex reduce/reduce conflicts
781@cindex shift/reduce conflicts
782
8a4281b9
JD
783In the simplest cases, you can use the GLR algorithm
784to parse grammars that are unambiguous but fail to be LR(1).
eb45ef3b 785Such grammars typically require more than one symbol of lookahead.
fa7e68c3
PE
786
787Consider a problem that
788arises in the declaration of enumerated and subrange types in the
789programming language Pascal. Here are some examples:
790
791@example
792type subrange = lo .. hi;
793type enum = (a, b, c);
794@end example
795
796@noindent
797The original language standard allows only numeric
798literals and constant identifiers for the subrange bounds (@samp{lo}
8a4281b9 799and @samp{hi}), but Extended Pascal (ISO/IEC
fa7e68c3
PE
80010206) and many other
801Pascal implementations allow arbitrary expressions there. This gives
802rise to the following situation, containing a superfluous pair of
803parentheses:
804
805@example
806type subrange = (a) .. b;
807@end example
808
809@noindent
810Compare this to the following declaration of an enumerated
811type with only one value:
812
813@example
814type enum = (a);
815@end example
816
817@noindent
818(These declarations are contrived, but they are syntactically
819valid, and more-complicated cases can come up in practical programs.)
820
821These two declarations look identical until the @samp{..} token.
8a4281b9 822With normal LR(1) one-token lookahead it is not
fa7e68c3
PE
823possible to decide between the two forms when the identifier
824@samp{a} is parsed. It is, however, desirable
825for a parser to decide this, since in the latter case
826@samp{a} must become a new identifier to represent the enumeration
827value, while in the former case @samp{a} must be evaluated with its
828current meaning, which may be a constant or even a function call.
829
830You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
831to be resolved later, but this typically requires substantial
832contortions in both semantic actions and large parts of the
833grammar, where the parentheses are nested in the recursive rules for
834expressions.
835
836You might think of using the lexer to distinguish between the two
837forms by returning different tokens for currently defined and
838undefined identifiers. But if these declarations occur in a local
839scope, and @samp{a} is defined in an outer scope, then both forms
840are possible---either locally redefining @samp{a}, or using the
841value of @samp{a} from the outer scope. So this approach cannot
842work.
843
e757bb10 844A simple solution to this problem is to declare the parser to
8a4281b9
JD
845use the GLR algorithm.
846When the GLR parser reaches the critical state, it
fa7e68c3
PE
847merely splits into two branches and pursues both syntax rules
848simultaneously. Sooner or later, one of them runs into a parsing
849error. If there is a @samp{..} token before the next
850@samp{;}, the rule for enumerated types fails since it cannot
851accept @samp{..} anywhere; otherwise, the subrange type rule
852fails since it requires a @samp{..} token. So one of the branches
853fails silently, and the other one continues normally, performing
854all the intermediate actions that were postponed during the split.
855
856If the input is syntactically incorrect, both branches fail and the parser
857reports a syntax error as usual.
858
859The effect of all this is that the parser seems to ``guess'' the
860correct branch to take, or in other words, it seems to use more
8a4281b9
JD
861lookahead than the underlying LR(1) algorithm actually allows
862for. In this example, LR(2) would suffice, but also some cases
863that are not LR(@math{k}) for any @math{k} can be handled this way.
fa7e68c3 864
8a4281b9 865In general, a GLR parser can take quadratic or cubic worst-case time,
fa7e68c3
PE
866and the current Bison parser even takes exponential time and space
867for some grammars. In practice, this rarely happens, and for many
868grammars it is possible to prove that it cannot happen.
869The present example contains only one conflict between two
870rules, and the type-declaration context containing the conflict
871cannot be nested. So the number of
872branches that can exist at any time is limited by the constant 2,
873and the parsing time is still linear.
874
875Here is a Bison grammar corresponding to the example above. It
876parses a vastly simplified form of Pascal type declarations.
877
878@example
879%token TYPE DOTDOT ID
880
881@group
882%left '+' '-'
883%left '*' '/'
884@end group
885
886%%
887
888@group
889type_decl : TYPE ID '=' type ';'
890 ;
891@end group
892
893@group
894type : '(' id_list ')'
895 | expr DOTDOT expr
896 ;
897@end group
898
899@group
900id_list : ID
901 | id_list ',' ID
902 ;
903@end group
904
905@group
906expr : '(' expr ')'
907 | expr '+' expr
908 | expr '-' expr
909 | expr '*' expr
910 | expr '/' expr
911 | ID
912 ;
913@end group
914@end example
915
8a4281b9 916When used as a normal LR(1) grammar, Bison correctly complains
fa7e68c3
PE
917about one reduce/reduce conflict. In the conflicting situation the
918parser chooses one of the alternatives, arbitrarily the one
919declared first. Therefore the following correct input is not
920recognized:
921
922@example
923type t = (a) .. b;
924@end example
925
8a4281b9 926The parser can be turned into a GLR parser, while also telling Bison
ff7571c0
JD
927to be silent about the one known reduce/reduce conflict, by adding
928these two declarations to the Bison grammar file (before the first
fa7e68c3
PE
929@samp{%%}):
930
931@example
932%glr-parser
933%expect-rr 1
934@end example
935
936@noindent
937No change in the grammar itself is required. Now the
938parser recognizes all valid declarations, according to the
939limited syntax above, transparently. In fact, the user does not even
940notice when the parser splits.
941
8a4281b9 942So here we have a case where we can use the benefits of GLR,
f8e1c9e5
AD
943almost without disadvantages. Even in simple cases like this, however,
944there are at least two potential problems to beware. First, always
8a4281b9
JD
945analyze the conflicts reported by Bison to make sure that GLR
946splitting is only done where it is intended. A GLR parser
f8e1c9e5 947splitting inadvertently may cause problems less obvious than an
8a4281b9 948LR parser statically choosing the wrong alternative in a
f8e1c9e5
AD
949conflict. Second, consider interactions with the lexer (@pxref{Semantic
950Tokens}) with great care. Since a split parser consumes tokens without
951performing any actions during the split, the lexer cannot obtain
952information via parser actions. Some cases of lexer interactions can be
8a4281b9 953eliminated by using GLR to shift the complications from the
f8e1c9e5
AD
954lexer to the parser. You must check the remaining cases for
955correctness.
956
957In our example, it would be safe for the lexer to return tokens based on
958their current meanings in some symbol table, because no new symbols are
959defined in the middle of a type declaration. Though it is possible for
960a parser to define the enumeration constants as they are parsed, before
961the type declaration is completed, it actually makes no difference since
962they cannot be used within the same enumerated type declaration.
fa7e68c3
PE
963
964@node Merging GLR Parses
8a4281b9
JD
965@subsection Using GLR to Resolve Ambiguities
966@cindex GLR parsing, ambiguous grammars
967@cindex generalized LR (GLR) parsing, ambiguous grammars
fa7e68c3
PE
968@findex %dprec
969@findex %merge
970@cindex conflicts
971@cindex reduce/reduce conflicts
972
2a8d363a 973Let's consider an example, vastly simplified from a C++ grammar.
676385e2
PH
974
975@example
976%@{
38a92d50
PE
977 #include <stdio.h>
978 #define YYSTYPE char const *
979 int yylex (void);
980 void yyerror (char const *);
676385e2
PH
981%@}
982
983%token TYPENAME ID
984
985%right '='
986%left '+'
987
988%glr-parser
989
990%%
991
fae437e8 992prog :
676385e2
PH
993 | prog stmt @{ printf ("\n"); @}
994 ;
995
996stmt : expr ';' %dprec 1
997 | decl %dprec 2
998 ;
999
2a8d363a 1000expr : ID @{ printf ("%s ", $$); @}
fae437e8 1001 | TYPENAME '(' expr ')'
2a8d363a
AD
1002 @{ printf ("%s <cast> ", $1); @}
1003 | expr '+' expr @{ printf ("+ "); @}
1004 | expr '=' expr @{ printf ("= "); @}
676385e2
PH
1005 ;
1006
fae437e8 1007decl : TYPENAME declarator ';'
2a8d363a 1008 @{ printf ("%s <declare> ", $1); @}
676385e2 1009 | TYPENAME declarator '=' expr ';'
2a8d363a 1010 @{ printf ("%s <init-declare> ", $1); @}
676385e2
PH
1011 ;
1012
2a8d363a 1013declarator : ID @{ printf ("\"%s\" ", $1); @}
676385e2
PH
1014 | '(' declarator ')'
1015 ;
1016@end example
1017
1018@noindent
1019This models a problematic part of the C++ grammar---the ambiguity between
1020certain declarations and statements. For example,
1021
1022@example
1023T (x) = y+z;
1024@end example
1025
1026@noindent
1027parses as either an @code{expr} or a @code{stmt}
c827f760
PE
1028(assuming that @samp{T} is recognized as a @code{TYPENAME} and
1029@samp{x} as an @code{ID}).
676385e2 1030Bison detects this as a reduce/reduce conflict between the rules
fae437e8 1031@code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
e757bb10 1032time it encounters @code{x} in the example above. Since this is a
8a4281b9 1033GLR parser, it therefore splits the problem into two parses, one for
fa7e68c3
PE
1034each choice of resolving the reduce/reduce conflict.
1035Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1036however, neither of these parses ``dies,'' because the grammar as it stands is
e757bb10
AD
1037ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1038the other reduces @code{stmt : decl}, after which both parsers are in an
1039identical state: they've seen @samp{prog stmt} and have the same unprocessed
1040input remaining. We say that these parses have @dfn{merged.}
fa7e68c3 1041
8a4281b9 1042At this point, the GLR parser requires a specification in the
fa7e68c3
PE
1043grammar of how to choose between the competing parses.
1044In the example above, the two @code{%dprec}
e757bb10 1045declarations specify that Bison is to give precedence
fa7e68c3 1046to the parse that interprets the example as a
676385e2
PH
1047@code{decl}, which implies that @code{x} is a declarator.
1048The parser therefore prints
1049
1050@example
fae437e8 1051"x" y z + T <init-declare>
676385e2
PH
1052@end example
1053
fa7e68c3
PE
1054The @code{%dprec} declarations only come into play when more than one
1055parse survives. Consider a different input string for this parser:
676385e2
PH
1056
1057@example
1058T (x) + y;
1059@end example
1060
1061@noindent
8a4281b9 1062This is another example of using GLR to parse an unambiguous
fa7e68c3 1063construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
676385e2
PH
1064Here, there is no ambiguity (this cannot be parsed as a declaration).
1065However, at the time the Bison parser encounters @code{x}, it does not
1066have enough information to resolve the reduce/reduce conflict (again,
1067between @code{x} as an @code{expr} or a @code{declarator}). In this
fa7e68c3 1068case, no precedence declaration is used. Again, the parser splits
676385e2
PH
1069into two, one assuming that @code{x} is an @code{expr}, and the other
1070assuming @code{x} is a @code{declarator}. The second of these parsers
1071then vanishes when it sees @code{+}, and the parser prints
1072
1073@example
fae437e8 1074x T <cast> y +
676385e2
PH
1075@end example
1076
1077Suppose that instead of resolving the ambiguity, you wanted to see all
fa7e68c3 1078the possibilities. For this purpose, you must merge the semantic
676385e2
PH
1079actions of the two possible parsers, rather than choosing one over the
1080other. To do so, you could change the declaration of @code{stmt} as
1081follows:
1082
1083@example
1084stmt : expr ';' %merge <stmtMerge>
1085 | decl %merge <stmtMerge>
1086 ;
1087@end example
1088
1089@noindent
676385e2
PH
1090and define the @code{stmtMerge} function as:
1091
1092@example
38a92d50
PE
1093static YYSTYPE
1094stmtMerge (YYSTYPE x0, YYSTYPE x1)
676385e2
PH
1095@{
1096 printf ("<OR> ");
1097 return "";
1098@}
1099@end example
1100
1101@noindent
1102with an accompanying forward declaration
1103in the C declarations at the beginning of the file:
1104
1105@example
1106%@{
38a92d50 1107 #define YYSTYPE char const *
676385e2
PH
1108 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1109%@}
1110@end example
1111
1112@noindent
fa7e68c3
PE
1113With these declarations, the resulting parser parses the first example
1114as both an @code{expr} and a @code{decl}, and prints
676385e2
PH
1115
1116@example
fae437e8 1117"x" y z + T <init-declare> x T <cast> y z + = <OR>
676385e2
PH
1118@end example
1119
fa7e68c3 1120Bison requires that all of the
e757bb10 1121productions that participate in any particular merge have identical
fa7e68c3
PE
1122@samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1123and the parser will report an error during any parse that results in
1124the offending merge.
9501dc6e 1125
32c29292
JD
1126@node GLR Semantic Actions
1127@subsection GLR Semantic Actions
1128
8a4281b9 1129The nature of GLR parsing and the structure of the generated
20be2f92
PH
1130parsers give rise to certain restrictions on semantic values and actions.
1131
1132@subsubsection Deferred semantic actions
32c29292
JD
1133@cindex deferred semantic actions
1134By definition, a deferred semantic action is not performed at the same time as
1135the associated reduction.
1136This raises caveats for several Bison features you might use in a semantic
8a4281b9 1137action in a GLR parser.
32c29292
JD
1138
1139@vindex yychar
8a4281b9 1140@cindex GLR parsers and @code{yychar}
32c29292 1141@vindex yylval
8a4281b9 1142@cindex GLR parsers and @code{yylval}
32c29292 1143@vindex yylloc
8a4281b9 1144@cindex GLR parsers and @code{yylloc}
32c29292 1145In any semantic action, you can examine @code{yychar} to determine the type of
742e4900 1146the lookahead token present at the time of the associated reduction.
32c29292
JD
1147After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1148you can then examine @code{yylval} and @code{yylloc} to determine the
742e4900 1149lookahead token's semantic value and location, if any.
32c29292
JD
1150In a nondeferred semantic action, you can also modify any of these variables to
1151influence syntax analysis.
742e4900 1152@xref{Lookahead, ,Lookahead Tokens}.
32c29292
JD
1153
1154@findex yyclearin
8a4281b9 1155@cindex GLR parsers and @code{yyclearin}
32c29292
JD
1156In a deferred semantic action, it's too late to influence syntax analysis.
1157In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1158shallow copies of the values they had at the time of the associated reduction.
1159For this reason alone, modifying them is dangerous.
1160Moreover, the result of modifying them is undefined and subject to change with
1161future versions of Bison.
1162For example, if a semantic action might be deferred, you should never write it
1163to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1164memory referenced by @code{yylval}.
1165
20be2f92 1166@subsubsection YYERROR
32c29292 1167@findex YYERROR
8a4281b9 1168@cindex GLR parsers and @code{YYERROR}
32c29292 1169Another Bison feature requiring special consideration is @code{YYERROR}
8710fc41 1170(@pxref{Action Features}), which you can invoke in a semantic action to
32c29292 1171initiate error recovery.
8a4281b9 1172During deterministic GLR operation, the effect of @code{YYERROR} is
eb45ef3b 1173the same as its effect in a deterministic parser.
20be2f92
PH
1174The effect in a deferred action is similar, but the precise point of the
1175error is undefined; instead, the parser reverts to deterministic operation,
1176selecting an unspecified stack on which to continue with a syntax error.
1177In a semantic predicate (see @ref{Semantic Predicates}) during nondeterministic
1178parsing, @code{YYERROR} silently prunes
1179the parse that invoked the test.
1180
1181@subsubsection Restrictions on semantic values and locations
8a4281b9 1182GLR parsers require that you use POD (Plain Old Data) types for
20be2f92
PH
1183semantic values and location types when using the generated parsers as
1184C++ code.
8710fc41 1185
ca2a6d15
PH
1186@node Semantic Predicates
1187@subsection Controlling a Parse with Arbitrary Predicates
1188@findex %?
8a4281b9 1189@cindex Semantic predicates in GLR parsers
ca2a6d15
PH
1190
1191In addition to the @code{%dprec} and @code{%merge} directives,
8a4281b9 1192GLR parsers
ca2a6d15
PH
1193allow you to reject parses on the basis of arbitrary computations executed
1194in user code, without having Bison treat this rejection as an error
1195if there are alternative parses. (This feature is experimental and may
1196evolve. We welcome user feedback.) For example,
1197
1198@smallexample
1199widget :
1200 %?@{ new_syntax @} "widget" id new_args @{ $$ = f($3, $4); @}
1201 | %?@{ !new_syntax @} "widget" id old_args @{ $$ = f($3, $4); @}
1202 ;
1203@end smallexample
1204
1205@noindent
1206is one way to allow the same parser to handle two different syntaxes for
1207widgets. The clause preceded by @code{%?} is treated like an ordinary
1208action, except that its text is treated as an expression and is always
1209evaluated immediately (even when in nondeterministic mode). If the
1210expression yields 0 (false), the clause is treated as a syntax error,
1211which, in a nondeterministic parser, causes the stack in which it is reduced
1212to die. In a deterministic parser, it acts like YYERROR.
1213
1214As the example shows, predicates otherwise look like semantic actions, and
1215therefore you must be take them into account when determining the numbers
1216to use for denoting the semantic values of right-hand side symbols.
1217Predicate actions, however, have no defined value, and may not be given
1218labels.
1219
1220There is a subtle difference between semantic predicates and ordinary
1221actions in nondeterministic mode, since the latter are deferred.
1222For example, we could try to rewrite the previous example as
1223
1224@smallexample
1225widget :
1226 @{ if (!new_syntax) YYERROR; @} "widget" id new_args @{ $$ = f($3, $4); @}
1227 | @{ if (new_syntax) YYERROR; @} "widget" id old_args @{ $$ = f($3, $4); @}
1228 ;
1229@end smallexample
1230
1231@noindent
1232(reversing the sense of the predicate tests to cause an error when they are
1233false). However, this
1234does @emph{not} have the same effect if @code{new_args} and @code{old_args}
1235have overlapping syntax.
1236Since the mid-rule actions testing @code{new_syntax} are deferred,
8a4281b9 1237a GLR parser first encounters the unresolved ambiguous reduction
ca2a6d15
PH
1238for cases where @code{new_args} and @code{old_args} recognize the same string
1239@emph{before} performing the tests of @code{new_syntax}. It therefore
1240reports an error.
1241
1242Finally, be careful in writing predicates: deferred actions have not been
1243evaluated, so that using them in a predicate will have undefined effects.
1244
fa7e68c3 1245@node Compiler Requirements
8a4281b9 1246@subsection Considerations when Compiling GLR Parsers
fa7e68c3 1247@cindex @code{inline}
8a4281b9 1248@cindex GLR parsers and @code{inline}
fa7e68c3 1249
8a4281b9 1250The GLR parsers require a compiler for ISO C89 or
38a92d50
PE
1251later. In addition, they use the @code{inline} keyword, which is not
1252C89, but is C99 and is a common extension in pre-C99 compilers. It is
1253up to the user of these parsers to handle
9501dc6e
AD
1254portability issues. For instance, if using Autoconf and the Autoconf
1255macro @code{AC_C_INLINE}, a mere
1256
1257@example
1258%@{
38a92d50 1259 #include <config.h>
9501dc6e
AD
1260%@}
1261@end example
1262
1263@noindent
1264will suffice. Otherwise, we suggest
1265
1266@example
1267%@{
38a92d50
PE
1268 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1269 #define inline
1270 #endif
9501dc6e
AD
1271%@}
1272@end example
676385e2 1273
342b8b6e 1274@node Locations Overview
847bf1f5
AD
1275@section Locations
1276@cindex location
95923bd6
AD
1277@cindex textual location
1278@cindex location, textual
847bf1f5
AD
1279
1280Many applications, like interpreters or compilers, have to produce verbose
72d2299c 1281and useful error messages. To achieve this, one must be able to keep track of
95923bd6 1282the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
847bf1f5
AD
1283Bison provides a mechanism for handling these locations.
1284
72d2299c 1285Each token has a semantic value. In a similar fashion, each token has an
847bf1f5 1286associated location, but the type of locations is the same for all tokens and
72d2299c 1287groupings. Moreover, the output parser is equipped with a default data
847bf1f5
AD
1288structure for storing locations (@pxref{Locations}, for more details).
1289
1290Like semantic values, locations can be reached in actions using a dedicated
72d2299c 1291set of constructs. In the example above, the location of the whole grouping
847bf1f5
AD
1292is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1293@code{@@3}.
1294
1295When a rule is matched, a default action is used to compute the semantic value
72d2299c
PE
1296of its left hand side (@pxref{Actions}). In the same way, another default
1297action is used for locations. However, the action for locations is general
847bf1f5 1298enough for most cases, meaning there is usually no need to describe for each
72d2299c 1299rule how @code{@@$} should be formed. When building a new location for a given
847bf1f5
AD
1300grouping, the default behavior of the output parser is to take the beginning
1301of the first symbol, and the end of the last symbol.
1302
342b8b6e 1303@node Bison Parser
ff7571c0 1304@section Bison Output: the Parser Implementation File
bfa74976
RS
1305@cindex Bison parser
1306@cindex Bison utility
1307@cindex lexical analyzer, purpose
1308@cindex parser
1309
ff7571c0
JD
1310When you run Bison, you give it a Bison grammar file as input. The
1311most important output is a C source file that implements a parser for
1312the language described by the grammar. This parser is called a
1313@dfn{Bison parser}, and this file is called a @dfn{Bison parser
1314implementation file}. Keep in mind that the Bison utility and the
1315Bison parser are two distinct programs: the Bison utility is a program
1316whose output is the Bison parser implementation file that becomes part
1317of your program.
bfa74976
RS
1318
1319The job of the Bison parser is to group tokens into groupings according to
1320the grammar rules---for example, to build identifiers and operators into
1321expressions. As it does this, it runs the actions for the grammar rules it
1322uses.
1323
704a47c4
AD
1324The tokens come from a function called the @dfn{lexical analyzer} that
1325you must supply in some fashion (such as by writing it in C). The Bison
1326parser calls the lexical analyzer each time it wants a new token. It
1327doesn't know what is ``inside'' the tokens (though their semantic values
1328may reflect this). Typically the lexical analyzer makes the tokens by
1329parsing characters of text, but Bison does not depend on this.
1330@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
bfa74976 1331
ff7571c0
JD
1332The Bison parser implementation file is C code which defines a
1333function named @code{yyparse} which implements that grammar. This
1334function does not make a complete C program: you must supply some
1335additional functions. One is the lexical analyzer. Another is an
1336error-reporting function which the parser calls to report an error.
1337In addition, a complete C program must start with a function called
1338@code{main}; you have to provide this, and arrange for it to call
1339@code{yyparse} or the parser will never run. @xref{Interface, ,Parser
1340C-Language Interface}.
bfa74976 1341
f7ab6a50 1342Aside from the token type names and the symbols in the actions you
ff7571c0
JD
1343write, all symbols defined in the Bison parser implementation file
1344itself begin with @samp{yy} or @samp{YY}. This includes interface
1345functions such as the lexical analyzer function @code{yylex}, the
1346error reporting function @code{yyerror} and the parser function
1347@code{yyparse} itself. This also includes numerous identifiers used
1348for internal purposes. Therefore, you should avoid using C
1349identifiers starting with @samp{yy} or @samp{YY} in the Bison grammar
1350file except for the ones defined in this manual. Also, you should
1351avoid using the C identifiers @samp{malloc} and @samp{free} for
1352anything other than their usual meanings.
1353
1354In some cases the Bison parser implementation file includes system
1355headers, and in those cases your code should respect the identifiers
1356reserved by those headers. On some non-GNU hosts, @code{<alloca.h>},
1357@code{<malloc.h>}, @code{<stddef.h>}, and @code{<stdlib.h>} are
1358included as needed to declare memory allocators and related types.
1359@code{<libintl.h>} is included if message translation is in use
1360(@pxref{Internationalization}). Other system headers may be included
1361if you define @code{YYDEBUG} to a nonzero value (@pxref{Tracing,
1362,Tracing Your Parser}).
7093d0f5 1363
342b8b6e 1364@node Stages
bfa74976
RS
1365@section Stages in Using Bison
1366@cindex stages in using Bison
1367@cindex using Bison
1368
1369The actual language-design process using Bison, from grammar specification
1370to a working compiler or interpreter, has these parts:
1371
1372@enumerate
1373@item
1374Formally specify the grammar in a form recognized by Bison
704a47c4
AD
1375(@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1376in the language, describe the action that is to be taken when an
1377instance of that rule is recognized. The action is described by a
1378sequence of C statements.
bfa74976
RS
1379
1380@item
704a47c4
AD
1381Write a lexical analyzer to process input and pass tokens to the parser.
1382The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1383Lexical Analyzer Function @code{yylex}}). It could also be produced
1384using Lex, but the use of Lex is not discussed in this manual.
bfa74976
RS
1385
1386@item
1387Write a controlling function that calls the Bison-produced parser.
1388
1389@item
1390Write error-reporting routines.
1391@end enumerate
1392
1393To turn this source code as written into a runnable program, you
1394must follow these steps:
1395
1396@enumerate
1397@item
1398Run Bison on the grammar to produce the parser.
1399
1400@item
1401Compile the code output by Bison, as well as any other source files.
1402
1403@item
1404Link the object files to produce the finished product.
1405@end enumerate
1406
342b8b6e 1407@node Grammar Layout
bfa74976
RS
1408@section The Overall Layout of a Bison Grammar
1409@cindex grammar file
1410@cindex file format
1411@cindex format of grammar file
1412@cindex layout of Bison grammar
1413
1414The input file for the Bison utility is a @dfn{Bison grammar file}. The
1415general form of a Bison grammar file is as follows:
1416
1417@example
1418%@{
08e49d20 1419@var{Prologue}
bfa74976
RS
1420%@}
1421
1422@var{Bison declarations}
1423
1424%%
1425@var{Grammar rules}
1426%%
08e49d20 1427@var{Epilogue}
bfa74976
RS
1428@end example
1429
1430@noindent
1431The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1432in every Bison grammar file to separate the sections.
1433
72d2299c 1434The prologue may define types and variables used in the actions. You can
342b8b6e 1435also use preprocessor commands to define macros used there, and use
bfa74976 1436@code{#include} to include header files that do any of these things.
38a92d50
PE
1437You need to declare the lexical analyzer @code{yylex} and the error
1438printer @code{yyerror} here, along with any other global identifiers
1439used by the actions in the grammar rules.
bfa74976
RS
1440
1441The Bison declarations declare the names of the terminal and nonterminal
1442symbols, and may also describe operator precedence and the data types of
1443semantic values of various symbols.
1444
1445The grammar rules define how to construct each nonterminal symbol from its
1446parts.
1447
38a92d50
PE
1448The epilogue can contain any code you want to use. Often the
1449definitions of functions declared in the prologue go here. In a
1450simple program, all the rest of the program can go here.
bfa74976 1451
342b8b6e 1452@node Examples
bfa74976
RS
1453@chapter Examples
1454@cindex simple examples
1455@cindex examples, simple
1456
1457Now we show and explain three sample programs written using Bison: a
1458reverse polish notation calculator, an algebraic (infix) notation
1459calculator, and a multi-function calculator. All three have been tested
1460under BSD Unix 4.3; each produces a usable, though limited, interactive
1461desk-top calculator.
1462
1463These examples are simple, but Bison grammars for real programming
aa08666d
AD
1464languages are written the same way. You can copy these examples into a
1465source file to try them.
bfa74976
RS
1466
1467@menu
f5f419de
DJ
1468* RPN Calc:: Reverse polish notation calculator;
1469 a first example with no operator precedence.
1470* Infix Calc:: Infix (algebraic) notation calculator.
1471 Operator precedence is introduced.
bfa74976 1472* Simple Error Recovery:: Continuing after syntax errors.
342b8b6e 1473* Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
f5f419de
DJ
1474* Multi-function Calc:: Calculator with memory and trig functions.
1475 It uses multiple data-types for semantic values.
1476* Exercises:: Ideas for improving the multi-function calculator.
bfa74976
RS
1477@end menu
1478
342b8b6e 1479@node RPN Calc
bfa74976
RS
1480@section Reverse Polish Notation Calculator
1481@cindex reverse polish notation
1482@cindex polish notation calculator
1483@cindex @code{rpcalc}
1484@cindex calculator, simple
1485
1486The first example is that of a simple double-precision @dfn{reverse polish
1487notation} calculator (a calculator using postfix operators). This example
1488provides a good starting point, since operator precedence is not an issue.
1489The second example will illustrate how operator precedence is handled.
1490
1491The source code for this calculator is named @file{rpcalc.y}. The
ff7571c0 1492@samp{.y} extension is a convention used for Bison grammar files.
bfa74976
RS
1493
1494@menu
f5f419de
DJ
1495* Rpcalc Declarations:: Prologue (declarations) for rpcalc.
1496* Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
1497* Rpcalc Lexer:: The lexical analyzer.
1498* Rpcalc Main:: The controlling function.
1499* Rpcalc Error:: The error reporting function.
1500* Rpcalc Generate:: Running Bison on the grammar file.
1501* Rpcalc Compile:: Run the C compiler on the output code.
bfa74976
RS
1502@end menu
1503
f5f419de 1504@node Rpcalc Declarations
bfa74976
RS
1505@subsection Declarations for @code{rpcalc}
1506
1507Here are the C and Bison declarations for the reverse polish notation
1508calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1509
1510@example
72d2299c 1511/* Reverse polish notation calculator. */
bfa74976
RS
1512
1513%@{
38a92d50
PE
1514 #define YYSTYPE double
1515 #include <math.h>
1516 int yylex (void);
1517 void yyerror (char const *);
bfa74976
RS
1518%@}
1519
1520%token NUM
1521
72d2299c 1522%% /* Grammar rules and actions follow. */
bfa74976
RS
1523@end example
1524
75f5aaea 1525The declarations section (@pxref{Prologue, , The prologue}) contains two
38a92d50 1526preprocessor directives and two forward declarations.
bfa74976
RS
1527
1528The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1964ad8c
AD
1529specifying the C data type for semantic values of both tokens and
1530groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1531Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1532don't define it, @code{int} is the default. Because we specify
1533@code{double}, each token and each expression has an associated value,
1534which is a floating point number.
bfa74976
RS
1535
1536The @code{#include} directive is used to declare the exponentiation
1537function @code{pow}.
1538
38a92d50
PE
1539The forward declarations for @code{yylex} and @code{yyerror} are
1540needed because the C language requires that functions be declared
1541before they are used. These functions will be defined in the
1542epilogue, but the parser calls them so they must be declared in the
1543prologue.
1544
704a47c4
AD
1545The second section, Bison declarations, provides information to Bison
1546about the token types (@pxref{Bison Declarations, ,The Bison
1547Declarations Section}). Each terminal symbol that is not a
1548single-character literal must be declared here. (Single-character
bfa74976
RS
1549literals normally don't need to be declared.) In this example, all the
1550arithmetic operators are designated by single-character literals, so the
1551only terminal symbol that needs to be declared is @code{NUM}, the token
1552type for numeric constants.
1553
342b8b6e 1554@node Rpcalc Rules
bfa74976
RS
1555@subsection Grammar Rules for @code{rpcalc}
1556
1557Here are the grammar rules for the reverse polish notation calculator.
1558
1559@example
1560input: /* empty */
1561 | input line
1562;
1563
1564line: '\n'
18b519c0 1565 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
bfa74976
RS
1566;
1567
18b519c0
AD
1568exp: NUM @{ $$ = $1; @}
1569 | exp exp '+' @{ $$ = $1 + $2; @}
1570 | exp exp '-' @{ $$ = $1 - $2; @}
1571 | exp exp '*' @{ $$ = $1 * $2; @}
1572 | exp exp '/' @{ $$ = $1 / $2; @}
1573 /* Exponentiation */
1574 | exp exp '^' @{ $$ = pow ($1, $2); @}
1575 /* Unary minus */
1576 | exp 'n' @{ $$ = -$1; @}
bfa74976
RS
1577;
1578%%
1579@end example
1580
1581The groupings of the rpcalc ``language'' defined here are the expression
1582(given the name @code{exp}), the line of input (@code{line}), and the
1583complete input transcript (@code{input}). Each of these nonterminal
8c5b881d 1584symbols has several alternate rules, joined by the vertical bar @samp{|}
bfa74976
RS
1585which is read as ``or''. The following sections explain what these rules
1586mean.
1587
1588The semantics of the language is determined by the actions taken when a
1589grouping is recognized. The actions are the C code that appears inside
1590braces. @xref{Actions}.
1591
1592You must specify these actions in C, but Bison provides the means for
1593passing semantic values between the rules. In each action, the
1594pseudo-variable @code{$$} stands for the semantic value for the grouping
1595that the rule is going to construct. Assigning a value to @code{$$} is the
1596main job of most actions. The semantic values of the components of the
1597rule are referred to as @code{$1}, @code{$2}, and so on.
1598
1599@menu
13863333
AD
1600* Rpcalc Input::
1601* Rpcalc Line::
1602* Rpcalc Expr::
bfa74976
RS
1603@end menu
1604
342b8b6e 1605@node Rpcalc Input
bfa74976
RS
1606@subsubsection Explanation of @code{input}
1607
1608Consider the definition of @code{input}:
1609
1610@example
1611input: /* empty */
1612 | input line
1613;
1614@end example
1615
1616This definition reads as follows: ``A complete input is either an empty
1617string, or a complete input followed by an input line''. Notice that
1618``complete input'' is defined in terms of itself. This definition is said
1619to be @dfn{left recursive} since @code{input} appears always as the
1620leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1621
1622The first alternative is empty because there are no symbols between the
1623colon and the first @samp{|}; this means that @code{input} can match an
1624empty string of input (no tokens). We write the rules this way because it
1625is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1626It's conventional to put an empty alternative first and write the comment
1627@samp{/* empty */} in it.
1628
1629The second alternate rule (@code{input line}) handles all nontrivial input.
1630It means, ``After reading any number of lines, read one more line if
1631possible.'' The left recursion makes this rule into a loop. Since the
1632first alternative matches empty input, the loop can be executed zero or
1633more times.
1634
1635The parser function @code{yyparse} continues to process input until a
1636grammatical error is seen or the lexical analyzer says there are no more
72d2299c 1637input tokens; we will arrange for the latter to happen at end-of-input.
bfa74976 1638
342b8b6e 1639@node Rpcalc Line
bfa74976
RS
1640@subsubsection Explanation of @code{line}
1641
1642Now consider the definition of @code{line}:
1643
1644@example
1645line: '\n'
1646 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1647;
1648@end example
1649
1650The first alternative is a token which is a newline character; this means
1651that rpcalc accepts a blank line (and ignores it, since there is no
1652action). The second alternative is an expression followed by a newline.
1653This is the alternative that makes rpcalc useful. The semantic value of
1654the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1655question is the first symbol in the alternative. The action prints this
1656value, which is the result of the computation the user asked for.
1657
1658This action is unusual because it does not assign a value to @code{$$}. As
1659a consequence, the semantic value associated with the @code{line} is
1660uninitialized (its value will be unpredictable). This would be a bug if
1661that value were ever used, but we don't use it: once rpcalc has printed the
1662value of the user's input line, that value is no longer needed.
1663
342b8b6e 1664@node Rpcalc Expr
bfa74976
RS
1665@subsubsection Explanation of @code{expr}
1666
1667The @code{exp} grouping has several rules, one for each kind of expression.
1668The first rule handles the simplest expressions: those that are just numbers.
1669The second handles an addition-expression, which looks like two expressions
1670followed by a plus-sign. The third handles subtraction, and so on.
1671
1672@example
1673exp: NUM
1674 | exp exp '+' @{ $$ = $1 + $2; @}
1675 | exp exp '-' @{ $$ = $1 - $2; @}
1676 @dots{}
1677 ;
1678@end example
1679
1680We have used @samp{|} to join all the rules for @code{exp}, but we could
1681equally well have written them separately:
1682
1683@example
1684exp: NUM ;
1685exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1686exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1687 @dots{}
1688@end example
1689
1690Most of the rules have actions that compute the value of the expression in
1691terms of the value of its parts. For example, in the rule for addition,
1692@code{$1} refers to the first component @code{exp} and @code{$2} refers to
1693the second one. The third component, @code{'+'}, has no meaningful
1694associated semantic value, but if it had one you could refer to it as
1695@code{$3}. When @code{yyparse} recognizes a sum expression using this
1696rule, the sum of the two subexpressions' values is produced as the value of
1697the entire expression. @xref{Actions}.
1698
1699You don't have to give an action for every rule. When a rule has no
1700action, Bison by default copies the value of @code{$1} into @code{$$}.
1701This is what happens in the first rule (the one that uses @code{NUM}).
1702
1703The formatting shown here is the recommended convention, but Bison does
72d2299c 1704not require it. You can add or change white space as much as you wish.
bfa74976
RS
1705For example, this:
1706
1707@example
99a9344e 1708exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
bfa74976
RS
1709@end example
1710
1711@noindent
1712means the same thing as this:
1713
1714@example
1715exp: NUM
1716 | exp exp '+' @{ $$ = $1 + $2; @}
1717 | @dots{}
99a9344e 1718;
bfa74976
RS
1719@end example
1720
1721@noindent
1722The latter, however, is much more readable.
1723
342b8b6e 1724@node Rpcalc Lexer
bfa74976
RS
1725@subsection The @code{rpcalc} Lexical Analyzer
1726@cindex writing a lexical analyzer
1727@cindex lexical analyzer, writing
1728
704a47c4
AD
1729The lexical analyzer's job is low-level parsing: converting characters
1730or sequences of characters into tokens. The Bison parser gets its
1731tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1732Analyzer Function @code{yylex}}.
bfa74976 1733
8a4281b9 1734Only a simple lexical analyzer is needed for the RPN
c827f760 1735calculator. This
bfa74976
RS
1736lexical analyzer skips blanks and tabs, then reads in numbers as
1737@code{double} and returns them as @code{NUM} tokens. Any other character
1738that isn't part of a number is a separate token. Note that the token-code
1739for such a single-character token is the character itself.
1740
1741The return value of the lexical analyzer function is a numeric code which
1742represents a token type. The same text used in Bison rules to stand for
1743this token type is also a C expression for the numeric code for the type.
1744This works in two ways. If the token type is a character literal, then its
e966383b 1745numeric code is that of the character; you can use the same
bfa74976
RS
1746character literal in the lexical analyzer to express the number. If the
1747token type is an identifier, that identifier is defined by Bison as a C
1748macro whose definition is the appropriate number. In this example,
1749therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1750
1964ad8c
AD
1751The semantic value of the token (if it has one) is stored into the
1752global variable @code{yylval}, which is where the Bison parser will look
1753for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
f5f419de 1754defined at the beginning of the grammar; @pxref{Rpcalc Declarations,
1964ad8c 1755,Declarations for @code{rpcalc}}.)
bfa74976 1756
72d2299c
PE
1757A token type code of zero is returned if the end-of-input is encountered.
1758(Bison recognizes any nonpositive value as indicating end-of-input.)
bfa74976
RS
1759
1760Here is the code for the lexical analyzer:
1761
1762@example
1763@group
72d2299c 1764/* The lexical analyzer returns a double floating point
e966383b 1765 number on the stack and the token NUM, or the numeric code
72d2299c
PE
1766 of the character read if not a number. It skips all blanks
1767 and tabs, and returns 0 for end-of-input. */
bfa74976
RS
1768
1769#include <ctype.h>
1770@end group
1771
1772@group
13863333
AD
1773int
1774yylex (void)
bfa74976
RS
1775@{
1776 int c;
1777
72d2299c 1778 /* Skip white space. */
13863333 1779 while ((c = getchar ()) == ' ' || c == '\t')
bfa74976
RS
1780 ;
1781@end group
1782@group
72d2299c 1783 /* Process numbers. */
13863333 1784 if (c == '.' || isdigit (c))
bfa74976
RS
1785 @{
1786 ungetc (c, stdin);
1787 scanf ("%lf", &yylval);
1788 return NUM;
1789 @}
1790@end group
1791@group
72d2299c 1792 /* Return end-of-input. */
13863333 1793 if (c == EOF)
bfa74976 1794 return 0;
72d2299c 1795 /* Return a single char. */
13863333 1796 return c;
bfa74976
RS
1797@}
1798@end group
1799@end example
1800
342b8b6e 1801@node Rpcalc Main
bfa74976
RS
1802@subsection The Controlling Function
1803@cindex controlling function
1804@cindex main function in simple example
1805
1806In keeping with the spirit of this example, the controlling function is
1807kept to the bare minimum. The only requirement is that it call
1808@code{yyparse} to start the process of parsing.
1809
1810@example
1811@group
13863333
AD
1812int
1813main (void)
bfa74976 1814@{
13863333 1815 return yyparse ();
bfa74976
RS
1816@}
1817@end group
1818@end example
1819
342b8b6e 1820@node Rpcalc Error
bfa74976
RS
1821@subsection The Error Reporting Routine
1822@cindex error reporting routine
1823
1824When @code{yyparse} detects a syntax error, it calls the error reporting
13863333 1825function @code{yyerror} to print an error message (usually but not
6e649e65 1826always @code{"syntax error"}). It is up to the programmer to supply
13863333
AD
1827@code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1828here is the definition we will use:
bfa74976
RS
1829
1830@example
1831@group
1832#include <stdio.h>
1833
38a92d50 1834/* Called by yyparse on error. */
13863333 1835void
38a92d50 1836yyerror (char const *s)
bfa74976 1837@{
4e03e201 1838 fprintf (stderr, "%s\n", s);
bfa74976
RS
1839@}
1840@end group
1841@end example
1842
1843After @code{yyerror} returns, the Bison parser may recover from the error
1844and continue parsing if the grammar contains a suitable error rule
1845(@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1846have not written any error rules in this example, so any invalid input will
1847cause the calculator program to exit. This is not clean behavior for a
9ecbd125 1848real calculator, but it is adequate for the first example.
bfa74976 1849
f5f419de 1850@node Rpcalc Generate
bfa74976
RS
1851@subsection Running Bison to Make the Parser
1852@cindex running Bison (introduction)
1853
ceed8467
AD
1854Before running Bison to produce a parser, we need to decide how to
1855arrange all the source code in one or more source files. For such a
ff7571c0
JD
1856simple example, the easiest thing is to put everything in one file,
1857the grammar file. The definitions of @code{yylex}, @code{yyerror} and
1858@code{main} go at the end, in the epilogue of the grammar file
75f5aaea 1859(@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
bfa74976
RS
1860
1861For a large project, you would probably have several source files, and use
1862@code{make} to arrange to recompile them.
1863
ff7571c0
JD
1864With all the source in the grammar file, you use the following command
1865to convert it into a parser implementation file:
bfa74976
RS
1866
1867@example
fa4d969f 1868bison @var{file}.y
bfa74976
RS
1869@end example
1870
1871@noindent
ff7571c0
JD
1872In this example, the grammar file is called @file{rpcalc.y} (for
1873``Reverse Polish @sc{calc}ulator''). Bison produces a parser
1874implementation file named @file{@var{file}.tab.c}, removing the
1875@samp{.y} from the grammar file name. The parser implementation file
1876contains the source code for @code{yyparse}. The additional functions
1877in the grammar file (@code{yylex}, @code{yyerror} and @code{main}) are
1878copied verbatim to the parser implementation file.
bfa74976 1879
342b8b6e 1880@node Rpcalc Compile
ff7571c0 1881@subsection Compiling the Parser Implementation File
bfa74976
RS
1882@cindex compiling the parser
1883
ff7571c0 1884Here is how to compile and run the parser implementation file:
bfa74976
RS
1885
1886@example
1887@group
1888# @r{List files in current directory.}
9edcd895 1889$ @kbd{ls}
bfa74976
RS
1890rpcalc.tab.c rpcalc.y
1891@end group
1892
1893@group
1894# @r{Compile the Bison parser.}
1895# @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
b56471a6 1896$ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
bfa74976
RS
1897@end group
1898
1899@group
1900# @r{List files again.}
9edcd895 1901$ @kbd{ls}
bfa74976
RS
1902rpcalc rpcalc.tab.c rpcalc.y
1903@end group
1904@end example
1905
1906The file @file{rpcalc} now contains the executable code. Here is an
1907example session using @code{rpcalc}.
1908
1909@example
9edcd895
AD
1910$ @kbd{rpcalc}
1911@kbd{4 9 +}
bfa74976 191213
9edcd895 1913@kbd{3 7 + 3 4 5 *+-}
bfa74976 1914-13
9edcd895 1915@kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
bfa74976 191613
9edcd895 1917@kbd{5 6 / 4 n +}
bfa74976 1918-3.166666667
9edcd895 1919@kbd{3 4 ^} @r{Exponentiation}
bfa74976 192081
9edcd895
AD
1921@kbd{^D} @r{End-of-file indicator}
1922$
bfa74976
RS
1923@end example
1924
342b8b6e 1925@node Infix Calc
bfa74976
RS
1926@section Infix Notation Calculator: @code{calc}
1927@cindex infix notation calculator
1928@cindex @code{calc}
1929@cindex calculator, infix notation
1930
1931We now modify rpcalc to handle infix operators instead of postfix. Infix
1932notation involves the concept of operator precedence and the need for
1933parentheses nested to arbitrary depth. Here is the Bison code for
1934@file{calc.y}, an infix desk-top calculator.
1935
1936@example
38a92d50 1937/* Infix notation calculator. */
bfa74976
RS
1938
1939%@{
38a92d50
PE
1940 #define YYSTYPE double
1941 #include <math.h>
1942 #include <stdio.h>
1943 int yylex (void);
1944 void yyerror (char const *);
bfa74976
RS
1945%@}
1946
38a92d50 1947/* Bison declarations. */
bfa74976
RS
1948%token NUM
1949%left '-' '+'
1950%left '*' '/'
d78f0ac9
AD
1951%precedence NEG /* negation--unary minus */
1952%right '^' /* exponentiation */
bfa74976 1953
38a92d50
PE
1954%% /* The grammar follows. */
1955input: /* empty */
bfa74976
RS
1956 | input line
1957;
1958
1959line: '\n'
1960 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1961;
1962
1963exp: NUM @{ $$ = $1; @}
1964 | exp '+' exp @{ $$ = $1 + $3; @}
1965 | exp '-' exp @{ $$ = $1 - $3; @}
1966 | exp '*' exp @{ $$ = $1 * $3; @}
1967 | exp '/' exp @{ $$ = $1 / $3; @}
1968 | '-' exp %prec NEG @{ $$ = -$2; @}
1969 | exp '^' exp @{ $$ = pow ($1, $3); @}
1970 | '(' exp ')' @{ $$ = $2; @}
1971;
1972%%
1973@end example
1974
1975@noindent
ceed8467
AD
1976The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1977same as before.
bfa74976
RS
1978
1979There are two important new features shown in this code.
1980
1981In the second section (Bison declarations), @code{%left} declares token
1982types and says they are left-associative operators. The declarations
1983@code{%left} and @code{%right} (right associativity) take the place of
1984@code{%token} which is used to declare a token type name without
d78f0ac9 1985associativity/precedence. (These tokens are single-character literals, which
bfa74976 1986ordinarily don't need to be declared. We declare them here to specify
d78f0ac9 1987the associativity/precedence.)
bfa74976
RS
1988
1989Operator precedence is determined by the line ordering of the
1990declarations; the higher the line number of the declaration (lower on
1991the page or screen), the higher the precedence. Hence, exponentiation
1992has the highest precedence, unary minus (@code{NEG}) is next, followed
d78f0ac9
AD
1993by @samp{*} and @samp{/}, and so on. Unary minus is not associative,
1994only precedence matters (@code{%precedence}. @xref{Precedence, ,Operator
704a47c4 1995Precedence}.
bfa74976 1996
704a47c4
AD
1997The other important new feature is the @code{%prec} in the grammar
1998section for the unary minus operator. The @code{%prec} simply instructs
1999Bison that the rule @samp{| '-' exp} has the same precedence as
2000@code{NEG}---in this case the next-to-highest. @xref{Contextual
2001Precedence, ,Context-Dependent Precedence}.
bfa74976
RS
2002
2003Here is a sample run of @file{calc.y}:
2004
2005@need 500
2006@example
9edcd895
AD
2007$ @kbd{calc}
2008@kbd{4 + 4.5 - (34/(8*3+-3))}
bfa74976 20096.880952381
9edcd895 2010@kbd{-56 + 2}
bfa74976 2011-54
9edcd895 2012@kbd{3 ^ 2}
bfa74976
RS
20139
2014@end example
2015
342b8b6e 2016@node Simple Error Recovery
bfa74976
RS
2017@section Simple Error Recovery
2018@cindex error recovery, simple
2019
2020Up to this point, this manual has not addressed the issue of @dfn{error
2021recovery}---how to continue parsing after the parser detects a syntax
ceed8467
AD
2022error. All we have handled is error reporting with @code{yyerror}.
2023Recall that by default @code{yyparse} returns after calling
2024@code{yyerror}. This means that an erroneous input line causes the
2025calculator program to exit. Now we show how to rectify this deficiency.
bfa74976
RS
2026
2027The Bison language itself includes the reserved word @code{error}, which
2028may be included in the grammar rules. In the example below it has
2029been added to one of the alternatives for @code{line}:
2030
2031@example
2032@group
2033line: '\n'
2034 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2035 | error '\n' @{ yyerrok; @}
2036;
2037@end group
2038@end example
2039
ceed8467 2040This addition to the grammar allows for simple error recovery in the
6e649e65 2041event of a syntax error. If an expression that cannot be evaluated is
ceed8467
AD
2042read, the error will be recognized by the third rule for @code{line},
2043and parsing will continue. (The @code{yyerror} function is still called
2044upon to print its message as well.) The action executes the statement
2045@code{yyerrok}, a macro defined automatically by Bison; its meaning is
2046that error recovery is complete (@pxref{Error Recovery}). Note the
2047difference between @code{yyerrok} and @code{yyerror}; neither one is a
e0c471a9 2048misprint.
bfa74976
RS
2049
2050This form of error recovery deals with syntax errors. There are other
2051kinds of errors; for example, division by zero, which raises an exception
2052signal that is normally fatal. A real calculator program must handle this
2053signal and use @code{longjmp} to return to @code{main} and resume parsing
2054input lines; it would also have to discard the rest of the current line of
2055input. We won't discuss this issue further because it is not specific to
2056Bison programs.
2057
342b8b6e
AD
2058@node Location Tracking Calc
2059@section Location Tracking Calculator: @code{ltcalc}
2060@cindex location tracking calculator
2061@cindex @code{ltcalc}
2062@cindex calculator, location tracking
2063
9edcd895
AD
2064This example extends the infix notation calculator with location
2065tracking. This feature will be used to improve the error messages. For
2066the sake of clarity, this example is a simple integer calculator, since
2067most of the work needed to use locations will be done in the lexical
72d2299c 2068analyzer.
342b8b6e
AD
2069
2070@menu
f5f419de
DJ
2071* Ltcalc Declarations:: Bison and C declarations for ltcalc.
2072* Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
2073* Ltcalc Lexer:: The lexical analyzer.
342b8b6e
AD
2074@end menu
2075
f5f419de 2076@node Ltcalc Declarations
342b8b6e
AD
2077@subsection Declarations for @code{ltcalc}
2078
9edcd895
AD
2079The C and Bison declarations for the location tracking calculator are
2080the same as the declarations for the infix notation calculator.
342b8b6e
AD
2081
2082@example
2083/* Location tracking calculator. */
2084
2085%@{
38a92d50
PE
2086 #define YYSTYPE int
2087 #include <math.h>
2088 int yylex (void);
2089 void yyerror (char const *);
342b8b6e
AD
2090%@}
2091
2092/* Bison declarations. */
2093%token NUM
2094
2095%left '-' '+'
2096%left '*' '/'
d78f0ac9 2097%precedence NEG
342b8b6e
AD
2098%right '^'
2099
38a92d50 2100%% /* The grammar follows. */
342b8b6e
AD
2101@end example
2102
9edcd895
AD
2103@noindent
2104Note there are no declarations specific to locations. Defining a data
2105type for storing locations is not needed: we will use the type provided
2106by default (@pxref{Location Type, ,Data Types of Locations}), which is a
2107four member structure with the following integer fields:
2108@code{first_line}, @code{first_column}, @code{last_line} and
cd48d21d
AD
2109@code{last_column}. By conventions, and in accordance with the GNU
2110Coding Standards and common practice, the line and column count both
2111start at 1.
342b8b6e
AD
2112
2113@node Ltcalc Rules
2114@subsection Grammar Rules for @code{ltcalc}
2115
9edcd895
AD
2116Whether handling locations or not has no effect on the syntax of your
2117language. Therefore, grammar rules for this example will be very close
2118to those of the previous example: we will only modify them to benefit
2119from the new information.
342b8b6e 2120
9edcd895
AD
2121Here, we will use locations to report divisions by zero, and locate the
2122wrong expressions or subexpressions.
342b8b6e
AD
2123
2124@example
2125@group
2126input : /* empty */
2127 | input line
2128;
2129@end group
2130
2131@group
2132line : '\n'
2133 | exp '\n' @{ printf ("%d\n", $1); @}
2134;
2135@end group
2136
2137@group
2138exp : NUM @{ $$ = $1; @}
2139 | exp '+' exp @{ $$ = $1 + $3; @}
2140 | exp '-' exp @{ $$ = $1 - $3; @}
2141 | exp '*' exp @{ $$ = $1 * $3; @}
2142@end group
342b8b6e 2143@group
9edcd895 2144 | exp '/' exp
342b8b6e
AD
2145 @{
2146 if ($3)
2147 $$ = $1 / $3;
2148 else
2149 @{
2150 $$ = 1;
9edcd895
AD
2151 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2152 @@3.first_line, @@3.first_column,
2153 @@3.last_line, @@3.last_column);
342b8b6e
AD
2154 @}
2155 @}
2156@end group
2157@group
178e123e 2158 | '-' exp %prec NEG @{ $$ = -$2; @}
342b8b6e
AD
2159 | exp '^' exp @{ $$ = pow ($1, $3); @}
2160 | '(' exp ')' @{ $$ = $2; @}
2161@end group
2162@end example
2163
2164This code shows how to reach locations inside of semantic actions, by
2165using the pseudo-variables @code{@@@var{n}} for rule components, and the
2166pseudo-variable @code{@@$} for groupings.
2167
9edcd895
AD
2168We don't need to assign a value to @code{@@$}: the output parser does it
2169automatically. By default, before executing the C code of each action,
2170@code{@@$} is set to range from the beginning of @code{@@1} to the end
2171of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2172can be redefined (@pxref{Location Default Action, , Default Action for
2173Locations}), and for very specific rules, @code{@@$} can be computed by
2174hand.
342b8b6e
AD
2175
2176@node Ltcalc Lexer
2177@subsection The @code{ltcalc} Lexical Analyzer.
2178
9edcd895 2179Until now, we relied on Bison's defaults to enable location
72d2299c 2180tracking. The next step is to rewrite the lexical analyzer, and make it
9edcd895
AD
2181able to feed the parser with the token locations, as it already does for
2182semantic values.
342b8b6e 2183
9edcd895
AD
2184To this end, we must take into account every single character of the
2185input text, to avoid the computed locations of being fuzzy or wrong:
342b8b6e
AD
2186
2187@example
2188@group
2189int
2190yylex (void)
2191@{
2192 int c;
18b519c0 2193@end group
342b8b6e 2194
18b519c0 2195@group
72d2299c 2196 /* Skip white space. */
342b8b6e
AD
2197 while ((c = getchar ()) == ' ' || c == '\t')
2198 ++yylloc.last_column;
18b519c0 2199@end group
342b8b6e 2200
18b519c0 2201@group
72d2299c 2202 /* Step. */
342b8b6e
AD
2203 yylloc.first_line = yylloc.last_line;
2204 yylloc.first_column = yylloc.last_column;
2205@end group
2206
2207@group
72d2299c 2208 /* Process numbers. */
342b8b6e
AD
2209 if (isdigit (c))
2210 @{
2211 yylval = c - '0';
2212 ++yylloc.last_column;
2213 while (isdigit (c = getchar ()))
2214 @{
2215 ++yylloc.last_column;
2216 yylval = yylval * 10 + c - '0';
2217 @}
2218 ungetc (c, stdin);
2219 return NUM;
2220 @}
2221@end group
2222
72d2299c 2223 /* Return end-of-input. */
342b8b6e
AD
2224 if (c == EOF)
2225 return 0;
2226
72d2299c 2227 /* Return a single char, and update location. */
342b8b6e
AD
2228 if (c == '\n')
2229 @{
2230 ++yylloc.last_line;
2231 yylloc.last_column = 0;
2232 @}
2233 else
2234 ++yylloc.last_column;
2235 return c;
2236@}
2237@end example
2238
9edcd895
AD
2239Basically, the lexical analyzer performs the same processing as before:
2240it skips blanks and tabs, and reads numbers or single-character tokens.
2241In addition, it updates @code{yylloc}, the global variable (of type
2242@code{YYLTYPE}) containing the token's location.
342b8b6e 2243
9edcd895 2244Now, each time this function returns a token, the parser has its number
72d2299c 2245as well as its semantic value, and its location in the text. The last
9edcd895
AD
2246needed change is to initialize @code{yylloc}, for example in the
2247controlling function:
342b8b6e
AD
2248
2249@example
9edcd895 2250@group
342b8b6e
AD
2251int
2252main (void)
2253@{
2254 yylloc.first_line = yylloc.last_line = 1;
2255 yylloc.first_column = yylloc.last_column = 0;
2256 return yyparse ();
2257@}
9edcd895 2258@end group
342b8b6e
AD
2259@end example
2260
9edcd895
AD
2261Remember that computing locations is not a matter of syntax. Every
2262character must be associated to a location update, whether it is in
2263valid input, in comments, in literal strings, and so on.
342b8b6e
AD
2264
2265@node Multi-function Calc
bfa74976
RS
2266@section Multi-Function Calculator: @code{mfcalc}
2267@cindex multi-function calculator
2268@cindex @code{mfcalc}
2269@cindex calculator, multi-function
2270
2271Now that the basics of Bison have been discussed, it is time to move on to
2272a more advanced problem. The above calculators provided only five
2273functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2274be nice to have a calculator that provides other mathematical functions such
2275as @code{sin}, @code{cos}, etc.
2276
2277It is easy to add new operators to the infix calculator as long as they are
2278only single-character literals. The lexical analyzer @code{yylex} passes
9d9b8b70 2279back all nonnumeric characters as tokens, so new grammar rules suffice for
bfa74976
RS
2280adding a new operator. But we want something more flexible: built-in
2281functions whose syntax has this form:
2282
2283@example
2284@var{function_name} (@var{argument})
2285@end example
2286
2287@noindent
2288At the same time, we will add memory to the calculator, by allowing you
2289to create named variables, store values in them, and use them later.
2290Here is a sample session with the multi-function calculator:
2291
2292@example
9edcd895
AD
2293$ @kbd{mfcalc}
2294@kbd{pi = 3.141592653589}
bfa74976 22953.1415926536
9edcd895 2296@kbd{sin(pi)}
bfa74976 22970.0000000000
9edcd895 2298@kbd{alpha = beta1 = 2.3}
bfa74976 22992.3000000000
9edcd895 2300@kbd{alpha}
bfa74976 23012.3000000000
9edcd895 2302@kbd{ln(alpha)}
bfa74976 23030.8329091229
9edcd895 2304@kbd{exp(ln(beta1))}
bfa74976 23052.3000000000
9edcd895 2306$
bfa74976
RS
2307@end example
2308
2309Note that multiple assignment and nested function calls are permitted.
2310
2311@menu
f5f419de
DJ
2312* Mfcalc Declarations:: Bison declarations for multi-function calculator.
2313* Mfcalc Rules:: Grammar rules for the calculator.
2314* Mfcalc Symbol Table:: Symbol table management subroutines.
bfa74976
RS
2315@end menu
2316
f5f419de 2317@node Mfcalc Declarations
bfa74976
RS
2318@subsection Declarations for @code{mfcalc}
2319
2320Here are the C and Bison declarations for the multi-function calculator.
2321
2322@smallexample
18b519c0 2323@group
bfa74976 2324%@{
38a92d50
PE
2325 #include <math.h> /* For math functions, cos(), sin(), etc. */
2326 #include "calc.h" /* Contains definition of `symrec'. */
2327 int yylex (void);
2328 void yyerror (char const *);
bfa74976 2329%@}
18b519c0
AD
2330@end group
2331@group
bfa74976 2332%union @{
38a92d50
PE
2333 double val; /* For returning numbers. */
2334 symrec *tptr; /* For returning symbol-table pointers. */
bfa74976 2335@}
18b519c0 2336@end group
38a92d50
PE
2337%token <val> NUM /* Simple double precision number. */
2338%token <tptr> VAR FNCT /* Variable and Function. */
bfa74976
RS
2339%type <val> exp
2340
18b519c0 2341@group
bfa74976
RS
2342%right '='
2343%left '-' '+'
2344%left '*' '/'
d78f0ac9
AD
2345%precedence NEG /* negation--unary minus */
2346%right '^' /* exponentiation */
18b519c0 2347@end group
38a92d50 2348%% /* The grammar follows. */
bfa74976
RS
2349@end smallexample
2350
2351The above grammar introduces only two new features of the Bison language.
2352These features allow semantic values to have various data types
2353(@pxref{Multiple Types, ,More Than One Value Type}).
2354
2355The @code{%union} declaration specifies the entire list of possible types;
2356this is instead of defining @code{YYSTYPE}. The allowable types are now
2357double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2358the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2359
2360Since values can now have various types, it is necessary to associate a
2361type with each grammar symbol whose semantic value is used. These symbols
2362are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2363declarations are augmented with information about their data type (placed
2364between angle brackets).
2365
704a47c4
AD
2366The Bison construct @code{%type} is used for declaring nonterminal
2367symbols, just as @code{%token} is used for declaring token types. We
2368have not used @code{%type} before because nonterminal symbols are
2369normally declared implicitly by the rules that define them. But
2370@code{exp} must be declared explicitly so we can specify its value type.
2371@xref{Type Decl, ,Nonterminal Symbols}.
bfa74976 2372
342b8b6e 2373@node Mfcalc Rules
bfa74976
RS
2374@subsection Grammar Rules for @code{mfcalc}
2375
2376Here are the grammar rules for the multi-function calculator.
2377Most of them are copied directly from @code{calc}; three rules,
2378those which mention @code{VAR} or @code{FNCT}, are new.
2379
2380@smallexample
18b519c0 2381@group
bfa74976
RS
2382input: /* empty */
2383 | input line
2384;
18b519c0 2385@end group
bfa74976 2386
18b519c0 2387@group
bfa74976
RS
2388line:
2389 '\n'
2390 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2391 | error '\n' @{ yyerrok; @}
2392;
18b519c0 2393@end group
bfa74976 2394
18b519c0 2395@group
bfa74976
RS
2396exp: NUM @{ $$ = $1; @}
2397 | VAR @{ $$ = $1->value.var; @}
2398 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2399 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2400 | exp '+' exp @{ $$ = $1 + $3; @}
2401 | exp '-' exp @{ $$ = $1 - $3; @}
2402 | exp '*' exp @{ $$ = $1 * $3; @}
2403 | exp '/' exp @{ $$ = $1 / $3; @}
2404 | '-' exp %prec NEG @{ $$ = -$2; @}
2405 | exp '^' exp @{ $$ = pow ($1, $3); @}
2406 | '(' exp ')' @{ $$ = $2; @}
2407;
18b519c0 2408@end group
38a92d50 2409/* End of grammar. */
bfa74976
RS
2410%%
2411@end smallexample
2412
f5f419de 2413@node Mfcalc Symbol Table
bfa74976
RS
2414@subsection The @code{mfcalc} Symbol Table
2415@cindex symbol table example
2416
2417The multi-function calculator requires a symbol table to keep track of the
2418names and meanings of variables and functions. This doesn't affect the
2419grammar rules (except for the actions) or the Bison declarations, but it
2420requires some additional C functions for support.
2421
2422The symbol table itself consists of a linked list of records. Its
2423definition, which is kept in the header @file{calc.h}, is as follows. It
2424provides for either functions or variables to be placed in the table.
2425
2426@smallexample
2427@group
38a92d50 2428/* Function type. */
32dfccf8 2429typedef double (*func_t) (double);
72f889cc 2430@end group
32dfccf8 2431
72f889cc 2432@group
38a92d50 2433/* Data type for links in the chain of symbols. */
bfa74976
RS
2434struct symrec
2435@{
38a92d50 2436 char *name; /* name of symbol */
bfa74976 2437 int type; /* type of symbol: either VAR or FNCT */
32dfccf8
AD
2438 union
2439 @{
38a92d50
PE
2440 double var; /* value of a VAR */
2441 func_t fnctptr; /* value of a FNCT */
bfa74976 2442 @} value;
38a92d50 2443 struct symrec *next; /* link field */
bfa74976
RS
2444@};
2445@end group
2446
2447@group
2448typedef struct symrec symrec;
2449
38a92d50 2450/* The symbol table: a chain of `struct symrec'. */
bfa74976
RS
2451extern symrec *sym_table;
2452
a730d142 2453symrec *putsym (char const *, int);
38a92d50 2454symrec *getsym (char const *);
bfa74976
RS
2455@end group
2456@end smallexample
2457
2458The new version of @code{main} includes a call to @code{init_table}, a
2459function that initializes the symbol table. Here it is, and
2460@code{init_table} as well:
2461
2462@smallexample
bfa74976
RS
2463#include <stdio.h>
2464
18b519c0 2465@group
38a92d50 2466/* Called by yyparse on error. */
13863333 2467void
38a92d50 2468yyerror (char const *s)
bfa74976
RS
2469@{
2470 printf ("%s\n", s);
2471@}
18b519c0 2472@end group
bfa74976 2473
18b519c0 2474@group
bfa74976
RS
2475struct init
2476@{
38a92d50
PE
2477 char const *fname;
2478 double (*fnct) (double);
bfa74976
RS
2479@};
2480@end group
2481
2482@group
38a92d50 2483struct init const arith_fncts[] =
13863333 2484@{
32dfccf8
AD
2485 "sin", sin,
2486 "cos", cos,
13863333 2487 "atan", atan,
32dfccf8
AD
2488 "ln", log,
2489 "exp", exp,
13863333
AD
2490 "sqrt", sqrt,
2491 0, 0
2492@};
18b519c0 2493@end group
bfa74976 2494
18b519c0 2495@group
bfa74976 2496/* The symbol table: a chain of `struct symrec'. */
38a92d50 2497symrec *sym_table;
bfa74976
RS
2498@end group
2499
2500@group
72d2299c 2501/* Put arithmetic functions in table. */
13863333
AD
2502void
2503init_table (void)
bfa74976
RS
2504@{
2505 int i;
2506 symrec *ptr;
2507 for (i = 0; arith_fncts[i].fname != 0; i++)
2508 @{
2509 ptr = putsym (arith_fncts[i].fname, FNCT);
2510 ptr->value.fnctptr = arith_fncts[i].fnct;
2511 @}
2512@}
2513@end group
38a92d50
PE
2514
2515@group
2516int
2517main (void)
2518@{
2519 init_table ();
2520 return yyparse ();
2521@}
2522@end group
bfa74976
RS
2523@end smallexample
2524
2525By simply editing the initialization list and adding the necessary include
2526files, you can add additional functions to the calculator.
2527
2528Two important functions allow look-up and installation of symbols in the
2529symbol table. The function @code{putsym} is passed a name and the type
2530(@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2531linked to the front of the list, and a pointer to the object is returned.
2532The function @code{getsym} is passed the name of the symbol to look up. If
2533found, a pointer to that symbol is returned; otherwise zero is returned.
2534
2535@smallexample
2536symrec *
38a92d50 2537putsym (char const *sym_name, int sym_type)
bfa74976
RS
2538@{
2539 symrec *ptr;
2540 ptr = (symrec *) malloc (sizeof (symrec));
2541 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2542 strcpy (ptr->name,sym_name);
2543 ptr->type = sym_type;
72d2299c 2544 ptr->value.var = 0; /* Set value to 0 even if fctn. */
bfa74976
RS
2545 ptr->next = (struct symrec *)sym_table;
2546 sym_table = ptr;
2547 return ptr;
2548@}
2549
2550symrec *
38a92d50 2551getsym (char const *sym_name)
bfa74976
RS
2552@{
2553 symrec *ptr;
2554 for (ptr = sym_table; ptr != (symrec *) 0;
2555 ptr = (symrec *)ptr->next)
2556 if (strcmp (ptr->name,sym_name) == 0)
2557 return ptr;
2558 return 0;
2559@}
2560@end smallexample
2561
2562The function @code{yylex} must now recognize variables, numeric values, and
2563the single-character arithmetic operators. Strings of alphanumeric
9d9b8b70 2564characters with a leading letter are recognized as either variables or
bfa74976
RS
2565functions depending on what the symbol table says about them.
2566
2567The string is passed to @code{getsym} for look up in the symbol table. If
2568the name appears in the table, a pointer to its location and its type
2569(@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2570already in the table, then it is installed as a @code{VAR} using
2571@code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
e0c471a9 2572returned to @code{yyparse}.
bfa74976
RS
2573
2574No change is needed in the handling of numeric values and arithmetic
2575operators in @code{yylex}.
2576
2577@smallexample
2578@group
2579#include <ctype.h>
18b519c0 2580@end group
13863333 2581
18b519c0 2582@group
13863333
AD
2583int
2584yylex (void)
bfa74976
RS
2585@{
2586 int c;
2587
72d2299c 2588 /* Ignore white space, get first nonwhite character. */
bfa74976
RS
2589 while ((c = getchar ()) == ' ' || c == '\t');
2590
2591 if (c == EOF)
2592 return 0;
2593@end group
2594
2595@group
2596 /* Char starts a number => parse the number. */
2597 if (c == '.' || isdigit (c))
2598 @{
2599 ungetc (c, stdin);
2600 scanf ("%lf", &yylval.val);
2601 return NUM;
2602 @}
2603@end group
2604
2605@group
2606 /* Char starts an identifier => read the name. */
2607 if (isalpha (c))
2608 @{
2609 symrec *s;
2610 static char *symbuf = 0;
2611 static int length = 0;
2612 int i;
2613@end group
2614
2615@group
2616 /* Initially make the buffer long enough
2617 for a 40-character symbol name. */
2618 if (length == 0)
2619 length = 40, symbuf = (char *)malloc (length + 1);
2620
2621 i = 0;
2622 do
2623@end group
2624@group
2625 @{
2626 /* If buffer is full, make it bigger. */
2627 if (i == length)
2628 @{
2629 length *= 2;
18b519c0 2630 symbuf = (char *) realloc (symbuf, length + 1);
bfa74976
RS
2631 @}
2632 /* Add this character to the buffer. */
2633 symbuf[i++] = c;
2634 /* Get another character. */
2635 c = getchar ();
2636 @}
2637@end group
2638@group
72d2299c 2639 while (isalnum (c));
bfa74976
RS
2640
2641 ungetc (c, stdin);
2642 symbuf[i] = '\0';
2643@end group
2644
2645@group
2646 s = getsym (symbuf);
2647 if (s == 0)
2648 s = putsym (symbuf, VAR);
2649 yylval.tptr = s;
2650 return s->type;
2651 @}
2652
2653 /* Any other character is a token by itself. */
2654 return c;
2655@}
2656@end group
2657@end smallexample
2658
72d2299c 2659This program is both powerful and flexible. You may easily add new
704a47c4
AD
2660functions, and it is a simple job to modify this code to install
2661predefined variables such as @code{pi} or @code{e} as well.
bfa74976 2662
342b8b6e 2663@node Exercises
bfa74976
RS
2664@section Exercises
2665@cindex exercises
2666
2667@enumerate
2668@item
2669Add some new functions from @file{math.h} to the initialization list.
2670
2671@item
2672Add another array that contains constants and their values. Then
2673modify @code{init_table} to add these constants to the symbol table.
2674It will be easiest to give the constants type @code{VAR}.
2675
2676@item
2677Make the program report an error if the user refers to an
2678uninitialized variable in any way except to store a value in it.
2679@end enumerate
2680
342b8b6e 2681@node Grammar File
bfa74976
RS
2682@chapter Bison Grammar Files
2683
2684Bison takes as input a context-free grammar specification and produces a
2685C-language function that recognizes correct instances of the grammar.
2686
ff7571c0 2687The Bison grammar file conventionally has a name ending in @samp{.y}.
234a3be3 2688@xref{Invocation, ,Invoking Bison}.
bfa74976
RS
2689
2690@menu
2691* Grammar Outline:: Overall layout of the grammar file.
2692* Symbols:: Terminal and nonterminal symbols.
2693* Rules:: How to write grammar rules.
2694* Recursion:: Writing recursive rules.
2695* Semantics:: Semantic values and actions.
847bf1f5 2696* Locations:: Locations and actions.
bfa74976
RS
2697* Declarations:: All kinds of Bison declarations are described here.
2698* Multiple Parsers:: Putting more than one Bison parser in one program.
2699@end menu
2700
342b8b6e 2701@node Grammar Outline
bfa74976
RS
2702@section Outline of a Bison Grammar
2703
2704A Bison grammar file has four main sections, shown here with the
2705appropriate delimiters:
2706
2707@example
2708%@{
38a92d50 2709 @var{Prologue}
bfa74976
RS
2710%@}
2711
2712@var{Bison declarations}
2713
2714%%
2715@var{Grammar rules}
2716%%
2717
75f5aaea 2718@var{Epilogue}
bfa74976
RS
2719@end example
2720
2721Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
8a4281b9 2722As a GNU extension, @samp{//} introduces a comment that
2bfc2e2a 2723continues until end of line.
bfa74976
RS
2724
2725@menu
f5f419de 2726* Prologue:: Syntax and usage of the prologue.
2cbe6b7f 2727* Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
f5f419de
DJ
2728* Bison Declarations:: Syntax and usage of the Bison declarations section.
2729* Grammar Rules:: Syntax and usage of the grammar rules section.
2730* Epilogue:: Syntax and usage of the epilogue.
bfa74976
RS
2731@end menu
2732
38a92d50 2733@node Prologue
75f5aaea
MA
2734@subsection The prologue
2735@cindex declarations section
2736@cindex Prologue
2737@cindex declarations
bfa74976 2738
f8e1c9e5
AD
2739The @var{Prologue} section contains macro definitions and declarations
2740of functions and variables that are used in the actions in the grammar
ff7571c0
JD
2741rules. These are copied to the beginning of the parser implementation
2742file so that they precede the definition of @code{yyparse}. You can
2743use @samp{#include} to get the declarations from a header file. If
2744you don't need any C declarations, you may omit the @samp{%@{} and
f8e1c9e5 2745@samp{%@}} delimiters that bracket this section.
bfa74976 2746
9c437126 2747The @var{Prologue} section is terminated by the first occurrence
287c78f6
PE
2748of @samp{%@}} that is outside a comment, a string literal, or a
2749character constant.
2750
c732d2c6
AD
2751You may have more than one @var{Prologue} section, intermixed with the
2752@var{Bison declarations}. This allows you to have C and Bison
2753declarations that refer to each other. For example, the @code{%union}
2754declaration may use types defined in a header file, and you may wish to
2755prototype functions that take arguments of type @code{YYSTYPE}. This
2756can be done with two @var{Prologue} blocks, one before and one after the
2757@code{%union} declaration.
2758
2759@smallexample
2760%@{
aef3da86 2761 #define _GNU_SOURCE
38a92d50
PE
2762 #include <stdio.h>
2763 #include "ptypes.h"
c732d2c6
AD
2764%@}
2765
2766%union @{
779e7ceb 2767 long int n;
c732d2c6
AD
2768 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2769@}
2770
2771%@{
38a92d50
PE
2772 static void print_token_value (FILE *, int, YYSTYPE);
2773 #define YYPRINT(F, N, L) print_token_value (F, N, L)
c732d2c6
AD
2774%@}
2775
2776@dots{}
2777@end smallexample
2778
aef3da86
PE
2779When in doubt, it is usually safer to put prologue code before all
2780Bison declarations, rather than after. For example, any definitions
2781of feature test macros like @code{_GNU_SOURCE} or
2782@code{_POSIX_C_SOURCE} should appear before all Bison declarations, as
2783feature test macros can affect the behavior of Bison-generated
2784@code{#include} directives.
2785
2cbe6b7f
JD
2786@node Prologue Alternatives
2787@subsection Prologue Alternatives
2788@cindex Prologue Alternatives
2789
136a0f76 2790@findex %code
16dc6a9e
JD
2791@findex %code requires
2792@findex %code provides
2793@findex %code top
85894313 2794
2cbe6b7f 2795The functionality of @var{Prologue} sections can often be subtle and
ff7571c0
JD
2796inflexible. As an alternative, Bison provides a @code{%code}
2797directive with an explicit qualifier field, which identifies the
2798purpose of the code and thus the location(s) where Bison should
2799generate it. For C/C++, the qualifier can be omitted for the default
2800location, or it can be one of @code{requires}, @code{provides},
e0c07222 2801@code{top}. @xref{%code Summary}.
2cbe6b7f
JD
2802
2803Look again at the example of the previous section:
2804
2805@smallexample
2806%@{
2807 #define _GNU_SOURCE
2808 #include <stdio.h>
2809 #include "ptypes.h"
2810%@}
2811
2812%union @{
2813 long int n;
2814 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2815@}
2816
2817%@{
2818 static void print_token_value (FILE *, int, YYSTYPE);
2819 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2820%@}
2821
2822@dots{}
2823@end smallexample
2824
2825@noindent
ff7571c0
JD
2826Notice that there are two @var{Prologue} sections here, but there's a
2827subtle distinction between their functionality. For example, if you
2828decide to override Bison's default definition for @code{YYLTYPE}, in
2829which @var{Prologue} section should you write your new definition?
2830You should write it in the first since Bison will insert that code
2831into the parser implementation file @emph{before} the default
2832@code{YYLTYPE} definition. In which @var{Prologue} section should you
2833prototype an internal function, @code{trace_token}, that accepts
2834@code{YYLTYPE} and @code{yytokentype} as arguments? You should
2835prototype it in the second since Bison will insert that code
2cbe6b7f
JD
2836@emph{after} the @code{YYLTYPE} and @code{yytokentype} definitions.
2837
2838This distinction in functionality between the two @var{Prologue} sections is
2839established by the appearance of the @code{%union} between them.
a501eca9 2840This behavior raises a few questions.
2cbe6b7f
JD
2841First, why should the position of a @code{%union} affect definitions related to
2842@code{YYLTYPE} and @code{yytokentype}?
2843Second, what if there is no @code{%union}?
2844In that case, the second kind of @var{Prologue} section is not available.
2845This behavior is not intuitive.
2846
8e0a5e9e 2847To avoid this subtle @code{%union} dependency, rewrite the example using a
16dc6a9e 2848@code{%code top} and an unqualified @code{%code}.
2cbe6b7f
JD
2849Let's go ahead and add the new @code{YYLTYPE} definition and the
2850@code{trace_token} prototype at the same time:
2851
2852@smallexample
16dc6a9e 2853%code top @{
2cbe6b7f
JD
2854 #define _GNU_SOURCE
2855 #include <stdio.h>
8e0a5e9e
JD
2856
2857 /* WARNING: The following code really belongs
16dc6a9e 2858 * in a `%code requires'; see below. */
8e0a5e9e 2859
2cbe6b7f
JD
2860 #include "ptypes.h"
2861 #define YYLTYPE YYLTYPE
2862 typedef struct YYLTYPE
2863 @{
2864 int first_line;
2865 int first_column;
2866 int last_line;
2867 int last_column;
2868 char *filename;
2869 @} YYLTYPE;
2870@}
2871
2872%union @{
2873 long int n;
2874 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2875@}
2876
2877%code @{
2878 static void print_token_value (FILE *, int, YYSTYPE);
2879 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2880 static void trace_token (enum yytokentype token, YYLTYPE loc);
2881@}
2882
2883@dots{}
2884@end smallexample
2885
2886@noindent
16dc6a9e
JD
2887In this way, @code{%code top} and the unqualified @code{%code} achieve the same
2888functionality as the two kinds of @var{Prologue} sections, but it's always
8e0a5e9e 2889explicit which kind you intend.
2cbe6b7f
JD
2890Moreover, both kinds are always available even in the absence of @code{%union}.
2891
ff7571c0
JD
2892The @code{%code top} block above logically contains two parts. The
2893first two lines before the warning need to appear near the top of the
2894parser implementation file. The first line after the warning is
2895required by @code{YYSTYPE} and thus also needs to appear in the parser
2896implementation file. However, if you've instructed Bison to generate
2897a parser header file (@pxref{Decl Summary, ,%defines}), you probably
2898want that line to appear before the @code{YYSTYPE} definition in that
2899header file as well. The @code{YYLTYPE} definition should also appear
2900in the parser header file to override the default @code{YYLTYPE}
2901definition there.
2cbe6b7f 2902
16dc6a9e 2903In other words, in the @code{%code top} block above, all but the first two
8e0a5e9e
JD
2904lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
2905definitions.
16dc6a9e 2906Thus, they belong in one or more @code{%code requires}:
9bc0dd67
JD
2907
2908@smallexample
16dc6a9e 2909%code top @{
2cbe6b7f
JD
2910 #define _GNU_SOURCE
2911 #include <stdio.h>
2912@}
2913
16dc6a9e 2914%code requires @{
9bc0dd67
JD
2915 #include "ptypes.h"
2916@}
2917%union @{
2918 long int n;
2919 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2920@}
2921
16dc6a9e 2922%code requires @{
2cbe6b7f
JD
2923 #define YYLTYPE YYLTYPE
2924 typedef struct YYLTYPE
2925 @{
2926 int first_line;
2927 int first_column;
2928 int last_line;
2929 int last_column;
2930 char *filename;
2931 @} YYLTYPE;
2932@}
2933
136a0f76 2934%code @{
2cbe6b7f
JD
2935 static void print_token_value (FILE *, int, YYSTYPE);
2936 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2937 static void trace_token (enum yytokentype token, YYLTYPE loc);
2938@}
2939
2940@dots{}
2941@end smallexample
2942
2943@noindent
ff7571c0
JD
2944Now Bison will insert @code{#include "ptypes.h"} and the new
2945@code{YYLTYPE} definition before the Bison-generated @code{YYSTYPE}
2946and @code{YYLTYPE} definitions in both the parser implementation file
2947and the parser header file. (By the same reasoning, @code{%code
2948requires} would also be the appropriate place to write your own
2949definition for @code{YYSTYPE}.)
2950
2951When you are writing dependency code for @code{YYSTYPE} and
2952@code{YYLTYPE}, you should prefer @code{%code requires} over
2953@code{%code top} regardless of whether you instruct Bison to generate
2954a parser header file. When you are writing code that you need Bison
2955to insert only into the parser implementation file and that has no
2956special need to appear at the top of that file, you should prefer the
2957unqualified @code{%code} over @code{%code top}. These practices will
2958make the purpose of each block of your code explicit to Bison and to
2959other developers reading your grammar file. Following these
2960practices, we expect the unqualified @code{%code} and @code{%code
2961requires} to be the most important of the four @var{Prologue}
16dc6a9e 2962alternatives.
a501eca9 2963
ff7571c0
JD
2964At some point while developing your parser, you might decide to
2965provide @code{trace_token} to modules that are external to your
2966parser. Thus, you might wish for Bison to insert the prototype into
2967both the parser header file and the parser implementation file. Since
2968this function is not a dependency required by @code{YYSTYPE} or
8e0a5e9e 2969@code{YYLTYPE}, it doesn't make sense to move its prototype to a
ff7571c0
JD
2970@code{%code requires}. More importantly, since it depends upon
2971@code{YYLTYPE} and @code{yytokentype}, @code{%code requires} is not
2972sufficient. Instead, move its prototype from the unqualified
2973@code{%code} to a @code{%code provides}:
2cbe6b7f
JD
2974
2975@smallexample
16dc6a9e 2976%code top @{
2cbe6b7f 2977 #define _GNU_SOURCE
136a0f76 2978 #include <stdio.h>
2cbe6b7f 2979@}
136a0f76 2980
16dc6a9e 2981%code requires @{
2cbe6b7f
JD
2982 #include "ptypes.h"
2983@}
2984%union @{
2985 long int n;
2986 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2987@}
2988
16dc6a9e 2989%code requires @{
2cbe6b7f
JD
2990 #define YYLTYPE YYLTYPE
2991 typedef struct YYLTYPE
2992 @{
2993 int first_line;
2994 int first_column;
2995 int last_line;
2996 int last_column;
2997 char *filename;
2998 @} YYLTYPE;
2999@}
3000
16dc6a9e 3001%code provides @{
2cbe6b7f
JD
3002 void trace_token (enum yytokentype token, YYLTYPE loc);
3003@}
3004
3005%code @{
9bc0dd67
JD
3006 static void print_token_value (FILE *, int, YYSTYPE);
3007 #define YYPRINT(F, N, L) print_token_value (F, N, L)
34f98f46 3008@}
9bc0dd67
JD
3009
3010@dots{}
3011@end smallexample
3012
2cbe6b7f 3013@noindent
ff7571c0
JD
3014Bison will insert the @code{trace_token} prototype into both the
3015parser header file and the parser implementation file after the
3016definitions for @code{yytokentype}, @code{YYLTYPE}, and
3017@code{YYSTYPE}.
2cbe6b7f 3018
ff7571c0
JD
3019The above examples are careful to write directives in an order that
3020reflects the layout of the generated parser implementation and header
3021files: @code{%code top}, @code{%code requires}, @code{%code provides},
3022and then @code{%code}. While your grammar files may generally be
3023easier to read if you also follow this order, Bison does not require
3024it. Instead, Bison lets you choose an organization that makes sense
3025to you.
2cbe6b7f 3026
a501eca9 3027You may declare any of these directives multiple times in the grammar file.
2cbe6b7f
JD
3028In that case, Bison concatenates the contained code in declaration order.
3029This is the only way in which the position of one of these directives within
3030the grammar file affects its functionality.
3031
3032The result of the previous two properties is greater flexibility in how you may
3033organize your grammar file.
3034For example, you may organize semantic-type-related directives by semantic
3035type:
3036
3037@smallexample
16dc6a9e 3038%code requires @{ #include "type1.h" @}
2cbe6b7f
JD
3039%union @{ type1 field1; @}
3040%destructor @{ type1_free ($$); @} <field1>
3041%printer @{ type1_print ($$); @} <field1>
3042
16dc6a9e 3043%code requires @{ #include "type2.h" @}
2cbe6b7f
JD
3044%union @{ type2 field2; @}
3045%destructor @{ type2_free ($$); @} <field2>
3046%printer @{ type2_print ($$); @} <field2>
3047@end smallexample
3048
3049@noindent
3050You could even place each of the above directive groups in the rules section of
3051the grammar file next to the set of rules that uses the associated semantic
3052type.
61fee93e
JD
3053(In the rules section, you must terminate each of those directives with a
3054semicolon.)
2cbe6b7f
JD
3055And you don't have to worry that some directive (like a @code{%union}) in the
3056definitions section is going to adversely affect their functionality in some
3057counter-intuitive manner just because it comes first.
3058Such an organization is not possible using @var{Prologue} sections.
3059
a501eca9 3060This section has been concerned with explaining the advantages of the four
8e0a5e9e 3061@var{Prologue} alternatives over the original Yacc @var{Prologue}.
a501eca9
JD
3062However, in most cases when using these directives, you shouldn't need to
3063think about all the low-level ordering issues discussed here.
3064Instead, you should simply use these directives to label each block of your
3065code according to its purpose and let Bison handle the ordering.
3066@code{%code} is the most generic label.
16dc6a9e
JD
3067Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
3068as needed.
a501eca9 3069
342b8b6e 3070@node Bison Declarations
bfa74976
RS
3071@subsection The Bison Declarations Section
3072@cindex Bison declarations (introduction)
3073@cindex declarations, Bison (introduction)
3074
3075The @var{Bison declarations} section contains declarations that define
3076terminal and nonterminal symbols, specify precedence, and so on.
3077In some simple grammars you may not need any declarations.
3078@xref{Declarations, ,Bison Declarations}.
3079
342b8b6e 3080@node Grammar Rules
bfa74976
RS
3081@subsection The Grammar Rules Section
3082@cindex grammar rules section
3083@cindex rules section for grammar
3084
3085The @dfn{grammar rules} section contains one or more Bison grammar
3086rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
3087
3088There must always be at least one grammar rule, and the first
3089@samp{%%} (which precedes the grammar rules) may never be omitted even
3090if it is the first thing in the file.
3091
38a92d50 3092@node Epilogue
75f5aaea 3093@subsection The epilogue
bfa74976 3094@cindex additional C code section
75f5aaea 3095@cindex epilogue
bfa74976
RS
3096@cindex C code, section for additional
3097
ff7571c0
JD
3098The @var{Epilogue} is copied verbatim to the end of the parser
3099implementation file, just as the @var{Prologue} is copied to the
3100beginning. This is the most convenient place to put anything that you
3101want to have in the parser implementation file but which need not come
3102before the definition of @code{yyparse}. For example, the definitions
3103of @code{yylex} and @code{yyerror} often go here. Because C requires
3104functions to be declared before being used, you often need to declare
3105functions like @code{yylex} and @code{yyerror} in the Prologue, even
3106if you define them in the Epilogue. @xref{Interface, ,Parser
3107C-Language Interface}.
bfa74976
RS
3108
3109If the last section is empty, you may omit the @samp{%%} that separates it
3110from the grammar rules.
3111
f8e1c9e5
AD
3112The Bison parser itself contains many macros and identifiers whose names
3113start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3114any such names (except those documented in this manual) in the epilogue
3115of the grammar file.
bfa74976 3116
342b8b6e 3117@node Symbols
bfa74976
RS
3118@section Symbols, Terminal and Nonterminal
3119@cindex nonterminal symbol
3120@cindex terminal symbol
3121@cindex token type
3122@cindex symbol
3123
3124@dfn{Symbols} in Bison grammars represent the grammatical classifications
3125of the language.
3126
3127A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
3128class of syntactically equivalent tokens. You use the symbol in grammar
3129rules to mean that a token in that class is allowed. The symbol is
3130represented in the Bison parser by a numeric code, and the @code{yylex}
f8e1c9e5
AD
3131function returns a token type code to indicate what kind of token has
3132been read. You don't need to know what the code value is; you can use
3133the symbol to stand for it.
bfa74976 3134
f8e1c9e5
AD
3135A @dfn{nonterminal symbol} stands for a class of syntactically
3136equivalent groupings. The symbol name is used in writing grammar rules.
3137By convention, it should be all lower case.
bfa74976 3138
82f3355e
JD
3139Symbol names can contain letters, underscores, periods, and non-initial
3140digits and dashes. Dashes in symbol names are a GNU extension, incompatible
3141with POSIX Yacc. Periods and dashes make symbol names less convenient to
3142use with named references, which require brackets around such names
3143(@pxref{Named References}). Terminal symbols that contain periods or dashes
3144make little sense: since they are not valid symbols (in most programming
3145languages) they are not exported as token names.
bfa74976 3146
931c7513 3147There are three ways of writing terminal symbols in the grammar:
bfa74976
RS
3148
3149@itemize @bullet
3150@item
3151A @dfn{named token type} is written with an identifier, like an
c827f760 3152identifier in C@. By convention, it should be all upper case. Each
bfa74976
RS
3153such name must be defined with a Bison declaration such as
3154@code{%token}. @xref{Token Decl, ,Token Type Names}.
3155
3156@item
3157@cindex character token
3158@cindex literal token
3159@cindex single-character literal
931c7513
RS
3160A @dfn{character token type} (or @dfn{literal character token}) is
3161written in the grammar using the same syntax used in C for character
3162constants; for example, @code{'+'} is a character token type. A
3163character token type doesn't need to be declared unless you need to
3164specify its semantic value data type (@pxref{Value Type, ,Data Types of
3165Semantic Values}), associativity, or precedence (@pxref{Precedence,
3166,Operator Precedence}).
bfa74976
RS
3167
3168By convention, a character token type is used only to represent a
3169token that consists of that particular character. Thus, the token
3170type @code{'+'} is used to represent the character @samp{+} as a
3171token. Nothing enforces this convention, but if you depart from it,
3172your program will confuse other readers.
3173
3174All the usual escape sequences used in character literals in C can be
3175used in Bison as well, but you must not use the null character as a
72d2299c
PE
3176character literal because its numeric code, zero, signifies
3177end-of-input (@pxref{Calling Convention, ,Calling Convention
2bfc2e2a
PE
3178for @code{yylex}}). Also, unlike standard C, trigraphs have no
3179special meaning in Bison character literals, nor is backslash-newline
3180allowed.
931c7513
RS
3181
3182@item
3183@cindex string token
3184@cindex literal string token
9ecbd125 3185@cindex multicharacter literal
931c7513
RS
3186A @dfn{literal string token} is written like a C string constant; for
3187example, @code{"<="} is a literal string token. A literal string token
3188doesn't need to be declared unless you need to specify its semantic
14ded682 3189value data type (@pxref{Value Type}), associativity, or precedence
931c7513
RS
3190(@pxref{Precedence}).
3191
3192You can associate the literal string token with a symbolic name as an
3193alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
3194Declarations}). If you don't do that, the lexical analyzer has to
3195retrieve the token number for the literal string token from the
3196@code{yytname} table (@pxref{Calling Convention}).
3197
c827f760 3198@strong{Warning}: literal string tokens do not work in Yacc.
931c7513
RS
3199
3200By convention, a literal string token is used only to represent a token
3201that consists of that particular string. Thus, you should use the token
3202type @code{"<="} to represent the string @samp{<=} as a token. Bison
9ecbd125 3203does not enforce this convention, but if you depart from it, people who
931c7513
RS
3204read your program will be confused.
3205
3206All the escape sequences used in string literals in C can be used in
92ac3705
PE
3207Bison as well, except that you must not use a null character within a
3208string literal. Also, unlike Standard C, trigraphs have no special
2bfc2e2a
PE
3209meaning in Bison string literals, nor is backslash-newline allowed. A
3210literal string token must contain two or more characters; for a token
3211containing just one character, use a character token (see above).
bfa74976
RS
3212@end itemize
3213
3214How you choose to write a terminal symbol has no effect on its
3215grammatical meaning. That depends only on where it appears in rules and
3216on when the parser function returns that symbol.
3217
72d2299c
PE
3218The value returned by @code{yylex} is always one of the terminal
3219symbols, except that a zero or negative value signifies end-of-input.
3220Whichever way you write the token type in the grammar rules, you write
3221it the same way in the definition of @code{yylex}. The numeric code
3222for a character token type is simply the positive numeric code of the
3223character, so @code{yylex} can use the identical value to generate the
3224requisite code, though you may need to convert it to @code{unsigned
3225char} to avoid sign-extension on hosts where @code{char} is signed.
ff7571c0
JD
3226Each named token type becomes a C macro in the parser implementation
3227file, so @code{yylex} can use the name to stand for the code. (This
3228is why periods don't make sense in terminal symbols.) @xref{Calling
3229Convention, ,Calling Convention for @code{yylex}}.
bfa74976
RS
3230
3231If @code{yylex} is defined in a separate file, you need to arrange for the
3232token-type macro definitions to be available there. Use the @samp{-d}
3233option when you run Bison, so that it will write these macro definitions
3234into a separate header file @file{@var{name}.tab.h} which you can include
3235in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
3236
72d2299c 3237If you want to write a grammar that is portable to any Standard C
9d9b8b70 3238host, you must use only nonnull character tokens taken from the basic
c827f760 3239execution character set of Standard C@. This set consists of the ten
72d2299c
PE
3240digits, the 52 lower- and upper-case English letters, and the
3241characters in the following C-language string:
3242
3243@example
3244"\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3245@end example
3246
f8e1c9e5
AD
3247The @code{yylex} function and Bison must use a consistent character set
3248and encoding for character tokens. For example, if you run Bison in an
8a4281b9 3249ASCII environment, but then compile and run the resulting
f8e1c9e5 3250program in an environment that uses an incompatible character set like
8a4281b9
JD
3251EBCDIC, the resulting program may not work because the tables
3252generated by Bison will assume ASCII numeric values for
f8e1c9e5
AD
3253character tokens. It is standard practice for software distributions to
3254contain C source files that were generated by Bison in an
8a4281b9
JD
3255ASCII environment, so installers on platforms that are
3256incompatible with ASCII must rebuild those files before
f8e1c9e5 3257compiling them.
e966383b 3258
bfa74976
RS
3259The symbol @code{error} is a terminal symbol reserved for error recovery
3260(@pxref{Error Recovery}); you shouldn't use it for any other purpose.
23c5a174
AD
3261In particular, @code{yylex} should never return this value. The default
3262value of the error token is 256, unless you explicitly assigned 256 to
3263one of your tokens with a @code{%token} declaration.
bfa74976 3264
342b8b6e 3265@node Rules
bfa74976
RS
3266@section Syntax of Grammar Rules
3267@cindex rule syntax
3268@cindex grammar rule syntax
3269@cindex syntax of grammar rules
3270
3271A Bison grammar rule has the following general form:
3272
3273@example
e425e872 3274@group
bfa74976
RS
3275@var{result}: @var{components}@dots{}
3276 ;
e425e872 3277@end group
bfa74976
RS
3278@end example
3279
3280@noindent
9ecbd125 3281where @var{result} is the nonterminal symbol that this rule describes,
bfa74976 3282and @var{components} are various terminal and nonterminal symbols that
13863333 3283are put together by this rule (@pxref{Symbols}).
bfa74976
RS
3284
3285For example,
3286
3287@example
3288@group
3289exp: exp '+' exp
3290 ;
3291@end group
3292@end example
3293
3294@noindent
3295says that two groupings of type @code{exp}, with a @samp{+} token in between,
3296can be combined into a larger grouping of type @code{exp}.
3297
72d2299c
PE
3298White space in rules is significant only to separate symbols. You can add
3299extra white space as you wish.
bfa74976
RS
3300
3301Scattered among the components can be @var{actions} that determine
3302the semantics of the rule. An action looks like this:
3303
3304@example
3305@{@var{C statements}@}
3306@end example
3307
3308@noindent
287c78f6
PE
3309@cindex braced code
3310This is an example of @dfn{braced code}, that is, C code surrounded by
3311braces, much like a compound statement in C@. Braced code can contain
3312any sequence of C tokens, so long as its braces are balanced. Bison
3313does not check the braced code for correctness directly; it merely
ff7571c0
JD
3314copies the code to the parser implementation file, where the C
3315compiler can check it.
287c78f6
PE
3316
3317Within braced code, the balanced-brace count is not affected by braces
3318within comments, string literals, or character constants, but it is
3319affected by the C digraphs @samp{<%} and @samp{%>} that represent
3320braces. At the top level braced code must be terminated by @samp{@}}
3321and not by a digraph. Bison does not look for trigraphs, so if braced
3322code uses trigraphs you should ensure that they do not affect the
3323nesting of braces or the boundaries of comments, string literals, or
3324character constants.
3325
bfa74976
RS
3326Usually there is only one action and it follows the components.
3327@xref{Actions}.
3328
3329@findex |
3330Multiple rules for the same @var{result} can be written separately or can
3331be joined with the vertical-bar character @samp{|} as follows:
3332
bfa74976
RS
3333@example
3334@group
3335@var{result}: @var{rule1-components}@dots{}
3336 | @var{rule2-components}@dots{}
3337 @dots{}
3338 ;
3339@end group
3340@end example
bfa74976
RS
3341
3342@noindent
3343They are still considered distinct rules even when joined in this way.
3344
3345If @var{components} in a rule is empty, it means that @var{result} can
3346match the empty string. For example, here is how to define a
3347comma-separated sequence of zero or more @code{exp} groupings:
3348
3349@example
3350@group
3351expseq: /* empty */
3352 | expseq1
3353 ;
3354@end group
3355
3356@group
3357expseq1: exp
3358 | expseq1 ',' exp
3359 ;
3360@end group
3361@end example
3362
3363@noindent
3364It is customary to write a comment @samp{/* empty */} in each rule
3365with no components.
3366
342b8b6e 3367@node Recursion
bfa74976
RS
3368@section Recursive Rules
3369@cindex recursive rule
3370
f8e1c9e5
AD
3371A rule is called @dfn{recursive} when its @var{result} nonterminal
3372appears also on its right hand side. Nearly all Bison grammars need to
3373use recursion, because that is the only way to define a sequence of any
3374number of a particular thing. Consider this recursive definition of a
9ecbd125 3375comma-separated sequence of one or more expressions:
bfa74976
RS
3376
3377@example
3378@group
3379expseq1: exp
3380 | expseq1 ',' exp
3381 ;
3382@end group
3383@end example
3384
3385@cindex left recursion
3386@cindex right recursion
3387@noindent
3388Since the recursive use of @code{expseq1} is the leftmost symbol in the
3389right hand side, we call this @dfn{left recursion}. By contrast, here
3390the same construct is defined using @dfn{right recursion}:
3391
3392@example
3393@group
3394expseq1: exp
3395 | exp ',' expseq1
3396 ;
3397@end group
3398@end example
3399
3400@noindent
ec3bc396
AD
3401Any kind of sequence can be defined using either left recursion or right
3402recursion, but you should always use left recursion, because it can
3403parse a sequence of any number of elements with bounded stack space.
3404Right recursion uses up space on the Bison stack in proportion to the
3405number of elements in the sequence, because all the elements must be
3406shifted onto the stack before the rule can be applied even once.
3407@xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3408of this.
bfa74976
RS
3409
3410@cindex mutual recursion
3411@dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3412rule does not appear directly on its right hand side, but does appear
3413in rules for other nonterminals which do appear on its right hand
13863333 3414side.
bfa74976
RS
3415
3416For example:
3417
3418@example
3419@group
3420expr: primary
3421 | primary '+' primary
3422 ;
3423@end group
3424
3425@group
3426primary: constant
3427 | '(' expr ')'
3428 ;
3429@end group
3430@end example
3431
3432@noindent
3433defines two mutually-recursive nonterminals, since each refers to the
3434other.
3435
342b8b6e 3436@node Semantics
bfa74976
RS
3437@section Defining Language Semantics
3438@cindex defining language semantics
13863333 3439@cindex language semantics, defining
bfa74976
RS
3440
3441The grammar rules for a language determine only the syntax. The semantics
3442are determined by the semantic values associated with various tokens and
3443groupings, and by the actions taken when various groupings are recognized.
3444
3445For example, the calculator calculates properly because the value
3446associated with each expression is the proper number; it adds properly
3447because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3448the numbers associated with @var{x} and @var{y}.
3449
3450@menu
3451* Value Type:: Specifying one data type for all semantic values.
3452* Multiple Types:: Specifying several alternative data types.
3453* Actions:: An action is the semantic definition of a grammar rule.
3454* Action Types:: Specifying data types for actions to operate on.
3455* Mid-Rule Actions:: Most actions go at the end of a rule.
3456 This says when, why and how to use the exceptional
3457 action in the middle of a rule.
d013372c 3458* Named References:: Using named references in actions.
bfa74976
RS
3459@end menu
3460
342b8b6e 3461@node Value Type
bfa74976
RS
3462@subsection Data Types of Semantic Values
3463@cindex semantic value type
3464@cindex value type, semantic
3465@cindex data types of semantic values
3466@cindex default data type
3467
3468In a simple program it may be sufficient to use the same data type for
3469the semantic values of all language constructs. This was true in the
8a4281b9 3470RPN and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
1964ad8c 3471Notation Calculator}).
bfa74976 3472
ddc8ede1
PE
3473Bison normally uses the type @code{int} for semantic values if your
3474program uses the same data type for all language constructs. To
bfa74976
RS
3475specify some other type, define @code{YYSTYPE} as a macro, like this:
3476
3477@example
3478#define YYSTYPE double
3479@end example
3480
3481@noindent
50cce58e
PE
3482@code{YYSTYPE}'s replacement list should be a type name
3483that does not contain parentheses or square brackets.
342b8b6e 3484This macro definition must go in the prologue of the grammar file
75f5aaea 3485(@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
bfa74976 3486
342b8b6e 3487@node Multiple Types
bfa74976
RS
3488@subsection More Than One Value Type
3489
3490In most programs, you will need different data types for different kinds
3491of tokens and groupings. For example, a numeric constant may need type
f8e1c9e5
AD
3492@code{int} or @code{long int}, while a string constant needs type
3493@code{char *}, and an identifier might need a pointer to an entry in the
3494symbol table.
bfa74976
RS
3495
3496To use more than one data type for semantic values in one parser, Bison
3497requires you to do two things:
3498
3499@itemize @bullet
3500@item
ddc8ede1 3501Specify the entire collection of possible data types, either by using the
704a47c4 3502@code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
ddc8ede1
PE
3503Value Types}), or by using a @code{typedef} or a @code{#define} to
3504define @code{YYSTYPE} to be a union type whose member names are
3505the type tags.
bfa74976
RS
3506
3507@item
14ded682
AD
3508Choose one of those types for each symbol (terminal or nonterminal) for
3509which semantic values are used. This is done for tokens with the
3510@code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3511and for groupings with the @code{%type} Bison declaration (@pxref{Type
3512Decl, ,Nonterminal Symbols}).
bfa74976
RS
3513@end itemize
3514
342b8b6e 3515@node Actions
bfa74976
RS
3516@subsection Actions
3517@cindex action
3518@vindex $$
3519@vindex $@var{n}
d013372c
AR
3520@vindex $@var{name}
3521@vindex $[@var{name}]
bfa74976
RS
3522
3523An action accompanies a syntactic rule and contains C code to be executed
3524each time an instance of that rule is recognized. The task of most actions
3525is to compute a semantic value for the grouping built by the rule from the
3526semantic values associated with tokens or smaller groupings.
3527
287c78f6
PE
3528An action consists of braced code containing C statements, and can be
3529placed at any position in the rule;
704a47c4
AD
3530it is executed at that position. Most rules have just one action at the
3531end of the rule, following all the components. Actions in the middle of
3532a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3533Actions, ,Actions in Mid-Rule}).
bfa74976 3534
ff7571c0
JD
3535The C code in an action can refer to the semantic values of the
3536components matched by the rule with the construct @code{$@var{n}},
3537which stands for the value of the @var{n}th component. The semantic
3538value for the grouping being constructed is @code{$$}. In addition,
3539the semantic values of symbols can be accessed with the named
3540references construct @code{$@var{name}} or @code{$[@var{name}]}.
3541Bison translates both of these constructs into expressions of the
3542appropriate type when it copies the actions into the parser
3543implementation file. @code{$$} (or @code{$@var{name}}, when it stands
3544for the current grouping) is translated to a modifiable lvalue, so it
3545can be assigned to.
bfa74976
RS
3546
3547Here is a typical example:
3548
3549@example
3550@group
3551exp: @dots{}
3552 | exp '+' exp
3553 @{ $$ = $1 + $3; @}
3554@end group
3555@end example
3556
d013372c
AR
3557Or, in terms of named references:
3558
3559@example
3560@group
3561exp[result]: @dots{}
3562 | exp[left] '+' exp[right]
3563 @{ $result = $left + $right; @}
3564@end group
3565@end example
3566
bfa74976
RS
3567@noindent
3568This rule constructs an @code{exp} from two smaller @code{exp} groupings
3569connected by a plus-sign token. In the action, @code{$1} and @code{$3}
d013372c 3570(@code{$left} and @code{$right})
bfa74976
RS
3571refer to the semantic values of the two component @code{exp} groupings,
3572which are the first and third symbols on the right hand side of the rule.
d013372c
AR
3573The sum is stored into @code{$$} (@code{$result}) so that it becomes the
3574semantic value of
bfa74976
RS
3575the addition-expression just recognized by the rule. If there were a
3576useful semantic value associated with the @samp{+} token, it could be
e0c471a9 3577referred to as @code{$2}.
bfa74976 3578
d013372c
AR
3579@xref{Named References,,Using Named References}, for more information
3580about using the named references construct.
3581
3ded9a63
AD
3582Note that the vertical-bar character @samp{|} is really a rule
3583separator, and actions are attached to a single rule. This is a
3584difference with tools like Flex, for which @samp{|} stands for either
3585``or'', or ``the same action as that of the next rule''. In the
3586following example, the action is triggered only when @samp{b} is found:
3587
3588@example
3589@group
3590a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3591@end group
3592@end example
3593
bfa74976
RS
3594@cindex default action
3595If you don't specify an action for a rule, Bison supplies a default:
72f889cc
AD
3596@w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3597becomes the value of the whole rule. Of course, the default action is
3598valid only if the two data types match. There is no meaningful default
3599action for an empty rule; every empty rule must have an explicit action
3600unless the rule's value does not matter.
bfa74976
RS
3601
3602@code{$@var{n}} with @var{n} zero or negative is allowed for reference
3603to tokens and groupings on the stack @emph{before} those that match the
3604current rule. This is a very risky practice, and to use it reliably
3605you must be certain of the context in which the rule is applied. Here
3606is a case in which you can use this reliably:
3607
3608@example
3609@group
3610foo: expr bar '+' expr @{ @dots{} @}
3611 | expr bar '-' expr @{ @dots{} @}
3612 ;
3613@end group
3614
3615@group
3616bar: /* empty */
3617 @{ previous_expr = $0; @}
3618 ;
3619@end group
3620@end example
3621
3622As long as @code{bar} is used only in the fashion shown here, @code{$0}
3623always refers to the @code{expr} which precedes @code{bar} in the
3624definition of @code{foo}.
3625
32c29292 3626@vindex yylval
742e4900 3627It is also possible to access the semantic value of the lookahead token, if
32c29292
JD
3628any, from a semantic action.
3629This semantic value is stored in @code{yylval}.
3630@xref{Action Features, ,Special Features for Use in Actions}.
3631
342b8b6e 3632@node Action Types
bfa74976
RS
3633@subsection Data Types of Values in Actions
3634@cindex action data types
3635@cindex data types in actions
3636
3637If you have chosen a single data type for semantic values, the @code{$$}
3638and @code{$@var{n}} constructs always have that data type.
3639
3640If you have used @code{%union} to specify a variety of data types, then you
3641must declare a choice among these types for each terminal or nonterminal
3642symbol that can have a semantic value. Then each time you use @code{$$} or
3643@code{$@var{n}}, its data type is determined by which symbol it refers to
e0c471a9 3644in the rule. In this example,
bfa74976
RS
3645
3646@example
3647@group
3648exp: @dots{}
3649 | exp '+' exp
3650 @{ $$ = $1 + $3; @}
3651@end group
3652@end example
3653
3654@noindent
3655@code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3656have the data type declared for the nonterminal symbol @code{exp}. If
3657@code{$2} were used, it would have the data type declared for the
e0c471a9 3658terminal symbol @code{'+'}, whatever that might be.
bfa74976
RS
3659
3660Alternatively, you can specify the data type when you refer to the value,
3661by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3662reference. For example, if you have defined types as shown here:
3663
3664@example
3665@group
3666%union @{
3667 int itype;
3668 double dtype;
3669@}
3670@end group
3671@end example
3672
3673@noindent
3674then you can write @code{$<itype>1} to refer to the first subunit of the
3675rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3676
342b8b6e 3677@node Mid-Rule Actions
bfa74976
RS
3678@subsection Actions in Mid-Rule
3679@cindex actions in mid-rule
3680@cindex mid-rule actions
3681
3682Occasionally it is useful to put an action in the middle of a rule.
3683These actions are written just like usual end-of-rule actions, but they
3684are executed before the parser even recognizes the following components.
3685
3686A mid-rule action may refer to the components preceding it using
3687@code{$@var{n}}, but it may not refer to subsequent components because
3688it is run before they are parsed.
3689
3690The mid-rule action itself counts as one of the components of the rule.
3691This makes a difference when there is another action later in the same rule
3692(and usually there is another at the end): you have to count the actions
3693along with the symbols when working out which number @var{n} to use in
3694@code{$@var{n}}.
3695
3696The mid-rule action can also have a semantic value. The action can set
3697its value with an assignment to @code{$$}, and actions later in the rule
3698can refer to the value using @code{$@var{n}}. Since there is no symbol
3699to name the action, there is no way to declare a data type for the value
fdc6758b
MA
3700in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3701specify a data type each time you refer to this value.
bfa74976
RS
3702
3703There is no way to set the value of the entire rule with a mid-rule
3704action, because assignments to @code{$$} do not have that effect. The
3705only way to set the value for the entire rule is with an ordinary action
3706at the end of the rule.
3707
3708Here is an example from a hypothetical compiler, handling a @code{let}
3709statement that looks like @samp{let (@var{variable}) @var{statement}} and
3710serves to create a variable named @var{variable} temporarily for the
3711duration of @var{statement}. To parse this construct, we must put
3712@var{variable} into the symbol table while @var{statement} is parsed, then
3713remove it afterward. Here is how it is done:
3714
3715@example
3716@group
3717stmt: LET '(' var ')'
3718 @{ $<context>$ = push_context ();
3719 declare_variable ($3); @}
3720 stmt @{ $$ = $6;
3721 pop_context ($<context>5); @}
3722@end group
3723@end example
3724
3725@noindent
3726As soon as @samp{let (@var{variable})} has been recognized, the first
3727action is run. It saves a copy of the current semantic context (the
3728list of accessible variables) as its semantic value, using alternative
3729@code{context} in the data-type union. Then it calls
3730@code{declare_variable} to add the new variable to that list. Once the
3731first action is finished, the embedded statement @code{stmt} can be
3732parsed. Note that the mid-rule action is component number 5, so the
3733@samp{stmt} is component number 6.
3734
3735After the embedded statement is parsed, its semantic value becomes the
3736value of the entire @code{let}-statement. Then the semantic value from the
3737earlier action is used to restore the prior list of variables. This
3738removes the temporary @code{let}-variable from the list so that it won't
3739appear to exist while the rest of the program is parsed.
3740
841a7737
JD
3741@findex %destructor
3742@cindex discarded symbols, mid-rule actions
3743@cindex error recovery, mid-rule actions
3744In the above example, if the parser initiates error recovery (@pxref{Error
3745Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3746it might discard the previous semantic context @code{$<context>5} without
3747restoring it.
3748Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3749Discarded Symbols}).
ec5479ce
JD
3750However, Bison currently provides no means to declare a destructor specific to
3751a particular mid-rule action's semantic value.
841a7737
JD
3752
3753One solution is to bury the mid-rule action inside a nonterminal symbol and to
3754declare a destructor for that symbol:
3755
3756@example
3757@group
3758%type <context> let
3759%destructor @{ pop_context ($$); @} let
3760
3761%%
3762
3763stmt: let stmt
3764 @{ $$ = $2;
3765 pop_context ($1); @}
3766 ;
3767
3768let: LET '(' var ')'
3769 @{ $$ = push_context ();
3770 declare_variable ($3); @}
3771 ;
3772
3773@end group
3774@end example
3775
3776@noindent
3777Note that the action is now at the end of its rule.
3778Any mid-rule action can be converted to an end-of-rule action in this way, and
3779this is what Bison actually does to implement mid-rule actions.
3780
bfa74976
RS
3781Taking action before a rule is completely recognized often leads to
3782conflicts since the parser must commit to a parse in order to execute the
3783action. For example, the following two rules, without mid-rule actions,
3784can coexist in a working parser because the parser can shift the open-brace
3785token and look at what follows before deciding whether there is a
3786declaration or not:
3787
3788@example
3789@group
3790compound: '@{' declarations statements '@}'
3791 | '@{' statements '@}'
3792 ;
3793@end group
3794@end example
3795
3796@noindent
3797But when we add a mid-rule action as follows, the rules become nonfunctional:
3798
3799@example
3800@group
3801compound: @{ prepare_for_local_variables (); @}
3802 '@{' declarations statements '@}'
3803@end group
3804@group
3805 | '@{' statements '@}'
3806 ;
3807@end group
3808@end example
3809
3810@noindent
3811Now the parser is forced to decide whether to run the mid-rule action
3812when it has read no farther than the open-brace. In other words, it
3813must commit to using one rule or the other, without sufficient
3814information to do it correctly. (The open-brace token is what is called
742e4900
JD
3815the @dfn{lookahead} token at this time, since the parser is still
3816deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.)
bfa74976
RS
3817
3818You might think that you could correct the problem by putting identical
3819actions into the two rules, like this:
3820
3821@example
3822@group
3823compound: @{ prepare_for_local_variables (); @}
3824 '@{' declarations statements '@}'
3825 | @{ prepare_for_local_variables (); @}
3826 '@{' statements '@}'
3827 ;
3828@end group
3829@end example
3830
3831@noindent
3832But this does not help, because Bison does not realize that the two actions
3833are identical. (Bison never tries to understand the C code in an action.)
3834
3835If the grammar is such that a declaration can be distinguished from a
3836statement by the first token (which is true in C), then one solution which
3837does work is to put the action after the open-brace, like this:
3838
3839@example
3840@group
3841compound: '@{' @{ prepare_for_local_variables (); @}
3842 declarations statements '@}'
3843 | '@{' statements '@}'
3844 ;
3845@end group
3846@end example
3847
3848@noindent
3849Now the first token of the following declaration or statement,
3850which would in any case tell Bison which rule to use, can still do so.
3851
3852Another solution is to bury the action inside a nonterminal symbol which
3853serves as a subroutine:
3854
3855@example
3856@group
3857subroutine: /* empty */
3858 @{ prepare_for_local_variables (); @}
3859 ;
3860
3861@end group
3862
3863@group
3864compound: subroutine
3865 '@{' declarations statements '@}'
3866 | subroutine
3867 '@{' statements '@}'
3868 ;
3869@end group
3870@end example
3871
3872@noindent
3873Now Bison can execute the action in the rule for @code{subroutine} without
841a7737 3874deciding which rule for @code{compound} it will eventually use.
bfa74976 3875
d013372c
AR
3876@node Named References
3877@subsection Using Named References
3878@cindex named references
3879
3880While every semantic value can be accessed with positional references
3881@code{$@var{n}} and @code{$$}, it's often much more convenient to refer to
3882them by name. First of all, original symbol names may be used as named
3883references. For example:
3884
3885@example
3886@group
3887invocation: op '(' args ')'
3888 @{ $invocation = new_invocation ($op, $args, @@invocation); @}
3889@end group
3890@end example
3891
3892@noindent
3893The positional @code{$$}, @code{@@$}, @code{$n}, and @code{@@n} can be
3894mixed with @code{$name} and @code{@@name} arbitrarily. For example:
3895
3896@example
3897@group
3898invocation: op '(' args ')'
3899 @{ $$ = new_invocation ($op, $args, @@$); @}
3900@end group
3901@end example
3902
3903@noindent
3904However, sometimes regular symbol names are not sufficient due to
3905ambiguities:
3906
3907@example
3908@group
3909exp: exp '/' exp
3910 @{ $exp = $exp / $exp; @} // $exp is ambiguous.
3911
3912exp: exp '/' exp
3913 @{ $$ = $1 / $exp; @} // One usage is ambiguous.
3914
3915exp: exp '/' exp
3916 @{ $$ = $1 / $3; @} // No error.
3917@end group
3918@end example
3919
3920@noindent
3921When ambiguity occurs, explicitly declared names may be used for values and
3922locations. Explicit names are declared as a bracketed name after a symbol
3923appearance in rule definitions. For example:
3924@example
3925@group
3926exp[result]: exp[left] '/' exp[right]
3927 @{ $result = $left / $right; @}
3928@end group
3929@end example
3930
3931@noindent
3932Explicit names may be declared for RHS and for LHS symbols as well. In order
3933to access a semantic value generated by a mid-rule action, an explicit name
3934may also be declared by putting a bracketed name after the closing brace of
3935the mid-rule action code:
3936@example
3937@group
3938exp[res]: exp[x] '+' @{$left = $x;@}[left] exp[right]
3939 @{ $res = $left + $right; @}
3940@end group
3941@end example
3942
3943@noindent
3944
3945In references, in order to specify names containing dots and dashes, an explicit
3946bracketed syntax @code{$[name]} and @code{@@[name]} must be used:
3947@example
3948@group
3949if-stmt: IF '(' expr ')' THEN then.stmt ';'
3950 @{ $[if-stmt] = new_if_stmt ($expr, $[then.stmt]); @}
3951@end group
3952@end example
3953
3954It often happens that named references are followed by a dot, dash or other
3955C punctuation marks and operators. By default, Bison will read
3956@code{$name.suffix} as a reference to symbol value @code{$name} followed by
3957@samp{.suffix}, i.e., an access to the @samp{suffix} field of the semantic
3958value. In order to force Bison to recognize @code{name.suffix} in its entirety
3959as the name of a semantic value, bracketed syntax @code{$[name.suffix]}
3960must be used.
3961
3962
342b8b6e 3963@node Locations
847bf1f5
AD
3964@section Tracking Locations
3965@cindex location
95923bd6
AD
3966@cindex textual location
3967@cindex location, textual
847bf1f5
AD
3968
3969Though grammar rules and semantic actions are enough to write a fully
72d2299c 3970functional parser, it can be useful to process some additional information,
3e259915
MA
3971especially symbol locations.
3972
704a47c4
AD
3973The way locations are handled is defined by providing a data type, and
3974actions to take when rules are matched.
847bf1f5
AD
3975
3976@menu
3977* Location Type:: Specifying a data type for locations.
3978* Actions and Locations:: Using locations in actions.
3979* Location Default Action:: Defining a general way to compute locations.
3980@end menu
3981
342b8b6e 3982@node Location Type
847bf1f5
AD
3983@subsection Data Type of Locations
3984@cindex data type of locations
3985@cindex default location type
3986
3987Defining a data type for locations is much simpler than for semantic values,
3988since all tokens and groupings always use the same type.
3989
50cce58e
PE
3990You can specify the type of locations by defining a macro called
3991@code{YYLTYPE}, just as you can specify the semantic value type by
ddc8ede1 3992defining a @code{YYSTYPE} macro (@pxref{Value Type}).
847bf1f5
AD
3993When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3994four members:
3995
3996@example
6273355b 3997typedef struct YYLTYPE
847bf1f5
AD
3998@{
3999 int first_line;
4000 int first_column;
4001 int last_line;
4002 int last_column;
6273355b 4003@} YYLTYPE;
847bf1f5
AD
4004@end example
4005
d59e456d
AD
4006When @code{YYLTYPE} is not defined, at the beginning of the parsing, Bison
4007initializes all these fields to 1 for @code{yylloc}. To initialize
4008@code{yylloc} with a custom location type (or to chose a different
4009initialization), use the @code{%initial-action} directive. @xref{Initial
4010Action Decl, , Performing Actions before Parsing}.
cd48d21d 4011
342b8b6e 4012@node Actions and Locations
847bf1f5
AD
4013@subsection Actions and Locations
4014@cindex location actions
4015@cindex actions, location
4016@vindex @@$
4017@vindex @@@var{n}
d013372c
AR
4018@vindex @@@var{name}
4019@vindex @@[@var{name}]
847bf1f5
AD
4020
4021Actions are not only useful for defining language semantics, but also for
4022describing the behavior of the output parser with locations.
4023
4024The most obvious way for building locations of syntactic groupings is very
72d2299c 4025similar to the way semantic values are computed. In a given rule, several
847bf1f5
AD
4026constructs can be used to access the locations of the elements being matched.
4027The location of the @var{n}th component of the right hand side is
4028@code{@@@var{n}}, while the location of the left hand side grouping is
4029@code{@@$}.
4030
d013372c
AR
4031In addition, the named references construct @code{@@@var{name}} and
4032@code{@@[@var{name}]} may also be used to address the symbol locations.
4033@xref{Named References,,Using Named References}, for more information
4034about using the named references construct.
4035
3e259915 4036Here is a basic example using the default data type for locations:
847bf1f5
AD
4037
4038@example
4039@group
4040exp: @dots{}
3e259915 4041 | exp '/' exp
847bf1f5 4042 @{
3e259915
MA
4043 @@$.first_column = @@1.first_column;
4044 @@$.first_line = @@1.first_line;
847bf1f5
AD
4045 @@$.last_column = @@3.last_column;
4046 @@$.last_line = @@3.last_line;
3e259915
MA
4047 if ($3)
4048 $$ = $1 / $3;
4049 else
4050 @{
4051 $$ = 1;
4e03e201
AD
4052 fprintf (stderr,
4053 "Division by zero, l%d,c%d-l%d,c%d",
4054 @@3.first_line, @@3.first_column,
4055 @@3.last_line, @@3.last_column);
3e259915 4056 @}
847bf1f5
AD
4057 @}
4058@end group
4059@end example
4060
3e259915 4061As for semantic values, there is a default action for locations that is
72d2299c 4062run each time a rule is matched. It sets the beginning of @code{@@$} to the
3e259915 4063beginning of the first symbol, and the end of @code{@@$} to the end of the
79282c6c 4064last symbol.
3e259915 4065
72d2299c 4066With this default action, the location tracking can be fully automatic. The
3e259915
MA
4067example above simply rewrites this way:
4068
4069@example
4070@group
4071exp: @dots{}
4072 | exp '/' exp
4073 @{
4074 if ($3)
4075 $$ = $1 / $3;
4076 else
4077 @{
4078 $$ = 1;
4e03e201
AD
4079 fprintf (stderr,
4080 "Division by zero, l%d,c%d-l%d,c%d",
4081 @@3.first_line, @@3.first_column,
4082 @@3.last_line, @@3.last_column);
3e259915
MA
4083 @}
4084 @}
4085@end group
4086@end example
847bf1f5 4087
32c29292 4088@vindex yylloc
742e4900 4089It is also possible to access the location of the lookahead token, if any,
32c29292
JD
4090from a semantic action.
4091This location is stored in @code{yylloc}.
4092@xref{Action Features, ,Special Features for Use in Actions}.
4093
342b8b6e 4094@node Location Default Action
847bf1f5
AD
4095@subsection Default Action for Locations
4096@vindex YYLLOC_DEFAULT
8a4281b9 4097@cindex GLR parsers and @code{YYLLOC_DEFAULT}
847bf1f5 4098
72d2299c 4099Actually, actions are not the best place to compute locations. Since
704a47c4
AD
4100locations are much more general than semantic values, there is room in
4101the output parser to redefine the default action to take for each
72d2299c 4102rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
96b93a3d
PE
4103matched, before the associated action is run. It is also invoked
4104while processing a syntax error, to compute the error's location.
8a4281b9 4105Before reporting an unresolvable syntactic ambiguity, a GLR
8710fc41
JD
4106parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
4107of that ambiguity.
847bf1f5 4108
3e259915 4109Most of the time, this macro is general enough to suppress location
79282c6c 4110dedicated code from semantic actions.
847bf1f5 4111
72d2299c 4112The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
96b93a3d 4113the location of the grouping (the result of the computation). When a
766de5eb 4114rule is matched, the second parameter identifies locations of
96b93a3d 4115all right hand side elements of the rule being matched, and the third
8710fc41 4116parameter is the size of the rule's right hand side.
8a4281b9 4117When a GLR parser reports an ambiguity, which of multiple candidate
8710fc41
JD
4118right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
4119When processing a syntax error, the second parameter identifies locations
4120of the symbols that were discarded during error processing, and the third
96b93a3d 4121parameter is the number of discarded symbols.
847bf1f5 4122
766de5eb 4123By default, @code{YYLLOC_DEFAULT} is defined this way:
847bf1f5 4124
766de5eb 4125@smallexample
847bf1f5 4126@group
766de5eb
PE
4127# define YYLLOC_DEFAULT(Current, Rhs, N) \
4128 do \
4129 if (N) \
4130 @{ \
4131 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
4132 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
4133 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
4134 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
4135 @} \
4136 else \
4137 @{ \
4138 (Current).first_line = (Current).last_line = \
4139 YYRHSLOC(Rhs, 0).last_line; \
4140 (Current).first_column = (Current).last_column = \
4141 YYRHSLOC(Rhs, 0).last_column; \
4142 @} \
4143 while (0)
847bf1f5 4144@end group
766de5eb 4145@end smallexample
676385e2 4146
766de5eb
PE
4147where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
4148in @var{rhs} when @var{k} is positive, and the location of the symbol
f28ac696 4149just before the reduction when @var{k} and @var{n} are both zero.
676385e2 4150
3e259915 4151When defining @code{YYLLOC_DEFAULT}, you should consider that:
847bf1f5 4152
3e259915 4153@itemize @bullet
79282c6c 4154@item
72d2299c 4155All arguments are free of side-effects. However, only the first one (the
3e259915 4156result) should be modified by @code{YYLLOC_DEFAULT}.
847bf1f5 4157
3e259915 4158@item
766de5eb
PE
4159For consistency with semantic actions, valid indexes within the
4160right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
4161valid index, and it refers to the symbol just before the reduction.
4162During error processing @var{n} is always positive.
0ae99356
PE
4163
4164@item
4165Your macro should parenthesize its arguments, if need be, since the
4166actual arguments may not be surrounded by parentheses. Also, your
4167macro should expand to something that can be used as a single
4168statement when it is followed by a semicolon.
3e259915 4169@end itemize
847bf1f5 4170
342b8b6e 4171@node Declarations
bfa74976
RS
4172@section Bison Declarations
4173@cindex declarations, Bison
4174@cindex Bison declarations
4175
4176The @dfn{Bison declarations} section of a Bison grammar defines the symbols
4177used in formulating the grammar and the data types of semantic values.
4178@xref{Symbols}.
4179
4180All token type names (but not single-character literal tokens such as
4181@code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
4182declared if you need to specify which data type to use for the semantic
4183value (@pxref{Multiple Types, ,More Than One Value Type}).
4184
ff7571c0
JD
4185The first rule in the grammar file also specifies the start symbol, by
4186default. If you want some other symbol to be the start symbol, you
4187must declare it explicitly (@pxref{Language and Grammar, ,Languages
4188and Context-Free Grammars}).
bfa74976
RS
4189
4190@menu
b50d2359 4191* Require Decl:: Requiring a Bison version.
bfa74976
RS
4192* Token Decl:: Declaring terminal symbols.
4193* Precedence Decl:: Declaring terminals with precedence and associativity.
4194* Union Decl:: Declaring the set of all semantic value types.
4195* Type Decl:: Declaring the choice of type for a nonterminal symbol.
18d192f0 4196* Initial Action Decl:: Code run before parsing starts.
72f889cc 4197* Destructor Decl:: Declaring how symbols are freed.
d6328241 4198* Expect Decl:: Suppressing warnings about parsing conflicts.
bfa74976
RS
4199* Start Decl:: Specifying the start symbol.
4200* Pure Decl:: Requesting a reentrant parser.
9987d1b3 4201* Push Decl:: Requesting a push parser.
bfa74976 4202* Decl Summary:: Table of all Bison declarations.
35c1e5f0 4203* %define Summary:: Defining variables to adjust Bison's behavior.
e0c07222 4204* %code Summary:: Inserting code into the parser source.
bfa74976
RS
4205@end menu
4206
b50d2359
AD
4207@node Require Decl
4208@subsection Require a Version of Bison
4209@cindex version requirement
4210@cindex requiring a version of Bison
4211@findex %require
4212
4213You may require the minimum version of Bison to process the grammar. If
9b8a5ce0
AD
4214the requirement is not met, @command{bison} exits with an error (exit
4215status 63).
b50d2359
AD
4216
4217@example
4218%require "@var{version}"
4219@end example
4220
342b8b6e 4221@node Token Decl
bfa74976
RS
4222@subsection Token Type Names
4223@cindex declaring token type names
4224@cindex token type names, declaring
931c7513 4225@cindex declaring literal string tokens
bfa74976
RS
4226@findex %token
4227
4228The basic way to declare a token type name (terminal symbol) is as follows:
4229
4230@example
4231%token @var{name}
4232@end example
4233
4234Bison will convert this into a @code{#define} directive in
4235the parser, so that the function @code{yylex} (if it is in this file)
4236can use the name @var{name} to stand for this token type's code.
4237
d78f0ac9
AD
4238Alternatively, you can use @code{%left}, @code{%right},
4239@code{%precedence}, or
14ded682
AD
4240@code{%nonassoc} instead of @code{%token}, if you wish to specify
4241associativity and precedence. @xref{Precedence Decl, ,Operator
4242Precedence}.
bfa74976
RS
4243
4244You can explicitly specify the numeric code for a token type by appending
b1cc23c4 4245a nonnegative decimal or hexadecimal integer value in the field immediately
1452af69 4246following the token name:
bfa74976
RS
4247
4248@example
4249%token NUM 300
1452af69 4250%token XNUM 0x12d // a GNU extension
bfa74976
RS
4251@end example
4252
4253@noindent
4254It is generally best, however, to let Bison choose the numeric codes for
4255all token types. Bison will automatically select codes that don't conflict
e966383b 4256with each other or with normal characters.
bfa74976
RS
4257
4258In the event that the stack type is a union, you must augment the
4259@code{%token} or other token declaration to include the data type
704a47c4
AD
4260alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
4261Than One Value Type}).
bfa74976
RS
4262
4263For example:
4264
4265@example
4266@group
4267%union @{ /* define stack type */
4268 double val;
4269 symrec *tptr;
4270@}
4271%token <val> NUM /* define token NUM and its type */
4272@end group
4273@end example
4274
931c7513
RS
4275You can associate a literal string token with a token type name by
4276writing the literal string at the end of a @code{%token}
4277declaration which declares the name. For example:
4278
4279@example
4280%token arrow "=>"
4281@end example
4282
4283@noindent
4284For example, a grammar for the C language might specify these names with
4285equivalent literal string tokens:
4286
4287@example
4288%token <operator> OR "||"
4289%token <operator> LE 134 "<="
4290%left OR "<="
4291@end example
4292
4293@noindent
4294Once you equate the literal string and the token name, you can use them
4295interchangeably in further declarations or the grammar rules. The
4296@code{yylex} function can use the token name or the literal string to
4297obtain the token type code number (@pxref{Calling Convention}).
b1cc23c4
JD
4298Syntax error messages passed to @code{yyerror} from the parser will reference
4299the literal string instead of the token name.
4300
4301The token numbered as 0 corresponds to end of file; the following line
4302allows for nicer error messages referring to ``end of file'' instead
4303of ``$end'':
4304
4305@example
4306%token END 0 "end of file"
4307@end example
931c7513 4308
342b8b6e 4309@node Precedence Decl
bfa74976
RS
4310@subsection Operator Precedence
4311@cindex precedence declarations
4312@cindex declaring operator precedence
4313@cindex operator precedence, declaring
4314
d78f0ac9
AD
4315Use the @code{%left}, @code{%right}, @code{%nonassoc}, or
4316@code{%precedence} declaration to
bfa74976
RS
4317declare a token and specify its precedence and associativity, all at
4318once. These are called @dfn{precedence declarations}.
704a47c4
AD
4319@xref{Precedence, ,Operator Precedence}, for general information on
4320operator precedence.
bfa74976 4321
ab7f29f8 4322The syntax of a precedence declaration is nearly the same as that of
bfa74976
RS
4323@code{%token}: either
4324
4325@example
4326%left @var{symbols}@dots{}
4327@end example
4328
4329@noindent
4330or
4331
4332@example
4333%left <@var{type}> @var{symbols}@dots{}
4334@end example
4335
4336And indeed any of these declarations serves the purposes of @code{%token}.
4337But in addition, they specify the associativity and relative precedence for
4338all the @var{symbols}:
4339
4340@itemize @bullet
4341@item
4342The associativity of an operator @var{op} determines how repeated uses
4343of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
4344@var{z}} is parsed by grouping @var{x} with @var{y} first or by
4345grouping @var{y} with @var{z} first. @code{%left} specifies
4346left-associativity (grouping @var{x} with @var{y} first) and
4347@code{%right} specifies right-associativity (grouping @var{y} with
4348@var{z} first). @code{%nonassoc} specifies no associativity, which
4349means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
4350considered a syntax error.
4351
d78f0ac9
AD
4352@code{%precedence} gives only precedence to the @var{symbols}, and
4353defines no associativity at all. Use this to define precedence only,
4354and leave any potential conflict due to associativity enabled.
4355
bfa74976
RS
4356@item
4357The precedence of an operator determines how it nests with other operators.
4358All the tokens declared in a single precedence declaration have equal
4359precedence and nest together according to their associativity.
4360When two tokens declared in different precedence declarations associate,
4361the one declared later has the higher precedence and is grouped first.
4362@end itemize
4363
ab7f29f8
JD
4364For backward compatibility, there is a confusing difference between the
4365argument lists of @code{%token} and precedence declarations.
4366Only a @code{%token} can associate a literal string with a token type name.
4367A precedence declaration always interprets a literal string as a reference to a
4368separate token.
4369For example:
4370
4371@example
4372%left OR "<=" // Does not declare an alias.
4373%left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
4374@end example
4375
342b8b6e 4376@node Union Decl
bfa74976
RS
4377@subsection The Collection of Value Types
4378@cindex declaring value types
4379@cindex value types, declaring
4380@findex %union
4381
287c78f6
PE
4382The @code{%union} declaration specifies the entire collection of
4383possible data types for semantic values. The keyword @code{%union} is
4384followed by braced code containing the same thing that goes inside a
4385@code{union} in C@.
bfa74976
RS
4386
4387For example:
4388
4389@example
4390@group
4391%union @{
4392 double val;
4393 symrec *tptr;
4394@}
4395@end group
4396@end example
4397
4398@noindent
4399This says that the two alternative types are @code{double} and @code{symrec
4400*}. They are given names @code{val} and @code{tptr}; these names are used
4401in the @code{%token} and @code{%type} declarations to pick one of the types
4402for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
4403
8a4281b9 4404As an extension to POSIX, a tag is allowed after the
6273355b
PE
4405@code{union}. For example:
4406
4407@example
4408@group
4409%union value @{
4410 double val;
4411 symrec *tptr;
4412@}
4413@end group
4414@end example
4415
d6ca7905 4416@noindent
6273355b
PE
4417specifies the union tag @code{value}, so the corresponding C type is
4418@code{union value}. If you do not specify a tag, it defaults to
4419@code{YYSTYPE}.
4420
8a4281b9 4421As another extension to POSIX, you may specify multiple
d6ca7905
PE
4422@code{%union} declarations; their contents are concatenated. However,
4423only the first @code{%union} declaration can specify a tag.
4424
6273355b 4425Note that, unlike making a @code{union} declaration in C, you need not write
bfa74976
RS
4426a semicolon after the closing brace.
4427
ddc8ede1
PE
4428Instead of @code{%union}, you can define and use your own union type
4429@code{YYSTYPE} if your grammar contains at least one
4430@samp{<@var{type}>} tag. For example, you can put the following into
4431a header file @file{parser.h}:
4432
4433@example
4434@group
4435union YYSTYPE @{
4436 double val;
4437 symrec *tptr;
4438@};
4439typedef union YYSTYPE YYSTYPE;
4440@end group
4441@end example
4442
4443@noindent
4444and then your grammar can use the following
4445instead of @code{%union}:
4446
4447@example
4448@group
4449%@{
4450#include "parser.h"
4451%@}
4452%type <val> expr
4453%token <tptr> ID
4454@end group
4455@end example
4456
342b8b6e 4457@node Type Decl
bfa74976
RS
4458@subsection Nonterminal Symbols
4459@cindex declaring value types, nonterminals
4460@cindex value types, nonterminals, declaring
4461@findex %type
4462
4463@noindent
4464When you use @code{%union} to specify multiple value types, you must
4465declare the value type of each nonterminal symbol for which values are
4466used. This is done with a @code{%type} declaration, like this:
4467
4468@example
4469%type <@var{type}> @var{nonterminal}@dots{}
4470@end example
4471
4472@noindent
704a47c4
AD
4473Here @var{nonterminal} is the name of a nonterminal symbol, and
4474@var{type} is the name given in the @code{%union} to the alternative
4475that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
4476can give any number of nonterminal symbols in the same @code{%type}
4477declaration, if they have the same value type. Use spaces to separate
4478the symbol names.
bfa74976 4479
931c7513
RS
4480You can also declare the value type of a terminal symbol. To do this,
4481use the same @code{<@var{type}>} construction in a declaration for the
4482terminal symbol. All kinds of token declarations allow
4483@code{<@var{type}>}.
4484
18d192f0
AD
4485@node Initial Action Decl
4486@subsection Performing Actions before Parsing
4487@findex %initial-action
4488
4489Sometimes your parser needs to perform some initializations before
4490parsing. The @code{%initial-action} directive allows for such arbitrary
4491code.
4492
4493@deffn {Directive} %initial-action @{ @var{code} @}
4494@findex %initial-action
287c78f6 4495Declare that the braced @var{code} must be invoked before parsing each time
451364ed 4496@code{yyparse} is called. The @var{code} may use @code{$$} and
742e4900 4497@code{@@$} --- initial value and location of the lookahead --- and the
451364ed 4498@code{%parse-param}.
18d192f0
AD
4499@end deffn
4500
451364ed
AD
4501For instance, if your locations use a file name, you may use
4502
4503@example
48b16bbc 4504%parse-param @{ char const *file_name @};
451364ed
AD
4505%initial-action
4506@{
4626a15d 4507 @@$.initialize (file_name);
451364ed
AD
4508@};
4509@end example
4510
18d192f0 4511
72f889cc
AD
4512@node Destructor Decl
4513@subsection Freeing Discarded Symbols
4514@cindex freeing discarded symbols
4515@findex %destructor
12e35840 4516@findex <*>
3ebecc24 4517@findex <>
a85284cf
AD
4518During error recovery (@pxref{Error Recovery}), symbols already pushed
4519on the stack and tokens coming from the rest of the file are discarded
4520until the parser falls on its feet. If the parser runs out of memory,
9d9b8b70 4521or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
a85284cf
AD
4522symbols on the stack must be discarded. Even if the parser succeeds, it
4523must discard the start symbol.
258b75ca
PE
4524
4525When discarded symbols convey heap based information, this memory is
4526lost. While this behavior can be tolerable for batch parsers, such as
4b367315
AD
4527in traditional compilers, it is unacceptable for programs like shells or
4528protocol implementations that may parse and execute indefinitely.
258b75ca 4529
a85284cf
AD
4530The @code{%destructor} directive defines code that is called when a
4531symbol is automatically discarded.
72f889cc
AD
4532
4533@deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
4534@findex %destructor
287c78f6
PE
4535Invoke the braced @var{code} whenever the parser discards one of the
4536@var{symbols}.
4b367315 4537Within @var{code}, @code{$$} designates the semantic value associated
ec5479ce
JD
4538with the discarded symbol, and @code{@@$} designates its location.
4539The additional parser parameters are also available (@pxref{Parser Function, ,
4540The Parser Function @code{yyparse}}).
ec5479ce 4541
b2a0b7ca
JD
4542When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
4543per-symbol @code{%destructor}.
4544You may also define a per-type @code{%destructor} by listing a semantic type
12e35840 4545tag among @var{symbols}.
b2a0b7ca 4546In that case, the parser will invoke this @var{code} whenever it discards any
12e35840 4547grammar symbol that has that semantic type tag unless that symbol has its own
b2a0b7ca
JD
4548per-symbol @code{%destructor}.
4549
12e35840 4550Finally, you can define two different kinds of default @code{%destructor}s.
85894313
JD
4551(These default forms are experimental.
4552More user feedback will help to determine whether they should become permanent
4553features.)
3ebecc24 4554You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
12e35840
JD
4555exactly one @code{%destructor} declaration in your grammar file.
4556The parser will invoke the @var{code} associated with one of these whenever it
4557discards any user-defined grammar symbol that has no per-symbol and no per-type
4558@code{%destructor}.
4559The parser uses the @var{code} for @code{<*>} in the case of such a grammar
4560symbol for which you have formally declared a semantic type tag (@code{%type}
4561counts as such a declaration, but @code{$<tag>$} does not).
3ebecc24 4562The parser uses the @var{code} for @code{<>} in the case of such a grammar
12e35840 4563symbol that has no declared semantic type tag.
72f889cc
AD
4564@end deffn
4565
b2a0b7ca 4566@noindent
12e35840 4567For example:
72f889cc
AD
4568
4569@smallexample
ec5479ce
JD
4570%union @{ char *string; @}
4571%token <string> STRING1
4572%token <string> STRING2
4573%type <string> string1
4574%type <string> string2
b2a0b7ca
JD
4575%union @{ char character; @}
4576%token <character> CHR
4577%type <character> chr
12e35840
JD
4578%token TAGLESS
4579
b2a0b7ca 4580%destructor @{ @} <character>
12e35840
JD
4581%destructor @{ free ($$); @} <*>
4582%destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
3ebecc24 4583%destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
72f889cc
AD
4584@end smallexample
4585
4586@noindent
b2a0b7ca
JD
4587guarantees that, when the parser discards any user-defined symbol that has a
4588semantic type tag other than @code{<character>}, it passes its semantic value
12e35840 4589to @code{free} by default.
ec5479ce
JD
4590However, when the parser discards a @code{STRING1} or a @code{string1}, it also
4591prints its line number to @code{stdout}.
4592It performs only the second @code{%destructor} in this case, so it invokes
4593@code{free} only once.
12e35840
JD
4594Finally, the parser merely prints a message whenever it discards any symbol,
4595such as @code{TAGLESS}, that has no semantic type tag.
4596
4597A Bison-generated parser invokes the default @code{%destructor}s only for
4598user-defined as opposed to Bison-defined symbols.
4599For example, the parser will not invoke either kind of default
4600@code{%destructor} for the special Bison-defined symbols @code{$accept},
4601@code{$undefined}, or @code{$end} (@pxref{Table of Symbols, ,Bison Symbols}),
4602none of which you can reference in your grammar.
4603It also will not invoke either for the @code{error} token (@pxref{Table of
4604Symbols, ,error}), which is always defined by Bison regardless of whether you
4605reference it in your grammar.
4606However, it may invoke one of them for the end token (token 0) if you
4607redefine it from @code{$end} to, for example, @code{END}:
3508ce36
JD
4608
4609@smallexample
4610%token END 0
4611@end smallexample
4612
12e35840
JD
4613@cindex actions in mid-rule
4614@cindex mid-rule actions
4615Finally, Bison will never invoke a @code{%destructor} for an unreferenced
4616mid-rule semantic value (@pxref{Mid-Rule Actions,,Actions in Mid-Rule}).
4617That is, Bison does not consider a mid-rule to have a semantic value if you do
4618not reference @code{$$} in the mid-rule's action or @code{$@var{n}} (where
4619@var{n} is the RHS symbol position of the mid-rule) in any later action in that
4620rule.
4621However, if you do reference either, the Bison-generated parser will invoke the
3ebecc24 4622@code{<>} @code{%destructor} whenever it discards the mid-rule symbol.
12e35840 4623
3508ce36
JD
4624@ignore
4625@noindent
4626In the future, it may be possible to redefine the @code{error} token as a
4627nonterminal that captures the discarded symbols.
4628In that case, the parser will invoke the default destructor for it as well.
4629@end ignore
4630
e757bb10
AD
4631@sp 1
4632
4633@cindex discarded symbols
4634@dfn{Discarded symbols} are the following:
4635
4636@itemize
4637@item
4638stacked symbols popped during the first phase of error recovery,
4639@item
4640incoming terminals during the second phase of error recovery,
4641@item
742e4900 4642the current lookahead and the entire stack (except the current
9d9b8b70 4643right-hand side symbols) when the parser returns immediately, and
258b75ca
PE
4644@item
4645the start symbol, when the parser succeeds.
e757bb10
AD
4646@end itemize
4647
9d9b8b70
PE
4648The parser can @dfn{return immediately} because of an explicit call to
4649@code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
4650exhaustion.
4651
29553547 4652Right-hand side symbols of a rule that explicitly triggers a syntax
9d9b8b70
PE
4653error via @code{YYERROR} are not discarded automatically. As a rule
4654of thumb, destructors are invoked only when user actions cannot manage
a85284cf 4655the memory.
e757bb10 4656
342b8b6e 4657@node Expect Decl
bfa74976
RS
4658@subsection Suppressing Conflict Warnings
4659@cindex suppressing conflict warnings
4660@cindex preventing warnings about conflicts
4661@cindex warnings, preventing
4662@cindex conflicts, suppressing warnings of
4663@findex %expect
d6328241 4664@findex %expect-rr
bfa74976
RS
4665
4666Bison normally warns if there are any conflicts in the grammar
7da99ede
AD
4667(@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4668have harmless shift/reduce conflicts which are resolved in a predictable
4669way and would be difficult to eliminate. It is desirable to suppress
4670the warning about these conflicts unless the number of conflicts
4671changes. You can do this with the @code{%expect} declaration.
bfa74976
RS
4672
4673The declaration looks like this:
4674
4675@example
4676%expect @var{n}
4677@end example
4678
035aa4a0
PE
4679Here @var{n} is a decimal integer. The declaration says there should
4680be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4681Bison reports an error if the number of shift/reduce conflicts differs
4682from @var{n}, or if there are any reduce/reduce conflicts.
bfa74976 4683
eb45ef3b 4684For deterministic parsers, reduce/reduce conflicts are more
035aa4a0 4685serious, and should be eliminated entirely. Bison will always report
8a4281b9 4686reduce/reduce conflicts for these parsers. With GLR
035aa4a0 4687parsers, however, both kinds of conflicts are routine; otherwise,
8a4281b9 4688there would be no need to use GLR parsing. Therefore, it is
035aa4a0 4689also possible to specify an expected number of reduce/reduce conflicts
8a4281b9 4690in GLR parsers, using the declaration:
d6328241
PH
4691
4692@example
4693%expect-rr @var{n}
4694@end example
4695
bfa74976
RS
4696In general, using @code{%expect} involves these steps:
4697
4698@itemize @bullet
4699@item
4700Compile your grammar without @code{%expect}. Use the @samp{-v} option
4701to get a verbose list of where the conflicts occur. Bison will also
4702print the number of conflicts.
4703
4704@item
4705Check each of the conflicts to make sure that Bison's default
4706resolution is what you really want. If not, rewrite the grammar and
4707go back to the beginning.
4708
4709@item
4710Add an @code{%expect} declaration, copying the number @var{n} from the
8a4281b9 4711number which Bison printed. With GLR parsers, add an
035aa4a0 4712@code{%expect-rr} declaration as well.
bfa74976
RS
4713@end itemize
4714
93d7dde9
JD
4715Now Bison will report an error if you introduce an unexpected conflict,
4716but will keep silent otherwise.
bfa74976 4717
342b8b6e 4718@node Start Decl
bfa74976
RS
4719@subsection The Start-Symbol
4720@cindex declaring the start symbol
4721@cindex start symbol, declaring
4722@cindex default start symbol
4723@findex %start
4724
4725Bison assumes by default that the start symbol for the grammar is the first
4726nonterminal specified in the grammar specification section. The programmer
4727may override this restriction with the @code{%start} declaration as follows:
4728
4729@example
4730%start @var{symbol}
4731@end example
4732
342b8b6e 4733@node Pure Decl
bfa74976
RS
4734@subsection A Pure (Reentrant) Parser
4735@cindex reentrant parser
4736@cindex pure parser
d9df47b6 4737@findex %define api.pure
bfa74976
RS
4738
4739A @dfn{reentrant} program is one which does not alter in the course of
4740execution; in other words, it consists entirely of @dfn{pure} (read-only)
4741code. Reentrancy is important whenever asynchronous execution is possible;
9d9b8b70
PE
4742for example, a nonreentrant program may not be safe to call from a signal
4743handler. In systems with multiple threads of control, a nonreentrant
bfa74976
RS
4744program must be called only within interlocks.
4745
70811b85 4746Normally, Bison generates a parser which is not reentrant. This is
c827f760
PE
4747suitable for most uses, and it permits compatibility with Yacc. (The
4748standard Yacc interfaces are inherently nonreentrant, because they use
70811b85
RS
4749statically allocated variables for communication with @code{yylex},
4750including @code{yylval} and @code{yylloc}.)
bfa74976 4751
70811b85 4752Alternatively, you can generate a pure, reentrant parser. The Bison
67501061 4753declaration @samp{%define api.pure} says that you want the parser to be
70811b85 4754reentrant. It looks like this:
bfa74976
RS
4755
4756@example
d9df47b6 4757%define api.pure
bfa74976
RS
4758@end example
4759
70811b85
RS
4760The result is that the communication variables @code{yylval} and
4761@code{yylloc} become local variables in @code{yyparse}, and a different
4762calling convention is used for the lexical analyzer function
4763@code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
f4101aa6
AD
4764Parsers}, for the details of this. The variable @code{yynerrs}
4765becomes local in @code{yyparse} in pull mode but it becomes a member
9987d1b3 4766of yypstate in push mode. (@pxref{Error Reporting, ,The Error
70811b85
RS
4767Reporting Function @code{yyerror}}). The convention for calling
4768@code{yyparse} itself is unchanged.
4769
4770Whether the parser is pure has nothing to do with the grammar rules.
4771You can generate either a pure parser or a nonreentrant parser from any
4772valid grammar.
bfa74976 4773
9987d1b3
JD
4774@node Push Decl
4775@subsection A Push Parser
4776@cindex push parser
4777@cindex push parser
67212941 4778@findex %define api.push-pull
9987d1b3 4779
59da312b
JD
4780(The current push parsing interface is experimental and may evolve.
4781More user feedback will help to stabilize it.)
4782
f4101aa6
AD
4783A pull parser is called once and it takes control until all its input
4784is completely parsed. A push parser, on the other hand, is called
9987d1b3
JD
4785each time a new token is made available.
4786
f4101aa6 4787A push parser is typically useful when the parser is part of a
9987d1b3 4788main event loop in the client's application. This is typically
f4101aa6
AD
4789a requirement of a GUI, when the main event loop needs to be triggered
4790within a certain time period.
9987d1b3 4791
d782395d
JD
4792Normally, Bison generates a pull parser.
4793The following Bison declaration says that you want the parser to be a push
35c1e5f0 4794parser (@pxref{%define Summary,,api.push-pull}):
9987d1b3
JD
4795
4796@example
cf499cff 4797%define api.push-pull push
9987d1b3
JD
4798@end example
4799
4800In almost all cases, you want to ensure that your push parser is also
4801a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). The only
f4101aa6 4802time you should create an impure push parser is to have backwards
9987d1b3
JD
4803compatibility with the impure Yacc pull mode interface. Unless you know
4804what you are doing, your declarations should look like this:
4805
4806@example
d9df47b6 4807%define api.pure
cf499cff 4808%define api.push-pull push
9987d1b3
JD
4809@end example
4810
f4101aa6
AD
4811There is a major notable functional difference between the pure push parser
4812and the impure push parser. It is acceptable for a pure push parser to have
9987d1b3
JD
4813many parser instances, of the same type of parser, in memory at the same time.
4814An impure push parser should only use one parser at a time.
4815
4816When a push parser is selected, Bison will generate some new symbols in
f4101aa6
AD
4817the generated parser. @code{yypstate} is a structure that the generated
4818parser uses to store the parser's state. @code{yypstate_new} is the
9987d1b3
JD
4819function that will create a new parser instance. @code{yypstate_delete}
4820will free the resources associated with the corresponding parser instance.
f4101aa6 4821Finally, @code{yypush_parse} is the function that should be called whenever a
9987d1b3
JD
4822token is available to provide the parser. A trivial example
4823of using a pure push parser would look like this:
4824
4825@example
4826int status;
4827yypstate *ps = yypstate_new ();
4828do @{
4829 status = yypush_parse (ps, yylex (), NULL);
4830@} while (status == YYPUSH_MORE);
4831yypstate_delete (ps);
4832@end example
4833
4834If the user decided to use an impure push parser, a few things about
f4101aa6 4835the generated parser will change. The @code{yychar} variable becomes
9987d1b3
JD
4836a global variable instead of a variable in the @code{yypush_parse} function.
4837For this reason, the signature of the @code{yypush_parse} function is
f4101aa6 4838changed to remove the token as a parameter. A nonreentrant push parser
9987d1b3
JD
4839example would thus look like this:
4840
4841@example
4842extern int yychar;
4843int status;
4844yypstate *ps = yypstate_new ();
4845do @{
4846 yychar = yylex ();
4847 status = yypush_parse (ps);
4848@} while (status == YYPUSH_MORE);
4849yypstate_delete (ps);
4850@end example
4851
f4101aa6 4852That's it. Notice the next token is put into the global variable @code{yychar}
9987d1b3
JD
4853for use by the next invocation of the @code{yypush_parse} function.
4854
f4101aa6 4855Bison also supports both the push parser interface along with the pull parser
9987d1b3 4856interface in the same generated parser. In order to get this functionality,
cf499cff
JD
4857you should replace the @samp{%define api.push-pull push} declaration with the
4858@samp{%define api.push-pull both} declaration. Doing this will create all of
c373bf8b 4859the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
f4101aa6
AD
4860and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
4861would be used. However, the user should note that it is implemented in the
d782395d
JD
4862generated parser by calling @code{yypull_parse}.
4863This makes the @code{yyparse} function that is generated with the
cf499cff 4864@samp{%define api.push-pull both} declaration slower than the normal
d782395d
JD
4865@code{yyparse} function. If the user
4866calls the @code{yypull_parse} function it will parse the rest of the input
f4101aa6
AD
4867stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
4868and then @code{yypull_parse} the rest of the input stream. If you would like
4869to switch back and forth between between parsing styles, you would have to
4870write your own @code{yypull_parse} function that knows when to quit looking
4871for input. An example of using the @code{yypull_parse} function would look
9987d1b3
JD
4872like this:
4873
4874@example
4875yypstate *ps = yypstate_new ();
4876yypull_parse (ps); /* Will call the lexer */
4877yypstate_delete (ps);
4878@end example
4879
67501061 4880Adding the @samp{%define api.pure} declaration does exactly the same thing to
cf499cff
JD
4881the generated parser with @samp{%define api.push-pull both} as it did for
4882@samp{%define api.push-pull push}.
9987d1b3 4883
342b8b6e 4884@node Decl Summary
bfa74976
RS
4885@subsection Bison Declaration Summary
4886@cindex Bison declaration summary
4887@cindex declaration summary
4888@cindex summary, Bison declaration
4889
d8988b2f 4890Here is a summary of the declarations used to define a grammar:
bfa74976 4891
18b519c0 4892@deffn {Directive} %union
bfa74976
RS
4893Declare the collection of data types that semantic values may have
4894(@pxref{Union Decl, ,The Collection of Value Types}).
18b519c0 4895@end deffn
bfa74976 4896
18b519c0 4897@deffn {Directive} %token
bfa74976
RS
4898Declare a terminal symbol (token type name) with no precedence
4899or associativity specified (@pxref{Token Decl, ,Token Type Names}).
18b519c0 4900@end deffn
bfa74976 4901
18b519c0 4902@deffn {Directive} %right
bfa74976
RS
4903Declare a terminal symbol (token type name) that is right-associative
4904(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4905@end deffn
bfa74976 4906
18b519c0 4907@deffn {Directive} %left
bfa74976
RS
4908Declare a terminal symbol (token type name) that is left-associative
4909(@pxref{Precedence Decl, ,Operator Precedence}).
18b519c0 4910@end deffn
bfa74976 4911
18b519c0 4912@deffn {Directive} %nonassoc
bfa74976 4913Declare a terminal symbol (token type name) that is nonassociative
bfa74976 4914(@pxref{Precedence Decl, ,Operator Precedence}).
39a06c25
PE
4915Using it in a way that would be associative is a syntax error.
4916@end deffn
4917
91d2c560 4918@ifset defaultprec
39a06c25 4919@deffn {Directive} %default-prec
22fccf95 4920Assign a precedence to rules lacking an explicit @code{%prec} modifier
39a06c25
PE
4921(@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4922@end deffn
91d2c560 4923@end ifset
bfa74976 4924
18b519c0 4925@deffn {Directive} %type
bfa74976
RS
4926Declare the type of semantic values for a nonterminal symbol
4927(@pxref{Type Decl, ,Nonterminal Symbols}).
18b519c0 4928@end deffn
bfa74976 4929
18b519c0 4930@deffn {Directive} %start
89cab50d
AD
4931Specify the grammar's start symbol (@pxref{Start Decl, ,The
4932Start-Symbol}).
18b519c0 4933@end deffn
bfa74976 4934
18b519c0 4935@deffn {Directive} %expect
bfa74976
RS
4936Declare the expected number of shift-reduce conflicts
4937(@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
18b519c0
AD
4938@end deffn
4939
bfa74976 4940
d8988b2f
AD
4941@sp 1
4942@noindent
4943In order to change the behavior of @command{bison}, use the following
4944directives:
4945
148d66d8 4946@deffn {Directive} %code @{@var{code}@}
e0c07222 4947@deffnx {Directive} %code @var{qualifier} @{@var{code}@}
148d66d8 4948@findex %code
e0c07222
JD
4949Insert @var{code} verbatim into the output parser source at the
4950default location or at the location specified by @var{qualifier}.
4951@xref{%code Summary}.
148d66d8
JD
4952@end deffn
4953
18b519c0 4954@deffn {Directive} %debug
fa819509
AD
4955Instrument the output parser for traces. Obsoleted by @samp{%define
4956parse.trace}.
ec3bc396 4957@xref{Tracing, ,Tracing Your Parser}.
f7dae1ea 4958@end deffn
d8988b2f 4959
35c1e5f0
JD
4960@deffn {Directive} %define @var{variable}
4961@deffnx {Directive} %define @var{variable} @var{value}
4962@deffnx {Directive} %define @var{variable} "@var{value}"
4963Define a variable to adjust Bison's behavior. @xref{%define Summary}.
4964@end deffn
4965
4966@deffn {Directive} %defines
4967Write a parser header file containing macro definitions for the token
4968type names defined in the grammar as well as a few other declarations.
4969If the parser implementation file is named @file{@var{name}.c} then
4970the parser header file is named @file{@var{name}.h}.
4971
4972For C parsers, the parser header file declares @code{YYSTYPE} unless
4973@code{YYSTYPE} is already defined as a macro or you have used a
4974@code{<@var{type}>} tag without using @code{%union}. Therefore, if
4975you are using a @code{%union} (@pxref{Multiple Types, ,More Than One
4976Value Type}) with components that require other definitions, or if you
4977have defined a @code{YYSTYPE} macro or type definition (@pxref{Value
4978Type, ,Data Types of Semantic Values}), you need to arrange for these
4979definitions to be propagated to all modules, e.g., by putting them in
4980a prerequisite header that is included both by your parser and by any
4981other module that needs @code{YYSTYPE}.
4982
4983Unless your parser is pure, the parser header file declares
4984@code{yylval} as an external variable. @xref{Pure Decl, ,A Pure
4985(Reentrant) Parser}.
4986
4987If you have also used locations, the parser header file declares
4988@code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
4989the @code{YYSTYPE} macro and @code{yylval}. @xref{Locations,
4990,Tracking Locations}.
4991
4992This parser header file is normally essential if you wish to put the
4993definition of @code{yylex} in a separate source file, because
4994@code{yylex} typically needs to be able to refer to the
4995above-mentioned declarations and to the token type codes. @xref{Token
4996Values, ,Semantic Values of Tokens}.
4997
4998@findex %code requires
4999@findex %code provides
5000If you have declared @code{%code requires} or @code{%code provides}, the output
5001header also contains their code.
5002@xref{%code Summary}.
5003@end deffn
5004
5005@deffn {Directive} %defines @var{defines-file}
5006Same as above, but save in the file @var{defines-file}.
5007@end deffn
5008
5009@deffn {Directive} %destructor
5010Specify how the parser should reclaim the memory associated to
5011discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
5012@end deffn
5013
5014@deffn {Directive} %file-prefix "@var{prefix}"
5015Specify a prefix to use for all Bison output file names. The names
5016are chosen as if the grammar file were named @file{@var{prefix}.y}.
5017@end deffn
5018
5019@deffn {Directive} %language "@var{language}"
5020Specify the programming language for the generated parser. Currently
5021supported languages include C, C++, and Java.
5022@var{language} is case-insensitive.
5023
5024This directive is experimental and its effect may be modified in future
5025releases.
5026@end deffn
5027
5028@deffn {Directive} %locations
5029Generate the code processing the locations (@pxref{Action Features,
5030,Special Features for Use in Actions}). This mode is enabled as soon as
5031the grammar uses the special @samp{@@@var{n}} tokens, but if your
5032grammar does not use it, using @samp{%locations} allows for more
5033accurate syntax error messages.
5034@end deffn
5035
5036@deffn {Directive} %name-prefix "@var{prefix}"
5037Rename the external symbols used in the parser so that they start with
5038@var{prefix} instead of @samp{yy}. The precise list of symbols renamed
5039in C parsers
5040is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
5041@code{yylval}, @code{yychar}, @code{yydebug}, and
5042(if locations are used) @code{yylloc}. If you use a push parser,
5043@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5044@code{yypstate_new} and @code{yypstate_delete} will
5045also be renamed. For example, if you use @samp{%name-prefix "c_"}, the
5046names become @code{c_parse}, @code{c_lex}, and so on.
5047For C++ parsers, see the @samp{%define api.namespace} documentation in this
5048section.
5049@xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
5050@end deffn
5051
5052@ifset defaultprec
5053@deffn {Directive} %no-default-prec
5054Do not assign a precedence to rules lacking an explicit @code{%prec}
5055modifier (@pxref{Contextual Precedence, ,Context-Dependent
5056Precedence}).
5057@end deffn
5058@end ifset
5059
5060@deffn {Directive} %no-lines
5061Don't generate any @code{#line} preprocessor commands in the parser
5062implementation file. Ordinarily Bison writes these commands in the
5063parser implementation file so that the C compiler and debuggers will
5064associate errors and object code with your source file (the grammar
5065file). This directive causes them to associate errors with the parser
5066implementation file, treating it as an independent source file in its
5067own right.
5068@end deffn
5069
5070@deffn {Directive} %output "@var{file}"
5071Specify @var{file} for the parser implementation file.
5072@end deffn
5073
5074@deffn {Directive} %pure-parser
5075Deprecated version of @samp{%define api.pure} (@pxref{%define
5076Summary,,api.pure}), for which Bison is more careful to warn about
5077unreasonable usage.
5078@end deffn
5079
5080@deffn {Directive} %require "@var{version}"
5081Require version @var{version} or higher of Bison. @xref{Require Decl, ,
5082Require a Version of Bison}.
5083@end deffn
5084
5085@deffn {Directive} %skeleton "@var{file}"
5086Specify the skeleton to use.
5087
5088@c You probably don't need this option unless you are developing Bison.
5089@c You should use @code{%language} if you want to specify the skeleton for a
5090@c different language, because it is clearer and because it will always choose the
5091@c correct skeleton for non-deterministic or push parsers.
5092
5093If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5094file in the Bison installation directory.
5095If it does, @var{file} is an absolute file name or a file name relative to the
5096directory of the grammar file.
5097This is similar to how most shells resolve commands.
5098@end deffn
5099
5100@deffn {Directive} %token-table
5101Generate an array of token names in the parser implementation file.
5102The name of the array is @code{yytname}; @code{yytname[@var{i}]} is
5103the name of the token whose internal Bison token code number is
5104@var{i}. The first three elements of @code{yytname} correspond to the
5105predefined tokens @code{"$end"}, @code{"error"}, and
5106@code{"$undefined"}; after these come the symbols defined in the
5107grammar file.
5108
5109The name in the table includes all the characters needed to represent
5110the token in Bison. For single-character literals and literal
5111strings, this includes the surrounding quoting characters and any
5112escape sequences. For example, the Bison single-character literal
5113@code{'+'} corresponds to a three-character name, represented in C as
5114@code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
5115corresponds to a five-character name, represented in C as
5116@code{"\"\\\\/\""}.
5117
5118When you specify @code{%token-table}, Bison also generates macro
5119definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
5120@code{YYNRULES}, and @code{YYNSTATES}:
5121
5122@table @code
5123@item YYNTOKENS
5124The highest token number, plus one.
5125@item YYNNTS
5126The number of nonterminal symbols.
5127@item YYNRULES
5128The number of grammar rules,
5129@item YYNSTATES
5130The number of parser states (@pxref{Parser States}).
5131@end table
5132@end deffn
5133
5134@deffn {Directive} %verbose
5135Write an extra output file containing verbose descriptions of the
5136parser states and what is done for each type of lookahead token in
5137that state. @xref{Understanding, , Understanding Your Parser}, for more
5138information.
5139@end deffn
5140
5141@deffn {Directive} %yacc
5142Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
5143including its naming conventions. @xref{Bison Options}, for more.
5144@end deffn
5145
5146
5147@node %define Summary
5148@subsection %define Summary
51151d91
JD
5149
5150There are many features of Bison's behavior that can be controlled by
5151assigning the feature a single value. For historical reasons, some
5152such features are assigned values by dedicated directives, such as
5153@code{%start}, which assigns the start symbol. However, newer such
5154features are associated with variables, which are assigned by the
5155@code{%define} directive:
5156
c1d19e10 5157@deffn {Directive} %define @var{variable}
cf499cff 5158@deffnx {Directive} %define @var{variable} @var{value}
c1d19e10 5159@deffnx {Directive} %define @var{variable} "@var{value}"
51151d91 5160Define @var{variable} to @var{value}.
9611cfa2 5161
51151d91
JD
5162@var{value} must be placed in quotation marks if it contains any
5163character other than a letter, underscore, period, or non-initial dash
5164or digit. Omitting @code{"@var{value}"} entirely is always equivalent
5165to specifying @code{""}.
9611cfa2 5166
51151d91
JD
5167It is an error if a @var{variable} is defined by @code{%define}
5168multiple times, but see @ref{Bison Options,,-D
5169@var{name}[=@var{value}]}.
5170@end deffn
cf499cff 5171
51151d91
JD
5172The rest of this section summarizes variables and values that
5173@code{%define} accepts.
9611cfa2 5174
51151d91
JD
5175Some @var{variable}s take Boolean values. In this case, Bison will
5176complain if the variable definition does not meet one of the following
5177four conditions:
9611cfa2
JD
5178
5179@enumerate
cf499cff 5180@item @code{@var{value}} is @code{true}
9611cfa2 5181
cf499cff
JD
5182@item @code{@var{value}} is omitted (or @code{""} is specified).
5183This is equivalent to @code{true}.
9611cfa2 5184
cf499cff 5185@item @code{@var{value}} is @code{false}.
9611cfa2
JD
5186
5187@item @var{variable} is never defined.
c6abeab1 5188In this case, Bison selects a default value.
9611cfa2 5189@end enumerate
148d66d8 5190
c6abeab1
JD
5191What @var{variable}s are accepted, as well as their meanings and default
5192values, depend on the selected target language and/or the parser
5193skeleton (@pxref{Decl Summary,,%language}, @pxref{Decl
5194Summary,,%skeleton}).
5195Unaccepted @var{variable}s produce an error.
793fbca5
JD
5196Some of the accepted @var{variable}s are:
5197
fa819509 5198@table @code
6b5a0de9 5199@c ================================================== api.namespace
67501061
AD
5200@item api.namespace
5201@findex %define api.namespace
5202@itemize
5203@item Languages(s): C++
5204
f1b238df 5205@item Purpose: Specify the namespace for the parser class.
67501061
AD
5206For example, if you specify:
5207
5208@smallexample
5209%define api.namespace "foo::bar"
5210@end smallexample
5211
5212Bison uses @code{foo::bar} verbatim in references such as:
5213
5214@smallexample
5215foo::bar::parser::semantic_type
5216@end smallexample
5217
5218However, to open a namespace, Bison removes any leading @code{::} and then
5219splits on any remaining occurrences:
5220
5221@smallexample
5222namespace foo @{ namespace bar @{
5223 class position;
5224 class location;
5225@} @}
5226@end smallexample
5227
5228@item Accepted Values:
5229Any absolute or relative C++ namespace reference without a trailing
5230@code{"::"}. For example, @code{"foo"} or @code{"::foo::bar"}.
5231
5232@item Default Value:
5233The value specified by @code{%name-prefix}, which defaults to @code{yy}.
5234This usage of @code{%name-prefix} is for backward compatibility and can
5235be confusing since @code{%name-prefix} also specifies the textual prefix
5236for the lexical analyzer function. Thus, if you specify
5237@code{%name-prefix}, it is best to also specify @samp{%define
5238api.namespace} so that @code{%name-prefix} @emph{only} affects the
5239lexical analyzer function. For example, if you specify:
5240
5241@smallexample
5242%define api.namespace "foo"
5243%name-prefix "bar::"
5244@end smallexample
5245
5246The parser namespace is @code{foo} and @code{yylex} is referenced as
5247@code{bar::lex}.
5248@end itemize
5249@c namespace
5250
5251
5252
5253@c ================================================== api.pure
d9df47b6
JD
5254@item api.pure
5255@findex %define api.pure
5256
5257@itemize @bullet
5258@item Language(s): C
5259
5260@item Purpose: Request a pure (reentrant) parser program.
5261@xref{Pure Decl, ,A Pure (Reentrant) Parser}.
5262
5263@item Accepted Values: Boolean
5264
cf499cff 5265@item Default Value: @code{false}
d9df47b6 5266@end itemize
71b00ed8 5267@c api.pure
d9df47b6 5268
67501061
AD
5269
5270
5271@c ================================================== api.push-pull
67212941
JD
5272@item api.push-pull
5273@findex %define api.push-pull
793fbca5
JD
5274
5275@itemize @bullet
eb45ef3b 5276@item Language(s): C (deterministic parsers only)
793fbca5 5277
f1b238df 5278@item Purpose: Request a pull parser, a push parser, or both.
d782395d 5279@xref{Push Decl, ,A Push Parser}.
59da312b
JD
5280(The current push parsing interface is experimental and may evolve.
5281More user feedback will help to stabilize it.)
793fbca5 5282
cf499cff 5283@item Accepted Values: @code{pull}, @code{push}, @code{both}
793fbca5 5284
cf499cff 5285@item Default Value: @code{pull}
793fbca5 5286@end itemize
67212941 5287@c api.push-pull
71b00ed8 5288
6b5a0de9
AD
5289
5290
5291@c ================================================== api.tokens.prefix
4c6622c2
AD
5292@item api.tokens.prefix
5293@findex %define api.tokens.prefix
5294
5295@itemize
5296@item Languages(s): all
5297
5298@item Purpose:
5299Add a prefix to the token names when generating their definition in the
5300target language. For instance
5301
5302@example
5303%token FILE for ERROR
5304%define api.tokens.prefix "TOK_"
5305%%
5306start: FILE for ERROR;
5307@end example
5308
5309@noindent
5310generates the definition of the symbols @code{TOK_FILE}, @code{TOK_for},
5311and @code{TOK_ERROR} in the generated source files. In particular, the
5312scanner must use these prefixed token names, while the grammar itself
5313may still use the short names (as in the sample rule given above). The
5314generated informational files (@file{*.output}, @file{*.xml},
5315@file{*.dot}) are not modified by this prefix. See @ref{Calc++ Parser}
5316and @ref{Calc++ Scanner}, for a complete example.
5317
5318@item Accepted Values:
5319Any string. Should be a valid identifier prefix in the target language,
5320in other words, it should typically be an identifier itself (sequence of
5321letters, underscores, and ---not at the beginning--- digits).
5322
5323@item Default Value:
5324empty
5325@end itemize
5326@c api.tokens.prefix
5327
5328
3cdc21cf 5329@c ================================================== lex_symbol
84072495 5330@item lex_symbol
3cdc21cf
AD
5331@findex %define lex_symbol
5332
5333@itemize @bullet
5334@item Language(s):
5335C++
5336
5337@item Purpose:
5338When variant-based semantic values are enabled (@pxref{C++ Variants}),
5339request that symbols be handled as a whole (type, value, and possibly
5340location) in the scanner. @xref{Complete Symbols}, for details.
5341
5342@item Accepted Values:
5343Boolean.
5344
5345@item Default Value:
5346@code{false}
5347@end itemize
5348@c lex_symbol
5349
5350
6b5a0de9
AD
5351@c ================================================== lr.default-reductions
5352
5bab9d08 5353@item lr.default-reductions
110ef36a 5354@cindex default reductions
5bab9d08 5355@findex %define lr.default-reductions
eb45ef3b
JD
5356@cindex delayed syntax errors
5357@cindex syntax errors delayed
8a4281b9 5358@cindex LAC
fcf834f9 5359@findex %nonassoc
eb45ef3b
JD
5360
5361@itemize @bullet
5362@item Language(s): all
5363
fcf834f9 5364@item Purpose: Specify the kind of states that are permitted to
110ef36a 5365contain default reductions.
fcf834f9
JD
5366That is, in such a state, Bison selects the reduction with the largest
5367lookahead set to be the default parser action and then removes that
110ef36a 5368lookahead set.
fcf834f9
JD
5369(The ability to specify where default reductions should be used is
5370experimental.
eb45ef3b
JD
5371More user feedback will help to stabilize it.)
5372
5373@item Accepted Values:
5374@itemize
cf499cff 5375@item @code{all}.
35c1e5f0
JD
5376This is the traditional Bison behavior. The main advantage is a
5377significant decrease in the size of the parser tables. The
5378disadvantage is that, when the generated parser encounters a
fcf834f9
JD
5379syntactically unacceptable token, the parser might then perform
5380unnecessary default reductions before it can detect the syntax error.
35c1e5f0
JD
5381Such delayed syntax error detection is usually inherent in LALR and
5382IELR parser tables anyway due to LR state merging (@pxref{%define
5383Summary,,lr.type}). Furthermore, the use of @code{%nonassoc} can
5384contribute to delayed syntax error detection even in the case of
5385canonical LR. As an experimental feature, delayed syntax error
5386detection can be overcome in all cases by enabling LAC (@pxref{%define
5387Summary,,parse.lac}, for details, including a discussion of the
5388effects of delayed syntax error detection).
eb45ef3b 5389
cf499cff 5390@item @code{consistent}.
eb45ef3b
JD
5391@cindex consistent states
5392A consistent state is a state that has only one possible action.
5393If that action is a reduction, then the parser does not need to request
5394a lookahead token from the scanner before performing that action.
fcf834f9
JD
5395However, the parser recognizes the ability to ignore the lookahead token
5396in this way only when such a reduction is encoded as a default
5397reduction.
5398Thus, if default reductions are permitted only in consistent states,
8a4281b9 5399then a canonical LR parser that does not employ
fcf834f9
JD
5400@code{%nonassoc} detects a syntax error as soon as it @emph{needs} the
5401syntactically unacceptable token from the scanner.
eb45ef3b 5402
cf499cff 5403@item @code{accepting}.
eb45ef3b 5404@cindex accepting state
fcf834f9
JD
5405In the accepting state, the default reduction is actually the accept
5406action.
8a4281b9 5407In this case, a canonical LR parser that does not employ
fcf834f9
JD
5408@code{%nonassoc} detects a syntax error as soon as it @emph{reaches} the
5409syntactically unacceptable token in the input.
5410That is, it does not perform any extra reductions.
eb45ef3b
JD
5411@end itemize
5412
5413@item Default Value:
5414@itemize
cf499cff
JD
5415@item @code{accepting} if @code{lr.type} is @code{canonical-lr}.
5416@item @code{all} otherwise.
eb45ef3b
JD
5417@end itemize
5418@end itemize
5419
6b5a0de9
AD
5420@c ============================================ lr.keep-unreachable-states
5421
67212941
JD
5422@item lr.keep-unreachable-states
5423@findex %define lr.keep-unreachable-states
31984206
JD
5424
5425@itemize @bullet
5426@item Language(s): all
5427
f1b238df
JD
5428@item Purpose: Request that Bison allow unreachable parser states to
5429remain in the parser tables.
31984206
JD
5430Bison considers a state to be unreachable if there exists no sequence of
5431transitions from the start state to that state.
5432A state can become unreachable during conflict resolution if Bison disables a
5433shift action leading to it from a predecessor state.
5434Keeping unreachable states is sometimes useful for analysis purposes, but they
5435are useless in the generated parser.
5436
5437@item Accepted Values: Boolean
5438
cf499cff 5439@item Default Value: @code{false}
31984206
JD
5440
5441@item Caveats:
5442
5443@itemize @bullet
cff03fb2
JD
5444
5445@item Unreachable states may contain conflicts and may use rules not used in
5446any other state.
31984206
JD
5447Thus, keeping unreachable states may induce warnings that are irrelevant to
5448your parser's behavior, and it may eliminate warnings that are relevant.
5449Of course, the change in warnings may actually be relevant to a parser table
5450analysis that wants to keep unreachable states, so this behavior will likely
5451remain in future Bison releases.
5452
5453@item While Bison is able to remove unreachable states, it is not guaranteed to
5454remove other kinds of useless states.
5455Specifically, when Bison disables reduce actions during conflict resolution,
5456some goto actions may become useless, and thus some additional states may
5457become useless.
5458If Bison were to compute which goto actions were useless and then disable those
5459actions, it could identify such states as unreachable and then remove those
5460states.
5461However, Bison does not compute which goto actions are useless.
5462@end itemize
5463@end itemize
67212941 5464@c lr.keep-unreachable-states
31984206 5465
6b5a0de9
AD
5466@c ================================================== lr.type
5467
eb45ef3b
JD
5468@item lr.type
5469@findex %define lr.type
8a4281b9
JD
5470@cindex LALR
5471@cindex IELR
5472@cindex LR
eb45ef3b
JD
5473
5474@itemize @bullet
5475@item Language(s): all
5476
f1b238df 5477@item Purpose: Specify the type of parser tables within the
8a4281b9 5478LR(1) family.
eb45ef3b
JD
5479(This feature is experimental.
5480More user feedback will help to stabilize it.)
5481
5482@item Accepted Values:
5483@itemize
cf499cff 5484@item @code{lalr}.
8a4281b9
JD
5485While Bison generates LALR parser tables by default for
5486historical reasons, IELR or canonical LR is almost
eb45ef3b 5487always preferable for deterministic parsers.
8a4281b9 5488The trouble is that LALR parser tables can suffer from
110ef36a 5489mysterious conflicts and thus may not accept the full set of sentences
8a4281b9 5490that IELR and canonical LR accept.
eb45ef3b 5491@xref{Mystery Conflicts}, for details.
8a4281b9 5492However, there are at least two scenarios where LALR may be
eb45ef3b
JD
5493worthwhile:
5494@itemize
8a4281b9
JD
5495@cindex GLR with LALR
5496@item When employing GLR parsers (@pxref{GLR Parsers}), if you
eb45ef3b
JD
5497do not resolve any conflicts statically (for example, with @code{%left}
5498or @code{%prec}), then the parser explores all potential parses of any
5499given input.
8a4281b9 5500In this case, the use of LALR parser tables is guaranteed not
110ef36a 5501to alter the language accepted by the parser.
8a4281b9 5502LALR parser tables are the smallest parser tables Bison can
eb45ef3b 5503currently generate, so they may be preferable.
f1b238df 5504Nevertheless, once you begin to resolve conflicts statically,
8a4281b9
JD
5505GLR begins to behave more like a deterministic parser, and so
5506IELR and canonical LR can be helpful to avoid
5507LALR's mysterious behavior.
eb45ef3b
JD
5508
5509@item Occasionally during development, an especially malformed grammar
8a4281b9
JD
5510with a major recurring flaw may severely impede the IELR or
5511canonical LR parser table generation algorithm.
5512LALR can be a quick way to generate parser tables in order to
eb45ef3b 5513investigate such problems while ignoring the more subtle differences
8a4281b9 5514from IELR and canonical LR.
eb45ef3b
JD
5515@end itemize
5516
cf499cff 5517@item @code{ielr}.
8a4281b9
JD
5518IELR is a minimal LR algorithm.
5519That is, given any grammar (LR or non-LR),
5520IELR and canonical LR always accept exactly the same
eb45ef3b 5521set of sentences.
8a4281b9
JD
5522However, as for LALR, the number of parser states is often an
5523order of magnitude less for IELR than for canonical
5524LR.
5525More importantly, because canonical LR's extra parser states
5526may contain duplicate conflicts in the case of non-LR
5527grammars, the number of conflicts for IELR is often an order
eb45ef3b
JD
5528of magnitude less as well.
5529This can significantly reduce the complexity of developing of a grammar.
5530
cf499cff 5531@item @code{canonical-lr}.
eb45ef3b
JD
5532@cindex delayed syntax errors
5533@cindex syntax errors delayed
8a4281b9 5534@cindex LAC
fcf834f9 5535@findex %nonassoc
35c1e5f0
JD
5536While inefficient, canonical LR parser tables can be an interesting
5537means to explore a grammar because they have a property that IELR and
5538LALR tables do not. That is, if @code{%nonassoc} is not used and
5539default reductions are left disabled (@pxref{%define
5540Summary,,lr.default-reductions}), then, for every left context of
5541every canonical LR state, the set of tokens accepted by that state is
5542guaranteed to be the exact set of tokens that is syntactically
5543acceptable in that left context. It might then seem that an advantage
5544of canonical LR parsers in production is that, under the above
5545constraints, they are guaranteed to detect a syntax error as soon as
5546possible without performing any unnecessary reductions. However, IELR
5547parsers using LAC (@pxref{%define Summary,,parse.lac}) are also able
5548to achieve this behavior without sacrificing @code{%nonassoc} or
5549default reductions.
eb45ef3b
JD
5550@end itemize
5551
cf499cff 5552@item Default Value: @code{lalr}
eb45ef3b
JD
5553@end itemize
5554
67501061
AD
5555
5556@c ================================================== namespace
793fbca5
JD
5557@item namespace
5558@findex %define namespace
67501061 5559Obsoleted by @code{api.namespace}
fa819509
AD
5560@c namespace
5561
31b850d2
AD
5562
5563@c ================================================== parse.assert
0c90a1f5
AD
5564@item parse.assert
5565@findex %define parse.assert
5566
5567@itemize
5568@item Languages(s): C++
5569
5570@item Purpose: Issue runtime assertions to catch invalid uses.
3cdc21cf
AD
5571In C++, when variants are used (@pxref{C++ Variants}), symbols must be
5572constructed and
0c90a1f5
AD
5573destroyed properly. This option checks these constraints.
5574
5575@item Accepted Values: Boolean
5576
5577@item Default Value: @code{false}
5578@end itemize
5579@c parse.assert
5580
31b850d2
AD
5581
5582@c ================================================== parse.error
5583@item parse.error
5584@findex %define parse.error
5585@itemize
5586@item Languages(s):
fcf834f9 5587all
31b850d2
AD
5588@item Purpose:
5589Control the kind of error messages passed to the error reporting
5590function. @xref{Error Reporting, ,The Error Reporting Function
5591@code{yyerror}}.
5592@item Accepted Values:
5593@itemize
cf499cff 5594@item @code{simple}
31b850d2
AD
5595Error messages passed to @code{yyerror} are simply @w{@code{"syntax
5596error"}}.
cf499cff 5597@item @code{verbose}
31b850d2
AD
5598Error messages report the unexpected token, and possibly the expected
5599ones.
5600@end itemize
5601
5602@item Default Value:
5603@code{simple}
5604@end itemize
5605@c parse.error
5606
5607
fcf834f9
JD
5608@c ================================================== parse.lac
5609@item parse.lac
5610@findex %define parse.lac
8a4281b9 5611@cindex LAC
fcf834f9
JD
5612@cindex lookahead correction
5613
5614@itemize
5615@item Languages(s): C
5616
8a4281b9 5617@item Purpose: Enable LAC (lookahead correction) to improve
fcf834f9
JD
5618syntax error handling.
5619
8a4281b9 5620Canonical LR, IELR, and LALR can suffer
fcf834f9
JD
5621from a couple of problems upon encountering a syntax error. First, the
5622parser might perform additional parser stack reductions before
5623discovering the syntax error. Such reductions perform user semantic
5624actions that are unexpected because they are based on an invalid token,
5625and they cause error recovery to begin in a different syntactic context
5626than the one in which the invalid token was encountered. Second, when
5627verbose error messages are enabled (with @code{%error-verbose} or
5628@code{#define YYERROR_VERBOSE}), the expected token list in the syntax
5629error message can both contain invalid tokens and omit valid tokens.
5630
5631The culprits for the above problems are @code{%nonassoc}, default
5632reductions in inconsistent states, and parser state merging. Thus,
8a4281b9
JD
5633IELR and LALR suffer the most. Canonical
5634LR can suffer only if @code{%nonassoc} is used or if default
fcf834f9
JD
5635reductions are enabled for inconsistent states.
5636
8a4281b9
JD
5637LAC is a new mechanism within the parsing algorithm that
5638completely solves these problems for canonical LR,
5639IELR, and LALR without sacrificing @code{%nonassoc},
fcf834f9
JD
5640default reductions, or state mering. Conceptually, the mechanism is
5641straight-forward. Whenever the parser fetches a new token from the
5642scanner so that it can determine the next parser action, it immediately
5643suspends normal parsing and performs an exploratory parse using a
5644temporary copy of the normal parser state stack. During this
5645exploratory parse, the parser does not perform user semantic actions.
5646If the exploratory parse reaches a shift action, normal parsing then
5647resumes on the normal parser stacks. If the exploratory parse reaches
5648an error instead, the parser reports a syntax error. If verbose syntax
5649error messages are enabled, the parser must then discover the list of
5650expected tokens, so it performs a separate exploratory parse for each
5651token in the grammar.
5652
35c1e5f0
JD
5653There is one subtlety about the use of LAC. That is, when in a
5654consistent parser state with a default reduction, the parser will not
5655attempt to fetch a token from the scanner because no lookahead is
5656needed to determine the next parser action. Thus, whether default
5657reductions are enabled in consistent states (@pxref{%define
fcf834f9
JD
5658Summary,,lr.default-reductions}) affects how soon the parser detects a
5659syntax error: when it @emph{reaches} an erroneous token or when it
35c1e5f0
JD
5660eventually @emph{needs} that token as a lookahead. The latter
5661behavior is probably more intuitive, so Bison currently provides no
5662way to achieve the former behavior while default reductions are fully
5663enabled.
fcf834f9 5664
8a4281b9 5665Thus, when LAC is in use, for some fixed decision of whether
fcf834f9 5666to enable default reductions in consistent states, canonical
8a4281b9 5667LR and IELR behave exactly the same for both
fcf834f9 5668syntactically acceptable and syntactically unacceptable input. While
8a4281b9
JD
5669LALR still does not support the full language-recognition
5670power of canonical LR and IELR, LAC at
5671least enables LALR's syntax error handling to correctly
5672reflect LALR's language-recognition power.
fcf834f9 5673
8a4281b9 5674Because LAC requires many parse actions to be performed twice,
fcf834f9
JD
5675it can have a performance penalty. However, not all parse actions must
5676be performed twice. Specifically, during a series of default reductions
5677in consistent states and shift actions, the parser never has to initiate
5678an exploratory parse. Moreover, the most time-consuming tasks in a
5679parse are often the file I/O, the lexical analysis performed by the
5680scanner, and the user's semantic actions, but none of these are
5681performed during the exploratory parse. Finally, the base of the
5682temporary stack used during an exploratory parse is a pointer into the
5683normal parser state stack so that the stack is never physically copied.
8a4281b9 5684In our experience, the performance penalty of LAC has proven
fcf834f9
JD
5685insignificant for practical grammars.
5686
5687@item Accepted Values: @code{none}, @code{full}
5688
5689@item Default Value: @code{none}
5690@end itemize
5691@c parse.lac
5692
31b850d2 5693@c ================================================== parse.trace
fa819509
AD
5694@item parse.trace
5695@findex %define parse.trace
5696
5697@itemize
5698@item Languages(s): C, C++
5699
5700@item Purpose: Require parser instrumentation for tracing.
ff7571c0
JD
5701In C/C++, define the macro @code{YYDEBUG} to 1 in the parser implementation
5702file if it is not already defined, so that the debugging facilities are
5703compiled. @xref{Tracing, ,Tracing Your Parser}.
793fbca5 5704
fa819509
AD
5705@item Accepted Values: Boolean
5706
5707@item Default Value: @code{false}
5708@end itemize
fa819509 5709@c parse.trace
99c08fb6 5710
3cdc21cf
AD
5711@c ================================================== variant
5712@item variant
5713@findex %define variant
5714
5715@itemize @bullet
5716@item Language(s):
5717C++
5718
5719@item Purpose:
f1b238df 5720Request variant-based semantic values.
3cdc21cf
AD
5721@xref{C++ Variants}.
5722
5723@item Accepted Values:
5724Boolean.
5725
5726@item Default Value:
5727@code{false}
5728@end itemize
5729@c variant
99c08fb6 5730@end table
592d0b1e 5731
d8988b2f 5732
e0c07222
JD
5733@node %code Summary
5734@subsection %code Summary
e0c07222 5735@findex %code
e0c07222 5736@cindex Prologue
51151d91
JD
5737
5738The @code{%code} directive inserts code verbatim into the output
5739parser source at any of a predefined set of locations. It thus serves
5740as a flexible and user-friendly alternative to the traditional Yacc
5741prologue, @code{%@{@var{code}%@}}. This section summarizes the
5742functionality of @code{%code} for the various target languages
5743supported by Bison. For a detailed discussion of how to use
5744@code{%code} in place of @code{%@{@var{code}%@}} for C/C++ and why it
5745is advantageous to do so, @pxref{Prologue Alternatives}.
5746
5747@deffn {Directive} %code @{@var{code}@}
5748This is the unqualified form of the @code{%code} directive. It
5749inserts @var{code} verbatim at a language-dependent default location
5750in the parser implementation.
5751
e0c07222 5752For C/C++, the default location is the parser implementation file
51151d91
JD
5753after the usual contents of the parser header file. Thus, the
5754unqualified form replaces @code{%@{@var{code}%@}} for most purposes.
e0c07222
JD
5755
5756For Java, the default location is inside the parser class.
5757@end deffn
5758
5759@deffn {Directive} %code @var{qualifier} @{@var{code}@}
5760This is the qualified form of the @code{%code} directive.
51151d91
JD
5761@var{qualifier} identifies the purpose of @var{code} and thus the
5762location(s) where Bison should insert it. That is, if you need to
5763specify location-sensitive @var{code} that does not belong at the
5764default location selected by the unqualified @code{%code} form, use
5765this form instead.
5766@end deffn
5767
5768For any particular qualifier or for the unqualified form, if there are
5769multiple occurrences of the @code{%code} directive, Bison concatenates
5770the specified code in the order in which it appears in the grammar
5771file.
e0c07222 5772
51151d91
JD
5773Not all qualifiers are accepted for all target languages. Unaccepted
5774qualifiers produce an error. Some of the accepted qualifiers are:
e0c07222 5775
84072495 5776@table @code
e0c07222
JD
5777@item requires
5778@findex %code requires
5779
5780@itemize @bullet
5781@item Language(s): C, C++
5782
5783@item Purpose: This is the best place to write dependency code required for
5784@code{YYSTYPE} and @code{YYLTYPE}.
5785In other words, it's the best place to define types referenced in @code{%union}
5786directives, and it's the best place to override Bison's default @code{YYSTYPE}
5787and @code{YYLTYPE} definitions.
5788
5789@item Location(s): The parser header file and the parser implementation file
5790before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
5791definitions.
5792@end itemize
5793
5794@item provides
5795@findex %code provides
5796
5797@itemize @bullet
5798@item Language(s): C, C++
5799
5800@item Purpose: This is the best place to write additional definitions and
5801declarations that should be provided to other modules.
5802
5803@item Location(s): The parser header file and the parser implementation
5804file after the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and
5805token definitions.
5806@end itemize
5807
5808@item top
5809@findex %code top
5810
5811@itemize @bullet
5812@item Language(s): C, C++
5813
5814@item Purpose: The unqualified @code{%code} or @code{%code requires}
5815should usually be more appropriate than @code{%code top}. However,
5816occasionally it is necessary to insert code much nearer the top of the
5817parser implementation file. For example:
5818
5819@smallexample
5820%code top @{
5821 #define _GNU_SOURCE
5822 #include <stdio.h>
5823@}
5824@end smallexample
5825
5826@item Location(s): Near the top of the parser implementation file.
5827@end itemize
5828
5829@item imports
5830@findex %code imports
5831
5832@itemize @bullet
5833@item Language(s): Java
5834
5835@item Purpose: This is the best place to write Java import directives.
5836
5837@item Location(s): The parser Java file after any Java package directive and
5838before any class definitions.
5839@end itemize
84072495 5840@end table
e0c07222 5841
51151d91
JD
5842Though we say the insertion locations are language-dependent, they are
5843technically skeleton-dependent. Writers of non-standard skeletons
5844however should choose their locations consistently with the behavior
5845of the standard Bison skeletons.
e0c07222 5846
d8988b2f 5847
342b8b6e 5848@node Multiple Parsers
bfa74976
RS
5849@section Multiple Parsers in the Same Program
5850
5851Most programs that use Bison parse only one language and therefore contain
5852only one Bison parser. But what if you want to parse more than one
5853language with the same program? Then you need to avoid a name conflict
5854between different definitions of @code{yyparse}, @code{yylval}, and so on.
5855
5856The easy way to do this is to use the option @samp{-p @var{prefix}}
704a47c4
AD
5857(@pxref{Invocation, ,Invoking Bison}). This renames the interface
5858functions and variables of the Bison parser to start with @var{prefix}
5859instead of @samp{yy}. You can use this to give each parser distinct
5860names that do not conflict.
bfa74976
RS
5861
5862The precise list of symbols renamed is @code{yyparse}, @code{yylex},
2a8d363a 5863@code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
f4101aa6
AD
5864@code{yychar} and @code{yydebug}. If you use a push parser,
5865@code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
9987d1b3 5866@code{yypstate_new} and @code{yypstate_delete} will also be renamed.
f4101aa6 5867For example, if you use @samp{-p c}, the names become @code{cparse},
9987d1b3 5868@code{clex}, and so on.
bfa74976
RS
5869
5870@strong{All the other variables and macros associated with Bison are not
5871renamed.} These others are not global; there is no conflict if the same
5872name is used in different parsers. For example, @code{YYSTYPE} is not
5873renamed, but defining this in different ways in different parsers causes
5874no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
5875
ff7571c0
JD
5876The @samp{-p} option works by adding macro definitions to the
5877beginning of the parser implementation file, defining @code{yyparse}
5878as @code{@var{prefix}parse}, and so on. This effectively substitutes
5879one name for the other in the entire parser implementation file.
bfa74976 5880
342b8b6e 5881@node Interface
bfa74976
RS
5882@chapter Parser C-Language Interface
5883@cindex C-language interface
5884@cindex interface
5885
5886The Bison parser is actually a C function named @code{yyparse}. Here we
5887describe the interface conventions of @code{yyparse} and the other
5888functions that it needs to use.
5889
5890Keep in mind that the parser uses many C identifiers starting with
5891@samp{yy} and @samp{YY} for internal purposes. If you use such an
75f5aaea
MA
5892identifier (aside from those in this manual) in an action or in epilogue
5893in the grammar file, you are likely to run into trouble.
bfa74976
RS
5894
5895@menu
f5f419de
DJ
5896* Parser Function:: How to call @code{yyparse} and what it returns.
5897* Push Parser Function:: How to call @code{yypush_parse} and what it returns.
5898* Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
5899* Parser Create Function:: How to call @code{yypstate_new} and what it returns.
5900* Parser Delete Function:: How to call @code{yypstate_delete} and what it returns.
5901* Lexical:: You must supply a function @code{yylex}
5902 which reads tokens.
5903* Error Reporting:: You must supply a function @code{yyerror}.
5904* Action Features:: Special features for use in actions.
5905* Internationalization:: How to let the parser speak in the user's
5906 native language.
bfa74976
RS
5907@end menu
5908
342b8b6e 5909@node Parser Function
bfa74976
RS
5910@section The Parser Function @code{yyparse}
5911@findex yyparse
5912
5913You call the function @code{yyparse} to cause parsing to occur. This
5914function reads tokens, executes actions, and ultimately returns when it
5915encounters end-of-input or an unrecoverable syntax error. You can also
14ded682
AD
5916write an action which directs @code{yyparse} to return immediately
5917without reading further.
bfa74976 5918
2a8d363a
AD
5919
5920@deftypefun int yyparse (void)
bfa74976
RS
5921The value returned by @code{yyparse} is 0 if parsing was successful (return
5922is due to end-of-input).
5923
b47dbebe
PE
5924The value is 1 if parsing failed because of invalid input, i.e., input
5925that contains a syntax error or that causes @code{YYABORT} to be
5926invoked.
5927
5928The value is 2 if parsing failed due to memory exhaustion.
2a8d363a 5929@end deftypefun
bfa74976
RS
5930
5931In an action, you can cause immediate return from @code{yyparse} by using
5932these macros:
5933
2a8d363a 5934@defmac YYACCEPT
bfa74976
RS
5935@findex YYACCEPT
5936Return immediately with value 0 (to report success).
2a8d363a 5937@end defmac
bfa74976 5938
2a8d363a 5939@defmac YYABORT
bfa74976
RS
5940@findex YYABORT
5941Return immediately with value 1 (to report failure).
2a8d363a
AD
5942@end defmac
5943
5944If you use a reentrant parser, you can optionally pass additional
5945parameter information to it in a reentrant way. To do so, use the
5946declaration @code{%parse-param}:
5947
2055a44e 5948@deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
2a8d363a 5949@findex %parse-param
2055a44e
AD
5950Declare that one or more
5951@var{argument-declaration} are additional @code{yyparse} arguments.
94175978 5952The @var{argument-declaration} is used when declaring
feeb0eda
PE
5953functions or prototypes. The last identifier in
5954@var{argument-declaration} must be the argument name.
2a8d363a
AD
5955@end deffn
5956
5957Here's an example. Write this in the parser:
5958
5959@example
2055a44e 5960%parse-param @{int *nastiness@} @{int *randomness@}
2a8d363a
AD
5961@end example
5962
5963@noindent
5964Then call the parser like this:
5965
5966@example
5967@{
5968 int nastiness, randomness;
5969 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
5970 value = yyparse (&nastiness, &randomness);
5971 @dots{}
5972@}
5973@end example
5974
5975@noindent
5976In the grammar actions, use expressions like this to refer to the data:
5977
5978@example
5979exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
5980@end example
5981
9987d1b3
JD
5982@node Push Parser Function
5983@section The Push Parser Function @code{yypush_parse}
5984@findex yypush_parse
5985
59da312b
JD
5986(The current push parsing interface is experimental and may evolve.
5987More user feedback will help to stabilize it.)
5988
f4101aa6 5989You call the function @code{yypush_parse} to parse a single token. This
cf499cff
JD
5990function is available if either the @samp{%define api.push-pull push} or
5991@samp{%define api.push-pull both} declaration is used.
9987d1b3
JD
5992@xref{Push Decl, ,A Push Parser}.
5993
5994@deftypefun int yypush_parse (yypstate *yyps)
f4101aa6 5995The value returned by @code{yypush_parse} is the same as for yyparse with the
9987d1b3
JD
5996following exception. @code{yypush_parse} will return YYPUSH_MORE if more input
5997is required to finish parsing the grammar.
5998@end deftypefun
5999
6000@node Pull Parser Function
6001@section The Pull Parser Function @code{yypull_parse}
6002@findex yypull_parse
6003
59da312b
JD
6004(The current push parsing interface is experimental and may evolve.
6005More user feedback will help to stabilize it.)
6006
f4101aa6 6007You call the function @code{yypull_parse} to parse the rest of the input
cf499cff 6008stream. This function is available if the @samp{%define api.push-pull both}
f4101aa6 6009declaration is used.
9987d1b3
JD
6010@xref{Push Decl, ,A Push Parser}.
6011
6012@deftypefun int yypull_parse (yypstate *yyps)
6013The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
6014@end deftypefun
6015
6016@node Parser Create Function
6017@section The Parser Create Function @code{yystate_new}
6018@findex yypstate_new
6019
59da312b
JD
6020(The current push parsing interface is experimental and may evolve.
6021More user feedback will help to stabilize it.)
6022
f4101aa6 6023You call the function @code{yypstate_new} to create a new parser instance.
cf499cff
JD
6024This function is available if either the @samp{%define api.push-pull push} or
6025@samp{%define api.push-pull both} declaration is used.
9987d1b3
JD
6026@xref{Push Decl, ,A Push Parser}.
6027
6028@deftypefun yypstate *yypstate_new (void)
f50bfcd6 6029The function will return a valid parser instance if there was memory available
333e670c
JD
6030or 0 if no memory was available.
6031In impure mode, it will also return 0 if a parser instance is currently
6032allocated.
9987d1b3
JD
6033@end deftypefun
6034
6035@node Parser Delete Function
6036@section The Parser Delete Function @code{yystate_delete}
6037@findex yypstate_delete
6038
59da312b
JD
6039(The current push parsing interface is experimental and may evolve.
6040More user feedback will help to stabilize it.)
6041
9987d1b3 6042You call the function @code{yypstate_delete} to delete a parser instance.
cf499cff
JD
6043function is available if either the @samp{%define api.push-pull push} or
6044@samp{%define api.push-pull both} declaration is used.
9987d1b3
JD
6045@xref{Push Decl, ,A Push Parser}.
6046
6047@deftypefun void yypstate_delete (yypstate *yyps)
6048This function will reclaim the memory associated with a parser instance.
6049After this call, you should no longer attempt to use the parser instance.
6050@end deftypefun
bfa74976 6051
342b8b6e 6052@node Lexical
bfa74976
RS
6053@section The Lexical Analyzer Function @code{yylex}
6054@findex yylex
6055@cindex lexical analyzer
6056
6057The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
6058the input stream and returns them to the parser. Bison does not create
6059this function automatically; you must write it so that @code{yyparse} can
6060call it. The function is sometimes referred to as a lexical scanner.
6061
ff7571c0
JD
6062In simple programs, @code{yylex} is often defined at the end of the
6063Bison grammar file. If @code{yylex} is defined in a separate source
6064file, you need to arrange for the token-type macro definitions to be
6065available there. To do this, use the @samp{-d} option when you run
6066Bison, so that it will write these macro definitions into the separate
6067parser header file, @file{@var{name}.tab.h}, which you can include in
6068the other source files that need it. @xref{Invocation, ,Invoking
6069Bison}.
bfa74976
RS
6070
6071@menu
6072* Calling Convention:: How @code{yyparse} calls @code{yylex}.
f5f419de
DJ
6073* Token Values:: How @code{yylex} must return the semantic value
6074 of the token it has read.
6075* Token Locations:: How @code{yylex} must return the text location
6076 (line number, etc.) of the token, if the
6077 actions want that.
6078* Pure Calling:: How the calling convention differs in a pure parser
6079 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
bfa74976
RS
6080@end menu
6081
342b8b6e 6082@node Calling Convention
bfa74976
RS
6083@subsection Calling Convention for @code{yylex}
6084
72d2299c
PE
6085The value that @code{yylex} returns must be the positive numeric code
6086for the type of token it has just found; a zero or negative value
6087signifies end-of-input.
bfa74976
RS
6088
6089When a token is referred to in the grammar rules by a name, that name
ff7571c0
JD
6090in the parser implementation file becomes a C macro whose definition
6091is the proper numeric code for that token type. So @code{yylex} can
6092use the name to indicate that type. @xref{Symbols}.
bfa74976
RS
6093
6094When a token is referred to in the grammar rules by a character literal,
6095the numeric code for that character is also the code for the token type.
72d2299c
PE
6096So @code{yylex} can simply return that character code, possibly converted
6097to @code{unsigned char} to avoid sign-extension. The null character
6098must not be used this way, because its code is zero and that
bfa74976
RS
6099signifies end-of-input.
6100
6101Here is an example showing these things:
6102
6103@example
13863333
AD
6104int
6105yylex (void)
bfa74976
RS
6106@{
6107 @dots{}
72d2299c 6108 if (c == EOF) /* Detect end-of-input. */
bfa74976
RS
6109 return 0;
6110 @dots{}
6111 if (c == '+' || c == '-')
72d2299c 6112 return c; /* Assume token type for `+' is '+'. */
bfa74976 6113 @dots{}
72d2299c 6114 return INT; /* Return the type of the token. */
bfa74976
RS
6115 @dots{}
6116@}
6117@end example
6118
6119@noindent
6120This interface has been designed so that the output from the @code{lex}
6121utility can be used without change as the definition of @code{yylex}.
6122
931c7513
RS
6123If the grammar uses literal string tokens, there are two ways that
6124@code{yylex} can determine the token type codes for them:
6125
6126@itemize @bullet
6127@item
6128If the grammar defines symbolic token names as aliases for the
6129literal string tokens, @code{yylex} can use these symbolic names like
6130all others. In this case, the use of the literal string tokens in
6131the grammar file has no effect on @code{yylex}.
6132
6133@item
9ecbd125 6134@code{yylex} can find the multicharacter token in the @code{yytname}
931c7513 6135table. The index of the token in the table is the token type's code.
9ecbd125 6136The name of a multicharacter token is recorded in @code{yytname} with a
931c7513 6137double-quote, the token's characters, and another double-quote. The
9e0876fb
PE
6138token's characters are escaped as necessary to be suitable as input
6139to Bison.
931c7513 6140
9e0876fb
PE
6141Here's code for looking up a multicharacter token in @code{yytname},
6142assuming that the characters of the token are stored in
6143@code{token_buffer}, and assuming that the token does not contain any
6144characters like @samp{"} that require escaping.
931c7513
RS
6145
6146@smallexample
6147for (i = 0; i < YYNTOKENS; i++)
6148 @{
6149 if (yytname[i] != 0
6150 && yytname[i][0] == '"'
68449b3a
PE
6151 && ! strncmp (yytname[i] + 1, token_buffer,
6152 strlen (token_buffer))
931c7513
RS
6153 && yytname[i][strlen (token_buffer) + 1] == '"'
6154 && yytname[i][strlen (token_buffer) + 2] == 0)
6155 break;
6156 @}
6157@end smallexample
6158
6159The @code{yytname} table is generated only if you use the
8c9a50be 6160@code{%token-table} declaration. @xref{Decl Summary}.
931c7513
RS
6161@end itemize
6162
342b8b6e 6163@node Token Values
bfa74976
RS
6164@subsection Semantic Values of Tokens
6165
6166@vindex yylval
9d9b8b70 6167In an ordinary (nonreentrant) parser, the semantic value of the token must
bfa74976
RS
6168be stored into the global variable @code{yylval}. When you are using
6169just one data type for semantic values, @code{yylval} has that type.
6170Thus, if the type is @code{int} (the default), you might write this in
6171@code{yylex}:
6172
6173@example
6174@group
6175 @dots{}
72d2299c
PE
6176 yylval = value; /* Put value onto Bison stack. */
6177 return INT; /* Return the type of the token. */
bfa74976
RS
6178 @dots{}
6179@end group
6180@end example
6181
6182When you are using multiple data types, @code{yylval}'s type is a union
704a47c4
AD
6183made from the @code{%union} declaration (@pxref{Union Decl, ,The
6184Collection of Value Types}). So when you store a token's value, you
6185must use the proper member of the union. If the @code{%union}
6186declaration looks like this:
bfa74976
RS
6187
6188@example
6189@group
6190%union @{
6191 int intval;
6192 double val;
6193 symrec *tptr;
6194@}
6195@end group
6196@end example
6197
6198@noindent
6199then the code in @code{yylex} might look like this:
6200
6201@example
6202@group
6203 @dots{}
72d2299c
PE
6204 yylval.intval = value; /* Put value onto Bison stack. */
6205 return INT; /* Return the type of the token. */
bfa74976
RS
6206 @dots{}
6207@end group
6208@end example
6209
95923bd6
AD
6210@node Token Locations
6211@subsection Textual Locations of Tokens
bfa74976
RS
6212
6213@vindex yylloc
847bf1f5 6214If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
f8e1c9e5
AD
6215Tracking Locations}) in actions to keep track of the textual locations
6216of tokens and groupings, then you must provide this information in
6217@code{yylex}. The function @code{yyparse} expects to find the textual
6218location of a token just parsed in the global variable @code{yylloc}.
6219So @code{yylex} must store the proper data in that variable.
847bf1f5
AD
6220
6221By default, the value of @code{yylloc} is a structure and you need only
89cab50d
AD
6222initialize the members that are going to be used by the actions. The
6223four members are called @code{first_line}, @code{first_column},
6224@code{last_line} and @code{last_column}. Note that the use of this
6225feature makes the parser noticeably slower.
bfa74976
RS
6226
6227@tindex YYLTYPE
6228The data type of @code{yylloc} has the name @code{YYLTYPE}.
6229
342b8b6e 6230@node Pure Calling
c656404a 6231@subsection Calling Conventions for Pure Parsers
bfa74976 6232
67501061 6233When you use the Bison declaration @samp{%define api.pure} to request a
e425e872
RS
6234pure, reentrant parser, the global communication variables @code{yylval}
6235and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
6236Parser}.) In such parsers the two global variables are replaced by
6237pointers passed as arguments to @code{yylex}. You must declare them as
6238shown here, and pass the information back by storing it through those
6239pointers.
bfa74976
RS
6240
6241@example
13863333
AD
6242int
6243yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
bfa74976
RS
6244@{
6245 @dots{}
6246 *lvalp = value; /* Put value onto Bison stack. */
6247 return INT; /* Return the type of the token. */
6248 @dots{}
6249@}
6250@end example
6251
6252If the grammar file does not use the @samp{@@} constructs to refer to
95923bd6 6253textual locations, then the type @code{YYLTYPE} will not be defined. In
bfa74976
RS
6254this case, omit the second argument; @code{yylex} will be called with
6255only one argument.
6256
2055a44e 6257If you wish to pass additional arguments to @code{yylex}, use
2a8d363a 6258@code{%lex-param} just like @code{%parse-param} (@pxref{Parser
2055a44e
AD
6259Function}). To pass additional arguments to both @code{yylex} and
6260@code{yyparse}, use @code{%param}.
e425e872 6261
2055a44e 6262@deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
2a8d363a 6263@findex %lex-param
2055a44e
AD
6264Specify that @var{argument-declaration} are additional @code{yylex} argument
6265declarations. You may pass one or more such declarations, which is
6266equivalent to repeating @code{%lex-param}.
6267@end deffn
6268
6269@deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
6270@findex %param
6271Specify that @var{argument-declaration} are additional
6272@code{yylex}/@code{yyparse} argument declaration. This is equivalent to
6273@samp{%lex-param @{@var{argument-declaration}@} @dots{} %parse-param
6274@{@var{argument-declaration}@} @dots{}}. You may pass one or more
6275declarations, which is equivalent to repeating @code{%param}.
2a8d363a 6276@end deffn
e425e872 6277
2a8d363a 6278For instance:
e425e872
RS
6279
6280@example
2055a44e
AD
6281%lex-param @{scanner_mode *mode@}
6282%parse-param @{parser_mode *mode@}
6283%param @{environment_type *env@}
e425e872
RS
6284@end example
6285
6286@noindent
2a8d363a 6287results in the following signature:
e425e872
RS
6288
6289@example
2055a44e
AD
6290int yylex (scanner_mode *mode, environment_type *env);
6291int yyparse (parser_mode *mode, environment_type *env);
e425e872
RS
6292@end example
6293
67501061 6294If @samp{%define api.pure} is added:
c656404a
RS
6295
6296@example
2055a44e
AD
6297int yylex (YYSTYPE *lvalp, scanner_mode *mode, environment_type *env);
6298int yyparse (parser_mode *mode, environment_type *env);
c656404a
RS
6299@end example
6300
2a8d363a 6301@noindent
67501061 6302and finally, if both @samp{%define api.pure} and @code{%locations} are used:
c656404a 6303
2a8d363a 6304@example
2055a44e
AD
6305int yylex (YYSTYPE *lvalp, YYLTYPE *llocp,
6306 scanner_mode *mode, environment_type *env);
6307int yyparse (parser_mode *mode, environment_type *env);
2a8d363a 6308@end example
931c7513 6309
342b8b6e 6310@node Error Reporting
bfa74976
RS
6311@section The Error Reporting Function @code{yyerror}
6312@cindex error reporting function
6313@findex yyerror
6314@cindex parse error
6315@cindex syntax error
6316
31b850d2 6317The Bison parser detects a @dfn{syntax error} (or @dfn{parse error})
9ecbd125 6318whenever it reads a token which cannot satisfy any syntax rule. An
bfa74976 6319action in the grammar can also explicitly proclaim an error, using the
ceed8467
AD
6320macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
6321in Actions}).
bfa74976
RS
6322
6323The Bison parser expects to report the error by calling an error
6324reporting function named @code{yyerror}, which you must supply. It is
6325called by @code{yyparse} whenever a syntax error is found, and it
6e649e65
PE
6326receives one argument. For a syntax error, the string is normally
6327@w{@code{"syntax error"}}.
bfa74976 6328
31b850d2 6329@findex %define parse.error
cf499cff 6330If you invoke @samp{%define parse.error verbose} in the Bison
2a8d363a
AD
6331declarations section (@pxref{Bison Declarations, ,The Bison Declarations
6332Section}), then Bison provides a more verbose and specific error message
6e649e65 6333string instead of just plain @w{@code{"syntax error"}}.
bfa74976 6334
1a059451
PE
6335The parser can detect one other kind of error: memory exhaustion. This
6336can happen when the input contains constructions that are very deeply
bfa74976 6337nested. It isn't likely you will encounter this, since the Bison
1a059451
PE
6338parser normally extends its stack automatically up to a very large limit. But
6339if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
6340fashion, except that the argument string is @w{@code{"memory exhausted"}}.
6341
6342In some cases diagnostics like @w{@code{"syntax error"}} are
6343translated automatically from English to some other language before
6344they are passed to @code{yyerror}. @xref{Internationalization}.
bfa74976
RS
6345
6346The following definition suffices in simple programs:
6347
6348@example
6349@group
13863333 6350void
38a92d50 6351yyerror (char const *s)
bfa74976
RS
6352@{
6353@end group
6354@group
6355 fprintf (stderr, "%s\n", s);
6356@}
6357@end group
6358@end example
6359
6360After @code{yyerror} returns to @code{yyparse}, the latter will attempt
6361error recovery if you have written suitable error recovery grammar rules
6362(@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
6363immediately return 1.
6364
93724f13 6365Obviously, in location tracking pure parsers, @code{yyerror} should have
fa7e68c3 6366an access to the current location.
8a4281b9 6367This is indeed the case for the GLR
2a8d363a 6368parsers, but not for the Yacc parser, for historical reasons. I.e., if
d9df47b6 6369@samp{%locations %define api.pure} is passed then the prototypes for
2a8d363a
AD
6370@code{yyerror} are:
6371
6372@example
38a92d50
PE
6373void yyerror (char const *msg); /* Yacc parsers. */
6374void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
2a8d363a
AD
6375@end example
6376
feeb0eda 6377If @samp{%parse-param @{int *nastiness@}} is used, then:
2a8d363a
AD
6378
6379@example
b317297e
PE
6380void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
6381void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
2a8d363a
AD
6382@end example
6383
8a4281b9 6384Finally, GLR and Yacc parsers share the same @code{yyerror} calling
2a8d363a
AD
6385convention for absolutely pure parsers, i.e., when the calling
6386convention of @code{yylex} @emph{and} the calling convention of
67501061 6387@samp{%define api.pure} are pure.
d9df47b6 6388I.e.:
2a8d363a
AD
6389
6390@example
6391/* Location tracking. */
6392%locations
6393/* Pure yylex. */
d9df47b6 6394%define api.pure
feeb0eda 6395%lex-param @{int *nastiness@}
2a8d363a 6396/* Pure yyparse. */
feeb0eda
PE
6397%parse-param @{int *nastiness@}
6398%parse-param @{int *randomness@}
2a8d363a
AD
6399@end example
6400
6401@noindent
6402results in the following signatures for all the parser kinds:
6403
6404@example
6405int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
6406int yyparse (int *nastiness, int *randomness);
93724f13
AD
6407void yyerror (YYLTYPE *locp,
6408 int *nastiness, int *randomness,
38a92d50 6409 char const *msg);
2a8d363a
AD
6410@end example
6411
1c0c3e95 6412@noindent
38a92d50
PE
6413The prototypes are only indications of how the code produced by Bison
6414uses @code{yyerror}. Bison-generated code always ignores the returned
6415value, so @code{yyerror} can return any type, including @code{void}.
6416Also, @code{yyerror} can be a variadic function; that is why the
6417message is always passed last.
6418
6419Traditionally @code{yyerror} returns an @code{int} that is always
6420ignored, but this is purely for historical reasons, and @code{void} is
6421preferable since it more accurately describes the return type for
6422@code{yyerror}.
93724f13 6423
bfa74976
RS
6424@vindex yynerrs
6425The variable @code{yynerrs} contains the number of syntax errors
8a2800e7 6426reported so far. Normally this variable is global; but if you
704a47c4
AD
6427request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
6428then it is a local variable which only the actions can access.
bfa74976 6429
342b8b6e 6430@node Action Features
bfa74976
RS
6431@section Special Features for Use in Actions
6432@cindex summary, action features
6433@cindex action features summary
6434
6435Here is a table of Bison constructs, variables and macros that
6436are useful in actions.
6437
18b519c0 6438@deffn {Variable} $$
bfa74976
RS
6439Acts like a variable that contains the semantic value for the
6440grouping made by the current rule. @xref{Actions}.
18b519c0 6441@end deffn
bfa74976 6442
18b519c0 6443@deffn {Variable} $@var{n}
bfa74976
RS
6444Acts like a variable that contains the semantic value for the
6445@var{n}th component of the current rule. @xref{Actions}.
18b519c0 6446@end deffn
bfa74976 6447
18b519c0 6448@deffn {Variable} $<@var{typealt}>$
bfa74976 6449Like @code{$$} but specifies alternative @var{typealt} in the union
704a47c4
AD
6450specified by the @code{%union} declaration. @xref{Action Types, ,Data
6451Types of Values in Actions}.
18b519c0 6452@end deffn
bfa74976 6453
18b519c0 6454@deffn {Variable} $<@var{typealt}>@var{n}
bfa74976 6455Like @code{$@var{n}} but specifies alternative @var{typealt} in the
13863333 6456union specified by the @code{%union} declaration.
e0c471a9 6457@xref{Action Types, ,Data Types of Values in Actions}.
18b519c0 6458@end deffn
bfa74976 6459
18b519c0 6460@deffn {Macro} YYABORT;
bfa74976
RS
6461Return immediately from @code{yyparse}, indicating failure.
6462@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 6463@end deffn
bfa74976 6464
18b519c0 6465@deffn {Macro} YYACCEPT;
bfa74976
RS
6466Return immediately from @code{yyparse}, indicating success.
6467@xref{Parser Function, ,The Parser Function @code{yyparse}}.
18b519c0 6468@end deffn
bfa74976 6469
18b519c0 6470@deffn {Macro} YYBACKUP (@var{token}, @var{value});
bfa74976
RS
6471@findex YYBACKUP
6472Unshift a token. This macro is allowed only for rules that reduce
742e4900 6473a single value, and only when there is no lookahead token.
8a4281b9 6474It is also disallowed in GLR parsers.
742e4900 6475It installs a lookahead token with token type @var{token} and
bfa74976
RS
6476semantic value @var{value}; then it discards the value that was
6477going to be reduced by this rule.
6478
6479If the macro is used when it is not valid, such as when there is
742e4900 6480a lookahead token already, then it reports a syntax error with
bfa74976
RS
6481a message @samp{cannot back up} and performs ordinary error
6482recovery.
6483
6484In either case, the rest of the action is not executed.
18b519c0 6485@end deffn
bfa74976 6486
18b519c0 6487@deffn {Macro} YYEMPTY
bfa74976 6488@vindex YYEMPTY
742e4900 6489Value stored in @code{yychar} when there is no lookahead token.
18b519c0 6490@end deffn
bfa74976 6491
32c29292
JD
6492@deffn {Macro} YYEOF
6493@vindex YYEOF
742e4900 6494Value stored in @code{yychar} when the lookahead is the end of the input
32c29292
JD
6495stream.
6496@end deffn
6497
18b519c0 6498@deffn {Macro} YYERROR;
bfa74976
RS
6499@findex YYERROR
6500Cause an immediate syntax error. This statement initiates error
6501recovery just as if the parser itself had detected an error; however, it
6502does not call @code{yyerror}, and does not print any message. If you
6503want to print an error message, call @code{yyerror} explicitly before
6504the @samp{YYERROR;} statement. @xref{Error Recovery}.
18b519c0 6505@end deffn
bfa74976 6506
18b519c0 6507@deffn {Macro} YYRECOVERING
02103984
PE
6508@findex YYRECOVERING
6509The expression @code{YYRECOVERING ()} yields 1 when the parser
6510is recovering from a syntax error, and 0 otherwise.
bfa74976 6511@xref{Error Recovery}.
18b519c0 6512@end deffn
bfa74976 6513
18b519c0 6514@deffn {Variable} yychar
742e4900
JD
6515Variable containing either the lookahead token, or @code{YYEOF} when the
6516lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
32c29292
JD
6517has been performed so the next token is not yet known.
6518Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
6519Actions}).
742e4900 6520@xref{Lookahead, ,Lookahead Tokens}.
18b519c0 6521@end deffn
bfa74976 6522
18b519c0 6523@deffn {Macro} yyclearin;
742e4900 6524Discard the current lookahead token. This is useful primarily in
32c29292
JD
6525error rules.
6526Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
6527Semantic Actions}).
6528@xref{Error Recovery}.
18b519c0 6529@end deffn
bfa74976 6530
18b519c0 6531@deffn {Macro} yyerrok;
bfa74976 6532Resume generating error messages immediately for subsequent syntax
13863333 6533errors. This is useful primarily in error rules.
bfa74976 6534@xref{Error Recovery}.
18b519c0 6535@end deffn
bfa74976 6536
32c29292 6537@deffn {Variable} yylloc
742e4900 6538Variable containing the lookahead token location when @code{yychar} is not set
32c29292
JD
6539to @code{YYEMPTY} or @code{YYEOF}.
6540Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
6541Actions}).
6542@xref{Actions and Locations, ,Actions and Locations}.
6543@end deffn
6544
6545@deffn {Variable} yylval
742e4900 6546Variable containing the lookahead token semantic value when @code{yychar} is
32c29292
JD
6547not set to @code{YYEMPTY} or @code{YYEOF}.
6548Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
6549Actions}).
6550@xref{Actions, ,Actions}.
6551@end deffn
6552
18b519c0 6553@deffn {Value} @@$
847bf1f5 6554@findex @@$
95923bd6 6555Acts like a structure variable containing information on the textual location
847bf1f5
AD
6556of the grouping made by the current rule. @xref{Locations, ,
6557Tracking Locations}.
bfa74976 6558
847bf1f5
AD
6559@c Check if those paragraphs are still useful or not.
6560
6561@c @example
6562@c struct @{
6563@c int first_line, last_line;
6564@c int first_column, last_column;
6565@c @};
6566@c @end example
6567
6568@c Thus, to get the starting line number of the third component, you would
6569@c use @samp{@@3.first_line}.
bfa74976 6570
847bf1f5
AD
6571@c In order for the members of this structure to contain valid information,
6572@c you must make @code{yylex} supply this information about each token.
6573@c If you need only certain members, then @code{yylex} need only fill in
6574@c those members.
bfa74976 6575
847bf1f5 6576@c The use of this feature makes the parser noticeably slower.
18b519c0 6577@end deffn
847bf1f5 6578
18b519c0 6579@deffn {Value} @@@var{n}
847bf1f5 6580@findex @@@var{n}
95923bd6 6581Acts like a structure variable containing information on the textual location
847bf1f5
AD
6582of the @var{n}th component of the current rule. @xref{Locations, ,
6583Tracking Locations}.
18b519c0 6584@end deffn
bfa74976 6585
f7ab6a50
PE
6586@node Internationalization
6587@section Parser Internationalization
6588@cindex internationalization
6589@cindex i18n
6590@cindex NLS
6591@cindex gettext
6592@cindex bison-po
6593
6594A Bison-generated parser can print diagnostics, including error and
6595tracing messages. By default, they appear in English. However, Bison
f8e1c9e5
AD
6596also supports outputting diagnostics in the user's native language. To
6597make this work, the user should set the usual environment variables.
6598@xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
6599For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
8a4281b9 6600set the user's locale to French Canadian using the UTF-8
f7ab6a50
PE
6601encoding. The exact set of available locales depends on the user's
6602installation.
6603
6604The maintainer of a package that uses a Bison-generated parser enables
6605the internationalization of the parser's output through the following
8a4281b9
JD
6606steps. Here we assume a package that uses GNU Autoconf and
6607GNU Automake.
f7ab6a50
PE
6608
6609@enumerate
6610@item
30757c8c 6611@cindex bison-i18n.m4
8a4281b9 6612Into the directory containing the GNU Autoconf macros used
f7ab6a50
PE
6613by the package---often called @file{m4}---copy the
6614@file{bison-i18n.m4} file installed by Bison under
6615@samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
6616For example:
6617
6618@example
6619cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
6620@end example
6621
6622@item
30757c8c
PE
6623@findex BISON_I18N
6624@vindex BISON_LOCALEDIR
6625@vindex YYENABLE_NLS
f7ab6a50
PE
6626In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
6627invocation, add an invocation of @code{BISON_I18N}. This macro is
6628defined in the file @file{bison-i18n.m4} that you copied earlier. It
6629causes @samp{configure} to find the value of the
30757c8c
PE
6630@code{BISON_LOCALEDIR} variable, and it defines the source-language
6631symbol @code{YYENABLE_NLS} to enable translations in the
6632Bison-generated parser.
f7ab6a50
PE
6633
6634@item
6635In the @code{main} function of your program, designate the directory
6636containing Bison's runtime message catalog, through a call to
6637@samp{bindtextdomain} with domain name @samp{bison-runtime}.
6638For example:
6639
6640@example
6641bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
6642@end example
6643
6644Typically this appears after any other call @code{bindtextdomain
6645(PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
6646@samp{BISON_LOCALEDIR} to be defined as a string through the
6647@file{Makefile}.
6648
6649@item
6650In the @file{Makefile.am} that controls the compilation of the @code{main}
6651function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
6652either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
6653
6654@example
6655DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6656@end example
6657
6658or:
6659
6660@example
6661AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
6662@end example
6663
6664@item
6665Finally, invoke the command @command{autoreconf} to generate the build
6666infrastructure.
6667@end enumerate
6668
bfa74976 6669
342b8b6e 6670@node Algorithm
13863333
AD
6671@chapter The Bison Parser Algorithm
6672@cindex Bison parser algorithm
bfa74976
RS
6673@cindex algorithm of parser
6674@cindex shifting
6675@cindex reduction
6676@cindex parser stack
6677@cindex stack, parser
6678
6679As Bison reads tokens, it pushes them onto a stack along with their
6680semantic values. The stack is called the @dfn{parser stack}. Pushing a
6681token is traditionally called @dfn{shifting}.
6682
6683For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
6684@samp{3} to come. The stack will have four elements, one for each token
6685that was shifted.
6686
6687But the stack does not always have an element for each token read. When
6688the last @var{n} tokens and groupings shifted match the components of a
6689grammar rule, they can be combined according to that rule. This is called
6690@dfn{reduction}. Those tokens and groupings are replaced on the stack by a
6691single grouping whose symbol is the result (left hand side) of that rule.
6692Running the rule's action is part of the process of reduction, because this
6693is what computes the semantic value of the resulting grouping.
6694
6695For example, if the infix calculator's parser stack contains this:
6696
6697@example
66981 + 5 * 3
6699@end example
6700
6701@noindent
6702and the next input token is a newline character, then the last three
6703elements can be reduced to 15 via the rule:
6704
6705@example
6706expr: expr '*' expr;
6707@end example
6708
6709@noindent
6710Then the stack contains just these three elements:
6711
6712@example
67131 + 15
6714@end example
6715
6716@noindent
6717At this point, another reduction can be made, resulting in the single value
671816. Then the newline token can be shifted.
6719
6720The parser tries, by shifts and reductions, to reduce the entire input down
6721to a single grouping whose symbol is the grammar's start-symbol
6722(@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
6723
6724This kind of parser is known in the literature as a bottom-up parser.
6725
6726@menu
742e4900 6727* Lookahead:: Parser looks one token ahead when deciding what to do.
bfa74976
RS
6728* Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
6729* Precedence:: Operator precedence works by resolving conflicts.
6730* Contextual Precedence:: When an operator's precedence depends on context.
6731* Parser States:: The parser is a finite-state-machine with stack.
6732* Reduce/Reduce:: When two rules are applicable in the same situation.
f5f419de 6733* Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
676385e2 6734* Generalized LR Parsing:: Parsing arbitrary context-free grammars.
1a059451 6735* Memory Management:: What happens when memory is exhausted. How to avoid it.
bfa74976
RS
6736@end menu
6737
742e4900
JD
6738@node Lookahead
6739@section Lookahead Tokens
6740@cindex lookahead token
bfa74976
RS
6741
6742The Bison parser does @emph{not} always reduce immediately as soon as the
6743last @var{n} tokens and groupings match a rule. This is because such a
6744simple strategy is inadequate to handle most languages. Instead, when a
6745reduction is possible, the parser sometimes ``looks ahead'' at the next
6746token in order to decide what to do.
6747
6748When a token is read, it is not immediately shifted; first it becomes the
742e4900 6749@dfn{lookahead token}, which is not on the stack. Now the parser can
bfa74976 6750perform one or more reductions of tokens and groupings on the stack, while
742e4900
JD
6751the lookahead token remains off to the side. When no more reductions
6752should take place, the lookahead token is shifted onto the stack. This
bfa74976 6753does not mean that all possible reductions have been done; depending on the
742e4900 6754token type of the lookahead token, some rules may choose to delay their
bfa74976
RS
6755application.
6756
742e4900 6757Here is a simple case where lookahead is needed. These three rules define
bfa74976
RS
6758expressions which contain binary addition operators and postfix unary
6759factorial operators (@samp{!}), and allow parentheses for grouping.
6760
6761@example
6762@group
6763expr: term '+' expr
6764 | term
6765 ;
6766@end group
6767
6768@group
6769term: '(' expr ')'
6770 | term '!'
6771 | NUMBER
6772 ;
6773@end group
6774@end example
6775
6776Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
6777should be done? If the following token is @samp{)}, then the first three
6778tokens must be reduced to form an @code{expr}. This is the only valid
6779course, because shifting the @samp{)} would produce a sequence of symbols
6780@w{@code{term ')'}}, and no rule allows this.
6781
6782If the following token is @samp{!}, then it must be shifted immediately so
6783that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
6784parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
6785@code{expr}. It would then be impossible to shift the @samp{!} because
6786doing so would produce on the stack the sequence of symbols @code{expr
6787'!'}. No rule allows that sequence.
6788
6789@vindex yychar
32c29292
JD
6790@vindex yylval
6791@vindex yylloc
742e4900 6792The lookahead token is stored in the variable @code{yychar}.
32c29292
JD
6793Its semantic value and location, if any, are stored in the variables
6794@code{yylval} and @code{yylloc}.
bfa74976
RS
6795@xref{Action Features, ,Special Features for Use in Actions}.
6796
342b8b6e 6797@node Shift/Reduce
bfa74976
RS
6798@section Shift/Reduce Conflicts
6799@cindex conflicts
6800@cindex shift/reduce conflicts
6801@cindex dangling @code{else}
6802@cindex @code{else}, dangling
6803
6804Suppose we are parsing a language which has if-then and if-then-else
6805statements, with a pair of rules like this:
6806
6807@example
6808@group
6809if_stmt:
6810 IF expr THEN stmt
6811 | IF expr THEN stmt ELSE stmt
6812 ;
6813@end group
6814@end example
6815
6816@noindent
6817Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
6818terminal symbols for specific keyword tokens.
6819
742e4900 6820When the @code{ELSE} token is read and becomes the lookahead token, the
bfa74976
RS
6821contents of the stack (assuming the input is valid) are just right for
6822reduction by the first rule. But it is also legitimate to shift the
6823@code{ELSE}, because that would lead to eventual reduction by the second
6824rule.
6825
6826This situation, where either a shift or a reduction would be valid, is
6827called a @dfn{shift/reduce conflict}. Bison is designed to resolve
6828these conflicts by choosing to shift, unless otherwise directed by
6829operator precedence declarations. To see the reason for this, let's
6830contrast it with the other alternative.
6831
6832Since the parser prefers to shift the @code{ELSE}, the result is to attach
6833the else-clause to the innermost if-statement, making these two inputs
6834equivalent:
6835
6836@example
6837if x then if y then win (); else lose;
6838
6839if x then do; if y then win (); else lose; end;
6840@end example
6841
6842But if the parser chose to reduce when possible rather than shift, the
6843result would be to attach the else-clause to the outermost if-statement,
6844making these two inputs equivalent:
6845
6846@example
6847if x then if y then win (); else lose;
6848
6849if x then do; if y then win (); end; else lose;
6850@end example
6851
6852The conflict exists because the grammar as written is ambiguous: either
6853parsing of the simple nested if-statement is legitimate. The established
6854convention is that these ambiguities are resolved by attaching the
6855else-clause to the innermost if-statement; this is what Bison accomplishes
6856by choosing to shift rather than reduce. (It would ideally be cleaner to
6857write an unambiguous grammar, but that is very hard to do in this case.)
6858This particular ambiguity was first encountered in the specifications of
6859Algol 60 and is called the ``dangling @code{else}'' ambiguity.
6860
6861To avoid warnings from Bison about predictable, legitimate shift/reduce
93d7dde9
JD
6862conflicts, use the @code{%expect @var{n}} declaration.
6863There will be no warning as long as the number of shift/reduce conflicts
6864is exactly @var{n}, and Bison will report an error if there is a
6865different number.
bfa74976
RS
6866@xref{Expect Decl, ,Suppressing Conflict Warnings}.
6867
6868The definition of @code{if_stmt} above is solely to blame for the
6869conflict, but the conflict does not actually appear without additional
ff7571c0
JD
6870rules. Here is a complete Bison grammar file that actually manifests
6871the conflict:
bfa74976
RS
6872
6873@example
6874@group
6875%token IF THEN ELSE variable
6876%%
6877@end group
6878@group
6879stmt: expr
6880 | if_stmt
6881 ;
6882@end group
6883
6884@group
6885if_stmt:
6886 IF expr THEN stmt
6887 | IF expr THEN stmt ELSE stmt
6888 ;
6889@end group
6890
6891expr: variable
6892 ;
6893@end example
6894
342b8b6e 6895@node Precedence
bfa74976
RS
6896@section Operator Precedence
6897@cindex operator precedence
6898@cindex precedence of operators
6899
6900Another situation where shift/reduce conflicts appear is in arithmetic
6901expressions. Here shifting is not always the preferred resolution; the
6902Bison declarations for operator precedence allow you to specify when to
6903shift and when to reduce.
6904
6905@menu
6906* Why Precedence:: An example showing why precedence is needed.
d78f0ac9
AD
6907* Using Precedence:: How to specify precedence and associativity.
6908* Precedence Only:: How to specify precedence only.
bfa74976
RS
6909* Precedence Examples:: How these features are used in the previous example.
6910* How Precedence:: How they work.
6911@end menu
6912
342b8b6e 6913@node Why Precedence
bfa74976
RS
6914@subsection When Precedence is Needed
6915
6916Consider the following ambiguous grammar fragment (ambiguous because the
6917input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
6918
6919@example
6920@group
6921expr: expr '-' expr
6922 | expr '*' expr
6923 | expr '<' expr
6924 | '(' expr ')'
6925 @dots{}
6926 ;
6927@end group
6928@end example
6929
6930@noindent
6931Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
14ded682
AD
6932should it reduce them via the rule for the subtraction operator? It
6933depends on the next token. Of course, if the next token is @samp{)}, we
6934must reduce; shifting is invalid because no single rule can reduce the
6935token sequence @w{@samp{- 2 )}} or anything starting with that. But if
6936the next token is @samp{*} or @samp{<}, we have a choice: either
6937shifting or reduction would allow the parse to complete, but with
6938different results.
6939
6940To decide which one Bison should do, we must consider the results. If
6941the next operator token @var{op} is shifted, then it must be reduced
6942first in order to permit another opportunity to reduce the difference.
6943The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
6944hand, if the subtraction is reduced before shifting @var{op}, the result
6945is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
6946reduce should depend on the relative precedence of the operators
6947@samp{-} and @var{op}: @samp{*} should be shifted first, but not
6948@samp{<}.
bfa74976
RS
6949
6950@cindex associativity
6951What about input such as @w{@samp{1 - 2 - 5}}; should this be
14ded682
AD
6952@w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
6953operators we prefer the former, which is called @dfn{left association}.
6954The latter alternative, @dfn{right association}, is desirable for
6955assignment operators. The choice of left or right association is a
6956matter of whether the parser chooses to shift or reduce when the stack
742e4900 6957contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
14ded682 6958makes right-associativity.
bfa74976 6959
342b8b6e 6960@node Using Precedence
bfa74976
RS
6961@subsection Specifying Operator Precedence
6962@findex %left
bfa74976 6963@findex %nonassoc
d78f0ac9
AD
6964@findex %precedence
6965@findex %right
bfa74976
RS
6966
6967Bison allows you to specify these choices with the operator precedence
6968declarations @code{%left} and @code{%right}. Each such declaration
6969contains a list of tokens, which are operators whose precedence and
6970associativity is being declared. The @code{%left} declaration makes all
6971those operators left-associative and the @code{%right} declaration makes
6972them right-associative. A third alternative is @code{%nonassoc}, which
6973declares that it is a syntax error to find the same operator twice ``in a
6974row''.
d78f0ac9
AD
6975The last alternative, @code{%precedence}, allows to define only
6976precedence and no associativity at all. As a result, any
6977associativity-related conflict that remains will be reported as an
6978compile-time error. The directive @code{%nonassoc} creates run-time
6979error: using the operator in a associative way is a syntax error. The
6980directive @code{%precedence} creates compile-time errors: an operator
6981@emph{can} be involved in an associativity-related conflict, contrary to
6982what expected the grammar author.
bfa74976
RS
6983
6984The relative precedence of different operators is controlled by the
d78f0ac9
AD
6985order in which they are declared. The first precedence/associativity
6986declaration in the file declares the operators whose
bfa74976
RS
6987precedence is lowest, the next such declaration declares the operators
6988whose precedence is a little higher, and so on.
6989
d78f0ac9
AD
6990@node Precedence Only
6991@subsection Specifying Precedence Only
6992@findex %precedence
6993
8a4281b9 6994Since POSIX Yacc defines only @code{%left}, @code{%right}, and
d78f0ac9
AD
6995@code{%nonassoc}, which all defines precedence and associativity, little
6996attention is paid to the fact that precedence cannot be defined without
6997defining associativity. Yet, sometimes, when trying to solve a
6998conflict, precedence suffices. In such a case, using @code{%left},
6999@code{%right}, or @code{%nonassoc} might hide future (associativity
7000related) conflicts that would remain hidden.
7001
7002The dangling @code{else} ambiguity (@pxref{Shift/Reduce, , Shift/Reduce
f50bfcd6 7003Conflicts}) can be solved explicitly. This shift/reduce conflicts occurs
d78f0ac9
AD
7004in the following situation, where the period denotes the current parsing
7005state:
7006
7007@example
7008if @var{e1} then if @var{e2} then @var{s1} . else @var{s2}
7009@end example
7010
7011The conflict involves the reduction of the rule @samp{IF expr THEN
7012stmt}, which precedence is by default that of its last token
7013(@code{THEN}), and the shifting of the token @code{ELSE}. The usual
7014disambiguation (attach the @code{else} to the closest @code{if}),
7015shifting must be preferred, i.e., the precedence of @code{ELSE} must be
7016higher than that of @code{THEN}. But neither is expected to be involved
7017in an associativity related conflict, which can be specified as follows.
7018
7019@example
7020%precedence THEN
7021%precedence ELSE
7022@end example
7023
7024The unary-minus is another typical example where associativity is
7025usually over-specified, see @ref{Infix Calc, , Infix Notation
f50bfcd6 7026Calculator: @code{calc}}. The @code{%left} directive is traditionally
d78f0ac9
AD
7027used to declare the precedence of @code{NEG}, which is more than needed
7028since it also defines its associativity. While this is harmless in the
7029traditional example, who knows how @code{NEG} might be used in future
7030evolutions of the grammar@dots{}
7031
342b8b6e 7032@node Precedence Examples
bfa74976
RS
7033@subsection Precedence Examples
7034
7035In our example, we would want the following declarations:
7036
7037@example
7038%left '<'
7039%left '-'
7040%left '*'
7041@end example
7042
7043In a more complete example, which supports other operators as well, we
7044would declare them in groups of equal precedence. For example, @code{'+'} is
7045declared with @code{'-'}:
7046
7047@example
7048%left '<' '>' '=' NE LE GE
7049%left '+' '-'
7050%left '*' '/'
7051@end example
7052
7053@noindent
7054(Here @code{NE} and so on stand for the operators for ``not equal''
7055and so on. We assume that these tokens are more than one character long
7056and therefore are represented by names, not character literals.)
7057
342b8b6e 7058@node How Precedence
bfa74976
RS
7059@subsection How Precedence Works
7060
7061The first effect of the precedence declarations is to assign precedence
7062levels to the terminal symbols declared. The second effect is to assign
704a47c4
AD
7063precedence levels to certain rules: each rule gets its precedence from
7064the last terminal symbol mentioned in the components. (You can also
7065specify explicitly the precedence of a rule. @xref{Contextual
7066Precedence, ,Context-Dependent Precedence}.)
7067
7068Finally, the resolution of conflicts works by comparing the precedence
742e4900 7069of the rule being considered with that of the lookahead token. If the
704a47c4
AD
7070token's precedence is higher, the choice is to shift. If the rule's
7071precedence is higher, the choice is to reduce. If they have equal
7072precedence, the choice is made based on the associativity of that
7073precedence level. The verbose output file made by @samp{-v}
7074(@pxref{Invocation, ,Invoking Bison}) says how each conflict was
7075resolved.
bfa74976
RS
7076
7077Not all rules and not all tokens have precedence. If either the rule or
742e4900 7078the lookahead token has no precedence, then the default is to shift.
bfa74976 7079
342b8b6e 7080@node Contextual Precedence
bfa74976
RS
7081@section Context-Dependent Precedence
7082@cindex context-dependent precedence
7083@cindex unary operator precedence
7084@cindex precedence, context-dependent
7085@cindex precedence, unary operator
7086@findex %prec
7087
7088Often the precedence of an operator depends on the context. This sounds
7089outlandish at first, but it is really very common. For example, a minus
7090sign typically has a very high precedence as a unary operator, and a
7091somewhat lower precedence (lower than multiplication) as a binary operator.
7092
d78f0ac9
AD
7093The Bison precedence declarations
7094can only be used once for a given token; so a token has
bfa74976
RS
7095only one precedence declared in this way. For context-dependent
7096precedence, you need to use an additional mechanism: the @code{%prec}
e0c471a9 7097modifier for rules.
bfa74976
RS
7098
7099The @code{%prec} modifier declares the precedence of a particular rule by
7100specifying a terminal symbol whose precedence should be used for that rule.
7101It's not necessary for that symbol to appear otherwise in the rule. The
7102modifier's syntax is:
7103
7104@example
7105%prec @var{terminal-symbol}
7106@end example
7107
7108@noindent
7109and it is written after the components of the rule. Its effect is to
7110assign the rule the precedence of @var{terminal-symbol}, overriding
7111the precedence that would be deduced for it in the ordinary way. The
7112altered rule precedence then affects how conflicts involving that rule
7113are resolved (@pxref{Precedence, ,Operator Precedence}).
7114
7115Here is how @code{%prec} solves the problem of unary minus. First, declare
7116a precedence for a fictitious terminal symbol named @code{UMINUS}. There
7117are no tokens of this type, but the symbol serves to stand for its
7118precedence:
7119
7120@example
7121@dots{}
7122%left '+' '-'
7123%left '*'
7124%left UMINUS
7125@end example
7126
7127Now the precedence of @code{UMINUS} can be used in specific rules:
7128
7129@example
7130@group
7131exp: @dots{}
7132 | exp '-' exp
7133 @dots{}
7134 | '-' exp %prec UMINUS
7135@end group
7136@end example
7137
91d2c560 7138@ifset defaultprec
39a06c25
PE
7139If you forget to append @code{%prec UMINUS} to the rule for unary
7140minus, Bison silently assumes that minus has its usual precedence.
7141This kind of problem can be tricky to debug, since one typically
7142discovers the mistake only by testing the code.
7143
22fccf95 7144The @code{%no-default-prec;} declaration makes it easier to discover
39a06c25
PE
7145this kind of problem systematically. It causes rules that lack a
7146@code{%prec} modifier to have no precedence, even if the last terminal
7147symbol mentioned in their components has a declared precedence.
7148
22fccf95 7149If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
39a06c25
PE
7150for all rules that participate in precedence conflict resolution.
7151Then you will see any shift/reduce conflict until you tell Bison how
7152to resolve it, either by changing your grammar or by adding an
7153explicit precedence. This will probably add declarations to the
7154grammar, but it helps to protect against incorrect rule precedences.
7155
22fccf95
PE
7156The effect of @code{%no-default-prec;} can be reversed by giving
7157@code{%default-prec;}, which is the default.
91d2c560 7158@end ifset
39a06c25 7159
342b8b6e 7160@node Parser States
bfa74976
RS
7161@section Parser States
7162@cindex finite-state machine
7163@cindex parser state
7164@cindex state (of parser)
7165
7166The function @code{yyparse} is implemented using a finite-state machine.
7167The values pushed on the parser stack are not simply token type codes; they
7168represent the entire sequence of terminal and nonterminal symbols at or
7169near the top of the stack. The current state collects all the information
7170about previous input which is relevant to deciding what to do next.
7171
742e4900
JD
7172Each time a lookahead token is read, the current parser state together
7173with the type of lookahead token are looked up in a table. This table
7174entry can say, ``Shift the lookahead token.'' In this case, it also
bfa74976
RS
7175specifies the new parser state, which is pushed onto the top of the
7176parser stack. Or it can say, ``Reduce using rule number @var{n}.''
7177This means that a certain number of tokens or groupings are taken off
7178the top of the stack, and replaced by one grouping. In other words,
7179that number of states are popped from the stack, and one new state is
7180pushed.
7181
742e4900 7182There is one other alternative: the table can say that the lookahead token
bfa74976
RS
7183is erroneous in the current state. This causes error processing to begin
7184(@pxref{Error Recovery}).
7185
342b8b6e 7186@node Reduce/Reduce
bfa74976
RS
7187@section Reduce/Reduce Conflicts
7188@cindex reduce/reduce conflict
7189@cindex conflicts, reduce/reduce
7190
7191A reduce/reduce conflict occurs if there are two or more rules that apply
7192to the same sequence of input. This usually indicates a serious error
7193in the grammar.
7194
7195For example, here is an erroneous attempt to define a sequence
7196of zero or more @code{word} groupings.
7197
7198@example
7199sequence: /* empty */
7200 @{ printf ("empty sequence\n"); @}
7201 | maybeword
7202 | sequence word
7203 @{ printf ("added word %s\n", $2); @}
7204 ;
7205
7206maybeword: /* empty */
7207 @{ printf ("empty maybeword\n"); @}
7208 | word
7209 @{ printf ("single word %s\n", $1); @}
7210 ;
7211@end example
7212
7213@noindent
7214The error is an ambiguity: there is more than one way to parse a single
7215@code{word} into a @code{sequence}. It could be reduced to a
7216@code{maybeword} and then into a @code{sequence} via the second rule.
7217Alternatively, nothing-at-all could be reduced into a @code{sequence}
7218via the first rule, and this could be combined with the @code{word}
7219using the third rule for @code{sequence}.
7220
7221There is also more than one way to reduce nothing-at-all into a
7222@code{sequence}. This can be done directly via the first rule,
7223or indirectly via @code{maybeword} and then the second rule.
7224
7225You might think that this is a distinction without a difference, because it
7226does not change whether any particular input is valid or not. But it does
7227affect which actions are run. One parsing order runs the second rule's
7228action; the other runs the first rule's action and the third rule's action.
7229In this example, the output of the program changes.
7230
7231Bison resolves a reduce/reduce conflict by choosing to use the rule that
7232appears first in the grammar, but it is very risky to rely on this. Every
7233reduce/reduce conflict must be studied and usually eliminated. Here is the
7234proper way to define @code{sequence}:
7235
7236@example
7237sequence: /* empty */
7238 @{ printf ("empty sequence\n"); @}
7239 | sequence word
7240 @{ printf ("added word %s\n", $2); @}
7241 ;
7242@end example
7243
7244Here is another common error that yields a reduce/reduce conflict:
7245
7246@example
7247sequence: /* empty */
7248 | sequence words
7249 | sequence redirects
7250 ;
7251
7252words: /* empty */
7253 | words word
7254 ;
7255
7256redirects:/* empty */
7257 | redirects redirect
7258 ;
7259@end example
7260
7261@noindent
7262The intention here is to define a sequence which can contain either
7263@code{word} or @code{redirect} groupings. The individual definitions of
7264@code{sequence}, @code{words} and @code{redirects} are error-free, but the
7265three together make a subtle ambiguity: even an empty input can be parsed
7266in infinitely many ways!
7267
7268Consider: nothing-at-all could be a @code{words}. Or it could be two
7269@code{words} in a row, or three, or any number. It could equally well be a
7270@code{redirects}, or two, or any number. Or it could be a @code{words}
7271followed by three @code{redirects} and another @code{words}. And so on.
7272
7273Here are two ways to correct these rules. First, to make it a single level
7274of sequence:
7275
7276@example
7277sequence: /* empty */
7278 | sequence word
7279 | sequence redirect
7280 ;
7281@end example
7282
7283Second, to prevent either a @code{words} or a @code{redirects}
7284from being empty:
7285
7286@example
7287sequence: /* empty */
7288 | sequence words
7289 | sequence redirects
7290 ;
7291
7292words: word
7293 | words word
7294 ;
7295
7296redirects:redirect
7297 | redirects redirect
7298 ;
7299@end example
7300
342b8b6e 7301@node Mystery Conflicts
bfa74976
RS
7302@section Mysterious Reduce/Reduce Conflicts
7303
7304Sometimes reduce/reduce conflicts can occur that don't look warranted.
7305Here is an example:
7306
7307@example
7308@group
7309%token ID
7310
7311%%
7312def: param_spec return_spec ','
7313 ;
7314param_spec:
7315 type
7316 | name_list ':' type
7317 ;
7318@end group
7319@group
7320return_spec:
7321 type
7322 | name ':' type
7323 ;
7324@end group
7325@group
7326type: ID
7327 ;
7328@end group
7329@group
7330name: ID
7331 ;
7332name_list:
7333 name
7334 | name ',' name_list
7335 ;
7336@end group
7337@end example
7338
7339It would seem that this grammar can be parsed with only a single token
742e4900 7340of lookahead: when a @code{param_spec} is being read, an @code{ID} is
bfa74976 7341a @code{name} if a comma or colon follows, or a @code{type} if another
8a4281b9 7342@code{ID} follows. In other words, this grammar is LR(1).
bfa74976 7343
8a4281b9
JD
7344@cindex LR(1)
7345@cindex LALR(1)
eb45ef3b 7346However, for historical reasons, Bison cannot by default handle all
8a4281b9 7347LR(1) grammars.
eb45ef3b
JD
7348In this grammar, two contexts, that after an @code{ID} at the beginning
7349of a @code{param_spec} and likewise at the beginning of a
7350@code{return_spec}, are similar enough that Bison assumes they are the
7351same.
7352They appear similar because the same set of rules would be
bfa74976
RS
7353active---the rule for reducing to a @code{name} and that for reducing to
7354a @code{type}. Bison is unable to determine at that stage of processing
742e4900 7355that the rules would require different lookahead tokens in the two
bfa74976
RS
7356contexts, so it makes a single parser state for them both. Combining
7357the two contexts causes a conflict later. In parser terminology, this
8a4281b9 7358occurrence means that the grammar is not LALR(1).
bfa74976 7359
eb45ef3b 7360For many practical grammars (specifically those that fall into the
35c1e5f0
JD
7361non-LR(1) class), the limitations of LALR(1) result in difficulties
7362beyond just mysterious reduce/reduce conflicts. The best way to fix
7363all these problems is to select a different parser table generation
7364algorithm. Either IELR(1) or canonical LR(1) would suffice, but the
7365former is more efficient and easier to debug during development.
7366@xref{%define Summary,,lr.type}, for details. (Bison's IELR(1) and
7367canonical LR(1) implementations are experimental. More user feedback
7368will help to stabilize them.)
eb45ef3b 7369
8a4281b9 7370If you instead wish to work around LALR(1)'s limitations, you
eb45ef3b
JD
7371can often fix a mysterious conflict by identifying the two parser states
7372that are being confused, and adding something to make them look
7373distinct. In the above example, adding one rule to
bfa74976
RS
7374@code{return_spec} as follows makes the problem go away:
7375
7376@example
7377@group
7378%token BOGUS
7379@dots{}
7380%%
7381@dots{}
7382return_spec:
7383 type
7384 | name ':' type
7385 /* This rule is never used. */
7386 | ID BOGUS
7387 ;
7388@end group
7389@end example
7390
7391This corrects the problem because it introduces the possibility of an
7392additional active rule in the context after the @code{ID} at the beginning of
7393@code{return_spec}. This rule is not active in the corresponding context
7394in a @code{param_spec}, so the two contexts receive distinct parser states.
7395As long as the token @code{BOGUS} is never generated by @code{yylex},
7396the added rule cannot alter the way actual input is parsed.
7397
7398In this particular example, there is another way to solve the problem:
7399rewrite the rule for @code{return_spec} to use @code{ID} directly
7400instead of via @code{name}. This also causes the two confusing
7401contexts to have different sets of active rules, because the one for
7402@code{return_spec} activates the altered rule for @code{return_spec}
7403rather than the one for @code{name}.
7404
7405@example
7406param_spec:
7407 type
7408 | name_list ':' type
7409 ;
7410return_spec:
7411 type
7412 | ID ':' type
7413 ;
7414@end example
7415
8a4281b9 7416For a more detailed exposition of LALR(1) parsers and parser
e054b190
PE
7417generators, please see:
7418Frank DeRemer and Thomas Pennello, Efficient Computation of
8a4281b9 7419LALR(1) Look-Ahead Sets, @cite{ACM Transactions on
e054b190
PE
7420Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
7421pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
7422
fae437e8 7423@node Generalized LR Parsing
8a4281b9
JD
7424@section Generalized LR (GLR) Parsing
7425@cindex GLR parsing
7426@cindex generalized LR (GLR) parsing
676385e2 7427@cindex ambiguous grammars
9d9b8b70 7428@cindex nondeterministic parsing
676385e2 7429
fae437e8
AD
7430Bison produces @emph{deterministic} parsers that choose uniquely
7431when to reduce and which reduction to apply
742e4900 7432based on a summary of the preceding input and on one extra token of lookahead.
676385e2
PH
7433As a result, normal Bison handles a proper subset of the family of
7434context-free languages.
fae437e8 7435Ambiguous grammars, since they have strings with more than one possible
676385e2
PH
7436sequence of reductions cannot have deterministic parsers in this sense.
7437The same is true of languages that require more than one symbol of
742e4900 7438lookahead, since the parser lacks the information necessary to make a
676385e2 7439decision at the point it must be made in a shift-reduce parser.
fae437e8 7440Finally, as previously mentioned (@pxref{Mystery Conflicts}),
eb45ef3b 7441there are languages where Bison's default choice of how to
676385e2
PH
7442summarize the input seen so far loses necessary information.
7443
7444When you use the @samp{%glr-parser} declaration in your grammar file,
7445Bison generates a parser that uses a different algorithm, called
8a4281b9 7446Generalized LR (or GLR). A Bison GLR
c827f760 7447parser uses the same basic
676385e2
PH
7448algorithm for parsing as an ordinary Bison parser, but behaves
7449differently in cases where there is a shift-reduce conflict that has not
fae437e8 7450been resolved by precedence rules (@pxref{Precedence}) or a
8a4281b9 7451reduce-reduce conflict. When a GLR parser encounters such a
c827f760 7452situation, it
fae437e8 7453effectively @emph{splits} into a several parsers, one for each possible
676385e2
PH
7454shift or reduction. These parsers then proceed as usual, consuming
7455tokens in lock-step. Some of the stacks may encounter other conflicts
fae437e8 7456and split further, with the result that instead of a sequence of states,
8a4281b9 7457a Bison GLR parsing stack is what is in effect a tree of states.
676385e2
PH
7458
7459In effect, each stack represents a guess as to what the proper parse
7460is. Additional input may indicate that a guess was wrong, in which case
7461the appropriate stack silently disappears. Otherwise, the semantics
fae437e8 7462actions generated in each stack are saved, rather than being executed
676385e2 7463immediately. When a stack disappears, its saved semantic actions never
fae437e8 7464get executed. When a reduction causes two stacks to become equivalent,
676385e2
PH
7465their sets of semantic actions are both saved with the state that
7466results from the reduction. We say that two stacks are equivalent
fae437e8 7467when they both represent the same sequence of states,
676385e2
PH
7468and each pair of corresponding states represents a
7469grammar symbol that produces the same segment of the input token
7470stream.
7471
7472Whenever the parser makes a transition from having multiple
eb45ef3b 7473states to having one, it reverts to the normal deterministic parsing
676385e2
PH
7474algorithm, after resolving and executing the saved-up actions.
7475At this transition, some of the states on the stack will have semantic
7476values that are sets (actually multisets) of possible actions. The
7477parser tries to pick one of the actions by first finding one whose rule
7478has the highest dynamic precedence, as set by the @samp{%dprec}
fae437e8 7479declaration. Otherwise, if the alternative actions are not ordered by
676385e2 7480precedence, but there the same merging function is declared for both
fae437e8 7481rules by the @samp{%merge} declaration,
676385e2
PH
7482Bison resolves and evaluates both and then calls the merge function on
7483the result. Otherwise, it reports an ambiguity.
7484
8a4281b9
JD
7485It is possible to use a data structure for the GLR parsing tree that
7486permits the processing of any LR(1) grammar in linear time (in the
c827f760 7487size of the input), any unambiguous (not necessarily
8a4281b9 7488LR(1)) grammar in
fae437e8 7489quadratic worst-case time, and any general (possibly ambiguous)
676385e2
PH
7490context-free grammar in cubic worst-case time. However, Bison currently
7491uses a simpler data structure that requires time proportional to the
7492length of the input times the maximum number of stacks required for any
9d9b8b70 7493prefix of the input. Thus, really ambiguous or nondeterministic
676385e2
PH
7494grammars can require exponential time and space to process. Such badly
7495behaving examples, however, are not generally of practical interest.
9d9b8b70 7496Usually, nondeterminism in a grammar is local---the parser is ``in
676385e2 7497doubt'' only for a few tokens at a time. Therefore, the current data
8a4281b9 7498structure should generally be adequate. On LR(1) portions of a
eb45ef3b 7499grammar, in particular, it is only slightly slower than with the
8a4281b9 7500deterministic LR(1) Bison parser.
676385e2 7501
8a4281b9 7502For a more detailed exposition of GLR parsers, please see: Elizabeth
f6481e2f 7503Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
8a4281b9 7504Generalised LR Parsers, Royal Holloway, University of
f6481e2f
PE
7505London, Department of Computer Science, TR-00-12,
7506@uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps},
7507(2000-12-24).
7508
1a059451
PE
7509@node Memory Management
7510@section Memory Management, and How to Avoid Memory Exhaustion
7511@cindex memory exhaustion
7512@cindex memory management
bfa74976
RS
7513@cindex stack overflow
7514@cindex parser stack overflow
7515@cindex overflow of parser stack
7516
1a059451 7517The Bison parser stack can run out of memory if too many tokens are shifted and
bfa74976 7518not reduced. When this happens, the parser function @code{yyparse}
1a059451 7519calls @code{yyerror} and then returns 2.
bfa74976 7520
c827f760 7521Because Bison parsers have growing stacks, hitting the upper limit
d1a1114f
AD
7522usually results from using a right recursion instead of a left
7523recursion, @xref{Recursion, ,Recursive Rules}.
7524
bfa74976
RS
7525@vindex YYMAXDEPTH
7526By defining the macro @code{YYMAXDEPTH}, you can control how deep the
1a059451 7527parser stack can become before memory is exhausted. Define the
bfa74976
RS
7528macro with a value that is an integer. This value is the maximum number
7529of tokens that can be shifted (and not reduced) before overflow.
bfa74976
RS
7530
7531The stack space allowed is not necessarily allocated. If you specify a
1a059451 7532large value for @code{YYMAXDEPTH}, the parser normally allocates a small
bfa74976
RS
7533stack at first, and then makes it bigger by stages as needed. This
7534increasing allocation happens automatically and silently. Therefore,
7535you do not need to make @code{YYMAXDEPTH} painfully small merely to save
7536space for ordinary inputs that do not need much stack.
7537
d7e14fc0
PE
7538However, do not allow @code{YYMAXDEPTH} to be a value so large that
7539arithmetic overflow could occur when calculating the size of the stack
7540space. Also, do not allow @code{YYMAXDEPTH} to be less than
7541@code{YYINITDEPTH}.
7542
bfa74976
RS
7543@cindex default stack limit
7544The default value of @code{YYMAXDEPTH}, if you do not define it, is
754510000.
7546
7547@vindex YYINITDEPTH
7548You can control how much stack is allocated initially by defining the
eb45ef3b
JD
7549macro @code{YYINITDEPTH} to a positive integer. For the deterministic
7550parser in C, this value must be a compile-time constant
d7e14fc0
PE
7551unless you are assuming C99 or some other target language or compiler
7552that allows variable-length arrays. The default is 200.
7553
1a059451 7554Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
bfa74976 7555
20be2f92
PH
7556You can generate a deterministic parser containing C++ user code from
7557the default (C) skeleton, as well as from the C++ skeleton
7558(@pxref{C++ Parsers}). However, if you do use the default skeleton
7559and want to allow the parsing stack to grow,
7560be careful not to use semantic types or location types that require
7561non-trivial copy constructors.
7562The C skeleton bypasses these constructors when copying data to
7563new, larger stacks.
d1a1114f 7564
342b8b6e 7565@node Error Recovery
bfa74976
RS
7566@chapter Error Recovery
7567@cindex error recovery
7568@cindex recovery from errors
7569
6e649e65 7570It is not usually acceptable to have a program terminate on a syntax
bfa74976
RS
7571error. For example, a compiler should recover sufficiently to parse the
7572rest of the input file and check it for errors; a calculator should accept
7573another expression.
7574
7575In a simple interactive command parser where each input is one line, it may
7576be sufficient to allow @code{yyparse} to return 1 on error and have the
7577caller ignore the rest of the input line when that happens (and then call
7578@code{yyparse} again). But this is inadequate for a compiler, because it
7579forgets all the syntactic context leading up to the error. A syntax error
7580deep within a function in the compiler input should not cause the compiler
7581to treat the following line like the beginning of a source file.
7582
7583@findex error
7584You can define how to recover from a syntax error by writing rules to
7585recognize the special token @code{error}. This is a terminal symbol that
7586is always defined (you need not declare it) and reserved for error
7587handling. The Bison parser generates an @code{error} token whenever a
7588syntax error happens; if you have provided a rule to recognize this token
13863333 7589in the current context, the parse can continue.
bfa74976
RS
7590
7591For example:
7592
7593@example
7594stmnts: /* empty string */
7595 | stmnts '\n'
7596 | stmnts exp '\n'
7597 | stmnts error '\n'
7598@end example
7599
7600The fourth rule in this example says that an error followed by a newline
7601makes a valid addition to any @code{stmnts}.
7602
7603What happens if a syntax error occurs in the middle of an @code{exp}? The
7604error recovery rule, interpreted strictly, applies to the precise sequence
7605of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
7606the middle of an @code{exp}, there will probably be some additional tokens
7607and subexpressions on the stack after the last @code{stmnts}, and there
7608will be tokens to read before the next newline. So the rule is not
7609applicable in the ordinary way.
7610
7611But Bison can force the situation to fit the rule, by discarding part of
72f889cc
AD
7612the semantic context and part of the input. First it discards states
7613and objects from the stack until it gets back to a state in which the
bfa74976 7614@code{error} token is acceptable. (This means that the subexpressions
72f889cc
AD
7615already parsed are discarded, back to the last complete @code{stmnts}.)
7616At this point the @code{error} token can be shifted. Then, if the old
742e4900 7617lookahead token is not acceptable to be shifted next, the parser reads
bfa74976 7618tokens and discards them until it finds a token which is acceptable. In
72f889cc
AD
7619this example, Bison reads and discards input until the next newline so
7620that the fourth rule can apply. Note that discarded symbols are
7621possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
7622Discarded Symbols}, for a means to reclaim this memory.
bfa74976
RS
7623
7624The choice of error rules in the grammar is a choice of strategies for
7625error recovery. A simple and useful strategy is simply to skip the rest of
7626the current input line or current statement if an error is detected:
7627
7628@example
72d2299c 7629stmnt: error ';' /* On error, skip until ';' is read. */
bfa74976
RS
7630@end example
7631
7632It is also useful to recover to the matching close-delimiter of an
7633opening-delimiter that has already been parsed. Otherwise the
7634close-delimiter will probably appear to be unmatched, and generate another,
7635spurious error message:
7636
7637@example
7638primary: '(' expr ')'
7639 | '(' error ')'
7640 @dots{}
7641 ;
7642@end example
7643
7644Error recovery strategies are necessarily guesses. When they guess wrong,
7645one syntax error often leads to another. In the above example, the error
7646recovery rule guesses that an error is due to bad input within one
7647@code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
7648middle of a valid @code{stmnt}. After the error recovery rule recovers
7649from the first error, another syntax error will be found straightaway,
7650since the text following the spurious semicolon is also an invalid
7651@code{stmnt}.
7652
7653To prevent an outpouring of error messages, the parser will output no error
7654message for another syntax error that happens shortly after the first; only
7655after three consecutive input tokens have been successfully shifted will
7656error messages resume.
7657
7658Note that rules which accept the @code{error} token may have actions, just
7659as any other rules can.
7660
7661@findex yyerrok
7662You can make error messages resume immediately by using the macro
7663@code{yyerrok} in an action. If you do this in the error rule's action, no
7664error messages will be suppressed. This macro requires no arguments;
7665@samp{yyerrok;} is a valid C statement.
7666
7667@findex yyclearin
742e4900 7668The previous lookahead token is reanalyzed immediately after an error. If
bfa74976
RS
7669this is unacceptable, then the macro @code{yyclearin} may be used to clear
7670this token. Write the statement @samp{yyclearin;} in the error rule's
7671action.
32c29292 7672@xref{Action Features, ,Special Features for Use in Actions}.
bfa74976 7673
6e649e65 7674For example, suppose that on a syntax error, an error handling routine is
bfa74976
RS
7675called that advances the input stream to some point where parsing should
7676once again commence. The next symbol returned by the lexical scanner is
742e4900 7677probably correct. The previous lookahead token ought to be discarded
bfa74976
RS
7678with @samp{yyclearin;}.
7679
7680@vindex YYRECOVERING
02103984
PE
7681The expression @code{YYRECOVERING ()} yields 1 when the parser
7682is recovering from a syntax error, and 0 otherwise.
7683Syntax error diagnostics are suppressed while recovering from a syntax
7684error.
bfa74976 7685
342b8b6e 7686@node Context Dependency
bfa74976
RS
7687@chapter Handling Context Dependencies
7688
7689The Bison paradigm is to parse tokens first, then group them into larger
7690syntactic units. In many languages, the meaning of a token is affected by
7691its context. Although this violates the Bison paradigm, certain techniques
7692(known as @dfn{kludges}) may enable you to write Bison parsers for such
7693languages.
7694
7695@menu
7696* Semantic Tokens:: Token parsing can depend on the semantic context.
7697* Lexical Tie-ins:: Token parsing can depend on the syntactic context.
7698* Tie-in Recovery:: Lexical tie-ins have implications for how
7699 error recovery rules must be written.
7700@end menu
7701
7702(Actually, ``kludge'' means any technique that gets its job done but is
7703neither clean nor robust.)
7704
342b8b6e 7705@node Semantic Tokens
bfa74976
RS
7706@section Semantic Info in Token Types
7707
7708The C language has a context dependency: the way an identifier is used
7709depends on what its current meaning is. For example, consider this:
7710
7711@example
7712foo (x);
7713@end example
7714
7715This looks like a function call statement, but if @code{foo} is a typedef
7716name, then this is actually a declaration of @code{x}. How can a Bison
7717parser for C decide how to parse this input?
7718
8a4281b9 7719The method used in GNU C is to have two different token types,
bfa74976
RS
7720@code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
7721identifier, it looks up the current declaration of the identifier in order
7722to decide which token type to return: @code{TYPENAME} if the identifier is
7723declared as a typedef, @code{IDENTIFIER} otherwise.
7724
7725The grammar rules can then express the context dependency by the choice of
7726token type to recognize. @code{IDENTIFIER} is accepted as an expression,
7727but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
7728@code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
7729is @emph{not} significant, such as in declarations that can shadow a
7730typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
7731accepted---there is one rule for each of the two token types.
7732
7733This technique is simple to use if the decision of which kinds of
7734identifiers to allow is made at a place close to where the identifier is
7735parsed. But in C this is not always so: C allows a declaration to
7736redeclare a typedef name provided an explicit type has been specified
7737earlier:
7738
7739@example
3a4f411f
PE
7740typedef int foo, bar;
7741int baz (void)
7742@{
7743 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
7744 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
7745 return foo (bar);
7746@}
bfa74976
RS
7747@end example
7748
7749Unfortunately, the name being declared is separated from the declaration
7750construct itself by a complicated syntactic structure---the ``declarator''.
7751
9ecbd125 7752As a result, part of the Bison parser for C needs to be duplicated, with
14ded682
AD
7753all the nonterminal names changed: once for parsing a declaration in
7754which a typedef name can be redefined, and once for parsing a
7755declaration in which that can't be done. Here is a part of the
7756duplication, with actions omitted for brevity:
bfa74976
RS
7757
7758@example
7759initdcl:
7760 declarator maybeasm '='
7761 init
7762 | declarator maybeasm
7763 ;
7764
7765notype_initdcl:
7766 notype_declarator maybeasm '='
7767 init
7768 | notype_declarator maybeasm
7769 ;
7770@end example
7771
7772@noindent
7773Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
7774cannot. The distinction between @code{declarator} and
7775@code{notype_declarator} is the same sort of thing.
7776
7777There is some similarity between this technique and a lexical tie-in
7778(described next), in that information which alters the lexical analysis is
7779changed during parsing by other parts of the program. The difference is
7780here the information is global, and is used for other purposes in the
7781program. A true lexical tie-in has a special-purpose flag controlled by
7782the syntactic context.
7783
342b8b6e 7784@node Lexical Tie-ins
bfa74976
RS
7785@section Lexical Tie-ins
7786@cindex lexical tie-in
7787
7788One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
7789which is set by Bison actions, whose purpose is to alter the way tokens are
7790parsed.
7791
7792For example, suppose we have a language vaguely like C, but with a special
7793construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
7794an expression in parentheses in which all integers are hexadecimal. In
7795particular, the token @samp{a1b} must be treated as an integer rather than
7796as an identifier if it appears in that context. Here is how you can do it:
7797
7798@example
7799@group
7800%@{
38a92d50
PE
7801 int hexflag;
7802 int yylex (void);
7803 void yyerror (char const *);
bfa74976
RS
7804%@}
7805%%
7806@dots{}
7807@end group
7808@group
7809expr: IDENTIFIER
7810 | constant
7811 | HEX '('
7812 @{ hexflag = 1; @}
7813 expr ')'
7814 @{ hexflag = 0;
7815 $$ = $4; @}
7816 | expr '+' expr
7817 @{ $$ = make_sum ($1, $3); @}
7818 @dots{}
7819 ;
7820@end group
7821
7822@group
7823constant:
7824 INTEGER
7825 | STRING
7826 ;
7827@end group
7828@end example
7829
7830@noindent
7831Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
7832it is nonzero, all integers are parsed in hexadecimal, and tokens starting
7833with letters are parsed as integers if possible.
7834
ff7571c0
JD
7835The declaration of @code{hexflag} shown in the prologue of the grammar
7836file is needed to make it accessible to the actions (@pxref{Prologue,
7837,The Prologue}). You must also write the code in @code{yylex} to obey
7838the flag.
bfa74976 7839
342b8b6e 7840@node Tie-in Recovery
bfa74976
RS
7841@section Lexical Tie-ins and Error Recovery
7842
7843Lexical tie-ins make strict demands on any error recovery rules you have.
7844@xref{Error Recovery}.
7845
7846The reason for this is that the purpose of an error recovery rule is to
7847abort the parsing of one construct and resume in some larger construct.
7848For example, in C-like languages, a typical error recovery rule is to skip
7849tokens until the next semicolon, and then start a new statement, like this:
7850
7851@example
7852stmt: expr ';'
7853 | IF '(' expr ')' stmt @{ @dots{} @}
7854 @dots{}
7855 error ';'
7856 @{ hexflag = 0; @}
7857 ;
7858@end example
7859
7860If there is a syntax error in the middle of a @samp{hex (@var{expr})}
7861construct, this error rule will apply, and then the action for the
7862completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
7863remain set for the entire rest of the input, or until the next @code{hex}
7864keyword, causing identifiers to be misinterpreted as integers.
7865
7866To avoid this problem the error recovery rule itself clears @code{hexflag}.
7867
7868There may also be an error recovery rule that works within expressions.
7869For example, there could be a rule which applies within parentheses
7870and skips to the close-parenthesis:
7871
7872@example
7873@group
7874expr: @dots{}
7875 | '(' expr ')'
7876 @{ $$ = $2; @}
7877 | '(' error ')'
7878 @dots{}
7879@end group
7880@end example
7881
7882If this rule acts within the @code{hex} construct, it is not going to abort
7883that construct (since it applies to an inner level of parentheses within
7884the construct). Therefore, it should not clear the flag: the rest of
7885the @code{hex} construct should be parsed with the flag still in effect.
7886
7887What if there is an error recovery rule which might abort out of the
7888@code{hex} construct or might not, depending on circumstances? There is no
7889way you can write the action to determine whether a @code{hex} construct is
7890being aborted or not. So if you are using a lexical tie-in, you had better
7891make sure your error recovery rules are not of this kind. Each rule must
7892be such that you can be sure that it always will, or always won't, have to
7893clear the flag.
7894
ec3bc396
AD
7895@c ================================================== Debugging Your Parser
7896
342b8b6e 7897@node Debugging
bfa74976 7898@chapter Debugging Your Parser
ec3bc396
AD
7899
7900Developing a parser can be a challenge, especially if you don't
7901understand the algorithm (@pxref{Algorithm, ,The Bison Parser
7902Algorithm}). Even so, sometimes a detailed description of the automaton
7903can help (@pxref{Understanding, , Understanding Your Parser}), or
7904tracing the execution of the parser can give some insight on why it
7905behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
7906
7907@menu
7908* Understanding:: Understanding the structure of your parser.
7909* Tracing:: Tracing the execution of your parser.
7910@end menu
7911
7912@node Understanding
7913@section Understanding Your Parser
7914
7915As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
7916Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
7917frequent than one would hope), looking at this automaton is required to
7918tune or simply fix a parser. Bison provides two different
35fe0834 7919representation of it, either textually or graphically (as a DOT file).
ec3bc396
AD
7920
7921The textual file is generated when the options @option{--report} or
7922@option{--verbose} are specified, see @xref{Invocation, , Invoking
7923Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
ff7571c0
JD
7924the parser implementation file name, and adding @samp{.output}
7925instead. Therefore, if the grammar file is @file{foo.y}, then the
7926parser implementation file is called @file{foo.tab.c} by default. As
7927a consequence, the verbose output file is called @file{foo.output}.
ec3bc396
AD
7928
7929The following grammar file, @file{calc.y}, will be used in the sequel:
7930
7931@example
7932%token NUM STR
7933%left '+' '-'
7934%left '*'
7935%%
7936exp: exp '+' exp
7937 | exp '-' exp
7938 | exp '*' exp
7939 | exp '/' exp
7940 | NUM
7941 ;
7942useless: STR;
7943%%
7944@end example
7945
88bce5a2
AD
7946@command{bison} reports:
7947
7948@example
8f0d265e
JD
7949calc.y: warning: 1 nonterminal useless in grammar
7950calc.y: warning: 1 rule useless in grammar
cff03fb2
JD
7951calc.y:11.1-7: warning: nonterminal useless in grammar: useless
7952calc.y:11.10-12: warning: rule useless in grammar: useless: STR
5a99098d 7953calc.y: conflicts: 7 shift/reduce
88bce5a2
AD
7954@end example
7955
7956When given @option{--report=state}, in addition to @file{calc.tab.c}, it
7957creates a file @file{calc.output} with contents detailed below. The
7958order of the output and the exact presentation might vary, but the
7959interpretation is the same.
ec3bc396
AD
7960
7961The first section includes details on conflicts that were solved thanks
7962to precedence and/or associativity:
7963
7964@example
7965Conflict in state 8 between rule 2 and token '+' resolved as reduce.
7966Conflict in state 8 between rule 2 and token '-' resolved as reduce.
7967Conflict in state 8 between rule 2 and token '*' resolved as shift.
7968@exdent @dots{}
7969@end example
7970
7971@noindent
7972The next section lists states that still have conflicts.
7973
7974@example
5a99098d
PE
7975State 8 conflicts: 1 shift/reduce
7976State 9 conflicts: 1 shift/reduce
7977State 10 conflicts: 1 shift/reduce
7978State 11 conflicts: 4 shift/reduce
ec3bc396
AD
7979@end example
7980
7981@noindent
7982@cindex token, useless
7983@cindex useless token
7984@cindex nonterminal, useless
7985@cindex useless nonterminal
7986@cindex rule, useless
7987@cindex useless rule
7988The next section reports useless tokens, nonterminal and rules. Useless
7989nonterminals and rules are removed in order to produce a smaller parser,
7990but useless tokens are preserved, since they might be used by the
d80fb37a 7991scanner (note the difference between ``useless'' and ``unused''
ec3bc396
AD
7992below):
7993
7994@example
d80fb37a 7995Nonterminals useless in grammar:
ec3bc396
AD
7996 useless
7997
d80fb37a 7998Terminals unused in grammar:
ec3bc396
AD
7999 STR
8000
cff03fb2 8001Rules useless in grammar:
ec3bc396
AD
8002#6 useless: STR;
8003@end example
8004
8005@noindent
8006The next section reproduces the exact grammar that Bison used:
8007
8008@example
8009Grammar
8010
8011 Number, Line, Rule
88bce5a2 8012 0 5 $accept -> exp $end
ec3bc396
AD
8013 1 5 exp -> exp '+' exp
8014 2 6 exp -> exp '-' exp
8015 3 7 exp -> exp '*' exp
8016 4 8 exp -> exp '/' exp
8017 5 9 exp -> NUM
8018@end example
8019
8020@noindent
8021and reports the uses of the symbols:
8022
8023@example
8024Terminals, with rules where they appear
8025
88bce5a2 8026$end (0) 0
ec3bc396
AD
8027'*' (42) 3
8028'+' (43) 1
8029'-' (45) 2
8030'/' (47) 4
8031error (256)
8032NUM (258) 5
8033
8034Nonterminals, with rules where they appear
8035
88bce5a2 8036$accept (8)
ec3bc396
AD
8037 on left: 0
8038exp (9)
8039 on left: 1 2 3 4 5, on right: 0 1 2 3 4
8040@end example
8041
8042@noindent
8043@cindex item
8044@cindex pointed rule
8045@cindex rule, pointed
8046Bison then proceeds onto the automaton itself, describing each state
8047with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
8048item is a production rule together with a point (marked by @samp{.})
8049that the input cursor.
8050
8051@example
8052state 0
8053
88bce5a2 8054 $accept -> . exp $ (rule 0)
ec3bc396 8055
2a8d363a 8056 NUM shift, and go to state 1
ec3bc396 8057
2a8d363a 8058 exp go to state 2
ec3bc396
AD
8059@end example
8060
8061This reads as follows: ``state 0 corresponds to being at the very
8062beginning of the parsing, in the initial rule, right before the start
8063symbol (here, @code{exp}). When the parser returns to this state right
8064after having reduced a rule that produced an @code{exp}, the control
8065flow jumps to state 2. If there is no such transition on a nonterminal
742e4900 8066symbol, and the lookahead is a @code{NUM}, then this token is shifted on
ec3bc396 8067the parse stack, and the control flow jumps to state 1. Any other
742e4900 8068lookahead triggers a syntax error.''
ec3bc396
AD
8069
8070@cindex core, item set
8071@cindex item set core
8072@cindex kernel, item set
8073@cindex item set core
8074Even though the only active rule in state 0 seems to be rule 0, the
742e4900 8075report lists @code{NUM} as a lookahead token because @code{NUM} can be
ec3bc396
AD
8076at the beginning of any rule deriving an @code{exp}. By default Bison
8077reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
8078you want to see more detail you can invoke @command{bison} with
8079@option{--report=itemset} to list all the items, include those that can
8080be derived:
8081
8082@example
8083state 0
8084
88bce5a2 8085 $accept -> . exp $ (rule 0)
ec3bc396
AD
8086 exp -> . exp '+' exp (rule 1)
8087 exp -> . exp '-' exp (rule 2)
8088 exp -> . exp '*' exp (rule 3)
8089 exp -> . exp '/' exp (rule 4)
8090 exp -> . NUM (rule 5)
8091
8092 NUM shift, and go to state 1
8093
8094 exp go to state 2
8095@end example
8096
8097@noindent
8098In the state 1...
8099
8100@example
8101state 1
8102
8103 exp -> NUM . (rule 5)
8104
2a8d363a 8105 $default reduce using rule 5 (exp)
ec3bc396
AD
8106@end example
8107
8108@noindent
742e4900 8109the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
ec3bc396
AD
8110(@samp{$default}), the parser will reduce it. If it was coming from
8111state 0, then, after this reduction it will return to state 0, and will
8112jump to state 2 (@samp{exp: go to state 2}).
8113
8114@example
8115state 2
8116
88bce5a2 8117 $accept -> exp . $ (rule 0)
ec3bc396
AD
8118 exp -> exp . '+' exp (rule 1)
8119 exp -> exp . '-' exp (rule 2)
8120 exp -> exp . '*' exp (rule 3)
8121 exp -> exp . '/' exp (rule 4)
8122
2a8d363a
AD
8123 $ shift, and go to state 3
8124 '+' shift, and go to state 4
8125 '-' shift, and go to state 5
8126 '*' shift, and go to state 6
8127 '/' shift, and go to state 7
ec3bc396
AD
8128@end example
8129
8130@noindent
8131In state 2, the automaton can only shift a symbol. For instance,
742e4900 8132because of the item @samp{exp -> exp . '+' exp}, if the lookahead if
ec3bc396
AD
8133@samp{+}, it will be shifted on the parse stack, and the automaton
8134control will jump to state 4, corresponding to the item @samp{exp -> exp
8135'+' . exp}. Since there is no default action, any other token than
6e649e65 8136those listed above will trigger a syntax error.
ec3bc396 8137
eb45ef3b 8138@cindex accepting state
ec3bc396
AD
8139The state 3 is named the @dfn{final state}, or the @dfn{accepting
8140state}:
8141
8142@example
8143state 3
8144
88bce5a2 8145 $accept -> exp $ . (rule 0)
ec3bc396 8146
2a8d363a 8147 $default accept
ec3bc396
AD
8148@end example
8149
8150@noindent
8151the initial rule is completed (the start symbol and the end
8152of input were read), the parsing exits successfully.
8153
8154The interpretation of states 4 to 7 is straightforward, and is left to
8155the reader.
8156
8157@example
8158state 4
8159
8160 exp -> exp '+' . exp (rule 1)
8161
2a8d363a 8162 NUM shift, and go to state 1
ec3bc396 8163
2a8d363a 8164 exp go to state 8
ec3bc396
AD
8165
8166state 5
8167
8168 exp -> exp '-' . exp (rule 2)
8169
2a8d363a 8170 NUM shift, and go to state 1
ec3bc396 8171
2a8d363a 8172 exp go to state 9
ec3bc396
AD
8173
8174state 6
8175
8176 exp -> exp '*' . exp (rule 3)
8177
2a8d363a 8178 NUM shift, and go to state 1
ec3bc396 8179
2a8d363a 8180 exp go to state 10
ec3bc396
AD
8181
8182state 7
8183
8184 exp -> exp '/' . exp (rule 4)
8185
2a8d363a 8186 NUM shift, and go to state 1
ec3bc396 8187
2a8d363a 8188 exp go to state 11
ec3bc396
AD
8189@end example
8190
5a99098d
PE
8191As was announced in beginning of the report, @samp{State 8 conflicts:
81921 shift/reduce}:
ec3bc396
AD
8193
8194@example
8195state 8
8196
8197 exp -> exp . '+' exp (rule 1)
8198 exp -> exp '+' exp . (rule 1)
8199 exp -> exp . '-' exp (rule 2)
8200 exp -> exp . '*' exp (rule 3)
8201 exp -> exp . '/' exp (rule 4)
8202
2a8d363a
AD
8203 '*' shift, and go to state 6
8204 '/' shift, and go to state 7
ec3bc396 8205
2a8d363a
AD
8206 '/' [reduce using rule 1 (exp)]
8207 $default reduce using rule 1 (exp)
ec3bc396
AD
8208@end example
8209
742e4900 8210Indeed, there are two actions associated to the lookahead @samp{/}:
ec3bc396
AD
8211either shifting (and going to state 7), or reducing rule 1. The
8212conflict means that either the grammar is ambiguous, or the parser lacks
8213information to make the right decision. Indeed the grammar is
8214ambiguous, as, since we did not specify the precedence of @samp{/}, the
8215sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
8216NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
8217NUM}, which corresponds to reducing rule 1.
8218
eb45ef3b 8219Because in deterministic parsing a single decision can be made, Bison
ec3bc396
AD
8220arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
8221Shift/Reduce Conflicts}. Discarded actions are reported in between
8222square brackets.
8223
8224Note that all the previous states had a single possible action: either
8225shifting the next token and going to the corresponding state, or
8226reducing a single rule. In the other cases, i.e., when shifting
8227@emph{and} reducing is possible or when @emph{several} reductions are
742e4900
JD
8228possible, the lookahead is required to select the action. State 8 is
8229one such state: if the lookahead is @samp{*} or @samp{/} then the action
ec3bc396
AD
8230is shifting, otherwise the action is reducing rule 1. In other words,
8231the first two items, corresponding to rule 1, are not eligible when the
742e4900 8232lookahead token is @samp{*}, since we specified that @samp{*} has higher
8dd162d3 8233precedence than @samp{+}. More generally, some items are eligible only
742e4900
JD
8234with some set of possible lookahead tokens. When run with
8235@option{--report=lookahead}, Bison specifies these lookahead tokens:
ec3bc396
AD
8236
8237@example
8238state 8
8239
88c78747 8240 exp -> exp . '+' exp (rule 1)
ec3bc396
AD
8241 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
8242 exp -> exp . '-' exp (rule 2)
8243 exp -> exp . '*' exp (rule 3)
8244 exp -> exp . '/' exp (rule 4)
8245
8246 '*' shift, and go to state 6
8247 '/' shift, and go to state 7
8248
8249 '/' [reduce using rule 1 (exp)]
8250 $default reduce using rule 1 (exp)
8251@end example
8252
8253The remaining states are similar:
8254
8255@example
8256state 9
8257
8258 exp -> exp . '+' exp (rule 1)
8259 exp -> exp . '-' exp (rule 2)
8260 exp -> exp '-' exp . (rule 2)
8261 exp -> exp . '*' exp (rule 3)
8262 exp -> exp . '/' exp (rule 4)
8263
2a8d363a
AD
8264 '*' shift, and go to state 6
8265 '/' shift, and go to state 7
ec3bc396 8266
2a8d363a
AD
8267 '/' [reduce using rule 2 (exp)]
8268 $default reduce using rule 2 (exp)
ec3bc396
AD
8269
8270state 10
8271
8272 exp -> exp . '+' exp (rule 1)
8273 exp -> exp . '-' exp (rule 2)
8274 exp -> exp . '*' exp (rule 3)
8275 exp -> exp '*' exp . (rule 3)
8276 exp -> exp . '/' exp (rule 4)
8277
2a8d363a 8278 '/' shift, and go to state 7
ec3bc396 8279
2a8d363a
AD
8280 '/' [reduce using rule 3 (exp)]
8281 $default reduce using rule 3 (exp)
ec3bc396
AD
8282
8283state 11
8284
8285 exp -> exp . '+' exp (rule 1)
8286 exp -> exp . '-' exp (rule 2)
8287 exp -> exp . '*' exp (rule 3)
8288 exp -> exp . '/' exp (rule 4)
8289 exp -> exp '/' exp . (rule 4)
8290
2a8d363a
AD
8291 '+' shift, and go to state 4
8292 '-' shift, and go to state 5
8293 '*' shift, and go to state 6
8294 '/' shift, and go to state 7
ec3bc396 8295
2a8d363a
AD
8296 '+' [reduce using rule 4 (exp)]
8297 '-' [reduce using rule 4 (exp)]
8298 '*' [reduce using rule 4 (exp)]
8299 '/' [reduce using rule 4 (exp)]
8300 $default reduce using rule 4 (exp)
ec3bc396
AD
8301@end example
8302
8303@noindent
fa7e68c3
PE
8304Observe that state 11 contains conflicts not only due to the lack of
8305precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
8306@samp{*}, but also because the
ec3bc396
AD
8307associativity of @samp{/} is not specified.
8308
8309
8310@node Tracing
8311@section Tracing Your Parser
bfa74976
RS
8312@findex yydebug
8313@cindex debugging
8314@cindex tracing the parser
8315
8316If a Bison grammar compiles properly but doesn't do what you want when it
8317runs, the @code{yydebug} parser-trace feature can help you figure out why.
8318
3ded9a63
AD
8319There are several means to enable compilation of trace facilities:
8320
8321@table @asis
8322@item the macro @code{YYDEBUG}
8323@findex YYDEBUG
8324Define the macro @code{YYDEBUG} to a nonzero value when you compile the
8a4281b9 8325parser. This is compliant with POSIX Yacc. You could use
3ded9a63
AD
8326@samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
8327YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
8328Prologue}).
8329
8330@item the option @option{-t}, @option{--debug}
8331Use the @samp{-t} option when you run Bison (@pxref{Invocation,
8a4281b9 8332,Invoking Bison}). This is POSIX compliant too.
3ded9a63
AD
8333
8334@item the directive @samp{%debug}
8335@findex %debug
fa819509
AD
8336Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison Declaration
8337Summary}). This Bison extension is maintained for backward
8338compatibility with previous versions of Bison.
8339
8340@item the variable @samp{parse.trace}
8341@findex %define parse.trace
35c1e5f0
JD
8342Add the @samp{%define parse.trace} directive (@pxref{%define
8343Summary,,parse.trace}), or pass the @option{-Dparse.trace} option
fa819509 8344(@pxref{Bison Options}). This is a Bison extension, which is especially
35c1e5f0
JD
8345useful for languages that don't use a preprocessor. Unless POSIX and Yacc
8346portability matter to you, this is the preferred solution.
3ded9a63
AD
8347@end table
8348
fa819509 8349We suggest that you always enable the trace option so that debugging is
3ded9a63 8350always possible.
bfa74976 8351
02a81e05 8352The trace facility outputs messages with macro calls of the form
e2742e46 8353@code{YYFPRINTF (stderr, @var{format}, @var{args})} where
f57a7536 8354@var{format} and @var{args} are the usual @code{printf} format and variadic
4947ebdb
PE
8355arguments. If you define @code{YYDEBUG} to a nonzero value but do not
8356define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
9c437126 8357and @code{YYFPRINTF} is defined to @code{fprintf}.
bfa74976
RS
8358
8359Once you have compiled the program with trace facilities, the way to
8360request a trace is to store a nonzero value in the variable @code{yydebug}.
8361You can do this by making the C code do it (in @code{main}, perhaps), or
8362you can alter the value with a C debugger.
8363
8364Each step taken by the parser when @code{yydebug} is nonzero produces a
8365line or two of trace information, written on @code{stderr}. The trace
8366messages tell you these things:
8367
8368@itemize @bullet
8369@item
8370Each time the parser calls @code{yylex}, what kind of token was read.
8371
8372@item
8373Each time a token is shifted, the depth and complete contents of the
8374state stack (@pxref{Parser States}).
8375
8376@item
8377Each time a rule is reduced, which rule it is, and the complete contents
8378of the state stack afterward.
8379@end itemize
8380
8381To make sense of this information, it helps to refer to the listing file
704a47c4
AD
8382produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
8383Bison}). This file shows the meaning of each state in terms of
8384positions in various rules, and also what each state will do with each
8385possible input token. As you read the successive trace messages, you
8386can see that the parser is functioning according to its specification in
8387the listing file. Eventually you will arrive at the place where
8388something undesirable happens, and you will see which parts of the
8389grammar are to blame.
bfa74976 8390
ff7571c0
JD
8391The parser implementation file is a C program and you can use C
8392debuggers on it, but it's not easy to interpret what it is doing. The
8393parser function is a finite-state machine interpreter, and aside from
8394the actions it executes the same code over and over. Only the values
8395of variables show where in the grammar it is working.
bfa74976
RS
8396
8397@findex YYPRINT
8398The debugging information normally gives the token type of each token
8399read, but not its semantic value. You can optionally define a macro
8400named @code{YYPRINT} to provide a way to print the value. If you define
8401@code{YYPRINT}, it should take three arguments. The parser will pass a
8402standard I/O stream, the numeric code for the token type, and the token
8403value (from @code{yylval}).
8404
8405Here is an example of @code{YYPRINT} suitable for the multi-function
f5f419de 8406calculator (@pxref{Mfcalc Declarations, ,Declarations for @code{mfcalc}}):
bfa74976
RS
8407
8408@smallexample
38a92d50
PE
8409%@{
8410 static void print_token_value (FILE *, int, YYSTYPE);
8411 #define YYPRINT(file, type, value) print_token_value (file, type, value)
8412%@}
8413
8414@dots{} %% @dots{} %% @dots{}
bfa74976
RS
8415
8416static void
831d3c99 8417print_token_value (FILE *file, int type, YYSTYPE value)
bfa74976
RS
8418@{
8419 if (type == VAR)
d3c4e709 8420 fprintf (file, "%s", value.tptr->name);
bfa74976 8421 else if (type == NUM)
d3c4e709 8422 fprintf (file, "%d", value.val);
bfa74976
RS
8423@}
8424@end smallexample
8425
ec3bc396
AD
8426@c ================================================= Invoking Bison
8427
342b8b6e 8428@node Invocation
bfa74976
RS
8429@chapter Invoking Bison
8430@cindex invoking Bison
8431@cindex Bison invocation
8432@cindex options for invoking Bison
8433
8434The usual way to invoke Bison is as follows:
8435
8436@example
8437bison @var{infile}
8438@end example
8439
8440Here @var{infile} is the grammar file name, which usually ends in
ff7571c0
JD
8441@samp{.y}. The parser implementation file's name is made by replacing
8442the @samp{.y} with @samp{.tab.c} and removing any leading directory.
8443Thus, the @samp{bison foo.y} file name yields @file{foo.tab.c}, and
8444the @samp{bison hack/foo.y} file name yields @file{foo.tab.c}. It's
8445also possible, in case you are writing C++ code instead of C in your
8446grammar file, to name it @file{foo.ypp} or @file{foo.y++}. Then, the
8447output files will take an extension like the given one as input
8448(respectively @file{foo.tab.cpp} and @file{foo.tab.c++}). This
8449feature takes effect with all options that manipulate file names like
234a3be3
AD
8450@samp{-o} or @samp{-d}.
8451
8452For example :
8453
8454@example
8455bison -d @var{infile.yxx}
8456@end example
84163231 8457@noindent
72d2299c 8458will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
234a3be3
AD
8459
8460@example
b56471a6 8461bison -d -o @var{output.c++} @var{infile.y}
234a3be3 8462@end example
84163231 8463@noindent
234a3be3
AD
8464will produce @file{output.c++} and @file{outfile.h++}.
8465
8a4281b9 8466For compatibility with POSIX, the standard Bison
397ec073
PE
8467distribution also contains a shell script called @command{yacc} that
8468invokes Bison with the @option{-y} option.
8469
bfa74976 8470@menu
13863333 8471* Bison Options:: All the options described in detail,
c827f760 8472 in alphabetical order by short options.
bfa74976 8473* Option Cross Key:: Alphabetical list of long options.
93dd49ab 8474* Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
bfa74976
RS
8475@end menu
8476
342b8b6e 8477@node Bison Options
bfa74976
RS
8478@section Bison Options
8479
8480Bison supports both traditional single-letter options and mnemonic long
8481option names. Long option names are indicated with @samp{--} instead of
8482@samp{-}. Abbreviations for option names are allowed as long as they
8483are unique. When a long option takes an argument, like
8484@samp{--file-prefix}, connect the option name and the argument with
8485@samp{=}.
8486
8487Here is a list of options that can be used with Bison, alphabetized by
8488short option. It is followed by a cross key alphabetized by long
8489option.
8490
89cab50d
AD
8491@c Please, keep this ordered as in `bison --help'.
8492@noindent
8493Operations modes:
8494@table @option
8495@item -h
8496@itemx --help
8497Print a summary of the command-line options to Bison and exit.
bfa74976 8498
89cab50d
AD
8499@item -V
8500@itemx --version
8501Print the version number of Bison and exit.
bfa74976 8502
f7ab6a50
PE
8503@item --print-localedir
8504Print the name of the directory containing locale-dependent data.
8505
a0de5091
JD
8506@item --print-datadir
8507Print the name of the directory containing skeletons and XSLT.
8508
89cab50d
AD
8509@item -y
8510@itemx --yacc
ff7571c0
JD
8511Act more like the traditional Yacc command. This can cause different
8512diagnostics to be generated, and may change behavior in other minor
8513ways. Most importantly, imitate Yacc's output file name conventions,
8514so that the parser implementation file is called @file{y.tab.c}, and
8515the other outputs are called @file{y.output} and @file{y.tab.h}.
8516Also, if generating a deterministic parser in C, generate
8517@code{#define} statements in addition to an @code{enum} to associate
8518token numbers with token names. Thus, the following shell script can
8519substitute for Yacc, and the Bison distribution contains such a script
8520for compatibility with POSIX:
bfa74976 8521
89cab50d 8522@example
397ec073 8523#! /bin/sh
26e06a21 8524bison -y "$@@"
89cab50d 8525@end example
54662697
PE
8526
8527The @option{-y}/@option{--yacc} option is intended for use with
8528traditional Yacc grammars. If your grammar uses a Bison extension
8529like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
8530this option is specified.
8531
1d5b3c08
JD
8532@item -W [@var{category}]
8533@itemx --warnings[=@var{category}]
118d4978
AD
8534Output warnings falling in @var{category}. @var{category} can be one
8535of:
8536@table @code
8537@item midrule-values
8e55b3aa
JD
8538Warn about mid-rule values that are set but not used within any of the actions
8539of the parent rule.
8540For example, warn about unused @code{$2} in:
118d4978
AD
8541
8542@example
8543exp: '1' @{ $$ = 1; @} '+' exp @{ $$ = $1 + $4; @};
8544@end example
8545
8e55b3aa
JD
8546Also warn about mid-rule values that are used but not set.
8547For example, warn about unset @code{$$} in the mid-rule action in:
118d4978
AD
8548
8549@example
8550 exp: '1' @{ $1 = 1; @} '+' exp @{ $$ = $2 + $4; @};
8551@end example
8552
8553These warnings are not enabled by default since they sometimes prove to
8554be false alarms in existing grammars employing the Yacc constructs
8e55b3aa 8555@code{$0} or @code{$-@var{n}} (where @var{n} is some positive integer).
118d4978
AD
8556
8557
8558@item yacc
8a4281b9 8559Incompatibilities with POSIX Yacc.
118d4978
AD
8560
8561@item all
8e55b3aa 8562All the warnings.
118d4978 8563@item none
8e55b3aa 8564Turn off all the warnings.
118d4978 8565@item error
8e55b3aa 8566Treat warnings as errors.
118d4978
AD
8567@end table
8568
8569A category can be turned off by prefixing its name with @samp{no-}. For
93d7dde9 8570instance, @option{-Wno-yacc} will hide the warnings about
8a4281b9 8571POSIX Yacc incompatibilities.
89cab50d
AD
8572@end table
8573
8574@noindent
8575Tuning the parser:
8576
8577@table @option
8578@item -t
8579@itemx --debug
ff7571c0
JD
8580In the parser implementation file, define the macro @code{YYDEBUG} to
85811 if it is not already defined, so that the debugging facilities are
8582compiled. @xref{Tracing, ,Tracing Your Parser}.
89cab50d 8583
58697c6d
AD
8584@item -D @var{name}[=@var{value}]
8585@itemx --define=@var{name}[=@var{value}]
17aed602 8586@itemx -F @var{name}[=@var{value}]
de5ab940
JD
8587@itemx --force-define=@var{name}[=@var{value}]
8588Each of these is equivalent to @samp{%define @var{name} "@var{value}"}
35c1e5f0 8589(@pxref{%define Summary}) except that Bison processes multiple
de5ab940
JD
8590definitions for the same @var{name} as follows:
8591
8592@itemize
8593@item
0b6d43c5
JD
8594Bison quietly ignores all command-line definitions for @var{name} except
8595the last.
de5ab940 8596@item
0b6d43c5
JD
8597If that command-line definition is specified by a @code{-D} or
8598@code{--define}, Bison reports an error for any @code{%define}
8599definition for @var{name}.
de5ab940 8600@item
0b6d43c5
JD
8601If that command-line definition is specified by a @code{-F} or
8602@code{--force-define} instead, Bison quietly ignores all @code{%define}
8603definitions for @var{name}.
8604@item
8605Otherwise, Bison reports an error if there are multiple @code{%define}
8606definitions for @var{name}.
de5ab940
JD
8607@end itemize
8608
8609You should avoid using @code{-F} and @code{--force-define} in your
ff7571c0
JD
8610make files unless you are confident that it is safe to quietly ignore
8611any conflicting @code{%define} that may be added to the grammar file.
58697c6d 8612
0e021770
PE
8613@item -L @var{language}
8614@itemx --language=@var{language}
8615Specify the programming language for the generated parser, as if
8616@code{%language} was specified (@pxref{Decl Summary, , Bison Declaration
59da312b 8617Summary}). Currently supported languages include C, C++, and Java.
e6e704dc 8618@var{language} is case-insensitive.
0e021770 8619
ed4d67dc
JD
8620This option is experimental and its effect may be modified in future
8621releases.
8622
89cab50d 8623@item --locations
d8988b2f 8624Pretend that @code{%locations} was specified. @xref{Decl Summary}.
89cab50d
AD
8625
8626@item -p @var{prefix}
8627@itemx --name-prefix=@var{prefix}
02975b9a 8628Pretend that @code{%name-prefix "@var{prefix}"} was specified.
d8988b2f 8629@xref{Decl Summary}.
bfa74976
RS
8630
8631@item -l
8632@itemx --no-lines
ff7571c0
JD
8633Don't put any @code{#line} preprocessor commands in the parser
8634implementation file. Ordinarily Bison puts them in the parser
8635implementation file so that the C compiler and debuggers will
8636associate errors with your source file, the grammar file. This option
8637causes them to associate errors with the parser implementation file,
8638treating it as an independent source file in its own right.
bfa74976 8639
e6e704dc
JD
8640@item -S @var{file}
8641@itemx --skeleton=@var{file}
a7867f53 8642Specify the skeleton to use, similar to @code{%skeleton}
e6e704dc
JD
8643(@pxref{Decl Summary, , Bison Declaration Summary}).
8644
ed4d67dc
JD
8645@c You probably don't need this option unless you are developing Bison.
8646@c You should use @option{--language} if you want to specify the skeleton for a
8647@c different language, because it is clearer and because it will always
8648@c choose the correct skeleton for non-deterministic or push parsers.
e6e704dc 8649
a7867f53
JD
8650If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
8651file in the Bison installation directory.
8652If it does, @var{file} is an absolute file name or a file name relative to the
8653current working directory.
8654This is similar to how most shells resolve commands.
8655
89cab50d
AD
8656@item -k
8657@itemx --token-table
d8988b2f 8658Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
89cab50d 8659@end table
bfa74976 8660
89cab50d
AD
8661@noindent
8662Adjust the output:
bfa74976 8663
89cab50d 8664@table @option
8e55b3aa 8665@item --defines[=@var{file}]
d8988b2f 8666Pretend that @code{%defines} was specified, i.e., write an extra output
6deb4447 8667file containing macro definitions for the token type names defined in
4bfd5e4e 8668the grammar, as well as a few other declarations. @xref{Decl Summary}.
931c7513 8669
8e55b3aa
JD
8670@item -d
8671This is the same as @code{--defines} except @code{-d} does not accept a
8672@var{file} argument since POSIX Yacc requires that @code{-d} can be bundled
8673with other short options.
342b8b6e 8674
89cab50d
AD
8675@item -b @var{file-prefix}
8676@itemx --file-prefix=@var{prefix}
9c437126 8677Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
72d2299c 8678for all Bison output file names. @xref{Decl Summary}.
bfa74976 8679
ec3bc396
AD
8680@item -r @var{things}
8681@itemx --report=@var{things}
8682Write an extra output file containing verbose description of the comma
8683separated list of @var{things} among:
8684
8685@table @code
8686@item state
8687Description of the grammar, conflicts (resolved and unresolved), and
eb45ef3b 8688parser's automaton.
ec3bc396 8689
742e4900 8690@item lookahead
ec3bc396 8691Implies @code{state} and augments the description of the automaton with
742e4900 8692each rule's lookahead set.
ec3bc396
AD
8693
8694@item itemset
8695Implies @code{state} and augments the description of the automaton with
8696the full set of items for each state, instead of its core only.
8697@end table
8698
1bb2bd75
JD
8699@item --report-file=@var{file}
8700Specify the @var{file} for the verbose description.
8701
bfa74976
RS
8702@item -v
8703@itemx --verbose
9c437126 8704Pretend that @code{%verbose} was specified, i.e., write an extra output
6deb4447 8705file containing verbose descriptions of the grammar and
72d2299c 8706parser. @xref{Decl Summary}.
bfa74976 8707
fa4d969f
PE
8708@item -o @var{file}
8709@itemx --output=@var{file}
ff7571c0 8710Specify the @var{file} for the parser implementation file.
bfa74976 8711
fa4d969f 8712The other output files' names are constructed from @var{file} as
d8988b2f 8713described under the @samp{-v} and @samp{-d} options.
342b8b6e 8714
a7c09cba 8715@item -g [@var{file}]
8e55b3aa 8716@itemx --graph[=@var{file}]
eb45ef3b 8717Output a graphical representation of the parser's
35fe0834 8718automaton computed by Bison, in @uref{http://www.graphviz.org/, Graphviz}
8a4281b9 8719@uref{http://www.graphviz.org/doc/info/lang.html, DOT} format.
8e55b3aa
JD
8720@code{@var{file}} is optional.
8721If omitted and the grammar file is @file{foo.y}, the output file will be
8722@file{foo.dot}.
59da312b 8723
a7c09cba 8724@item -x [@var{file}]
8e55b3aa 8725@itemx --xml[=@var{file}]
eb45ef3b 8726Output an XML report of the parser's automaton computed by Bison.
8e55b3aa 8727@code{@var{file}} is optional.
59da312b
JD
8728If omitted and the grammar file is @file{foo.y}, the output file will be
8729@file{foo.xml}.
8730(The current XML schema is experimental and may evolve.
8731More user feedback will help to stabilize it.)
bfa74976
RS
8732@end table
8733
342b8b6e 8734@node Option Cross Key
bfa74976
RS
8735@section Option Cross Key
8736
8737Here is a list of options, alphabetized by long option, to help you find
de5ab940 8738the corresponding short option and directive.
bfa74976 8739
de5ab940 8740@multitable {@option{--force-define=@var{name}[=@var{value}]}} {@option{-F @var{name}[=@var{value}]}} {@code{%nondeterministic-parser}}
a7c09cba 8741@headitem Long Option @tab Short Option @tab Bison Directive
f4101aa6 8742@include cross-options.texi
aa08666d 8743@end multitable
bfa74976 8744
93dd49ab
PE
8745@node Yacc Library
8746@section Yacc Library
8747
8748The Yacc library contains default implementations of the
8749@code{yyerror} and @code{main} functions. These default
8a4281b9 8750implementations are normally not useful, but POSIX requires
93dd49ab
PE
8751them. To use the Yacc library, link your program with the
8752@option{-ly} option. Note that Bison's implementation of the Yacc
8a4281b9 8753library is distributed under the terms of the GNU General
93dd49ab
PE
8754Public License (@pxref{Copying}).
8755
8756If you use the Yacc library's @code{yyerror} function, you should
8757declare @code{yyerror} as follows:
8758
8759@example
8760int yyerror (char const *);
8761@end example
8762
8763Bison ignores the @code{int} value returned by this @code{yyerror}.
8764If you use the Yacc library's @code{main} function, your
8765@code{yyparse} function should have the following type signature:
8766
8767@example
8768int yyparse (void);
8769@end example
8770
12545799
AD
8771@c ================================================= C++ Bison
8772
8405b70c
PB
8773@node Other Languages
8774@chapter Parsers Written In Other Languages
12545799
AD
8775
8776@menu
8777* C++ Parsers:: The interface to generate C++ parser classes
8405b70c 8778* Java Parsers:: The interface to generate Java parser classes
12545799
AD
8779@end menu
8780
8781@node C++ Parsers
8782@section C++ Parsers
8783
8784@menu
8785* C++ Bison Interface:: Asking for C++ parser generation
8786* C++ Semantic Values:: %union vs. C++
8787* C++ Location Values:: The position and location classes
8788* C++ Parser Interface:: Instantiating and running the parser
8789* C++ Scanner Interface:: Exchanges between yylex and parse
8405b70c 8790* A Complete C++ Example:: Demonstrating their use
12545799
AD
8791@end menu
8792
8793@node C++ Bison Interface
8794@subsection C++ Bison Interface
ed4d67dc 8795@c - %skeleton "lalr1.cc"
12545799
AD
8796@c - Always pure
8797@c - initial action
8798
eb45ef3b 8799The C++ deterministic parser is selected using the skeleton directive,
86e5b440
AD
8800@samp{%skeleton "lalr1.cc"}, or the synonymous command-line option
8801@option{--skeleton=lalr1.cc}.
e6e704dc 8802@xref{Decl Summary}.
0e021770 8803
793fbca5
JD
8804When run, @command{bison} will create several entities in the @samp{yy}
8805namespace.
67501061 8806@findex %define api.namespace
35c1e5f0
JD
8807Use the @samp{%define api.namespace} directive to change the namespace name,
8808see @ref{%define Summary,,api.namespace}. The various classes are generated
8809in the following files:
aa08666d 8810
12545799
AD
8811@table @file
8812@item position.hh
8813@itemx location.hh
8814The definition of the classes @code{position} and @code{location},
3cdc21cf 8815used for location tracking when enabled. @xref{C++ Location Values}.
12545799
AD
8816
8817@item stack.hh
8818An auxiliary class @code{stack} used by the parser.
8819
fa4d969f
PE
8820@item @var{file}.hh
8821@itemx @var{file}.cc
ff7571c0 8822(Assuming the extension of the grammar file was @samp{.yy}.) The
cd8b5791
AD
8823declaration and implementation of the C++ parser class. The basename
8824and extension of these two files follow the same rules as with regular C
8825parsers (@pxref{Invocation}).
12545799 8826
cd8b5791
AD
8827The header is @emph{mandatory}; you must either pass
8828@option{-d}/@option{--defines} to @command{bison}, or use the
12545799
AD
8829@samp{%defines} directive.
8830@end table
8831
8832All these files are documented using Doxygen; run @command{doxygen}
8833for a complete and accurate documentation.
8834
8835@node C++ Semantic Values
8836@subsection C++ Semantic Values
8837@c - No objects in unions
178e123e 8838@c - YYSTYPE
12545799
AD
8839@c - Printer and destructor
8840
3cdc21cf
AD
8841Bison supports two different means to handle semantic values in C++. One is
8842alike the C interface, and relies on unions (@pxref{C++ Unions}). As C++
8843practitioners know, unions are inconvenient in C++, therefore another
8844approach is provided, based on variants (@pxref{C++ Variants}).
8845
8846@menu
8847* C++ Unions:: Semantic values cannot be objects
8848* C++ Variants:: Using objects as semantic values
8849@end menu
8850
8851@node C++ Unions
8852@subsubsection C++ Unions
8853
12545799
AD
8854The @code{%union} directive works as for C, see @ref{Union Decl, ,The
8855Collection of Value Types}. In particular it produces a genuine
3cdc21cf 8856@code{union}, which have a few specific features in C++.
12545799
AD
8857@itemize @minus
8858@item
fb9712a9
AD
8859The type @code{YYSTYPE} is defined but its use is discouraged: rather
8860you should refer to the parser's encapsulated type
8861@code{yy::parser::semantic_type}.
12545799
AD
8862@item
8863Non POD (Plain Old Data) types cannot be used. C++ forbids any
8864instance of classes with constructors in unions: only @emph{pointers}
8865to such objects are allowed.
8866@end itemize
8867
8868Because objects have to be stored via pointers, memory is not
8869reclaimed automatically: using the @code{%destructor} directive is the
8870only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
8871Symbols}.
8872
3cdc21cf
AD
8873@node C++ Variants
8874@subsubsection C++ Variants
8875
8876Starting with version 2.6, Bison provides a @emph{variant} based
8877implementation of semantic values for C++. This alleviates all the
8878limitations reported in the previous section, and in particular, object
8879types can be used without pointers.
8880
8881To enable variant-based semantic values, set @code{%define} variable
35c1e5f0 8882@code{variant} (@pxref{%define Summary,, variant}). Once this defined,
3cdc21cf
AD
8883@code{%union} is ignored, and instead of using the name of the fields of the
8884@code{%union} to ``type'' the symbols, use genuine types.
8885
8886For instance, instead of
8887
8888@example
8889%union
8890@{
8891 int ival;
8892 std::string* sval;
8893@}
8894%token <ival> NUMBER;
8895%token <sval> STRING;
8896@end example
8897
8898@noindent
8899write
8900
8901@example
8902%token <int> NUMBER;
8903%token <std::string> STRING;
8904@end example
8905
8906@code{STRING} is no longer a pointer, which should fairly simplify the user
8907actions in the grammar and in the scanner (in particular the memory
8908management).
8909
8910Since C++ features destructors, and since it is customary to specialize
8911@code{operator<<} to support uniform printing of values, variants also
8912typically simplify Bison printers and destructors.
8913
8914Variants are stricter than unions. When based on unions, you may play any
8915dirty game with @code{yylval}, say storing an @code{int}, reading a
8916@code{char*}, and then storing a @code{double} in it. This is no longer
8917possible with variants: they must be initialized, then assigned to, and
8918eventually, destroyed.
8919
8920@deftypemethod {semantic_type} {T&} build<T> ()
8921Initialize, but leave empty. Returns the address where the actual value may
8922be stored. Requires that the variant was not initialized yet.
8923@end deftypemethod
8924
8925@deftypemethod {semantic_type} {T&} build<T> (const T& @var{t})
8926Initialize, and copy-construct from @var{t}.
8927@end deftypemethod
8928
8929
8930@strong{Warning}: We do not use Boost.Variant, for two reasons. First, it
8931appeared unacceptable to require Boost on the user's machine (i.e., the
8932machine on which the generated parser will be compiled, not the machine on
8933which @command{bison} was run). Second, for each possible semantic value,
8934Boost.Variant not only stores the value, but also a tag specifying its
8935type. But the parser already ``knows'' the type of the semantic value, so
8936that would be duplicating the information.
8937
8938Therefore we developed light-weight variants whose type tag is external (so
8939they are really like @code{unions} for C++ actually). But our code is much
8940less mature that Boost.Variant. So there is a number of limitations in
8941(the current implementation of) variants:
8942@itemize
8943@item
8944Alignment must be enforced: values should be aligned in memory according to
8945the most demanding type. Computing the smallest alignment possible requires
8946meta-programming techniques that are not currently implemented in Bison, and
8947therefore, since, as far as we know, @code{double} is the most demanding
8948type on all platforms, alignments are enforced for @code{double} whatever
8949types are actually used. This may waste space in some cases.
8950
8951@item
8952Our implementation is not conforming with strict aliasing rules. Alias
8953analysis is a technique used in optimizing compilers to detect when two
8954pointers are disjoint (they cannot ``meet''). Our implementation breaks
8955some of the rules that G++ 4.4 uses in its alias analysis, so @emph{strict
8956alias analysis must be disabled}. Use the option
8957@option{-fno-strict-aliasing} to compile the generated parser.
8958
8959@item
8960There might be portability issues we are not aware of.
8961@end itemize
8962
a6ca4ce2 8963As far as we know, these limitations @emph{can} be alleviated. All it takes
3cdc21cf 8964is some time and/or some talented C++ hacker willing to contribute to Bison.
12545799
AD
8965
8966@node C++ Location Values
8967@subsection C++ Location Values
8968@c - %locations
8969@c - class Position
8970@c - class Location
16dc6a9e 8971@c - %define filename_type "const symbol::Symbol"
12545799
AD
8972
8973When the directive @code{%locations} is used, the C++ parser supports
8974location tracking, see @ref{Locations, , Locations Overview}. Two
8975auxiliary classes define a @code{position}, a single point in a file,
8976and a @code{location}, a range composed of a pair of
8977@code{position}s (possibly spanning several files).
8978
fa4d969f 8979@deftypemethod {position} {std::string*} file
12545799
AD
8980The name of the file. It will always be handled as a pointer, the
8981parser will never duplicate nor deallocate it. As an experimental
8982feature you may change it to @samp{@var{type}*} using @samp{%define
16dc6a9e 8983filename_type "@var{type}"}.
12545799
AD
8984@end deftypemethod
8985
8986@deftypemethod {position} {unsigned int} line
8987The line, starting at 1.
8988@end deftypemethod
8989
8990@deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
8991Advance by @var{height} lines, resetting the column number.
8992@end deftypemethod
8993
8994@deftypemethod {position} {unsigned int} column
8995The column, starting at 0.
8996@end deftypemethod
8997
8998@deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
8999Advance by @var{width} columns, without changing the line number.
9000@end deftypemethod
9001
9002@deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{width})
9003@deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
9004@deftypemethodx {position} {position&} operator-= (const position& @var{pos}, int @var{width})
9005@deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{width})
9006Various forms of syntactic sugar for @code{columns}.
9007@end deftypemethod
9008
9009@deftypemethod {position} {position} operator<< (std::ostream @var{o}, const position& @var{p})
9010Report @var{p} on @var{o} like this:
fa4d969f
PE
9011@samp{@var{file}:@var{line}.@var{column}}, or
9012@samp{@var{line}.@var{column}} if @var{file} is null.
12545799
AD
9013@end deftypemethod
9014
9015@deftypemethod {location} {position} begin
9016@deftypemethodx {location} {position} end
9017The first, inclusive, position of the range, and the first beyond.
9018@end deftypemethod
9019
9020@deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
9021@deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
9022Advance the @code{end} position.
9023@end deftypemethod
9024
9025@deftypemethod {location} {location} operator+ (const location& @var{begin}, const location& @var{end})
9026@deftypemethodx {location} {location} operator+ (const location& @var{begin}, int @var{width})
9027@deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
9028Various forms of syntactic sugar.
9029@end deftypemethod
9030
9031@deftypemethod {location} {void} step ()
9032Move @code{begin} onto @code{end}.
9033@end deftypemethod
9034
9035
9036@node C++ Parser Interface
9037@subsection C++ Parser Interface
9038@c - define parser_class_name
9039@c - Ctor
9040@c - parse, error, set_debug_level, debug_level, set_debug_stream,
9041@c debug_stream.
9042@c - Reporting errors
9043
9044The output files @file{@var{output}.hh} and @file{@var{output}.cc}
9045declare and define the parser class in the namespace @code{yy}. The
9046class name defaults to @code{parser}, but may be changed using
16dc6a9e 9047@samp{%define parser_class_name "@var{name}"}. The interface of
9d9b8b70 9048this class is detailed below. It can be extended using the
12545799
AD
9049@code{%parse-param} feature: its semantics is slightly changed since
9050it describes an additional member of the parser class, and an
9051additional argument for its constructor.
9052
3cdc21cf
AD
9053@defcv {Type} {parser} {semantic_type}
9054@defcvx {Type} {parser} {location_type}
9055The types for semantic values and locations (if enabled).
9056@end defcv
9057
86e5b440
AD
9058@defcv {Type} {parser} {token}
9059A structure that contains (only) the definition of the tokens as the
9060@code{yytokentype} enumeration. To refer to the token @code{FOO}, the
9061scanner should use @code{yy::parser::token::FOO}. The scanner can use
9062@samp{typedef yy::parser::token token;} to ``import'' the token enumeration
9063(@pxref{Calc++ Scanner}).
9064@end defcv
9065
3cdc21cf
AD
9066@defcv {Type} {parser} {syntax_error}
9067This class derives from @code{std::runtime_error}. Throw instances of it
9068from user actions to raise parse errors. This is equivalent with first
9069invoking @code{error} to report the location and message of the syntax
9070error, and then to invoke @code{YYERROR} to enter the error-recovery mode.
9071But contrary to @code{YYERROR} which can only be invoked from user actions
9072(i.e., written in the action itself), the exception can be thrown from
9073function invoked from the user action.
8a0adb01 9074@end defcv
12545799
AD
9075
9076@deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
9077Build a new parser object. There are no arguments by default, unless
9078@samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
9079@end deftypemethod
9080
3cdc21cf
AD
9081@deftypemethod {syntax_error} {} syntax_error (const location_type& @var{l}, const std::string& @var{m})
9082@deftypemethodx {syntax_error} {} syntax_error (const std::string& @var{m})
9083Instantiate a syntax-error exception.
9084@end deftypemethod
9085
12545799
AD
9086@deftypemethod {parser} {int} parse ()
9087Run the syntactic analysis, and return 0 on success, 1 otherwise.
9088@end deftypemethod
9089
9090@deftypemethod {parser} {std::ostream&} debug_stream ()
9091@deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
9092Get or set the stream used for tracing the parsing. It defaults to
9093@code{std::cerr}.
9094@end deftypemethod
9095
9096@deftypemethod {parser} {debug_level_type} debug_level ()
9097@deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
9098Get or set the tracing level. Currently its value is either 0, no trace,
9d9b8b70 9099or nonzero, full tracing.
12545799
AD
9100@end deftypemethod
9101
9102@deftypemethod {parser} {void} error (const location_type& @var{l}, const std::string& @var{m})
3cdc21cf 9103@deftypemethodx {parser} {void} error (const std::string& @var{m})
12545799
AD
9104The definition for this member function must be supplied by the user:
9105the parser uses it to report a parser error occurring at @var{l},
3cdc21cf
AD
9106described by @var{m}. If location tracking is not enabled, the second
9107signature is used.
12545799
AD
9108@end deftypemethod
9109
9110
9111@node C++ Scanner Interface
9112@subsection C++ Scanner Interface
9113@c - prefix for yylex.
9114@c - Pure interface to yylex
9115@c - %lex-param
9116
9117The parser invokes the scanner by calling @code{yylex}. Contrary to C
9118parsers, C++ parsers are always pure: there is no point in using the
3cdc21cf
AD
9119@samp{%define api.pure} directive. The actual interface with @code{yylex}
9120depends whether you use unions, or variants.
12545799 9121
3cdc21cf
AD
9122@menu
9123* Split Symbols:: Passing symbols as two/three components
9124* Complete Symbols:: Making symbols a whole
9125@end menu
9126
9127@node Split Symbols
9128@subsubsection Split Symbols
9129
9130Therefore the interface is as follows.
9131
86e5b440
AD
9132@deftypemethod {parser} {int} yylex (semantic_type* @var{yylval}, location_type* @var{yylloc}, @var{type1} @var{arg1}, ...)
9133@deftypemethodx {parser} {int} yylex (semantic_type* @var{yylval}, @var{type1} @var{arg1}, ...)
3cdc21cf
AD
9134Return the next token. Its type is the return value, its semantic value and
9135location (if enabled) being @var{yylval} and @var{yylloc}. Invocations of
12545799
AD
9136@samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
9137@end deftypemethod
9138
3cdc21cf
AD
9139Note that when using variants, the interface for @code{yylex} is the same,
9140but @code{yylval} is handled differently.
9141
9142Regular union-based code in Lex scanner typically look like:
9143
9144@example
9145[0-9]+ @{
9146 yylval.ival = text_to_int (yytext);
9147 return yy::parser::INTEGER;
9148 @}
9149[a-z]+ @{
9150 yylval.sval = new std::string (yytext);
9151 return yy::parser::IDENTIFIER;
9152 @}
9153@end example
9154
9155Using variants, @code{yylval} is already constructed, but it is not
9156initialized. So the code would look like:
9157
9158@example
9159[0-9]+ @{
9160 yylval.build<int>() = text_to_int (yytext);
9161 return yy::parser::INTEGER;
9162 @}
9163[a-z]+ @{
9164 yylval.build<std::string> = yytext;
9165 return yy::parser::IDENTIFIER;
9166 @}
9167@end example
9168
9169@noindent
9170or
9171
9172@example
9173[0-9]+ @{
9174 yylval.build(text_to_int (yytext));
9175 return yy::parser::INTEGER;
9176 @}
9177[a-z]+ @{
9178 yylval.build(yytext);
9179 return yy::parser::IDENTIFIER;
9180 @}
9181@end example
9182
9183
9184@node Complete Symbols
9185@subsubsection Complete Symbols
9186
9187If you specified both @code{%define variant} and @code{%define lex_symbol},
9188the @code{parser} class also defines the class @code{parser::symbol_type}
9189which defines a @emph{complete} symbol, aggregating its type (i.e., the
9190traditional value returned by @code{yylex}), its semantic value (i.e., the
9191value passed in @code{yylval}, and possibly its location (@code{yylloc}).
9192
9193@deftypemethod {symbol_type} {} symbol_type (token_type @var{type}, const semantic_type& @var{value}, const location_type& @var{location})
9194Build a complete terminal symbol which token type is @var{type}, and which
9195semantic value is @var{value}. If location tracking is enabled, also pass
9196the @var{location}.
9197@end deftypemethod
9198
9199This interface is low-level and should not be used for two reasons. First,
9200it is inconvenient, as you still have to build the semantic value, which is
9201a variant, and second, because consistency is not enforced: as with unions,
9202it is still possible to give an integer as semantic value for a string.
9203
9204So for each token type, Bison generates named constructors as follows.
9205
9206@deftypemethod {symbol_type} {} make_@var{token} (const @var{value_type}& @var{value}, const location_type& @var{location})
9207@deftypemethodx {symbol_type} {} make_@var{token} (const location_type& @var{location})
9208Build a complete terminal symbol for the token type @var{token} (not
9209including the @code{api.tokens.prefix}) whose possible semantic value is
9210@var{value} of adequate @var{value_type}. If location tracking is enabled,
9211also pass the @var{location}.
9212@end deftypemethod
9213
9214For instance, given the following declarations:
9215
9216@example
9217%define api.tokens.prefix "TOK_"
9218%token <std::string> IDENTIFIER;
9219%token <int> INTEGER;
9220%token COLON;
9221@end example
9222
9223@noindent
9224Bison generates the following functions:
9225
9226@example
9227symbol_type make_IDENTIFIER(const std::string& v,
9228 const location_type& l);
9229symbol_type make_INTEGER(const int& v,
9230 const location_type& loc);
9231symbol_type make_COLON(const location_type& loc);
9232@end example
9233
9234@noindent
9235which should be used in a Lex-scanner as follows.
9236
9237@example
9238[0-9]+ return yy::parser::make_INTEGER(text_to_int (yytext), loc);
9239[a-z]+ return yy::parser::make_IDENTIFIER(yytext, loc);
9240":" return yy::parser::make_COLON(loc);
9241@end example
9242
9243Tokens that do not have an identifier are not accessible: you cannot simply
9244use characters such as @code{':'}, they must be declared with @code{%token}.
12545799
AD
9245
9246@node A Complete C++ Example
8405b70c 9247@subsection A Complete C++ Example
12545799
AD
9248
9249This section demonstrates the use of a C++ parser with a simple but
9250complete example. This example should be available on your system,
3cdc21cf 9251ready to compile, in the directory @dfn{.../bison/examples/calc++}. It
12545799
AD
9252focuses on the use of Bison, therefore the design of the various C++
9253classes is very naive: no accessors, no encapsulation of members etc.
9254We will use a Lex scanner, and more precisely, a Flex scanner, to
3cdc21cf 9255demonstrate the various interactions. A hand-written scanner is
12545799
AD
9256actually easier to interface with.
9257
9258@menu
9259* Calc++ --- C++ Calculator:: The specifications
9260* Calc++ Parsing Driver:: An active parsing context
9261* Calc++ Parser:: A parser class
9262* Calc++ Scanner:: A pure C++ Flex scanner
9263* Calc++ Top Level:: Conducting the band
9264@end menu
9265
9266@node Calc++ --- C++ Calculator
8405b70c 9267@subsubsection Calc++ --- C++ Calculator
12545799
AD
9268
9269Of course the grammar is dedicated to arithmetics, a single
9d9b8b70 9270expression, possibly preceded by variable assignments. An
12545799
AD
9271environment containing possibly predefined variables such as
9272@code{one} and @code{two}, is exchanged with the parser. An example
9273of valid input follows.
9274
9275@example
9276three := 3
9277seven := one + two * three
9278seven * seven
9279@end example
9280
9281@node Calc++ Parsing Driver
8405b70c 9282@subsubsection Calc++ Parsing Driver
12545799
AD
9283@c - An env
9284@c - A place to store error messages
9285@c - A place for the result
9286
9287To support a pure interface with the parser (and the scanner) the
9288technique of the ``parsing context'' is convenient: a structure
9289containing all the data to exchange. Since, in addition to simply
9290launch the parsing, there are several auxiliary tasks to execute (open
9291the file for parsing, instantiate the parser etc.), we recommend
9292transforming the simple parsing context structure into a fully blown
9293@dfn{parsing driver} class.
9294
9295The declaration of this driver class, @file{calc++-driver.hh}, is as
9296follows. The first part includes the CPP guard and imports the
fb9712a9
AD
9297required standard library components, and the declaration of the parser
9298class.
12545799 9299
1c59e0a1 9300@comment file: calc++-driver.hh
12545799
AD
9301@example
9302#ifndef CALCXX_DRIVER_HH
9303# define CALCXX_DRIVER_HH
9304# include <string>
9305# include <map>
fb9712a9 9306# include "calc++-parser.hh"
12545799
AD
9307@end example
9308
12545799
AD
9309
9310@noindent
9311Then comes the declaration of the scanning function. Flex expects
9312the signature of @code{yylex} to be defined in the macro
9313@code{YY_DECL}, and the C++ parser expects it to be declared. We can
9314factor both as follows.
1c59e0a1
AD
9315
9316@comment file: calc++-driver.hh
12545799 9317@example
3dc5e96b 9318// Tell Flex the lexer's prototype ...
3cdc21cf
AD
9319# define YY_DECL \
9320 yy::calcxx_parser::symbol_type yylex (calcxx_driver& driver)
12545799
AD
9321// ... and declare it for the parser's sake.
9322YY_DECL;
9323@end example
9324
9325@noindent
9326The @code{calcxx_driver} class is then declared with its most obvious
9327members.
9328
1c59e0a1 9329@comment file: calc++-driver.hh
12545799
AD
9330@example
9331// Conducting the whole scanning and parsing of Calc++.
9332class calcxx_driver
9333@{
9334public:
9335 calcxx_driver ();
9336 virtual ~calcxx_driver ();
9337
9338 std::map<std::string, int> variables;
9339
9340 int result;
9341@end example
9342
9343@noindent
3cdc21cf
AD
9344To encapsulate the coordination with the Flex scanner, it is useful to have
9345member functions to open and close the scanning phase.
12545799 9346
1c59e0a1 9347@comment file: calc++-driver.hh
12545799
AD
9348@example
9349 // Handling the scanner.
9350 void scan_begin ();
9351 void scan_end ();
9352 bool trace_scanning;
9353@end example
9354
9355@noindent
9356Similarly for the parser itself.
9357
1c59e0a1 9358@comment file: calc++-driver.hh
12545799 9359@example
3cdc21cf
AD
9360 // Run the parser on file F.
9361 // Return 0 on success.
bb32f4f2 9362 int parse (const std::string& f);
3cdc21cf
AD
9363 // The name of the file being parsed.
9364 // Used later to pass the file name to the location tracker.
12545799 9365 std::string file;
3cdc21cf 9366 // Whether parser traces should be generated.
12545799
AD
9367 bool trace_parsing;
9368@end example
9369
9370@noindent
9371To demonstrate pure handling of parse errors, instead of simply
9372dumping them on the standard error output, we will pass them to the
9373compiler driver using the following two member functions. Finally, we
9374close the class declaration and CPP guard.
9375
1c59e0a1 9376@comment file: calc++-driver.hh
12545799
AD
9377@example
9378 // Error handling.
9379 void error (const yy::location& l, const std::string& m);
9380 void error (const std::string& m);
9381@};
9382#endif // ! CALCXX_DRIVER_HH
9383@end example
9384
9385The implementation of the driver is straightforward. The @code{parse}
9386member function deserves some attention. The @code{error} functions
9387are simple stubs, they should actually register the located error
9388messages and set error state.
9389
1c59e0a1 9390@comment file: calc++-driver.cc
12545799
AD
9391@example
9392#include "calc++-driver.hh"
9393#include "calc++-parser.hh"
9394
9395calcxx_driver::calcxx_driver ()
9396 : trace_scanning (false), trace_parsing (false)
9397@{
9398 variables["one"] = 1;
9399 variables["two"] = 2;
9400@}
9401
9402calcxx_driver::~calcxx_driver ()
9403@{
9404@}
9405
bb32f4f2 9406int
12545799
AD
9407calcxx_driver::parse (const std::string &f)
9408@{
9409 file = f;
9410 scan_begin ();
9411 yy::calcxx_parser parser (*this);
9412 parser.set_debug_level (trace_parsing);
bb32f4f2 9413 int res = parser.parse ();
12545799 9414 scan_end ();
bb32f4f2 9415 return res;
12545799
AD
9416@}
9417
9418void
9419calcxx_driver::error (const yy::location& l, const std::string& m)
9420@{
9421 std::cerr << l << ": " << m << std::endl;
9422@}
9423
9424void
9425calcxx_driver::error (const std::string& m)
9426@{
9427 std::cerr << m << std::endl;
9428@}
9429@end example
9430
9431@node Calc++ Parser
8405b70c 9432@subsubsection Calc++ Parser
12545799 9433
ff7571c0
JD
9434The grammar file @file{calc++-parser.yy} starts by asking for the C++
9435deterministic parser skeleton, the creation of the parser header file,
9436and specifies the name of the parser class. Because the C++ skeleton
9437changed several times, it is safer to require the version you designed
9438the grammar for.
1c59e0a1
AD
9439
9440@comment file: calc++-parser.yy
12545799 9441@example
ed4d67dc 9442%skeleton "lalr1.cc" /* -*- C++ -*- */
e6e704dc 9443%require "@value{VERSION}"
12545799 9444%defines
16dc6a9e 9445%define parser_class_name "calcxx_parser"
fb9712a9
AD
9446@end example
9447
3cdc21cf
AD
9448@noindent
9449@findex %define variant
9450@findex %define lex_symbol
9451This example will use genuine C++ objects as semantic values, therefore, we
9452require the variant-based interface. To make sure we properly use it, we
9453enable assertions. To fully benefit from type-safety and more natural
9454definition of ``symbol'', we enable @code{lex_symbol}.
9455
9456@comment file: calc++-parser.yy
9457@example
9458%define variant
9459%define parse.assert
9460%define lex_symbol
9461@end example
9462
fb9712a9 9463@noindent
16dc6a9e 9464@findex %code requires
3cdc21cf
AD
9465Then come the declarations/inclusions needed by the semantic values.
9466Because the parser uses the parsing driver and reciprocally, both would like
a6ca4ce2 9467to include the header of the other, which is, of course, insane. This
3cdc21cf 9468mutual dependency will be broken using forward declarations. Because the
fb9712a9 9469driver's header needs detailed knowledge about the parser class (in
3cdc21cf 9470particular its inner types), it is the parser's header which will use a
e0c07222 9471forward declaration of the driver. @xref{%code Summary}.
fb9712a9
AD
9472
9473@comment file: calc++-parser.yy
9474@example
3cdc21cf
AD
9475%code requires
9476@{
12545799 9477# include <string>
fb9712a9 9478class calcxx_driver;
9bc0dd67 9479@}
12545799
AD
9480@end example
9481
9482@noindent
9483The driver is passed by reference to the parser and to the scanner.
9484This provides a simple but effective pure interface, not relying on
9485global variables.
9486
1c59e0a1 9487@comment file: calc++-parser.yy
12545799
AD
9488@example
9489// The parsing context.
2055a44e 9490%param @{ calcxx_driver& driver @}
12545799
AD
9491@end example
9492
9493@noindent
2055a44e 9494Then we request location tracking, and initialize the
f50bfcd6 9495first location's file name. Afterward new locations are computed
12545799 9496relatively to the previous locations: the file name will be
2055a44e 9497propagated.
12545799 9498
1c59e0a1 9499@comment file: calc++-parser.yy
12545799
AD
9500@example
9501%locations
9502%initial-action
9503@{
9504 // Initialize the initial location.
b47dbebe 9505 @@$.begin.filename = @@$.end.filename = &driver.file;
12545799
AD
9506@};
9507@end example
9508
9509@noindent
2055a44e 9510Use the following two directives to enable parser tracing and verbose
12545799
AD
9511error messages.
9512
1c59e0a1 9513@comment file: calc++-parser.yy
12545799 9514@example
fa819509 9515%define parse.trace
cf499cff 9516%define parse.error verbose
12545799
AD
9517@end example
9518
fb9712a9 9519@noindent
136a0f76
PB
9520@findex %code
9521The code between @samp{%code @{} and @samp{@}} is output in the
34f98f46 9522@file{*.cc} file; it needs detailed knowledge about the driver.
fb9712a9
AD
9523
9524@comment file: calc++-parser.yy
9525@example
3cdc21cf
AD
9526%code
9527@{
fb9712a9 9528# include "calc++-driver.hh"
34f98f46 9529@}
fb9712a9
AD
9530@end example
9531
9532
12545799
AD
9533@noindent
9534The token numbered as 0 corresponds to end of file; the following line
99c08fb6 9535allows for nicer error messages referring to ``end of file'' instead of
35c1e5f0
JD
9536``$end''. Similarly user friendly names are provided for each symbol. To
9537avoid name clashes in the generated files (@pxref{Calc++ Scanner}), prefix
9538tokens with @code{TOK_} (@pxref{%define Summary,,api.tokens.prefix}).
12545799 9539
1c59e0a1 9540@comment file: calc++-parser.yy
12545799 9541@example
4c6622c2 9542%define api.tokens.prefix "TOK_"
3cdc21cf
AD
9543%token
9544 END 0 "end of file"
9545 ASSIGN ":="
9546 MINUS "-"
9547 PLUS "+"
9548 STAR "*"
9549 SLASH "/"
9550 LPAREN "("
9551 RPAREN ")"
9552;
12545799
AD
9553@end example
9554
9555@noindent
3cdc21cf
AD
9556Since we use variant-based semantic values, @code{%union} is not used, and
9557both @code{%type} and @code{%token} expect genuine types, as opposed to type
9558tags.
12545799 9559
1c59e0a1 9560@comment file: calc++-parser.yy
12545799 9561@example
3cdc21cf
AD
9562%token <std::string> IDENTIFIER "identifier"
9563%token <int> NUMBER "number"
9564%type <int> exp
9565@end example
9566
9567@noindent
9568No @code{%destructor} is needed to enable memory deallocation during error
9569recovery; the memory, for strings for instance, will be reclaimed by the
9570regular destructors. All the values are printed using their
9571@code{operator<<}.
12545799 9572
3cdc21cf
AD
9573@c FIXME: Document %printer, and mention that it takes a braced-code operand.
9574@comment file: calc++-parser.yy
9575@example
9576%printer @{ debug_stream () << $$; @} <*>;
12545799
AD
9577@end example
9578
9579@noindent
3cdc21cf
AD
9580The grammar itself is straightforward (@pxref{Location Tracking Calc, ,
9581Location Tracking Calculator: @code{ltcalc}}).
12545799 9582
1c59e0a1 9583@comment file: calc++-parser.yy
12545799
AD
9584@example
9585%%
9586%start unit;
9587unit: assignments exp @{ driver.result = $2; @};
9588
99c08fb6
AD
9589assignments:
9590 assignments assignment @{@}
9591| /* Nothing. */ @{@};
12545799 9592
3dc5e96b 9593assignment:
3cdc21cf 9594 "identifier" ":=" exp @{ driver.variables[$1] = $3; @};
12545799 9595
3cdc21cf
AD
9596%left "+" "-";
9597%left "*" "/";
99c08fb6 9598exp:
3cdc21cf
AD
9599 exp "+" exp @{ $$ = $1 + $3; @}
9600| exp "-" exp @{ $$ = $1 - $3; @}
9601| exp "*" exp @{ $$ = $1 * $3; @}
9602| exp "/" exp @{ $$ = $1 / $3; @}
298e8ad9 9603| "(" exp ")" @{ std::swap ($$, $2); @}
3cdc21cf 9604| "identifier" @{ $$ = driver.variables[$1]; @}
298e8ad9 9605| "number" @{ std::swap ($$, $1); @};
12545799
AD
9606%%
9607@end example
9608
9609@noindent
9610Finally the @code{error} member function registers the errors to the
9611driver.
9612
1c59e0a1 9613@comment file: calc++-parser.yy
12545799
AD
9614@example
9615void
3cdc21cf 9616yy::calcxx_parser::error (const location_type& l,
1c59e0a1 9617 const std::string& m)
12545799
AD
9618@{
9619 driver.error (l, m);
9620@}
9621@end example
9622
9623@node Calc++ Scanner
8405b70c 9624@subsubsection Calc++ Scanner
12545799
AD
9625
9626The Flex scanner first includes the driver declaration, then the
9627parser's to get the set of defined tokens.
9628
1c59e0a1 9629@comment file: calc++-scanner.ll
12545799
AD
9630@example
9631%@{ /* -*- C++ -*- */
3c248d70
AD
9632# include <cerrno>
9633# include <climits>
3cdc21cf 9634# include <cstdlib>
12545799
AD
9635# include <string>
9636# include "calc++-driver.hh"
9637# include "calc++-parser.hh"
eaea13f5 9638
3cdc21cf
AD
9639// Work around an incompatibility in flex (at least versions
9640// 2.5.31 through 2.5.33): it generates code that does
9641// not conform to C89. See Debian bug 333231
9642// <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>.
7870f699
PE
9643# undef yywrap
9644# define yywrap() 1
eaea13f5 9645
3cdc21cf
AD
9646// The location of the current token.
9647static yy::location loc;
12545799
AD
9648%@}
9649@end example
9650
9651@noindent
9652Because there is no @code{#include}-like feature we don't need
9653@code{yywrap}, we don't need @code{unput} either, and we parse an
9654actual file, this is not an interactive session with the user.
3cdc21cf 9655Finally, we enable scanner tracing.
12545799 9656
1c59e0a1 9657@comment file: calc++-scanner.ll
12545799
AD
9658@example
9659%option noyywrap nounput batch debug
9660@end example
9661
9662@noindent
9663Abbreviations allow for more readable rules.
9664
1c59e0a1 9665@comment file: calc++-scanner.ll
12545799
AD
9666@example
9667id [a-zA-Z][a-zA-Z_0-9]*
9668int [0-9]+
9669blank [ \t]
9670@end example
9671
9672@noindent
9d9b8b70 9673The following paragraph suffices to track locations accurately. Each
12545799 9674time @code{yylex} is invoked, the begin position is moved onto the end
3cdc21cf
AD
9675position. Then when a pattern is matched, its width is added to the end
9676column. When matching ends of lines, the end
12545799
AD
9677cursor is adjusted, and each time blanks are matched, the begin cursor
9678is moved onto the end cursor to effectively ignore the blanks
9679preceding tokens. Comments would be treated equally.
9680
1c59e0a1 9681@comment file: calc++-scanner.ll
12545799 9682@example
828c373b 9683%@{
3cdc21cf
AD
9684 // Code run each time a pattern is matched.
9685 # define YY_USER_ACTION loc.columns (yyleng);
828c373b 9686%@}
12545799
AD
9687%%
9688%@{
3cdc21cf
AD
9689 // Code run each time yylex is called.
9690 loc.step ();
12545799 9691%@}
3cdc21cf
AD
9692@{blank@}+ loc.step ();
9693[\n]+ loc.lines (yyleng); loc.step ();
12545799
AD
9694@end example
9695
9696@noindent
3cdc21cf 9697The rules are simple. The driver is used to report errors.
12545799 9698
1c59e0a1 9699@comment file: calc++-scanner.ll
12545799 9700@example
3cdc21cf
AD
9701"-" return yy::calcxx_parser::make_MINUS(loc);
9702"+" return yy::calcxx_parser::make_PLUS(loc);
9703"*" return yy::calcxx_parser::make_STAR(loc);
9704"/" return yy::calcxx_parser::make_SLASH(loc);
9705"(" return yy::calcxx_parser::make_LPAREN(loc);
9706")" return yy::calcxx_parser::make_RPAREN(loc);
9707":=" return yy::calcxx_parser::make_ASSIGN(loc);
9708
04098407
PE
9709@{int@} @{
9710 errno = 0;
9711 long n = strtol (yytext, NULL, 10);
9712 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
3cdc21cf
AD
9713 driver.error (loc, "integer is out of range");
9714 return yy::calcxx_parser::make_NUMBER(n, loc);
04098407 9715@}
3cdc21cf
AD
9716@{id@} return yy::calcxx_parser::make_IDENTIFIER(yytext, loc);
9717. driver.error (loc, "invalid character");
9718<<EOF>> return yy::calcxx_parser::make_END(loc);
12545799
AD
9719%%
9720@end example
9721
9722@noindent
3cdc21cf 9723Finally, because the scanner-related driver's member-functions depend
12545799
AD
9724on the scanner's data, it is simpler to implement them in this file.
9725
1c59e0a1 9726@comment file: calc++-scanner.ll
12545799
AD
9727@example
9728void
9729calcxx_driver::scan_begin ()
9730@{
9731 yy_flex_debug = trace_scanning;
bb32f4f2
AD
9732 if (file == "-")
9733 yyin = stdin;
9734 else if (!(yyin = fopen (file.c_str (), "r")))
9735 @{
3cdc21cf 9736 error (std::string ("cannot open ") + file + ": " + strerror(errno));
bb32f4f2
AD
9737 exit (1);
9738 @}
12545799
AD
9739@}
9740
9741void
9742calcxx_driver::scan_end ()
9743@{
9744 fclose (yyin);
9745@}
9746@end example
9747
9748@node Calc++ Top Level
8405b70c 9749@subsubsection Calc++ Top Level
12545799
AD
9750
9751The top level file, @file{calc++.cc}, poses no problem.
9752
1c59e0a1 9753@comment file: calc++.cc
12545799
AD
9754@example
9755#include <iostream>
9756#include "calc++-driver.hh"
9757
9758int
fa4d969f 9759main (int argc, char *argv[])
12545799 9760@{
414c76a4 9761 int res = 0;
12545799
AD
9762 calcxx_driver driver;
9763 for (++argv; argv[0]; ++argv)
9764 if (*argv == std::string ("-p"))
9765 driver.trace_parsing = true;
9766 else if (*argv == std::string ("-s"))
9767 driver.trace_scanning = true;
bb32f4f2
AD
9768 else if (!driver.parse (*argv))
9769 std::cout << driver.result << std::endl;
414c76a4
AD
9770 else
9771 res = 1;
9772 return res;
12545799
AD
9773@}
9774@end example
9775
8405b70c
PB
9776@node Java Parsers
9777@section Java Parsers
9778
9779@menu
f5f419de
DJ
9780* Java Bison Interface:: Asking for Java parser generation
9781* Java Semantic Values:: %type and %token vs. Java
9782* Java Location Values:: The position and location classes
9783* Java Parser Interface:: Instantiating and running the parser
9784* Java Scanner Interface:: Specifying the scanner for the parser
9785* Java Action Features:: Special features for use in actions
9786* Java Differences:: Differences between C/C++ and Java Grammars
9787* Java Declarations Summary:: List of Bison declarations used with Java
8405b70c
PB
9788@end menu
9789
9790@node Java Bison Interface
9791@subsection Java Bison Interface
9792@c - %language "Java"
8405b70c 9793
59da312b
JD
9794(The current Java interface is experimental and may evolve.
9795More user feedback will help to stabilize it.)
9796
e254a580
DJ
9797The Java parser skeletons are selected using the @code{%language "Java"}
9798directive or the @option{-L java}/@option{--language=java} option.
8405b70c 9799
e254a580 9800@c FIXME: Documented bug.
ff7571c0
JD
9801When generating a Java parser, @code{bison @var{basename}.y} will
9802create a single Java source file named @file{@var{basename}.java}
9803containing the parser implementation. Using a grammar file without a
9804@file{.y} suffix is currently broken. The basename of the parser
9805implementation file can be changed by the @code{%file-prefix}
9806directive or the @option{-p}/@option{--name-prefix} option. The
9807entire parser implementation file name can be changed by the
9808@code{%output} directive or the @option{-o}/@option{--output} option.
9809The parser implementation file contains a single class for the parser.
8405b70c 9810
e254a580 9811You can create documentation for generated parsers using Javadoc.
8405b70c 9812
e254a580
DJ
9813Contrary to C parsers, Java parsers do not use global variables; the
9814state of the parser is always local to an instance of the parser class.
9815Therefore, all Java parsers are ``pure'', and the @code{%pure-parser}
67501061 9816and @samp{%define api.pure} directives does not do anything when used in
e254a580 9817Java.
8405b70c 9818
e254a580 9819Push parsers are currently unsupported in Java and @code{%define
67212941 9820api.push-pull} have no effect.
01b477c6 9821
8a4281b9 9822GLR parsers are currently unsupported in Java. Do not use the
e254a580
DJ
9823@code{glr-parser} directive.
9824
9825No header file can be generated for Java parsers. Do not use the
9826@code{%defines} directive or the @option{-d}/@option{--defines} options.
9827
9828@c FIXME: Possible code change.
fa819509
AD
9829Currently, support for tracing is always compiled
9830in. Thus the @samp{%define parse.trace} and @samp{%token-table}
9831directives and the
e254a580
DJ
9832@option{-t}/@option{--debug} and @option{-k}/@option{--token-table}
9833options have no effect. This may change in the future to eliminate
fa819509
AD
9834unused code in the generated parser, so use @samp{%define parse.trace}
9835explicitly
1979121c 9836if needed. Also, in the future the
e254a580
DJ
9837@code{%token-table} directive might enable a public interface to
9838access the token names and codes.
8405b70c 9839
09ccae9b 9840Getting a ``code too large'' error from the Java compiler means the code
f50bfcd6 9841hit the 64KB bytecode per method limitation of the Java class file.
09ccae9b
DJ
9842Try reducing the amount of code in actions and static initializers;
9843otherwise, report a bug so that the parser skeleton will be improved.
9844
9845
8405b70c
PB
9846@node Java Semantic Values
9847@subsection Java Semantic Values
9848@c - No %union, specify type in %type/%token.
9849@c - YYSTYPE
9850@c - Printer and destructor
9851
9852There is no @code{%union} directive in Java parsers. Instead, the
9853semantic values' types (class names) should be specified in the
9854@code{%type} or @code{%token} directive:
9855
9856@example
9857%type <Expression> expr assignment_expr term factor
9858%type <Integer> number
9859@end example
9860
9861By default, the semantic stack is declared to have @code{Object} members,
9862which means that the class types you specify can be of any class.
9863To improve the type safety of the parser, you can declare the common
67501061 9864superclass of all the semantic values using the @samp{%define stype}
e254a580 9865directive. For example, after the following declaration:
8405b70c
PB
9866
9867@example
e254a580 9868%define stype "ASTNode"
8405b70c
PB
9869@end example
9870
9871@noindent
9872any @code{%type} or @code{%token} specifying a semantic type which
9873is not a subclass of ASTNode, will cause a compile-time error.
9874
e254a580 9875@c FIXME: Documented bug.
8405b70c
PB
9876Types used in the directives may be qualified with a package name.
9877Primitive data types are accepted for Java version 1.5 or later. Note
9878that in this case the autoboxing feature of Java 1.5 will be used.
e254a580
DJ
9879Generic types may not be used; this is due to a limitation in the
9880implementation of Bison, and may change in future releases.
8405b70c
PB
9881
9882Java parsers do not support @code{%destructor}, since the language
9883adopts garbage collection. The parser will try to hold references
9884to semantic values for as little time as needed.
9885
9886Java parsers do not support @code{%printer}, as @code{toString()}
9887can be used to print the semantic values. This however may change
9888(in a backwards-compatible way) in future versions of Bison.
9889
9890
9891@node Java Location Values
9892@subsection Java Location Values
9893@c - %locations
9894@c - class Position
9895@c - class Location
9896
9897When the directive @code{%locations} is used, the Java parser
9898supports location tracking, see @ref{Locations, , Locations Overview}.
9899An auxiliary user-defined class defines a @dfn{position}, a single point
9900in a file; Bison itself defines a class representing a @dfn{location},
9901a range composed of a pair of positions (possibly spanning several
9902files). The location class is an inner class of the parser; the name
e254a580 9903is @code{Location} by default, and may also be renamed using
cf499cff 9904@samp{%define location_type "@var{class-name}"}.
8405b70c
PB
9905
9906The location class treats the position as a completely opaque value.
9907By default, the class name is @code{Position}, but this can be changed
67501061 9908with @samp{%define position_type "@var{class-name}"}. This class must
e254a580 9909be supplied by the user.
8405b70c
PB
9910
9911
e254a580
DJ
9912@deftypeivar {Location} {Position} begin
9913@deftypeivarx {Location} {Position} end
8405b70c 9914The first, inclusive, position of the range, and the first beyond.
e254a580
DJ
9915@end deftypeivar
9916
9917@deftypeop {Constructor} {Location} {} Location (Position @var{loc})
c265fd6b 9918Create a @code{Location} denoting an empty range located at a given point.
e254a580 9919@end deftypeop
8405b70c 9920
e254a580
DJ
9921@deftypeop {Constructor} {Location} {} Location (Position @var{begin}, Position @var{end})
9922Create a @code{Location} from the endpoints of the range.
9923@end deftypeop
9924
9925@deftypemethod {Location} {String} toString ()
8405b70c
PB
9926Prints the range represented by the location. For this to work
9927properly, the position class should override the @code{equals} and
9928@code{toString} methods appropriately.
9929@end deftypemethod
9930
9931
9932@node Java Parser Interface
9933@subsection Java Parser Interface
9934@c - define parser_class_name
9935@c - Ctor
9936@c - parse, error, set_debug_level, debug_level, set_debug_stream,
9937@c debug_stream.
9938@c - Reporting errors
9939
e254a580
DJ
9940The name of the generated parser class defaults to @code{YYParser}. The
9941@code{YY} prefix may be changed using the @code{%name-prefix} directive
9942or the @option{-p}/@option{--name-prefix} option. Alternatively, use
67501061 9943@samp{%define parser_class_name "@var{name}"} to give a custom name to
e254a580 9944the class. The interface of this class is detailed below.
8405b70c 9945
e254a580 9946By default, the parser class has package visibility. A declaration
67501061 9947@samp{%define public} will change to public visibility. Remember that,
e254a580
DJ
9948according to the Java language specification, the name of the @file{.java}
9949file should match the name of the class in this case. Similarly, you can
9950use @code{abstract}, @code{final} and @code{strictfp} with the
9951@code{%define} declaration to add other modifiers to the parser class.
67501061 9952A single @samp{%define annotations "@var{annotations}"} directive can
1979121c 9953be used to add any number of annotations to the parser class.
e254a580
DJ
9954
9955The Java package name of the parser class can be specified using the
67501061 9956@samp{%define package} directive. The superclass and the implemented
e254a580 9957interfaces of the parser class can be specified with the @code{%define
67501061 9958extends} and @samp{%define implements} directives.
e254a580
DJ
9959
9960The parser class defines an inner class, @code{Location}, that is used
9961for location tracking (see @ref{Java Location Values}), and a inner
9962interface, @code{Lexer} (see @ref{Java Scanner Interface}). Other than
9963these inner class/interface, and the members described in the interface
9964below, all the other members and fields are preceded with a @code{yy} or
9965@code{YY} prefix to avoid clashes with user code.
9966
e254a580
DJ
9967The parser class can be extended using the @code{%parse-param}
9968directive. Each occurrence of the directive will add a @code{protected
9969final} field to the parser class, and an argument to its constructor,
9970which initialize them automatically.
9971
e254a580
DJ
9972@deftypeop {Constructor} {YYParser} {} YYParser (@var{lex_param}, @dots{}, @var{parse_param}, @dots{})
9973Build a new parser object with embedded @code{%code lexer}. There are
2055a44e
AD
9974no parameters, unless @code{%param}s and/or @code{%parse-param}s and/or
9975@code{%lex-param}s are used.
1979121c
DJ
9976
9977Use @code{%code init} for code added to the start of the constructor
9978body. This is especially useful to initialize superclasses. Use
f50bfcd6 9979@samp{%define init_throws} to specify any uncaught exceptions.
e254a580
DJ
9980@end deftypeop
9981
9982@deftypeop {Constructor} {YYParser} {} YYParser (Lexer @var{lexer}, @var{parse_param}, @dots{})
9983Build a new parser object using the specified scanner. There are no
2055a44e
AD
9984additional parameters unless @code{%param}s and/or @code{%parse-param}s are
9985used.
e254a580
DJ
9986
9987If the scanner is defined by @code{%code lexer}, this constructor is
9988declared @code{protected} and is called automatically with a scanner
2055a44e 9989created with the correct @code{%param}s and/or @code{%lex-param}s.
1979121c
DJ
9990
9991Use @code{%code init} for code added to the start of the constructor
9992body. This is especially useful to initialize superclasses. Use
67501061 9993@samp{%define init_throws} to specify any uncatch exceptions.
e254a580 9994@end deftypeop
8405b70c
PB
9995
9996@deftypemethod {YYParser} {boolean} parse ()
9997Run the syntactic analysis, and return @code{true} on success,
9998@code{false} otherwise.
9999@end deftypemethod
10000
1979121c
DJ
10001@deftypemethod {YYParser} {boolean} getErrorVerbose ()
10002@deftypemethodx {YYParser} {void} setErrorVerbose (boolean @var{verbose})
10003Get or set the option to produce verbose error messages. These are only
cf499cff 10004available with @samp{%define parse.error verbose}, which also turns on
1979121c
DJ
10005verbose error messages.
10006@end deftypemethod
10007
10008@deftypemethod {YYParser} {void} yyerror (String @var{msg})
10009@deftypemethodx {YYParser} {void} yyerror (Position @var{pos}, String @var{msg})
10010@deftypemethodx {YYParser} {void} yyerror (Location @var{loc}, String @var{msg})
10011Print an error message using the @code{yyerror} method of the scanner
10012instance in use. The @code{Location} and @code{Position} parameters are
10013available only if location tracking is active.
10014@end deftypemethod
10015
01b477c6 10016@deftypemethod {YYParser} {boolean} recovering ()
8405b70c 10017During the syntactic analysis, return @code{true} if recovering
e254a580
DJ
10018from a syntax error.
10019@xref{Error Recovery}.
8405b70c
PB
10020@end deftypemethod
10021
10022@deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
10023@deftypemethodx {YYParser} {void} setDebugStream (java.io.printStream @var{o})
10024Get or set the stream used for tracing the parsing. It defaults to
10025@code{System.err}.
10026@end deftypemethod
10027
10028@deftypemethod {YYParser} {int} getDebugLevel ()
10029@deftypemethodx {YYParser} {void} setDebugLevel (int @var{l})
10030Get or set the tracing level. Currently its value is either 0, no trace,
10031or nonzero, full tracing.
10032@end deftypemethod
10033
1979121c
DJ
10034@deftypecv {Constant} {YYParser} {String} {bisonVersion}
10035@deftypecvx {Constant} {YYParser} {String} {bisonSkeleton}
10036Identify the Bison version and skeleton used to generate this parser.
10037@end deftypecv
10038
8405b70c
PB
10039
10040@node Java Scanner Interface
10041@subsection Java Scanner Interface
01b477c6 10042@c - %code lexer
8405b70c 10043@c - %lex-param
01b477c6 10044@c - Lexer interface
8405b70c 10045
e254a580
DJ
10046There are two possible ways to interface a Bison-generated Java parser
10047with a scanner: the scanner may be defined by @code{%code lexer}, or
10048defined elsewhere. In either case, the scanner has to implement the
1979121c
DJ
10049@code{Lexer} inner interface of the parser class. This interface also
10050contain constants for all user-defined token names and the predefined
10051@code{EOF} token.
e254a580
DJ
10052
10053In the first case, the body of the scanner class is placed in
10054@code{%code lexer} blocks. If you want to pass parameters from the
10055parser constructor to the scanner constructor, specify them with
10056@code{%lex-param}; they are passed before @code{%parse-param}s to the
10057constructor.
01b477c6 10058
59c5ac72 10059In the second case, the scanner has to implement the @code{Lexer} interface,
01b477c6
PB
10060which is defined within the parser class (e.g., @code{YYParser.Lexer}).
10061The constructor of the parser object will then accept an object
10062implementing the interface; @code{%lex-param} is not used in this
10063case.
10064
10065In both cases, the scanner has to implement the following methods.
10066
e254a580
DJ
10067@deftypemethod {Lexer} {void} yyerror (Location @var{loc}, String @var{msg})
10068This method is defined by the user to emit an error message. The first
10069parameter is omitted if location tracking is not active. Its type can be
67501061 10070changed using @samp{%define location_type "@var{class-name}".}
8405b70c
PB
10071@end deftypemethod
10072
e254a580 10073@deftypemethod {Lexer} {int} yylex ()
8405b70c 10074Return the next token. Its type is the return value, its semantic
f50bfcd6 10075value and location are saved and returned by the their methods in the
e254a580
DJ
10076interface.
10077
67501061 10078Use @samp{%define lex_throws} to specify any uncaught exceptions.
e254a580 10079Default is @code{java.io.IOException}.
8405b70c
PB
10080@end deftypemethod
10081
10082@deftypemethod {Lexer} {Position} getStartPos ()
10083@deftypemethodx {Lexer} {Position} getEndPos ()
01b477c6
PB
10084Return respectively the first position of the last token that
10085@code{yylex} returned, and the first position beyond it. These
10086methods are not needed unless location tracking is active.
8405b70c 10087
67501061 10088The return type can be changed using @samp{%define position_type
8405b70c
PB
10089"@var{class-name}".}
10090@end deftypemethod
10091
10092@deftypemethod {Lexer} {Object} getLVal ()
f50bfcd6 10093Return the semantic value of the last token that yylex returned.
8405b70c 10094
67501061 10095The return type can be changed using @samp{%define stype
8405b70c
PB
10096"@var{class-name}".}
10097@end deftypemethod
10098
10099
e254a580
DJ
10100@node Java Action Features
10101@subsection Special Features for Use in Java Actions
10102
10103The following special constructs can be uses in Java actions.
10104Other analogous C action features are currently unavailable for Java.
10105
67501061 10106Use @samp{%define throws} to specify any uncaught exceptions from parser
e254a580
DJ
10107actions, and initial actions specified by @code{%initial-action}.
10108
10109@defvar $@var{n}
10110The semantic value for the @var{n}th component of the current rule.
10111This may not be assigned to.
10112@xref{Java Semantic Values}.
10113@end defvar
10114
10115@defvar $<@var{typealt}>@var{n}
10116Like @code{$@var{n}} but specifies a alternative type @var{typealt}.
10117@xref{Java Semantic Values}.
10118@end defvar
10119
10120@defvar $$
10121The semantic value for the grouping made by the current rule. As a
10122value, this is in the base type (@code{Object} or as specified by
67501061 10123@samp{%define stype}) as in not cast to the declared subtype because
e254a580
DJ
10124casts are not allowed on the left-hand side of Java assignments.
10125Use an explicit Java cast if the correct subtype is needed.
10126@xref{Java Semantic Values}.
10127@end defvar
10128
10129@defvar $<@var{typealt}>$
10130Same as @code{$$} since Java always allow assigning to the base type.
10131Perhaps we should use this and @code{$<>$} for the value and @code{$$}
10132for setting the value but there is currently no easy way to distinguish
10133these constructs.
10134@xref{Java Semantic Values}.
10135@end defvar
10136
10137@defvar @@@var{n}
10138The location information of the @var{n}th component of the current rule.
10139This may not be assigned to.
10140@xref{Java Location Values}.
10141@end defvar
10142
10143@defvar @@$
10144The location information of the grouping made by the current rule.
10145@xref{Java Location Values}.
10146@end defvar
10147
10148@deffn {Statement} {return YYABORT;}
10149Return immediately from the parser, indicating failure.
10150@xref{Java Parser Interface}.
10151@end deffn
8405b70c 10152
e254a580
DJ
10153@deffn {Statement} {return YYACCEPT;}
10154Return immediately from the parser, indicating success.
10155@xref{Java Parser Interface}.
10156@end deffn
8405b70c 10157
e254a580 10158@deffn {Statement} {return YYERROR;}
c265fd6b 10159Start error recovery without printing an error message.
e254a580
DJ
10160@xref{Error Recovery}.
10161@end deffn
8405b70c 10162
e254a580
DJ
10163@deftypefn {Function} {boolean} recovering ()
10164Return whether error recovery is being done. In this state, the parser
10165reads token until it reaches a known state, and then restarts normal
10166operation.
10167@xref{Error Recovery}.
10168@end deftypefn
8405b70c 10169
1979121c
DJ
10170@deftypefn {Function} {void} yyerror (String @var{msg})
10171@deftypefnx {Function} {void} yyerror (Position @var{loc}, String @var{msg})
10172@deftypefnx {Function} {void} yyerror (Location @var{loc}, String @var{msg})
e254a580 10173Print an error message using the @code{yyerror} method of the scanner
1979121c
DJ
10174instance in use. The @code{Location} and @code{Position} parameters are
10175available only if location tracking is active.
e254a580 10176@end deftypefn
8405b70c 10177
8405b70c 10178
8405b70c
PB
10179@node Java Differences
10180@subsection Differences between C/C++ and Java Grammars
10181
10182The different structure of the Java language forces several differences
10183between C/C++ grammars, and grammars designed for Java parsers. This
29553547 10184section summarizes these differences.
8405b70c
PB
10185
10186@itemize
10187@item
01b477c6 10188Java lacks a preprocessor, so the @code{YYERROR}, @code{YYACCEPT},
8405b70c 10189@code{YYABORT} symbols (@pxref{Table of Symbols}) cannot obviously be
01b477c6
PB
10190macros. Instead, they should be preceded by @code{return} when they
10191appear in an action. The actual definition of these symbols is
8405b70c
PB
10192opaque to the Bison grammar, and it might change in the future. The
10193only meaningful operation that you can do, is to return them.
e254a580 10194See @pxref{Java Action Features}.
8405b70c
PB
10195
10196Note that of these three symbols, only @code{YYACCEPT} and
10197@code{YYABORT} will cause a return from the @code{yyparse}
10198method@footnote{Java parsers include the actions in a separate
10199method than @code{yyparse} in order to have an intuitive syntax that
10200corresponds to these C macros.}.
10201
e254a580
DJ
10202@item
10203Java lacks unions, so @code{%union} has no effect. Instead, semantic
10204values have a common base type: @code{Object} or as specified by
f50bfcd6 10205@samp{%define stype}. Angle brackets on @code{%token}, @code{type},
e254a580
DJ
10206@code{$@var{n}} and @code{$$} specify subtypes rather than fields of
10207an union. The type of @code{$$}, even with angle brackets, is the base
10208type since Java casts are not allow on the left-hand side of assignments.
10209Also, @code{$@var{n}} and @code{@@@var{n}} are not allowed on the
10210left-hand side of assignments. See @pxref{Java Semantic Values} and
10211@pxref{Java Action Features}.
10212
8405b70c 10213@item
f50bfcd6 10214The prologue declarations have a different meaning than in C/C++ code.
01b477c6
PB
10215@table @asis
10216@item @code{%code imports}
10217blocks are placed at the beginning of the Java source code. They may
10218include copyright notices. For a @code{package} declarations, it is
67501061 10219suggested to use @samp{%define package} instead.
8405b70c 10220
01b477c6
PB
10221@item unqualified @code{%code}
10222blocks are placed inside the parser class.
10223
10224@item @code{%code lexer}
10225blocks, if specified, should include the implementation of the
10226scanner. If there is no such block, the scanner can be any class
10227that implements the appropriate interface (see @pxref{Java Scanner
10228Interface}).
29553547 10229@end table
8405b70c
PB
10230
10231Other @code{%code} blocks are not supported in Java parsers.
e254a580
DJ
10232In particular, @code{%@{ @dots{} %@}} blocks should not be used
10233and may give an error in future versions of Bison.
10234
01b477c6 10235The epilogue has the same meaning as in C/C++ code and it can
e254a580
DJ
10236be used to define other classes used by the parser @emph{outside}
10237the parser class.
8405b70c
PB
10238@end itemize
10239
e254a580
DJ
10240
10241@node Java Declarations Summary
10242@subsection Java Declarations Summary
10243
10244This summary only include declarations specific to Java or have special
10245meaning when used in a Java parser.
10246
10247@deffn {Directive} {%language "Java"}
10248Generate a Java class for the parser.
10249@end deffn
10250
10251@deffn {Directive} %lex-param @{@var{type} @var{name}@}
10252A parameter for the lexer class defined by @code{%code lexer}
10253@emph{only}, added as parameters to the lexer constructor and the parser
10254constructor that @emph{creates} a lexer. Default is none.
10255@xref{Java Scanner Interface}.
10256@end deffn
10257
10258@deffn {Directive} %name-prefix "@var{prefix}"
10259The prefix of the parser class name @code{@var{prefix}Parser} if
67501061 10260@samp{%define parser_class_name} is not used. Default is @code{YY}.
e254a580
DJ
10261@xref{Java Bison Interface}.
10262@end deffn
10263
10264@deffn {Directive} %parse-param @{@var{type} @var{name}@}
10265A parameter for the parser class added as parameters to constructor(s)
10266and as fields initialized by the constructor(s). Default is none.
10267@xref{Java Parser Interface}.
10268@end deffn
10269
10270@deffn {Directive} %token <@var{type}> @var{token} @dots{}
10271Declare tokens. Note that the angle brackets enclose a Java @emph{type}.
10272@xref{Java Semantic Values}.
10273@end deffn
10274
10275@deffn {Directive} %type <@var{type}> @var{nonterminal} @dots{}
10276Declare the type of nonterminals. Note that the angle brackets enclose
10277a Java @emph{type}.
10278@xref{Java Semantic Values}.
10279@end deffn
10280
10281@deffn {Directive} %code @{ @var{code} @dots{} @}
10282Code appended to the inside of the parser class.
10283@xref{Java Differences}.
10284@end deffn
10285
10286@deffn {Directive} {%code imports} @{ @var{code} @dots{} @}
10287Code inserted just after the @code{package} declaration.
10288@xref{Java Differences}.
10289@end deffn
10290
1979121c
DJ
10291@deffn {Directive} {%code init} @{ @var{code} @dots{} @}
10292Code inserted at the beginning of the parser constructor body.
10293@xref{Java Parser Interface}.
10294@end deffn
10295
e254a580
DJ
10296@deffn {Directive} {%code lexer} @{ @var{code} @dots{} @}
10297Code added to the body of a inner lexer class within the parser class.
10298@xref{Java Scanner Interface}.
10299@end deffn
10300
10301@deffn {Directive} %% @var{code} @dots{}
10302Code (after the second @code{%%}) appended to the end of the file,
10303@emph{outside} the parser class.
10304@xref{Java Differences}.
10305@end deffn
10306
10307@deffn {Directive} %@{ @var{code} @dots{} %@}
1979121c 10308Not supported. Use @code{%code imports} instead.
e254a580
DJ
10309@xref{Java Differences}.
10310@end deffn
10311
10312@deffn {Directive} {%define abstract}
10313Whether the parser class is declared @code{abstract}. Default is false.
10314@xref{Java Bison Interface}.
10315@end deffn
10316
1979121c
DJ
10317@deffn {Directive} {%define annotations} "@var{annotations}"
10318The Java annotations for the parser class. Default is none.
10319@xref{Java Bison Interface}.
10320@end deffn
10321
e254a580
DJ
10322@deffn {Directive} {%define extends} "@var{superclass}"
10323The superclass of the parser class. Default is none.
10324@xref{Java Bison Interface}.
10325@end deffn
10326
10327@deffn {Directive} {%define final}
10328Whether the parser class is declared @code{final}. Default is false.
10329@xref{Java Bison Interface}.
10330@end deffn
10331
10332@deffn {Directive} {%define implements} "@var{interfaces}"
10333The implemented interfaces of the parser class, a comma-separated list.
10334Default is none.
10335@xref{Java Bison Interface}.
10336@end deffn
10337
1979121c
DJ
10338@deffn {Directive} {%define init_throws} "@var{exceptions}"
10339The exceptions thrown by @code{%code init} from the parser class
10340constructor. Default is none.
10341@xref{Java Parser Interface}.
10342@end deffn
10343
e254a580
DJ
10344@deffn {Directive} {%define lex_throws} "@var{exceptions}"
10345The exceptions thrown by the @code{yylex} method of the lexer, a
10346comma-separated list. Default is @code{java.io.IOException}.
10347@xref{Java Scanner Interface}.
10348@end deffn
10349
10350@deffn {Directive} {%define location_type} "@var{class}"
10351The name of the class used for locations (a range between two
10352positions). This class is generated as an inner class of the parser
10353class by @command{bison}. Default is @code{Location}.
10354@xref{Java Location Values}.
10355@end deffn
10356
10357@deffn {Directive} {%define package} "@var{package}"
10358The package to put the parser class in. Default is none.
10359@xref{Java Bison Interface}.
10360@end deffn
10361
10362@deffn {Directive} {%define parser_class_name} "@var{name}"
10363The name of the parser class. Default is @code{YYParser} or
10364@code{@var{name-prefix}Parser}.
10365@xref{Java Bison Interface}.
10366@end deffn
10367
10368@deffn {Directive} {%define position_type} "@var{class}"
10369The name of the class used for positions. This class must be supplied by
10370the user. Default is @code{Position}.
10371@xref{Java Location Values}.
10372@end deffn
10373
10374@deffn {Directive} {%define public}
10375Whether the parser class is declared @code{public}. Default is false.
10376@xref{Java Bison Interface}.
10377@end deffn
10378
10379@deffn {Directive} {%define stype} "@var{class}"
10380The base type of semantic values. Default is @code{Object}.
10381@xref{Java Semantic Values}.
10382@end deffn
10383
10384@deffn {Directive} {%define strictfp}
10385Whether the parser class is declared @code{strictfp}. Default is false.
10386@xref{Java Bison Interface}.
10387@end deffn
10388
10389@deffn {Directive} {%define throws} "@var{exceptions}"
10390The exceptions thrown by user-supplied parser actions and
10391@code{%initial-action}, a comma-separated list. Default is none.
10392@xref{Java Parser Interface}.
10393@end deffn
10394
10395
12545799 10396@c ================================================= FAQ
d1a1114f
AD
10397
10398@node FAQ
10399@chapter Frequently Asked Questions
10400@cindex frequently asked questions
10401@cindex questions
10402
10403Several questions about Bison come up occasionally. Here some of them
10404are addressed.
10405
10406@menu
55ba27be
AD
10407* Memory Exhausted:: Breaking the Stack Limits
10408* How Can I Reset the Parser:: @code{yyparse} Keeps some State
10409* Strings are Destroyed:: @code{yylval} Loses Track of Strings
10410* Implementing Gotos/Loops:: Control Flow in the Calculator
ed2e6384 10411* Multiple start-symbols:: Factoring closely related grammars
8a4281b9 10412* Secure? Conform?:: Is Bison POSIX safe?
55ba27be
AD
10413* I can't build Bison:: Troubleshooting
10414* Where can I find help?:: Troubleshouting
10415* Bug Reports:: Troublereporting
8405b70c 10416* More Languages:: Parsers in C++, Java, and so on
55ba27be
AD
10417* Beta Testing:: Experimenting development versions
10418* Mailing Lists:: Meeting other Bison users
d1a1114f
AD
10419@end menu
10420
1a059451
PE
10421@node Memory Exhausted
10422@section Memory Exhausted
d1a1114f
AD
10423
10424@display
1a059451 10425My parser returns with error with a @samp{memory exhausted}
d1a1114f
AD
10426message. What can I do?
10427@end display
10428
10429This question is already addressed elsewhere, @xref{Recursion,
10430,Recursive Rules}.
10431
e64fec0a
PE
10432@node How Can I Reset the Parser
10433@section How Can I Reset the Parser
5b066063 10434
0e14ad77
PE
10435The following phenomenon has several symptoms, resulting in the
10436following typical questions:
5b066063
AD
10437
10438@display
10439I invoke @code{yyparse} several times, and on correct input it works
10440properly; but when a parse error is found, all the other calls fail
0e14ad77 10441too. How can I reset the error flag of @code{yyparse}?
5b066063
AD
10442@end display
10443
10444@noindent
10445or
10446
10447@display
0e14ad77 10448My parser includes support for an @samp{#include}-like feature, in
5b066063 10449which case I run @code{yyparse} from @code{yyparse}. This fails
67501061 10450although I did specify @samp{%define api.pure}.
5b066063
AD
10451@end display
10452
0e14ad77
PE
10453These problems typically come not from Bison itself, but from
10454Lex-generated scanners. Because these scanners use large buffers for
5b066063
AD
10455speed, they might not notice a change of input file. As a
10456demonstration, consider the following source file,
10457@file{first-line.l}:
10458
10459@verbatim
10460%{
10461#include <stdio.h>
10462#include <stdlib.h>
10463%}
10464%%
10465.*\n ECHO; return 1;
10466%%
10467int
0e14ad77 10468yyparse (char const *file)
5b066063
AD
10469{
10470 yyin = fopen (file, "r");
10471 if (!yyin)
10472 exit (2);
fa7e68c3 10473 /* One token only. */
5b066063 10474 yylex ();
0e14ad77 10475 if (fclose (yyin) != 0)
5b066063
AD
10476 exit (3);
10477 return 0;
10478}
10479
10480int
0e14ad77 10481main (void)
5b066063
AD
10482{
10483 yyparse ("input");
10484 yyparse ("input");
10485 return 0;
10486}
10487@end verbatim
10488
10489@noindent
10490If the file @file{input} contains
10491
10492@verbatim
10493input:1: Hello,
10494input:2: World!
10495@end verbatim
10496
10497@noindent
0e14ad77 10498then instead of getting the first line twice, you get:
5b066063
AD
10499
10500@example
10501$ @kbd{flex -ofirst-line.c first-line.l}
10502$ @kbd{gcc -ofirst-line first-line.c -ll}
10503$ @kbd{./first-line}
10504input:1: Hello,
10505input:2: World!
10506@end example
10507
0e14ad77
PE
10508Therefore, whenever you change @code{yyin}, you must tell the
10509Lex-generated scanner to discard its current buffer and switch to the
10510new one. This depends upon your implementation of Lex; see its
10511documentation for more. For Flex, it suffices to call
10512@samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
10513Flex-generated scanner needs to read from several input streams to
10514handle features like include files, you might consider using Flex
10515functions like @samp{yy_switch_to_buffer} that manipulate multiple
10516input buffers.
5b066063 10517
b165c324
AD
10518If your Flex-generated scanner uses start conditions (@pxref{Start
10519conditions, , Start conditions, flex, The Flex Manual}), you might
10520also want to reset the scanner's state, i.e., go back to the initial
10521start condition, through a call to @samp{BEGIN (0)}.
10522
fef4cb51
AD
10523@node Strings are Destroyed
10524@section Strings are Destroyed
10525
10526@display
c7e441b4 10527My parser seems to destroy old strings, or maybe it loses track of
fef4cb51
AD
10528them. Instead of reporting @samp{"foo", "bar"}, it reports
10529@samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
10530@end display
10531
10532This error is probably the single most frequent ``bug report'' sent to
10533Bison lists, but is only concerned with a misunderstanding of the role
8c5b881d 10534of the scanner. Consider the following Lex code:
fef4cb51
AD
10535
10536@verbatim
10537%{
10538#include <stdio.h>
10539char *yylval = NULL;
10540%}
10541%%
10542.* yylval = yytext; return 1;
10543\n /* IGNORE */
10544%%
10545int
10546main ()
10547{
fa7e68c3 10548 /* Similar to using $1, $2 in a Bison action. */
fef4cb51
AD
10549 char *fst = (yylex (), yylval);
10550 char *snd = (yylex (), yylval);
10551 printf ("\"%s\", \"%s\"\n", fst, snd);
10552 return 0;
10553}
10554@end verbatim
10555
10556If you compile and run this code, you get:
10557
10558@example
10559$ @kbd{flex -osplit-lines.c split-lines.l}
10560$ @kbd{gcc -osplit-lines split-lines.c -ll}
10561$ @kbd{printf 'one\ntwo\n' | ./split-lines}
10562"one
10563two", "two"
10564@end example
10565
10566@noindent
10567this is because @code{yytext} is a buffer provided for @emph{reading}
10568in the action, but if you want to keep it, you have to duplicate it
10569(e.g., using @code{strdup}). Note that the output may depend on how
10570your implementation of Lex handles @code{yytext}. For instance, when
10571given the Lex compatibility option @option{-l} (which triggers the
10572option @samp{%array}) Flex generates a different behavior:
10573
10574@example
10575$ @kbd{flex -l -osplit-lines.c split-lines.l}
10576$ @kbd{gcc -osplit-lines split-lines.c -ll}
10577$ @kbd{printf 'one\ntwo\n' | ./split-lines}
10578"two", "two"
10579@end example
10580
10581
2fa09258
AD
10582@node Implementing Gotos/Loops
10583@section Implementing Gotos/Loops
a06ea4aa
AD
10584
10585@display
10586My simple calculator supports variables, assignments, and functions,
2fa09258 10587but how can I implement gotos, or loops?
a06ea4aa
AD
10588@end display
10589
10590Although very pedagogical, the examples included in the document blur
a1c84f45 10591the distinction to make between the parser---whose job is to recover
a06ea4aa 10592the structure of a text and to transmit it to subsequent modules of
a1c84f45 10593the program---and the processing (such as the execution) of this
a06ea4aa
AD
10594structure. This works well with so called straight line programs,
10595i.e., precisely those that have a straightforward execution model:
10596execute simple instructions one after the others.
10597
10598@cindex abstract syntax tree
8a4281b9 10599@cindex AST
a06ea4aa
AD
10600If you want a richer model, you will probably need to use the parser
10601to construct a tree that does represent the structure it has
10602recovered; this tree is usually called the @dfn{abstract syntax tree},
8a4281b9 10603or @dfn{AST} for short. Then, walking through this tree,
a06ea4aa
AD
10604traversing it in various ways, will enable treatments such as its
10605execution or its translation, which will result in an interpreter or a
10606compiler.
10607
10608This topic is way beyond the scope of this manual, and the reader is
10609invited to consult the dedicated literature.
10610
10611
ed2e6384
AD
10612@node Multiple start-symbols
10613@section Multiple start-symbols
10614
10615@display
10616I have several closely related grammars, and I would like to share their
10617implementations. In fact, I could use a single grammar but with
10618multiple entry points.
10619@end display
10620
10621Bison does not support multiple start-symbols, but there is a very
10622simple means to simulate them. If @code{foo} and @code{bar} are the two
10623pseudo start-symbols, then introduce two new tokens, say
10624@code{START_FOO} and @code{START_BAR}, and use them as switches from the
10625real start-symbol:
10626
10627@example
10628%token START_FOO START_BAR;
10629%start start;
10630start: START_FOO foo
10631 | START_BAR bar;
10632@end example
10633
10634These tokens prevents the introduction of new conflicts. As far as the
10635parser goes, that is all that is needed.
10636
10637Now the difficult part is ensuring that the scanner will send these
10638tokens first. If your scanner is hand-written, that should be
10639straightforward. If your scanner is generated by Lex, them there is
10640simple means to do it: recall that anything between @samp{%@{ ... %@}}
10641after the first @code{%%} is copied verbatim in the top of the generated
10642@code{yylex} function. Make sure a variable @code{start_token} is
10643available in the scanner (e.g., a global variable or using
10644@code{%lex-param} etc.), and use the following:
10645
10646@example
10647 /* @r{Prologue.} */
10648%%
10649%@{
10650 if (start_token)
10651 @{
10652 int t = start_token;
10653 start_token = 0;
10654 return t;
10655 @}
10656%@}
10657 /* @r{The rules.} */
10658@end example
10659
10660
55ba27be
AD
10661@node Secure? Conform?
10662@section Secure? Conform?
10663
10664@display
10665Is Bison secure? Does it conform to POSIX?
10666@end display
10667
10668If you're looking for a guarantee or certification, we don't provide it.
10669However, Bison is intended to be a reliable program that conforms to the
8a4281b9 10670POSIX specification for Yacc. If you run into problems,
55ba27be
AD
10671please send us a bug report.
10672
10673@node I can't build Bison
10674@section I can't build Bison
10675
10676@display
8c5b881d
PE
10677I can't build Bison because @command{make} complains that
10678@code{msgfmt} is not found.
55ba27be
AD
10679What should I do?
10680@end display
10681
10682Like most GNU packages with internationalization support, that feature
10683is turned on by default. If you have problems building in the @file{po}
10684subdirectory, it indicates that your system's internationalization
10685support is lacking. You can re-configure Bison with
10686@option{--disable-nls} to turn off this support, or you can install GNU
10687gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
10688Bison. See the file @file{ABOUT-NLS} for more information.
10689
10690
10691@node Where can I find help?
10692@section Where can I find help?
10693
10694@display
10695I'm having trouble using Bison. Where can I find help?
10696@end display
10697
10698First, read this fine manual. Beyond that, you can send mail to
10699@email{help-bison@@gnu.org}. This mailing list is intended to be
10700populated with people who are willing to answer questions about using
10701and installing Bison. Please keep in mind that (most of) the people on
10702the list have aspects of their lives which are not related to Bison (!),
10703so you may not receive an answer to your question right away. This can
10704be frustrating, but please try not to honk them off; remember that any
10705help they provide is purely voluntary and out of the kindness of their
10706hearts.
10707
10708@node Bug Reports
10709@section Bug Reports
10710
10711@display
10712I found a bug. What should I include in the bug report?
10713@end display
10714
10715Before you send a bug report, make sure you are using the latest
10716version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
10717mirrors. Be sure to include the version number in your bug report. If
10718the bug is present in the latest version but not in a previous version,
10719try to determine the most recent version which did not contain the bug.
10720
10721If the bug is parser-related, you should include the smallest grammar
10722you can which demonstrates the bug. The grammar file should also be
10723complete (i.e., I should be able to run it through Bison without having
10724to edit or add anything). The smaller and simpler the grammar, the
10725easier it will be to fix the bug.
10726
10727Include information about your compilation environment, including your
10728operating system's name and version and your compiler's name and
10729version. If you have trouble compiling, you should also include a
10730transcript of the build session, starting with the invocation of
10731`configure'. Depending on the nature of the bug, you may be asked to
10732send additional files as well (such as `config.h' or `config.cache').
10733
10734Patches are most welcome, but not required. That is, do not hesitate to
10735send a bug report just because you can not provide a fix.
10736
10737Send bug reports to @email{bug-bison@@gnu.org}.
10738
8405b70c
PB
10739@node More Languages
10740@section More Languages
55ba27be
AD
10741
10742@display
8405b70c 10743Will Bison ever have C++ and Java support? How about @var{insert your
55ba27be
AD
10744favorite language here}?
10745@end display
10746
8405b70c 10747C++ and Java support is there now, and is documented. We'd love to add other
55ba27be
AD
10748languages; contributions are welcome.
10749
10750@node Beta Testing
10751@section Beta Testing
10752
10753@display
10754What is involved in being a beta tester?
10755@end display
10756
10757It's not terribly involved. Basically, you would download a test
10758release, compile it, and use it to build and run a parser or two. After
10759that, you would submit either a bug report or a message saying that
10760everything is okay. It is important to report successes as well as
10761failures because test releases eventually become mainstream releases,
10762but only if they are adequately tested. If no one tests, development is
10763essentially halted.
10764
10765Beta testers are particularly needed for operating systems to which the
10766developers do not have easy access. They currently have easy access to
10767recent GNU/Linux and Solaris versions. Reports about other operating
10768systems are especially welcome.
10769
10770@node Mailing Lists
10771@section Mailing Lists
10772
10773@display
10774How do I join the help-bison and bug-bison mailing lists?
10775@end display
10776
10777See @url{http://lists.gnu.org/}.
a06ea4aa 10778
d1a1114f
AD
10779@c ================================================= Table of Symbols
10780
342b8b6e 10781@node Table of Symbols
bfa74976
RS
10782@appendix Bison Symbols
10783@cindex Bison symbols, table of
10784@cindex symbols in Bison, table of
10785
18b519c0 10786@deffn {Variable} @@$
3ded9a63 10787In an action, the location of the left-hand side of the rule.
88bce5a2 10788@xref{Locations, , Locations Overview}.
18b519c0 10789@end deffn
3ded9a63 10790
18b519c0 10791@deffn {Variable} @@@var{n}
3ded9a63
AD
10792In an action, the location of the @var{n}-th symbol of the right-hand
10793side of the rule. @xref{Locations, , Locations Overview}.
18b519c0 10794@end deffn
3ded9a63 10795
d013372c
AR
10796@deffn {Variable} @@@var{name}
10797In an action, the location of a symbol addressed by name.
10798@xref{Locations, , Locations Overview}.
10799@end deffn
10800
10801@deffn {Variable} @@[@var{name}]
10802In an action, the location of a symbol addressed by name.
10803@xref{Locations, , Locations Overview}.
10804@end deffn
10805
18b519c0 10806@deffn {Variable} $$
3ded9a63
AD
10807In an action, the semantic value of the left-hand side of the rule.
10808@xref{Actions}.
18b519c0 10809@end deffn
3ded9a63 10810
18b519c0 10811@deffn {Variable} $@var{n}
3ded9a63
AD
10812In an action, the semantic value of the @var{n}-th symbol of the
10813right-hand side of the rule. @xref{Actions}.
18b519c0 10814@end deffn
3ded9a63 10815
d013372c
AR
10816@deffn {Variable} $@var{name}
10817In an action, the semantic value of a symbol addressed by name.
10818@xref{Actions}.
10819@end deffn
10820
10821@deffn {Variable} $[@var{name}]
10822In an action, the semantic value of a symbol addressed by name.
10823@xref{Actions}.
10824@end deffn
10825
dd8d9022
AD
10826@deffn {Delimiter} %%
10827Delimiter used to separate the grammar rule section from the
10828Bison declarations section or the epilogue.
10829@xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
18b519c0 10830@end deffn
bfa74976 10831
dd8d9022
AD
10832@c Don't insert spaces, or check the DVI output.
10833@deffn {Delimiter} %@{@var{code}%@}
ff7571c0
JD
10834All code listed between @samp{%@{} and @samp{%@}} is copied verbatim
10835to the parser implementation file. Such code forms the prologue of
10836the grammar file. @xref{Grammar Outline, ,Outline of a Bison
dd8d9022 10837Grammar}.
18b519c0 10838@end deffn
bfa74976 10839
ca2a6d15
PH
10840@deffn {Directive} %?@{@var{expression}@}
10841Predicate actions. This is a type of action clause that may appear in
10842rules. The expression is evaluated, and if false, causes a syntax error. In
8a4281b9 10843GLR parsers during nondeterministic operation,
ca2a6d15
PH
10844this silently causes an alternative parse to die. During deterministic
10845operation, it is the same as the effect of YYERROR.
10846@xref{Semantic Predicates}.
10847
10848This feature is experimental.
10849More user feedback will help to determine whether it should become a permanent
10850feature.
10851@end deffn
10852
dd8d9022
AD
10853@deffn {Construct} /*@dots{}*/
10854Comment delimiters, as in C.
18b519c0 10855@end deffn
bfa74976 10856
dd8d9022
AD
10857@deffn {Delimiter} :
10858Separates a rule's result from its components. @xref{Rules, ,Syntax of
10859Grammar Rules}.
18b519c0 10860@end deffn
bfa74976 10861
dd8d9022
AD
10862@deffn {Delimiter} ;
10863Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 10864@end deffn
bfa74976 10865
dd8d9022
AD
10866@deffn {Delimiter} |
10867Separates alternate rules for the same result nonterminal.
10868@xref{Rules, ,Syntax of Grammar Rules}.
18b519c0 10869@end deffn
bfa74976 10870
12e35840
JD
10871@deffn {Directive} <*>
10872Used to define a default tagged @code{%destructor} or default tagged
10873@code{%printer}.
85894313
JD
10874
10875This feature is experimental.
10876More user feedback will help to determine whether it should become a permanent
10877feature.
10878
12e35840
JD
10879@xref{Destructor Decl, , Freeing Discarded Symbols}.
10880@end deffn
10881
3ebecc24 10882@deffn {Directive} <>
12e35840
JD
10883Used to define a default tagless @code{%destructor} or default tagless
10884@code{%printer}.
85894313
JD
10885
10886This feature is experimental.
10887More user feedback will help to determine whether it should become a permanent
10888feature.
10889
12e35840
JD
10890@xref{Destructor Decl, , Freeing Discarded Symbols}.
10891@end deffn
10892
dd8d9022
AD
10893@deffn {Symbol} $accept
10894The predefined nonterminal whose only rule is @samp{$accept: @var{start}
10895$end}, where @var{start} is the start symbol. @xref{Start Decl, , The
10896Start-Symbol}. It cannot be used in the grammar.
18b519c0 10897@end deffn
bfa74976 10898
136a0f76 10899@deffn {Directive} %code @{@var{code}@}
148d66d8 10900@deffnx {Directive} %code @var{qualifier} @{@var{code}@}
51151d91
JD
10901Insert @var{code} verbatim into the output parser source at the
10902default location or at the location specified by @var{qualifier}.
e0c07222 10903@xref{%code Summary}.
9bc0dd67
JD
10904@end deffn
10905
10906@deffn {Directive} %debug
10907Equip the parser for debugging. @xref{Decl Summary}.
10908@end deffn
10909
91d2c560 10910@ifset defaultprec
22fccf95
PE
10911@deffn {Directive} %default-prec
10912Assign a precedence to rules that lack an explicit @samp{%prec}
10913modifier. @xref{Contextual Precedence, ,Context-Dependent
10914Precedence}.
39a06c25 10915@end deffn
91d2c560 10916@end ifset
39a06c25 10917
148d66d8
JD
10918@deffn {Directive} %define @var{define-variable}
10919@deffnx {Directive} %define @var{define-variable} @var{value}
cf499cff 10920@deffnx {Directive} %define @var{define-variable} "@var{value}"
35c1e5f0 10921Define a variable to adjust Bison's behavior. @xref{%define Summary}.
148d66d8
JD
10922@end deffn
10923
18b519c0 10924@deffn {Directive} %defines
ff7571c0
JD
10925Bison declaration to create a parser header file, which is usually
10926meant for the scanner. @xref{Decl Summary}.
18b519c0 10927@end deffn
6deb4447 10928
02975b9a
JD
10929@deffn {Directive} %defines @var{defines-file}
10930Same as above, but save in the file @var{defines-file}.
10931@xref{Decl Summary}.
10932@end deffn
10933
18b519c0 10934@deffn {Directive} %destructor
258b75ca 10935Specify how the parser should reclaim the memory associated to
fa7e68c3 10936discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
18b519c0 10937@end deffn
72f889cc 10938
18b519c0 10939@deffn {Directive} %dprec
676385e2 10940Bison declaration to assign a precedence to a rule that is used at parse
c827f760 10941time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
8a4281b9 10942GLR Parsers}.
18b519c0 10943@end deffn
676385e2 10944
dd8d9022
AD
10945@deffn {Symbol} $end
10946The predefined token marking the end of the token stream. It cannot be
10947used in the grammar.
10948@end deffn
10949
10950@deffn {Symbol} error
10951A token name reserved for error recovery. This token may be used in
10952grammar rules so as to allow the Bison parser to recognize an error in
10953the grammar without halting the process. In effect, a sentence
10954containing an error may be recognized as valid. On a syntax error, the
742e4900
JD
10955token @code{error} becomes the current lookahead token. Actions
10956corresponding to @code{error} are then executed, and the lookahead
dd8d9022
AD
10957token is reset to the token that originally caused the violation.
10958@xref{Error Recovery}.
18d192f0
AD
10959@end deffn
10960
18b519c0 10961@deffn {Directive} %error-verbose
cf499cff 10962An obsolete directive standing for @samp{%define parse.error verbose}.
18b519c0 10963@end deffn
2a8d363a 10964
02975b9a 10965@deffn {Directive} %file-prefix "@var{prefix}"
72d2299c 10966Bison declaration to set the prefix of the output files. @xref{Decl
d8988b2f 10967Summary}.
18b519c0 10968@end deffn
d8988b2f 10969
18b519c0 10970@deffn {Directive} %glr-parser
8a4281b9
JD
10971Bison declaration to produce a GLR parser. @xref{GLR
10972Parsers, ,Writing GLR Parsers}.
18b519c0 10973@end deffn
676385e2 10974
dd8d9022
AD
10975@deffn {Directive} %initial-action
10976Run user code before parsing. @xref{Initial Action Decl, , Performing Actions before Parsing}.
10977@end deffn
10978
e6e704dc
JD
10979@deffn {Directive} %language
10980Specify the programming language for the generated parser.
10981@xref{Decl Summary}.
10982@end deffn
10983
18b519c0 10984@deffn {Directive} %left
d78f0ac9 10985Bison declaration to assign precedence and left associativity to token(s).
bfa74976 10986@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 10987@end deffn
bfa74976 10988
2055a44e
AD
10989@deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
10990Bison declaration to specifying additional arguments that
2a8d363a
AD
10991@code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
10992for Pure Parsers}.
18b519c0 10993@end deffn
2a8d363a 10994
18b519c0 10995@deffn {Directive} %merge
676385e2 10996Bison declaration to assign a merging function to a rule. If there is a
fae437e8 10997reduce/reduce conflict with a rule having the same merging function, the
676385e2 10998function is applied to the two semantic values to get a single result.
8a4281b9 10999@xref{GLR Parsers, ,Writing GLR Parsers}.
18b519c0 11000@end deffn
676385e2 11001
02975b9a 11002@deffn {Directive} %name-prefix "@var{prefix}"
72d2299c 11003Bison declaration to rename the external symbols. @xref{Decl Summary}.
18b519c0 11004@end deffn
d8988b2f 11005
91d2c560 11006@ifset defaultprec
22fccf95
PE
11007@deffn {Directive} %no-default-prec
11008Do not assign a precedence to rules that lack an explicit @samp{%prec}
11009modifier. @xref{Contextual Precedence, ,Context-Dependent
11010Precedence}.
11011@end deffn
91d2c560 11012@end ifset
22fccf95 11013
18b519c0 11014@deffn {Directive} %no-lines
931c7513 11015Bison declaration to avoid generating @code{#line} directives in the
ff7571c0 11016parser implementation file. @xref{Decl Summary}.
18b519c0 11017@end deffn
931c7513 11018
18b519c0 11019@deffn {Directive} %nonassoc
d78f0ac9 11020Bison declaration to assign precedence and nonassociativity to token(s).
bfa74976 11021@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 11022@end deffn
bfa74976 11023
02975b9a 11024@deffn {Directive} %output "@var{file}"
ff7571c0
JD
11025Bison declaration to set the name of the parser implementation file.
11026@xref{Decl Summary}.
18b519c0 11027@end deffn
d8988b2f 11028
2055a44e
AD
11029@deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
11030Bison declaration to specify additional arguments that both
11031@code{yylex} and @code{yyparse} should accept. @xref{Parser Function,, The
11032Parser Function @code{yyparse}}.
11033@end deffn
11034
11035@deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
11036Bison declaration to specify additional arguments that @code{yyparse}
11037should accept. @xref{Parser Function,, The Parser Function @code{yyparse}}.
18b519c0 11038@end deffn
2a8d363a 11039
18b519c0 11040@deffn {Directive} %prec
bfa74976
RS
11041Bison declaration to assign a precedence to a specific rule.
11042@xref{Contextual Precedence, ,Context-Dependent Precedence}.
18b519c0 11043@end deffn
bfa74976 11044
d78f0ac9
AD
11045@deffn {Directive} %precedence
11046Bison declaration to assign precedence to token(s), but no associativity
11047@xref{Precedence Decl, ,Operator Precedence}.
11048@end deffn
11049
18b519c0 11050@deffn {Directive} %pure-parser
35c1e5f0
JD
11051Deprecated version of @samp{%define api.pure} (@pxref{%define
11052Summary,,api.pure}), for which Bison is more careful to warn about
11053unreasonable usage.
18b519c0 11054@end deffn
bfa74976 11055
b50d2359 11056@deffn {Directive} %require "@var{version}"
9b8a5ce0
AD
11057Require version @var{version} or higher of Bison. @xref{Require Decl, ,
11058Require a Version of Bison}.
b50d2359
AD
11059@end deffn
11060
18b519c0 11061@deffn {Directive} %right
d78f0ac9 11062Bison declaration to assign precedence and right associativity to token(s).
bfa74976 11063@xref{Precedence Decl, ,Operator Precedence}.
18b519c0 11064@end deffn
bfa74976 11065
e6e704dc
JD
11066@deffn {Directive} %skeleton
11067Specify the skeleton to use; usually for development.
11068@xref{Decl Summary}.
11069@end deffn
11070
18b519c0 11071@deffn {Directive} %start
704a47c4
AD
11072Bison declaration to specify the start symbol. @xref{Start Decl, ,The
11073Start-Symbol}.
18b519c0 11074@end deffn
bfa74976 11075
18b519c0 11076@deffn {Directive} %token
bfa74976
RS
11077Bison declaration to declare token(s) without specifying precedence.
11078@xref{Token Decl, ,Token Type Names}.
18b519c0 11079@end deffn
bfa74976 11080
18b519c0 11081@deffn {Directive} %token-table
ff7571c0
JD
11082Bison declaration to include a token name table in the parser
11083implementation file. @xref{Decl Summary}.
18b519c0 11084@end deffn
931c7513 11085
18b519c0 11086@deffn {Directive} %type
704a47c4
AD
11087Bison declaration to declare nonterminals. @xref{Type Decl,
11088,Nonterminal Symbols}.
18b519c0 11089@end deffn
bfa74976 11090
dd8d9022
AD
11091@deffn {Symbol} $undefined
11092The predefined token onto which all undefined values returned by
11093@code{yylex} are mapped. It cannot be used in the grammar, rather, use
11094@code{error}.
11095@end deffn
11096
18b519c0 11097@deffn {Directive} %union
bfa74976
RS
11098Bison declaration to specify several possible data types for semantic
11099values. @xref{Union Decl, ,The Collection of Value Types}.
18b519c0 11100@end deffn
bfa74976 11101
dd8d9022
AD
11102@deffn {Macro} YYABORT
11103Macro to pretend that an unrecoverable syntax error has occurred, by
11104making @code{yyparse} return 1 immediately. The error reporting
11105function @code{yyerror} is not called. @xref{Parser Function, ,The
11106Parser Function @code{yyparse}}.
8405b70c
PB
11107
11108For Java parsers, this functionality is invoked using @code{return YYABORT;}
11109instead.
dd8d9022 11110@end deffn
3ded9a63 11111
dd8d9022
AD
11112@deffn {Macro} YYACCEPT
11113Macro to pretend that a complete utterance of the language has been
11114read, by making @code{yyparse} return 0 immediately.
11115@xref{Parser Function, ,The Parser Function @code{yyparse}}.
8405b70c
PB
11116
11117For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
11118instead.
dd8d9022 11119@end deffn
bfa74976 11120
dd8d9022 11121@deffn {Macro} YYBACKUP
742e4900 11122Macro to discard a value from the parser stack and fake a lookahead
dd8d9022 11123token. @xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 11124@end deffn
bfa74976 11125
dd8d9022 11126@deffn {Variable} yychar
32c29292 11127External integer variable that contains the integer value of the
742e4900 11128lookahead token. (In a pure parser, it is a local variable within
dd8d9022
AD
11129@code{yyparse}.) Error-recovery rule actions may examine this variable.
11130@xref{Action Features, ,Special Features for Use in Actions}.
18b519c0 11131@end deffn
bfa74976 11132
dd8d9022
AD
11133@deffn {Variable} yyclearin
11134Macro used in error-recovery rule actions. It clears the previous
742e4900 11135lookahead token. @xref{Error Recovery}.
18b519c0 11136@end deffn
bfa74976 11137
dd8d9022
AD
11138@deffn {Macro} YYDEBUG
11139Macro to define to equip the parser with tracing code. @xref{Tracing,
11140,Tracing Your Parser}.
18b519c0 11141@end deffn
bfa74976 11142
dd8d9022
AD
11143@deffn {Variable} yydebug
11144External integer variable set to zero by default. If @code{yydebug}
11145is given a nonzero value, the parser will output information on input
11146symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
18b519c0 11147@end deffn
bfa74976 11148
dd8d9022
AD
11149@deffn {Macro} yyerrok
11150Macro to cause parser to recover immediately to its normal mode
11151after a syntax error. @xref{Error Recovery}.
11152@end deffn
11153
11154@deffn {Macro} YYERROR
11155Macro to pretend that a syntax error has just been detected: call
11156@code{yyerror} and then perform normal error recovery if possible
11157(@pxref{Error Recovery}), or (if recovery is impossible) make
11158@code{yyparse} return 1. @xref{Error Recovery}.
8405b70c
PB
11159
11160For Java parsers, this functionality is invoked using @code{return YYERROR;}
11161instead.
dd8d9022
AD
11162@end deffn
11163
11164@deffn {Function} yyerror
11165User-supplied function to be called by @code{yyparse} on error.
71b00ed8 11166@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
dd8d9022
AD
11167@end deffn
11168
11169@deffn {Macro} YYERROR_VERBOSE
71b00ed8
AD
11170An obsolete macro used in the @file{yacc.c} skeleton, that you define
11171with @code{#define} in the prologue to request verbose, specific error
11172message strings when @code{yyerror} is called. It doesn't matter what
11173definition you use for @code{YYERROR_VERBOSE}, just whether you define
cf499cff 11174it. Using @samp{%define parse.error verbose} is preferred
31b850d2 11175(@pxref{Error Reporting, ,The Error Reporting Function @code{yyerror}}).
dd8d9022
AD
11176@end deffn
11177
11178@deffn {Macro} YYINITDEPTH
11179Macro for specifying the initial size of the parser stack.
1a059451 11180@xref{Memory Management}.
dd8d9022
AD
11181@end deffn
11182
11183@deffn {Function} yylex
11184User-supplied lexical analyzer function, called with no arguments to get
11185the next token. @xref{Lexical, ,The Lexical Analyzer Function
11186@code{yylex}}.
11187@end deffn
11188
11189@deffn {Macro} YYLEX_PARAM
11190An obsolete macro for specifying an extra argument (or list of extra
32c29292 11191arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
dd8d9022
AD
11192macro is deprecated, and is supported only for Yacc like parsers.
11193@xref{Pure Calling,, Calling Conventions for Pure Parsers}.
11194@end deffn
11195
11196@deffn {Variable} yylloc
11197External variable in which @code{yylex} should place the line and column
11198numbers associated with a token. (In a pure parser, it is a local
11199variable within @code{yyparse}, and its address is passed to
32c29292
JD
11200@code{yylex}.)
11201You can ignore this variable if you don't use the @samp{@@} feature in the
11202grammar actions.
11203@xref{Token Locations, ,Textual Locations of Tokens}.
742e4900 11204In semantic actions, it stores the location of the lookahead token.
32c29292 11205@xref{Actions and Locations, ,Actions and Locations}.
dd8d9022
AD
11206@end deffn
11207
11208@deffn {Type} YYLTYPE
11209Data type of @code{yylloc}; by default, a structure with four
11210members. @xref{Location Type, , Data Types of Locations}.
11211@end deffn
11212
11213@deffn {Variable} yylval
11214External variable in which @code{yylex} should place the semantic
11215value associated with a token. (In a pure parser, it is a local
11216variable within @code{yyparse}, and its address is passed to
32c29292
JD
11217@code{yylex}.)
11218@xref{Token Values, ,Semantic Values of Tokens}.
742e4900 11219In semantic actions, it stores the semantic value of the lookahead token.
32c29292 11220@xref{Actions, ,Actions}.
dd8d9022
AD
11221@end deffn
11222
11223@deffn {Macro} YYMAXDEPTH
1a059451
PE
11224Macro for specifying the maximum size of the parser stack. @xref{Memory
11225Management}.
dd8d9022
AD
11226@end deffn
11227
11228@deffn {Variable} yynerrs
8a2800e7 11229Global variable which Bison increments each time it reports a syntax error.
f4101aa6 11230(In a pure parser, it is a local variable within @code{yyparse}. In a
9987d1b3 11231pure push parser, it is a member of yypstate.)
dd8d9022
AD
11232@xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
11233@end deffn
11234
11235@deffn {Function} yyparse
11236The parser function produced by Bison; call this function to start
11237parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
11238@end deffn
11239
9987d1b3 11240@deffn {Function} yypstate_delete
f4101aa6 11241The function to delete a parser instance, produced by Bison in push mode;
9987d1b3 11242call this function to delete the memory associated with a parser.
f4101aa6 11243@xref{Parser Delete Function, ,The Parser Delete Function
9987d1b3 11244@code{yypstate_delete}}.
59da312b
JD
11245(The current push parsing interface is experimental and may evolve.
11246More user feedback will help to stabilize it.)
9987d1b3
JD
11247@end deffn
11248
11249@deffn {Function} yypstate_new
f4101aa6 11250The function to create a parser instance, produced by Bison in push mode;
9987d1b3 11251call this function to create a new parser.
f4101aa6 11252@xref{Parser Create Function, ,The Parser Create Function
9987d1b3 11253@code{yypstate_new}}.
59da312b
JD
11254(The current push parsing interface is experimental and may evolve.
11255More user feedback will help to stabilize it.)
9987d1b3
JD
11256@end deffn
11257
11258@deffn {Function} yypull_parse
f4101aa6
AD
11259The parser function produced by Bison in push mode; call this function to
11260parse the rest of the input stream.
11261@xref{Pull Parser Function, ,The Pull Parser Function
9987d1b3 11262@code{yypull_parse}}.
59da312b
JD
11263(The current push parsing interface is experimental and may evolve.
11264More user feedback will help to stabilize it.)
9987d1b3
JD
11265@end deffn
11266
11267@deffn {Function} yypush_parse
f4101aa6
AD
11268The parser function produced by Bison in push mode; call this function to
11269parse a single token. @xref{Push Parser Function, ,The Push Parser Function
9987d1b3 11270@code{yypush_parse}}.
59da312b
JD
11271(The current push parsing interface is experimental and may evolve.
11272More user feedback will help to stabilize it.)
9987d1b3
JD
11273@end deffn
11274
dd8d9022
AD
11275@deffn {Macro} YYPARSE_PARAM
11276An obsolete macro for specifying the name of a parameter that
11277@code{yyparse} should accept. The use of this macro is deprecated, and
11278is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
11279Conventions for Pure Parsers}.
11280@end deffn
11281
11282@deffn {Macro} YYRECOVERING
02103984
PE
11283The expression @code{YYRECOVERING ()} yields 1 when the parser
11284is recovering from a syntax error, and 0 otherwise.
11285@xref{Action Features, ,Special Features for Use in Actions}.
dd8d9022
AD
11286@end deffn
11287
11288@deffn {Macro} YYSTACK_USE_ALLOCA
eb45ef3b
JD
11289Macro used to control the use of @code{alloca} when the
11290deterministic parser in C needs to extend its stacks. If defined to 0,
d7e14fc0
PE
11291the parser will use @code{malloc} to extend its stacks. If defined to
112921, the parser will use @code{alloca}. Values other than 0 and 1 are
11293reserved for future Bison extensions. If not defined,
11294@code{YYSTACK_USE_ALLOCA} defaults to 0.
11295
55289366 11296In the all-too-common case where your code may run on a host with a
d7e14fc0
PE
11297limited stack and with unreliable stack-overflow checking, you should
11298set @code{YYMAXDEPTH} to a value that cannot possibly result in
11299unchecked stack overflow on any of your target hosts when
11300@code{alloca} is called. You can inspect the code that Bison
11301generates in order to determine the proper numeric values. This will
11302require some expertise in low-level implementation details.
dd8d9022
AD
11303@end deffn
11304
11305@deffn {Type} YYSTYPE
11306Data type of semantic values; @code{int} by default.
11307@xref{Value Type, ,Data Types of Semantic Values}.
18b519c0 11308@end deffn
bfa74976 11309
342b8b6e 11310@node Glossary
bfa74976
RS
11311@appendix Glossary
11312@cindex glossary
11313
11314@table @asis
eb45ef3b
JD
11315@item Accepting State
11316A state whose only action is the accept action.
11317The accepting state is thus a consistent state.
11318@xref{Understanding,,}.
11319
8a4281b9 11320@item Backus-Naur Form (BNF; also called ``Backus Normal Form'')
c827f760
PE
11321Formal method of specifying context-free grammars originally proposed
11322by John Backus, and slightly improved by Peter Naur in his 1960-01-02
11323committee document contributing to what became the Algol 60 report.
11324@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976 11325
eb45ef3b 11326@item Consistent State
35c1e5f0
JD
11327A state containing only one possible action. @xref{%define
11328Summary,,lr.default-reductions}.
eb45ef3b 11329
bfa74976
RS
11330@item Context-free grammars
11331Grammars specified as rules that can be applied regardless of context.
11332Thus, if there is a rule which says that an integer can be used as an
11333expression, integers are allowed @emph{anywhere} an expression is
89cab50d
AD
11334permitted. @xref{Language and Grammar, ,Languages and Context-Free
11335Grammars}.
bfa74976 11336
110ef36a
JD
11337@item Default Reduction
11338The reduction that a parser should perform if the current parser state
35c1e5f0
JD
11339contains no other action for the lookahead token. In permitted parser
11340states, Bison declares the reduction with the largest lookahead set to
11341be the default reduction and removes that lookahead set.
11342@xref{%define Summary,,lr.default-reductions}.
eb45ef3b 11343
bfa74976
RS
11344@item Dynamic allocation
11345Allocation of memory that occurs during execution, rather than at
11346compile time or on entry to a function.
11347
11348@item Empty string
11349Analogous to the empty set in set theory, the empty string is a
11350character string of length zero.
11351
11352@item Finite-state stack machine
11353A ``machine'' that has discrete states in which it is said to exist at
11354each instant in time. As input to the machine is processed, the
11355machine moves from state to state as specified by the logic of the
11356machine. In the case of the parser, the input is the language being
11357parsed, and the states correspond to various stages in the grammar
c827f760 11358rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976 11359
8a4281b9 11360@item Generalized LR (GLR)
676385e2 11361A parsing algorithm that can handle all context-free grammars, including those
8a4281b9 11362that are not LR(1). It resolves situations that Bison's
eb45ef3b 11363deterministic parsing
676385e2
PH
11364algorithm cannot by effectively splitting off multiple parsers, trying all
11365possible parsers, and discarding those that fail in the light of additional
c827f760 11366right context. @xref{Generalized LR Parsing, ,Generalized
8a4281b9 11367LR Parsing}.
676385e2 11368
bfa74976
RS
11369@item Grouping
11370A language construct that is (in general) grammatically divisible;
c827f760 11371for example, `expression' or `declaration' in C@.
bfa74976
RS
11372@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11373
8a4281b9 11374@item IELR(1)
35c1e5f0
JD
11375A minimal LR(1) parser table generation algorithm. That is, given any
11376context-free grammar, IELR(1) generates parser tables with the full
11377language recognition power of canonical LR(1) but with nearly the same
11378number of parser states as LALR(1). This reduction in parser states
11379is often an order of magnitude. More importantly, because canonical
11380LR(1)'s extra parser states may contain duplicate conflicts in the
11381case of non-LR(1) grammars, the number of conflicts for IELR(1) is
11382often an order of magnitude less as well. This can significantly
11383reduce the complexity of developing of a grammar. @xref{%define
11384Summary,,lr.type}.
eb45ef3b 11385
bfa74976
RS
11386@item Infix operator
11387An arithmetic operator that is placed between the operands on which it
11388performs some operation.
11389
11390@item Input stream
11391A continuous flow of data between devices or programs.
11392
8a4281b9 11393@item LAC (Lookahead Correction)
fcf834f9 11394A parsing mechanism that fixes the problem of delayed syntax error
35c1e5f0
JD
11395detection, which is caused by LR state merging, default reductions,
11396and the use of @code{%nonassoc}. Delayed syntax error detection
11397results in unexpected semantic actions, initiation of error recovery
11398in the wrong syntactic context, and an incorrect list of expected
11399tokens in a verbose syntax error message. @xref{%define
11400Summary,,parse.lac}.
fcf834f9 11401
bfa74976
RS
11402@item Language construct
11403One of the typical usage schemas of the language. For example, one of
11404the constructs of the C language is the @code{if} statement.
11405@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
11406
11407@item Left associativity
11408Operators having left associativity are analyzed from left to right:
11409@samp{a+b+c} first computes @samp{a+b} and then combines with
11410@samp{c}. @xref{Precedence, ,Operator Precedence}.
11411
11412@item Left recursion
89cab50d
AD
11413A rule whose result symbol is also its first component symbol; for
11414example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
11415Rules}.
bfa74976
RS
11416
11417@item Left-to-right parsing
11418Parsing a sentence of a language by analyzing it token by token from
c827f760 11419left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
11420
11421@item Lexical analyzer (scanner)
11422A function that reads an input stream and returns tokens one by one.
11423@xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
11424
11425@item Lexical tie-in
11426A flag, set by actions in the grammar rules, which alters the way
11427tokens are parsed. @xref{Lexical Tie-ins}.
11428
931c7513 11429@item Literal string token
14ded682 11430A token which consists of two or more fixed characters. @xref{Symbols}.
931c7513 11431
742e4900
JD
11432@item Lookahead token
11433A token already read but not yet shifted. @xref{Lookahead, ,Lookahead
89cab50d 11434Tokens}.
bfa74976 11435
8a4281b9 11436@item LALR(1)
bfa74976 11437The class of context-free grammars that Bison (like most other parser
8a4281b9 11438generators) can handle by default; a subset of LR(1).
eb45ef3b 11439@xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
bfa74976 11440
8a4281b9 11441@item LR(1)
bfa74976 11442The class of context-free grammars in which at most one token of
742e4900 11443lookahead is needed to disambiguate the parsing of any piece of input.
bfa74976
RS
11444
11445@item Nonterminal symbol
11446A grammar symbol standing for a grammatical construct that can
11447be expressed through rules in terms of smaller constructs; in other
11448words, a construct that is not a token. @xref{Symbols}.
11449
bfa74976
RS
11450@item Parser
11451A function that recognizes valid sentences of a language by analyzing
11452the syntax structure of a set of tokens passed to it from a lexical
11453analyzer.
11454
11455@item Postfix operator
11456An arithmetic operator that is placed after the operands upon which it
11457performs some operation.
11458
11459@item Reduction
11460Replacing a string of nonterminals and/or terminals with a single
89cab50d 11461nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
c827f760 11462Parser Algorithm}.
bfa74976
RS
11463
11464@item Reentrant
11465A reentrant subprogram is a subprogram which can be in invoked any
11466number of times in parallel, without interference between the various
11467invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
11468
11469@item Reverse polish notation
11470A language in which all operators are postfix operators.
11471
11472@item Right recursion
89cab50d
AD
11473A rule whose result symbol is also its last component symbol; for
11474example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
11475Rules}.
bfa74976
RS
11476
11477@item Semantics
11478In computer languages, the semantics are specified by the actions
11479taken for each instance of the language, i.e., the meaning of
11480each statement. @xref{Semantics, ,Defining Language Semantics}.
11481
11482@item Shift
11483A parser is said to shift when it makes the choice of analyzing
11484further input from the stream rather than reducing immediately some
c827f760 11485already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
bfa74976
RS
11486
11487@item Single-character literal
11488A single character that is recognized and interpreted as is.
11489@xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
11490
11491@item Start symbol
11492The nonterminal symbol that stands for a complete valid utterance in
11493the language being parsed. The start symbol is usually listed as the
13863333 11494first nonterminal symbol in a language specification.
bfa74976
RS
11495@xref{Start Decl, ,The Start-Symbol}.
11496
11497@item Symbol table
11498A data structure where symbol names and associated data are stored
11499during parsing to allow for recognition and use of existing
11500information in repeated uses of a symbol. @xref{Multi-function Calc}.
11501
6e649e65
PE
11502@item Syntax error
11503An error encountered during parsing of an input stream due to invalid
11504syntax. @xref{Error Recovery}.
11505
bfa74976
RS
11506@item Token
11507A basic, grammatically indivisible unit of a language. The symbol
11508that describes a token in the grammar is a terminal symbol.
11509The input of the Bison parser is a stream of tokens which comes from
11510the lexical analyzer. @xref{Symbols}.
11511
11512@item Terminal symbol
89cab50d
AD
11513A grammar symbol that has no rules in the grammar and therefore is
11514grammatically indivisible. The piece of text it represents is a token.
11515@xref{Language and Grammar, ,Languages and Context-Free Grammars}.
bfa74976
RS
11516@end table
11517
342b8b6e 11518@node Copying This Manual
f2b5126e 11519@appendix Copying This Manual
f2b5126e
PB
11520@include fdl.texi
11521
342b8b6e 11522@node Index
bfa74976
RS
11523@unnumbered Index
11524
11525@printindex cp
11526
bfa74976 11527@bye
a06ea4aa 11528
6b5a0de9
AD
11529@c LocalWords: texinfo setfilename settitle setchapternewpage finalout texi FSF
11530@c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex FSF's
11531@c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry Naur
11532@c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa Multi
11533@c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc multi
11534@c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex defaultprec Donnelly Gotos
11535@c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref yypush
11536@c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex lr
11537@c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge POSIX
11538@c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG yypull
11539@c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit nonfree
11540@c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok rr
11541@c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln Stallman Destructor
11542@c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym enum
11543@c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof Lex
11544@c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum DOTDOT
11545@c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype Unary
11546@c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs nonterminal
11547@c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES reentrant
11548@c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param yypstate
11549@c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP subrange
11550@c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword loc
11551@c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH inline
11552@c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype Lookahead
11553@c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args Autoconf
11554@c LocalWords: infile ypp yxx outfile itemx tex leaderfill Troubleshouting sqrt
11555@c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll lookahead
11556@c LocalWords: nbar yytext fst snd osplit ntwo strdup AST Troublereporting th
11557@c LocalWords: YYSTACK DVI fdl printindex IELR nondeterministic nonterminals ps
fcf834f9 11558@c LocalWords: subexpressions declarator nondeferred config libintl postfix LAC
6b5a0de9 11559@c LocalWords: preprocessor nonpositive unary nonnumeric typedef extern rhs
ff7571c0 11560@c LocalWords: yytokentype destructor multicharacter nonnull EBCDIC
6b5a0de9
AD
11561@c LocalWords: lvalue nonnegative XNUM CHR chr TAGLESS tagless stdout api TOK
11562@c LocalWords: destructors Reentrancy nonreentrant subgrammar nonassociative
11563@c LocalWords: deffnx namespace xml goto lalr ielr runtime lex yacc yyps env
11564@c LocalWords: yystate variadic Unshift NLS gettext po UTF Automake LOCALEDIR
11565@c LocalWords: YYENABLE bindtextdomain Makefile DEFS CPPFLAGS DBISON DeRemer
11566@c LocalWords: autoreconf Pennello multisets nondeterminism Generalised baz
11567@c LocalWords: redeclare automata Dparse localedir datadir XSLT midrule Wno
ff7571c0 11568@c LocalWords: Graphviz multitable headitem hh basename Doxygen fno
6b5a0de9
AD
11569@c LocalWords: doxygen ival sval deftypemethod deallocate pos deftypemethodx
11570@c LocalWords: Ctor defcv defcvx arg accessors arithmetics CPP ifndef CALCXX
11571@c LocalWords: lexer's calcxx bool LPAREN RPAREN deallocation cerrno climits
11572@c LocalWords: cstdlib Debian undef yywrap unput noyywrap nounput zA yyleng
11573@c LocalWords: errno strtol ERANGE str strerror iostream argc argv Javadoc
11574@c LocalWords: bytecode initializers superclass stype ASTNode autoboxing nls
11575@c LocalWords: toString deftypeivar deftypeivarx deftypeop YYParser strictfp
11576@c LocalWords: superclasses boolean getErrorVerbose setErrorVerbose deftypecv
11577@c LocalWords: getDebugStream setDebugStream getDebugLevel setDebugLevel url
11578@c LocalWords: bisonVersion deftypecvx bisonSkeleton getStartPos getEndPos
679e9935 11579@c LocalWords: getLVal defvar deftypefn deftypefnx gotos msgfmt Corbett
6b5a0de9 11580@c LocalWords: subdirectory Solaris nonassociativity
e944aaff
AD
11581
11582@c Local Variables:
11583@c ispell-dictionary: "american"
11584@c fill-column: 76
11585@c End: